(34 days)
The PhiCal™ test is a quantitative ELISA for measuring, in human stool, concentrations of fecal calprotectin, a neutrophilic protein that is a marker of mucosal inflammation. The PhiCal™ test can be used as an in vitro diagnostic to aid in the diagnosis of inflammatory bowel diseases (IBD): Crohn's disease and ulcerative colitis, and to differentiate IBD from irritable bowel syndrome; when used in conjunction with other diagnostic testing and the total clinical picture.
The PhiCal test assay consists of: microtiter plate coated with polyclonal rabbit antibodies for calprotectin (12 strips, 8 wells per strip); alkaline phosphatase labeled rabbit anti-calprotectin IgG in buffer with Proclin 300; substrate in buffer; 20X washing solution; 10X dilution solution with Proclin 300; 5X extraction solution with Proclin 300; 5 vials of calprotectin solution at concentrations of 6.25, 12.5, 25, 50 and 100 ng/mL; a low and a high control.
Acceptance Criteria and Study for the PhiCal™ Test
1. Table of Acceptance Criteria and Reported Device Performance
The submission document primarily focuses on demonstrating the analytical and clinical performance of the PhiCal™ Test. The acceptance criteria are implicitly derived from the established performance metrics and the intended use of the device.
| Performance Metric | Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|---|
| Analytical Performance | ||
| Intra-Assay Precision (%CV) | Low CV, indicating high repeatability within a single assay run. | Single extraction, 20x assay reads: Low matrix: 4.07%, High matrix: 6.17%. |
| 12 samples, 6 replicates: Range 2.9% - 14.3%. | ||
| Inter-Assay Precision (%CV) | Low CV, indicating high reproducibility across different runs. | 6 stool pools, 20 separate extraction & assay runs: Range 8.9% - 18.1%. |
| 10 samples (5+, 5-), 5 extractions, 4 replicates each, 5 different days: Range 5.8% - 20.1%. | ||
| Extraction Repeat Reproducibility | Low CV, demonstrating consistent extraction process. | Low range sample: 12.60% CV, High range sample: 12.13% CV. |
| Split Sample Comparison (correlation) | High correlation (r²) with a reference laboratory. | y = 0.9603x + 9.7691, r² = 0.9618 (comparison with Fagerhol laboratory in Norway). |
| Aqueous Linearity (R²) | High R² and acceptable linearity. | y = 0.8771x + 2.8302, R² = 0.9962 (from 6.25 - 100 ng/ml). |
| Matrix Linearity (R²) | High R² and acceptable linearity. | y = 1.0569x - 0.1658, R² = 0.9866 (from 7.73 - 114.90 ng/ml). |
| Accuracy/Recovery (% Recovery) | % Recovery within an acceptable range for spiked samples. | Recoveries ranged from 99.1% to 118.6%. |
| Analyte Stability | Stable under recommended storage and transport conditions. | Calprotectin levels remained relatively consistent for up to 11 days under variable temperature conditions simulating specimen transport (2-8°C, 37°C, room temperature). Extract stability also demonstrated at -20°C for 11 days. |
| Minimum Detection Limit (MDL) | Result below the functional sensitivity of the assay. | Determined by 20 replicates of assay buffer, resulted in MDT below functional sensitivity. |
| Functional Sensitivity (%CV) | %CV 120 mcg/gm cut-off)** | High sensitivity for detecting IBD/Inflamed. |
| Overall Clinical Specificity (at >120 mcg/gm cut-off) | High specificity for differentiating IBS/Normal. | 94% (451/480) when borderline cases are excluded (n=132). |
| ROC Analysis (at 120 µg/g cut-off) | Acceptable sensitivity and specificity. | 95% specificity and 70% sensitivity at 120 µg/g. |
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size for Clinical Performance (Test Set): 908 subjects (total cohort used for clinical sensitivity and specificity).
- IBD: 255
- IBS: 410
- Other bowel diseases: 82
- Normal subjects: 161
- Data Provenance: Not explicitly stated regarding country of origin or specific study sites. The Split Sample Comparison Study mentions the "Fagerhol laboratory in Norway" as a reference for comparison, suggesting at least some interaction with data or expertise from Norway. The clinical study itself is presented as a single comprehensive cohort without geographical breakdown. Given the nature of a 510(k) submission for a new device, it is likely a retrospective analysis of collected samples for this initial submission, or a prospective collection for this specific study. The document does not specify "retrospective" or "prospective."
3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications
The document does not explicitly state the number of experts used to establish the ground truth for the clinical test set. The clinical study categorizes subjects into "IBD/Inflamed 'Organic'" and "IBS/Normal 'Non-Organic'". The diagnosis of these conditions (IBD, IBS, other bowel diseases, normal) would typically be established by Gastroenterologists based on a combination of clinical history, endoscopic findings, histopathology, imaging, and other diagnostic tests. However, the exact process or number/qualifications of experts involved in the classification of these 908 subjects (which serves as the "ground truth" for the device's clinical performance) are not detailed in this summary.
Regarding the calibrators reference standard, it mentions comparison to the "Gold Standard" preparation provided by Dr. Magne Fagerhol, Norway. Dr. Fagerhol is a prominent researcher in the field of calprotectin, implying his expertise in establishing this "gold standard."
4. Adjudication Method for the Test Set
The document does not describe a formal adjudication method (e.g., 2+1, 3+1) for establishing the diagnostic classifications of the 908 subjects. The diagnoses used as ground truth are presented as given, presumably established via standard diagnostic clinical practice.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No Multi-Reader Multi-Case (MRMC) comparative effectiveness study was mentioned. The device is a quantitative ELISA test, not an imaging-based AI system that would typically involve human readers. Therefore, the concept of human readers improving with AI vs. without AI assistance is not applicable in this context.
6. Standalone (Algorithm Only) Performance
Yes, a standalone performance study was done. The entire submission describes the performance of the PhiCal™ test as a "quantitative ELISA for measuring... fecal calprotectin," which is an in vitro diagnostic device. The clinical sensitivity and specificity are reported based on the device's measurements compared to the established clinical diagnoses (ground truth) of the 908 subjects. This is the definition of standalone performance for this type of diagnostic test – the assay provides a result without direct human interpretation in the loop that influences the result itself.
7. Type of Ground Truth Used
The ground truth for the clinical performance evaluation (sensitivity and specificity) was based on clinical diagnoses. Subjects were classified as "IBD/Inflamed 'Organic'," "IBS/Normal 'Non-Organic'," or "other bowel diseases" and "normal subjects." These classifications would typically involve a combination of:
- Clinical outcomes data: Patient symptoms, disease course.
- Pathology: Biopsy results (especially for IBD).
- Endoscopy: Visual assessment of the bowel lining.
- Other diagnostic testing: Blood tests, imaging, etc.
- Expert consensus: The overall clinical picture, as defined by medical professionals.
8. Sample Size for the Training Set
The document does not explicitly delineate a "training set" in the context of traditional machine learning where an algorithm is explicitly trained on data. This is an ELISA assay, not an AI/ML model. The "training" for such an assay primarily involves:
- Assay Development and Optimization: Iterative laboratory work to establish optimal reagent concentrations, incubation times, standard curves, etc. The document mentions "Establishing the cut-off for the assay was done by testing 124 normal sera." This group of 124 normal sera was likely used to help define the initial cut-off, which could be considered a form of "training" or optimization data for the clinical interpretation.
- Calibrators: The 5 calibrators used in each assay run (6.25, 12.5, 25, 50, and 100 ng/mL) are integral to "training" the curve from which sample concentrations are derived.
9. How the Ground Truth for the Training Set Was Established
As noted above, for an ELISA assay, the concept of a "training set" and associated ground truth is different from AI/ML.
- For the 124 normal sera used to establish the cut-off: The "ground truth" would be that these individuals were clinically healthy or considered "normal," likely confirmed through absence of symptoms and standard clinical assessment.
- For the Calibrators: The ground truth for the concentrations of the calibrators (6.25, 12.5, 25, 50, and 100 ng/mL) is established by careful gravimetric and volumetric preparation and validated against the internal primary reference, which itself is compared to the "Gold Standard" preparation provided by Dr. Magne Fagerhol, Norway. This "Gold Standard" essentially defines the true concentration of calprotectin.
§ 866.5180 Fecal calprotectin immunological test system.
(a)
Identification. A fecal calprotectin immunological test system is anin vitro diagnostic device that consists of reagents used to quantitatively measure, by immunochemical techniques, fecal calprotectin in human stool specimens. The device is intended forin vitro diagnostic use as an aid in the diagnosis of inflammatory bowel diseases (IBD), specifically Crohn's disease and ulcerative colitis, and as an aid in differentiation of IBD from irritable bowel syndrome.(b)
Classification. Class II (special controls). The special control for these devices is FDA's guidance document entitled “Class II Special Controls Guidance Document: Fecal Calprotectin Immunological Test Systems.” For the availability of this guidance document, see § 866.1(e).