(263 days)
The ALPHA Cryptococcal Antigen enzyme immunoassay (CrAg EIA) is a qualitative or semi-quantitative (titration) test system for the detection of capsular polysaccharide antigens of Cryptococcus species complex (Cryptococcus neoformans and Cryptococcus gattii) in serum and cerebrospinal fluid (CSF). The ALPHA Cryptococcal Antigen Enzyme Immunoassay is an assay which can be used as an aid in the diagnosis of cryptococcosis. Test results are to be used in conjunction with information available from the patient clinical evaluation and other diagnostic procedures.
The ALPHA Cryptococcal Antigen Enzyme Immunoassay (EIA) is a direct immunoenzymatic sandwich microplate assay which detects Cryptococcus antigens in serum and CSF. Anti-Cryptococcus antibodies bound to microwell plates are used as capture antibodies, and horseradish peroxidase (HRP)-conjugated anti-Cryptococcus antibodies are used as detect antibodies. The positive control and standard curve material are composed of cryptococcal capsular polysaccharide antigen in a buffered protein solution with a preservative. In the qualitative procedure, specimens are analyzed undiluted. In the titration procedure, specimens are analyzed after serial dilution in specimen diluent. Either serum or CSF is added to the microwells coated with the capture antibodies and incubated. If the patient specimen contains cryptococcal antigens that are recognized by the capture antibodies, those antigens will become bound to the microwells. The microwells are washed to remove unbound patient material, and HRP-conjugated detect antibody is added to the wells. If Cryptococcus antigens are bound to the microwells by the capture antibodies, the detect antibody will also become bound to the microwells. The wells are then washed to remove any unbound detect antibody. Next, tetramethylbenzidine (TMB) substrate is added to the microwells, and in the presence of HRP, a blue color will develop. The reaction is stopped by the addition of a stop solution. The optical density (OD) is determined with a microplate reader at 450 nm with reference at 630 nm (reference is optional).
Here's an analysis of the ALPHA Cryptococcal Antigen EIA based on the provided 510(k) summary, structured to address your specific points:
Acceptance Criteria and Device Performance Study
The document describes several analytical performance studies rather than clinical studies with patient outcomes or a specific "acceptance criteria" table for diagnostic accuracy. The primary criteria for device approval appear to be substantial equivalence to the predicate device (Meridian's Premier™ Cryptococcal Antigen EIA, K904393) and meeting internal laboratory performance benchmarks for precision, sensitivity, specificity, and non-interference.
However, based on the ROC analysis and method comparison studies, we can infer some performance expectations for sensitivity and specificity.
1. Table of Acceptance Criteria and Reported Device Performance
| Performance Metric | Acceptance Criteria (Inferred from Predicate Equivalence and ROC analysis) | Reported Device Performance (ALPHA Cryptococcal Antigen EIA) |
|---|---|---|
| Qualitative Agreement vs. Commercial EIA (Serum) | High agreement (similar to predicate) | % Positive Agreement: 98.5% (95% CI: 94.7% - 99.6%) |
| % Negative Agreement: 97.7% (95% CI: 96.7% - 98.4%) | ||
| Qualitative Agreement vs. Commercial EIA (CSF) | High agreement (similar to predicate) | % Positive Agreement: 93.5% (95% CI: 79.3% - 98.2%) |
| % Negative Agreement: 99.2% (95% CI: 97.8% - 99.7%) | ||
| Qualitative Agreement vs. IMMY CrAg LFA (Serum) | High agreement (similar to companion LFA) | % Positive Agreement: 96.9% |
| % Negative Agreement: 99.8% | ||
| Qualitative Agreement vs. IMMY CrAg LFA (CSF) | High agreement (similar to companion LFA) | % Positive Agreement: 93.8% |
| % Negative Agreement: 99.5% | ||
| Analytical Sensitivity (LoD) - Serum | Not explicitly stated as acceptance criteria, but demonstrates detection capability. | 5.3 ng/ml |
| Analytical Sensitivity (LoD) - CSF | Not explicitly stated as acceptance criteria, but demonstrates detection capability. | 1.7 ng/ml |
| Analytical Specificity (Cross-Reactivity) | Total positive for fungal pathologies 0.265) | |
| ROC Cutoff Specificity (vs. IMMY CrAg LFA) | Not explicitly stated, but target was to match LFA. | 99.9% (for blanked OD > 0.265) |
2. Sample Size Used for the Test Set and Data Provenance
The "test set" for performance evaluation consists of several cohorts:
- Precision Studies: Spiked serum and mock CSF samples. The number of unique samples is not given, but they were tested twice per day over five days across three sites. Each panel consisted of negative, high negative, low positive, and moderate positive samples for both serum and CSF.
- Analytical Sensitivity (LoD): 80 replicates of normal human serum, 80 replicates of artificial CSF for LoB. For LoD, initially 20 replicates of 4 concentrations, then 24 replicates of serum and CSF spiked with antigen near initial LoD.
- Analytical Specificity (Cross-Reactivity): 118 serum specimens and 15 fungal culture filtrates.
- Interference Studies: Five icteric, five hemolyzed, and five lipemic serum specimens.
- High Dose Hook Effect: One negative serum specimen pool spiked with antigen.
- Freeze-Thaw Study: Three CrAg-Negative human CSF specimens and four CrAg-Negative human serum specimens, spiked.
- ROC Analysis (for Cutoff Determination): 995 combined serum and CSF specimens.
- Method Comparison (vs. Commercial EIA and IMMY LFA): 1782 specimens (CSF n=426; serum n=1356).
- Data Provenance: The studies were conducted at Immuno-Mycologics, Inc. (IMMY) and two national reference laboratories, as well as a clinical lab for precision. The specimens for the method comparison were "submitted for cryptococcal antigen EIA testing," implying they were clinical samples, likely retrospective or leftover clinical samples. The country of origin is not explicitly stated, but the locations (Norman, OK; national reference labs) suggest United States origin. The nature of the samples (spiked or "submitted for testing") suggests retrospective use of clinical samples or artificially prepared samples.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The ground truth for the test sets was primarily established through:
- Spiking: For precision, LoD, interference, high dose hook, and freeze-thaw studies, samples were spiked with known concentrations of cryptococcal antigen, establishing a de facto ground truth.
- Reference Methods:
- For ROC analysis and comparison to LFA, the IMMY CrAg Lateral Flow Assay (LFA) was used as the reference method to classify specimens as true positive or negative. The expertise behind the LFA's accuracy would pertain to its own validation.
- For comparison to another manufacturer's device, the commercial Cryptococcal antigen EIA (the predicate device, Meridian's Premier™ Cryptococcal Antigen EIA) was used as the comparator.
- For analytical specificity, known positive/negative states for cross-reactants (e.g., Aspergillus galactomannan positive confirmed by Bio-Rad's Platelia™ Aspergillus EIA) were used.
The document does not mention the use of human experts (e.g., radiologists, pathologists) to establish ground truth for the test sets in the traditional sense of medical image analysis or complex diagnostic interpretation. The focus is on analytical performance and agreement with established laboratory tests.
4. Adjudication Method for the Test Set
No explicit adjudication method (e.g., 2+1, 3+1) is mentioned for the test sets. The ground truth was primarily defined by:
- Known concentrations (for spiked samples).
- Results from a single reference laboratory test (IMMY CrAg LFA or the commercial predicate EIA).
- Known status for cross-reactivity samples (e.g., confirmed Aspergillus positive).
There is no indication of multiple human readers or a consensus process for determining the ground truth for any of these studies.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
No, an MRMC comparative effectiveness study was not done. This device is an in vitro diagnostic (IVD) immunoassay that provides a quantitative or qualitative result. It is not an AI-powered diagnostic tool designed to assist human readers in interpreting images or complex data, nor does it involve "human readers" in the sense of medical specialists interpreting results that the device then supports or enhances.
6. If a Standalone (i.e. algorithm only without human-in-the loop performance) was done
This device is a standalone diagnostic test. It functions as an "algorithm only" in the sense that the assay procedure and subsequent optical density (OD) readings lead to a quantitative value or a positive/negative determination based on a defined cutoff. The performance studies described (precision, LoD, cross-reactivity, method comparison) are reflections of its standalone performance. There isn't a "human-in-the-loop" component in interpreting the EIA's direct output beyond standard laboratory practice of running the assay and reporting the result.
7. The Type of Ground Truth Used
The ground truth used for these studies can be categorized as:
- Known Spiking Concentrations: For analytical sensitivity, precision, interference, high-dose hook, and freeze-thaw studies.
- Reference Laboratory Assay Results:
- IMMY CrAg Lateral Flow Assay (LFA): Used as the reference for ROC analysis and comparison studies against the LFA.
- Commercial Cryptococcal Antigen EIA (Predicate Device): Used as the reference for method comparison studies to demonstrate substantial equivalence.
- External reference assays: For confirming known states of potentially cross-reacting substances (e.g., Bio-Rad's Platelia™ Aspergillus EIA).
8. The Sample Size for the Training Set
No explicit "training set" for an AI algorithm is mentioned as this is a traditional immunoassay.
However, if we interpret "training set" in the context of establishing assay parameters or a cutoff:
- The ROC analysis to determine the cutoff (0.265 blanked OD) was performed on a dataset of 995 combined serum and CSF specimens. This dataset could be considered analogous to a "training set" for defining the assay's threshold for positivity.
9. How the Ground Truth for the Training Set was Established
For the 995 specimens used in the ROC analysis to establish the cutoff:
- The ground truth was established by comparing the ALPHA Cryptococcal Antigen EIA results to the IMMY CrAg Lateral Flow Assay (LFA). The LFA was used to classify a specimen as a "true positive or negative" for the purpose of matching the sensitivity and specificity of the new EIA to the LFA. This means the LFA's results were accepted as the ground truth for this particular analysis aiming for concordance.
§ 866.3165
Cryptococcus neoformans serological reagents.(a)
Identification. Cryptococcus neoformans serological reagents are devices that consist of antigens used in serological tests to identify antibodies toCryptococcus neoformans in serum. Additionally, some of these reagents consist of antisera conjugated with a fluorescent dye (immunofluorescent reagents) and are used to identifyCryptococcus neoformans directly from clinical specimens or from cultured isolates derived from clinical specimens. The identification aids in the diagnosis of cryptococcosis and provides epidemiological information on this type of disease. Cryptococcosis infections are found most often as chronic meningitis (inflammation of brain membranes) and, if not treated, are usually fatal.(b)
Classification. Class II (special controls). The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to § 866.9.