(134 days)
The iCubate, Inc. iC-GN Assay™ for use on the iC-System™ is a qualitative, multiplexed, in vitro diagnostic test for the detection and identification of potentially pathogenic gram negative bacteria, which may cause bloodstream infection (BSI). The iC-GN Assay™ is performed directly on positive blood cultures, confirmed by Gram stain to contain gram negative bacilli. Cultures demonstrating mixed Gram stain results should not be tested on the assay. The iC-GN Assay™ is validated for use with select BACTEC™, BacT/ALERT® and VersaTREK® blood culture bottles. The iC-GN Assay™ is indicated for use in conjunction with other clinical and laboratory findings, such as culture, to aid in the diagnosis of bacterial bloodstream infections; however, it is not used to monitor bloodstream infections.
The iC-GN Assay™ detects target DNA and identifies the following:
Bacterial Genera and Species: Acinetobacter baumannii complex, Enterobacter cloacae complex, Escherichia coli, Klebsiella oxytoca, Klebsiella pneumoniae, Pseudomonas aeruginosa, Proteus species, Serratia marcescens
Resistance Markers: KPC (blaKPC)- associated with resistance to carbapenems, NDM (blaNDM)- associated with resistance to carbapenems, CTX-M group 1(blaCTX-M group 1)- associated with resistance to extended spectrum beta-lactams
In mixed growth, the iC-GN Assay™ does not specifically attribute detection of KPC, NDM, or CTX-M group 1 to a specific genera or species.
Sub-culturing of positive blood cultures is necessary to recover organisms for susceptibility testing, identification of organisms not detected by the iC-GN Assay™, differentiation of mixed growth, association of antimicrobial resistance marker genes to a specific organism, or for epidemiological typing.
The iC-GN Assay™ utilizes polymerase chain reaction (PCR) for the multiplex amplification of specific targets and detects the amplified targets with microarray hybridization. Targets are detected directly from patient positive blood cultures confirmed by Gram stain to contain gram negative bacilli. The iC-GN Assay utilizes proprietary ARM-PCR (Amplicon Rescued Multiplex PCR) technology allowing for multiple targets to be amplified in one reaction. Testing is done in a self-contained, automated, disposable cassette using the iCubate™ processor (iC-Processor™). After the reaction is complete, the cassette is read on the iCubate® reader (iC-Reader™). Results from the iC-Reader™ are interpreted by iC-Report™ software and a final report is displayed on the iMac® computer.
To operate, the user opens the iC-Cassette™ cap and pipettes an aliquot of the diluted positive blood culture sample into the sample/PCR well in the bottom well plate of the cassette. Once inoculated, the cassette cap is closed, and all extraction, amplification and detection processes are completed in the cassette, a closed system. Extraction, amplification and detection sequences are defined by an assay script controlled by the iC-Processor™.
The processing script is defined within a barcode label positioned on the top of each iC-Cassette™ which communicates with the iC-Processor™. To access and pierce the foilsealed reagent wells located in the bottom well plate of the cassette, the processor manipulates the cassette to move the cassette pipette horizontally and vertically. The script directs the transfer of reagents between the wells in the bottom well plate and finally to the array within the cassette. The iC-Processor™ is capable of processing four (4) iC-Cassettes™ with random access.
Once processing is complete, the cassette is manually transferred from the iC-Processor™ to the iC-Reader™ where the microarray within the cassette is read. The iC-Reader™ is capable of reading up to four (4) iC-Cassettes™ at one time. The results are interpreted via the iC-Report™ software and displayed for the user on the iMac®. Raw data and result interpretations are stored within the iMac®; raw data is accessible to iCubate® service personnel only and not to the end user.
When finished with a loaded iC-GN Cassette™, it should be disposed as biohazardous waste.
Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria are generally implied by the "Percent Agreement" values (positive and negative) to reference methods. While specific numeric acceptance thresholds are not explicitly stated as "acceptance criteria," performance metrics of 95% or higher with tight confidence intervals are typically considered strong evidence of meeting performance expectations for such devices. For reproducibility and equivalency, "≥ 95% performance" is explicitly stated as the acceptance criteria.
| Metric / Target | Acceptance Criteria (Implied/Explicit) | Reported Device Performance (Positive Percent Agreement) | Reported Device Performance (Negative Percent Agreement) |
|---|---|---|---|
| Reproducibility | ≥ 95% performance (explicit) | 96.7% - 100% (Overall for individual targets) | 96.7% - 100% (Overall for individual targets) |
| Blood Culture Bottle Equivalency | ≥ 95% performance (explicit) | 97.9% - 100.0% | 98.8% - 100.0% |
| Method Comparison (Organisms) | High agreement with reference method (implied) | ||
| Acinetobacter baumannii complex | High agreement with reference method (implied) | 100% (7/7 fresh, 45/45 contrived) | 99.9% (968-969/969 fresh, 125/125 contrived) |
| Enterobacter cloacae complex | High agreement with reference method (implied) | 94.5% (52/55 fresh), 100% (5/5 frozen, 17/17 contrived) | 100% (921/921 fresh, 153/153 contrived) |
| Escherichia coli | High agreement with reference method (implied) | 98.4% (480/488 fresh), 100% (6/6 frozen, 15/15 contrived) | 100% (488/488 fresh, 155/155 contrived) |
| Klebsiella oxytoca | High agreement with reference method (implied) | 95.8% (23/24 fresh), 100% (30/30 contrived) | 99.7% (949/952 fresh, 140/140 contrived) |
| Klebsiella pneumoniae | High agreement with reference method (implied) | 96.8% (150/155 fresh), 100% (3/3 frozen, 21/21 contrived) | 99.3% (815/821 fresh, 148/149 contrived) |
| Proteus mirabilis | High agreement with reference method (implied) | 97.4% (37/38 fresh), 100% (9/9 frozen, 12/12 contrived) | 99.5% (914/919 fresh, 158/158 contrived) |
| Pseudomonas aeruginosa | High agreement with reference method (implied) | 95.1% (78/82 fresh), 100% (1/1 frozen, 10/10 contrived) | 99.8% (892/894 fresh, 160/160 contrived) |
| Serratia marcescens | High agreement with reference method (implied) | 100% (29/29 fresh), 100% (20/20 contrived) | 99.6% (943/947 fresh, 149/150 contrived) |
| Method Comparison (Resistance Markers) | High agreement with reference method (implied) | ||
| CTX-M | High agreement with reference method (implied) | 97.0% (64/66 fresh), 100% (1/1 frozen, 15/15 contrived) | 99.9% (909/910 fresh, 155/155 contrived) |
| KPC | High agreement with reference method (implied) | 100% (1/1 fresh), 100% (50/50 contrived) | 99.9% (974/975 fresh), 99.2% (119/120 contrived) |
| NDM | High agreement with reference method (implied) | 0/0 (fresh/frozen) - not detected, 100% (50/50 contrived) | 100% (976/976 fresh), 100% (120/120 contrived) |
2. Sample size used for the test set and the data provenance
-
Method Comparison Study (Clinical Study):
- Total specimens enrolled: 1107
- Specimens included in performance analysis: 1002
- Fresh prospective specimens: 976
- Frozen prospective specimens (retrospectively tested): 26 (2.6%)
- Contrived samples: 170
- Data Provenance: Five geographically dispersed clinical sites in the U.S. (NY, WI, NM, FL, IN). The specimens were "leftover de-identified specimens from anaerobic blood culture bottles flagged as positive." This indicates a retrospective collection of clinical samples taken from patients, which were then de-identified and used for the study.
-
Reproducibility Study:
- Eighteen-organism panel tested in triplicate across five, non-consecutive days by two independent operators at each of three sites. This implies approximately 18 organisms * 3 replicates * 5 days * 2 operators * 3 sites = 1620 tests, but the table breaks down performance by target with denominators of ~90, suggesting a more specific number per target/concentration.
-
Limit of Detection (LoD) Study:
- Twenty-seven representative strains, a minimum of three per target. Tested in a minimum of twenty replicates (for confirmation phase) on each of three unique cassette lots.
-
Inclusivity Study:
- Eighty-two (82) representative strains, a minimum of ten strains for each target analyte. Each strain tested in triplicate.
-
Exclusivity Study:
- A total of 114 strains. Each strain tested in triplicate (retested in replicates of 3 or 10 for discordant results).
-
Microbial Interference Study:
- Sixty (60) gram negative exclusivity strains in combination with eight (8) iC-GN target organisms. Each combination tested in triplicate (retested in replicates of 3 or 10 for discordant results).
-
Competitive Inhibition Study:
- One target organism (low concentration) combined with a second target organism (high concentration). Each combination tested in triplicate (retested in replicates of 10 for false negatives).
-
Interfering Substances Study:
- Eight representative target organisms plus one non-target organism. Organism/interferent combination tested in triplicate (retested in replicates of 10 or 3 for discordant results).
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
The document does not specify the number or qualifications of experts used to establish the ground truth. It states that the ground truth for the clinical method comparison was based on:
- "reference culture followed by MALDI identification" for organism targets.
- "PCR amplification followed by confirmatory bidirectional sequencing" for resistance markers.
- "Phenotypic antimicrobial susceptibility testing (AST)" was also performed to identify samples requiring sequencing.
While these are standard laboratory methods, the involvement of specific "experts" (e.g., microbiologists, infectious disease specialists) beyond performing these standard laboratory procedures is not detailed.
4. Adjudication method for the test set
The document does not describe a formal adjudication method (like 2+1 or 3+1). For the method comparison study, discordant samples were "sequenced" (for resistance markers) or further analyzed by PCR/bi-directional sequencing for organism targets. For other performance studies (inclusivity, exclusivity, microbial interference), discordant results led to retesting in replicates (3 or 10). This indicates a re-testing approach for discrepancies rather than a multi-expert consensus.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
No, a multi-reader multi-case (MRMC) comparative effectiveness study was not done. The iC-GN Assay is an in vitro diagnostic (IVD) device, specifically a molecular diagnostic test for identifying microorganisms and resistance markers from positive blood cultures. It does not involve "human readers" interpreting images or data in a way that would typically be evaluated in an MRMC study for AI assistance. Its performance is compared against laboratory reference methods.
6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done
Yes, the studies presented evaluate the standalone performance of the iC-GN Assay. The assay is an automated system (iC-System) that detects and identifies targets directly from positive blood cultures using PCR and microarray hybridization. The results are interpreted by software (iC-Report™) and displayed for the user. The performance data (reproducibility, LoD, inclusivity, exclusivity, microbial interference, competitive inhibition, and method comparison) all reflect the device's inherent analytical and clinical performance without a human interpreting the primary result, but rather the system's output. The human role is operational (loading samples, reading the final report), not interpretive of the raw data.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
The ground truth used for the clinical method comparison study was established using:
- Reference culture followed by MALDI identification: This is a combination of microbiological culture methods and advanced biochemical analysis to identify bacterial species.
- PCR amplification followed by confirmatory bidirectional sequencing: This is a molecular method used to confirm the presence and identity of specific genes (e.g., resistance markers) or bacterial species through DNA sequencing.
- Phenotypic antimicrobial susceptibility testing (AST): Used to guide which resistance markers might need further sequencing for confirmation.
For other analytical studies (Inclusivity, Exclusivity, LoD, etc.), the ground truth was based on the known identity and concentration of the characterized bacterial strains used.
8. The sample size for the training set
The document does not provide details about a "training set" in the context of an algorithm or AI development. This device is a molecular diagnostic test, not an AI/ML-based diagnostic that would typically involve a separate training, validation, and test set for algorithm development. The studies described are performance validation studies for the finished medical device.
9. How the ground truth for the training set was established
As there's no explicit mention of a "training set" for an algorithm, the method for establishing its ground truth is not described. The ground truth for the verification and validation studies (as detailed in point 7) was established through standard microbiological and molecular laboratory techniques, which served as the comparator for assessing device performance.
§ 866.3365 Multiplex nucleic acid assay for identification of microorganisms and resistance markers from positive blood cultures.
(a)
Identification. A multiplex nucleic acid assay for identification of microorganisms and resistance markers from positive blood cultures is a qualitative in vitro device intended to simultaneously detect and identify microorganism nucleic acids from blood cultures that test positive by Gram stain or other microbiological stains. The device detects specific nucleic acid sequences for microorganism identification as well as for antimicrobial resistance. This device aids in the diagnosis of bloodstream infections when used in conjunction with other clinical and laboratory findings. However, the device does not replace traditional methods for culture and susceptibility testing.(b)
Classification. Class II (special controls). The special control for this device is FDA's guideline document entitled “Class II Special Controls Guideline: Multiplex Nucleic Acid Assay for Identification of Microorganisms and Resistance Markers from Positive Blood Cultures.” For availability of the guideline document, see § 866.1(e).