(110 days)
The Colibrí is an automated in vitro diagnostic specimen preparation system for use with WASPLab to prepare MALDI-TOF targets for the bioMérieux VITEK MS systems or Bruker MALDI Biotyper CA mass spectrometry systems for qualitative identification and microbial suspension for the bioMérieux VITEK 2 systems or Beckman Coulter MicroScan WalkAway Antimicrobial Susceptibility Testing (AST) systems for qualitative testing of isolated colonies of gram-negative and gram-positive bacterial species grown on solid culture media.
The Colibrí is an automated pre-analytical processor that picks isolated colonies designated by the operator and uses a pipetting system to prepare MALDI-TOF MS (Matrix-Assisted Laser Desorption/lonization-Time of Flight Mass Spectrometry) target slides for bacterial identification and microbial suspension at known concentration for Antimicrobial Susceptibility Testing and purity assessment.
The Colibrí software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.
Bacterial suspensions for AST and purity plates are identified by barcode label.
The Colibrí is intended for use by trained healthcare professionals in clinical laboratories in conjunction with other clinical and laboratory findings, including Gram staining, to aid in the diagnosis of bacterial infections.
The Colibrí has not been validated for use in the identification or processing of yeast species, molds, Nocardia, or mycobacteria.
The Colibrí is an instrument which automates the picking of selected colonies from plated media and prepares MALDI target slides for the bioMérieux VITEK MS systems or the Bruker MALDI Biotyper CA systems that are used in clinical laboratories for identification and differentiation of organisms grown on plated media by Matrix-Assisted Laser Desorption/Jonization Time-of Flight Mass Spectrometry (MALDI-TOF MS). The Colibri automates the preparation of microbial suspensions at known concentration for bioMérieux VITEK 2 systems and Beckman Coulter MicroScan WalkAway systems that are used in clinical laboratories for AST analyses. Moreover, the Colibrí is used for Purity Plates preparation for purity assessments.
The Colibrí includes the following components:
- Colibrí instrument and software with on-board pipetting system and nephelometer .
- Colibrí Primary Tubes
- Colibrí Spreader
- Colibrí Daily Verification kit.
Colibri is designed to be used in conjunction with the WASPLab device for culture plate incubation and image analysis. After appropriate plate incubation, the operator selects the colonies from a digital image of culture media plate streaked with microbiological human specimen, available through WebApp software, the WASPLab User Interface.
The operator assigns the automatic ID or AST tasks to the isolated colonies to be processed. Then, the operator loads the plates in the Collbri where colonies are automatically picked, spotted on the target slide and overlayed with the matrix or suspended into the dedicated solution for the preparation of the microbial suspension for AST purposes (Secondary Tube).
When used in conjunction with the bioMérieux VITEK MS systems, the Colibri can prepare the 48-spot target slides by performing the direct spotting of colonies. The calibrator used for quality control is manually applied by the operator at the end of the automated colony spotting. When used in conjunction with the Bruker MALDI Biotyper CA systems, the Colibri can prepare either reusable 48-spot or disposable 96-spot targets by performing the Direct Transfer Sample Procedure. The BTS used for quality control is manually applied by the operator at the end of the automated colony spotting.
When used in conjunction with the bioMérieux VITEK 2 systems or the Beckman Coulter MicroScan WalkAway systems, the Colibri can prepare the microbial suspension at the proper concentration by direct colony suspension method. The onboard nephelometer allows the preparation of Secondary Tubes (AST suspensions) at the correct concentration and the Colibri Spreader is used for Purity Plates preparation.
The Colibrí software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.
The traceability of prepared Secondary Tube and Purity Plates is maintained by dedicated labels applications.
Colibrí requires four different calibrations, one on the nephelometer, three on the cameras. None of these calibration activities require user intervention if not in terms of periodical cleaning of the mechanical component as described in the dedicated section of the User Manual. The Set-up calibration of nephelometer and camera units are performed during the device initial setup. Auto-calibration is performed at the end of the initial set-up and periodically during the preventive maintenance to check that all the mechanical references can be found inside the positioning tolerances, that the I/Os are responsive. Runtime calibration is performed during the normal usage to automatically check the proper functioning of the Colibrí.
Colibrí requires a daily nephelometer verification to check the proper reading of suspensions at different turbidity values.
The Colibrí device is an automated in vitro diagnostic specimen preparation system. The provided text describes the acceptance criteria and the study that proves the device meets these criteria for preparing microbial suspensions for Antimicrobial Susceptibility Testing (AST) using Beckman Coulter MicroScan WalkAway systems.
Here's a breakdown of the requested information:
1. A table of acceptance criteria and the reported device performance:
The acceptance criteria are implicitly derived from the successful outcomes of the analytical studies. The performance is reported as the percentage of successful outcomes for each metric.
| Metric | Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|---|
| Preparation of Microbial Suspensions for AST | Microbial concentration within acceptable limits: |
- E. coli ATCC 25922: 3-7 x 105 CFU/mL
- Other bacteria: 2-8 x 105 CFU/mL | 98.5% of prepared suspensions had microbial concentration within acceptable limits.
- E. coli: 100% (36/36)
- Pseudomonas aeruginosa: 96.7% (29/30)
- Staphylococcus aureus: 97.6% (41/42)
- Enterococcus faecalis: 100% (30/30) |
| AST Challenge Test (Agreement with Manual Preparation) | Essential Agreement (EA) of MICs: High agreement
Category Agreement (CA): High agreement
Discrepancies (vmj, maj): Low/none | Overall EA: 100% (1232/1232 evaluable MIC results within 1 two-fold dilution)
Overall CA: 98.4% (4187/4254 SIR categorizations in agreement)
Very Major discrepancy (vmj): 0
Major discrepancy (maj): 0 |
| Reproducibility Study | Best-case reproducibility: ≥95% (implied)
Worst-case reproducibility: ≥89% (implied) | Best-case reproducibility: ≥99.8% (all panels combined)
Worst-case reproducibility: ≥94.3% (all panels combined) |
| Sample preparation for Quality Control | 100% of MIC values within CLSI/panel IFU QC range | 100% (all tested organisms and antimicrobial agents) |
| Purity Plates Evaluation (Cross-contamination) | Absence of cross-contamination (100% monomicrobial growth) | 100% (453/453 Purity Plates showed monomicrobial growth) |
2. Sample sizes used for the test set and the data provenance:
-
Preparation of Microbial Suspensions for AST:
- Test Set Size: 132 microbial suspensions (36 E. coli, 30 Pseudomonas aeruginosa, 42 Staphylococcus aureus, 30 Enterococcus faecalis).
- Data Provenance: Not explicitly stated (e.g., country of origin). The study involved three Colibrí instruments, suggesting internal validation. Retrospective or prospective is not specified, but the nature of the validation suggests prospective testing.
-
AST Challenge Test:
- Test Set Size: Different species: Enterobacterales (n=50 isolates), Staphylococcus (n=20 isolates), Streptococcus (n=12 isolates), Enterococcus (n=18 isolates), non-fermenters (n=10 isolates). Each processed by three Colibrí instruments, yielding varying numbers of MIC results and SIR categorizations across different panels (e.g., 2454 total MIC results for Enterobacterales on NM-NF50 panel).
- Data Provenance: Not explicitly stated (e.g., country of origin). The study design implies prospective testing within the manufacturer's validation process.
-
Reproducibility Study:
- Test Set Size: 9 gram-positive and 9 gram-negative strains, processed on 3 Colibrí instruments over 3 days, with each condition tested in triplicate (total of 27 replicates for each strain-antimicrobial agent combination).
- Data Provenance: Not explicitly stated (e.g., country of origin, retrospective or prospective). Implied prospective.
-
Sample preparation for Quality Control:
- Test Set Size: CLSI-recommended reference strains (Escherichia coli ATCC 25922, Pseudomonas aeruginosa ATCC 27853, Staphylococcus aureus ATCC 29213, Enterococcus faecalis ATCC 29212). The "No. MIC within QC range" indicates a total of 30 or 36 or 42 tests for each drug-organism combination, per three instruments.
- Data Provenance: Not explicitly stated. Implied prospective.
-
Purity Plates Evaluation:
- Test Set Size: 453 purity plates (150 from AST Challenge, 162 from AST Reproducibility, 141 from Quality Control studies).
- Data Provenance: Not explicitly stated. Implied prospective.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- The ground truth for AST results (MICs and SIR categories) generally refers to the results obtained from a reference method. In this case, "Manual suspension was used as comparative method" for the AST Challenge Test. This implies that manually prepared suspensions, processed by the MicroScan WalkAway, served as the reference standard.
- The text does not specify the number of experts or their qualifications for establishing this manual ground truth. It mentions that three different technicians operated the Colibrí machines, but it doesn't detail the personnel for the manual comparative method or for interpreting the results as ground truth beyond the "FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria" and CLSI guidelines.
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set:
- The document does not describe any expert adjudication process for the test set results. The comparison is made against a "manual result" (ground truth). The discrepancies (vmj, maj, min) are simply categorized and reported, implying a direct comparison without further expert review for resolving initial disagreements.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No, an MRMC study was not conducted. This device (Colibrí) is an automated system for sample preparation and does not involve human "readers" or "AI assistance" in the typical sense of image analysis for diagnosis. Its role is to automate a laboratory process, and the performance is measured against reference methods, not human interpretation.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- The studies presented are primarily standalone (algorithm only without human-in-the-loop performance) in terms of the Colibrí device's automated functions. The device picks colonies, prepares suspensions, and records data automatically. The performance metrics (inoculum density, MIC accuracy, reproducibility, purity) assess the device's output against established standards and manual methods.
- While an operator designates colonies for picking, the act of preparation itself is automated and evaluated for its accuracy. The "manual suspension" used for comparison acts as the reference for the "algorithm only" performance of the Colibrí in producing the suspension.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc):
- The ground truth for the analytical studies combines reference methods/standards and established guidelines:
- For microbial suspension concentration: Viable cell count (CFU/mL) against CLSI and FDA guidelines.
- For AST results: Manual suspension preparation as the comparative method, and comparison of MICs and SIR categories against FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria and CLSI guideline M07.
- For reproducibility: Comparison to the "mode result" (most frequent MIC value) and established reproducibility criteria (e.g., within one doubling dilution).
- For Quality Control: CLSI-recommended QC ranges and MicroScan panel IFU values.
- For Purity Plates: Visual assessment (implied) to confirm monomicrobial growth.
8. The sample size for the training set:
- This document describes performance validation studies for a medical device (Colibrí), not a machine learning model. Therefore, there is no "training set" in the context of data used to train an AI algorithm. The Colibrí is an automated instrument with pre-programmed functions, not a learning algorithm that requires a training dataset.
9. How the ground truth for the training set was established:
- As explained above, there is no training set for this device in the context of AI/ML.
§ 866.1645 Fully automated short-term incubation cycle antimicrobial susceptibility system.
(a)
Identification. A fully automated short-term incubation cycle antimicrobial susceptibility system is a device that incorporates concentrations of antimicrobial agents into a system for the purpose of determining in vitro susceptibility of bacterial pathogens isolated from clinical specimens. Test results obtained from short-term (less than 16 hours) incubation are used to determine the antimicrobial agent of choice to treat bacterial diseases.(b)
Classification. Class II (special controls). The special control for this device is FDA's guidance document entitled “Class II Special Controls Guidance Document: Antimicrobial Susceptibility Test (AST) Systems; Guidance for Industry and FDA.”