(151 days)
The Colibri™ is an automated in vitro diagnostic specimen preparation system for use with WASPLab® to prepare MALDI-TOF targets for the bioMérieux VITEK® MS or Bruker MALDI Biotyper® CA mass spectrometry systems for qualitative identification and microbial suspension for the bioMérieux VITEK® 2 Antimicrobial Susceptibility Testing (AST) system for qualitative testing of isolated colonies of gram-negative and gram-positive bacterial species grown on solid culture media.
The Colibri™ is an automated pre-analytical processor that nicks isolated colonies designated by the operator and uses a pipetting system to prepare MALDI-TOF MS (Matrix-Assisted Laser Desorption/lonization-Time of Flight Mass Spectrometry) target slides for bacterial identification and microbial suspension at known concentration for Antimicrobial Susceptibility Testing and purity assessment.
The Collori™ software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.
Bacterial suspensions for AST and purity plates are identified by barcode label.
The Colibri™ is intended for use by trained healthcare professionals in clinical laboratories in conjunction with other clinical and laboratory findings, including Gram staining, to aid in the diagnosis of bacterial infections.
The Colibri™ has not been validated for use in the identification or processing of yeast species, or mycobacteria.
The Colibrí is an instrument which automates the picking of selected colonies from plated media and prepares MALDI target slides for the bioMérieux VITEK MS or the Bruker MALDI Biotyper CA System that are used in clinical laboratories for identification and differentiation of organisms grown on plated media by Matrix-Assisted Laser Desorption/Tonization Time-of Flight Mass Spectrometry (MALDI-TOF MS). The Colibrí automates the preparation of microbial suspensions at known concentration for bioMérieux VITEK 2 System that is used in clinical laboratories for AST analyses. Moreover, the Colibrí is used for Purity Plates preparation for purity assessments.
The Colibrí includes the following components:
- . Colibrí instrument and software with on-board pipetting system and nephelometer
- Colibrí Primary Tubes ●
- . Colibrí Spreader
- Colibrí Daily Verification kit. ●
Colibri is designed to be used in conjunction with the WASPLab device for culture plate incubation and image analysis. After appropriate plate incubation, the operator selects the colonies from a digital image of culture media plate streaked with microbiological human specimen, available through WebApp software, the WASPLab User Interface.
The operator assigns the automatic ID or AST tasks to the isolated colonies to be processed. Then, the operator loads the plates in the Colibri where colonies are automatically picked, spotted on the target slide and overlayed with the matrix or suspended into the dedicated solution for the preparation of the microbial suspension for AST purposes (Secondary Tube).
When used in conjunction with the bioMérieux VITEK MS, the Colibrí can prepare the 48-spot target slides by performing the direct spotting of colonies. The calibrator used for quality control is manually applied by the operator at the end of the automated colony spotting. When used in conjunction with the Bruker MALDI Biotyper CA System, the Colibri can prepare either reusable 48-spot or disposable 96-spot targets by performing the Direct Transfer Sample Procedure. The BTS used for quality control is manually applied by the operator at the end of the automated colony spotting.
When used in conjunction with the bioMérieux VITEK 2, the Colibrí can prepare the microbial suspension at the proper concentration by direct colony suspension method. The onboard nephelometer allows the preparation of Secondary Tubes (AST suspensions) at the correct concentration and the Colibrí Spreader is used for Purity Plates preparation.
The Colibrí software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.
The traceability of prepared Secondary Tube and Purity Plates is maintained by dedicated labels applications.
Colibrí requires four different calibrations, one on the nephelometer, three on the cameras. None of these calibration activities require user intervention if not in terms of periodical cleaning of the mechanical component as described in the dedicated section of the User Manual. The Set-up calibration of nephelometer and camera units are performed during the device initial setup. Autocalibration is performed at the end of the initial set-up and periodically during the preventive maintenance to check that all the mechanical references can be found inside the positioning tolerances, that the I/Os are responsive. Run-time calibration is performed during the normal usage to automatically check the proper functioning of the Colibrí.
Colibrí requires a daily nephelometer verification to check the proper reading of suspensions at different turbidity values.
Acceptance Criteria and Device Performance for Colibrí
The Colibrí is an automated in vitro diagnostic specimen preparation system for use with WASPLab to prepare MALDI-TOF targets for microbial identification and microbial suspensions for Antimicrobial Susceptibility Testing (AST).
1. Table of Acceptance Criteria and Reported Device Performance
| Parameter | Acceptance Criteria | Reported Device Performance |
|---|---|---|
| Colony Picking Accuracy | 100% correct picking of designated colonies. | 100% of designated colonies were correctly picked without any event of picking a wrong colony. |
| VITEK MS Identification Agreement (Gram-Negative) | High agreement with expected strain identity. | Gram-Negative Species: All species (Citrobacter koseri, Escherichia coli, Klebsiella pneumoniae, Proteus mirabilis, Pseudomonas aeruginosa) showed 100% agreement with good confidence for VITEK MS identification. Across 92 picked colonies, there was a 100% agreement on species/group identification. |
| VITEK MS Identification Agreement (Gram-Positive) | High agreement with expected strain identity. | Gram-Positive Species: Overall, 98.0% agreement with good confidence across 200 picked colonies. Individual species agreement ranged from 95.0% (Enterococcus faecium) to 100% (Staphylococcus aureus, Staphylococcus saprophyticus). |
| AST Essential Agreement (EA) | >98% agreement between Colibrí prepared samples and manually prepared samples. | For all species and antimicrobial agents combined, 100% (1315/1315) of on-scale MIC results were in Essential Agreement (EA). |
| AST Category Agreement (CA) | >98% agreement between Colibrí prepared samples and manually prepared samples. | For all species and antimicrobial agents combined, 99.4% (2703/2720) of results were in Category Agreement (CA). |
| AST Very Major Category Error | 0% | 0% |
| AST Major Category Error | 0% | 0% |
| AST Minor Category Error | Acceptable level (calculated based on the differences in CA from 100%). | 17 minor errors occurred out of 2720 tested, resulting in a minor error rate of 0.6%. This is considered acceptable given the high overall CA. |
2. Sample Size Used for the Test Set and Data Provenance
The study utilized a test set consisting of various bacterial strains to evaluate the Colibrí's performance in preparing samples for MALDI-TOF identification and AST.
- Sample Size for Identification:
- Gram-Negative Species: 5 species (4 Enterobacterales, 1 Non-fermenter) with 92 colonies picked and analyzed.
- Gram-Positive Species: 5 species (2 Enterococcus, 2 Staphylococcus, 1 Streptococcus) with 200 colonies picked and analyzed.
- Total for Identification: 292 colonies.
- Sample Size for AST:
- Total AST Tested: 2720 tests (sum of "Total tested" column from AST summary tables).
- This included: 1400 Enterobacterales, 200 Non-fermenters, 520 Staphylococci, 380 Enterococci, and 220 Streptococci.
- Data Provenance: The document does not explicitly state the country of origin of the data or whether it was retrospective or prospective. However, given that it is a submission to the U.S. Food & Drug Administration (FDA) by an Italian company (Copan WASP Srl, Brescia, Italy), the study was likely conducted to meet international regulatory standards, potentially with data from one or more clinical laboratories. The nature of the "Full workflow validation" study described suggests a prospective experimental design to assess the device's performance under controlled conditions.
3. Number of Experts Used to Establish Ground Truth and Qualifications
The document does not explicitly state the number of experts used or their specific qualifications for establishing ground truth. However, the study focuses on the accuracy of identification results against "expected strain identity" and comparison of MICs and SIR categories against "MICs obtained by bioMérieux VITEK 2 using manual sample preparation" and "FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria."
This implies that:
- For identification, the "expected strain identity" would have been established using a gold standard method, likely by skilled microbiologists or reference laboratories with established credentials in microbial identification.
- For AST, the "manual sample preparation" by bioMérieux VITEK 2 would serve as the comparator, and the interpretation of results would follow "FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria," likely applied by qualified laboratory personnel.
4. Adjudication Method
The document does not describe a formal adjudication method (e.g., 2+1, 3+1) for the test set.
- For identification, performance was calculated as "percentage of spotted colonies matching the expected identity," suggesting a direct comparison to a single reference (ground truth).
- For AST, the "MICs obtained by bioMérieux VITEK 2 using Colibrí were compared to the MICs obtained by bioMérieux VITEK 2 using manual sample preparation," indicating a direct method-to-method comparison. Discrepant SIR results were categorized, implying a systematic evaluation rather than a consensus-based adjudication in the traditional sense.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
There is no mention of a Multi-Reader Multi-Case (MRMC) comparative effectiveness study or any effect size of how much human readers improve with AI vs. without AI assistance. The Colibrí device is a pre-analytical automated system for sample preparation, not an AI-assisted diagnostic interpretation tool for human readers. Its function is to automate the preparation steps for subsequent analysis by other IVD systems (MALDI-TOF MS and VITEK 2 AST).
6. Standalone Performance Study
Yes, a standalone performance study was done for the algorithm/system. The "Full workflow validation" study directly assesses the Colibrí device's ability to accurately pick colonies and prepare samples for downstream analysis without human intervention in the picking and preparation steps themselves. The results presented for "VITEK MS Identification Result" and "AST summary of results" directly reflect the performance of the Colibrí-prepared samples when analyzed by the respective analytical instruments.
7. Type of Ground Truth Used
- For microbial identification: The ground truth was based on "expected strain identity," which implies confirmed identification results likely obtained through established, highly accurate microbiological methods serving as a reference.
- For AST: The ground truth for comparative purposes was "MICs obtained by bioMérieux VITEK 2 using manual sample preparation," interpreted according to "FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria."
8. Sample Size for the Training Set
The document explicitly states that the "Colibrí uses equally designed and developed hardware and software modules as the Colibrí System. Therefore, the performance of the predicate device represents the performance of the new device." It also mentions that "The results of the analytical studies were submitted to support the 510(k) Premarket Notifications K193138 and K220546." This indicates that the current submission (K223245) relies on the performance data of the predicate device (Colibrí System, K220546). The document does not provide details on the training set sample size for the Colibrí System's development.
9. How the Ground Truth for the Training Set Was Established
As the current submission relies on the predicate device's performance, the ground truth for any training set related to the predicate Colibrí System would have been established during its development and prior FDA submission (K220546). The current document does not provide details of how the ground truth for the predicate device's training set was established. However, based on the performance parameters listed (e.g., accuracy of colony picking, reproducibility of identification, accuracy of nephelometer), it likely involved:
- Microbial cultures: Using well-characterized microbial strains with confirmed identities.
- Manual reference methods: Comparing automated colony picking against visual confirmation by trained personnel.
- Reference AST methods: Comparing automated suspension preparation and subsequent AST results against established manual AST methods and interpretive criteria to assess agreement in MICs and categorical susceptibility (Susceptible, Intermediate, Resistant).
§ 866.3378 Clinical mass spectrometry microorganism identification and differentiation system.
(a)
Identification. A clinical mass spectrometry microorganism identification and differentiation system is a qualitative in vitro diagnostic device intended for the identification and differentiation of microorganisms from processed human specimens. The system acquires, processes, and analyzes spectra to generate data specific to a microorganism(s). The device is indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial and fungal infection.(b)
Classification. Class II (special controls). The special controls for this device are:(1) The intended use statement must include a detailed description of what the device detects, the type of results provided to the user, the clinical indications appropriate for test use, and the specific population(s) for which the device is intended, when applicable.
(2) Any sample collection device used must be FDA-cleared, -approved, or -classified as 510(k) exempt with an indication for in vitro diagnostic use.
(3) The labeling required under § 809.10(b) of this chapter must include:
(i) A detailed device description, including all device components, control elements incorporated into the test procedure, instrument requirements, ancillary reagents required but not provided, and a detailed explanation of the methodology and all pre-analytical methods for processing of specimens, and algorithm used to generate a final result. This must include a description of validated inactivation procedure(s) that are confirmed through a viability testing protocol, as applicable.
(ii) Performance characteristics for all claimed sample types from clinical studies with clinical specimens that include prospective samples and/or, if appropriate, characterized samples.
(iii) Performance characteristics of the device for all claimed sample types based on analytical studies, including limit of detection, inclusivity, reproducibility, interference, cross-reactivity, interfering substances, carryover/cross-contamination, sample stability, and additional studies regarding processed specimen type and intended use claims, as applicable.
(iv) A detailed explanation of the interpretation of test results for clinical specimens and acceptance criteria for any quality control testing.
(4) The device's labeling must include a prominent hyperlink to the manufacturer's website where the manufacturer must make available their most recent version of the device's labeling required under § 809.10(b) of this chapter, which must reflect any changes in the performance characteristics of the device. FDA must have unrestricted access to this website, or manufacturers must provide this information to FDA through an alternative method that is considered and determined by FDA to be acceptable and appropriate.
(5) Design verification and validation must include:
(i) Any clinical studies must be performed with samples representative of the intended use population and compare the device performance to results obtained from an FDA-accepted reference method and/or FDA-accepted comparator method, as appropriate. Documentation from the clinical studies must include the clinical study protocol (including predefined statistical analysis plan, if applicable), clinical study report, and results of all statistical analyses.
(ii) Performance characteristics for analytical and clinical studies for specific identification processes for the following, as appropriate:
(A) Bacteria,
(B) Yeasts,
(C) Molds,
(D) Mycobacteria,
(E) Nocardia,
(F) Direct sample testing (
e.g., blood culture),(G) Antibiotic resistance markers, and
(H) Select agents (
e.g., pathogens of high consequence).(iii) Documentation that the manufacturer's risk mitigation strategy ensures that their device does not prevent any device(s) with which it is indicated for use, including incorporated device(s), from achieving their intended use (
e.g., safety and effectiveness of the functions of the indicated device(s) remain unaffected).(iv) A detailed device description, including the following:
(A) Overall device design, including all device components and all control elements incorporated into the testing procedure.
(B) Algorithm used to generate a final result from raw data (
e.g., how raw signals are converted into a reported result).(C) A detailed description of device software, including validation activities and outcomes.
(D) Acquisition parameters (
e.g., mass range, laser power, laser profile and number of laser shots per profile, raster scan, signal-to-noise threshold) used to generate data specific to a microorganism.(E) Implementation methodology, construction parameters, and quality assurance protocols, including the standard operating protocol for generation of reference entries for the device.
(F) For each claimed microorganism characteristic, a minimum of five reference entries for each organism (including the type strain for microorganism identification), or, if there are fewer reference entries, a clinical and/or technical justification, determined by FDA to be acceptable and appropriate, for why five reference entries are not needed.
(G) DNA sequence analysis characterizing all type strains and at least 20 percent of the non-type strains of a species detected by the device, or, if there are fewer strain sequences, then a clinical and/or technical justification, determined by FDA to be acceptable and appropriate, must be provided for the reduced number of strains sequenced.
(H) As part of the risk management activities, an appropriate end user device training program, which must be offered as an effort to mitigate the risk of failure from user error.