Search Results

The Colibrí System is an in vitro diagnostic device comprised of the Colibrí Vision System and Collbrí Preparation Station for use with the bioMérieux VITEK MS or Bruker MALDI Biotyper CA mass spectrometry systems for qualitative identification of isolated colonies of Gram-negative bacterial species grown on solid culture media. The Collbri System is a semi-automated pre-analytical processor that picks isolated by the operator and uses a pipetting system to prepare MALDI-TOF MS (Matrix-Assisted Laser Desorption/Ionization-Time Of Flight Mass Spectrometry) target slides. The Colibri software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzer.

The Colibrí System is intended for use by trained healthcare professionals in clinical laboratories in conjunction with other clinical and laboratory finding Gram staining, to aid in the diagnosis of bacterial infections.

The Colibrí System has not been validated for use in identification of yeast species.

The Copan Colibrí System is designed to be used as an accessory of the downstream MALDI-TOF analyzers automating various manual steps in the workflow for the preparation of samples for the identification of isolated colonies of microorganisms cultured from the human body.

The Colibrí System automates the preparation of MALDI target slides for the bioMérieux VITEK MS or the Bruker MALDI Biotyper CA System that are used in clinical laboratories for identification and differentiation of organisms grown on plated media by Matrix-Assisted Laser Desorption/Ionization Time-of Flight Mass Spectrometry (MALDI-TOF MS). The system comprises the Colibrí Vision System and Colibrí Preparation Station and pipette tips as consumables. After appropriate plate incubation, the operator using the graphical User Interface (Image Reading Interface) chooses the plates exhibiting adequate growth and selects the isolated colonies to be processed assigning the automatic ID tasks. By using the Collbrí Vision System, specific colonies to be picked are designated by the operator on a digital plate. The Operator manually loads the plates in the Colibri Preparation Station where colonies are automatically picked, spotted on the target slide and overlayed with the matrix.

When used in conjunction with the bioMérieux VITEK MS, the Colibrí System can prepare the 48spot target slides by performing the direct spotting of colonies. The calibrator used for quality control is manually applied by the operator at the end of the automated colony spotting. When used in conjunction with the Bruker MALDI Biotyper CA System, the Colibrí System can prepare either reusable 48-spot or disposable 96-spot targets by performing the Direct Transfer Sample Procedure. The BTS used for quality control is manually applied by the operator at the and of the automated colony spotting.

The Colibrí software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.

Colibri System requires three different calibrations. None of these calibration activities require user intervention if not in terms of periodical cleaning of the mechanical component as described in the dedicated section of the User Manual. Set-up calibration is performed during the device initial setup for the camera units positioned on the Colibrí Vision System and on the Colibrí Preparation Station. Auto-calibration is performed at the end of the initial set-up and periodically during the preventive maintenance to check that, in the Colibri Preparation Station, all the mechanical references can be found inside the positioning tolerances, that the I/Os are responsive. Run-time calibration is performed during the normal usage to automatically check the proper functioning of the Colibrí Vision System and the Colibri Preparation Station.

The provided text describes the performance data for the Colibrí System, an in vitro diagnostic device. The acceptance criteria are implicitly defined by the performance observed in various analytical studies, with the goal of demonstrating substantial equivalence to predicate devices for qualitative identification of isolated colonies of Gram-negative and Gram-positive bacterial species.

Here's an analysis of the acceptance criteria and study proving the device meets them:

1. Table of Acceptance Criteria (Implicit) and Reported Device Performance

The document does not explicitly state pre-defined acceptance criteria in a dedicated table with specific thresholds. However, based on the studies conducted, the implicit acceptance criteria appear to be:

• High agreement with expected identification (ground truth).
• Absence of wrong identifications (false positives).
• Maintenance of performance across different conditions (e.g., positional effects, culture age, different MALDI-TOF MS systems and target types, different operators, different Colibrí systems).
• Comparable performance to manual preparation.
• No cross-contamination.

Here's a table summarizing the reported device performance, which serves as the evidence that these implicit criteria were met. The percentages below represent the agreement with the expected ID (ground truth) for high-confidence results where available, or overall agreement as calculated by the study.

2. Sample Sizes Used for the Test Set and Data Provenance

• Test Set (Analytical Studies):

  • Colony Picking: 1390 spots (VITEK MS) and 1690 spots (Bruker MALDI Biotyper CA).
  • Positional Effect: 864 spots (VITEK MS), 864 spots (Bruker MALDI Biotyper CA US IVD 48 Spot), 1692 spots (Bruker MALDI Biotyper CA MBT Biotarget 96 US IVD).
  • Inclusivity: 392 spots/strains (VITEK MS) and 468 spots/strains for each Bruker MALDI Biotyper CA system (US IVD 48 Spot and MBT Biotarget 96 US IVD).
  • Specificity: 20 spots (VITEK MS) and 20 spots (Bruker MALDI Biotyper CA).
  • Reproducibility: 1800 spots (VITEK MS) and 1800 spots (Bruker MALDI Biotyper CA).
  • Cross-Contamination: 572 spots (VITEK MS) and 686 spots (Bruker MALDI Biotyper CA total for both systems).
  • Colony Stability: 576 spots (VITEK MS) and 1440 spots (Bruker MALDI Biotyper CA).

• Data Provenance: The document generally indicates "on-panel" and "off-panel" strains, often referring to common clinical isolates in the US. No explicit country of origin for the clinical samples/strains themselves is stated beyond generic "human specimens" or "commonly isolated Gram-positive and Gram-negative species in the US". The studies were conducted by a single operator (Inclusivity, Specificity, Cross-Contamination, Colony Stability) or two operators (Reproducibility) on multiple Colibrí systems (Reproducibility, Colony Picking, Inclusivity, Cross-Contamination) in laboratory settings. The studies are prospective as they are designed experiments to evaluate the device performance.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

• The document implies that the ground truth for identification was established by the expected strain identity of the stock cultures used in the analytical studies. This means the identities of the bacterial strains were known beforehand (e.g., ATCC strains or well-characterized lab strains).
• For the performance results themselves, identification was based on the output of the VITEK MS or Bruker MALDI Biotyper CA systems, which generate confidence values or log scores. The interpretation of these scores (e.g., what constitutes a "correct single choice") is intrinsic to the performance specifications of these established MALDI-TOF MS systems.
• No human experts (e.g., radiologists) were used to establish the ground truth for these microbial identification studies. The "ground truth" is the presumed identity of the bacterial strain used in the experiment.

4. Adjudication Method for the Test Set

• Not applicable in the conventional sense. Since the ground truth for the test set was the expected strain identity of known bacterial cultures, there was no need for human expert adjudication to resolve discrepancies in interpretation. The output of the MALDI-TOF MS system for each sample was compared directly to the known identity of the inoculated strain. Discrepancies (low confidence, no ID, wrong ID) were noted and analyzed against the overall performance.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done

• No. An MRMC study is typically performed for image-based diagnostic aids where multiple human readers interpret cases with and without AI assistance. This study involves microorganism identification, a laboratory process, not direct human interpretation of images for diagnosis.
• The study compares the performance of the automated Colibrí System to manual preparation (the standard of care, as listed in the predicate device comparison tables), which is a different type of comparison than human readers with/without AI assistance.
• Effect size of human reader improvement: Not applicable, as no human reader study was conducted. The comparison is between an automated sample preparation method (Colibrí) and a manual sample preparation method for laboratory identification.

6. If a Standalone (i.e. algorithm only without human-in-the loop performance) was Done

• Yes, indirectly. The Colibrí System is a semi-automated pre-analytical processor. While an operator designates the colony, the picking, spotting, and matrix application are automated.
• The performance metrics provided (e.g., % agreement, no wrong ID) are for the Colibrí System's ability to prepare samples for subsequent MALDI-TOF MS analysis, which then generates the identification result. The Colibrí software records location and transmits information to the MALDI-TOF MS analyzer.
• The study evaluated the effectiveness of the automated sample preparation step compared to manual preparation, which directly impacts the downstream MALDI-TOF MS identification. However, the identification itself is performed by the MALDI-TOF MS analyzer, not the Colibrí system's "algorithm" in isolation, except for tracking sample positions. The Colibrí's key "algorithm" is in its vision system and robotic picking/spotting, which impacts the quality of the sample for the MALDI-TOF MS.

7. The Type of Ground Truth Used

• Expected Strain Identity: The ground truth for the analytical studies was the known, expected identity of the bacterial strains used. These were presumably well-characterized laboratory strains (e.g., ATCC cultures) with a confirmed identity. This is a highly controlled form of ground truth.

8. The Sample Size for the Training Set

• Not explicitly stated in terms of a "training set" for an AI/machine learning model used within Colibrí's core functions. The Colibrí System is described as a "semi-automated pre-analytical processor" that uses "a pipetting system" and "Colibrí software records the identity of each sample and its position". It's unclear if the "Colibrí Vision System" employs complex machine learning that would require a dedicated training set beyond basic image processing for colony detection and localization.
• The document implies that the system is automating a manual process, meaning its "training" pertains more to engineering and calibration rather than a machine learning model for diagnosis. The performance studies focus on the system's accuracy in physical manipulation and compatibility with existing MALDI-TOF MS systems.

9. How the Ground Truth for the Training Set Was Established

• Not applicable as a distinct ML training set is not described. If the "Colibrí Vision System" used internal algorithms that were "trained" to recognize colony morphology for picking, the ground truth would likely have been established by manually annotating colonies on images or physical plates as "pickable" or "not pickable" based on expert microbiological judgment, but this is not mentioned in the provided text. The current text suggests the operator "chooses the plates exhibiting adequate growth and selects the isolated colonies to be processed assigning the automatic ID tasks" on a "digital plate", implying human-in-the-loop for key decisions.

Ask a specific question about this device