Search Results

The Colibrí is an automated in vitro diagnostic specimen preparation system for use with WASPLab to prepare MALDI-TOF targets for the bioMérieux VITEK MS systems or Bruker MALDI Biotyper CA mass spectrometry systems for qualitative identification and microbial suspension for the bioMérieux VITEK 2 systems or Beckman Coulter MicroScan WalkAway Antimicrobial Susceptibility Testing (AST) systems for qualitative testing of isolated colonies of gram-negative and gram-positive bacterial species grown on solid culture media.

The Colibrí is an automated pre-analytical processor that picks isolated colonies designated by the operator and uses a pipetting system to prepare MALDI-TOF MS (Matrix-Assisted Laser Desorption/lonization-Time of Flight Mass Spectrometry) target slides for bacterial identification and microbial suspension at known concentration for Antimicrobial Susceptibility Testing and purity assessment.

The Colibrí software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.

Bacterial suspensions for AST and purity plates are identified by barcode label.

The Colibrí is intended for use by trained healthcare professionals in clinical laboratories in conjunction with other clinical and laboratory findings, including Gram staining, to aid in the diagnosis of bacterial infections.

The Colibrí has not been validated for use in the identification or processing of yeast species, molds, Nocardia, or mycobacteria.

The Colibrí is an instrument which automates the picking of selected colonies from plated media and prepares MALDI target slides for the bioMérieux VITEK MS systems or the Bruker MALDI Biotyper CA systems that are used in clinical laboratories for identification and differentiation of organisms grown on plated media by Matrix-Assisted Laser Desorption/Jonization Time-of Flight Mass Spectrometry (MALDI-TOF MS). The Colibri automates the preparation of microbial suspensions at known concentration for bioMérieux VITEK 2 systems and Beckman Coulter MicroScan WalkAway systems that are used in clinical laboratories for AST analyses. Moreover, the Colibrí is used for Purity Plates preparation for purity assessments.

The Colibrí includes the following components:

• Colibrí instrument and software with on-board pipetting system and nephelometer .
• Colibrí Primary Tubes
• Colibrí Spreader
• Colibrí Daily Verification kit.

Colibri is designed to be used in conjunction with the WASPLab device for culture plate incubation and image analysis. After appropriate plate incubation, the operator selects the colonies from a digital image of culture media plate streaked with microbiological human specimen, available through WebApp software, the WASPLab User Interface.

The operator assigns the automatic ID or AST tasks to the isolated colonies to be processed. Then, the operator loads the plates in the Collbri where colonies are automatically picked, spotted on the target slide and overlayed with the matrix or suspended into the dedicated solution for the preparation of the microbial suspension for AST purposes (Secondary Tube).

When used in conjunction with the bioMérieux VITEK MS systems, the Colibri can prepare the 48-spot target slides by performing the direct spotting of colonies. The calibrator used for quality control is manually applied by the operator at the end of the automated colony spotting. When used in conjunction with the Bruker MALDI Biotyper CA systems, the Colibri can prepare either reusable 48-spot or disposable 96-spot targets by performing the Direct Transfer Sample Procedure. The BTS used for quality control is manually applied by the operator at the end of the automated colony spotting.

When used in conjunction with the bioMérieux VITEK 2 systems or the Beckman Coulter MicroScan WalkAway systems, the Colibri can prepare the microbial suspension at the proper concentration by direct colony suspension method. The onboard nephelometer allows the preparation of Secondary Tubes (AST suspensions) at the correct concentration and the Colibri Spreader is used for Purity Plates preparation.

The Colibrí software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.

The traceability of prepared Secondary Tube and Purity Plates is maintained by dedicated labels applications.

Colibrí requires four different calibrations, one on the nephelometer, three on the cameras. None of these calibration activities require user intervention if not in terms of periodical cleaning of the mechanical component as described in the dedicated section of the User Manual. The Set-up calibration of nephelometer and camera units are performed during the device initial setup. Auto-calibration is performed at the end of the initial set-up and periodically during the preventive maintenance to check that all the mechanical references can be found inside the positioning tolerances, that the I/Os are responsive. Runtime calibration is performed during the normal usage to automatically check the proper functioning of the Colibrí.

Colibrí requires a daily nephelometer verification to check the proper reading of suspensions at different turbidity values.

The Colibrí device is an automated in vitro diagnostic specimen preparation system. The provided text describes the acceptance criteria and the study that proves the device meets these criteria for preparing microbial suspensions for Antimicrobial Susceptibility Testing (AST) using Beckman Coulter MicroScan WalkAway systems.

Here's a breakdown of the requested information:

1. A table of acceptance criteria and the reported device performance:

The acceptance criteria are implicitly derived from the successful outcomes of the analytical studies. The performance is reported as the percentage of successful outcomes for each metric.

• E. coli ATCC 25922: 3-7 x 105 CFU/mL
• Other bacteria: 2-8 x 105 CFU/mL | 98.5% of prepared suspensions had microbial concentration within acceptable limits.
• E. coli: 100% (36/36)
• Pseudomonas aeruginosa: 96.7% (29/30)
• Staphylococcus aureus: 97.6% (41/42)
• Enterococcus faecalis: 100% (30/30) |
  | AST Challenge Test (Agreement with Manual Preparation) | Essential Agreement (EA) of MICs: High agreement
  Category Agreement (CA): High agreement
  Discrepancies (vmj, maj): Low/none | Overall EA: 100% (1232/1232 evaluable MIC results within 1 two-fold dilution)
  Overall CA: 98.4% (4187/4254 SIR categorizations in agreement)
  Very Major discrepancy (vmj): 0
  Major discrepancy (maj): 0 |
  | Reproducibility Study | Best-case reproducibility: ≥95% (implied)
  Worst-case reproducibility: ≥89% (implied) | Best-case reproducibility: ≥99.8% (all panels combined)
  Worst-case reproducibility: ≥94.3% (all panels combined) |
  | Sample preparation for Quality Control | 100% of MIC values within CLSI/panel IFU QC range | 100% (all tested organisms and antimicrobial agents) |
  | Purity Plates Evaluation (Cross-contamination) | Absence of cross-contamination (100% monomicrobial growth) | 100% (453/453 Purity Plates showed monomicrobial growth) |

2. Sample sizes used for the test set and the data provenance:

• Preparation of Microbial Suspensions for AST:

  • Test Set Size: 132 microbial suspensions (36 E. coli, 30 Pseudomonas aeruginosa, 42 Staphylococcus aureus, 30 Enterococcus faecalis).
  • Data Provenance: Not explicitly stated (e.g., country of origin). The study involved three Colibrí instruments, suggesting internal validation. Retrospective or prospective is not specified, but the nature of the validation suggests prospective testing.

• AST Challenge Test:

  • Test Set Size: Different species: Enterobacterales (n=50 isolates), Staphylococcus (n=20 isolates), Streptococcus (n=12 isolates), Enterococcus (n=18 isolates), non-fermenters (n=10 isolates). Each processed by three Colibrí instruments, yielding varying numbers of MIC results and SIR categorizations across different panels (e.g., 2454 total MIC results for Enterobacterales on NM-NF50 panel).
  • Data Provenance: Not explicitly stated (e.g., country of origin). The study design implies prospective testing within the manufacturer's validation process.

• Reproducibility Study:

  • Test Set Size: 9 gram-positive and 9 gram-negative strains, processed on 3 Colibrí instruments over 3 days, with each condition tested in triplicate (total of 27 replicates for each strain-antimicrobial agent combination).
  • Data Provenance: Not explicitly stated (e.g., country of origin, retrospective or prospective). Implied prospective.

• Sample preparation for Quality Control:

  • Test Set Size: CLSI-recommended reference strains (Escherichia coli ATCC 25922, Pseudomonas aeruginosa ATCC 27853, Staphylococcus aureus ATCC 29213, Enterococcus faecalis ATCC 29212). The "No. MIC within QC range" indicates a total of 30 or 36 or 42 tests for each drug-organism combination, per three instruments.
  • Data Provenance: Not explicitly stated. Implied prospective.

• Purity Plates Evaluation:

  • Test Set Size: 453 purity plates (150 from AST Challenge, 162 from AST Reproducibility, 141 from Quality Control studies).
  • Data Provenance: Not explicitly stated. Implied prospective.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

• The ground truth for AST results (MICs and SIR categories) generally refers to the results obtained from a reference method. In this case, "Manual suspension was used as comparative method" for the AST Challenge Test. This implies that manually prepared suspensions, processed by the MicroScan WalkAway, served as the reference standard.
• The text does not specify the number of experts or their qualifications for establishing this manual ground truth. It mentions that three different technicians operated the Colibrí machines, but it doesn't detail the personnel for the manual comparative method or for interpreting the results as ground truth beyond the "FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria" and CLSI guidelines.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set:

• The document does not describe any expert adjudication process for the test set results. The comparison is made against a "manual result" (ground truth). The discrepancies (vmj, maj, min) are simply categorized and reported, implying a direct comparison without further expert review for resolving initial disagreements.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

• No, an MRMC study was not conducted. This device (Colibrí) is an automated system for sample preparation and does not involve human "readers" or "AI assistance" in the typical sense of image analysis for diagnosis. Its role is to automate a laboratory process, and the performance is measured against reference methods, not human interpretation.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

• The studies presented are primarily standalone (algorithm only without human-in-the-loop performance) in terms of the Colibrí device's automated functions. The device picks colonies, prepares suspensions, and records data automatically. The performance metrics (inoculum density, MIC accuracy, reproducibility, purity) assess the device's output against established standards and manual methods.
• While an operator designates colonies for picking, the act of preparation itself is automated and evaluated for its accuracy. The "manual suspension" used for comparison acts as the reference for the "algorithm only" performance of the Colibrí in producing the suspension.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc):

• The ground truth for the analytical studies combines reference methods/standards and established guidelines:
  • For microbial suspension concentration: Viable cell count (CFU/mL) against CLSI and FDA guidelines.
  • For AST results: Manual suspension preparation as the comparative method, and comparison of MICs and SIR categories against FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria and CLSI guideline M07.
  • For reproducibility: Comparison to the "mode result" (most frequent MIC value) and established reproducibility criteria (e.g., within one doubling dilution).
  • For Quality Control: CLSI-recommended QC ranges and MicroScan panel IFU values.
  • For Purity Plates: Visual assessment (implied) to confirm monomicrobial growth.

8. The sample size for the training set:

• This document describes performance validation studies for a medical device (Colibrí), not a machine learning model. Therefore, there is no "training set" in the context of data used to train an AI algorithm. The Colibrí is an automated instrument with pre-programmed functions, not a learning algorithm that requires a training dataset.

9. How the ground truth for the training set was established:

• As explained above, there is no training set for this device in the context of AI/ML.

Ask a specific question about this device

The Colibri™ is an automated in vitro diagnostic specimen preparation system for use with WASPLab® to prepare MALDI-TOF targets for the bioMérieux VITEK® MS or Bruker MALDI Biotyper® CA mass spectrometry systems for qualitative identification and microbial suspension for the bioMérieux VITEK® 2 Antimicrobial Susceptibility Testing (AST) system for qualitative testing of isolated colonies of gram-negative and gram-positive bacterial species grown on solid culture media.

The Colibri™ is an automated pre-analytical processor that nicks isolated colonies designated by the operator and uses a pipetting system to prepare MALDI-TOF MS (Matrix-Assisted Laser Desorption/lonization-Time of Flight Mass Spectrometry) target slides for bacterial identification and microbial suspension at known concentration for Antimicrobial Susceptibility Testing and purity assessment.

The Collori™ software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.

Bacterial suspensions for AST and purity plates are identified by barcode label.

The Colibri™ is intended for use by trained healthcare professionals in clinical laboratories in conjunction with other clinical and laboratory findings, including Gram staining, to aid in the diagnosis of bacterial infections.

The Colibri™ has not been validated for use in the identification or processing of yeast species, or mycobacteria.

The Colibrí is an instrument which automates the picking of selected colonies from plated media and prepares MALDI target slides for the bioMérieux VITEK MS or the Bruker MALDI Biotyper CA System that are used in clinical laboratories for identification and differentiation of organisms grown on plated media by Matrix-Assisted Laser Desorption/Tonization Time-of Flight Mass Spectrometry (MALDI-TOF MS). The Colibrí automates the preparation of microbial suspensions at known concentration for bioMérieux VITEK 2 System that is used in clinical laboratories for AST analyses. Moreover, the Colibrí is used for Purity Plates preparation for purity assessments.

The Colibrí includes the following components:

• . Colibrí instrument and software with on-board pipetting system and nephelometer
• Colibrí Primary Tubes ●
• . Colibrí Spreader
• Colibrí Daily Verification kit. ●

Colibri is designed to be used in conjunction with the WASPLab device for culture plate incubation and image analysis. After appropriate plate incubation, the operator selects the colonies from a digital image of culture media plate streaked with microbiological human specimen, available through WebApp software, the WASPLab User Interface.

The operator assigns the automatic ID or AST tasks to the isolated colonies to be processed. Then, the operator loads the plates in the Colibri where colonies are automatically picked, spotted on the target slide and overlayed with the matrix or suspended into the dedicated solution for the preparation of the microbial suspension for AST purposes (Secondary Tube).

When used in conjunction with the bioMérieux VITEK MS, the Colibrí can prepare the 48-spot target slides by performing the direct spotting of colonies. The calibrator used for quality control is manually applied by the operator at the end of the automated colony spotting. When used in conjunction with the Bruker MALDI Biotyper CA System, the Colibri can prepare either reusable 48-spot or disposable 96-spot targets by performing the Direct Transfer Sample Procedure. The BTS used for quality control is manually applied by the operator at the end of the automated colony spotting.

When used in conjunction with the bioMérieux VITEK 2, the Colibrí can prepare the microbial suspension at the proper concentration by direct colony suspension method. The onboard nephelometer allows the preparation of Secondary Tubes (AST suspensions) at the correct concentration and the Colibrí Spreader is used for Purity Plates preparation.

The Colibrí software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.

The traceability of prepared Secondary Tube and Purity Plates is maintained by dedicated labels applications.

Colibrí requires four different calibrations, one on the nephelometer, three on the cameras. None of these calibration activities require user intervention if not in terms of periodical cleaning of the mechanical component as described in the dedicated section of the User Manual. The Set-up calibration of nephelometer and camera units are performed during the device initial setup. Autocalibration is performed at the end of the initial set-up and periodically during the preventive maintenance to check that all the mechanical references can be found inside the positioning tolerances, that the I/Os are responsive. Run-time calibration is performed during the normal usage to automatically check the proper functioning of the Colibrí.

Colibrí requires a daily nephelometer verification to check the proper reading of suspensions at different turbidity values.

Acceptance Criteria and Device Performance for Colibrí

The Colibrí is an automated in vitro diagnostic specimen preparation system for use with WASPLab to prepare MALDI-TOF targets for microbial identification and microbial suspensions for Antimicrobial Susceptibility Testing (AST).

1. Table of Acceptance Criteria and Reported Device Performance

2. Sample Size Used for the Test Set and Data Provenance

The study utilized a test set consisting of various bacterial strains to evaluate the Colibrí's performance in preparing samples for MALDI-TOF identification and AST.

• Sample Size for Identification:
  • Gram-Negative Species: 5 species (4 Enterobacterales, 1 Non-fermenter) with 92 colonies picked and analyzed.
  • Gram-Positive Species: 5 species (2 Enterococcus, 2 Staphylococcus, 1 Streptococcus) with 200 colonies picked and analyzed.
  • Total for Identification: 292 colonies.
• Sample Size for AST:
  • Total AST Tested: 2720 tests (sum of "Total tested" column from AST summary tables).
  • This included: 1400 Enterobacterales, 200 Non-fermenters, 520 Staphylococci, 380 Enterococci, and 220 Streptococci.
• Data Provenance: The document does not explicitly state the country of origin of the data or whether it was retrospective or prospective. However, given that it is a submission to the U.S. Food & Drug Administration (FDA) by an Italian company (Copan WASP Srl, Brescia, Italy), the study was likely conducted to meet international regulatory standards, potentially with data from one or more clinical laboratories. The nature of the "Full workflow validation" study described suggests a prospective experimental design to assess the device's performance under controlled conditions.

3. Number of Experts Used to Establish Ground Truth and Qualifications

The document does not explicitly state the number of experts used or their specific qualifications for establishing ground truth. However, the study focuses on the accuracy of identification results against "expected strain identity" and comparison of MICs and SIR categories against "MICs obtained by bioMérieux VITEK 2 using manual sample preparation" and "FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria."

This implies that:

• For identification, the "expected strain identity" would have been established using a gold standard method, likely by skilled microbiologists or reference laboratories with established credentials in microbial identification.
• For AST, the "manual sample preparation" by bioMérieux VITEK 2 would serve as the comparator, and the interpretation of results would follow "FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria," likely applied by qualified laboratory personnel.

4. Adjudication Method

The document does not describe a formal adjudication method (e.g., 2+1, 3+1) for the test set.

• For identification, performance was calculated as "percentage of spotted colonies matching the expected identity," suggesting a direct comparison to a single reference (ground truth).
• For AST, the "MICs obtained by bioMérieux VITEK 2 using Colibrí were compared to the MICs obtained by bioMérieux VITEK 2 using manual sample preparation," indicating a direct method-to-method comparison. Discrepant SIR results were categorized, implying a systematic evaluation rather than a consensus-based adjudication in the traditional sense.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

There is no mention of a Multi-Reader Multi-Case (MRMC) comparative effectiveness study or any effect size of how much human readers improve with AI vs. without AI assistance. The Colibrí device is a pre-analytical automated system for sample preparation, not an AI-assisted diagnostic interpretation tool for human readers. Its function is to automate the preparation steps for subsequent analysis by other IVD systems (MALDI-TOF MS and VITEK 2 AST).

6. Standalone Performance Study

Yes, a standalone performance study was done for the algorithm/system. The "Full workflow validation" study directly assesses the Colibrí device's ability to accurately pick colonies and prepare samples for downstream analysis without human intervention in the picking and preparation steps themselves. The results presented for "VITEK MS Identification Result" and "AST summary of results" directly reflect the performance of the Colibrí-prepared samples when analyzed by the respective analytical instruments.

7. Type of Ground Truth Used

• For microbial identification: The ground truth was based on "expected strain identity," which implies confirmed identification results likely obtained through established, highly accurate microbiological methods serving as a reference.
• For AST: The ground truth for comparative purposes was "MICs obtained by bioMérieux VITEK 2 using manual sample preparation," interpreted according to "FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria."

8. Sample Size for the Training Set

The document explicitly states that the "Colibrí uses equally designed and developed hardware and software modules as the Colibrí System. Therefore, the performance of the predicate device represents the performance of the new device." It also mentions that "The results of the analytical studies were submitted to support the 510(k) Premarket Notifications K193138 and K220546." This indicates that the current submission (K223245) relies on the performance data of the predicate device (Colibrí System, K220546). The document does not provide details on the training set sample size for the Colibrí System's development.

9. How the Ground Truth for the Training Set Was Established

As the current submission relies on the predicate device's performance, the ground truth for any training set related to the predicate Colibrí System would have been established during its development and prior FDA submission (K220546). The current document does not provide details of how the ground truth for the predicate device's training set was established. However, based on the performance parameters listed (e.g., accuracy of colony picking, reproducibility of identification, accuracy of nephelometer), it likely involved:

• Microbial cultures: Using well-characterized microbial strains with confirmed identities.
• Manual reference methods: Comparing automated colony picking against visual confirmation by trained personnel.
• Reference AST methods: Comparing automated suspension preparation and subsequent AST results against established manual AST methods and interpretive criteria to assess agreement in MICs and categorical susceptibility (Susceptible, Intermediate, Resistant).

Ask a specific question about this device

The Collbri™ System is an in vitro diagnostic device comprised of the Collbri™ Preparation Station for use with the bioMérieux VITEK® MS or Bruker MALDI Biotyper® CA mass spectrometry systems for qualitative identification and with the bioMérieux VITEK® 2 Antimicrobial Susceptibility Testing (AST) system for qualitative testing of isolated colonies of gram-negative and gram-positive bacterial species grown on solid culture media. The Collbri™ System is a semi-automated pre-analytical processor that picks isolated colonies designated by the operator and uses a pipetting system to prepare MALDI-TOF MS (Matrix-Assisted Laser Desorption/Jonization-Time of Flight Mass Spectrometry) target slides for bacterial identification and microbial suspension at known concentration for Antimicrobial Susceptibility Testing and purity assessment.

The Colibrí software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.

Bacterial suspensions for AST and purity plates are identified by barcode label.

The Colibr™ System is intended for use by trained healthcare professionals in clinical laboratories in conjunction with other clinical and laboratory finding Gram staining, to aid in the diagnosis of bacterial infections.

The Collbri™ System has not been validated for use in the identification or processing of yeast species, Mocardia, or mycobacteria.

The Copan Colibrí System is designed to be used as an accessory of the downstream MALDI-TOF MS and antimicrobial susceptibility testing (AST) analyzers automating various manual steps in the workflow for the preparation of samples for the identification of isolated colonies and for AST of isolated colonies of gram-negative and gram-positive bacterial species grown on solid culture media.

The Colibrí System automates the preparation of MALDI target slides for the bioMérieux VITEK MS or the Bruker MALDI Biotyper CA System that are used in clinical laboratories for identification (ID) of organisms grown on plated media by Matrix-Assisted Laser Desorption/Jonization Time-of Flight Mass Spectrometry (MALDI-TOF MS). The Colibri System automates the preparation of microbial suspensions at known concentration for bioMérieux VITEK 2 System that is used in clinical laboratories for AST analyses. Moreover, the Colibri System is used for Purity Plates preparation for purity assessments.

The Colibrí System comprises the Colibrí Vision System and Colibrí Preparation Station hardware modules and pipette tips, Primary Tubes, Spreader and nephelometer Verification Kit as consumables. After appropriate plate incubation, the operator using the graphical User Interface (Image Reading Interface) chooses the plates exhibiting adequate growth and selects the isolated colonies to be processed assigning the automatic ID or AST tasks. By using the Colibrí Vision System, specific colonies to be picked are designated by the operator on a digital plate. The Operator manually loads the plates in the Colibri Preparation Station where colonies are automatically picked, spotted on the target slide and overlayed with the matrix or suspended into the dedicated solution for the preparation of the microbial suspension for AST purposes (Secondary Tube).

When used in conjunction with the bioMérieux VITEK MS, the Colibrí System can prepare the 48spot target slides by performing the direct spotting of colonies. The calibrator used for quality control is manually applied by the operator at the end of the automated colony spotting. When used in conjunction with the Bruker MALDI Biotyper CA System, the Colibrí System can prepare either reusable 48-spot or disposable 96-spot targets by performing the Direct Transfer Sample Procedure. The BTS used for quality control is manually applied by the operator at the automated colony spotting.

When used in conjunction with the bioMérieux VITEK 2, the Colibrí System can prepare the microbial suspension at the proper concentration by direct colony suspension method. The onboard nephelometer allows the preparation of Secondary Tubes (AST suspensions) at the correct concentration and the Colibrí Spreader is used for Purity Plates preparation.

Copan WASP S.r.l., Traditional 510(k)- Colibrí System

The Colibrí software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.

The traceability of prepared Secondary Tube and Purity Plates is maintained by dedicated labels applications.

Colibri System requires four different calibrations, one on the nephelometer, three on the cameras. None of these calibration activities require user intervention if not in terms of periodical cleaning of the mechanical component as described in the dedicated section of the User Manual. The Set-up calibration of nephelometer and camera units positioned on the Colibrí Vision System and on the Colibrí Preparation Station are performed during the device initial setup. Auto-calibration is performed at the end of the initial set-up and periodically during the preventive maintenance to check that, in the Collbrí Preparation all the mechanical references can be found inside the positioning tolerances, that the I/Os are responsive. Run-time calibration is performed during the normal usage to automatically check the proper functioning of the Colibrí Vision System and the Colibrí Preparation Station.

Colibrí System requires a daily nephelometer verification to check the proper reading of suspensions at different turbidity values.

The provided text describes the performance data for the Colibrí System, an in vitro diagnostic device. The acceptance criteria and performance are primarily focused on its ability to accurately prepare microbial suspensions for Antimicrobial Susceptibility Testing (AST) and for MALDI-TOF MS identification, compared to manual methods.

Here's an attempt to extract the requested information. Please note that the document is a 510(k) summary, which often provides summarized performance data rather than detailed study protocols. Therefore, some information might be explicitly stated as "not applicable" or inferred based on common practices for such device clearances.

1. A table of acceptance criteria and the reported device performance

The document outlines several analytical studies. While explicit "acceptance criteria" are not always presented as target percentages, the performance results are given, implying that these results met the internal pre-defined acceptance thresholds for substantial equivalence.

2. Sample sizes used for the test set and the data provenance

• Nephelometer Calibration Verification: 300 suspensions were prepared (20 suspensions for each of 5 concentrations, across 3 operators and 3 Colibrí systems, though the calculation isn't directly 2053*3).
• Pipettor Trueness and Precision: 10 measurements for each of 4 volumes, across 3 Colibrí System pipettors (10 * 4 * 3 = 120 measurements).
• E. coli Suspensions Preparation Verification: The exact number of suspensions isn't provided, but it states "A variable number of colonies was selected... to create different suspensions... Three Colibrí Systems run by three different operators were included."
• Colony Picking and Microbial Suspensions for AST: 6 bacterial species (3 Gram-Negative and 3 Gram-Positive) grown in 2 polymicrobial mixtures on different culture media.
• AST Challenge Test:
  • Total Tested: 5991 (across various antibiotics and organism groups within the challenge test).
  • Evaluated MIC results: 1883 evaluable MIC results.
  • Organism groups: Enterobacterales (n=62 strains), Staphylococcus (n=16 strains), Streptococcus (n=30 strains), Enterococcus (n=16 strains), and non-fermenters (n=32 strains).
  • Strains: Both susceptible and resistant strains, exhibiting a range of on-scale MIC values.
  • Media: Trypticase Soy Agar + 5% Sheep Blood, MacConkey Agar, and Columbia agar + 5% sheep blood.
  • Incubation times: Varied (e.g., 14h, 24h for Enterobacterales/Non-fermenters, 18h for Staphylococcus/Enterococcus/Streptococcus).
  • Data Provenance: Not explicitly stated (e.g., country of origin, retrospective/prospective). However, given it's a 510(k) submission for a medical device manufacturer (Copan WASP S.r.l., Italy), it's highly likely to be internal, prospective studies conducted at their facilities or collaborator sites.
• Reproducibility Study: Each microorganism was tested with the appropriate antibiotic panel, with each condition tested in triplicate, for a total of 81 replicates for each combination strain-antimicrobial agent across three Colibrí Systems and three operators over three days.
• Purity Plate Growth: 2,364 purity plates.
• QC Sample Preparation: Conducted daily at the beginning of the working session on each instrument involved in the Analytical Studies.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

• This information is not explicitly provided in the document.
• For the AST Challenge Test, the "ground truth" (or comparator method) was the MICs obtained by the bioMérieux VITEK 2 using manual sample preparation, interpreted according to FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria. This implies the ground truth relies on established, validated laboratory methods and interpretations, rather than subjective expert consensus.
• For other analytical studies (e.g., nephelometer accuracy, pipettor precision), the ground truth generally relies on quantitative measurements using calibrated instruments and standard protocols (e.g., viable cell counts for McFarland turbidity verification, gravimetric measurements for pipettor accuracy).

4. Adjudication method for the test set

• This is not applicable in the context of this device's performance validation. The device automates a pre-analytical step. The performance is assessed by comparing its output (prepared microbial suspensions) to a reference method (manual preparation for VITEK 2). It's not a diagnostic AI system requiring expert adjudication of image interpretations or clinical diagnoses.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• No, an MRMC comparative effectiveness study was not done. This type of study is relevant for AI systems that assist human readers in interpreting medical images (e.g., radiology AI). The Colibrí System is a laboratory automation device for preparing samples. It does not involve human "readers" of AI outputs in a diagnostic context that would require such a study design.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

• The performance data are essentially standalone (algorithm/device only) in terms of its ability to perform the physical process of colonial picking and suspension preparation. The device's output is then fed into other IVD analyzers (VITEK 2, bioMérieux VITEK MS, Bruker MALDI Biotyper CA System) for identification and AST.
• The comparison in the AST Challenge Test is between the Colibrí System's automated preparation and manual preparation which is the existing standard. So, it's comparing automated device output to a manual, human-executed process, where the subsequent analysis (VITEK 2) is the same. The data provided (EA, CA, error rates) represent the performance of the Colibrí system's prepared samples, which is a form of standalone performance.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

• Comparatative Ground Truth/Reference Method: For the AST Challenge Test, the ground truth was derived from the MICs obtained by the bioMérieux VITEK 2 using manual sample preparation, interpreted according to FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria. This is a well-established and standardized laboratory reference method.
• Quantitative Ground Truth: For other analytical studies, the ground truth was based on quantitative laboratory measurements, such as:
  • Viable cell counts (CFU/mL) for confirming bacterial concentration for nephelometry.
  • Gravimetric measurements for pipetting accuracy.
  • Visual inspection of purity plates and accepted microbiological methods to ensure monomicrobial suspensions and absence of contamination.

8. The sample size for the training set

• The document describes performance validation studies, not product development or AI model training. Therefore, information about a "training set" (in the context of machine learning) is not applicable or provided. This device automates a physical process, not a machine learning model that needs training data in the traditional sense. The "training" of the device likely refers to physical calibration and quality control.

9. How the ground truth for the training set was established

• As per point 8, the concept of a "training set" for an AI/machine learning model is not applicable to the description of this device's validation. The device's mechanics and software are validated against established engineering and microbiology standards.

Ask a specific question about this device