Search Results

The BD Kiestra IdentifA module is an automated in vitro diagnostic specimen preparation system for use with the BD Kiestra Laboratory Automation Solution to prepare MALDI targets for the Bruker MALDI Biotyper System (CA, sirius CA, or sirius one CA) for the qualitative identification of microorganisms using matrix-assisted laser desorption/ionization-time of flight mass spectrometry (MALDI-TOF MS) analysis of colonies grown on plated culture media from human specimens.

The BD Kiestra IdentifA is indicated for use in the clinical laboratory with the BD Kiestra ReadA Compact and Bruker MALDI Biotyper System (CA, sirius one CA) to aid in the diagnosis of bacterial and fungal infections.

The BD Kiestra™ IdentifA automates preparation of MALDI targets for the Bruker MALDI Biotyper® CA System, sirius CA system, and/or sirius one CA System that are used in clinical laboratories for identification and differentiation of organisms grown on plated media by Matrix-Assisted Laser Desorption/lonization Time-of-Flight Mass Spectrometry (MALDI-TOF MS). The system comprises of the BD Kiestra™ IdentifA module (including the associated software and onboard nephelometers and pipetting system), formic acid and automation-compatible transfer vials (for HCCA matrix and Bacterial Test Standard (BTS), which are obtained directly from Bruker and manually transferred to the vials for use on the instrument), consumables (pipette tips and cuvette arrays for preparation of organism suspensions and fluid movement), and nephelometer calibration standards (McFarland standard vials for measuring turbidity of microbial suspensions).

When identification of an organism growing on a culture medium plate is required, a technologist designates specific colonies for picking by the BD Kiestra™ IdentifA module using a digital image of the plate obtained using the BD Kiestra™ ReadA Compact module. The BD Kiestra™ IdentifA automatically suspends the designated colonized water and uses an onboard nephelometer to determine the resulting turbidity. The organism concentration is adjusted automatically by picking additional designated colonies or by appropriate dilution of the suspension to achieve a turbidity within a targeted range of McFarland values. Based on the final organism concentration, the BD Kiestra™ IdentifA pipets one or more aliquots of the microbial suspension onto a MALDI target (either reusable 48-spot or disposable 96-spot targets) and dries the spots at elevated temperature.

The BD Kiestra™ IdentifA performs the extended Direct Transfer (eDT) Sample Preparation Procedure from Bruker whereby the instrument overlays the dried sample spot on the MALDI target with formic acid and matrix. The BD Kiestra™ IdentifA also spots the BTS used for quality control of MALDI-TOF MS organism identification. Once spots are dry, the technologist manually removes the target from the BD Kiestra™ IdentifA and loads it into the Bruker MALDI Biotyper® System for analysis. Information regarding the location of each sample and BTS on the targets and the associated MALDI-TOF MS results are transmitted between the BD Kiestra™ IdentifA and Bruker MALDI Biotyper® System via the Synapsys Informatics, the main software interface, and the BD Kiestra™ BeA, the data interface hub module that communicates with all the other modules including the BD Kiestra™ IdentifA. In addition to preparing the MALDI target, if requested, the BD Kiestra™ IdentifA will also dilute the organism suspension to a standardized turbidity of 0.5 McFarland.

Modules of the BD Kiestra™ System each have their own operating software that communicates via the central BeA data interface hub module with the Synapsys user interface which in turn sends and receives information to/from the Laboratory Information System (LIS).

The BD KiestraTM ReadA or ReadA Compact module is required for use in conjunction with the BD Kiestra™ IdentifA module for image capture. Culture plate incubation may be done offline and then moved to the BD Kiestra™ ReadA or ReadA Compact module for imaging, or incubation can be done in the ReadA and then remain in the ReadA for imaging. Additional software modules (BD Synapsys Informatics and BD Kiestra™ BeA) are also required for the function of the BD Kiestra™ IdentifA, and these modules reside on the BD Kiestra™ Laboratory Automation Solution. The digital image is used by the BD Kiestra™ IdentifA for image analysis and colony designation by the operator.

The provided text outlines the performance characteristics and acceptance criteria for the BD Kiestra IdentifA device. This device is an automated in vitro diagnostic specimen preparation system for MALDI-TOF MS analysis of microorganisms. The submission focuses on demonstrating substantial equivalence to a predicate device (K191964) with the addition of the BD Kiestra ReadA camera system and the Bruker MALDI Biotyper sirius CA and sirius one CA systems.

Here's a breakdown of the requested information based on the provided document:

Acceptance Criteria and Reported Device Performance

The acceptance criteria are implicitly defined by the successful outcomes of the studies demonstrating equivalency or accuracy. The performance is reported as meeting these criteria.

Table 1: Acceptance Criteria and Reported Device Performance

Study Details:

2. Sample Size and Data Provenance:

• Study 1 (Colony Picking Accuracy):
  • Sample Size: 205 mixed cultures (Escherichia coli and Streptococcus pyogenes). 1230 colonies were picked, and 408 isolate MALDI ID results were analyzed.
  • Data Provenance: Not explicitly stated regarding country of origin, but generally, clinical device studies for FDA submissions are conducted in the US or in countries with comparable regulatory and quality standards. The data provenance is prospective as it involves experimental setup and data collection for the purpose of the study.
• Study 2 (Camera Equivalency):
  • Sample Size: 15 microorganisms, 3 dilutions (10^3, 10^4, 10^5 CFU/mL), 2 solid media types (MAC, TSA II).
    • BD Kiestra ReadA (25MP): 270 plate images (15 organisms x 3 dilutions x 2 media types x 3 systems).
    • BD Kiestra ReadA Compact (5MP): 90 plate images (15 organisms x 3 dilutions x 2 media types x 1 system).
  • Data Provenance: Not explicitly stated regarding country of origin. Prospective.
• Study 3 (Identification Accuracy - Bruker MALDI Biotyper systems):
  • Sample Size: 37 Gram-positive bacteria, Gram-negative bacteria, and yeast organisms. Two spots per suspension, tested for three days for repeatability.
  • Data Provenance: Not explicitly stated regarding country of origin. Prospective.
• Quality Control (BTS Workflow):
  • Sample Size: 102 spots (standard workflow) and 430 spots (modified workflow).
  • Data Provenance: Not explicitly stated regarding country of origin. Prospective.

3. Number of Experts Used and Qualifications:

• Study 2 (Camera Equivalency): "Three operators representative of the intended user interpreted the culture plate images and assigned a minimal morphological identification (MMI) code."
• Other Studies: The document does not specify the number or qualifications of experts (e.g., microbiologists, lab technologists) involved in establishing ground truth or performing manual inspections (like in Study 1 for visual confirmation of picking). However, for a device in microbiology, it is standard that trained laboratory personnel or microbiologists perform these tasks.

4. Adjudication Method for the Test Set:

• The document does not explicitly describe a formal adjudication method (e.g., 2+1, 3+1 consensus) for the test set results.
• For Study 2, the "Minimal Morphological Identification (MMI) code" was assigned by "three operators," and agreement was measured. It implies independent assessment rather than a consensus/adjudication process.
• For other studies, the "expected identification" or "Bruker interpretive criteria" served as the reference standard, rather than an adjudicated human consensus.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:

• No MRMC comparative effectiveness study was done to show how human readers improve with AI vs. without AI assistance. This device is an automated preparation system, aiding in sample processing for subsequent human evaluation via MALDI-TOF MS results, rather than an AI diagnostic interpretation tool for images. Study 2 did involve multiple readers evaluating images from different camera systems, but it focused on camera equivalency for MMI interpretation, not on AI assistance for readers.

6. Standalone Performance:

• The device (BD Kiestra IdentifA) is an automated system for sample preparation. Its "standalone performance" is implicitly demonstrated through the accuracy of its automated functions (colony picking, suspension preparation, spotting) and the subsequent successful identification by the Bruker MALDI Biotyper systems, which are separate components. The studies confirm the accuracy of the BD Kiestra IdentifA's automated steps in preparing samples suitable for the MALDI-TOF MS system. The MALDI-TOF MS system itself has its own performance characteristics.

7. Type of Ground Truth Used:

• Study 1 (Colony Picking Accuracy):
  • Initial confirmation of picking: Visual inspection of plates and comparison to original digital images.
  • Confirmation of identification success: Bruker MALDI identification using established interpretive criteria (Log(score) values).
• Study 2 (Camera Equivalency): Minimal Morphological Identification (MMI) codes assigned by trained operators. This is a form of expert interpretation/consensus (though agreement was measured, not a formal consensus process).
• Study 3 (Identification Accuracy - Bruker MALDI Biotyper systems): Bruker interpretive criteria for species identification (Log(score) values). This relies on the established performance of the MALDI-TOF MS system and comparison of results between different versions of the system.
• Quality Control (BTS Workflow): Bruker interpretive criteria for species identification (Log(score) values) for the Bacterial Test Standard (BTS).

In summary, the ground truth for microorganism identification is based on the established MALDI-TOF MS interpretive criteria (log scores for identification confidence), as the device's role is to prepare samples for this established identification method. For colony picking and image interpretation, the ground truth is based on visual inspection, comparison to expected outcomes, and expert human interpretation (for MMI).

8. Sample Size for the Training Set:

• The document does not mention a "training set" in the context of an AI/ML model for the BD Kiestra IdentifA. This device is described as an automated specimen preparation system utilizing robotics, nephelometry, and image processing, rather than a system based on machine learning that requires a separate training phase. The studies described are performance validation studies.

9. How the Ground Truth for the Training Set was Established:

• As the document does not describe an AI/ML model with a distinct training set, this question is not applicable. The device's functionality is based on predefined algorithms and hardware performance, validated against established laboratory methods and interpretive criteria, as detailed in point 7.

Ask a specific question about this device

The Colibri™ is an automated in vitro diagnostic specimen preparation system for use with WASPLab® to prepare MALDI-TOF targets for the bioMérieux VITEK® MS or Bruker MALDI Biotyper® CA mass spectrometry systems for qualitative identification and microbial suspension for the bioMérieux VITEK® 2 Antimicrobial Susceptibility Testing (AST) system for qualitative testing of isolated colonies of gram-negative and gram-positive bacterial species grown on solid culture media.

The Colibri™ is an automated pre-analytical processor that nicks isolated colonies designated by the operator and uses a pipetting system to prepare MALDI-TOF MS (Matrix-Assisted Laser Desorption/lonization-Time of Flight Mass Spectrometry) target slides for bacterial identification and microbial suspension at known concentration for Antimicrobial Susceptibility Testing and purity assessment.

The Collori™ software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.

Bacterial suspensions for AST and purity plates are identified by barcode label.

The Colibri™ is intended for use by trained healthcare professionals in clinical laboratories in conjunction with other clinical and laboratory findings, including Gram staining, to aid in the diagnosis of bacterial infections.

The Colibri™ has not been validated for use in the identification or processing of yeast species, or mycobacteria.

The Colibrí is an instrument which automates the picking of selected colonies from plated media and prepares MALDI target slides for the bioMérieux VITEK MS or the Bruker MALDI Biotyper CA System that are used in clinical laboratories for identification and differentiation of organisms grown on plated media by Matrix-Assisted Laser Desorption/Tonization Time-of Flight Mass Spectrometry (MALDI-TOF MS). The Colibrí automates the preparation of microbial suspensions at known concentration for bioMérieux VITEK 2 System that is used in clinical laboratories for AST analyses. Moreover, the Colibrí is used for Purity Plates preparation for purity assessments.

The Colibrí includes the following components:

• . Colibrí instrument and software with on-board pipetting system and nephelometer
• Colibrí Primary Tubes ●
• . Colibrí Spreader
• Colibrí Daily Verification kit. ●

Colibri is designed to be used in conjunction with the WASPLab device for culture plate incubation and image analysis. After appropriate plate incubation, the operator selects the colonies from a digital image of culture media plate streaked with microbiological human specimen, available through WebApp software, the WASPLab User Interface.

The operator assigns the automatic ID or AST tasks to the isolated colonies to be processed. Then, the operator loads the plates in the Colibri where colonies are automatically picked, spotted on the target slide and overlayed with the matrix or suspended into the dedicated solution for the preparation of the microbial suspension for AST purposes (Secondary Tube).

When used in conjunction with the bioMérieux VITEK MS, the Colibrí can prepare the 48-spot target slides by performing the direct spotting of colonies. The calibrator used for quality control is manually applied by the operator at the end of the automated colony spotting. When used in conjunction with the Bruker MALDI Biotyper CA System, the Colibri can prepare either reusable 48-spot or disposable 96-spot targets by performing the Direct Transfer Sample Procedure. The BTS used for quality control is manually applied by the operator at the end of the automated colony spotting.

When used in conjunction with the bioMérieux VITEK 2, the Colibrí can prepare the microbial suspension at the proper concentration by direct colony suspension method. The onboard nephelometer allows the preparation of Secondary Tubes (AST suspensions) at the correct concentration and the Colibrí Spreader is used for Purity Plates preparation.

The Colibrí software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.

The traceability of prepared Secondary Tube and Purity Plates is maintained by dedicated labels applications.

Colibrí requires four different calibrations, one on the nephelometer, three on the cameras. None of these calibration activities require user intervention if not in terms of periodical cleaning of the mechanical component as described in the dedicated section of the User Manual. The Set-up calibration of nephelometer and camera units are performed during the device initial setup. Autocalibration is performed at the end of the initial set-up and periodically during the preventive maintenance to check that all the mechanical references can be found inside the positioning tolerances, that the I/Os are responsive. Run-time calibration is performed during the normal usage to automatically check the proper functioning of the Colibrí.

Colibrí requires a daily nephelometer verification to check the proper reading of suspensions at different turbidity values.

Acceptance Criteria and Device Performance for Colibrí

The Colibrí is an automated in vitro diagnostic specimen preparation system for use with WASPLab to prepare MALDI-TOF targets for microbial identification and microbial suspensions for Antimicrobial Susceptibility Testing (AST).

1. Table of Acceptance Criteria and Reported Device Performance

2. Sample Size Used for the Test Set and Data Provenance

The study utilized a test set consisting of various bacterial strains to evaluate the Colibrí's performance in preparing samples for MALDI-TOF identification and AST.

• Sample Size for Identification:
  • Gram-Negative Species: 5 species (4 Enterobacterales, 1 Non-fermenter) with 92 colonies picked and analyzed.
  • Gram-Positive Species: 5 species (2 Enterococcus, 2 Staphylococcus, 1 Streptococcus) with 200 colonies picked and analyzed.
  • Total for Identification: 292 colonies.
• Sample Size for AST:
  • Total AST Tested: 2720 tests (sum of "Total tested" column from AST summary tables).
  • This included: 1400 Enterobacterales, 200 Non-fermenters, 520 Staphylococci, 380 Enterococci, and 220 Streptococci.
• Data Provenance: The document does not explicitly state the country of origin of the data or whether it was retrospective or prospective. However, given that it is a submission to the U.S. Food & Drug Administration (FDA) by an Italian company (Copan WASP Srl, Brescia, Italy), the study was likely conducted to meet international regulatory standards, potentially with data from one or more clinical laboratories. The nature of the "Full workflow validation" study described suggests a prospective experimental design to assess the device's performance under controlled conditions.

3. Number of Experts Used to Establish Ground Truth and Qualifications

The document does not explicitly state the number of experts used or their specific qualifications for establishing ground truth. However, the study focuses on the accuracy of identification results against "expected strain identity" and comparison of MICs and SIR categories against "MICs obtained by bioMérieux VITEK 2 using manual sample preparation" and "FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria."

This implies that:

• For identification, the "expected strain identity" would have been established using a gold standard method, likely by skilled microbiologists or reference laboratories with established credentials in microbial identification.
• For AST, the "manual sample preparation" by bioMérieux VITEK 2 would serve as the comparator, and the interpretation of results would follow "FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria," likely applied by qualified laboratory personnel.

4. Adjudication Method

The document does not describe a formal adjudication method (e.g., 2+1, 3+1) for the test set.

• For identification, performance was calculated as "percentage of spotted colonies matching the expected identity," suggesting a direct comparison to a single reference (ground truth).
• For AST, the "MICs obtained by bioMérieux VITEK 2 using Colibrí were compared to the MICs obtained by bioMérieux VITEK 2 using manual sample preparation," indicating a direct method-to-method comparison. Discrepant SIR results were categorized, implying a systematic evaluation rather than a consensus-based adjudication in the traditional sense.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

There is no mention of a Multi-Reader Multi-Case (MRMC) comparative effectiveness study or any effect size of how much human readers improve with AI vs. without AI assistance. The Colibrí device is a pre-analytical automated system for sample preparation, not an AI-assisted diagnostic interpretation tool for human readers. Its function is to automate the preparation steps for subsequent analysis by other IVD systems (MALDI-TOF MS and VITEK 2 AST).

6. Standalone Performance Study

Yes, a standalone performance study was done for the algorithm/system. The "Full workflow validation" study directly assesses the Colibrí device's ability to accurately pick colonies and prepare samples for downstream analysis without human intervention in the picking and preparation steps themselves. The results presented for "VITEK MS Identification Result" and "AST summary of results" directly reflect the performance of the Colibrí-prepared samples when analyzed by the respective analytical instruments.

7. Type of Ground Truth Used

• For microbial identification: The ground truth was based on "expected strain identity," which implies confirmed identification results likely obtained through established, highly accurate microbiological methods serving as a reference.
• For AST: The ground truth for comparative purposes was "MICs obtained by bioMérieux VITEK 2 using manual sample preparation," interpreted according to "FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria."

8. Sample Size for the Training Set

The document explicitly states that the "Colibrí uses equally designed and developed hardware and software modules as the Colibrí System. Therefore, the performance of the predicate device represents the performance of the new device." It also mentions that "The results of the analytical studies were submitted to support the 510(k) Premarket Notifications K193138 and K220546." This indicates that the current submission (K223245) relies on the performance data of the predicate device (Colibrí System, K220546). The document does not provide details on the training set sample size for the Colibrí System's development.

9. How the Ground Truth for the Training Set Was Established

As the current submission relies on the predicate device's performance, the ground truth for any training set related to the predicate Colibrí System would have been established during its development and prior FDA submission (K220546). The current document does not provide details of how the ground truth for the predicate device's training set was established. However, based on the performance parameters listed (e.g., accuracy of colony picking, reproducibility of identification, accuracy of nephelometer), it likely involved:

• Microbial cultures: Using well-characterized microbial strains with confirmed identities.
• Manual reference methods: Comparing automated colony picking against visual confirmation by trained personnel.
• Reference AST methods: Comparing automated suspension preparation and subsequent AST results against established manual AST methods and interpretive criteria to assess agreement in MICs and categorical susceptibility (Susceptible, Intermediate, Resistant).

Ask a specific question about this device

The Colibrí System is an in vitro diagnostic device comprised of the Colibrí Vision System and Collbrí Preparation Station for use with the bioMérieux VITEK MS or Bruker MALDI Biotyper CA mass spectrometry systems for qualitative identification of isolated colonies of Gram-negative bacterial species grown on solid culture media. The Collbri System is a semi-automated pre-analytical processor that picks isolated by the operator and uses a pipetting system to prepare MALDI-TOF MS (Matrix-Assisted Laser Desorption/Ionization-Time Of Flight Mass Spectrometry) target slides. The Colibri software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzer.

The Colibrí System is intended for use by trained healthcare professionals in clinical laboratories in conjunction with other clinical and laboratory finding Gram staining, to aid in the diagnosis of bacterial infections.

The Colibrí System has not been validated for use in identification of yeast species.

The Copan Colibrí System is designed to be used as an accessory of the downstream MALDI-TOF analyzers automating various manual steps in the workflow for the preparation of samples for the identification of isolated colonies of microorganisms cultured from the human body.

The Colibrí System automates the preparation of MALDI target slides for the bioMérieux VITEK MS or the Bruker MALDI Biotyper CA System that are used in clinical laboratories for identification and differentiation of organisms grown on plated media by Matrix-Assisted Laser Desorption/Ionization Time-of Flight Mass Spectrometry (MALDI-TOF MS). The system comprises the Colibrí Vision System and Colibrí Preparation Station and pipette tips as consumables. After appropriate plate incubation, the operator using the graphical User Interface (Image Reading Interface) chooses the plates exhibiting adequate growth and selects the isolated colonies to be processed assigning the automatic ID tasks. By using the Collbrí Vision System, specific colonies to be picked are designated by the operator on a digital plate. The Operator manually loads the plates in the Colibri Preparation Station where colonies are automatically picked, spotted on the target slide and overlayed with the matrix.

When used in conjunction with the bioMérieux VITEK MS, the Colibrí System can prepare the 48spot target slides by performing the direct spotting of colonies. The calibrator used for quality control is manually applied by the operator at the end of the automated colony spotting. When used in conjunction with the Bruker MALDI Biotyper CA System, the Colibrí System can prepare either reusable 48-spot or disposable 96-spot targets by performing the Direct Transfer Sample Procedure. The BTS used for quality control is manually applied by the operator at the and of the automated colony spotting.

The Colibrí software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.

Colibri System requires three different calibrations. None of these calibration activities require user intervention if not in terms of periodical cleaning of the mechanical component as described in the dedicated section of the User Manual. Set-up calibration is performed during the device initial setup for the camera units positioned on the Colibrí Vision System and on the Colibrí Preparation Station. Auto-calibration is performed at the end of the initial set-up and periodically during the preventive maintenance to check that, in the Colibri Preparation Station, all the mechanical references can be found inside the positioning tolerances, that the I/Os are responsive. Run-time calibration is performed during the normal usage to automatically check the proper functioning of the Colibrí Vision System and the Colibri Preparation Station.

The provided text describes the performance data for the Colibrí System, an in vitro diagnostic device. The acceptance criteria are implicitly defined by the performance observed in various analytical studies, with the goal of demonstrating substantial equivalence to predicate devices for qualitative identification of isolated colonies of Gram-negative and Gram-positive bacterial species.

Here's an analysis of the acceptance criteria and study proving the device meets them:

1. Table of Acceptance Criteria (Implicit) and Reported Device Performance

The document does not explicitly state pre-defined acceptance criteria in a dedicated table with specific thresholds. However, based on the studies conducted, the implicit acceptance criteria appear to be:

• High agreement with expected identification (ground truth).
• Absence of wrong identifications (false positives).
• Maintenance of performance across different conditions (e.g., positional effects, culture age, different MALDI-TOF MS systems and target types, different operators, different Colibrí systems).
• Comparable performance to manual preparation.
• No cross-contamination.

Here's a table summarizing the reported device performance, which serves as the evidence that these implicit criteria were met. The percentages below represent the agreement with the expected ID (ground truth) for high-confidence results where available, or overall agreement as calculated by the study.

2. Sample Sizes Used for the Test Set and Data Provenance

• Test Set (Analytical Studies):

  • Colony Picking: 1390 spots (VITEK MS) and 1690 spots (Bruker MALDI Biotyper CA).
  • Positional Effect: 864 spots (VITEK MS), 864 spots (Bruker MALDI Biotyper CA US IVD 48 Spot), 1692 spots (Bruker MALDI Biotyper CA MBT Biotarget 96 US IVD).
  • Inclusivity: 392 spots/strains (VITEK MS) and 468 spots/strains for each Bruker MALDI Biotyper CA system (US IVD 48 Spot and MBT Biotarget 96 US IVD).
  • Specificity: 20 spots (VITEK MS) and 20 spots (Bruker MALDI Biotyper CA).
  • Reproducibility: 1800 spots (VITEK MS) and 1800 spots (Bruker MALDI Biotyper CA).
  • Cross-Contamination: 572 spots (VITEK MS) and 686 spots (Bruker MALDI Biotyper CA total for both systems).
  • Colony Stability: 576 spots (VITEK MS) and 1440 spots (Bruker MALDI Biotyper CA).

• Data Provenance: The document generally indicates "on-panel" and "off-panel" strains, often referring to common clinical isolates in the US. No explicit country of origin for the clinical samples/strains themselves is stated beyond generic "human specimens" or "commonly isolated Gram-positive and Gram-negative species in the US". The studies were conducted by a single operator (Inclusivity, Specificity, Cross-Contamination, Colony Stability) or two operators (Reproducibility) on multiple Colibrí systems (Reproducibility, Colony Picking, Inclusivity, Cross-Contamination) in laboratory settings. The studies are prospective as they are designed experiments to evaluate the device performance.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

• The document implies that the ground truth for identification was established by the expected strain identity of the stock cultures used in the analytical studies. This means the identities of the bacterial strains were known beforehand (e.g., ATCC strains or well-characterized lab strains).
• For the performance results themselves, identification was based on the output of the VITEK MS or Bruker MALDI Biotyper CA systems, which generate confidence values or log scores. The interpretation of these scores (e.g., what constitutes a "correct single choice") is intrinsic to the performance specifications of these established MALDI-TOF MS systems.
• No human experts (e.g., radiologists) were used to establish the ground truth for these microbial identification studies. The "ground truth" is the presumed identity of the bacterial strain used in the experiment.

4. Adjudication Method for the Test Set

• Not applicable in the conventional sense. Since the ground truth for the test set was the expected strain identity of known bacterial cultures, there was no need for human expert adjudication to resolve discrepancies in interpretation. The output of the MALDI-TOF MS system for each sample was compared directly to the known identity of the inoculated strain. Discrepancies (low confidence, no ID, wrong ID) were noted and analyzed against the overall performance.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done

• No. An MRMC study is typically performed for image-based diagnostic aids where multiple human readers interpret cases with and without AI assistance. This study involves microorganism identification, a laboratory process, not direct human interpretation of images for diagnosis.
• The study compares the performance of the automated Colibrí System to manual preparation (the standard of care, as listed in the predicate device comparison tables), which is a different type of comparison than human readers with/without AI assistance.
• Effect size of human reader improvement: Not applicable, as no human reader study was conducted. The comparison is between an automated sample preparation method (Colibrí) and a manual sample preparation method for laboratory identification.

6. If a Standalone (i.e. algorithm only without human-in-the loop performance) was Done

• Yes, indirectly. The Colibrí System is a semi-automated pre-analytical processor. While an operator designates the colony, the picking, spotting, and matrix application are automated.
• The performance metrics provided (e.g., % agreement, no wrong ID) are for the Colibrí System's ability to prepare samples for subsequent MALDI-TOF MS analysis, which then generates the identification result. The Colibrí software records location and transmits information to the MALDI-TOF MS analyzer.
• The study evaluated the effectiveness of the automated sample preparation step compared to manual preparation, which directly impacts the downstream MALDI-TOF MS identification. However, the identification itself is performed by the MALDI-TOF MS analyzer, not the Colibrí system's "algorithm" in isolation, except for tracking sample positions. The Colibrí's key "algorithm" is in its vision system and robotic picking/spotting, which impacts the quality of the sample for the MALDI-TOF MS.

7. The Type of Ground Truth Used

• Expected Strain Identity: The ground truth for the analytical studies was the known, expected identity of the bacterial strains used. These were presumably well-characterized laboratory strains (e.g., ATCC cultures) with a confirmed identity. This is a highly controlled form of ground truth.

8. The Sample Size for the Training Set

• Not explicitly stated in terms of a "training set" for an AI/machine learning model used within Colibrí's core functions. The Colibrí System is described as a "semi-automated pre-analytical processor" that uses "a pipetting system" and "Colibrí software records the identity of each sample and its position". It's unclear if the "Colibrí Vision System" employs complex machine learning that would require a dedicated training set beyond basic image processing for colony detection and localization.
• The document implies that the system is automating a manual process, meaning its "training" pertains more to engineering and calibration rather than a machine learning model for diagnosis. The performance studies focus on the system's accuracy in physical manipulation and compatibility with existing MALDI-TOF MS systems.

9. How the Ground Truth for the Training Set Was Established

• Not applicable as a distinct ML training set is not described. If the "Colibrí Vision System" used internal algorithms that were "trained" to recognize colony morphology for picking, the ground truth would likely have been established by manually annotating colonies on images or physical plates as "pickable" or "not pickable" based on expert microbiological judgment, but this is not mentioned in the provided text. The current text suggests the operator "chooses the plates exhibiting adequate growth and selects the isolated colonies to be processed assigning the automatic ID tasks" on a "digital plate", implying human-in-the-loop for key decisions.

Ask a specific question about this device

The BD Kiestra IdentifA module is an automated in vitro diagnostic specimen preparation system for use with the BD Kiestra Laboratory Automation Solution to prepare MALDI targets for the Bruker MALDI Biotyper CA System for the qualitative identification and differentiation of microorganisms using matrix-assisted laser desorption/ionization-time of flight mass spectrometry (MALDI-TOF MS) analysis of colonies grown on plated culture media from human specimens.

The BD Kiestra IdentifA is indicated for use in the clinical laboratory with the BD Kiestra Read Compact and Bruker MALDI Biotyper CA System to aid in the diagnosis of bacterial and fungal infections.

The BD Kiestra™ IdentifA is an instrument which automates picking of technologist-selected colonies from plated media and prepares a Bruker MALDI target for identification and differentiation of organisms. The BD Kiestra IdentifA includes the following components (Note: Bruker MALDI targets, Matrix and Bacterial Test Standard (BTS) are required, however, they are obtained directly from Bruker Daltonik GmbH):

• . BD Kiestra IdentifA instrument and software with onboard pipetting and nephelometry.
• . BD formic acid, deionized water, pipet tips, Matrix and BTS transfer vials.
• . BD Kiestra IdentifA nephelometer calibration standards (0.2, 0.5, 1.0 and 3.0 McFarland).
• BD Kiestra IdentifA cuvette array.

When a MALDI identification is ordered by a technologist selects the colonies from an image of a plated medium obtained using the BD Kiestra™ ReadA Compact. The coordinates of the colonies and the plated medium are transferred to BD Kiestra IdentifA where the colonies are picked. The colonies are suspended in deionized water and the onboard nephelometer determines the McFarland turbidity. Based on the McFarland, BD Kiestra IdentifA pipets the organism suspension onto a Bruker MALDI target. The BD Kiestra IdentifA uses the Bruker extended Direct Transfer method for preparation of the MALDI target by overlaying formic acid and Bruker Matrix onto the target spot. In addition, BTS spots are prepared on the target slide for quality control. Once dried, the technologist manually removes the target and loads onto the Bruker MALDI Biotyper CA System. The BD Kiestra IdentifA transfers the location of sample and BTS spots to the MALDI Biotyper CA. If requested by the technologist, BD Kiestra IdentifA will also dilute the organism suspension to a target of 0.5 McFarland.

The BD Kiestra IdentifA can be used as a standalone instrument or integrated into the BD Kiestra Laboratory Automation System. The standalone instrument utilizes an input/output module for manual plate loading, which handles de-stacking and stacking of plates. When physically integrated into the BD Kiestra Laboratory Automation System, BD Kiestra IdentifA is connected to a track by way of a connection module for automatic plate transfer. BD Kiestra IdentifA software is responsible for the instrument functionality and a touchscreen is mounted on the instrument for user interface.

Here's an analysis of the acceptance criteria and the studies performed for the BD Kiestra IdentifA device, based on the provided text:

Acceptance Criteria and Reported Device Performance

Study Details

2. Sample sizes used for the test set and the data provenance

• Colony Picking Accuracy: 200 mixed culture plates were used, from which 1200 colonies were selected and picked. The data provenance is not explicitly stated (e.g., country of origin), though it is noted as "internal analytical testing." The study design appears to be prospective (experimental).
• Organism Identification Accuracy: A total of 464 isolates of Gram-positive bacteria, Gram-negative bacteria, and yeasts were tested. The data provenance is not explicitly stated (e.g., country of origin), but it is referred to as "internal analytical testing." The study design appears to be prospective (experimental).
• Reproducibility: For each of the 15 strains, 27 tests were performed (3 days x 3 replicates x 3 instruments). This totals 15 strains * 27 tests/strain = 405 tests. The data provenance is not explicitly stated. The study design appears to be prospective (experimental).
• Limit of Detection: 9 organisms were tested, with 8 MALDI target spots inoculated for each. This totals 9 organisms * 8 spots/organism = 72 tests. The data provenance is not explicitly stated. The study design appears to be prospective (experimental).
• Cross-contamination: 100 plates inoculated with Staphylococcus aureus and 100 plates with Klebsiella pneumoniae. These 200 inoculated plates were alternated with 200 uninoculated media, for a total of 400 media processed. An additional field examination included "over 58,000 samples" processed in Europe since January 2020. The study design combined prospective experimental testing with retrospective field data analysis.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

• The text does not explicitly state the number of experts or their qualifications for establishing ground truth for the test set. However, for the colony picking accuracy, it mentions "Colonies from both isolates on each plate were selected by a technologist". The "Organism Identification" study implicitly used comparison to an "expected result," which would typically be based on a validated identification method and potentially confirmed by expert review, but this is not detailed. The "Reproducibility" study used "Strains with known identifications."

4. Adjudication method for the test set

• The document does not describe a formal adjudication method (like 2+1, 3+1) for disagreements or ambiguous cases in the test set. For the "Organism Identification Accuracy," results from the BD Kiestra IdentifA were "compared to the expected result for each isolate." For colony picking, it was confirmed visually and by the Bruker MALDI Biotyper CA identification.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• No, a multi-reader multi-case (MRMC) comparative effectiveness study focusing on human reader performance improvement with AI assistance was not done. This device is an automated specimen preparation system, not an AI for interpretation that would assist human readers in diagnosing. Its function is to automate the preparation stage for downstream MALDI-TOF MS analysis. The comparison is between automated preparation vs. manual preparation using the predicate device.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

• Yes, the performance studies (Colony Picking, Organism Identification Accuracy, Reproducibility, LoD, Cross-contamination) evaluate the standalone performance of the BD Kiestra IdentifA in preparing samples. The output of the BD Kiestra IdentifA is a prepared MALDI target, which is then loaded onto the Bruker MALDI Biotyper CA System for organism identification. While a technologist selects colonies from a digital image, the subsequent steps of picking, suspension, turbidity measurement, and spotting are automated without human intervention. The performance metrics are based on the results obtained from the prepared targets by the downstream MALDI-TOF MS system.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

• The ground truth primarily appears to be "expected identification" or "known identifications" of microorganisms. This implies an established and validated identification for each isolate, likely obtained through standard microbiological methods or a reference MALDI-TOF MS system. For the "Organism Identification Accuracy" study, the comparison was between the BD Kiestra IdentifA prepared samples and "manual sample preparation, i.e. performing the extended Direct Transfer (eDT) Procedure and spotting on a MALDI target according to the previously FDA-cleared Bruker MALDI Biotyper CA user manual." This suggests the "expected result" was either based on prior rigorous identification or the result from the manual predicate method.

8. The sample size for the training set

• The document does not explicitly state the sample size for a training set. As an automated specimen preparation system, it's possible its internal algorithms (e.g., for colony picking, turbidity estimation, pipetting precision) were developed and optimized using various datasets, but these "training sets" are not described in terms of size or content. The provided studies focus on validation/test set performance.

9. How the ground truth for the training set was established

• Not applicable, as a specific "training set" and its ground truth establishment are not described in the provided text. The device's function is mechanical automation of a known lab procedure, not an AI model that learns from training data in the same way. Any inherent 'intelligence' (e.g., image processing for colony identification) would have been programmed based on established features rather than learned through labeled training data.

Ask a specific question about this device