The MBT Sepsityper is a qualitative in vitro diagnostic device consisting of an MBT-CA (Sepsityper) software extension and a reagent kit (MBT Sepsityper Kit US IVD) for use in conjunction with other clinical and laboratory findings to aid in the early diagnosis of bacterial and yeast infections from positively flagged blood cultures using the MALDI Biotyper CA System.

The MBT Sepsityper Kit US IVD is a disposable blood culture processing device that includes associated reagents that are intended to concentrate and purify microbial cells from blood culture samples identified as positive by a continuous monitoring blood culture system and confirmed to demonstrate the presence of a single organism as determined by Gram stain. This sample preparation manual method is performed by laboratory health professionals in a clinical diagnostic setting. Subculturing of positive blood cultures is necessary to recover organisms for identification of organisms not identified by the MBT-CA System, for susceptibility testing and for differentiation of mixed growth.

Positive MBT Sepsityper results do not rule out co-infection with organisms that may not be detected by the MBT-CA System. Results of the MBT Sepsityper should not be used as the sole basis for diagnosis, treatment, or other patient management decisions. Results of the MBT Sepsityper should be correlated with Gram stain results and used in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial and yeast bloodstream infections.

Organisms recovered from positive blood culture bottles that are suitable for identification using the MBT Sepsityper Kit US IVD and MBT-CA Systems are listed in the MALDI Biotyper CA System Package Insert Reference Library.

The MALDI Biotyper CA System uses MALDI (matrix-assisted laser desorption/ionization) TOF (time of flight) mass spectrometry technology for the identification of organisms isolated from clinical samples. Identification can be performed from an isolated colony or from a cell extract. The sample material is transferred to a target plate, dried and overlaid with a matrix. The MBT FAST Shuttle US IVD is an optional hardware tool that may be used for drying the samples deposited on the MALDI target plate under controlled conditions.

The MALDI process transforms the proteins and peptides from the isolated microorganisms into positively charged ions. This is achieved by irradiating the matrix-sample composite with a UV laser. The matrix absorbs laser energy and transfers protons to the intact proteins or peptides in the gas phase. These ions are electrostatically accelerated and arrive in the flight tube at a mass-dependent speed. Because different proteins/peptides have different masses, ions arrive at the detector at different times (time of flight). The MBT-CA System measures the time (in the nanosecond range) between pulsed acceleration and the corresponding detector signal of the ions, and the time is converted into an exact molecular mass.

The highly abundant microbial ribosomal proteins result in a mass spectrum with a characteristic mass and intensity distribution pattern. This pattern is species-specific for many bacteria and yeasts and can be used as a 'molecular fingerprint' to identify a test organism. The spectrum of the unknown test organism, acquired through the software MBT Compass HT CA of the MBT-CA System, is electronically transformed into a peak list. Using a biostatistical algorithm, this peak list is compared to reference peak lists of organisms in the MBT-CA Reference Library and a log(score) between 0.00 and 3.00 is calculated. The higher the log(score), the higher the degree of similarity to a given organism in the MBT-CA Reference Library. The log(score) ranges reflect the probability of organism identification.

The FDA 510(k) submission document focuses on demonstrating substantial equivalence to an existing predicate device rather than presenting a traditional acceptance criteria study for a new device. Therefore, the "acceptance criteria" discussed are largely driven by proving that the new components (MBT Compass HT CA software and MBT FAST Shuttle US IVD) maintain or improve the performance and safety established by the predicate device.

Here's an analysis of the provided text to fulfill your request:

Acceptance Criteria and Reported Device Performance

The concept of "acceptance criteria" in this context isn't a single set of predefined thresholds for a novel device's performance against a clinical gold standard (e.g., sensitivity/specificity targets). Instead, it's about demonstrating that the new components do not negatively impact the established performance of the predicate device and potentially offer improvements (like accelerated drying time). The "reported device performance" is presented as evidence that these conditions are met.

Table 1: Acceptance Criteria (Implied) and Reported Device Performance

Study Details

Based on the provided text:

1. Sample sizes used for the test set and the data provenance:

   • MBT FAST Shuttle - Repeatability: 120 mass spectra (presumably from 12 bacterial/yeast strains * 10 repeats * 2 runs across DT, eDT, Ext, Sepsityper workflows as indicated in Table 3 headers, although the text says "each out of 2 runs" for "each workflow/method" - suggesting 10 per method/workflow per run).
   • MBT FAST Shuttle - Reproducibility (Site-to-Site): 2700 samples for MBT workflow (900 samples per study site * 3 sites) and 1350 samples for Sepsityper workflow (450 samples per study site * 3 sites). The document mentions "10 microorganisms" used per study site.
   • MBT FAST Shuttle - Reproducibility (Device-to-device): 1080 samples for MBT workflow (360 samples per MBT FAST Shuttle * 3 shuttles) and 540 samples for Sepsityper workflow (180 samples per MBT FAST Shuttle * 3 shuttles).
   • MBT FAST Shuttle - Reproducibility (Operator-to-operator): 900 samples for MBT workflow (450 samples per operator * 2 operators) and 450 samples for Sepsityper workflow (225 samples per operator * 2 operators).
   • MBT FAST Shuttle - Reproducibility (Day-to-day): 900 samples for MBT workflow (180 samples per day * 5 days) and 450 samples for Sepsityper workflow (90 samples per day * 5 days).
   • MBT FAST Shuttle - Method Comparison (Drying): 279 mass spectra for air-dried and 279 mass spectra for MBT FAST Shuttle dried from three study sites (93 mass spectra per site per drying method). Ten (10) microorganisms and a blood culture, each spotted in triplicates.
   • MBT Compass HT CA - Low Confidence Results: 15,270 spectra in total, with 1,670 yellow log(scores) re-analyzed.
   • MBT Compass HT CA - IDealTune: Data collected from 133 BTS-QC runs at Site 1 (over 17 months) and 76 BTS-QC runs at Site 2 (over 14 months).

   Data Provenance: The studies were performed at multiple sites (at least 3 for reproducibility studies), and one study explicitly mentions that microorganisms were shipped to both US study sites. This implies the data is, at least in part, prospectively collected in a multi-center setting for verification/validation. The "low confidence results" study was a retrospective non-interventional validation using data from previous clearances.

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • The document does not specify the number or qualifications of experts used to establish ground truth for most of these performance studies. The studies primarily focus on performance consistency and equivalence compared to established methods using what appears to be common laboratory standards (e.g., identity confirmed organisms, BTS quality checks).
   • For the "low confidence results" study, it states: "Isolates from clinical routine were used to compare the results of the MBT-CA System against a gold standard (16S sequencing)." This suggests the ground truth was established by 16S sequencing, a molecular method, rather than solely by human experts, and then potentially interpreted by experts.

3. Adjudication method (e.g. 2+1, 3+1, none) for the test set:

   • No adjudication method involving multiple human readers for conflict resolution is mentioned or appears to be applicable given the nature of the device (mass spectrometry-based organism identification). The performance is assessed on the agreement with an expected identification or log(score) thresholds.

4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No MRMC or human-in-the-loop comparative effectiveness study with human readers assisting or being assisted by AI is described in this document. The device is a "clinical mass spectrometry microorganism identification and differentiation system," not an AI-assisted diagnostic imaging tool.

5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

   • Yes, the performance validation studies of the MBT FAST Shuttle US IVD and MBT Compass HT CA software are essentially standalone performance evaluations of these components within the overall MALDI Biotyper CA System. The "outputs" (identification results, log(scores)) are generated by the system (including the hardware, software, and reference library) without direct human interpretation of the raw mass spectra. Human involvement is in sample preparation and operating the system, but the core identification is algorithmic.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc):

   • For the analytical performance of organism identification, the ground truth appears to be based on:
     • Reference strains/known microorganisms: Used in repeatability and reproducibility studies.
     • 16S sequencing: Explicitly stated as the "gold standard" for comparing results in the "low confidence results" study.
     • Internal quality control standards: Like the Bacterial Test Standard (BTS) for IDealTune validation.
   • This is primarily laboratory-based "gold standard" ground truth (molecular methods, established reference cultures), rather than expert consensus on clinical cases.

7. The sample size for the training set:

   • This document describes the validation of new components for an existing system. It does not provide details about the training set size for the underlying MALDI Biotyper CA System's reference library or analytical algorithms. The "reference library" (which acts as a form of "training data" for identifying unknown spectra) is mentioned as being continually updated, but its size is not specified.

8. How the ground truth for the training set was established:

   • Similar to the above, the document does not detail how the ground truth was established for the training data (the reference library) of the overall MALDI Biotyper CA System. However, standard practice for building such libraries involves:
     • Well-characterized bacterial and yeast strains: Often from culture collections, with identity confirmed by a variety of methods including 16S rRNA gene sequencing, traditional biochemical tests, and possibly whole-genome sequencing.
     • Internal validation and verification: Ensuring the spectral patterns are consistent and representative for each species.

In summary, this 510(k) submission successfully demonstrates substantial equivalence by showing that the new components (MBT Compass HT CA software and MBT FAST Shuttle US IVD) maintain the safety and effectiveness of the predicate device, and in some cases, enhance usability (faster drying time, improved instrument maintenance) without introducing new risks or compromising diagnostic accuracy. The studies presented are analytical validations focusing on performance characteristics relevant to microorganism identification in a laboratory setting.

Ask a specific question about this device

The Accelerate Arc system is an automated sample preparation device that uses lysis and centrifugation to prepare concentrated microbial suspensions from positive blood culture samples that can be used for bacterial and yeast identification with the Bruker MALDI Biotyper CA System (MBT-CA System) with MBT-CA Sepsityper software extension. Samples are processed directly from BD BACTEC blood culture bottles identified as positive by a continuous monitoring blood culture system. Samples should be confirmed as monomicrobial by Gram stain.

The Accelerate Arc system is an in vitro diagnostic device comprised of the Accelerate Arc system software, and the Accelerate Arc BC kit. The Accelerate Arc BC kit is a disposable consumable that includes reagents to concentrate and purify microbial cells from positive blood culture samples.

Microbial suspensions prepared by the Accelerate Arc system can be used to identify bacterial species and yeasts in accordance with the Bruker MBT-CA reference library.

Subculture of positive blood culture is necessary to recover organisms not identified by the Bruker with MBT-CA Sepsityper software extension, species not indicated for testing with the Bruker MBT-CA System with MBT-CA Sepsityper software extension, for susceptibility testing, and for differentiation/recovery of organisms present in polymicrobial samples.

The Accelerate Arc system is intended for use by trained healthcare professionals in clinical laboratories in conjunction with other clinical and laboratory findings, including Gram staining, to aid in the diagnosis of bloodstream infections.

The Accelerate Arc system is an automated sample preparation device with associated consumables that uses lysis and centrifugation to prepare microbial suspensions from positive blood culture (PBC) samples from BD BACTEC™ bottles that have rung positive on a continuous monitoring system and confirmed to be monomicrobial by Gram stain. Suspensions containing concentrated, monomicrobial microorganisms are intended for use with the downstream mass spectrometry (MS) analyzer Bruker MALDI Biotyper® CA System with MBT-CA Sepsityper® software extension for qualitative identification and differentiation of microorganisms to aid in the early diagnosis of bacterial and yeast infections. This device is comprised of an automated sample preparation instrument (Accelerate Arc instrument), system software (Accelerate Arc system software), and sample preparation kit (Accelerate Arc BC kit).

The Accelerate Arc system was designed to standardize workflow to minimize operator error and variability. The Accelerate Arc instrument, system software and BC kit rapidly clean up and concentrate microorganisms from positive blood culture samples for downstream identification of the microorganism using the Bruker MALDI Biotyper® CA System with MBT-CA Sepsityper® software extension. The confidence score range from the MBT-CA Sepsityper® software extension is used to denote high confidence (1.8 to 3), low confidence (1.6 to 1.79), and no identification (0 to 1.59). Altogether, rapid microorganism identification direct from PBC can be achieved in about 1 ½ hours following this workflow.

The maximum system configuration of eight Accelerate Arc modules can process greater than 150 PBCs in a single day.

The Accelerate ArcTM system is comprised of:

. Accelerate Arc™ instrument
Accelerate Arc™ system software .
. Accelerate Arc™ BC kit

Samples prepared by the Accelerate Arc system are intended for use with:

Bruker MALDI Biotyper® CA System with MBT-CA Sepsityper® software . extension

Here's a summary of the acceptance criteria and the study that proves the device meets them, based on the provided text:

Acceptance Criteria and Device Performance for Accelerate Arc System

1. Table of Acceptance Criteria and Reported Device Performance

The document does not explicitly present a table of acceptance criteria. Instead, performance characteristics are described qualitatively and with percentages for identification rates. Based on the "Performance Characteristics" section, key aspects for successful identification (High or Low Confidence ID) and accuracy are derived as acceptance indicators.

Ask a specific question about this device

The Colibrí is an automated in vitro diagnostic specimen preparation system for use with WASPLab to prepare MALDI-TOF targets for the bioMérieux VITEK MS systems or Bruker MALDI Biotyper CA mass spectrometry systems for qualitative identification and microbial suspension for the bioMérieux VITEK 2 systems or Beckman Coulter MicroScan WalkAway Antimicrobial Susceptibility Testing (AST) systems for qualitative testing of isolated colonies of gram-negative and gram-positive bacterial species grown on solid culture media.

The Colibrí is an automated pre-analytical processor that picks isolated colonies designated by the operator and uses a pipetting system to prepare MALDI-TOF MS (Matrix-Assisted Laser Desorption/lonization-Time of Flight Mass Spectrometry) target slides for bacterial identification and microbial suspension at known concentration for Antimicrobial Susceptibility Testing and purity assessment.

The Colibrí software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.

Bacterial suspensions for AST and purity plates are identified by barcode label.

The Colibrí is intended for use by trained healthcare professionals in clinical laboratories in conjunction with other clinical and laboratory findings, including Gram staining, to aid in the diagnosis of bacterial infections.

The Colibrí has not been validated for use in the identification or processing of yeast species, molds, Nocardia, or mycobacteria.

The Colibrí is an instrument which automates the picking of selected colonies from plated media and prepares MALDI target slides for the bioMérieux VITEK MS systems or the Bruker MALDI Biotyper CA systems that are used in clinical laboratories for identification and differentiation of organisms grown on plated media by Matrix-Assisted Laser Desorption/Jonization Time-of Flight Mass Spectrometry (MALDI-TOF MS). The Colibri automates the preparation of microbial suspensions at known concentration for bioMérieux VITEK 2 systems and Beckman Coulter MicroScan WalkAway systems that are used in clinical laboratories for AST analyses. Moreover, the Colibrí is used for Purity Plates preparation for purity assessments.

The Colibrí includes the following components:

• Colibrí instrument and software with on-board pipetting system and nephelometer .
• Colibrí Primary Tubes
• Colibrí Spreader
• Colibrí Daily Verification kit.

Colibri is designed to be used in conjunction with the WASPLab device for culture plate incubation and image analysis. After appropriate plate incubation, the operator selects the colonies from a digital image of culture media plate streaked with microbiological human specimen, available through WebApp software, the WASPLab User Interface.

The operator assigns the automatic ID or AST tasks to the isolated colonies to be processed. Then, the operator loads the plates in the Collbri where colonies are automatically picked, spotted on the target slide and overlayed with the matrix or suspended into the dedicated solution for the preparation of the microbial suspension for AST purposes (Secondary Tube).

When used in conjunction with the bioMérieux VITEK MS systems, the Colibri can prepare the 48-spot target slides by performing the direct spotting of colonies. The calibrator used for quality control is manually applied by the operator at the end of the automated colony spotting. When used in conjunction with the Bruker MALDI Biotyper CA systems, the Colibri can prepare either reusable 48-spot or disposable 96-spot targets by performing the Direct Transfer Sample Procedure. The BTS used for quality control is manually applied by the operator at the end of the automated colony spotting.

When used in conjunction with the bioMérieux VITEK 2 systems or the Beckman Coulter MicroScan WalkAway systems, the Colibri can prepare the microbial suspension at the proper concentration by direct colony suspension method. The onboard nephelometer allows the preparation of Secondary Tubes (AST suspensions) at the correct concentration and the Colibri Spreader is used for Purity Plates preparation.

The Colibrí software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.

The traceability of prepared Secondary Tube and Purity Plates is maintained by dedicated labels applications.

Colibrí requires four different calibrations, one on the nephelometer, three on the cameras. None of these calibration activities require user intervention if not in terms of periodical cleaning of the mechanical component as described in the dedicated section of the User Manual. The Set-up calibration of nephelometer and camera units are performed during the device initial setup. Auto-calibration is performed at the end of the initial set-up and periodically during the preventive maintenance to check that all the mechanical references can be found inside the positioning tolerances, that the I/Os are responsive. Runtime calibration is performed during the normal usage to automatically check the proper functioning of the Colibrí.

Colibrí requires a daily nephelometer verification to check the proper reading of suspensions at different turbidity values.

The Colibrí device is an automated in vitro diagnostic specimen preparation system. The provided text describes the acceptance criteria and the study that proves the device meets these criteria for preparing microbial suspensions for Antimicrobial Susceptibility Testing (AST) using Beckman Coulter MicroScan WalkAway systems.

Here's a breakdown of the requested information:

1. A table of acceptance criteria and the reported device performance:

The acceptance criteria are implicitly derived from the successful outcomes of the analytical studies. The performance is reported as the percentage of successful outcomes for each metric.

• E. coli ATCC 25922: 3-7 x 105 CFU/mL
• Other bacteria: 2-8 x 105 CFU/mL | 98.5% of prepared suspensions had microbial concentration within acceptable limits.
• E. coli: 100% (36/36)
• Pseudomonas aeruginosa: 96.7% (29/30)
• Staphylococcus aureus: 97.6% (41/42)
• Enterococcus faecalis: 100% (30/30) |
  | AST Challenge Test (Agreement with Manual Preparation) | Essential Agreement (EA) of MICs: High agreement
  Category Agreement (CA): High agreement
  Discrepancies (vmj, maj): Low/none | Overall EA: 100% (1232/1232 evaluable MIC results within 1 two-fold dilution)
  Overall CA: 98.4% (4187/4254 SIR categorizations in agreement)
  Very Major discrepancy (vmj): 0
  Major discrepancy (maj): 0 |
  | Reproducibility Study | Best-case reproducibility: ≥95% (implied)
  Worst-case reproducibility: ≥89% (implied) | Best-case reproducibility: ≥99.8% (all panels combined)
  Worst-case reproducibility: ≥94.3% (all panels combined) |
  | Sample preparation for Quality Control | 100% of MIC values within CLSI/panel IFU QC range | 100% (all tested organisms and antimicrobial agents) |
  | Purity Plates Evaluation (Cross-contamination) | Absence of cross-contamination (100% monomicrobial growth) | 100% (453/453 Purity Plates showed monomicrobial growth) |

2. Sample sizes used for the test set and the data provenance:

• Preparation of Microbial Suspensions for AST:

  • Test Set Size: 132 microbial suspensions (36 E. coli, 30 Pseudomonas aeruginosa, 42 Staphylococcus aureus, 30 Enterococcus faecalis).
  • Data Provenance: Not explicitly stated (e.g., country of origin). The study involved three Colibrí instruments, suggesting internal validation. Retrospective or prospective is not specified, but the nature of the validation suggests prospective testing.

• AST Challenge Test:

  • Test Set Size: Different species: Enterobacterales (n=50 isolates), Staphylococcus (n=20 isolates), Streptococcus (n=12 isolates), Enterococcus (n=18 isolates), non-fermenters (n=10 isolates). Each processed by three Colibrí instruments, yielding varying numbers of MIC results and SIR categorizations across different panels (e.g., 2454 total MIC results for Enterobacterales on NM-NF50 panel).
  • Data Provenance: Not explicitly stated (e.g., country of origin). The study design implies prospective testing within the manufacturer's validation process.

• Reproducibility Study:

  • Test Set Size: 9 gram-positive and 9 gram-negative strains, processed on 3 Colibrí instruments over 3 days, with each condition tested in triplicate (total of 27 replicates for each strain-antimicrobial agent combination).
  • Data Provenance: Not explicitly stated (e.g., country of origin, retrospective or prospective). Implied prospective.

• Sample preparation for Quality Control:

  • Test Set Size: CLSI-recommended reference strains (Escherichia coli ATCC 25922, Pseudomonas aeruginosa ATCC 27853, Staphylococcus aureus ATCC 29213, Enterococcus faecalis ATCC 29212). The "No. MIC within QC range" indicates a total of 30 or 36 or 42 tests for each drug-organism combination, per three instruments.
  • Data Provenance: Not explicitly stated. Implied prospective.

• Purity Plates Evaluation:

  • Test Set Size: 453 purity plates (150 from AST Challenge, 162 from AST Reproducibility, 141 from Quality Control studies).
  • Data Provenance: Not explicitly stated. Implied prospective.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

• The ground truth for AST results (MICs and SIR categories) generally refers to the results obtained from a reference method. In this case, "Manual suspension was used as comparative method" for the AST Challenge Test. This implies that manually prepared suspensions, processed by the MicroScan WalkAway, served as the reference standard.
• The text does not specify the number of experts or their qualifications for establishing this manual ground truth. It mentions that three different technicians operated the Colibrí machines, but it doesn't detail the personnel for the manual comparative method or for interpreting the results as ground truth beyond the "FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria" and CLSI guidelines.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set:

• The document does not describe any expert adjudication process for the test set results. The comparison is made against a "manual result" (ground truth). The discrepancies (vmj, maj, min) are simply categorized and reported, implying a direct comparison without further expert review for resolving initial disagreements.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

• No, an MRMC study was not conducted. This device (Colibrí) is an automated system for sample preparation and does not involve human "readers" or "AI assistance" in the typical sense of image analysis for diagnosis. Its role is to automate a laboratory process, and the performance is measured against reference methods, not human interpretation.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

• The studies presented are primarily standalone (algorithm only without human-in-the-loop performance) in terms of the Colibrí device's automated functions. The device picks colonies, prepares suspensions, and records data automatically. The performance metrics (inoculum density, MIC accuracy, reproducibility, purity) assess the device's output against established standards and manual methods.
• While an operator designates colonies for picking, the act of preparation itself is automated and evaluated for its accuracy. The "manual suspension" used for comparison acts as the reference for the "algorithm only" performance of the Colibrí in producing the suspension.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc):

• The ground truth for the analytical studies combines reference methods/standards and established guidelines:
  • For microbial suspension concentration: Viable cell count (CFU/mL) against CLSI and FDA guidelines.
  • For AST results: Manual suspension preparation as the comparative method, and comparison of MICs and SIR categories against FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria and CLSI guideline M07.
  • For reproducibility: Comparison to the "mode result" (most frequent MIC value) and established reproducibility criteria (e.g., within one doubling dilution).
  • For Quality Control: CLSI-recommended QC ranges and MicroScan panel IFU values.
  • For Purity Plates: Visual assessment (implied) to confirm monomicrobial growth.

8. The sample size for the training set:

• This document describes performance validation studies for a medical device (Colibrí), not a machine learning model. Therefore, there is no "training set" in the context of data used to train an AI algorithm. The Colibrí is an automated instrument with pre-programmed functions, not a learning algorithm that requires a training dataset.

9. How the ground truth for the training set was established:

• As explained above, there is no training set for this device in the context of AI/ML.

Ask a specific question about this device

The BD Kiestra IdentifA module is an automated in vitro diagnostic specimen preparation system for use with the BD Kiestra Laboratory Automation Solution to prepare MALDI targets for the Bruker MALDI Biotyper System (CA, sirius CA, or sirius one CA) for the qualitative identification of microorganisms using matrix-assisted laser desorption/ionization-time of flight mass spectrometry (MALDI-TOF MS) analysis of colonies grown on plated culture media from human specimens.

The BD Kiestra IdentifA is indicated for use in the clinical laboratory with the BD Kiestra ReadA Compact and Bruker MALDI Biotyper System (CA, sirius one CA) to aid in the diagnosis of bacterial and fungal infections.

The BD Kiestra™ IdentifA automates preparation of MALDI targets for the Bruker MALDI Biotyper® CA System, sirius CA system, and/or sirius one CA System that are used in clinical laboratories for identification and differentiation of organisms grown on plated media by Matrix-Assisted Laser Desorption/lonization Time-of-Flight Mass Spectrometry (MALDI-TOF MS). The system comprises of the BD Kiestra™ IdentifA module (including the associated software and onboard nephelometers and pipetting system), formic acid and automation-compatible transfer vials (for HCCA matrix and Bacterial Test Standard (BTS), which are obtained directly from Bruker and manually transferred to the vials for use on the instrument), consumables (pipette tips and cuvette arrays for preparation of organism suspensions and fluid movement), and nephelometer calibration standards (McFarland standard vials for measuring turbidity of microbial suspensions).

When identification of an organism growing on a culture medium plate is required, a technologist designates specific colonies for picking by the BD Kiestra™ IdentifA module using a digital image of the plate obtained using the BD Kiestra™ ReadA Compact module. The BD Kiestra™ IdentifA automatically suspends the designated colonized water and uses an onboard nephelometer to determine the resulting turbidity. The organism concentration is adjusted automatically by picking additional designated colonies or by appropriate dilution of the suspension to achieve a turbidity within a targeted range of McFarland values. Based on the final organism concentration, the BD Kiestra™ IdentifA pipets one or more aliquots of the microbial suspension onto a MALDI target (either reusable 48-spot or disposable 96-spot targets) and dries the spots at elevated temperature.

The BD Kiestra™ IdentifA performs the extended Direct Transfer (eDT) Sample Preparation Procedure from Bruker whereby the instrument overlays the dried sample spot on the MALDI target with formic acid and matrix. The BD Kiestra™ IdentifA also spots the BTS used for quality control of MALDI-TOF MS organism identification. Once spots are dry, the technologist manually removes the target from the BD Kiestra™ IdentifA and loads it into the Bruker MALDI Biotyper® System for analysis. Information regarding the location of each sample and BTS on the targets and the associated MALDI-TOF MS results are transmitted between the BD Kiestra™ IdentifA and Bruker MALDI Biotyper® System via the Synapsys Informatics, the main software interface, and the BD Kiestra™ BeA, the data interface hub module that communicates with all the other modules including the BD Kiestra™ IdentifA. In addition to preparing the MALDI target, if requested, the BD Kiestra™ IdentifA will also dilute the organism suspension to a standardized turbidity of 0.5 McFarland.

Modules of the BD Kiestra™ System each have their own operating software that communicates via the central BeA data interface hub module with the Synapsys user interface which in turn sends and receives information to/from the Laboratory Information System (LIS).

The BD KiestraTM ReadA or ReadA Compact module is required for use in conjunction with the BD Kiestra™ IdentifA module for image capture. Culture plate incubation may be done offline and then moved to the BD Kiestra™ ReadA or ReadA Compact module for imaging, or incubation can be done in the ReadA and then remain in the ReadA for imaging. Additional software modules (BD Synapsys Informatics and BD Kiestra™ BeA) are also required for the function of the BD Kiestra™ IdentifA, and these modules reside on the BD Kiestra™ Laboratory Automation Solution. The digital image is used by the BD Kiestra™ IdentifA for image analysis and colony designation by the operator.

The provided text outlines the performance characteristics and acceptance criteria for the BD Kiestra IdentifA device. This device is an automated in vitro diagnostic specimen preparation system for MALDI-TOF MS analysis of microorganisms. The submission focuses on demonstrating substantial equivalence to a predicate device (K191964) with the addition of the BD Kiestra ReadA camera system and the Bruker MALDI Biotyper sirius CA and sirius one CA systems.

Here's a breakdown of the requested information based on the provided document:

Acceptance Criteria and Reported Device Performance

The acceptance criteria are implicitly defined by the successful outcomes of the studies demonstrating equivalency or accuracy. The performance is reported as meeting these criteria.

Table 1: Acceptance Criteria and Reported Device Performance

Study Details:

2. Sample Size and Data Provenance:

• Study 1 (Colony Picking Accuracy):
  • Sample Size: 205 mixed cultures (Escherichia coli and Streptococcus pyogenes). 1230 colonies were picked, and 408 isolate MALDI ID results were analyzed.
  • Data Provenance: Not explicitly stated regarding country of origin, but generally, clinical device studies for FDA submissions are conducted in the US or in countries with comparable regulatory and quality standards. The data provenance is prospective as it involves experimental setup and data collection for the purpose of the study.
• Study 2 (Camera Equivalency):
  • Sample Size: 15 microorganisms, 3 dilutions (10^3, 10^4, 10^5 CFU/mL), 2 solid media types (MAC, TSA II).
    • BD Kiestra ReadA (25MP): 270 plate images (15 organisms x 3 dilutions x 2 media types x 3 systems).
    • BD Kiestra ReadA Compact (5MP): 90 plate images (15 organisms x 3 dilutions x 2 media types x 1 system).
  • Data Provenance: Not explicitly stated regarding country of origin. Prospective.
• Study 3 (Identification Accuracy - Bruker MALDI Biotyper systems):
  • Sample Size: 37 Gram-positive bacteria, Gram-negative bacteria, and yeast organisms. Two spots per suspension, tested for three days for repeatability.
  • Data Provenance: Not explicitly stated regarding country of origin. Prospective.
• Quality Control (BTS Workflow):
  • Sample Size: 102 spots (standard workflow) and 430 spots (modified workflow).
  • Data Provenance: Not explicitly stated regarding country of origin. Prospective.

3. Number of Experts Used and Qualifications:

• Study 2 (Camera Equivalency): "Three operators representative of the intended user interpreted the culture plate images and assigned a minimal morphological identification (MMI) code."
• Other Studies: The document does not specify the number or qualifications of experts (e.g., microbiologists, lab technologists) involved in establishing ground truth or performing manual inspections (like in Study 1 for visual confirmation of picking). However, for a device in microbiology, it is standard that trained laboratory personnel or microbiologists perform these tasks.

4. Adjudication Method for the Test Set:

• The document does not explicitly describe a formal adjudication method (e.g., 2+1, 3+1 consensus) for the test set results.
• For Study 2, the "Minimal Morphological Identification (MMI) code" was assigned by "three operators," and agreement was measured. It implies independent assessment rather than a consensus/adjudication process.
• For other studies, the "expected identification" or "Bruker interpretive criteria" served as the reference standard, rather than an adjudicated human consensus.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:

• No MRMC comparative effectiveness study was done to show how human readers improve with AI vs. without AI assistance. This device is an automated preparation system, aiding in sample processing for subsequent human evaluation via MALDI-TOF MS results, rather than an AI diagnostic interpretation tool for images. Study 2 did involve multiple readers evaluating images from different camera systems, but it focused on camera equivalency for MMI interpretation, not on AI assistance for readers.

6. Standalone Performance:

• The device (BD Kiestra IdentifA) is an automated system for sample preparation. Its "standalone performance" is implicitly demonstrated through the accuracy of its automated functions (colony picking, suspension preparation, spotting) and the subsequent successful identification by the Bruker MALDI Biotyper systems, which are separate components. The studies confirm the accuracy of the BD Kiestra IdentifA's automated steps in preparing samples suitable for the MALDI-TOF MS system. The MALDI-TOF MS system itself has its own performance characteristics.

7. Type of Ground Truth Used:

• Study 1 (Colony Picking Accuracy):
  • Initial confirmation of picking: Visual inspection of plates and comparison to original digital images.
  • Confirmation of identification success: Bruker MALDI identification using established interpretive criteria (Log(score) values).
• Study 2 (Camera Equivalency): Minimal Morphological Identification (MMI) codes assigned by trained operators. This is a form of expert interpretation/consensus (though agreement was measured, not a formal consensus process).
• Study 3 (Identification Accuracy - Bruker MALDI Biotyper systems): Bruker interpretive criteria for species identification (Log(score) values). This relies on the established performance of the MALDI-TOF MS system and comparison of results between different versions of the system.
• Quality Control (BTS Workflow): Bruker interpretive criteria for species identification (Log(score) values) for the Bacterial Test Standard (BTS).

In summary, the ground truth for microorganism identification is based on the established MALDI-TOF MS interpretive criteria (log scores for identification confidence), as the device's role is to prepare samples for this established identification method. For colony picking and image interpretation, the ground truth is based on visual inspection, comparison to expected outcomes, and expert human interpretation (for MMI).

8. Sample Size for the Training Set:

• The document does not mention a "training set" in the context of an AI/ML model for the BD Kiestra IdentifA. This device is described as an automated specimen preparation system utilizing robotics, nephelometry, and image processing, rather than a system based on machine learning that requires a separate training phase. The studies described are performance validation studies.

9. How the Ground Truth for the Training Set was Established:

• As the document does not describe an AI/ML model with a distinct training set, this question is not applicable. The device's functionality is based on predefined algorithms and hardware performance, validated against established laboratory methods and interpretive criteria, as detailed in point 7.

Ask a specific question about this device

The Colibri™ is an automated in vitro diagnostic specimen preparation system for use with WASPLab® to prepare MALDI-TOF targets for the bioMérieux VITEK® MS or Bruker MALDI Biotyper® CA mass spectrometry systems for qualitative identification and microbial suspension for the bioMérieux VITEK® 2 Antimicrobial Susceptibility Testing (AST) system for qualitative testing of isolated colonies of gram-negative and gram-positive bacterial species grown on solid culture media.

The Colibri™ is an automated pre-analytical processor that nicks isolated colonies designated by the operator and uses a pipetting system to prepare MALDI-TOF MS (Matrix-Assisted Laser Desorption/lonization-Time of Flight Mass Spectrometry) target slides for bacterial identification and microbial suspension at known concentration for Antimicrobial Susceptibility Testing and purity assessment.

The Collori™ software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.

Bacterial suspensions for AST and purity plates are identified by barcode label.

The Colibri™ is intended for use by trained healthcare professionals in clinical laboratories in conjunction with other clinical and laboratory findings, including Gram staining, to aid in the diagnosis of bacterial infections.

The Colibri™ has not been validated for use in the identification or processing of yeast species, or mycobacteria.

The Colibrí is an instrument which automates the picking of selected colonies from plated media and prepares MALDI target slides for the bioMérieux VITEK MS or the Bruker MALDI Biotyper CA System that are used in clinical laboratories for identification and differentiation of organisms grown on plated media by Matrix-Assisted Laser Desorption/Tonization Time-of Flight Mass Spectrometry (MALDI-TOF MS). The Colibrí automates the preparation of microbial suspensions at known concentration for bioMérieux VITEK 2 System that is used in clinical laboratories for AST analyses. Moreover, the Colibrí is used for Purity Plates preparation for purity assessments.

The Colibrí includes the following components:

• . Colibrí instrument and software with on-board pipetting system and nephelometer
• Colibrí Primary Tubes ●
• . Colibrí Spreader
• Colibrí Daily Verification kit. ●

Colibri is designed to be used in conjunction with the WASPLab device for culture plate incubation and image analysis. After appropriate plate incubation, the operator selects the colonies from a digital image of culture media plate streaked with microbiological human specimen, available through WebApp software, the WASPLab User Interface.

The operator assigns the automatic ID or AST tasks to the isolated colonies to be processed. Then, the operator loads the plates in the Colibri where colonies are automatically picked, spotted on the target slide and overlayed with the matrix or suspended into the dedicated solution for the preparation of the microbial suspension for AST purposes (Secondary Tube).

When used in conjunction with the bioMérieux VITEK MS, the Colibrí can prepare the 48-spot target slides by performing the direct spotting of colonies. The calibrator used for quality control is manually applied by the operator at the end of the automated colony spotting. When used in conjunction with the Bruker MALDI Biotyper CA System, the Colibri can prepare either reusable 48-spot or disposable 96-spot targets by performing the Direct Transfer Sample Procedure. The BTS used for quality control is manually applied by the operator at the end of the automated colony spotting.

When used in conjunction with the bioMérieux VITEK 2, the Colibrí can prepare the microbial suspension at the proper concentration by direct colony suspension method. The onboard nephelometer allows the preparation of Secondary Tubes (AST suspensions) at the correct concentration and the Colibrí Spreader is used for Purity Plates preparation.

The Colibrí software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.

The traceability of prepared Secondary Tube and Purity Plates is maintained by dedicated labels applications.

Colibrí requires four different calibrations, one on the nephelometer, three on the cameras. None of these calibration activities require user intervention if not in terms of periodical cleaning of the mechanical component as described in the dedicated section of the User Manual. The Set-up calibration of nephelometer and camera units are performed during the device initial setup. Autocalibration is performed at the end of the initial set-up and periodically during the preventive maintenance to check that all the mechanical references can be found inside the positioning tolerances, that the I/Os are responsive. Run-time calibration is performed during the normal usage to automatically check the proper functioning of the Colibrí.

Colibrí requires a daily nephelometer verification to check the proper reading of suspensions at different turbidity values.

Acceptance Criteria and Device Performance for Colibrí

The Colibrí is an automated in vitro diagnostic specimen preparation system for use with WASPLab to prepare MALDI-TOF targets for microbial identification and microbial suspensions for Antimicrobial Susceptibility Testing (AST).

1. Table of Acceptance Criteria and Reported Device Performance

2. Sample Size Used for the Test Set and Data Provenance

The study utilized a test set consisting of various bacterial strains to evaluate the Colibrí's performance in preparing samples for MALDI-TOF identification and AST.

• Sample Size for Identification:
  • Gram-Negative Species: 5 species (4 Enterobacterales, 1 Non-fermenter) with 92 colonies picked and analyzed.
  • Gram-Positive Species: 5 species (2 Enterococcus, 2 Staphylococcus, 1 Streptococcus) with 200 colonies picked and analyzed.
  • Total for Identification: 292 colonies.
• Sample Size for AST:
  • Total AST Tested: 2720 tests (sum of "Total tested" column from AST summary tables).
  • This included: 1400 Enterobacterales, 200 Non-fermenters, 520 Staphylococci, 380 Enterococci, and 220 Streptococci.
• Data Provenance: The document does not explicitly state the country of origin of the data or whether it was retrospective or prospective. However, given that it is a submission to the U.S. Food & Drug Administration (FDA) by an Italian company (Copan WASP Srl, Brescia, Italy), the study was likely conducted to meet international regulatory standards, potentially with data from one or more clinical laboratories. The nature of the "Full workflow validation" study described suggests a prospective experimental design to assess the device's performance under controlled conditions.

3. Number of Experts Used to Establish Ground Truth and Qualifications

The document does not explicitly state the number of experts used or their specific qualifications for establishing ground truth. However, the study focuses on the accuracy of identification results against "expected strain identity" and comparison of MICs and SIR categories against "MICs obtained by bioMérieux VITEK 2 using manual sample preparation" and "FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria."

This implies that:

• For identification, the "expected strain identity" would have been established using a gold standard method, likely by skilled microbiologists or reference laboratories with established credentials in microbial identification.
• For AST, the "manual sample preparation" by bioMérieux VITEK 2 would serve as the comparator, and the interpretation of results would follow "FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria," likely applied by qualified laboratory personnel.

4. Adjudication Method

The document does not describe a formal adjudication method (e.g., 2+1, 3+1) for the test set.

• For identification, performance was calculated as "percentage of spotted colonies matching the expected identity," suggesting a direct comparison to a single reference (ground truth).
• For AST, the "MICs obtained by bioMérieux VITEK 2 using Colibrí were compared to the MICs obtained by bioMérieux VITEK 2 using manual sample preparation," indicating a direct method-to-method comparison. Discrepant SIR results were categorized, implying a systematic evaluation rather than a consensus-based adjudication in the traditional sense.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

There is no mention of a Multi-Reader Multi-Case (MRMC) comparative effectiveness study or any effect size of how much human readers improve with AI vs. without AI assistance. The Colibrí device is a pre-analytical automated system for sample preparation, not an AI-assisted diagnostic interpretation tool for human readers. Its function is to automate the preparation steps for subsequent analysis by other IVD systems (MALDI-TOF MS and VITEK 2 AST).

6. Standalone Performance Study

Yes, a standalone performance study was done for the algorithm/system. The "Full workflow validation" study directly assesses the Colibrí device's ability to accurately pick colonies and prepare samples for downstream analysis without human intervention in the picking and preparation steps themselves. The results presented for "VITEK MS Identification Result" and "AST summary of results" directly reflect the performance of the Colibrí-prepared samples when analyzed by the respective analytical instruments.

7. Type of Ground Truth Used

• For microbial identification: The ground truth was based on "expected strain identity," which implies confirmed identification results likely obtained through established, highly accurate microbiological methods serving as a reference.
• For AST: The ground truth for comparative purposes was "MICs obtained by bioMérieux VITEK 2 using manual sample preparation," interpreted according to "FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria."

8. Sample Size for the Training Set

The document explicitly states that the "Colibrí uses equally designed and developed hardware and software modules as the Colibrí System. Therefore, the performance of the predicate device represents the performance of the new device." It also mentions that "The results of the analytical studies were submitted to support the 510(k) Premarket Notifications K193138 and K220546." This indicates that the current submission (K223245) relies on the performance data of the predicate device (Colibrí System, K220546). The document does not provide details on the training set sample size for the Colibrí System's development.

9. How the Ground Truth for the Training Set Was Established

As the current submission relies on the predicate device's performance, the ground truth for any training set related to the predicate Colibrí System would have been established during its development and prior FDA submission (K220546). The current document does not provide details of how the ground truth for the predicate device's training set was established. However, based on the performance parameters listed (e.g., accuracy of colony picking, reproducibility of identification, accuracy of nephelometer), it likely involved:

• Microbial cultures: Using well-characterized microbial strains with confirmed identities.
• Manual reference methods: Comparing automated colony picking against visual confirmation by trained personnel.
• Reference AST methods: Comparing automated suspension preparation and subsequent AST results against established manual AST methods and interpretive criteria to assess agreement in MICs and categorical susceptibility (Susceptible, Intermediate, Resistant).

Ask a specific question about this device

The Collbri™ System is an in vitro diagnostic device comprised of the Collbri™ Preparation Station for use with the bioMérieux VITEK® MS or Bruker MALDI Biotyper® CA mass spectrometry systems for qualitative identification and with the bioMérieux VITEK® 2 Antimicrobial Susceptibility Testing (AST) system for qualitative testing of isolated colonies of gram-negative and gram-positive bacterial species grown on solid culture media. The Collbri™ System is a semi-automated pre-analytical processor that picks isolated colonies designated by the operator and uses a pipetting system to prepare MALDI-TOF MS (Matrix-Assisted Laser Desorption/Jonization-Time of Flight Mass Spectrometry) target slides for bacterial identification and microbial suspension at known concentration for Antimicrobial Susceptibility Testing and purity assessment.

The Colibrí software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.

Bacterial suspensions for AST and purity plates are identified by barcode label.

The Colibr™ System is intended for use by trained healthcare professionals in clinical laboratories in conjunction with other clinical and laboratory finding Gram staining, to aid in the diagnosis of bacterial infections.

The Collbri™ System has not been validated for use in the identification or processing of yeast species, Mocardia, or mycobacteria.

The Copan Colibrí System is designed to be used as an accessory of the downstream MALDI-TOF MS and antimicrobial susceptibility testing (AST) analyzers automating various manual steps in the workflow for the preparation of samples for the identification of isolated colonies and for AST of isolated colonies of gram-negative and gram-positive bacterial species grown on solid culture media.

The Colibrí System automates the preparation of MALDI target slides for the bioMérieux VITEK MS or the Bruker MALDI Biotyper CA System that are used in clinical laboratories for identification (ID) of organisms grown on plated media by Matrix-Assisted Laser Desorption/Jonization Time-of Flight Mass Spectrometry (MALDI-TOF MS). The Colibri System automates the preparation of microbial suspensions at known concentration for bioMérieux VITEK 2 System that is used in clinical laboratories for AST analyses. Moreover, the Colibri System is used for Purity Plates preparation for purity assessments.

The Colibrí System comprises the Colibrí Vision System and Colibrí Preparation Station hardware modules and pipette tips, Primary Tubes, Spreader and nephelometer Verification Kit as consumables. After appropriate plate incubation, the operator using the graphical User Interface (Image Reading Interface) chooses the plates exhibiting adequate growth and selects the isolated colonies to be processed assigning the automatic ID or AST tasks. By using the Colibrí Vision System, specific colonies to be picked are designated by the operator on a digital plate. The Operator manually loads the plates in the Colibri Preparation Station where colonies are automatically picked, spotted on the target slide and overlayed with the matrix or suspended into the dedicated solution for the preparation of the microbial suspension for AST purposes (Secondary Tube).

When used in conjunction with the bioMérieux VITEK MS, the Colibrí System can prepare the 48spot target slides by performing the direct spotting of colonies. The calibrator used for quality control is manually applied by the operator at the end of the automated colony spotting. When used in conjunction with the Bruker MALDI Biotyper CA System, the Colibrí System can prepare either reusable 48-spot or disposable 96-spot targets by performing the Direct Transfer Sample Procedure. The BTS used for quality control is manually applied by the operator at the automated colony spotting.

When used in conjunction with the bioMérieux VITEK 2, the Colibrí System can prepare the microbial suspension at the proper concentration by direct colony suspension method. The onboard nephelometer allows the preparation of Secondary Tubes (AST suspensions) at the correct concentration and the Colibrí Spreader is used for Purity Plates preparation.

Copan WASP S.r.l., Traditional 510(k)- Colibrí System

The Colibrí software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.

The traceability of prepared Secondary Tube and Purity Plates is maintained by dedicated labels applications.

Colibri System requires four different calibrations, one on the nephelometer, three on the cameras. None of these calibration activities require user intervention if not in terms of periodical cleaning of the mechanical component as described in the dedicated section of the User Manual. The Set-up calibration of nephelometer and camera units positioned on the Colibrí Vision System and on the Colibrí Preparation Station are performed during the device initial setup. Auto-calibration is performed at the end of the initial set-up and periodically during the preventive maintenance to check that, in the Collbrí Preparation all the mechanical references can be found inside the positioning tolerances, that the I/Os are responsive. Run-time calibration is performed during the normal usage to automatically check the proper functioning of the Colibrí Vision System and the Colibrí Preparation Station.

Colibrí System requires a daily nephelometer verification to check the proper reading of suspensions at different turbidity values.

The provided text describes the performance data for the Colibrí System, an in vitro diagnostic device. The acceptance criteria and performance are primarily focused on its ability to accurately prepare microbial suspensions for Antimicrobial Susceptibility Testing (AST) and for MALDI-TOF MS identification, compared to manual methods.

Here's an attempt to extract the requested information. Please note that the document is a 510(k) summary, which often provides summarized performance data rather than detailed study protocols. Therefore, some information might be explicitly stated as "not applicable" or inferred based on common practices for such device clearances.

1. A table of acceptance criteria and the reported device performance

The document outlines several analytical studies. While explicit "acceptance criteria" are not always presented as target percentages, the performance results are given, implying that these results met the internal pre-defined acceptance thresholds for substantial equivalence.

2. Sample sizes used for the test set and the data provenance

• Nephelometer Calibration Verification: 300 suspensions were prepared (20 suspensions for each of 5 concentrations, across 3 operators and 3 Colibrí systems, though the calculation isn't directly 2053*3).
• Pipettor Trueness and Precision: 10 measurements for each of 4 volumes, across 3 Colibrí System pipettors (10 * 4 * 3 = 120 measurements).
• E. coli Suspensions Preparation Verification: The exact number of suspensions isn't provided, but it states "A variable number of colonies was selected... to create different suspensions... Three Colibrí Systems run by three different operators were included."
• Colony Picking and Microbial Suspensions for AST: 6 bacterial species (3 Gram-Negative and 3 Gram-Positive) grown in 2 polymicrobial mixtures on different culture media.
• AST Challenge Test:
  • Total Tested: 5991 (across various antibiotics and organism groups within the challenge test).
  • Evaluated MIC results: 1883 evaluable MIC results.
  • Organism groups: Enterobacterales (n=62 strains), Staphylococcus (n=16 strains), Streptococcus (n=30 strains), Enterococcus (n=16 strains), and non-fermenters (n=32 strains).
  • Strains: Both susceptible and resistant strains, exhibiting a range of on-scale MIC values.
  • Media: Trypticase Soy Agar + 5% Sheep Blood, MacConkey Agar, and Columbia agar + 5% sheep blood.
  • Incubation times: Varied (e.g., 14h, 24h for Enterobacterales/Non-fermenters, 18h for Staphylococcus/Enterococcus/Streptococcus).
  • Data Provenance: Not explicitly stated (e.g., country of origin, retrospective/prospective). However, given it's a 510(k) submission for a medical device manufacturer (Copan WASP S.r.l., Italy), it's highly likely to be internal, prospective studies conducted at their facilities or collaborator sites.
• Reproducibility Study: Each microorganism was tested with the appropriate antibiotic panel, with each condition tested in triplicate, for a total of 81 replicates for each combination strain-antimicrobial agent across three Colibrí Systems and three operators over three days.
• Purity Plate Growth: 2,364 purity plates.
• QC Sample Preparation: Conducted daily at the beginning of the working session on each instrument involved in the Analytical Studies.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

• This information is not explicitly provided in the document.
• For the AST Challenge Test, the "ground truth" (or comparator method) was the MICs obtained by the bioMérieux VITEK 2 using manual sample preparation, interpreted according to FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria. This implies the ground truth relies on established, validated laboratory methods and interpretations, rather than subjective expert consensus.
• For other analytical studies (e.g., nephelometer accuracy, pipettor precision), the ground truth generally relies on quantitative measurements using calibrated instruments and standard protocols (e.g., viable cell counts for McFarland turbidity verification, gravimetric measurements for pipettor accuracy).

4. Adjudication method for the test set

• This is not applicable in the context of this device's performance validation. The device automates a pre-analytical step. The performance is assessed by comparing its output (prepared microbial suspensions) to a reference method (manual preparation for VITEK 2). It's not a diagnostic AI system requiring expert adjudication of image interpretations or clinical diagnoses.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• No, an MRMC comparative effectiveness study was not done. This type of study is relevant for AI systems that assist human readers in interpreting medical images (e.g., radiology AI). The Colibrí System is a laboratory automation device for preparing samples. It does not involve human "readers" of AI outputs in a diagnostic context that would require such a study design.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

• The performance data are essentially standalone (algorithm/device only) in terms of its ability to perform the physical process of colonial picking and suspension preparation. The device's output is then fed into other IVD analyzers (VITEK 2, bioMérieux VITEK MS, Bruker MALDI Biotyper CA System) for identification and AST.
• The comparison in the AST Challenge Test is between the Colibrí System's automated preparation and manual preparation which is the existing standard. So, it's comparing automated device output to a manual, human-executed process, where the subsequent analysis (VITEK 2) is the same. The data provided (EA, CA, error rates) represent the performance of the Colibrí system's prepared samples, which is a form of standalone performance.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

• Comparatative Ground Truth/Reference Method: For the AST Challenge Test, the ground truth was derived from the MICs obtained by the bioMérieux VITEK 2 using manual sample preparation, interpreted according to FDA-Recognized Antimicrobial Susceptibility Test Interpretive Criteria. This is a well-established and standardized laboratory reference method.
• Quantitative Ground Truth: For other analytical studies, the ground truth was based on quantitative laboratory measurements, such as:
  • Viable cell counts (CFU/mL) for confirming bacterial concentration for nephelometry.
  • Gravimetric measurements for pipetting accuracy.
  • Visual inspection of purity plates and accepted microbiological methods to ensure monomicrobial suspensions and absence of contamination.

8. The sample size for the training set

• The document describes performance validation studies, not product development or AI model training. Therefore, information about a "training set" (in the context of machine learning) is not applicable or provided. This device automates a physical process, not a machine learning model that needs training data in the traditional sense. The "training" of the device likely refers to physical calibration and quality control.

9. How the ground truth for the training set was established

• As per point 8, the concept of a "training set" for an AI/machine learning model is not applicable to the description of this device's validation. The device's mechanics and software are validated against established engineering and microbiology standards.

Ask a specific question about this device

VITEK® MS PRIME is a mass spectrometry system using matrix-assisted laser desorption/ionization time of flight mass spectrometry (MALDI-TOF MS) for the identification of microorganisms cultured from human specimens.

The VITEK® MS PRIME system is a qualitative in vitro diagnostic device indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial and fungal infections.

This 510(k) submission introduces the VITEK®MS PRIME System. The VITEK® MS PRIME is intended for laboratory use by professional users who are trained in microbiology and good laboratory practices.

The VITEK® MS PRIME makes microorganism identifications via matrix-assisted laser desorption/ionization - time of flight mass spectrometry (MALDI-TOF MS) technology, which includes the three basic principles of ionization, separation, and detection,

As a first step, a VITEK® MS-DS Target Slide is prepared in accordance with the instructions for use.

NOTE: Depending on the culture, the analyte sample (i.e. microorganism from cultured media) may be directly spotted to a target slide, or for Mycobacterium. Nocardia and mould it must be processed/inactivated before adding to the target slide.

Once the specimen (cultured from the appropriate media) is spotted to the target slide, a matrix is added for the purpose of easy sublimation and strong absorbance in the laser wavelength employed by theinstrument.

NOTE: The VITEK® MS PRIME is a Class 1 laser product, containing a Class 4 Neodymium-doped yttrium lithium fluoride (Nd:YLF) laser – the laser operates at a wavelength of 349 nm.

The prepared slide is then loaded onto the VITEK®MS PRIME instrument. where a laser targets the sample spot and pulses the isolate spot, resulting in vibrational excitation of matrix and analyte molecules. The matrix transfer protons to the analyte resulting in a positive charge. So as part of the first basic principle, the ionized molecules are then accelerated in an electromagnetic field and a grid electrode in the ionization chamber.

The acceleration in the electromagnetic field is the beginning of the second basic principle (i.e. the separation process that is based of the time-of-flight principle). The velocity of the molecules depends on the mass-to-charge (m/z) ratio of the analyte, with heavier molecules having a higher moment of inertia resulting in a lower velocity.

As a final step in the basic principle of MALDI-ToF technology (i.e. detection) the time of flight is measured precisely by the ions arrival at a particle detector. This speed of the ions in flight depends on their mass - with heavier molecules having a higher moment of inertia resulting in a lower velocity. The time of transit is measured precisely by the ions' arrival at a particle detector. Based on the time of flight, the m/z ratio of each particle can be determined, and a mass spectrum of the analyte sample mixture is generated. The recorded signal is processed and presented as a spectrum of intensity versus mass in Daltons (Da). The mass spectrum displays results as a series of peaks (spectrum) which correspond to the ionized proteins derived from the analyte sample. And for identification of an unknown organism, the resulting mass spectra are sufficiently distinctive to allow taxonomic characterization at the genus and species when compared against the VITEK® MS Knowledge Base.

Here's a breakdown of the acceptance criteria and study information for the VITEK® MS PRIME, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance:

2. Sample Sizes Used for the Test Set and Data Provenance:

• Biological Equivalency Study:

  • Sample Size: 1461 samples (representing 487 unique tests in triplicate).
  • Data Provenance: Not explicitly stated, but the strains tested included "critical pathogens" for the 479 claimed species. It is likely a combination of well-characterized laboratory strains and potentially some clinical isolates, given the reference to "clinically validated isolates." retrospective or prospective is not mentioned.

• Clinical Performance Evaluation:

  • Sample Size: 500 clinical isolates (from 100 species with five strains each).
  • Data Provenance: "clinical isolates tested from all sites combined." The specific countries of origin are not specified, but the data is explicitly from "clinical isolates," suggesting data from human specimens. The study design of using "clinical isolates" usually implies retrospective or prospectively collected samples from clinical settings. It refers to "reference identification obtained during previous clinical studies," suggesting a retrospective use of previously characterized clinical data.

• Challenge Isolate Results:

  • Sample Size: 100 challenge strains.
  • Data Provenance: Not specified, but "challenge strains" often refers to a curated set of difficult-to-identify or representative strains used for rigorous testing.

• Quality Control Results:

  • Sample Size: Not explicitly stated, but refers to "all quality control strains tested at all sites."

• Reproducibility Results:

  • Sample Size: Not explicitly stated, but refers to "Reproducibility strains."

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications:

• The document does not directly state the number of experts or their qualifications.
• For the Clinical Performance Evaluation, the ground truth was established by "a one choice or multiple choice (more than one species) reference identification obtained during previous clinical studies." This implies that the reference identifications were well-established and accepted, likely through conventional microbiological methods and expert interpretation from those previous studies. The nature of these "previous clinical studies" (e.g., whether they involved expert consensus or a gold standard method) is not detailed.

4. Adjudication Method for the Test Set:

• The document does not explicitly describe an adjudication method for the test set results. The ground truth for the clinical performance evaluation was "reference identification obtained during previous clinical studies," implying that disagreements with a single reference standard were likely noted as discordant results rather than undergoing a separate adjudication process within this specific study.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:

• No MRMC comparative effectiveness study is mentioned. This device (VITEK® MS PRIME) is an automated system for microorganism identification using MALDI-TOF MS technology, which does not involve human readers interpreting results in the same way an imaging AI algorithm would. Its performance is compared to a reference method, not to human readers' performance with and without AI assistance.

6. Standalone Performance:

• Yes, a standalone performance study was done. Both the "Biological Equivalency study" and "Clinical Performance Evaluation" describe the performance of the VITEK® MS PRIME system alone (algorithm only) in identifying microorganisms, comparing its results to a ground truth or reference identification. The stated agreement rates (e.g., 98.4% clinical agreement) are measures of this standalone performance. The device is described as "a qualitative in vitro diagnostic device indicated for use in conjunction with other clinical and laboratory findings," but the performance metrics provided are for the device's identification capability itself.

7. Type of Ground Truth Used:

• The ground truth used appears to be reference identification by accepted microbiological methods (which implicitly includes expert consensus in their establishment).
  • For the Biological Equivalency study, performance was measured "in comparison with the reference method."
  • For the Clinical Performance Evaluation, performance was determined by comparing the VITEK® MS PRIME identification to "a one choice or multiple choice (more than one species) reference identification obtained during previous clinical studies." This "reference identification" would be established through a combination of traditional culture-based methods, molecular methods, and expert interpretation/consensus over time, serving as the gold standard for organism identification. "Outcomes data" or "pathology" as the direct ground truth are not mentioned for identifying the microorganisms themselves, though the device aids in diagnosis of infections.

8. Sample Size for the Training Set:

• The document does not explicitly state the sample size for the training set. It refers to a "VITEK® MS Knowledge Base" (KB v3.2) against which the mass spectra are compared. This knowledge base is the "training set" or reference library. The complexity and size of this knowledge base are not detailed in terms of number of samples/isolates used to build it.

9. How the Ground Truth for the Training Set Was Established:

• The document states that identifications are made "when compared against the VITEK® MS Knowledge Base." While it doesn't describe the exact process for building this KB, such knowledge bases for MALDI-TOF MS systems are typically built by:
  • Acquisition of mass spectra from a large collection of well-characterized and phenotypically/genetically confirmed (often by sequencing, biochemical tests, or other gold-standard methods) reference strains across various species.
  • Each reference strain's identity is verified by expert microbiologists using established methods before being added to the database.
  • The collection process ensures reproducibility and representation of intra-species variability.
  • Thus, the ground truth for the training set (Knowledge Base) is established through a rigorous process of expert-validated identification using traditional and molecular microbiological gold standards.

Ask a specific question about this device

The Colibrí System is an in vitro diagnostic device comprised of the Colibrí Vision System and Collbrí Preparation Station for use with the bioMérieux VITEK MS or Bruker MALDI Biotyper CA mass spectrometry systems for qualitative identification of isolated colonies of Gram-negative bacterial species grown on solid culture media. The Collbri System is a semi-automated pre-analytical processor that picks isolated by the operator and uses a pipetting system to prepare MALDI-TOF MS (Matrix-Assisted Laser Desorption/Ionization-Time Of Flight Mass Spectrometry) target slides. The Colibri software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzer.

The Colibrí System is intended for use by trained healthcare professionals in clinical laboratories in conjunction with other clinical and laboratory finding Gram staining, to aid in the diagnosis of bacterial infections.

The Colibrí System has not been validated for use in identification of yeast species.

The Copan Colibrí System is designed to be used as an accessory of the downstream MALDI-TOF analyzers automating various manual steps in the workflow for the preparation of samples for the identification of isolated colonies of microorganisms cultured from the human body.

The Colibrí System automates the preparation of MALDI target slides for the bioMérieux VITEK MS or the Bruker MALDI Biotyper CA System that are used in clinical laboratories for identification and differentiation of organisms grown on plated media by Matrix-Assisted Laser Desorption/Ionization Time-of Flight Mass Spectrometry (MALDI-TOF MS). The system comprises the Colibrí Vision System and Colibrí Preparation Station and pipette tips as consumables. After appropriate plate incubation, the operator using the graphical User Interface (Image Reading Interface) chooses the plates exhibiting adequate growth and selects the isolated colonies to be processed assigning the automatic ID tasks. By using the Collbrí Vision System, specific colonies to be picked are designated by the operator on a digital plate. The Operator manually loads the plates in the Colibri Preparation Station where colonies are automatically picked, spotted on the target slide and overlayed with the matrix.

When used in conjunction with the bioMérieux VITEK MS, the Colibrí System can prepare the 48spot target slides by performing the direct spotting of colonies. The calibrator used for quality control is manually applied by the operator at the end of the automated colony spotting. When used in conjunction with the Bruker MALDI Biotyper CA System, the Colibrí System can prepare either reusable 48-spot or disposable 96-spot targets by performing the Direct Transfer Sample Procedure. The BTS used for quality control is manually applied by the operator at the and of the automated colony spotting.

The Colibrí software records the identity of each sample and its position on the target slide and communicates this information electronically to the MALDI-TOF MS analyzers.

Colibri System requires three different calibrations. None of these calibration activities require user intervention if not in terms of periodical cleaning of the mechanical component as described in the dedicated section of the User Manual. Set-up calibration is performed during the device initial setup for the camera units positioned on the Colibrí Vision System and on the Colibrí Preparation Station. Auto-calibration is performed at the end of the initial set-up and periodically during the preventive maintenance to check that, in the Colibri Preparation Station, all the mechanical references can be found inside the positioning tolerances, that the I/Os are responsive. Run-time calibration is performed during the normal usage to automatically check the proper functioning of the Colibrí Vision System and the Colibri Preparation Station.

The provided text describes the performance data for the Colibrí System, an in vitro diagnostic device. The acceptance criteria are implicitly defined by the performance observed in various analytical studies, with the goal of demonstrating substantial equivalence to predicate devices for qualitative identification of isolated colonies of Gram-negative and Gram-positive bacterial species.

Here's an analysis of the acceptance criteria and study proving the device meets them:

1. Table of Acceptance Criteria (Implicit) and Reported Device Performance

The document does not explicitly state pre-defined acceptance criteria in a dedicated table with specific thresholds. However, based on the studies conducted, the implicit acceptance criteria appear to be:

• High agreement with expected identification (ground truth).
• Absence of wrong identifications (false positives).
• Maintenance of performance across different conditions (e.g., positional effects, culture age, different MALDI-TOF MS systems and target types, different operators, different Colibrí systems).
• Comparable performance to manual preparation.
• No cross-contamination.

Here's a table summarizing the reported device performance, which serves as the evidence that these implicit criteria were met. The percentages below represent the agreement with the expected ID (ground truth) for high-confidence results where available, or overall agreement as calculated by the study.

2. Sample Sizes Used for the Test Set and Data Provenance

• Test Set (Analytical Studies):

  • Colony Picking: 1390 spots (VITEK MS) and 1690 spots (Bruker MALDI Biotyper CA).
  • Positional Effect: 864 spots (VITEK MS), 864 spots (Bruker MALDI Biotyper CA US IVD 48 Spot), 1692 spots (Bruker MALDI Biotyper CA MBT Biotarget 96 US IVD).
  • Inclusivity: 392 spots/strains (VITEK MS) and 468 spots/strains for each Bruker MALDI Biotyper CA system (US IVD 48 Spot and MBT Biotarget 96 US IVD).
  • Specificity: 20 spots (VITEK MS) and 20 spots (Bruker MALDI Biotyper CA).
  • Reproducibility: 1800 spots (VITEK MS) and 1800 spots (Bruker MALDI Biotyper CA).
  • Cross-Contamination: 572 spots (VITEK MS) and 686 spots (Bruker MALDI Biotyper CA total for both systems).
  • Colony Stability: 576 spots (VITEK MS) and 1440 spots (Bruker MALDI Biotyper CA).

• Data Provenance: The document generally indicates "on-panel" and "off-panel" strains, often referring to common clinical isolates in the US. No explicit country of origin for the clinical samples/strains themselves is stated beyond generic "human specimens" or "commonly isolated Gram-positive and Gram-negative species in the US". The studies were conducted by a single operator (Inclusivity, Specificity, Cross-Contamination, Colony Stability) or two operators (Reproducibility) on multiple Colibrí systems (Reproducibility, Colony Picking, Inclusivity, Cross-Contamination) in laboratory settings. The studies are prospective as they are designed experiments to evaluate the device performance.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

• The document implies that the ground truth for identification was established by the expected strain identity of the stock cultures used in the analytical studies. This means the identities of the bacterial strains were known beforehand (e.g., ATCC strains or well-characterized lab strains).
• For the performance results themselves, identification was based on the output of the VITEK MS or Bruker MALDI Biotyper CA systems, which generate confidence values or log scores. The interpretation of these scores (e.g., what constitutes a "correct single choice") is intrinsic to the performance specifications of these established MALDI-TOF MS systems.
• No human experts (e.g., radiologists) were used to establish the ground truth for these microbial identification studies. The "ground truth" is the presumed identity of the bacterial strain used in the experiment.

4. Adjudication Method for the Test Set

• Not applicable in the conventional sense. Since the ground truth for the test set was the expected strain identity of known bacterial cultures, there was no need for human expert adjudication to resolve discrepancies in interpretation. The output of the MALDI-TOF MS system for each sample was compared directly to the known identity of the inoculated strain. Discrepancies (low confidence, no ID, wrong ID) were noted and analyzed against the overall performance.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done

• No. An MRMC study is typically performed for image-based diagnostic aids where multiple human readers interpret cases with and without AI assistance. This study involves microorganism identification, a laboratory process, not direct human interpretation of images for diagnosis.
• The study compares the performance of the automated Colibrí System to manual preparation (the standard of care, as listed in the predicate device comparison tables), which is a different type of comparison than human readers with/without AI assistance.
• Effect size of human reader improvement: Not applicable, as no human reader study was conducted. The comparison is between an automated sample preparation method (Colibrí) and a manual sample preparation method for laboratory identification.

6. If a Standalone (i.e. algorithm only without human-in-the loop performance) was Done

• Yes, indirectly. The Colibrí System is a semi-automated pre-analytical processor. While an operator designates the colony, the picking, spotting, and matrix application are automated.
• The performance metrics provided (e.g., % agreement, no wrong ID) are for the Colibrí System's ability to prepare samples for subsequent MALDI-TOF MS analysis, which then generates the identification result. The Colibrí software records location and transmits information to the MALDI-TOF MS analyzer.
• The study evaluated the effectiveness of the automated sample preparation step compared to manual preparation, which directly impacts the downstream MALDI-TOF MS identification. However, the identification itself is performed by the MALDI-TOF MS analyzer, not the Colibrí system's "algorithm" in isolation, except for tracking sample positions. The Colibrí's key "algorithm" is in its vision system and robotic picking/spotting, which impacts the quality of the sample for the MALDI-TOF MS.

7. The Type of Ground Truth Used

• Expected Strain Identity: The ground truth for the analytical studies was the known, expected identity of the bacterial strains used. These were presumably well-characterized laboratory strains (e.g., ATCC cultures) with a confirmed identity. This is a highly controlled form of ground truth.

8. The Sample Size for the Training Set

• Not explicitly stated in terms of a "training set" for an AI/machine learning model used within Colibrí's core functions. The Colibrí System is described as a "semi-automated pre-analytical processor" that uses "a pipetting system" and "Colibrí software records the identity of each sample and its position". It's unclear if the "Colibrí Vision System" employs complex machine learning that would require a dedicated training set beyond basic image processing for colony detection and localization.
• The document implies that the system is automating a manual process, meaning its "training" pertains more to engineering and calibration rather than a machine learning model for diagnosis. The performance studies focus on the system's accuracy in physical manipulation and compatibility with existing MALDI-TOF MS systems.

9. How the Ground Truth for the Training Set Was Established

• Not applicable as a distinct ML training set is not described. If the "Colibrí Vision System" used internal algorithms that were "trained" to recognize colony morphology for picking, the ground truth would likely have been established by manually annotating colonies on images or physical plates as "pickable" or "not pickable" based on expert microbiological judgment, but this is not mentioned in the provided text. The current text suggests the operator "chooses the plates exhibiting adequate growth and selects the isolated colonies to be processed assigning the automatic ID tasks" on a "digital plate", implying human-in-the-loop for key decisions.

Ask a specific question about this device

The BD Kiestra IdentifA module is an automated in vitro diagnostic specimen preparation system for use with the BD Kiestra Laboratory Automation Solution to prepare MALDI targets for the Bruker MALDI Biotyper CA System for the qualitative identification and differentiation of microorganisms using matrix-assisted laser desorption/ionization-time of flight mass spectrometry (MALDI-TOF MS) analysis of colonies grown on plated culture media from human specimens.

The BD Kiestra IdentifA is indicated for use in the clinical laboratory with the BD Kiestra Read Compact and Bruker MALDI Biotyper CA System to aid in the diagnosis of bacterial and fungal infections.

The BD Kiestra™ IdentifA is an instrument which automates picking of technologist-selected colonies from plated media and prepares a Bruker MALDI target for identification and differentiation of organisms. The BD Kiestra IdentifA includes the following components (Note: Bruker MALDI targets, Matrix and Bacterial Test Standard (BTS) are required, however, they are obtained directly from Bruker Daltonik GmbH):

• . BD Kiestra IdentifA instrument and software with onboard pipetting and nephelometry.
• . BD formic acid, deionized water, pipet tips, Matrix and BTS transfer vials.
• . BD Kiestra IdentifA nephelometer calibration standards (0.2, 0.5, 1.0 and 3.0 McFarland).
• BD Kiestra IdentifA cuvette array.

When a MALDI identification is ordered by a technologist selects the colonies from an image of a plated medium obtained using the BD Kiestra™ ReadA Compact. The coordinates of the colonies and the plated medium are transferred to BD Kiestra IdentifA where the colonies are picked. The colonies are suspended in deionized water and the onboard nephelometer determines the McFarland turbidity. Based on the McFarland, BD Kiestra IdentifA pipets the organism suspension onto a Bruker MALDI target. The BD Kiestra IdentifA uses the Bruker extended Direct Transfer method for preparation of the MALDI target by overlaying formic acid and Bruker Matrix onto the target spot. In addition, BTS spots are prepared on the target slide for quality control. Once dried, the technologist manually removes the target and loads onto the Bruker MALDI Biotyper CA System. The BD Kiestra IdentifA transfers the location of sample and BTS spots to the MALDI Biotyper CA. If requested by the technologist, BD Kiestra IdentifA will also dilute the organism suspension to a target of 0.5 McFarland.

The BD Kiestra IdentifA can be used as a standalone instrument or integrated into the BD Kiestra Laboratory Automation System. The standalone instrument utilizes an input/output module for manual plate loading, which handles de-stacking and stacking of plates. When physically integrated into the BD Kiestra Laboratory Automation System, BD Kiestra IdentifA is connected to a track by way of a connection module for automatic plate transfer. BD Kiestra IdentifA software is responsible for the instrument functionality and a touchscreen is mounted on the instrument for user interface.

Here's an analysis of the acceptance criteria and the studies performed for the BD Kiestra IdentifA device, based on the provided text:

Acceptance Criteria and Reported Device Performance

Study Details

2. Sample sizes used for the test set and the data provenance

• Colony Picking Accuracy: 200 mixed culture plates were used, from which 1200 colonies were selected and picked. The data provenance is not explicitly stated (e.g., country of origin), though it is noted as "internal analytical testing." The study design appears to be prospective (experimental).
• Organism Identification Accuracy: A total of 464 isolates of Gram-positive bacteria, Gram-negative bacteria, and yeasts were tested. The data provenance is not explicitly stated (e.g., country of origin), but it is referred to as "internal analytical testing." The study design appears to be prospective (experimental).
• Reproducibility: For each of the 15 strains, 27 tests were performed (3 days x 3 replicates x 3 instruments). This totals 15 strains * 27 tests/strain = 405 tests. The data provenance is not explicitly stated. The study design appears to be prospective (experimental).
• Limit of Detection: 9 organisms were tested, with 8 MALDI target spots inoculated for each. This totals 9 organisms * 8 spots/organism = 72 tests. The data provenance is not explicitly stated. The study design appears to be prospective (experimental).
• Cross-contamination: 100 plates inoculated with Staphylococcus aureus and 100 plates with Klebsiella pneumoniae. These 200 inoculated plates were alternated with 200 uninoculated media, for a total of 400 media processed. An additional field examination included "over 58,000 samples" processed in Europe since January 2020. The study design combined prospective experimental testing with retrospective field data analysis.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

• The text does not explicitly state the number of experts or their qualifications for establishing ground truth for the test set. However, for the colony picking accuracy, it mentions "Colonies from both isolates on each plate were selected by a technologist". The "Organism Identification" study implicitly used comparison to an "expected result," which would typically be based on a validated identification method and potentially confirmed by expert review, but this is not detailed. The "Reproducibility" study used "Strains with known identifications."

4. Adjudication method for the test set

• The document does not describe a formal adjudication method (like 2+1, 3+1) for disagreements or ambiguous cases in the test set. For the "Organism Identification Accuracy," results from the BD Kiestra IdentifA were "compared to the expected result for each isolate." For colony picking, it was confirmed visually and by the Bruker MALDI Biotyper CA identification.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• No, a multi-reader multi-case (MRMC) comparative effectiveness study focusing on human reader performance improvement with AI assistance was not done. This device is an automated specimen preparation system, not an AI for interpretation that would assist human readers in diagnosing. Its function is to automate the preparation stage for downstream MALDI-TOF MS analysis. The comparison is between automated preparation vs. manual preparation using the predicate device.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

• Yes, the performance studies (Colony Picking, Organism Identification Accuracy, Reproducibility, LoD, Cross-contamination) evaluate the standalone performance of the BD Kiestra IdentifA in preparing samples. The output of the BD Kiestra IdentifA is a prepared MALDI target, which is then loaded onto the Bruker MALDI Biotyper CA System for organism identification. While a technologist selects colonies from a digital image, the subsequent steps of picking, suspension, turbidity measurement, and spotting are automated without human intervention. The performance metrics are based on the results obtained from the prepared targets by the downstream MALDI-TOF MS system.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

• The ground truth primarily appears to be "expected identification" or "known identifications" of microorganisms. This implies an established and validated identification for each isolate, likely obtained through standard microbiological methods or a reference MALDI-TOF MS system. For the "Organism Identification Accuracy" study, the comparison was between the BD Kiestra IdentifA prepared samples and "manual sample preparation, i.e. performing the extended Direct Transfer (eDT) Procedure and spotting on a MALDI target according to the previously FDA-cleared Bruker MALDI Biotyper CA user manual." This suggests the "expected result" was either based on prior rigorous identification or the result from the manual predicate method.

8. The sample size for the training set

• The document does not explicitly state the sample size for a training set. As an automated specimen preparation system, it's possible its internal algorithms (e.g., for colony picking, turbidity estimation, pipetting precision) were developed and optimized using various datasets, but these "training sets" are not described in terms of size or content. The provided studies focus on validation/test set performance.

9. How the ground truth for the training set was established

• Not applicable, as a specific "training set" and its ground truth establishment are not described in the provided text. The device's function is mechanical automation of a known lab procedure, not an AI model that learns from training data in the same way. Any inherent 'intelligence' (e.g., image processing for colony identification) would have been programmed based on established features rather than learned through labeled training data.

Ask a specific question about this device

The MBT Sepsityper is a qualitative in vitro diagnostic device consisting of a MBT-CA (Sepsityper) software extension and a reagent kit (MBT Sepsityper Kit US IVD) for use in conjunction with other clinical and laboratory findings to aid in the early diagnosis of bacterial and yeast infections from positively flagged blood cultures using the MALDI Biotyper CA System.

The MBT Sepsityper Kit US IVD is a disposable blood culture processing device that includes associated reagents that are intended to concentrate and purify microbial cells from blood culture samples identified as positive by a continuous monitoring blood culture system and confirmed to demonstrate the presence of a single organism as determined by Gram stain. This sample preparation manual method is performed by laboratory health a clinical diagnostic setting.

Subculturing of positive blood cultures is necessary to recover organisms for identification of organisms not identified by the MBT-CA System, for susceptibility testing and for differentiation of mixed growth.

Positive MBT Sepsityper results do not rule out co-infection with organisms that may not be detected by the MBT-CA System. Results of the MBT Sepsityper should not be used as the sole basis for diagnosis, treatment, or other patient management decisions. Results of the MBT Sepsityper should be correlated with Gram stain results and used in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial and yeast bloodstream infections.

The MBT Sepsityper is a qualitative in vitro diagnostic device consisting of a MBT-CA (Sepsityper) software extension and a reagent kit (MBT Sepsityper Kit US IVD). The MBT Sepsityper Kit US IVD is a disposable blood culture processing device that includes associated reagents that are intended to concentrate and purify microbial cells from blood culture samples identified as positive by a continuous monitoring blood culture system and confirmed to demonstrate the presence of a single organism as determined by Gram stain. This sample preparation manual method is performed by laboratory health a clinical diagnostic setting.

The provided text is related to an FDA 510(k) clearance for a medical device (MBT Sepsityper) and primarily describes its indications for use, regulatory classification, and the types of organisms it can identify. It does NOT contain the detailed information necessary to fully address all parts of your request regarding acceptance criteria and the study proving the device meets those criteria.

Specifically, the document does not include:

• A table of acceptance criteria and reported device performance.
• Sample sizes used for test and training sets, or data provenance.
• Information on expert ground truth establishment (number of experts, qualifications, adjudication).
• Details about MRMC comparative effectiveness studies or standalone algorithm performance.
• Specifics on how ground truth was established for training or test sets (e.g., pathology, outcomes data).

Therefore, I can only address the parts for which information is implicitly or explicitly available in the provided text.

Based on the provided information, here's what can be gathered:

1. A table of acceptance criteria and the reported device performance:

This information is not provided in the document. The document is an FDA clearance letter, which typically summarizes the outcome of the review rather than providing the raw performance data or the detailed acceptance criteria used in the validation study.

2. Sample sized used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective):

This information is not provided in the document. The document mentions that the device is "for use in conjunction with other clinical and laboratory findings to aid in the early diagnosis of bacterial and yeast infections from positively flagged blood cultures." It also lists a wide range of bacteria and yeasts the device is intended to identify, implying that a significant amount of data was used for validation, but specific sample sizes and provenance are absent.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience):

This information is not provided in the document. For in vitro diagnostic microbiology devices like this, ground truth is typically established by definitive laboratory methods (e.g., sequencing, advanced biochemical tests) rather than expert human interpretation of images, but the specifics are not detailed here.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set:

This information is not provided in the document. Adjudication methods are typically relevant for studies involving human interpretation of complex medical images, which is not the primary function of this device.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

This information is not provided in the document. MRMC studies are generally applicable to imaging devices where human interpretation plays a significant role. This device is an in vitro diagnostic system for microorganism identification using mass spectrometry, not an AI-assisted diagnostic imaging tool.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

The device is described as "a qualitative in vitro diagnostic device consisting of a MBT-CA (Sepsityper) software extension and a reagent kit." This implies that the software performs the identification based on mass spectrometry data, making it a standalone algorithm in terms of the identification process itself. However, the clearance states it is "for use in conjunction with other clinical and laboratory findings," and that "results should not be used as the sole basis for diagnosis," indicating that human oversight and integration with other clinical data are required downstream. The document does not explicitly state if standalone performance metrics (e.g., sensitivity, specificity) of the algorithm alone were evaluated as a distinct part of the study, separate from its overall clinical utility.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

While not explicitly stated in detail, for an in vitro diagnostic device identifying microorganisms, the ground truth would almost certainly be established by definitive microbiological methods, such as:

• Reference culture methods: Gold standard growth and biochemical identification.
• Molecular methods: DNA sequencing (e.g., 16S rRNA gene sequencing for bacteria, ITS region sequencing for fungi) which provides highly accurate species-level identification.
  The document lists specific organisms the device can identify, implying that the ground truth for these organisms was established by such highly accurate methods.

8. The sample size for the training set:

This information is not provided in the document.

9. How the ground truth for the training set was established:

This information is not provided in the document, but similar to point 7, it would logically be established by definitive microbiological or molecular methods.

Ask a specific question about this device