Search Results

The MALDI Biotyper CA System is a mass spectrometer systems using matrix-assisted laser desorption/ionization - time of flight (MALDI-TOF) for the identification of microorganisms cultured from human specimens.

The MALDI Biotyper CA System is a qualitative in vitro diagnostic device indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial and yeast infections.

The MBT-CA System is a mass spectrometer system using matrix-assisted laser desorption/ionization - time of flight (MALDI-TOF) for the identification of microorganisms cultured from human specimens. The system uses a different methodology for organism identification based on unique protein patterns of the microorganisms obtained from mass spectrometry. The test organism's spectrum (a pattern of mass peaks) is compared with a reference spectra library (database). Using biostatistical analysis, a probability ranking of the organism identification is generated. The probability ranking is represented as a log(score) between 0.00 and 3.00. Organism identification is reported with high confidence if the log(score) is ≥2.00. An organism identification is reported with low confidence if the log(score) is between 1.70 and <2.00.

Here's a breakdown of the acceptance criteria and the study details for the MALDI Biotyper CA (MBT-CA) System, based on the provided document:

1. Table of Acceptance Criteria and Reported Device Performance

The provided document does not explicitly state pre-defined acceptance criteria in terms of numerical thresholds for sensitivity, specificity, or overall accuracy. Instead, it presents performance results from various studies (reproducibility, challenge panel, method comparison) and concludes that the device performs acceptably.

However, based on the reported performance in the Method Comparison study, common metrics for identification systems would be:

Note: The "acceptance criteria" presented above are inferred from the strong performance and conclusions drawn in the document, rather than explicitly stated numerical targets prior to testing.

2. Sample Size Used for the Test Set and Data Provenance

• Method Comparison Study (Overall Isolate Performance):

  • Sample Size: 2091 fresh and stored organisms.
  • Data Provenance: Organisms were tested at four (4) US clinical test sites and an in-house laboratory. Isolates were sub-cultured and sent to an interim reference laboratory and then to a sequencing reference laboratory for ground truth determination. This indicates prospective and retrospective data collection with a US origin.

• Reproducibility Study:

  • Sample Size: 9 unique organisms (REPRO-02 excluded). Each organism tested in duplicate, 5 days, 2 runs/day, 3 sites (9 organisms x 2 replicates x 5 days x 2 runs x 3 sites = 540 measurements). Total MBT-CA IDs for summary = 179/180 per site.
  • Data Provenance: Conducted at three (3) clinical study sites (US, likely, given the FDA submission context). The organisms were "well-characterized," suggesting they might be reference strains or previously identified clinical isolates.

• Challenge Panel Study:

  • Sample Size: 46 organisms.
  • Data Provenance: Selected from stored organisms from the clinical study, prepared by the interim reference laboratory. Tested at three (3) study sites (US, likely).

• Biological/Technical Equivalency Studies:

  • Sample Size: 34 species for laser equivalency (4080 spectra). Multiple species for target equivalency (e.g., 1000 measurements for repeatability/precision, 1500 for LOD, 2500 for sample stability prior to matrix, 3000 for post-matrix stability, 50 FDA cleared organisms, 1500 for mass accuracy/edge effects). Nocardia Study: 30 strains covering 6 species, resulting in ~15,000 measurements.
  • Data Provenance: Not explicitly stated for specific origin, but these are technical validation studies performed by the manufacturer, likely controlled lab settings.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications

• Method Comparison Study:

  • The ground truth was established by sequencing (16S rRNA or ITS sequencing and protein gene sequencing). This relies on established molecular biology techniques, not human expert interpretation. While experts run and interpret these sequences, the core ground truth is the genetic information itself. The document does not specify a number or qualification of "experts" in the sense of clinical reviewers for ground truth determination but implies reliance on the robust and objective results of gene sequencing performed by a sequencing reference laboratory.

• Reproducibility Study:

  • Organisms were "well-characterized." The ground truth was presumably established by prior definitive identification methods, likely including gene sequencing or reputable reference lab methods. No mention of independent experts for this study's ground truth.

• Challenge Panel Study:

  • Organism identifications were "blinded to test sites," and the panel was prepared by the study interim reference laboratory. The ground truth was established by the reference lab, again likely through gold standard methods like sequencing.

4. Adjudication Method for the Test Set

The document does not describe an explicit adjudication method involving multiple human readers or a specific consensus process for discrepancies in the generated log(scores) or identifications against a human-read ground truth. Instead:

• The ground truth for the organism identity itself (reference algorithm) was established by molecular sequencing.
• The device's log(score) provides a quantitative measure of confidence. If the log(score) is too low (<1.70), the result is "no ID." If it's between 1.70 and <2.00, it's "low confidence."
• For discrepancies in the device's workflow (e.g., if Direct Transfer yields a low score), the protocol specifies progressive steps (extended Direct Transfer, then Extraction procedure) until a high confidence log(score) is achieved or the organism remains unidentified. This is an internal algorithmic "adjudication" within the device's workflow.
• The final interpretation is stated to be by a "trained microbiologist," who should consider "all relevant information available" (Gram staining, colony morphology, growth characteristics, sample matrix, etc.) in conjunction with the device's results. This implies human oversight and potential adjudication of the device's output in a clinical setting, but not as part of the ground truth establishment for the studies described.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No MRMC comparative effectiveness study is described where human readers' performance with and without AI (MALDI-TOF) assistance is measured to determine an effect size. The studies focus on the standalone performance of the MALDI Biotyper CA System against a molecularly derived ground truth. The device is intended to aid in diagnosis, implying it would be used by humans, but the studies do not quantify the improvement in human performance with the device.

6. Standalone Performance Study (Algorithm Only)

Yes, extensive standalone performance studies were done. The "Method Comparison" study, "Reproducibility" study, and "Challenge Panel" study, as well as the various "Equivalency Studies," all evaluate the MALDI Biotyper CA System's performance (algorithm only, with human operation but without human interpretation as the ground truth) against a molecularly derived ground truth or against itself under different conditions. The tables showing "Overall Isolate Performance" (Table 6, 7, 8) are prime examples of standalone performance evaluation.

7. Type of Ground Truth Used

• Method Comparison and Challenge Panel Studies: The primary ground truth for organism identification was established by molecular sequencing, specifically 16S rRNA or ITS sequencing and protein gene sequencing.
• Reproducibility Study: Organisms were "well-characterized," implying a gold standard identification method, most likely molecular sequencing.
• Equivalency Studies: These compared the performance of different components of the MBT-CA system (e.g., laser types, target plates) against each other or against established performance metrics, rather than identifying unknown organisms from scratch using an external ground truth. The underlying ground truth for the organisms used in these studies would still be molecular identification.

8. Sample Size for the Training Set

The document does not explicitly state the sample size used for training the MBT-CA System's reference database. It mentions the "MALDI Biotyper Reference Library for Clinical Applications (MBT-CA)" and states it's an "update" (K163536 vs. K142677). This reference library is what the device compares new spectra against. The training set would be the collection of reference spectra used to build this database. The provided text refers to new organisms added to the library but does not give a total size or the methodology used to build the original or updated library from a training perspective.

9. How the Ground Truth for the Training Set Was Established

The document implies that the reference library is built from well-characterized and known organisms. For the specific organisms added to the claimed list (Table 5), they represent further development/validation. The initial development of the database entries would have likely involved:

• Definitive microbiological identification methods: Classic biochemical tests, microscopy, and, critically, molecular sequencing (e.g., 16S rRNA sequencing), which is mentioned as the ground truth method elsewhere (e.g., Acinetobacter baumannii complex resolution, Method Comparison).
• The "Matching Algorithm" calculates matches by comparing a new spectrum against each single reference entry of a reference database. This suggests that each entry in the database itself originates from a definitively identified organism whose mass spectrum was collected and included.

Ask a specific question about this device

The Bruker Daltonics, Inc MALDI Biotyper CA System is a qualitative in vitro diagnostic mass spectrometer system for the identification of Gram-negative bacterial colonies cultured from human specimens using matrix-assisted laser desorption/ ionization - time of flight (MALDI-TOF) mass spectrometry technology.

The MALDI Biotyper CA System is indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of Gram negative bacterial infections.

The MBT-CA System uses a different methodology for organism identification based on unique protein patterns of the microorganisms obtained from mass spectrometry. The test organism's spectrum (a pattern of mass peaks) is compared with a reference spectra library (database). Using biostatistical analysis, a probability ranking of the organism identification is generated. The probability ranking is represented as a log(score) between 0.00 and 3.00. Organism identification is reported with high confidence if the log(score) is ≥ 2.00. An organism identification is reported with low confidence if the log(score) is between 1.70 and <2.00.

The provided text describes the acceptance criteria and a detailed study for the MALDI Biotyper CA System, a mass spectrometer used for identifying Gram-negative bacterial colonies.

Here's the breakdown of the information requested:

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria are generally implied throughout the performance studies, aiming for high correct identification rates and low false identification/no identification rates for organisms within the claim, and "No Identification" for organisms outside the claim.

2. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

• Precision/Repeatability: 10 test organisms, 72 samples measured per organism (total 720 samples). Data provenance is not specified, but the study was performed in the US (implied by "Bruker US IVD Bacterial Test Standard (BTS)"). It appears to be prospective, laboratory-controlled testing.
• Limit of Detection: 7 frequently occurring clinically relevant test organisms. Each cell stock was diluted to a minimum of 5 concentrations and tested in duplicate (minimum 70 measurements). Data provenance is not specified, likely laboratory.
• Specificity (Phase 1): 16 strains of organisms not in the library, each tested in duplicate (DT + Ext), so 32 direct transfer and 32 extraction tests for identification (total 64). Data provenance is not specified, likely laboratory.
• Specificity (Phase 2): 6 strains of closely related organisms, each tested in duplicate (DT + Ext), so 12 direct transfer and 12 extraction tests for identification (total 24). Data provenance is not specified, likely laboratory.
• Mixed Culture: 1 target organism (Pseudomonas aeruginosa) with 4 non-target organisms. Tested at 4 concentration conditions, 32 measurements per condition (total 128 tests). Data provenance is not specified, likely laboratory.
• Media and Colony Stability: 7 Gram-negative organisms, 2 incubation time points (18h, 24h), 2 temperatures (18°C, 25°C), 4 media types. Each condition likely involves multiple replicates. The table shows totals of 288 tests per media type (DT) and 288 per media type (Ext). (Total 2304 tests displayed). Data provenance is not specified, likely laboratory.
• Influence of Agar Media: 4 agar media types, each inoculated alone (12 replicates), with target organism alone (12 replicates), and with target organism + agar (10-12 replicates). This involves 3 frequently occurring Enterobacteriacea and non-fermenting Gram Negative Bacteria (36 total replicates for agar alone. 36 replicates for target organism alone. ~36 replicates for target organism + agar). Data provenance is not specified, likely laboratory.
• Organism Stability Prior to MBT-CA Analysis: 3 common gram-negative bacteria. DT: 8 replicates per organism, 5 time points (120 tests). Ext: 6 replicates per time point for 5 time points (30 tests). Extracted Material: 2 extracts, 24 replicates per extract, 5 time points (240 tests). Data provenance is not specified, likely laboratory.
• Sample Stability overlaid with Matrix: 3 gram negative target organisms. Multiple conditions (temperature, humidity), 4 time points. Each condition involved 4 target spots of directly transferred, 4 spots of extracted, and 8 spots of matrix solution alone. (e.g., 24/24 or 18/24 for organisms, 8/8 for matrix). Data provenance is not specified, likely laboratory.
• Bacterial Test Standard (BTS) Stability:
  • Accelerated/Shipping: 3 lots of BTS, 8 vials/lot. Tested at 3 time intervals (1, 2, 3 weeks). Each time interval, 2 vials/lot reconstituted, 16 target plate positions + 3 cross-joint positions spotted (approx. 19 tests * 3 lots * 3 timepoints + initial = 171 tests).
  • Real-Time: 3 lots of BTS. Tested at 5 time intervals (3, 6, 9, 12, 18 months). (approx. 19 tests * 3 lots * 5 timepoints + initial = 285 tests).
  • In-Use: 1 lot, 4 vials. Tested at 6 time points (1-6 months). (approx. 8 replicates * 6 timepoints + initial = 56 tests).
  • Data provenance is not specified, likely laboratory.
• HCCA portioned (Matrix) Stability:
  • Accelerated/Shipping: 1 lot matrix. Tested at 6 time intervals (1, 2, 4, 7, 16 weeks). Each interval, 16 replicates of ID (DT/EXT methods) for two targets (32 total), plus matrix alone.
  • Real-Time: 3 lots matrix. Tested at 5 time intervals (3, 6, 9, 12, 18 months). 16 replicates of ID per target (32 total).
  • In-Use (Controlled RT): 3 lots matrix. Tested at 3 time intervals (1, 3, 7 days). 24 replicates of ID per target (48 total) plus 8 matrix alone.
  • In-Use (Stressed Temps): 1 lot matrix. Tested at 2 time intervals (6, 12 hours). 24 replicates of ID per target (48 total) plus 8 matrix alone.
  • Data provenance is not specified, likely laboratory.
• Carry-Over and Cross Contamination: 2 target plates. 2 frequently occurring Gram negative bacteria. Each target inoculated 4x via DT and Ext in alternating patterns. All sample positions overlaid with matrix, remaining unused positions as blanks. Repeated 4 times on each target plate (5 cycles total). Results show 16 test organisms per run and 26-27 matrix-only spots per run (approx. 240 organism identification tests and 265 matrix blanks across the summary tables). Data provenance is not specified, likely laboratory.
• Proficiency/Familiarity: 4 US study sites, multiple operators (16 operators shown in table). Each operator tested 5 QC organisms. The # samples passed varies per operator (e.g., 25/25, 30/31, 32/32), indicating varying numbers of replicates for these 5 QC organisms (ranging from 25 to 35 samples per operator for each method, DT and Ext).
• Reproducibility: 4 clinical study sites, 2 operators/site, 10 well-characterized organisms. Tested in duplicate via direct transfer. 80 samples per organism (4 sites * 2 operators * 5 days * 2 runs/day * 2 replicates = 80 per organism) (Total 800 tests for 10 organisms). Data provenance is US clinical sites.
• Challenge Panel: 100 organisms (80 from clinical study, 20 from strain collections). Tested at 5 study sites. Site E only received 87 samples. (4 sites * 100 organisms, 1 site * 87 organisms = 487 organism identifications). Data provenance is US clinical sites for some organisms, and strain collections for others.
• Method Comparison: 2263 fresh and stored isolates (Gram-negatives) tested in total. Performance compared to sequencing and biochemical identification. Data from 4 clinical test sites and Bruker (Bremen, Germany). Provenance is a mix of fresh (498 never frozen) and stored isolates, from clinical test sites (US) and potentially other sources. Appears to be prospective (fresh isolates) and retrospective (stored isolates).

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

The ground truth for the method comparison study (the primary clinical performance study) was established by:

• Sequencing: Specifically, "16s bi-directional sequencing" and "protein sequencing."
• Biochemical identification: "Vitek 2".
• The text does not specify the number or qualifications of experts who performed or interpreted these ground truth methods. It only mentions that the isolates were sent to a "sequencing reference laboratory" for sequencing in accordance with "MM-18 A guidelines" (which are CLSI M-18 guidelines for bacterial identification systems using nucleic acid sequencing).

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

The text does not describe an adjudication method for disagreements in ground truth or between the device and ground truth. It states that performance was "compared to sequencing and when necessary to biochemical identification (i.e: Vitek 2) and protein sequencing." There's no mention of a human review or adjudication process for conflicting results between these reference methods or between the device and the reference.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• No, an MRMC comparative effectiveness study was not done.
• This device is an automated mass spectrometry system for identifying microorganisms. It does not involve human "readers" or "AI assistance" in the typical sense applied to medical imaging or diagnostics where human interpretation is enhanced by AI. The device itself performs the identification based on mass spectrometry data and a biostatistical algorithm comparing to a reference library.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

• Yes, a standalone (algorithm only) performance study was done. The entire performance section (Precision, LOD, Specificity, Mixed Culture, Media/Colony Stability, Organism Stability, Matrix Stability, Carry-Over, Challenge Panel, Reproducibility, and Method Comparison) describes the performance of the MALDI Biotyper CA System itself, which is an automated system generating organism IDs with a log(score).
• The "Performance Data" and "Method Comparison" sections explicitly evaluate the device's ability to identify organisms compared to reference methods. The system reports an organism identification with high confidence if the log(score) is ≥ 2.00 and low confidence if the log(score) is between 1.70 and <2.00. This is the "algorithm only" performance.
• While the text mentions that "Results should be reviewed by a trained microbiologist and final organism identification should be based on all relevant information available," the reported performance metrics (e.g., 96.07% high confidence ID rate in method comparison) reflect the device's standalone output before any human review or integration with other clinical findings.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

The ground truth used for the method comparison study (the central performance evaluation) was a combination of:

• 16s bi-directional sequencing (primary molecular method)
• Biochemical identification (Vitek 2) (conventional phenotypic method, used "when necessary")
• Protein sequencing (another molecular method, used "when necessary")
  This constitutes a robust, laboratory-based reference standard for microbial identification.

8. The sample size for the training set

The document does not explicitly state the size of the training set used to build the MALDI Biotyper CA System's reference library (database). It mentions that the "spectrum of the unknown organism is first transformed into a peak list. Using a biostatistical algorithm, this peak list is compared to the reference peak lists of organisms in the reference library (database)."

The method comparison study states, "In total, 2263 fresh and stored isolates were tested to support the initial reference library claim." This phrasing suggests these 2263 isolates might have been used for validation of the library's performance, but not necessarily for its training or development, as the library would presumably exist prior to this validation study. The content does not elaborate on the training data used to create the specific reference library ("MALDI Biotyper for Clinical Applications (MBT-CA)") or the database itself.

9. How the ground truth for the training set was established

Since the size and specific details of the training set are not provided, the method for establishing its ground truth is also not explicitly detailed in this summary. However, it's reasonable to infer that the reference library would have been built using well-characterized microbial strains, with their identities confirmed by gold-standard methods similar to, or even more comprehensive than, those used for the test set (e.g., definitive sequencing and classical microbiological techniques). The primary ground truth for the system's performance validation relies on 16s bi-directional sequencing, biochemical methods, and protein sequencing.

Ask a specific question about this device