Search Results

The Bruker Daltonics, Inc. MALDI Biotyper CA System is a mass spectrometer system using matrix-assisted laser desorption/ionization – time-of-flight (MALDI-TOF) for the identification of microorganisms cultured from human specimens.

The MALDI Biotyper CA System is a qualitative in vitro diagnostic device indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial and yeast infections.

The Bruker Daltonics, Inc. MALDI Biotyper CA System is a mass spectrometer system using matrix-assisted laser desorption/ionization – time-of-flight (MALDI-TOF) for the identification of microorganisms cultured from human specimens. The system measures the time (in the nanosecond range) between pulsed acceleration and the corresponding detector signal, the speed is converted into an exact molecular mass. The mass-to-charge ratio of an ion is proportional to the square of its drift time. Highly abundant microbial proteins (mainly ribosomal proteins) result in a mass spectrum with characteristic mass and intensity distribution. It is specific for many bacteria and is interpreted as a molecular fingerprint to identify the test organism.

Here's an analysis of the provided text to extract the acceptance criteria and study details:

1. Table of Acceptance Criteria and Reported Device Performance

The document doesn't explicitly state "acceptance criteria" in a formal table format with numerical targets. However, based on the performance data presented, the implicit acceptance criteria for identification confidence (log(score) ≥ 2.00) are assessed. The reported performance for "Overall Isolate Performance" (Table 6) shows the aggregate performance across all claims.

2. Sample Sizes and Data Provenance

• Test Set Sample Size: 4,395 fresh and stored organisms, generating 4,399 data points for the method comparison study (which serves as the primary test set for demonstrating performance against a reference). This also includes 3,802 replicate testing results reported separately.
• Data Provenance: The method comparison study was performed at six (6) US clinical test sites and an in-house laboratory. The organisms included both fresh and stored organisms. The document indicates that due to the rarity of some organisms, replicates were tested by multiple sites.

3. Number of Experts and Qualifications for Ground Truth of Test Set

The document does not explicitly state the number of experts or their specific qualifications (e.g., "radiologist with 10 years of experience") for establishing the ground truth of the test set organisms. However, it indicates:

• "The interim reference laboratory stored all organisms included in the study and sent all organisms to the sequencing reference laboratory for sequencing and protein sequencing when requested."
• This implies that a reference laboratory, likely staffed by expert microbiologists and molecular biologists, was responsible for confirming the identity of the organisms used as ground truth, primarily through sequencing and protein sequencing.

4. Adjudication Method for the Test Set

The document does not mention a specific adjudication method like "2+1" or "3+1" for discrepancies in the test set. The ground truth was established by a "sequencing reference laboratory for sequencing and protein sequencing." This suggests that the final identification was based on these molecular methods, rather than an expert consensus process between multiple readers.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not done. This device is an automated mass spectrometry system for identifying microorganisms. The evaluation focuses on the device's ability to identify organisms compared to a reference method (sequencing/protein sequencing), not on how it assists human readers or improves their performance.

6. Standalone (Algorithm Only) Performance

Yes, a standalone performance study was done. The entire Method Comparison section (pages 20-25) describes the performance of the MALDI Biotyper CA System (the algorithm/device) in identifying microorganisms without human intervention in the identification process other than following the described workflow. The log(score) system is an output of the algorithm.

7. Type of Ground Truth Used

The primary type of ground truth used for the test set was molecular sequencing (sequencing and protein sequencing), which is considered a highly accurate and definitive method for microorganism identification. This was handled by a "sequencing reference laboratory."

8. Sample Size for the Training Set

The document does not explicitly state the sample size for the training set. It refers to a "reference spectra library (database)" against which the test organism's spectrum is compared. This library is implicitly the training data, but its specific size or composition is not detailed in terms of number of unique isolates or spectra.

9. How the Ground Truth for the Training Set Was Established

The document states: "Calculates matches by comparing a new spectrum against each single reference entry of a reference database." It also mentions "MALDI Biotyper for Clinical Applications (MBT-CA)" as the database. While it doesn't describe the exact process for establishing ground truth for each entry within that database, it's generally understood that such reference libraries are built using highly characterized strains whose identities have been confirmed through gold-standard methods (e.g., 16S rRNA gene sequencing, whole-genome sequencing, or polyphasic taxonomy). The context implies that the internal reference library's entries serve as the established ground truth for comparison.

Ask a specific question about this device

The Bruker Daltonics, Inc MALDI Biotyper CA System is a qualitative in vitro diagnostic mass spectrometer system for the identification of Gram-negative bacterial colonies cultured from human specimens using matrix-assisted laser desorption/ ionization - time of flight (MALDI-TOF) mass spectrometry technology.

The MALDI Biotyper CA System is indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of Gram negative bacterial infections.

The MBT-CA System uses a different methodology for organism identification based on unique protein patterns of the microorganisms obtained from mass spectrometry. The test organism's spectrum (a pattern of mass peaks) is compared with a reference spectra library (database). Using biostatistical analysis, a probability ranking of the organism identification is generated. The probability ranking is represented as a log(score) between 0.00 and 3.00. Organism identification is reported with high confidence if the log(score) is ≥ 2.00. An organism identification is reported with low confidence if the log(score) is between 1.70 and <2.00.

The provided text describes the acceptance criteria and a detailed study for the MALDI Biotyper CA System, a mass spectrometer used for identifying Gram-negative bacterial colonies.

Here's the breakdown of the information requested:

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria are generally implied throughout the performance studies, aiming for high correct identification rates and low false identification/no identification rates for organisms within the claim, and "No Identification" for organisms outside the claim.

2. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

• Precision/Repeatability: 10 test organisms, 72 samples measured per organism (total 720 samples). Data provenance is not specified, but the study was performed in the US (implied by "Bruker US IVD Bacterial Test Standard (BTS)"). It appears to be prospective, laboratory-controlled testing.
• Limit of Detection: 7 frequently occurring clinically relevant test organisms. Each cell stock was diluted to a minimum of 5 concentrations and tested in duplicate (minimum 70 measurements). Data provenance is not specified, likely laboratory.
• Specificity (Phase 1): 16 strains of organisms not in the library, each tested in duplicate (DT + Ext), so 32 direct transfer and 32 extraction tests for identification (total 64). Data provenance is not specified, likely laboratory.
• Specificity (Phase 2): 6 strains of closely related organisms, each tested in duplicate (DT + Ext), so 12 direct transfer and 12 extraction tests for identification (total 24). Data provenance is not specified, likely laboratory.
• Mixed Culture: 1 target organism (Pseudomonas aeruginosa) with 4 non-target organisms. Tested at 4 concentration conditions, 32 measurements per condition (total 128 tests). Data provenance is not specified, likely laboratory.
• Media and Colony Stability: 7 Gram-negative organisms, 2 incubation time points (18h, 24h), 2 temperatures (18°C, 25°C), 4 media types. Each condition likely involves multiple replicates. The table shows totals of 288 tests per media type (DT) and 288 per media type (Ext). (Total 2304 tests displayed). Data provenance is not specified, likely laboratory.
• Influence of Agar Media: 4 agar media types, each inoculated alone (12 replicates), with target organism alone (12 replicates), and with target organism + agar (10-12 replicates). This involves 3 frequently occurring Enterobacteriacea and non-fermenting Gram Negative Bacteria (36 total replicates for agar alone. 36 replicates for target organism alone. ~36 replicates for target organism + agar). Data provenance is not specified, likely laboratory.
• Organism Stability Prior to MBT-CA Analysis: 3 common gram-negative bacteria. DT: 8 replicates per organism, 5 time points (120 tests). Ext: 6 replicates per time point for 5 time points (30 tests). Extracted Material: 2 extracts, 24 replicates per extract, 5 time points (240 tests). Data provenance is not specified, likely laboratory.
• Sample Stability overlaid with Matrix: 3 gram negative target organisms. Multiple conditions (temperature, humidity), 4 time points. Each condition involved 4 target spots of directly transferred, 4 spots of extracted, and 8 spots of matrix solution alone. (e.g., 24/24 or 18/24 for organisms, 8/8 for matrix). Data provenance is not specified, likely laboratory.
• Bacterial Test Standard (BTS) Stability:
  • Accelerated/Shipping: 3 lots of BTS, 8 vials/lot. Tested at 3 time intervals (1, 2, 3 weeks). Each time interval, 2 vials/lot reconstituted, 16 target plate positions + 3 cross-joint positions spotted (approx. 19 tests * 3 lots * 3 timepoints + initial = 171 tests).
  • Real-Time: 3 lots of BTS. Tested at 5 time intervals (3, 6, 9, 12, 18 months). (approx. 19 tests * 3 lots * 5 timepoints + initial = 285 tests).
  • In-Use: 1 lot, 4 vials. Tested at 6 time points (1-6 months). (approx. 8 replicates * 6 timepoints + initial = 56 tests).
  • Data provenance is not specified, likely laboratory.
• HCCA portioned (Matrix) Stability:
  • Accelerated/Shipping: 1 lot matrix. Tested at 6 time intervals (1, 2, 4, 7, 16 weeks). Each interval, 16 replicates of ID (DT/EXT methods) for two targets (32 total), plus matrix alone.
  • Real-Time: 3 lots matrix. Tested at 5 time intervals (3, 6, 9, 12, 18 months). 16 replicates of ID per target (32 total).
  • In-Use (Controlled RT): 3 lots matrix. Tested at 3 time intervals (1, 3, 7 days). 24 replicates of ID per target (48 total) plus 8 matrix alone.
  • In-Use (Stressed Temps): 1 lot matrix. Tested at 2 time intervals (6, 12 hours). 24 replicates of ID per target (48 total) plus 8 matrix alone.
  • Data provenance is not specified, likely laboratory.
• Carry-Over and Cross Contamination: 2 target plates. 2 frequently occurring Gram negative bacteria. Each target inoculated 4x via DT and Ext in alternating patterns. All sample positions overlaid with matrix, remaining unused positions as blanks. Repeated 4 times on each target plate (5 cycles total). Results show 16 test organisms per run and 26-27 matrix-only spots per run (approx. 240 organism identification tests and 265 matrix blanks across the summary tables). Data provenance is not specified, likely laboratory.
• Proficiency/Familiarity: 4 US study sites, multiple operators (16 operators shown in table). Each operator tested 5 QC organisms. The # samples passed varies per operator (e.g., 25/25, 30/31, 32/32), indicating varying numbers of replicates for these 5 QC organisms (ranging from 25 to 35 samples per operator for each method, DT and Ext).
• Reproducibility: 4 clinical study sites, 2 operators/site, 10 well-characterized organisms. Tested in duplicate via direct transfer. 80 samples per organism (4 sites * 2 operators * 5 days * 2 runs/day * 2 replicates = 80 per organism) (Total 800 tests for 10 organisms). Data provenance is US clinical sites.
• Challenge Panel: 100 organisms (80 from clinical study, 20 from strain collections). Tested at 5 study sites. Site E only received 87 samples. (4 sites * 100 organisms, 1 site * 87 organisms = 487 organism identifications). Data provenance is US clinical sites for some organisms, and strain collections for others.
• Method Comparison: 2263 fresh and stored isolates (Gram-negatives) tested in total. Performance compared to sequencing and biochemical identification. Data from 4 clinical test sites and Bruker (Bremen, Germany). Provenance is a mix of fresh (498 never frozen) and stored isolates, from clinical test sites (US) and potentially other sources. Appears to be prospective (fresh isolates) and retrospective (stored isolates).

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

The ground truth for the method comparison study (the primary clinical performance study) was established by:

• Sequencing: Specifically, "16s bi-directional sequencing" and "protein sequencing."
• Biochemical identification: "Vitek 2".
• The text does not specify the number or qualifications of experts who performed or interpreted these ground truth methods. It only mentions that the isolates were sent to a "sequencing reference laboratory" for sequencing in accordance with "MM-18 A guidelines" (which are CLSI M-18 guidelines for bacterial identification systems using nucleic acid sequencing).

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

The text does not describe an adjudication method for disagreements in ground truth or between the device and ground truth. It states that performance was "compared to sequencing and when necessary to biochemical identification (i.e: Vitek 2) and protein sequencing." There's no mention of a human review or adjudication process for conflicting results between these reference methods or between the device and the reference.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• No, an MRMC comparative effectiveness study was not done.
• This device is an automated mass spectrometry system for identifying microorganisms. It does not involve human "readers" or "AI assistance" in the typical sense applied to medical imaging or diagnostics where human interpretation is enhanced by AI. The device itself performs the identification based on mass spectrometry data and a biostatistical algorithm comparing to a reference library.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

• Yes, a standalone (algorithm only) performance study was done. The entire performance section (Precision, LOD, Specificity, Mixed Culture, Media/Colony Stability, Organism Stability, Matrix Stability, Carry-Over, Challenge Panel, Reproducibility, and Method Comparison) describes the performance of the MALDI Biotyper CA System itself, which is an automated system generating organism IDs with a log(score).
• The "Performance Data" and "Method Comparison" sections explicitly evaluate the device's ability to identify organisms compared to reference methods. The system reports an organism identification with high confidence if the log(score) is ≥ 2.00 and low confidence if the log(score) is between 1.70 and <2.00. This is the "algorithm only" performance.
• While the text mentions that "Results should be reviewed by a trained microbiologist and final organism identification should be based on all relevant information available," the reported performance metrics (e.g., 96.07% high confidence ID rate in method comparison) reflect the device's standalone output before any human review or integration with other clinical findings.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

The ground truth used for the method comparison study (the central performance evaluation) was a combination of:

• 16s bi-directional sequencing (primary molecular method)
• Biochemical identification (Vitek 2) (conventional phenotypic method, used "when necessary")
• Protein sequencing (another molecular method, used "when necessary")
  This constitutes a robust, laboratory-based reference standard for microbial identification.

8. The sample size for the training set

The document does not explicitly state the size of the training set used to build the MALDI Biotyper CA System's reference library (database). It mentions that the "spectrum of the unknown organism is first transformed into a peak list. Using a biostatistical algorithm, this peak list is compared to the reference peak lists of organisms in the reference library (database)."

The method comparison study states, "In total, 2263 fresh and stored isolates were tested to support the initial reference library claim." This phrasing suggests these 2263 isolates might have been used for validation of the library's performance, but not necessarily for its training or development, as the library would presumably exist prior to this validation study. The content does not elaborate on the training data used to create the specific reference library ("MALDI Biotyper for Clinical Applications (MBT-CA)") or the database itself.

9. How the ground truth for the training set was established

Since the size and specific details of the training set are not provided, the method for establishing its ground truth is also not explicitly detailed in this summary. However, it's reasonable to infer that the reference library would have been built using well-characterized microbial strains, with their identities confirmed by gold-standard methods similar to, or even more comprehensive than, those used for the test set (e.g., definitive sequencing and classical microbiological techniques). The primary ground truth for the system's performance validation relies on 16s bi-directional sequencing, biochemical methods, and protein sequencing.

Ask a specific question about this device