Search Results

The MALDI Biotyper CA System is a mass spectrometer systems using matrix-assisted laser desorption/ionization - time of flight (MALDI-TOF) for the identification of microorganisms cultured from human specimens.

The MALDI Biotyper CA System is a qualitative in vitro diagnostic device indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial and yeast infections.

The MBT-CA System is a mass spectrometer system using matrix-assisted laser desorption/ionization - time of flight (MALDI-TOF) for the identification of microorganisms cultured from human specimens. The system uses a different methodology for organism identification based on unique protein patterns of the microorganisms obtained from mass spectrometry. The test organism's spectrum (a pattern of mass peaks) is compared with a reference spectra library (database). Using biostatistical analysis, a probability ranking of the organism identification is generated. The probability ranking is represented as a log(score) between 0.00 and 3.00. Organism identification is reported with high confidence if the log(score) is ≥2.00. An organism identification is reported with low confidence if the log(score) is between 1.70 and <2.00.

Here's a breakdown of the acceptance criteria and the study details for the MALDI Biotyper CA (MBT-CA) System, based on the provided document:

1. Table of Acceptance Criteria and Reported Device Performance

The provided document does not explicitly state pre-defined acceptance criteria in terms of numerical thresholds for sensitivity, specificity, or overall accuracy. Instead, it presents performance results from various studies (reproducibility, challenge panel, method comparison) and concludes that the device performs acceptably.

However, based on the reported performance in the Method Comparison study, common metrics for identification systems would be:

Note: The "acceptance criteria" presented above are inferred from the strong performance and conclusions drawn in the document, rather than explicitly stated numerical targets prior to testing.

2. Sample Size Used for the Test Set and Data Provenance

• Method Comparison Study (Overall Isolate Performance):

  • Sample Size: 2091 fresh and stored organisms.
  • Data Provenance: Organisms were tested at four (4) US clinical test sites and an in-house laboratory. Isolates were sub-cultured and sent to an interim reference laboratory and then to a sequencing reference laboratory for ground truth determination. This indicates prospective and retrospective data collection with a US origin.

• Reproducibility Study:

  • Sample Size: 9 unique organisms (REPRO-02 excluded). Each organism tested in duplicate, 5 days, 2 runs/day, 3 sites (9 organisms x 2 replicates x 5 days x 2 runs x 3 sites = 540 measurements). Total MBT-CA IDs for summary = 179/180 per site.
  • Data Provenance: Conducted at three (3) clinical study sites (US, likely, given the FDA submission context). The organisms were "well-characterized," suggesting they might be reference strains or previously identified clinical isolates.

• Challenge Panel Study:

  • Sample Size: 46 organisms.
  • Data Provenance: Selected from stored organisms from the clinical study, prepared by the interim reference laboratory. Tested at three (3) study sites (US, likely).

• Biological/Technical Equivalency Studies:

  • Sample Size: 34 species for laser equivalency (4080 spectra). Multiple species for target equivalency (e.g., 1000 measurements for repeatability/precision, 1500 for LOD, 2500 for sample stability prior to matrix, 3000 for post-matrix stability, 50 FDA cleared organisms, 1500 for mass accuracy/edge effects). Nocardia Study: 30 strains covering 6 species, resulting in ~15,000 measurements.
  • Data Provenance: Not explicitly stated for specific origin, but these are technical validation studies performed by the manufacturer, likely controlled lab settings.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications

• Method Comparison Study:

  • The ground truth was established by sequencing (16S rRNA or ITS sequencing and protein gene sequencing). This relies on established molecular biology techniques, not human expert interpretation. While experts run and interpret these sequences, the core ground truth is the genetic information itself. The document does not specify a number or qualification of "experts" in the sense of clinical reviewers for ground truth determination but implies reliance on the robust and objective results of gene sequencing performed by a sequencing reference laboratory.

• Reproducibility Study:

  • Organisms were "well-characterized." The ground truth was presumably established by prior definitive identification methods, likely including gene sequencing or reputable reference lab methods. No mention of independent experts for this study's ground truth.

• Challenge Panel Study:

  • Organism identifications were "blinded to test sites," and the panel was prepared by the study interim reference laboratory. The ground truth was established by the reference lab, again likely through gold standard methods like sequencing.

4. Adjudication Method for the Test Set

The document does not describe an explicit adjudication method involving multiple human readers or a specific consensus process for discrepancies in the generated log(scores) or identifications against a human-read ground truth. Instead:

• The ground truth for the organism identity itself (reference algorithm) was established by molecular sequencing.
• The device's log(score) provides a quantitative measure of confidence. If the log(score) is too low (<1.70), the result is "no ID." If it's between 1.70 and <2.00, it's "low confidence."
• For discrepancies in the device's workflow (e.g., if Direct Transfer yields a low score), the protocol specifies progressive steps (extended Direct Transfer, then Extraction procedure) until a high confidence log(score) is achieved or the organism remains unidentified. This is an internal algorithmic "adjudication" within the device's workflow.
• The final interpretation is stated to be by a "trained microbiologist," who should consider "all relevant information available" (Gram staining, colony morphology, growth characteristics, sample matrix, etc.) in conjunction with the device's results. This implies human oversight and potential adjudication of the device's output in a clinical setting, but not as part of the ground truth establishment for the studies described.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No MRMC comparative effectiveness study is described where human readers' performance with and without AI (MALDI-TOF) assistance is measured to determine an effect size. The studies focus on the standalone performance of the MALDI Biotyper CA System against a molecularly derived ground truth. The device is intended to aid in diagnosis, implying it would be used by humans, but the studies do not quantify the improvement in human performance with the device.

6. Standalone Performance Study (Algorithm Only)

Yes, extensive standalone performance studies were done. The "Method Comparison" study, "Reproducibility" study, and "Challenge Panel" study, as well as the various "Equivalency Studies," all evaluate the MALDI Biotyper CA System's performance (algorithm only, with human operation but without human interpretation as the ground truth) against a molecularly derived ground truth or against itself under different conditions. The tables showing "Overall Isolate Performance" (Table 6, 7, 8) are prime examples of standalone performance evaluation.

7. Type of Ground Truth Used

• Method Comparison and Challenge Panel Studies: The primary ground truth for organism identification was established by molecular sequencing, specifically 16S rRNA or ITS sequencing and protein gene sequencing.
• Reproducibility Study: Organisms were "well-characterized," implying a gold standard identification method, most likely molecular sequencing.
• Equivalency Studies: These compared the performance of different components of the MBT-CA system (e.g., laser types, target plates) against each other or against established performance metrics, rather than identifying unknown organisms from scratch using an external ground truth. The underlying ground truth for the organisms used in these studies would still be molecular identification.

8. Sample Size for the Training Set

The document does not explicitly state the sample size used for training the MBT-CA System's reference database. It mentions the "MALDI Biotyper Reference Library for Clinical Applications (MBT-CA)" and states it's an "update" (K163536 vs. K142677). This reference library is what the device compares new spectra against. The training set would be the collection of reference spectra used to build this database. The provided text refers to new organisms added to the library but does not give a total size or the methodology used to build the original or updated library from a training perspective.

9. How the Ground Truth for the Training Set Was Established

The document implies that the reference library is built from well-characterized and known organisms. For the specific organisms added to the claimed list (Table 5), they represent further development/validation. The initial development of the database entries would have likely involved:

• Definitive microbiological identification methods: Classic biochemical tests, microscopy, and, critically, molecular sequencing (e.g., 16S rRNA sequencing), which is mentioned as the ground truth method elsewhere (e.g., Acinetobacter baumannii complex resolution, Method Comparison).
• The "Matching Algorithm" calculates matches by comparing a new spectrum against each single reference entry of a reference database. This suggests that each entry in the database itself originates from a definitively identified organism whose mass spectrum was collected and included.

Ask a specific question about this device

VITEK® MS is a mass spectrometry system using matrix-assisted laser desorption/ionization – time of flight mass spectrometry (MALDI-TOF MS) for the identification of microorganisms cultured from human specimens.

The VITEK® MS is a qualitative in vitro diagnostic device in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial, yeast and mould infections.

VITEK® MS is mass spectrometry system using matrix-assisted laser desorption/ionization-time of flight mass spectrometry (MALDI-TOF MS) for the identification of microorganisms cultured from human specimens. The VITEK® MS is a qualitative in vitro diagnostic device indicated for use in coniunction with other clinical and laboratory findings to aid in the diagnosis of bacterial, yeast and mould infections.

The VITEK® MS is intended for laboratory use by professional users who are trained in microbiology and good laboratory practices.

This 510(k) is an update to the VITEK® MS (Mass Spectrometry) clinical knowledge base (KB v3.0.0) for the purposes of identifying Mycobacterium, Nocardia, and mould isolates. As the VITEK® MS KB v3.0.0 update includes new indications for use on the VITEK® MS system, new clinical data was required to establish safety and effectiveness. To account for the detection of higher mass peaks, relevant for some moulds and Mycobacterium, the VITEK® MS acquisition station software was optimized (in v1.5.0) .

Microorqanism identifications are are made via matrix-assisted laser desorption/ionization - time of flight mass spectrometry (MALDI-TOF MS) technology, which includes the three basic principles of ionization, separation, and detection. Depending on the isolate culture, the analyte sample may be directly spotted to a target slide, or for Mycobacterium, Nocardia and mould it must be processed/inactivated before adding to the target slide. Once spotted to the target slide, a matrix is added for the purpose of easy sublimation and strong absorbance in the laser wavelength employed by the instrument.

The slide is then loaded onto the instrument, where a laser targets the sample spot and pulses the isolate spot, resulting in vibrational excitation of matrix and analyte molecules. The matrix transfer protons to the analyte resulting in a positive charge. The ionized molecules are then accelerated in an electromagnetic field and a grid electrode in the ionization chamber. The velocity of the molecules depends on the mass-to-charge (m/z) ratio of the analyte, with heavier molecules having a higher moment of inertia resulting in a lower velocity.

The time of flight is measured precisely by the ions arrival at a particle detector. Based on the time of flight, the m/z ratio of each particle can be determined, and a mass spectrum of the analyte sample mixture is generated. The mass spectrum displays results as a series of peaks (spectrum) which correspond to the ionized proteins derived from the analyte sample. The mass spectra are sufficiently distinctive to allow taxonomic characterization at the genus and species.

The VITEK® MS system is a mass spectrometry system using matrix-assisted laser desorption/ionization – time of flight mass spectrometry (MALDI-TOF MS) for the identification of microorganisms cultured from human specimens. It is a qualitative in vitro diagnostic device used in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial, yeast, and mold infections. This summary specifically details the performance of the VITEK® MS with an updated clinical knowledge base (KB v3.0.0) for identifying Mycobacterium, Nocardia, and mold isolates.

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria for the VITEK® MS updated with KB v3.0.0 are implied by the reported performance, aiming for high agreement rates and very low error rates when compared against a reference method (molecular sequencing).

Reported Device Performance for Challenge Strains (Mycobacterium, Nocardia, and Mold Combined):

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size (Clinical Strains): A total of 2,695 isolates (Mycobacterium, Nocardia, and mold) were tested.
• Sample Size (Challenge Strains): 100 well-characterized challenge strains were included (50 molds, 35 Mycobacterium, and 15 Nocardia).
• Data Provenance: The study was conducted at five clinical sites. The document does not specify the country of origin of the data, but it is implied to be clinical data from human specimens. The data is prospective in the sense that it was collected specifically for this performance testing, but the isolates themselves would have been collected as part of routine clinical practice.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts

The document does not specify the number or qualifications of experts used to establish the ground truth beyond stating that the reference method was "molecular sequencing methods." This suggests that the ground truth was established through laboratory testing rather than human expert consensus on morphological characteristics, for example.

4. Adjudication Method for the Test Set

The document does not explicitly state an adjudication method. The performance evaluation was a direct comparison of the VITEK MS identification results to molecular sequencing methods.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done

No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not done. This device identifies microorganisms and does not involve human readers interpreting images or data directly in a way that an MRMC study would typically assess. The performance is assessed on the accuracy of the device's identification against a reference standard.

6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was Done

Yes, the performance study described is a standalone performance assessment of the VITEK® MS system. The device automatically generates an identification based on MALDI-TOF MS data, and its results are compared directly to a molecular sequencing reference method. While professional users trained in microbiology operate the device and interpret its output in a laboratory setting, the core performance reported (agreement, error, no-ID rates) reflects the algorithm's capability without direct human interpretation influencing the primary identification result itself.

7. The Type of Ground Truth Used

The ground truth used was molecular sequencing methods. This is considered a highly accurate and definitive method for identifying microorganisms at the species level. Clinical strains included samples from both solid and liquid culture media.

8. The Sample Size for the Training Set

The document does not explicitly state the sample size for the training set. It refers to the "VITEK® MS clinical knowledge base (KB v3.0.0)," implying a database of known mass spectra used for identification. The development of this knowledge base would have involved a large and diverse collection of characterized organisms, but the specific numbers for the training set are not provided in this summary.

9. How the Ground Truth for the Training Set was Established

The document does not explicitly detail how the ground truth for the training set (clinical knowledge base) was established. However, for such diagnostic platforms, the ground truth for training data is typically established through a combination of:

• Reference methods: Including molecular sequencing (e.g., 16S rRNA gene sequencing for bacteria, ITS sequencing for fungi), conventional biochemical tests, and expert microbiological characterization.
• Pure, well-characterized strains: Often sourced from culture collections (e.g., ATCC).
• Clinical isolates: Confirmed by multiple reliable methods.

The knowledge base would be built by acquiring mass spectral profiles from these ground-truthed organisms.

Ask a specific question about this device

The Bruker Daltonics, Inc. MALDI Biotyper CA System is a mass spectrometer system using matrix-assisted laser desorption/ionization – time-of-flight (MALDI-TOF) for the identification of microorganisms cultured from human specimens.

The MALDI Biotyper CA System is a qualitative in vitro diagnostic device indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial and yeast infections.

The Bruker Daltonics, Inc. MALDI Biotyper CA System is a mass spectrometer system using matrix-assisted laser desorption/ionization – time-of-flight (MALDI-TOF) for the identification of microorganisms cultured from human specimens. The system measures the time (in the nanosecond range) between pulsed acceleration and the corresponding detector signal, the speed is converted into an exact molecular mass. The mass-to-charge ratio of an ion is proportional to the square of its drift time. Highly abundant microbial proteins (mainly ribosomal proteins) result in a mass spectrum with characteristic mass and intensity distribution. It is specific for many bacteria and is interpreted as a molecular fingerprint to identify the test organism.

Here's an analysis of the provided text to extract the acceptance criteria and study details:

1. Table of Acceptance Criteria and Reported Device Performance

The document doesn't explicitly state "acceptance criteria" in a formal table format with numerical targets. However, based on the performance data presented, the implicit acceptance criteria for identification confidence (log(score) ≥ 2.00) are assessed. The reported performance for "Overall Isolate Performance" (Table 6) shows the aggregate performance across all claims.

2. Sample Sizes and Data Provenance

• Test Set Sample Size: 4,395 fresh and stored organisms, generating 4,399 data points for the method comparison study (which serves as the primary test set for demonstrating performance against a reference). This also includes 3,802 replicate testing results reported separately.
• Data Provenance: The method comparison study was performed at six (6) US clinical test sites and an in-house laboratory. The organisms included both fresh and stored organisms. The document indicates that due to the rarity of some organisms, replicates were tested by multiple sites.

3. Number of Experts and Qualifications for Ground Truth of Test Set

The document does not explicitly state the number of experts or their specific qualifications (e.g., "radiologist with 10 years of experience") for establishing the ground truth of the test set organisms. However, it indicates:

• "The interim reference laboratory stored all organisms included in the study and sent all organisms to the sequencing reference laboratory for sequencing and protein sequencing when requested."
• This implies that a reference laboratory, likely staffed by expert microbiologists and molecular biologists, was responsible for confirming the identity of the organisms used as ground truth, primarily through sequencing and protein sequencing.

4. Adjudication Method for the Test Set

The document does not mention a specific adjudication method like "2+1" or "3+1" for discrepancies in the test set. The ground truth was established by a "sequencing reference laboratory for sequencing and protein sequencing." This suggests that the final identification was based on these molecular methods, rather than an expert consensus process between multiple readers.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not done. This device is an automated mass spectrometry system for identifying microorganisms. The evaluation focuses on the device's ability to identify organisms compared to a reference method (sequencing/protein sequencing), not on how it assists human readers or improves their performance.

6. Standalone (Algorithm Only) Performance

Yes, a standalone performance study was done. The entire Method Comparison section (pages 20-25) describes the performance of the MALDI Biotyper CA System (the algorithm/device) in identifying microorganisms without human intervention in the identification process other than following the described workflow. The log(score) system is an output of the algorithm.

7. Type of Ground Truth Used

The primary type of ground truth used for the test set was molecular sequencing (sequencing and protein sequencing), which is considered a highly accurate and definitive method for microorganism identification. This was handled by a "sequencing reference laboratory."

8. Sample Size for the Training Set

The document does not explicitly state the sample size for the training set. It refers to a "reference spectra library (database)" against which the test organism's spectrum is compared. This library is implicitly the training data, but its specific size or composition is not detailed in terms of number of unique isolates or spectra.

9. How the Ground Truth for the Training Set Was Established

The document states: "Calculates matches by comparing a new spectrum against each single reference entry of a reference database." It also mentions "MALDI Biotyper for Clinical Applications (MBT-CA)" as the database. While it doesn't describe the exact process for establishing ground truth for each entry within that database, it's generally understood that such reference libraries are built using highly characterized strains whose identities have been confirmed through gold-standard methods (e.g., 16S rRNA gene sequencing, whole-genome sequencing, or polyphasic taxonomy). The context implies that the internal reference library's entries serve as the established ground truth for comparison.

Ask a specific question about this device

The Bruker Daltonics, Inc MALDI Biotyper CA System is a qualitative in vitro diagnostic mass spectrometer system for the identification of Gram-negative bacterial colonies cultured from human specimens using matrix-assisted laser desorption/ ionization - time of flight (MALDI-TOF) mass spectrometry technology.

The MALDI Biotyper CA System is indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of Gram negative bacterial infections.

The MBT-CA System uses a different methodology for organism identification based on unique protein patterns of the microorganisms obtained from mass spectrometry. The test organism's spectrum (a pattern of mass peaks) is compared with a reference spectra library (database). Using biostatistical analysis, a probability ranking of the organism identification is generated. The probability ranking is represented as a log(score) between 0.00 and 3.00. Organism identification is reported with high confidence if the log(score) is ≥ 2.00. An organism identification is reported with low confidence if the log(score) is between 1.70 and <2.00.

The provided text describes the acceptance criteria and a detailed study for the MALDI Biotyper CA System, a mass spectrometer used for identifying Gram-negative bacterial colonies.

Here's the breakdown of the information requested:

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria are generally implied throughout the performance studies, aiming for high correct identification rates and low false identification/no identification rates for organisms within the claim, and "No Identification" for organisms outside the claim.

2. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

• Precision/Repeatability: 10 test organisms, 72 samples measured per organism (total 720 samples). Data provenance is not specified, but the study was performed in the US (implied by "Bruker US IVD Bacterial Test Standard (BTS)"). It appears to be prospective, laboratory-controlled testing.
• Limit of Detection: 7 frequently occurring clinically relevant test organisms. Each cell stock was diluted to a minimum of 5 concentrations and tested in duplicate (minimum 70 measurements). Data provenance is not specified, likely laboratory.
• Specificity (Phase 1): 16 strains of organisms not in the library, each tested in duplicate (DT + Ext), so 32 direct transfer and 32 extraction tests for identification (total 64). Data provenance is not specified, likely laboratory.
• Specificity (Phase 2): 6 strains of closely related organisms, each tested in duplicate (DT + Ext), so 12 direct transfer and 12 extraction tests for identification (total 24). Data provenance is not specified, likely laboratory.
• Mixed Culture: 1 target organism (Pseudomonas aeruginosa) with 4 non-target organisms. Tested at 4 concentration conditions, 32 measurements per condition (total 128 tests). Data provenance is not specified, likely laboratory.
• Media and Colony Stability: 7 Gram-negative organisms, 2 incubation time points (18h, 24h), 2 temperatures (18°C, 25°C), 4 media types. Each condition likely involves multiple replicates. The table shows totals of 288 tests per media type (DT) and 288 per media type (Ext). (Total 2304 tests displayed). Data provenance is not specified, likely laboratory.
• Influence of Agar Media: 4 agar media types, each inoculated alone (12 replicates), with target organism alone (12 replicates), and with target organism + agar (10-12 replicates). This involves 3 frequently occurring Enterobacteriacea and non-fermenting Gram Negative Bacteria (36 total replicates for agar alone. 36 replicates for target organism alone. ~36 replicates for target organism + agar). Data provenance is not specified, likely laboratory.
• Organism Stability Prior to MBT-CA Analysis: 3 common gram-negative bacteria. DT: 8 replicates per organism, 5 time points (120 tests). Ext: 6 replicates per time point for 5 time points (30 tests). Extracted Material: 2 extracts, 24 replicates per extract, 5 time points (240 tests). Data provenance is not specified, likely laboratory.
• Sample Stability overlaid with Matrix: 3 gram negative target organisms. Multiple conditions (temperature, humidity), 4 time points. Each condition involved 4 target spots of directly transferred, 4 spots of extracted, and 8 spots of matrix solution alone. (e.g., 24/24 or 18/24 for organisms, 8/8 for matrix). Data provenance is not specified, likely laboratory.
• Bacterial Test Standard (BTS) Stability:
  • Accelerated/Shipping: 3 lots of BTS, 8 vials/lot. Tested at 3 time intervals (1, 2, 3 weeks). Each time interval, 2 vials/lot reconstituted, 16 target plate positions + 3 cross-joint positions spotted (approx. 19 tests * 3 lots * 3 timepoints + initial = 171 tests).
  • Real-Time: 3 lots of BTS. Tested at 5 time intervals (3, 6, 9, 12, 18 months). (approx. 19 tests * 3 lots * 5 timepoints + initial = 285 tests).
  • In-Use: 1 lot, 4 vials. Tested at 6 time points (1-6 months). (approx. 8 replicates * 6 timepoints + initial = 56 tests).
  • Data provenance is not specified, likely laboratory.
• HCCA portioned (Matrix) Stability:
  • Accelerated/Shipping: 1 lot matrix. Tested at 6 time intervals (1, 2, 4, 7, 16 weeks). Each interval, 16 replicates of ID (DT/EXT methods) for two targets (32 total), plus matrix alone.
  • Real-Time: 3 lots matrix. Tested at 5 time intervals (3, 6, 9, 12, 18 months). 16 replicates of ID per target (32 total).
  • In-Use (Controlled RT): 3 lots matrix. Tested at 3 time intervals (1, 3, 7 days). 24 replicates of ID per target (48 total) plus 8 matrix alone.
  • In-Use (Stressed Temps): 1 lot matrix. Tested at 2 time intervals (6, 12 hours). 24 replicates of ID per target (48 total) plus 8 matrix alone.
  • Data provenance is not specified, likely laboratory.
• Carry-Over and Cross Contamination: 2 target plates. 2 frequently occurring Gram negative bacteria. Each target inoculated 4x via DT and Ext in alternating patterns. All sample positions overlaid with matrix, remaining unused positions as blanks. Repeated 4 times on each target plate (5 cycles total). Results show 16 test organisms per run and 26-27 matrix-only spots per run (approx. 240 organism identification tests and 265 matrix blanks across the summary tables). Data provenance is not specified, likely laboratory.
• Proficiency/Familiarity: 4 US study sites, multiple operators (16 operators shown in table). Each operator tested 5 QC organisms. The # samples passed varies per operator (e.g., 25/25, 30/31, 32/32), indicating varying numbers of replicates for these 5 QC organisms (ranging from 25 to 35 samples per operator for each method, DT and Ext).
• Reproducibility: 4 clinical study sites, 2 operators/site, 10 well-characterized organisms. Tested in duplicate via direct transfer. 80 samples per organism (4 sites * 2 operators * 5 days * 2 runs/day * 2 replicates = 80 per organism) (Total 800 tests for 10 organisms). Data provenance is US clinical sites.
• Challenge Panel: 100 organisms (80 from clinical study, 20 from strain collections). Tested at 5 study sites. Site E only received 87 samples. (4 sites * 100 organisms, 1 site * 87 organisms = 487 organism identifications). Data provenance is US clinical sites for some organisms, and strain collections for others.
• Method Comparison: 2263 fresh and stored isolates (Gram-negatives) tested in total. Performance compared to sequencing and biochemical identification. Data from 4 clinical test sites and Bruker (Bremen, Germany). Provenance is a mix of fresh (498 never frozen) and stored isolates, from clinical test sites (US) and potentially other sources. Appears to be prospective (fresh isolates) and retrospective (stored isolates).

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

The ground truth for the method comparison study (the primary clinical performance study) was established by:

• Sequencing: Specifically, "16s bi-directional sequencing" and "protein sequencing."
• Biochemical identification: "Vitek 2".
• The text does not specify the number or qualifications of experts who performed or interpreted these ground truth methods. It only mentions that the isolates were sent to a "sequencing reference laboratory" for sequencing in accordance with "MM-18 A guidelines" (which are CLSI M-18 guidelines for bacterial identification systems using nucleic acid sequencing).

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

The text does not describe an adjudication method for disagreements in ground truth or between the device and ground truth. It states that performance was "compared to sequencing and when necessary to biochemical identification (i.e: Vitek 2) and protein sequencing." There's no mention of a human review or adjudication process for conflicting results between these reference methods or between the device and the reference.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• No, an MRMC comparative effectiveness study was not done.
• This device is an automated mass spectrometry system for identifying microorganisms. It does not involve human "readers" or "AI assistance" in the typical sense applied to medical imaging or diagnostics where human interpretation is enhanced by AI. The device itself performs the identification based on mass spectrometry data and a biostatistical algorithm comparing to a reference library.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

• Yes, a standalone (algorithm only) performance study was done. The entire performance section (Precision, LOD, Specificity, Mixed Culture, Media/Colony Stability, Organism Stability, Matrix Stability, Carry-Over, Challenge Panel, Reproducibility, and Method Comparison) describes the performance of the MALDI Biotyper CA System itself, which is an automated system generating organism IDs with a log(score).
• The "Performance Data" and "Method Comparison" sections explicitly evaluate the device's ability to identify organisms compared to reference methods. The system reports an organism identification with high confidence if the log(score) is ≥ 2.00 and low confidence if the log(score) is between 1.70 and <2.00. This is the "algorithm only" performance.
• While the text mentions that "Results should be reviewed by a trained microbiologist and final organism identification should be based on all relevant information available," the reported performance metrics (e.g., 96.07% high confidence ID rate in method comparison) reflect the device's standalone output before any human review or integration with other clinical findings.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

The ground truth used for the method comparison study (the central performance evaluation) was a combination of:

• 16s bi-directional sequencing (primary molecular method)
• Biochemical identification (Vitek 2) (conventional phenotypic method, used "when necessary")
• Protein sequencing (another molecular method, used "when necessary")
  This constitutes a robust, laboratory-based reference standard for microbial identification.

8. The sample size for the training set

The document does not explicitly state the size of the training set used to build the MALDI Biotyper CA System's reference library (database). It mentions that the "spectrum of the unknown organism is first transformed into a peak list. Using a biostatistical algorithm, this peak list is compared to the reference peak lists of organisms in the reference library (database)."

The method comparison study states, "In total, 2263 fresh and stored isolates were tested to support the initial reference library claim." This phrasing suggests these 2263 isolates might have been used for validation of the library's performance, but not necessarily for its training or development, as the library would presumably exist prior to this validation study. The content does not elaborate on the training data used to create the specific reference library ("MALDI Biotyper for Clinical Applications (MBT-CA)") or the database itself.

9. How the ground truth for the training set was established

Since the size and specific details of the training set are not provided, the method for establishing its ground truth is also not explicitly detailed in this summary. However, it's reasonable to infer that the reference library would have been built using well-characterized microbial strains, with their identities confirmed by gold-standard methods similar to, or even more comprehensive than, those used for the test set (e.g., definitive sequencing and classical microbiological techniques). The primary ground truth for the system's performance validation relies on 16s bi-directional sequencing, biochemical methods, and protein sequencing.

Ask a specific question about this device

VITEK MS is a mass spectrometer system using matrix-assisted laser desorption/ionization time to flight (MALDI-TOF) for the identification of microorganisms cultured from human specimens.

The VITEK®MS is a qualitative in vitro diagnostic device indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial and yeast infections.

The VITEK® MS v2.0 system is a system consisting of kit reagents (VITEK MS-CHCA, VITEK MS-FA), VITEK MS-DS target slides, VITEK® MS Prep Station, Knowledge Base, software, and the VITEK®MS (original equipment manufacturer (OEM)-labeled Shimadzu AXIMA® Assurance mass spectrometer).

The VITEK® MS v2.0 system includes an OEM-labeled Shimadzu AXIMA® Assurance mass spectrometer linked to a reference database, referred to as Knowledge Base. Matrix assisted laser desorption ionization (MALDI) is the process used to ionize a sample in to the gas phase. A pulsed laser beam is directed on to the sample. Energy from the laser beam desorbs and ionizes the sample. Extraction plates provide high-voltage electrical fields to accelerate the ionized particles upwards through the time-of-flight (TOF) vacuum tube. An ion lens focuses the ions. Deflector plates steer the ions on a path towards the linear detector at the flight-tube. An ion gate blanks out low mass ions (for example, derived from the matrix). The detector detects the ions directly from the sample (lower-molecular weight ions followed by highermolecular weight ions). Ions hitting the detector cause an electrical signal which is recorded. The recorded signal is processed by the software and presented as a spectrum of intensity versus mass, in Daltons (Da).

During target ionization, mass spectra within a range of 2,000-20,000 Daltons are recorded in linear positive mode at a laser frequency of 50 Hz. For each interrogation, laser shots at different positions within the target well produce up to 100 mass profiles that are summed into a single, raw mass spectrum. The spectrum is then processed by baseline correction, de-noising, and peak detection to identify well-defined peaks. The list of these significant peaks is subjected to a proprietary process called "mass binning". The processed (binned) data are used to query the Knowledge Base to determine the unknown's taxonomic identity. These results are then provided in the form of a single, species-level (and sometimes subspecies-level) identification, a split (low discrimination) identification with up to four species-level alternatives displayed, or no identification.

VITEK®MS Device Acceptance Criteria and Study Summary

Here's an analysis of the VITEK®MS device's acceptance criteria and the study data provided, structured as requested:

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria are not explicitly stated as distinct numerical thresholds in the provided document, but rather implied by the overall performance demonstrated in the clinical study and the defined operational characteristics. The key performance indicators for a diagnostic device like VITEK®MS are typically related to its accuracy in identifying organisms. Based on the clinical study results, here's a table summarizing the performance in terms of "Correct Identification":

Note: Specific, pre-defined numerical acceptance criteria for each organism group (e.g., "Gram-positive bacteria must have >90% combined correct identification") are not explicitly listed as "acceptance criteria" in the text, but the demonstrated performance would be evaluated against what is considered clinically acceptable for such a device. The FDA's decision to classify it as Class II with special controls suggests these performance levels were deemed acceptable.

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size for the Test Set:
  • Challenge Study: 300 isolates (3 panels of 100 strains each).
  • Prospective Clinical Study: 7068 isolates in total, broken down as:
    • Gram-positive bacteria: 2256 isolates
    • Gram-negative bacteria: 3656 isolates
    • Yeasts: 1156 isolates
• Data Provenance: Prospective clinical study data was collected from five clinical microbiology laboratories in the United States. The study involved patient cultures.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

The document states that the reference identification for the prospective clinical study was "determined by molecular sequencing supplemented as needed by additional molecular sequencing and/or biochemical testing."

• Number of Experts: Not explicitly stated. The process "determined by molecular sequencing" implies laboratory personnel trained in molecular biology and biochemical identification methods, but not a specific count of "experts."
• Qualifications of Experts: Not explicitly stated, but inferred to be laboratory professionals skilled in molecular sequencing and biochemical testing for microbial identification. No specific years of experience or board certifications (e.g., radiologist) are mentioned, as this is a microbiology device.

4. Adjudication Method for the Test Set

The document does not explicitly describe a formal "adjudication method" in the sense of multiple experts reviewing and resolving discrepancies in the VITEK®MS results against a ground truth.

Instead, the process described for establishing ground truth is:

• "molecular sequencing supplemented as needed by additional molecular sequencing and/or biochemical testing."
• This suggests that molecular sequencing was the primary reference method, and if there were ambiguities or needs for further confirmation for the ground truth itself, additional molecular or biochemical tests were used. The document does not describe a process for resolving disagreements between the VITEK®MS result and the reference identification by a panel of human experts. Discrepancies are simply reported.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and Effect Size

No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not explicitly reported or described in the provided document. The study focuses on evaluating the standalone performance of the VITEK®MS device against a reference method (molecular sequencing), not on how human readers' performance improves with or without AI assistance.

6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Was Done

Yes, a standalone performance study was clearly done. The "Prospective Clinical Study" and the "Challenge Study" evaluate the VITEK®MS device's identification performance (algorithm only) against a reference standard. The results are presented as percentages of correct identification by the device itself. The device is described as a "mass spectrometer system for clinical use for the identification of microorganisms...a qualitative in vitro diagnostic device intended for the identification of microorganisms cultured from human specimens." Its output is an identification with a confidence value.

7. The Type of Ground Truth Used

The ground truth used for both the challenge study and the prospective clinical study was molecular sequencing supplemented as needed by additional molecular sequencing and/or biochemical testing.

8. The Sample Size for the Training Set

The training set details are described under "Knowledge Base":

• Reference database: Data representing 755 taxa, including 645 bacteria and 110 fungi.
• Isolates per species group: An average of 10 isolates (range 2-475) per species group.
• Reference spectra per species: An average of 40 reference spectra per species.

Therefore, the training set effectively included data from at least 755 taxa, with extensive spectral data (average 40 spectra per species) generated from these isolates. The total number of individual isolates used for training would be at least 755 taxa * (average 10 isolates/taxa) = ~7550 isolates if each isolate was distinct. The total number of spectra would be at least 755 taxa * (average 40 spectra/taxa) = ~30,200 spectra.

9. How the Ground Truth for the Training Set Was Established

For the training set (Knowledge Base development):

• Each reference isolate was grown on multiple media types under several growth conditions.
• Raw spectra were acquired by more than one technician using multiple instruments.
• The raw spectra were processed (baseline correction, de-noising, peak detection) to identify well-defined peaks.
• The list of significant peaks was subjected to a proprietary process called "mass binning."
• These processed (binned) data were then used to build the Knowledge Base, implying that the taxonomic identity of these reference isolates was established through standard microbiological and molecular methods prior to their inclusion in the database. While the specific methods for establishing the identity of each isolate for the knowledge base are not explicitly detailed in the same way as the clinical study's ground truth, it is understood that such reference strains would have their identities confirmed by established, reliable methods (e.g., 16S rRNA gene sequencing, traditional biochemical tests, etc.) before being used to build a diagnostic database.

Ask a specific question about this device