(231 days)
VITEK MS is a mass spectrometer system using matrix-assisted laser desorption/ionization time to flight (MALDI-TOF) for the identification of microorganisms cultured from human specimens.
The VITEK®MS is a qualitative in vitro diagnostic device indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial and yeast infections.
The VITEK® MS v2.0 system is a system consisting of kit reagents (VITEK MS-CHCA, VITEK MS-FA), VITEK MS-DS target slides, VITEK® MS Prep Station, Knowledge Base, software, and the VITEK®MS (original equipment manufacturer (OEM)-labeled Shimadzu AXIMA® Assurance mass spectrometer).
The VITEK® MS v2.0 system includes an OEM-labeled Shimadzu AXIMA® Assurance mass spectrometer linked to a reference database, referred to as Knowledge Base. Matrix assisted laser desorption ionization (MALDI) is the process used to ionize a sample in to the gas phase. A pulsed laser beam is directed on to the sample. Energy from the laser beam desorbs and ionizes the sample. Extraction plates provide high-voltage electrical fields to accelerate the ionized particles upwards through the time-of-flight (TOF) vacuum tube. An ion lens focuses the ions. Deflector plates steer the ions on a path towards the linear detector at the flight-tube. An ion gate blanks out low mass ions (for example, derived from the matrix). The detector detects the ions directly from the sample (lower-molecular weight ions followed by highermolecular weight ions). Ions hitting the detector cause an electrical signal which is recorded. The recorded signal is processed by the software and presented as a spectrum of intensity versus mass, in Daltons (Da).
During target ionization, mass spectra within a range of 2,000-20,000 Daltons are recorded in linear positive mode at a laser frequency of 50 Hz. For each interrogation, laser shots at different positions within the target well produce up to 100 mass profiles that are summed into a single, raw mass spectrum. The spectrum is then processed by baseline correction, de-noising, and peak detection to identify well-defined peaks. The list of these significant peaks is subjected to a proprietary process called "mass binning". The processed (binned) data are used to query the Knowledge Base to determine the unknown's taxonomic identity. These results are then provided in the form of a single, species-level (and sometimes subspecies-level) identification, a split (low discrimination) identification with up to four species-level alternatives displayed, or no identification.
VITEK®MS Device Acceptance Criteria and Study Summary
Here's an analysis of the VITEK®MS device's acceptance criteria and the study data provided, structured as requested:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria are not explicitly stated as distinct numerical thresholds in the provided document, but rather implied by the overall performance demonstrated in the clinical study and the defined operational characteristics. The key performance indicators for a diagnostic device like VITEK®MS are typically related to its accuracy in identifying organisms. Based on the clinical study results, here's a table summarizing the performance in terms of "Correct Identification":
| Performance Metric | Acceptance Criteria (Implied) | Reported Device Performance (Overall) |
|---|---|---|
| Combined Correct Identification | High percentage of correct identifications (single choice + low discrimination correct genus) | 93.6% (6614/7068) |
| - Gram-positive bacteria | High percentage of correct identifications | 93.5% (2110/2256) |
| - Gram-negative bacteria | High percentage of correct identifications | 92.8% (3391/3656) |
| - Yeasts | High percentage of correct identifications | 96.3% (1113/1156) |
| Reproducibility | High reproducibility rate | 99.7% (598/600) |
Note: Specific, pre-defined numerical acceptance criteria for each organism group (e.g., "Gram-positive bacteria must have >90% combined correct identification") are not explicitly listed as "acceptance criteria" in the text, but the demonstrated performance would be evaluated against what is considered clinically acceptable for such a device. The FDA's decision to classify it as Class II with special controls suggests these performance levels were deemed acceptable.
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size for the Test Set:
- Challenge Study: 300 isolates (3 panels of 100 strains each).
- Prospective Clinical Study: 7068 isolates in total, broken down as:
- Gram-positive bacteria: 2256 isolates
- Gram-negative bacteria: 3656 isolates
- Yeasts: 1156 isolates
- Data Provenance: Prospective clinical study data was collected from five clinical microbiology laboratories in the United States. The study involved patient cultures.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The document states that the reference identification for the prospective clinical study was "determined by molecular sequencing supplemented as needed by additional molecular sequencing and/or biochemical testing."
- Number of Experts: Not explicitly stated. The process "determined by molecular sequencing" implies laboratory personnel trained in molecular biology and biochemical identification methods, but not a specific count of "experts."
- Qualifications of Experts: Not explicitly stated, but inferred to be laboratory professionals skilled in molecular sequencing and biochemical testing for microbial identification. No specific years of experience or board certifications (e.g., radiologist) are mentioned, as this is a microbiology device.
4. Adjudication Method for the Test Set
The document does not explicitly describe a formal "adjudication method" in the sense of multiple experts reviewing and resolving discrepancies in the VITEK®MS results against a ground truth.
Instead, the process described for establishing ground truth is:
- "molecular sequencing supplemented as needed by additional molecular sequencing and/or biochemical testing."
- This suggests that molecular sequencing was the primary reference method, and if there were ambiguities or needs for further confirmation for the ground truth itself, additional molecular or biochemical tests were used. The document does not describe a process for resolving disagreements between the VITEK®MS result and the reference identification by a panel of human experts. Discrepancies are simply reported.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and Effect Size
No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not explicitly reported or described in the provided document. The study focuses on evaluating the standalone performance of the VITEK®MS device against a reference method (molecular sequencing), not on how human readers' performance improves with or without AI assistance.
6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Was Done
Yes, a standalone performance study was clearly done. The "Prospective Clinical Study" and the "Challenge Study" evaluate the VITEK®MS device's identification performance (algorithm only) against a reference standard. The results are presented as percentages of correct identification by the device itself. The device is described as a "mass spectrometer system for clinical use for the identification of microorganisms...a qualitative in vitro diagnostic device intended for the identification of microorganisms cultured from human specimens." Its output is an identification with a confidence value.
7. The Type of Ground Truth Used
The ground truth used for both the challenge study and the prospective clinical study was molecular sequencing supplemented as needed by additional molecular sequencing and/or biochemical testing.
8. The Sample Size for the Training Set
The training set details are described under "Knowledge Base":
- Reference database: Data representing 755 taxa, including 645 bacteria and 110 fungi.
- Isolates per species group: An average of 10 isolates (range 2-475) per species group.
- Reference spectra per species: An average of 40 reference spectra per species.
Therefore, the training set effectively included data from at least 755 taxa, with extensive spectral data (average 40 spectra per species) generated from these isolates. The total number of individual isolates used for training would be at least 755 taxa * (average 10 isolates/taxa) = ~7550 isolates if each isolate was distinct. The total number of spectra would be at least 755 taxa * (average 40 spectra/taxa) = ~30,200 spectra.
9. How the Ground Truth for the Training Set Was Established
For the training set (Knowledge Base development):
- Each reference isolate was grown on multiple media types under several growth conditions.
- Raw spectra were acquired by more than one technician using multiple instruments.
- The raw spectra were processed (baseline correction, de-noising, peak detection) to identify well-defined peaks.
- The list of significant peaks was subjected to a proprietary process called "mass binning."
- These processed (binned) data were then used to build the Knowledge Base, implying that the taxonomic identity of these reference isolates was established through standard microbiological and molecular methods prior to their inclusion in the database. While the specific methods for establishing the identity of each isolate for the knowledge base are not explicitly detailed in the same way as the clinical study's ground truth, it is understood that such reference strains would have their identities confirmed by established, reliable methods (e.g., 16S rRNA gene sequencing, traditional biochemical tests, etc.) before being used to build a diagnostic database.
§ 866.3361 Mass spectrometer system for clinical use for the identification of microorganisms.
(a)
Identification. A mass spectrometer system for clinical use for the identification of microorganisms is a qualitative in vitro diagnostic device intended for the identification of microorganisms cultured from human specimens. The device is comprised of an ionization source, a mass analyzer, and a spectral database. The device is indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial and fungal infections.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Premarket notification submissions must include detailed documentation for device software, including, but not limited to, standalone software applications and hardware-based devices that incorporate software.
(2) Premarket notification submissions must include database implementation methodology, construction parameters, and quality assurance protocols.
(3) A detailed explanation of the interpretation of results and acceptance criteria must be included in the device's 21 CFR 809.10(b)(9) compliant labeling.
(4) As part of the risk management activities performed as part of your 21 CFR 820.30 design controls, you must document an appropriate end user device training program that will be offered as part of your efforts to mitigate the risk of failure to correctly operate the instrument.
(5) Premarket notification submissions must include details on the appropriate end user device training program that will be offered while marketing the device.