Search Results
Found 4 results
510(k) Data Aggregation
(90 days)
QBN
The MBT Sepsityper is a qualitative in vitro diagnostic device consisting of an MBT-CA (Sepsityper) software extension and a reagent kit (MBT Sepsityper Kit US IVD) for use in conjunction with other clinical and laboratory findings to aid in the early diagnosis of bacterial and yeast infections from positively flagged blood cultures using the MALDI Biotyper CA System.
The MBT Sepsityper Kit US IVD is a disposable blood culture processing device that includes associated reagents that are intended to concentrate and purify microbial cells from blood culture samples identified as positive by a continuous monitoring blood culture system and confirmed to demonstrate the presence of a single organism as determined by Gram stain. This sample preparation manual method is performed by laboratory health professionals in a clinical diagnostic setting. Subculturing of positive blood cultures is necessary to recover organisms for identification of organisms not identified by the MBT-CA System, for susceptibility testing and for differentiation of mixed growth.
Positive MBT Sepsityper results do not rule out co-infection with organisms that may not be detected by the MBT-CA System. Results of the MBT Sepsityper should not be used as the sole basis for diagnosis, treatment, or other patient management decisions. Results of the MBT Sepsityper should be correlated with Gram stain results and used in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial and yeast bloodstream infections.
Organisms recovered from positive blood culture bottles that are suitable for identification using the MBT Sepsityper Kit US IVD and MBT-CA Systems are listed in the MALDI Biotyper CA System Package Insert Reference Library.
The MALDI Biotyper CA System uses MALDI (matrix-assisted laser desorption/ionization) TOF (time of flight) mass spectrometry technology for the identification of organisms isolated from clinical samples. Identification can be performed from an isolated colony or from a cell extract. The sample material is transferred to a target plate, dried and overlaid with a matrix. The MBT FAST Shuttle US IVD is an optional hardware tool that may be used for drying the samples deposited on the MALDI target plate under controlled conditions.
The MALDI process transforms the proteins and peptides from the isolated microorganisms into positively charged ions. This is achieved by irradiating the matrix-sample composite with a UV laser. The matrix absorbs laser energy and transfers protons to the intact proteins or peptides in the gas phase. These ions are electrostatically accelerated and arrive in the flight tube at a mass-dependent speed. Because different proteins/peptides have different masses, ions arrive at the detector at different times (time of flight). The MBT-CA System measures the time (in the nanosecond range) between pulsed acceleration and the corresponding detector signal of the ions, and the time is converted into an exact molecular mass.
The highly abundant microbial ribosomal proteins result in a mass spectrum with a characteristic mass and intensity distribution pattern. This pattern is species-specific for many bacteria and yeasts and can be used as a 'molecular fingerprint' to identify a test organism. The spectrum of the unknown test organism, acquired through the software MBT Compass HT CA of the MBT-CA System, is electronically transformed into a peak list. Using a biostatistical algorithm, this peak list is compared to reference peak lists of organisms in the MBT-CA Reference Library and a log(score) between 0.00 and 3.00 is calculated. The higher the log(score), the higher the degree of similarity to a given organism in the MBT-CA Reference Library. The log(score) ranges reflect the probability of organism identification.
The FDA 510(k) submission document focuses on demonstrating substantial equivalence to an existing predicate device rather than presenting a traditional acceptance criteria study for a new device. Therefore, the "acceptance criteria" discussed are largely driven by proving that the new components (MBT Compass HT CA software and MBT FAST Shuttle US IVD) maintain or improve the performance and safety established by the predicate device.
Here's an analysis of the provided text to fulfill your request:
Acceptance Criteria and Reported Device Performance
The concept of "acceptance criteria" in this context isn't a single set of predefined thresholds for a novel device's performance against a clinical gold standard (e.g., sensitivity/specificity targets). Instead, it's about demonstrating that the new components do not negatively impact the established performance of the predicate device and potentially offer improvements (like accelerated drying time). The "reported device performance" is presented as evidence that these conditions are met.
Table 1: Acceptance Criteria (Implied) and Reported Device Performance
Acceptance Criteria (Implied from Study Design) | Reported Device Performance |
---|---|
MBT FAST Shuttle US IVD: | |
Safety and Compliance: Meets EMC, Electrical, Mechanical, and Thermal standards. | Complies with IEC 61326-1, IEC 61326-2-6, IEC 60601-1-2 regarding EMC. Complies with IEC 61010-1, IEC 61010-2-010, IEC 61010-2-101 and national versions (UL, CAN/CSA) for safety. |
Repeatability of Microorganism Identification: High percentage of correct identifications across different preparation methods and workflows. | MBT Workflow: Overall average 96.67% correct ID. Sepsityper Workflow: Overall average 100% correct ID. (Table 3) |
Reproducibility of Microorganism Identification: Consistent correct identification across sites, FAST Shuttle units, operators, and days. | Site-to-Site: MBT: 96.9% correct; Sepsityper: 89.1% correct. (Table 4) |
FAST Shuttle-to-FAST Shuttle: MBT: 95.6% correct; Sepsityper: 98.3% correct. (Table 5) | |
Operator-to-Operator: MBT: 98.3% correct; Sepsityper: 98.9% correct. (Table 6) | |
Day-to-Day: MBT: 98.2% correct; Sepsityper: 98.9% correct. (Table 7) | |
Equivalence of Drying Methods (FAST Shuttle vs. Air-drying): No significant difference in log(score) results. | Mean log(scores) for air-dried (2.32 ± 0.25) and MBT FAST Shuttle dried (2.34 ± 0.28) samples were very similar across all methods and study sites. (Table 8) Differences in means were minimal (e.g., -0.02). |
Accelerated Drying Time: Significantly shorter drying time with MBT FAST Shuttle. | MBT FAST Shuttle drying times (average 7.63 minutes) were significantly shorter than air drying times (average 17.59 minutes). (Table 9) |
MBT Compass HT CA Software: | |
Software Verification & Validation: Meets software standards and requirements with established traceability. | Conducted and documented in accordance with 2023 FDA guidance. Includes code review, unit level, and system level testing. |
Cybersecurity: Vulnerability and penetration testing conducted, controls implemented and verified. | Conducted and documented in accordance with 2023 FDA guidance. All appropriate controls implemented and verified. |
Analytical Performance (Low Confidence Results as Final): Low confidence results (DT/eDT) show no significant difference in species identification compared to the Ext method. | Of 1,670 yellow log(scores), 1,269 showed high-confidence species ID after Ext. Only 7 samples (0.55%) showed a different result with Ext, which were justified by polyphasic taxonomic rules or library improvement. |
IDealTune Functionality: Improves and maintains mass spectrometer performance, reducing need for manual tune-ups. | High BTS-QC passing rates (99% and 100%) observed over 14-17 months, with only 9-24 IDealTune adjustments. (Table 10) Confirmed no manual tune-ups needed for over a year with IDealTune. |
Study Details
Based on the provided text:
-
Sample sizes used for the test set and the data provenance:
- MBT FAST Shuttle - Repeatability: 120 mass spectra (presumably from 12 bacterial/yeast strains * 10 repeats * 2 runs across DT, eDT, Ext, Sepsityper workflows as indicated in Table 3 headers, although the text says "each out of 2 runs" for "each workflow/method" - suggesting 10 per method/workflow per run).
- MBT FAST Shuttle - Reproducibility (Site-to-Site): 2700 samples for MBT workflow (900 samples per study site * 3 sites) and 1350 samples for Sepsityper workflow (450 samples per study site * 3 sites). The document mentions "10 microorganisms" used per study site.
- MBT FAST Shuttle - Reproducibility (Device-to-device): 1080 samples for MBT workflow (360 samples per MBT FAST Shuttle * 3 shuttles) and 540 samples for Sepsityper workflow (180 samples per MBT FAST Shuttle * 3 shuttles).
- MBT FAST Shuttle - Reproducibility (Operator-to-operator): 900 samples for MBT workflow (450 samples per operator * 2 operators) and 450 samples for Sepsityper workflow (225 samples per operator * 2 operators).
- MBT FAST Shuttle - Reproducibility (Day-to-day): 900 samples for MBT workflow (180 samples per day * 5 days) and 450 samples for Sepsityper workflow (90 samples per day * 5 days).
- MBT FAST Shuttle - Method Comparison (Drying): 279 mass spectra for air-dried and 279 mass spectra for MBT FAST Shuttle dried from three study sites (93 mass spectra per site per drying method). Ten (10) microorganisms and a blood culture, each spotted in triplicates.
- MBT Compass HT CA - Low Confidence Results: 15,270 spectra in total, with 1,670 yellow log(scores) re-analyzed.
- MBT Compass HT CA - IDealTune: Data collected from 133 BTS-QC runs at Site 1 (over 17 months) and 76 BTS-QC runs at Site 2 (over 14 months).
Data Provenance: The studies were performed at multiple sites (at least 3 for reproducibility studies), and one study explicitly mentions that microorganisms were shipped to both US study sites. This implies the data is, at least in part, prospectively collected in a multi-center setting for verification/validation. The "low confidence results" study was a retrospective non-interventional validation using data from previous clearances.
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- The document does not specify the number or qualifications of experts used to establish ground truth for most of these performance studies. The studies primarily focus on performance consistency and equivalence compared to established methods using what appears to be common laboratory standards (e.g., identity confirmed organisms, BTS quality checks).
- For the "low confidence results" study, it states: "Isolates from clinical routine were used to compare the results of the MBT-CA System against a gold standard (16S sequencing)." This suggests the ground truth was established by 16S sequencing, a molecular method, rather than solely by human experts, and then potentially interpreted by experts.
-
Adjudication method (e.g. 2+1, 3+1, none) for the test set:
- No adjudication method involving multiple human readers for conflict resolution is mentioned or appears to be applicable given the nature of the device (mass spectrometry-based organism identification). The performance is assessed on the agreement with an expected identification or log(score) thresholds.
-
If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No MRMC or human-in-the-loop comparative effectiveness study with human readers assisting or being assisted by AI is described in this document. The device is a "clinical mass spectrometry microorganism identification and differentiation system," not an AI-assisted diagnostic imaging tool.
-
If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- Yes, the performance validation studies of the MBT FAST Shuttle US IVD and MBT Compass HT CA software are essentially standalone performance evaluations of these components within the overall MALDI Biotyper CA System. The "outputs" (identification results, log(scores)) are generated by the system (including the hardware, software, and reference library) without direct human interpretation of the raw mass spectra. Human involvement is in sample preparation and operating the system, but the core identification is algorithmic.
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc):
- For the analytical performance of organism identification, the ground truth appears to be based on:
- Reference strains/known microorganisms: Used in repeatability and reproducibility studies.
- 16S sequencing: Explicitly stated as the "gold standard" for comparing results in the "low confidence results" study.
- Internal quality control standards: Like the Bacterial Test Standard (BTS) for IDealTune validation.
- This is primarily laboratory-based "gold standard" ground truth (molecular methods, established reference cultures), rather than expert consensus on clinical cases.
- For the analytical performance of organism identification, the ground truth appears to be based on:
-
The sample size for the training set:
- This document describes the validation of new components for an existing system. It does not provide details about the training set size for the underlying MALDI Biotyper CA System's reference library or analytical algorithms. The "reference library" (which acts as a form of "training data" for identifying unknown spectra) is mentioned as being continually updated, but its size is not specified.
-
How the ground truth for the training set was established:
- Similar to the above, the document does not detail how the ground truth was established for the training data (the reference library) of the overall MALDI Biotyper CA System. However, standard practice for building such libraries involves:
- Well-characterized bacterial and yeast strains: Often from culture collections, with identity confirmed by a variety of methods including 16S rRNA gene sequencing, traditional biochemical tests, and possibly whole-genome sequencing.
- Internal validation and verification: Ensuring the spectral patterns are consistent and representative for each species.
- Similar to the above, the document does not detail how the ground truth was established for the training data (the reference library) of the overall MALDI Biotyper CA System. However, standard practice for building such libraries involves:
In summary, this 510(k) submission successfully demonstrates substantial equivalence by showing that the new components (MBT Compass HT CA software and MBT FAST Shuttle US IVD) maintain the safety and effectiveness of the predicate device, and in some cases, enhance usability (faster drying time, improved instrument maintenance) without introducing new risks or compromising diagnostic accuracy. The studies presented are analytical validations focusing on performance characteristics relevant to microorganism identification in a laboratory setting.
Ask a specific question about this device
(221 days)
QBN
VITEK® MS PRIME is a mass spectrometry system using matrix-assisted laser desorption/ionization time of flight mass spectrometry (MALDI-TOF MS) for the identification of microorganisms cultured from human specimens.
The VITEK® MS PRIME system is a qualitative in vitro diagnostic device indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial and fungal infections.
This 510(k) submission introduces the VITEK®MS PRIME System. The VITEK® MS PRIME is intended for laboratory use by professional users who are trained in microbiology and good laboratory practices.
The VITEK® MS PRIME makes microorganism identifications via matrix-assisted laser desorption/ionization - time of flight mass spectrometry (MALDI-TOF MS) technology, which includes the three basic principles of ionization, separation, and detection,
As a first step, a VITEK® MS-DS Target Slide is prepared in accordance with the instructions for use.
NOTE: Depending on the culture, the analyte sample (i.e. microorganism from cultured media) may be directly spotted to a target slide, or for Mycobacterium. Nocardia and mould it must be processed/inactivated before adding to the target slide.
Once the specimen (cultured from the appropriate media) is spotted to the target slide, a matrix is added for the purpose of easy sublimation and strong absorbance in the laser wavelength employed by theinstrument.
NOTE: The VITEK® MS PRIME is a Class 1 laser product, containing a Class 4 Neodymium-doped yttrium lithium fluoride (Nd:YLF) laser – the laser operates at a wavelength of 349 nm.
The prepared slide is then loaded onto the VITEK®MS PRIME instrument. where a laser targets the sample spot and pulses the isolate spot, resulting in vibrational excitation of matrix and analyte molecules. The matrix transfer protons to the analyte resulting in a positive charge. So as part of the first basic principle, the ionized molecules are then accelerated in an electromagnetic field and a grid electrode in the ionization chamber.
The acceleration in the electromagnetic field is the beginning of the second basic principle (i.e. the separation process that is based of the time-of-flight principle). The velocity of the molecules depends on the mass-to-charge (m/z) ratio of the analyte, with heavier molecules having a higher moment of inertia resulting in a lower velocity.
As a final step in the basic principle of MALDI-ToF technology (i.e. detection) the time of flight is measured precisely by the ions arrival at a particle detector. This speed of the ions in flight depends on their mass - with heavier molecules having a higher moment of inertia resulting in a lower velocity. The time of transit is measured precisely by the ions' arrival at a particle detector. Based on the time of flight, the m/z ratio of each particle can be determined, and a mass spectrum of the analyte sample mixture is generated. The recorded signal is processed and presented as a spectrum of intensity versus mass in Daltons (Da). The mass spectrum displays results as a series of peaks (spectrum) which correspond to the ionized proteins derived from the analyte sample. And for identification of an unknown organism, the resulting mass spectra are sufficiently distinctive to allow taxonomic characterization at the genus and species when compared against the VITEK® MS Knowledge Base.
Here's a breakdown of the acceptance criteria and study information for the VITEK® MS PRIME, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance:
Performance Metric | Acceptance Criteria (Minimum Agreement) | Reported Device Performance (VITEK® MS PRIME) |
---|---|---|
Biological Equivalency | 95% Agreement (compared to reference method) | 99.7% Agreement (1456/1461, excluding discordant IDs and No IDs) |
Correct single choice ID or low discrimination to correct genus (Gram-positive) | Not explicitly stated, but within overall 95% | 99.0% |
Correct single choice ID or low discrimination to correct genus (Gram-negative) | Not explicitly stated, but within overall 95% | 97.2% |
Correct single choice ID or low discrimination to correct genus (Yeast) | Not explicitly stated, but within overall 95% | 100% |
Correct single choice ID or low discrimination to correct genus (Mycobacteria - solid culture) | Not explicitly stated, but within overall 95% | 100% |
Correct single choice ID or low discrimination to correct genus (Mycobacteria - liquid culture) | Not explicitly stated, but within overall 95% | 97.62% |
Correct single choice ID or low discrimination to correct genus (Moulds) | Not explicitly stated, but within overall 95% | 97.4% |
Correct single choice ID or low discrimination to correct genus (Nocardia) | Not explicitly stated, but within overall 95% | 100% |
Discordant Identification Rate (Biological Equivalency) | Not explicitly stated, but implied to be low | 0.3% (5/1461) |
No Identification Rate (Biological Equivalency) | Not explicitly stated, but implied to be low | 1.6% (23/1461) |
Clinical Performance Evaluation (Overall, including/excluding No IDs) | 95% Agreement (compared to reference method) | 98.4% (492/500) |
Clinical Performance (Overall, excluding No ID results) | 95% Agreement (compared to reference method) | 99.6% (492/494) |
Correct single choice ID or low discrimination to correct genus (Gram-positive) | Not explicitly stated, but within overall 95% | 99.3% |
Correct single choice ID or low discrimination to correct genus (Gram-negative) | Not explicitly stated, but within overall 95% | 98.8% |
Correct single choice ID or low discrimination to correct genus (Yeast) | Not explicitly stated, but within overall 95% | 95.3% |
Correct single choice ID or low discrimination to correct genus (Mycobacteria) | Not explicitly stated, but within overall 95% | 100% |
Correct single choice ID or low discrimination to correct genus (Moulds) | Not explicitly stated, but within overall 95% | 98.0% |
Correct single choice ID or low discrimination to correct genus (Nocardia) | Not explicitly stated, but within overall 95% | 100% |
Discordant Identification Rate (Clinical Performance) | Not explicitly stated, but implied to be low | 0.4% (2/500) |
No Identification Rate (Clinical Performance) | Not explicitly stated, but implied to be low | 1.2% (6/500) |
Challenge Isolate Results | Not explicitly stated, but implied to be high agreement, no misidentifications/no IDs | 100.0% (100/100) agreement, no No IDs, no discrepant results |
Quality Control Results | Not explicitly stated, but implied to be high agreement | 98.3% agreement |
Reproducibility Results | Not explicitly stated, but implied to be high agreement | 99.5% agreement |
2. Sample Sizes Used for the Test Set and Data Provenance:
-
Biological Equivalency Study:
- Sample Size: 1461 samples (representing 487 unique tests in triplicate).
- Data Provenance: Not explicitly stated, but the strains tested included "critical pathogens" for the 479 claimed species. It is likely a combination of well-characterized laboratory strains and potentially some clinical isolates, given the reference to "clinically validated isolates." retrospective or prospective is not mentioned.
-
Clinical Performance Evaluation:
- Sample Size: 500 clinical isolates (from 100 species with five strains each).
- Data Provenance: "clinical isolates tested from all sites combined." The specific countries of origin are not specified, but the data is explicitly from "clinical isolates," suggesting data from human specimens. The study design of using "clinical isolates" usually implies retrospective or prospectively collected samples from clinical settings. It refers to "reference identification obtained during previous clinical studies," suggesting a retrospective use of previously characterized clinical data.
-
Challenge Isolate Results:
- Sample Size: 100 challenge strains.
- Data Provenance: Not specified, but "challenge strains" often refers to a curated set of difficult-to-identify or representative strains used for rigorous testing.
-
Quality Control Results:
- Sample Size: Not explicitly stated, but refers to "all quality control strains tested at all sites."
-
Reproducibility Results:
- Sample Size: Not explicitly stated, but refers to "Reproducibility strains."
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications:
- The document does not directly state the number of experts or their qualifications.
- For the Clinical Performance Evaluation, the ground truth was established by "a one choice or multiple choice (more than one species) reference identification obtained during previous clinical studies." This implies that the reference identifications were well-established and accepted, likely through conventional microbiological methods and expert interpretation from those previous studies. The nature of these "previous clinical studies" (e.g., whether they involved expert consensus or a gold standard method) is not detailed.
4. Adjudication Method for the Test Set:
- The document does not explicitly describe an adjudication method for the test set results. The ground truth for the clinical performance evaluation was "reference identification obtained during previous clinical studies," implying that disagreements with a single reference standard were likely noted as discordant results rather than undergoing a separate adjudication process within this specific study.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:
- No MRMC comparative effectiveness study is mentioned. This device (VITEK® MS PRIME) is an automated system for microorganism identification using MALDI-TOF MS technology, which does not involve human readers interpreting results in the same way an imaging AI algorithm would. Its performance is compared to a reference method, not to human readers' performance with and without AI assistance.
6. Standalone Performance:
- Yes, a standalone performance study was done. Both the "Biological Equivalency study" and "Clinical Performance Evaluation" describe the performance of the VITEK® MS PRIME system alone (algorithm only) in identifying microorganisms, comparing its results to a ground truth or reference identification. The stated agreement rates (e.g., 98.4% clinical agreement) are measures of this standalone performance. The device is described as "a qualitative in vitro diagnostic device indicated for use in conjunction with other clinical and laboratory findings," but the performance metrics provided are for the device's identification capability itself.
7. Type of Ground Truth Used:
- The ground truth used appears to be reference identification by accepted microbiological methods (which implicitly includes expert consensus in their establishment).
- For the Biological Equivalency study, performance was measured "in comparison with the reference method."
- For the Clinical Performance Evaluation, performance was determined by comparing the VITEK® MS PRIME identification to "a one choice or multiple choice (more than one species) reference identification obtained during previous clinical studies." This "reference identification" would be established through a combination of traditional culture-based methods, molecular methods, and expert interpretation/consensus over time, serving as the gold standard for organism identification. "Outcomes data" or "pathology" as the direct ground truth are not mentioned for identifying the microorganisms themselves, though the device aids in diagnosis of infections.
8. Sample Size for the Training Set:
- The document does not explicitly state the sample size for the training set. It refers to a "VITEK® MS Knowledge Base" (KB v3.2) against which the mass spectra are compared. This knowledge base is the "training set" or reference library. The complexity and size of this knowledge base are not detailed in terms of number of samples/isolates used to build it.
9. How the Ground Truth for the Training Set Was Established:
- The document states that identifications are made "when compared against the VITEK® MS Knowledge Base." While it doesn't describe the exact process for building this KB, such knowledge bases for MALDI-TOF MS systems are typically built by:
- Acquisition of mass spectra from a large collection of well-characterized and phenotypically/genetically confirmed (often by sequencing, biochemical tests, or other gold-standard methods) reference strains across various species.
- Each reference strain's identity is verified by expert microbiologists using established methods before being added to the database.
- The collection process ensures reproducibility and representation of intra-species variability.
- Thus, the ground truth for the training set (Knowledge Base) is established through a rigorous process of expert-validated identification using traditional and molecular microbiological gold standards.
Ask a specific question about this device
(205 days)
QBN
VITEK® MS is a mass spectrometry system using matrix-assisted laser desorption/ionization time of flight mass spectrometry (MALDI-TOF MS) for the identification of microorganisms cultured from human specimens.
The VITEK® MS system is a qualitative in vitro diagnostic device indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial, yeast and mould infections.
The VITEK® MS v3.0 system is a system consisting of kit reagents (VITEK® MS-CHCA, VITEK® MS-FA, VITEK® MS Mycobacterium/Nocardia Kit, VITEK® MS Mould Kit), VITEK® MS-DS target slides, and the VITEK® MS (original equipment VITEK® MS Prep Station, Knowledge Base v3.2.0, software, and the VITEK" MS (original eq
This document focuses on the VITEK® MS system, a mass spectrometry system for microorganism identification, and its performance characteristics. It outlines the acceptance criteria and details of the study conducted to prove the device meets these criteria.
1. Acceptance Criteria and Reported Device Performance
The acceptance criteria are implicitly defined by the reported performance targets which show very high agreement rates for identification. The primary performance metric is the "Total Correct Genus ID (One Choice and Low Discrimination)".
Here's a table summarizing criteria derived from the reported performance, reflecting typical expectations for such devices, and the observed performance for different organism groups.
Organism Group | Acceptance Criteria (Implied) - Total Correct Genus ID | Reported Device Performance (Total Correct Genus ID) |
---|---|---|
Gram-positive bacteria | >95% | 95.8% (3594/3750) |
Gram-negative bacteria | >95% | 95.2% (5788/6079) |
Gram-negative bacteria (Brucella only) | >90% | 91.7% (220/240) |
Yeasts | >95% | 96.8% (1316/1360) |
Moulds | >90% | 92.7% (1398/1508) |
Mycobacterium | >95% | 96.5% (777/805) |
Nocardia | >95% | 97.9% (374/382) |
Overall (All Organisms Total) | >95% | 95.4% (13247/13884) |
Note: The reported performance also details "One Choice Correct" and "Low Discrimination Correct" breakouts, as well as discordant and no-ID rates, providing a comprehensive view of the device's accuracy. For example, for "All" clinical bacteria and yeast isolates tested, the agreement with reference identification was 98.8% (4189/4241).
2. Sample Size Used for the Test Set and Data Provenance
Test Set Sample Size:
- Overall Performance (Routine fresh and stock strains): 13,884 samples across various organism groups.
- Clinical Studies (Clinical isolates): 4,241 test results.
- Brucella specific testing (ATCC): 240 strains for clinical trial, 180 for reproducibility, 45 for challenge testing.
Data Provenance:
The data was collected from:
- Routine fresh and stock strains from patient cultures in clinical microbiology laboratories and bioMérieux laboratories in the United States and France. This indicates a mix of prospective and retrospective (stock strains) data and geographic diversity.
- Clinical isolates were tested at "one external clinical trial site and one internal site" for bacteria (other than Brucella) and yeasts.
- For Brucella, clinical trial, reproducibility, and challenge testing were performed at "one external clinical trial site" (specifically, ATCC is mentioned for Brucella testing). This suggests a focus on specific, well-characterized strains for Brucella.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
The document does not explicitly state the number of experts used or their specific qualifications (e.g., "radiologist with 10 years of experience"). However, it mentions that "Samples were sequenced by the appropriate reference laboratory, and if needed, additional analysis was performed internally to obtain a reference identification (GenBank, dendrogram analysis)."
This implies that the ground truth was established by:
- Personnel at "appropriate reference laboratories" (implying qualified microbiologists/scientists).
- Internal personnel at bioMérieux.
- The use of "proficiency panels tested by all participating technologists at the external sites" for the clinical trial suggests that the individuals performing the tests were trained and qualified.
While specific numbers and detailed qualifications of individual experts are not provided, the methodology points to reliance on established laboratory practices and reference techniques for accurate identification.
4. Adjudication Method for the Test Set
The document does not explicitly describe a formal adjudication method (like 2+1 or 3+1 for human readers of images). For microbiological identification, the "adjudication" is inherent in the ground truth establishment process:
- Reference methods (DNA sequencing analysis, supplemental testing) are considered the gold standard.
- If sequencing was insufficient, "additional analysis was performed internally to obtain a reference identification (GenBank, dendrogram analysis)." This suggests a process where discrepancies or difficult identifications are resolved by deeper, more definitive, and comparative analysis rather than a simple consensus among multiple readers.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No, a multi-reader multi-case (MRMC) comparative effectiveness study was not done in the traditional sense of comparing human readers with and without AI assistance for interpretation tasks. This device is an automated system for identifying microorganisms, not an AI for image interpretation that assists human diagnosticians. The study focuses on the accuracy of the automated system against a definitive reference method.
6. Standalone (Algorithm Only Without Human-in-the-Loop Performance)
Yes, the study primarily evaluates the standalone performance of the VITEK® MS system. The reported performance metrics (e.g., "Total Correct Genus ID") reflect the accuracy of the device's algorithm in identifying microorganisms when presented with samples.
While the "Interpretation of results and use of the VITEK® MS system require a competent laboratorian" is mentioned in the device description, the performance tables quantify the output of the machine itself compared to the ground truth, effectively reporting its standalone accuracy.
7. The Type of Ground Truth Used
The ground truth for microorganism identification was established using:
- DNA sequencing analysis: This is the primary reference method.
- Supplemental testing (when necessary): This could include biochemical tests, serology, or other definitive microbiological techniques.
- Previously well-characterized strains (ATCC or equivalent): For certain challenge or quality control tests, established strains were used, for which the identity is already known and highly reliable.
- GenBank and dendrogram analysis: Used for additional internal analysis to obtain reference identification when sequencing was not fully conclusive, reinforcing the ground truth.
This indicates a robust, multi-faceted approach to ground truth establishment, relying on genetic and phenotypic gold standards.
8. The Sample Size for the Training Set
The document does not explicitly state the sample size used for the training set for the VITEK® MS Knowledge Base (KB) v3.2.0. It mentions that the "Knowledge Base includes data representing 1316 species and 1158 taxa displayed" and "1095 species of bacteria" and "232 species of fungi." This describes the content of the database used for identification, which is analogous to the "training data" that built the system's ability to recognize these species. However, the exact number of samples or spectra used to build this database is not provided in this excerpt.
9. How the Ground Truth for the Training Set Was Established
The document describes the "Knowledge Base" as having been "developed from spectra of a number of microbial species." While it doesn't detail the training process or the ground truth establishment for the training set specifically, it's highly implied that the same rigorous methods used for the test set (DNA sequencing, supplemental testing, well-characterized strains) would have been used to curate the reference spectra within the Knowledge Base. This ensures that the foundational data used by the device for identification is accurate and reliable. The continuous updates to the "Knowledge Base v3.2.0" suggest an ongoing process of incorporating new and validated spectral data.
Ask a specific question about this device
(203 days)
QBN
The MALDI Biotyper CA System is a mass spectrometer system using matrix-assisted laser desorption/ionization - time of flight (MALDI-TOF) for the identification and differentiation of microorganisms cultured from human specimens.
The MALDI Biotyper CA System is a qualitative in vitro diagnostic device indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial and fungal infections.
The MBT-CA System consists of the Microflex LT/SH mass spectrometer, reference library, kit reagents (US IVD HCCA, US IVD Bacterial Test Standard), US IVD 48 Spot Target or MBT Biotarget 96 US IVD plate, and software. The MALDI Biotyper CA System with closed safety covers is a Class 1 Laser product. With the safety cover opened it becomes a Class 4 Laser product.
The MALDI Biotyper CA System reference library was established by analyzing the type strain from each claimed species combined with 4 to ~30 additional strains from the same species provided by clinical laboratories or commercial strain collections. Currently a total of 3029 strains (covering 334 species / groups with 294 bacteria plus 40 yeasts) are contained in the clinically validated MBT-CA library.
Implementation methodology, construction parameters and quality assurance protocols use a standard operating protocol for generation of reference entries and all testing parameters are the same.
MBT-CA microorganism identification is based on isolate MALDI spectra using Bruker reference libraries with a 1:1 comparison of unknown MALDI spectra against each single entry of a given reference library. During a single identification event, an unknown MALDI spectra is compared against each single reference entry producing individual log(score) results. This number of log(scores) is sorted based on their value and the highest one is used to generate the final result. The addition of new reference entries does not influence the already included entries. If no reference entries are removed within a library update the log(score) calculation remains unchanged for the same MALDI spectra.
MALDI Biotyper CA System client software displays a user-interface which guides the user through the MALDI Biotyper CA System workflow. The MALDI Biotyper CA System client also interfaces to the flexControl software for automated acquisition of mass spectra on the microflex LT/SH instrument.
The MALDI Biotyper CA System server communicates with the MALDI Biotyper CA System client and the MBT-DB server. It performs preprocessing on acquired spectra, and matches peak lists against the Main Spectrum (reference pattern, (MSP)) for matching and calculates the score value (log (score)).
The MBT-DB server stores all information for the MALDI Biotyper CA System. The MBT-DB maintains spectra data (creation information and mass/intensity lists), project data (results of defined and executed runs), method data (parameter lists for spectra preprocessing and identification), user management data, reference patterns and other peak lists plus additional maintenance data.
GTPS firmware communicates with the flexControl PC software, controls and monitors the vacuum, moves the sample carrier and performs the docking of the target plate, controls and monitors high voltages in the ion source, generates trigger signals, and monitors instrument status.
The flexControl acquisition software communicates with the MALDI Biotyper CA System client, loads automatic run jobs, communicates with the GTPS firmware, communicates with the laser in the microflex LT/SH instrument, sets the acquisition parameters in the digitizer and reads the acquired data from the digitizer, performs automated data acquisition, evaluates acquired spectra, adjusts the laser power during automatic data acquisition, performs a re-calibration of the time-of-flight to mass transformation, stored acquired spectra on disk and performs source cleaning. The flexControl software does not display a user interface.
The optional Honeywell (Hyperion 1300g) Barcode Reader USB cable is connected to the MALDI Biotyper CA System computer. The barcode reader scans the unique ten-digit target ID which appears in the Target ID box on the target plate. After the target ID has been entered, the a new Run page opens and the ten-digit target ID appears as the Plate ID and is appended to the Run name. Sample identifications are entered into the computer corresponding to the target plate position for that run.
Here's a breakdown of the acceptance criteria and study information for the MALDI Biotyper CA System, extracted from the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The document describes performance for the addition of Candida auris to the existing MBT-CA reference library, rather than a full de novo clearance study of the entire device. Therefore, the "acceptance criteria" here refer to the performance required for the inclusion of this new organism into the established system.
Criteria (for C. auris identification) | Reported Device Performance (C. auris identification) |
---|---|
High confidence organism ID (log(score) $\ge$ 2.0) | 22 out of 22 (100%) correctly identified |
Low confidence organism ID (log(score) $\ge$ 1.7 - |
Ask a specific question about this device
Page 1 of 1