(270 days)
BacT/ALERT® VIRTUO(TM) Microbial Detection System is an automated microbial test system capable of incubating, agitating, and continuously monitoring for the detection of aerobic microorganism growth from blood and other normally sterile body fluids.
The VIRTUO Instrument is the next generation of the bioMerieux BacT/ALERT Microbial Detection System. This blood culture instrument consists of an incubator, agitation mechanism, robotic apparatus for automated loading and unloading of bottles and a tactile graphical interface. The VIRTUO is used in conjunction with existing, commercialized BacT/ALERT reagent bottles for clinical use (BacT/ALERT SA, SN, FA Plus, FN Plus and PF Plus culture bottles). The VIRTUO system utilizes a colorimetric sensor and reflected light to monitor the presence and production of carbon dioxide (CO2) dissolved in the culture medium. If microorganisms are present in the inoculated sample, carbon dioxide is produced as the organisms metabolize the substrates in the culture medium. When growth of the microorganisms produces CO2, the color of the gas-permeable sensor installed in the bottom of each culture bottle changes from blue-green to yellow. The color change results in an increase of reflectance units monitored by the system. The VIRTUO optically monitors the reflectance of each bottle over time and will store and interpret readings against algorithms, which are embedded in the firmware and/or software.
Here's a breakdown of the BacT/ALERT VIRTUO Bacterial Detection System's acceptance criteria and study proving its performance, based on the provided FDA 510(k) Summary.
This device is not an AI/ML device in the modern sense (e.g., using deep learning for diagnostic imaging), but rather an automated system that uses embedded algorithms to detect microbial growth. Therefore, some of the requested information regarding AI/ML-specific concepts (like MRMC studies, training set details for complex models) might not be directly applicable or detailed in the provided document. The study performed is a comparative clinical study against a predicate device.
Device Name: BacT/ALERT® VIRTUO™ Microbial Detection System
Device Description: An automated microbial test system capable of incubating, agitating, and continuously monitoring for the detection of aerobic, facultative, and anaerobic microorganism growth from blood and other normally sterile body fluids. It utilizes a colorimetric sensor and reflected light to monitor CO2 production as an indicator of microbial growth. Its detection algorithms are proprietary to the system.
1. Table of Acceptance Criteria and Reported Device Performance
The provided document doesn't explicitly state the "acceptance criteria" for performance as a numerical threshold (e.g., "sensitivity must be > X%"). Instead, it demonstrates substantial equivalence to the predicate device (BacT/ALERT® 3D Microbial Detection System) by comparing their performance head-to-head. The key performance metric assessed is the "Ratio of True Positives" and its 95% Confidence Interval (CI), with the implicit acceptance being that the performance of the VIRTUO is comparable to the established predicate device, and the false positive/negative rates are within acceptable limits.
Performance Goal: Demonstrate substantial equivalence to the BacT/ALERT® 3D Microbial Detection System in detecting microbial growth.
| Performance Metric | Acceptance Criteria (Implied) | Reported Device Performance (BacT/ALERT® VIRTUO™ vs. BacT/ALERT® 3D™) |
|---|---|---|
| Ratio of True Positives (VIRTUO / BTA3D) – Overall | 95% CI for the ratio of true positives should largely encompass or be close to 1.0, indicating comparable detection rates between the two systems for various specimen types and clinical determinations (Significant, Contaminant, Unknown). | Blood Cultures (Compliant) |
| For FA Plus (Blood) | Ratio of True Positives should be close to 1.0, and 95% CI should indicate comparable performance. | Significant: 0.970 (95% CI: 0.821, 1.119) |
| Contaminant: 0.636 | ||
| Total: 0.924 (95% CI: 0.766, 1.082) | ||
| For FN Plus (Blood) | Same as above. | Significant: 0.979 (95% CI: 0.823, 1.135) |
| Contaminant: 1.300 | ||
| Total: 1.034 (95% CI: 0.852, 1.216) | ||
| For PF Plus (Blood) | Due to low numbers, potentially wider CIs are accepted, but still demonstrate detection. | Significant: 1.000 (95% CI: -1.772, 3.772*) |
| Total: 1.000 (95% CI: -1.772, 3.772*) | ||
| For Low Fill FA Plus (Blood) | Same as above. | Significant: 0.966 (95% CI: 0.790, 1.142) |
| Contaminant: 1.333 | ||
| Total: 1.031 (95% CI: 0.790, 1.272) | ||
| For SA (Blood) | Same as above. | Significant: 1.026 (95% CI: 0.857, 1.195) |
| Contaminant: 0.400 | ||
| Total: 0.920 (95% CI: 0.736, 1.104) | ||
| For SN (Blood) | Same as above. | Significant: 1.067 (95% CI: 0.797, 1.337) |
| Contaminant: 0.250 | ||
| Total: 1.000 (95% CI: 0.730, 1.270) | ||
| Sterile Body Fluid Cultures | ||
| For FA Plus (SBF) | Same as above. | Significant: 1.000 (95% CI: 0.852, 1.148) |
| Contaminant: 1.333 | ||
| Total: 1.058 (95% CI: 0.864, 1.252) | ||
| For FN Plus (SBF) | Same as above. | Significant: 1.026 (95% CI: 0.838, 1.214) |
| Contaminant: 1.125 | ||
| Total: 1.000 (95% CI: 0.794, 1.206) | ||
| For PF Plus (SBF) | Same as above. | Significant: 0.933 (95% CI: 0.807, 1.059) |
| Contaminant: 0.000 | ||
| Total: 0.882 (95% CI: 0.665, 1.099) | ||
| For SA (SBF) | Same as above. | Significant: 1.000 (95% CI: 0.837, 1.163) |
| Contaminant: 1.000 | ||
| Total: 1.056 (95% CI: 0.806, 1.306) | ||
| False Positive Rate (Overall) | Should be low and comparable to the predicate device. | VIRTUO: 0.09% (5/5862) |
| BTA3D: 0.19% (11/5862) | ||
| False Negative Rate (Overall) | Should be low and comparable to the predicate device. | VIRTUO: 0.38% (22/5862) |
| BTA3D: 0.38% (22/5862) | ||
| Positive Control Performance | High percentage of positive controls detected. | 1487/1490 (99.8%) signaled positive. |
| Negative Control Performance | High percentage of negative controls detected as negative. | 1486/1486 (100%) signaled negative. |
| Overall Control Performance | High percentage of controls giving expected results. | 2973/2976 (99.9%) of controls gave expected results. |
Note on CI for PF Plus (Blood): The document explicitly states "Since the confidence interval contains a negative, the interval does not provide a meaningful interpretation" due to extremely low positive counts for this specific category. However, the 1:1 ratio of detected positives itself implies equivalence for the limited positive cases observed.
2. Sample Size Used for the Test Set and Data Provenance
-
Test Set (Clinical Evaluation):
- Total Bottle Pairs Tested (Clinical Samples): 5862 bottle pairs (one VIRTUO bottle and one BTA3D bottle per patient sample).
- Detailed Sample Sizes per Category:
- BacT/ALERT FA Plus (Blood): 1057 (compliant)
- BacT/ALERT FN Plus (Blood): 912 (compliant)
- BacT/ALERT PF Plus (Blood): 161 (compliant)
- Low Fill BacT/ALERT FA Plus (Blood): 379 (compliant)
- BacT/ALERT SA (Blood): 780 (compliant)
- BacT/ALERT SN (Blood): 830 (compliant)
- BacT/ALERT FA Plus (SBF): 374
- BacT/ALERT FN Plus (SBF): 437
- BacT/ALERT PF Plus (SBF): 77
- BacT/ALERT SA (SBF): 81
- BacT/ALERT SN (SBF): 81
- Data Provenance: "External performance evaluations were conducted at eight external clinical sites." The document does not specify the country of origin, but given the FDA submission, it is typically either US-based or multi-national with significant US participation. The data is prospective as it involves clinical evaluation performed specifically for the 510(k) submission.
-
Control Sample Size:
- 1490 positive controls
- 1486 negative controls
- Total controls: 2976
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The document does not specify the number or qualifications of experts used. However, for a device detecting microbial growth, the "ground truth" for positive culture would typically be established by standard microbiological laboratory procedures:
- Positive Culture: Visual detection of growth indicators (e.g., turbidity, gas production, or color change) in the culture bottle, followed by subculture onto agar plates and identification of microbial isolates using standard laboratory techniques (e.g., Gram stain, biochemical tests, mass spectrometry).
- Negative Culture: No growth detected after a specified incubation period, confirmed by terminal subculture on rich media.
- Clinical Determination (Significant/Contaminant/Unknown): This classification would typically be made by a qualified microbiologist or infectious disease physician, based on the identified organism, patient's clinical presentation, and other relevant factors.
4. Adjudication Method (e.g., 2+1, 3+1, none) for the Test Set
The document does not explicitly describe an adjudication method. For comparison studies of this nature, discrepancies between the investigational device and the predicate (or deviations from expected results) would generally be investigated by the clinical sites' microbiology laboratories to determine the definitive "ground truth" (e.g., re-subculture, review of patient charts).
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
No, an MRMC comparative effectiveness study was not done. This type of study (assessing human reader performance with and without AI assistance on diagnostic images) is not applicable to this device. The BacT/ALERT VIRTUO is an automated microbial detection system, not an imaging AI diagnostic aid for human readers. Its primary function is to automatically detect the presence of microbial growth, replacing or assisting a manual observation process, rather than augmenting a radiologist's interpretation of an image.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done
Yes, in essence, the primary performance evaluation measures the standalone performance of the BacT/ALERT VIRTUO system against the predicate BacT/ALERT 3D system. Both systems operate autonomously to detect microbial growth. The reported "True Positives," "False Positives," and "False Negatives" for VIRTUO directly reflect its algorithm-only performance in detecting growth from clinical samples, without human intervention required for the detection itself (humans are involved in loading bottles and interpreting the overall clinical significance, but not in the detection of growth by the device).
7. The Type of Ground Truth Used (expert consensus, pathology, outcomes data, etc.)
The ground truth was established by standard microbiological culture and identification methods, combined with a clinical review to classify isolates as "significant," "contaminant," or "unknown." This is essentially a definitive laboratory diagnosis of microbial presence and identity.
8. The Sample Size for the Training Set
The document does not specify a separate "training set" sample size in the context of typical AI/ML model development. The device uses "proprietary algorithms unique for BacT/ALERT VIRTUO," which implies embedded logic or rules rather than a dynamically trained machine learning model in the contemporary sense. The algorithms would have been developed and validated internally by the manufacturer using a combination of laboratory studies, simulated data, and potentially a distinct internal dataset. The data presented in the 510(k) is for the clinical validation (test set performance) of the final device.
9. How the Ground Truth for the Training Set was Established
Given that this is not a modern AI/ML device with a distinct training phase documented for public review, the methods for establishing ground truth for any internal algorithm development ("training") would also implicitly rely on controlled laboratory studies and established microbiological techniques to generate data on microbial growth, CO2 production kinetics, and associated reflectance changes. This would involve inoculating bottles with known strains and concentrations of microorganisms and observing their growth characteristics under various conditions.
§ 866.2560 Microbial growth monitor.
(a)
Identification. A microbial growth monitor is a device intended for medical purposes that measures the concentration of bacteria suspended in a liquid medium by measuring changes in light scattering properties, optical density, electrical impedance, or by making direct bacterial counts. The device aids in the diagnosis of disease caused by pathogenic microorganisms.(b)
Classification. Class I. With the exception of automated blood culturing system devices that are used in testing for bacteria, fungi, and other microorganisms in blood and other normally sterile body fluids, this device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter.