(260 days)
INTENDED USE: The BBLCRYSTAL™ Gram Positive (GP) Identification (ID) System is a miniaturized identification method employing modified conventional, fluorogenic, and chromogenic substrates. It is intended for the identification of frequently isolated aerobic gram-positive bacteria from clinical specimens.
INDICATIONS FOR USE: Use of the BBLCRYSTAL™ Gram Positive Identification System is indicated when the aerobic gram-positive organisms described in the attached table have been isolated in pure culture from clinical specimens in a clinical laboratory, and identification of the microorganisms is desired.
The main component of the BBLCRYSTAL™ GP ID System is the BBLCRYSTAL GP panel assembly, consisting of the CRYSTAL base and lid. The BBLCRYSTAL lid consists of 29 dehydrated biochemical/chromogenic/ fluorogenic substrates and one fluorogenic negative control, on the ends of plastic prongs. The CRYSTAL base consists of 30 matching wells; its design allows inoculation of all 30 wells in a single step by pouring the suspension of pure culture into the target area and tilting the base. The test inoculum rehydrates the dried substrates and initiates test reactions.
The pure culture suspension is prepared by picking several small colonies of the same morphology from media such as Trypticase® Soy Agar with 5% Sheep Blood or Columbia Agar with 5% Sheep Blood, or alternatively selective media such as Phenylethyl Alcolol Agar with 5% Sheep Blood or Columbia CNA Agar with 5% Sheep Blood. A standardized suspension of this culture is prepared in the BBLCRYSTAL™ ANR, GP, RGP, N/H ID Inoculum Fluid provided. The suspension is added to the target area of the panel base, which the use then rocks back and forth to inoculate all the wells contained in the base.
After the base/lid assembly has been incubated for 4 hours at 35-37°C, the assembly is placed on the BBLCRYSTAL Panel Viewer and the color reactions are visually compared to the BBLCRYSTAL GP Color Chart provided. Each reaction is scored as a positive (+) or negative (-) and recorded on the BBLCRYSTAL GP Report Form.
After all results are read, a 10-digit numerical profile is calculated by assigning a value of 4, 2, or 1 to each positive reaction. (Negative reactions are scored as "O".) The values for each column are then added together to obtain the 10 digit Profile Number.
The BBLCRYSTAL ID System Electronic Codebook is loaded into the user's PC and the appropriate database is selected. Then the Profile Number and results of any off-line tests are entered, and the Codebook gives one of the following three results:
- (a) a definitive ID;
- (b) a tie between two or more species; or
- no ID possible with data submitted. (c)
In the case of a definitive ID or a tie between two or more potential ID's, the user can access the statistics for that ID as well as background information for the species identified.
Here's a breakdown of the acceptance criteria and study details for the BBLCRYSTAL™ Gram Positive ID System, based on the provided text:
1. Acceptance Criteria and Reported Device Performance
The document doesn't explicitly state formal "acceptance criteria" with numerical thresholds. However, based on the clinical correlation study results, we can infer the performance targets that were likely considered acceptable for an identification system:
| Acceptance Criteria Category | Reported Device Performance (BBLCRYSTAL™ GP ID System) |
|---|---|
| Correct Identification Rate | 90% (668/735) including supplemental testing |
| Incorrect Identification Rate | 7.6% (56/735) |
| "No Identification" Rate | 1.5% (11/735) |
| Overall Reproducibility | 96.7% |
| Individual Substrate Reaction Reproducibility | 79.2% to 100% |
| Individual QC Organism Reaction Reproducibility | 91.4% to 99.8% |
| Individual Site Reproducibility | 95.5% to 97.8% |
2. Sample Size and Data Provenance for the Test Set
- Sample Size for Test Set: 735 gram positive aerobic isolates.
- Data Provenance: The isolates were a mix of:
- "Fresh, routine isolates arriving in the clinical laboratory."
- "Previously identified isolates of the clinical trial sites' choice."
- The study was conducted at "four clinical laboratories," implying data from multiple locations. The country of origin is not specified but is implicitly the United States, given the FDA submission.
- Retrospective or Prospective: The "fresh, routine isolates" suggest a prospective component, while "previously identified isolates" suggest a retrospective component. The overall study appears to be a hybrid.
3. Number of Experts and Qualifications for Ground Truth - Test Set
- The document states the BBLCRYSTAL™ system's performance was evaluated "against a combination of the API Staph, API 20 Strep, and API Coryne Identification Systems, and conventional methodologies."
- While it doesn't explicitly mention "experts" establishing ground truth in the same way one might for image interpretation, the "conventional methodologies" and predicate devices (API systems) served as the current gold standard. This implies that the ground truth was established by standard laboratory practices and the accepted performance of these established systems, which would involve trained microbiology laboratory personnel.
- No specific number or qualifications of individual "experts" (like radiologists with X years of experience) are provided, as the ground truth was based on established laboratory methods rather than subjective human interpretation.
4. Adjudication Method for the Test Set
- The document implies that the ground truth was established by comparing the BBLCRYSTAL™ results to the results obtained from the predicate API systems and "conventional methodologies."
- There is no mention of a specific "adjudication method" like 2+1 or 3+1 for resolving discrepancies. The conventional methodologies and predicate devices implicitly served as the authoritative reference for identification. If there were discrepancies, the conventional methods would likely be the arbiter, as the new device's performance was being measured against them.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- This device is an automated identification system for microorganisms, not an imaging device requiring human interpretation in the same way.
- Therefore, an MRMC comparative effectiveness study comparing human readers with and without AI assistance is not applicable and was not conducted. The "human-in-the-loop" aspect primarily involves preparing the sample, inoculating the panel, and entering results, not interpreting complex outputs.
6. Standalone (Algorithm Only) Performance Study
- Yes, the performance study described is essentially a standalone evaluation of the BBLCRYSTAL™ GP ID System. The "system" (BBLCRYSTAL™) generates an identification, which is then compared to the ground truth established by other validated methods.
- While human users prepare the inoculum, read the reactions, and enter the profile number, the core "identification" logic (the electronic codebook) operates independently to provide the final result. The reported percentages (90% correct, 7.6% incorrect, 1.5% no ID) are indicative of the algorithm's standalone performance within the context of its intended use.
7. Type of Ground Truth Used
- The ground truth was established using a combination of:
- Predicate device results: BioMerieux Vitek, Inc., API Staph, API 20 Strep, and API Coryne.
- Conventional methodologies: This would typically include established biochemical tests, phenotypic characteristics, and potentially genetic methods used in clinical microbiology for definitive identification.
8. Sample Size for the Training Set
- The document does not specify the sample size used for the training set. It focuses entirely on the performance evaluation cohort (the test set). The "Electronic Codebook" implies a database was built, but details on its creation are not provided.
9. How Ground Truth for the Training Set Was Established
- The document does not provide details on how the ground truth for the training set (i.e., the database building for the BBLCRYSTAL™ Electronic Codebook) was established. It describes the product itself and its performance evaluation. Typically, such databases are built using well-characterized, often type strains and a large collection of clinical isolates identified by a comprehensive battery of conventional and molecular methods.
§ 866.2660 Microorganism differentiation and identification device.
(a)
Identification. A microorganism differentiation and identification device is a device intended for medical purposes that consists of one or more components, such as differential culture media, biochemical reagents, and paper discs or paper strips impregnated with test reagents, that are usually contained in individual compartments and used to differentiate and identify selected microorganisms. The device aids in the diagnosis of disease.(b)
Classification. Class I (general controls). The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to § 866.9.