(834 days)
The BD Kiestra IdentifA module is an automated in vitro diagnostic specimen preparation system for use with the BD Kiestra Laboratory Automation Solution to prepare MALDI targets for the Bruker MALDI Biotyper CA System for the qualitative identification and differentiation of microorganisms using matrix-assisted laser desorption/ionization-time of flight mass spectrometry (MALDI-TOF MS) analysis of colonies grown on plated culture media from human specimens.
The BD Kiestra IdentifA is indicated for use in the clinical laboratory with the BD Kiestra Read Compact and Bruker MALDI Biotyper CA System to aid in the diagnosis of bacterial and fungal infections.
The BD Kiestra™ IdentifA is an instrument which automates picking of technologist-selected colonies from plated media and prepares a Bruker MALDI target for identification and differentiation of organisms. The BD Kiestra IdentifA includes the following components (Note: Bruker MALDI targets, Matrix and Bacterial Test Standard (BTS) are required, however, they are obtained directly from Bruker Daltonik GmbH):
- . BD Kiestra IdentifA instrument and software with onboard pipetting and nephelometry.
- . BD formic acid, deionized water, pipet tips, Matrix and BTS transfer vials.
- . BD Kiestra IdentifA nephelometer calibration standards (0.2, 0.5, 1.0 and 3.0 McFarland).
- BD Kiestra IdentifA cuvette array.
When a MALDI identification is ordered by a technologist selects the colonies from an image of a plated medium obtained using the BD Kiestra™ ReadA Compact. The coordinates of the colonies and the plated medium are transferred to BD Kiestra IdentifA where the colonies are picked. The colonies are suspended in deionized water and the onboard nephelometer determines the McFarland turbidity. Based on the McFarland, BD Kiestra IdentifA pipets the organism suspension onto a Bruker MALDI target. The BD Kiestra IdentifA uses the Bruker extended Direct Transfer method for preparation of the MALDI target by overlaying formic acid and Bruker Matrix onto the target spot. In addition, BTS spots are prepared on the target slide for quality control. Once dried, the technologist manually removes the target and loads onto the Bruker MALDI Biotyper CA System. The BD Kiestra IdentifA transfers the location of sample and BTS spots to the MALDI Biotyper CA. If requested by the technologist, BD Kiestra IdentifA will also dilute the organism suspension to a target of 0.5 McFarland.
The BD Kiestra IdentifA can be used as a standalone instrument or integrated into the BD Kiestra Laboratory Automation System. The standalone instrument utilizes an input/output module for manual plate loading, which handles de-stacking and stacking of plates. When physically integrated into the BD Kiestra Laboratory Automation System, BD Kiestra IdentifA is connected to a track by way of a connection module for automatic plate transfer. BD Kiestra IdentifA software is responsible for the instrument functionality and a touchscreen is mounted on the instrument for user interface.
Here's an analysis of the acceptance criteria and the studies performed for the BD Kiestra IdentifA device, based on the provided text:
Acceptance Criteria and Reported Device Performance
| Criteria | Acceptance Criteria (Explicit or Implied) | Reported Device Performance |
|---|---|---|
| Colony Picking Accuracy | 100% of colonies successfully selected and picked. 100% of prepared target spots provide the expected identification with Log(score) values ≥ 2.00. | 1200 (100%) colonies successfully selected and picked. 400 (100%) target spots provided the expected identification with Log(score) values ≥ 2.00. |
| Organism Identification Accuracy | BD Kiestra IdentifA processing yields equivalent or better identification accuracy compared to manual sample preparation. Specifically, for samples with positive organism identification (Log(score) ≥ 2.00), the percentage matching the expected identification should be comparable to manual preparation. | Of 397 samples with positive organism identification (Log(score) ≥ 2.00), BD Kiestra IdentifA processing yielded 388 (97.7%) matching the expected identification. Manual eDT method yielded 387 (97.5%). This demonstrates equivalency. Individual species results are detailed in the tables for Gram-negative, Gram-positive, and yeast species, indicating high concordance rates. |
| Reproducibility | For most strains, Log(score) ≥ 2.00 > 95% of the time across BD Kiestra IdentifA modules, replicates, groups, and lots. (Note: Acknowledged exceptions for strains where the predicate also performs poorly). | 13 out of 15 strains showed 100% (or 96%) agreement for Log(score) ≥ 2.00. |
| Corynebacterium jeikeium (37%) and Candida albicans (74%) did not meet the >95% criterion, but this was attributed to the original Bruker system's performance for these strains, as manual preparation also failed to meet the criterion. | ||
| Limit of Detection (LoD) | For each organism tested at or above the LoD (0.2 McFarland), at least 6/8 replicates should result in a correct identification. (Implied acceptance based on comparison to manual eDT method's LoD.) | All organisms except Saccharomyces cerevisiae (3/8) achieved 8/8 acceptable MALDI identifications at 0.2-0.3 McFarland. The low performance for S. cerevisiae was noted to be consistent with the original Bruker system's limitations. Demonstrated equivalency to the claimed LoD for the manual eDT method (CFU/target spot). |
| Cross-contamination | No cross-contamination within and between culture plates, and between spots on the MALDI target. 100% correct results for inoculated and uninoculated samples, with "No peaks" or "No identification" for uninoculated and Log(score) > 2.00 for inoculated. Zero reported cross-contamination events in field use (European data). | For the study, 100% of inoculated and uninoculated samples yielded the correct results (no contamination). |
| Over 58,000 samples processed across 3 BD Kiestra IdentifA instruments in Europe since January 2020 without any reported cross-contamination events. |
Study Details
2. Sample sizes used for the test set and the data provenance
- Colony Picking Accuracy: 200 mixed culture plates were used, from which 1200 colonies were selected and picked. The data provenance is not explicitly stated (e.g., country of origin), though it is noted as "internal analytical testing." The study design appears to be prospective (experimental).
- Organism Identification Accuracy: A total of 464 isolates of Gram-positive bacteria, Gram-negative bacteria, and yeasts were tested. The data provenance is not explicitly stated (e.g., country of origin), but it is referred to as "internal analytical testing." The study design appears to be prospective (experimental).
- Reproducibility: For each of the 15 strains, 27 tests were performed (3 days x 3 replicates x 3 instruments). This totals 15 strains * 27 tests/strain = 405 tests. The data provenance is not explicitly stated. The study design appears to be prospective (experimental).
- Limit of Detection: 9 organisms were tested, with 8 MALDI target spots inoculated for each. This totals 9 organisms * 8 spots/organism = 72 tests. The data provenance is not explicitly stated. The study design appears to be prospective (experimental).
- Cross-contamination: 100 plates inoculated with Staphylococcus aureus and 100 plates with Klebsiella pneumoniae. These 200 inoculated plates were alternated with 200 uninoculated media, for a total of 400 media processed. An additional field examination included "over 58,000 samples" processed in Europe since January 2020. The study design combined prospective experimental testing with retrospective field data analysis.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
- The text does not explicitly state the number of experts or their qualifications for establishing ground truth for the test set. However, for the colony picking accuracy, it mentions "Colonies from both isolates on each plate were selected by a technologist". The "Organism Identification" study implicitly used comparison to an "expected result," which would typically be based on a validated identification method and potentially confirmed by expert review, but this is not detailed. The "Reproducibility" study used "Strains with known identifications."
4. Adjudication method for the test set
- The document does not describe a formal adjudication method (like 2+1, 3+1) for disagreements or ambiguous cases in the test set. For the "Organism Identification Accuracy," results from the BD Kiestra IdentifA were "compared to the expected result for each isolate." For colony picking, it was confirmed visually and by the Bruker MALDI Biotyper CA identification.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
- No, a multi-reader multi-case (MRMC) comparative effectiveness study focusing on human reader performance improvement with AI assistance was not done. This device is an automated specimen preparation system, not an AI for interpretation that would assist human readers in diagnosing. Its function is to automate the preparation stage for downstream MALDI-TOF MS analysis. The comparison is between automated preparation vs. manual preparation using the predicate device.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
- Yes, the performance studies (Colony Picking, Organism Identification Accuracy, Reproducibility, LoD, Cross-contamination) evaluate the standalone performance of the BD Kiestra IdentifA in preparing samples. The output of the BD Kiestra IdentifA is a prepared MALDI target, which is then loaded onto the Bruker MALDI Biotyper CA System for organism identification. While a technologist selects colonies from a digital image, the subsequent steps of picking, suspension, turbidity measurement, and spotting are automated without human intervention. The performance metrics are based on the results obtained from the prepared targets by the downstream MALDI-TOF MS system.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
- The ground truth primarily appears to be "expected identification" or "known identifications" of microorganisms. This implies an established and validated identification for each isolate, likely obtained through standard microbiological methods or a reference MALDI-TOF MS system. For the "Organism Identification Accuracy" study, the comparison was between the BD Kiestra IdentifA prepared samples and "manual sample preparation, i.e. performing the extended Direct Transfer (eDT) Procedure and spotting on a MALDI target according to the previously FDA-cleared Bruker MALDI Biotyper CA user manual." This suggests the "expected result" was either based on prior rigorous identification or the result from the manual predicate method.
8. The sample size for the training set
- The document does not explicitly state the sample size for a training set. As an automated specimen preparation system, it's possible its internal algorithms (e.g., for colony picking, turbidity estimation, pipetting precision) were developed and optimized using various datasets, but these "training sets" are not described in terms of size or content. The provided studies focus on validation/test set performance.
9. How the ground truth for the training set was established
- Not applicable, as a specific "training set" and its ground truth establishment are not described in the provided text. The device's function is mechanical automation of a known lab procedure, not an AI model that learns from training data in the same way. Any inherent 'intelligence' (e.g., image processing for colony identification) would have been programmed based on established features rather than learned through labeled training data.
§ 866.3378 Clinical mass spectrometry microorganism identification and differentiation system.
(a)
Identification. A clinical mass spectrometry microorganism identification and differentiation system is a qualitative in vitro diagnostic device intended for the identification and differentiation of microorganisms from processed human specimens. The system acquires, processes, and analyzes spectra to generate data specific to a microorganism(s). The device is indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial and fungal infection.(b)
Classification. Class II (special controls). The special controls for this device are:(1) The intended use statement must include a detailed description of what the device detects, the type of results provided to the user, the clinical indications appropriate for test use, and the specific population(s) for which the device is intended, when applicable.
(2) Any sample collection device used must be FDA-cleared, -approved, or -classified as 510(k) exempt with an indication for in vitro diagnostic use.
(3) The labeling required under § 809.10(b) of this chapter must include:
(i) A detailed device description, including all device components, control elements incorporated into the test procedure, instrument requirements, ancillary reagents required but not provided, and a detailed explanation of the methodology and all pre-analytical methods for processing of specimens, and algorithm used to generate a final result. This must include a description of validated inactivation procedure(s) that are confirmed through a viability testing protocol, as applicable.
(ii) Performance characteristics for all claimed sample types from clinical studies with clinical specimens that include prospective samples and/or, if appropriate, characterized samples.
(iii) Performance characteristics of the device for all claimed sample types based on analytical studies, including limit of detection, inclusivity, reproducibility, interference, cross-reactivity, interfering substances, carryover/cross-contamination, sample stability, and additional studies regarding processed specimen type and intended use claims, as applicable.
(iv) A detailed explanation of the interpretation of test results for clinical specimens and acceptance criteria for any quality control testing.
(4) The device's labeling must include a prominent hyperlink to the manufacturer's website where the manufacturer must make available their most recent version of the device's labeling required under § 809.10(b) of this chapter, which must reflect any changes in the performance characteristics of the device. FDA must have unrestricted access to this website, or manufacturers must provide this information to FDA through an alternative method that is considered and determined by FDA to be acceptable and appropriate.
(5) Design verification and validation must include:
(i) Any clinical studies must be performed with samples representative of the intended use population and compare the device performance to results obtained from an FDA-accepted reference method and/or FDA-accepted comparator method, as appropriate. Documentation from the clinical studies must include the clinical study protocol (including predefined statistical analysis plan, if applicable), clinical study report, and results of all statistical analyses.
(ii) Performance characteristics for analytical and clinical studies for specific identification processes for the following, as appropriate:
(A) Bacteria,
(B) Yeasts,
(C) Molds,
(D) Mycobacteria,
(E) Nocardia,
(F) Direct sample testing (
e.g., blood culture),(G) Antibiotic resistance markers, and
(H) Select agents (
e.g., pathogens of high consequence).(iii) Documentation that the manufacturer's risk mitigation strategy ensures that their device does not prevent any device(s) with which it is indicated for use, including incorporated device(s), from achieving their intended use (
e.g., safety and effectiveness of the functions of the indicated device(s) remain unaffected).(iv) A detailed device description, including the following:
(A) Overall device design, including all device components and all control elements incorporated into the testing procedure.
(B) Algorithm used to generate a final result from raw data (
e.g., how raw signals are converted into a reported result).(C) A detailed description of device software, including validation activities and outcomes.
(D) Acquisition parameters (
e.g., mass range, laser power, laser profile and number of laser shots per profile, raster scan, signal-to-noise threshold) used to generate data specific to a microorganism.(E) Implementation methodology, construction parameters, and quality assurance protocols, including the standard operating protocol for generation of reference entries for the device.
(F) For each claimed microorganism characteristic, a minimum of five reference entries for each organism (including the type strain for microorganism identification), or, if there are fewer reference entries, a clinical and/or technical justification, determined by FDA to be acceptable and appropriate, for why five reference entries are not needed.
(G) DNA sequence analysis characterizing all type strains and at least 20 percent of the non-type strains of a species detected by the device, or, if there are fewer strain sequences, then a clinical and/or technical justification, determined by FDA to be acceptable and appropriate, must be provided for the reduced number of strains sequenced.
(H) As part of the risk management activities, an appropriate end user device training program, which must be offered as an effort to mitigate the risk of failure from user error.