Search Results

The BD Kiestra™ Methicillin-resistant Staphylococcus aureus (MRSA) Application is an in-vitro diagnostic software program that requires the BD Kiestra™ Laboratory Automation Solution in order to operate.

The BD Kiestra™ Methicillin-resistant Staphylococcus aureus (MRSA) Application is applied to digital images of BD BBL™ CHROMagar™ MRSA II culture plates inoculated with anterior nares samples.

Algorithms are applied to digital images to provide a qualitative assessment of colony growth and colorimetric detection of target colonies for the detection of nasal colonization by MRSA and to serve as an aid in the prevention and control of MRSA infection. Applied algorithms provide the following results:

• "No growth", which will be manually released individually or as a batch (with other no growth samples) by . a trained microbiologist upon review of the digital plate images.
• . "Growth - other" (growth without mauve color), which digital plate images will be manually reviewed by a trained microbiologist.
• "Growth MRSA Mauve" (growth with mauve color), which digital plate images will be manually reviewed ● by a trained microbiologist.

The assay is not intended to guide, diagnose, or monitor treatment for MRSA infections. It is not intended to provide results of susceptibility to oxacillin/methicillin.

The BD Kiestra™ Methicillin-resistant Staphylococcus aureus (MRSA) Application is indicated for use in the clinical laboratory.

The BD Kiestra™ Methicillin-resistant Staphylococcus aureus (MRSA) Application will be optional for the BD Kiestra™ Laboratory Automation Solution and will support laboratory technologists in batching no growth on the BD BBL™ CHROMagar™ MRSA II, growth with no key colony color detected for MRSA ("Growth – other"), and growth with key colony color detected for MRSA ("Growth MRSA Mauve"). These classifications will be characterized as "no growth" and "growth with mauve color" from BD BBLTM CHROMagar™ MRSA II media, from anterior nares samples.

The technologist has the ability to create work lists in BD Synapsys™ informatics solution based on the classifications (growth, no growth or growth with mauve color). These work lists will be used for followup work and batching of results, at the sample level.

The BD Kiestra™ Methicillin-resistant Staphylococcus aureus (MRSA) Application will apply Image Algorithms to the digital images to determine if the plate contains "growth" or "no growth". At the individual plate level when the Image Algorithms detects colony growth and potential mauve color the classification will be "growth with mauve color".

When the BD Kiestra™ Methicillin-resistant Staphylococcus aureus (MRSA) Application is not capable of automatically generating the outputs (visual attributes: growth with or without mauve color/no growth), the laboratory technologist will be required to read the digital image of the plate on the computer screen and decide on follow-up action as is the current standard laboratory practice.

Here's a summary of the acceptance criteria and the study details for the BD Kiestra™ Methicillin-resistant Staphylococcus aureus (MRSA) Application, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria are not explicitly stated as distinct numerical targets in the provided document. However, the studies demonstrate performance metrics related to agreement with manual interpretation and reproducibility.

2. Sample sizes used for the test set and the data provenance

• Digital Quality Image Study:
  • Sample Size: 521 plate images (across 3 microbiologists, so 174, 172, and 175 plates respectively for each microbiologist's manual review).
  • Data Provenance: Internal digital image quality study, using simulated surveillance samples (MRSA, non-MRSA, saline controls). Implies data was generated specifically for this study. The location or country of origin is not explicitly stated, but it's an "internal" study. The nature (simulated surveillance samples) suggests it was a prospective generation of samples for the study.
• Digital Image Reproducibility Study:
  • Sample Size: 518 plate images (3 images excluded due to invalid results from at least one microbiologist).
  • Data Provenance: Same as the Digital Quality Image Study, as it re-analyzed the results from that study. "Internal" study, likely newly generated for this purpose.
• Reproducibility Study (Seeded Samples):
  • Sample Size: Variable, ranging from 55 to 1056 individual observations per dilution/organism combination across two sites. Total observations are in the thousands (e.g., 2089 for saline controls).
  • Data Provenance: Internal reproducibility study using seeded samples (bacterial strains grown in saline). Conducted at two internal sites (BD Sparks, MD location). This is also prospective generation of data.
• Clinical Performance Studies:
  • Sample Size: Approximately 1,800 clinical anterior nares specimens.
  • Data Provenance: Clinical anterior nares specimens. Collected at "three clinical sites." The text does not specify the country of origin, but "clinical sites" generally refer to real-world healthcare settings. This is a prospective collection of real patient samples.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

• Digital Quality Image Study & Digital Image Reproducibility Study:
  • Number of Experts: 3.
  • Qualifications: "Three trained clinical microbiologists." Specific years of experience or board certifications are not provided.
• Reproducibility Study (Seeded Samples):
  • The ground truth for seeded samples is inherently defined by the known organisms and dilutions used to create the samples. The study then assesses the automated system's agreement with this known "ground truth" for growth and color. Human experts here are likely involved in verifying the initial seeding and later in the interpretation of the results, but the definition of "mauve" or "non-mauve" for specific strains is pre-defined.
• Clinical Performance Studies:
  • Number of Experts: Not explicitly stated as a fixed number, but the images were "manually read by trained microbiologists at those sites." This implies multiple microbiologists across the three clinical sites.
  • Qualifications: "Trained microbiologists." Specific years of experience or board certifications are not provided.

4. Adjudication method for the test set

• Digital Image Reproducibility Study: The "final digital image result" was "determined by 2/3 majority microbiologist result." This indicates a form of consensus-based adjudication, specifically a majority vote among the three microbiologists.
• Other studies (Digital Quality Image, Reproducibility with Seeded Samples, Clinical Performance): The primary comparison for these studies appears to be between the device's output and individual manual reads by microbiologists or known characteristics of seeded samples. Explicit adjudication methods for generating a single "ground truth" reference for these specific studies are not detailed, though for the clinical study, the manual read at each site serves as the reference.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• A MRMC comparative effectiveness study, where human readers' performance with and without AI assistance is quantitatively measured and compared, is not explicitly described in the provided text.
• The studies focus on the agreement of the device with manual reads (Digital Quality Image, Clinical Performance) and the reproducibility of human reads of digital images (Digital Image Reproducibility). While these demonstrate the device's capability to produce similar results to human reads, they don't directly quantify the improvement of human readers due to AI assistance. The device's stated purpose is to "aid in the prevention and control of MRSA infection" and to support laboratory technologists in "batching" and "streamline and optimize the reading workflow," suggesting an assistive role. However, the magnitude of this assistance in terms of effect size on human reader performance is not presented.

6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done

• Yes, a standalone performance analysis was conducted. The tables in the "Analytical Performance" and "Clinical Performance Studies" sections (e.g., Table 1, Table 6, and the final clinical performance table on page 17) directly compare the BD Kiestra™ MRSA App's output to the manual plate reads or ground truth. The "Percent Agreement" values (e.g., "No Growth Percent Agreement 75.6% (773/1023)") represent the algorithm's performance against those references.
• It's important to note that even when the app provides results, the indications for use state that "No growth", "Growth - other", and "Growth MRSA Mauve" classifications "will be manually reviewed by a trained microbiologist." This means that while the algorithm provides a classification, a human-in-the-loop is always part of the final release process. However, the data presented directly assesses the algorithm's standalone accuracy in classifying the images.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

• Expert Consensus: Used in the "Digital Image Reproducibility Study," where a 2/3 majority microbiologist result formed the ground truth for comparing individual microbiologist reproducibility.
• Manual Expert Reading: For the "Digital Quality Image Study" and the "Clinical Performance Studies," the ground truth was established by the manual interpretation of the plates (or digital images of plates) by trained microbiologists. This acts as the "gold standard" for comparison.
• Known Sample Characteristics: For the "Reproducibility Study (Seeded Samples)," the ground truth was defined by the known bacterial strains, dilutions, and expected growth/color characteristics of the seeded samples.

8. The sample size for the training set

• The document does not provide information on the sample size used for the training set. It focuses solely on the performance of the already-trained and developed algorithm.

9. How the ground truth for the training set was established

• The document does not describe how the ground truth for the training set was established. Information regarding the training data, annotation process, or expert involvement in labeling training data is not included in the provided text.

Ask a specific question about this device

The APAS Independence is an in vitro diagnostic system comprised of an instrument for automated imaging of agar plates and a software analysis module for the following uses:

1. The APAS Independence, when using its IC MRSA Chromogenic BD Analysis Module, automates culture plate imaging and interpretation to detect the presence of colonies with colors suggestive of methicillin-resistant Staphylococcus aureus (MRSA) growth on Beckton Dickinson BBL™ CHROMagar™ MRSA II agar that has been inoculated with anterior nares swabs and incubated at 36°C ± 1°C for 24 hours. The APAS Independence, when using its IC MRSA Chromogenic BD Analysis Module, provides an aid in routine screening for colonization with MRSA. It provides one of two screening results: Presumptive MRSA or Negative. All culture plates that are identified as Presumptive MRSA by the APAS Independence, when using the IC MRSA Chromogenic BD Analysis Module require review by a trained microbiologist.

2. The APAS Independence, when using its IC MRSA Chromogenic TFS/S Analysis Module, automates culture plate imaging and interpretation to detect the presence of colonies with colors suggestive of methicillin-resistant Staphylococus aureus (MRSA) growth on Thermo-Fisher Spectra™ MRSA agar that has been inoculated with anterior nares swabs and incubated at 36°C ± 1°C for 24 hours. The APAS Independence, when using its IC MRSA Chromogenic TFS/S Analysis Module, provides an aid in routine screening for colonization with MRSA. It provides one of three screening results: Presumptive MRSA, Presumptive non-MRSA, or Negative. All culture plates that are identified as Presumptive MRSA or Presumptive non-MRSA by the APAS Independence, when using the IC MRSA Chromogenic TFS/S Analysis Module, require review by a trained microbiologist.

APAS Independence is a device designed to be used in a microbiology laboratory to automate the initial screening of specimens for the presence of growth on culture plates. It is an in vitro diagnostic device and has no direct contact with patients.

The APAS Independence consists of an automated plate handling mechanism to move culture plates through the instrument, an imaging station to capture images of culture plates, and software for image analysis (e.g., determination of growth) and presentation of reports.

The APAS Independence is intended to be installed with multiple software (analysis) modules, each of which will provide an assessment of growth for a specific clinical indication. More than one analysis module may be developed for the same indication to allow APAS to assess growth on culture plates from multiple agar manufacturers sold for the same indications.

This submission includes two MRSA analysis modules that have been developed for the same indication, which is to be used in a microbiology laboratory to automate the initial screening for the presence of presumptive methicillin-resistant Staphylococcus aureus (MRSA) growth on culture plates. It is indicated for the screening of MRSA colonization from swabs, where a specimen is collected from the anterior nares by non-invasive sampling techniques and plated onto specified chromogenic MRSA agars. No quantification of growth is required.

APAS Independence with IC MRSA Chromogenic BD Analysis Module is designed to interpret growth on BBL™ CHROMagar™ MRSA II agar from Becton Dickinson, and APAS Independence with IC MRSA Chromogenic TFS/S Analysis Module is designed to interpret growth on Spectra™ MRSA agar from Thermo Fisher Scientific (Remel).

Here's a breakdown of the acceptance criteria and study findings for the APAS Independence with MRSA Analysis Modules, based on the provided document:

1. Table of Acceptance Criteria and Reported Device Performance

The document defines acceptance for "Presumptive MRSA growth" based on agreement with a reference panel. The distinction between a 2-designation and 3-designation rule set is also tied to specific performance metrics.

2. Sample Size Used for the Test Set and Data Provenance

• BD Analysis Module & TFS/S Analysis Module (Clinical Study):

  • Total Samples: 1590 enrolled.
  • BD Analyzed: 1573 (1118 non-simulated, 395 simulated MRSA-positive, 60 simulated negative MRSA).
  • TFS/S Analyzed: 1580 (1122 non-simulated, 398 simulated MRSA-positive, 60 simulated negative MRSA).
  • Data Provenance: A single site in Australia collected left-over specimens from standard-of-care screening. Simulated samples were also used to supplement the study, incorporating unique MRSA strains for global representation, including the United States. This indicates a mix of prospective (clinical) and engineered (simulated) data.

• Digital Image Quality Study (Reproducibility & Equivalency):

  • BD Media: 263 plates for "No growth", 82 for "Presumptive non-MRSA", 55 for "Presumptive MRSA" (Reproducibility); 263, 77, 60 respectively for "Microbiologist Image vs Plate-in-Hand" (Equivalency).
  • TFS/S Media: 196 plates for "No growth", 103 for "Presumptive non-MRSA", 101 for "Presumptive MRSA" (Reproducibility); 194, 101, 105 respectively for "Microbiologist Image vs Plate-in-Hand" (Equivalency).
  • Data Provenance: Not explicitly stated, but likely laboratory-generated images and physical plates used by microbiologists during the study.

• Accuracy - Trueness Study:

  • BD Analysis Module: 1106 colonies evaluated (798 MRSA, 308 Non-MRSA).
  • TFS/S Analysis Module: 1320 colonies evaluated (776 MRSA, 544 Non-MRSA).
  • Data Provenance: Laboratory-produced pure cultures of MRSA and non-MRSA organisms.

• Accuracy - Precision Study:

  • Sample Size: For each organism (MRSA wild strain, MRSA ATCC 43300, S. haemolyticus wild strains 1 & 2), 3 dilutions were used, with each diluted sample inoculated in triplicate. Five replicate images of each plate were taken at 3 different orientations on 3 different APAS Independence instruments. For each organism/dilution/instrument combination, there are 45 images * (e.g. MRSA Wild Strain 1, dilution 1 on Instrument 1 has 45 images).
  • Data Provenance: Laboratory-produced cultures.

• Analytical Sensitivity - Limit of Detection of Colony Size:

  • Sample Size: Two MRSA strains (one ATCC, one wild). Six replicate plates prepared for each organism, with images taken at 1-hour intervals until colonies were approx. 2mm. Multiple isolated colonies digitally labeled for size measurement.
  • Data Provenance: Laboratory-produced cultures.

• Analytical Specificity - Limit of Blank:

  • Sample Size: 96 plates (3 applicators x 4 media x 4 labels x 2 marks).
  • Data Provenance: Laboratory-prepared plates with no growth.

• Analytical Specificity - Interference with detection of MRSA:

  • Sample Size: 48 plates per organism (3 applicators x 2 media x 4 labels x 2 marks) for MRSA and for S. haemolyticus heavy growth.
  • Data Provenance: Laboratory-prepared cultures with known interfering substances/conditions.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications

• Clinical Study (IVD Studies) & Analytical Sensitivity - Detection of MRSA in Mixtures:

  • Number of Experts: Two reference panels, each consisting of 3 clinical microbiologists.
  • Qualifications: "Clinical microbiologists." Specific years of experience or board certifications are not explicitly stated in the document.

• Digital Image Quality Study (Reproducibility & Equivalency):

  • Number of Experts: 3 microbiologists.
  • Qualifications: "Microbiologists." Specific years of experience or board certifications are not explicitly stated.

• Accuracy - Trueness Study:

  • Number of Experts: One microbiologist for digitally labeling isolated colonies.
  • Qualifications: "Microbiologist." Specific years of experience or board certifications are not explicitly stated.

4. Adjudication Method for the Test Set

• Clinical Study (IVD Studies) & Analytical Sensitivity - Detection of MRSA in Mixtures:

  • Adjudication Method (Reference Panels): Majority vote (likely 3+0 or 2+1, meaning at least 2 out of 3 agreed) from the respective panel of 3 independent microbiologists. This established the "truth state."

• Digital Image Quality Study (Reproducibility):

  • Adjudication Method: Single panel result (i.e. majority vote) for each image among the three microbiologists.

• Digital Image Quality Study (Equivalency):

  • Adjudication Method: The plate-in-hand interpretation was considered the "truth state."

• Accuracy - Trueness Study:

  • Adjudication Method: Ground truth for individual colonies was established by digital labeling by a microbiologist.

5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study was done

• Yes, in part. The clinical studies (IVD studies) for both analysis modules involved multiple readers (the 3 microbiologists in each of the two panels) and multiple cases (the clinical and simulated samples) to establish the reference standard against which the APAS device was compared.
• Effect Size of Human Readers Improve with AI vs. without AI assistance:
  • The study design was a standalone performance evaluation of the AI without human assistance, compared to a human consensus reference. It did not evaluate the comparative effectiveness of human readers with AI vs. without AI assistance. Therefore, there is no reported effect size for how much human readers improve with AI assistance from this document. The device instead functions as an aid in routine screening, flagging plates that require review by a trained microbiologist.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done

• Yes, a standalone performance evaluation was largely conducted. The core performance objective was to compare the APAS-generated result (algorithm-only) against the reference panel of microbiologists. The study determined the agreement of the APAS final designation with the consensus of human experts.
• The "Indications for Use" explicitly state: "All culture plates that are identified as Presumptive MRSA by the APAS Independence... require review by a trained microbiologist." This indicates that the device operates as a screening/triage tool, and positive APAS results are not considered final without human review.

7. The Type of Ground Truth Used

• Expert Consensus (Microbiologist Panel): For the clinical studies (IVD) and some analytical studies (e.g., Detection of MRSA in Mixtures), the ground truth was established by the majority vote of two independent panels of 3 clinical microbiologists interpreting digital images.
• Plate-in-Hand Interpretation: For the "Equivalency of Plate-in-Hand vs. Plate Image" component of the digital image quality study, the plate-in-hand interpretation by a microbiologist was considered the truth state.
• Microbiologist Digital Labeling: For the "Accuracy - Trueness" and "Analytical Sensitivity - Limit of Detection of Colony Size" studies, the ground truth at the colony level was established by a microbiologist digitally labeling colonies.
• Expected Outcome (Laboratory-Controlled): For "Precision" and "Analytical Specificity" studies, the "expected outcome" (presence/absence of presumptive MRSA/growth) derived from controlled laboratory preparations served as the ground truth.

8. The Sample Size for the Training Set

• The document does not explicitly state the sample size for the training set used to develop the APAS analysis modules. The provided data focuses entirely on the validation/test sets.

9. How the Ground Truth for the Training Set was Established

• The document does not explicitly describe how the ground truth for the training set was established. It concentrates on the methodologies for the performance evaluation and validation data.

Ask a specific question about this device