Search Results

The APAS Independence is an in vitro diagnostic system comprised of an instrument for automated imaging of agar plates and a software analysis module for the following uses:

1. The APAS Independence, when using its IC MRSA Chromogenic BD Analysis Module, automates culture plate imaging and interpretation to detect the presence of colonies with colors suggestive of methicillin-resistant Staphylococcus aureus (MRSA) growth on Beckton Dickinson BBL™ CHROMagar™ MRSA II agar that has been inoculated with anterior nares swabs and incubated at 36°C ± 1°C for 24 hours. The APAS Independence, when using its IC MRSA Chromogenic BD Analysis Module, provides an aid in routine screening for colonization with MRSA. It provides one of two screening results: Presumptive MRSA or Negative. All culture plates that are identified as Presumptive MRSA by the APAS Independence, when using the IC MRSA Chromogenic BD Analysis Module require review by a trained microbiologist.

2. The APAS Independence, when using its IC MRSA Chromogenic TFS/S Analysis Module, automates culture plate imaging and interpretation to detect the presence of colonies with colors suggestive of methicillin-resistant Staphylococus aureus (MRSA) growth on Thermo-Fisher Spectra™ MRSA agar that has been inoculated with anterior nares swabs and incubated at 36°C ± 1°C for 24 hours. The APAS Independence, when using its IC MRSA Chromogenic TFS/S Analysis Module, provides an aid in routine screening for colonization with MRSA. It provides one of three screening results: Presumptive MRSA, Presumptive non-MRSA, or Negative. All culture plates that are identified as Presumptive MRSA or Presumptive non-MRSA by the APAS Independence, when using the IC MRSA Chromogenic TFS/S Analysis Module, require review by a trained microbiologist.

APAS Independence is a device designed to be used in a microbiology laboratory to automate the initial screening of specimens for the presence of growth on culture plates. It is an in vitro diagnostic device and has no direct contact with patients.

The APAS Independence consists of an automated plate handling mechanism to move culture plates through the instrument, an imaging station to capture images of culture plates, and software for image analysis (e.g., determination of growth) and presentation of reports.

The APAS Independence is intended to be installed with multiple software (analysis) modules, each of which will provide an assessment of growth for a specific clinical indication. More than one analysis module may be developed for the same indication to allow APAS to assess growth on culture plates from multiple agar manufacturers sold for the same indications.

This submission includes two MRSA analysis modules that have been developed for the same indication, which is to be used in a microbiology laboratory to automate the initial screening for the presence of presumptive methicillin-resistant Staphylococcus aureus (MRSA) growth on culture plates. It is indicated for the screening of MRSA colonization from swabs, where a specimen is collected from the anterior nares by non-invasive sampling techniques and plated onto specified chromogenic MRSA agars. No quantification of growth is required.

APAS Independence with IC MRSA Chromogenic BD Analysis Module is designed to interpret growth on BBL™ CHROMagar™ MRSA II agar from Becton Dickinson, and APAS Independence with IC MRSA Chromogenic TFS/S Analysis Module is designed to interpret growth on Spectra™ MRSA agar from Thermo Fisher Scientific (Remel).

Here's a breakdown of the acceptance criteria and study findings for the APAS Independence with MRSA Analysis Modules, based on the provided document:

1. Table of Acceptance Criteria and Reported Device Performance

The document defines acceptance for "Presumptive MRSA growth" based on agreement with a reference panel. The distinction between a 2-designation and 3-designation rule set is also tied to specific performance metrics.

2. Sample Size Used for the Test Set and Data Provenance

• BD Analysis Module & TFS/S Analysis Module (Clinical Study):

  • Total Samples: 1590 enrolled.
  • BD Analyzed: 1573 (1118 non-simulated, 395 simulated MRSA-positive, 60 simulated negative MRSA).
  • TFS/S Analyzed: 1580 (1122 non-simulated, 398 simulated MRSA-positive, 60 simulated negative MRSA).
  • Data Provenance: A single site in Australia collected left-over specimens from standard-of-care screening. Simulated samples were also used to supplement the study, incorporating unique MRSA strains for global representation, including the United States. This indicates a mix of prospective (clinical) and engineered (simulated) data.

• Digital Image Quality Study (Reproducibility & Equivalency):

  • BD Media: 263 plates for "No growth", 82 for "Presumptive non-MRSA", 55 for "Presumptive MRSA" (Reproducibility); 263, 77, 60 respectively for "Microbiologist Image vs Plate-in-Hand" (Equivalency).
  • TFS/S Media: 196 plates for "No growth", 103 for "Presumptive non-MRSA", 101 for "Presumptive MRSA" (Reproducibility); 194, 101, 105 respectively for "Microbiologist Image vs Plate-in-Hand" (Equivalency).
  • Data Provenance: Not explicitly stated, but likely laboratory-generated images and physical plates used by microbiologists during the study.

• Accuracy - Trueness Study:

  • BD Analysis Module: 1106 colonies evaluated (798 MRSA, 308 Non-MRSA).
  • TFS/S Analysis Module: 1320 colonies evaluated (776 MRSA, 544 Non-MRSA).
  • Data Provenance: Laboratory-produced pure cultures of MRSA and non-MRSA organisms.

• Accuracy - Precision Study:

  • Sample Size: For each organism (MRSA wild strain, MRSA ATCC 43300, S. haemolyticus wild strains 1 & 2), 3 dilutions were used, with each diluted sample inoculated in triplicate. Five replicate images of each plate were taken at 3 different orientations on 3 different APAS Independence instruments. For each organism/dilution/instrument combination, there are 45 images * (e.g. MRSA Wild Strain 1, dilution 1 on Instrument 1 has 45 images).
  • Data Provenance: Laboratory-produced cultures.

• Analytical Sensitivity - Limit of Detection of Colony Size:

  • Sample Size: Two MRSA strains (one ATCC, one wild). Six replicate plates prepared for each organism, with images taken at 1-hour intervals until colonies were approx. 2mm. Multiple isolated colonies digitally labeled for size measurement.
  • Data Provenance: Laboratory-produced cultures.

• Analytical Specificity - Limit of Blank:

  • Sample Size: 96 plates (3 applicators x 4 media x 4 labels x 2 marks).
  • Data Provenance: Laboratory-prepared plates with no growth.

• Analytical Specificity - Interference with detection of MRSA:

  • Sample Size: 48 plates per organism (3 applicators x 2 media x 4 labels x 2 marks) for MRSA and for S. haemolyticus heavy growth.
  • Data Provenance: Laboratory-prepared cultures with known interfering substances/conditions.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications

• Clinical Study (IVD Studies) & Analytical Sensitivity - Detection of MRSA in Mixtures:

  • Number of Experts: Two reference panels, each consisting of 3 clinical microbiologists.
  • Qualifications: "Clinical microbiologists." Specific years of experience or board certifications are not explicitly stated in the document.

• Digital Image Quality Study (Reproducibility & Equivalency):

  • Number of Experts: 3 microbiologists.
  • Qualifications: "Microbiologists." Specific years of experience or board certifications are not explicitly stated.

• Accuracy - Trueness Study:

  • Number of Experts: One microbiologist for digitally labeling isolated colonies.
  • Qualifications: "Microbiologist." Specific years of experience or board certifications are not explicitly stated.

4. Adjudication Method for the Test Set

• Clinical Study (IVD Studies) & Analytical Sensitivity - Detection of MRSA in Mixtures:

  • Adjudication Method (Reference Panels): Majority vote (likely 3+0 or 2+1, meaning at least 2 out of 3 agreed) from the respective panel of 3 independent microbiologists. This established the "truth state."

• Digital Image Quality Study (Reproducibility):

  • Adjudication Method: Single panel result (i.e. majority vote) for each image among the three microbiologists.

• Digital Image Quality Study (Equivalency):

  • Adjudication Method: The plate-in-hand interpretation was considered the "truth state."

• Accuracy - Trueness Study:

  • Adjudication Method: Ground truth for individual colonies was established by digital labeling by a microbiologist.

5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study was done

• Yes, in part. The clinical studies (IVD studies) for both analysis modules involved multiple readers (the 3 microbiologists in each of the two panels) and multiple cases (the clinical and simulated samples) to establish the reference standard against which the APAS device was compared.
• Effect Size of Human Readers Improve with AI vs. without AI assistance:
  • The study design was a standalone performance evaluation of the AI without human assistance, compared to a human consensus reference. It did not evaluate the comparative effectiveness of human readers with AI vs. without AI assistance. Therefore, there is no reported effect size for how much human readers improve with AI assistance from this document. The device instead functions as an aid in routine screening, flagging plates that require review by a trained microbiologist.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done

• Yes, a standalone performance evaluation was largely conducted. The core performance objective was to compare the APAS-generated result (algorithm-only) against the reference panel of microbiologists. The study determined the agreement of the APAS final designation with the consensus of human experts.
• The "Indications for Use" explicitly state: "All culture plates that are identified as Presumptive MRSA by the APAS Independence... require review by a trained microbiologist." This indicates that the device operates as a screening/triage tool, and positive APAS results are not considered final without human review.

7. The Type of Ground Truth Used

• Expert Consensus (Microbiologist Panel): For the clinical studies (IVD) and some analytical studies (e.g., Detection of MRSA in Mixtures), the ground truth was established by the majority vote of two independent panels of 3 clinical microbiologists interpreting digital images.
• Plate-in-Hand Interpretation: For the "Equivalency of Plate-in-Hand vs. Plate Image" component of the digital image quality study, the plate-in-hand interpretation by a microbiologist was considered the truth state.
• Microbiologist Digital Labeling: For the "Accuracy - Trueness" and "Analytical Sensitivity - Limit of Detection of Colony Size" studies, the ground truth at the colony level was established by a microbiologist digitally labeling colonies.
• Expected Outcome (Laboratory-Controlled): For "Precision" and "Analytical Specificity" studies, the "expected outcome" (presence/absence of presumptive MRSA/growth) derived from controlled laboratory preparations served as the ground truth.

8. The Sample Size for the Training Set

• The document does not explicitly state the sample size for the training set used to develop the APAS analysis modules. The provided data focuses entirely on the validation/test sets.

9. How the Ground Truth for the Training Set was Established

• The document does not explicitly describe how the ground truth for the training set was established. It concentrates on the methodologies for the performance evaluation and validation data.

Ask a specific question about this device