Search Results

The BD Kiestra™ Methicillin-resistant Staphylococcus aureus (MRSA) Application is an in-vitro diagnostic software program that requires the BD Kiestra™ Laboratory Automation Solution in order to operate.

The BD Kiestra™ Methicillin-resistant Staphylococcus aureus (MRSA) Application is applied to digital images of BD BBL™ CHROMagar™ MRSA II culture plates inoculated with anterior nares samples.

Algorithms are applied to digital images to provide a qualitative assessment of colony growth and colorimetric detection of target colonies for the detection of nasal colonization by MRSA and to serve as an aid in the prevention and control of MRSA infection. Applied algorithms provide the following results:

• "No growth", which will be manually released individually or as a batch (with other no growth samples) by . a trained microbiologist upon review of the digital plate images.
• . "Growth - other" (growth without mauve color), which digital plate images will be manually reviewed by a trained microbiologist.
• "Growth MRSA Mauve" (growth with mauve color), which digital plate images will be manually reviewed ● by a trained microbiologist.

The assay is not intended to guide, diagnose, or monitor treatment for MRSA infections. It is not intended to provide results of susceptibility to oxacillin/methicillin.

The BD Kiestra™ Methicillin-resistant Staphylococcus aureus (MRSA) Application is indicated for use in the clinical laboratory.

The BD Kiestra™ Methicillin-resistant Staphylococcus aureus (MRSA) Application will be optional for the BD Kiestra™ Laboratory Automation Solution and will support laboratory technologists in batching no growth on the BD BBL™ CHROMagar™ MRSA II, growth with no key colony color detected for MRSA ("Growth – other"), and growth with key colony color detected for MRSA ("Growth MRSA Mauve"). These classifications will be characterized as "no growth" and "growth with mauve color" from BD BBLTM CHROMagar™ MRSA II media, from anterior nares samples.

The technologist has the ability to create work lists in BD Synapsys™ informatics solution based on the classifications (growth, no growth or growth with mauve color). These work lists will be used for followup work and batching of results, at the sample level.

The BD Kiestra™ Methicillin-resistant Staphylococcus aureus (MRSA) Application will apply Image Algorithms to the digital images to determine if the plate contains "growth" or "no growth". At the individual plate level when the Image Algorithms detects colony growth and potential mauve color the classification will be "growth with mauve color".

When the BD Kiestra™ Methicillin-resistant Staphylococcus aureus (MRSA) Application is not capable of automatically generating the outputs (visual attributes: growth with or without mauve color/no growth), the laboratory technologist will be required to read the digital image of the plate on the computer screen and decide on follow-up action as is the current standard laboratory practice.

Here's a summary of the acceptance criteria and the study details for the BD Kiestra™ Methicillin-resistant Staphylococcus aureus (MRSA) Application, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria are not explicitly stated as distinct numerical targets in the provided document. However, the studies demonstrate performance metrics related to agreement with manual interpretation and reproducibility.

2. Sample sizes used for the test set and the data provenance

• Digital Quality Image Study:
  • Sample Size: 521 plate images (across 3 microbiologists, so 174, 172, and 175 plates respectively for each microbiologist's manual review).
  • Data Provenance: Internal digital image quality study, using simulated surveillance samples (MRSA, non-MRSA, saline controls). Implies data was generated specifically for this study. The location or country of origin is not explicitly stated, but it's an "internal" study. The nature (simulated surveillance samples) suggests it was a prospective generation of samples for the study.
• Digital Image Reproducibility Study:
  • Sample Size: 518 plate images (3 images excluded due to invalid results from at least one microbiologist).
  • Data Provenance: Same as the Digital Quality Image Study, as it re-analyzed the results from that study. "Internal" study, likely newly generated for this purpose.
• Reproducibility Study (Seeded Samples):
  • Sample Size: Variable, ranging from 55 to 1056 individual observations per dilution/organism combination across two sites. Total observations are in the thousands (e.g., 2089 for saline controls).
  • Data Provenance: Internal reproducibility study using seeded samples (bacterial strains grown in saline). Conducted at two internal sites (BD Sparks, MD location). This is also prospective generation of data.
• Clinical Performance Studies:
  • Sample Size: Approximately 1,800 clinical anterior nares specimens.
  • Data Provenance: Clinical anterior nares specimens. Collected at "three clinical sites." The text does not specify the country of origin, but "clinical sites" generally refer to real-world healthcare settings. This is a prospective collection of real patient samples.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

• Digital Quality Image Study & Digital Image Reproducibility Study:
  • Number of Experts: 3.
  • Qualifications: "Three trained clinical microbiologists." Specific years of experience or board certifications are not provided.
• Reproducibility Study (Seeded Samples):
  • The ground truth for seeded samples is inherently defined by the known organisms and dilutions used to create the samples. The study then assesses the automated system's agreement with this known "ground truth" for growth and color. Human experts here are likely involved in verifying the initial seeding and later in the interpretation of the results, but the definition of "mauve" or "non-mauve" for specific strains is pre-defined.
• Clinical Performance Studies:
  • Number of Experts: Not explicitly stated as a fixed number, but the images were "manually read by trained microbiologists at those sites." This implies multiple microbiologists across the three clinical sites.
  • Qualifications: "Trained microbiologists." Specific years of experience or board certifications are not provided.

4. Adjudication method for the test set

• Digital Image Reproducibility Study: The "final digital image result" was "determined by 2/3 majority microbiologist result." This indicates a form of consensus-based adjudication, specifically a majority vote among the three microbiologists.
• Other studies (Digital Quality Image, Reproducibility with Seeded Samples, Clinical Performance): The primary comparison for these studies appears to be between the device's output and individual manual reads by microbiologists or known characteristics of seeded samples. Explicit adjudication methods for generating a single "ground truth" reference for these specific studies are not detailed, though for the clinical study, the manual read at each site serves as the reference.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• A MRMC comparative effectiveness study, where human readers' performance with and without AI assistance is quantitatively measured and compared, is not explicitly described in the provided text.
• The studies focus on the agreement of the device with manual reads (Digital Quality Image, Clinical Performance) and the reproducibility of human reads of digital images (Digital Image Reproducibility). While these demonstrate the device's capability to produce similar results to human reads, they don't directly quantify the improvement of human readers due to AI assistance. The device's stated purpose is to "aid in the prevention and control of MRSA infection" and to support laboratory technologists in "batching" and "streamline and optimize the reading workflow," suggesting an assistive role. However, the magnitude of this assistance in terms of effect size on human reader performance is not presented.

6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done

• Yes, a standalone performance analysis was conducted. The tables in the "Analytical Performance" and "Clinical Performance Studies" sections (e.g., Table 1, Table 6, and the final clinical performance table on page 17) directly compare the BD Kiestra™ MRSA App's output to the manual plate reads or ground truth. The "Percent Agreement" values (e.g., "No Growth Percent Agreement 75.6% (773/1023)") represent the algorithm's performance against those references.
• It's important to note that even when the app provides results, the indications for use state that "No growth", "Growth - other", and "Growth MRSA Mauve" classifications "will be manually reviewed by a trained microbiologist." This means that while the algorithm provides a classification, a human-in-the-loop is always part of the final release process. However, the data presented directly assesses the algorithm's standalone accuracy in classifying the images.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

• Expert Consensus: Used in the "Digital Image Reproducibility Study," where a 2/3 majority microbiologist result formed the ground truth for comparing individual microbiologist reproducibility.
• Manual Expert Reading: For the "Digital Quality Image Study" and the "Clinical Performance Studies," the ground truth was established by the manual interpretation of the plates (or digital images of plates) by trained microbiologists. This acts as the "gold standard" for comparison.
• Known Sample Characteristics: For the "Reproducibility Study (Seeded Samples)," the ground truth was defined by the known bacterial strains, dilutions, and expected growth/color characteristics of the seeded samples.

8. The sample size for the training set

• The document does not provide information on the sample size used for the training set. It focuses solely on the performance of the already-trained and developed algorithm.

9. How the ground truth for the training set was established

• The document does not describe how the ground truth for the training set was established. Information regarding the training data, annotation process, or expert involvement in labeling training data is not included in the provided text.

Ask a specific question about this device

The APAS Independence is an in vitro diagnostic system comprised of an instrument for automated imaging of agar plates and a software analysis module for the following uses:

1. The APAS Independence, when using its IC MRSA Chromogenic BD Analysis Module, automates culture plate imaging and interpretation to detect the presence of colonies with colors suggestive of methicillin-resistant Staphylococcus aureus (MRSA) growth on Beckton Dickinson BBL™ CHROMagar™ MRSA II agar that has been inoculated with anterior nares swabs and incubated at 36°C ± 1°C for 24 hours. The APAS Independence, when using its IC MRSA Chromogenic BD Analysis Module, provides an aid in routine screening for colonization with MRSA. It provides one of two screening results: Presumptive MRSA or Negative. All culture plates that are identified as Presumptive MRSA by the APAS Independence, when using the IC MRSA Chromogenic BD Analysis Module require review by a trained microbiologist.

2. The APAS Independence, when using its IC MRSA Chromogenic TFS/S Analysis Module, automates culture plate imaging and interpretation to detect the presence of colonies with colors suggestive of methicillin-resistant Staphylococus aureus (MRSA) growth on Thermo-Fisher Spectra™ MRSA agar that has been inoculated with anterior nares swabs and incubated at 36°C ± 1°C for 24 hours. The APAS Independence, when using its IC MRSA Chromogenic TFS/S Analysis Module, provides an aid in routine screening for colonization with MRSA. It provides one of three screening results: Presumptive MRSA, Presumptive non-MRSA, or Negative. All culture plates that are identified as Presumptive MRSA or Presumptive non-MRSA by the APAS Independence, when using the IC MRSA Chromogenic TFS/S Analysis Module, require review by a trained microbiologist.

APAS Independence is a device designed to be used in a microbiology laboratory to automate the initial screening of specimens for the presence of growth on culture plates. It is an in vitro diagnostic device and has no direct contact with patients.

The APAS Independence consists of an automated plate handling mechanism to move culture plates through the instrument, an imaging station to capture images of culture plates, and software for image analysis (e.g., determination of growth) and presentation of reports.

The APAS Independence is intended to be installed with multiple software (analysis) modules, each of which will provide an assessment of growth for a specific clinical indication. More than one analysis module may be developed for the same indication to allow APAS to assess growth on culture plates from multiple agar manufacturers sold for the same indications.

This submission includes two MRSA analysis modules that have been developed for the same indication, which is to be used in a microbiology laboratory to automate the initial screening for the presence of presumptive methicillin-resistant Staphylococcus aureus (MRSA) growth on culture plates. It is indicated for the screening of MRSA colonization from swabs, where a specimen is collected from the anterior nares by non-invasive sampling techniques and plated onto specified chromogenic MRSA agars. No quantification of growth is required.

APAS Independence with IC MRSA Chromogenic BD Analysis Module is designed to interpret growth on BBL™ CHROMagar™ MRSA II agar from Becton Dickinson, and APAS Independence with IC MRSA Chromogenic TFS/S Analysis Module is designed to interpret growth on Spectra™ MRSA agar from Thermo Fisher Scientific (Remel).

Here's a breakdown of the acceptance criteria and study findings for the APAS Independence with MRSA Analysis Modules, based on the provided document:

1. Table of Acceptance Criteria and Reported Device Performance

The document defines acceptance for "Presumptive MRSA growth" based on agreement with a reference panel. The distinction between a 2-designation and 3-designation rule set is also tied to specific performance metrics.

2. Sample Size Used for the Test Set and Data Provenance

• BD Analysis Module & TFS/S Analysis Module (Clinical Study):

  • Total Samples: 1590 enrolled.
  • BD Analyzed: 1573 (1118 non-simulated, 395 simulated MRSA-positive, 60 simulated negative MRSA).
  • TFS/S Analyzed: 1580 (1122 non-simulated, 398 simulated MRSA-positive, 60 simulated negative MRSA).
  • Data Provenance: A single site in Australia collected left-over specimens from standard-of-care screening. Simulated samples were also used to supplement the study, incorporating unique MRSA strains for global representation, including the United States. This indicates a mix of prospective (clinical) and engineered (simulated) data.

• Digital Image Quality Study (Reproducibility & Equivalency):

  • BD Media: 263 plates for "No growth", 82 for "Presumptive non-MRSA", 55 for "Presumptive MRSA" (Reproducibility); 263, 77, 60 respectively for "Microbiologist Image vs Plate-in-Hand" (Equivalency).
  • TFS/S Media: 196 plates for "No growth", 103 for "Presumptive non-MRSA", 101 for "Presumptive MRSA" (Reproducibility); 194, 101, 105 respectively for "Microbiologist Image vs Plate-in-Hand" (Equivalency).
  • Data Provenance: Not explicitly stated, but likely laboratory-generated images and physical plates used by microbiologists during the study.

• Accuracy - Trueness Study:

  • BD Analysis Module: 1106 colonies evaluated (798 MRSA, 308 Non-MRSA).
  • TFS/S Analysis Module: 1320 colonies evaluated (776 MRSA, 544 Non-MRSA).
  • Data Provenance: Laboratory-produced pure cultures of MRSA and non-MRSA organisms.

• Accuracy - Precision Study:

  • Sample Size: For each organism (MRSA wild strain, MRSA ATCC 43300, S. haemolyticus wild strains 1 & 2), 3 dilutions were used, with each diluted sample inoculated in triplicate. Five replicate images of each plate were taken at 3 different orientations on 3 different APAS Independence instruments. For each organism/dilution/instrument combination, there are 45 images * (e.g. MRSA Wild Strain 1, dilution 1 on Instrument 1 has 45 images).
  • Data Provenance: Laboratory-produced cultures.

• Analytical Sensitivity - Limit of Detection of Colony Size:

  • Sample Size: Two MRSA strains (one ATCC, one wild). Six replicate plates prepared for each organism, with images taken at 1-hour intervals until colonies were approx. 2mm. Multiple isolated colonies digitally labeled for size measurement.
  • Data Provenance: Laboratory-produced cultures.

• Analytical Specificity - Limit of Blank:

  • Sample Size: 96 plates (3 applicators x 4 media x 4 labels x 2 marks).
  • Data Provenance: Laboratory-prepared plates with no growth.

• Analytical Specificity - Interference with detection of MRSA:

  • Sample Size: 48 plates per organism (3 applicators x 2 media x 4 labels x 2 marks) for MRSA and for S. haemolyticus heavy growth.
  • Data Provenance: Laboratory-prepared cultures with known interfering substances/conditions.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications

• Clinical Study (IVD Studies) & Analytical Sensitivity - Detection of MRSA in Mixtures:

  • Number of Experts: Two reference panels, each consisting of 3 clinical microbiologists.
  • Qualifications: "Clinical microbiologists." Specific years of experience or board certifications are not explicitly stated in the document.

• Digital Image Quality Study (Reproducibility & Equivalency):

  • Number of Experts: 3 microbiologists.
  • Qualifications: "Microbiologists." Specific years of experience or board certifications are not explicitly stated.

• Accuracy - Trueness Study:

  • Number of Experts: One microbiologist for digitally labeling isolated colonies.
  • Qualifications: "Microbiologist." Specific years of experience or board certifications are not explicitly stated.

4. Adjudication Method for the Test Set

• Clinical Study (IVD Studies) & Analytical Sensitivity - Detection of MRSA in Mixtures:

  • Adjudication Method (Reference Panels): Majority vote (likely 3+0 or 2+1, meaning at least 2 out of 3 agreed) from the respective panel of 3 independent microbiologists. This established the "truth state."

• Digital Image Quality Study (Reproducibility):

  • Adjudication Method: Single panel result (i.e. majority vote) for each image among the three microbiologists.

• Digital Image Quality Study (Equivalency):

  • Adjudication Method: The plate-in-hand interpretation was considered the "truth state."

• Accuracy - Trueness Study:

  • Adjudication Method: Ground truth for individual colonies was established by digital labeling by a microbiologist.

5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study was done

• Yes, in part. The clinical studies (IVD studies) for both analysis modules involved multiple readers (the 3 microbiologists in each of the two panels) and multiple cases (the clinical and simulated samples) to establish the reference standard against which the APAS device was compared.
• Effect Size of Human Readers Improve with AI vs. without AI assistance:
  • The study design was a standalone performance evaluation of the AI without human assistance, compared to a human consensus reference. It did not evaluate the comparative effectiveness of human readers with AI vs. without AI assistance. Therefore, there is no reported effect size for how much human readers improve with AI assistance from this document. The device instead functions as an aid in routine screening, flagging plates that require review by a trained microbiologist.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done

• Yes, a standalone performance evaluation was largely conducted. The core performance objective was to compare the APAS-generated result (algorithm-only) against the reference panel of microbiologists. The study determined the agreement of the APAS final designation with the consensus of human experts.
• The "Indications for Use" explicitly state: "All culture plates that are identified as Presumptive MRSA by the APAS Independence... require review by a trained microbiologist." This indicates that the device operates as a screening/triage tool, and positive APAS results are not considered final without human review.

7. The Type of Ground Truth Used

• Expert Consensus (Microbiologist Panel): For the clinical studies (IVD) and some analytical studies (e.g., Detection of MRSA in Mixtures), the ground truth was established by the majority vote of two independent panels of 3 clinical microbiologists interpreting digital images.
• Plate-in-Hand Interpretation: For the "Equivalency of Plate-in-Hand vs. Plate Image" component of the digital image quality study, the plate-in-hand interpretation by a microbiologist was considered the truth state.
• Microbiologist Digital Labeling: For the "Accuracy - Trueness" and "Analytical Sensitivity - Limit of Detection of Colony Size" studies, the ground truth at the colony level was established by a microbiologist digitally labeling colonies.
• Expected Outcome (Laboratory-Controlled): For "Precision" and "Analytical Specificity" studies, the "expected outcome" (presence/absence of presumptive MRSA/growth) derived from controlled laboratory preparations served as the ground truth.

8. The Sample Size for the Training Set

• The document does not explicitly state the sample size for the training set used to develop the APAS analysis modules. The provided data focuses entirely on the validation/test sets.

9. How the Ground Truth for the Training Set was Established

• The document does not explicitly describe how the ground truth for the training set was established. It concentrates on the methodologies for the performance evaluation and validation data.

Ask a specific question about this device

The APAS Independence is an in vitro diagnostic system comprised of an instrument and software analysis module(s) for specific indications that are used to automate imaging and interpretation of microbial colonies on plates of solid culture media.

The APAS Independence, when using its urine analysis module, automates urine culture plate imaging and interpretation to detect the presence or absence of microbial growth on sheep blood and MacConkey agar culture plates that are inoculated with a 1µL sample volume. The APAS Independence, when using its urine analysis module, provides a semi-quantitative assessment of colony counts that are used as an aid in the diagnosis of urinary tract infection. All urine culture plates that are identified as positive for growth by the APAS Independence, when using its urine analysis module, must be reviewed by a trained microbiologist.

APAS Independence with Urine Analysis Module is a device designed to be used in a microbiology laboratory to automate the initial screening for the presence of growth on urine culture plates. It is an in vitro diagnostic device and has no direct contact with patients.

APAS Independence consists of an automated plate handling mechanism to move the plates through the instrument, an imaging station to capture an image of the culture plate, combined with software for analysis of the image, determination of growth and presentation of reports.

The APAS Independence with Urine Analysis Module is intended to determine whether growth is present or not, and to provide a semi-quantitative assessment of the colony count (if present). This information will then be combined with other available clinical information to screen out biological samples without growth. All other plates will be presented to a microbiologist for examination, determination of status and further testing according to conventional laboratory practice. This enables the microbiologist to focus on plates with potentially significant growth, thereby reducing the time until results can be reported.

The APAS Independence is intended to have different software modules, each of which will provide an assessment of growth for specific clinical indications. This submission covers only the APAS Independence with Urine Analysis Module. The APAS Independence with Urine Analysis Module is indicated for screening of culture plates for assessment of urinary tract infections where the urine specimens are collected and 1μl is plated onto Blood and MacConkey Agars and incubated at 35±2°C for 18 to 22 hours.

The provided text describes the acceptance criteria and the study proving the device meets these criteria for the APAS Independence with Urine Analysis Module (K183648).

1. Table of Acceptance Criteria (Implicit) and Reported Device Performance:

The document establishes substantial equivalence to a predicate device, APAS Compact with Urine Analysis Module. Therefore, the acceptance criteria are implicitly tied to the performance of the predicate device. The performance data presented focuses on agreement with the predicate.

2. Sample Size and Data Provenance for Test Set:

• Sample Size: 350 leftover clinical urine samples.
• Data Provenance: The document states "leftover clinical urine samples." The country of origin is not explicitly stated but is implicitly Australia, given the submitter's country for the predicate device's de novo application (DEN150059) was Australia. The data is retrospective as it uses "leftover clinical urine samples."

3. Number of Experts and Qualifications for Ground Truth for Test Set:

Not applicable for this section as the study's ground truth for the comparison was the predicate device's performance, not human expert consensus, for the new device's equivalence study.

4. Adjudication Method for the Test Set:

Not applicable. The study compares the new device (APAS Independence) to the predicate device (APAS Compact) itself, not to human expert reads requiring adjudication. Discrepancies between the two devices are analyzed, and the clinical implications (e.g., misclassification leading to microbiologist review) are discussed.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:

• Was it done? No, an MRMC study comparing human readers with and without AI assistance was not conducted directly for the APAS Independence with Urine Analysis Module in this submission.
• The document states that the predicate device (APAS Compact with Urine Analysis Module) had three clinical studies (LBT001, LBT002, LBT003) submitted for its de novo applicationDEN150059, which "showed that APAS performed similarly to a microbiologist in reading and interpreting agar plates." This indicates that the predicate device's performance was evaluated against human microbiologists. The current submission established equivalence to this predicate.
• Effect Size: Not applicable since an MRMC study was not performed as part of this specific submission.

6. Standalone (Algorithm Only) Performance:

• Was it done? Yes, the entire study presented for the APAS Independence with Urine Analysis Module is a standalone (algorithm only) performance comparison between the new device and the predicate device. Both the APAS Independence and the APAS Compact are automated systems that read and interpret images without direct human intervention in the initial classification process.

7. Type of Ground Truth Used:

The ground truth for the APAS Independence performance evaluation was the performance of the predicate device (APAS Compact with Urine Analysis Module). The study aimed to demonstrate that the new device performs equivalently to the already-cleared predicate.

8. Sample Size for the Training Set:

The document does not explicitly state the sample size for the training set of the APAS Independence with Urine Analysis Module. It only details the test set used for the comparison with the predicate device (350 samples). However, it mentions that both devices "utilize the same core APAS technology and the same urine analysis module," implying shared underlying algorithmic development and training.

9. How the Ground Truth for the Training Set was Established:

The document does not provide details on how the ground truth for the training set (if any specific to APAS Independence's development rather than its use of the predicate's technology) was established. However, given that the "same core APAS technology and the same urine analysis module" are utilized as the predicate, it is highly likely that the training methodologies and ground truth establishment for the predicate device would apply. For the predicate device (APAS Compact), the clinical studies (LBT001, LBT002, LBT003) referenced for the de novo application would have established ground truth, likely through expert consensus of microbiologists, as it stated "APAS performed similarly to a microbiologist."

Ask a specific question about this device

The APAS Compact is an in vitro diagnostic system comprised of an instrument for automated imaging of agar culture plates and a software analysis module for the following use:

The APAS Compact, when using its urine analysis module, automates urine culture plate imaging and interpretation to detect the presence of microbial growth on sheep blood and MacConkey agar culture plates that are inoculated with a 1uL sample volume. The APAS Compact, when using its urine analysis module, provides a semiquantitative assessment of colony counts that are used as an aid in the diagnosis of urinary tract infection. All urine culture plates that are identified as positive for growth by the APAS Compact, when using its urine analysis module, must be reviewed by a trained microbiologist.

The APAS Compact with Urine Analysis Module is an instrumented system that is designed for screening of urine culture plates for the presence of microbial growth. The device comprises an imaging station for capture of digital images of the culture plates, together with software for analysis of the images, the determination and enumeration of microbial growth and reporting of results.

This document outlines the acceptance criteria and study findings for the APAS Compact with Urine Analysis Module.

1. Acceptance Criteria and Reported Device Performance

2. Sample Sizes and Data Provenance

• Test Set (Clinical Study):

  • Total Enrolled Samples: 10,100 urine samples (5835 from Site 1, 2117 from Site 2, 2148 from Site 3).
  • Included in Performance Analysis: 9,224 samples (5634 from Site 1, 1769 from Site 2, 1821 from Site 3).
  • Data Provenance: Three clinical sites: one in the US (Site 1) and two ex-US (Sites 2 and 3). The study used remnant urine samples from standard of care culture. This indicates a retrospective data collection approach for the clinical study.

• Training Set: The document does not explicitly state the sample size for the training set.

3. Number of Experts and Qualifications for Ground Truth (Test Set)

• Number of Experts: Three microbiologists.
• Qualifications: "Trained microbiologist" is mentioned multiple times. For the clinical study, each microbiologist was "trained to read the urine culture plates in a standard fashion." Specific years of experience are not provided.

4. Adjudication Method (Test Set)

• The document states that for the reference method in the clinical study, "each microbiologist... was blinded to the results from the other panel members and to those obtained by the APAS Compact."
• The ground truth for the clinical study was established by a "reference microbiologist panel," meaning the consensus among these three microbiologists. The exact method of achieving consensus (e.g., 2+1, 3+1, none) is not explicitly detailed, but it implies a shared understanding to form the "reference method."

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• No, a MRMC comparative effectiveness study was not done. The clinical study compared the APAS Compact's performance against a panel of microbiologists (serving as the reference standard), not against human readers with and without AI assistance to measure improvement. The primary goal of the APAS Compact is to automate initial screening to reduce microbiologist workload, rather than directly augment human reading for every plate.

6. Standalone Performance Study

• Yes, a standalone performance study was done. The entire clinical study (Section L.3) and analytical performance studies (Section L.1) evaluate the APAS Compact's performance as an algorithm-only device (without human-in-the-loop during the initial assessment and designation phase). The performance tables (e.g., Tables 19-24) directly report the APAS Compact's designations and colony counts compared to the reference standard. While the device's indications for use emphasize that growth-positive plates "must be reviewed by a trained microbiologist," the reported performance metrics are for the algorithm's initial assessment before this mandatory human review.

7. Type of Ground Truth Used

• Clinical Study: Expert Consensus (Microbiologist Panel): The ground truth for the clinical study was established by "an independent panel of three microbiologists," who read the urine culture plates in a "standard fashion" and were blinded to other results. Their combined assessment served as the reference method.
• Analytical Study (Linearity/Assay Reportable Range): Expert Consensus (Manual Colony Counts): The mean of two independent manual colony counts performed by two microbiologists was used as the reference result for each plate.

8. Sample Size for the Training Set

• The document does not explicitly state the sample size for the training set. It describes the "APAS Controller Software" and "Urine Analysis Module Software" functions, implying that these algorithms were developed and likely trained, but the specific training data volume is not provided in this excerpt.

9. How Ground Truth for the Training Set Was Established

• The document does not explicitly describe how the ground truth for the training set was established. While it details the methods for establishing ground truth in the analytical and clinical validation studies (using expert microbiologist consensus), it does not provide information specific to the data used for training the algorithms.

Ask a specific question about this device