Search Results

The LIAISON PLEX® Gram-Positive Blood Culture Assay (BCP), performed using the automated, sample-to-result LIAISON PLEX® System, is a qualitative multiplexed in vitro diagnostic test for the simultaneous detection and identification of selected gram-positive pathogens and/or selected genetic determinants associated with antimicrobial resistance in positive blood culture bottles. BCP is performed directly on blood culture media using blood culture bottles identified as positive by a continuous monitoring blood culture system and which contain gram-positive bacteria as determined by Gram stain.

The BCP Assay detects and identifies the following:

Gram Positive Resistance Markers:

• mecA/mecC
• vanA
• vanB

Genera and Species:

• Bacillus spp.
• Enterococcus faecalis
• Enterococcus faecium
• Listeria spp.
• Staphylococcus spp.
• Staphylococcus aureus
• Staphylococcus epidermidis
• Staphylococcus lugdunensis
• Streptococcus spp.
• Streptococcus agalactiae
• Streptococcus anginosus group
• Streptococcus pneumoniae
• Streptococcus pyogenes

Negative results for antimicrobial resistance genes do not indicate bacterial susceptibility as there are multiple mechanisms that can contribute to resistance.

The LIAISON PLEX® BCP Assay contains targets for the detection of genetic determinants associated with resistance to methicillin (mecA/C) and vancomycin (vanA and vanB) to aid in the identification of potentially antimicrobial-resistant organisms in positive blood culture samples. In mixed growth, the LIAISON PLEX BCP Assay does not specifically attribute vanA/vanB-mediated vancomycin resistance to either E. faecalis or E. faecium, or mecA/mecC-mediated methicillin resistance to either Staphylococcus spp., S. aureus, S. epidermidis or S. lugdunensis.

The antimicrobial resistance gene or marker detected may or may not be associated with the agent responsible for disease. Negative results for these select antimicrobial resistance gene and marker assays do not indicate susceptibility, as multiple mechanisms of methicillin and vancomycin resistance exist.

The LIAISON PLEX® BCP Assay is indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial bloodstream infections (BSI). The LIAISON PLEX® BCP Assay is not intended to monitor these infections. Sub-culturing of positive blood cultures is necessary to recover organisms for antimicrobial susceptibility testing (AST), for identification of organisms not detected by the LIAISON PLEX BCP Assay, to detect mixed infections that may not be detected by the LIAISON PLEX BCP Assay, for association of antimicrobial resistance genes to a specific organism, or for epidemiological typing.

The LIAISON PLEX® Gram-Positive Blood Culture Assay (BCP Assay) is an automated test for the detection and identification of nucleic acid from gram-positive bacteria in a positive blood culture media sample. The BCP Assay is performed directly on blood culture media using blood culture bottles identified as positive by a continuous monitoring blood culture system, and which contain gram-positive bacteria, as determined by a Gram stain.

The LIAISON PLEX® System is a fully automated, bench-top "sample-to-answer" device that performs sample preparation and microarray-based hybridization for the detection of target-specific nucleic acids. The test reagents are supplied as a single, disposable test cartridge. Target amplification is not performed as part of the BCP Assay workflow, as it is a non-amplified, direct detection test performed on the LIAISON PLEX® System.

The provided text describes the acceptance criteria and the study that proves the LIAISON PLEX® Gram-Positive Blood Culture Assay meets those criteria. However, it's important to note that this document is a 510(k) Clearance Letter and 510(k) Summary, which focuses on demonstrating substantial equivalence to a predicate device for regulatory clearance. It is not an academic paper detailing a clinical trial with specific acceptance criteria that would typically be associated with AI/ML diagnostic performance metrics like those in a multi-reader multi-case (MRMC) study or the establishment of ground truth by multiple experts.

The study described here is for an in vitro diagnostic test for the detection of nucleic acid sequences, not for an AI/ML-based diagnostic imaging device. Therefore, many of the requested points (e.g., number of experts for ground truth, adjudication method, MRMC study, human reader improvement with AI assistance) are not applicable to this type of device and study.

I will interpret the provided information in the context of the device described (a multiplex nucleic acid assay) and its verification and validation.

Acceptance Criteria and Device Performance

The acceptance criteria for this device are implicitly derived from the performance goals demonstrated in the analytical and clinical studies, aiming for high sensitivity and specificity in detecting specified microorganisms and resistance markers. The reported performance is presented in several tables throughout the document.

Table 1: Acceptance Criteria (Implicit) and Reported Device Performance

2. Sample Sizes and Data Provenance

• Test Set (Clinical Study):
  • Prospective Arm (Arm 1): 562 unique specimens enrolled, 509 included in analysis.
  • Pre-selected Arm (Arm 2): 163 pre-selected left-over specimens, 162 included in analysis.
  • Contrived Arm (Arm 3): 225 contrived specimens.
  • Total specimens analyzed for clinical performance: 509 (prospective) + 162 (pre-selected) + 225 (contrived) = 896 unique samples.
• Data Provenance:
  • Prospective Arm: Collected between April 2024 and August 2024 from four geographically diverse clinical sites within the United States. Data is prospective.
  • Pre-selected Arm: Sourced from ten vendors in the United States and one site in Italy. Data are retrospective (left-over, de-identified specimens).
  • Contrived Arm: Specimens were prepared, blinded, randomized, and tested at three external testing sites and one internal testing site between June 2024 to September 2024. These are contrived (synthetic) samples.

3. Number of Experts and Qualifications for Ground Truth

• Not Applicable in the traditional sense for this IVD device. The ground truth for this nucleic acid assay is established through reference methods such as culture with automated microbiological/biochemical identification (VITEK 2), PCR followed by bi-directional sequencing (BDS), or a combination thereof. This is a laboratory-based diagnostic test, not an imaging device requiring human expert interpretation for ground truth.

4. Adjudication Method for the Test Set

• Not Applicable in the traditional sense. The "adjudication" for the gold standard (reference method) involves a hierarchical algorithm (Table 19). If initial VITEK 2 results were insufficient or for specific targets like Bacillus spp. and resistance markers (mecA/mecC, vanA, vanB), PCR followed by bi-directional sequencing was used. This is a technical validation process against established laboratory methods rather than a consensus among clinical experts interpreting an output.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Not done. This type of study (MRMC) is typically used for diagnostic imaging devices where human readers interpret images with and without AI assistance. This device is a molecular diagnostic assay that provides direct results, and therefore, an MRMC study is not relevant.

6. Standalone (i.e., algorithm only without human-in-the-loop performance)

• Yes, in essence. The LIAISON PLEX® BCP Assay is an automated, sample-to-result system. Its performance (sensitivity, specificity) is evaluated independently against a reference method. While trained operators are involved in loading samples and running the system, the diagnostic result itself is generated by the "system" (instrument + assay + software algorithm) without human diagnostic interpretation of the raw signal data.

7. The Type of Ground Truth Used

• Laboratory Reference Methods:
  • Culture followed by Automated microbiological/biochemical identification using VITEK 2 for most bacterial species.
  • PCR followed by bi-directional sequencing (BDS) for Bacillus spp. and resistance markers (mecA/mecC, vanA, vanB). This method was also used to confirm discordant results or for targets with low prevalence in initial testing.
  • In some cases, MALDI-ToF assay (Matrix-Assisted Laser Desorption/Ionization-Time Of Flight) was also mentioned as a Standard of Care method in the footnotes for discordant result explanations.

8. The Sample Size for the Training Set

• Not explicitly stated in terms of a "training set" for an AI/ML model. This device is a direct detection assay based on hybridization, not a machine learning model that undergoes a training phase with a distinct dataset. Therefore, the concept of a "training set" in the context of data used to train an AI model is not applicable to this traditional IVD. The development process would involve extensive analytical characterization, probe design, and optimization using various microbial strains and clinical samples, but this is distinct from "training data" for an AI algorithm.

9. How the Ground Truth for the Training Set Was Established

• Not applicable for this type of device. As explained above, this is not an AI/ML device that requires a "training set" with ground truth established in the AI/ML context. The assay's design and optimization would rely on established microbiological methods and genetic sequencing to define target sequences and ensure specificity and inclusivity.

Ask a specific question about this device

The LIAISON PLEX® Gram‐Negative Blood Culture (BCN) Assay, performed using the automated, sample‐to‐result LIAISON PLEX® System, is a qualitative multiplexed in vitro diagnostic test for the simultaneous detection and identification of selected gram‐negative pathogens and/or selected genetic determinants associated with antimicrobial resistance in positive blood culture bottles. LIAISON PLEX® BCN Assay is performed directly on blood culture media using blood culture bottles identified as positive by a continuous monitoring blood culture system which contain gram‐negative bacteria as determined by Gram stain.

The LIAISON PLEX® BCN Assay detects and identifies the following:

Resistance Markers:

• CTX‐M (blaCTX‐M)
• IMP (blaIMP)
• KPC (blaKPC)
• NDM (blaNDM)
• OXA (blaOXA)
• VIM (blaVIM)
• MCR
• SME (blaSME)

Gram Negative Genera and Species:

• Enterobacteriaceae / Morganellaceae
• Acinetobacter baumannii
• Acinetobacter spp.
• Citrobacter spp.
• Enterobacter spp. (1)
• Escherichia coli (2)
• Haemophilus influenzae
• Klebsiella oxytoca
• Klebsiella pneumoniae
• Klebsiella variicola
• Morganella morganii
• Neisseria meningitidis
• Proteus spp.
• Pseudomonas aeruginosa
• Pseudomonas spp.
• Salmonella spp.
• Serratia marcescens
• Stenotrophomonas maltophilia

(1) Due to reclassification, Klebsiella aerogenes will be reported Enterobacter spp.
(2) LIAISON PLEX® BCN Assay will not distinguish between Escherichia coli and Shigella spp. (S. dysenteriae, S. boydii, S. flexneri and S. sonnei)

LIAISON PLEX® BCN Assay contains targets for the detection of genetic determinants associated with resistance to carbapenems (blaCTX‐M, blaIMP, blaKPC, blaNDM, blaOXA48‐like, blaVIM, blaSME) to aid in the identification of potentially antimicrobial‐resistant organisms in positive blood culture samples. In addition, the panel includes an assay for the detection of the mobilized genetic determinant MCR, an emerging marker of public health importance. The antimicrobial resistance gene or marker detected may or may not be associated with the agent responsible for disease. Negative results for these select antimicrobial resistance gene and marker assays do not indicate susceptibility, as multiple mechanisms of resistance to ß‐lactams and colistin exist.

LIAISON PLEX® BCN Assay is indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial bloodstream infections (BSI). LIAISON PLEX® BCN Assay is not intended to monitor treatment of these infections. Sub‐culturing of positive blood cultures is necessary to recover organisms for antimicrobial susceptibility testing (AST), for identification of organisms not detected by LIAISON PLEX® BCN Assay, to detect mixed infections that may not be detected by LIAISON PLEX® BCN Assay, for association of antimicrobial resistance marker genes to a specific organism, or for epidemiological typing.

The LIAISON PLEX® Gram‐Negative Blood Culture (BCN) Assay is an automated test for the detection and identification of nucleic acid from gram‐negative bacteria in a positive blood culture media sample. The BCN Assay is performed directly on blood culture media using blood culture bottles identified as positive by a continuous monitoring blood culture system, and which contain gram‐negative bacteria, as determined by a Gram stain.

The LIAISON PLEX® System is a fully automated, bench‐top "sample‐to‐answer" device that performs sample preparation, polymerase chain reaction (PCR) and microarray‐based hybridization for the detection of target‐specific nucleic acids. The test reagents are supplied as a single, disposable test cartridge. PCR is not performed on the LIAISON PLEX® BCN Assay, as it is a non‐amplified, direct detection test performed on the LIAISON PLEX® System.

Here's a breakdown of the acceptance criteria and study details for the LIAISON PLEX Gram-Negative Blood Culture Assay, based on the provided FDA 510(k) clearance letter:

1. Table of Acceptance Criteria and Reported Device Performance

Note: The reported performance for individual targets (Sensitivity/PPA and Specificity/NPA) is explicitly given for the "Combined" data set (Prospective + Pre-selected). For Resistance Markers, some N/A entries indicate insufficient positive reference cases in the clinical study.

2. Sample Size and Data Provenance

• Test Set (Clinical Study):
  • Prospective Samples: 381 unique specimens enrolled, 351 included in analysis (30 excluded due to duplicates or incomplete reference testing).
  • Pre-selected Samples: 231 left-over, de-identified specimens.
  • Contrived Samples: 746 specimens.
  • Total Clinical Samples (Prospective + Pre-selected): 582 specimens included in analysis.
  • Total Contrived Samples: 746 specimens included in analysis.
  • Data Provenance:
    • Prospective: Collected from four geographically diverse clinical sites within the United States between March 2024 and July 2024.
    • Pre-selected: Sourced from seven vendors in the United States and one site in Italy.
    • Contrived: Tested at four testing sites during April 2024 – August 2024.
  • Retrospective/Prospective: Prospective data was collected prospectively. Pre-selected and Contrived data were essentially retrospective/prepared, then tested blindly and randomized.

3. Number of Experts and Qualifications for Ground Truth

The document does not mention the use of "experts" in the sense of clinical specialists (e.g., radiologists) establishing ground truth for the test set. Instead, the ground truth was established by laboratory methods.

4. Adjudication Method for the Test Set

The document describes the "Reference Method Algorithm" (Table 19) where the LIAISON PLEX BCN Assay results were compared to:

• Culture followed by Automated microbiological/biochemical identification using VITEK 2 for most organism targets.
• Culture followed by Automated microbiological/biochemical identification using VITEK 2 with positive confirmation of the clinical isolate by PCR/BDS for Acinetobacter baumannii, Klebsiella oxytoca, Klebsiella pneumoniae, and Klebsiella variicola.
• PCR followed by bi-directional sequencing (BDS) for all resistance markers (CTX-M, IMP, KPC, NDM, OXA, VIM, MCR, SME).

This is a hierarchical or sequential method rather than an "adjudication" by multiple human readers for discrepancies. For pre-selected specimens, initial identification was by Standard of Care (SoC) and/or PCR followed by BDS, then confirmed.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• No, this was not an MRMC comparative effectiveness study where human readers' improvement with AI vs. without AI assistance was measured. This device is an in vitro diagnostic test for direct detection of nucleic acids, not an AI software intended to assist human readers.

6. Standalone Performance (Algorithm Only)

• Yes, this was a standalone performance study. The LIAISON PLEX BCN Assay is an automated, qualitative multiplexed in vitro diagnostic test performed on the LIAISON PLEX System. The performance metrics (Sensitivity/PPA and Specificity/NPA) described in the clinical study (Table 20 and 21) represent the standalone performance of the device without human interpretation of its results.

7. Type of Ground Truth Used

The ground truth for the clinical study (test set) was established using a combination of:

• Culture followed by Automated microbiological/biochemical identification (VITEK 2).
• PCR followed by bi-directional sequencing (BDS).
• In some cases, a combination of VITEK 2 and PCR/BDS for confirmation.
• For resistance markers, ground truth was solely by PCR followed by bi-directional sequencing (BDS).

8. Sample Size for the Training Set

The document does not explicitly state the sample size used for the training set of the LIAISON PLEX BCN Assay. The provided information focuses on the analytical and clinical validation of the final device. For IVD products like this, internal development and optimization (which would involve data that could be considered a "training set" in a broad sense for assay design) are typically not detailed in the same way as machine learning model training sets in FDA submissions. The inclusivity and exclusivity studies (Analytical Reactivity and Specificity) use laboratory-tested strains and in silico analysis respectively, which are key parts of ensuring the assay's design covers the intended targets and avoids false positives.

9. How the Ground Truth for the Training Set Was Established

Since an explicit "training set" sample size isn't provided, the method for establishing its ground truth isn't detailed. However, the comprehensive Analytical Reactivity (Inclusivity) and Analytical Specificity (Exclusivity) studies describe how the assay's design was validated:

• Analytical Reactivity (Inclusivity): Laboratory testing of 246 on-panel organisms (in triplicate) and in silico analysis using sequences from GenBank and WGS databases (February to April 2024). The ground truth for these would be the known identity of the characterized strains and the genetic sequences from public databases.
• Analytical Specificity (Exclusivity): Laboratory testing of 113 off-panel species (in triplicate) and in silico exclusivity assessment against on-panel and off-panel organisms from GenBank (June 7, 2024). The ground truth here is the known identity of these off-panel organisms and their genetic sequences.

These analytical studies effectively serve as the validation of the assay's "knowledge" or "training" on what to detect and what not to detect.

Ask a specific question about this device

The LIAISON PLEX Yeast Blood Culture (BCY) Assay is a qualitative nucleic acid multiplex in vitro diagnostic test intended for use on the LIAISON PLEX System for simultaneous detection and identification of multiple potentially pathogenic fungal organisms in positive blood culture. The LIAISON PLEX BCY Assay is performed directly on blood culture samples identified as positive by a continuous monitoring blood culture system and which contain fungal organisms as determined by Gram Stain. The LIAISON PLEX BCY Assay detects and identifies the following fungal organisms:

Candida albicans
Candida auris
Candida dubliniensis
Candida famata
Candida glabrata
Candida guilliermondii
Candida kefyr
Candida krusei
Candida lipolytica
Candida lusitaniae
Candida parapsilosis
Candida tropicalis
Candida haemulonii / duobushaemulonii
Cryptococcus neoformans / gattii

The detection and identification of specific fungal nucleic acids from individuals exhibiting signs and/or symptoms of bloodstream infection aids in the diagnosis of bloodstream infection when used in conjunction with other clinical information. The results from LIAISON PLEX BCY Assay are intended to be interpreted in conjunction with Gram stain results and should not be used as the sole basis for diagnosis, treatment management decisions.

Negative results in the setting of a suspected bloodstream infection with pathogens that are not detected by this test. Positive results do not rule out co-infection with other organisms; the organism(s) detected by LIAISON PLEX BCY Assay may not be the definite cause of disease. Additional laboratory testing (e.g. sub-culturing of positive blood cultures for identification of organisms not detected by LIAISON PLEX BCY Assay, susceptibility testing and differentiation of mixed growth) and clinical presentation must be taken into consideration in the final diagnosis of bloodstream infection.

The LIAISON PLEX "Yeast Blood Culture Assay (BCY Assay) is performed directly on blood culture media using blood culture bottles identified as positive by a continuous monitoring blood culture system, and which contain a fungal organism, as determined by a Gram stain. The system consists of an instrument, a single-use disposable test cartridge, and a transfer pipette. The user loads the sample into the sample port of the LIAISON PLEX Yeast Blood Culture Assay Cartridge. Next, the user sets up the sample order on the LIAISON PLEX System by first entering the sample information or scanning the barcode ID located on the sample tube, then scanning the barcode ID located on the test cartridge. Last, the user inserts the test cartridge into the processing module to initiate the test. The LIAISON PLEX System identifies the assay being run and automatically initiates the proper testing protocol to process the sample, analyze the data, and generate test results.

The LIAISON PLEX System automates the BCY Assay sample analysis through the following steps: a) Sample Preparation: Nucleic acid extraction via mechanical and chemical cell lysis and magnetic bead-based nucleic acid isolation; b) Amplification: Multiplex PCR based amplification of the extracted nucleic acid to generate target specific amplicons; c) Hybridization: Amplified DNA hybridizes to specific capture DNA arrayed on a glass slide in a microarray format and the bound target DNA, in turn, hybridizes with mediator and gold-nanoparticle probes; d) Signal Analysis: Gold nanoparticle probes bound specifically to target-containing spots in the microarray are silver-enhanced, and light scatter from the spots is measured and further analyzed to determine the presence (Detected) or absence (Not Detected) of a target.

Acceptance Criteria and Device Performance for LIAISON PLEX Yeast Blood Culture Assay

The LIAISON PLEX Yeast Blood Culture (BCY) Assay is a qualitative nucleic acid multiplex in vitro diagnostic test for the simultaneous detection and identification of multiple potentially pathogenic fungal organisms in positive blood cultures. The study summarized below aimed to demonstrate the device meets its acceptance criteria through analytical and clinical performance evaluations.

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria for the LIAISON PLEX BCY Assay clinical study were predefined as:

• Sensitivity: ≥ 90% for all targets
• Specificity: ≥ 95% for each target
• Failure rate: ≤ 15%

The clinical performance data, combining prospective and pre-selected specimens, is summarized in the table below, comparing it against the acceptance criteria.

Table 1: LIAISON PLEX BCY Assay Clinical Performance (Acceptance Criteria vs. Reported Performance)

NA = Not applicable, as there were no positive cases for this target in the combined prospective/pre-selected dataset to calculate sensitivity, or no negative cases to calculate specificity.

The reported performance demonstrates that the LIAISON PLEX BCY Assay met all defined acceptance criteria.

2. Sample Sizes Used for the Test Set and Data Provenance

The test set for the clinical performance evaluation utilized a combination of prospective, pre-selected, and contrived specimens:

• Prospective Specimens (Clinical Study): 69 unique specimens from four geographically diverse clinical sites in the United States. These were collected prospectively between June 2023 and October 2023. One initial specimen was excluded due to an inconclusive Gram stain result.
• Pre-selected Specimens (Clinical Study): 63 remnant, de-identified specimens sourced from 6 different sites/vendors in the United States. The data provenance is retrospective clinical specimens that were pre-characterized.
• Contrived Specimens: 829 specimens were artificially generated. These were blinded, randomized, and tested at all four testing sites during June 2023. These are contrived data.

3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications

The provided document does not explicitly state the number of experts or their specific qualifications (e.g., radiologist with 10 years of experience) used to establish the ground truth for the test set.

However, the ground truth for the clinical study was established by comparing the LIAISON PLEX BCY Assay's performance against a Standard of Care culture followed by identification by Matrix Assisted Laser Desorption/Ionization coupled to time-of-flight Mass Spectrometry (MALDI-TOF MS) for all fungal targets. For the pre-selected specimens, it is mentioned that they were "characterized by an FDA cleared molecular assay prior to enrollment in the study." This implies that the ground truth relied on established laboratory methodologies and potentially an FDA-cleared reference method.

4. Adjudication Method for the Test Set

The document does not describe an adjudication method like "2+1" or "3+1" using human experts for the clinical study. The comparison was made against a "Standard of Care culture followed by identification by MALDI-TOF MS" as the reference method, which serves as the definitive ground truth for the presence and identification of the fungal organisms. For the pre-selected specimens, an "FDA cleared molecular assay" was used for characterization prior to study entry. These are objective laboratory methods rather than subjective expert interpretations requiring adjudication.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done

No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not reported. This device is an in vitro diagnostic (IVD) assay designed for automated detection and identification of fungal organisms directly from blood culture samples. Its performance is evaluated against laboratory reference methods (culture and MALDI-TOF MS, or existing molecular assays), not against human reader interpretation of images or other subjective assessments. Therefore, there is no discussion of how human readers improve with or without AI assistance, as AI assistance is not part of the described use case for this IVD device.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done

Yes, the performance presented for the LIAISON PLEX BCY Assay in the analytical and clinical performance sections is for the standalone algorithm/device performance directly on the sample, without human interpretation or intervention in the diagnostic output. The system automates sample analysis, including nucleic acid extraction, amplification, hybridization, and signal analysis, to generate detected or not detected results. The indication for use clearly states the results are "intended to be interpreted in conjunction with Gram stain results and should not be used as the sole basis for diagnosis, treatment management decisions," but this refers to the clinical application context, not a human-in-the-loop for the device's fundamental diagnostic accuracy calculation.

7. The Type of Ground Truth Used

The primary ground truth used for the clinical study was:

• Standard of Care culture followed by identification by Matrix Assisted Laser Desorption/Ionization coupled to time-of-flight Mass Spectrometry (MALDI-TOF MS). This is a laboratory-based, objective method for identifying microorganisms.
• For pre-selected specimens, ground truth was established by an previously FDA-cleared molecular assay.
• For contrived specimens, the ground truth was inherently known based on how the samples were prepared (spiking known organisms).

8. The Sample Size for the Training Set

The document does not report specific sample sizes for a separate training set for this device. The information provided heavily details analytical and clinical verification/validation (test set) studies. For diagnostic devices like this (nucleic acid assays), the development process typically involves extensive analytical characterization (Limit of Detection, Inclusivity, Exclusivity, Interference, Reproducibility) and then clinical validation. It is probable that internal development, optimization, and early verification studies used various sets of samples, but these are not explicitly termed "training sets" in the context of machine learning model training as one might expect for AI/ML-based diagnostic software.

9. How the Ground Truth for the Training Set Was Established

As no explicit "training set" is documented (in the context of AI/ML), there is no description of how ground truth was established for such a set. However, for the analytical studies and development of the assay, ground truth would have been established through well-characterized reference strains and clinical isolates, quantified using standard microbiology techniques (e.g., CFU/mL for Limit of Detection). These methods involve culturing, molecular characterization, and established laboratory practices to confirm the identity and concentration of the microorganisms.

Ask a specific question about this device

The LIAISON PLEX Respiratory Flex (RSP Flex) Assay is a multiplexed qualitative test for the simultaneous in vitro detection and identification of multiple bacterial and viral nucleic acids in nasopharyngeal swabs (NPS) obtained from individuals with clinical signs and symptoms of respiratory tract infection, including SARS-CoV-2. The test is performed on the automated LIAISON PLEX System utilizing reverse transcription (RT), polymerase chain reaction (PCR), and array hybridization to detect specific nucleic acid gene sequences of the following organism types and subtypes:

Viruses: Adenovirus Human Coronavirus (HKU1, NL63, OC43, and 229E not differentiated) Human Enterovirus/Rhinovirus (not differentiated) Human Metapneumovirus, Influenza A Influenza A (subtype H1) Influenza A (subtype H3) Influenza B Parainfluenza 1 Parainfluenza 2 Parainfluenza 3 Parainfluenza 4 Respiratory Syncytial Virus Severe Acute Respiratory Syndrome Coronavirus (SARS-CoV-2)

Bacteria: Bordetella holmesii Bordetella parapertussis Bordetella pertussis Chlamydia pneumoniae Mycoplasma pneumoniae

Nucleic acids from the bacterial and viral organisms identified by this test are generally detectable in NPS specimens during the acute phase of infection. Detecting and identifying specific bacterial and viral nucleic acids from individuals exhibiting signs and symptoms of respiratory infection aids in the diagnosis of respiratory infection, if used in conjunction with other clinical, epidemiological, and laboratory findings. The results of this test should not be used as the sole basis for diagnosis, treatment, or patient management decisions.

Negative results in the presence of a respiratory illness may be due to infection with pathogens that are not detected by this test or due to lower respiratory that is not detected by an NPS specimen. Conversely, positive results do not rule out infection or co-infection with organisms not detected by the LIAISON PLEX Respiratory Flex (RSP Flex) Assay. The agent(s) detected may not be the definite cause of disease.

The use of additional laboratory testing (e.g., bacterial and viral culture, immunofluorescence, and radiography), may be necessary when evaluating a patient with possible respiratory tract infection.

The LIAISON PLEX® Respiratory Flex Assay is a multiplexed nucleic acid test system composed of the LIAISON PLEX Instrument, the LIAISON PLEX® System Software (preinstalled on the LIAISON PLEX Instrument), the LIAISON PLEX® Respiratory Flex Assay cartridge, and the LIAISON PLEX® Respiratory Flex Assay File. The LIAISON PLEX® Respiratory Flex Assay cartridge contains the reagents to perform nucleic acid extraction and purification, reverse transcription, PCR, and array hybridization. Specifically, the LIAISON PLEX® Respiratory Flex Assay detects bacteria and viruses from nasopharyngeal swab (NPS) specimens collected from individuals with signs and symptoms of respiratory infection.

The LIAISON PLEX System consists of a touchscreen user interface that includes the software for running and analyzing assay results, one to six processing/imaging LIAISON PLEX modules, and a handheld barcode reader. Each LIAISON PLEX module processes one sample at a time under the control of the LIAISON PLEX System software.

LIAISON PLEX® automates the sample processing through analysis within a single cartridge. Processing steps include 1.) Sample Preparation: Nucleic acid extraction from organisms by chemical and mechanical means and isolation of nucleic acid on magnetic beads 2.) Target Amplification: Multiplex PCR and RT-PCR based amplification of extracted nucleic acid to generate target specific amplicons 3.) Hybridization: Amplicons hybridize with their target specific DNA probe arranged in a microarray format and that are attached to mediator and gold nanoparticles 4.) Analysis: Gold nanoparticles specifically bound to target amplicons are silver enhanced and the light scatter from microarray spot is measured and analyzed to confirm presence (Detected) or absence (not Detected) of a target.

The LIAISON PLEX Respiratory Flex Assay has the option of creating and processing results for custom panels using Flex® Software. Flex Software allows users to randomly select and group targets in tiers for result processing. Up to 7 targets may be selected for the initial test tier. After the first tier, each additional tier requires a specific number of credits. Flex™ credits allow the end-user to create custom panels and pay for a smaller subset of results tailored to the individual patient's clinical presentation. Alternatively, a laboratory may choose the fixed price option where all target results are processed at the same time.

The LIAISON PLEX Respiratory Flex Assay is a multiplexed qualitative test for the simultaneous in vitro detection and identification of multiple bacterial and viral nucleic acids in nasopharyngeal swabs (NPS) from individuals with clinical signs and symptoms of respiratory tract infection, including SARS-CoV-2.

Here's an analysis of the acceptance criteria and study proving its performance:

1. Table of Acceptance Criteria and Reported Device Performance

The provided document doesn't explicitly state "acceptance criteria" with numerical thresholds for performance metrics. However, regulatory bodies like the FDA typically expect high sensitivity (Positive Percent Agreement - PPA) and specificity (Negative Percent Agreement - NPA) for diagnostic assays. Based on the clinical performance summary, we can infer the achieved performance.

Infered Acceptance Criteria and Reported Device Performance (Summary for Key Analytes):

Note: For analytes with 0/0 positive cases (Bordetella holmesii, Bordetella pertussis, Chlamydia pneumoniae, Mycoplasma pneumoniae in the prospective study), performance is "Not Evaluable" (NE) but the NPA is 100%. These were supplemented with contrived specimens.

2. Sample Size and Data Provenance

Prospective Clinical Study:

• Sample Size (Test Set): 1843 unique clinical specimens initially enrolled, with 1832 specimens yielding valid results after retests.
• Data Provenance: Prospectively collected between October 2022 to April 2023 from six geographically diverse clinical sites within the United States. Specimens were remnant and de-identified, collected from pediatric and adult patients.

Archived Specimen Testing (Retrospective):

• Sample Size (Test Set): 256 pre-selected, left-over, frozen, de-identified specimens, all yielding valid results after retests.
• Data Provenance: Retrospectively collected from November 2013 through June 2023 from four sites/vendors in the United States.

Contrived Specimen Testing:

• Sample Size (Test Set): 300 contrived specimens, all yielding valid results after retests.
• Data Provenance: Artificially created samples to cover low prevalence targets. Tested at two US sites (details about origin of base matrix not specified beyond "simulated NPS matrix").

3. Number of Experts and Qualifications for Ground Truth

The document does not specify the number of experts used and their exact qualifications (e.g., "radiologist with 10 years of experience") for establishing ground truth. Instead, it indicates that the ground truth was established by comparator methods, which are themselves FDA-cleared molecular panels or analytically validated assays. These methods inherently rely on expertise for their development and validation but do not require additional human expert adjudication for each case in this study.

4. Adjudication Method for the Test Set

Specimens that obtained discordant results between the LIAISON PLEX Respiratory Flex Assay and the comparator method underwent additional testing for investigation.

• For targets typically compared against FDA-cleared molecular respiratory panels, discordant samples were re-tested with an FDA-cleared molecular respiratory panel or PCR/BDS.
• For Bordetella holmesii, Bordetella parapertussis, Bordetella pertussis, comparator performance was based on "well-validated Fragment Analysis (FA) assays followed by PCR/Bi-Directional Sequencing (PCR/BDS) assays."

This implies an adjudication method involving a third, more definitive or confirmatory test (often a "tie-breaker" or "gold standard" method like PCR/BDS) for discordant results. This is a common practice in diagnostic device studies. The document does not specify a "2+1" or "3+1" structure as those typically refer to multiple human readers or interpretations.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not done. This study evaluates the performance of a diagnostic assay itself for detecting nucleic acids, not the impact of AI assistance on human readers' interpretation of images or other data. Therefore, there is no effect size reported for how much human readers improve with AI vs. without AI assistance.

6. Standalone (Algorithm Only) Performance

Yes, a standalone performance was done. The entire clinical performance study (prospective, archived, and contrived) evaluates the LIAISON PLEX Respiratory Flex Assay as an algorithm-only or device-only diagnostic tool without human-in-the-loop interpretation being part of the primary evaluation. The results presented for PPA and NPA are based solely on the device's output compared to the ground truth.

7. Type of Ground Truth Used

The ground truth used was primarily established through comparator laboratory methods:

• FDA-cleared molecular respiratory panels for most viral and bacterial targets.
• FDA-cleared molecular SARS-CoV-2 assay for SARS-CoV-2.
• Analytically Validated Fragment Analysis (FA) assays followed by PCR/Bi-Directional Sequencing (PCR/BDS) assays for Bordetella species.
• For discordant results, additional testing with FDA-cleared molecular respiratory panels or PCR/BDS was performed for investigation.

This combination of highly sensitive and specific molecular diagnostic assays, with confirmatory testing for discordance, serves as the ground truth.

8. Sample Size for the Training Set

The document does not explicitly state the sample size for a separate "training set" for the LIAISON PLEX Respiratory Flex Assay. This is characteristic of molecular diagnostic assays which are typically developed and optimized through analytical studies (e.g., limit of detection, inclusivity, exclusivity) and then validated in clinical performance studies without a distinct "AI training set" in the common machine learning sense. The performance characteristics (analytical and clinical) presented are for the final, locked-down assay.

9. How the Ground Truth for the Training Set was Established

Since a distinct "training set" in the AI/machine learning context is not specified, the method for establishing its ground truth is not detailed. However, the development of such assays involves extensive analytical testing using characterized isolates, strains, and clinical samples to define reactivity, specificity, and sensitivity. The ground truth for this analytical development would typically be based on:

• Known concentrations of purified nucleic acids or organisms.
• Well-characterized reference materials and clinical specimens with confirmed presence/absence of targets via established methods (e.g., culture, sequencing, reference PCR).

Ask a specific question about this device

The ARIES® MRSA Assay is an integrated, real-time, polymerase chain reaction (PCR) based qualitative in vitro diagnostic test for the direct detection of methicillinresistant Staphylococcus aureus (MRSA) DNA from nasal swabs in patients at risk for nasal colonization.

The ARIES® MRSA Assay is intended to aid in the prevention and control of MRSA infections in healthcare settings.

The assay is not intended to guide, diagnose, or monitor treatment for MRSA infections. It is not intended to provide results of susceptibility to oxacillin/methicillin. A negative result does not preclude MRSA nasal colonization. Concomitant cultures are necessary to recover organisms for epidemiological typing or for further susceptibility testing.

The ARIES® MRSA Assay is indicated for use with ARIES® Systems.

The ARIES® MRSA Assay is a polymerase chain reaction (PCR)-based qualitative in vitro diagnostic test system which consists of the ARIES® System or the ARIES® M1 System with their included ARIES® Software, a sample processing tube, an assay-specific cassette, and an assay-specific protocol file. The ARIES® MRSA Assay cassette is a disposable, single-use cassette containing nucleic acid purification reagents, internal sample process control (SPC), and an assay-specific master mix capable of performing the designated assay on one sample. The ARIES® MRSA Assay cassette directly detects methicillin-resistant Staphylococcus aureus (MRSA) from nasal swabs in patients at risk of nasal colonization. Specifically, the ARIES® MRSA Assay cassette detects the methicillin resistance genes (mecA and mecC), Staphylococcus aureus orfX gene, the SCCmec junction region, and a DNA Sample Processing Control.

Nasal swab specimens are collected using the Liquid Amies Elution Swab (ESwab™) Collection and Transport System, or equivalent. A portion of the sample is transferred to the provided 2 mL ARIES MRSA Sample Processing Tube and vortexed. The processed sample is then transferred to the ARIES® MRSA Assay cassette.

The specimen is lysed and nucleic acid is extracted using an ARIES® instrument. An extractable sample processing control (SPC) target present in the ARIES® MRSA Assay cassette is processed with the specimen. The SPC controls for recovery of extracted nucleic acid, for inhibitory substances and for PCR reagent and instrument integrity. The Ct value of the SPC is designed to verify nucleic acid extraction, to identify PCR inhibition, if any, and verify proper function of the extraction system and real-time instrument.

The extracted nucleic acid and SPC are transferred via magnetic beads through the cassette to the ARIES® MRSA Assay lyophilized PCR reagents in the PCR tube that contain primer pairs and probes specific to mecA/mecC, orfX, SCCmec and the SPC sequence. Each probe is labeled with a distinct fluorophore and detected in a distinct channel of an ARIES® System. PCR amplification is performed and assay fluorescence is monitored. Hybridization of a fluorescently labeled probe to the amplified target results in the release of quenching and generation of fluorescence signal that is indicative of PCR generated amplicon. Following amplification, the reaction is heated to separate the fluorescentlabeled probe from the amplified target, a process that results in a decrease in the fluorescence signal. The reaction fluorescence is measured during this process and the temperature at which the change in fluorescence is the maximum is the Tm of the amplicon. The instrument fluorescence output is analyzed and test results are determined using the ARIES® System software and the ARIES® MRSA Assay protocol and run files. ARIES® MRSA Assay results may be reported from the ARIES® Software or from the optional SYNCT® Software.

Here's a breakdown of the acceptance criteria and study information for the ARIES® MRSA Assay, extracted from the provided FDA 510(k) summary:

Acceptance Criteria and Reported Device Performance

Study Information

The provided document describes studies primarily for analytical and clinical performance.

1. Sample sizes used for the test set and the data provenance:

   • Analytical Studies (Test Set):

     • Precision/Reproducibility: A blinded panel of 5 samples (2 MRSA strains at 1x LoD and 5x LoD, and 1 negative sample). Each tested in triplicate by 2 operators over 5 days (within-lab) or at 3 sites by 2 operators over 5 days (site-to-site). Total replicates: 30 per condition for within-lab, 90 per condition for site-to-site.
     • Limit of Detection (LoD): 20 replicates for each MRSA strain at suspected LoD concentration.
     • Inclusivity: 55 MRSA strains, each tested in triplicate. Total: 165 replicates.
     • Challenge Study: 16 MRSA strains with high MICs (some retested), 17 MRSA strains with low MICs (one retested), 4 BORSA strains, 16 empty cassette SA variants, 1 MRSE strain. Each tested in triplicate.
     • Cross-Reactivity: 99 organisms, each tested in triplicate with MRSA positive and negative samples.
     • Interfering Substances: 24 substances, each tested in triplicate with MRSA positive and negative samples.
     • Nasal Swab Equivalency: 1 MRSA strain at 3 concentrations, 1 negative sample. Each tested with two swab types, likely in replicates (e.g., 6/6 for MRSA positivity suggests 6 replicates for each condition).
     • Data Provenance: Not explicitly stated for analytical studies, but often performed in-house by the manufacturer. The strains used are from established collections (BEI, ATCC, ARLG, CDC AR Bank, Lyon University), suggesting controlled laboratory conditions.

   • Clinical Performance Study (Test Set):

     • Total specimens collected: 2254 nasal swab specimens.
     • Specimens meeting eligibility criteria and included in analysis: 1762 unique specimens.
     • Data Provenance: Prospective collection from August 2018 to February 2019 at four (4) geographically distinct clinical sites within the United States.

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • Analytical Studies: Ground truth was established by known organism concentrations and characteristics (e.g., specific MRSA strains, non-MRSA organisms, substances). No human experts are typically involved in establishing ground truth for these types of analytical tests.
   • Clinical Study: The "reference method" for ground truth was direct and enriched bacterial culture. This process is laboratory-based and while it involves trained microbiologists, it does not typically involve "experts" in the sense of clinical reviewers or adjudicators for each case. The results of the culture are the ground truth.

3. Adjudication method (e.g. 2+1, 3+1, none) for the test set:

   • Analytical Studies: No adjudication method described as ground truth is based on known laboratory preparations and characteristics.
   • Clinical Study: No explicit adjudication method for the reference method (bacterial culture) is described. The culture results are presented as the definitive ground truth.

4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No MRMC comparative effectiveness study was mentioned. The ARIES® MRSA Assay is a diagnostic molecular assay that directly detects target DNA; it is not an AI-based imaging or interpretive device that assists human readers.

5. If a standalone (i.e. algorithm only without human-in-the loop performance) was done:

   • Yes, the ARIES® MRSA Assay is a standalone diagnostic device. The performance data presented (sensitivity, specificity, agreement) directly reflects the algorithm's performance (the assay and its associated software) compared to the reference method (bacterial culture). There is no "human-in-the-loop" component for interpretation of the assay results, as the software determines whether the result is Positive, Negative, or Invalid based on predefined internal cut-offs.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

   • Analytical Studies: Ground truth was based on known bacterial strains, concentrations, and characteristics (e.g., presence or absence of specific genes like mecA/mecC, orfX, SCCmec; known categories of non-MRSA organisms or interfering substances).
   • Clinical Study: Ground truth was established by direct and enriched bacterial culture for MRSA.

7. The sample size for the training set:

   • The document implies that an "Initial Assay Protocol File parameters were set during internal optimization and benchmarking studies" and "The final Assay Protocol File parameters were then established during internal verification studies using data from optimization, benchmarking and verification." However, specific sample sizes for these internal training or development phases are not explicitly provided in the 510(k) summary. The listed analytical studies are effectively "test sets" for various performance metrics, not necessarily for training.

8. How the ground truth for the training set was established:

   • As detailed above, specific training sets and their ground truth establishment methods are not explicitly described in this summary. For molecular diagnostic assays like this, the "training" typically involves empirical testing with known positive and negative controls, spiked samples, and characterized clinical samples during assay development to optimize PCR parameters, probe design, and define interpretation algorithms (Ct cut-offs, Tm windows). The "ground truth" during this development phase would be based on well-characterized strains and samples, similar to the analytical studies described.

Ask a specific question about this device

The ARIES® Group A Strep Assay is a real-time polymerase chain reaction (PCR) based qualitative in vitro diagnostic test for the direct detection of Streptococus pyogenes (Group A beta-hemolytic Streptococcus) in throat swab specimens from patients with signs and symptoms of pharyngitis.

The ARIES® Group A Strep Assay can be used as an aid in the diagnosis of Group A Streptococcal pharyngitis. The assay is not intended to monitor treatment for Group A Streptococcus infections.

The ARIES® Group A Strep Assay is indicated for use with ARIES® Systems.

The ARIES® Group A Strep Assay is a polymerase chain reaction (PCR)-based qualitative in vitro diagnostic test system that consists of the ARIES® System or the ARIES® M1 System with their included ARIES® Software, an assay-specific cassette, and an assay-specific protocol file. The ARIES® Group A Strep Assay cassette is a disposable, single-use cassette containing nucleic acid purification reagents, internal sample process control (SPC), and an assay-specific master mix capable of performing the designated assay on one sample. The ARIES® Group A Strep Assay cassette directly detects Streptococcus pyoqenes (Group A ß-hemolytic Streptococcus) in throat swab specimens collected from the surface of human tonsils and posterior pharyngeal wall.

Throat swab specimens are collected from patients using a commercially available Liquid Amies based transport system (Nylon Flocked Swab with 1 mL modified Liquid Amies (ESwab™). The specimen is then transported to the laboratory for testing.

The specimen is lysed and nucleic acid is extracted using an ARIES® System. An extractable sample processing control (SPC) target is present in the ARIES® Group A Strep Assay cassette and is processed with the specimen. The SPC controls for recovery of extracted nucleic acid, the presence of inhibitory substances and for PCR reagent and instrument integrity. The Ct value of the SPC is designed to verify nucleic acid extraction, to identify PCR inhibition, if any, and verify proper function of the extraction system and real-time instrument. The Tm value of the SPC is used as a reference for determining the target Tm.

The extracted nucleic acid and SPC are transferred via magnetic beads through the cassette to the ARIES® Group A Strep Assay lyophilized PCR reagents in the PCR tube that contains primer pairs specific to the S. pyogenes DNaseB (sdaB) gene and the SPC sequence. Each of the primer pairs is labeled with a distinct fluorophore and detected in distinct channels of the ARIES® Systems. PCR amplification is performed and assay fluorescence is monitored. Incorporation of a quencher-labeled nucleotide results in a decrease in fluorescence for the associated primer pair. Following amplification, the reaction is slowly heated to separate the fluorescent-labeled strand from the quencher-labeled strand, a process that results in an increase in the fluorescence signal. The reaction fluorescence is measured during this process and the temperature at which the change in fluorescence is the maximum is the T ,, of the amplicon. The instrument fluorescence output is analyzed and test results are determined using the ARIES® System software and the ARIES® Group A Strep Assay protocol and run files. ARIES® Group A Strep Assay results may be reported from the ARIES® Software or from the optional SYNCT® Software.

Here's a breakdown of the acceptance criteria and the study details for the ARIES® Group A Strep Assay, based on the provided document:

1. Table of Acceptance Criteria and Reported Device Performance

Note: For Lot-to-Lot Reproducibility (Low Positive) and Within-Laboratory Precision/Repeatability (Low Positive), additional replicates were tested to achieve an overall positivity >94%, indicating acceptable performance at the LoD.

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size for Clinical Performance (Test Set): A total of 618 unique specimens were available for analysis in the clinical performance study. Initially, 735 specimens were collected, but 112 were excluded due to various reasons (protocol non-compliance, insufficient volume, incorrect device, etc.).
• Data Provenance: The data was prospectively collected from patients suspected of pharyngitis in four geographically distinct clinical sites within the United States.

3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications of Those Experts

The document does not explicitly state the "number of experts" or their specific "qualifications" (e.g., "Radiologist with 10 years of experience") for establishing the ground truth of the clinical test set.

However, the ground truth for the clinical test set was established by a "reference method (bacterial culture followed by organism identification by Matrix-Assisted Laser Desorption/Ionization - Time-of-Flight Mass Spectrometry (MALDI-TOF MS))". This work was "performed at a centralized testing facility." It is implied that trained laboratory personnel and microbiologists conducted these reference tests, as is standard practice for such methods, but their specific experience levels are not detailed.

4. Adjudication Method for the Test Set

The document describes a robust "ground truth" establishment process using a reference laboratory technique (bacterial culture + MALDI-TOF MS). For specimens where the ARIES assay and the reference method disagreed, an adjudication method was employed:

• Bidirectional sequencing analysis using analytically validated primers that targeted genomic regions distinct from the ARIES Group A Strep Assay was used.
  • For 2 false negative ARIES results (ARIES negative, culture positive), sequencing showed them to be GAS negative.
  • For 7 false positive ARIES results (ARIES positive, culture negative), sequencing showed them to be GAS positive.
    This indicates a method similar to "Truth by Consensus" with an expert-level tie-breaker (sequencing), effectively refining the ground truth against discrepancies.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not done. This device is an in vitro diagnostic (IVD) assay that machine-interprets results (qualitative PCR). It does not involve human "readers" assessing images or data in a way that would be "assisted" by AI, hence the concept of a human reader improvement effect size does not apply.

6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was done

Yes, a standalone performance evaluation was done. The ARIES® Group A Strep Assay is an automated qualitative in vitro diagnostic test system. The clinical performance study directly assessed the agreement of the ARIES® Group A Strep Assay (algorithm only within the ARIES® Systems) against the established reference method, without human interpretation of the assay's raw output. The results (Positive, Negative, Invalid) are determined by the ARIES® System software.

7. The Type of Ground Truth Used (expert consensus, pathology, outcomes data, etc)

The ground truth used for the clinical performance study was primarily bacterial culture followed by organism identification by Matrix-Assisted Laser Desorption/Ionization - Time-of-Flight Mass Spectrometry (MALDI-TOF MS). In cases of discordance with the assay, bidirectional sequencing analysis was used as a confirmatory "higher truth" method. This can be considered a form of expert consensus/gold standard laboratory method.

8. The Sample Size for the Training Set

The document does not explicitly state a sample size for a training set. The ARIES® Group A Strep Assay is a PCR-based diagnostic test, where the "training" typically involves empirical optimization of assay parameters (primers, probes, reaction conditions, and cut-offs) during product development and internal verification studies, rather than machine learning on a distinct "training set" of patient data. The "Initial Assay Protocol File parameters were set during internal optimization studies" and "The final Assay Protocol File parameters were then established during internal verification studies." So, the optimization and verification process served a similar function to "training" for machine learning algorithms.

9. How the Ground Truth for the Training Set Was Established

As noted in point 8, a traditional "training set" with separate ground truth establishment isn't described in the context of a PCR assay. Instead, the "ground truth" for optimizing the assay protocol file parameters (Ct cut-off, Tm window, Tm Peak Threshold) was established through internal optimization and verification studies. These studies likely involved testing known positive and negative S. pyogenes samples (e.g., well-characterized strains, spiked samples) to define the optimal performance characteristics of the assay before clinical validation.

Ask a specific question about this device

The ARIES® C. difficile Assay is a real-time polymerase chain reaction (PCR) based qualitative in vitro diagnostic test for the direct detection of toxigenic Clostridium difficile (C. difficile) nucleic acid in unpreserved, unformed (liquid or soft) stool specimens obtained from patients suspected of having Clostridium difficile infection (CDI). The test targets the C. difficile toxin A gene (tcdA) and toxin B gene (tcdB) and is indicated for use as an aid in the diagnosis of C. difficile infection (CDI). The ARIES® C. difficile Assay is indicated for use with ARIES® Systems.

The ARIES® C. difficile Assay is a polymerase chain reaction (PCR)-based qualitative in vitro diagnostic test system which consists of the ARIES® System or the ARIES® M1 System with their included ARIES® Software, a stool resuspension kit, an assay-specific cassette, and an assay-specific protocol file. The ARIES® C. difficile Assay cassette is a disposable, single-use cassette containing nucleic acid purification reagents, internal sample process control (SPC), and an assay-specific master mix capable of performing the designated assay on one sample. The ARIES® C. difficile Assay cassette directly detects toxigenic Clostridium difficile (C. difficile) from unformed stool specimens obtained from patients suspected of having Clostridium difficile infection. Specifically, the ARIES® C. difficile Assay cassette detects the C. difficile toxin A gene (tcdA) and toxin B gene (tcdB) and a DNA Sample Processing Control. Unpreserved raw stool is processed using the ARIES® Stool Resuspension Kit. The ARIES® Stool Resuspension Kit includes a flocked swab, a tube containing preprocessing beads, and Stool Resuspension Buffer. The preprocessing method involves transfer of a swab of stool specimen into a tube containing preprocessing beads and Stool Resuspension Buffer. The swab is mixed in the tube by vortexing and then centrifuged. Finally, the preprocessed sample is added to the ARIES® C. difficile Assay cassette. The specimen is lysed and nucleic acid is extracted using an ARIES® instrument. An extractable sample processing control (SPC) target is also present in the ARIES® C. difficile Assay cassette and is processed with the specimen. The SPC controls for recovery of extracted nucleic acid, for inhibitory substances and for PCR reagent and instrument integrity. The Ct value of the SPC is designed to verify nucleic acid extraction, to identify PCR inhibition, if any, and verify proper function of the extraction system and real-time instrument. The Tm value of the SPC is used as a reference for determining the Tm of the tcdA and tcdB targets (if present). The extracted nucleic acid and SPC are transferred via magnetic beads through the cassette to the ARIES® C. difficile Assay lyophilized PCR reagents in the PCR tube that contain primer pairs specific to tcdA, tcdB, and the SPC sequence. Each of the primer pairs is labeled with a distinct fluorophore and detected in distinct channels of an ARIES® System. PCR amplification is performed and assay fluorescence is monitored. Incorporation of a quencher-labeled nucleotide results in a decrease in fluorescence for the associated primer pair. Following amplification, the reaction is slowly heated to separate the fluorescent-labeled strand from the quencher-labeled strand, a process that results in an increase in the fluorescence signal. The reaction fluorescence is measured during this process and the temperature at which the change in fluorescence is the maximum Tm of the amplicon. The strands of the amplicons will separate at a specific melting temperature (Tm) and an increase in fluorescence is observed. The instrument fluorescence output is analyzed and test results are determined using the ARIES® System software and the ARIES® C. difficile Assay protocol and run files. ARIES® C. difficile Assay results may be reported from the ARIES Software or from the optional SYNCT® Software.

Here's an analysis of the acceptance criteria and the study that proves the device meets them, based on the provided FDA 510(k) summary for the ARIES® C. difficile Assay:

1. Table of Acceptance Criteria and Reported Device Performance

The FDA 510(k) summary does not explicitly state pre-defined acceptance criteria values for clinical performance as a single table. Instead, it presents the assay's performance metrics against a reference method and implicitly indicates that these results were considered "acceptable" for substantial equivalence. Based on the clinical performance section, we can infer the key performance metrics reported.

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size:

  • Clinical Study: 984 unique specimens were included in the data analysis. After exclusions and re-tests, 979 eligible specimens were used for performance calculations.
  • Analytical Precision/Reproducibility:
    • Within-Laboratory: 252 samples (36 for each of 7 panel members).
    • Reproducibility (3 sites x 5 days x 6 panel members x 3 replicates): 270 samples (90 for each of 7 panel members, assuming negative also had 90 replicates).
  • Limit of Detection (LoD): Not specified precisely, but implied to be multiple replicates at various dilutions.
  • Analytical Reactivity: 15 distinct strains of toxigenic C. difficile (in addition to those in LoD).
  • Analytical Specificity (Cross-Reactivity/Interference): 61 microorganisms/viruses + human DNA, each tested with three replicates of two C. difficile strains (BAA-1870 and BAA-1871) and three negative replicates.
  • Carry-Over/Cross-Contamination: 30 high positive and 30 negative samples.

• Data Provenance:

  • Clinical Study: Prospective. The specimens were collected from 31-October-2016 to 21-February-2017 at four geographically distinct clinical sites within the United States. The specimens consisted of excess leftover de-identified, unpreserved, unformed stool specimens from patients suspected of having Clostridium difficile infection (CDI).

3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications of Experts

The document does not explicitly state the "number of experts" or their specific "qualifications" for establishing the ground truth using the reference method.

• Ground Truth Method: The reference method for the clinical study was "direct and enriched toxigenic culture." This is a laboratory-based method.
• Location of Ground Truth Establishment: "Reference method testing was performed at a centralized testing facility." It is implied that qualified laboratory personnel perform these culture tests, but specific number or qualifications are not provided.

4. Adjudication Method for the Test Set

The document does not explicitly describe an adjudication method (like 2+1, 3+1, none) for discordant results between the ARIES® C. difficile Assay and the reference method.

However, for discordant cases, further characterization was performed:

• For specimens negative by direct toxigenic culture but positive by ARIES® C. difficile Assay (False Positives based on direct culture), enriched toxigenic culture was used, and for some, bi-directional sequencing with analytically validated primers was performed.
• For specimens positive by direct toxigenic culture but negative by ARIES® C. difficile Assay (False Negatives based on direct culture), bi-directional sequencing with analytically validated primers was used.
• Similar follow-up testing (bi-directional sequencing) was done for discords when comparing to "Direct and Enriched Toxigenic Culture."

This suggests a form of further characterization for discordant results rather than an adjudication panel of experts re-reading primary data.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done

No, an MRMC comparative effectiveness study involving human readers with and without AI assistance was not done. This device is an in vitro diagnostic (IVD) assay, not an AI imaging or diagnostic algorithm designed to assist human readers. Its performance is evaluated stand-alone against a reference laboratory method.

6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was done

Yes, the clinical performance study evaluated the ARIES® C. difficile Assay as a standalone algorithm/device. The assay generates a qualitative result (POSITIVE, NEGATIVE, or INVALID) based on its internal processing and software analysis, without human interpretation of raw data for diagnosis. The ARIES® System software automatically determines and reports results.

7. The Type of Ground Truth Used

The primary ground truth used for the clinical performance study was direct and enriched toxigenic culture.

• Direct Toxigenic Culture: This involves culturing the C. difficile from the stool sample directly to identify toxigenic strains.
• Enriched Toxigenic Culture: This involves an enrichment step before culturing to increase the yield of C. difficile, potentially improving sensitivity.

For discordant results, further analysis using bi-directional sequencing with analytically validated primers was employed.

8. The Sample Size for the Training Set

The document does not specify a distinct training set sample size. For IVD submissions like this one, it's common that assay parameters (like Ct cut-offs and Tm windows) are established during "internal optimization studies" and "internal verification studies." The document explicitly states:

• "Initial Assay Protocol File parameters were set during internal optimization studies."
• "The final Assay Protocol File parameters were then established during internal verification studies."

The sizes of these internal optimization and verification studies are not provided but would have involved various analytical samples (LoD, inclusivity, exclusivity, etc.) rather than a dedicated "training set" of clinical samples in the same way an AI model might be trained. The clinical trial data (979 samples) served as a test set for validation, not for training the assay's parameters.

9. How the Ground Truth for the Training Set Was Established

Since no distinct "training set" of clinical samples is explicitly described for algorithm development in the AI sense, the ground truth establishment for the internal optimization and verification studies likely involved:

• Carefully characterized reference strains of C. difficile (toxigenic and non-toxigenic).
• Spiked stool matrixes with known concentrations of C. difficile to determine detection limits and reactivity.
• Potentially, a smaller set of pre-characterized clinical samples to refine initial assay parameters.

Again, these studies would rely on established microbiological and molecular biology techniques (culture, PCR, sequencing) to define the presence/absence and characteristics of C. difficile in the samples used for internal development.

Ask a specific question about this device

The ARIES® Bordetella Assay is a real-time polymerase chain reaction (PCR) based qualitative in vitro diagnostic test for the direct detection and identification of Bordetella pertussis (B. pertussis) and Bordetella parapertussis (B. parapertussis) nucleic acid in nasopharyngeal swab (NPS) specimens obtained from individuals suspected of having a respiratory tract infection attributable to B. pertussis or B. parapertussis.

The ARIES® Bordetella Assay targets the B. pertussis toxin promoter and the B. parapertussis IS1001 insertion element in the genomes. When clinical factors suggest that B. pertussis or B. parapertussis may not be the cause of respiratory infection, other clinically appropriate investigation(s) should be carried out in accordance with published guidelines.

Negative results for the ARIES® Bordetella Assay do not preclude B. pertussis infection and positive results do not rule out co-infections with other respiratory pathogens. The direction and identification of B. pertussis and B. parapertussis nucleic acids from symptomatic patients aids in the diagnosis of B. perfussis and B. parapertussis respiratory infection with other clinical findings and epidemiological information.

The ARIES® Bordetella Assay is indicated for use with the ARIES® Systems.

The ARIES® Bordetella Assay is a polymerase chain reaction (PCR)-based qualitative in vitro diagnostic test system that consists of the ARIES® System or the ARIES® M1 System with their included ARIES® Software, an assay-specific cassette, and an assay-specific protocol file. The ARIES® Bordetella Assay cassette is a disposable, single-use cassette containing nucleic acid purification reagents, internal sample process control (SPC), and an assay-specific master mix capable of performing the designated assay on one sample. The ARIES® Bordetella Assay cassette directly detects and identifies B. pertussis and B. parapertussis DNA from nasopharyngeal swab (NPS) specimens collected from the human nasopharynx region.

Nasopharyngeal swab specimens are collected from patients using a commercially available E-Swab™ (Nylon® Flocked Swab along with modified Liquid Amies) or a commercially available nasopharyngeal swab (i.e. rayon, flocked, nylon, plastic shaft, etc.) placed into an approved transport media (i.e UTM, M5, M6, or equivalent). The specimen is then transported to the laboratory for testing. The specimen is lysed and nucleic acid is extracted using an ARIES® System. An extractable sample processing control (SPC) target is present in the ARIES® Bordetella Assay cassette and is processed with the specimen. The SPC controls for specimen lysis, for recovery of extracted nucleic acid, for inhibitory substances and for PCR reagent and instrument integrity. The Ct value of the SPC is designed to verify proper specimen lysis and nucleic acid extraction, to identify PCR inhibition, if any, and verify proper function of the extraction system and real-time instrument. The Tm value of the SPC is used as a reference for determining the target Tm.

The extracted nucleic acid and SPC are transferred via magnetic beads through the cassette to the ARIES® Bordetella Assay lyophilized PCR reagents in the PCR tube that contains primer pairs specific to the B. pertussis toxin promoter (ptxA-pr), the B. parapertussis IS1001 insertion element, and the SPC sequence. Each of the primer pairs are labeled with a distinct fluorophore and detected in distinct channels of the ARIES® Systems. PCR amplification is performed and assay fluorescence is monitored. Incorporation of a quencher-labeled nucleotide results in a decrease in fluorescence for the associated primer pair. Following amplification, the reaction is slowly heated to separate the fluorescent-labeled strand from the quencher-labeled strand, a process that results in an increase in the fluorescence signal. The reaction fluorescence is measured during this process and the temperature at which the change in fluorescence is the maximum Tm of the amplicon. The strands of the amplicons will separate at a specific melting temperature (Tm) and an increase in fluorescence is observed. The instrument fluorescence output is analyzed and test results are determined using the ARIES® System software and the ARIES® Bordetella Assay protocol and run files. ARIES® Bordetella Assay results may be reported from the ARIES® Software or from the optional SYNCT® Software.

Here's an analysis of the acceptance criteria and study detailed in the provided document, structured as requested:

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria provided are predominantly for analytical performance (reproducibility) and general performance goals for clinical studies (e.g., 95% confidence intervals). Specific quantitative acceptance criteria for clinical performance (PPA, NPA) are not explicitly stated as distinct numerical targets in the text, but rather implied by the successful outcomes of the studies.

2. Sample Size Used for the Test Set and Data Provenance

• Test Set (Clinical Performance Study):

  • Prospective Cohort: 1052 unique nasopharyngeal swab (NPS) specimens.
    • Positive B. pertussis: 32 specimens
    • Positive B. parapertussis: 2 specimens
    • Negative for both: 1018 specimens (calculated from B. pertussis NPA of 1009/1020 and B. parapertussis NPA of 1048/1050, roughly consistent with the total size)
  • Supplemental Cohort (Banked/Pre-selected):
    • 37 B. pertussis positive specimens
    • 20 B. parapertussis positive specimens
    • Paired with an equal number of unique negative clinical specimens (not explicitly stated for each organism, but implied for blinded testing).
  • Supplemental Cohort (Contrived):
    • 50 B. parapertussis contrived specimens
  • Data Provenance:
    • Country of Origin: United States. Specimens were collected from five (5) geographically distinct clinical sites within the United States.
    • Retrospective/Prospective: The study combines both prospective and pre-selected (banked) specimens.
      • "Leftover de-identified nasopharyngeal swab (NPS) specimens prospectively collected from pediatric and adult patients suspected of having respiratory tract infection attributable to B. pertussis or B. parapertussis."
      • "Due to the low prevalence B. pertussis and B. parapertussis observed in the prospective study, the clinical sample set was supplemented with banked (pre-selected) B. pertussis (N=37) and B. parapertussis (N=20) positive specimens as well as contrived B. parapertussis specimens (N=50)."

• Test Set (Analytical Studies):

  • Reproducibility (Site): 30 replicates per condition (low positive, moderate positive, negative) per site (3 sites), across 5 days (e.g., 30/30 positive reported for each condition per site, meaning 30 tests performed per condition). Total = 90 tests per condition for overall result.
  • Reproducibility (Lot-to-Lot): 15 replicates per condition (low positive, moderate positive, negative) per lot (3 lots). Total = 45 tests per condition for overall result.
  • Within-Laboratory Precision: 30 replicates per condition (low positive, moderate positive, negative) (30/30 positive/negative reported).
  • Limit of Detection: 20 replicates per strain/concentration per organism type (e.g., 19/20 or 20/20 positive reported).
  • Inclusivity: 3 replicates per strain per organism (3/3 positive/negative reported).
  • Interfering Substances: 3 replicates per Bordetella type (with interferant) and 3 replicates of negative matrix (with interferant) for each of 19 substances.
  • Cross-Reactivity (Exclusivity): 3 replicates for each of 71 unique microorganisms.
  • Microbial Interference/Co-Infection: 3 replicates per target strain + CRO combination, plus specific re-testing for certain CROs.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts

The ground truth for the clinical test set was established using a "composite comparator assay consisting of two well-characterized real-time PCR assays (for each bacterial pathogen) followed by confirmation of positive PCR amplification product with bi-directional sequencing."

The document does not explicitly state the "number of experts" or their "qualifications" involved in establishing this ground truth. It refers to "well-characterized real-time PCR assays" and "bi-directional sequencing" performed at a "centralized testing facility," implying laboratory professionals skilled in molecular diagnostics. It does not mention clinical experts (e.g., radiologists, pathologists) directly establishing the ground truth for this in vitro diagnostic device.

4. Adjudication Method for the Test Set

The adjudication method for determining the composite comparator ground truth was:

• Positive: If one out of two comparator PCR assays was positive (Ct values ≤40) and confirmed by bi-directional sequencing, OR if both comparator PCR assays were positive.
• Negative: If one out of two comparator PCR assays was negative (Ct values >40) and confirmed by bi-directional sequencing, OR if both comparator PCR assays were negative.

This is a form of multimethod consensus rather than human expert adjudication.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs without AI Assistance

No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not done. This device is an in vitro diagnostic (IVD) test, specifically a real-time PCR assay. It does not involve human "readers" interpreting images or clinical data in the way AI in diagnostics typically does. Therefore, the concept of human readers improving with or without AI assistance is not applicable to this type of device.

6. If a Standalone (i.e. algorithm only without human-in-the loop performance) Was Done

Yes, the studies presented here are primarily standalone (algorithm only) performance studies. The ARIES® Bordetella Assay is an automated PCR-based system. Its "performance" refers to its analytical and clinical accuracy in detecting specific nucleic acid sequences directly, interpreted by the ARIES® System software. There is no human-in-the-loop performance component in the sense of a diagnostic interpretation loop that the device assists. Operators load samples, but the detection and interpretation are automated by the instrument and its software, making it a standalone device performance evaluation.

7. The Type of Ground Truth Used

The ground truth used for the clinical performance study was a composite comparator method. This method consisted of:

• Two independent, well-characterized real-time PCR assays (for each bacterial pathogen).
• Followed by confirmation of positive PCR amplification product with bi-directional sequencing.

In essence, it's a molecular gold standard combining highly sensitive and specific laboratory techniques.

8. The Sample Size for the Training Set

The document does not explicitly state a separate "training set" sample size for the ARIES® Bordetella Assay. For IVD submissions like this, the "training" aspect often refers to internal optimization and verification studies that determine assay parameters (like Ct cut-offs) before formal performance evaluation. The text mentions:

• "Initial Assay Protocol File parameters were set during internal optimization studies"
• "The final Assay Protocol File parameters were then established during internal verification studies"

These internal studies likely used various samples for development and refinement, but a distinct, quantified "training set" in the context of machine learning model development is not detailed or specified. The clinical trial data (prospective, pre-selected, contrived) is presented primarily as a test set for validation.

9. How the Ground Truth for the Training Set Was Established

Given that a specific, quantified "training set" is not detailed in the document in the context of a machine learning paradigm, the method for establishing its ground truth is also not explicitly described. However, if "training set" conceptually refers to the samples used during "internal optimization studies" and "internal verification studies" mentioned in section M.1.g, it can be inferred that these would have used:

• Well-characterized bacterial strains: Diluted into appropriate matrices (like native nasopharyngeal matrix) at known concentrations.
• Known negative samples: Such as culture media or confirmed pathogen-free clinical matrices.
• The ground truth for these samples would be based on known organism identity and concentration (e.g., quantified using standard culture techniques like CFU/mL, verified by plating and colony counting). This is how analytical studies (like LoD, inclusivity) established their ground truth.
• For any clinical samples used in early internal verification, the ground truth would likely have been established using similar reference methods (e.g., culture, reference PCR, sequencing) as those later used for the clinical test set.

Ask a specific question about this device

The ARIES GBS Assay, performed on ARIES Systems, is a real-time polymerase chain reaction (RT-PCR) based qualitative in vitro diagnostic test. The ARIES GBS Assay is designed to detect Group B Streptococcus (GBS) nucleic acid from 18-24 hour Lim broth enrichments of vaginal-rectal specimen swabs obtained from pregnant women.

The ARIES GBS Assay is intended for use as a method for detection in antepartum women. It is not intended to diagnose or monitor treatment of a GBS infection.

The ARIES GBS Assay does not provide susceptibility results. Culture isolates are needed for performing susceptibility testing as recommended for penicillin-allergic women.

Not Found

This is a letter for the ARIES GBS Assay from the FDA and does not contain the information required to answer the question, as it is a device approval letter and not a study.

Ask a specific question about this device

The ARIES® Flu A/B & RSV Assay is a polymerase chain reaction (PCR) based qualitative in vitro diagnostic test for the direct detection and differentiation of influenza A virus, influenza B virus, and respiratory syncytial virus (RSV) nucleic acid in nasopharyngeal swab (NPS) specimens from patients with signs and symptoms of respiratory tract infection in conjunction with clinical and laboratory findings. The test is intended for use as an aid in the differential diagnosis of Influenza A, Influenza B, and RSV in humans and is not intended to detect Influenza C.

Negative results do not preclude influenza virus or RSV infection and should not be used as the sole basis for diagnosis, treatment or other management decisions. Conversely, positive results do not rule-out bacterial infection or co-infection with other viruses. The agent detected may not be the definite cause of disease. The use of additional laboratory testing (e.g. bacterial culture, immunofluorescence, X-ray findings) and clinical presentation must be taken into consideration in order to obtain the final diagnosis of respiratory viral infection.

Performance characteristics for influenza A were established during the 2015-2016 influenza seasons when influenza A/H3N2 and A/H1N1 pandemic were the predominant influenza A viruses in circulation. When other Influenza A viruses are emerging, performance characteristics may vary.

If infection with a novel Influenza A virus is suspected based on current clinical and epidemiological screening criteria recommended by public health authorities, specimens should be collected with appropriate infection control precautions for novel virulent influenza viruses and sent to state or local health departments for testing. Viral culture should not be attempted in these cases unless a BSL 3+ facility is available to receive and culture specimens.

The ARIES® Flu A/B & RSV Assay is indicated for use with the ARIES® Systems.

The ARIES® Flu A/B & RSV Assay is a polymerase chain reaction (PCR)-based qualitative in vitro diagnostic test system that will consist of an ARIES® System with its included software, an assay-specific cassette, and an assay-specific protocol file. The ARIES® Flu A/B & RSV Assay cassette is a disposable, single-use cassette containing nucleic acid purification reagents, internal sample process control (SPC), and an assay-specific master mix capable of performing the designated assay on one sample. The ARIES® Flu A/B & RSV Assay cassette directly detects and differentiates influenza A virus, influenza B virus, and respiratory syncytial virus (RSV) nucleic acid in nasopharyngeal swabs (NPS) specimens from patients with signs and symptoms of respiratory tract infection in conjunction with clinical and laboratory findings. Specifically, the ARIES® Flu A/B & RSV Assay cassette detects the matrix protein genes of influenza A and influenza B viruses, and the fusion gene of RSV and a RNA Sample Processing Control.

The specimen is lysed and nucleic acid is extracted using an ARIES® System. An extractable sample processing control (SPC) target is present in the ARIES® Flu A/B & RSV assay cassette and is processed with the specimen. The SPC controls for specimen lysis, for recovery of extracted nucleic acid, for inhibitory substances and for PCR reagent and instrument integrity. The Ct value of the SPC is designed to verify proper specimen lysis and nucleic acid extraction, to identify PCR inhibition, if any, and verify proper function of the extraction system and real-time instrument. The Tm value of the SPC is used as a reference for determining the target Tm.

The extracted nucleic acid and SPC are transferred via magnetic beads through the cassette to the ARIES® Flu A/B & RSV Assay lyophilized PCR reagents in the PCR tube that contain primer pairs specific to influenza A, influenza B, RSV, and the SPC sequence. The specific primer pairs are labeled with distinct fluorophore labels. PCR amplification is performed and assay fluorescence is monitored on an ARIES® System. Incorporation of the quencherlabeled nucleotide causes a decrease in assay fluorescence. Following amplification, the reaction is slowly heated and fluorescence is monitored. The strands of the amplification products will separate at a specific melting temperature (Tm) that is determined by an increase in fluorescence as the strands are separated. The instrument fluorescence output is analyzed and test results are determined using the ARIES® Flu A/B & RSV Assay protocol file. A printed results report is generated.

Here's a breakdown of the acceptance criteria and study information for the ARIES® Flu A/B & RSV Assay, based on the provided document:

1. Table of Acceptance Criteria and Reported Device Performance

Clinical Performance Acceptance Criteria:

Reported Device Performance (Clinical Study):

Additional Influenza B Performance (Supplemented Study):

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size for Prospective Clinical Study: 2479 eligible unique nasopharyngeal swab (NPS) specimens.
• Sample Size for Supplemented Influenza B Study: 40 pre-selected Influenza B positive specimens + 40 unique negative clinical specimens (total 80).
• Data Provenance: Retrospective and prospective.
  • Prospective Data: Specimens collected during the 2014/2015 (Jan 18, 2015 to Mar 20, 2015) and 2015/2016 (Nov 02, 2015 to Feb 29, 2016) flu seasons.
  • Geographical Origin: Collected at 3 clinical sites (2014/2015) and 4 clinical sites (2015/2016) in the United States and Canada.
  • Retrospective/Pre-selected Data: 40 banked (pre-selected) Influenza B positive specimens collected at a single clinical laboratory (collection site) in Canada for the supplemental study.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

The document does not explicitly state the number or qualifications of "experts" used to establish ground truth in the traditional sense (e.g., radiologists interpreting images).

Instead, the ground truth for the clinical study was established using:

• An FDA-cleared molecular comparator assay (primary reference standard).
• Bi-directional sequencing for discordant results.

This approach means the "ground truth" relies on the performance of another validated diagnostic device and a molecular sequencing technique, rather than human expert consensus on clinical presentation or imaging.

4. Adjudication Method for the Test Set

• Primary Comparison: All 2479 eligible clinical specimens were initially tested by an FDA-cleared molecular comparator and the ARIES® Flu A/B & RSV Assay.
• Discordant Analysis: For specimens where the ARIES® Flu A/B & RSV Assay results differed from the comparator assay result, they were further assessed by bi-directional sequencing using analytically validated primers that targeted genomic regions distinct from the ARIES® Flu A/B & RSV Assay.
• Reporting: Results from discordant testing analysis were not included in the initial calculation of Positive Percent Agreement and Negative Percent Agreement for each target. These discordant results were noted as footnotes in the performance evaluation tables (e.g., "False Negative by bi-directional sequencing").

This indicates a form of secondary adjudication using an independent, highly specific molecular method to resolve discrepancies between the new device and the predicate.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done

No, this was not a Multi-Reader Multi-Case (MRMC) comparative effectiveness study involving human readers. This study evaluated the standalone performance of a molecular diagnostic assay against a comparator assay and sequencing, not the improvement of human readers with AI assistance.

6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was done

Yes, a standalone performance study was conducted. The ARIES® Flu A/B & RSV Assay is an automated molecular diagnostic test system. The clinical performance evaluation directly tested the device's output against a molecular comparator and sequencing, without human interpretation of the assay's raw output determining the final diagnostic call. The assay itself provides a qualitative "Positive" or "Negative" result.

7. The Type of Ground Truth Used

The ground truth for the test set was an combination of:

• Molecular Comparator Assay Results: An FDA-cleared molecular test used as the primary reference standard.
• Bi-directional Sequencing: Used as a confirmatory "tie-breaker" or definitive reference for specimens yielding discordant results between the ARIES® assay and the primary comparator.

8. The Sample Size for the Training Set

The document does not directly specify a "training set" sample size in the context of machine learning model development. For in vitro diagnostic devices like the ARIES® Flu A/B & RSV Assay, "training" typically refers to the process of developing and optimizing the assay's chemical reagents, amplification protocols, and the algorithms for interpreting raw signal (e.g., Ct values, Tm values) into a qualitative result. This development process uses numerous laboratory experiments with contrived samples and potentially some early clinical samples, but these are generally not referred to as a "training set" in the same way as in AI/ML literature.

The document describes extensive analytical performance studies (e.g., Limit of Detection, Inclusivity, Specificity, Precision, Stability) that are crucial for defining the assay's operating characteristics and establishing the cut-offs and parameters that lead to accurate results. These analytical studies utilize:

• Precision/Reproducibility: 675 replicates (initial precision) and 1260 replicates (site-to-site reproducibility) using contrived samples (viral cultures at specific concentrations diluted in simulated nasal matrix).
• Limit of Detection: At least 20 replicates for each of 7 viral strains.
• Inclusivity: Triplicate testing for each of 34 different viral strains (some required additional testing).
• Interfering Substances: Triplicate testing for viral targets with 18 different interfering substances.
• Cross-Reactivity: Triplicate testing for 32 microorganisms (some required additional testing).
• Microbial Interference/Co-infection: Triplicate testing for various pathogen combinations.

These extensive studies with known concentrations and types of pathogens and interferents essentially "train" and validate the design and interpretation rules of the assay.

9. How the Ground Truth for the Training Set was Established

Again, the concept of a "training set" with ground truth in the AI/ML sense is not explicitly detailed. However, for the analytical studies (which inform the assay's design and operating parameters), the "ground truth" was established based on:

• Known Concentrations of Characterized Viral Stocks/Strains: Viral cultures were quantified and diluted to precise concentrations (e.g., TCID50/mL) to create samples with a known positive status and concentration.
• Known Negative Samples: Simulated nasal matrix (SNM) or Universal Transport Medium (UTM) were used as known negative samples.
• Known Interfering/Cross-Reactive Agents: Specific microorganisms or substances were added at defined concentrations to evaluate their impact.

Essentially, the ground truth for these analytical studies was derived from laboratory-controlled conditions where the presence, identity, and concentration of analytes were precisely known.

Ask a specific question about this device