(131 days)
The ARIES® Bordetella Assay is a real-time polymerase chain reaction (PCR) based qualitative in vitro diagnostic test for the direct detection and identification of Bordetella pertussis (B. pertussis) and Bordetella parapertussis (B. parapertussis) nucleic acid in nasopharyngeal swab (NPS) specimens obtained from individuals suspected of having a respiratory tract infection attributable to B. pertussis or B. parapertussis.
The ARIES® Bordetella Assay targets the B. pertussis toxin promoter and the B. parapertussis IS1001 insertion element in the genomes. When clinical factors suggest that B. pertussis or B. parapertussis may not be the cause of respiratory infection, other clinically appropriate investigation(s) should be carried out in accordance with published guidelines.
Negative results for the ARIES® Bordetella Assay do not preclude B. pertussis infection and positive results do not rule out co-infections with other respiratory pathogens. The direction and identification of B. pertussis and B. parapertussis nucleic acids from symptomatic patients aids in the diagnosis of B. perfussis and B. parapertussis respiratory infection with other clinical findings and epidemiological information.
The ARIES® Bordetella Assay is indicated for use with the ARIES® Systems.
The ARIES® Bordetella Assay is a polymerase chain reaction (PCR)-based qualitative in vitro diagnostic test system that consists of the ARIES® System or the ARIES® M1 System with their included ARIES® Software, an assay-specific cassette, and an assay-specific protocol file. The ARIES® Bordetella Assay cassette is a disposable, single-use cassette containing nucleic acid purification reagents, internal sample process control (SPC), and an assay-specific master mix capable of performing the designated assay on one sample. The ARIES® Bordetella Assay cassette directly detects and identifies B. pertussis and B. parapertussis DNA from nasopharyngeal swab (NPS) specimens collected from the human nasopharynx region.
Nasopharyngeal swab specimens are collected from patients using a commercially available E-Swab™ (Nylon® Flocked Swab along with modified Liquid Amies) or a commercially available nasopharyngeal swab (i.e. rayon, flocked, nylon, plastic shaft, etc.) placed into an approved transport media (i.e UTM, M5, M6, or equivalent). The specimen is then transported to the laboratory for testing. The specimen is lysed and nucleic acid is extracted using an ARIES® System. An extractable sample processing control (SPC) target is present in the ARIES® Bordetella Assay cassette and is processed with the specimen. The SPC controls for specimen lysis, for recovery of extracted nucleic acid, for inhibitory substances and for PCR reagent and instrument integrity. The Ct value of the SPC is designed to verify proper specimen lysis and nucleic acid extraction, to identify PCR inhibition, if any, and verify proper function of the extraction system and real-time instrument. The Tm value of the SPC is used as a reference for determining the target Tm.
The extracted nucleic acid and SPC are transferred via magnetic beads through the cassette to the ARIES® Bordetella Assay lyophilized PCR reagents in the PCR tube that contains primer pairs specific to the B. pertussis toxin promoter (ptxA-pr), the B. parapertussis IS1001 insertion element, and the SPC sequence. Each of the primer pairs are labeled with a distinct fluorophore and detected in distinct channels of the ARIES® Systems. PCR amplification is performed and assay fluorescence is monitored. Incorporation of a quencher-labeled nucleotide results in a decrease in fluorescence for the associated primer pair. Following amplification, the reaction is slowly heated to separate the fluorescent-labeled strand from the quencher-labeled strand, a process that results in an increase in the fluorescence signal. The reaction fluorescence is measured during this process and the temperature at which the change in fluorescence is the maximum Tm of the amplicon. The strands of the amplicons will separate at a specific melting temperature (Tm) and an increase in fluorescence is observed. The instrument fluorescence output is analyzed and test results are determined using the ARIES® System software and the ARIES® Bordetella Assay protocol and run files. ARIES® Bordetella Assay results may be reported from the ARIES® Software or from the optional SYNCT® Software.
Here's an analysis of the acceptance criteria and study detailed in the provided document, structured as requested:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria provided are predominantly for analytical performance (reproducibility) and general performance goals for clinical studies (e.g., 95% confidence intervals). Specific quantitative acceptance criteria for clinical performance (PPA, NPA) are not explicitly stated as distinct numerical targets in the text, but rather implied by the successful outcomes of the studies.
| Acceptance Criteria Category | Specific Criteria (from document) | Reported Device Performance (from document) |
|---|---|---|
| Reproducibility (Site) | 100% positive for moderate positive samples, ≥ 95% positive for low positive samples, and 100% negative for negative samples. | B. pertussis (Low Positive): Site 1: 100%, Site 2: 100%, Site 3: 100%; Overall: 100% (90/90) with 95% CI (96.0% - 100.0%) |
| B. pertussis (Moderate Positive): Site 1: 100%, Site 2: 100%, Site 3: 100%; Overall: 100% (90/90) with 95% CI (96.0% - 100.0%) | ||
| B. parapertussis (Low Positive): Site 1: 100%, Site 2: 100%, Site 3: 100%; Overall: 100% (90/90) with 95% CI (96.0% - 100.0%) | ||
| B. parapertussis (Moderate Positive): Site 1: 100%, Site 2: 100%, Site 3: 100%; Overall: 100% (90/90) with 95% CI (96.0% - 100.0%) | ||
| Negative: Site 1: 0%, Site 2: 0%, Site 3: 0%; Overall: 0% (0/90) with 95% CI (0.0% - 4.0%) | ||
| Reproducibility (Lot-to-Lot) | 100% positive for moderate positive samples, ≥ 95% positive for low positive samples, and 100% negative for negative samples. | B. pertussis (Low Positive): 100% (45/45) |
| B. pertussis (Moderate Positive): 100% (45/45) | ||
| B. parapertussis (Low Positive): 100% (45/45) | ||
| B. parapertussis (Moderate Positive): 100% (45/45) | ||
| Negative: 0% (0/45) | ||
| Precision (Within-Lab) | Expected Positivity: Approx 95% for Low Positive, 100% for Moderate Positive, 0% for Negative. | B. pertussis (Low Positive): 100% (30/30) with 95% CI (88.4% - 100.0%) |
| B. pertussis (Moderate Positive): 100% (30/30) with 95% CI (88.4% - 100.0%) | ||
| B. parapertussis (Low Positive): 100% (30/30) with 95% CI (88.4% - 100.0%) | ||
| B. parapertussis (Moderate Positive): 100% (30/30) with 95% CI (88.4% - 100.0%) | ||
| Negative: 0% (0/30) with 95% CI (0.0% - 11.6%) | ||
| Limit of Detection (LoD) | Positivity rate of ≥ 95% at the lowest concentration. | B. pertussis (A639): 95% (19/20) at 1,640 CFU/mL |
| B. pertussis (BAA-589): 95% (19/20) at 1,800 CFU/mL | ||
| B. parapertussis (A747): 100% (20/20) at 172 CFU/mL | ||
| B. parapertussis (BAA-587): 95% (19/20) at 213 CFU/mL | ||
| Overall LoD: B. pertussis 1,800 CFU/mL, B. parapertussis 213 CFU/mL | ||
| Inclusivity | 100% positivity at 3x LoD for various Bordetella strains. | Seven B. parapertussis strains: 100% positivity at 3x LoD. |
| Nine of eleven B. pertussis strains: 100% positivity at 3x LoD. | ||
| Two B. pertussis strains (ATCC 8478, ATCC 9797) not detected at 3x, 10x, or 100x LoD due to primer mismatches (low prevalence, old strains). | ||
| Stability (Cassette Shelf-Life) | 100% positivity for all Bordetella replicates and 100% negativity for all negative replicates at each time point and temperature. | Data collected up to 7 months gave expected results (100% positivity for controls, 100% negativity). Studies ongoing. |
| Carry-Over / Cross-Contamination | No carry-over or cross-contamination observed. | No carry-over or cross-contamination observed. |
| Clinical Performance (Prospective) | Implied to demonstrate acceptable diagnostic accuracy. | B. pertussis: PPA 93.8% (30/32), 95% CI (79.2% - 99.2%); NPA 98.9% (1009/1020), 95% CI (98.1% - 99.5%) |
| B. parapertussis: PPA 100% (2/2), 95% CI (15.8% - 100%); NPA 99.8% (1048/1050), 95% CI (99.3% - 100%) | ||
| Clinical Performance (Overall Combined) | Implied to demonstrate acceptable diagnostic accuracy. | B. pertussis: PPA 97.1% (67/69), 95% CI (89.9% - 99.6%); NPA 99.0% (1086/1097), 95% CI (98.2% - 99.5%) |
| B. parapertussis: PPA 100% (72/72), 95% CI (95.0% - 100%); NPA 99.7% (1191/1194), 95% CI (99.3% - 99.9%) |
2. Sample Size Used for the Test Set and Data Provenance
-
Test Set (Clinical Performance Study):
- Prospective Cohort: 1052 unique nasopharyngeal swab (NPS) specimens.
- Positive B. pertussis: 32 specimens
- Positive B. parapertussis: 2 specimens
- Negative for both: 1018 specimens (calculated from B. pertussis NPA of 1009/1020 and B. parapertussis NPA of 1048/1050, roughly consistent with the total size)
- Supplemental Cohort (Banked/Pre-selected):
- 37 B. pertussis positive specimens
- 20 B. parapertussis positive specimens
- Paired with an equal number of unique negative clinical specimens (not explicitly stated for each organism, but implied for blinded testing).
- Supplemental Cohort (Contrived):
- 50 B. parapertussis contrived specimens
- Data Provenance:
- Country of Origin: United States. Specimens were collected from five (5) geographically distinct clinical sites within the United States.
- Retrospective/Prospective: The study combines both prospective and pre-selected (banked) specimens.
- "Leftover de-identified nasopharyngeal swab (NPS) specimens prospectively collected from pediatric and adult patients suspected of having respiratory tract infection attributable to B. pertussis or B. parapertussis."
- "Due to the low prevalence B. pertussis and B. parapertussis observed in the prospective study, the clinical sample set was supplemented with banked (pre-selected) B. pertussis (N=37) and B. parapertussis (N=20) positive specimens as well as contrived B. parapertussis specimens (N=50)."
- Prospective Cohort: 1052 unique nasopharyngeal swab (NPS) specimens.
-
Test Set (Analytical Studies):
- Reproducibility (Site): 30 replicates per condition (low positive, moderate positive, negative) per site (3 sites), across 5 days (e.g., 30/30 positive reported for each condition per site, meaning 30 tests performed per condition). Total = 90 tests per condition for overall result.
- Reproducibility (Lot-to-Lot): 15 replicates per condition (low positive, moderate positive, negative) per lot (3 lots). Total = 45 tests per condition for overall result.
- Within-Laboratory Precision: 30 replicates per condition (low positive, moderate positive, negative) (30/30 positive/negative reported).
- Limit of Detection: 20 replicates per strain/concentration per organism type (e.g., 19/20 or 20/20 positive reported).
- Inclusivity: 3 replicates per strain per organism (3/3 positive/negative reported).
- Interfering Substances: 3 replicates per Bordetella type (with interferant) and 3 replicates of negative matrix (with interferant) for each of 19 substances.
- Cross-Reactivity (Exclusivity): 3 replicates for each of 71 unique microorganisms.
- Microbial Interference/Co-Infection: 3 replicates per target strain + CRO combination, plus specific re-testing for certain CROs.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The ground truth for the clinical test set was established using a "composite comparator assay consisting of two well-characterized real-time PCR assays (for each bacterial pathogen) followed by confirmation of positive PCR amplification product with bi-directional sequencing."
The document does not explicitly state the "number of experts" or their "qualifications" involved in establishing this ground truth. It refers to "well-characterized real-time PCR assays" and "bi-directional sequencing" performed at a "centralized testing facility," implying laboratory professionals skilled in molecular diagnostics. It does not mention clinical experts (e.g., radiologists, pathologists) directly establishing the ground truth for this in vitro diagnostic device.
4. Adjudication Method for the Test Set
The adjudication method for determining the composite comparator ground truth was:
- Positive: If one out of two comparator PCR assays was positive (Ct values ≤40) and confirmed by bi-directional sequencing, OR if both comparator PCR assays were positive.
- Negative: If one out of two comparator PCR assays was negative (Ct values >40) and confirmed by bi-directional sequencing, OR if both comparator PCR assays were negative.
This is a form of multimethod consensus rather than human expert adjudication.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs without AI Assistance
No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not done. This device is an in vitro diagnostic (IVD) test, specifically a real-time PCR assay. It does not involve human "readers" interpreting images or clinical data in the way AI in diagnostics typically does. Therefore, the concept of human readers improving with or without AI assistance is not applicable to this type of device.
6. If a Standalone (i.e. algorithm only without human-in-the loop performance) Was Done
Yes, the studies presented here are primarily standalone (algorithm only) performance studies. The ARIES® Bordetella Assay is an automated PCR-based system. Its "performance" refers to its analytical and clinical accuracy in detecting specific nucleic acid sequences directly, interpreted by the ARIES® System software. There is no human-in-the-loop performance component in the sense of a diagnostic interpretation loop that the device assists. Operators load samples, but the detection and interpretation are automated by the instrument and its software, making it a standalone device performance evaluation.
7. The Type of Ground Truth Used
The ground truth used for the clinical performance study was a composite comparator method. This method consisted of:
- Two independent, well-characterized real-time PCR assays (for each bacterial pathogen).
- Followed by confirmation of positive PCR amplification product with bi-directional sequencing.
In essence, it's a molecular gold standard combining highly sensitive and specific laboratory techniques.
8. The Sample Size for the Training Set
The document does not explicitly state a separate "training set" sample size for the ARIES® Bordetella Assay. For IVD submissions like this, the "training" aspect often refers to internal optimization and verification studies that determine assay parameters (like Ct cut-offs) before formal performance evaluation. The text mentions:
- "Initial Assay Protocol File parameters were set during internal optimization studies"
- "The final Assay Protocol File parameters were then established during internal verification studies"
These internal studies likely used various samples for development and refinement, but a distinct, quantified "training set" in the context of machine learning model development is not detailed or specified. The clinical trial data (prospective, pre-selected, contrived) is presented primarily as a test set for validation.
9. How the Ground Truth for the Training Set Was Established
Given that a specific, quantified "training set" is not detailed in the document in the context of a machine learning paradigm, the method for establishing its ground truth is also not explicitly described. However, if "training set" conceptually refers to the samples used during "internal optimization studies" and "internal verification studies" mentioned in section M.1.g, it can be inferred that these would have used:
- Well-characterized bacterial strains: Diluted into appropriate matrices (like native nasopharyngeal matrix) at known concentrations.
- Known negative samples: Such as culture media or confirmed pathogen-free clinical matrices.
- The ground truth for these samples would be based on known organism identity and concentration (e.g., quantified using standard culture techniques like CFU/mL, verified by plating and colony counting). This is how analytical studies (like LoD, inclusivity) established their ground truth.
- For any clinical samples used in early internal verification, the ground truth would likely have been established using similar reference methods (e.g., culture, reference PCR, sequencing) as those later used for the clinical test set.
§ 866.3980 Respiratory viral panel multiplex nucleic acid assay.
(a)
Identification. A respiratory viral panel multiplex nucleic acid assay is a qualitative in vitro diagnostic device intended to simultaneously detect and identify multiple viral nucleic acids extracted from human respiratory specimens or viral culture. The detection and identification of a specific viral nucleic acid from individuals exhibiting signs and symptoms of respiratory infection aids in the diagnosis of respiratory viral infection when used in conjunction with other clinical and laboratory findings. The device is intended for detection and identification of a combination of the following viruses:(1) Influenza A and Influenza B;
(2) Influenza A subtype H1 and Influenza A subtype H3;
(3) Respiratory Syncytial Virus subtype A and Respiratory Syncytial Virus subtype B;
(4) Parainfluenza 1, Parainfluenza 2, and Parainfluenza 3 virus;
(5) Human Metapneumovirus;
(6) Rhinovirus; and
(7) Adenovirus.
(b)
Classification. Class II (special controls). The special controls are:(1) FDA's guidance document entitled “Class II Special Controls Guidance Document: Respiratory Viral Panel Multiplex Nucleic Acid Assay;”
(2) For a device that detects and identifies Human Metapneumovirus, FDA's guidance document entitled “Class II Special Controls Guidance Document: Testing for Human Metapneumovirus (hMPV) Using Nucleic Acid Assays;” and
(3) For a device that detects and differentiates Influenza A subtype H1 and subtype H3, FDA's guidance document entitled “Class II Special Controls Guidance Document: Testing for Detection and Differentiation of Influenza A Virus Subtypes Using Multiplex Nucleic Acid Assays.” See § 866.1(e) for the availability of these guidance documents.