(227 days)
The Quidel Molecular RSV + hMPV assay is a multiplex Real Time RT-PCR assay for the in vitro qualitative detection and identification of respiratory syncytial virus and human metapneumovirus viral RNA extracted from nasal and nasopharyngeal swabs specimens from patients with signs and symptoms of respiratory infection. This in vitro diagnostic test is intended to aid in the differential diagnosis of respiratory syncytial virus and human metapneumovirus infections. This test is not intended to differentiate the four genetic sub-lineages of hMPV.
Negative results do not preclude RSV or hMPV infection and should not be used as the sole basis for diagnosis, treatment or other patient management decisions.
The Quidel Molecular RSV + hMPV Assay detects viral nucleic acids that have been extracted from a patient sample using the NucliSENS® easyMAG® automated extraction platform. A multiplex RT-PCR reaction is carried out under optimized conditions in a single tube generating amplicons for each of the target viruses present in the sample. This reaction is performed utilizing either the Cepheid SmartCycler® II or the Applied Biosystems 7500 Fast DX. Identification of RSV and hMPV and the PRC occurs by the use of target-specific primers and fluorescent-labeled probes that hybridize to conserved regions in the genomes of RSV and hMPV and the PRC.
Here's a summary of the acceptance criteria and study details for the Quidel Molecular RSV + hMPV Assay, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria provided in the document are primarily for analytical performance (LoD, Reproducibility, Inclusivity, Specificity) and clinical performance (Sensitivity and Specificity/Positive and Negative Percent Agreement). The clinical performance is reported compared to predicate devices or established methods.
Device: Quidel Molecular RSV + hMPV Assay
| Performance Measure | Acceptance Criteria (Implicit from study results) | Reported Device Performance (Cepheid SmartCycler II) | Reported Device Performance (Applied Biosystems 7500 Fast DX) |
|---|---|---|---|
| Analytical Performance | |||
| Limit of Detection (LoD) | Defined as the lowest concentration at which 95% of replicates tested positive. | Ranges from 1.89E+00 TCID50/mL (RSV A) to 2.645E+01 TCID50/mL (hMPV-A1) | Ranges from 6.29E-01 TCID50/mL (RSV A) to 1.7E+01 TCID50/mL (hMPV-A1) |
| Reproducibility | High concordance for positive and negative controls/high and medium positives; acceptable %CV for Ct values. | RSV Low Positive 2x LoD: 89/89 (99-100%) | |
| RSV Med Positive 5x LoD: 90/90 (100%) | |||
| hMPV Low Positive 2x LoD: 90/90 (100%) | |||
| hMPV Med Positive 5x LoD: 90/90 (100%) | |||
| Negative Controls: 0/90 (0%) positive | RSV Low Positive 2x LoD: 90/90 (100%) | ||
| RSV Med Positive 5x LoD: 90/90 (100%) | |||
| hMPV Low Positive 2x LoD: 87/90 (96.7%) | |||
| hMPV Med Positive 5x LoD: 89/90 (98.9%) | |||
| Negative Controls: 0/90 (0%) positive | |||
| Analytical Reactivity (Inclusivity) | All tested strains of RSV and hMPV should be detected as positive. | All 13 RSV strains and 12 hMPV strains tested were Positive. | All 13 RSV strains and 12 hMPV strains tested were Positive. |
| Analytical Specificity (Cross-Reactivity) | No false positives with common respiratory pathogens or flora. | 100% specificity against 27 viruses, 24 bacteria, and 1 yeast strain. | 100% specificity against 27 viruses, 24 bacteria, and 1 yeast strain. |
| Clinical Performance (RSV - vs. DSFA & Cell Culture w/DFA) | Good sensitivity and specificity (implicitly high values) | Sensitivity: 97.9% (95% CI: 93.9% - 99.3%) | |
| Specificity: 97.6% (95% CI: 96.3% - 98.4%) | Sensitivity: 98.6% (95% CI: 94.9% - 99.6%) | ||
| Specificity: 96.8% (95% CI: 95.4% - 97.8%) | |||
| Clinical Performance (hMPV - vs. Pro hMPV+) | Good positive and negative percent agreement (implicitly high values) | Positive percent agreement: 96.7% (95% CI: 92.4% - 98.6%) | |
| Negative percent agreement: 99.6% (95% CI: 98.9% - 99.9%) | Positive percent agreement: 98.0% (95% CI: 94.3% - 99.3%) | ||
| Negative percent agreement: 99.3% (95% CI: 98.4% - 99.7%) |
2. Sample Size Used for the Test Set and Data Provenance
- Clinical Performance Test Set Samples:
- Total samples collected: 1014 specimens (414 fresh, 600 frozen) for RSV comparison.
- RSV testing (SmartCycler II): 1009 specimens after excluding contaminated cell cultures.
- hMPV testing (SmartCycler II): 951 specimens after excluding invalid comparative device results.
- RSV testing (7500 Fast Dx): 1007 specimens after excluding contaminated cell cultures and invalid subject method results.
- hMPV testing (7500 Fast Dx): 946 specimens after excluding invalid comparative and subject method results.
- Data Provenance: The samples were collected prospectively during the 2012 respiratory virus season (January to March 2012) from symptomatic patients at four sites across the United States. The study specifically used "fresh (414) and frozen (600) swab specimens."
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The document does not explicitly state the number or qualifications of experts used. However, the ground truth for RSV was established using "DSFA & Cell Culture w/DFA" (Direct Specimen Fluorescent Antibody and Cell Culture with DFA), which implies interpretation by trained laboratory personnel or specialists, although their specific qualifications or number are not detailed. For hMPV, the ground truth was established by comparing it to the "FDA Cleared hMPV molecular test" (Gen-Probe Prodesse Pro hMPV+, K082688), which is a molecular diagnostic method rather than expert interpretation of raw data.
4. Adjudication Method for the Test Set
- RSV Discrepant Results:
- For the SmartCycler II, all 21 originally discordant specimens (QM RSV + hMPV positive, reference method negative) were positive for RSV by an FDA-cleared RT-PCR assay and by bi-directional sequence analysis.
- For the 7500 Fast Dx, 25 of 28 originally discordant specimens were positive by an FDA-cleared RT-PCR assay, and 27 of 28 were positive by bi-directional sequence analysis.
- hMPV Discrepant Results:
- For both the SmartCycler II and the 7500 Fast Dx, all originally discordant specimens (QM RSV + hMPV positive, reference method negative) were positive for hMPV by bi-directional sequence analysis.
This indicates a form of post-hoc adjudication or discrepancy resolution using additional, more definitive molecular methods (FDA-cleared RT-PCR and bi-directional sequencing) for cases where the subject device and the initial reference method disagreed.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs. Without AI Assistance
No, this was not an MRMC study. The device is an in vitro diagnostic (molecular assay) for direct detection of viral RNA, not an imaging device requiring human reader interpretation or AI assistance in interpretation. Therefore, a multi-reader multi-case comparative effectiveness study on human reader improvement with or without AI assistance is not applicable here.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
Yes, the clinical performance study evaluates the standalone performance of the Quidel Molecular RSV + hMPV Assay. The results are presented as the device's agreement (sensitivity, specificity, positive/negative percent agreement) compared directly to the reference methods, without human interpretation of the assay results impacting the reported performance metrics. The assay itself provides a qualitative (positive/negative) result based on its internal thresholding (e.g., fluorescence achieved by 50 cycles on SmartCycler II or 35 cycles on ABI 7500 Fast Dx).
7. The Type of Ground Truth Used
- For RSV: The ground truth for clinical performance was established using Direct Specimen Fluorescent Antibody (DSFA) and Cell Culture with DFA.
- For hMPV: The ground truth for clinical performance was established using an FDA Cleared hMPV molecular test (Gen-Probe Prodesse Pro hMPV+).
- For discrepant results: Bi-directional sequence analysis and/or another FDA-cleared RT-PCR assay were used as definitive ground truth.
8. The Sample Size for the Training Set
The document does not explicitly state a sample size for a "training set" in the context of machine learning or algorithm development. For this type of molecular diagnostic assay, analytical studies (LoD, inclusivity, specificity) and clinical validation are performed. The LoD study involved replicates of serially diluted viral cultures, and inclusivity/specificity studies used panels of various strains/organisms. These analytical studies are analogous to "training" or "development" data in that they inform and validate the assay's operational parameters, but they are not framed as a classic machine learning training set.
9. How the Ground Truth for the Training Set Was Established
Given that this is a molecular diagnostic assay, the "ground truth" for establishing analytical parameters (like LoD, inclusivity, and specificity) is based on:
- Quantified viral cultures (TCID50/mL): Used for LoD studies, where the exact concentration of virus is known.
- Known viral strains or bacterial/yeast cultures: Used for inclusivity (known to contain the target virus) and specificity (known to contain other organisms to test for cross-reactivity) studies. The identity and concentration of these cultures are established through standard microbiological and virological techniques.
§ 866.3980 Respiratory viral panel multiplex nucleic acid assay.
(a)
Identification. A respiratory viral panel multiplex nucleic acid assay is a qualitative in vitro diagnostic device intended to simultaneously detect and identify multiple viral nucleic acids extracted from human respiratory specimens or viral culture. The detection and identification of a specific viral nucleic acid from individuals exhibiting signs and symptoms of respiratory infection aids in the diagnosis of respiratory viral infection when used in conjunction with other clinical and laboratory findings. The device is intended for detection and identification of a combination of the following viruses:(1) Influenza A and Influenza B;
(2) Influenza A subtype H1 and Influenza A subtype H3;
(3) Respiratory Syncytial Virus subtype A and Respiratory Syncytial Virus subtype B;
(4) Parainfluenza 1, Parainfluenza 2, and Parainfluenza 3 virus;
(5) Human Metapneumovirus;
(6) Rhinovirus; and
(7) Adenovirus.
(b)
Classification. Class II (special controls). The special controls are:(1) FDA's guidance document entitled “Class II Special Controls Guidance Document: Respiratory Viral Panel Multiplex Nucleic Acid Assay;”
(2) For a device that detects and identifies Human Metapneumovirus, FDA's guidance document entitled “Class II Special Controls Guidance Document: Testing for Human Metapneumovirus (hMPV) Using Nucleic Acid Assays;” and
(3) For a device that detects and differentiates Influenza A subtype H1 and subtype H3, FDA's guidance document entitled “Class II Special Controls Guidance Document: Testing for Detection and Differentiation of Influenza A Virus Subtypes Using Multiplex Nucleic Acid Assays.” See § 866.1(e) for the availability of these guidance documents.