(28 days)
The Influenza B Lineage Genotyping Kit contains reagents and controls of the CDC Human Influenza Virus Real-Time RT-PCR Diagnostic Panel and is intended for use in real-time RT-PCR (rRT-PCR) assays on an Applied Biosystems (ABI) 7500 Fast Dx Real-Time PCR instrument in conjunction with clinical and epidemiological information:
For the determination of the genetic lineage of human influenza B viruses as B/Victoria or B/Yamagata lineage from · viral RNA in upper respiratory tract clinical specimens (including nasopharyngeal swabs [NS], throat swabs [TS], nasal aspirates [NA], nasal washes [NW] and dual nasopharyngeal/throat swabs [NPS/TS]) from human patients with signs and symptoms of respiratory infection and/or from viral culture;
· To provide epidemiologic information for surveillance of circulating influenza viruses.
Performance characteristics for influenza B lineage genotyping were established during a season when influenza B/ Victoria and B/Yamagata lineages were in circulation.
Negative results do not preclude influenza virus infection and should not be used as the sole basis for treatment or other patient management decisions. Conversely, positive results do not rule out bacterial infection with other viruses. The agent detected may not be the definite cause of disease.
All users, and any person reporting results from use of this device should be trained to perform and interpret the results from this procedure by a competent instructor prior to use. CDC Influenza Division will limit the distribution of this device to only those users who have successfully completed a training course provided by CDC instructors or designees.
The CDC Human Influenza Real-Time RT-PCR Diagnostic Panel is used in real-time RT-PCR (rRT-PCR) assays on the Applied Biosystems® (ABI) 7500 Fast Dx Real-time PCR system. The panel is configured in four separate kits. Each kit consists of oligonucleotide primers. fluorescently labeled hydrolysis probes, and controls which are used in rRT-PCR assays for the in vitro qualitative detection and characterization of influenza virus RNA in respiratory specimens from patients presenting with influenza-like illness (ILI). Oligonucleotide primers and probes included in the Influenza B Lineage Genotyping Kit for detection of influenza B and the two major genetic lineages of influenza B were selected from highly conserved regions of the non-structural (NS) and HA genes, respectively. Oligonucleotide primers and probes to detect the human RNase P gene (RP) in control samples and clinical specimens are also included in the kit.
Here's a summary of the acceptance criteria and study details for the CDC Human Influenza Virus Real-Time RT-PCR Diagnostic Panel: Influenza B Lineage Genotyping Kit, based on the provided document:
This document describes a 510(k) premarket notification for modifications to an existing influenza B lineage genotyping kit (K172091). The changes primarily address recent evolutionary changes in circulating influenza B viruses that could impact the reactivity of the current kit, and the evaluation of an alternative fluorescent hydrolysis probe quencher chemistry (ZEN™). The study aims to demonstrate that the modified device remains substantially equivalent to the predicate device.
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria are generally demonstrated through 100% agreement or high percentage sensitivity/specificity with a clear lower bound for the 95% Confidence Interval (CI).
| Test Type | Acceptance Criteria | Reported Device Performance |
|---|---|---|
| Analytical Sensitivity (LOD Equivalency) | Demonstrating 100% positivity (3 out of 3 replicates) at either the same endpoint LOD concentration or within a 5-fold dilution of each other. | Influenza B Assay (BHQ vs. ZEN quencher): Both InfB (BHQ) and InfB (ZEN) showed similar reactivity and endpoint range with Influenza B/Nevada/03/2011 and B/Wisconsin/1/2010. For B/Nevada/03/2011, LOD for both was generally around 10^2.8 EID50/mL with Invitrogen Superscript™ and 10^3.5 EID50/mL with Quanta qScript™. For B/Wisconsin/1/2010, LOD for both was generally around 10^3.5 EID50/mL with Invitrogen Superscript™ and 10^4.2 EID50/mL with Quanta qScript™. |
Influenza B Lineage Genotyping Assay (VER 1.1 and VER 2 VIC & YAM assays): All assays demonstrated similar reactivity with apparent LOD endpoints within one 5-fold dilution of each other for various B/Victoria and B/Yamagata strains across both enzyme systems (Invitrogen Superscript™ and Quanta qScript™). For example, LOD for B/Maryland/15/2016 (VER 1.1 & VER 2) was 10^1.7 EID50/mL for both enzymes. |
| Analytical Sensitivity (LOD Confirmation) | Lowest virus concentration where InfB and VIC or InfB and YAM primer and probe sets demonstrated ≥ 95% detection. | Demonstrated LODs for B/Maryland/15/2016 (Victoria) were 10^1.7 EID50/mL for both VER 1.1 and VER 2 with both enzyme systems. Demonstrated LODs for B/Texas/81/2016 (Yamagata) were 10^2.2 EID50/mL (VER 1.1 & VER 2 with Invitrogen) and 10^1.5 EID50/mL (VER 1.1 & VER 2 with Quanta). |
| Analytical Sensitivity (Inclusivity) | 100% positivity (3 out of 3 replicates) for the tested strains at or near the established LOD. | B/Victoria (VER 1.1 & VER 2): All 10 tested B/Victoria strains showed 3/3 positive replicates (100% inclusivity) at or near their LOD for both VER 1.1 and VER 2, across both enzyme systems.
B/Yamagata (VER 1.1 & VER 2): All 10 tested B/Yamagata strains showed 3/3 positive replicates (100% inclusivity) at or near their LOD for both VER 1.1 and VER 2, across both enzyme systems. |
| Analytical Specificity (Cross-Reactivity) | No cross-reactivity detected with the opposite influenza B lineage. | No cross-reactivity was detected with either VER 1.1 or VER 2 VIC and YAM primer/probe sets when tested against 10 high-titer B/Yamagata lineage viruses (for VIC assays) and 10 high-titer B/Victoria lineage viruses (for YAM assays), across both enzyme systems. (Results consistently showed '-') |
| Analytical Specificity (Exclusivity with Influenza A Viruses) | No cross-reactivity detected with various influenza A viruses. | No cross-reactivity was detected with either VER 1.1 or VER 2 VIC and YAM assays when tested against 8 high-titer Influenza A viruses of various human and animal subtypes, across both enzyme systems. (Results consistently showed '-') |
| Analytical Specificity (Exclusivity with Non-Influenza Respiratory Pathogens) | No cross-reactivity detected with common non-influenza respiratory pathogens or flora. | No cross-reactivity was detected with VER 2 VIC and YAM assays when tested against 36 organisms (16 viruses, 19 bacteria, and 1 yeast) at high titers, across both enzyme systems. (Results consistently showed '-') |
| Clinical Performance (Retrospective InfB Assay) | 100% positive and negative agreement with the cleared oligonucleotide probe (BHQ quencher). | 100.0% Positive Agreement (95% CI: 88.7-100.0) for 30/30 positive samples. 100.0% Negative Agreement (95% CI: 92.9-100.0) for 50/50 negative samples. (Both enzyme systems showed identical results). |
| Clinical Performance (VER 2 B/Victoria Prospective Study) | High sensitivity and specificity, typically aiming for >90% sensitivity and >95% specificity with a tight 95% CI. | Sensitivity: 100.0% (5/5 positives) (95% CI: 56.6 – 100.0).
Specificity: 100.0% (574/574 negatives) (95% CI: 99.3 – 100.0). (Note: Low number of positives impacts CI width for sensitivity). |
| Clinical Performance (VER 2 B/Yamagata Prospective Study) | High sensitivity and specificity, typically aiming for >90% sensitivity and >95% specificity with a tight 95% CI. | Sensitivity: 100.0% (38/38 positives) (95% CI: 90.8 – 100.0).
Specificity: 99.8% (540/541 negatives) (95% CI: 99.0 – 100.0). |
| Clinical Performance (Retrospective VER 1.1 & VER 2 VIC & YAM assays) | 100% positive and negative agreement with the FDA-cleared Influenza B Lineage Genotyping Kit. | VIC VER 1.1: 100.0% Positive Agreement (30/30) (95% CI: 88.7 - 100.0); 100.0% Negative Agreement (157/157) (95% CI: 97.6 - 100.0).
YAM VER 1.1: 100.0% Positive Agreement (96/96) (95% CI: 96.2 - 100.0); 100.0% Negative Agreement (91/91) (95% CI: 96.0 - 100.0).
VIC VER 2: 100.0% Positive Agreement (30/30) (95% CI: 88.7 - 100.0); 100.0% Negative Agreement (157/157) (95% CI: 97.6 - 100.0).
YAM VER 2: 100.0% Positive Agreement (96/96) (95% CI: 96.2 - 100.0); 100.0% Negative Agreement (91/91) (95% CI: 96.0 - 100.0). |
2. Sample Size for the Test Set and Data Provenance
Test Set Sample Sizes:
- Analytical Sensitivity (LOD Equivalency & Inclusivity): Varied. Typically, 3 replicates per dilution/strain. For LOD confirmation, 20 extraction replicates per dilution.
- Analytical Specificity (Cross-Reactivity & Exclusivity): For cross-reactivity, 10 B/Yamagata strains for VIC assays and 10 B/Victoria strains for YAM assays. For exclusivity with Influenza A, 8 Influenza A strains. For exclusivity with non-influenza pathogens, 36 organisms (16 viruses, 19 bacteria, 1 yeast). All tested with both enzyme systems.
- Clinical Performance (InfB Assay Retrospective): 30 positive and 50 negative upper respiratory tract clinical samples.
- Clinical Performance (VER 2 Prospective Study):
- Total collected: 592 samples.
- Analyzed: 579 upper respiratory tract clinical specimens.
- B/Victoria: 5 positive, 574 negative.
- B/Yamagata: 38 positive, 541 negative.
- Clinical Performance (VER 1.1 & VER 2 Retrospective Study):
- 126 specimens positive for either influenza B/Victoria or B/Yamagata viruses.
- 61 specimens negative for influenza B viruses.
- (An additional 4 lung tissue specimens were tested for informational purposes but not included in performance calculations).
Data Provenance:
- Analytical studies: Laboratory-characterized virus stocks and reference strains (e.g., CDC repositories, ATCC).
- Clinical Performance (InfB Assay Retrospective): Retrospective clinical samples collected during the 2011-2012 influenza season.
- Clinical Performance (VER 2 Prospective Study): Prospective clinical investigation conducted at 3 U.S. public health laboratories using upper respiratory tract specimens collected during the 2016-2017 influenza season.
- Clinical Performance (VER 1.1 & VER 2 Retrospective Study): Retrospective upper respiratory tract clinical samples collected during the 2017 and 2017-2018 influenza seasons.
Country of Origin: Primarily the United States (CDC, U.S. public health laboratories) with some virus strains noted as originating from various international locations (e.g., Hong Kong, Bolivia, Laos, Michigan, etc.) for inclusivity and exclusivity testing, reflecting global diversity.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
The ground truth for the test sets (both analytical and clinical) appears to be established by molecular diagnostic methods using an FDA-cleared predicate device, the CDC Human Influenza Virus Real-Time RT-PCR Diagnostic Panel: Influenza B Lineage Genotyping Kit (K172091), and/or by using well-characterized virus stocks with known titers (EID50/mL or TCID50/mL).
For clinical samples, they were "previously determined to be positive or negative for influenza B virus" or "determined to be positive for influenza B/Victoria or B/Yamagata viruses using the FDA-cleared Influenza B Lineage Genotyping Kit."
The document does not explicitly mention the use of a specific "number of experts" (e.g., clinicians, virologists, or laboratory professionals) for establishing the individual patient-level clinical ground truth for the test sets in the same way one might describe image interpretation by radiologists. Instead, the ground truth is primarily based on the results of the already FDA-cleared predicate diagnostic test or characterized reference materials. The "competent instructor" trained individuals (users, analysts, and those reporting results) on how to perform and interpret the device results.
4. Adjudication Method for the Test Set
The document does not describe an adjudication method for conflicting results in the test set, such as a 2+1 or 3+1 system. This is typical for molecular diagnostic studies where the comparison is generally made against a well-established reference method (the predicate device or characterized stocks). Any discrepancies would likely be investigated to determine the cause (e.g., retesting, review of sample integrity, or confirmation by alternative gold standard) rather than being resolved by expert consensus in a traditional adjudication panel format. For example, in the LOD studies, "3 out of 3 replicates" or "≥ 95% detection" criteria are used, indicating a reliance on consistent assay performance rather than expert adjudication.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No, a multi-reader multi-case (MRMC) comparative effectiveness study was not explicitly described. MRMC studies are typically performed for imaging devices or other diagnostics where human interpretation is a key component and the effect of AI assistance on human performance is being evaluated. This submission is for a molecular diagnostic (RT-PCR kit) where the output is a quantitative measurement (Ct values) and qualitative result (positive/negative, lineage determination) interpreted directly by the instrument software or a trained individual following predefined rules, not a subjective interpretation by multiple human readers.
Therefore, there is no mention of an effect size of how much human readers improve with AI vs. without AI assistance.
6. Standalone (Algorithm Only) Performance
Yes, the studies presented primarily represent standalone performance of the device (or its modified components). The analytical performance evaluations (sensitivity, inclusivity, specificity, exclusivity) directly assess the algorithm's (reagent kit's) ability to detect and differentiate influenza B lineages from viral RNA. The clinical performance studies compare the results of the investigational device directly against the FDA-cleared predicate device on clinical samples. While human technicians perform the laboratory procedures, the "algorithm" here refers to the specific RT-PCR assay chemistry, probes, and primer sets that yield a result, which is then interpreted according to established cut-offs/rules, not an AI algorithm that assists human interpretation.
7. Type of Ground Truth Used
The ground truth used for the studies is a combination of:
- Reference standard/Predicate device: For clinical samples, the ground truth was largely established by the "FDA-cleared CDC Human Influenza Virus Real-Time RT-PCR Diagnostic Panel: Influenza B Lineage Genotyping Kit."
- Characterized virus stocks: For analytical studies (LOD, inclusivity, cross-reactivity, exclusivity), the ground truth was established using well-characterized influenza virus stocks and other respiratory pathogens with known strain designations, subtypes, and titers (EID50/mL, TCID50/mL, CFU/mL, or ng/µL). This implies a high level of confidence in the identity and concentration of the biological material.
8. Sample Size for the Training Set
The document does not explicitly state a sample size for a "training set" in the context of an algorithm's development. This is because the device described is a molecular diagnostic kit (RT-PCR) with specific primers and probes designed based on biological knowledge of viral sequences, rather than a machine learning or artificial intelligence algorithm that requires a distinct training phase with labeled data.
The "modifications" were "primarily to address recent evolutionary changes in circulating influenza B viruses that may impact the reactivity of the current Influenza B Lineage Genotyping Kit. Two design approaches were evaluated that address specific genetic mutations in the targeted hemagglutinin (HA) gene..." This indicates a design process driven by biological knowledge and engineering based on viral genetics, not data-driven machine learning training.
9. How the Ground Truth for the Training Set was Established
Since there's no explicitly defined "training set" for an AI algorithm in the provided text, the concept of establishing ground truth for it doesn't directly apply. The design of the new VER 1.1 and VER 2 assays was based on scientific understanding of circulating influenza B virus evolutionary changes and specific genetic mutations in the HA gene. This informed the selection of the oligonucleotide primers and probes. The efficacy of these new designs was then evaluated using the analytical and clinical studies described in the rest of the document, where the ground truth was established as detailed in point 7.
§ 866.3980 Respiratory viral panel multiplex nucleic acid assay.
(a)
Identification. A respiratory viral panel multiplex nucleic acid assay is a qualitative in vitro diagnostic device intended to simultaneously detect and identify multiple viral nucleic acids extracted from human respiratory specimens or viral culture. The detection and identification of a specific viral nucleic acid from individuals exhibiting signs and symptoms of respiratory infection aids in the diagnosis of respiratory viral infection when used in conjunction with other clinical and laboratory findings. The device is intended for detection and identification of a combination of the following viruses:(1) Influenza A and Influenza B;
(2) Influenza A subtype H1 and Influenza A subtype H3;
(3) Respiratory Syncytial Virus subtype A and Respiratory Syncytial Virus subtype B;
(4) Parainfluenza 1, Parainfluenza 2, and Parainfluenza 3 virus;
(5) Human Metapneumovirus;
(6) Rhinovirus; and
(7) Adenovirus.
(b)
Classification. Class II (special controls). The special controls are:(1) FDA's guidance document entitled “Class II Special Controls Guidance Document: Respiratory Viral Panel Multiplex Nucleic Acid Assay;”
(2) For a device that detects and identifies Human Metapneumovirus, FDA's guidance document entitled “Class II Special Controls Guidance Document: Testing for Human Metapneumovirus (hMPV) Using Nucleic Acid Assays;” and
(3) For a device that detects and differentiates Influenza A subtype H1 and subtype H3, FDA's guidance document entitled “Class II Special Controls Guidance Document: Testing for Detection and Differentiation of Influenza A Virus Subtypes Using Multiplex Nucleic Acid Assays.” See § 866.1(e) for the availability of these guidance documents.