INFLUENZA A/H5 (ASIAN LINEAGE) VIRUS REAL-TIME RT-PCR PRIMER AND PROBE SET

• K Number: DEN060008
• Device Name: INFLUENZA A/H5 (ASIAN LINEAGE) VIRUS REAL-TIME RT-PCR PRIMER AND PROBE SET
• Manufacturer: CENTERS FOR DISEASE CONTROL AND PREVENTION
• Date Cleared: 2006-02-03 ( 7 days)
• Product Code: NXD
• Regulation Number: 866.3332
• Type: Post-NSE
• Panel: MI
• Reference & Predicate Devices: N/A

Intended Use

The Influenza A/H5 (Asian Iineage) Virus Real-time RT-PCR Primer and Probe Set is intended for the in vitro qualitative detection of Influenza A/H5 (Asian lineage) virus RNA either directly in patient respiratory specimens or in viral cultures for the presumptive laboratory identification of Influenza A/H5 (Asian lineage) virus.

Testing with the Influenza A/H5 (Asian lineage) Virus Real-time RT-PCR Primer and Probe Set should be used in conjunction with other laboratory testing and clinical observations for the following indications:

• Providing epidemiological information for the surveillance of human infection with Influenza A/H5 (Asian lineage) virus
• Identifying patients who may be infected with Influenza A/H5 (Asian lineage) virus based on clinical and epidemiological risk factors
  The definitive identification of influenza A/H5 (Asian lineage) either directly from patient specimens or from viral cultures requires additional laboratory testing, along with clinical and epidemiological assessment in consultation with national influenza surveillance experts.

Negative results do not preclude influenza virus infection and should not be used as the sole basis for treatment or other patient management decisions.

Use of this assay is limited to Laboratory Response Network (LRN) designated laboratories.

Device Description

The Influenza A/H5 (Asian lineage) Virus Real-time RT-PCR Primer and Probe Set is composed of 2 primer pairs (4 unlabeled oligonucleotides) and 3 labeled probes, along with an inactivated virus control. These reagents are for use in single-tube realtime RT-PCR testing to directly detect novel Influenza A virus specific RNA in human respiratory specimens, or in viral cultures. The real-time PCR process simultaneously amplifies and detects nucleic acid targets in the same reaction.

The primer and probe sets (FluA2 and FluA3) target two distinct RNA regions that are both present in the influenza A/H5 (Asian lineage) hemagglutinin (HA) gene of highly pathogenic H5N1 viruses from the Asian lineage. These target regions were chosen to allow specific detection of Asian lineage influenza A/H5 viruses without detection of other influenza virus subtypes, including the North American lineage influenza A/H5 viruses (e.g., avian H5N2 strains).

Note: There are two lineages of avian influenza A/H5 viruses: the Eurasian (Asian) and North American (American) lineages. Viruses from these two lineages are genetically different. All known human influenza H5 infections have been caused by highly pathogenic viruses of the Asian lineage.

Primers/Probes: The 2 primer and probe sets (FluA2 and FluA3), each target a different region within the HA gene. The FluA2 target is in the HA2 region of the HA gene and the FluA3 target spans the cleavage site of the HA gene. These sets were selected from multiple candidates based on preliminary assessment of reaction efficiency and primer-dimer effects. FluA2 contains an equal mixture of two probes, to ensure optimum homology with viruses within both clades. The probes are labeled with FAM (6-carboxyfluorescein) and Blackhole Quencher™ 1 (BHQTM1). The BHQ chemistry is designed to minimize non-specific fluorescence. Experimental efficiency values of 100% +/- 5% are considered optimal. Tagman reaction efficiencies of FluA2 and FluA3 sets were demonstrated to be 100.6% (R2=0.996) and 100.3% (R2=0.996).
Influenza A/H5N1 positive control (500 µL of virus suspended in 0.01 M PBS): a reverse-engineered vaccine candidate virus that may safely be handled in BSL-2. Inactivated with beta- propiolactone (0.05%). The reassortant virus is noninfectious in chickens and USDA has removed it from the select agent list, classifying it as a BSL2 infectious agent. Additionally, the inactivated virus preparation is non-infectious in embryonated chicken eggs.

Other reagents or accessories required to perform testing with the device are:

Qiagen QuantiTect™ Probe RT-PCR Kit (Qiagen), a master-mix for reversetranscription and cDNA amplification.
Materials for extraction and purification of specimens (three commercial extraction kits are suggested for use in isolating RNA in the test procedure based on specifications meeting the requirements for quality of RNA to be used in the FluA2 and FluA3 reactions). These extraction procedures were used during development and testing of the Influenza A/H5 (Asian lineage) Virus Real-time RT-PCR Primer and Probe Set.
An RNase P (RP) Real-time PCR Primer and probe set that targets the human ribonuclease P (RP) sequence. Extracted clinical specimens should contain human RNA. The RP assay thus serves as a control to ensure RNA resulted from the extraction of clinical specimens.

AI/ML Overview

1. A table of acceptance criteria and the reported device performance

The document does not explicitly state pre-defined acceptance criteria for device performance. However, based on the reproducibility study results, the implied criteria for positive detection (presumptive positive) appear to relate to the virus concentration and agreement with expected results. For negative samples, the acceptance criterion is a 100% agreement with expected results or near 100% given the one equivocal observation.

Performance Metric	Acceptance Criteria (Implied)	Reported Device Performance
Reproducibility (Positive Samples)	High agreement with expected positive results at various concentrations, especially at higher concentrations. Specific thresholds are not explicitly stated, but generally, high accuracy is expected.	Overall (across all instruments):

• 400 TCID50/mL: All 36 samples detected (100%)
• 40 TCID50/mL: 47/55 (85.5%) positive; 52/55 (94.5%) positive or equivocal
• 4 TCID50/mL: 32/36 (88.9%) positive; 35/36 (97.2%) positive or equivocal
  Specific Instrument Performance (Agreement with Expected Results for Positive Samples):
• ABI 7000: 100% at 400, 40, and 4 TCID50/mL (Swab, low 0.4 TCID50/mL: 10% positive, 30% equivocal, 70% negative)
• ABI 7700: 100% at 400, 40, and 4 TCID50/mL (Swab, low 0.4 TCID50/mL: 0% positive, 28.5% equivocal, 71.5% negative)
• LightCycler®: 100% at 400 TCID50/mL, 78% at 40 TCID50/mL, 89% at 40 TCID50/mL (liquid), 67% at 4 TCID50/mL
• SmartCycler®: 100% at 400 TCID50/mL, 67% at 40 TCID50/mL, 100% at 40 TCID50/mL (liquid), 90% at 4 TCID50/mL |
  | Reproducibility (Negative Samples) | High agreement with expected negative results (e.g., 100% negative). | Overall (across all instruments): 71 negative control samples had 1 equivocal result, yielding 98.6% agreement with expected results.
  Specific Instrument Performance (Agreement with Expected Results for Negative Samples):
• ABI 7000: 100% for both liquid and swab negative samples
• ABI 7700: 100% for both liquid and swab negative samples
• LightCycler®: 100% for liquid negative, 89% for swab negative (due to 1 equivocal)
• SmartCycler®: 100% for both liquid and swab negative samples |
  | Detection Limit | Not explicitly stated as a criterion, but values are given as a performance characteristic. | - ABI 7000 and ABI 7700: 4 TCID50/mL for each target
• LightCycler® and SmartCycler®: 40 TCID50/mL for each target |
  | Analytical Specificity (Cross-reactivity) | No cross-reactivity with non-H5 Asian HA types, non-influenza respiratory viruses, or bacteria. | No cross-reactivity observed with the tested viral strains or bacterial species (17 stock strains of non-H5 influenza, 8 non-influenza respiratory viruses, 11 bacterial species). |
  | Purity of Virus Control | Non-infectious | The inactivated virus control is non-infectious in chickens and embryonated chicken eggs. |
  | Clinical Sensitivity | Not estimable due to rarity of Asian H5 virus infections. | Not estimated. |
  | Clinical Specificity | Not estimable due to rarity of Asian H5 virus infections. | Not estimated. |
  | Agreement with Lab-Confirmed Cases (H5N1) | High agreement with confirmed patient diagnosis. | 10/22 (45%) specimens were presumptive positive, and another 9 were equivocal. Thus, 19/22 (86%, 65.1-97.1% CI) specimens showed A/H5 (Asian lineage) virus detection in one or both probe-set reactions, agreeing with the patient case diagnosis (from 10 patients). All lab-confirmed cases had at least one positive test when multiple specimens were tested. |
  | Agreement with Banked Respiratory Specimens (Negative) | High agreement with expected negative results. | Overall agreement was 98.5% (201/204, 95.8-99.7% CI). One sample was positive (confirmed H5N1), and two were equivocal (contained A/H3). |
  | Agreement with Banked Tissue Culture Samples (Negative) | High agreement with expected negative results. | Agreement was 95.2% (20/21, 76.2-99.9% CI). One sample was equivocal (contained A/H3). |

2. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

• Reproducibility Study (Simulated Specimens):

  • Test Set Size: 6 simulated respiratory specimens (3 swabs, 3 liquid) at various virus concentrations (including negative), tested in 10 laboratories (ABI 7000: 60 tests), 7 laboratories (ABI 7700: 42 tests), 9 laboratories (LightCycler®: 54 tests), and 10 laboratories (SmartCycler®: 60 tests).
  • Total Samples Tested: 216 tests distributed across the instruments. Each simulated specimen type was likely tested multiple times within each lab or across labs.
  • Data Provenance: The simulated specimens were "spiked with both human cellular material and titered amounts of the virus control material." The control virus itself is a "reverse-engineered vaccine candidate virus." This indicates contrived samples, likely prepared in a laboratory setting (e.g., CDC). This is a prospective study design for performance evaluation.

• Clinical Study (Laboratory-confirmed patient cases):

  • Test Set Size: 27 specimens from 10 laboratory-confirmed H5N1 patients and 2 specimens from 1 contact patient (not confirmed for H5N1).
  • Data Provenance: Retrospective. Specimens were from "laboratory-confirmed patient cases (from Indonesia) of influenza H5N1 virus."

• Clinical Study (Banked human respiratory specimens):

  • Test Set Size: 204 banked human respiratory specimens.
  • Data Provenance: Retrospective. Specimens were received by CDC Influenza Branch Laboratory from Vietnam (168), United States (21), Thailand (11), Kuwait (4).

• Clinical Study (Banked tissue culture samples):

  • Test Set Size: 21 banked tissue culture samples.
  • Data Provenance: Retrospective. Received by CDC Influenza Branch Laboratory from international locations (18) and the United States (3).

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

• Reproducibility Study: No human experts were involved in establishing ground truth for the simulated test set. The ground truth was based on the known concentration of the spiked virus and the expected negative status of non-spiked samples.

• Clinical Study (Laboratory-confirmed patient cases):

  • Number of Experts: Not explicitly stated, but "Confirmation by independent (WHO or Indonesian) laboratory or sequence analysis" indicates multiple experts and methodologies were involved. "Multiple WHO methods (culture, sequencing, RT-PCR testing for multiple HA targets) were used to characterize H5N1 in these patients."
  • Qualifications: "National influenza surveillance experts" and staff in "WHO-recognized RT-PCR assays" and the "WHO Collaborating Center for Surveillance, Epidemiology and Control of Influenza" are implied. These would be highly qualified virologists, microbiologists, and molecular biologists.

• Clinical Study (Banked human respiratory specimens):

  • Number of Experts: Unspecified, but confirmed by "Influenza Branch4 Lab Real-time RT-PCR Panel Result." This refers to the CDC's Influenza Branch Laboratory, a WHO Collaborating Center, implying a team of experts.
  • Qualifications: Experts at a "WHO Collaborating Center for Surveillance, Epidemiology and Control of Influenza" and a "Bioterrorism Rapid Response and Advanced Technology Laboratory" would be highly qualified specialists in virology, molecular diagnostics, and public health.

• Clinical Study (Banked tissue culture samples):

  • Number of Experts: Unspecified, but "Influenza Branch3 Lab Real-time RT-PCR Panel Result" implies experts from the CDC Influenza Branch Laboratory.
  • Qualifications: Similar to the above, highly qualified specialists in virology and molecular diagnostics.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

• Reproducibility Study: No adjudication method was necessary for the simulated test set, as the ground truth was based on engineered virus concentrations.

• Clinical Studies (Laboratory-confirmed patient cases, Banked human respiratory specimens, Banked tissue culture samples):

  • The term "adjudication" in the sense of resolving discrepancies between multiple readers is not directly applicable. For patient cases, confirmation was achieved through "independent (WHO or Indonesian) laboratory or sequence analysis" and "Multiple WHO methods (culture, sequencing, RT-PCR testing for multiple HA targets)." This suggests a consensus-based approach or reconciliation of results by reference methods rather than an explicit "X+Y" adjudication.
  • For banked specimens, the "Influenza Branch Lab Real-time RT-PCR Panel Result" serves as the reference standard, implying a thorough process by a specialized laboratory.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• There was no MRMC comparative effectiveness study done.
• This device is a diagnostic kit (RT-PCR primer and probe set), not an AI or imaging device that would typically involve human "readers" in the sense of interpreting outputs that AI could aid. The output is a quantitative (CT value) and qualitative (positive/equivocal/negative) result based on amplification.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

• Yes, the performance studies primarily assess the standalone performance of the diagnostic kit. The "reproducibility study" evaluates the kit's performance across different laboratories and instruments, and the "clinical studies" evaluate its ability to detect the virus in patient and banked samples, independent of human interpretation improvements.
• While laboratory technicians operate the equipment, the interpretation of the RT-PCR results (based on crossing thresholds and specific target detection) follows established protocols without a human "in-the-loop" decision-making process in the same way an AI-assisted diagnostic device might be evaluated. The result is directly derived from the reaction.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)

• Reproducibility Study: Engineered/Known Concentration. The ground truth was based on the known, spiked concentrations of the Influenza A/H5N1 virus into simulated respiratory specimens.

• Clinical Study (Laboratory-confirmed patient cases): Multi-method Laboratory Confirmation / Consensus. The ground truth was established by "independent (WHO or Indonesian) laboratory or sequence analysis" and "Multiple WHO methods (culture, sequencing, RT-PCR testing for multiple HA targets)." This represents a form of expert-driven, multi-modal laboratory confirmation, akin to an "expert consensus" based on advanced diagnostic techniques.

• Clinical Studies (Banked human respiratory specimens and tissue culture samples): Reference Laboratory Panel Result. The ground truth was established by the "Influenza Branch Lab Real-time RT-PCR Panel Result" at CDC (a WHO Collaborating Center). This acts as a highly reliable reference standard, representing the most definitive laboratory diagnosis available.

8. The sample size for the training set

• The document describes a diagnostic kit (reagents), not an AI or machine learning model. Therefore, there is no specific "training set" in the conventional sense of data used to train an algorithm.
• The "development" of the primer and probe sets involved selecting them from "multiple candidates based on preliminary assessment of reaction efficiency and primer-dimer effects." This involved testing, but it's part of the assay design and optimization, not machine learning model training.

9. How the ground truth for the training set was established

• As stated above, there is no "training set" for an algorithm in this context. The development process for the primer and probe sets involved analytical testing with known viral strains and concentrations to ensure specificity and efficiency, which inherently relies on established knowledge of viral genetics and laboratory standards (e.g., known positive controls, established culture methods).

Regulation Number and Section

§ 866.3332 Reagents for detection of specific novel influenza A viruses.

(a)
Identification. Reagents for detection of specific novel influenza A viruses are devices that are intended for use in a nucleic acid amplification test to directly detect specific virus RNA in human respiratory specimens or viral cultures. Detection of specific virus RNA aids in the diagnosis of influenza caused by specific novel influenza A viruses in patients with clinical risk of infection with these viruses, and also aids in the presumptive laboratory identification of specific novel influenza A viruses to provide epidemiological information on influenza. These reagents include primers, probes, and specific influenza A virus controls.(b)
Classification. Class II (special controls). The special controls are:(1) FDA's guidance document entitled “Class II Special Controls Guidance Document: Reagents for Detection of Specific Novel Influenza A Viruses.” See § 866.1(e) for information on obtaining this document.
(2) The distribution of these devices is limited to laboratories with experienced personnel who have training in standardized molecular testing procedures and expertise in viral diagnosis, and appropriate biosafety equipment and containment.