(112 days)
The FilmArray Respiratory Panel (RP) is a multiplexed nucleic acid test intended for use with the FilmArray instrument for the simultaneous qualitative detection and identification of multiple respiratory viral nucleic acids in nasopharyngeal swabs (NPS) obtained from individuals suspected of respiratory tract infections. The following virus types and subtypes are identified using the FilmArray RP: Adenovirus, Coronavirus HKU1, Coronavirus NL63, Human Metapneumovirus, Influenza A, Influenza A subtype H1, Influenza A subtype H3, Influenza A subtype 2009 H1, Influenza B, Parainfluenza virus 3, Rhinovirus/Enterovirus, and Respiratory Syncytial Virus. The detection and identification of specific viral nucleic acids from individuals exhibiting signs and symptoms of a respiratory infection aids in the diagnosis of respiratory infection if used in conjunction with other clinical and epidemiological information. Negative results do not preclude respiratory infection and should not be used as the sole basis for diagnosis, treatment or other management decisions. Positive results do not rule out bacterial infection or co-infection with other organisms. The agent detected may not be the definite cause of disease. The use of additional laboratory testing (e.g. bacterial and viral culture, immunofluorescence, and radiography) and clinical presentation must be taken into consideration in order to obtain the final diagnosis of respiratory infection.
Due to seasonal prevalence, performance characteristics for Influenza A/H1, Influenza A/H3, Influenza A/2009 H1, and Influenza B were established primarily with retrospective clinical specimens.
Due to the genetic similarity between human Rhinovirus and Enterovirus, the FilmArray RP cannot reliably differentiate them. A positive FilmArray RP Rhinovirus/Enterovirus result should be followed-up using an alternate method (e.g. cell culture or sequence analysis).
The FilmArray RP detects Adenovirus species C serotype 6 with reduced sensitivity. It is recommended that specimens found to be negative for Adenovirus after examination using FilmArray RP be confirmed by an alternate method (e.g. FDA cleared molecular test or cell culture).
Performance characteristics for influenza A were established when influenza A/2009 H1N1, A/H1, and A/H3 were the predominant influenza A viruses in circulation. When other influenza A viruses are emerging, performance characteristics may vary. If infection with a novel influenza A virus is suspected based on current clinical and epidemiological screening criteria recommended by public health authorities, specimens should be collected with appropriate infection control precautions for novel virulent influenza viruses and sent to state or local health departments for testing. Viral culture should not be attempted in these cases unless a BSL 3+ facility is available to receive and culture specimens.
The FilmArray RP System is multiplex nucleic acid test system composed of the FilmArray instrument, the FilmArray software (preinstalled on a laptop computer) and the FilmArray RP pouch. The FilmArray RP pouch contains freeze-dried reagents to perform nucleic acid purification, reverse transcription, and nested, multiplex PCR with DNA melt analysis. The Respiratory Panel (RP) pouch identifies 12 common and emerging viral respiratory pathogens (see Table 1).
A test is initiated by loading Hydration Solution and an unprocessed patient nasopharyngeal swab (NPS) specimen (i.e. specimen mixed with Sample Buffer) into the FilmArray RP pouch. The pouch contains all of the reagents required for specimen testing and analysis in a freeze-dried format; the addition of Hydration Solution and Sample/Buffer Mix rehydrates the reagents. After the pouch is prepared, the FilmArrav software guides the user though the steps of placing the pouch into the instrument, scanning the pouch barcode, entering the sample identification and initiating the run.
The FilmArray instrument contains a coordinated system of inflatable bladders and seal points, which act on the pouch to control the movement of liquid between the pouch blisters. When a bladder is inflated over a reagent blister, it forces liquid from the blister into connecting channels. Alternatively, when a seal is placed over a connecting channel it acts as a valve to open or close a channel. In addition, electronically controlled pneumatic pistons are positioned over multiple plungers in order to deliver the rehydrated reagents into the blisters at the appropriate times. Two Peltier devices control heating and cooling of the pouch to drive the reverse transcription reactions, the PCR reactions, and the melting curve analysis.
Nucleic acid extraction occurs within the FilmArray pouch using mechanical lysis and standard magnetic bead technology. After extracting and purifying nucleic acids from the unprocessed sample, the FilmArray performs a nested multiplex PCR that is executed in two stages. During the first stage, the FilmArray performs a single, large volume, highly multiplexed reverse transcription PCR (rt-PCR) reaction. The products from first stage PCR are then diluted and combined with a fresh, primer-free master mix and a fluorescent double stranded DNA binding dye (LC Green@Plus, Idaho Technology). This second master mix solution, is then distributed to each well of the array. Array wells contain sets of primers designed specifically to amplify sequences internal to the PCR products generated during the first stage PCR reaction. The second stage PCR, or nested PCR, is performed in singleplex fashion in each well of the array. At the conclusion of the 200 stage PCR, the array is interrogated by melting curve analysis for the detection of signature amplicons denoting the presence of specific viral or bacterial targets A digital camera placed in front of the second stage PCR captures fluorescent images of the PCR reactions in real time.
The FilmArray software automatically interprets the results of each DNA melting curve analysis and combines the data with the results of the internal pouch controls to provide a test result for each organism on the panel.
Here's an analysis of the provided text, focusing on acceptance criteria and supporting study details:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state pre-defined "acceptance criteria" in a numerical target for metrics like sensitivity or specificity. However, it presents the reported clinical sensitivity (Positive Percent Agreement - PPA) and specificity (Negative Percent Agreement - NPA) for each detected organism from two studies: a prospective clinical study and an archived specimen study. The implicit acceptance criteria would be for these values to be high, ideally close to 100%, and for the 95% Confidence Intervals (CI) to be acceptably narrow.
Table: Reported Device Performance (Clinical Studies)
| Organism/Virus | Study Type | Sensitivity (PPA) | 95% CI | Specificity (NPA) | 95% CI |
|---|---|---|---|---|---|
| Adenovirus | Prospective Clinical | 88.9% (24/27) | 70.8 - 97.7% | 98.3% (812/826) | 97.2 - 99.1% |
| Adenovirus | Archived Specimen | 100.0% (27/27) | 87.2 - 100% | 100.0% (28/28) | 87.7 - 100% |
| Coronavirus HKU1 | Prospective Clinical | 95.8% (23/24) | 78.9 - 99.9% | 99.8% (827/829) | 99.1 - 100% |
| Coronavirus NL63 | Prospective Clinical | 95.8% (23/24) | 78.9 - 99.9% | 100.0% (829/829) | 99.6 - 100% |
| Human Metapneumovirus | Prospective Clinical | 94.6% (88/93) | 87.9 - 98.2% | 99.2% (754/760) | 98.3 - 99.7% |
| Human Rhinovirus/Enterovirus | Prospective Clinical | 92.7% (190/205) | 88.2 - 95.8% | 94.6% (613/648) | 92.6 - 96.2% |
| Enterovirus | Archived Specimen | 95.7% (22/23) | 78.0 - 99.9% | 100.0% (90/90) | 96.0 - 100% |
| Influenza A | Prospective Clinical | 90.0% (9/10) | 55.5 - 99.8% | 99.8% (841/843) | 99.2 - 100% |
| Influenza A/H1 | Prospective Clinical | 0/0 | n/a | 100.0% (853/853) | 99.6 - 100% |
| Influenza A/H1 | Archived Specimen | 100.0% (32/32) | 89.1 - 100% | 100.0% (127/127) | 97.1 - 100% |
| Influenza A/H3 | Prospective Clinical | 0/0 | n/a | 100.0% (853/853) | 99.6 - 100% |
| Influenza A/H3 | Archived Specimen | 100.0% (54/54) | 93.4 - 100% | 100.0% (105/105) | 96.5 - 100% |
| Influenza A/2009 H1 | Prospective Clinical | 88.9% (8/9) | 51.8 - 99.7% | 99.6% (841/844) | 99.0 - 99.9% |
| Influenza A/H1-2009 | Archived Specimen | 100.0% (34/34) | 89.7 - 100% | 100.0% (125/125) | 97.1 - 100% |
| Influenza B | Prospective Clinical | 0/0 | n/a | 100.0% (853/853) | 99.6 - 100% |
| Influenza B | Archived Specimen | 100.0% (30/30) | 88.4 - 100% | 100.0% (129/129) | 97.2 - 100% |
| Parainfluenza Virus 3 | Prospective Clinical | 95.8% (23/24) | 78.9 - 99.9% | 98.8% (819/829) | 97.8 - 99.4% |
| PIV3 | Archived Specimen | 100.0% (36/36) | 90.3 - 100% | 100.0% (93/93) | 96.1 - 100% |
| Respiratory Syncytial Virus | Prospective Clinical | 100.0% (52/52) | 93.2 - 100% | 89.1% (714/801) | 86.8 - 91.2% |
2. Sample Sizes and Data Provenance
- Test Set (Clinical Performance):
- Prospective Clinical Study:
- Sample Size: 853 subjects (from an initial enrollment of 857).
- Data Provenance: 3 U.S. clinical sites, prospective data collected from December 2009 through May 2010.
- Archived Specimen Study:
- Sample Size: 349 archived clinical NPS specimens were confirmed to contain or lack the analyte of interest (from 400 initially selected).
- Data Provenance: Retrospective, preselected archived clinical NPS specimens. Age demographics provided (e.g., 89 (25.5%) ≤5 years, 188 (53.9%) unknown age from a pediatric hospital). Country of origin is implied to be U.S. from clinical sites and some strains are from particular US states, but not explicitly stated for all archived samples.
- Prospective Clinical Study:
3. Number of Experts and Qualifications for Ground Truth (Test Set)
The document primarily relies on comparator methods for establishing ground truth, rather than a direct count of human expert consensus.
- For Adenovirus, Influenza A, Influenza B, Parainfluenza virus 3, Respiratory Syncytial Virus (Prospective Study):
- Ground Truth Method: Viral culture followed by DFA (Fluorescent Antibody) identification. "True" positives/negatives were defined based on these comparator methods.
- For FluA/H1, FluA/H3, FluA/2009 H1 (Prospective Study):
- Ground Truth Method: Viral culture followed by one analytically validated PCR assay with bi-directional sequence confirmation. The comparator PCR assays were designed to amplify different sequences than the FilmArray assays. "True" positives/negatives were based on cultural positivity for Influenza A and bi-directional sequencing results matching NCBI GenBank entries.
- For Human Rhinovirus, Coronavirus NL63, Coronavirus HKU1, Human Metapneumovirus (Prospective Study):
- Ground Truth Method: A "predetermined algorithm that used composite comparator methods" consisting of two analytically validated PCR assays followed by bi-directional sequencing. "True" positives/negatives were based on bi-directional sequencing data matching NCBI GenBank entries or negative results from both comparator PCR assays.
- For Archived Specimen Study:
- Ground Truth Method: Analyte-specific PCR and bi-directional sequencing were used to confirm the presence or absence of the analyte of interest in each specimen prior to FilmArray RP testing.
The document does not specify the number of experts or their qualifications for interpreting the DFA, viral culture, PCR, or sequencing results. It states these are "analytically validated" assays, implying expert input in their validation and interpretation, but provides no direct detail on the human experts involved in establishing the ground truth for this specific study.
4. Adjudication Method for the Test Set
The document describes comparator methods for defining "true" positive/negative results. For the clinical performance sections:
- For categories like Adenovirus through RSV in the prospective study, ground truth was set by viral culture + DFA. Discrepancies between FilmArray RP and the comparator method were further investigated (e.g., footnoted retesting and bi-directional sequence analysis for false positives/negatives). For example, "Adenoviruses were identified in 13/14 false positive specimens using bi-directional sequence analysis." This implies adjudication by molecular methods (PCR and sequencing) was used to resolve discrepancies or to further confirm findings where FilmArray differed from the initial comparator method.
- For Influenza A subtypes and other viruses (Rhinovirus, etc.), the ground truth was already a composite of PCR and bi-directional sequencing, suggesting internal resolution of complex cases within that ground truth establishment process.
The "archived specimen study" explicitly states that the presence or absence of the analyte was confirmed using analyte-specific PCR and bi-directional sequencing prior to FilmArray RP testing, which serves as a highly robust ground truth.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No. The study is a diagnostic performance evaluation comparing the device (FilmArray RP) against established laboratory methods (viral culture, PCR, sequencing) for detecting respiratory pathogens. It does not involve human readers interpreting results with or without AI assistance, or quantify the effect size of human improvement. This is a standalone diagnostic test evaluation.
6. Standalone Performance Study (Algorithm Only)
Yes, the clinical performance studies (prospective and archived specimen) describe the standalone performance of the FilmArray RP system. The device is automated with "automated test interpretation and report generation" where the "User cannot access raw data" (Table 3), indicating the algorithm's interpretation delivers the final result.
7. Type of Ground Truth Used
The ground truth used is a combination of:
- Expert Consensus/Reference Methods (Viral Culture + DFA, PCR + Bi-directional Sequencing): This reflects standard laboratory diagnostic methods considered to be gold standards or highly accurate comparators at the time.
- "True" positives/negatives were determined by these reference methods.
- For discrepancies, especially for "false positives" or "false negatives" where the FilmArray RP result differed from the initial comparator, further molecular analysis (bi-directional sequence analysis) was often performed, indicating a robust process to confirm ground truth when needed.
8. Sample Size for the Training Set
The document explicitly describes studies for clinical performance evaluation and analytical validation (LoD, inclusivity, specificity, precision, interference). It does not provide details on a specific "training set" or its size, as would be typical for machine learning models. This suggests that the FilmArray RP system's interpretive algorithm (which automatically interprets melting curve analysis results) was developed and validated internally using methods not detailed in this summary (e.g., extensive analytical testing, algorithm tuning, and potentially internal datasets before these formal clinical studies). The "analytical studies" sections (LoD, inclusivity, etc.) would contribute to refining and confirming the algorithm's performance.
9. How the Ground Truth for the Training Set Was Established
Since a "training set" is not explicitly defined or detailed in this summary, the method for establishing its ground truth is also not provided. However, based on the analytical studies described:
- LoD (Limit of Detection): Established by testing limiting dilutions of live, quantified viruses (or clinical specimens in one case) where the presence and concentration are precisely known.
- Analytical Reactivity (Inclusivity): Evaluated using known strains/isolates, often quantified. Bi-directional sequencing and bioinformatics (in silico analyses) were used to confirm specific viral identities and predict reactivity.
- Analytical Specificity (Cross-reactivity and Exclusivity): Tested against high concentrations of known interfering organisms or substances.
These analytical studies established ground truth for known viral identities and concentrations under controlled conditions, which would underpin the fundamental ability of the algorithm to accurately detect and identify targets. While not a "training set" in the modern ML sense, these analytical validations serve a similar purpose in confirming the algorithm's foundational accuracy against known truths.
§ 866.3980 Respiratory viral panel multiplex nucleic acid assay.
(a)
Identification. A respiratory viral panel multiplex nucleic acid assay is a qualitative in vitro diagnostic device intended to simultaneously detect and identify multiple viral nucleic acids extracted from human respiratory specimens or viral culture. The detection and identification of a specific viral nucleic acid from individuals exhibiting signs and symptoms of respiratory infection aids in the diagnosis of respiratory viral infection when used in conjunction with other clinical and laboratory findings. The device is intended for detection and identification of a combination of the following viruses:(1) Influenza A and Influenza B;
(2) Influenza A subtype H1 and Influenza A subtype H3;
(3) Respiratory Syncytial Virus subtype A and Respiratory Syncytial Virus subtype B;
(4) Parainfluenza 1, Parainfluenza 2, and Parainfluenza 3 virus;
(5) Human Metapneumovirus;
(6) Rhinovirus; and
(7) Adenovirus.
(b)
Classification. Class II (special controls). The special controls are:(1) FDA's guidance document entitled “Class II Special Controls Guidance Document: Respiratory Viral Panel Multiplex Nucleic Acid Assay;”
(2) For a device that detects and identifies Human Metapneumovirus, FDA's guidance document entitled “Class II Special Controls Guidance Document: Testing for Human Metapneumovirus (hMPV) Using Nucleic Acid Assays;” and
(3) For a device that detects and differentiates Influenza A subtype H1 and subtype H3, FDA's guidance document entitled “Class II Special Controls Guidance Document: Testing for Detection and Differentiation of Influenza A Virus Subtypes Using Multiplex Nucleic Acid Assays.” See § 866.1(e) for the availability of these guidance documents.