(113 days)
The FilmArray Respiratory Panel (RP) is a multiplexed nucleic acid test intended for use with FilmArray systems for the simultaneous qualitative detection and identification of multiple respiratory viral and bacterial nucleic acids in nasopharyngeal swabs (NPS) obtained from individuals suspected of respiratory tract infections. The following organism types and subtypes are identified using the FilmArray RP: Adenovirus 229E, Coronavirus HKU1, Coronavirus NL63, Coronavirus OC43, Human Metapneumovirus, Influenza A subtype H1, Influenza A subtype H3, Influenza A subtype 2009 H1, Influenza Virus 1, Parainfluenza Virus 2, Parainfluenza Virus 3, Parainfluenza Virus 4, Human Rhinovirus/Enterovirus, Respiratory Syncytial Virus, Bordetella pertussis, Chlamydophila pneumoniae, and Mycoplasma pneumoniae. The detection of specific viral and bacterial nucleic acids from individuals exhibiting signs and symptoms of a respiratory infection aids in the diagnosis of respiratory infection if used in conjunction with other clinical and epidemiological information. The results of this test should not be used as the sole basis for diagnosis, treatment, or other management decisions. Negative results in the setting of a respiratory illness may be due to infection with pathogens that are not detected by this test or, lower respiratory tract infection that is not detected by a nasopharyngeal swab specimen. Positive results do not rule other organisms: the agent(s) detected by the Film Array RP may not be the definite cause of disease. Additional laboratory testing (e.g. bacterial and viral culturescence, and radiography) may be necessary when evaluating a patient with possible respiratory tract infection.
Due to the small number of positive specimens collected for certain organisms during the prospective clinical study, performance characteristics for Bordetella pertussis, Coronavirus OC43, Influenza A H1, Influenza A H3, Influenza A H1-2009, Influenza B, Mycoplasma pneumoniae, Parainfluenza Virus 1, Parainfluenza Virus 2, and Parainfluenza Virus 4 were established primarily with retrospective clinical specimens. Performance characteristics for Chlamydophila pneumoniae were established primarily using contrived clinical specimens.
Due to the genetic similarity between Human Rhinovirus, the FilmArray RP cannot reliably differentiate them. A positive FilmArray RP Rhinovirus result should be followed-up using an alternate method (e.g., cell culture or sequence analysis).
The FilmArray RP assay for Coronavirus OC43 may cross-react with some isolates of Coronavirus HKU1. A dual positive result may be due to cross-reactivity or may indicate a co-infection.
Performance characteristics for Influenza A were established when Influenza A 2009 HIN1, A H1, and A H3 were the predominant Influenza A viruses in circulation. Performance of detecting Influenza A may vary if other Influenza A strains are circulating or a novel Influenza A virus emerges. If influenza A virus is suspected based on current clinical and epidemiological screening criteria recommended by public health authorities, specimens should be collected with appropriate infection control precautions for novel virulent Influenza viruses and sent to state or local health departments for testing. Viral culture should not be attempted in these cases unless a BSL 3+ facility is available to receive and culture specimens.
The FilmArray Respiratory Panel is a multiplex nucleic acid test designed to be used with FilmArray systems. The FilmArray RP pouch contains freeze-dried reagents to perform nucleic acid purification, reverse transcription, and nested, multiplex PCR with DNA melt analysis. FilmArray RP simultaneously conducts 20 tests for the identification of respiratory pathogens from nasopharyngeal swabs (NPS) obtained from individuals suspected of respiratory tract infections (Table 1). Results from the FilmArray RP test are available within about one hour.
A test is initiated by loading Hydration Solution and an unprocessed patient nasopharyngeal swab (NPS) specimen (i.e. specimen mixed with Sample Buffer) into the FilmArray RP pouch. The pouch contains all of the reagents required for specimen testing and analysis in a freezedried format; the addition of Hydration Solution and specimen/Sample Buffer Mix rehydrates the reagents. After the pouch is prepared, the FilmArray software guides the user though the steps of placing the pouch into the instrument, scanning the pouch barcode, entering the sample identification, and initiating the run.
The FilmArray instrument contains a coordinated system of inflatable bladders and seal points. which act on the pouch to control the movement of liquid between the pouch blisters. Alternatively, when a seal is placed over a connecting channel it acts as a valve to open or close a channel. In addition, electronically controlled pneumatic pistons are positioned over multiple plungers in order to deliver the rehydrated reagents into the blisters at the appropriate times. Two Peltier devices control heating and cooling of the pouch to drive the PCR reactions and the melt curve analysis.
Nucleic acid extraction occurs within the FilmArray pouch using mechanical and chemical Ivsis followed by purification using standard magnetic bead technology. After extracting and purifying nucleic acids from the unprocessed sample, a nested multiplex PCR is executed in two stages. During the first stage, a single, large volume, highly multiplexed reverse transcription PCR (rt-PCR) reaction is performed. The products from first stage PCR are then diluted and combined with a fresh, primer-free master mix and a fluorescent double stranded DNA binding dye (LC Green® Plus, BioFire Defense, LLC). The solution is then distributed to each well of the array. Array wells contain sets of primers designed specifically to amplify sequences internal to the PCR products generated during the first stage PCR reaction. The 2nd stage PCR, or nested PCR, is performed in singleplex fashion in each well of the array. At the conclusion of the 2ª0 stage PCR, the array is interrogated by melt curve analysis for the detection of signature amplicons denoting the presence of specific targets. A digital camera placed in front of the array captures fluorescent images of the PCR reactions and software interprets the data.
The FilmArray software automatically interprets the results of each DNA melt curve analysis and combines the data with the results of the internal pouch controls to provide a test result for each organism on the panel.
Here's a summary of the acceptance criteria and study details for the FilmArray Respiratory Panel (RP), based on the provided document:
The study is designed to demonstrate that the performance of the FilmArray RP on the new FilmArray 2.0 system and with the new injection vial loading method is equivalent to its performance on the original FilmArray system with syringe loading.
1. Table of Acceptance Criteria and Reported Device Performance
The document doesn't explicitly state "acceptance criteria" in a table format with pass/fail metrics. Instead, it presents performance data (Positive Percent Agreement - PPA, Negative Percent Agreement - NPA, and % Detection) and implies that 100% concordance for most analytes and overall PPA of ≥96.8% and NPA of ≥99.2% are considered acceptable. For some analytes, slightly lower, but statistically comparable, performance with overlapping 95% CIs is accepted.
For the Low Analyte study, the implied acceptance criterion is equivalent detection on all three configurations, evidenced by >95% detection or overlapping 2-sided 95% confidence intervals and comparable amplification/detection in titration series.
For the Tm values, the acceptance criterion is that the mean Tm values for all FilmArray RP assays on the modified configurations should be ±0.5°C or less compared to the same samples tested on the current configuration.
For the Reproducibility study, the implied acceptance criterion is high reproducibility and consistency with previously collected data on the single-instrument system, demonstrated by high percentage agreement with expected results and low standard deviations for Tm values.
Here's a table summarizing the reported device performance, which the manufacturer presents as evidence of meeting their equivalence criteria:
| Study/Performance Metric | Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|---|
| Clinical Performance (Comparison Studies) | ||
| Overall Positive Percent Agreement (PPA) | ≥92.0% (lower bound of 95% CI) | ≥96.8% (100% for 16/20 analytes) |
| Overall Negative Percent Agreement (NPA) | ≥99.5% (lower bound of 95% CI) | ≥99.2% (100% for 16/20 analytes) |
| System Performance (Run Failures) | Low run failure rate | Current system: 3.7%; Modified system: 0% |
| Low Analyte Study (LoD) | ||
| Detection Rate at LoD | >95% or overlapping 95% CI for all configurations | Most analytes 100%, some 75-95% with overlapping CIs |
| Tm Values (Mean Tm difference) | ≤ ±0.5°C between configurations | All configurations ≤ ±0.5°C difference |
| Reproducibility Study | ||
| % Agreement with Expected Results (Positive) | High agreement to expected, consistent with predicate | Range from 90% to 100% (mostly 100%) with tight 95% CIs |
| % Agreement with Expected Results (Negative) | 100% for all sites/systems | 100% |
| Tm Reproducibility (StDev Tm) | Low standard deviation (implying consistency) | Generally ±0.1 to ±0.5 for all analytes and configurations |
2. Sample Size Used for the Test Set and Data Provenance
Clinical Performance Study:
- Sample Size: A total of 102 specimens were selected for testing.
- Data Provenance: The specimens were:
- Previously obtained during the FilmArray RP prospective clinical evaluations.
- Supplemented with other archived specimens collected from external medical facilities and reference laboratories.
- This indicates a mix of prospective and retrospective clinical specimens from various unspecified countries/regions (implied to be from clinical facilities).
Low Analyte Study:
- Sample Size:
- Titration series: Samples at concentrations above, at, and below (10x, 1x, 0.1x, and 0.01x) LoD (specific number of samples not given, but likely multiple for each concentration and analyte).
- Additional side-by-side testing at LoD: 20 replicates per analyte on each system.
- Data Provenance: Not specified, but given the nature of the study, these were likely contrived samples made in a laboratory setting, not clinical specimens.
Reproducibility Study:
- Sample Size: Contrived nasopharyngeal swab samples spiked with four different RP analytes (Bordetella pertussis, Adenovirus Species C, Influenza A H1N1-2009, Respiratory Syncytial Virus Type A). Each analyte was evaluated at three concentrations (Negative, Low Positive, Moderate Positive).
- 90 data points per sample, per loading procedure. This equates to 90 runs for each of the 4 analytes x 3 concentrations x 2 loading procedures = 2160 individual results.
- For the single-instrument comparison, the total negative replicates were 540, and positive were 60-180 depending on the analyte.
- Data Provenance: Contrived nasopharyngeal swab samples, likely prepared in a laboratory setting.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The document does not provide information on the number or qualifications of experts used to establish the ground truth for the test sets. It refers to specimens being "originally characterized" for the clinical performance study and "expected result" for the reproducibility study. In the context of in-vitro diagnostics, the ground truth for clinical specimens is typically established by well-characterized reference methods (e.g., culture, sequencing) at the originating clinical or reference laboratories. For contrived samples, the truth is known by design.
4. Adjudication Method for the Test Set
The document does not explicitly describe an adjudication method for the test set results. For the clinical performance study, it notes "occasional discrepant results were observed where an analyte was detected by one or two out of three runs; in all cases the discrepant results were attributed to analyte levels below the limit of detection (LoD) for a particular assay in specimens that had previously been characterized as positive for that analyte, or due to a known cross-reactivity." This suggests internal review and technical explanation for discrepancies rather than an independent expert adjudication process as might be seen for imaging studies.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done
No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not done. This document describes the validation of an in vitro diagnostic device (a molecular diagnostic test), not an imaging or AI-assisted diagnostic tool that would involve human readers. The study compares the device's performance across different system configurations and loading methods, not human reader performance with or without AI assistance.
6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) Was Done
The studies described are for a standalone diagnostic device (the FilmArray Respiratory Panel cartridge and system). The system automatically interprets the results of the DNA melt curve analysis and combines data with internal controls to provide a test result. Human interaction is primarily for loading the sample and initiating the run, not for interpreting raw data or making diagnostic decisions that the device output alone would provide. Therefore, the device operates autonomously in terms of result generation.
7. The Type of Ground Truth Used
- Clinical Performance Study:
- For positive specimens: "originally characterized as positive for that analyte" or "not been reported as present in these specimens by the source laboratory." This implies ground truth was established by reference methods (e.g., other molecular tests, culture, sequencing) at the originating labs, or clinical diagnosis.
- For low-level positives/cross-reactivity: explanations are provided for discrepancies.
- Low Analyte Study: Ground truth was established by contriving samples with known concentrations of analytes.
- Reproducibility Study: Ground truth was established by contriving samples with known analytes and concentrations (Negative, Low Positive, Moderate Positive).
8. The Sample Size for the Training Set
The document does not mention a training set sample size. This is expected because the FilmArray Respiratory Panel is a molecular diagnostic assay using PCR and melt curve analysis, not an AI/machine learning algorithm that requires a separate training set for model development. The assays are designed based on known genetic sequences for the pathogens.
9. How the Ground Truth for the Training Set Was Established
As there is no explicit training set for an AI/machine learning model, this question is not applicable to the FilmArray RP device described in this document. The "training" for such a device would be the extensive R&D process to design and optimize the primers, probes, and reaction conditions for each target pathogen, along with establishing detection thresholds, likely validated against characterized reference materials and strains.
§ 866.3980 Respiratory viral panel multiplex nucleic acid assay.
(a)
Identification. A respiratory viral panel multiplex nucleic acid assay is a qualitative in vitro diagnostic device intended to simultaneously detect and identify multiple viral nucleic acids extracted from human respiratory specimens or viral culture. The detection and identification of a specific viral nucleic acid from individuals exhibiting signs and symptoms of respiratory infection aids in the diagnosis of respiratory viral infection when used in conjunction with other clinical and laboratory findings. The device is intended for detection and identification of a combination of the following viruses:(1) Influenza A and Influenza B;
(2) Influenza A subtype H1 and Influenza A subtype H3;
(3) Respiratory Syncytial Virus subtype A and Respiratory Syncytial Virus subtype B;
(4) Parainfluenza 1, Parainfluenza 2, and Parainfluenza 3 virus;
(5) Human Metapneumovirus;
(6) Rhinovirus; and
(7) Adenovirus.
(b)
Classification. Class II (special controls). The special controls are:(1) FDA's guidance document entitled “Class II Special Controls Guidance Document: Respiratory Viral Panel Multiplex Nucleic Acid Assay;”
(2) For a device that detects and identifies Human Metapneumovirus, FDA's guidance document entitled “Class II Special Controls Guidance Document: Testing for Human Metapneumovirus (hMPV) Using Nucleic Acid Assays;” and
(3) For a device that detects and differentiates Influenza A subtype H1 and subtype H3, FDA's guidance document entitled “Class II Special Controls Guidance Document: Testing for Detection and Differentiation of Influenza A Virus Subtypes Using Multiplex Nucleic Acid Assays.” See § 866.1(e) for the availability of these guidance documents.