(263 days)
The PLEX-ID Flu assay is a qualitative nucleic acid in vitro diagnostic test intended for the detection and differentiation of influenza A HINI (2009), influenza A H1N1 (seasonal), influenza A H3N2 (seasonal) and influenza B viral nucleic acids in nasopharyngeal swab specimens from patients symptomatic for respiratory tract infection. The PLEX-ID Flu assay is intended for use on the PLEX-ID System (version 1.2) as an aid in the diagnosis of influenza infection in conjunction with clinical and epidemiological information. This assay is not intended to detect influenza C virus.
Negative results do not preclude influenza virus infection and should not be used as the sole basis for diagnosis, treatment, or other patient management decisions,
If infection with a novel influenza A virus is suspected based on current clinical and epidemiological screening criteria recommended by public health authorities, specimens should be collected with appropriate infection control precautions for novel virulent influenza viruses and sent to state or local health departments for testing. Viral culture should not be attempted in these cases unless a BSL 3+ facility is available to receive and culture specimens.
The PLEX-ID Flu assay consists of the following kits:
- . PLEX-ID Flu Amplification Kit (List No. 05N21-91)
- . PLEX-ID Flu Control Kit (List No. 05N21-80)
The PLEX-ID Flu assay is a qualitative in vitro diagnostic test used for the detection and identification of influenza A and B viruses directly from human samples. First, nucleic acids are extracted from samples. Nucleic acids are then amplified via a reverse transcription polymerase chain reaction (RT-PCR). The PCR products are subsequently desalted and analyzed in a mass spectrometer to determine the base composition of the PCR products. Analysis of the base composition of PCR products is used to determine identity by comparison to a proprietary database. Desalting, mass spectrometry and data analysis are conducted on the PLEX-ID Analyzer. Reported results include an identification of influenza virus species and subtypes of influenza A virus.
Here's a breakdown of the acceptance criteria and the study details for the PLEX-ID Flu assay, based on the provided text:
1. Acceptance Criteria and Reported Device Performance
The document does not explicitly state pre-defined acceptance criteria in terms of specific sensitivity and specificity thresholds. However, the reported performance metrics (Positive Percent Agreement - PPA and Negative Percent Agreement - NPA) serve as de-facto acceptance criteria demonstrated by the device. The study design aims to show a high level of agreement with a comparator device.
| Target | Performance Metric | Reported Performance (Prospective Study) | Reported Performance (Retrospective Study) |
|---|---|---|---|
| Influenza A H1N1 (2009) | Positive % Agreement | 100.0% (95% CI: 89.3%, 100.0%) (32/32) | 98.2% (95% CI: 96.7%, 99.0%) (538/548) |
| Influenza A H1N1 (2009) | Negative % Agreement | 99.6% (95% CI: 99.1%, 99.8%) (1239/1244) | 94.3% (95% CI: 92.5%, 95.7%) (746/791) |
| Influenza A H1N1 (seasonal) | Positive % Agreement | 57.1% (95% CI: 25.0%, 84.2%) (4/7) | 96.4% (95% CI: 82.3%, 99.4%) (27/28) |
| Influenza A H1N1 (seasonal) | Negative % Agreement | 100.0% (95% CI: 99.7%, 100.0%) (1269/1269) | 100.0% (95% CI: 99.7%, 100.0%) (1311/1311) |
| Influenza A H3N2 (seasonal) | Positive % Agreement | 98.0% (95% CI: 89.3%, 99.6%) (48/49) | 100.0% (95% CI: 34.2%, 100.0%) (2/2) |
| Influenza A H3N2 (seasonal) | Negative % Agreement | 99.8% (95% CI: 99.4%, 100.0%) (1225/1227) | 100.0% (95% CI: 99.7%, 100.0%) (1337/1337) |
| Influenza B | Positive % Agreement | 100.0% (95% CI: 85.1%, 100.0%) (22/22) | 92.9% (95% CI: 77.4%, 98.0%) (26/28) |
| Influenza B | Negative % Agreement | 99.8% (95% CI: 99.4%, 100.0%) (1252/1254) | 99.9% (95% CI: 99.6%, 100.0%) (1312/1313) |
2. Sample Size Used for the Test Set and Data Provenance
- Prospective Study:
- Sample Size: 1276 unique nasopharyngeal swab specimens were included in the final analysis (starting from 1287 collected, with exclusions).
- Data Provenance: Prospectively collected from two U.S. clinical sites and three specimen suppliers from September 2009 through June 2010.
- Retrospective Study:
- Sample Size: 1339 unique specimens were included in the final analysis (starting from 1378 collected, with exclusions).
- Data Provenance: Banked specimens collected in February 2008, January 2009 through March 2009, and leftover archived specimens from April 2009 through December 2009. The country of origin for these specimens is not explicitly stated but can be inferred to be largely U.S. given the U.S. clinical sites for prospective collection.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
The ground truth was established using FDA-cleared nucleic acid assays as comparators, not human experts.
- Comparator 1: An FDA-cleared assay for the detection and discrimination of influenza A virus, influenza B virus, and Respiratory Syncytial virus nucleic acids.
- Comparator 2: An FDA-cleared assay for the detection and discrimination of influenza A virus subtypes [influenza A H1N1 (2009), influenza A H1N1 (seasonal), influenza A H3N2 (seasonal)].
Therefore, the concept of "number of experts" and their "qualifications" for ground truth establishment is not applicable in this context, as the ground truth was based on established laboratory methods.
4. Adjudication Method for the Test Set
For discrepant samples between Comparator 1 and Comparator 2, Comparator 1 vs. PLEX-ID Flu, and Comparator 2 vs. PLEX-ID Flu, PCR followed by bi-directional sequencing was used as an adjudication method. However, the results from bi-directional sequence testing were not used to calculate the Positive Percent Agreement and Negative Percent Agreement of the PLEX-ID Flu assay presented in the tables. This implies that the primary agreement calculations were against the Comparator assays, with sequencing used for further investigation of discrepancies but not as the sole ground truth for the primary performance metrics.
5. Multi Reader Multi Case (MRMC) Comparative Effectiveness Study
No, an MRMC comparative effectiveness study involving human readers with and without AI assistance was not done. This study is for an in vitro diagnostic (IVD) assay designed to detect viral nucleic acids, which does not involve human interpretation in the same way an imaging AI device would.
6. Standalone (i.e. algorithm only without human-in-the loop performance) Study
Yes, this was a standalone performance study. The PLEX-ID Flu assay automatically detects and identifies influenza viral nucleic acids based on the mass spectrometry analysis of PCR products, comparing base composition to a proprietary database. The results are reported directly by the system without human interpretation as part of the core diagnostic function. The "human-in-the-loop" concept, as typically applied to AI in imaging, is not relevant here; the device itself performs the analysis.
7. The Type of Ground Truth Used
The primary ground truth for the clinical performance evaluation was based on results from FDA-cleared comparator nucleic acid assays. For samples with discordant results between the PLEX-ID Flu assay and the comparator assays, PCR followed by bi-directional sequencing was used as an additional, higher-resolution method for further investigation and characterization of the discrepant sample's true status. However, as noted in point 4, the primary agreement metrics were calculated against the comparator assays.
8. The Sample Size for the Training Set
The document does not explicitly state the sample size for a training set. This is common for IVD assays where "training" might refer to internal assay development and optimization rather than a distinct, separate training set for a machine learning algorithm as would be described for AI devices in imaging or other fields. The performance characteristics sections focus on validation against clinical and contrived samples.
9. How the Ground Truth for the Training Set Was Established
Since a specific "training set" with established ground truth is not explicitly described in the context of an AI training paradigm, this information is not applicable in the conventional sense. The assay's "training" or development would involve molecular biology techniques, primer design, and database development, likely drawing upon known influenza sequences and characterized viral samples, rather than a separate dataset for explicit AI algorithm training. The proprietary database against which base compositions are compared would have been developed using known viral sequences and characteristics.
§ 866.3980 Respiratory viral panel multiplex nucleic acid assay.
(a)
Identification. A respiratory viral panel multiplex nucleic acid assay is a qualitative in vitro diagnostic device intended to simultaneously detect and identify multiple viral nucleic acids extracted from human respiratory specimens or viral culture. The detection and identification of a specific viral nucleic acid from individuals exhibiting signs and symptoms of respiratory infection aids in the diagnosis of respiratory viral infection when used in conjunction with other clinical and laboratory findings. The device is intended for detection and identification of a combination of the following viruses:(1) Influenza A and Influenza B;
(2) Influenza A subtype H1 and Influenza A subtype H3;
(3) Respiratory Syncytial Virus subtype A and Respiratory Syncytial Virus subtype B;
(4) Parainfluenza 1, Parainfluenza 2, and Parainfluenza 3 virus;
(5) Human Metapneumovirus;
(6) Rhinovirus; and
(7) Adenovirus.
(b)
Classification. Class II (special controls). The special controls are:(1) FDA's guidance document entitled “Class II Special Controls Guidance Document: Respiratory Viral Panel Multiplex Nucleic Acid Assay;”
(2) For a device that detects and identifies Human Metapneumovirus, FDA's guidance document entitled “Class II Special Controls Guidance Document: Testing for Human Metapneumovirus (hMPV) Using Nucleic Acid Assays;” and
(3) For a device that detects and differentiates Influenza A subtype H1 and subtype H3, FDA's guidance document entitled “Class II Special Controls Guidance Document: Testing for Detection and Differentiation of Influenza A Virus Subtypes Using Multiplex Nucleic Acid Assays.” See § 866.1(e) for the availability of these guidance documents.