(249 days)
The Accula Flu A/Flu B Test performed on the Accula Dock is a molecular in vitro diagnostic test utilizing polymerase chain reaction (PCR) and lateral flow technologies for the qualitative, visual detection and differentiation of influenza A and influenza B viral RNA. The Accula Flu B Test uses a nasal swab specimen collected from patients with signs and symptoms of respiratory infection. The Accula Flu A/Flu B assay is intended as an aid in the diagnosis of influenza infection in conjunction with clinical and epidemiological risk factors. The assay is not intended to detect the presence of influenza C virus.
Negative results do not preclude influenza virus infection and should not be used as the sole basis for treatment or other patient management decisions.
Performance characteristics for influenza A were established during the 2016-2017 influenza season. When other influenza A viruses are emerging, performance characteristics may vary.
If infection with a novel influenza A virus is suspected based on current clinical and epidemiological screening criteria recommended by public health authorities, specimens should be collected with appropriate infection control precautions for novel virulent influenza viruses and sent to state or local public health department for testing. Viral culture should not be attempted in these cases unless a BSL 3+ facility is available to receive and culture specimens.
The Accula Flu A/Flu B Test is a semi-automated, colorimetric, multiplex reverse-transcription polymerase chain reaction (RT-PCR) nucleic acid amplification test to qualitatively detect influenza A and B viral RNA from unprocessed nasal swabs that have not undergone prior nucleic acid extraction. The system integrates nucleic acid extraction, reverse transcription, a novel Mesa Biotech PCR nucleic acid amplification technology named OscARTM, and hybridization-based visual detection into a completely self-contained and automated system. The Accula Flu A/Flu B system consists of a small reusable Dock to drive the automated testing process, and a single-use disposable test cassette that contains all the enzymes and reagents.
The Accula Dock is an electronic module which executes in vitro diagnostic tests on compatible Mesa Biotech Test Cassettes. It consists of an electro-mechanical interface to a single Test Cassette. The Dock contains all electrical systems, controls and logic necessary to orchestrate in vitro diagnostic tests within the inserted Test Cassette.
Upon insertion of a Test Cassette, the Dock will detect and identify the Cassette type. After the user transfers a clinical test sample into the Cassette and closes the Dock lid, embedded firmware in the Dock will control fluid flow of the sample into the various chambers of the Cassette, apply controlled voltage signals to the various Cassette heaters (monitored by sensors within the Dock), and provide visual status to the user with critical information such as estimated time to read, and various error states, should they be encountered.
Here's a breakdown of the acceptance criteria and study details for the Accula Flu A/Flu B Test, based on the provided document:
Acceptance Criteria and Device Performance
| Acceptance Criteria Category | Specific Metric | Acceptance Criteria (Implicit) | Reported Device Performance (Accula Flu A/Flu B Test) |
|---|---|---|---|
| CLIA-Waived Clinical Study | Influenza A: | ||
| Sensitivity | Not explicitly stated as a numerical acceptance criterion, but results are compared to an FDA-cleared molecular comparator. | 97% (95% CI: 94.4% - 98.4%) against comparator | |
| Specificity | Not explicitly stated as a numerical acceptance criterion, but results are compared to an FDA-cleared molecular comparator. | 94% (95% CI: 92.0% - 95.1%) against comparator | |
| Influenza B: | |||
| Sensitivity | Not explicitly stated as a numerical acceptance criterion, but results are compared to an FDA-cleared molecular comparator. | 94% (95% CI: 88.7% - 97.0%) against comparator | |
| Specificity | Not explicitly stated as a numerical acceptance criterion, but results are compared to an FDA-cleared molecular comparator. | 99% (95% CI: 97.9% - 99.3%) against comparator | |
| Reproducibility | Percent Agreement (Overall) | Not explicitly stated as a numerical acceptance criterion, but "Agreement was 100% across all sites, operators and days." (for specific panel samples) | 100% (Overall for pre-defined panel samples: Flu A Low/Mod Positive, Flu B Low/Mod Positive, True Negative) |
| Limit of Detection (LoD) | Detection Rate | ≥ 95% detection at the determined LoD concentration | Confirmed for Influenza A/California/07/2009 (H1N1), A/Texas/50/2012 (H3N2), B/Nevada/3/2011 (Victoria), B/Massachusetts/2/2012 (Yamagata) |
| Analytical Reactivity (Inclusivity) | Detection Rate | 100% detection for 2x LoD concentration for all tested strains | 100% (All 23 influenza strains detected at approx. 2x LoD) |
| Analytical Specificity (Cross-Reactivity) | No false positives | No false positives at tested concentrations | 100% (All 33 exclusivity organisms were negative) |
| Interfering Substances | No negative effect on performance | 100% agreement with expected results in presence of interfering substances | 100% agreement (3/3) for all tested targets and interfering substances |
| Performance Near Cut-Off (CLIA-Waived Study) | Agreement (Low Positive & Negative) | Not explicitly stated, but high agreement expected for untrained users | Low A Positive: 97% (58/60) |
| Low B Positive: 97% (58/60) | |||
| Negative: 100% (59/59) |
Study Details
-
Sample size used for the test set and the data provenance:
- Clinical Test Set: 1258 evaluable specimens (out of 1331 enrolled subjects).
- Provenance: Prospective clinical study conducted during the 2016-2017 flu season in the U.S. Nasal swabs were collected from patients presenting with flu-like symptoms.
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- The document does not specify the number of experts or their qualifications for establishing the initial ground truth.
- The comparator method used for the clinical study was an "FDA-cleared molecular influenza assay" performed at two central laboratories.
- For discrepant results, an "alternative FDA-cleared molecular assay" was used at the reference laboratory for analysis. This implies the ground truth for clinical performance was established using the results of these molecular assays, rather than expert consensus on individual cases.
-
Adjudication method (e.g. 2+1, 3+1, none) for the test set:
- The method described is: "All discrepant results were analyzed using an alternative FDA-cleared molecular assay at the reference laboratory." This is a form of discrepant analysis where a third, presumably more definitive, method is used to resolve conflicts between the investigational device and the primary comparator.
-
If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No, an MRMC comparative effectiveness study involving human readers and AI assistance was not done.
- This device is an in vitro diagnostic test (a molecular assay) and does not involve human interpretation assisted by AI. The product itself processes samples and provides a qualitative, visual result. The "CLIA Waiver Studies" section refers to performance by "untrained intended operators" but this is to assess the usability and accuracy of the device itself by non-laboratory personnel, not human readers interpreting images with AI.
-
If a standalone (i.e. algorithm only without human-in-the loop performance) was done:
- Yes, the primary clinical performance and analytical studies (LoD, inclusivity, cross-reactivity, interfering substances) effectively represent "standalone" performance of the device's diagnostic capability, as it's a fully automated molecular test providing a visual result. The "CLIA Waiver Studies" assessed performance of the device when used by non-laboratory personnel, but the output is still generated by the device's inherent algorithms and reagents.
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
- For the clinical performance evaluation: The ground truth was established by an FDA-cleared molecular influenza assay, with discrepant results being resolved by an "alternative FDA-cleared molecular assay."
- For analytical studies (LoD, inclusivity, cross-reactivity, interfering substances): Ground truth was established by known concentrations of spiked viral strains or organisms into negative clinical matrix or buffer.
-
The sample size for the training set:
- The document does not provide information on a training set. For in vitro diagnostic devices like this one, development and validation typically involves analytical studies and clinical performance studies, rather than a machine learning "training set" in the conventional sense. The "training" for such a device is in its design and optimization of reagents and protocols.
-
How the ground truth for the training set was established:
- As no training set is mentioned or implied for this type of device, this question is not applicable based on the provided document. The device's "training" equivalent would be the R&D and optimization process, where ground truth would be based on well-characterized viral samples and synthetic controls.
§ 866.3980 Respiratory viral panel multiplex nucleic acid assay.
(a)
Identification. A respiratory viral panel multiplex nucleic acid assay is a qualitative in vitro diagnostic device intended to simultaneously detect and identify multiple viral nucleic acids extracted from human respiratory specimens or viral culture. The detection and identification of a specific viral nucleic acid from individuals exhibiting signs and symptoms of respiratory infection aids in the diagnosis of respiratory viral infection when used in conjunction with other clinical and laboratory findings. The device is intended for detection and identification of a combination of the following viruses:(1) Influenza A and Influenza B;
(2) Influenza A subtype H1 and Influenza A subtype H3;
(3) Respiratory Syncytial Virus subtype A and Respiratory Syncytial Virus subtype B;
(4) Parainfluenza 1, Parainfluenza 2, and Parainfluenza 3 virus;
(5) Human Metapneumovirus;
(6) Rhinovirus; and
(7) Adenovirus.
(b)
Classification. Class II (special controls). The special controls are:(1) FDA's guidance document entitled “Class II Special Controls Guidance Document: Respiratory Viral Panel Multiplex Nucleic Acid Assay;”
(2) For a device that detects and identifies Human Metapneumovirus, FDA's guidance document entitled “Class II Special Controls Guidance Document: Testing for Human Metapneumovirus (hMPV) Using Nucleic Acid Assays;” and
(3) For a device that detects and differentiates Influenza A subtype H1 and subtype H3, FDA's guidance document entitled “Class II Special Controls Guidance Document: Testing for Detection and Differentiation of Influenza A Virus Subtypes Using Multiplex Nucleic Acid Assays.” See § 866.1(e) for the availability of these guidance documents.