(110 days)
The BinaxNOW® Influenza A & B Test is an in vitro immunochromatographic assay for the qualitative detection of influenza A and B nucleoprotein antigens in nasopharyngeal (NP) swab and nasal wash/aspirate specimens. It is intended to aid in the rapid differential diagnosis of influenza A and B viral infections. Negative test results should be confirmed by culture.
The BinaxNOW® Influenza A & B Test is an immunochromatographic membrane assay that uses highly sensitive monoclonal antibodies to detect influenza type A & B nucleoprotein antigens in nasopharyngeal specimens. These antibodies and a control antibody are immobilized onto a membrane support as three distinct lines and combined with other reagents/pads to construct a test strip. This test strip is mounted inside a cardboard, book-shaped hinged test device. Swab specimens require a sample preparation step, in which the sample is eluted off the swab into elution solution or transport media. Nasal wash/aspirate samples require no preparation. Sample is added to the top of the test strip and the test device is closed. Test results are interpreted at 15 minutes based on the presence or absence of pink-to-purple colored Sample Lines. The blue Control Line turns pink in a valid assay.
Here's a breakdown of the acceptance criteria and study details based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state pre-defined acceptance criteria in terms of numerical thresholds for sensitivity and specificity. Instead, it demonstrates performance by comparing the new BinaxNOW® Influenza A & B Test to existing predicate devices (Binax NOW® Flu A Test and Binax NOW® Flu B Test) and viral culture. The key indication of "acceptance" is the determination of "substantial equivalence" to the predicate devices by the FDA.
Based on the performance data presented, here's a summary:
| Performance Metric | Target/Comparison for "Acceptance" | Reported Device Performance (BinaxNOW® Influenza A & B Test) |
|---|---|---|
| Clinical Performance | Equivalent to individual NOW® Flu A and Flu B Tests | Vs. NOW® Flu A Test (for Influenza A): |
- Sensitivity: 100%
- Specificity: 96%
Vs. NOW® Flu B Test (for Influenza B): - Sensitivity: 93%
- Specificity: 97% |
| | Compared to viral culture (historical data from original A & B tests) | Historical Data (from original A & B tests vs. viral culture, 2002 study): - Flu A Sensitivity (nasal wash): 82%
- Flu A Sensitivity (NP swab): 78%
- Flu B Sensitivity (nasal wash): 71%
- Flu B Sensitivity (NP swab): 58%
- Specificity (washes and swabs): 92% to 97% |
| Analytical Sensitivity| Equivalent to individual NOW® Flu A and Flu B Tests | LOD for Flu A/Beijing: 1.03 x 10^2 ng/ml
LOD for Flu B/Harbin: 6.05 x 10^1 ng/ml
"Cutoff" sample detection rates comparable to predicate devices (50% for Flu A, 46% vs. 10% for Flu B) |
| Reactivity | Positive detection for common influenza A and B strains | Positive detection for 7 live influenza A strains and 5 live influenza B strains at various concentrations. |
| Analytical Specificity / Cross-Reactivity | No cross-reactivity with common respiratory microorganisms | No cross-reactivity with 27 bacteria, 8 viruses, and 1 yeast. |
| Interfering Substances| No interference with test interpretation | No interference with listed substances at specified concentrations (except 1% whole blood interfering with Flu A LOD negative samples). |
| Transport Media | No impact on test performance | Media alone tested negative; media inoculated with LOD levels tested positive. |
| Reproducibility | High agreement between runs, operators, and sites | 97% agreement with expected test results across multiple runs, operators, and 3 sites. |
2. Sample Sizes Used for the Test Set and Data Provenance
-
Clinical Sample Comparison (BinaxNOW® A & B vs. individual NOW® A & B tests):
- Influenza A comparison: 306 retrospective frozen clinical samples.
- Influenza B comparison: 303 retrospective frozen clinical samples.
- Data Provenance: Retrospective frozen clinical samples collected from symptomatic patients at multiple physician offices, clinics, and hospitals in the Southern, Northeastern, and Midwestern regions of the United States, and one hospital in Sweden.
-
Original Multi-site Prospective Clinical Study (comparing individual NOW® Flu A & B Tests to viral culture, 2002):
- 191 nasal wash specimens
- 182 nasopharyngeal (NP) swab specimens
- Data Provenance: Multi-center prospective study during the 2002 flu season at physician offices and clinics located in the United States.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The document does not specify the number or qualifications of experts used to establish the ground truth for the clinical comparison directly assessing the BinaxNOW® Influenza A & B Test.
-
For the comparison against the predicate devices: The predicate devices (individual NOW® Flu A and NOW® Flu B Tests) were used as the reference standard (ground truth). The ground truth for these predicate devices themselves would have been established historically (likely via viral culture).
-
For the historical 2002 multi-site prospective clinical study: The ground truth was viral culture, which is considered an objective laboratory method, not reliant on expert interpretation of the rapid test results.
4. Adjudication Method for the Test Set
The document does not describe an adjudication method for the clinical test sets in terms of resolving discrepancies between readers or between the device and ground truth. The comparisons are presented as direct measures against a reference standard (predicate devices or viral culture).
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and the Effect Size of How Much Human Readers Improve with AI vs. Without AI Assistance
- No, an MRMC comparative effectiveness study was not done as described in the context of assistance from AI.
- This device is a rapid diagnostic test (immunochromatographic assay), not an AI-powered diagnostic imaging or interpretation system. The "readers" for the analytical sensitivity experiment were "operators" interpreting the device results, not expert diagnosticians being assisted by AI.
6. If a Standalone (i.e. Algorithm Only Without Human-in-the-Loop Performance) Was Done
- Yes, standalone performance was assessed in the context of a rapid diagnostic test. The results (sensitivity, specificity) presented for the BinaxNOW® Influenza A & B Test were generated by having operators read the test strip, not by integrating it with human interpretive assistance beyond the basic instruction of how to read the pink-to-purple lines.
- The "Analytical Sensitivity Comparison" section involved 12 different operators interpreting devices for LOD and cutoff levels. This is a form of standalone performance evaluation for the test itself.
7. The Type of Ground Truth Used
- Clinical Sample Comparison (BinaxNOW® A & B vs. individual NOW® A & B tests): The ground truth was the result from the predicate devices (Binax NOW® Flu A Test and Binax NOW® Flu B Test). This implies that the predicate devices were considered the accepted standard for influenza detection in these samples.
- Original Multi-site Prospective Clinical Study (of individual NOW® Flu A & B Tests): The ground truth was viral culture. Viral culture is generally considered a gold standard for influenza diagnosis.
8. The Sample Size for the Training Set
The document does not specify a training set in the context of machine learning or algorithm development. This device is an immunochromatographic assay, which is a chemical and biological test, not typically "trained" in the way an AI algorithm is. The "development" of the test would involve optimization of its biological components and chemical reactions.
9. How the Ground Truth for the Training Set Was Established
As there is no mention of a training set for an algorithm, this question is not applicable. The development of the immunochromatographic assay relies on chemical and biological principles and optimization, not on a "training set" with established ground truth in the AI sense.
§ 866.3328 Influenza virus antigen detection test system.
(a)
Identification. An influenza virus antigen detection test system is a device intended for the qualitative detection of influenza viral antigens directly from clinical specimens in patients with signs and symptoms of respiratory infection. The test aids in the diagnosis of influenza infection and provides epidemiological information on influenza. Due to the propensity of the virus to mutate, new strains emerge over time which may potentially affect the performance of these devices. Because influenza is highly contagious and may lead to an acute respiratory tract infection causing severe illness and even death, the accuracy of these devices has serious public health implications.(b)
Classification. Class II (special controls). The special controls for this device are:(1) The device's sensitivity and specificity performance characteristics or positive percent agreement and negative percent agreement, for each specimen type claimed in the intended use of the device, must meet one of the following two minimum clinical performance criteria:
(i) For devices evaluated as compared to an FDA-cleared nucleic acid based-test or other currently appropriate and FDA accepted comparator method other than correctly performed viral culture method:
(A) The positive percent agreement estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 80 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 70 percent.
(B) The negative percent agreement estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 95 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 90 percent.
(ii) For devices evaluated as compared to correctly performed viral culture method as the comparator method:
(A) The sensitivity estimate for the device when testing for influenza A must be at the point estimate of at least 90 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 80 percent. The sensitivity estimate for the device when testing for influenza B must be at the point estimate of at least 80 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 70 percent.
(B) The specificity estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 95 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 90 percent.
(2) When performing testing to demonstrate the device meets the requirements in paragraph (b)(1) of this section, a currently appropriate and FDA accepted comparator method must be used to establish assay performance in clinical studies.
(3) Annual analytical reactivity testing of the device must be performed with contemporary influenza strains. This annual analytical reactivity testing must meet the following criteria:
(i) The appropriate strains to be tested will be identified by FDA in consultation with the Centers for Disease Control and Prevention (CDC) and sourced from CDC or an FDA-designated source. If the annual strains are not available from CDC, FDA will identify an alternative source for obtaining the requisite strains.
(ii) The testing must be conducted according to a standardized protocol considered and determined by FDA to be acceptable and appropriate.
(iii) By July 31 of each calendar year, the results of the last 3 years of annual analytical reactivity testing must be included as part of the device's labeling. If a device has not been on the market long enough for 3 years of annual analytical reactivity testing to have been conducted since the device received marketing authorization from FDA, then the results of every annual analytical reactivity testing since the device received marketing authorization from FDA must be included. The results must be presented as part of the device's labeling in a tabular format, which includes the detailed information for each virus tested as described in the certificate of authentication, either by:
(A) Placing the results directly in the device's § 809.10(b) of this chapter compliant labeling that physically accompanies the device in a separate section of the labeling where the analytical reactivity testing data can be found; or
(B) In the device's label or in other labeling that physically accompanies the device, prominently providing a hyperlink to the manufacturer's public Web site where the analytical reactivity testing data can be found. The manufacturer's home page, as well as the primary part of the manufacturer's Web site that discusses the device, must provide a prominently placed hyperlink to the Web page containing this information and must allow unrestricted viewing access.
(4) If one of the actions listed at section 564(b)(1)(A)-(D) of the Federal Food, Drug, and Cosmetic Act occurs with respect to an influenza viral strain, or if the Secretary of Health and Human Services (HHS) determines, under section 319(a) of the Public Health Service Act, that a disease or disorder presents a public health emergency, or that a public health emergency otherwise exists, with respect to an influenza viral strain:
(i) Within 30 days from the date that FDA notifies manufacturers that characterized viral samples are available for test evaluation, the manufacturer must have testing performed on the device with those viral samples in accordance with a standardized protocol considered and determined by FDA to be acceptable and appropriate. The procedure and location of testing may depend on the nature of the emerging virus.
(ii) Within 60 days from the date that FDA notifies manufacturers that characterized viral samples are available for test evaluation and continuing until 3 years from that date, the results of the influenza emergency analytical reactivity testing, including the detailed information for the virus tested as described in the certificate of authentication, must be included as part of the device's labeling in a tabular format, either by:
(A) Placing the results directly in the device's § 809.10(b) of this chapter compliant labeling that physically accompanies the device in a separate section of the labeling where analytical reactivity testing data can be found, but separate from the annual analytical reactivity testing results; or
(B) In a section of the device's label or in other labeling that physically accompanies the device, prominently providing a hyperlink to the manufacturer's public Web site where the analytical reactivity testing data can be found. The manufacturer's home page, as well as the primary part of the manufacturer's Web site that discusses the device, must provide a prominently placed hyperlink to the Web page containing this information and must allow unrestricted viewing access.