(80 days)
The BD Veritor™ System for Rapid Detection of Flu A+B is a rapid chromatographic immunoassay for the direct and qualitative detection of influenza A and B viral nucleoprotein antigens from nasopharyngeal wash/aspirate and swab samples in transport media from symptomatic patients. The BD Veritor System for Rapid Detection of Flu A+B is a differentiated test, such that influenza A viral antigens can be distinguished from influenza B viral antigens from a single processed sample using a single device. The test is to be used as an aid in the diagnosis of influenza A and B viral infections. A negative test is presumptive and it is recommended that these results be confirmed by viral culture or an FDA-cleared influenza A and B molecular assay. Negative test results do not preclude influenza viral infection and should not be used as the sole basis for treatment or other patient management decisions. The test is not intended to detect influenza C antigens.
The BD Flu A+B test is a chromatographic assay to qualitatively detect influenza A and B viral antigens in samples processed from respiratory specimens. The processed specimen is added to the test device where influenza A or influenza B viral antigens bind to antiinfluenza antibodies conjugated to detector particles on the A+B test strip. The antigenconjugate complex migrates across the test strip to the reaction area and is captured by an antibody line on the membrane. Results are interpreted by the BD Veritor™ System Reader, a portable electronic device which uses a reflectance-based measurement method to evaluate the line signal intensities on the assay test strip, and applies specific algorithms to determine the presence or absence of any target analyte(s). A liquid crystal display (LCD) on the instrument communicates the results to the operator.
Here's a breakdown of the acceptance criteria and the study details for the BD Veritor™ System for Rapid Detection of Flu A+B, based on the provided document:
1. Table of Acceptance Criteria and Reported Device Performance
The document doesn't explicitly state "acceptance criteria" numerical targets in the same way a device like an imaging AI might. However, performance metrics for clinical studies are presented as primary indicators of the device's capability. The "reference method" of PCR is the de facto gold standard against which the device's accuracy is measured. Implicitly, these reported performance values are what were deemed acceptable for FDA clearance.
| Metric | Acceptance Criteria (Implicit from Context) | Reported Device Performance (NP Swab in Transport Media - U.S. and Japan Combined) |
|---|---|---|
| Influenza A: | ||
| Positive Percent Agreement (PPA) / Sensitivity | Adequate agreement with FDA-cleared molecular assay (PCR) | 81.3% (95% CI: 70.0%, 88.9%) |
| Negative Percent Agreement (NPA) / Specificity | Adequate agreement with FDA-cleared molecular assay (PCR) | 97.4% (94.4%, 98.8%) |
| Influenza B: | ||
| Positive Percent Agreement (PPA) / Sensitivity | Adequate agreement with FDA-cleared molecular assay (PCR) | 85.6% (76.8%, 91.4%) |
| Negative Percent Agreement (NPA) / Specificity | Adequate agreement with FDA-cleared molecular assay (PCR) | 99.0% (96.5%, 99.7%) |
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size:
- Combined U.S. and Japan (NP Swab in Transport Media): 292 samples
- U.S. Only (NP Swab in Transport Media): 217 prospective specimens (201 evaluable after exclusions)
- Japan Only (NP Swab in Transport Media): 93 prospective specimens (91 evaluable after exclusions)
- Data Provenance:
- Countries of Origin: United States (six clinical trial sites) and Japan (five clinical sites).
- Retrospective or Prospective: The clinical studies were prospective. This is explicitly stated: "A total of 217 prospective specimens were evaluated..." for the U.S. study, and "A total of 93 prospective specimens were evaluated..." for the Japan study.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
- The document does not explicitly state the number of experts used to establish the ground truth or their qualifications.
- The ground truth was established by an "FDA-cleared influenza A and B molecular assay (PCR)." This means the ground truth was based on the results of a validated laboratory test, not on human expert consensus or interpretation of images. Therefore, the concept of "experts establishing ground truth" in the traditional sense (e.g., radiologists, pathologists) is not directly applicable here. The expertise lies in the validated PCR method.
4. Adjudication Method for the Test Set
- The document does not describe an adjudication method for the test set.
- Since the ground truth was derived from an FDA-cleared molecular assay (PCR), human adjudication as typically understood (e.g., 2+1, 3+1 consensus reads) would not be part of establishing the ground truth for this type of device. The PCR result is the definitive reference.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, and the effect size of how much human readers improve with AI vs without AI assistance
- No, an MRMC comparative effectiveness study was not performed.
- This study evaluates a standalone diagnostic device (an in-vitro diagnostic test) directly against a reference laboratory method (PCR), not a device designed to assist human readers or interpreters. Therefore, the concept of human readers improving with or without AI assistance is not applicable to this submission.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done
- Yes, a standalone performance study was done.
- The BD Veritor™ System for Rapid Detection of Flu A+B test is interpreted by the "BD Veritor™ System Reader, a portable electronic device which uses a reflectance-based measurement method to evaluate the line signal intensities on the assay test strip, and applies specific algorithms to determine the presence or absence of any target analyte(s)." The performance data (PPA, NPA) directly reflects the output of this automated system as compared to the PCR reference, making it a standalone evaluation.
7. The Type of Ground Truth Used
- The ground truth used was an FDA-cleared influenza A and B molecular assay (PCR). This is a laboratory-based, objective measure, often considered a highly accurate gold standard for viral detection.
8. The Sample Size for the Training Set
- The document does not explicitly state a separate "training set" sample size for the device's algorithm.
- For in-vitro diagnostics (IVDs) like this, algorithm development typically involves laboratory-based studies (e.g., analytical sensitivity, specificity, cross-reactivity, interfering substances, reproducibility) using characterized samples and controlled experiments, rather than a distinct "training set" of patient data in the way a machine learning algorithm might be trained. The summary provides details on:
- Analytical Sensitivity (LOD): Used 7 influenza strains (4 Flu A, 3 Flu B) tested with 20-60 replicates each.
- Analytical Specificity: 52 influenza viral strains tested in triplicate.
- Cross Reactivity: 51 microorganisms tested in triplicate.
- Reproducibility: 30 simulated influenza A or B samples tested at three sites.
- The "clinical performance data" presented are effectively the test set data, used to evaluate the final, developed device.
9. How the Ground Truth for the Training Set Was Established
- As noted above, the concept of a "training set" in the machine learning sense is not explicitly presented for this IVD. Instead, the device's detection algorithms and parameters would have been developed and refined using controlled laboratory studies.
- For analytical sensitivity (LOD), the ground truth for determining the lowest detectable concentration for each strain would have been based on establishing known concentrations of the target virus.
- For analytical specificity and cross-reactivity, the ground truth would have been based on the known presence or absence of specific influenza strains or other microorganisms in the tested samples.
- These rigorous laboratory studies ensure the underlying scientific validity and robust performance of the detection mechanism and its associated algorithm.
§ 866.3328 Influenza virus antigen detection test system.
(a)
Identification. An influenza virus antigen detection test system is a device intended for the qualitative detection of influenza viral antigens directly from clinical specimens in patients with signs and symptoms of respiratory infection. The test aids in the diagnosis of influenza infection and provides epidemiological information on influenza. Due to the propensity of the virus to mutate, new strains emerge over time which may potentially affect the performance of these devices. Because influenza is highly contagious and may lead to an acute respiratory tract infection causing severe illness and even death, the accuracy of these devices has serious public health implications.(b)
Classification. Class II (special controls). The special controls for this device are:(1) The device's sensitivity and specificity performance characteristics or positive percent agreement and negative percent agreement, for each specimen type claimed in the intended use of the device, must meet one of the following two minimum clinical performance criteria:
(i) For devices evaluated as compared to an FDA-cleared nucleic acid based-test or other currently appropriate and FDA accepted comparator method other than correctly performed viral culture method:
(A) The positive percent agreement estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 80 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 70 percent.
(B) The negative percent agreement estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 95 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 90 percent.
(ii) For devices evaluated as compared to correctly performed viral culture method as the comparator method:
(A) The sensitivity estimate for the device when testing for influenza A must be at the point estimate of at least 90 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 80 percent. The sensitivity estimate for the device when testing for influenza B must be at the point estimate of at least 80 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 70 percent.
(B) The specificity estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 95 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 90 percent.
(2) When performing testing to demonstrate the device meets the requirements in paragraph (b)(1) of this section, a currently appropriate and FDA accepted comparator method must be used to establish assay performance in clinical studies.
(3) Annual analytical reactivity testing of the device must be performed with contemporary influenza strains. This annual analytical reactivity testing must meet the following criteria:
(i) The appropriate strains to be tested will be identified by FDA in consultation with the Centers for Disease Control and Prevention (CDC) and sourced from CDC or an FDA-designated source. If the annual strains are not available from CDC, FDA will identify an alternative source for obtaining the requisite strains.
(ii) The testing must be conducted according to a standardized protocol considered and determined by FDA to be acceptable and appropriate.
(iii) By July 31 of each calendar year, the results of the last 3 years of annual analytical reactivity testing must be included as part of the device's labeling. If a device has not been on the market long enough for 3 years of annual analytical reactivity testing to have been conducted since the device received marketing authorization from FDA, then the results of every annual analytical reactivity testing since the device received marketing authorization from FDA must be included. The results must be presented as part of the device's labeling in a tabular format, which includes the detailed information for each virus tested as described in the certificate of authentication, either by:
(A) Placing the results directly in the device's § 809.10(b) of this chapter compliant labeling that physically accompanies the device in a separate section of the labeling where the analytical reactivity testing data can be found; or
(B) In the device's label or in other labeling that physically accompanies the device, prominently providing a hyperlink to the manufacturer's public Web site where the analytical reactivity testing data can be found. The manufacturer's home page, as well as the primary part of the manufacturer's Web site that discusses the device, must provide a prominently placed hyperlink to the Web page containing this information and must allow unrestricted viewing access.
(4) If one of the actions listed at section 564(b)(1)(A)-(D) of the Federal Food, Drug, and Cosmetic Act occurs with respect to an influenza viral strain, or if the Secretary of Health and Human Services (HHS) determines, under section 319(a) of the Public Health Service Act, that a disease or disorder presents a public health emergency, or that a public health emergency otherwise exists, with respect to an influenza viral strain:
(i) Within 30 days from the date that FDA notifies manufacturers that characterized viral samples are available for test evaluation, the manufacturer must have testing performed on the device with those viral samples in accordance with a standardized protocol considered and determined by FDA to be acceptable and appropriate. The procedure and location of testing may depend on the nature of the emerging virus.
(ii) Within 60 days from the date that FDA notifies manufacturers that characterized viral samples are available for test evaluation and continuing until 3 years from that date, the results of the influenza emergency analytical reactivity testing, including the detailed information for the virus tested as described in the certificate of authentication, must be included as part of the device's labeling in a tabular format, either by:
(A) Placing the results directly in the device's § 809.10(b) of this chapter compliant labeling that physically accompanies the device in a separate section of the labeling where analytical reactivity testing data can be found, but separate from the annual analytical reactivity testing results; or
(B) In a section of the device's label or in other labeling that physically accompanies the device, prominently providing a hyperlink to the manufacturer's public Web site where the analytical reactivity testing data can be found. The manufacturer's home page, as well as the primary part of the manufacturer's Web site that discusses the device, must provide a prominently placed hyperlink to the Web page containing this information and must allow unrestricted viewing access.