(198 days)
The BioStar® AB FLU OIA® assay is an Optical ImmunoAssay (OIA) test for the qualitative, rapid detection of influenza A and B viral antigen (nucleoprotein) from nasal aspirate, nasopharyngeal swab, throat swab, or sputum specimens. This test is intended for in vitro diagnostic use to aid in the diagnosis of influenza A and B viral infections. The AB FLU OIA test is not intended to detect influenza C. Negative test results should be confirmed by cell culture.
The AB FLU OIA test involves the extraction and detection of a protein antigen unique to influenza A or B (nucleoprotein). The Opical ImmunoAssay technology enables the direction of a physical change in the optical thickness of molecular thin films. This change is a result of antigen-antibody binding on an optical surface (silicon wafer). When extracted specimen is placed directly on the optical surface, the immobilized specific antibodies capture the antigen. After washing, the substrate is added, increasing the thickness (Mass Enhancement) of the molecular thin film. This change in thickness alters the reflected light path and is visually perceived as a color change. Slight changes in optical thickness produce a distinct visible color change. A positive result appears as a purple spot on the predominant gold background. When antigen is not present in the specimen, no binding takes place. Therefore, the optical thickness remains unchanged and the original gold indicating a negative result.
Here's an analysis of the provided text regarding the acceptance criteria and study for the BioStar® AB FLU OIA Test Kit:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state pre-defined acceptance criteria in terms of specific sensitivity and specificity thresholds. Instead, it presents the "relative sensitivity" and "relative specificity" as the results of the clinical study for different specimen types. The conclusion states that these results, combined with analytical sensitivity and specificity, demonstrate that the device is safe and effective. Therefore, the reported performance itself serves as the basis for acceptance. I will present the reported clinical performance.
| Test Parameter / Specimen Type | Reported Device Performance |
|---|---|
| Clinical Performance | |
| Nasal Aspirate (NA) | Relative Sensitivity: 88.4% |
| Relative Specificity: 69.4% | |
| Nasopharyngeal Swab (NPS) | Relative Sensitivity: 83.3% |
| Relative Specificity: 76.2% | |
| Throat Swab (TS) | Relative Sensitivity: 62.1% |
| Relative Specificity: 79.5% | |
| Sputum (SP) | Relative Sensitivity: 81.1% |
| Relative Specificity: 51.5% | |
| Analytical Sensitivity | |
| Hong Kong/68 H3N2 (A) | $2.6 \times 10^3$ TCID/test |
| Shangdong/9/93 H3N2 (A) | $1.2 \times 10^4$ TCID/test |
| Texas/36/91 H1N1 (A) | $7.1 \times 10^3$ TCID/test |
| Wuhan/359/95 H3N2 (A) | $1.3 \times 10^3$ TCID/test |
| Bayern/7/95 H1N1 (A) | $2.3 \times 10^3$ TCID/test |
| Singapore/1/57 H2N2 (A) | $5.0 \times 10^2$ TCID/test |
| Hong Kong/156/97 H5N1 (A) | $5.3 \times 10^3$ TCID/test |
| Panama/45/90 (B) | $1.3 \times 10^4$ TCID/test |
| Beijing/184/93 (B) | $9.4 \times 10^2$ TCID/test |
| Guangdong/5/94 (B) | $6.0 \times 10^3$ TCID/test |
| Victoria/2/87 (B) | $1.2 \times 10^3$ TCID/test |
| Analytical Specificity | No positive result for listed organisms/viruses |
| Interfering Substances | No decrease in signal for Influenza A or B in presence of blood, mouthwashes, throat sprays, cough lozenges, or cough/cold elixirs. No positive signal without virus present. |
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size for Test Set: 184 individuals exhibiting influenza-like symptoms.
- Data Provenance:
- Country of Origin: United States (indicated by "Rocky Mountain region," "Midwest region," and "Southwestern region" for the three study sites).
- Retrospective or Prospective: Prospective, as specimens were collected from individuals exhibiting symptoms as part of the evaluation.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The document does not specify the number or qualifications of experts used to establish the ground truth. It states that the ground truth was determined by "confirmed culture from any specimen type." This implies trained laboratory personnel performed and interpreted the cultures, but their specific qualifications (e.g., years of experience, specific certifications) are not provided.
4. Adjudication Method for the Test Set
The document does not describe a formal adjudication method (like 2+1 or 3+1) for discordant results between the device and the ground truth (cell culture). The "relative sensitivity and specificity" were calculated by comparing the AB FLU OIA assay results directly to the 14-day culture results.
5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
No, an MRMC comparative effectiveness study involving human readers and AI assistance was not performed or described. This is an in vitro diagnostic device (test kit) meant for laboratory use, not a device that directly assists human readers/clinicians in interpreting images or data. The comparison is between the test kit and a conventional laboratory method (cell culture).
6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was done
Yes, this study represents a standalone performance evaluation of the device. The AB FLU OIA assay is a laboratory test kit, and its performance was assessed independently against cell culture without direct human interpretation influencing its results.
7. The Type of Ground Truth Used
The primary ground truth used for determining clinical performance (relative sensitivity and specificity) was cell culture (specifically, "14-day culture"). For analytical sensitivity, the ground truth was also cell culture (TCID/test values estimated by inoculating dilutions into cell culture).
8. The Sample Size for the Training Set
The document does not explicitly mention a "training set" for the device, as it describes a finished diagnostic test kit rather than a machine learning algorithm that undergoes a distinct training phase with a specific dataset. The evaluation presented focuses on the device's performance characteristics.
9. How the Ground Truth for the Training Set Was Established
As no specific training set is identified in the context of machine learning, this question is not directly applicable. If considering the development process of the assay, the ground truth for establishing acceptable performance characteristics (e.g., antibody binding, optical properties) would have been determined through internal validation using known positive and negative controls, spiked samples, and reference strains, likely confirmed by gold standard methods (like cell culture or genetic sequencing) during the assay's development. However, these specific developmental "training" details are not provided in this 510(k) summary.
§ 866.3328 Influenza virus antigen detection test system.
(a)
Identification. An influenza virus antigen detection test system is a device intended for the qualitative detection of influenza viral antigens directly from clinical specimens in patients with signs and symptoms of respiratory infection. The test aids in the diagnosis of influenza infection and provides epidemiological information on influenza. Due to the propensity of the virus to mutate, new strains emerge over time which may potentially affect the performance of these devices. Because influenza is highly contagious and may lead to an acute respiratory tract infection causing severe illness and even death, the accuracy of these devices has serious public health implications.(b)
Classification. Class II (special controls). The special controls for this device are:(1) The device's sensitivity and specificity performance characteristics or positive percent agreement and negative percent agreement, for each specimen type claimed in the intended use of the device, must meet one of the following two minimum clinical performance criteria:
(i) For devices evaluated as compared to an FDA-cleared nucleic acid based-test or other currently appropriate and FDA accepted comparator method other than correctly performed viral culture method:
(A) The positive percent agreement estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 80 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 70 percent.
(B) The negative percent agreement estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 95 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 90 percent.
(ii) For devices evaluated as compared to correctly performed viral culture method as the comparator method:
(A) The sensitivity estimate for the device when testing for influenza A must be at the point estimate of at least 90 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 80 percent. The sensitivity estimate for the device when testing for influenza B must be at the point estimate of at least 80 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 70 percent.
(B) The specificity estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 95 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 90 percent.
(2) When performing testing to demonstrate the device meets the requirements in paragraph (b)(1) of this section, a currently appropriate and FDA accepted comparator method must be used to establish assay performance in clinical studies.
(3) Annual analytical reactivity testing of the device must be performed with contemporary influenza strains. This annual analytical reactivity testing must meet the following criteria:
(i) The appropriate strains to be tested will be identified by FDA in consultation with the Centers for Disease Control and Prevention (CDC) and sourced from CDC or an FDA-designated source. If the annual strains are not available from CDC, FDA will identify an alternative source for obtaining the requisite strains.
(ii) The testing must be conducted according to a standardized protocol considered and determined by FDA to be acceptable and appropriate.
(iii) By July 31 of each calendar year, the results of the last 3 years of annual analytical reactivity testing must be included as part of the device's labeling. If a device has not been on the market long enough for 3 years of annual analytical reactivity testing to have been conducted since the device received marketing authorization from FDA, then the results of every annual analytical reactivity testing since the device received marketing authorization from FDA must be included. The results must be presented as part of the device's labeling in a tabular format, which includes the detailed information for each virus tested as described in the certificate of authentication, either by:
(A) Placing the results directly in the device's § 809.10(b) of this chapter compliant labeling that physically accompanies the device in a separate section of the labeling where the analytical reactivity testing data can be found; or
(B) In the device's label or in other labeling that physically accompanies the device, prominently providing a hyperlink to the manufacturer's public Web site where the analytical reactivity testing data can be found. The manufacturer's home page, as well as the primary part of the manufacturer's Web site that discusses the device, must provide a prominently placed hyperlink to the Web page containing this information and must allow unrestricted viewing access.
(4) If one of the actions listed at section 564(b)(1)(A)-(D) of the Federal Food, Drug, and Cosmetic Act occurs with respect to an influenza viral strain, or if the Secretary of Health and Human Services (HHS) determines, under section 319(a) of the Public Health Service Act, that a disease or disorder presents a public health emergency, or that a public health emergency otherwise exists, with respect to an influenza viral strain:
(i) Within 30 days from the date that FDA notifies manufacturers that characterized viral samples are available for test evaluation, the manufacturer must have testing performed on the device with those viral samples in accordance with a standardized protocol considered and determined by FDA to be acceptable and appropriate. The procedure and location of testing may depend on the nature of the emerging virus.
(ii) Within 60 days from the date that FDA notifies manufacturers that characterized viral samples are available for test evaluation and continuing until 3 years from that date, the results of the influenza emergency analytical reactivity testing, including the detailed information for the virus tested as described in the certificate of authentication, must be included as part of the device's labeling in a tabular format, either by:
(A) Placing the results directly in the device's § 809.10(b) of this chapter compliant labeling that physically accompanies the device in a separate section of the labeling where analytical reactivity testing data can be found, but separate from the annual analytical reactivity testing results; or
(B) In a section of the device's label or in other labeling that physically accompanies the device, prominently providing a hyperlink to the manufacturer's public Web site where the analytical reactivity testing data can be found. The manufacturer's home page, as well as the primary part of the manufacturer's Web site that discusses the device, must provide a prominently placed hyperlink to the Web page containing this information and must allow unrestricted viewing access.