(281 days)
The OSOM Influenza A&B Test is an in vitro diagnostic immunochromatographic assay intended for the qualitative detection of influenza A and influenza B viral nucleoprotein antigens from nasal swab specimens in symptomatic patients. It is intended to aid in the rapid differential diagnosis of influenza A and/or B viral infections. This test is not intended for the detection of influenza C viruses. A negative test is presumptive and it is recommended these results be confirmed by cell culture.
Cross-reactivity with respiratory viruses other than influenza viruses has not been evaluated. The user is responsible for determining the cross-reactivity of other respiratory viruses with this test.
The OSOM Influenza A&B Test consists of a test stick that separately detects influenza A and B. The test procedure requires the solubilization of the nucleoproteins from a swab by mixing the swab in Extraction Buffer. The test stick is then placed in the sample mixture, which then migrates along the membrane surface. If influenza A and/or B viral antigens are present in the sample, it will form a complex with mouse monoclonal IgG antibodies to influenza A and/or B nucleoproteins conjugated to colloidal gold. The complex will then be bound by another mouse anti-influenza A and/or B antibody coated on the nitrocellulose membrane. A pink to purple control line must appear in the control region of the stick for results to be valid. The appearance of a second and possibly a third light pink to purple line will appear in the test line region indicating an A, B or A and B positive result.
Here's an analysis of the acceptance criteria and study that proves the OSOM Influenza A&B Test meets them, based on the provided text:
1. A table of acceptance criteria and the reported device performance
The document does not explicitly state pre-defined acceptance criteria in terms of numerical thresholds for sensitivity, specificity, and agreement. However, the study results are presented as the device's performance. For the purpose of this response, I will interpret the reported performance values as implicitly meeting the unstated acceptance criteria for substantial equivalence to the predicate device.
| Metric (vs. Viral Culture) | Acceptance Criteria (Implicit) | Reported Device Performance |
|---|---|---|
| Influenza A | (Not explicitly stated) | |
| Sensitivity | Acceptable | 73.8% (95% CI 64.4% - 81.9%) |
| Specificity | Acceptable | 96.4% (95% CI 93.4% - 98.2%) |
| Agreement | Acceptable | 90.1% |
| Influenza B | (Not explicitly stated) | |
| Sensitivity | Acceptable | 60.0% (95% CI 45.2-73.6%) |
| Specificity | Acceptable | 96.4% (95% CI 93.8% - 98.1%) |
| Agreement | Acceptable | 91.6% |
Additional Performance Data:
| Study/Test | Acceptance Criteria (Implicit) | Reported Device Performance |
|---|---|---|
| Assay Reproducibility | Acceptable | |
| Overall Accuracy Flu A | Acceptable | 97% |
| Overall Accuracy Flu B | Acceptable | 94% |
| Analytical Sensitivity | Acceptable | |
| Detection Limit Influenza A | Acceptable | 4.4 x 10^4 TCID50/test |
| Detection Limit Influenza B | Acceptable | 1.44 x 10^5 TCID50/test |
| Analytical Specificity/Cross-reactivity | Acceptable | No false positives from 24/25 bacterial isolates (1 S. aureus strain provided false positive at very high concentration). All 46 influenza strains tested positive. |
| Interfering Substances | Acceptable | No effect on performance from various common medications/substances. |
2. Sample sized used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)
- Sample Size for Clinical Performance (Test Set): 383 subjects.
- 383 total samples for comparison to viral culture for both Influenza A and B.
- Of these, 132 samples were from pediatric subjects (2-19 years) and 251 samples were from adults (> 20 years).
- Data Provenance: The document does not explicitly state the country of origin or whether the study was retrospective or prospective. Given the context of a 510(k) submission to the FDA in the US, it is highly likely that the clinical study was conducted in the United States. The study involved enrollment of subjects, which suggests a prospective collection of samples for the clinical performance evaluation.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)
- The ground truth for the clinical performance study (sensitivity, specificity, agreement) was established using viral cell culture as the reference method. This is a laboratory-based method, not dependent on human expert interpretation of the final result for the ground truth. Therefore, the concept of "number of experts" and their "qualifications" for establishing the ground truth does not directly apply here in the traditional sense of image or clinical interpretation.
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set
- The document does not describe any human adjudication method for the ground truth (viral culture) or for the device's results. Viral culture results are objective laboratory findings. For the OSOM test results, it is a rapid diagnostic test with visually interpreted lines, implying a single interpretation per test, though reproducibility was assessed across different operators.
- Polymerase Chain Reaction (PCR) was performed on specimens that gave inconsistent results between the OSOM test and viral culture. This was done "for information only" and PCR was not FDA approved/cleared for this purpose at the time, meaning it was not used as a primary adjudication method for the final, reported clinical performance metrics directly, but rather for investigational purposes.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
- No, a MRMC comparative effectiveness study was not done. This device is an in vitro diagnostic immunochromatographic assay (a rapid point-of-care test), not an AI-powered diagnostic imaging or interpretation system. It does not involve human readers interpreting complex images with or without AI assistance, so the concept of "effect size of how much human readers improve with AI" is not applicable.
- However, an Assay Reproducibility study was conducted to demonstrate that the test performs acceptably in the hands of various operators (nurses, nurse practitioners, physician's office personnel). This involved multiple operators interpreting coded and masked samples, which is a form of multi-reader evaluation for the device's interpretability. The overall accuracy was 97% for Flu A and 94% for Flu B in this reproducibility study.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
- Yes, in essence, the primary clinical performance evaluation is a standalone study. The OSOM Influenza A&B Test is a lateral flow immunoassay that provides a visual reading (pink to purple lines). Although a human interprets these lines, the core "algorithm" (the immunochromatographic assay itself) operates independently. The sensitivity, specificity, and agreement reported in the "Agreement with Viral Culture" section represent the performance of the device on its own, with human interpretation assumed to be done according to instructions. The test is designed to be read directly by an operator, not to be an "AI algorithm" that outputs a result for a human to then validate or integrate into a diagnosis.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
- The primary ground truth for the clinical performance (sensitivity, specificity) was viral cell culture. This is considered a gold standard for influenza virus detection.
- For the analytical specificity and cross-reactivity studies, the ground truth was the known identity of the bacterial isolates and influenza virus strains.
8. The sample size for the training set
- The document describes the clinical performance evaluation and various analytical studies. It does not describe a separate "training set" in the context of machine learning or AI models, as this is an immunoassay. The tests performed are validations of the developed assay, not training phases for an algorithm.
9. How the ground truth for the training set was established
- As concluded in point 8, there isn't a "training set" for an AI model mentioned in the document. The development of an immunoassay like the OSOM test involves extensive laboratory work, including using known positive and negative samples, and samples spiked with varying concentrations of analytes, to optimize the assay's components and parameters (antibodies, reagents, flow characteristics, etc.). This iterative process would utilize known viral cultures and other characterized samples to ensure the assay functions as intended, but it's not typically referred to as a "training set" with ground truth in the same way as in AI/ML development.
§ 866.3328 Influenza virus antigen detection test system.
(a)
Identification. An influenza virus antigen detection test system is a device intended for the qualitative detection of influenza viral antigens directly from clinical specimens in patients with signs and symptoms of respiratory infection. The test aids in the diagnosis of influenza infection and provides epidemiological information on influenza. Due to the propensity of the virus to mutate, new strains emerge over time which may potentially affect the performance of these devices. Because influenza is highly contagious and may lead to an acute respiratory tract infection causing severe illness and even death, the accuracy of these devices has serious public health implications.(b)
Classification. Class II (special controls). The special controls for this device are:(1) The device's sensitivity and specificity performance characteristics or positive percent agreement and negative percent agreement, for each specimen type claimed in the intended use of the device, must meet one of the following two minimum clinical performance criteria:
(i) For devices evaluated as compared to an FDA-cleared nucleic acid based-test or other currently appropriate and FDA accepted comparator method other than correctly performed viral culture method:
(A) The positive percent agreement estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 80 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 70 percent.
(B) The negative percent agreement estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 95 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 90 percent.
(ii) For devices evaluated as compared to correctly performed viral culture method as the comparator method:
(A) The sensitivity estimate for the device when testing for influenza A must be at the point estimate of at least 90 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 80 percent. The sensitivity estimate for the device when testing for influenza B must be at the point estimate of at least 80 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 70 percent.
(B) The specificity estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 95 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 90 percent.
(2) When performing testing to demonstrate the device meets the requirements in paragraph (b)(1) of this section, a currently appropriate and FDA accepted comparator method must be used to establish assay performance in clinical studies.
(3) Annual analytical reactivity testing of the device must be performed with contemporary influenza strains. This annual analytical reactivity testing must meet the following criteria:
(i) The appropriate strains to be tested will be identified by FDA in consultation with the Centers for Disease Control and Prevention (CDC) and sourced from CDC or an FDA-designated source. If the annual strains are not available from CDC, FDA will identify an alternative source for obtaining the requisite strains.
(ii) The testing must be conducted according to a standardized protocol considered and determined by FDA to be acceptable and appropriate.
(iii) By July 31 of each calendar year, the results of the last 3 years of annual analytical reactivity testing must be included as part of the device's labeling. If a device has not been on the market long enough for 3 years of annual analytical reactivity testing to have been conducted since the device received marketing authorization from FDA, then the results of every annual analytical reactivity testing since the device received marketing authorization from FDA must be included. The results must be presented as part of the device's labeling in a tabular format, which includes the detailed information for each virus tested as described in the certificate of authentication, either by:
(A) Placing the results directly in the device's § 809.10(b) of this chapter compliant labeling that physically accompanies the device in a separate section of the labeling where the analytical reactivity testing data can be found; or
(B) In the device's label or in other labeling that physically accompanies the device, prominently providing a hyperlink to the manufacturer's public Web site where the analytical reactivity testing data can be found. The manufacturer's home page, as well as the primary part of the manufacturer's Web site that discusses the device, must provide a prominently placed hyperlink to the Web page containing this information and must allow unrestricted viewing access.
(4) If one of the actions listed at section 564(b)(1)(A)-(D) of the Federal Food, Drug, and Cosmetic Act occurs with respect to an influenza viral strain, or if the Secretary of Health and Human Services (HHS) determines, under section 319(a) of the Public Health Service Act, that a disease or disorder presents a public health emergency, or that a public health emergency otherwise exists, with respect to an influenza viral strain:
(i) Within 30 days from the date that FDA notifies manufacturers that characterized viral samples are available for test evaluation, the manufacturer must have testing performed on the device with those viral samples in accordance with a standardized protocol considered and determined by FDA to be acceptable and appropriate. The procedure and location of testing may depend on the nature of the emerging virus.
(ii) Within 60 days from the date that FDA notifies manufacturers that characterized viral samples are available for test evaluation and continuing until 3 years from that date, the results of the influenza emergency analytical reactivity testing, including the detailed information for the virus tested as described in the certificate of authentication, must be included as part of the device's labeling in a tabular format, either by:
(A) Placing the results directly in the device's § 809.10(b) of this chapter compliant labeling that physically accompanies the device in a separate section of the labeling where analytical reactivity testing data can be found, but separate from the annual analytical reactivity testing results; or
(B) In a section of the device's label or in other labeling that physically accompanies the device, prominently providing a hyperlink to the manufacturer's public Web site where the analytical reactivity testing data can be found. The manufacturer's home page, as well as the primary part of the manufacturer's Web site that discusses the device, must provide a prominently placed hyperlink to the Web page containing this information and must allow unrestricted viewing access.