(190 days)
The OSOM® ULTRA PLUS FLU A&B Test is an in vitro rapid diagnostic immunochromatographic assay intended for the qualitative detection of influenza type A and type B nucleoprotein antigens directly from nasal and nasopharyngeal swab specimens from patients with signs and symptoms of respiratory infection.
It is intended to aid in the rapid differential diagnosis of influenza A and B viral infections. This test is not intended for the detection of influenza C viruses.
A negative test result is presumptive, and it is recommended these results be confirmed by viral culture or an FDA-cleared influenza A and B molecular assay. Negative test results do not preclude influenza virus infection and should not be used as the sole basis for treatment or other patient management decisions.
Performance characteristics for influenza A were established during the US 2018-2019 influenza season when A/H1N1pdm09 and influenza A/H3N2 were the predominant influenza A viruses in circulation, and the influenza B Yamagata and Victoria lineages were in co-circulation. When other influenza A or B viruses are emerging, performance characteristics may vary.
If infection with a novel influenza virus is suspected based on current clinical and epidemiological screening criteria recommended by public health authorities, specimens should be collected with appropriate infection control precautions for novel virulent influenza viruses and sent to state or local health department for testing. Viral culture should not be attempted in these cases unless a BSL 3+ facility is available to receive and culture specimens.
The OSOM® ULTRA PLUS FLU A&B Test consists of a test stick that separately detects influenza A and B. The test procedure requires the solubilization of the nucleoproteins from a swab by mixing the swab in Extraction Buffer. The test stick is then placed in the sample mixture, which then migrates along the membrane surface. If influenza A and/or B viral antigens are present in the sample, it will form a complex with mouse monoclonal IgG antibodies to influenza A and/or B nucleoproteins conjugated to colloidal gold. The complex will then be bound by another a rat anti-influenza A and/or mouse anti-influenza B antibody coated on the nitrocellulose membrane. A pink to purple control line must appear in the control region of the stick for results to be valid. The appearance of a second and possibly a third light pink to purple line in the test line region indicates an A, B or A and B positive result. A visible control line with no test line is a negative result.
The Sekisui Diagnostics OSOM ULTRA PLUS FLU A&B Test is an in vitro rapid diagnostic immunochromatographic assay intended for the qualitative detection of influenza A and B nucleoprotein antigens from nasal and nasopharyngeal swab specimens.
Here's an analysis of the acceptance criteria and study data:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state pre-defined acceptance criteria in terms of numerical thresholds for sensitivity and specificity. Instead, the study results are presented as the device's performance characteristics. Regulatory bodies typically expect high sensitivity and specificity for such diagnostic tests. For the purpose of summarization, we will extract the reported clinical performance as the device's demonstrated performance.
| Performance Characteristic | Target (Implicit/Assumed for FDA Clearance) | Reported Device Performance (Influenza A) | Reported Device Performance (Influenza B) |
|---|---|---|---|
| Sensitivity | High (e.g., typically >80%) | 90.3% (95% CI: 87.0%-92.8%) | 88.0% (95% CI: 81.8%-92.3%) |
| Specificity | High (e.g., typically >95%) | 96.7% (95% CI: 95.5%-97.6%) | 99.2% (95% CI: 98.6%-99.6%) |
| Invalid Rate | Low (e.g., 95%) | 98.9% - 100% (for different categories) | 100% (for different categories) |
| Analytical Sensitivity (LoD) | Defined for specific strains | A/H1N1: 7.1x10¹ TCID50/mL; A/H3N2: 2.2x10⁵ CEID50/mL | B/Victoria: 3.5x10³ TCID50/mL; B/Yamagata: 1.6x10² TCID50/mL |
| Analytical Reactivity (Detection of various strains) | 100% detection of tested strains at LoD | Detected all 16 tested influenza A strains | Detected all 8 tested influenza B strains |
| Analytical Specificity (Cross-Reactivity) | 0% cross-reactivity with non-target organisms and human DNA | No cross-reactivity with 41 tested organisms/DNA | No cross-reactivity with 41 tested organisms/DNA |
| Interfering Substances | No interference at specified concentrations | No interference observed for 26 tested substances | No interference observed for 26 tested substances |
| Competitive Interference | No competitive interference | No competitive interference observed | No competitive interference observed |
2. Sample Size Used for the Test Set and Data Provenance
- Test Set Sample Size:
- Prospective study: 1210 evaluable prospective samples.
- Banked samples: 316 evaluable banked samples.
- Total evaluable samples for performance evaluation: 1526 samples (1210 prospective + 316 banked).
- Data Provenance:
- Country of Origin: United States. The prospective study was conducted at 21 point-of-care (POC) sites across the United States.
- Retrospective or Prospective: The primary clinical study was prospective, collecting samples from January 2019 to May 2019. This prospective dataset was supplemented with 317 banked samples (retrospective) collected from previous influenza seasons due to atypically low prevalence of influenza B in the prospective study period.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The document does not explicitly state the number of experts used or their specific qualifications for establishing the ground truth. However, the ground truth was established by "an FDA-cleared molecular test" (a reference method) and further "another FDA-cleared molecular test for discrepant analysis." This implies that the ground truth was determined by validated laboratory methods rather than direct expert interpretation of the rapid test results. The operators at the CLIA waived sites were "untrained operators with no laboratory training or experience," but they were performing the device test, not establishing ground truth.
4. Adjudication Method for the Test Set
The adjudication method used was a "discrepant analysis" involving a second FDA-cleared molecular test.
- For Influenza A: If the OSOM ULTRA PLUS FLU A&B Test results differed from the initial FDA-cleared molecular comparator method, a second FDA-cleared molecular test was used for adjudication. For instance, out of 37 false positive specimens, Flu A was detected in 23 using the second molecular test. For 39 false negative specimens, Flu A was not detected in 7 using the second molecular test.
- For Influenza B: Similarly, FLu B was detected in 3 of 11 false positive specimens using a second FDA-cleared molecular test, and not detected in 2 of 18 false negative specimens using the second test.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done
No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not explicitly described in the provided text in the context of comparing human readers with and without AI assistance. This device is a rapid diagnostic immunochromatographic assay, which is read visually by a single operator (or potentially multiple operators for reproducibility studies, but not in a comparative effectiveness study against AI). The study involved "untrained operators" at POC sites, but their performance was compared against a molecular reference method, not against an AI.
6. If a Standalone (i.e. algorithm only, without human-in-the-loop performance) was done
This question is not applicable as the device is a manual, visually interpreted rapid diagnostic test, not an AI-powered algorithm. The device's performance is its standalone performance without human input beyond sample application and visual interpretation. The performance metrics presented (sensitivity, specificity) reflect the device's ability to accurately detect the antigen when operated by intended users (untrained operators).
7. The Type of Ground Truth Used
The ground truth for the clinical performance study was established using:
- An FDA-cleared molecular test (as the primary comparator method).
- Another FDA-cleared molecular test for discrepant analysis.
This indicates a highly reliable laboratory-based ground truth, often considered superior to expert consensus for objective biological markers like viral antigens.
8. The Sample Size for the Training Set
The document does not provide information on a training set sample size. This is expected for laboratory diagnostic devices where development involves analytical studies (LoD, reactivity, specificity) to optimize the assay, followed by clinical validation. 'Training set' is a term primarily used in machine learning and AI development, which does not apply directly to this type of traditional in vitro diagnostic device validation.
9. How the Ground Truth for the Training Set Was Established
As there is no mention of a "training set" in the context of AI or machine learning for this device, information on how its ground truth was established is not applicable/provided. The analytical studies (LoD, reactivity, specificity, interference) involve preparing samples with known concentrations of analytes, where the "ground truth" is determined by the preparation method itself and confirmed by standard laboratory techniques.
§ 866.3328 Influenza virus antigen detection test system.
(a)
Identification. An influenza virus antigen detection test system is a device intended for the qualitative detection of influenza viral antigens directly from clinical specimens in patients with signs and symptoms of respiratory infection. The test aids in the diagnosis of influenza infection and provides epidemiological information on influenza. Due to the propensity of the virus to mutate, new strains emerge over time which may potentially affect the performance of these devices. Because influenza is highly contagious and may lead to an acute respiratory tract infection causing severe illness and even death, the accuracy of these devices has serious public health implications.(b)
Classification. Class II (special controls). The special controls for this device are:(1) The device's sensitivity and specificity performance characteristics or positive percent agreement and negative percent agreement, for each specimen type claimed in the intended use of the device, must meet one of the following two minimum clinical performance criteria:
(i) For devices evaluated as compared to an FDA-cleared nucleic acid based-test or other currently appropriate and FDA accepted comparator method other than correctly performed viral culture method:
(A) The positive percent agreement estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 80 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 70 percent.
(B) The negative percent agreement estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 95 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 90 percent.
(ii) For devices evaluated as compared to correctly performed viral culture method as the comparator method:
(A) The sensitivity estimate for the device when testing for influenza A must be at the point estimate of at least 90 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 80 percent. The sensitivity estimate for the device when testing for influenza B must be at the point estimate of at least 80 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 70 percent.
(B) The specificity estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 95 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 90 percent.
(2) When performing testing to demonstrate the device meets the requirements in paragraph (b)(1) of this section, a currently appropriate and FDA accepted comparator method must be used to establish assay performance in clinical studies.
(3) Annual analytical reactivity testing of the device must be performed with contemporary influenza strains. This annual analytical reactivity testing must meet the following criteria:
(i) The appropriate strains to be tested will be identified by FDA in consultation with the Centers for Disease Control and Prevention (CDC) and sourced from CDC or an FDA-designated source. If the annual strains are not available from CDC, FDA will identify an alternative source for obtaining the requisite strains.
(ii) The testing must be conducted according to a standardized protocol considered and determined by FDA to be acceptable and appropriate.
(iii) By July 31 of each calendar year, the results of the last 3 years of annual analytical reactivity testing must be included as part of the device's labeling. If a device has not been on the market long enough for 3 years of annual analytical reactivity testing to have been conducted since the device received marketing authorization from FDA, then the results of every annual analytical reactivity testing since the device received marketing authorization from FDA must be included. The results must be presented as part of the device's labeling in a tabular format, which includes the detailed information for each virus tested as described in the certificate of authentication, either by:
(A) Placing the results directly in the device's § 809.10(b) of this chapter compliant labeling that physically accompanies the device in a separate section of the labeling where the analytical reactivity testing data can be found; or
(B) In the device's label or in other labeling that physically accompanies the device, prominently providing a hyperlink to the manufacturer's public Web site where the analytical reactivity testing data can be found. The manufacturer's home page, as well as the primary part of the manufacturer's Web site that discusses the device, must provide a prominently placed hyperlink to the Web page containing this information and must allow unrestricted viewing access.
(4) If one of the actions listed at section 564(b)(1)(A)-(D) of the Federal Food, Drug, and Cosmetic Act occurs with respect to an influenza viral strain, or if the Secretary of Health and Human Services (HHS) determines, under section 319(a) of the Public Health Service Act, that a disease or disorder presents a public health emergency, or that a public health emergency otherwise exists, with respect to an influenza viral strain:
(i) Within 30 days from the date that FDA notifies manufacturers that characterized viral samples are available for test evaluation, the manufacturer must have testing performed on the device with those viral samples in accordance with a standardized protocol considered and determined by FDA to be acceptable and appropriate. The procedure and location of testing may depend on the nature of the emerging virus.
(ii) Within 60 days from the date that FDA notifies manufacturers that characterized viral samples are available for test evaluation and continuing until 3 years from that date, the results of the influenza emergency analytical reactivity testing, including the detailed information for the virus tested as described in the certificate of authentication, must be included as part of the device's labeling in a tabular format, either by:
(A) Placing the results directly in the device's § 809.10(b) of this chapter compliant labeling that physically accompanies the device in a separate section of the labeling where analytical reactivity testing data can be found, but separate from the annual analytical reactivity testing results; or
(B) In a section of the device's label or in other labeling that physically accompanies the device, prominently providing a hyperlink to the manufacturer's public Web site where the analytical reactivity testing data can be found. The manufacturer's home page, as well as the primary part of the manufacturer's Web site that discusses the device, must provide a prominently placed hyperlink to the Web page containing this information and must allow unrestricted viewing access.