(273 days)
ImmunoCard STAT!® Flu A&B is a rapid, qualitative, lateral-flow immunoassay for detecting both influenza A and influenza B viral nucleoprotein antigens in human nasal wash, nasopharyngeal aspirate and nasopharyngeal swab samples. It is designed to test samples from symptomatic patients. It is recommended that all negative test results be confirmed by cell culture.
ImmunoCard STAT! Flu A & B is distributed as a test kit that includes the following reagents: ImmunoCard STAT! Flu A & B Test Device: A chromatography strip housed in a plastic frame and enclosed in a foil pouch with a desiccant. The strip contains immobilized monoclonal antibody at the CONTROL line. The strip also contains colloidal gold conjugated anti-mouse antibody at the CONTROL line. The strip also contains colloidal gold conjugated anti-influenza A and anti-influenza B as the detection antibodies. A buffered protein solution containing sodium azide (0.095%) as a preservative. Positive Control: Inactivated influenza A and influenza B virus in a buffered solution containing sodium azide (0.095%) as a preservative.
Here's a breakdown of the acceptance criteria and study information for the ImmunoCard STAT! Flu A & B device, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state pre-defined acceptance criteria in terms of sensitivity and specificity targets. Instead, it presents the performance of the ImmunoCard STAT! Flu A & B device and compares it to a predicate device (Binax NOW Flu A + B) and the gold standard (cell culture). The efficacy is demonstrated by the clinical trial results showing its ability to detect Influenza A and B.
Given the context of a 510(k) summary for a diagnostic device, the implied acceptance criteria would be that the device performs similarly to legally marketed predicate devices and demonstrates reasonable accuracy against a "gold standard" (cell culture).
Here's a summary of the reported performance, derived from Table 6-2 (Overall Performance)
Performance of ImmunoCard STAT! Flu A & B vs. Tissue Culture (Gold Standard)
| Specimen Type & Flu Type | Performance Metric | ImmunoCard STAT! Flu A & B | Binax NOW Flu A + B (Predicate) |
|---|---|---|---|
| Wash/Aspirate - Flu A | True Positives (TP) | 35 | 34 |
| False Negatives (FN) | 4 | 5 | |
| False Positives (FP) | 14 | 10 | |
| True Negatives (TN) | 162 | 166 | |
| Total Samples | 215 | 215 | |
| Sensitivity (TP / (TP+FN)) | 35 / 39 = 89.7% | 34 / 39 = 87.2% | |
| Specificity (TN / (TN+FP)) | 162 / 176 = 92.0% | 166 / 176 = 94.3% | |
| Wash/Aspirate - Flu B | True Positives (TP) | 4 | 4 |
| False Negatives (FN) | 3 | 3 | |
| False Positives (FP) | 0 | 2 | |
| True Negatives (TN) | 208 | 205 | |
| Total Samples | 215 | 214 (1 Binax Flu B Invalid) | |
| Sensitivity | 4 / 7 = 57.1% | 4 / 7 = 57.1% | |
| Specificity | 208 / 208 = 100% | 205 / 207 = 99.0% | |
| Swab - Flu A | True Positives (TP) | 66 | 53 |
| False Negatives (FN) | 24 | 37 | |
| False Positives (FP) | 9 | 4 | |
| True Negatives (TN) | 329 | 333 | |
| Total Samples | 428 | 427 (1 Binax Flu A Invalid) | |
| Sensitivity | 66 / 90 = 73.3% | 53 / 90 = 58.9% | |
| Specificity | 329 / 338 = 97.3% | 333 / 337 = 98.8% | |
| Swab - Flu B | True Positives (TP) | 25 | 13 |
| False Negatives (FN) | 12 | 24 | |
| False Positives (FP) | 0 | 31 | |
| True Negatives (TN) | 391 | 359 | |
| Total Samples | 428 | 427 | |
| Sensitivity | 25 / 37 = 67.6% | 13 / 37 = 35.1% | |
| Specificity | 391 / 391 = 100% | 359 / 390 = 92.1% |
(Note: Sensitivity and Specificity are calculated based on the provided counts).
The document concludes that the ImmunoCard STAT! Flu A & B "Performs similarly to the predicate devices Binax NOW Flu A and NOW Flu B." This statement serves as the primary evidence that it meets the implicit acceptance criteria of demonstrating comparable effectiveness.
2. Sample Sizes Used for the Test Set and Data Provenance
-
Test Set Sample Sizes (Clinical Trials):
- Wash/Aspirate Samples: 215 total (across Flu A and Flu B evaluations)
- Swab Samples: 428 total (across Flu A and Flu B evaluations)
- Total Clinical Samples: 643
-
Data Provenance:
- "archival frozen (retrospective) or fresh (prospective) samples"
- "collected from symptomatic patients."
- The study was performed by "Three independent laboratories and Meridian's Development Laboratory." The country of origin is not explicitly stated, but Meridian Bioscience, Inc. is located in Cincinnati, OH, USA.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
- The ground truth for the clinical trials was established by cell culture. There is no mention of human experts being used to establish the ground truth for the clinical samples themselves. The interpretation of the cell culture results would have been performed by laboratory professionals, but these are not explicitly termed "experts" with specific qualifications in the context of ground truth establishment for the test set.
4. Adjudication Method for the Test Set
- The document implies that cell culture was the definitive gold standard. For "discordant results" between ImmunoCard STAT! Flu A & B and cell culture, PCR testing was used as an additional confirmatory method (Table 6-4). This acts as a form of adjudication for discordant results by introducing a third, more sensitive method to resolve discrepancies between the index test and the primary gold standard.
5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs. Without AI Assistance
- No, an MRMC comparative effectiveness study involving human readers with and without AI assistance was NOT done. This device is a rapid, qualitative lateral-flow immunoassay, which does not involve "human readers" in the sense of interpreting complex images or data that AI would assist with. The results are typically read visually as the presence or absence of a line. The document does mention "reproducibility" testing where clinical sites graded reactions, but this is to assess inter-site and inter-day variability, not human performance with/without AI.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
- This device is a manual, non-instrumented test. Its "standalone" performance is its performance, as it does not rely on an algorithm or a human-in-the-loop for interpretation beyond visual reading of the lines presented on the test strip. The clinical trial data presented reflects the standalone performance of the device when used by laboratory professionals.
7. The Type of Ground Truth Used
- The primary ground truth used for the clinical trials was cell culture isolation, which is explicitly referred to as the "gold standard for the detection of influenza."
- For discordant results, PCR results were also used for further characterization.
8. The Sample Size for the Training Set
- The document does not explicitly describe a separate training set or its sample size. This is common for traditional immunoassay development, where often a "development" or "optimization" phase occurs, but not necessarily a distinct, formalized "training set" in the same way as machine learning models. The LOD (Limit of Detection) study used laboratory-prepared antigen preparations (ATCC strains), which could be considered part of the development/training process.
9. How the Ground Truth for the Training Set Was Established
- As there's no explicitly defined "training set" in the context of the clinical evaluation, the method for establishing its ground truth is not detailed. For the LOD study, the "ground truth" was the known concentration of virions/mL in the prepared antigen samples.
§ 866.3328 Influenza virus antigen detection test system.
(a)
Identification. An influenza virus antigen detection test system is a device intended for the qualitative detection of influenza viral antigens directly from clinical specimens in patients with signs and symptoms of respiratory infection. The test aids in the diagnosis of influenza infection and provides epidemiological information on influenza. Due to the propensity of the virus to mutate, new strains emerge over time which may potentially affect the performance of these devices. Because influenza is highly contagious and may lead to an acute respiratory tract infection causing severe illness and even death, the accuracy of these devices has serious public health implications.(b)
Classification. Class II (special controls). The special controls for this device are:(1) The device's sensitivity and specificity performance characteristics or positive percent agreement and negative percent agreement, for each specimen type claimed in the intended use of the device, must meet one of the following two minimum clinical performance criteria:
(i) For devices evaluated as compared to an FDA-cleared nucleic acid based-test or other currently appropriate and FDA accepted comparator method other than correctly performed viral culture method:
(A) The positive percent agreement estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 80 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 70 percent.
(B) The negative percent agreement estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 95 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 90 percent.
(ii) For devices evaluated as compared to correctly performed viral culture method as the comparator method:
(A) The sensitivity estimate for the device when testing for influenza A must be at the point estimate of at least 90 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 80 percent. The sensitivity estimate for the device when testing for influenza B must be at the point estimate of at least 80 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 70 percent.
(B) The specificity estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 95 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 90 percent.
(2) When performing testing to demonstrate the device meets the requirements in paragraph (b)(1) of this section, a currently appropriate and FDA accepted comparator method must be used to establish assay performance in clinical studies.
(3) Annual analytical reactivity testing of the device must be performed with contemporary influenza strains. This annual analytical reactivity testing must meet the following criteria:
(i) The appropriate strains to be tested will be identified by FDA in consultation with the Centers for Disease Control and Prevention (CDC) and sourced from CDC or an FDA-designated source. If the annual strains are not available from CDC, FDA will identify an alternative source for obtaining the requisite strains.
(ii) The testing must be conducted according to a standardized protocol considered and determined by FDA to be acceptable and appropriate.
(iii) By July 31 of each calendar year, the results of the last 3 years of annual analytical reactivity testing must be included as part of the device's labeling. If a device has not been on the market long enough for 3 years of annual analytical reactivity testing to have been conducted since the device received marketing authorization from FDA, then the results of every annual analytical reactivity testing since the device received marketing authorization from FDA must be included. The results must be presented as part of the device's labeling in a tabular format, which includes the detailed information for each virus tested as described in the certificate of authentication, either by:
(A) Placing the results directly in the device's § 809.10(b) of this chapter compliant labeling that physically accompanies the device in a separate section of the labeling where the analytical reactivity testing data can be found; or
(B) In the device's label or in other labeling that physically accompanies the device, prominently providing a hyperlink to the manufacturer's public Web site where the analytical reactivity testing data can be found. The manufacturer's home page, as well as the primary part of the manufacturer's Web site that discusses the device, must provide a prominently placed hyperlink to the Web page containing this information and must allow unrestricted viewing access.
(4) If one of the actions listed at section 564(b)(1)(A)-(D) of the Federal Food, Drug, and Cosmetic Act occurs with respect to an influenza viral strain, or if the Secretary of Health and Human Services (HHS) determines, under section 319(a) of the Public Health Service Act, that a disease or disorder presents a public health emergency, or that a public health emergency otherwise exists, with respect to an influenza viral strain:
(i) Within 30 days from the date that FDA notifies manufacturers that characterized viral samples are available for test evaluation, the manufacturer must have testing performed on the device with those viral samples in accordance with a standardized protocol considered and determined by FDA to be acceptable and appropriate. The procedure and location of testing may depend on the nature of the emerging virus.
(ii) Within 60 days from the date that FDA notifies manufacturers that characterized viral samples are available for test evaluation and continuing until 3 years from that date, the results of the influenza emergency analytical reactivity testing, including the detailed information for the virus tested as described in the certificate of authentication, must be included as part of the device's labeling in a tabular format, either by:
(A) Placing the results directly in the device's § 809.10(b) of this chapter compliant labeling that physically accompanies the device in a separate section of the labeling where analytical reactivity testing data can be found, but separate from the annual analytical reactivity testing results; or
(B) In a section of the device's label or in other labeling that physically accompanies the device, prominently providing a hyperlink to the manufacturer's public Web site where the analytical reactivity testing data can be found. The manufacturer's home page, as well as the primary part of the manufacturer's Web site that discusses the device, must provide a prominently placed hyperlink to the Web page containing this information and must allow unrestricted viewing access.