(256 days)
The CareStart™ Flu A&B Plus is an in vitro rapid immunochromatographic assay for the qualitative detection of influenza virus type A and B nucleoprotein antigens directly from nasopharyngeal swab specimens of symptomatic patients.
The test is intended for use as an aid in the rapid differential diagnosis of acute influenza type A and B viral infections. This test is intended to distinguish between influenza type A and/or B virus in a single test. This test is not intended to detect influenza type C viral antigens. Negative test results are presumptive and should be confirmed by viral culture or an FDA-cleared influenza A and B molecular assay. Negative results do not preclude influenza virus infections and should not be used as the basis for treatment or other patient management decisions.
Performance characteristics for influenza A and B were established during the 2018-2019 influenza season when influenza A/H3N2, A/H1N1pdm09, and B/Victoria were the predominant influenza viruses in circulation. When other influenza viruses are emerging, performance characteristics may vary.
If infection with a novel influenza virus is suspected based on current clinical and epidemiological screening criteria recommended by public health authorities, specimens should be collected with appropriate infection control precautions for novel virulent influenza viruses and sent to the state or local health department for testing. Viral culture should not be attempted in these cases unless a BSL 3+ facility is available to received and culture specimens.
The CareStart™ Flu A&B Plus test is an immunochromatographic assay for detection of extracted influenza type A and B virus nucleoprotein antigens in nasopharyngeal specimens.
Nasopharyngeal swabs require a sample preparation step in which the sample is eluted and washed off into the extraction buffer solution. Extracted swab sample is added to the sample well of the test device to initiate the test. When the swab sample migrates in the test strip, influenza A or B viral antigens bind to anti-influenza antibodies conjugated to indicator particles in the test strip forming an immune complex. The immune complex is then captured by each test line and control line on the membrane as it migrates through the strip.
Test results are interpreted at 10 minutes. The presence of two colored lines, a purplecolored line in the control region "C" and a red-colored line in the influenza A test region "A", indicates influenza A positive. The presence of two colored lines, a purplecolored line in the control region "C" and a blue-colored line in the influenza B test region "B", indicates influenza B positive. The presence of three colored lines, a purple-colored line in the control region "C", a red-colored line in the influenza A test region "A", and a blue-colored line in the influenza B test region "B indicates, influenza A and B dual positive result. The absence of a line on both influenza A and B test regions with a purple-colored line in the control region "C" indicates negative. No appearance of purple-colored line in the control region "C" indicates invalid test.
Here's an analysis of the acceptance criteria and the study that proves the device meets them, based on the provided text:
Device Name: CareStart™ Flu A&B Plus (Influenza Virus Antigen Detection Test System)
General Acceptance Criteria (Implied by FDA 510(k) Clearance):
The primary acceptance criterion for a 510(k) submission is that the device is substantially equivalent to a legally marketed predicate device. This is demonstrated by showing that the new device has the same intended use and technological characteristics, and that its performance is at least as safe and effective as the predicate device. Specific performance criteria are established through analytical and clinical studies.
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state pre-defined acceptance criteria values (e.g., "PPA must be >X%"). Instead, the performance estimates are presented, and the implication is that these results demonstrate substantial equivalence to the predicate device. For the purpose of this analysis, I will treat the reported performance as the "met criteria" that led to substantial equivalence.
| Performance Metric | Acceptance Criteria (Implied/Demonstrated) | Reported Device Performance |
|---|---|---|
| Clinical Performance (Influenza A) | Sufficiently high PPA and NPA compared to molecular assay | PPA: 79.9% (95% CI: 75.7% – 83.7%) |
| NPA: 98.4% (95% CI: 97.0% – 99.2%) | ||
| Clinical Performance (Influenza B) | Sufficiently high PPA and NPA compared to molecular assay | PPA: 88.2% (95% CI: 65.7% – 96.7%) (Prospective) |
| PPA: 96.6% (95% CI: 91.5% – 98.7%) (Retrospective) | ||
| NPA: 100.0% (95% CI: 99.6% – 100.0%) (Prospective) | ||
| NPA: 97.8% (95% CI: 88.4% – 99.6%) (Retrospective) | ||
| Analytical Sensitivity (LoD) | Detection of virus strains at specified concentrations | All tested strains detected at concentrations ranging from $2.0 \times 10^{5.2}$ to $1.6 \times 10^{6.4}$ for Influenza A and $2.0 \times 10^{5.5}$ to $1.6 \times 10^{6.4}$ for Influenza B, with 95-100% reactivity. |
| Reactivity (Inclusivity) | Detection of various influenza A and B strains | All 15 Influenza A and 10 Influenza B strains detected in 3/3 replicates at specified concentrations. |
| Analytical Specificity (Cross-Reactivity/Interference) | No false positives (cross-reactivity); no interference with positive samples | No false positives with 31 bacteria and 15 non-influenza viruses. No interference with influenza A/B positive samples. Biotin concentrations >500 ng/ml can cause false negative influenza A results (caveat). |
| Reproducibility | High agreement across sites, operators, and days | 100.0% overall agreement for all sample categories across 3 sites, 3 operators, and 5 days. |
| Lot-to-Lot Precision | Consistent results across different reagent lots | 100.0% agreement across 3 reagent lots for all sample categories. |
2. Sample Size Used for the Test Set and Data Provenance
-
Clinical Test Set:
- Prospective Clinical Study: 944 evaluable nasopharyngeal swab specimens.
- Provenance: Collected from symptomatic patients during the 2018-2019 influenza season at 10 Point-of-Care investigational sites throughout the U.S. (prospective, U.S. origin).
- Retrospective Study (supplemental for Influenza B): 162 swab samples prepared from archived respiratory specimens.
- Provenance: Archived respiratory specimens from patients with influenza-like symptoms, confirmed positive or negative by an FDA-cleared molecular assay. Samples were distributed among four investigational sites (retrospective, likely U.S. origin, as it supplemental for the U.S. prospective study).
- Prospective Clinical Study: 944 evaluable nasopharyngeal swab specimens.
-
Analytical Sensitivity (LoD): 20 replicates per virus strain (8 strains tested).
-
Analytical Sensitivity (Reactivity/Inclusivity): 3 replicates per virus strain (25 strains tested).
-
Analytical Specificity (Cross-Reactivity/Interference): 3 replicates per organism/virus, both with and without influenza viruses (31 bacteria, 15 non-influenza viruses, 30 interfering substances).
-
Reproducibility Study: 7 sample categories tested in a blinded manner by 3 operators at 3 sites on 5 non-consecutive days (7 samples * 3 operators * 3 sites * 5 days = 315 tests, per virus type if applicable). Counted as 135/135 for each category overall.
-
Lot-to-Lot Precision: 7 sample categories tested across 3 reagent lots (counted as 27/27 for each category overall).
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The document does not specify the number or qualifications of experts for establishing ground truth.
-
Clinical Study Ground Truth: The ground truth for the clinical studies (prospective and retrospective) was established using an FDA-cleared influenza A and B molecular assay (comparator method). For discrepant results in the prospective study, an alternative FDA-cleared molecular assay was used for investigation. These are laboratory-based, highly sensitive and specific molecular tests for influenza, which are considered a gold standard for pathogen detection.
-
Analytical Studies Ground Truth: For analytical studies (LoD, Reactivity, Cross-Reactivity, Interference, Reproducibility, Lot-to-Lot Precision), the ground truth was inherent to the carefully prepared samples (e.g., known virus concentrations, known interfering substances).
4. Adjudication Method for the Test Set
- Clinical Studies: For the prospective clinical study, discrepancies between the CareStart™ Flu A&B Plus results and the initial molecular comparator assay results were investigated using an alternative FDA-cleared molecular influenza A/B assay. The results of this secondary testing were captured in footnotes but not included in the primary calculations of the performance estimates. This suggests a form of 2+1 adjudication not directly applied to the final performance metrics but used for understanding discrepancies.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, what was the effect size of how much human readers improve with AI vs without AI assistance
- Not applicable. This device is an in vitro rapid immunochromatographic assay (a rapid diagnostic test kit) and does not involve AI assistance or human readers interpreting AI outputs. The results are interpreted visually by a human or using a simple instrument (like the predicate, though the proposed device is visually interpreted). Therefore, an MRMC study related to AI assistance for human readers was not performed.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
- Yes, in essence, the "device performance" in the clinical and analytical studies represents the standalone performance of the assay. The CareStart™ Flu A&B Plus itself is a rapid test that produces visual results. The clinical studies compare the device's output (positive/negative for Flu A/B) directly against a molecular truth, effectively testing its "standalone" diagnostic capabilities without a human interpretation model beyond reading a test line. The detection format is "Visual determination of presence or absence of colored line indicators" for the proposed device, indicating it functions as a standalone diagnostic tool without requiring an additional human interpretation step or an algorithm to read the result in a complex way.
7. The type of ground truth used
- Molecular Assay: The primary ground truth for clinical performance evaluation (prospective and retrospective studies) was an FDA-cleared influenza A and B molecular assay. An alternative FDA-cleared molecular assay was used to investigate discrepant results.
- Known Concentrations/Absence of Analytes: For analytical studies (LoD, Reactivity, Cross-Reactivity, Interference, Reproducibility, Lot-to-Lot Precision), the ground truth was based on samples with known concentrations of specific virus strains or known absence of target analytes/presence of interfering substances.
8. The sample size for the training set
The document does not explicitly describe a separate "training set" in the context of machine learning or AI. As a rapid diagnostic test, the device's development typically involves iterative design and testing using various lab-prepared and clinical samples, which implicitly serve as a development/training phase. However, there isn't a formally described training set as one might see for an AI algorithm. The performance evaluation presented focuses on the validation of the device.
9. How the ground truth for the training set was established
As there is no explicitly defined "training set" in the provided document, the method for establishing its ground truth is not described. The analytical and clinical studies described serve as validation of the final device. The ground truth for those validation studies was established through FDA-cleared molecular assays or by controlled preparation of samples with known viral content.
§ 866.3328 Influenza virus antigen detection test system.
(a)
Identification. An influenza virus antigen detection test system is a device intended for the qualitative detection of influenza viral antigens directly from clinical specimens in patients with signs and symptoms of respiratory infection. The test aids in the diagnosis of influenza infection and provides epidemiological information on influenza. Due to the propensity of the virus to mutate, new strains emerge over time which may potentially affect the performance of these devices. Because influenza is highly contagious and may lead to an acute respiratory tract infection causing severe illness and even death, the accuracy of these devices has serious public health implications.(b)
Classification. Class II (special controls). The special controls for this device are:(1) The device's sensitivity and specificity performance characteristics or positive percent agreement and negative percent agreement, for each specimen type claimed in the intended use of the device, must meet one of the following two minimum clinical performance criteria:
(i) For devices evaluated as compared to an FDA-cleared nucleic acid based-test or other currently appropriate and FDA accepted comparator method other than correctly performed viral culture method:
(A) The positive percent agreement estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 80 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 70 percent.
(B) The negative percent agreement estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 95 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 90 percent.
(ii) For devices evaluated as compared to correctly performed viral culture method as the comparator method:
(A) The sensitivity estimate for the device when testing for influenza A must be at the point estimate of at least 90 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 80 percent. The sensitivity estimate for the device when testing for influenza B must be at the point estimate of at least 80 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 70 percent.
(B) The specificity estimate for the device when testing for influenza A and influenza B must be at the point estimate of at least 95 percent with a lower bound of the 95 percent confidence interval that is greater than or equal to 90 percent.
(2) When performing testing to demonstrate the device meets the requirements in paragraph (b)(1) of this section, a currently appropriate and FDA accepted comparator method must be used to establish assay performance in clinical studies.
(3) Annual analytical reactivity testing of the device must be performed with contemporary influenza strains. This annual analytical reactivity testing must meet the following criteria:
(i) The appropriate strains to be tested will be identified by FDA in consultation with the Centers for Disease Control and Prevention (CDC) and sourced from CDC or an FDA-designated source. If the annual strains are not available from CDC, FDA will identify an alternative source for obtaining the requisite strains.
(ii) The testing must be conducted according to a standardized protocol considered and determined by FDA to be acceptable and appropriate.
(iii) By July 31 of each calendar year, the results of the last 3 years of annual analytical reactivity testing must be included as part of the device's labeling. If a device has not been on the market long enough for 3 years of annual analytical reactivity testing to have been conducted since the device received marketing authorization from FDA, then the results of every annual analytical reactivity testing since the device received marketing authorization from FDA must be included. The results must be presented as part of the device's labeling in a tabular format, which includes the detailed information for each virus tested as described in the certificate of authentication, either by:
(A) Placing the results directly in the device's § 809.10(b) of this chapter compliant labeling that physically accompanies the device in a separate section of the labeling where the analytical reactivity testing data can be found; or
(B) In the device's label or in other labeling that physically accompanies the device, prominently providing a hyperlink to the manufacturer's public Web site where the analytical reactivity testing data can be found. The manufacturer's home page, as well as the primary part of the manufacturer's Web site that discusses the device, must provide a prominently placed hyperlink to the Web page containing this information and must allow unrestricted viewing access.
(4) If one of the actions listed at section 564(b)(1)(A)-(D) of the Federal Food, Drug, and Cosmetic Act occurs with respect to an influenza viral strain, or if the Secretary of Health and Human Services (HHS) determines, under section 319(a) of the Public Health Service Act, that a disease or disorder presents a public health emergency, or that a public health emergency otherwise exists, with respect to an influenza viral strain:
(i) Within 30 days from the date that FDA notifies manufacturers that characterized viral samples are available for test evaluation, the manufacturer must have testing performed on the device with those viral samples in accordance with a standardized protocol considered and determined by FDA to be acceptable and appropriate. The procedure and location of testing may depend on the nature of the emerging virus.
(ii) Within 60 days from the date that FDA notifies manufacturers that characterized viral samples are available for test evaluation and continuing until 3 years from that date, the results of the influenza emergency analytical reactivity testing, including the detailed information for the virus tested as described in the certificate of authentication, must be included as part of the device's labeling in a tabular format, either by:
(A) Placing the results directly in the device's § 809.10(b) of this chapter compliant labeling that physically accompanies the device in a separate section of the labeling where analytical reactivity testing data can be found, but separate from the annual analytical reactivity testing results; or
(B) In a section of the device's label or in other labeling that physically accompanies the device, prominently providing a hyperlink to the manufacturer's public Web site where the analytical reactivity testing data can be found. The manufacturer's home page, as well as the primary part of the manufacturer's Web site that discusses the device, must provide a prominently placed hyperlink to the Web page containing this information and must allow unrestricted viewing access.