(268 days)
The EUROIMMUN Anti-West Nile Virus ELISA (IgG) is intended for the qualitative detection of IgG antibodies to West Nile virus in human serum and plasma (K+EDTA, Li+-heparin). This test is intended as an aid in the presumptive laboratory diagnosis of West Nile virus infection in patients with clinical symptoms consistent with meningitis encephalitis, in conjunction with other laboratory and clinical findings. Positive results must be confirmed by the plaque reduction test (PRNT) or by using the current CDC guidelines for diagnosis of this disease.
The assay characteristics have not been established for testing cord blood, neonates, prenatal screening, and general population screening of patients without symptoms of meningoencephalitis. This assay is not FDA cleared or approved for testing blood or plasma donors.
Warning: Cross-reactivity with IgG to Dengue, Chikungunya, Zika and Tick-borne Encephalitis (TBE) viruses has been observed with the EUROIMMUN Anti-West Nile Virus ELISA (IgG). Reactive results must be reported with a caution statement regarding possible IgG cross-reactivity with other flaviviruses.
Patient samples are diluted 1:101 in sample buffer, 100 ul of each diluted patient sample and pre-diluted controls and the calibrator are added to the antigen coated microtiter wells and incubated for 60 minutes at +37°C. After incubation the microtiter well strips are washed 3 times with wash buffer to remove unbound antibodies and 100 ul of the anti-human IgG enzyme conjugate reagent is added to each microtiter well. After an additional 30-minutes incubation at room temperature, the microtiter wells are again washed 3 times with wash buffer to remove any unbound enzyme conjugate and 100 ul of the chromogen substrate is added. The strips are incubated for 15 minutes at room temperature and 100 ul stop solution is added. The microtiter plates are placed in an ELISA reader and read at a wavelength of 450 nm and a reference wavelength of between 620 nm and 650 nm within 30 minutes.
The provided text describes the EUROIMMUN Anti-West Nile Virus ELISA (IgG) device, which is an in vitro diagnostic immunoassay. Therefore, the questions related to AI/MRMC studies, human-in-the-loop performance, and expert ground truth are not applicable in the context of this device. The acceptance criteria and study data are presented for the assay's analytical and clinical performance.
Here's the breakdown of the information based on the provided document:
1. Acceptance Criteria and Reported Device Performance
The acceptance criteria for the EUROIMMUN Anti-West Nile Virus ELISA (IgG) are primarily based on its analytical performance (repeatability, reproducibility, specificity) and clinical performance (sensitivity and specificity compared to a predicate device and PRNT).
| Acceptance Criteria Category | Specific Metric/Target | Reported Device Performance |
|---|---|---|
| Analytical Performance | Repeatability (Intra-assay precision): Demonstrated low variability across multiple runs and days. No specific %CV target is stated as an acceptance criterion, but low %CV values indicate acceptable repeatability. | Repeatability: For 7 seropositive samples tested over 14 runs on 7 days (3 replicates/run), the total %CV ranged from 6.1% to 13.6%. For negative samples (0.1 ratio) it was 13.3%. |
| Reproducibility (Inter-site precision): Demonstrated low variability across multiple sites. No specific %CV target is stated as an acceptance criterion, but low %CV values indicate acceptable reproducibility. | Reproducibility: For 7 seropositive samples tested at 3 different sites over 5 days (2 runs/day, 2 replicates/run), the total %CV ranged from 10.1% to 16.3%. For negative samples (0.1 ratio) it was 16.3%. | |
| Analytical Specificity/Cross-Reactivity: Evaluate cross-reactivity with other common infections and autoimmune markers. While no specific rejection threshold is given, the warning in the Indications for Use (cross-reactivity with other flaviviruses) suggests that some level of cross-reactivity is acknowledged and managed through labeling. The goal is to minimize non-specific reactivity. | Cross-Reactivity: Tested against 910 serologically characterized specimens from various diseases. Overall, 164 (18.0%) of these showed positive results with the WNV ELISA. Notably high cross-reactivity was observed with: | |
| - Anti-Chikungunya virus: 23/72 (31.9% positive, 68.1% negative) | ||
| - Anti-Dengue virus: 50/58 (86.2% positive, 13.8% negative) | ||
| - Anti-TBE virus: 33/118 (28.0% positive, 72.0% negative) | ||
| - Anti-Zika virus: 47/47 (100% positive, 0% negative) | ||
| Interferences: Demonstrate no significant interference from common biological substances (hemoglobin, triglycerides, bilirubin). | Interferences: No influence on results up to 1000 mg/dL hemoglobin, 2000 mg/dL triglycerides, and 40 mg/dL bilirubin. (Note: Interferences from high protein, cholesterol, and intralipids were not investigated). | |
| Assay Cut-off: Establish a cut-off with optimal sensitivity and specificity (e.g., 100% sensitivity and 100% specificity in the initial ROC analysis). The borderline range should cover a high percentage of negative samples (e.g., 98%). | Assay Cut-off: ROC analysis on 18 positive and 150 negative samples achieved 100.0% sensitivity and 100.0% specificity at OD 0.475. The Borderline range (ratio 0.8 to 1.1) covered 98% of negative samples. Using this cut-off/borderline with the same groups: Sensitivity 100.0% (95% C.I.: 81.5 - 100.0%), and specificity 98.7% (95% C.I.: 95.3 - 99.8% if borderline samples counted as positive). | |
| Clinical Performance | Positive Agreement (PPA) with Predicate Device: High agreement is expected for positive samples. No explicit acceptance percentage is stated, but typically values above 90% are sought for substantial equivalence. | Clinical Study I (Prospective, US): Positive agreement: 93.3% (42/45), 95% C.I. 81.7-98.6% vs. predicate. |
| Negative Agreement (NPA) with Predicate Device: High agreement is expected for negative samples. No explicit acceptance percentage is stated, but typically values above 90% are sought for substantial equivalence. | Clinical Study I (Prospective, US): Negative agreement: 94.4% (101/107), 95% C.I. 88.2-97.9% vs. predicate. | |
| Sensitivity vs. PRNT (Gold Standard): High sensitivity is critical for identifying infected individuals. For presumptive positives. | Clinical Study I (Prospective, US): For 27 presumptive positives by the predicate, tested against PRNT (ground truth): Sensitivity 92.6% (25/27), 95% C.I. 75.7-99.1%. | |
| Positive Agreement (PPA) with Predicate Device (Study II): (Replication/validation of agreement). | Clinical Study II (Prospective, US): Positive agreement: 95.6% (43/45), 95% C.I. 84.9-99.5% vs. predicate. | |
| Negative Agreement (NPA) with Predicate Device (Study II): (Replication/validation of agreement). | Clinical Study II (Prospective, US): Negative agreement: 98.9% (352/356), 95% C.I. 97.1-99.7% vs. predicate. | |
| Positive Agreement vs. PRNT (Retrospective, Non-US): High agreement with an established reference method, especially for positive samples. For outbreak samples. | Clinical Study III (Retrospective, Germany/South Africa): Positive agreement: 99.5% (194/195), 95% C.I. 97.2-100.0% vs. RKI Anti-West Nile Virus PRNT. | |
| Negative Agreement vs. PRNT (Retrospective, Non-US): High agreement with an established reference method, especially for negative samples. | Clinical Study III (Retrospective, Germany/South Africa): Negative agreement: 97.0% (97/100), 95% C.I. 91.5-99.4% vs. RKI Anti-West Nile Virus PRNT. | |
| Matrix Comparison | Equivalence of Serum vs. Plasma: The device should perform equivalently across tested sample matrices (serum, K+EDTA plasma, Li-heparin plasma). Regression equation should be near ideal (y=x), and % recovery should be within an acceptable range (e.g., 90-110%). | Matrix Comparison: Based on 20 spiked paired samples each for EDTA and Li-heparin plasma vs. serum. |
| - EDTA plasma: Regression equation y = -0.01 + 1.02x (near ideal). R2 = 0.9983. Mean % recovery = 101% (Range 98-105%). | ||
| - Li-heparin plasma: Regression equation y = 0.04 + 0.96x (near ideal). R2 = 0.9759. Mean % recovery = 100% (Range 92-109%). |
2. Sample Size Used for the Test Set and Data Provenance
The "test set" here refers to the clinical samples used for performance validation against a predicate device and ground truth (PRNT).
- Clinical Study I (Predicate Comparison and PRNT Confirmation):
- Sample Size: 152 samples for predicate comparison. Of these, 45 were presumptive positives by the predicate, and 27 of these were further tested by PRNT.
- Data Provenance: Prospective collection from hospitals and clinics across the US in 2015.
- Clinical Study II (Predicate Comparison):
- Sample Size: 401 serum samples.
- Data Provenance: Prospective collection from a clinical laboratory in the midwest (US).
- Clinical Study III (PRNT Comparison):
- Sample Size: 295 serum samples.
- Data Provenance: Retrospective collection in cooperation with the Robert Koch Institute (RKI), Berlin, Germany, including 200 samples from major outbreaks of West Nile fever in South Africa in 1974 and 1984.
- Analytical Specificity/Cross-Reactivity Study:
- Sample Size: 910 serologically characterized seropositive specimens.
- Data Provenance: Not explicitly stated, but implies diverse sources given the range of conditions.
- Assay Cut-off Determination:
- Sample Size: 18 sera from clinically characterized positive WNV patients and 150 sera from normal healthy blood donors.
- Data Provenance: Implied clinical and healthy populations, source not specified.
- Expected Values (US Studies):
- Sample Size: 553 samples.
- Data Provenance: Prospectively collected from US population.
- Matrix Comparison:
- Sample Size: 20 paired samples (serum, EDTA plasma, Li-heparin plasma), created by spiking 5 different sets of normal blood donor samples with 5 different positive serum samples.
- Data Provenance: Normal blood donors (source not specified), spiked with serum samples (source not specified if human/animal).
3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts
- This is an in vitro diagnostic (IVD) device, not an AI/imaging device requiring expert reads for ground truth in the same way.
- The ground truth for the clinical studies was established by Plaque Reduction Neutralization Test (PRNT), which is considered the gold standard for serological confirmation of West Nile Virus infection.
- For the comparison studies, a legally marketed predicate device (Focus Diagnostics West Nile Virus IgG DxSelect™ (K031953)) also served as a comparative "ground truth" or reference, as is common in 510(k) submissions for substantial equivalence.
- The document does not mention the use of human experts or their qualifications for establishing the ground truth for data used in the clinical studies. The ground truth relies on established laboratory tests (PRNT) or comparison to an already FDA-cleared device.
4. Adjudication Method for the Test Set
- No explicit "adjudication method" in the sense of multiple human readers resolving discrepancies is mentioned for this IVD device.
- For Clinical Study I, it states: "Average of three results for each clinical specimen tested at three sites was considered to calculate the positive percent and negative percent agreement between the EUROIMMUN Anti-West Nile Virus ELISA (IgG) vs the predicate assay." This indicates internal averaging/calculation, not expert adjudication.
- The ground truth for discordant results was resolved by PRNT where applicable (e.g., of the 45 presumptive positives by predicate in Study I, 27 were sent for PRNT).
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance
- Not applicable. This is an in vitro diagnostic (ELISA) device, not an AI or imaging device that uses human readers.
6. If a Standalone (i.e., Algorithm Only Without Human-in-the-Loop Performance) Was Done
- Not applicable. This is an automated ELISA assay. Its "standalone performance" is what is described in the analytical and clinical studies. There is no human-in-the-loop component in the interpretation of the assay results, other than a lab technician running the test and reporting the quantitative ratio/index, which then falls into predefined qualitative categories (negative, borderline, positive).
7. The Type of Ground Truth Used (expert consensus, pathology, outcomes data, etc.)
The types of ground truth used were primarily:
- Reference Laboratory Assay: Plaque Reduction Neutralization Test (PRNT) is considered the gold standard for confirming West Nile Virus infection. (Used in Clinical Study I and III).
- Predicate Device Comparison: The Focus Diagnostics West Nile Virus IgG DxSelect™ (K031953), an already legally marketed device, served as a reference standard for comparison in Clinical Study I and II to demonstrate substantial equivalence.
- Serological Characterization: For the cross-reactivity study, specimens were "serologically characterized," implying prior established diagnostic results for the various pathogens.
- Clinically Characterized Patients: For the initial assay cut-off determination, samples were from "clinically characterized positive West Nile virus patients" and "normal healthy blood donors," suggesting a clinical diagnosis or healthy status served as the ground truth for defining the cut-off.
8. The Sample Size for the Training Set
- This is an IVD kit, not a machine learning model, so there isn't a "training set" in the typical AI sense.
- However, the assay cut-off determination can be seen as analogous to a model's calibration or training phase. For this, 18 sera from clinically characterized WNV positive patients and 150 sera from normal healthy blood donors were used to determine the optimal OD cut-off and borderline range.
9. How the Ground Truth for the Training Set Was Established
- As noted above, for the assay cut-off determination (analogous to training/calibration):
- The "positive" ground truth was established by using "clinically characterized positive West Nile virus patients." Further details about how these patients were characterized (e.g., by PRNT, by symptoms, by other diagnostics) are not provided in this document but are assumed to be based on an established clinical diagnosis of WNV infection.
- The "negative" ground truth was established by using "normal healthy blood donors from a non-endemic region." These individuals are assumed to be free of WNV infection based on their health status and origin from a non-endemic area.
§ 866.3940 West Nile virus serological reagents.
(a)
Identification. West Nile virus serological reagents are devices that consist of antigens and antisera for the detection of anti-West Nile virus IgM antibodies, in human serum, from individuals who have signs and symptoms consistent with viral meningitis/encephalitis. The detection aids in the clinical laboratory diagnosis of viral meningitis/encephalitis caused by West Nile virus.(b)
Classification. Class II (special controls). The special control is FDA's guidance entitled “Class II Special Controls Guidance Document: Serological Reagents for the Laboratory Diagnosis of West Nile Virus.” See § 866.1(e) for the availability of this guidance document.