Search Results

The EUROIMMUN Anti-West Nile Virus ELISA (IgM) is intended for the qualitative detection of IgM antibodies to West Nile virus in human serum and plasma (K+EDTA, Li+-heparin). This test is intended as an aid in the presumptive laboratory diagnosis of West Nile virus infection in patients with clinical symptoms consistent with meningitis/encephalitis, in conjunction with other laboratory and clinical findings. Positive results must be confirmed by the plaque reduction test (PRNT) or by using the current CDC guidelines for diagnosis of this disease.

The assay characteristics have not been established for testing cord blood, neonates, prenatal screening, and general population screening of patients without symptoms of meningoencephalitis. This assay is not FDA cleared or approved for testing blood or plasma donors.

Warning: Cross-reactivity with IgM to Dengue virus, Malarialanti-Plasmodium falciparum and Parvovirus B19 has been observed with the EUROIMMUN Anti-West Nile Virus ELISA (IgM). Reactive results must be reported with a caution statement regarding possible IgM cross-reactivity with other flaviviruses.

Patient serum or plasma samples are diluted 1:101 in sample buffer and incubated for 10 minutes at room temperature to allow IgG/RF separation. 100 ul of each diluted patient sample and pre-diluted controls and calibrator are added to the antigen coated microtiter wells and incubated for 60 minutes at +37°C. After incubation the microtiter well strips are washed 3 times with wash buffer to remove unbound antibodies and 100 ul of the anti-human IgM enzyme conjugate reagent is added to each microtiter well. After an additional 30 minutes incubation at room temperature, the microtiter wells are again washed 3 times with wash buffer to remove any unbound enzyme conjugate and 100 ul of the chromogen substrate is added. The strips are incubated for 15 minutes at room temperature and 100 ul stop solution is added. The microtiter placed in an ELISA reader and read at a wavelength of 450 nm and a reference wavelength of between 620 nm and 650 nm within 30 minutes.

Here's an analysis of the provided text regarding the acceptance criteria and study for the EUROIMMUN Anti-West Nile Virus ELISA (IgM):

Note: This document describes an in vitro diagnostic (IVD) device, not a typical AI/ML device. Therefore, the concepts of "test set," "training set," "experts to establish ground truth," "adjudication method," and "MRMC comparative effectiveness study" do not directly apply in the same way they would to an AI-powered image analysis or diagnostic tool. Instead, the performance is evaluated against a predicate device and/or a reference standard like PRNT (Plaque Reduction Neutralization Test), and clinical studies involve comparisons of the device's results with these established methods.

Acceptance Criteria and Reported Device Performance

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria for this device, being a diagnostic test, are generally based on sensitivity and specificity relative to a reference standard or a predicate device. The document explicitly states the ranges for these measures.

Ask a specific question about this device

The EUROIMMUN Anti-West Nile Virus ELISA (IgG) is intended for the qualitative detection of IgG antibodies to West Nile virus in human serum and plasma (K+EDTA, Li+-heparin). This test is intended as an aid in the presumptive laboratory diagnosis of West Nile virus infection in patients with clinical symptoms consistent with meningitis encephalitis, in conjunction with other laboratory and clinical findings. Positive results must be confirmed by the plaque reduction test (PRNT) or by using the current CDC guidelines for diagnosis of this disease.

The assay characteristics have not been established for testing cord blood, neonates, prenatal screening, and general population screening of patients without symptoms of meningoencephalitis. This assay is not FDA cleared or approved for testing blood or plasma donors.

Warning: Cross-reactivity with IgG to Dengue, Chikungunya, Zika and Tick-borne Encephalitis (TBE) viruses has been observed with the EUROIMMUN Anti-West Nile Virus ELISA (IgG). Reactive results must be reported with a caution statement regarding possible IgG cross-reactivity with other flaviviruses.

Patient samples are diluted 1:101 in sample buffer, 100 ul of each diluted patient sample and pre-diluted controls and the calibrator are added to the antigen coated microtiter wells and incubated for 60 minutes at +37°C. After incubation the microtiter well strips are washed 3 times with wash buffer to remove unbound antibodies and 100 ul of the anti-human IgG enzyme conjugate reagent is added to each microtiter well. After an additional 30-minutes incubation at room temperature, the microtiter wells are again washed 3 times with wash buffer to remove any unbound enzyme conjugate and 100 ul of the chromogen substrate is added. The strips are incubated for 15 minutes at room temperature and 100 ul stop solution is added. The microtiter plates are placed in an ELISA reader and read at a wavelength of 450 nm and a reference wavelength of between 620 nm and 650 nm within 30 minutes.

The provided text describes the EUROIMMUN Anti-West Nile Virus ELISA (IgG) device, which is an in vitro diagnostic immunoassay. Therefore, the questions related to AI/MRMC studies, human-in-the-loop performance, and expert ground truth are not applicable in the context of this device. The acceptance criteria and study data are presented for the assay's analytical and clinical performance.

Here's the breakdown of the information based on the provided document:

1. Acceptance Criteria and Reported Device Performance

The acceptance criteria for the EUROIMMUN Anti-West Nile Virus ELISA (IgG) are primarily based on its analytical performance (repeatability, reproducibility, specificity) and clinical performance (sensitivity and specificity compared to a predicate device and PRNT).

2. Sample Size Used for the Test Set and Data Provenance

The "test set" here refers to the clinical samples used for performance validation against a predicate device and ground truth (PRNT).

• Clinical Study I (Predicate Comparison and PRNT Confirmation):
  • Sample Size: 152 samples for predicate comparison. Of these, 45 were presumptive positives by the predicate, and 27 of these were further tested by PRNT.
  • Data Provenance: Prospective collection from hospitals and clinics across the US in 2015.
• Clinical Study II (Predicate Comparison):
  • Sample Size: 401 serum samples.
  • Data Provenance: Prospective collection from a clinical laboratory in the midwest (US).
• Clinical Study III (PRNT Comparison):
  • Sample Size: 295 serum samples.
  • Data Provenance: Retrospective collection in cooperation with the Robert Koch Institute (RKI), Berlin, Germany, including 200 samples from major outbreaks of West Nile fever in South Africa in 1974 and 1984.
• Analytical Specificity/Cross-Reactivity Study:
  • Sample Size: 910 serologically characterized seropositive specimens.
  • Data Provenance: Not explicitly stated, but implies diverse sources given the range of conditions.
• Assay Cut-off Determination:
  • Sample Size: 18 sera from clinically characterized positive WNV patients and 150 sera from normal healthy blood donors.
  • Data Provenance: Implied clinical and healthy populations, source not specified.
• Expected Values (US Studies):
  • Sample Size: 553 samples.
  • Data Provenance: Prospectively collected from US population.
• Matrix Comparison:
  • Sample Size: 20 paired samples (serum, EDTA plasma, Li-heparin plasma), created by spiking 5 different sets of normal blood donor samples with 5 different positive serum samples.
  • Data Provenance: Normal blood donors (source not specified), spiked with serum samples (source not specified if human/animal).

3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts

• This is an in vitro diagnostic (IVD) device, not an AI/imaging device requiring expert reads for ground truth in the same way.
• The ground truth for the clinical studies was established by Plaque Reduction Neutralization Test (PRNT), which is considered the gold standard for serological confirmation of West Nile Virus infection.
• For the comparison studies, a legally marketed predicate device (Focus Diagnostics West Nile Virus IgG DxSelect™ (K031953)) also served as a comparative "ground truth" or reference, as is common in 510(k) submissions for substantial equivalence.
• The document does not mention the use of human experts or their qualifications for establishing the ground truth for data used in the clinical studies. The ground truth relies on established laboratory tests (PRNT) or comparison to an already FDA-cleared device.

4. Adjudication Method for the Test Set

• No explicit "adjudication method" in the sense of multiple human readers resolving discrepancies is mentioned for this IVD device.
• For Clinical Study I, it states: "Average of three results for each clinical specimen tested at three sites was considered to calculate the positive percent and negative percent agreement between the EUROIMMUN Anti-West Nile Virus ELISA (IgG) vs the predicate assay." This indicates internal averaging/calculation, not expert adjudication.
• The ground truth for discordant results was resolved by PRNT where applicable (e.g., of the 45 presumptive positives by predicate in Study I, 27 were sent for PRNT).

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance

• Not applicable. This is an in vitro diagnostic (ELISA) device, not an AI or imaging device that uses human readers.

6. If a Standalone (i.e., Algorithm Only Without Human-in-the-Loop Performance) Was Done

• Not applicable. This is an automated ELISA assay. Its "standalone performance" is what is described in the analytical and clinical studies. There is no human-in-the-loop component in the interpretation of the assay results, other than a lab technician running the test and reporting the quantitative ratio/index, which then falls into predefined qualitative categories (negative, borderline, positive).

7. The Type of Ground Truth Used (expert consensus, pathology, outcomes data, etc.)

The types of ground truth used were primarily:

• Reference Laboratory Assay: Plaque Reduction Neutralization Test (PRNT) is considered the gold standard for confirming West Nile Virus infection. (Used in Clinical Study I and III).
• Predicate Device Comparison: The Focus Diagnostics West Nile Virus IgG DxSelect™ (K031953), an already legally marketed device, served as a reference standard for comparison in Clinical Study I and II to demonstrate substantial equivalence.
• Serological Characterization: For the cross-reactivity study, specimens were "serologically characterized," implying prior established diagnostic results for the various pathogens.
• Clinically Characterized Patients: For the initial assay cut-off determination, samples were from "clinically characterized positive West Nile virus patients" and "normal healthy blood donors," suggesting a clinical diagnosis or healthy status served as the ground truth for defining the cut-off.

8. The Sample Size for the Training Set

• This is an IVD kit, not a machine learning model, so there isn't a "training set" in the typical AI sense.
• However, the assay cut-off determination can be seen as analogous to a model's calibration or training phase. For this, 18 sera from clinically characterized WNV positive patients and 150 sera from normal healthy blood donors were used to determine the optimal OD cut-off and borderline range.

9. How the Ground Truth for the Training Set Was Established

• As noted above, for the assay cut-off determination (analogous to training/calibration):
  • The "positive" ground truth was established by using "clinically characterized positive West Nile virus patients." Further details about how these patients were characterized (e.g., by PRNT, by symptoms, by other diagnostics) are not provided in this document but are assumed to be based on an established clinical diagnosis of WNV infection.
  • The "negative" ground truth was established by using "normal healthy blood donors from a non-endemic region." These individuals are assumed to be free of WNV infection based on their health status and origin from a non-endemic area.

Ask a specific question about this device

The Spectral West Nile virus IgM STATus™ test is a rapid immunochromatographic lateral flow assay that utilizes recombinant West Nile virus (WNV) antigen (E glycoprotein) for the qualitative detection of IgM antibodies to WNV in human serum or plasma (sodium heparin or sodium citrate). This test is for the presumptive laboratory diagnosis of West Nile virus infection in patients having signs and symptoms of meningoencephalitis. Positive results must be confirmed by the PRNT (Plaque Neutralization Reduction Test), or by using the current CDC guidelines for diagnosis of this disease. The Spectral test is not intended for point of care testing, home use or in screening blood donor samples.

A Simple and Rapid Immunoassay for the Qualitative Detection of West Nile Virus IgM Antibodies in Human Serum or Plasma. The Spectral WNV IgM STATus™ test employs solid-phase immunochromatographic assay technology to qualitatively detect the presence of WNV IgM antibodies in serum or plasma. When the specimen to be tested is dispensed into the sample well of the Spectral device, anti-WNV IgM in the sample will bind to the recombinant WNV antigen (envelop glycoprotein (E) of West Nile virus, NY99 strain) to form a tertiary complex with gold-labeled monoclonal murine antibody against flavivirus family glycoprotein E. This tertiary complex will migrate through reaction strip and be captured by goat anti-human IgM antibodies at the Test area. Excess, unreacted gold complex detector is captured by immobilized anti-mouse IgG antibodies at the Control area. A visible pinkish-purple horizontal band will appear in the Test area within 15 minutes following the addition of a sample if the level of the WNV IgM antibodies in the human serum sample is above the cut-off level. A pinkish-purple band in the Control area indicates that the test is working properly and such a band must always appear, irrespective of the WNV IgM levels, in order for the test to be valid.

Acceptance Criteria and Device Performance Study for Spectral West Nile Virus IgM STATus™ Test

This document outlines the acceptance criteria and the studies that demonstrate the Spectral West Nile Virus IgM STATus™ Test meets these criteria, based on the provided 510(k) summary (K052519).

1. Table of Acceptance Criteria and Reported Device Performance

The document does not explicitly state pre-defined acceptance criteria with specific numerical targets in a formal table. However, based on the performance data presented, the implicit acceptance criteria appear to be high agreement (sensitivity and specificity/negative agreement) with a legally marketed predicate device (Focus West Nile Virus IgM Capture ELISA) and reference methods like PRNT, along with robust reproducibility.

2. Sample Size Used for the Test Set and Data Provenance

The test set consisted of several distinct groups of samples tested at different study sites.

• Non-flavivirus IgM samples (for Negative Agreement):
  • Total Sample Size: 346 specimens.
  • Provenance: Prospectively collected from patients with a variety of non-flavivirus ailments (e.g., rash, febrile, bronchitis, diarrhea, drug, neuropathy, acute coronary syndrome) and from endemic normal populations.
  • Country of Origin:
    • Site 1: South-Western State of America (endemic region).
    • Sites 2 & 3: South Eastern Provinces of Canada (endemic regions).
• WNV PRNT Confirmed / CDC WNV IgM ELISA Negative specimens (Study Site 4):
  • Total Sample Size: 90 specimens (40 WNV PRNT confirmed; 50 CDC WNV IgM ELISA negative).
  • Provenance: Randomized retrospective patient serum specimens from a provincial health laboratory, sent with suspected WNV infection.
  • Country of Origin: South Eastern Province of Canada.
• Banked Panel of Clinical Serum Specimens (Study Site 5):
  • Total Sample Size: 79 specimens.
  • Provenance: Banked clinical serum specimens from patients with clinical symptomology consistent with WNV infection (including symptomatic acutely infected, neuro-invasive disease, and asymptomatic for previous exposures).
  • Country of Origin: Prairie Province at Central Canada.
• Reproducibility Panel: 15 clinical specimens.
• Interfering Substances Panel: Positive and negative serum specimens spiked with specific substances.
• IgM Specificity Panel: 14 paired positive samples.
• Cross-Reactivity Panel: Varied number of samples for each potentially cross-reactive agent (e.g., 10 for HSV, 28 for ANA, 33 for Rheumatoid Factor).
  • Provenance: Sera sero-positive to other potentially cross-reactive pathogens.
  • Country of Origin: North Eastern USA (site 1), Mid-West Canada (site 3 for Dengue), in-house (site 2 for ANA/RF), CDC laboratory at Mid-West USA (site 4 for EEE).

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications

The document does not specify the number or qualifications of experts used to establish the ground truth for the test set. Instead, it relies on established reference methods and the comparator predicate device.

4. Adjudication Method for the Test Set

The document does not mention an explicit adjudication method for the test set (e.g., 2+1, 3+1). The "ground truth" was established by reference assays or clinical characterization of samples.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

There is no indication of a multi-reader multi-case (MRMC) comparative effectiveness study or any assessment of human readers improving with AI vs. without AI assistance. This device is a diagnostic assay (rapid immunochromatographic test) designed for laboratory use, not an AI-assisted diagnostic imaging or interpretation tool for human readers.

6. Standalone Performance Study

Yes, a standalone performance study was conducted. The "Method Comparison" section explicitly tests the Spectral device against a comparator device (Focus West Nile Virus IgM Capture ELISA) and other reference methods like PRNT and CDC WNV IgM ELISA. The reported performance metrics (e.g., negative agreement, serological sensitivity) are measures of the algorithm's (device's) performance without human-in-the-loop intervention beyond reading the visible bands.

7. Type of Ground Truth Used

The ground truth for the different test sets was established using a combination of the following:

• Reference Devices/Assays:
  • Comparator device: Focus West Nile Virus IgM Capture ELISA (for negative agreement studies).
  • PRNT (Plaque Neutralization Reduction Test): Considered a gold standard for confirming WNV infection. Used for characterizing positive WNV samples at Study Site 4 and 5.
  • CDC WNV IgM ELISA and IgG ELISA: Used as reference assays for characterizing WNV positive and negative samples at Study Site 4 and 5.
• Clinical Characterization: Samples at Study Site 4 were from patients with "suspected WNV infection based on clinical signs and symptoms." Samples at Study Site 5 were from patients with "clinical symptomology consistent with the West Nile virus infection."
• Pathogen Sero-positivity: For cross-reactivity studies, samples were characterized as sero-positive to other potentially cross-reactive pathogens (e.g., Herpes Simplex Virus, Cytomegalovirus, Syphilis, Epstein Barr Virus, ANA, Rheumatoid Factor, HIV, certain Encephalitis viruses, Dengue Virus, Hepatitis B/C, Legionella, Yellow Fever, E. coli infection).

8. Sample Size for the Training Set

The document does not provide any information regarding a training set or its sample size. This is a rapid diagnostic test, and the reported studies focus on validation and performance characteristics as opposed to machine learning model development which would typically involve distinct training and test sets.

9. How the Ground Truth for the Training Set Was Established

Since no information on a training set is provided, the method for establishing its ground truth is also not available in the document.

Ask a specific question about this device

The West Nile Detect IgM Capture ELISA is for the qualitative detection of IgM antibodies to WNV recombinant antigens (WNRA) in serum for the presumptive clinical laboratory diagnosis of West Nile virus infection in patients with clinical symptoms consistent with meningoencephalitis. Positive results must be confirmed by Plaque Reduction Neutralization Test (PRNT), or by using the current CDC guidelines for diagnosis of this disease. Assay performance characteristics have not been established for testing cord blood, neonate, prenatal screening, general population screening without symptoms, or automated instruments. This assay is not FDA cleared or approved for testing blood or plasma donors.

West Nile Detect IgM Capture ELISA

I am sorry, but the provided text from the FDA 510(k) letter for the "West Nile Detect IgM Capture ELISA" device does not contain the specific information requested to describe the acceptance criteria and the study that proves the device meets those criteria, and therefore, I cannot fulfill your request to create the tables and sections you've outlined.

The document is a clearance letter indicating that the FDA found the device substantially equivalent to a predicate device. It defines the "Indications For Use" and general regulatory requirements but does not detail the performance study, acceptance criteria, or the specific results of such a study.

Specifically, the following information is missing from the provided text:

• Table of acceptance criteria and reported device performance: This information would typically be in the 510(k) submission's performance data section, not the clearance letter.
• Sample sizes used for the test set and data provenance: No details on the test set or its origin.
• Number of experts and their qualifications for ground truth: Not mentioned.
• Adjudication method for the test set: Not mentioned.
• Multi-reader multi-case (MRMC) comparative effectiveness study: No mention of such a study or any effect size.
• Standalone (algorithm-only) performance: This is an ELISA kit, not an algorithm in the typical sense for standalone performance.
• Type of ground truth used: While "presumptive clinical diagnosis" and "Plaque Reduction Neutralization Test (PRNT)" are mentioned as confirmation for positive results, the specifics of how the ground truth for the study was established are not provided.
• Sample size for the training set: Not applicable for an ELISA kit in the same way it would be for an AI algorithm, and in any case, not provided.
• How ground truth for the training set was established: Not applicable/not provided.

To obtain this information, one would typically need to consult the full 510(k) submission document, which often contains detailed study protocols and results, or scientific publications related to the validation of this specific assay. The provided FDA clearance letter is a summary of the regulatory outcome, not the detailed technical data.

Ask a specific question about this device

The PANBIO West Nile Virus IgG Indirect ELISA is for the qualitative presumptive detection of IgG antibodies to West Nile virus in serum. In conjunction with the PANBIO West Nile Virus IgM Capture ELISA, this test is intended as an aid in the clinical laboratory diagnosis of West Nile virus infection in patients with clinical symptoms consistent with encephalitis / meningitis. Positive results must be confirmed by plaque reduction neutralization test (PRNT), or by using the current Centers for Disease Control and Prevention (CDC) guidelines for diagnosis of this disease.

The PANBIO West Nile Virus IgG Indirect ELISA is for the qualitative detection of IgG antibodies to West Nile virus in serum. In conjunction with the PANBIO West Nile Virus IgM Capture ELISA, this test is intended as an aid in the presumptive laboratory diagnosis of West Nile virus infection in patients with clinical symptoms consistent with encephalitis / meningitis.

Acceptance Criteria and Device Performance for PANBIO West Nile Virus IgG Indirect ELISA

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria for this device are not explicitly stated as numerical targets in the provided text (e.g., "sensitivity must be > X%"). Rather, the studies aim to demonstrate adequate performance through descriptive statistical measures (sensitivity, specificity, positive and negative presumptive agreement) and reproducibility. The performance is reported with 95% Confidence Intervals. The cross-reactivity study aims to show the device's analytical specificity.

2. Sample Sizes and Data Provenance for Test Sets

• Study Site 1 (Louisiana, USA):

  • Sample Size: 300 retrospective sera.
  • Data Provenance: Louisiana, USA. Retrospective samples collected in 2002-2003.

• Study Site 2 (Utah, USA):

  • Sample Size: 325 retrospective sera.
  • Data Provenance: Utah, USA. Retrospective samples.

• Study Site 3 (Ohio, USA):

  • Sample Size: 284 retrospective sera.
  • Data Provenance: Ohio, USA. Retrospective samples collected in 2002.

• Reproducibility Study:

  • Sample Size: 8 sera, tested 3 times each on 3 different assays (total 72 tests per parameter).
  • Data Provenance: One Australian study site and two study sites in the USA.

• Cross-Reactivity Study:

  • Sample Size: 314 specimens in total, with varying numbers per disease state.
  • Data Provenance: Multiple sites including PANBIO (Site 5), a state health laboratory in Louisiana (Site 1), a private reference laboratory in Utah (Site 2), a hospital laboratory in Ohio (Site 3), and a private research laboratory in Maryland (Site 6). The exact timing of collection for these cross-reactive samples is not specified beyond being "from patients with confirmed diseases other than WNV."

3. Number of Experts and Qualifications for Ground Truth

The document does not explicitly state the "number of experts" or their specific "qualifications" (e.g., "radiologist with 10 years of experience") for establishing the ground truth. However, the ground truth was established by:

• Plaque Reduction Neutralization Test (PRNT): This is a gold standard serological test, implying expert virological laboratory practices and interpretation.
• West Nile Virus IgG IFA (Immunofluorescence Assay): This is another standard serological method, likely performed and interpreted by trained laboratory personnel or experts.
• Clinical and serological characterization: This suggests a combination of clinical diagnosis and laboratory test results, presumably by medical professionals and laboratory specialists.
• Western blot analysis: Used for confirmation of flavivirus positivity in cross-reactivity samples, indicating expert serological techniques.

4. Adjudication Method for the Test Set

The document does not describe a formal "adjudication method" involving multiple human readers or a consensus process for discrepant results in the context of device performance evaluation (e.g., 2+1, 3+1 for image interpretation). Instead, the comparison is directly between the PANBIO ELISA results and the established "gold standard" or "characterization" of the specimens (PRNT, IgG IFA, or clinical characterization).

Equivocal results from the PANBIO ELISA were generally "not retested" due to sample unavailability or because the cut-off was modified post-clinical trials, indicating that a formal re-adjudication process for equivocal outcomes was not systematically applied within these studies.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No Multi-Reader Multi-Case (MRMC) comparative effectiveness study was done. The device is an in-vitro diagnostic (IVD) assay designed for laboratory testing, not an imaging device requiring human-in-the-loop interpretation. Therefore, the concept of human readers improving with or without AI assistance is not applicable to this device.

6. Standalone Performance

Yes, a standalone performance study was done. The entire evaluation presented (sensitivity, specificity, presumptive agreements, reproducibility, and cross-reactivity) describes the performance of the PANBIO West Nile Virus IgG Indirect ELISA algorithm/assay itself, without human-in-the-loop assistance in its operation or primary interpretation. The results are based on how the assay performs when processing samples and generating qualitative results (Positive, Equivocal, Negative).

7. Type of Ground Truth Used

The ground truth used for the test sets included:

• Expert Consensus/Reference Methods: Primarily, Plaque Reduction Neutralization Test (PRNT) and West Nile Virus IgG Immunofluorescence Assay (IFA) were used as reference methods to characterize samples as positive or negative for West Nile Virus IgG. These are established laboratory methods considered reliable.
• Clinical Characterization: Samples from patients with "clinical symptoms consistent with encephalitis/meningitis" were also used, implying correlation with patient presentation alongside serological markers.
• Disease State Confirmation: For the cross-reactivity study, samples were characterized by their "disease state" (e.g., Dengue virus, St. Louis encephalitis, Cytomegalovirus) and for some flaviviruses, confirmed by Western blot analysis.

8. Sample Size for the Training Set

The document does not explicitly mention a "training set" or "validation set" in the context of machine learning. This is an IVD assay, and the "training" of the assay's cut-off or parameters would typically involve internal development studies and optimization rather than a distinct "training set" like in AI/ML contexts. The performance characteristics described are based on testing against retrospectively collected "test sets" or panels of samples.

9. How the Ground Truth for the Training Set was Established

Since a distinct "training set" for an AI/ML algorithm is not described, the method for establishing its ground truth is not applicable in this context. The assay's performance evaluation relies on comparing its output to established reference methods (PRNT, IFA, Western blot) and clinical characterization using the "test sets" described above. The statement that "cut-off was modified following clinical trials" in Study Site 2 hints at an iterative process of optimizing the assay's interpretive criteria based on initial performance data.

Ask a specific question about this device

The PANBIO West Nile Virus IgM Capture ELISA is for the qualitative presumptive detection of IoM antibodies to West Nile virus in serum as an aid in the clinical laboratory diagnosis of West Nile virus infection in patients with clinical symptoms consistent with encephalitis / meningitis. Positive results must be confirmed by plaque reduction neutralization test (PRNT), or by using the current Centers for Disease Control and Prevention (CDC) guidelines for diagnosis of this disease.

Assay performance characteristics have not been established for testing cord blood, neonate, prenatal screening, general population screening without symptoms of meningioencephalitis or automated instruments. The user is responsible for establishing these assay performance characteristics.

Caution: Cross-reactivity has been noted with the PANBIO West Nile IgM assay in specimens containing rheumatoid factor (RF). Reactive results must be reported with a caution statement regarding possible cross-reactivity with RF.

The West Nile Virus IgM Capture ELISA is an Enzyme Linked Immunosorbent Assay (ELISA) for the qualitative detection of IgM antibodies to West Nile virus in serum as an aid in the clinical laboratory diagnosis of West Nile virus in patients with clinical symptoms consistent with encephalitis / meningitis.

Acceptance Criteria and Device Performance Study for PANBIO West Nile Virus IgM Capture ELISA

1. Table of Acceptance Criteria and Reported Device Performance

The document does not explicitly state pre-defined acceptance criteria (e.g., "The device must achieve at least X% sensitivity and Y% specificity"). However, the performance characteristics obtained serve as the benchmark for clearance. Given the context of a 510(k) summary, the "reported device performance" essentially functions as the de facto acceptance criteria, demonstrating substantial equivalence to the predicate device.

*CI = Exact confidence interval
¹ Sixteen samples testing indeterminate by IFA and negative by PANBIO were assigned to "IgM positive" for worst-case.
² Seven samples testing indeterminate by IFA and positive by PANBIO were assigned to "IgM negative" for worst-case.

2. Sample Sizes Used for the Test Set and Data Provenance

• Study Site 1:
  • Total Samples: 420 retrospective sera
  • PRNT-confirmed subset: 130 samples (61 WNV positive, 69 WNV negative)
  • CDC MAC EIA presumptively characterized subset: 73 samples (11 WNV positive, 62 WNV negative)
  • IgM IFA presumptively characterized subset: 419 samples (96 IgM positive, 23 indeterminate, 300 negative) (Note: The sum is 419, not 420. One sample might have been excluded or lost from the initial 420).
  • Data Provenance (Country of Origin and Retrospective/Prospective): Retrospective sera from individuals in various locations:
    • California, USA (blood donation center)
    • Maryland, USA (private reference laboratory)
    • Utah, USA (private reference laboratory)
    • Texas, USA (university medical branch)
    • Minnesota, USA (private laboratory)
    • Canada (government medical laboratory)
• Study Site 2:
  • Total Samples: 51 retrospective sera confirmed by PRNT. These were encephalitis/meningitis patients.
  • Data Provenance (Country of Origin and Retrospective/Prospective): Retrospective sera from patients collected in 2002 at a hospital laboratory in Ohio, USA.
• Specificity of IgM Detection: 10 serum samples for DTT treatment, 8 serum samples for IgG interference. Provenance not specified but likely conducted in-house.
• Reproducibility: Not directly a test set for clinical performance; 27 observations for each of 9 samples.
• Cross-Reactivity: 160 specimens from patients with confirmed diseases other than WNV. Provenance not explicitly stated for individual samples, but the study was conducted at "Study Site 1" and "Study Site 2" (Ohio) and in-house at PANBIO.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

The document does not explicitly state the number of experts or their specific qualifications (e.g., radiologist with X years of experience) who established the ground truth for the test sets. Instead, the ground truth was established by:

• Plaque Reduction Neutralization Test (PRNT): A laboratory method considered the gold standard for WNV confirmation. This is a laboratory assay, not an expert panel judgment in the traditional sense.
• CDC MAC EIA: Presumptive characterization based on another laboratory assay.
• WNV IgM IFA: Presumptive characterization based on another laboratory assay.

The characterization of specimens was performed by recognized laboratory methods, not by human expert adjudication of individual cases.

4. Adjudication Method for the Test Set

The ground truth for the clinical performance studies was established using laboratory reference assays:

• Plaque Reduction Neutralization Test (PRNT): Used to "confirm" WNV positive/negative status for the primary sensitivity/specificity calculations. This acts as the independent "adjudicator."
• CDC MAC EIA and WNV IgM IFA: Used for presumptive characterization for additional agreement studies.

There was no human expert adjudication method described (e.g., 2+1, 3+1 consensus) for the test sets. The laboratory results themselves served as the reference standard.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve With AI vs. Without AI Assistance

No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not done.

This device is an Enzyme Linked Immunosorbent Assay (ELISA) for the qualitative detection of IgM antibodies, which means it is a laboratory diagnostic test. It is not an AI-powered device or an imaging interpretation system that would involve human readers or AI assistance in the way typically discussed in MRMC studies. The device itself performs the detection.

6. If a Standalone (i.e., Algorithm Only Without Human-in-the-Loop Performance) Was Done

Yes, the performance study was a standalone (algorithm only) evaluation.

The PANBIO West Nile Virus IgM Capture ELISA is an in-vitro diagnostic device that directly produces a result (positive, equivocal, negative) based on the biochemical reaction. Its performance was evaluated by directly comparing its output to established reference laboratory methods (PRNT, CDC MAC EIA, IgM IFA). There is no "human-in-the-loop" component in the sense of a human interpreting the device's output to make a primary diagnosis; the output is the diagnostic aid.

7. The Type of Ground Truth Used

The ground truth used in the performance studies was primarily laboratory reference assays:

• Plaque Reduction Neutralization Test (PRNT): Considered the gold standard for WNV confirmation.
• CDC MAC EIA (ELISA): Another established laboratory assay for WNV IgM detection.
• WNV IgM IFA (Immunofluorescence Assay): Another laboratory assay for WNV IgM detection.

For the cross-reactivity study, the ground truth was based on specimens from patients with confirmed diseases other than WNV.

8. The Sample Size for the Training Set

The document does not explicitly mention a "training set" in the context of device development. This is typical for traditional ELISA-based diagnostic kits, where the assay principles and reagents are developed and optimized rather than "trained" in the machine learning sense. The performance characteristics are then verified using independent test sets as described.

9. How the Ground Truth for the Training Set Was Established

Since a "training set" (as understood in machine learning/AI) is not typically applicable to this type of device, the concept of establishing ground truth for it is also not relevant here. The device's components and parameters are likely optimized through laboratory R&D processes based on known positive and negative controls.

Ask a specific question about this device

The Focus Technologies West Nile Virus IgM Capture ELISA is intended for qualitatively detecting IgM antibodies to West Nile virus in human serum. In conjunction with the Focus Technologies West Nile Virus ELISA IgG, the test is indicated for testing persons having symptoms of meningioencephalitis, as an aid in the presumptive laboratory diagnosis of West Nile virus infection. Positive results must be tested using the background subtraction method (either on the initial test or on a repeat test). Positive results must be confirmed by neutralization test, or by using the current CDC guidelines for diagnosing West Nile encephalitis. This test is not intended for self-testing, and this test is not FDA cleared nor approved for testing blood or plasma donors. Assay performance characteristics have not been established for automated instruments.

Indirect Enzyme-linked immunosorbent assay for qualitatively detecting human serum IgM antibodies to West Nile virus.

Here's a summary of the acceptance criteria and the studies that demonstrate the device meets those criteria, based on the provided text:

Acceptance Criteria and Device Performance

The provided document does not explicitly state "acceptance criteria" in a separate, clear table with pre-defined thresholds. However, the performance characteristics studies implicitly define the expected performance benchmarks based on agreement with reference assays and clinical classifications. For the purpose of this response, I will synthesize the performance results as the "reported device performance" and infer "acceptance criteria" from the documented agreements and sensitivities/specificities deemed acceptable for regulatory submission.

Note on Background Subtract: The device's performance is consistently evaluated both "without Background Subtract" and "with Background Subtract." The Intended Use explicitly states that "Positive results must be tested using the background subtraction method," suggesting that the performance with background subtract is the clinically relevant and accepted performance. Therefore, the table below will primarily focus on the "with Background Subtract" results as the crucial performance metrics, but acknowledge the "without" where significant differences exist.

Study Details:

1. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

   • Study Site 1 (Encephalitis/Meningitis Patients):
     • Sample Size: 300 patients. 44 confirmed positive (WNV encephalitis/meningitis symptoms, CDC IgM ELISA positive and WNV PRNT positive), 256 presumptive results (250 presumed negative, 2 presumed positive, 4 excluded).
     • Data Provenance: Sera were sequentially submitted to a state department of health laboratory located in the northeastern U.S., archived, and masked. Prospective collection from clinical suspicion.
   • Study Site 2 & 4 (WNV PRNT Positives):
     • Sample Size: 75 retrospective samples (70 ultimately tested with background subtract).
     • Data Provenance: Retrospective samples from a clinical laboratory (mid-western U.S.) with no clinical information, pre-screened positive by Focus and confirmed by WNV PRNT.
   • Study Site 3 (West Nile IFA Negatives):
     • Sample Size: 103 samples.
     • Data Provenance: Reactive samples that were West Nile IFA negative, from a clinical laboratory located in the southwestern U.S. (Retrospective implied by "assessed reactive samples").
   • Study Site 4 (Suspected Encephalitis/Meningitis Patients):
     • Sample Size: 50 samples.
     • Data Provenance: Archived and masked sera from a U.S. federal government laboratory. One confirmed positive by WNV PRNT, 49 presumptively negative by CDC ELISA for arboviruses.
   • Study Site 4 (Non-Flavivirus Test Samples):
     • Sample Size: 476 samples (4 samples were indeterminant with CDC IgM ELISA and excluded from final calculations).
     • Data Provenance: Prospectively collected from North America during August 2003. Submitted to a clinical laboratory in Southern California for non-flavivirus tests.
   • Cross-Reactivity Study (Study Site 1 & 4):
     • Sample Size: 75 samples sero-positive to other potentially cross-reactive pathogens.
     • Data Provenance: Retrospective and masked sera. DOH (northeastern U.S.) tested St. Louis encephalitis positives, Focus (Study Site 4) tested others.
   • Specificity (2-ME treatment, Study Site 4):
     • Sample Size: 15 sera.
     • Data Provenance: Sera positive for both WNV IgM and IgG.
   • Freeze-Thaw Study (Study Site 4):
     • Sample Size: 8 sera (5 positive, 3 negative).
     • Data Provenance: N/A (likely in-house selection).
   • Reproducibility Studies (Inter-lot, Inter/Intra-assay, Inter-laboratory):
     • Sample Size: Varies by study: 5 samples for inter-lot, 7 for inter/intra-assay (63 data points), 7 for inter-laboratory. Masked duplicates were used.
     • Data Provenance: Study Site 4 (Focus), Study Site 5 (mid-western U.S. clinical lab), Study Site 6 (northern California university lab), Study Site 1 (northeastern U.S. state DOH lab), Study Site 2 (mid-western U.S. clinical lab assumed different from site 5 for variety).

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)
   The document does not specify the "number of experts" or their "qualifications" involved in establishing the ground truth. Ground truth was primarily established using reference assays such as:

   • CDC West Nile Virus IgM ELISAs
   • West Nile Virus Plaque Reduction Neutralization Test (WNV PRNT)
   • West Nile IFA
   • Clinical criteria for encephalitis/meningitis (fever, altered mental status, CSF pleocytosis, etc.)
     These reference methods are established laboratory techniques, implying their performance is well-understood and accepted, but the exact human expert involvement in interpreting these for ground truth is not detailed.

3. Adjudication method (e.g. 2+1, 3+1, none) for the test set
   The document does not describe an explicit adjudication method (like 2+1 or 3+1 consensus). Ground truth was determined by comparing the device's results to a predefined reference standard (e.g., CDC IgM ELISA, WNV PRNT, WNV IFA, or clinical criteria), which implicitly serves as the adjudication or definitive determination.

4. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
   This is an in vitro diagnostic (IVD) device, specifically an ELISA assay. The concept of "human readers" for an "AI" assistance is not applicable in the context of this type of diagnostic test. The results of an ELISA are typically read by a spectrophotometer and interpreted based on a cutoff value, rather than a human "reader" making subjective interpretations that could be "assisted" by AI. Therefore, an MRMC comparative effectiveness study involving human readers with/without AI assistance was not done.

5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
   Yes, the studies presented are standalone performance evaluations of the Focus Technologies West Nile Virus IgM Capture ELISA. The device (an immunoassay kit) itself is the "algorithm" in this context, providing a quantitative output that is then interpreted against a cutoff. The performance metrics (sensitivity, specificity, agreement) directly reflect the device's ability to classify samples independently of human interpretational variability beyond reading the spectrophotometer. The "background subtract procedure" is an integral part of the assay's interpretive algorithm to mitigate false positives, as described in the "Test Principle" section.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
   The ground truth was established using reference laboratory assays (CDC WNV IgM ELISA, WNV PRNT, WNV IFA) and clinical diagnostic criteria for encephalitis/meningitis (which implicitly involves expert clinical judgment and outcomes). In some cases, previous positive screening results by other methods were also used to enrich the sample set. There is no mention of pathology (histopathological examination) or long-term outcomes data being used directly as ground truth for all studies, although the "confirmed" positive cases were patients with symptoms of meningioencephalitis.

7. The sample size for the training set
   The document does not specify a separate "training set" for the device. As an ELISA kit, it is a chemical/biological assay rather than a machine learning algorithm that requires explicit training data in the same sense. The performance characteristics were evaluated on various test sets as described above, but these are typically considered validation sets for an established assay methodology, rather than a "training set" that a machine learning model would use to learn parameters.

8. How the ground truth for the training set was established
   Since no explicit "training set" is mentioned in the context of a machine learning algorithm, this question is not directly applicable. If one considers the development and optimization of the ELISA assay itself, the ground truth would have been established during the research and development phase through various experiments using known positive and negative samples, similar to the reference assays used in the validation studies (e.g., using WNV PRNT confirmed samples, CDC ELISAs). However, the document does not detail this R&D process.

Ask a specific question about this device

The Focus Technologies West Nile Virus IgM Capture ELISA is intended for qualitatively detecting IgM antibodies to West Nile virus in human serum. In conjunction with the Focus Technologies West Nile Virus ELISA IgG, the test is indicated for testing persons having symptoms of meningioencephalitis, as an aid in the presumptive laboratory diagnosis of West Nile virus infection. Positive results must be confirmed by neutralization test, or by using the current CDC guidelines for diagnosing West Nile encephalitis. This test is not intended for self-testing, and this test is not FDA cleared nor approved for testing blood or plasma donors. Assay performance characteristics have not been established for automated instruments.

Indirect Enzyme-linked immunosorbent assay for qualitatively detecting human serum IgM antibodies to West Nile virus.

Here's a summary of the acceptance criteria and the studies performed for the Focus Technologies West Nile Virus IgM Capture ELISA, based on the provided text:

Acceptance Criteria and Device Performance

The acceptance criteria for this device are not explicitly stated in a quantitative table format (e.g., "sensitivity must be >X%"). Instead, the performance characteristics are presented as observed percentages and 95% confidence intervals from various studies, which are then compared to relevant reference methods. The implicit acceptance criteria are that the device performs comparably or acceptably against established reference methods like the CDC IgM ELISA and Plaque Reduction Neutralization Test (PRNT) for diagnosing West Nile virus infection.

Table of Acceptance Criteria (Implicit) and Reported Device Performance:

Study Details:

1. Sample sizes used for the test set and the data provenance:

   • Study Site 1 (Encephalitis/Meningitis Patients):
     • Test Set Size: 300 patients.
     • Data Provenance: Sera sequentially submitted to a state department of health laboratory in the northeastern U.S.; archived and masked. Likely retrospective, given the archival nature.
   • Study Site 2 (WNV PRNT Positives):
     • Test Set Size: 75 samples.
     • Data Provenance: Samples prescreened positive (by Focus) with a WNV native antigen ELISA and confirmed WNV positive by PRNT. From a clinical laboratory in the mid-western U.S.; archived. Likely retrospective.
   • Study Site 3 (West Nile IFA Negatives):
     • Test Set Size: 103 retrospective samples.
     • Data Provenance: From a clinical laboratory in the southwestern U.S.; retrospective.
   • Study Site 4 (Suspected Encephalitis/Meningitis Patients):
     • Test Set Size: 50 samples.
     • Data Provenance: Archived and masked sera provided by a U.S. federal government laboratory. Likely retrospective.
   • Study Site 4 (Non-Flavivirus Test Samples):
     • Test Set Size: 476 samples.
     • Data Provenance: Prospectively collected from North America during August 2003, submitted to a clinical laboratory in Southern California for non-flavivirus tests. Prospective.
   • Cross-reactivity Study:
     • Test Set Size: 75 samples (various pathogens).
     • Data Provenance: From Study Site 4 (Focus) and Study Site 1 (DOH for SLE positives). Sera were retrospective and masked.
   • Specificity of IgM Capture Wells Study:
     • Test Set Size: 15 sera.
     • Data Provenance: Not explicitly stated, but likely in-house from Focus (Study Site 4) using known WNV IgM and IgG positive samples.
   • Sera Freeze-Thaw Study:
     • Test Set Size: 8 sera (5 positive, 3 negative).
     • Data Provenance: Not explicitly stated, likely in-house from Focus (Study Site 4).
   • Reproducibility Studies:
     • Inter-lot and Inter/Intra-assay: 5-7 samples in duplicates/triplicates across different lots/days.
     • Inter-laboratory: Samples run in triplicate at 3 different labs on 3 different days.
     • Data Provenance: Focus (Study Site 4), a state department of health laboratory in the northeastern U.S. (Study Site 1), and a clinical laboratory in the mid-western U.S. (Study Site 2).

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • The document does not specify the number of experts or their qualifications. The ground truth was established by reference laboratory assays (e.g., CDC IgM ELISA, WNV PRNT, WNV IgM IFA), which were performed by trained personnel in state department of health or clinical laboratories. The expertise lies in the established protocols and interpretation guidelines of these reference methods rather than individual expert adjudication of each case.

3. Adjudication method (e.g. 2+1, 3+1, none) for the test set:

   • There was no explicit "adjudication method" in the sense of multiple human readers/experts evaluating the cases. The ground truth was determined by comparing the device's results against established reference laboratory tests (CDC IgM ELISA, WNV PRNT, WNV IgM IFA). The results of these reference tests served as the primary basis for classifying samples as positive or negative. Certain equivocal results from the Focus assay were calculated as positive or negative for performance metric purposes, indicating a predefined rule for handling these, but not human adjudication.

4. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No, a multi-reader multi-case (MRMC) comparative effectiveness study was not performed. This device is an in vitro diagnostic (IVD) ELISA kit, not an AI-powered image analysis tool or a system designed to assist human readers in interpreting complex images. Its output is a quantitative optical density that is used to derive a qualitative (positive, negative, equivocal) result, which is then interpreted by laboratory personnel. Therefore, the concept of "human readers improve with AI vs without AI assistance" is not applicable here.

5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

   • Yes, this study represents a standalone performance evaluation of the Focus Technologies West Nile Virus IgM Capture ELISA. The device is a laboratory assay (ELISA kit) designed to provide a result directly from a patient sample. Its performance is evaluated intrinsically through its chemical and immunological reactions, and the optical density reading is interpreted by a predefined algorithm/cutoff. Human involvement is in performing the assay and reading the result, but the "performance" itself is that of the assay kit. This is the typical way IVD assays are evaluated.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc):

   • The ground truth was primarily established using established reference laboratory tests:
     • Plaque Reduction Neutralization Test (PRNT): Considered the gold standard for confirming WNV infection.
     • CDC West Nile Virus IgM ELISA: A widely accepted and validated method for detecting WNV IgM antibodies.
     • West Nile IFA (Immunofluorescence Assay): Another method used for WNV IgM antibody detection.
     • Clinical Criteria: For the encephalitis/meningitis patient cohort, a combination of clinical symptoms (fever, altered mental status, CSF pleocytosis) in conjunction with laboratory results defined the diagnosis.

7. The sample size for the training set:

   • The document does not explicitly describe a "training set" for the device in the context of machine learning. For an ELISA kit, development typically involves internal optimization and validation using various samples, but these are not defined as a distinct "training set" like in AI/ML. The performance data presented are from validation/test sets.

8. How the ground truth for the training set was established:

   • Since a formal "training set" as understood in AI/ML is not described, the method for establishing its ground truth is not applicable or detailed in this document. The development and internal validation of such an assay would involve using well-characterized samples (often confirmed by gold-standard methods like PRNT or a previously validated ELISA) to optimize reagent concentrations, incubation times, and cutoff values, but this process isn't reported as a "training set" with established ground truth in the same way as for AI models.

Ask a specific question about this device

The Focus Technologies West Nile Virus ELISA IgG is intended for qualitatively detecting IgG antibodies to West Nile virus in human serum. In conjunction with the Focus Technologies West Nile Virus IgM Capture ELISA, the test is indicated for testing persons having symptoms of meningioencephalitis, as an aid in the presumptive laboratory diagnosis of West Nile virus infection. Positive results must be confirmed by neutralization test, or by using the current CDC guidelines for diagnosing West Nile encephalitis. This test is not intended for self-testing, and this test is not FDA cleared nor approved for testing blood or plasma donors. Assay performance characteristics have not been established for automated instruments.

Indirect Enzyme-linked immunosorbent assay for qualitatively detecting human serum IgG antibodies to West Nile virus.

Here's a detailed breakdown of the acceptance criteria and the study that proves the Focus Technologies West Nile Virus ELISA IgG device meets them, based on the provided text:

Acceptance Criteria and Device Performance for Focus Technologies West Nile Virus ELISA IgG

The document does not explicitly state pre-defined acceptance criteria (e.g., "sensitivity must be > X%") for the device. Instead, it reports the device's performance characteristics from various studies. Therefore, the "acceptance criteria" presented below are inferred from the reported performance, implying that these levels of performance were deemed acceptable for market clearance.

1. Table of Acceptance Criteria (Inferred) and Reported Device Performance

2. Sample Size Used for the Test Set and Data Provenance

• Study Site 1 (Encephalitis/Meningitis Patients):
  • Sample Size: 300 total patients (37 confirmed positive, 210 presumptive results, 53 unclassified). Performance calculated on 242 samples (37 confirmed positive + 205 presumptive negative).
  • Data Provenance: Sera were sequentially submitted to a state department of health laboratory in the northeastern U.S., archived, and masked. Retrospective.
• Study Site 2 (WNV PRNT Positives):
  • Sample Size: 75 retrospective sera.
  • Data Provenance: Sera were sequentially submitted to a clinical laboratory in the mid-western U.S., archived. Retrospective.
• Study Site 3 (WNV IFA Negatives):
  • Sample Size: 157 samples.
  • Data Provenance: Retrospective sera from a clinical laboratory in the southwestern U.S.
• Study Site 4 (Suspected Encephalitis/Meningitis Patients):
  • Sample Size: 50 sera (1 confirmed positive, 49 presumptively negative).
  • Data Provenance: Retrospective and masked sera provided by a U.S. federal government laboratory.
• Study Site 4 (Non-Flavivirus Test Samples):
  • Sample Size: 476 samples. Performance calculated on 461 samples (21 CDC ELISA positive + 440 CDC ELISA negative).
  • Data Provenance: Prospectively collected from North America (August 2003) and submitted to a clinical laboratory in Southern California for non-flavivirus tests. Prospective, but the "test set" for WNV performance was created using the CDC ELISA.
• Cross-reactivity Studies:
  • Sample Size: 75 sera (20 Dengue, 20 Japanese encephalitis, 21 St. Louis encephalitis, 20 Yellow fever, 17 Alphavirus, 15 Bunyavirus, 60 HSV-1, 12 EBV, 20 Cytomegalovirus, 20 Echovirus/Poliovirus, 20 Lyme disease).
  • Data Provenance: Sera were archived and masked, conducted at Focus (Study Site 4, for most pathogens) and a state department of health laboratory in the northeastern U.S. (DOH, for SLE positives). Retrospective.
• Freeze-Thaw Study:
  • Sample Size: 8 sera (5 positive, 3 negative).
  • Data Provenance: Conducted at Focus (Study Site 4).
• Reproducibility Studies:
  • Sample Size: Inter-lot: 5 samples; Inter/Intra-assay: 7 samples; Inter-laboratory: 7 samples.
  • Data Provenance: Conducted at Focus (Study Site 4), a state department of health laboratory in the northeastern U.S. (Study Site 1), and a clinical laboratory in the mid-western U.S. (Study Site 2).

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

The document does not explicitly state the "number of experts" or their "qualifications" involved in establishing the ground truth. However, it indicates:

• Reference Methods Used:
  • CDC West Nile Virus IgG Capture ELISA
  • Plaque Reduction Neutralization Test (PRNT) for West Nile virus
  • Dengue PRNT
  • St. Louis Encephalitis (SLE) PRNT
  • West Nile Virus native antigen ELISA
  • West Nile IFA

These are established laboratory assays, and their results are considered the "ground truth." The implication is that skilled laboratory personnel at the respective state, federal, and clinical laboratories performed these reference tests. The qualifications of these individuals (e.g., highly trained lab technologists, clinical microbiologists) are inherent to the operation of such labs but are not specified in terms of "years of experience" or "radiologist" equivalent, as this is an in vitro diagnostic device.

4. Adjudication Method for the Test Set

The document does not describe an explicit "adjudication method" in the sense of multiple experts independently evaluating cases and resolving discrepancies (e.g., 2+1, 3+1).

Instead, the "ground truth" was established by reference laboratory assays (CDC WNV IgG ELISA, PRNT, IFA), which inherently have their own established protocols for result interpretation. For example, "confirmed positive West Nile encephalitis patients" were defined by symptoms and positive results from both CDC WNV IgG ELISA and WNV PRNT. Equivocal or indeterminant results from reference assays were often excluded from performance calculations.

5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study Was Done

No, a Multi Reader Multi Case (MRMC) comparative effectiveness study was not done. This type of study typically involves comparing human reader performance with and without an AI-assisted device in diagnostic imaging contexts. The Focus Technologies West Nile Virus ELISA IgG is an in vitro diagnostic laboratory test, not an imaging device intended for human interpretation.

6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Was Done

Yes, the studies conducted evaluated the standalone performance of the Focus Technologies West Nile Virus ELISA IgG device (an algorithm/assay only). The device itself is an indirect ELISA designed to qualitatively detect antibodies, implying that the interpretation of the optical density readings and the final result (positive, negative, equivocal) is performed by the device's internal algorithms or by comparison to defined cut-offs, without a "human-in-the-loop" decision support role in the sense of AI for image interpretation. The performance metrics presented (sensitivity, specificity/agreement) directly reflect the device's ability to classify samples based on its own output against established reference methods.

7. The Type of Ground Truth Used

The primary types of ground truth used were:

• Reference Laboratory Assay Results:
  • CDC West Nile Virus IgG Capture ELISA: A highly respected and commonly used reference method for WNV IgG.
  • Plaque Reduction Neutralization Test (PRNT): Considered the gold standard for serological confirmation of arbovirus infections due to its high specificity.
  • West Nile IFA (Immunofluorescence Assay): Another serological test.
  • Dengue PRNT, SLE PRNT: Used for cross-reactivity assessment.
  • West Nile virus native antigen ELISA: Used as a screening method in Study Site 2 before PRNT confirmation.
• Clinical Criteria: For Study Site 1 and 4, patient presentation (meningioencephalitis symptoms, fever, altered mental status, CSF findings) was combined with laboratory results to define confirmed cases.

This combination of highly specific and sensitive reference assays, often coupled with relevant clinical symptoms, provides a robust ground truth for an in vitro diagnostic device.

8. The Sample Size for the Training Set

The document does not explicitly mention a "training set" in the context of machine learning or AI development. This device is an ELISA, which is typically a biochemically-based assay, not an AI/ML algorithm that requires a separate training phase with labeled data in the same way. The performance studies evaluate the assay's built characteristics against clinical samples and reference methods.

If one were to loosely interpret "training" as the internal development and calibration of the assay (e.g., setting cut-offs), the size of the samples used for that purpose is not detailed. The provided studies are primarily for performance validation.

9. How the Ground Truth for the Training Set Was Established

As noted above, there isn't a "training set" in the AI/ML sense for this ELISA device. The concept of "ground truth" for the internal development and establishment of assay cut-offs would typically involve:

• Well-characterized panels: Samples from individuals with known WNV infection status (confirmed by PRNT or PCR) and known negative individuals.
• Titration and statistical analysis: To determine optimal cut-off values for distinguishing positive, negative, and equivocal results based on optical density readings.

However, the document focuses on the validation studies performed to demonstrate the device's performance against established reference methods and clinical populations, rather than detailing its initial development and calibration.

Ask a specific question about this device

Ask a specific question about this device