Search Results

The PANBIO West Nile Virus IgM Capture ELISA is for the qualitative presumptive detection of IoM antibodies to West Nile virus in serum as an aid in the clinical laboratory diagnosis of West Nile virus infection in patients with clinical symptoms consistent with encephalitis / meningitis. Positive results must be confirmed by plaque reduction neutralization test (PRNT), or by using the current Centers for Disease Control and Prevention (CDC) guidelines for diagnosis of this disease.

Assay performance characteristics have not been established for testing cord blood, neonate, prenatal screening, general population screening without symptoms of meningioencephalitis or automated instruments. The user is responsible for establishing these assay performance characteristics.

Caution: Cross-reactivity has been noted with the PANBIO West Nile IgM assay in specimens containing rheumatoid factor (RF). Reactive results must be reported with a caution statement regarding possible cross-reactivity with RF.

The West Nile Virus IgM Capture ELISA is an Enzyme Linked Immunosorbent Assay (ELISA) for the qualitative detection of IgM antibodies to West Nile virus in serum as an aid in the clinical laboratory diagnosis of West Nile virus in patients with clinical symptoms consistent with encephalitis / meningitis.

Acceptance Criteria and Device Performance Study for PANBIO West Nile Virus IgM Capture ELISA

1. Table of Acceptance Criteria and Reported Device Performance

The document does not explicitly state pre-defined acceptance criteria (e.g., "The device must achieve at least X% sensitivity and Y% specificity"). However, the performance characteristics obtained serve as the benchmark for clearance. Given the context of a 510(k) summary, the "reported device performance" essentially functions as the de facto acceptance criteria, demonstrating substantial equivalence to the predicate device.

*CI = Exact confidence interval
¹ Sixteen samples testing indeterminate by IFA and negative by PANBIO were assigned to "IgM positive" for worst-case.
² Seven samples testing indeterminate by IFA and positive by PANBIO were assigned to "IgM negative" for worst-case.

2. Sample Sizes Used for the Test Set and Data Provenance

• Study Site 1:
  • Total Samples: 420 retrospective sera
  • PRNT-confirmed subset: 130 samples (61 WNV positive, 69 WNV negative)
  • CDC MAC EIA presumptively characterized subset: 73 samples (11 WNV positive, 62 WNV negative)
  • IgM IFA presumptively characterized subset: 419 samples (96 IgM positive, 23 indeterminate, 300 negative) (Note: The sum is 419, not 420. One sample might have been excluded or lost from the initial 420).
  • Data Provenance (Country of Origin and Retrospective/Prospective): Retrospective sera from individuals in various locations:
    • California, USA (blood donation center)
    • Maryland, USA (private reference laboratory)
    • Utah, USA (private reference laboratory)
    • Texas, USA (university medical branch)
    • Minnesota, USA (private laboratory)
    • Canada (government medical laboratory)
• Study Site 2:
  • Total Samples: 51 retrospective sera confirmed by PRNT. These were encephalitis/meningitis patients.
  • Data Provenance (Country of Origin and Retrospective/Prospective): Retrospective sera from patients collected in 2002 at a hospital laboratory in Ohio, USA.
• Specificity of IgM Detection: 10 serum samples for DTT treatment, 8 serum samples for IgG interference. Provenance not specified but likely conducted in-house.
• Reproducibility: Not directly a test set for clinical performance; 27 observations for each of 9 samples.
• Cross-Reactivity: 160 specimens from patients with confirmed diseases other than WNV. Provenance not explicitly stated for individual samples, but the study was conducted at "Study Site 1" and "Study Site 2" (Ohio) and in-house at PANBIO.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

The document does not explicitly state the number of experts or their specific qualifications (e.g., radiologist with X years of experience) who established the ground truth for the test sets. Instead, the ground truth was established by:

• Plaque Reduction Neutralization Test (PRNT): A laboratory method considered the gold standard for WNV confirmation. This is a laboratory assay, not an expert panel judgment in the traditional sense.
• CDC MAC EIA: Presumptive characterization based on another laboratory assay.
• WNV IgM IFA: Presumptive characterization based on another laboratory assay.

The characterization of specimens was performed by recognized laboratory methods, not by human expert adjudication of individual cases.

4. Adjudication Method for the Test Set

The ground truth for the clinical performance studies was established using laboratory reference assays:

• Plaque Reduction Neutralization Test (PRNT): Used to "confirm" WNV positive/negative status for the primary sensitivity/specificity calculations. This acts as the independent "adjudicator."
• CDC MAC EIA and WNV IgM IFA: Used for presumptive characterization for additional agreement studies.

There was no human expert adjudication method described (e.g., 2+1, 3+1 consensus) for the test sets. The laboratory results themselves served as the reference standard.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve With AI vs. Without AI Assistance

No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not done.

This device is an Enzyme Linked Immunosorbent Assay (ELISA) for the qualitative detection of IgM antibodies, which means it is a laboratory diagnostic test. It is not an AI-powered device or an imaging interpretation system that would involve human readers or AI assistance in the way typically discussed in MRMC studies. The device itself performs the detection.

6. If a Standalone (i.e., Algorithm Only Without Human-in-the-Loop Performance) Was Done

Yes, the performance study was a standalone (algorithm only) evaluation.

The PANBIO West Nile Virus IgM Capture ELISA is an in-vitro diagnostic device that directly produces a result (positive, equivocal, negative) based on the biochemical reaction. Its performance was evaluated by directly comparing its output to established reference laboratory methods (PRNT, CDC MAC EIA, IgM IFA). There is no "human-in-the-loop" component in the sense of a human interpreting the device's output to make a primary diagnosis; the output is the diagnostic aid.

7. The Type of Ground Truth Used

The ground truth used in the performance studies was primarily laboratory reference assays:

• Plaque Reduction Neutralization Test (PRNT): Considered the gold standard for WNV confirmation.
• CDC MAC EIA (ELISA): Another established laboratory assay for WNV IgM detection.
• WNV IgM IFA (Immunofluorescence Assay): Another laboratory assay for WNV IgM detection.

For the cross-reactivity study, the ground truth was based on specimens from patients with confirmed diseases other than WNV.

8. The Sample Size for the Training Set

The document does not explicitly mention a "training set" in the context of device development. This is typical for traditional ELISA-based diagnostic kits, where the assay principles and reagents are developed and optimized rather than "trained" in the machine learning sense. The performance characteristics are then verified using independent test sets as described.

9. How the Ground Truth for the Training Set Was Established

Since a "training set" (as understood in machine learning/AI) is not typically applicable to this type of device, the concept of establishing ground truth for it is also not relevant here. The device's components and parameters are likely optimized through laboratory R&D processes based on known positive and negative controls.

Ask a specific question about this device

The Focus Technologies West Nile Virus IgM Capture ELISA is intended for qualitatively detecting IgM antibodies to West Nile virus in human serum. In conjunction with the Focus Technologies West Nile Virus ELISA IgG, the test is indicated for testing persons having symptoms of meningioencephalitis, as an aid in the presumptive laboratory diagnosis of West Nile virus infection. Positive results must be tested using the background subtraction method (either on the initial test or on a repeat test). Positive results must be confirmed by neutralization test, or by using the current CDC guidelines for diagnosing West Nile encephalitis. This test is not intended for self-testing, and this test is not FDA cleared nor approved for testing blood or plasma donors. Assay performance characteristics have not been established for automated instruments.

Indirect Enzyme-linked immunosorbent assay for qualitatively detecting human serum IgM antibodies to West Nile virus.

Here's a summary of the acceptance criteria and the studies that demonstrate the device meets those criteria, based on the provided text:

Acceptance Criteria and Device Performance

The provided document does not explicitly state "acceptance criteria" in a separate, clear table with pre-defined thresholds. However, the performance characteristics studies implicitly define the expected performance benchmarks based on agreement with reference assays and clinical classifications. For the purpose of this response, I will synthesize the performance results as the "reported device performance" and infer "acceptance criteria" from the documented agreements and sensitivities/specificities deemed acceptable for regulatory submission.

Note on Background Subtract: The device's performance is consistently evaluated both "without Background Subtract" and "with Background Subtract." The Intended Use explicitly states that "Positive results must be tested using the background subtraction method," suggesting that the performance with background subtract is the clinically relevant and accepted performance. Therefore, the table below will primarily focus on the "with Background Subtract" results as the crucial performance metrics, but acknowledge the "without" where significant differences exist.

Study Details:

1. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

   • Study Site 1 (Encephalitis/Meningitis Patients):
     • Sample Size: 300 patients. 44 confirmed positive (WNV encephalitis/meningitis symptoms, CDC IgM ELISA positive and WNV PRNT positive), 256 presumptive results (250 presumed negative, 2 presumed positive, 4 excluded).
     • Data Provenance: Sera were sequentially submitted to a state department of health laboratory located in the northeastern U.S., archived, and masked. Prospective collection from clinical suspicion.
   • Study Site 2 & 4 (WNV PRNT Positives):
     • Sample Size: 75 retrospective samples (70 ultimately tested with background subtract).
     • Data Provenance: Retrospective samples from a clinical laboratory (mid-western U.S.) with no clinical information, pre-screened positive by Focus and confirmed by WNV PRNT.
   • Study Site 3 (West Nile IFA Negatives):
     • Sample Size: 103 samples.
     • Data Provenance: Reactive samples that were West Nile IFA negative, from a clinical laboratory located in the southwestern U.S. (Retrospective implied by "assessed reactive samples").
   • Study Site 4 (Suspected Encephalitis/Meningitis Patients):
     • Sample Size: 50 samples.
     • Data Provenance: Archived and masked sera from a U.S. federal government laboratory. One confirmed positive by WNV PRNT, 49 presumptively negative by CDC ELISA for arboviruses.
   • Study Site 4 (Non-Flavivirus Test Samples):
     • Sample Size: 476 samples (4 samples were indeterminant with CDC IgM ELISA and excluded from final calculations).
     • Data Provenance: Prospectively collected from North America during August 2003. Submitted to a clinical laboratory in Southern California for non-flavivirus tests.
   • Cross-Reactivity Study (Study Site 1 & 4):
     • Sample Size: 75 samples sero-positive to other potentially cross-reactive pathogens.
     • Data Provenance: Retrospective and masked sera. DOH (northeastern U.S.) tested St. Louis encephalitis positives, Focus (Study Site 4) tested others.
   • Specificity (2-ME treatment, Study Site 4):
     • Sample Size: 15 sera.
     • Data Provenance: Sera positive for both WNV IgM and IgG.
   • Freeze-Thaw Study (Study Site 4):
     • Sample Size: 8 sera (5 positive, 3 negative).
     • Data Provenance: N/A (likely in-house selection).
   • Reproducibility Studies (Inter-lot, Inter/Intra-assay, Inter-laboratory):
     • Sample Size: Varies by study: 5 samples for inter-lot, 7 for inter/intra-assay (63 data points), 7 for inter-laboratory. Masked duplicates were used.
     • Data Provenance: Study Site 4 (Focus), Study Site 5 (mid-western U.S. clinical lab), Study Site 6 (northern California university lab), Study Site 1 (northeastern U.S. state DOH lab), Study Site 2 (mid-western U.S. clinical lab assumed different from site 5 for variety).

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)
   The document does not specify the "number of experts" or their "qualifications" involved in establishing the ground truth. Ground truth was primarily established using reference assays such as:

   • CDC West Nile Virus IgM ELISAs
   • West Nile Virus Plaque Reduction Neutralization Test (WNV PRNT)
   • West Nile IFA
   • Clinical criteria for encephalitis/meningitis (fever, altered mental status, CSF pleocytosis, etc.)
     These reference methods are established laboratory techniques, implying their performance is well-understood and accepted, but the exact human expert involvement in interpreting these for ground truth is not detailed.

3. Adjudication method (e.g. 2+1, 3+1, none) for the test set
   The document does not describe an explicit adjudication method (like 2+1 or 3+1 consensus). Ground truth was determined by comparing the device's results to a predefined reference standard (e.g., CDC IgM ELISA, WNV PRNT, WNV IFA, or clinical criteria), which implicitly serves as the adjudication or definitive determination.

4. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
   This is an in vitro diagnostic (IVD) device, specifically an ELISA assay. The concept of "human readers" for an "AI" assistance is not applicable in the context of this type of diagnostic test. The results of an ELISA are typically read by a spectrophotometer and interpreted based on a cutoff value, rather than a human "reader" making subjective interpretations that could be "assisted" by AI. Therefore, an MRMC comparative effectiveness study involving human readers with/without AI assistance was not done.

5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
   Yes, the studies presented are standalone performance evaluations of the Focus Technologies West Nile Virus IgM Capture ELISA. The device (an immunoassay kit) itself is the "algorithm" in this context, providing a quantitative output that is then interpreted against a cutoff. The performance metrics (sensitivity, specificity, agreement) directly reflect the device's ability to classify samples independently of human interpretational variability beyond reading the spectrophotometer. The "background subtract procedure" is an integral part of the assay's interpretive algorithm to mitigate false positives, as described in the "Test Principle" section.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
   The ground truth was established using reference laboratory assays (CDC WNV IgM ELISA, WNV PRNT, WNV IFA) and clinical diagnostic criteria for encephalitis/meningitis (which implicitly involves expert clinical judgment and outcomes). In some cases, previous positive screening results by other methods were also used to enrich the sample set. There is no mention of pathology (histopathological examination) or long-term outcomes data being used directly as ground truth for all studies, although the "confirmed" positive cases were patients with symptoms of meningioencephalitis.

7. The sample size for the training set
   The document does not specify a separate "training set" for the device. As an ELISA kit, it is a chemical/biological assay rather than a machine learning algorithm that requires explicit training data in the same sense. The performance characteristics were evaluated on various test sets as described above, but these are typically considered validation sets for an established assay methodology, rather than a "training set" that a machine learning model would use to learn parameters.

8. How the ground truth for the training set was established
   Since no explicit "training set" is mentioned in the context of a machine learning algorithm, this question is not directly applicable. If one considers the development and optimization of the ELISA assay itself, the ground truth would have been established during the research and development phase through various experiments using known positive and negative samples, similar to the reference assays used in the validation studies (e.g., using WNV PRNT confirmed samples, CDC ELISAs). However, the document does not detail this R&D process.

Ask a specific question about this device

The Focus Technologies West Nile Virus IgM Capture ELISA is intended for qualitatively detecting IgM antibodies to West Nile virus in human serum. In conjunction with the Focus Technologies West Nile Virus ELISA IgG, the test is indicated for testing persons having symptoms of meningioencephalitis, as an aid in the presumptive laboratory diagnosis of West Nile virus infection. Positive results must be confirmed by neutralization test, or by using the current CDC guidelines for diagnosing West Nile encephalitis. This test is not intended for self-testing, and this test is not FDA cleared nor approved for testing blood or plasma donors. Assay performance characteristics have not been established for automated instruments.

Indirect Enzyme-linked immunosorbent assay for qualitatively detecting human serum IgM antibodies to West Nile virus.

Here's a summary of the acceptance criteria and the studies performed for the Focus Technologies West Nile Virus IgM Capture ELISA, based on the provided text:

Acceptance Criteria and Device Performance

The acceptance criteria for this device are not explicitly stated in a quantitative table format (e.g., "sensitivity must be >X%"). Instead, the performance characteristics are presented as observed percentages and 95% confidence intervals from various studies, which are then compared to relevant reference methods. The implicit acceptance criteria are that the device performs comparably or acceptably against established reference methods like the CDC IgM ELISA and Plaque Reduction Neutralization Test (PRNT) for diagnosing West Nile virus infection.

Table of Acceptance Criteria (Implicit) and Reported Device Performance:

Study Details:

1. Sample sizes used for the test set and the data provenance:

   • Study Site 1 (Encephalitis/Meningitis Patients):
     • Test Set Size: 300 patients.
     • Data Provenance: Sera sequentially submitted to a state department of health laboratory in the northeastern U.S.; archived and masked. Likely retrospective, given the archival nature.
   • Study Site 2 (WNV PRNT Positives):
     • Test Set Size: 75 samples.
     • Data Provenance: Samples prescreened positive (by Focus) with a WNV native antigen ELISA and confirmed WNV positive by PRNT. From a clinical laboratory in the mid-western U.S.; archived. Likely retrospective.
   • Study Site 3 (West Nile IFA Negatives):
     • Test Set Size: 103 retrospective samples.
     • Data Provenance: From a clinical laboratory in the southwestern U.S.; retrospective.
   • Study Site 4 (Suspected Encephalitis/Meningitis Patients):
     • Test Set Size: 50 samples.
     • Data Provenance: Archived and masked sera provided by a U.S. federal government laboratory. Likely retrospective.
   • Study Site 4 (Non-Flavivirus Test Samples):
     • Test Set Size: 476 samples.
     • Data Provenance: Prospectively collected from North America during August 2003, submitted to a clinical laboratory in Southern California for non-flavivirus tests. Prospective.
   • Cross-reactivity Study:
     • Test Set Size: 75 samples (various pathogens).
     • Data Provenance: From Study Site 4 (Focus) and Study Site 1 (DOH for SLE positives). Sera were retrospective and masked.
   • Specificity of IgM Capture Wells Study:
     • Test Set Size: 15 sera.
     • Data Provenance: Not explicitly stated, but likely in-house from Focus (Study Site 4) using known WNV IgM and IgG positive samples.
   • Sera Freeze-Thaw Study:
     • Test Set Size: 8 sera (5 positive, 3 negative).
     • Data Provenance: Not explicitly stated, likely in-house from Focus (Study Site 4).
   • Reproducibility Studies:
     • Inter-lot and Inter/Intra-assay: 5-7 samples in duplicates/triplicates across different lots/days.
     • Inter-laboratory: Samples run in triplicate at 3 different labs on 3 different days.
     • Data Provenance: Focus (Study Site 4), a state department of health laboratory in the northeastern U.S. (Study Site 1), and a clinical laboratory in the mid-western U.S. (Study Site 2).

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • The document does not specify the number of experts or their qualifications. The ground truth was established by reference laboratory assays (e.g., CDC IgM ELISA, WNV PRNT, WNV IgM IFA), which were performed by trained personnel in state department of health or clinical laboratories. The expertise lies in the established protocols and interpretation guidelines of these reference methods rather than individual expert adjudication of each case.

3. Adjudication method (e.g. 2+1, 3+1, none) for the test set:

   • There was no explicit "adjudication method" in the sense of multiple human readers/experts evaluating the cases. The ground truth was determined by comparing the device's results against established reference laboratory tests (CDC IgM ELISA, WNV PRNT, WNV IgM IFA). The results of these reference tests served as the primary basis for classifying samples as positive or negative. Certain equivocal results from the Focus assay were calculated as positive or negative for performance metric purposes, indicating a predefined rule for handling these, but not human adjudication.

4. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No, a multi-reader multi-case (MRMC) comparative effectiveness study was not performed. This device is an in vitro diagnostic (IVD) ELISA kit, not an AI-powered image analysis tool or a system designed to assist human readers in interpreting complex images. Its output is a quantitative optical density that is used to derive a qualitative (positive, negative, equivocal) result, which is then interpreted by laboratory personnel. Therefore, the concept of "human readers improve with AI vs without AI assistance" is not applicable here.

5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

   • Yes, this study represents a standalone performance evaluation of the Focus Technologies West Nile Virus IgM Capture ELISA. The device is a laboratory assay (ELISA kit) designed to provide a result directly from a patient sample. Its performance is evaluated intrinsically through its chemical and immunological reactions, and the optical density reading is interpreted by a predefined algorithm/cutoff. Human involvement is in performing the assay and reading the result, but the "performance" itself is that of the assay kit. This is the typical way IVD assays are evaluated.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc):

   • The ground truth was primarily established using established reference laboratory tests:
     • Plaque Reduction Neutralization Test (PRNT): Considered the gold standard for confirming WNV infection.
     • CDC West Nile Virus IgM ELISA: A widely accepted and validated method for detecting WNV IgM antibodies.
     • West Nile IFA (Immunofluorescence Assay): Another method used for WNV IgM antibody detection.
     • Clinical Criteria: For the encephalitis/meningitis patient cohort, a combination of clinical symptoms (fever, altered mental status, CSF pleocytosis) in conjunction with laboratory results defined the diagnosis.

7. The sample size for the training set:

   • The document does not explicitly describe a "training set" for the device in the context of machine learning. For an ELISA kit, development typically involves internal optimization and validation using various samples, but these are not defined as a distinct "training set" like in AI/ML. The performance data presented are from validation/test sets.

8. How the ground truth for the training set was established:

   • Since a formal "training set" as understood in AI/ML is not described, the method for establishing its ground truth is not applicable or detailed in this document. The development and internal validation of such an assay would involve using well-characterized samples (often confirmed by gold-standard methods like PRNT or a previously validated ELISA) to optimize reagent concentrations, incubation times, and cutoff values, but this process isn't reported as a "training set" with established ground truth in the same way as for AI models.

Ask a specific question about this device

Ask a specific question about this device