Search Results

The Focus Technologies West Nile Virus IgM Capture ELISA is intended for qualitatively detecting IgM antibodies to West Nile virus in human serum. In conjunction with the Focus Technologies West Nile Virus ELISA IgG, the test is indicated for testing persons having symptoms of meningioencephalitis, as an aid in the presumptive laboratory diagnosis of West Nile virus infection. Positive results must be tested using the background subtraction method (either on the initial test or on a repeat test). Positive results must be confirmed by neutralization test, or by using the current CDC guidelines for diagnosing West Nile encephalitis. This test is not intended for self-testing, and this test is not FDA cleared nor approved for testing blood or plasma donors. Assay performance characteristics have not been established for automated instruments.

Indirect Enzyme-linked immunosorbent assay for qualitatively detecting human serum IgM antibodies to West Nile virus.

Here's a summary of the acceptance criteria and the studies that demonstrate the device meets those criteria, based on the provided text:

Acceptance Criteria and Device Performance

The provided document does not explicitly state "acceptance criteria" in a separate, clear table with pre-defined thresholds. However, the performance characteristics studies implicitly define the expected performance benchmarks based on agreement with reference assays and clinical classifications. For the purpose of this response, I will synthesize the performance results as the "reported device performance" and infer "acceptance criteria" from the documented agreements and sensitivities/specificities deemed acceptable for regulatory submission.

Note on Background Subtract: The device's performance is consistently evaluated both "without Background Subtract" and "with Background Subtract." The Intended Use explicitly states that "Positive results must be tested using the background subtraction method," suggesting that the performance with background subtract is the clinically relevant and accepted performance. Therefore, the table below will primarily focus on the "with Background Subtract" results as the crucial performance metrics, but acknowledge the "without" where significant differences exist.

Study Details:

1. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

   • Study Site 1 (Encephalitis/Meningitis Patients):
     • Sample Size: 300 patients. 44 confirmed positive (WNV encephalitis/meningitis symptoms, CDC IgM ELISA positive and WNV PRNT positive), 256 presumptive results (250 presumed negative, 2 presumed positive, 4 excluded).
     • Data Provenance: Sera were sequentially submitted to a state department of health laboratory located in the northeastern U.S., archived, and masked. Prospective collection from clinical suspicion.
   • Study Site 2 & 4 (WNV PRNT Positives):
     • Sample Size: 75 retrospective samples (70 ultimately tested with background subtract).
     • Data Provenance: Retrospective samples from a clinical laboratory (mid-western U.S.) with no clinical information, pre-screened positive by Focus and confirmed by WNV PRNT.
   • Study Site 3 (West Nile IFA Negatives):
     • Sample Size: 103 samples.
     • Data Provenance: Reactive samples that were West Nile IFA negative, from a clinical laboratory located in the southwestern U.S. (Retrospective implied by "assessed reactive samples").
   • Study Site 4 (Suspected Encephalitis/Meningitis Patients):
     • Sample Size: 50 samples.
     • Data Provenance: Archived and masked sera from a U.S. federal government laboratory. One confirmed positive by WNV PRNT, 49 presumptively negative by CDC ELISA for arboviruses.
   • Study Site 4 (Non-Flavivirus Test Samples):
     • Sample Size: 476 samples (4 samples were indeterminant with CDC IgM ELISA and excluded from final calculations).
     • Data Provenance: Prospectively collected from North America during August 2003. Submitted to a clinical laboratory in Southern California for non-flavivirus tests.
   • Cross-Reactivity Study (Study Site 1 & 4):
     • Sample Size: 75 samples sero-positive to other potentially cross-reactive pathogens.
     • Data Provenance: Retrospective and masked sera. DOH (northeastern U.S.) tested St. Louis encephalitis positives, Focus (Study Site 4) tested others.
   • Specificity (2-ME treatment, Study Site 4):
     • Sample Size: 15 sera.
     • Data Provenance: Sera positive for both WNV IgM and IgG.
   • Freeze-Thaw Study (Study Site 4):
     • Sample Size: 8 sera (5 positive, 3 negative).
     • Data Provenance: N/A (likely in-house selection).
   • Reproducibility Studies (Inter-lot, Inter/Intra-assay, Inter-laboratory):
     • Sample Size: Varies by study: 5 samples for inter-lot, 7 for inter/intra-assay (63 data points), 7 for inter-laboratory. Masked duplicates were used.
     • Data Provenance: Study Site 4 (Focus), Study Site 5 (mid-western U.S. clinical lab), Study Site 6 (northern California university lab), Study Site 1 (northeastern U.S. state DOH lab), Study Site 2 (mid-western U.S. clinical lab assumed different from site 5 for variety).

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)
   The document does not specify the "number of experts" or their "qualifications" involved in establishing the ground truth. Ground truth was primarily established using reference assays such as:

   • CDC West Nile Virus IgM ELISAs
   • West Nile Virus Plaque Reduction Neutralization Test (WNV PRNT)
   • West Nile IFA
   • Clinical criteria for encephalitis/meningitis (fever, altered mental status, CSF pleocytosis, etc.)
     These reference methods are established laboratory techniques, implying their performance is well-understood and accepted, but the exact human expert involvement in interpreting these for ground truth is not detailed.

3. Adjudication method (e.g. 2+1, 3+1, none) for the test set
   The document does not describe an explicit adjudication method (like 2+1 or 3+1 consensus). Ground truth was determined by comparing the device's results to a predefined reference standard (e.g., CDC IgM ELISA, WNV PRNT, WNV IFA, or clinical criteria), which implicitly serves as the adjudication or definitive determination.

4. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
   This is an in vitro diagnostic (IVD) device, specifically an ELISA assay. The concept of "human readers" for an "AI" assistance is not applicable in the context of this type of diagnostic test. The results of an ELISA are typically read by a spectrophotometer and interpreted based on a cutoff value, rather than a human "reader" making subjective interpretations that could be "assisted" by AI. Therefore, an MRMC comparative effectiveness study involving human readers with/without AI assistance was not done.

5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
   Yes, the studies presented are standalone performance evaluations of the Focus Technologies West Nile Virus IgM Capture ELISA. The device (an immunoassay kit) itself is the "algorithm" in this context, providing a quantitative output that is then interpreted against a cutoff. The performance metrics (sensitivity, specificity, agreement) directly reflect the device's ability to classify samples independently of human interpretational variability beyond reading the spectrophotometer. The "background subtract procedure" is an integral part of the assay's interpretive algorithm to mitigate false positives, as described in the "Test Principle" section.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
   The ground truth was established using reference laboratory assays (CDC WNV IgM ELISA, WNV PRNT, WNV IFA) and clinical diagnostic criteria for encephalitis/meningitis (which implicitly involves expert clinical judgment and outcomes). In some cases, previous positive screening results by other methods were also used to enrich the sample set. There is no mention of pathology (histopathological examination) or long-term outcomes data being used directly as ground truth for all studies, although the "confirmed" positive cases were patients with symptoms of meningioencephalitis.

7. The sample size for the training set
   The document does not specify a separate "training set" for the device. As an ELISA kit, it is a chemical/biological assay rather than a machine learning algorithm that requires explicit training data in the same sense. The performance characteristics were evaluated on various test sets as described above, but these are typically considered validation sets for an established assay methodology, rather than a "training set" that a machine learning model would use to learn parameters.

8. How the ground truth for the training set was established
   Since no explicit "training set" is mentioned in the context of a machine learning algorithm, this question is not directly applicable. If one considers the development and optimization of the ELISA assay itself, the ground truth would have been established during the research and development phase through various experiments using known positive and negative samples, similar to the reference assays used in the validation studies (e.g., using WNV PRNT confirmed samples, CDC ELISAs). However, the document does not detail this R&D process.

Ask a specific question about this device

The Focus Technologies West Nile Virus IgM Capture ELISA is intended for qualitatively detecting IgM antibodies to West Nile virus in human serum. In conjunction with the Focus Technologies West Nile Virus ELISA IgG, the test is indicated for testing persons having symptoms of meningioencephalitis, as an aid in the presumptive laboratory diagnosis of West Nile virus infection. Positive results must be confirmed by neutralization test, or by using the current CDC guidelines for diagnosing West Nile encephalitis. This test is not intended for self-testing, and this test is not FDA cleared nor approved for testing blood or plasma donors. Assay performance characteristics have not been established for automated instruments.

Indirect Enzyme-linked immunosorbent assay for qualitatively detecting human serum IgM antibodies to West Nile virus.

Here's a summary of the acceptance criteria and the studies performed for the Focus Technologies West Nile Virus IgM Capture ELISA, based on the provided text:

Acceptance Criteria and Device Performance

The acceptance criteria for this device are not explicitly stated in a quantitative table format (e.g., "sensitivity must be >X%"). Instead, the performance characteristics are presented as observed percentages and 95% confidence intervals from various studies, which are then compared to relevant reference methods. The implicit acceptance criteria are that the device performs comparably or acceptably against established reference methods like the CDC IgM ELISA and Plaque Reduction Neutralization Test (PRNT) for diagnosing West Nile virus infection.

Table of Acceptance Criteria (Implicit) and Reported Device Performance:

Study Details:

1. Sample sizes used for the test set and the data provenance:

   • Study Site 1 (Encephalitis/Meningitis Patients):
     • Test Set Size: 300 patients.
     • Data Provenance: Sera sequentially submitted to a state department of health laboratory in the northeastern U.S.; archived and masked. Likely retrospective, given the archival nature.
   • Study Site 2 (WNV PRNT Positives):
     • Test Set Size: 75 samples.
     • Data Provenance: Samples prescreened positive (by Focus) with a WNV native antigen ELISA and confirmed WNV positive by PRNT. From a clinical laboratory in the mid-western U.S.; archived. Likely retrospective.
   • Study Site 3 (West Nile IFA Negatives):
     • Test Set Size: 103 retrospective samples.
     • Data Provenance: From a clinical laboratory in the southwestern U.S.; retrospective.
   • Study Site 4 (Suspected Encephalitis/Meningitis Patients):
     • Test Set Size: 50 samples.
     • Data Provenance: Archived and masked sera provided by a U.S. federal government laboratory. Likely retrospective.
   • Study Site 4 (Non-Flavivirus Test Samples):
     • Test Set Size: 476 samples.
     • Data Provenance: Prospectively collected from North America during August 2003, submitted to a clinical laboratory in Southern California for non-flavivirus tests. Prospective.
   • Cross-reactivity Study:
     • Test Set Size: 75 samples (various pathogens).
     • Data Provenance: From Study Site 4 (Focus) and Study Site 1 (DOH for SLE positives). Sera were retrospective and masked.
   • Specificity of IgM Capture Wells Study:
     • Test Set Size: 15 sera.
     • Data Provenance: Not explicitly stated, but likely in-house from Focus (Study Site 4) using known WNV IgM and IgG positive samples.
   • Sera Freeze-Thaw Study:
     • Test Set Size: 8 sera (5 positive, 3 negative).
     • Data Provenance: Not explicitly stated, likely in-house from Focus (Study Site 4).
   • Reproducibility Studies:
     • Inter-lot and Inter/Intra-assay: 5-7 samples in duplicates/triplicates across different lots/days.
     • Inter-laboratory: Samples run in triplicate at 3 different labs on 3 different days.
     • Data Provenance: Focus (Study Site 4), a state department of health laboratory in the northeastern U.S. (Study Site 1), and a clinical laboratory in the mid-western U.S. (Study Site 2).

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • The document does not specify the number of experts or their qualifications. The ground truth was established by reference laboratory assays (e.g., CDC IgM ELISA, WNV PRNT, WNV IgM IFA), which were performed by trained personnel in state department of health or clinical laboratories. The expertise lies in the established protocols and interpretation guidelines of these reference methods rather than individual expert adjudication of each case.

3. Adjudication method (e.g. 2+1, 3+1, none) for the test set:

   • There was no explicit "adjudication method" in the sense of multiple human readers/experts evaluating the cases. The ground truth was determined by comparing the device's results against established reference laboratory tests (CDC IgM ELISA, WNV PRNT, WNV IgM IFA). The results of these reference tests served as the primary basis for classifying samples as positive or negative. Certain equivocal results from the Focus assay were calculated as positive or negative for performance metric purposes, indicating a predefined rule for handling these, but not human adjudication.

4. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No, a multi-reader multi-case (MRMC) comparative effectiveness study was not performed. This device is an in vitro diagnostic (IVD) ELISA kit, not an AI-powered image analysis tool or a system designed to assist human readers in interpreting complex images. Its output is a quantitative optical density that is used to derive a qualitative (positive, negative, equivocal) result, which is then interpreted by laboratory personnel. Therefore, the concept of "human readers improve with AI vs without AI assistance" is not applicable here.

5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

   • Yes, this study represents a standalone performance evaluation of the Focus Technologies West Nile Virus IgM Capture ELISA. The device is a laboratory assay (ELISA kit) designed to provide a result directly from a patient sample. Its performance is evaluated intrinsically through its chemical and immunological reactions, and the optical density reading is interpreted by a predefined algorithm/cutoff. Human involvement is in performing the assay and reading the result, but the "performance" itself is that of the assay kit. This is the typical way IVD assays are evaluated.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc):

   • The ground truth was primarily established using established reference laboratory tests:
     • Plaque Reduction Neutralization Test (PRNT): Considered the gold standard for confirming WNV infection.
     • CDC West Nile Virus IgM ELISA: A widely accepted and validated method for detecting WNV IgM antibodies.
     • West Nile IFA (Immunofluorescence Assay): Another method used for WNV IgM antibody detection.
     • Clinical Criteria: For the encephalitis/meningitis patient cohort, a combination of clinical symptoms (fever, altered mental status, CSF pleocytosis) in conjunction with laboratory results defined the diagnosis.

7. The sample size for the training set:

   • The document does not explicitly describe a "training set" for the device in the context of machine learning. For an ELISA kit, development typically involves internal optimization and validation using various samples, but these are not defined as a distinct "training set" like in AI/ML. The performance data presented are from validation/test sets.

8. How the ground truth for the training set was established:

   • Since a formal "training set" as understood in AI/ML is not described, the method for establishing its ground truth is not applicable or detailed in this document. The development and internal validation of such an assay would involve using well-characterized samples (often confirmed by gold-standard methods like PRNT or a previously validated ELISA) to optimize reagent concentrations, incubation times, and cutoff values, but this process isn't reported as a "training set" with established ground truth in the same way as for AI models.

Ask a specific question about this device

The Focus Technologies West Nile Virus ELISA IgG is intended for qualitatively detecting IgG antibodies to West Nile virus in human serum. In conjunction with the Focus Technologies West Nile Virus IgM Capture ELISA, the test is indicated for testing persons having symptoms of meningioencephalitis, as an aid in the presumptive laboratory diagnosis of West Nile virus infection. Positive results must be confirmed by neutralization test, or by using the current CDC guidelines for diagnosing West Nile encephalitis. This test is not intended for self-testing, and this test is not FDA cleared nor approved for testing blood or plasma donors. Assay performance characteristics have not been established for automated instruments.

Indirect Enzyme-linked immunosorbent assay for qualitatively detecting human serum IgG antibodies to West Nile virus.

Here's a detailed breakdown of the acceptance criteria and the study that proves the Focus Technologies West Nile Virus ELISA IgG device meets them, based on the provided text:

Acceptance Criteria and Device Performance for Focus Technologies West Nile Virus ELISA IgG

The document does not explicitly state pre-defined acceptance criteria (e.g., "sensitivity must be > X%") for the device. Instead, it reports the device's performance characteristics from various studies. Therefore, the "acceptance criteria" presented below are inferred from the reported performance, implying that these levels of performance were deemed acceptable for market clearance.

1. Table of Acceptance Criteria (Inferred) and Reported Device Performance

2. Sample Size Used for the Test Set and Data Provenance

• Study Site 1 (Encephalitis/Meningitis Patients):
  • Sample Size: 300 total patients (37 confirmed positive, 210 presumptive results, 53 unclassified). Performance calculated on 242 samples (37 confirmed positive + 205 presumptive negative).
  • Data Provenance: Sera were sequentially submitted to a state department of health laboratory in the northeastern U.S., archived, and masked. Retrospective.
• Study Site 2 (WNV PRNT Positives):
  • Sample Size: 75 retrospective sera.
  • Data Provenance: Sera were sequentially submitted to a clinical laboratory in the mid-western U.S., archived. Retrospective.
• Study Site 3 (WNV IFA Negatives):
  • Sample Size: 157 samples.
  • Data Provenance: Retrospective sera from a clinical laboratory in the southwestern U.S.
• Study Site 4 (Suspected Encephalitis/Meningitis Patients):
  • Sample Size: 50 sera (1 confirmed positive, 49 presumptively negative).
  • Data Provenance: Retrospective and masked sera provided by a U.S. federal government laboratory.
• Study Site 4 (Non-Flavivirus Test Samples):
  • Sample Size: 476 samples. Performance calculated on 461 samples (21 CDC ELISA positive + 440 CDC ELISA negative).
  • Data Provenance: Prospectively collected from North America (August 2003) and submitted to a clinical laboratory in Southern California for non-flavivirus tests. Prospective, but the "test set" for WNV performance was created using the CDC ELISA.
• Cross-reactivity Studies:
  • Sample Size: 75 sera (20 Dengue, 20 Japanese encephalitis, 21 St. Louis encephalitis, 20 Yellow fever, 17 Alphavirus, 15 Bunyavirus, 60 HSV-1, 12 EBV, 20 Cytomegalovirus, 20 Echovirus/Poliovirus, 20 Lyme disease).
  • Data Provenance: Sera were archived and masked, conducted at Focus (Study Site 4, for most pathogens) and a state department of health laboratory in the northeastern U.S. (DOH, for SLE positives). Retrospective.
• Freeze-Thaw Study:
  • Sample Size: 8 sera (5 positive, 3 negative).
  • Data Provenance: Conducted at Focus (Study Site 4).
• Reproducibility Studies:
  • Sample Size: Inter-lot: 5 samples; Inter/Intra-assay: 7 samples; Inter-laboratory: 7 samples.
  • Data Provenance: Conducted at Focus (Study Site 4), a state department of health laboratory in the northeastern U.S. (Study Site 1), and a clinical laboratory in the mid-western U.S. (Study Site 2).

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

The document does not explicitly state the "number of experts" or their "qualifications" involved in establishing the ground truth. However, it indicates:

• Reference Methods Used:
  • CDC West Nile Virus IgG Capture ELISA
  • Plaque Reduction Neutralization Test (PRNT) for West Nile virus
  • Dengue PRNT
  • St. Louis Encephalitis (SLE) PRNT
  • West Nile Virus native antigen ELISA
  • West Nile IFA

These are established laboratory assays, and their results are considered the "ground truth." The implication is that skilled laboratory personnel at the respective state, federal, and clinical laboratories performed these reference tests. The qualifications of these individuals (e.g., highly trained lab technologists, clinical microbiologists) are inherent to the operation of such labs but are not specified in terms of "years of experience" or "radiologist" equivalent, as this is an in vitro diagnostic device.

4. Adjudication Method for the Test Set

The document does not describe an explicit "adjudication method" in the sense of multiple experts independently evaluating cases and resolving discrepancies (e.g., 2+1, 3+1).

Instead, the "ground truth" was established by reference laboratory assays (CDC WNV IgG ELISA, PRNT, IFA), which inherently have their own established protocols for result interpretation. For example, "confirmed positive West Nile encephalitis patients" were defined by symptoms and positive results from both CDC WNV IgG ELISA and WNV PRNT. Equivocal or indeterminant results from reference assays were often excluded from performance calculations.

5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study Was Done

No, a Multi Reader Multi Case (MRMC) comparative effectiveness study was not done. This type of study typically involves comparing human reader performance with and without an AI-assisted device in diagnostic imaging contexts. The Focus Technologies West Nile Virus ELISA IgG is an in vitro diagnostic laboratory test, not an imaging device intended for human interpretation.

6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Was Done

Yes, the studies conducted evaluated the standalone performance of the Focus Technologies West Nile Virus ELISA IgG device (an algorithm/assay only). The device itself is an indirect ELISA designed to qualitatively detect antibodies, implying that the interpretation of the optical density readings and the final result (positive, negative, equivocal) is performed by the device's internal algorithms or by comparison to defined cut-offs, without a "human-in-the-loop" decision support role in the sense of AI for image interpretation. The performance metrics presented (sensitivity, specificity/agreement) directly reflect the device's ability to classify samples based on its own output against established reference methods.

7. The Type of Ground Truth Used

The primary types of ground truth used were:

• Reference Laboratory Assay Results:
  • CDC West Nile Virus IgG Capture ELISA: A highly respected and commonly used reference method for WNV IgG.
  • Plaque Reduction Neutralization Test (PRNT): Considered the gold standard for serological confirmation of arbovirus infections due to its high specificity.
  • West Nile IFA (Immunofluorescence Assay): Another serological test.
  • Dengue PRNT, SLE PRNT: Used for cross-reactivity assessment.
  • West Nile virus native antigen ELISA: Used as a screening method in Study Site 2 before PRNT confirmation.
• Clinical Criteria: For Study Site 1 and 4, patient presentation (meningioencephalitis symptoms, fever, altered mental status, CSF findings) was combined with laboratory results to define confirmed cases.

This combination of highly specific and sensitive reference assays, often coupled with relevant clinical symptoms, provides a robust ground truth for an in vitro diagnostic device.

8. The Sample Size for the Training Set

The document does not explicitly mention a "training set" in the context of machine learning or AI development. This device is an ELISA, which is typically a biochemically-based assay, not an AI/ML algorithm that requires a separate training phase with labeled data in the same way. The performance studies evaluate the assay's built characteristics against clinical samples and reference methods.

If one were to loosely interpret "training" as the internal development and calibration of the assay (e.g., setting cut-offs), the size of the samples used for that purpose is not detailed. The provided studies are primarily for performance validation.

9. How the Ground Truth for the Training Set Was Established

As noted above, there isn't a "training set" in the AI/ML sense for this ELISA device. The concept of "ground truth" for the internal development and establishment of assay cut-offs would typically involve:

• Well-characterized panels: Samples from individuals with known WNV infection status (confirmed by PRNT or PCR) and known negative individuals.
• Titration and statistical analysis: To determine optimal cut-off values for distinguishing positive, negative, and equivocal results based on optical density readings.

However, the document focuses on the validation studies performed to demonstrate the device's performance against established reference methods and clinical populations, rather than detailing its initial development and calibration.

Ask a specific question about this device

Focus Technologies' HerpeSelect® 2 ELISA IgG test is intended for qualitatively detecting the presence or absence of human IgG class antibodies to HSV-2 in human sera. In conjunction with the Focus HerpeSelect® I ELISA IgG, the test is indicated for testing sexually active adults or expectant mothers for aiding in the presumptive digenosis of HSV infection. The assay can be used manually or in conjunction with an automated system as outlined in the package insert. The user is responsible for assay performance characteristics when an automated system is used. The performance of this assay has not been established for use in a pediatric population, for neonatal screening, or for testing of immunocompromised patients.

In the HerpeSelect 2 ELISA IgG assay, the polystyrene microwells are coated with recombinant gG-2 antigen. Diluted serum samples and controls are incubated in the wells to allow specific antibody present in the samples to react with the antigen. Nonspecific reactants are removed by washing, and peroxidase-conjugated anti-human IgG is added and reacts with specific IgG. Excess conjugate is removed by washing. Enzyme substrate and chromogen are added, and the color is allowed to develop. After adding the Stop Reagent, the resultant color change is quantified by a spectrophotometric reading of optical density (OD). Sample optical density readings are compared with reference cut-off OD readings to determine results.

The provided document describes the performance characteristics of the Focus Technologies HerpeSelect® 2 ELISA IgG assay for detecting HSV-2 IgG antibodies. The acceptance criteria are not explicitly stated in a single section but are implied through the reported performance metrics against a Western blot (WB) reference method across various populations.

1. Table of Acceptance Criteria and Reported Device Performance

Since explicit "acceptance criteria" are not given in a dedicated table, the reported performance metrics against the reference Western blot (considered the gold standard for serostatus) are used to infer them. Performance is presented for different populations, implying that satisfactory results in these different contexts are within the "acceptance criteria" for the device's intended use.

2. Sample Sizes and Data Provenance for Test Set

• Sexually Active Adults: n = 246 (retrospective, archived, masked sera)
• Expectant Mothers: n = 241 (retrospective, archived, masked sera)
• Culture Positive Patients: n = 63 (sera from culture positive patients)
• Low Prevalence Population: n = 81 (college students claiming to lack sexual experience)
• Type Specificity (HSV-1 WB positives): n = 287 (from expectant mothers, sexually active adults, low prevalence persons, and HSV-1 culture positives)
• Cross-reactivity with Taxonomically Related Viruses: n = 27 (HSV sero-negative by other FDA cleared HSV ELISAs, and IFA IgG positive for CMV, EBV VCA, HHV6, VZV)
• CDC Panel: Undisclosed specific number of samples (panel consists of 37% positive and 63% negative samples).
• Manual vs. Automated Method Agreement: n = 257 (adult samples submitted for HSV testing, 255 from US, 2 from outside US, retrospectively selected and manually tested by an external investigator as part of CLIA validation).

Data Provenance: The data primarily originated from a "Pacific Northwest university" for Western blot reference testing and a "major university located in the Northwestern United States" for discrepant analysis of cross-reactivity. One study on manual vs. automated method agreement involved a "major clinical laboratory located in Southern California" with samples predominantly from the US (255/257). The CDC panel information suggests samples from the US. The studies are described as using "masked, archived, and unselected sera" or "sequentially submitted" sera, indicating a retrospective data collection approach for most studies.

3. Number of Experts and Qualifications for Ground Truth

The document does not explicitly state the number of experts used to establish the ground truth for the test set or their specific qualifications (e.g., "radiologist with 10 years of experience"). However, it clearly states that the reference method for determining HSV-2 serostatus was an HSV-2 Western blot (WB) performed at a "Pacific Northwest university." For culture-positive patients, the reference methods included both culture (infection) and the HSV-2 Western blot. For cross-reactivity studies, a "type specific Western blot from a major university located in the Northwestern United States" was used for discrepant analysis.

For the "Agreement with CDC Panel," the ground truth was established by the CDC, implying a highly qualified and standardized reference.

The interpretation of Western blot results for HSV-2 would typically be performed by trained specialists in a reference laboratory, although their specific credentials are not detailed in this summary.

4. Adjudication Method for the Test Set

The document does not describe a formal adjudication method (like 2+1 or 3+1 consensus) for the initial Western blot interpretation or other reference methods. The Western blot results are presented as the established ground truth. Discrepant analysis between the HerpeSelect ELISA and other methods (e.g., other FDA cleared HSV ELISAs for cross-reactivity) was performed using a "type specific Western blot," suggesting this was the arbiter for discrepancies in those specific cases.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No multi-reader multi-case (MRMC) comparative effectiveness study comparing human readers with and without AI assistance (or in this case, the automated ELISA) is described. The studies focus on the performance of the HerpeSelect 2 ELISA IgG assay (both manual and automated options) against a reference standard (Western blot or culture), not on the improvement of human readers with AI assistance.

6. Standalone Performance Study

Yes, standalone performance studies were done. The entire document describes the performance of the HerpeSelect® 2 ELISA IgG device (both the manual and the automated option) as a standalone diagnostic tool. Its sensitivity, specificity, agreement, and reproducibility were evaluated independently against established reference methods (Western blot, culture) or in comparison with its own manual method.

7. Type of Ground Truth Used

The primary type of ground truth used was expert reference method, specifically:

• HSV-2 Western blot (WB): Considered the gold standard for HSV-2 antibody detection. This was used across most population studies (expectant mothers, sexually active adults, low prevalence population, type specificity).
• Culture (infection): Used as a reference alongside Western blot for the "culture positives" study.
• CDC Panel: A characterized serum panel from the CDC, implying a highly validated ground truth.
• Previously FDA cleared HSV ELISAs and IFA IgG for taxonomically similar viruses: Used as initial screening/reference for the cross-reactivity study, with Western blot for discrepant analysis.

8. Sample Size for the Training Set

The document does not provide information about a "training set" for the HerpeSelect® 2 ELISA IgG assay. This assay is a diagnostic laboratory test, not an AI or machine learning algorithm that typically requires distinct training sets. The studies described are performance evaluations of the final, already developed assay.

9. How Ground Truth for the Training Set Was Established

As there is no mention of a "training set" for algorithm development in this context (the device is an ELISA assay rather than an AI/ML algorithm), the method for establishing ground truth for a training set is not applicable or described. The performance data presented refers to the test set or validation set of samples on which the device's accuracy was demonstrated.

Ask a specific question about this device

Focus Technologies' HerpeSelect® 1 ELISA IgG test is intended for qualitatively detecting the presence or absence of human IgG class antibodies to HSV-1 in human sera. In conjunction with the Focus HerpeSelect® 2 ELISA IgG, the test is indicated for testing sexually active adults or expectant mothers for aiding in the presumptive diagnosis of HSV infection. The assay can be used manually or in conjunction with an automated system as outlined in the package insert. The user is responsible for assay performance characteristics when an automated system is used. The performance of this assay has not been established for use in a pediatric population, for neonatal screening, or for testing of immunocompromised patients.

In the HerpeSelect® I ELISA IgG IgG assay, the polystyrene microwells are coated with recombinant gG-1 antigen. Diluted serum samples and controls are incubated in the wells to allow specific antibody present in the samples to react with the antigen. Nonspecific reactants are removed by washing, and peroxidase-conjugated anti-human IgG is added and reacts with specific IgG. Excess conjugate is removed by washing. Enzyme substrate and chromogen are added, and the color is allowed to develop. After adding the Stop Reagent, the resultant color change is quantified by a spectrophotometric reading of optical density (OD). Sample optical density readings are compared with reference cut-off OD readings to determine results.

Here's an analysis of the provided text regarding the HerpeSelect® 1 ELISA IgG device, broken down by your requested criteria:

1. Table of Acceptance Criteria and Reported Device Performance

The document does not explicitly state "acceptance criteria" in terms of specific thresholds the device needed to meet for regulatory approval. Instead, it presents various "performance characteristics" studies. However, we can infer performance goals based on the reported results within the different populations. The primary comparative method used is the HSV-1 Western blot (WB).

2. Sample Size Used for the Test Set and Data Provenance

• Sexually Active Adults: n = 246 (archived, unselected sera)
• Expectant Mothers: n = 241 (archived, unselected sera)
• Culture Positives: n = 38 (sera from culture positive patients)
• Low Prevalence Population: n = 81 (sera from college students claiming to lack sexual experience)
• Type Specificity (HSV-2 Western blot positives / HSV-1 negatives): n = 90 (combined from expectant mothers, sexually active adults, low prevalence persons, and HSV-2 culture positives)
• Cross-reactivity (Taxonomically Related Viruses): n = 26 (HSV sero-negative by other FDA cleared ELISAs, IFA IgG positive for CMV, EBV VCA, HHV6, VZV)
• CDC Panel: Panel of unstated size (59% positive, 41% negative samples)
• Manual vs. Automated Methods Agreement: n = 248 (adult samples from US and 2 outside US, submitted for HSV testing)

Data Provenance:

• Most studies were conducted by an "outside investigator."
• The reference method for most studies (excluding cross-reactivity and agreement with CDC) was an HSV-1 Western blot from a Pacific Northwest university.
• The sera were "archived" and "masked" for several studies, indicating a retrospective design.
• The "outside investigator" for the manual vs. automated method comparison was part of a CLIA validation for a major clinical laboratory in Southern California.
• The CDC panel was obtained from the CDC.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts

The document does not explicitly state the number of experts used or their qualifications for establishing the ground truth. The ground truth for most studies was established using a "HSV-1 Western blot from a Pacific Northwest university" or "culture" results. Western blot interpretation is typically done by trained laboratory personnel or experts in serology, but specific details on the number and qualifications are not provided here. For the CDC panel, it states the panel was "characterized," implying expert review and consensus, but again, no specifics are given.

4. Adjudication Method for the Test Set

The document does not explicitly describe an adjudication method for the test set. It reports concordance with the reference methods (Western blot, culture, or CDC panel) and notes "equivocals" and "atypical Western blots" that were excluded from calculations, but it doesn't detail a process for resolving discrepancies.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs. without AI assistance

This device is an ELISA (Enzyme-linked Immunosorbent Assay), which is a laboratory test for detecting antibodies. It's not an imaging or diagnostic device that involves human "readers" in the context of interpreting complex medical images like radiology or pathology. Therefore, an MRMC comparative effectiveness study comparing human reader performance with and without AI assistance is not applicable to this type of device and was not performed.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done

Yes, the studies evaluating "Relative Sensitivity and Relative Specificity" with various populations (sexually active adults, expectant mothers, culture positives, low prevalence, type specificity) demonstrate the standalone performance of the HerpeSelect® 1 ELISA IgG device. The results (optical density readings) are automatically compared to reference cut-off OD readings to determine results, implying an algorithmic determination. The device's performance is compared directly against the reference standard (Western blot or culture), indicating a standalone assessment.

The document also describes "Reproducibility Using an Automated Instrument" and "% Agreement between the Manual and Automated Methods," which further supports assessment of the device's automated, standalone capabilities.

7. The Type of Ground Truth Used

The primary type of ground truth used was:

• Reference Method: HSV-1 Western blot from a Pacific Northwest university.
• Culture: For the "Relative Sensitivity with Culture Positives" study.
• CDC Panel Characterization: For the "Agreement with CDC Panel" study, indicating a highly characterized set of samples.
• Other FDA cleared HSV ELISAs and IFA IgG positive for taxonomically similar viruses: For the "Cross-reactivity with Taxonomically Related Viruses" study.

8. The Sample Size for the Training Set

The document describes performance characteristics using various test sets (as detailed in point 2). It does not explicitly mention a "training set" or a separate process for training the device. For ELISA tests, the "training" equivalent would typically involve the development and validation of the assay's reagents and cut-off values using known positive and negative samples during the assay development phase. This document focuses on the validation of the finalized device.

9. How the Ground Truth for the Training Set was Established

Since an explicit "training set" is not detailed, the method for establishing its ground truth is also not provided. However, the general principle for establishing ground truth during the development of an ELISA (which constitutes the implicit "training" for the assay's cut-offs and performance characteristics) would typically involve:

• Well-characterized clinical samples: Samples from individuals with confirmed HSV-1 infection (via culture or strong clinical evidence) and confirmed non-infected individuals.
• Comparative methods: Using established gold standard tests like Western blot, PCR, or viral culture to classify these samples.
• Expert Consensus: Review by infectious disease specialists or laboratory experts to ensure accurate classification of reference samples.

The studies presented here are mainly for the validation of the device rather than its initial development or training.

Ask a specific question about this device