(140 days)
The Focus Diagnostics Simplexa™ HSV 1 & 2 Direct assay is intended for use on the 3M Integrated Cycler instrument for the qualitative detection and differentiation of herpes simples virus (HSV-1 and HSV-2) DNA present in genital lesion swabs samples from patients with signs and symptoms of HSV-1 or HSV-2 infection of the genitalia. This test is an aid in the differential diagnosis of HSV-1 and HSV-2 genital infections.
The assay is not intended for use as a screening test for the presence of HSV-1 and HSV-2 in blood or blood products. The assay is for professional use only.
Simplexa™ HSV 1 & 2 Positive Control Pack
The Simplexa™ HSV 1 & 2 Positive Control Pack is intended to be used as a control with the Simplexa™ HSV 1 & 2 Direct kit. This control is not intended for use with other assays or systems.
The Simplexa™ HSV 1 & 2 Direct assay system is a real-time PCR that enables the direct amplification, detection and differentiation of HSV-1 and/or HSV-2 DNA from unprocessed genital swab specimens without nucleic acid extraction. The system consists of the Simplexa™ HSV 1 & 2 Direct assay, the 3M Integrated Cycler (with 3M Integrated Cycler Studio Software), the Direct Amplification Disc and associated accessories.
In the Simplexa™ HSV 1 & 2 Direct assay, bi-functional fluorescent probe-primers are used together with corresponding reverse primers to amplify HSV-1, HSV-2 and internal control targets. Well conserved regions of the HSV-1 and HSV-2 DNA polymerase genes are targeted to identify HSV-1 and HSV-2 DNA respectively in the specimen. An internal control is used to detect PCR failure and/or inhibition.
The provided document describes the Simplexa™ HSV 1 & 2 Direct assay and its performance evaluation. Here's a breakdown of the acceptance criteria and study details:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria are not explicitly stated as distinct "acceptance criteria" but can be inferred from the reported performance results and the common thresholds for such diagnostic tests (e.g., typically ≥ 95% agreement/sensitivity/specificity). The document presents the performance in terms of percent agreement with expected results for reproducibility and sensitivity/specificity for clinical agreement.
Reproducibility (Inter-site, Inter-day, Inter/Intra-assay)
| Performance Metric | Acceptance Criteria (Inferred) | Reported Device Performance |
|---|---|---|
| HSV-1 Result - Total % Agreement with Expected Results (all sites combined) | High agreement (e.g., ≥95%) | HSV-1 Low Positive: 100.0% (90/90) |
| HSV-1 Medium Positive: 100.0% (90/90) | ||
| High Negative (for HSV-1): 93.3% (84/90) | ||
| Positive Control (for HSV-1): 100.0% (89/89) | ||
| Total Agreement (HSV-1): 98.5% (531/539) | ||
| HSV-2 Result - Total % Agreement with Expected Results (all sites combined) | High agreement (e.g., ≥95%) | HSV-1 Low Positive (for HSV-2): 98.9% (89/90) |
| HSV-1 Medium Positive (for HSV-2): 100.0% (90/90) | ||
| HSV-2 Low Positive: 94.4% (85/90) | ||
| HSV-2 Medium Positive: 100.0% (90/90) | ||
| High Negative (for HSV-2): 94.4% (85/90) | ||
| Positive Control (for HSV-2): 100.0% (89/89) | ||
| Total Agreement (HSV-2): 98.0% (528/539) | ||
| DNA IC Result - Total % Agreement with Expected Results (all sites combined) | High agreement (e.g., ≥95%) | 100.0% across all sample types (e.g., HSV-1 Low Positive: 100.0% (90/90), Positive Control: 100.0% (89/89)) |
| Total Agreement (IC): 100.0% (539/539) |
Analytical Sensitivity/Limit of Detection (LoD)
| Performance Metric | Acceptance Criteria (Inferred) | Reported Device Performance |
|---|---|---|
| LoD for HSV-1 McIntyre | Detect positive ≥95% of the time | 32/32 (100%) at 4 TCID50/mL |
| LoD for HSV-1 HF | Detect positive ≥95% of the time | 32/32 (100%) at 160 TCID50/mL |
| LoD for HSV-2 G | Detect positive ≥95% of the time | 32/32 (100%) at 2 TCID50/mL |
| LoD for HSV-2 MS | Detect positive ≥95% of the time | 31/32 (~97%) at 10 TCID50/mL |
Clinical Agreement (Prospective Study vs. Composite Comparator)
| Performance Metric | Acceptance Criteria (Inferred) | Reported Device Performance (HSV-1) | Reported Device Performance (HSV-2) |
|---|---|---|---|
| Sensitivity | High sensitivity (e.g., ≥95%) | 97.4% (111/114) | |
| (95% CI: 92.5% to 99.1%) | 97.2% (175/180) | ||
| (95% CI: 93.7% to 98.8%) | |||
| Specificity | High specificity (e.g., ≥95%) | 98.2% (560/570) | |
| (95% CI: 96.8% to 99.0%) | 97.8% (497/508) | ||
| (95% CI: 96.2% to 98.8%) | |||
| Positive Predictive Value (PPV) | High predictive value (context-dependent) | 91.7% (111/121) | |
| (95% CI: 85.5% to 95.4%) | 94.1% (175/186) | ||
| (95% CI: 89.7% to 96.7%) | |||
| Negative Predictive Value (NPV) | High predictive value (context-dependent) | 99.5% (560/563) | |
| (95% CI: 98.4% to 99.8%) | 99.0% (497/502) | ||
| (95% CI: 97.7% to 99.6%) |
Clinical Agreement (Retrospective Study vs. Validated Bi-Directional Sequencing Assay)
| Performance Metric | Acceptance Criteria (Inferred) | Reported Device Performance (HSV-1) | Reported Device Performance (HSV-2) |
|---|---|---|---|
| Positive Percent Agreement (PPA) | High agreement (e.g., ≥95%) | 100.0% (14/14) | |
| (95% CI: 78.5% to 100.0%) | 100.0% (14/14) | ||
| (95% CI: 78.5% to 100.0%) | |||
| Negative Percent Agreement (NPA) | High agreement (e.g., ≥95%) | 92.9% (13/14) | |
| (95% CI: 68.5% to 98.7%) | 100.0% (14/14) | ||
| (95% CI: 78.5% to 100.0%) | |||
| Positive Predictive Value (PPV) | High predictive value (context-dependent) | 93.3% (14/15) | |
| (95% CI: 70.2% to 98.8%) | 100.0% (14/14) | ||
| (95% CI: 78.5% to 100.0%) | |||
| Negative Predictive Value (NPV) | High predictive value (context-dependent) | 100.0% (13/13) | |
| (95% CI: 77.2% to 100.0%) | 100.0% (14/14) | ||
| (95% CI: 78.5% to 100.0%) |
2. Sample Size Used for the Test Set and Data Provenance
-
Reproducibility Study:
- Test Set Size: For each of the six sample pools (HSV-1 Low Positive, HSV-1 Medium Positive, HSV-2 Low Positive, HSV-2 Medium Positive, High Negative, Positive Control), they were tested in triplicate on five different days by two operators at each of three sites. This amounts to: 6 samples * 3 replicates * 5 days * 2 operators * 3 sites = 540 data points (assuming complete datasets). The tables consolidate this, showing "Total Agreement" out of 539 samples for HSV-1, and 539 for HSV-2 and DNA IC across all categories (e.g., 531/539 for HSV-1 Total Agreement, 528/539 for HSV-2 Total Agreement, 539/539 for DNA IC Total Agreement).
- Data Provenance: The study was conducted at "Three investigative sites," implying a multi-center study within the U.S. (typical for FDA submissions). It's a prospective design as samples were "contrived" (prepared specifically for the study) and tests were performed over different days.
-
Clinical Agreement (Prospective Study):
- Test Set Size:
- Initially, 718 genital swab samples were prospectively collected.
- 9 samples were removed (not tested or invalid results on 3 assays).
- 13 samples were removed (not tested on comparator method sufficiently).
- 696 samples were used for the primary analysis.
- Further exclusions for specific analyses: 6 samples excluded from discordant analysis due to insufficient volume for FDA cleared NAAT; 2 samples excluded for HSV-1/HSV-2 discrepancy due to insufficient volume for FDA cleared NAAT.
- Final analysis for HSV-1: 684 samples.
- Final analysis for HSV-2: 688 samples.
- Data Provenance: "Prospectively collected from patients with signs and symptoms of genital herpes simplex virus (HSV) infection from 6 geographically diverse locations." This indicates the data is from the U.S. and is prospective.
- Test Set Size:
-
Clinical Agreement (Retrospective Study):
- Test Set Size: 28 genital swab samples (14 positive HSV-1 and 14 positive HSV-2).
- Data Provenance: "Retrospectively collected from male patients with signs and symptoms of genital herpes simplex virus (HSV) infection." Specific country of origin is not stated but implies within the U.S. given the FDA submission.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts
-
For the Reproducibility Study, the ground truth for contrived samples (low positive, medium positive, high negative, positive control) was established based on the known concentrations of HSV-1 and HSV-2 spiked into the matrix. This is an analytical study, not one requiring clinical experts for ground truth.
-
For the Clinical Agreement Studies (Prospective and Retrospective), the ground truth was established by a "composite comparator algorithm" and "validated bi-directional sequencing assay." This ground truth relies on laboratory results rather than expert interpretation of clinical data or images. Therefore, the document does not specify human experts for establishing ground truth, but rather relies on the interpretative methods of the comparator assays. Those interpreting the culture results, sequencing data, and FDA-cleared NAAT results would typically be trained laboratory personnel or clinical laboratorians.
4. Adjudication Method for the Test Set
-
Reproducibility Study: No explicit adjudication method is mentioned as the ground truth was "expected results" for contrived samples based on known concentrations. Adjudication wasn't necessary for these controlled samples.
-
Clinical Agreement (Prospective Study): An adjudication method was used for discordant results in the composite comparator algorithm:
- "A positive result for HSV-1 and/or HSV-2 was determined by a positive test result in either the culture or the bi-directional sequencing."
- "If both the culture and the bi-directional sequencing yielded positive results but disagreed in the differentiation of HSV-1 versus HSV-2, the results of the FDA cleared NAAT were used and a 2 out of 3 rule was followed to determine the type of the virus (e.g. if two of the methods were positive for HSV-1, the final comparator result was HSV-1 positive)." This is a form of 2-out-of-3 or majority rule adjudication.
-
Clinical Agreement (Retrospective Study): The ground truth was established by a "validated bi-directional sequencing assay." No specific adjudication process is described beyond the sequencing assay itself.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
- No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not done. This study focuses on an in vitro diagnostic (IVD) PCR assay for the detection of viral DNA, not on image interpretation or other tasks typically performed by human readers that could be augmented by AI. Therefore, there's no mention of human readers or AI assistance.
6. If a Standalone (i.e. algorithm only without human-in-the loop performance) was done
- Yes, the performance presented for the Simplexa™ HSV 1 & 2 Direct assay is standalone performance. It describes the diagnostic accuracy of the assay itself (device only) against a comparator method. The device is an automated, real-time PCR system, and its output (qualitative detection and differentiation of HSV-1 and HSV-2 DNA) is directly compared. The assay operates without human "in-the-loop" interpretation for the final diagnostic call.
7. The Type of Ground Truth Used
- Reproducibility Study: Known concentrations of spiked viral material (contrived samples) were used as ground truth.
- Clinical Agreement (Prospective Study): A composite comparator algorithm was used as ground truth, consisting of:
- Culture
- Bi-directional sequencing
- An FDA cleared NAAT (used for discordant resolution)
- Clinical Agreement (Retrospective Study): A validated bi-directional sequencing assay was used as ground truth.
8. The Sample Size for the Training Set
- The document describes performance evaluation studies (reproducibility and clinical agreement) for the Simplexa™ HSV 1 & 2 Direct assay. It does not explicitly mention a "training set" or "validation set" in the context of machine learning model development. This is typical for IVD assays which are developed and then verified/validated rather than 'trained'. The data presented is for validation of the a priori defined assay.
9. How the Ground Truth for the Training Set Was Established
- As no "training set" in the machine learning sense is described, this question is not applicable based on the provided document. The ground truth for the validation and reproducibility sets is detailed in point 7.
§ 866.3305 Herpes simplex virus serological assays.
(a)
Identification. Herpes simplex virus serological assays are devices that consist of antigens and antisera used in various serological tests to identify antibodies to herpes simplex virus in serum. Additionally, some of the assays consist of herpes simplex virus antisera conjugated with a fluorescent dye (immunofluorescent assays) used to identify herpes simplex virus directly from clinical specimens or tissue culture isolates derived from clinical specimens. The identification aids in the diagnosis of diseases caused by herpes simplex viruses and provides epidemiological information on these diseases. Herpes simplex viral infections range from common and mild lesions of the skin and mucous membranes to a severe form of encephalitis (inflammation of the brain). Neonatal herpes virus infections range from a mild infection to a severe generalized disease with a fatal outcome.(b)
Classification. Class II (special controls). The device is classified as class II (special controls). The special control for the device is FDA's revised guidance document entitled “Class II Special Controls Guidance Document: Herpes Simplex Virus Types 1 and 2 Serological Assays.” For availability of the guidance revised document, see § 866.1(e).