(88 days)
The illumigene HSV 1&2 DNA amplification assay, performed on the illumipro-10™, is a qualitative in vitro diagnostic test for the direct detection and differentiation of herpes simplex virus type 1 (HSV-1) and herpes simplex virus type 2 (HSV-2) DNA in cutaneous and mucocutaneous lesion specimens from male and female patients suspected of Herpetic infections.
illumigene HSV 1&2 utilizes loop-mediated isothermal DNA amplification (LAMP) technology to detect HSV-1 and HSV-2 by targeting segments of the herpes simplex virus 1 and herpes simplex virus 2 genomes. Results from illumigene HSV 1&2 are used as an aid in the diagnosis of HSV infection in symptomatic patients.
The assay is intended for use in hospital, reference or state laboratory settings. This device is not intended for nonlaboratory point-of-care use.
WARNING: illumigene HSV 1&2 is not FDA cleared for use with cerebrospinal fluid (CSF) or to aid in the diagnosis of HSV infections of the central nervous system (CNS). The device is not intended for prenatal screening.
The illumigene Molecular Diagnostic Test System is comprised of the illumigene® HSV 1&2 DNA Amplification Assay Test Kit, the illumigene HSV 1&2 External Controls Kit, and the illumipro-10™ Automated Isothermal Amplification and Detection System.
The illumigene HSV 1&2 molecular assay utilizes loop-mediated amplification (LAMP) technology to detect herpes simplex virus in cutaneous and mucocutaneous lesion swab specimens. Each illumigene HSV 1&2 assay is completed using illumigene Sample Preparation Apparatus III (SMP PREP III), illumigene HSV 1 Test Devices, illumigene HSV 2 Test Devices, Mineral Oil, illumigene Heat Treatment Tubes, and transfer pipettes. Using a transfer pipette, specimens are added to SMP PREP III and dispensed into Heat Treatment Tubes. Samples are heat-treated to make target and control DNA available for amplification. Each heat-treated sample is added to one illumigene HSV 1 and one illumigene HSV 2 Test Device. Mineral oil is added to each illumigene Test Device to prevent evaporation.
The illumipro-10™ heats each illumigene HSV 1 and HSV 2 Test Device containing prepared sample and control material, facilitating amplification of target DNA. When HSV-1 or HSV-2 is present in the specimen, a 208 base pair (bp) sequence of the HSV-1 glycoprotein G (US4) gene or a 189 bp sequence of the HSV-2 glycoprotein G (US4) gene is amplified and magnesium pyrophosphate is generated. Magnesium pyrophosphate forms a precipitate in the reaction mixture.
The illumipro-10™ monitors the absorbance characteristics of the reaction solutions at the assay Run Start (Signal initial, S initial) and at the assay Run End (Signal final, S final). The illumipro-10™ calculates the change in light transmission between Run End and Run Start (S final; S initial) and compares the ratio to an established cut-off value. The illumipro-10™ performs this ratio calculation to both the TEST chamber and the CONTROL chamber.
Fixed cut-off values for the TEST chamber are used to report sample results. TEST chamber S final:S initial ratios less than 82% are reported as 'POSITIVE'; TEST chamber S final:S initial ratios greater than or equal to 82% are reported as 'NEGATIVE'. Numerical values are not reported.
Fixed cut-off values for the CONTROL chamber are used to determine validity. CONTROL chamber S final:S initial ratios less than 90% are considered valid and allow for reporting of TEST chamber results (POSITIVE). CONTROL chamber S final:S initial ratios greater than or equal to 90% are considered invalid and prevent reporting of TEST chamber results. Invalid CONTROL chamber reactions are reported as "INVALID'. More stringent cut-off criteria are applied to the CONTROL chamber reaction is not inhibited, reagents are performing as intended and that sample processing was performed appropriately.
The illumigene HSV 1&2 External Controls Kit contains a combined HSV-1 and HSV-2 Positive Control and a Negative Control (Negative Control IV) for use in routine Quality Control testing. External Control reagents are provided to aid the user in detection of reagent deterioration, adverse environmental or test conditions, or variance in operator performance that may lead to test errors.
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria are not explicitly stated as distinct pass/fail thresholds in percentage terms for sensitivity and specificity. However, the clinical study results represent the device's performance against a reference method (ELVIS® HSV ID and D3 Typing Test System) and can be considered the demonstrated performance for substantial equivalence. The document describes combined site performance for both cutaneous and mucocutaneous lesions for HSV-1 and HSV-2. Because the data is presented separately for HSV-1 cutaneous, HSV-1 mucocutaneous, HSV-2 cutaneous, and HSV-2 mucocutaneous, I will aggregate the positive and negative counts to provide an overall performance.
Overall Aggregated illumigene® HSV 1&2 DNA Amplification Assay Performance vs. ELVIS® HSV ID and D3 Typing Test System (Clinical Study)
| Metric | HSV-1 Performance (Combined Lesions) | HSV-2 Performance (Combined Lesions) |
|---|---|---|
| Sensitivity | 200 / 211 = 94.8% | 179 / 181 = 98.9% |
| Specificity | 729 / 763 = 95.5% | 930 / 974 = 95.5% |
Breakdown by Lesion Type (as provided in the document)
| Target | Lesion Type | Cases Analyzed (ELVIS) | illumigene Pos (ELVIS Pos) | illumigene Pos (ELVIS Neg) | illumigene Neg (ELVIS Pos) | illumigene Neg (ELVIS Neg) | Sensitivity (95% CI) | Specificity (95% CI) |
|---|---|---|---|---|---|---|---|---|
| HSV-1 | Cutaneous | 264 | 48 | 6 | 3 | 207 | 94.1% (84.1-98.0%) | 97.2% (94.0-98.7%) |
| HSV-1 | Mucocutaneous | 710 | 152 | 28 | 8 | 522 | 95.0% (90.5-97.5%) | 94.9% (92.7-96.5%) |
| HSV-2 | Cutaneous | 306 | 42 | 13 | 0 | 251 | 100% (91.6-100.0%) | 95.1% (91.8-97.1%) |
| HSV-2 | Mucocutaneous | 849 | 137 | 31 | 2 | 679 | 98.6% (94.9-99.6%) | 95.6% (93.9-96.9%) |
2. Sample size used for the test set and the data provenance
- Sample Size (Test Set):
- Clinical Study: A total of 1158 leftover, de-identified lesion swab specimens were initially evaluated.
- For HSV-1 performance analysis, 974 specimens were included (after exclusions).
- For HSV-2 performance analysis, 1155 specimens were included (after exclusions).
- Data Provenance: The document does not explicitly state the country of origin. The study was conducted between October 2014 and March 2015. The specimens were "leftover, de-identified lesion swab specimens," indicating a retrospective study using previously collected samples.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
The ground truth for the clinical performance data was established by comparison to the ELVIS® HSV ID and D3 Typing Test System. This is a laboratory-based diagnostic system, not a visual assessment by individual human experts. Therefore, the concept of "number of experts" and "qualifications of those experts" for establishing ground truth in the traditional sense (e.g., radiologists) does not apply here. The "expert" in this context is the validated reference diagnostic system itself.
4. Adjudication method for the test set
Not applicable. The ground truth was established by comparing the device's results to a predicate diagnostic system (ELVIS® HSV ID and D3 Typing Test System). There was no adjudication process involving human experts resolving discrepancies for the ground truth of the clinical samples. For discrepancies between the illumigene assay and the ELVIS system, some samples were re-tested with an "alternative, FDA-cleared molecular assay," but this is a reconciliation method, not an adjudication of primary ground truth experts.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs. without AI assistance
Not applicable. This device is an in vitro diagnostic (IVD) assay for molecular detection of HSV DNA, not an AI-powered image analysis tool or system designed to assist human readers. Therefore, an MRMC study and evaluation of human reader improvement with/without AI assistance are not relevant to this submission.
6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done
Yes, the clinical evaluation of the illumigene® HSV 1&2 DNA Amplification Assay against the ELVIS® HSV ID and D3 Typing Test System represents a standalone performance study of the algorithm (the assay itself) as an in vitro diagnostic device without human-in-the-loop intervention for result interpretation; the illumipro-10™ system automatically interprets signals based on fixed cut-off values.
7. The type of ground truth used
The ground truth used for the clinical performance study was the ELVIS® HSV ID and D3 Typing Test System, which is a legally marketed predicate diagnostic device for HSV detection and typing. This is a form of reference standard comparison. Additionally, for discordant results, an "alternative, FDA-cleared molecular assay" was used for confirmation, further supporting the reference standard approach.
8. The sample size for the training set
The document does not explicitly state a separate "training set" sample size for the clinical study directly tied to an external ground truth comparison. However, the "Assay cut-off" section mentions that "Development optimization includes evaluation of characterized positive and negative clinical specimens. Amplification reagent concentrations are adjusted during design as needed to ensure illumigene results are aligned with clinical specimen reported results." This implies internal development and optimization based on clinical specimens, but a specific "training set" as commonly defined in machine learning or AI context, with a stated size, is not provided.
9. How the ground truth for the training set was established
As inferred above, the ground truth for any internal optimization/development (which might be considered analogous to a training set) was established through "characterized positive and negative clinical specimens." The characterization method for these specimens is not detailed beyond "reported results," but it would logically rely on established diagnostic methods.
§ 866.3309 Herpes virus nucleic acid-based cutaneous and mucocutaneous lesion panel.
(a)
Identification. A herpes virus nucleic acid-based cutaneous and mucocutaneous lesion panel is a qualitative in vitro diagnostic device intended for the simultaneous detection and differentiation of different herpes viruses in cutaneous and mucocutaneous lesion samples from symptomatic patients suspected of Herpetic infections. Negative results do not preclude infection and should not be used as the sole basis for treatment or other patient management decisions. The assay is not intended for use in cerebrospinal fluid samples.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Premarket notification submissions must include detailed documentation for the device description, including the device components, ancillary reagents required but not provided, and a detailed explanation of the methodology including primer design and selection.
(2) Premarket notification submissions must include detailed documentation from the following analytical and clinical performance studies: Analytical sensitivity (Limit of Detection), reactivity, inclusivity, precision, reproducibility, interference, cross reactivity, carry-over, and cross contamination.
(3) Premarket notification submissions must include detailed documentation of a clinical study using lesion samples in which Herpes Simplex Virus 1, Herpes Simplex Virus 2, or Varicella Zoster Virus DNA detection was requested. The study must compare the device performance to an appropriate well established reference method.
(4) A detailed explanation of the interpretation of results and acceptance criteria must be included in the device's 21 CFR 809.10(b)(9) compliant labeling.
(5) The device labeling must include a limitation statement that reads: “The device is not intended for use with cerebrospinal fluid or to aid in the diagnosis of HSV or VZV infections of the central nervous system (CNS).”
(6) Premarket notification submissions must include quality assurance protocols and a detailed documentation for device software, including, but not limited to, standalone software applications and hardware-based devices that incorporate software.
(7) The risk management activities performed as part of the manufacturer's 21 CFR 820.30 design controls must document an appropriate end user device training program that will be offered as part of efforts to mitigate the risk of failure to correctly operate the instrument.