(81 days)
The Lyra™ Direct HSV 1 + 2/VZV Assay is an in vitro multiplex Real-Time PCR test for qualitative detection and differentiation of herpes simplex virus type 1, herpes simplex virus type 2, and varicella-zoster virus DNA isolated and purified from cutaneous or mucocutaneous lesion samples obtained from symptomatic patients suspected of active herpes simplex virus 1. herpes simplex virus 2 and/or varicella-zoster infection. The Lyra™ Direct HSV 1 + 2/VZV Assay is intended to aid in the diagnosis of herpes simplex virus 1, herpes simplex virus 2 and varicella-zoster virus active cutaneous or mucocutaneous infections. Negative results do not preclude herpes simplex virus 1, herpes simplex virus 2 and varicella-zoster virus infections and should not be used as the sole basis for diagnosis, treatment or other management decisions. The Lyra™ Direct HSV 1 + 2/VZV Assay is not intended for use with cerebrospinal fluid or to aid in the diagnosis of HSV or VZV infections of the central nervous system (CNS). The Lyra™ Direct HSV 1 + 2/VZV Assay is not intended for use in prenatal screening. The device is not intended for point-of-care use.
The Lyra™ Direct HSV 1 + 2/VZV Assay detects viral nucleic acids from a patient sample. A multiplex Real-Time PCR reaction is carried out under optimized conditions in a single tube or well generating amplicons for HSV-1, HSV-2, VZV, and the Process Control (PRC). Identification of amplicons for HSV-1, HSV-2, VZV, and the PRC occurs by the use of target-specific primers and fluorescent-labeled probes that hybridize to conserved regions in the genomes of HSV-1, HSV-2, and VZV and to the PRC, respectively.
Here's an analysis of the acceptance criteria and study details for the Lyra™ Direct HSV 1 + 2/VZV Assay, formatted as requested:
Acceptance Criteria and Device Performance for Lyra™ Direct HSV 1 + 2/VZV Assay
The Lyra™ Direct HSV 1 + 2/VZV Assay is an in vitro multiplex Real-Time PCR test for qualitative detection and differentiation of HSV-1, HSV-2, and VZV DNA from cutaneous or mucocutaneous lesion samples. The regulatory classification is Class II with special controls (21 CFR 866.3309).
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state pre-defined acceptance criteria with numerical thresholds in a single table. Instead, performance is presented through various analytical and clinical studies. We will infer the de facto acceptance criteria based on the demonstrated performance that led to the device's classification. For clinical performance, the comparable effectiveness is evaluated against established methods (ELVIS® HSV ID and D3 Typing Test for HSV and DSFA and Culture with DFA for VZV). Successful performance in these comparisons implies the acceptance criteria were met.
Inferred Acceptance Criteria Table and Reported Device Performance:
| Performance Metric | Target Analyte | Acceptance Criteria (Inferred from successful study) | Reported Device Performance (Life Technologies QuantStudio™ Dx) | Reported Device Performance (Applied Biosystems® 7500 Fast Dx) | Reported Device Performance (Cepheid SmartCycler® II) |
|---|---|---|---|---|---|
| CLINICAL PERFORMANCE (Cutaneous Lesions) | |||||
| Sensitivity | HSV-1 | High, approaching 100% | 100% (24/24) | 100% (24/24) | 100% (24/24) |
| Specificity | HSV-1 | High, >95% | 98.4% (250/254) | 98.8% (252/254) | 98.8% (251/254) |
| Sensitivity | HSV-2 | High, approaching 100% | 100% (35/35) | 97.1% (34/35) | 100% (35/35) |
| Specificity | HSV-2 | High, >95% | 96.3% (234/243) | 96.7% (236/244) | 96.7% (235/243) |
| Sensitivity | VZV | High, approaching 100% | 100% (27/27) | 100% (26/27) | 100% (27/27) |
| Specificity | VZV | High, >90% | 95.9% (187/195) | 95.9% (189/196) | 94.9% (185/195) |
| CLINICAL PERFORMANCE (Mucocutaneous Lesions) | |||||
| Sensitivity | HSV-1 | High, >95% | 97.1% (100/103) | 95.1% (98/103) | 98.1% (101/103) |
| Specificity | HSV-1 | High, >95% | 97.1% (527/543) | 98.2% (531/541) | 97.2% (525/540) |
| Sensitivity | HSV-2 | High, approaching 100% | 100% (95/95) | 97.9% (93/95) | 98.9% (94/95) |
| Specificity | HSV-2 | High, >95% | 96.2% (530/551) | 97.1% (533/549) | 97.1% (532/548) |
| Sensitivity | VZV | High, approaching 100% | 100% (4/4) | 100% (4/4) | 100% (4/4) |
| Specificity | VZV | High, >97% | 98.8% (423/428) | 99.3% (423/426) | 98.8% (420/425) |
| ANALYTICAL PERFORMANCE | |||||
| Reproducibility (Detection Rate) | Low Positive | 100% for all analytes | 90/90 (HSV-1), 90/90 (HSV-2), 89/90 (VZV) | 90/90 (HSV-1), 90/90 (HSV-2), 88/90 (VZV) | 90/90 (HSV-1), 89/90 (HSV-2), 88/90 (VZV) |
| Reproducibility (Negative Samples) | Negative | 0% Detection | 0/90 (all analytes) | 0/90 (all analytes) | 0/90 (all analytes) |
| Limit of Detection (LoD) | All Analytes | LoD within 3 doubling dilutions across platforms | Achieved for various strains (see section 1.e for details) | Achieved for various strains (see section 1.e for details) | Achieved for various strains (see section 1.e for details) |
| Analytical Reactivity | Multiple strains | Detection at near LoD concentrations (100% positivity for all tested strains at specified concentrations) | All 16 tested strains were detected | All 16 tested strains were detected | All 16 tested strains were detected |
| Analytical Specificity (Cross-Reactivity/Inhibition) | Diverse Microorganisms, Endogenous Substances | No cross-reactivity with negative samples, no inhibition of positive samples | No cross-reactivity observed, no inhibition observed | No cross-reactivity observed, no inhibition observed | No cross-reactivity observed, no inhibition observed |
| Competitive Interference | Multiple analytes in same sample | No interference when multiple analytes present at varying concentrations | No competitive interference observed | No competitive interference observed | No competitive interference observed |
| Carry-over Contamination | PCR products | No cross-contamination from high positives to negative samples | No cross-contamination (0/48 tested negatives positive) | No cross-contamination (0/48 tested negatives positive) | No cross-contamination (0/48 tested negatives positive) |
Study Proving Acceptance Criteria Met:
The device's performance characteristics, including both analytical and clinical studies, were conducted to demonstrate it meets the requirements for a Class II designation with special controls.
2. Sample Sizes Used for the Test Set and Data Provenance
Test Set Sample Sizes (Clinical Study):
The clinical study was a multi-center study conducted between April 2013 and October 2013.
- Life Technologies QuantStudio™ Dx:
- Cutaneous Lesions: 279 specimens initially collected (278 for HSV-1/HSV-2, 222 for VZV after exclusions).
- Mucocutaneous Lesions: 650 specimens initially collected (646 for HSV-1/HSV-2, 432 for VZV after exclusions).
- Applied Biosystems® 7500 Fast Dx:
- Cutaneous Lesions: 279 specimens (279 for HSV-1/HSV-2, 223 for VZV after exclusions).
- Mucocutaneous Lesions: 650 specimens initially collected (644 for HSV-1/HSV-2, 430 for VZV after exclusions).
- Cepheid SmartCycler® II:
- Cutaneous Lesions: 279 specimens initially collected (278 for HSV-1/HSV-2, 222 for VZV after exclusions).
- Mucocutaneous Lesions: 650 specimens initially collected (643 for HSV-1/HSV-2, 429 for VZV after exclusions).
Data Provenance:
- Country of Origin: Not explicitly stated, but the submission is to the FDA, suggesting United States given the typical regulatory context for De Novo submissions. The multi-center nature implies data was collected from different clinical sites within the same regulatory jurisdiction.
- Retrospective or Prospective: The clinical study description states: "A multi-center study was performed between April, 2013 and October, 2013 to evaluate the Lyra™ Direct HSV 1 + 2/VZV Assay using lesion swab specimens obtained from cutaneous or mucocutaneous lesions and submitted for HSV and/or VZV culture." This clearly indicates a prospective collection of specimens for the purpose of the study.
3. Number of Experts Used to Establish Ground Truth for the Test Set and Their Qualifications
The ground truth for the clinical test set was established using FDA-cleared comparator methods, not directly by human experts in the sense of independent adjudication.
- For HSV-1 and HSV-2, the comparator method was the ELVIS® HSV ID and D3 Typing Test, an FDA-cleared cell culture system.
- For VZV, the comparator method involved staining cells present in the samples with an FDA-cleared VZV detection reagent (DSFA) and culturing the specimen using a mixed cell culture (H&V mixed cells) followed by staining with the same FDA-cleared reagent used for DSFA.
Therefore, there were no direct human experts establishing ground truth in the typical sense of radiologists or pathologists. The ground truth was based on the results of established, FDA-cleared laboratory methods, operated by trained laboratory personnel.
4. Adjudication Method for the Test Set
The primary adjudication method for discrepancies between the Lyra™ Direct Assay and the comparator methods was the use of an additional RT-PCR assay.
- For discrepancies in HSV-1 (e.g., Lyra™ positive, comparator negative), these cases were "positive by an additional RT-PCR assay."
- Similarly for HSV-2 and VZV discrepancies, "positives were positive by an additional RT-PCR assay."
- In some cases where Lyra™ was negative but the comparator was positive, these were also re-evaluated by an additional RT-PCR assay, with the statement "negatives were positive by an additional RT-PCR assay."
This acts as a tie-breaker or confirmatory test for discordant results, enhancing the robustness of the ground truth derived from the comparator methods.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs without AI Assistance
No, an MRMC comparative effectiveness study was not done. This device is an in vitro diagnostic (IVD) PCR assay, not an AI/CADe medical imaging device that assists human readers. Its performance is evaluated compared to established laboratory methods, not human interpretation of images. Therefore, the concept of "human readers improve with AI vs without AI assistance" is not applicable here.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
Yes, the primary clinical study represents standalone performance. The Lyra™ Direct HSV 1 + 2/VZV Assay is an automated real-time PCR test. Its results (positive/negative for each virus) are generated directly by the instrument and its software, without human interpretation of raw signals to determine the final viral status. The clinical sensitivity and specificity reported are the performance of the algorithm/device alone compared to the ground truth.
7. The Type of Ground Truth Used (Expert Consensus, Pathology, Outcomes Data, etc.)
The ground truth used for the clinical study was based on FDA-cleared conventional laboratory methods:
- Cell Culture with Immunofluorescence Staining (or similar detection):
- HSV-1 and HSV-2: ELVIS® HSV ID and D3 Typing Test.
- VZV: Direct Smear Fluorescent Antibody (DSFA) and mixed cell culture with DFA staining.
- Confirmatory RT-PCR/molecular assay: Used as an adjudication method for discordant results between the Lyra™ Direct Assay and the primary comparator methods.
8. The Sample Size for the Training Set
The document does not explicitly describe a separate "training set" for the clinical evaluation in the way machine learning algorithms typically use them. For IVD devices like this, the "development" or "training" process involves optimizing the assay components (primers, probes, reaction conditions) and setting analytical cut-offs based on analytical studies (e.g., Limit of Detection, reactivity, specificity studies) using characterized samples.
The reproducibility and precision studies used simulated samples (medium positive, low positive, high negative, negative, 5x LoD, 2x LoD,
§ 866.3309 Herpes virus nucleic acid-based cutaneous and mucocutaneous lesion panel.
(a)
Identification. A herpes virus nucleic acid-based cutaneous and mucocutaneous lesion panel is a qualitative in vitro diagnostic device intended for the simultaneous detection and differentiation of different herpes viruses in cutaneous and mucocutaneous lesion samples from symptomatic patients suspected of Herpetic infections. Negative results do not preclude infection and should not be used as the sole basis for treatment or other patient management decisions. The assay is not intended for use in cerebrospinal fluid samples.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Premarket notification submissions must include detailed documentation for the device description, including the device components, ancillary reagents required but not provided, and a detailed explanation of the methodology including primer design and selection.
(2) Premarket notification submissions must include detailed documentation from the following analytical and clinical performance studies: Analytical sensitivity (Limit of Detection), reactivity, inclusivity, precision, reproducibility, interference, cross reactivity, carry-over, and cross contamination.
(3) Premarket notification submissions must include detailed documentation of a clinical study using lesion samples in which Herpes Simplex Virus 1, Herpes Simplex Virus 2, or Varicella Zoster Virus DNA detection was requested. The study must compare the device performance to an appropriate well established reference method.
(4) A detailed explanation of the interpretation of results and acceptance criteria must be included in the device's 21 CFR 809.10(b)(9) compliant labeling.
(5) The device labeling must include a limitation statement that reads: “The device is not intended for use with cerebrospinal fluid or to aid in the diagnosis of HSV or VZV infections of the central nervous system (CNS).”
(6) Premarket notification submissions must include quality assurance protocols and a detailed documentation for device software, including, but not limited to, standalone software applications and hardware-based devices that incorporate software.
(7) The risk management activities performed as part of the manufacturer's 21 CFR 820.30 design controls must document an appropriate end user device training program that will be offered as part of efforts to mitigate the risk of failure to correctly operate the instrument.