(85 days)
The ARIES® HSV 1&2 Assay is a real-time polymerase chain reaction (PCR) based qualitative in vitro diagnostic test for the direct detection and differentiation of Herpes Simplex Virus 1 and HSV 2) DNA in cutaneous or mucocutaneous lesion specimens from symptomatic patients. The test is indicated for use as an aid in diagnosis of HSV infection in symptomatic patients. The ARIES® HSV 1&2 Assay is indicated for use on the ARIES® System.
WARNING: The ARIES® HSV 1&2 Assay is not FDA cleared for use with cerebrospinal fluid (CSF). The assay is not intended to be used for prenatal screening.
The Luminex ARIES® HSV 1 & 2 Kit is a polymerase chain reaction (PCR)-based qualitative in vitro diagnostic test for the direct detection and differentiation of herpes simplex virus (HSV) DNA using cutaneous and mucocutaneous lesion swab specimens. Patient lesion swab specimens are collected in Copan Universal Transport Medium, or identical Copan manufactured media formulations (Becton Dickinson Universal Viral Transport Media, Copan branded Universal Transport Medium for LabCorp, and the Quest Viral Culture Media) and transported to the laboratory. The specimen is pipetted into a cassette specific to the ARIES® HSV 1&2 Assay. In the cassette the specimen is lysed and nucleic acid is extracted using the ARIES® System and an extraction/ PCR cassette specific to the ARIES® HSV 1&2 Kit. An extractable sample processing control (SPC) target is present in the ARIES® HSV 1&2 assay cassette and is processed with the specimen. The Ct value of the SPC is designed to verify proper specimen lysis and nucleic acid extraction, to identify PCR inhibition, if any, and verify proper function of the extraction system and real-time instrument. The Tm value of the SPC is used as a reference for determining the target Tm.
The extracted nucleic acid is transferred via magnetic beads to the ARIES® HSV 1 & 2 Kit lyophilized PCR reagents in the cassette that contain a primer pair specific to HSV 1 and a primer pair specific to the SPC sequence. The specific primer pairs are labeled with distinct fluorophore labels. PCR amplification is performed and assay fluorescence is monitored on the Luminex ARIES® System. Incorporation of the quencher-labeled nucleotide causes a decrease in assay fluorescence. Following amplification, the reaction is slowly heated and fluorescence is monitored. The strands of the amplification products will separate at a specific melting temperature (Tm) that is determined by an increase in fluorescence as the strands are separated. The sequences between the PCR primer binding sites of the HSV 1 and HSV 2 amplicons have different base compositions that are distinguished by their different Tm. The instrument fluorescence output is analyzed and test results are determined using the ARIES® HSV 1 & 2 Kit assay protocol file. A printed results report is generated.
The Luminex ARIES® HSV 1&2 Assay chemistry is based on an expanded genetic alphabet technology, consisting of synthetic DNA base pair 2'-deoxy-5-methyl-isocytidine (iC): 2'deoxyisoguanosine (iG). The isobases ( iC and iG) pair specifically with each other and not with natural nucleotides. In addition isobases are efficiently incorporated during PCR. During PCR amplification, a quencher-modified iGTP is incorporated by the polymerase opposite an iC and a fluorophore reporter attached to a PCR primer. If the target is present and is amplified, assay fluorescence decreases with every cycle as amplification product accumulates. The decrease in assay fluorescence is monitored in real time using the Luminex ARIES® Instrument. Following PCR, the amplification products are thermally denatured and assay fluorescence is monitored. The strands of the amplification products are separated and assay fluorescence increases, thus enabling determination of the melting temperature (Tm) of the amplicon.
Here's an analysis of the acceptance criteria and study details for the ARIES® HSV 1&2 Assay, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state "acceptance criteria" in a single, defined table. However, analytical and clinical performance metrics are provided and can be inferred as the criteria against which the device's performance was judged.
| Parameter | Acceptance Criteria (Inferred) | Reported Device Performance |
|---|---|---|
| Analytical Sensitivity (LoD) | Lowest concentration with ≥ 95% positivity. | HSV-1 MacIntyre: 7.11E+03 TCID50/mL (100% positivity) |
| HSV-1 F: 16.5 TCID50/mL (95.8% positivity) | ||
| HSV-2 MS: 2.7 TCID50/mL (100% positivity) | ||
| HSV-2 G: 2.8 TCID50/mL (100% positivity) | ||
| Co-infection Detection | All analytes detected when present at equal concentrations. Analytes at low concentration may not be detected in presence of high concentration of another analyte. | HSV-1 High / HSV-2 Low: HSV 1 Positive (100%) |
| HSV-2 High / HSV-1 Low: HSV 2 Positive (100%) | ||
| HSV-1 High/HSV-2 High: HSV 1&2 Positive (100%) | ||
| Interfering Substances | 100% positive for HSV positive samples, 100% negative for HSV negative samples, no interference. | All HSV positive results were 100% positive, all HSV 1&2 negative results were 100% negative. (28 substances tested at specified concentrations). |
| Analytical Specificity (Cross-Reactivity) | 100% positive for HSV positive samples, 100% negative for HSV negative samples, no cross-reactivity/interference. | All HSV positive results were 100% positive, all HSV 1&2 negative results were 100% negative. (61 microorganisms tested at specified concentrations). |
| Reproducibility | High agreement with expected results for moderate positive and low positive samples; expected agreement for high negative. | HSV-1 Moderate Positive: 100% |
| HSV-1 Low Positive: 100% | ||
| HSV-1 High Negative: 32.2% (expected 20-80%) | ||
| HSV-2 Moderate Positive: 100% | ||
| HSV-2 Low Positive: 97.8% | ||
| HSV-2 High Negative: 92.2% (expected 20-80%) | ||
| HSV1&2 Negative: 100% | ||
| Precision (Repeatability) | High agreement with expected results for moderate positive and low positive samples; expected agreement for high negative. | HSV-1 Moderate Positive: 100% |
| HSV-1 Low Positive: 100% | ||
| HSV-1 High Negative: 45.8% (expected 20-80%) | ||
| HSV-2 Moderate Positive: 100% | ||
| HSV-2 Low Positive: 100% | ||
| HSV-2 High Negative: 97.4% (expected 20-80%) | ||
| HSV1&2 Negative: 100% | ||
| Carryover/Cross-Contamination | 0% carry-over/cross-contamination, 100% agreement for positive and negative samples. | 0% carry-over and cross-contamination observed. Overall percent agreement was 100% for positive and negative samples. |
| Fresh Specimen Stability | Expected results for all concentrations across tested time points (up to 15 days at 2-8°C). | Data supports stability for up to 15 days at 2-8°C. Moderate positive 100% positive, low positive ~95% positive, high negative 20-80% positive, negative 0% positive. |
| Frozen Specimen Stability | Expected results for all concentrations across tested time points (up to 12 months at -65 to -95°C). | Data collected up to 3 months shows all targets yielding expected results. Moderate positive 100% positive, low positive ~95% positive, high negative 20-80% positive, negative 100% negative. Stable for up to 3 months. |
| Reagent Stability | Expected results for all targets, lots, and storage temperatures across tested time points (up to 19 months at 2-8°C and 25°C). | Data collected up to 3 months shows all targets for all lots and storage temperatures giving expected results. HSV-1 & HSV-2 replicates 100% positive, negative replicates 100% negative. Stable for 3 months. |
| Clinical Performance | High sensitivity and specificity for HSV-1 and HSV-2 detection in cutaneous and mucocutaneous lesions. | HSV-1 Cutaneous (N=347): Sensitivity 91.1% (95% CI: 80.4-97.0%), Specificity 94.2% (95% CI: 90.8-96.6%) |
| HSV-1 Mucocutaneous (N=1190): Sensitivity 97.0% (95% CI: 94.2-98.7%), Specificity 95.4% (95% CI: 93.9-96.7%) | ||
| HSV-2 Cutaneous (N=448): Sensitivity 95.0% (95% CI: 88.8-98.4%), Specificity 88.8% (95% CI: 85.0-91.9%) | ||
| HSV-2 Mucocutaneous (N=1453): Sensitivity 98.5% (95% CI: 96.2-99.6%), Specificity 93.2% (95% CI: 91.6-94.6%) |
2. Sample Sizes Used for the Test Set and Data Provenance
- Clinical Test Set: A total of 1963 left-over clinical specimens were initially included.
- 1500 specimens were prospectively collected ("all comers").
- 463 specimens were pre-selected (also prospectively collected) to ensure representation of under-represented lesion types.
- 62 specimens were excluded from accuracy determinations (55 undeclared anatomical sites, 4 invalid upon re-testing, 3 unavailable for re-testing).
- Actual N values for clinical performance tables vary (e.g., N=347 for HSV-1 Cutaneous, N=1190 for HSV-1 Mucocutaneous, N=448 for HSV-2 Cutaneous, N=1453 for HSV-2 Mucocutaneous), reflecting subsets analyzed.
- Data Provenance: The clinical study was conducted at three (3) geographically diverse clinical sites in the United States. The data were collected prospectively.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The document does not specify the number or qualifications of experts used to establish the ground truth. The "reference/comparative method" used was the ELVIS HSV ID and D3 Typing Test System (a viral culture-based method). For discrepant results, bi-directional sequencing analysis using analytically validated primers was used as a supplemental reference. This implies that the ground truth was established by laboratory testing methods rather than by expert clinical consensus from a panel.
4. Adjudication Method for the Test Set
The primary reference method was the ELVIS viral culture. When the ARIES® HSV 1&2 Assay results differed from the ELVIS method, bi-directional sequencing analysis was performed as a confirmatory method for discrepancy resolution. This acts as a form of "adjudication" where a third, presumably more definitive, method is used to resolve conflicts between the index test and the primary reference.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance
No MRMC study was described. This device is a fully automated in vitro diagnostic test for direct detection of viral DNA, not an imaging or diagnostic aid that requires human interpretation. Therefore, the concept of "human readers improving with AI" is not applicable here.
6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) Was Done
Yes, the clinical performance study describes the standalone performance of the ARIES® HSV 1&2 Assay, comparing its results directly against the reference method (ELVIS viral culture, with sequencing for discrepancy resolution). The device itself is an automated system (ARIES® System) that performs nucleic acid extraction, amplification, and detection without human intervention in the result generation process.
7. The Type of Ground Truth Used
The ground truth for the clinical study was established using a reference laboratory method:
- Primary Reference: ELVIS HSV ID and D3 Typing Test System (viral culture).
- Discrepancy Resolution/Confirmatory: Bi-directional sequencing analysis using analytically validated primers that targeted genomic regions distinct from the ARIES® HSV 1&2 Assay. This can be considered a highly accurate laboratory-based ground truth.
8. The Sample Size for the Training Set
The document provided is a 510(k) summary for a diagnostic assay, which typically doesn't involve "training sets" in the same way machine learning algorithms do. The "assay" defines the biochemical and molecular components (primers, probes, reagents, protocol file) and the "system" (ARIES® System) is the instrument. These are developed and validated through analytical studies (LoD, specificity, reproducibility, etc.) and then clinically evaluated.
Therefore, there is no explicit "training set" sample size mentioned for a machine learning model, as this is a traditional in vitro diagnostic device, not an AI/ML-based one. The development of such assays involves extensive analytical testing of reagents and instrument parameters, which could be seen as an iterative "training" process for the assay design, but this is not reported in terms of "training set" sample sizes in the context of typical AI device submissions.
9. How the Ground Truth for the Training Set Was Established
Since there is no "training set" in the AI/ML sense, this question is not directly applicable. The assay's performance characteristics (e.g., optimal primer design, reaction conditions, melting temperature thresholds) would have been established during the development and optimization phases before the formal validation studies, using internal R&D samples and controls. The specifics of how these internal "ground truths" were established for assay development are not detailed in this regulatory submission.
§ 866.3309 Herpes virus nucleic acid-based cutaneous and mucocutaneous lesion panel.
(a)
Identification. A herpes virus nucleic acid-based cutaneous and mucocutaneous lesion panel is a qualitative in vitro diagnostic device intended for the simultaneous detection and differentiation of different herpes viruses in cutaneous and mucocutaneous lesion samples from symptomatic patients suspected of Herpetic infections. Negative results do not preclude infection and should not be used as the sole basis for treatment or other patient management decisions. The assay is not intended for use in cerebrospinal fluid samples.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Premarket notification submissions must include detailed documentation for the device description, including the device components, ancillary reagents required but not provided, and a detailed explanation of the methodology including primer design and selection.
(2) Premarket notification submissions must include detailed documentation from the following analytical and clinical performance studies: Analytical sensitivity (Limit of Detection), reactivity, inclusivity, precision, reproducibility, interference, cross reactivity, carry-over, and cross contamination.
(3) Premarket notification submissions must include detailed documentation of a clinical study using lesion samples in which Herpes Simplex Virus 1, Herpes Simplex Virus 2, or Varicella Zoster Virus DNA detection was requested. The study must compare the device performance to an appropriate well established reference method.
(4) A detailed explanation of the interpretation of results and acceptance criteria must be included in the device's 21 CFR 809.10(b)(9) compliant labeling.
(5) The device labeling must include a limitation statement that reads: “The device is not intended for use with cerebrospinal fluid or to aid in the diagnosis of HSV or VZV infections of the central nervous system (CNS).”
(6) Premarket notification submissions must include quality assurance protocols and a detailed documentation for device software, including, but not limited to, standalone software applications and hardware-based devices that incorporate software.
(7) The risk management activities performed as part of the manufacturer's 21 CFR 820.30 design controls must document an appropriate end user device training program that will be offered as part of efforts to mitigate the risk of failure to correctly operate the instrument.