(324 days)
The NucliSens® CMV pp67 assay is a nucleic acid amplification-based qualitative assay to be used in conjunction with the NucliSens® Reader for the detection of human cytomegalovirus (HCMV) pp67 mRNA in EDTA anticoagulated whole blood from adult transplant donors and HIV infected individuals. It is intended as an aid in the diagnosis of active (acute or reactivated) HCMV infection. This product is not intended for use in screening of blood or plasma donors.
The NucliSens® CMV pp67 assay is a nucleic acid amplification-based qualitative assay to be used in conjunction with the NucliSens®™ Reader for the detection of human cytomegalovirus (HCMV) pp67 mRNA in EDTA anticoagulated whole blood from adult transplant donors and HIV infected individuals. It is intended as an aid in the diagnosis of active (acute or reactivated) HCMV infection. This product is not intended for use in screening of blood or plasma donors.
The NucliSens® CMV pp67 assay is comprised of four separate stages:
- Nucleic acid release
- Nucleic acid isolation
- Nucleic acid amplification
- Nucleic acid detection
The provided text describes the NucliSens® CMV pp67 assay and its performance, primarily in comparison to existing methods (CMV pp65 antigenemia assay and cell culture) for diagnosing active HCMV infection in transplant and HIV-infected populations. However, it does not explicitly state "acceptance criteria" in the format of defined thresholds that the device had to meet. Instead, the study provides performance metrics (sensitivity, specificity, agreement) and concludes that the device performed "as well as" the predicate device.
Let's extract the relevant information based on your request, inferring "acceptance criteria" from the comparative performance.
1. Table of Acceptance Criteria and Reported Device Performance
As explicit acceptance criteria are not stated, the table below will summarize the comparative effectiveness results, which are analogous to demonstrating the device meets performance expectations for substantial equivalence. The predicate device's performance (or a combined reference standard) implicitly sets the bar.
| Performance Metric (As inferred) | Acceptance Criteria (Inferred from Predicate/Reference) | Reported Device Performance (NucliSens® CMV pp67) |
|---|---|---|
| Transplant Population (Per Subject) | - Sensitivity comparable to combined Antigenemia/Culture* | Sensitivity: 77% (95% CI: 60.7 to 88.9%) |
| Specificity: Not explicitly calculated for "per subject" in relation to negative subjects where both reference methods were negative (only "0" false positives given). | ||
| Overall Agreement: 82% | ||
| Transplant Population (Per Test) | - Overall Agreement comparable to combined Antigenemia/Culture* | Overall Agreement: 75% |
| HIV-1 Infected Population (Per Subject) | - Sensitivity comparable to combined Antigenemia/Culture* | Sensitivity: 78% |
| Specificity: 89% (95% CI: 51.8 to 99.7%) | ||
| Overall Agreement: 80% | ||
| HIV-1 Infected Population (Per Test) | - Overall Agreement comparable to combined Antigenemia/Culture* | Overall Agreement: 76% |
| Analytical Specificity (Blood Donors) | 100% agreement with CMV pp65 Antigenemia (negative population) | 100% agreement (50/50 negatives by both assays) |
| Analytical Sensitivity (CMV Infected Pop) | Overall agreement with Antigenemia and/or Culture | 84.3% (43/51) |
| Reproducibility | Consistent results across lots, sites, and technicians | Multiple logistic regression indicated no significant difference in detection probability across lots or test sites. (Specific quantitative acceptance criteria like % CV or reproducibility % are not explicitly stated, but the study concludes consistency.) |
| Limit of Detection (LoD) | High probability of detection at low RNA input | 95% response rate at 700 input RNA molecules (95% fiducial limits: 480 to 1158 RNA molecules). This indicates the assay's capability to detect low viral loads. |
| Interfering Substances | No interference from common interfering substances | No interference observed from icteric, hemolyzed, lipemic specimens, antinuclear antibodies, rheumatoid factor, multiparous women, related Herpes viruses (HHV-6A, HHV-6B, HHV-7, HHV-8), or common human bacterial flora for either negative or spiked positive samples (all 10/10 or 8/8 results were as expected). |
Note: The document states "The conclusions drawn from the clinical tests demonstrate that the NucliSens™ CMV pp67 assay is as safe, as effective, and performed as well as the legally marketed CMV Brite™ CMV Antigenemia Detection Test Kit and typical cell culture in the diagnosis of active HCMV infection." This implies that performance comparable to these reference methods serves as the de facto acceptance criteria.
2. Sample Size Used for the Test Set and Data Provenance
- Interfering Substances: 60 samples (10 unspiked negative, 10 spiked positive for each of 6 categories).
- Analytical Specificity (CMV Strains/Cross-reactivity):
- HCMV Strains: Number not specified, but 5 laboratory strains (Towne, C87, Davis, ESP, AD169) were tested.
- Cross-reactivity (other infections/diseases): 86 samples (various infections/diseases, tested unspiked and spiked with HCMV RNA; e.g., 10 HIV-1, 8 HTLV-I/II, 10 HSV-I, etc.)
- Cross-reactivity (related Herpes viruses): 4 cell lines (HHV-6A, HHV-6B, HHV-7, HHV-8 infected).
- Cross-reactivity (bacterial flora): 4 types of bacterial flora, each with 2 aliquots (unspiked and spiked).
- Analytical Specificity (Whole Blood Donors): 100 donors (50 HCMV seronegative, 50 HCMV seropositive).
- Analytical Sensitivity (CMV Infected Population): 51 clinical specimens.
- Reproducibility: 4 samples tested multiple times (repeated 8 times per lot/site/tech combination for some, 4 or 6 times for others). This involved 3 sites, with 1-3 technicians per site, and 3 lots of reagents. Total tests: (37+32+29) + (44+31+33) + (30+35+34) totals 325-334 specific tests for each specimen type (1-4).
- Limit of Detection: Based on the reproducibility data (4 samples with known RNA input, tested repeatedly).
- Clinical Performance (Transplant Population): 50 patients; multiple specimens collected over an extended period. Total valid tests considered: 368.
- Clinical Performance (HIV-1 Infected Population): 50 individuals; multiple specimens collected over an extended period. Total valid tests considered: 450.
Data Provenance:
- Analytical Specificity (CMV Strains/Cross-reactivity): North America and Europe for clinical/analytical studies; laboratory strains AD169, Towne, C87, Davis, ESP.
- Analytical Specificity (Whole Blood Donors): Not specified, but generally implies a US-based population if the study was for FDA clearance.
- Analytical Sensitivity (CMV Infected Population): United States (New England, Northeast, and West Coast regions).
- Reproducibility: 3 sites, likely in different geographical locations (internal to the company or contracted third parties).
- Clinical Performance (Transplant Population): Italy (longitudinal study).
- Clinical Performance (HIV-1 Infected Population): The Netherlands (longitudinal study).
- Overall: The data is primarily prospective (longitudinal studies) and retrospective (existing positive/negative panels) from multiple countries including the USA, Italy, and The Netherlands.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of those Experts
The document does not explicitly mention the number or specific qualifications of experts (e.g., radiologists with 10 years of experience) used to establish ground truth.
Instead, the "ground truth" (referred to as "true state of nature") for the clinical studies was established by conventional laboratory methods:
- For the analytical sensitivity and clinical performance studies, ground truth was determined by cell culture and/or an FDA-cleared CMV pp65 antigenemia assay.
- These are established diagnostic laboratory tests, and their interpretation would typically be performed by trained clinical laboratory professionals or physicians overseeing patient diagnosis.
4. Adjudication Method for the Test Set
The document describes the "ground truth" being established by either antigenemia OR culture being positive for a "CMV positive" designation, and BOTH antigenemia AND culture being negative for a "CMV negative" designation. This serves as the adjudication method for the reference standard.
- "If either antigenemia or culture were found positive for a specimen, that particular specimen was regarded as CMV positive. If both antigenemia and culture were negative, the state of nature for that specimen was considered CMV negative."
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
No. This device is a diagnostic assay (in vitro diagnostic device), not an imaging device or AI-assisted diagnostic tool that would involve human "readers" in the typical sense of an MRMC study assessing image interpretation. The NucliSens® CMV pp67 assay is an automated nucleic acid amplification test. It does not involve human readers interpreting AI output.
6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was done
Yes, the studies presented are for the standalone (algorithm only) performance of the NucliSens® CMV pp67 assay. It's an automated molecular diagnostic test; the "algorithm" is the entire test procedure from sample processing to result interpretation by the NucliSens® Reader. Its performance is compared directly against the established laboratory methods (antigenemia and cell culture).
7. The Type of Ground Truth Used (expert consensus, pathology, outcomes data, etc.)
The primary ground truth used in the clinical performance studies was a composite reference standard of:
- Traditional laboratory methods: Cell culture and an FDA-cleared CMV pp65 antigenemia assay results.
- For the analytical specificity and sensitivity studies, the ground truth involved known CMV strains, known negative panels, and clinical panels confirmed by existing diagnostic tests.
8. The Sample Size for the Training Set
The document does not explicitly describe a separate "training set" for the NucliSens® CMV pp67 assay in the context of machine learning or algorithm development. This is typical for a traditional in vitro diagnostic (IVD) assay which follows a pre-defined chemical and molecular reaction protocol rather than being "trained" iteratively on data. The validation studies (analytical and clinical) serve to demonstrate the performance of the fixed assay.
9. How the Ground Truth for the Training Set Was Established
Since there is no explicitly defined "training set" in the context of an algorithm that learns from data, this question is not directly applicable. If one were to consider any preliminary data used during the assay's development (e.g., to optimize primer design, reaction conditions), the ground truth for such internal development would typically come from well-characterized positive and negative samples, often using established methods like viral culture or PCR. However, this is not detailed in the 510(k) summary.
§ 866.3175 Cytomegalovirus serological reagents.
(a)
Identification. Cytomegalovirus serological reagents are devices that consist of antigens and antisera used in serological tests to identify antibodies to cytomegalovirus in serum. The identification aids in the diagnosis of diseases caused by cytomegaloviruses (principally cytomegalic inclusion disease) and provides epidemiological information on these diseases. Cytomegalic inclusion disease is a generalized infection of infants and is caused by intrauterine or early postnatal infection with the virus. The disease may cause severe congenital abnormalities, such as microcephaly (abnormal smallness of the head), motor disability, and mental retardation. Cytomegalovirus infection has also been associated with acquired hemolytic anemia, acute and chronic hepatitis, and an infectious mononucleosis-like syndrome.(b)
Classification. Class II (performance standards).