(299 days)
The EnLite™ Neonatal TREC Kit is an in vitro diagnostic device intended for the semiquantitative determination of TREC (T-cell receptor excision circle) DNA in blood specimens dried on filter paper. The test is for use on the VICTOR™ EnLite instrument. The test is indicated for use as an aid in screening newborns for severe combined immunodeficiency disorder (SCID).
This test is not intended for use as a diagnostic test or for screening of SCID-like Syndromes, such as DiGeorge Syndrome, or Omenn Syndrome. It is also not intended to screen for less acute SCID syndromes such as leaky-SCID or variant SCID.
The EnLite™ Neonatal TREC Kit is comprised of the EnLite™ Neonatal TREC Kit, the VICTOR™ EnLite instrument and the EnLite™ workstation software. The EnLite™ Neonatal TREC Kit contains reagents sufficient for 384 reactions or 1152 reactions, and multi-level, dried blood spot (DBS) calibrators and controls. The DBS calibrators and DBS controls have been prepared from porcine whole blood with a hematocrit value of 48-55%, and contain purified salmon-sperm, TREC, and beta-actin DNA.
The EnLite™ Neonatal TREC Kit is an in vitro diagnostic device for semi-quantitative determination of T-cell receptor excision circles (TRECs) in dried blood specimens, used as an aid in screening newborns for severe combined immunodeficiency disorder (SCID).
1. Acceptance Criteria and Reported Device Performance
The acceptance criteria for the EnLite™ Neonatal TREC Kit are outlined in the regulatory information, specifically within the "Special Controls" section (Section T, point 1(iii)). These criteria detail the required analytical and clinical performance characteristics for the device. The reported device performance is presented throughout the "Performance Characteristics" section (Section M).
Here's a table summarizing key acceptance criteria and reported performance, focusing on the clinical validation study as that directly addresses the intended use of screening:
Table of Acceptance Criteria and Reported Device Performance
| Performance Characteristic | Acceptance Criteria (from Special Controls) | Reported Performance (from Clinical Study) |
|---|---|---|
| Clinical Validity | Data demonstrating clinical validity using well-characterized prospectively or retrospectively obtained clinical specimens representative of the intended use population. |
A minimum of 10-15 confirmed positive specimens from more than one site, with relevant annotation, and SCID diagnosis by flow cytometry or clinically meaningful information regarding subject status at one year or beyond.
Additional specimens characterized by other disorders with low/absent TREC (e.g., other T-cell lymphopenic disorders) to supplement the range of results.
Pre-specified clinical decision point (cut-off) before studies.
Results summarized in tabular format comparing interpretation to reference method.
Point estimates and 95% CIs for PPA, NPA, and OPA.
Data must include retest rate, false positive rate before retest, final false positive rate, and false negative rate. | The primary clinical study objective was to demonstrate the EnLite™ Neonatal TREC Kit's screening performance in the intended use population and its ability to discriminate between normal and SCID cases. The study was conducted retrospectively.
SCID Positive Specimens: 17 archived confirmed SCID positive DBS specimens were obtained from newborn screening laboratories in the US. All 17 were confirmed for SCID by flow cytometry. These enriched the study due to the low incidence of SCID.
Other Low TREC Specimens: An additional 9 DBS specimens from babies with low TREC values (0 to 20 TREC Copies/uL) were included.
Comparator: For routine clinical study specimens, the comparator was the clinical assessment from medical records at one year of age or older (365 days), confirming the newborn was not identified with SCID, was not deceased from SCID-related complications, and was apparently healthy. For confirmed SCID cases, the comparator was the reference tests results for SCID confirmation.
Pre-specified Cut-off: The cut-off for TREC was pre-determined to be 36 copies/uL and for beta-actin as 56 copies/uL, based on the 2.5th percentile of normal distribution data from a separate cut-off confirmation study using 2846 archived, retrospective newborn specimens from the Danish Newborn Screening Biobank.
Retest Rate: The retest rate was 1.9%.
False Positive Rate: The false positive rate using the cut-off of 36 in the first round of testing was 1.5%. After repeat testing on follow-up cases, the final false positive rate was 0.5%.
False Negative Rate: The clinical data indicates 0 false negative results among the 16 confirmed SCID positives classified after the final testing round (Table 14).
Performance (from Table 14, excluding invalid results):
- Overall Percent Agreement (OPA): 99.7% (95% CI: 99.4% to 99.8%)
- Positive Percent Agreement (PPA): 100% (95% CI: 79.4% to 100%)
- Negative Percent Agreement (NPA): 99.7% (95% CI: 99.4% to 99.8%)
Note: One SCID positive specimen in the clinical study was classified as an invalid result, leading to 16 confirmed SCID positives being used for final agreement calculations. |
2. Sample Sizes and Data Provenance
-
Test Set Sample Size:
- Clinical Study: A total of 6,471 neonatal specimens were run, with 6,373 included in the final analysis. This included 6,389 routine Danish newborn screening biobanked newborn routine DBS samples and 82 enrichment samples (17 confirmed SCID positive samples, 9 confirmed low-level TREC specimens, and 56 samples used for blinding purposes). For the final agreement calculations, 5,454 specimens (after some exclusions and loss-to-follow-up) were used, specifically 5,442 after removing invalid results (16 confirmed SCID positives and 5,426 normal/presumptive normal).
- Cut-off Establishment Study: 3,243 archived, retrospective newborn specimens initially, with 2,846 included in the analysis after exclusions.
- Analytical Performance Studies (Examples):
- Reproducibility (Site-to-Site): 90 measurements per sample (6 unique TREC levels, 10 runs x 3 laboratories x 3 replicates/sample).
- Precision: 27 runs performed over 20 days. For TREC precision, 10 samples were assessed with 4 replicates/sample. For beta-actin, 7 samples were used.
- LoB/LoD/LoQ: 5 samples for LoB (60 results per sample); 5 samples for LoD/LoQ (108 results per sample).
-
Data Provenance:
- Clinical Study & Cut-off Establishment:
- Country of Origin: Denmark (samples from the Danish Newborn Screening Biobank, comprising the Danish population).
- Retrospective/Prospective: All samples were archived, retrospective.
- SCID Enrichment Samples: 17 confirmed SCID positive DBS specimens were obtained from newborn screening laboratories in the US (retrospective).
- Clinical Study & Cut-off Establishment:
3. Number of Experts and Qualifications for Ground Truth
The document does not explicitly state the number of experts used to establish the ground truth for the test set, nor their specific qualifications (e.g., "radiologist with 10 years of experience").
- However, the ground truth for the 17 confirmed SCID positive specimens was established by flow cytometry, which is a specialized laboratory test requiring expert interpretation, presumably by qualified clinical immunologists or pathologists.
- For the routine newborn specimens, the comparator for ground truth was the clinical assessment of the study subjects obtained from their medical records at one year of age or older (365 days), confirming they were not identified with SCID, were not deceased from SCID-related complications, and were apparently healthy. This clinical assessment would implicitly involve input from various medical professionals (pediatricians, specialists).
- The "expert" component primarily comes from the reference method (flow cytometry) for SCID diagnosis and the subsequent clinical follow-up for the larger cohort.
4. Adjudication Method for the Test Set
The adjudication method for the test set was not explicitly described as a multi-expert consensus process like "2+1" or "3+1" that is common in medical imaging studies. Instead, the ground truth for SCID confirmation was primarily based on:
- Laboratory Confirmation: Flow cytometry for the 17 confirmed SCID cases.
- Clinical Outcomes: Medical record review at one year of age or older for the large cohort of routine newborns to determine the absence of SCID.
The device itself has an internal retesting algorithm (Section P.4 and P.4, Figure 8). Initial results below the cut-off are "presumptive positive" and are retested in duplicate. This internal retesting acts as a form of "internal adjudication" for the device's own classification, but it's not external expert adjudication of the ground truth.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No MRMC comparative effectiveness study was mentioned. This device is a laboratory diagnostic kit and not an AI-assisted diagnostic tool for human readers (like a CAD system for radiologists). Therefore, a study to measure how much human readers improve with AI vs. without AI assistance is not applicable to this type of device.
6. Standalone Performance
The study primarily assessed the standalone performance of the device/kit (EnLite™ Neonatal TREC Kit, VICTOR™ EnLite instrument, and EnLite™ workstation software) in classifying samples as "presumptive positive" or "normal" based on its quantitative TREC and beta-actin measurements and the predefined cut-offs. The results (PPA, NPA, OPA) reflect the performance of the integrated system in a laboratory setting, without direct human cognitive interpretation of raw data for diagnosis. The output from the device is a quantitative TREC value, which is then used with a hard cut-off.
7. Type of Ground Truth Used
The ground truth used was a combination of:
- Laboratory Test Confirmation: For the known SCID positive cases, flow cytometry was used to confirm SCID.
- Clinical Outcomes Data: For the large cohort of routine newborn screening samples, the absence of SCID was determined through medical record review (vaccination records, national patient registry, civil registration system) at one year of age or older, looking for signs of SCID or SCID-related complications/death.
8. Sample Size for the Training Set
The document describes the evaluation of an already developed device/kit, not a machine learning model. Therefore, there is no explicit "training set" in the context of machine learning model development. The data used for establishing the clinical cut-off (2,846 samples from the Danish Newborn Screening Biobank) could be considered analogous to a "development" or "calibration" dataset, which informed the final cut-off value used in the pivotal study (test set).
9. How the Ground Truth for the Training Set Was Established
As noted above, there isn't a "training set" for a machine learning model. For the dataset used to establish the clinical cut-off (2,846 samples):
- These were archived, retrospective newborn specimens from the Danish Newborn Screening Biobank.
- The ground truth in this context was based on the distribution of TREC and beta-actin values in this "normal newborn population". The 2.5 percentile of this distribution was then chosen as the clinical cut-off for TREC (36 copies/uL) and beta-actin (56 copies/uL). This is a statistical approach to defining "normal" for screening purposes, rather than a direct disease diagnosis for each individual sample.
§ 866.5930 Newborn screening test for severe combined immunodeficiency disorder (SCID).
(a)
Identification. A newborn screening test for SCID is a prescription device intended to measure T-cell receptor excision circle (TREC) DNA obtained from dried blood spot specimens on filter paper using a polymerase chain reaction based test as an aid in screening newborns for SCID. Presumptive positive results must be followed up by diagnostic confirmatory testing. This test is not intended for use as a diagnostic test, or for screening of SCID-like syndromes, such as DiGeorge syndrome or Omenn syndrome. It is also not intended to screen for less acute SCID syndromes, such as leaky SCID or variant SCID.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Premarket notification submissions must include the following information:
(i) The intended use must indicate:
(A) The test is not intended for diagnostic use, or for screening of SCID-like syndromes, such as DiGeorge syndrome or Omenn syndrome; and
(B) The test is not intended to screen for less acute SCID syndromes, such as leaky SCID or variant SCID.
(ii) A detailed description of all components in the test that includes:
(A) A detailed description of the test components, all required reagents, instrumentation and equipment, including illustrations or photographs of nonstandard equipment or methods;
(B) Detailed documentation of the device software including, but not limited to, standalone software applications and hardware-based devices that incorporate software;
(C) Specifications for the filter paper, which must be appropriately labeled for in vitro diagnostic use, to be used in specimen collection and how it will be used in specimen collection validation. These specifications must include: descriptive characteristics of the filter paper, instructions on how a lab should choose the appropriate filter paper, chemical properties of the filter paper, interference concerns associated with the chemicals in the filter paper, absorption properties of the filter paper, punch size, absorption capacity, testing for homogeneity of punches, diameter of the circle for the dried blood spot aliquot, absorption time, physical composition, and number and size of punches to be tested;
(D) Methodology and protocols for detection of T-cell receptor excision circles and methods for determination of results. The cutoff must be selected before conducting clinical and analytical studies;
(E) A description of the result outputs along with sample reports. Sample reports must include the scale used in reporting of results (
e.g., TREC copies/µL) and the range of values that will be reported out; and(F) A description of appropriate internal and external controls that are recommended or provided. The description must identify those control elements that are incorporated into the testing procedure.
(iii) Information that demonstrates the performance characteristics of the test, including:
(A) Data that demonstrates the clinical validity of the device, using well characterized prospectively or retrospectively obtained clinical specimens representative of the intended use population. A minimum of 10 to 15 confirmed positive specimens must be obtained from more than 1 site, including relevant annotation, and, at 1 year or beyond, a SCID diagnosis by flow cytometry or clinically meaningful information regarding the status of the subject must be obtained. Additional specimens should have been obtained that are characterized by other disorders that can be found by screening specimens that have low or absent TREC (
e.g., other T-cell lymphopenic disorders) to supplement the range of results. The clinical validation study must have a pre-specified clinical decision point (i.e., cutoff to distinguish positive and negative results). Results must be summarized in tabular format comparing interpretation of results to the reference method. Point estimates together with two-sided 95 percent confidence intervals must be provided for the positive percent agreement, negative percent agreement, and overall percent agreement. Data must include the retest rate, the false positive rate before retest, the final false positive rate, and the false negative rate;(B) Device reproducibility data generated, using a minimum of three sites of which at least two must be external sites, with two operators at each site. Each site must conduct a minimum of five runs per operator over five nonconsecutive days evaluating a minimum of six different relevant TREC concentrations that span and are well distributed over the measuring range and include the clinical cutoff. Specimens must include cord blood and cord blood diluted with ABO matched adult blood specimens. Identical specimens from the same sample panel must be tested at each site. Each specimen must be run in triplicate and include controls run in triplicate. Results must be reported as the standard deviation and percentage coefficient of variation for each level tested. Results must also be displayed as a dichotomous variable around the cutoff. Total variation must be partitioned into the sum of within-lab and between-lab variations with pre-specified acceptance criteria and 95 percent confidence intervals for all data. Pre-specified acceptance criteria must be provided and followed;
(C) Device precision data using clinical samples to evaluate the within-lot, between-lot, within-run, between run, and total variation. A range of TREC levels of the specimen must include samples within the measuring range, samples above and below the measuring range, as well as with samples very near above and below the cutoff value. At least three replicates of each specimen must be tested with controls and calibrator(s) according to the device instructions for use. The precision study must use well characterized samples using different lots, instruments, and operators. Results must be summarized in tabular format. Pre-specified acceptance criteria must be provided and followed;
(D) Linearity of the test must be demonstrated using a dilution panel from clinical samples. The range of dilution samples must include samples within the measuring range, samples above and below the measuring range, as well as with samples very near above and below the cutoff value. Results of the regression analysis must be summarized in tabular format and fitted into a linear regression model with the individual measurement results against the dilution factors. Pre-specified acceptance criteria must be provided and followed;
(E) Device analytic sensitivity data, including limit of blank, limit of detection, and limit of quantification;
(F) Device specificity data, including interference, carryover, cross-contamination, and in silico analysis of potential off-target genomic sequences;
(G) Device stability data, including real-time stability of samples under various storage times, temperatures, and freeze-thaw conditions. A separate shipping stability study must be performed;
(H) Lot-to-lot reproducibility study of each filter paper that will be validated with the test. The lot-to-lot study must include a minimum of three lots of each blood spot card that will be validated with the test and be conducted over five nonconsecutive days. The sample panel must consist of specimens with a range of TREC levels and include samples within the measuring range, samples above and below the measuring range, and samples very near above and below the cutoff value. Multiple punches must be obtained from each card for demonstration of homogeneity of the analyte across the dried blood spot. Comparability of the test performance for each filter paper must be demonstrated. Stability and storage of TREC DNA on each blood spot card must be demonstrated. Results of the lot-to-lot study must be summarized providing the mean, standard deviation, and percentage coefficient of variation in a tabular format. Data must be calculated for within-run, between-run, within-lot, and between-lot. Data demonstrating the concordance between results across different filter papers must be provided. Study acceptance criteria must be provided and followed; and
(I) If applicable, a thermocycler reproducibility study must be performed using thermocyclers from three independent thermocyler manufacturers. The sample panel must consist of specimens with a range of TREC levels and must include samples within the measuring range, samples above and below the measuring range, and samples very near above and below the cutoff value. The study must be done using three filter paper lots and conducted over five nonconsecutive days. Results of the thermocycler reproducibility study must be summarized providing the mean, standard deviation, and percentage coefficient of variance in a tabular format. Data must be calculated for the within-run, between-run, within-lot, between-lot, and between thermocycler manufacturer study results. Study acceptance criteria must be provided and followed.
(iv) Identification of risk mitigation elements used by your device, including a description of all additional procedures, methods, and practices incorporated into the directions for use that mitigate risks associated with testing.
(2) Your § 809.10 compliant labeling must include:
(i) A warning statement that reads “This test is not intended for diagnostic use, preimplantation or prenatal testing, or for screening of SCID-like syndromes, such as DiGeorge syndrome or Omenn syndrome. It is also not intended to screen for less acute SCID syndromes, such as leaky SCID or variant SCID.”;
(ii) A warning statement that reads “Test results are intended to be used in conjunction with other clinical and diagnostic findings, consistent with professional standards of practice, including confirmation by alternative methods and clinical evaluation, as appropriate.”;
(iii) A description of the performance studies listed in paragraph (b)(1)(iii) and a summary of the results; and
(iv) A description of the filter paper specifications required for the test.