(288 days)
The NeoBase™ 2 Non-derivatized MSMS kit is intended for the measurement and evaluation of amino acid, succinylacetone, free carnitine, acylcarnitine, nucleoside and lysophospholipid concentrations (Table 1) with a tandem mass spectrometer from newborn heel prick blood specimens dried on filter paper. Quantitative analytis of these analytes and their relationship with each other is intended to provide analyte concentration profiles that may aid in screening newborns for metabolic disorders.
Not Found
The provided text describes the acceptance criteria and study results for the NeoBase 2 Non-derivatized MSMS kit.
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state numerical acceptance criteria for the screening performance studies (e.g., minimum sensitivity or specificity targets). Instead, it states that "All verification studies were successfully concluded and met the respective study's predetermined acceptance criteria." The clinical studies for screening performance are presented as agreement between the new device (NeoBase 2) and the predicate device (NeoBase). The agreement is presented as contingency tables (e.g., "Screening positive" vs "Screening negative" for both devices).
The performance is demonstrated by the agreement between the NeoBase 2 Non-derivatized MSMS kit and the predicate device, NeoBase Non-derivatized MSMS kit, in detecting various metabolic disorders in newborn screening. The results are presented in terms of the number of positive and negative screens detected by each device, along with the number of confirmed positive specimens.
Summary of Device Performance (from Tables A, B, C, D):
| Disorder Group | Cut-off Type (Percentile) | NeoBase 2 Screening Positive (with Predicate Positive) | NeoBase 2 Screening Negative (with Predicate Negative) | Total Specimens | Confirmed Positive Specimens (detected by both methods) |
|---|---|---|---|---|---|
| Study 1 | |||||
| Amino acid disorders | 99th | 621 | 1591 | 1751 | 15 |
| Amino acid disorders | 99.5th | 452 | 1645 | 1751 | 15 |
| Amino acid disorders | 1st | 161 | 1687 | 1737 | 1 (OTCD) |
| Fatty acid oxidation | 99th | 801 | 1581 | 1746 | 10 |
| Fatty acid oxidation | 99.5th | 451 | 1661 | 1746 | 10 |
| Fatty acid oxidation | Low Percentile | 1732 | 1386 | 1738 | 2 (CUD) |
| Organic acid condition | 99th | 571 | 1660 | 1751 | 15 |
| Organic acid condition | 99.5th | 361 | 1697 | 1751 | 15 |
| ADA-SCID | 99th | 2 | 1661 | 1738 | 2 |
| ADA-SCID | 99.5th | 2 | 1700 | 1738 | 2 |
| X-ALD | 99th | 2 | 1724 | 1738 | 2 |
| X-ALD | 99.5th | 2 | 1731 | 1738 | 2 |
| Study 2 | |||||
| Amino acid disorders | 99th | 1161 | 2353 | 2648 | 19 |
| Amino acid disorders | 99.5th | 782 | 2474 | 2648 | 18 |
| Amino acid disorders | 1st | 141 | 2571 | 2631 | 2 (OTCD) |
| Fatty acid oxidation | 99th | 1601 | 2326 | 2641 | 12 |
| Fatty acid oxidation | 99.5th | 1081 | 2442 | 2641 | 12 |
| Fatty acid oxidation | Low Percentile | 1581 | 2363 | 2632 | 3 |
| Organic acid condition | 99th | 861 | 2479 | 2642 | 13 |
| Organic acid condition | 99.5th | 422 | 2561 | 2642 | 12 |
| ADA-SCID | 99th | 2 | 2563 | 2631 | 2 |
| ADA-SCID | 99.5th | 2 | 2578 | 2631 | 2 |
| X-ALD | 99th | 2 | 2626 | 2631 | 2 |
| X-ALD | 99.5th | 2 | 2628 | 2631 | 2 |
2. Sample Size Used for the Test Set and Data Provenance
-
Study 1 Sample Size:
- Amino acid disorders, Fatty acid oxidation, Organic acid conditions: 1751 samples (for 99th and 99.5th percentile cut-offs) and 1737-1746 samples (for 1st and low percentile cut-offs).
- ADA-SCID and X-ALD: 1738 samples.
-
Study 2 Sample Size:
- Amino acid disorders, Fatty acid oxidation, Organic acid conditions: 2631-2648 samples.
- ADA-SCID and X-ALD: 2631 samples.
-
Data Provenance: The data was obtained from "routine newborn screening" in "two CLIA-certified state laboratories." The confirmed positive specimens were described as "retrospective" for Study 2. This suggests a retrospective study design using existing samples and accompanying diagnostic information. The country of origin is not explicitly stated but is implied to be the US due to "CLIA-certified state laboratories."
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The document does not specify the number or qualifications of experts used to establish the ground truth for the test set. It mentions "confirmed positive specimens," implying a definitive diagnostic process was followed to establish the true disease status of these samples, but details on the experts involved are not provided.
4. Adjudication Method for the Test Set
The document does not describe an adjudication method for the test set, such as 2+1 or 3+1. The acceptance is based on the agreement between the new device and the predicate device, using established cut-offs derived from routine newborn screening data.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No MRMC comparative effectiveness study was done. This device is a diagnostic kit measuring analyte concentrations, not an AI system assisting human readers. Therefore, the concept of "how much human readers improve with AI vs without AI assistance" is not applicable.
6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study
The study described is a comparison of the new device (NeoBase 2) to a predicate device (NeoBase) in obtaining analyte concentrations. While not explicitly stated as an "algorithm only" study, it's a standalone performance comparison of two test kits. The results (analyte concentrations and screening positive/negative classifications) are derived directly from the kit's operation with a tandem mass spectrometer, without human interpretation being part of the primary measurement process itself. The interpretation of the analyte profiles to aid in screening for metabolic disorders would typically involve medical professionals, but the performance data presented is on the analytical and classification output of the device.
7. Type of Ground Truth Used
The ground truth for the test set was based on "confirmed positive specimens." This implies that the true disease status of these specimens was established through clinical diagnosis and follow-up, which would typically involve a combination of clinical outcomes, biochemical testing, and/or genetic testing, ultimately confirmed by clinical experts. For ADA-SCID and X-ALD, it explicitly states "comparing the result... to the clinical condition."
8. Sample Size for the Training Set
The document does not explicitly mention a "training set" in the context of machine learning or AI. The term "cut-offs for both methods were determined by calculating the 99.5th and 99th percentile for all analytes" using "data from routine newborn screening." This large volume of routine newborn screening data could be considered analogous to a training or reference population used to establish the operating characteristics of the screening test. The specific sample size for this cut-off determination is not given, but it is implied to be a large dataset from the "two CLIA-certified state laboratories."
9. How the Ground Truth for the Training Set Was Established
As discussed in point 8, there isn't a traditional "training set" for an AI model. However, the cut-off values (e.g., 99th, 99.5th, 1st, 10th percentiles) used to define "screening positive" or "screening negative" were established using "data from routine newborn screening." This means the ground truth for establishing these cut-offs would inherently come from the statistical distribution of analyte levels in a large, presumably healthy and general newborn population, along with the understanding of what analyte levels are indicative of various metabolic disorders. The document states that the cut-off values "only apply to these studies."
§ 862.1055 Newborn screening test system for amino acids, free carnitine, and acylcarnitines using tandem mass spectrometry.
(a)
Identification. A newborn screening test system for amino acids, free carnitine, and acylcarnitines using tandem mass spectrometry is a device that consists of stable isotope internal standards, control materials, extraction solutions, flow solvents, instrumentation, software packages, and other reagents and materials. The device is intended for the measurement and evaluation of amino acids, free carnitine, and acylcarnitine concentrations from newborn whole blood filter paper samples. The quantitative analysis of amino acids, free carnitine, and acylcarnitines and their relationship with each other provides analyte concentration profiles that may aid in screening newborns for one or more inborn errors of amino acid, free carnitine, and acyl-carnitine metabolism.(b)
Classification. Class II (special controls). The special control is FDA's guidance document entitled “Class II Special Controls Guidance Document: Newborn Screening Test Systems for Amino Acids, Free Carnitine, and Acylcarnitines Using Tandem Mass Spectrometry.” See § 862.1(d) for the availability of this guidance document.