(34 days)
The Tag-It™ Cystic Fibrosis Kit is a device used to simultaneously detect and identify a panel of mutations and variants in the cvstic fibrosis transmembrane conductance regulator (CFTR) gene in human blood specimens. The panel includes mutations and variants currently recommended by the American College of Medical Genetics and American College of Obstetricians and Gynecologists (ACMG/ACOG), plus some of the worlds most common and North American-prevalent mutations. The Tag-It™ Cystic Fibrosis Kit is a qualitative genotyping test which provides information intended to be used for carrier testing in adults of reproductive age, as an aid in newborn screening, and in confirmatory diagnostic testing in newborns and children.
The kit is not indicated for use in fetal diagnostic or pre-implantation testing. This kit is also not indicated for stand-alone diagnostic purposes.
The Tag-It™ Cystic Fibrosis Kit includes the following components:
- Multiplex PCR Primer Mix including dNTPs designed to simultaneously produce 16 . amplimers of the CFTR gene
- . Multiplex ASPE Primer Mix including dNTPs (86 primers designed to hybridize to either wild-type or mutant alleles with proprietary sequences at their 5' ends designed to specifically hybridize to complementary sequences coupled to the bead component of the kit)
- Coupled Bead Suspension (86 spectrally distinguishable populations of 5.0 micron . polystyrene beads internally dyed with red and infrared fluorochromes coupled to proprietary DNA sequences designed to specifically hybridize to complementary sequences on the ASPE primers)
- 10X Wash Buffer
- . Tag-It™ Data Analysis Software (TDAS CF-I)
Here's an analysis of the provided text, outlining the acceptance criteria and the study data related to the device's performance:
1. Table of Acceptance Criteria and Reported Device Performance
| Performance Characteristic | Acceptance Criteria (Implicit) | Reported Device Performance |
|---|---|---|
| Reproducibility | >99.99% reproducibility | >99.99% reproducibility |
| Precision | >99.9% repeatability | >99.9% repeatability |
| Accuracy / Concordance | 100% agreement with DNA sequencing (for unambiguous calls) | 100% agreement with DNA sequencing (for unambiguous calls), 95% CI: 99.9% - 100% |
| Detection Limit | Function correctly with 1ng-200ng DNA input | Correct genotyping calls with 1ng-200ng DNA input |
| Analytical Specificity | Acceptable with purified genomic DNA (260/280 ratio 1.7-2.0) | Supported by literature, no interference studies performed as input is purified genomic DNA |
| Stability | 1 year shelf-life at -25°C to -15°C; temporary exposure up to 6 days at 37°C and 3 freeze-thaw cycles tolerated | Kit supports 1 year shelf-life and tolerates temporary exposure/freeze-thaw cycles |
Note on Acceptance Criteria: The document directly states the performance targets as "reproducibility of >99.99%" and "precision of >99.9% repeatability". For accuracy, it reports a 100% agreement, implying this was the target. Other criteria (detection limit, analytical specificity, stability) are described in terms of what the device achieves rather than explicit pre-defined criteria, suggesting they were established as acceptable based on the observed results.
2. Sample Sizes Used for the Test Set and Data Provenance
-
Reproducibility Test Set:
- Genomic Samples: 12 distinct purified genomic samples from Coriell Cell Repository (Camden, NJ). Each sample was assayed in quadruplicate.
- Total replicates tested: 828 replicates (12 samples x 4 replicates/sample x 3 sites x 3 lots = 432 replicates for the reproducibility study, but the text explicitly mentions "828 replicates tested").
- Total calls generated: 35,604 (828 replicates x 43 loci).
- Data Provenance: Genomic samples obtained from the Coriell Cell Repository (Camden, NJ). The study was Prospective, as it involved testing new kit lots at 3 sites over multiple days.
-
Precision Test Set:
- Samples: Replicates of a Coriell genomic sample and sequenced synthetic controls.
- Calls compared: Between 60 and 240 calls (WT, HET, Mu D, and variant detected) were compared per locus, depending on the experiment (day-to-day, user-to-user, lot-to-lot, reagent-to-reagent, machine-to-machine).
- Data Provenance: Coriell genomic samples and synthetic controls. The study appears to be prospective, designed to evaluate various sources of variability.
-
Accuracy / Method Comparison Test Set:
- Clinical Samples: 137 clinical samples (used in varying numbers for each mutation, as shown in Table 3). 68 of these clinical samples yielded unambiguous sequencing calls for all 43 loci. An additional 69 clinical samples were also tested but excluded from the 100% concordance calculation for Table 4 because DNA sequencing only provided unambiguous calls at a subset of the 43 loci.
- Genomic Replicates: Varies by mutation, up to 828 replicates (Table 3). These are purified genomic samples from the Coriell Institute (Camden, NJ).
- Synthetic Controls: Varies by mutation, up to 58 controls (Table 3).
- Data Provenance: A mix of clinical samples (likely from North America/various ethnicities given the context of CF mutation prevalence tables) and purified genomic samples from the Coriell Institute (Camden, NJ). The study design (comparing device calls to sequencing) suggests a retrospective analysis of previously characterized samples.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
The ground truth for the test set (reproducibility, precision, and accuracy) was established by bi-directional dideoxy terminal DNA sequencing. The document does not specify the number of experts used for establishing the ground truth via sequencing or their qualifications.
4. Adjudication Method for the Test Set
The document does not describe an adjudication method for the test set results. The ground truth was based on DNA sequencing. In cases where the Tag-It™ assay generated a "No Call" or an unexpected call for a replicate, it was re-run, and the re-run result was expected to match the sequencing. For clinical samples where sequencing was ambiguous, those samples were excluded from the primary accuracy table, suggesting an implicit "unambiguous sequencing" as the ultimate adjudicator.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not performed. This device is a genotyping kit, not an imaging or diagnostic AI system where human reader performance is typically assessed and compared with or without AI assistance. The performance is assessed against a gold standard (DNA sequencing), not human interpretation of diagnostic output.
6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study
Yes, a standalone performance study was done. The entire evaluation focuses on the performance of the "Tag-It™ Cystic Fibrosis Kit" and its accompanying "TDAS CF-I" software in making genotyping calls. The accuracy and reproducibility studies directly assess the algorithm's output (genotyping calls) against sequencing results, without requiring human intervention in result interpretation beyond initiating the test and reviewing the final software-generated genotype.
7. Type of Ground Truth Used
The primary and most robust ground truth used was bi-directional dideoxy terminal DNA sequencing. For rare alleles or cases where clinical samples were unavailable, synthetic oligonucleotide controls and purified genomic samples from the Coriell Institute (sequenced at all loci) were also used, all of which had their genotypes confirmed by sequencing.
8. Sample Size for the Training Set
The document does not explicitly state a sample size for a "training set." This type of molecular diagnostic kit relies on a fixed set of probes and a rule-based algorithm (thresholds for Median Fluorescence Intensity and allelic ratios) rather than a machine learning model that undergoes a distinct training phase. The "empirically derived ranges" for allelic ratios suggest an internal development and optimization process, but not a formally defined "training set" in the context of an FDA submission for this type of device.
9. How the Ground Truth for the Training Set Was Established
Since there isn't a formally defined "training set" in the machine learning sense, the concept of establishing ground truth for it is not explicitly applicable. However, the document mentions that:
- "The exact values for the thresholds have been empirically determined for each individual variation by running multiple genomic samples."
- "Synthetic controls were prepared by adding the DNA containing the mutation at a copy number equivalent to a natural sample blended in a genomic DNA matrix to represent all possible calls (WT, HET, Mu D, variant detected) for each mutation and variant probed for by the Tag-It™ Cystic Fibrosis Mutation Detection Kit."
This indicates that the thresholds for the algorithm were established through empirical testing with known genomic samples and synthetic controls, likely characterized by DNA sequencing, to ensure accurate genotype discrimination. This process serves a similar function to "training" in that it tunes the algorithm's parameters based on known data, but it's not a machine learning training set as commonly understood today.
§ 866.5900 Cystic fibrosis transmembrane conductance regulator (CFTR) gene mutation detection system.
(a)
Identification. The CFTR gene mutation detection system is a device used to simultaneously detect and identify a panel of mutations and variants in the CFTR gene. It is intended as an aid in confirmatory diagnostic testing of individuals with suspected cystic fibrosis (CF), carrier identification, and newborn screening. This device is not intended for stand-alone diagnostic purposes, prenatal diagnostic, pre-implantation, or population screening.(b)
Classification. Class II (special controls). The special control is FDA's guidance document entitled “Class II Special Controls Guidance Document: CFTR Gene Mutation Detection System.” See § 866.1(e) for the availability of this guidance document.