(328 days)
The Illumina MiSeqDx Cystic Fibrosis 139-Variant Assay is a qualitative in vitro diagnostic system used to simultaneously detect 139 clinically relevant cystic fibrosis disease-causing mutations and variants of the cystic fibrosis transmembrance regulator (CFTR) gene in genomic DNA isolated from human peripheral whole blood specimens. The variants include those recommended in 2004 by the American College of Medical Genetics (ACMG) and in 2011 by the American College of Obstetricians and Gynecologists (ACOG). The test is intended for carrier screening in adults of reproductive age, in confirmatory diagnostic testing of newborns and children, and as an initial test to aid in the diagnosis of individuals with suspected cystic fibrosis. The results of this test are intended to be interpreted by a board-certified clinical molecular geneticist or equivalent and should be used in conjunction with other available laboratory and clinical information. This test is not indicated for use for newborn screening, fetal diagnostic testing, pre-implantation testing, or for stand-alone diagnostic purposes.
The test is intended to be used on the Illumina MiSeqDxTM Instrument.
The Illumina MiSeqDx Cystic Fibrosis 139-Variant Assay consists of library preparation and sample indexing reagents, sequencing reagents and consumables, MiSeqDx instrument and data analysis software. Testing begins with genomic DNA from a peripheral whole blood sample. The genomic DNA is processed through the library preparation steps, which specifically amplifies the intended genomic regions of each sample while also adding the indexes for sample identification. Flow cell capture sequences are also added to the amplified products. The resulting sample libraries are then transferred into a MiSeqDx reagent cartridge which contains all of the reagents required for cluster generation and sequencing (Sequencing By Synthesis). The MiSeqDx Cartridge, MiSeqDx Flow Cell, and MiSeqDx SBS Solution (PR2) are then inserted into the MiSeqDx instrument, which performs cluster generation, sequencing and data analysis.
Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:
Acceptance Criteria and Device Performance
| Measure | Acceptance Criteria (Not explicitly stated as criteria, but implied by 100% and >99.99% results in accuracy study and high metrics in reproducibility) | Reported Device Performance (Accuracy Study) | Reported Device Performance (Reproducibility Study) |
|---|---|---|---|
| Genotype-level PA | High (implied 100% or near 100%) | 100% for all variants | 99.77% |
| Negative Agreement (NA) | High (implied >99%) | >99.99% for all wild types | 99.88% |
| Overall Agreement (OA) | High (implied >99%) | >99.99% for all reported positions | 99.88% |
| Sample Pass Rate | Not explicitly stated for accuracy study, but for reproducibility study, samples passing QC on first attempt | N/A for accuracy study (all tested samples were successfully genotyped) | 99.9% (number of samples passing QC metrics on the first attempt) |
| Accuracy (PolyTG/PolyT) | High (implied 100% in most cases, with explanations for deviations) | Varies by genotype, generally 100%, with exceptions explained (e.g., 50.0% for one less common genotype, 90.9% for another) | N/A for this specific sub-analysis in the reproducibility section, but overall reproducibility metrics apply. |
| Accuracy (Interfering Substances) | 100% call rate and 100% reproducibility | 100% call rate and 100% reproducibility in genotype calls | N/A (tested in interference study) |
| Call Rate (DNA Extraction) | 100% | 100% | N/A (tested in extraction study) |
| Accuracy (DNA Extraction) | 100% | 100% | N/A (tested in extraction study) |
| Sample First Pass Rate (DNA Extraction) | 100% | 100% | N/A (tested in extraction study) |
| Accuracy (DNA Input) | 100% | 100% | N/A (tested in DNA input study) |
| Sample First Pass Rate (DNA Input) | >95% for upper and lower bounds; 100% for specific tested input levels | >95% for 1250 ng and 25 ng; 100% for 1250 ng, 250 ng, 100 ng, and 25 ng samples | N/A (tested in DNA input study) |
| Reproducibility & Accuracy (Sample Indexing) | 100% | 100% | 100% (for sample/index primer combinations) |
Note regarding "Acceptance Criteria": The document K124006 primarily presents the results of validation studies rather than explicitly stating pre-defined "acceptance criteria" in a separate section. The "acceptance criteria" implied above are derived from the consistently high performance reported across various studies and the overall conclusion of the device being substantially equivalent.
Study Details
2. Sample Sizes Used for the Test Set and Data Provenance
- Accuracy Study (Primary Source):
- Test Set Size: 500 samples in total, comprised of:
- 366 samples from a clinical accuracy study (primary source).
- 68 cell line samples from the reproducibility study (supplementary).
- 14 clinical samples from the extraction method evaluation analytical study (supplementary).
- 52 synthetic plasmid samples (supplementary).
- Data Provenance:
- Clinical Samples: Majority (n=355) were archived, anonymized clinical gDNA specimens isolated from human blood. (No specific country of origin mentioned, but implied to be human clinical samples).
- Cell Line Samples: 11 samples were commercially available cell line specimens.
- Synthetic Plasmid Samples: Designed to include genomic context of rare variants, linearized, diluted, and blended with human genomic DNA (lab-generated, not from patients).
- Retrospective/Prospective: The 355 clinical gDNA specimens were "archived, anonymized clinical gDNA specimens," suggesting a retrospective approach for these samples. The cell line and synthetic samples were laboratory-generated.
- Test Set Size: 500 samples in total, comprised of:
- Reproducibility Study:
- Test Set Size: 46 samples (in each of two panels, total 92 unique samples) were tested. Each sample was tested for a total of 810 calls per site.
- Data Provenance: Genomic DNA from lymphoblastoid cell lines with known mutations and leukocyte-depleted blood spiked with lymphoblastoid cell lines with known mutations. This suggests laboratory-controlled samples, not directly from clinical patients in a prospective manner.
- DNA Extraction Study:
- Test Set Size: 14 unique blood samples per extraction method, with 2 operators performing 3 runs each, and each run having 2 replicates, totaling 168 (14 x 2 x 3 x 2) for each of the three extraction methods.
- Data Provenance: K2EDTA anti-coagulated whole blood samples, including wild type and three mutant genotypes (samples with F508del, I506V, and D110H).
- DNA Input Study:
- Test Set Size: 14 representative DNA samples (with 16 unique CF variants) tested in duplicate at 9 DNA input levels. Additional testing involved 4 representative samples with 20 replicates each (n=80), and 14 samples with 20 replicates each (n=280).
- Data Provenance: Representative DNA samples containing CF variants.
- Interfering Substances Study:
- Test Set Size: 8 whole blood specimens (including 3 CF positive samples with unique genotypes). 16 replicates each for specific interferents.
- Data Provenance: Whole blood specimens.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The document specifies the reference methods used to establish ground truth, but does not explicitly state the number or qualifications of experts involved in reviewing these reference results or establishing consensus.
- For 137 SNA/small InDel sites and PolyTG/Poly T region: Sanger bi-directional sequence analysis was the reference method.
- For two large deletions: A PCR-based assay was the reference method, confirmed for accuracy using Sanger Sequencing.
While Sanger sequencing and PCR assays are considered robust reference methods, the process of interpreting and confirming these results as "ground truth" and addressing any discrepancies would typically involve qualified molecular geneticists or lab personnel. However, this level of detail is not provided.
4. Adjudication Method for the Test Set
The document does not describe a formal adjudication method (e.g., 2+1, 3+1, none) for the test set. Ground truth was established by reference methods (Sanger bi-directional sequencing and a PCR-based assay). Any "miscalls" (one in the accuracy study and a few in the reproducibility study) or "no calls" are noted and sometimes explained (e.g., insufficient coverage, sample handling issues), implying internal review rather than a multi-expert adjudication process for primary ground truth establishment.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No Multi-Reader Multi-Case (MRMC) comparative effectiveness study was done. This device is an in vitro diagnostic system for detecting genetic mutations, not an imaging device or a diagnostic aid meant to directly assist human readers in interpreting complex visual data. Therefore, the concept of "human readers improve with AI vs. without AI assistance" is not applicable in this context.
6. Standalone (Algorithm Only Without Human-in-the-Loop Performance)
Yes, a standalone performance assessment was conducted. The accuracy and reproducibility studies directly evaluated the device's ability (including its data analysis software) to detect mutations and variants in genomic DNA. The "results of this test are intended to be interpreted by a board-certified clinical molecular geneticist or equivalent and should be used in conjunction with other available laboratory and clinical information," indicating that while human interpretation is part of the overall clinical process, the performance metrics (PA, NA, OA) are for the device (algorithm + instrument) alone.
7. Type of Ground Truth Used
The ground truth used was primarily based on:
- Sanger bi-directional sequence analysis: For 137 SNA/small InDel sites and the PolyTG/Poly T region. This is considered a gold standard for sequence verification.
- PCR-based assay: For two large deletions, confirmed by Sanger Sequencing.
This represents established molecular diagnostic laboratory methods, widely accepted as reliable for genetic variant detection.
8. Sample Size for the Training Set
The document does not report the sample size for a training set. This is typical for in vitro diagnostic devices based on established sequencing technology and a defined set of targets, where the "training" may involve internal development and optimization using design verification samples rather than a formally described, distinct "training set" in the context of machine learning model development. The focus is on validation against known samples to determine performance characteristics.
9. How the Ground Truth for the Training Set Was Established
As no explicit "training set" is described, how its ground truth might have been established is not mentioned in the document. Development and optimization would have relied on samples with known CFTR genotypes, likely confirmed by methods similar to those used for the test set (Sanger sequencing, PCR).
§ 866.5900 Cystic fibrosis transmembrane conductance regulator (CFTR) gene mutation detection system.
(a)
Identification. The CFTR gene mutation detection system is a device used to simultaneously detect and identify a panel of mutations and variants in the CFTR gene. It is intended as an aid in confirmatory diagnostic testing of individuals with suspected cystic fibrosis (CF), carrier identification, and newborn screening. This device is not intended for stand-alone diagnostic purposes, prenatal diagnostic, pre-implantation, or population screening.(b)
Classification. Class II (special controls). The special control is FDA's guidance document entitled “Class II Special Controls Guidance Document: CFTR Gene Mutation Detection System.” See § 866.1(e) for the availability of this guidance document.