(399 days)
The Invader® MTHFR 677 test is an in vitro diagnostic test intended for the detection and genotyping of a single point mutation (C to T at position 677) of the human 5.10-methylenetetrahydrofolate reductase (MTHFR) gene in isolated genomic DNA obtained from whole blood Potassium EDTA samples from patients with suspected thrombophilia.
The Invader MTHFR 677 test consists of the following components:
MTHFR 677 Oligo Mix
Universal Buffer
Universal Enzyme Mix
No DNA Control
MTHFR 677 Wild Type Control
MTHFR 677 Heterozygous Control
MTHFR 677 Mutant Control
Invader Call Reporter™ Software
The Invader® MTHFR 677 test is an in vitro diagnostic test for detecting and genotyping the C to T mutation at position 677 of the MTHFR gene.
1. Table of Acceptance Criteria and Reported Device Performance:
The document doesn't explicitly state "acceptance criteria" but demonstrates performance through several studies of 100% agreement with the ground truth for most tests. The primary performance metric is agreement with bi-directional sequencing.
| Performance Metric | Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|---|
| Inter-laboratory Reproducibility | 100% Agreement | 100% (n=540) |
| Lot-to-Lot Reproducibility | 100% Agreement | 100% (n=108) |
| Real-Time Stability | 100% Agreement | 100% (7 months interim) |
| Reagent Freeze-Thaw Stability | 100% Agreement | 100% (up to 15 cycles) |
| Analytical Sensitivity (5-80 ng/uL) | 100% Concordance | 100% (40/40 replicates) |
| Analytical Specificity (Interfering Substances) | 100% Agreement with Sequencing | 100% (18/18 for all substances) |
| Pre-Analytical Equivalency (Extraction Methods) | 100% Agreement with Sequencing | 100% (n=40 per method) |
| Instrument Equivalency | 100% Agreement with Sequencing | 100% (78/78 per instrument combination) |
| Method Comparison (Bi-directional Sequencing) | 100% Agreement (Final Run) | 100% (359/359) |
| Method Comparison (Bi-directional Sequencing) | >95% (95% LCB) | 99.45% First Run Agreement (359/361) |
2. Sample Sizes and Data Provenance:
- Inter-laboratory Reproducibility (Study #1):
- Test set sample size: 9 whole blood samples (3 wild type, 3 heterozygous, 3 homozygous mutant). Tested in duplicate over 5 non-consecutive days by 6 operators (2 from each of 3 sites), resulting in 90 samples tested per operator, totaling 540 data points.
- Data provenance: Not explicitly stated, but includes 2 external sites and 1 internal site, suggesting geographically diverse testing within a controlled study. Prospective based on the study design.
- Lot-to-Lot Reproducibility (Study #9):
- Test set sample size: 9 genomic DNA samples (3 wild type, 3 heterozygous, 3 mutant). Tested in quadruplicate using 3 different kit lots, totaling 108 data points.
- Data provenance: Not explicitly stated, but part of analytical performance studies, likely within a controlled laboratory setting.
- Real-Time Stability Study (Study #5):
- Test set sample size: Samples representing all 3 genotypes (including 3 controls and 4 gDNA samples) tested in quadruplicate at each time point.
- Data provenance: Not explicitly stated, likely internal laboratory testing.
- Reagent Freeze-Thaw Stability Study (Study #6):
- Test set sample size: Genomic DNA isolated from cell lines, representing all possible genotypes.
- Data provenance: Not explicitly stated, likely internal laboratory testing.
- Analytical Sensitivity (Study #3):
- Test set sample size: 3 genomic DNA samples (WT, HET, MUT). Each sample diluted to 8 different concentrations and tested in replicates of 40, totaling 960 data points.
- Data provenance: Whole blood collected in potassium EDTA, likely laboratory-controlled study.
- Analytical Specificity (Interfering Substances) (Study #4):
- Test set sample size: 9 whole blood samples of different genotypes (3 WT, 3 HET, 3 MUT).
- Data provenance: Not explicitly stated, likely laboratory-controlled study.
- Pre-Analytical Equivalency Study/Genomic DNA Extraction Reproducibility (Study #7):
- Test set sample size: 30 human whole blood samples and 10 leukocyte-depleted whole blood samples spiked with cell lines (total 40 samples). These were analyzed with 4 different extraction methods, leading to 160 extracted DNA samples tested.
- Data provenance: Human whole blood and spiked cell lines, likely from a controlled laboratory setting.
- Instrument Equivalency (Study #8):
- Test set sample size: 29 human whole blood samples and 10 leukocyte-depleted whole blood samples spiked with cell lines (total 39 samples). Tested across 3 thermal cyclers and 3 fluorometers (39*3 = 117 tests per fluorometer in Table 9, but recorded as 78 of 78 = 100% for each cell, implying 78 distinct samples or tests per fluorometer/thermal cycler combination).
- Data provenance: Human whole blood and spiked cell lines, likely from a controlled laboratory setting.
- Method Comparison (Bi-directional Sequencing) (Study #2):
- Test set sample size: 361 human whole blood samples.
- Data provenance: Human whole blood samples. Not specified if retrospective or prospective or country of origin, but generally such studies involve controlled collection.
3. Number of Experts and Qualifications for Ground Truth:
The document does not explicitly mention the number or qualifications of experts used to establish the ground truth. However, the ground truth for all performance studies is established by bi-directional DNA sequencing, which is a widely accepted gold standard for determining genetic variants. This method is inherently objective and does not rely on expert interpretation in the same way as, for example, image-based diagnostics.
4. Adjudication Method:
Given that the ground truth is established by bi-directional DNA sequencing, which is a definitive molecular method, there is no mention of an adjudication method in the context of human interpretation (e.g., 2+1, 3+1). The testing process involves comparing the device's output to the objective sequencing results. In cases of discrepancies or invalid results (e.g., in Study #2), these are noted and factored into the agreement calculations.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:
No, a multi-reader multi-case (MRMC) comparative effectiveness study was not done. The device is an in vitro diagnostic test for genetic analysis, not an imaging device or one requiring human interpretation of complex visual data. Therefore, the concept of "human readers improve with AI vs without AI assistance" is not applicable here.
6. Standalone Performance:
Yes, a standalone performance study (algorithm only without human-in-the-loop performance) was done. All the analytical performance studies (precision, stability, sensitivity, specificity, equivalency, and method comparison against sequencing) represent the standalone performance of the Invader® MTHFR 677 test, as it directly outputs genotype calls from fluorescence data without human interpretation influencing the result. The Invader Call Reporter™ software performs the data analysis and determines results autonomously following the import of fluorescence data.
7. Type of Ground Truth Used:
The primary type of ground truth used across all studies is bi-directional DNA sequencing. This is explicitly stated for the method comparison study and implied as the reference method for verifying all genotype calls throughout the analytical performance sections (e.g., "percent agreement between Invader® MTHFR 677 test and sequencing").
8. Sample Size for the Training Set:
The document does not specify the sample size used for a "training set." This device is based on a well-established molecular biology technique (Invader Plus® chemistry, PCR, FRET) and software for data interpretation, rather than a machine learning or AI model that typically requires a distinct training phase with large datasets. The controls included with the kit (Wild Type, Heterozygous, Mutant, No DNA Control) serve to ensure proper functioning and calibration for each run, effectively acting as internal references for the test.
9. How the Ground Truth for the Training Set Was Established:
Since there is no explicit mention of a "training set" in the context of machine learning, the question of how its ground truth was established is not directly applicable. The performance is validated against the gold standard of bi-directional DNA sequencing, which inherently provides the "ground truth" for the MTHFR 677 genotype. The development and optimization of the assay itself would have involved molecular biology techniques to ensure accurate detection of the specific SNP.
§ 864.7280 Factor V Leiden DNA mutation detection systems.
(a)
Identification. Factor V Leiden deoxyribonucleic acid (DNA) mutation detection systems are devices that consist of different reagents and instruments which include polymerase chain reaction (PCR) primers, hybridization matrices, thermal cyclers, imagers, and software packages. The detection of the Factor V Leiden mutation aids in the diagnosis of patients with suspected thrombophilia.(b)
Classification. Class II (special controls). The special control is FDA's guidance entitled “Class II Special Controls Guidance Document: Factor V Leiden DNA Mutation Detection Systems.” (See § 864.1(d) for the availability of this guidance document.)