(59 days)
INTROL™ CF Panel I Control is intended for in vitro diagnostic use as a quality control to monitor analytical performance of the extraction, amplification and detection steps of diagnostic assays used in the detection of the Cystic Fibrosis Transmembrane Conductance Regulator (CFTR) gene mutations and variants. This product is intended to be extracted and analyzed routinely with each CFTR assay run.
The INTROL™ CF Panel I Control is designed to monitor the detection of 38 CFTR mutations associated with cystic fibrosis, including the 23 mutations recommended for testing by American College of Medical Genetics (ACMG) and American College of Obstetricians and Gynecologists (ACOG). The INTROLTM CF Panel I Control also monitors 3 polymorphisms (1506V, 1507V, F508C) and the 5/7/9T variants.
INTROL™ CF Panel I Control consists of synthetic CFTR DNA suspended in a matrix of synthetic DNA targets, carrier DNA of a non-human species, preservatives, dye, and stabilizers. The synthetic DNA contains all 27 CFTR gene exons plus intronic borders, and contains specific mutations and polymorphisms which are divided among 5 bottles (bottles a, b, c, d, and e). The 5 bottles exist in two versions: Version G106ac includes bottles (a), (b), and (c), while Version G106de includes bottles (a), (b), (d), and (e). The specific mutations present in each bottle are listed below in Table 1; all other CFTR sequence is wild type. CFTR mutations that are not listed cannot be detected in the INTROL™ CF Panel I Control.
Control sequences include the mutations, wild type alleles, and polymorphisms recommended for testing by the American College of Medical Genetics (ACMG) and American College of Obstetricians and Gynecologists (ACOG). CFTR mutations that are not listed cannot be detected in the INTROL™ CF Panel I Control.
CFTR DNA is stabilized in the matrix and released when processed through common extraction methods as if it were a whole blood specimen. Following extraction, the released DNA can be used in common amplification based molecular assays techniques. Because INTROL™ CF Panel I Control is designed to mimic the whole blood sample, the resulting copy number of the artificial CFTR segment, after extraction, will be similar to that found in a processed human whole blood sample (v/v).
Here's a breakdown of the requested information based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
| Acceptance Criteria (from Product Acceptance Criteria and Testing section) | Reported Device Performance (from Performance Characteristics section) |
|---|---|
| All INTROL™ CF Panel I Control products are tested by an FDA-cleared CFTR mutation detection method before being released for market distribution. | External site evaluations show that across 10 sites and 6 different methods (including Tag-It™ and eSensor, both presumably FDA-cleared), there were 100% correct calls for 1133 calls over 66 runs and 11 lots. This indicates successful detection of mutations. |
| Any mutations not tested by the FDA-cleared method are sequenced bidirectionally before product is released. | "Bidirectional sequencing of INTROL™ CF Panel I DNA is used to validate the presence of mutant or wild type sequence." This confirms the method used for mutations not covered by FDA-cleared assays. |
| All mutations must be detected. | The external site evaluations consistently show 100% correct calls, indicating all intended detectable mutations were detected across various methods and sites. Additionally, for each bottle (a-e), the specific mutations and polymorphisms are designed to be present and are therefore expected to be detected. |
| Unopened INTROL™ CF Panel I Control material is stable through the expiration date printed on each bottle when stored refrigerated (2° - 8°C). | Real-time stability study (up to 12 months) showed 100% correct calls for 5 different lots at various time points (1, 2, 3, 4, and 12 months). |
| Opened material returned to the refrigerator (2° - 8°C) shortly after use is stable for thirty (30) days from the date of opening. | Two open vial stability studies demonstrated no loss of signal when used in CFTR assay at the end of the test period (49 days in one study, 35 days in another), exceeding the 30-day criterion. |
| Product stability during prolonged shipping without refrigeration (elevated temperature). | Elevated temperature studies (simulating shipping and incubation at 60°C for 21-24 days) showed no loss of signal. |
| Stability after one cycle of freeze/thaw. | Freeze/Thaw studies showed no loss of signal after one cycle. |
| The level of synthetic CFTR DNA present in the extracted control should be detectable and mimic whole blood samples as if it were a whole blood specimen (implicitly, through successful extraction and detection). | 96% successful laboratory extractions across 134 laboratories using 21 different methods. The 4% (5 labs) that didn't continue were due to a different quantitation method, not necessarily a failure of extraction itself. This suggests the device successfully mimics whole blood for extraction. |
2. Sample Size Used for the Test Set and Data Provenance
- Test Set Sample Size: In the external site evaluations, samples from 11 different manufacturing lots were tested. The total number of "Calls" (presumably individual tests for specific mutations) was 1133. It's not explicitly stated how many unique samples or cases were used for the test set, but rather the number of lots, runs, and calls.
- Data Provenance: The data was generated through external site evaluations (clinical laboratory study performed at external sites) involving 10 external sites, 8 of which were clinical laboratories representing intended users. The origin country is not specified, but the context of "FDA-cleared" suggests the study was conducted within the United States or under FDA guidelines. The study appears to be prospective as it involves evaluations of the manufactured control materials at various external sites.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The concept of "experts" establishing ground truth in the traditional sense (e.g., radiologists, pathologists) is not directly applicable here.
- For the test set validation (external site evaluations), FDA-cleared CFTR mutation detection methods and bidirectional sequencing were used to determine if the mutations were correctly detected. The "ground truth" for the content of the control material (which mutations and polymorphisms it contains) was established during the manufacturing process by Maine Molecular Quality Controls, Inc. (MMQCI) before product release.
- The control material itself contains predefined synthetic DNA with specific mutations and wild-type sequences (Table 1). The 'truth' of what the control contains is intrinsic to its design and manufacturing.
- The "experts" involved are those within MMQCI responsible for designing and manufacturing the synthetic DNA, and those performing the bidirectional sequencing, which is a highly accurate molecular technique for confirming DNA sequences. Their specific qualifications are not detailed beyond "experts."
4. Adjudication Method for the Test Set
Adjudication methods like "2+1" or "3+1" are typically used when multiple human readers or algorithms provide potentially discordant interpretations, which then need to be resolved. This is not directly applicable to this device.
- The "adjudication" for the performance study (external site evaluations) essentially involved checking if the CFTR assays correctly identified the mutations known to be present in the control material.
- The control materials have a pre-defined composition (Table 1) and are validated internally by FDA-cleared CFTR mutation detection methods and bidirectional sequencing before release. The process is a comparison against this known-good standard. Any discrepancy from the expected result would be a failure. There is no mention of a human review or adjudication of conflicting interpretations from different tests, but rather a direct comparison to the expected truth (the known composition of the control).
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance
- No Multi-Reader Multi-Case (MRMC) comparative effectiveness study was done.
- This device is a quality control material, not an AI-powered diagnostic device, so the concept of "human readers improve with AI vs. without AI assistance" is not relevant here. The studies focused on the analytical performance and stability of the control material itself within various CFTR assays.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
- A standalone performance evaluation was done, but it pertains to the performance of the various CFTR mutation detection assays (which are algorithms or protocols, but not "AI") when using the INTROL™ CF Panel I Control.
- The "Performance Characteristics" section details how the control material performed when tested by these various FDA-cleared and other amplification methods without direct human intervention in the interpretation of the control result itself. The control material is expected to produce a specific, interpretable result (detection of known mutations), and this was assessed at 10 external sites using different methods. The "100% correct calls" across these methods serve as the standalone analytical performance.
7. The Type of Ground Truth Used
- The ground truth used is primarily synthetic DNA with a precisely known sequence and mutation composition (defined at manufacturing) for the control material, validated by bidirectional sequencing and comparison to FDA-cleared CFTR mutation detection methods.
- This is a form of "known composition" or "reference standard" ground truth, rather than expert consensus, pathology, or outcomes data, which are typically found in clinical diagnostic studies.
8. The Sample Size for the Training Set
- Not applicable. This device is a quality control material and does not involve a "training set" in the context of machine learning or AI models. Its intended use is to monitor the performance of other diagnostic assays, not to be a diagnostic algorithm itself.
9. How the Ground Truth for the Training Set Was Established
- Not applicable, as there is no training set. The intrinsic "truth" of the control material is its precisely engineered synthetic DNA sequence designed to contain specific CFTR mutations and polymorphisms. This content is verified during manufacturing using established molecular biology techniques (sequencing and cleared CFTR assays).
§ 866.5910 Quality control material for cystic fibrosis nucleic acid assays.
(a)
Identification. Quality control material for cystic fibrosis nucleic acid assays. A quality control material for cystic fibrosis nucleic acid assays is a device intended to help monitor reliability of a test system by detecting analytical deviations such as those that may arise from reagent or instrument variation in genetic testing. This type of device includes recombinant, synthetic, and cell line-based DNA controls.(b)
Classification. Class II (special controls). The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 866.9. The special control is FDA's guidance document entitled “Class II Special Controls Guidance Document: Quality Control Material for Cystic Fibrosis Nucleic Acid Assays.” See § 866.1(e) for the availability of this guidance document.