Search Results
Found 6 results
510(k) Data Aggregation
(328 days)
ILLUMINA, INC.
The Illumina MiSeqDx Cystic Fibrosis 139-Variant Assay is a qualitative in vitro diagnostic system used to simultaneously detect 139 clinically relevant cystic fibrosis disease-causing mutations and variants of the cystic fibrosis transmembrance regulator (CFTR) gene in genomic DNA isolated from human peripheral whole blood specimens. The variants include those recommended in 2004 by the American College of Medical Genetics (ACMG) and in 2011 by the American College of Obstetricians and Gynecologists (ACOG). The test is intended for carrier screening in adults of reproductive age, in confirmatory diagnostic testing of newborns and children, and as an initial test to aid in the diagnosis of individuals with suspected cystic fibrosis. The results of this test are intended to be interpreted by a board-certified clinical molecular geneticist or equivalent and should be used in conjunction with other available laboratory and clinical information. This test is not indicated for use for newborn screening, fetal diagnostic testing, pre-implantation testing, or for stand-alone diagnostic purposes.
The test is intended to be used on the Illumina MiSeqDxTM Instrument.
The Illumina MiSeqDx Cystic Fibrosis 139-Variant Assay consists of library preparation and sample indexing reagents, sequencing reagents and consumables, MiSeqDx instrument and data analysis software. Testing begins with genomic DNA from a peripheral whole blood sample. The genomic DNA is processed through the library preparation steps, which specifically amplifies the intended genomic regions of each sample while also adding the indexes for sample identification. Flow cell capture sequences are also added to the amplified products. The resulting sample libraries are then transferred into a MiSeqDx reagent cartridge which contains all of the reagents required for cluster generation and sequencing (Sequencing By Synthesis). The MiSeqDx Cartridge, MiSeqDx Flow Cell, and MiSeqDx SBS Solution (PR2) are then inserted into the MiSeqDx instrument, which performs cluster generation, sequencing and data analysis.
Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:
Acceptance Criteria and Device Performance
Measure | Acceptance Criteria (Not explicitly stated as criteria, but implied by 100% and >99.99% results in accuracy study and high metrics in reproducibility) | Reported Device Performance (Accuracy Study) | Reported Device Performance (Reproducibility Study) |
---|---|---|---|
Genotype-level PA | High (implied 100% or near 100%) | 100% for all variants | 99.77% |
Negative Agreement (NA) | High (implied >99%) | >99.99% for all wild types | 99.88% |
Overall Agreement (OA) | High (implied >99%) | >99.99% for all reported positions | 99.88% |
Sample Pass Rate | Not explicitly stated for accuracy study, but for reproducibility study, samples passing QC on first attempt | N/A for accuracy study (all tested samples were successfully genotyped) | 99.9% (number of samples passing QC metrics on the first attempt) |
Accuracy (PolyTG/PolyT) | High (implied 100% in most cases, with explanations for deviations) | Varies by genotype, generally 100%, with exceptions explained (e.g., 50.0% for one less common genotype, 90.9% for another) | N/A for this specific sub-analysis in the reproducibility section, but overall reproducibility metrics apply. |
Accuracy (Interfering Substances) | 100% call rate and 100% reproducibility | 100% call rate and 100% reproducibility in genotype calls | N/A (tested in interference study) |
Call Rate (DNA Extraction) | 100% | 100% | N/A (tested in extraction study) |
Accuracy (DNA Extraction) | 100% | 100% | N/A (tested in extraction study) |
Sample First Pass Rate (DNA Extraction) | 100% | 100% | N/A (tested in extraction study) |
Accuracy (DNA Input) | 100% | 100% | N/A (tested in DNA input study) |
Sample First Pass Rate (DNA Input) | >95% for upper and lower bounds; 100% for specific tested input levels | >95% for 1250 ng and 25 ng; 100% for 1250 ng, 250 ng, 100 ng, and 25 ng samples | N/A (tested in DNA input study) |
Reproducibility & Accuracy (Sample Indexing) | 100% | 100% | 100% (for sample/index primer combinations) |
Note regarding "Acceptance Criteria": The document K124006 primarily presents the results of validation studies rather than explicitly stating pre-defined "acceptance criteria" in a separate section. The "acceptance criteria" implied above are derived from the consistently high performance reported across various studies and the overall conclusion of the device being substantially equivalent.
Study Details
2. Sample Sizes Used for the Test Set and Data Provenance
- Accuracy Study (Primary Source):
- Test Set Size: 500 samples in total, comprised of:
- 366 samples from a clinical accuracy study (primary source).
- 68 cell line samples from the reproducibility study (supplementary).
- 14 clinical samples from the extraction method evaluation analytical study (supplementary).
- 52 synthetic plasmid samples (supplementary).
- Data Provenance:
- Clinical Samples: Majority (n=355) were archived, anonymized clinical gDNA specimens isolated from human blood. (No specific country of origin mentioned, but implied to be human clinical samples).
- Cell Line Samples: 11 samples were commercially available cell line specimens.
- Synthetic Plasmid Samples: Designed to include genomic context of rare variants, linearized, diluted, and blended with human genomic DNA (lab-generated, not from patients).
- Retrospective/Prospective: The 355 clinical gDNA specimens were "archived, anonymized clinical gDNA specimens," suggesting a retrospective approach for these samples. The cell line and synthetic samples were laboratory-generated.
- Test Set Size: 500 samples in total, comprised of:
- Reproducibility Study:
- Test Set Size: 46 samples (in each of two panels, total 92 unique samples) were tested. Each sample was tested for a total of 810 calls per site.
- Data Provenance: Genomic DNA from lymphoblastoid cell lines with known mutations and leukocyte-depleted blood spiked with lymphoblastoid cell lines with known mutations. This suggests laboratory-controlled samples, not directly from clinical patients in a prospective manner.
- DNA Extraction Study:
- Test Set Size: 14 unique blood samples per extraction method, with 2 operators performing 3 runs each, and each run having 2 replicates, totaling 168 (14 x 2 x 3 x 2) for each of the three extraction methods.
- Data Provenance: K2EDTA anti-coagulated whole blood samples, including wild type and three mutant genotypes (samples with F508del, I506V, and D110H).
- DNA Input Study:
- Test Set Size: 14 representative DNA samples (with 16 unique CF variants) tested in duplicate at 9 DNA input levels. Additional testing involved 4 representative samples with 20 replicates each (n=80), and 14 samples with 20 replicates each (n=280).
- Data Provenance: Representative DNA samples containing CF variants.
- Interfering Substances Study:
- Test Set Size: 8 whole blood specimens (including 3 CF positive samples with unique genotypes). 16 replicates each for specific interferents.
- Data Provenance: Whole blood specimens.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The document specifies the reference methods used to establish ground truth, but does not explicitly state the number or qualifications of experts involved in reviewing these reference results or establishing consensus.
- For 137 SNA/small InDel sites and PolyTG/Poly T region: Sanger bi-directional sequence analysis was the reference method.
- For two large deletions: A PCR-based assay was the reference method, confirmed for accuracy using Sanger Sequencing.
While Sanger sequencing and PCR assays are considered robust reference methods, the process of interpreting and confirming these results as "ground truth" and addressing any discrepancies would typically involve qualified molecular geneticists or lab personnel. However, this level of detail is not provided.
4. Adjudication Method for the Test Set
The document does not describe a formal adjudication method (e.g., 2+1, 3+1, none) for the test set. Ground truth was established by reference methods (Sanger bi-directional sequencing and a PCR-based assay). Any "miscalls" (one in the accuracy study and a few in the reproducibility study) or "no calls" are noted and sometimes explained (e.g., insufficient coverage, sample handling issues), implying internal review rather than a multi-expert adjudication process for primary ground truth establishment.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No Multi-Reader Multi-Case (MRMC) comparative effectiveness study was done. This device is an in vitro diagnostic system for detecting genetic mutations, not an imaging device or a diagnostic aid meant to directly assist human readers in interpreting complex visual data. Therefore, the concept of "human readers improve with AI vs. without AI assistance" is not applicable in this context.
6. Standalone (Algorithm Only Without Human-in-the-Loop Performance)
Yes, a standalone performance assessment was conducted. The accuracy and reproducibility studies directly evaluated the device's ability (including its data analysis software) to detect mutations and variants in genomic DNA. The "results of this test are intended to be interpreted by a board-certified clinical molecular geneticist or equivalent and should be used in conjunction with other available laboratory and clinical information," indicating that while human interpretation is part of the overall clinical process, the performance metrics (PA, NA, OA) are for the device (algorithm + instrument) alone.
7. Type of Ground Truth Used
The ground truth used was primarily based on:
- Sanger bi-directional sequence analysis: For 137 SNA/small InDel sites and the PolyTG/Poly T region. This is considered a gold standard for sequence verification.
- PCR-based assay: For two large deletions, confirmed by Sanger Sequencing.
This represents established molecular diagnostic laboratory methods, widely accepted as reliable for genetic variant detection.
8. Sample Size for the Training Set
The document does not report the sample size for a training set. This is typical for in vitro diagnostic devices based on established sequencing technology and a defined set of targets, where the "training" may involve internal development and optimization using design verification samples rather than a formally described, distinct "training set" in the context of machine learning model development. The focus is on validation against known samples to determine performance characteristics.
9. How the Ground Truth for the Training Set Was Established
As no explicit "training set" is described, how its ground truth might have been established is not mentioned in the document. Development and optimization would have relied on samples with known CFTR genotypes, likely confirmed by methods similar to those used for the test set (Sanger sequencing, PCR).
Ask a specific question about this device
(57 days)
ILLUMINA, INC.
The MiSeqDx Platform is a sequencing instrument that measures fluorescence signals of labeled nucleotides through the use of instrument specific reagents and flow cells (MiSeqDx Universal Kit 1.0), imaging hardware, and data analysis software. The MiSeqDx Platform is intended for targeted sequencing of human genomic DNA from peripheral whole blood samples. The MiSeqDx Platform is not intended for whole genome or de novo sequencing.
The MiSeqDx Platform is a high throughput DNA sequence analyzer for clinical use.
The MiSeqDx Platform consists of the MiSeqDx instrument and data analysis software. It is for use with the MiSeqDx Universal Kit 1.0 [MiSeqDx reagent cartridge, MiSeqDx flow cell, SBS Solution (PR2 buffer)] for library preparation and sample indexing (K133136). The end-user inputs extracted genomic DNA to be sequenced and provides the Analyte Specific Reagents (ASRs) to develop a sequencing assay that targets their sequence of interest.
The Illumina MiSeqDx Platform is a high-throughput DNA sequencing instrument. The acceptance criteria and supporting studies are detailed below:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria for the MiSeqDx Platform are primarily derived from the special controls stipulated in 21 CFR 862.2265, focusing on accuracy and reproducibility across various genomic features.
Acceptance Criteria (from 21 CFR 862.2265 (B) items ii, iii, vii, viii) | Reported Device Performance (from Accuracy and Reproducibility Studies) |
---|---|
Accuracy: | |
- Ability to detect single nucleotide variants (SNVs). | Study 1: All SNVs had 100% agreement with the reference sequence. PPA ranged from 89.5% to 95.8% (due to missed indels, not SNVs). NPA was 100%. |
Study 2: PPA for SNVs and Indels was 94.1%, NPA was 100%. | |
- Ability to detect insertions and deletions (indels). | Study 1: Variants missed were 1-base insertions or 1-base deletions in homopolymer regions (e.g., Amplicon 9 and 95). |
Study 3 (CFTR Assay): 1-base insertion, 3-base deletion, and 2-base deletion were detected with 100% correct calls. | |
Overall: Validated for detection of SNVs and up to 3-base deletions. Evaluation of 1-base insertions was limited to 3 different insertions on 3 separate chromosomes. The system has problems detecting 1-base insertions or deletions in homopolymer tracts. 2 out of 3 1-base insertions tested were called correctly (those in non-homopolymer regions). 3 out of 4 1-base deletions called correctly (those in non-homopolymer regions). | |
- Performance across varying sequence context (e.g., GC-rich regions, homopolymer runs, different chromosomes). | Study 1 & 3 summary: |
- GC content > 19% and
Ask a specific question about this device
(46 days)
ILLUMINA, INC.
The MiSeqDx Universal Kit 1.0 is a set of reagents and consumables used in the processing of human genomic DNA samples derived from peripheral whole blood, and in the subsequent targeted re-sequencing of the resulting sample libraries. User-supplied analyte specific reagents are required for the preparation of libraries targeting specific genomic regions of interest. The MiSeqDx Universal Kit 1.0 is intended for use with the MiSeqDx instrument.
The MiSeqDx Universal Kit 1.0 is intended for use with the MiSeqDx Platform. The MiSeqDx Universal Kit 1.0 consists of 5 boxes.
This document describes the acceptance criteria and the study proving the MiSeqDx Universal Kit 1.0 meets those criteria, based on the provided text.
1. Table of Acceptance Criteria and Reported Device Performance
The provided text focuses on the classification and intended use of the MiSeqDx Universal Kit 1.0 as a component for high-throughput genomic sequencing. While specific numerical acceptance criteria for all performance characteristics are not explicitly detailed in the provided excerpt (many refer to an associated 510(k) K123989), the text does outline some critical operational specifications and general requirements.
Acceptance Criteria (from text) | Reported Device Performance (from text) |
---|---|
Operational Specifications: | |
Sequencing output > 1 Gb | The system is designed to deliver this. |
Reads > 3 million | The system is designed to deliver this. |
Read length (in paired end run) 2 x 150 bp | The system is designed to deliver this. |
Bases higher than Q30 > 75% (Greater than 75% of bases have Phred scale quality score greater than 30, indicating base call accuracy > 99.9%) | The system is designed to deliver this. |
Detection of SNVs and up to 3 base deletions | The system has been validated for this. |
Qualitative (genotype) results | The system is designed to deliver this. |
Recommended minimal coverage per amplicon for accurate variant calling (Q(max_gt | poly_site) >= 100) is 75x | This is a recommended use condition. |
Limitations/Known Performance: | |
Problems detecting 1 base insertions or deletions in homopolymer tracts (e.g., polyA) | This is a known limitation. |
Evaluation of 1 base insertions limited to 3 different insertions on 3 separate chromosomes | This states the extent of validation. |
Underlying polymorphisms or mutations in oligonucleotide-binding regions can affect calls | This is an inherent characteristic of hybridization-based workflows. |
Stability: | |
Shelf life of 12 months when stored at appropriate conditions | Real-time stability testing for 6 months supports this; ongoing for 12 months. |
2. Sample Size Used for the Test Set and Data Provenance
The provided text repeatedly refers to an associated 510(k) submission, K123989, for detailed performance studies. Therefore, the specific sample sizes used for the test set and the data provenance (e.g., country of origin, retrospective/prospective) are not detailed in this document. The studies mentioned in this document are for:
- Precision/Reproducibility
- Stability (related to blood samples)
- Detection Limit
- Analytical Specificity
- Accuracy
All these reference K123989, meaning the details about the specific test sets, sample sizes, and provenance would be found in that separate submission.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The document does not provide this information. The MiSeqDx Universal Kit 1.0 is a reagent kit that facilitates the creation of a library for genomic sequencing. The 'ground truth' for variant calling (SNVs, deletions, insertions) would typically be established based on orthogonal sequencing methods or well-characterized reference materials, rather than expert human interpretation of images or other subjective data. Details relating to the validation of variant calling would reside in K123989.
4. Adjudication Method for the Test Set
This information is not available in the provided text. Given the nature of a reagent kit for genomic sequencing, human adjudication methods like 2+1 or 3+1 are typically not applied to the primary sequencing data itself. The 'ground truth' for genomic variants is usually based on molecular methods.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve With AI vs Without AI Assistance
No MRMC comparative effectiveness study was mentioned or implied. The MiSeqDx Universal Kit 1.0 is a reagent kit used in a genomic sequencing workflow. It does not involve human "readers" in the context of interpreting diagnostic images or output in a way that would necessitate an MRMC study with AI assistance. The device's role is to prepare genetic material for sequencing, not to interpret the sequencing results themselves.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
The device itself is a reagent kit. It is not an algorithm, and therefore the concept of "standalone" algorithm performance as typically applied to AI/software clinical decision support systems does not directly apply here. The performance of the sequencing system (which the kit is a part of) would be evaluated in terms of its ability to accurately detect genetic variants, which is an "algorithm only" type of assessment in terms of sequence analysis (without human interpretation of the raw data). However, the specific details of this would be in K123989.
7. The Type of Ground Truth Used (expert consensus, pathology, outcomes data, etc.)
The document refers to studies establishing the "accuracy of high throughput DNA targeted sequencing of human genomic DNA" in K123989. For genomic sequencing, ground truth is typically established using:
- Orthogonal Sequencing Methods: Such as Sanger sequencing, whole genome sequencing (WGS), or other highly reliable sequencing technologies on the same samples.
- Reference Materials: Certified reference materials with known genetic variants.
- Synthetic DNA Constructs: With precisely known mutations.
The specific type of ground truth used would be detailed in K123989.
8. The Sample Size for the Training Set
The provided text does not contain any information regarding a "training set." The MiSeqDx Universal Kit 1.0 is a reagent kit, not a machine learning algorithm that requires a training set. The performance studies referenced in K123989 would evaluate the analytical performance of the kit and the sequencing platform, not the training of an algorithm.
9. How the Ground Truth for the Training Set Was Established
As no training set is mentioned or applicable to a reagent kit in the context of machine learning, this information is not provided and is irrelevant for this specific device.
Ask a specific question about this device
(77 days)
ILLUMINA, INC.
The Illumina MiSeqDx(TM) Cystic Fibrosis Clinical Sequencing Assay is a targeted sequencing in vitro diagnostic system that re-sequences the protein coding regions and intrones of the Cystic Fibrosis Transmembrane Conductance Regulator (CFTR) gene in genomic DNA isolated from human periphers collected in K2EDTA. The test detects single nucleotide variants, and small InDels within the region sequenced, and additionally reports on two deep intronic mutations and two large deletions. The test is intended to be used on the Illumina MiSeqDx Instrument.
The test is intended to be used as an aid in the diagnosis of individuals with suspected cystic fibrosis (CF). The test is most appropriate when the patient has an atypical or non-classic presentation of CF or when other mutation panels have failed to identify both causative mutations. The results of the test are intented by a board-certified clinical molecular geneticist or equivalent and should be used in conjunction with other available information including clinical symptoms, other diagnostic tess, and family history. This test is not indicated for use for stand-alone diagnostic testing, for pre-implantation testing, carrier screening, newborn screening, or population screening.
The Illumina MiSeqDx Cystic Fibrosis Clinical Sequencing Assay consists of library preparation and sample indexing reagents, sequencing reagents and consumables, MiSeqDx instrument and data analysis software. Testing begins with genomic DNA from a peripheral whole blood sample. The genomic DNA is processed through the library preparation steps, which specifically amplifies the intended genomic regions of each sample while also adding the indexes for sample identification. Flow cell capture sequences are also added to the amplified products. The resulting sample libraries are then transferred into a MiSeqDx reagent cartridge which contains all of the reagents required for cluster generation and sequencing (Sequencing By Synthesis). The MiSeqDx Cartridge, MiSeqDx Flow Cell, and MiSeqDx SBS Solution (PR2) are then inserted into the MiSeqDx instrument, which performs cluster generation, sequencing and data analysis.
Here's a summary of the acceptance criteria and study details for the Illumina MiSeqDx™ Cystic Fibrosis Clinical Sequencing Assay based on the provided text:
Acceptance Criteria and Device Performance
Acceptance Criteria | Reported Device Performance |
---|---|
Accuracy (Overall Agreement) | >99.99% |
Positive Agreement (PA) for Variants (excluding PolyTG/PolyT) | 100% |
Positive Agreement (PA) for Variants (including PolyTG/PolyT) | 99.66% |
Positive Agreement (PA) for PolyTG/PolyT variants alone | 98.44% |
Negative Agreement (NA) for Wild Type positions | >99.99% |
Reproducibility (Overall Agreement, All Sites) | 99.70% |
Reproducibility (PA) for Variants (excluding PolyTG/PolyT) | 99.60% |
Reproducibility (PA) for Variants (including PolyTG/PolyT) | 99.22% |
Reproducibility (PA) for PolyTG/PolyT variants alone | 97.83% |
Reproducibility (NA) for Wild Type | 99.70% |
DNA Extraction (Call Rate & Accuracy) | >99.99% (for all 3 methods tested) |
DNA Extraction (Sample First Pass Rate) | 100% (for all 3 methods tested) |
DNA Input (Accuracy & Call Rate) | 100% (for 25 ng to 1250 ng range) |
Interfering Substances (Call Rate & Reproducibility) | 100% |
Study Details
2. Sample Size Used for the Test Set and Data Provenance
-
Accuracy Study Test Set:
- Total Samples: 500
- Clinical Samples: 355 (archived, anonymized gDNA from human blood) - Country of origin not specified, but the review is by the US FDA.
- Commercial Cell Line Samples: 11 (from clinical accuracy study) + 68 (from reproducibility study) = 79
- Clinical Samples (from extraction method evaluation): 14
- Synthetic Plasmid Samples: 52
- Data Provenance: Retrospective for archived clinical samples, prospective for cell line and synthetic samples generated for the study.
-
Reproducibility Study Test Set:
- Total Samples: 46 distinct samples (implied from the HGVS tables, with many tested in multiple replicates). The tables show analysis for 68 samples, each tested across 3 sites, with 2 operators each doing 3 runs per site, and with some samples having multiple replicates within those runs. This results in a large number of individual tests. The text explicitly states 68 cell line samples were evaluated in the reproducibility study for accuracy data.
- Data Provenance: Prospective (generated specifically for the study). The operator details suggest this was done as part of the validation study.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
- The ground truth reference methods were Sanger bi-directional sequencing and validated PCR based assays.
- The text does not specify the number of experts used to establish the ground truth for Sanger sequencing or the PCR assays, nor their specific qualifications. It only states these were "reference methods."
4. Adjudication Method for the Test Set
- The text mentions that "All results are based on initial testing. No repeat testing was done for this study" for the accuracy study. For reproducibility, some samples had "no calls" or "miscalls" initially, and further investigation was done (e.g., misidentified variant, low level contamination, improperly prepared synthetic specimen, switched samples). However, a formal adjudication method (e.g., 2+1, 3+1) involving a specific number of experts to resolve discrepancies against the reference is not explicitly described. The determination of miscalls and no calls seems to stem from comparison with the reference methods.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done
- No, an MRMC comparative effectiveness study involving human readers with and without AI assistance was not done. This study focuses on the standalone analytical performance (accuracy, reproducibility) of the device (an in vitro diagnostic system for genetic sequencing), not on its impact on human reader performance.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
- Yes, a standalone performance study was done. The entire accuracy and reproducibility analysis describes the performance of the Illumina MiSeqDx Cystic Fibrosis Clinical Sequencing Assay (which includes the instrument and data analysis software) in detecting genetic variants. The results (PA, NA, OA, call rates) are indicative of the algorithm's performance compared to established reference methods (Sanger sequencing and PCR assays). The "human-in-the-loop" aspect for interpretation is mentioned in the intended use, where results are to be interpreted by a board-certified clinical molecular geneticist, but the performance metrics provided are for the device's ability to 'call' the variants.
7. The Type of Ground Truth Used
- Expert Consensus (implied via Gold Standard Methods): The primary ground truth for SNVs and small InDels was Sanger bi-directional sequencing. For large deletions, two validated PCR-based assays were used, with accuracy confirmed by Sanger Sequencing. These are considered gold-standard methods in molecular diagnostics, implying an expert-established and validated ground truth.
8. The Sample Size for the Training Set
- The document does not specify a training set size. As this is a performance validation document for a sequencing assay, it describes the testing of the device rather than the development or training of an AI model in the conventional sense. The "MiSeqDx instrument and data analysis software" are components, but details on their development/training are not provided in this regulatory summary.
9. How the Ground Truth for the Training Set Was Established
- Since a training set is not explicitly mentioned or detailed for the device's software/algorithm development, the method for establishing its ground truth is not provided in this document. The focus is on validating the final product's performance against reference standards.
Ask a specific question about this device
(208 days)
ILLUMINA, INC.
The VeraCode Genotyping Test for Factor V and Factor II is an in vitro diagnostic device for the detection and genotyping of Factor V Leiden G1691A and Factor II (Prothrombin) G20210A point mutations in DNA obtained from EDTA-anticoagulated human blood samples. The test is intended for use on the BeadXpress System. The VeraCode Genotyping Test for Factor V and Factor II on the BeadXpress System is indicated for use as an aid to diagnosis in the evaluation of patients with suspected thrombophilia.
The VeraCode Genotyping Test for Factor V and Factor II assay consists of reagents sufficient for 96 tests, consisting of two boxes containing pre-PCR and post-PCR reagents. The pre-PCR box contains the following reagents: MTR1 (1 x 1.2 mL). AB1 (1 x 4 mL), AOP1 (1 x 4.8 mL), ELM (1 x 4.8 mL), FSB (1 x 4.8mL), UB3 buffer (2 x 4.8 mL) and AE1 reagent (2 x 4.8 mL). The post-PCR box contains MSS reagent (1 x 4.8 mL) and Fast Start Taq DNA Polymerase (1 x 60 µL), VW2 buffer (1 x 60 mL), a VeraCode FV/FII Bead Plate with holographically inscribed glass microbeads aliquoted in strip-well plates, a test-specific kit manifest file and sample sheet files (containing test specific outcome specifications and sample plate layout files used to interpret and report genotype results). A magnet plate is also required but sold separately.
Here's a breakdown of the acceptance criteria and the study details for the Illumina VeraCode® Genotyping Test for Factor V and Factor II, based on the provided 510(k) summary:
1. Table of Acceptance Criteria and Reported Device Performance
The device's performance is primarily evaluated through reproducibility (precision) and accuracy (method comparison).
Acceptance Criteria for Reproducibility (Precision):
While explicit numerical acceptance criteria for reproducibility are not stated as "acceptance criteria," the study aimed to demonstrate high agreement for correct calls across different operators, sites, and days. The results presented in the summary show very high percentage agreements and 95% lower confidence bounds (LCB) nearing 100% after retesting, indicating that the implicit acceptance criterion was near-perfect agreement for calling genotypes correctly.
Test Parameter | Acceptance Criteria (Implicit) | Reported Device Performance (After Retest) |
---|---|---|
Reproducibility | High percentage agreement (>95%) and 95% LCB for correct genotype calls across sites, operators, and days. | Factor V: Wild Type : 100% agreement (99.38% LCB); Heterozygous : 99.58% agreement (98.04% LCB); Homozygous : 100% agreement (98.35% LCB). All initially "no calls" (failures) were resolved upon re-test. |
Factor II: Wild Type : 100% agreement (99.45% LCB); Heterozygous : 100% agreement (98.35% LCB); Homozygous : 100% agreement (98.35% LCB). All initially "no calls" (failures) were resolved upon re-test. | ||
Accuracy | High overall percent agreement (>95%) and 95% LCB compared to bi-directional DNA sequencing for genotype calls. | Factor V: Wild Type : 100% agreement (98.6% LCB); Heterozygous : 100% agreement (94.07% LCB); Homozygous : 92.86% agreement (70.33% LCB) with one of the 14 samples still being a "no call" after repeat testing. |
Factor II: Wild Type : 99.59% agreement (98.08% LCB); Heterozygous : 100% agreement (88.27% LCB); Homozygous : 100% agreement (65.18% LCB). |
Note: The 95% LCB for homozygous Factor V (70.33%) and Factor II (65.18%) in the accuracy study, while lower than other categories, likely reflect the smaller sample size for these less common genotypes.
2. Sample Size and Data Provenance for the Test Set
- Test Set Sample Size:
- Reproducibility (Precision) Study: A panel of 15 genomic DNA samples (3 for each of the 5 possible genotypes including Wild Type). Each sample was tested in duplicate once a day for 5 non-consecutive days at each of three sites by two operators. This results in a significant number of replicates (e.g., for Factor V Wild Type, 3 samples * 2 replicates * 5 days * 3 sites * 2 operators = 180 total instances, with slight variations in the tables indicating initial no-calls).
- Accuracy (Method Comparison) Study: 92 patient samples were accrued at each of the three sites, yielding a total of 276 patient samples. Additionally, each site received two archived DNA samples as positive controls.
- Data Provenance: The report does not explicitly state the country of origin. However, the samples for the accuracy study were "prospectively drawn blood samples" (for interference testing) and "pre-selected or unscreened patients undergoing Factor V testing" (for method comparison), suggesting a combination of prospective and retrospective (archived) samples for the accuracy study. For reproducibility, it uses "genomic DNA samples isolated from blood" and "commercially available cultured cells."
3. Number of Experts and Qualifications for Ground Truth
- Number of Experts: For the accuracy study (method comparison), the ground truth was established by "bi-directional DNA sequencing analysis performed at an independent reference laboratory." The number of individuals/experts involved in this independent sequencing and interpretation is not specified, nor are their specific qualifications (e.g., number of years of experience). However, "bi-directional DNA sequencing" is a well-established and highly accurate method for genotyping, implying expert-level analysis.
4. Adjudication Method for the Test Set
- There appears to be no explicit adjudication method described for resolving discrepancies between the device's results and the ground truth (bi-directional sequencing). The summary reports the raw agreement.
- For the device's own internal "no calls" (samples generating an invalid result), the "no calls" were retested once. This retesting resolved most, but not all, initial "no calls," as seen in the tables (e.g., one FVL Homozygous sample in the accuracy study remained a "no call" after retesting).
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- No MRMC comparative effectiveness study was done. This device is an in vitro diagnostic (IVD) genetic test, which typically has a standalone performance assessment against a gold standard (like sequencing) rather than evaluating human reader performance with or without AI assistance.
6. Standalone Performance Study (Algorithm Only)
- Yes, a standalone performance study was done. The entire analytical performance section (Precision/Reproducibility and Method Comparison) describes the performance of the device (VeraCode Genotyping Test for Factor V and Factor II on the BeadXpress System, with VeraScan software) as a standalone system. The results of the device were directly compared against the established ground truth (bi-directional sequencing), without human intervention in the interpretation process of the device's output. The software interprets and reports the genotype.
7. Type of Ground Truth Used
- The primary ground truth used for both the reproducibility and accuracy studies was bi-directional DNA sequencing. This is considered a highly reliable and definitive method for determining gene mutations.
8. Sample Size for the Training Set
- The document does not provide information on the sample size used for the training set. As a 510(k) submission for a diagnostic test kit based on established molecular biology principles (PCR, allele-specific primer extension, hybridization), the development of the "algorithm" (the VeraScan software and kit manifest file) would typically rely on well-characterized samples and biochemical parameters rather than a large machine learning training set in the modern sense. The "assay-specific kit manifest file" contains the parameters and cutoffs used to interpret and report genotype results, implying these were established during development.
9. How the Ground Truth for the Training Set Was Established
- Similar to the training set sample size, the document does not explicitly detail how ground truth for the training set (development of the kit manifest and software interpretation rules) was established. It's inferable that the development team used well-characterized reference samples (likely with known genotypes confirmed by sequencing or other gold-standard methods) to define the assay parameters, thresholds, and interpretation rules encoded in the kit manifest file that guides the VeraScan software. The "Preliminary passing criteria where all replicates of all genotypes produce the correct result when compared to bi-directional sequencing" mentioned in the detection limit section suggests an iterative process referencing sequencing during development.
Ask a specific question about this device
(208 days)
ILLUMINA, INC.
The BeadXpress® System is an in vitro diagnostic device intended for the simultaneous detection of multiple analytes in a DNA sample utilizing VeraCode holographic microbead technology. The BeadXpress System consists of the BeadXpress Reader and VeraScan software.
The BeadXpress® System is an open platform fluidic microbead reader which includes a dual-color laser detection system that enables optical scanning of multiplexed assays developed using the VeraCode digital microbead technology and VeraScan 2.0 software. The instrument performs a routine set of operating steps: Reader Initialization, Fluidic Initialization, Scanning, Data Consolidation and Flushing. Hardware is contained within a single instrument housing. The system consists of four sub-systems (fluidics, opto-mechanical, motion and electrical) that interact with each other to provide the desired results. The fluidic system consists of the parts that move fluids and beads through the system as well as the key groove plate upon which the beads lay to be scanned. The optomechanical system contains the two lasers and all the optical components that are involved with the optical alignment and delivery of the beams to the beads and collection of the signals generated. This motion system contains all the hardware that is involved with the physical movement of the internal parts of the reader and utilizes four major axes to control the movement and spacial orientation of the specific sub-systems. The electrical system contains all the electronic components that are responsible for control of the individual components and system overall.
The VeraScan software is installed on a PC directly connected to the BeadXpress Reader. It is used for operating the BeadXpress Reader and through the use of software modules, analyzing the scan results and genotype calls.
Here's an analysis of the provided text regarding the acceptance criteria and the study that proves the device meets them:
The provided document describes the Illumina, Inc. BeadXpress Reader System with VeraScan Software (v. 2.0.17.0). However, the document does not contain specific acceptance criteria or the study that directly proves the device meets those criteria for the BeadXpress Reader System itself.
Instead, the document consistently refers to the clearance of a separate assay (K093129), the "VeraCode® Genotyping Test for Factor V and Factor II," for which the BeadXpress System is intended to be used. The performance characteristics of the system (accuracy, precision, linearity, carryover, interfering substances) are stated to have been assessed during the clearance of that assay.
Therefore, I cannot provide the complete requested information for the BeadXpress Reader System itself based on this document alone, as the performance evaluation is explicitly tied to a different, separately cleared assay.
However, I can extract what is implied about the acceptance criteria and the study structure from the provided text, while making it clear that specific numbers are missing.
Implied Acceptance Criteria and Reported Device Performance (Table)
Since the document states that "Accuracy," "Precision/Reproducibility," "Linearity," and "Carryover" were "assessed during the clearance of the assay (K093129)," this implies general expectations for these performance metrics. The specific quantitative acceptance criteria or reported performance values for the BeadXpress system itself are not provided in this document.
Acceptance Criteria (Implied for the System) | Reported Device Performance (Implied, but specific metrics are absent in this document) |
---|---|
Accuracy: The system should accurately detect signals from hybridized probes. | Assessed during the clearance of assay K093129. (Specific values not provided here). |
Precision/Reproducibility: The system should provide consistent and repeatable results. | Assessed during the clearance of assay K093129 with separate instruments at each site. (Specific values not provided here). |
Linearity: The system should exhibit a linear response across its detection range. | Assessed during the clearance of assay K093129. (Specific values not provided here). |
Carryover: The system should minimize carryover between samples. | Assessed during the clearance of assay K093129; good laboratory practices are recommended. (Specific values not provided here). |
Interfering Substances: Performance should not be significantly impacted by common interfering substances. | Assessed during the clearance of assay K093129. (Specific values not provided here). |
Calibration: The calibration utility should ensure the reader functions within normal specifications. | The VeraScan test and calibration utility is performed, and results are measured to ensure the reader is "functioning within normal specifications." |
Detailed Study Information (Based on implications and absence of data)
-
Sample size used for the test set and the data provenance:
- Test Set Sample Size: Not specified in this document for the BeadXpress Reader System itself. The performance assessment refers to "clearance of the assay (K093129)," which would have its own test set.
- Data Provenance: Not specified in this document. Given it's an FDA 510(k) submission, it would likely involve controlled laboratory studies, but specific country of origin or whether it's retrospective/prospective is not mentioned for the system's performance.
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Not applicable/not specified for the BeadXpress Reader System's performance validation in this document. The "ground truth" would be related to the genetic mutations detected by Assay K093129, and the establishment of that ground truth (e.g., through Sanger sequencing or other gold standard methods) is not detailed here.
-
Adjudication method (e.g., 2+1, 3+1, none) for the test set:
- Not applicable/not specified. Clinical adjudication methods are typically for subjective interpretations (e.g., imaging), which is not the primary function of this instrument. For a genotyping system, "ground truth" is typically established by definitive molecular methods, not expert human adjudication of the instrument's output in the same way.
-
If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No. This is an instrument and software system for automated genotyping, not an AI-assisted diagnostic imaging or interpretation tool for human readers. Therefore, an MRMC study related to human reading improvement with AI assistance is not relevant and was not performed.
-
If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- Yes, implicitly. The "VeraScan software" with the "Genotyping (GT) Module" "analyzes scan data to call genotypes." This describes a standalone algorithmic performance. The document states: "Once analysis is completed, the software displays data results and graphical visualizations to help the user interpret run success." The system's primary function is automated detection and calling. The "performance characteristics" (accuracy, precision, etc.) of the system would be assessed in this standalone mode through the assay's validation (K093129).
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc):
- For genotyping, the ground truth would typically be established by a gold-standard molecular method (e.g., Sanger sequencing, another validated genotyping method, or verified reference materials). This document does not explicitly state the ground truth method used for the assay (K093129) to which the system's performance is tied.
-
The sample size for the training set:
- Not specified. The document does not describe a machine learning model developed with a training set for the BeadXpress Reader itself. The "Genotyping (GT) Module" configures run settings and calls genotypes "by using a process flow that associates fluorescence data in each color channel with calling thresholds supplied by the kit manifest." This implies a rule-based or threshold-based algorithm, rather than a machine learning model that undergoes a training phase with a specific dataset.
-
How the ground truth for the training set was established:
- Not applicable, as no machine learning training set is described for the system. The "kit manifest" provides the "calling thresholds," which represent the established parameters for interpreting the fluorescence data for specific assays.
Ask a specific question about this device
Page 1 of 1