(552 days)
The OncoMate™ MSI Dx Analysis System is a qualitative multiplex polymerase chain reaction (PCR) test intended to detect the deletion of mononucleotides in 5 microsatellite loci (BAT-25, NR-21, NR-24 and MONO-27) using matched tumor and normal DNA obtained from formalin fixed, paraffin-embedded (FFPE) colorectal tissue sections. The OncoMate™ MSI Dx Analysis System is for use with the Applied Biosystems® 3500Dx Genetic Analyzer and OncoMate™ MSI Dx Interpretive Software.
The OncoMate™ MSI Dx Analysis System is indicated in patients diagnosed with colorectal cancer (CRC) to detect microsatellite instability (MSI) as an aid in the identification of probable Lynch syndrome to help identify patients that would benefit from additional genetic testing to diagnose Lynch syndrome.
Results from the OncoMate™ MSI Dx Analysis System should be interpreted by healthcare professionals in conjunction with other clinical findings, family history, and other laboratory data.
The clinical performance of this device to guide treatment decision for MSI high patients has not been established.
The OncoMate™ MSI Dx Analysis System assay encompasses a complete workflow for MSI determination, from DNA extraction to data analysis. DNA is extracted from FFPE colorectal tissue samples (normal and tumor from the same patient) using the Maxwell® CSC DNA FFPE Kit and Maxwell® CSC Instrument. Double-stranded DNA (dsDNA) is then quantified using a fluorescence-based dsDNA quantification system of the user's choice. Next, amplification products are generated through multiplex PCR amplification of DNA microsatellite markers using the OncoMate™M MSI Dx Analysis System amplification kit. The PCR products are then mixed with Hi-Di™ Formamide and Size Standard 500 and heat-denatured. The resulting single-stranded DNA fragments are separated by size and detected via fluorescence using an Applied Biosystems® 3500Dx Genetic Analyzer. Following capillary electrophoresis, allele sizes from the CRC tumor DNA and the normal DNA are calculated and compared for each of the microsatellite markers using OncoMate™ MSI Dx Interpretive Software. If the length of two or more of the five mononucleotide-repeat marker alleles is changed by ≥2.75 base pairs (bp), the tumor is classified as MSI-H; if the allele length is changed for only one marker, or if the difference in allele lengths at the five markers is
Here's an analysis of the acceptance criteria and study data provided for the OncoMate™ MSI Dx Analysis System:
Acceptance Criteria and Reported Device Performance for OncoMate™ MSI Dx Analysis System
I. Acceptance Criteria Table:
Unfortunately, the provided text does not explicitly state a predefined set of quantitative acceptance criteria for the clinical performance endpoints (e.g., specific thresholds for PPA, NPA, or OPA). Instead, it presents the results of various analytical studies and then the "Method Comparison Study" against a predicate device and NGS. The conclusion then broadly states that the device is "substantially equivalent" based on these results.
However, we can infer some implicit performance expectations based on the presented results and the comparison to the predicate. The analytical studies demonstrate robust performance and reproducibility, which are foundational for a diagnostic device. The clinical method comparison aims to show comparable performance to the predicate.
Here's a table based on the reported performance in the method comparison study, which serves as the primary evidence for clinical acceptance:
| Acceptance Criteria (Implied / Demonstrated Performance) | Reported Device Performance (OncoMate™ MSI Dx Analysis System) |
|---|---|
| Method Comparison vs. VENTANA MMR IHC Panel (All Samples) | |
| Positive Percent Agreement (PPA) with MMR Loss | 97.8% (44/45; 95% CI: 88.4 – 99.6) |
| Negative Percent Agreement (NPA) with MMR Intact | 97.2% (106/109; 95% CI: 92.2 – 99.1) |
| Overall Percent Agreement (OPA) | 97.4% (150/154; 95% CI: 93.5 – 99.0) |
| Method Comparison vs. NGS Mismatch Repair Gene Mutations Results (All Samples) | |
| Positive Percent Agreement (PPA) with Pathogenic Mutation | 94.4% (17/18; 95% CI: 74.2 – 99.0) |
| Negative Percent Agreement (NPA) with No Pathogenic Mutation | 77.9% (106/136; 95% CI: 70.3 – 84.1) |
| Overall Percent Agreement (OPA) | 79.9% (123/154; 95% CI: 72.8 – 85.4) |
| Analytical Performance (Implicit Acceptance) | |
| DNA Extraction Success Rate | 96.4% concordant with predetermined MSI status (81/84) |
| Resolution of >3bp differences | Demonstrated measurement precision (stdev ≤0.13bp) |
| Limit of Blank (no MSI-H for MSS samples) | 100% MSS results (1440/1440) |
| Limit of Detection (LOD) - DNA Input | 100% MSI-H results at 0.5ng, 1ng, 2ng, 2.5ng DNA input |
| LOD - Tumor Content | Established at 30% tumor content for concordance across loci |
| Analytical Specificity (no interference) | Confirmed (100% agreement between expected and observed MSS calls across thermal cyclers) |
| Interfering Substances (no significant impact) | No statistically significant impact on assay performance |
| Cross-Contamination (no sample-to-sample CC) | 100% concordance with expected results (470 samples) |
| Reproducibility (PPA & NPA) | Overall PPA: 95.5% (92.4-97.6%), Overall NPA: 99.5% (97.4-100%) |
II. Sample Size and Data Provenance for the Test Set (Clinical Method Comparison):
- Sample Size: A total of 154 cases were used in the primary clinical method comparison study.
- Data Provenance: The samples consisted of Formalin-Fixed, Paraffin-Embedded (FFPE) colorectal cancer patient samples.
- The study design involved a "sequential series" of CRC patient samples enriched with a second set of suspected Lynch syndrome samples. This implies a mixture of prospective (sequential series) and potentially retrospective (enriched set sourced from a registry/archive of suspected Lynch cases) samples, although specific dates or collection methods aren't detailed.
- No specific country of origin is mentioned for the patient samples, but the context of an FDA submission for a US market suggests they were likely representative of a US population or a population broadly applicable to US clinical practice.
III. Number of Experts and Qualifications for Ground Truth Establishment (Clinical Method Comparison):
The ground truth for the clinical method comparison was primarily established by two reference methods:
- VENTANA MMR IHC Panel: This is a commercially available immunohistochemistry test, and its interpretation would typically be performed by qualified pathologists. The document states, "Immunohistochemistry was performed on all 154 tumor samples to determine protein expression... per the VENTANA instructions for use and the laboratory's Standard Operating Procedures." While the number of pathologists or their specific years of experience are not explicitly stated, the reference to SOPs suggests adherence to standard professional practice.
- Next Generation Sequencing (NGS) for DNA mismatch repair genes: This was performed by a "reference laboratory." The ground truth for Lynch syndrome confirmation was based on "Pathogenic or likely pathogenic mutations listed in the ClinVar database (21)." This implies that the interpretation of NGS results relied on an established public database and likely expert geneticists or molecular pathologists interpreting the findings according to recognized clinical significance criteria (e.g., ACMG guidelines). Again, the number of experts or their specific qualifications for interpreting the NGS results directly from this study is not explicitly stated, but the reliance on ClinVar and "reference laboratory" practices implicitly indicates expert interpretation.
IV. Adjudication Method for the Test Set:
The document does not describe an explicit adjudication method (e.g., 2+1, 3+1 consensus with multiple readers) for establishing the ground truth of the clinical samples within the context of the study.
- For the IHC comparison, the IHC results served as one gold standard, without mention of multiple independent IHC reads and subsequent adjudication if there were discrepancies.
- For the NGS comparison, the detection of pathogenic/likely pathogenic mutations in MMR genes (and BRAF exon 15) as recorded in ClinVar served as the ground truth. This is an objective molecular test result criterion rather than a subjective interpretation requiring adjudication.
V. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:
A traditional MRMC comparative effectiveness study, where human readers interpret cases with and without AI assistance to measure effect size, was not performed or described in this document.
The "OncoMate™ MSI Dx Analysis System" explicitly includes "OncoMate™ MSI Dx Interpretive Software" as part of the system. This implies that the software provides an "automated interpretive result, either MSI-H or MSS" (page 9). Therefore, the device itself is the interpretive system, rather than an AI assistance tool for human readers. The clinical study evaluated the performance of this system as a whole (kit + software) against other diagnostic methods (IHC and NGS).
VI. Standalone (Algorithm Only) Performance:
Yes, a standalone (algorithm only) performance study was performed. The entire analytical and clinical performance sections evaluate the OncoMate™ MSI Dx Analysis System, which includes the OncoMate™ MSI Dx Interpretive Software, as a self-contained diagnostic tool. The software provides an "automated interpretive result" (page 9) without requiring further human interpretation of the raw data. The performance metrics (PPA, NPA, OPA, reproducibility, LOD, etc.) are all measures of this standalone system's ability to classify MSI status.
VII. Type of Ground Truth Used:
The ground truth for the clinical method comparison study was established using two different methods:
- Expert Interpretation of a Predicate Device: The VENTANA MMR IHC Panel results, interpreted to determine MMR Loss or MMR Intact status. This represents an indirect measure of MSI status based on protein expression.
- NGS Mismatch Repair Gene Mutations and ClinVar Database: Direct molecular evidence of pathogenic or likely pathogenic germline mutations in MMR genes (MLH1, MSH2, MSH6, PMS2) and BRAF exon 15, cross-referenced with the ClinVar database. This is a direct genetic confirmation of Lynch syndrome.
Thus, the ground truth is a combination of established diagnostic methods: protein-level immunohistochemistry and germline genetic sequencing data verified against a public expert-curated database.
VIII. Sample Size for the Training Set:
The document does not explicitly describe a separate training set or its sample size for the OncoMate™ MSI Dx Analysis System. Diagnostic devices based on PCR and fragment analysis, like this one, typically rely on analytical validation (precision, accuracy of sizing, LOD, etc.) and clinical validation (method comparison) rather than traditional machine learning "training" sets in the same way an image-based AI algorithm might.
The development and optimization of the interpretive software's logic (e.g., the ≥2.75bp threshold for instability, the rule for MSI-H/MSS classification) would have been informed by extensive research and analytical studies, but these wouldn't be referred to as a "training set" in the machine learning sense. The provided performance studies are primarily about validation of the finalized system.
IX. How the Ground Truth for the Training Set Was Established:
As there is no explicitly stated "training set" in the context of machine learning, the concept of establishing ground truth for it as a separate phase is not detailed. The design of the system, including the software's interpretive rules, would have been based on established scientific principles of microsatellite instability analysis and potentially internal development data. For example, the definition of "marker instability as a 3bp change (implemented as ≥2.75bp to account for the sizing precision of capillary electrophoresis)" (page 9) is a pre-defined rule, not learned from a dataset. The analytical studies (like "Normal Range and Cutoff") verify these inherent rules and measurement capabilities.
§ 864.1866 Lynch syndrome test systems.
(a)
Identification. Lynch syndrome test systems are in vitro diagnostic tests for use with tumor tissue to identify previously diagnosed cancer patients at risk for having Lynch syndrome.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Premarket notification submissions must include the following information, as appropriate:
(i) A detailed description of all test components, including all provided reagents, and required but not provided, ancillary reagents.
(ii) A detailed description of instrumentation and equipment, including illustrations or photographs of non-standard equipment or manuals.
(iii) Detailed documentation of the device software, including, but not limited to, standalone software applications and hardware-based devices that incorporate software.
(iv) A detailed description of quality controls including appropriate positive and negative controls that are recommended or provided.
(v) Detailed specifications for sample collection, processing, and storage.
(vi) A detailed description of methodology and assay procedure.
(vii) A description of the assay cut-off (
i.e., the medical decision point between positive and negative results) or other relevant criteria that distinguishes positive and negative results, or ordinal classes of marker expression, including the rationale for the chosen cut-off or other relevant criteria and results supporting validation of the cut-off.(viii) Detailed specification of the criteria for test result interpretation and reporting.
(ix) Detailed information demonstrating the performance characteristics of the device, including:
(A) Data from an appropriate study demonstrating clinical accuracy using well-characterized clinical specimens representative of the intended use population (
i.e., concordance to Deoxyribonucleic Acid (DNA) sequencing results of the Lynch syndrome associated genes or method comparison to the predicate device using samples with known alterations in genes representative of Lynch syndrome). Pre-specified acceptance criteria must be provided and followed.(B) Appropriate device reproducibility data investigating all sources of variance (
e.g., for distributed tests, data generated using a minimum of three sites, of which at least two sites must be external sites). Each site must perform testing over a minimum of 5 nonconsecutive days evaluating a sample panel that spans the claimed measuring range, and includes the clinical threshold. Pre-specified acceptance criteria must be provided and followed.(C) Data demonstrating reader reproducibility, both within-reader and between-reader, assessed by three readers over 3 nonconsecutive days at each site, including a 2 week washout period between reads, as appropriate.
(D) Device precision data using clinical samples spanning the measuring range and controls to evaluate the within-lot, between-lot, within-run, between run, and total variation.
(E) Analytical specificity studies including as appropriate, western blots, peptide inhibition, testing in normal tissues and neoplastic tissues, interference by endogenous and exogenous substances, and cross-reactivity and cross contamination testing.
(F) Device analytical sensitivity data generated by testing an adequate number of samples from individuals with the target condition such that prevalence of the biomarker in the target population is established.
(G) Device stability data, including real-time stability and in-use stability, and stability evaluating various storage times, temperatures, and freeze-thaw conditions, as appropriate.
(H) The staining performance criteria assessed must include overall staining acceptability, background staining acceptability, and morphology acceptability, as appropriate.
(I) Appropriate training requirements for users, including interpretation manual, as applicable.
(J) Identification of risk mitigation elements used by the device, including a description of all additional procedures, methods, and practices incorporated into the instructions for use that mitigate risks associated with testing.
(2) The device's § 809.10(b) of this chapter compliant labeling must include a detailed description of the protocol, including the information described in paragraphs (b)(1)(i) through (viii) of this section, as appropriate, and a detailed description of the performance studies performed and the summary of the results, including those that relate to paragraph (b)(1)(ix) of this section, as appropriate.