Search Results

For in vitro diagnostic use. For use on the Biocartis Idylla™ System only.

The Idylla™ MSI Test, for use on the Idylla™ System, uses formalin-fixed, paraffin-embedded (FFPE) tissue sections of human CRC tumor, from which nucleic acids are liberated, then analyzed using PCR amplification of seven monomorphic biomarkers (ACVR2A, BTBD7, DID01, MRE11, RYR3, SEC31A and SULF2) and subsequent melt-curve analysis. The Idylla™MSI Test reports results as either microsatellite stable (MSS), or microsatellite instability high (MSI-H) or invalid.

Idylla™ MSI Test is indicated for use by healthcare professionals for the qualitative identification of microsatellite instability (MSI) in colorectal cancer (CRC) tumors, indicative of mismatch repair deficiency, as an and in the identification of potential Lynch syndrome to help identify patients that would benefit from additional genetic testing to diagnose Lynch syndrome.

The results from the Idylla™ MSI Test should be interpreted by healthcare professionals in conjunction with other clinical findings, family history, and other laboratory data. The Idyla™ MSI Test should not be used for diagnosis of CRC. The clinical performance of this device to guide treatment decision for MSI high patients has not been established.

The Biocartis Idylla™ System covers the entire process from sample to result with fully integrated sample preparation followed by PCR amplification and high-resolution melting detection of the targeted sequences. The Idylla™ System consists of the Idylla™ Console connected to one or more Idylla™ Instruments (up to eight instruments). Idylla™ Cartridges, designed for specific applications, can be processed by the Idylla System using test specific software (Test Type Package, MSI TTP). The Idylla™ MSI Test procedure and data analysis are validated for FFPE tissue sections.

The Idylla™ MSI Test detects a novel panel of seven monomorphic biomarkers.

The Idylla™ MSI Test Cartridges are ready-for-use and contain the necessary reagents to perform sample preparation, PCR amplification and high-resolution detection, starting from insertion of FFPE tissue sections. The MSI TTP directs the processing of the sample within the cartridge.

The process steps in the Idylla™ MSI Test are:

• FFPE liquefaction and cell lysis: After insertion of the FFPE tissue section into the cartridge, a combination of chemical reagents, enzymes, heat, and High Frequency Ultrasound (HIFU) induces deparaffinization, disruption of the tissue and lysis of the cells. The nucleic acids are liberated for subsequent PCR amplification.
• PCR using biomarker-specific primers: All necessary PCR reagents are present in a stable formulation and are used to amplify seven biomarkers indicative for MSI status.
• Detection and analysis: Detection of these specific targets is performed using fluorescently labeled molecular beacons after PCR amplification. These beacons differentially melt from the wild type or mutated amplicons with increasing temperature. The fluorescence differences at melting temperatures are further analyzed by the MSI TTP and translated into genetic calls on biomarker level and MSI status on sample level.
• Reporting: At the end of the run, the result, reporting the MSI status, the number of mutated biomarkers, and an MSI score range in the analyzed sample is displayed on the console screen.

Here's a breakdown of the acceptance criteria and study details for the Idylla™ MSI Test based on the provided FDA 510(k) summary:

1. Table of Acceptance Criteria & Reported Device Performance

The document doesn't explicitly state "acceptance criteria" as a separate section with numerical targets. Instead, it demonstrates performance through various analytical and clinical studies, showing high concordance and hit rates. Below is a summary of the performance metrics reported, which implicitly serve as the evidence for meeting acceptance for substantial equivalence.

2. Sample Size Used for the Test Set and Data Provenance

• Analytical Studies (LoD, Reproducibility, Interference):

  • LoD Estimation (Contrived): 24 replicates per allelic frequency level (Table 3), various levels (50%, 30%, 20%, 15%, 10%, 5%) for both high and low sample input.
  • LoD Estimation (Clinical): 4 FFPE samples (titration experiments), with 6 replicates per concentration level.
  • LoD Confirmation (Clinical): 7 FFPE clinical samples (MSI-H, MSS, borderline) tested in 10 replicates per lot (n=20) over 5 days on multiple instruments (total n=140 from all seven samples).
  • Reproducibility: 7 individual FFPE clinical specimens (MSI-H and MSS) tested at 3 laboratory sites, with 12 replicates per sample per site (total 36 tests/sample, 252 valid runs across all sites).
  • Interference (Hemoglobin, Triglycerides, Paraffin): 7 FFPE colorectal cancer clinical samples, tested in 5 replicates with each interfering substance.
  • Interference (Mucin): 19 FFPE clinical samples (MSI-H and MSS) with varying mucinous content, tested in 3 replicates each (total 57 runs, 55 valid).
  • Interference (Necrosis): 4 FFPE clinical samples with various necrosis levels, tested in triplicate with and without necrosis.
  • Data Provenance: The document states "clinical samples" and "hospital biobanks", indicating human sample origin. The reference to "Colorectal Cancer Family Registry" for the enrichment cohort suggests these are existing, well-characterized clinical samples. It is retrospective in nature, using archived FFPE tissue sections. Origin country is not explicitly stated but implies multicenter (multiple sites for reproducibility) data.

• Clinical Performance Study:

  • Total samples: 143 samples.
  • Sequential cohort: 123 samples (sequentially selected from two hospital biobanks).
  • Enrichment cohort: 20 confirmed Lynch cases (obtained from the Colorectal Cancer Family Registry).
  • Data Provenance: Retrospective, FFPE tissue sections of human CRC tumor.

3. Number of Experts Used to Establish Ground Truth for the Test Set and Their Qualifications

The document does not explicitly state the number of experts or their specific qualifications for establishing the ground truth for the test set as per a typical adjudication process for imaging or pathology.
The ground truth reference methods are:

• OncoMate™ MSI Dx Analysis System (Predicate Device): This is a legally marketed predicate device, implying its results are accepted as a reference standard.
• Germline NGS for MMR Genes: This is a molecular diagnostic method, considered a gold standard for identifying Lynch syndrome.
• Sample Characterization: For analytical studies, samples were characterized as MSI-H or MSS based on known properties (e.g., cell lines, clinical samples).

Therefore, the "experts" here are essentially the established and validated methods of the predicate device and germline NGS, rather than human expert readers adjudicating images.

4. Adjudication Method for the Test Set

No explicit adjudication method (like 2+1, 3+1) is mentioned or implied for the clinical performance study. The comparison is made directly against established reference methods: the OncoMate™ MSI Dx Analysis System and Germline NGS for MMR genes. These are direct comparisons between the device's output and the reference method's output.

5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study Was Done, and Effect Size of Human Readers Improvement with AI vs. Without AI Assistance

This device, the Idylla™ MSI Test, is an in vitro diagnostic (IVD) test that automatically reports results (MSS, MSI-H, or invalid) based on molecular analysis of tissue samples. It is not an AI-assisted diagnostic imaging device or a software that assists human readers in interpreting clinical data. Therefore, an MRMC comparative effectiveness study involving human readers with and without AI assistance is not applicable to this device. There is no human-in-the-loop component for result generation, so there's no "effect size of how much human readers improve with AI vs without AI assistance" to report.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done

Yes, the studies performed are inherently standalone performance of the Idylla™ MSI Test. The device provides an automated result (MSS, MSI-H, or Invalid) directly from the FFPE tissue sample without intermediate human intervention for interpretation of the molecular signals. The performance metrics (PPA, NPA, etc.) represent the algorithm's standalone accuracy against reference methods.

7. The Type of Ground Truth Used

The ground truth used for the clinical performance study was:

• Predicate Device Output: The results from the OncoMate™ MSI Dx Analysis System.
• Molecular Gold Standard: Germline Next Generation Sequencing (NGS) for DNA mismatch repair (MMR) genes, particularly for identifying Lynch cases.

For analytical studies, the ground truth was also based on well-characterized samples (e.g., cell lines with known allelic frequencies, clinical samples previously characterized as MSI-H or MSS).

8. The Sample Size for the Training Set

The document does not provide information about a separate "training set" or its sample size. This is typical for IVD submissions which validate a fixed algorithm or assay rather than continuously learning AI models. The development and optimization of such a test would involve internal R&D, but the submission focuses on the validation of the finalized product.

9. How the Ground Truth for the Training Set Was Established

As no specific training set is mentioned in the context of an AI/ML model, this question is not directly applicable. For the development of the Idylla™ MSI Test, the "ground truth" for optimizing the assay (e.g., primer design, melt curve analysis parameters) would have been established through a combination of scientific literature, internal studies using characterized cell lines, and reference clinical samples with known MSI status (likely determined by established methods like IHC or other PCR-based MSI tests).

Ask a specific question about this device

The BOND MMR Antibody Panel is intended to be used for the qualitative identification by light microscopy of human mismatch repair (MMR) proteins MLH1, MSH2, MSH6 and PMS2 in formalin-fixed, paraffin-embedded (FFPE) colorectal cancer (CRC) tissue sections by immunohistochemical staining. The BOND MMR Antibody Panel includes BOND Ready-to-Use Primary Antibody MLH1 (Mismatch Repair Protein) (ES05), BOND Ready-to-Use Primary Antibody MSH2 (Mismatch Repair Protein) (79H11), BOND Ready-to-Use Primary Antibody MSH6 (Mismatch Repair Protein) (EP49) and BOND Ready-to-Use Primary Antibody PMS2 (Mismatch Repair Protein) (EP51). The BOND MMR Antibody Panel is intended for use on the BOND-III or BOND-MAX fully automated systems with BOND Polymer Refine Detection.

The BOND MMR Antibody Panel is indicated for the detection of MMR protein deficiency as an aid in the identification of potential hereditary nonpolyposis colorectal cancer (HNPCC)/Lynch Syndrome in patients diagnosed with CRC. Patients with "MMR Loss" results should receive additional diagnostic testing consistent with clinical practice guidelines for diagnosis of Lynch syndrome. The BOND MMR Antibody Panel is not intended for use in indications other than CRC. This test should not be used for diagnosis of CRC.

The clinical interpretation of any staining or its absence when using the BOND MMR Antibody Panel should be complemented by morphological studies and proper controls and should be evaluated within the context of the patient's clinical history and other diagnostic tests by a qualified pathologist.

The clinical performance of this device to guide treatment of MMR deficient patients has not been established.

The BOND MMR Antibody Panel [subject device] consists of the following BOND Ready-to-Use (RTU) Primary Antibody (PA) products:

• MLH1 (Mismatch Repair Protein) (ES05) (PA0988-U) .
• MSH2 (Mismatch Repair Protein) (79H11) (PA0989-U) .
• . MSH6 (Mismatch Repair Protein) (EP49) (PA0990-U)
• . PMS2 (Mismatch Repair Protein) (EP51) (PA0991-U)

The BOND MMR Antibody Panel is intended for use on the BOND-III or BOND-MAX fully automated systems with BOND Polymer Refine Detection (DS9800). The BOND MMR Antibody Panel is indicated for the detection of mismatch repair protein deficiency as an aid in the identification of potential Hereditary Non-Polyposis Colorectal Cancer (HPNCC)/Lynch Syndrome in patients diagnosed with CRC.

MLH1 (Mismatch Repair Protein) (ES05) is a mouse anti-human monoclonal antibody produced as a tissue culture supernatant, and supplied in Tris buffered saline with carrier protein, containing 0.35 % ProClin™ 950 as a preservative and in a total volume of 7 mL. The antibody is optimally diluted for use on the automated BOND-MAX or BOND-III instrument staining platforms in combination with BOND Polymer Refine Detection (DS9800).

MSH2 (Mismatch Repair Protein) (79H11) is a mouse anti-human monoclonal antibody produced as a tissue culture supernatant, and supplied in Tris buffered saline with carrier protein, containing 0.35 % ProClin™ 950 as a preservative and in a total volume of 7ml. The antibody is optimally diluted for use on the automated BOND-MAX or BOND-III instrument staining platforms in combination with BOND Polymer Refine Detection (DS9800).

MSH6 (Mismatch Repair Protein) (EP49) is a rabbit anti-human monoclonal antibody produced as an affinity-purified tissue culture supernatant, and supplied in Tris buffered saline with carrier protein, containing 0.35 % ProClin™ 950 as a preservative and in a total volume of 7ml. The antibody is optimally diluted for use on the automated BOND-MAX or BOND-III instrument staining platforms in combination with BOND Polymer Refine Detection (DS9800).

PMS2 (Mismatch Repair Protein) (EP51) is a rabbit anti-human monoclonal antibody produced as an affinity purified tissue culture supernatant, and supplied in Tris buffered saline with carrier protein, containing 0.35 % ProClin™ 950 as a preservative and in a total volume of 7ml. The antibody is optimally diluted for use on the automated BOND-MAX or BOND-III instrument staining platforms in combination with BOND Polymer Refine Detection (DS9800).

Instrument and Software: The BOND-MAX and BOND-III instruments are fully automated slide stainers that perform automated deparaffinization (dewaxing), antigen retrieval, immunohistochemistry (IHC) staining/in situ hybridization (ISH) staining, and counterstaining. The major components of the BOND staining platforms are the processing module, computer (BOND controller), handheld ID scanner, and slide label printer. The BOND staining platforms are composed of a number of discrete software components including the BOND application software, BOND instrument/processing module software, BOND service software, and Laboratory interface system - integration package (LIS-IP).

Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided text:

Device: BOND MMR Antibody Panel

Intended Use: Qualitative identification by light microscopy of human mismatch repair (MMR) proteins (MLH1, MSH2, MSH6, PMS2) in formalin-fixed, paraffin-embedded (FFPE) colorectal cancer (CRC) tissue sections by immunohistochemical staining. It aids in identifying potential hereditary nonpolyposis colorectal cancer (HNPCC)/Lynch Syndrome in CRC patients.

1. Table of Acceptance Criteria and Reported Device Performance

The document doesn't explicitly state "acceptance criteria" in a single, consolidated table. However, the precision studies reference "pre-specified acceptance criteria of ≥85% lower bound confidence interval." The clinical performance section states the point estimates of agreement, which implicitly serve as performance benchmarks.

2. Sample Size Used for the Test Set and Data Provenance

Test Set Sample Size:

• Precision Studies (Intra-run, Between-day, Between-lot):
  • 40 FFPE CRC tissue cases (10 per MMR protein: 5 protein deficient, 5 intact) - used for intra-run, between-day, between-lot precision.
  • One MSH2 case was excluded in some analyses due to insufficient tumor.
• Reproducibility Studies (Pathologist & Laboratory):
  • 30 FFPE CRC tissue cases (specific breakdown for each MMR protein provided in Table 4, e.g., MLH1: 25 intact, 5 loss).
• Inter-day and Inter-site Reproducibility (BOND-III):
  • 24 FFPE CRC tissue cases (3 intact and 3 loss cases for each of the 4 MMR proteins).
• Clinical Performance Study:
  • Initially, 155 cases procured.
  • 143 cases were eligible and tested by both methods (BOND MMR Antibody Panel and DNA sequencing panel).
  • 133 cases had valid results by both methods and were evaluable for agreement analysis. These comprised:
    • Sequential cohort: 94 cases (unknown MMR status, sequentially obtained from a single US site).
    • Enrichment cohort: 49 cases (known MMR protein deficiencies from multiple sites).

Data Provenance:

• Clinical Performance Study: Specimens for the sequential cohort were obtained from a single US site. The enrichment cohort specimens were from multiple sites. The text indicates "Eligible remnant FFPE CRC tissues ("cases") were procured." This suggests the data are retrospective, using banked FFPE tissue samples.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

Precision & Reproducibility Studies:

• Intra-run, Between-day, Between-lot Precision: A single pathologist read and scored all stained slides. No specific qualifications are provided for this pathologist, other than being "a single pathologist."
• Intra- and Inter-Pathologist Reproducibility: 3 pathologists read and scored the 30 cases. No specific qualifications beyond "pathologist" are provided.
• Inter-Instrument Reproducibility: One pathologist evaluated the cases. No specific qualifications are provided.
• Inter-Laboratory Reproducibility: One pathologist at each of the 3 sites independently evaluated each case. No specific qualifications beyond "pathologist" are provided.
• Inter-day and Inter-site Reproducibility (BOND-III): One pathologist at each of the 3 sites read and scored the stained slides. No specific qualifications beyond "pathologist" are provided.

Clinical Performance Study:

• "One pathologist at the testing site read and scored BOND MMR Antibody Panel stained slides in accordance with the scoring guidance." No specific qualifications are provided for this pathologist.
• For the ground truth (DNA sequencing panel results): The ground truth was established by assessing "pathogenic mutation(s) likely to affect MMR protein expression in CRC" using a DNA sequencing panel. This is an objective molecular test, not dependent on expert visual interpretation.

4. Adjudication Method for the Test Set

Precision & Reproducibility Studies:

• For precision studies where a single pathologist scored, "majority score" was used as the reference where multiple replicates were performed. For reproducibility studies involving multiple pathologists or sites, concordance between pathologists/sites was evaluated. No explicit adjudication process like "2+1" or "3+1" is described for resolving discrepancies to establish a single ground truth from pathologist reads for performance metrics. Instead, the studies assess agreement between readers and sites. The "majority score" used in the precision studies implied that if there were discordant reads, the majority would determine the "true" result for that specific replicate.

Clinical Performance Study:

• The ground truth for the clinical performance study was the DNA sequencing panel result. The pathologist's interpretation of the BOND MMR Antibody Panel staining was compared against this molecular ground truth. Therefore, no pathologist adjudication of the test device results was used to establish the ground truth for this comparison.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Was an MRMC study done? Yes, aspects of MRMC designs are present in the reproducibility studies, particularly the "Intra- and Inter-Pathologist Reproducibility" and "Inter-Laboratory Reproducibility" sections. These studies involved multiple pathologists reading multiple cases to assess the reproducibility of the device's output.
• Effect size of human reader improvement with AI vs. without AI assistance: Not applicable. This device is an immunohistochemistry (IHC) panel, interpreted by pathologists using light microscopy. It is not an AI-assisted diagnostic device, nor does the study evaluate human reader performance with or without AI assistance. The study focuses on the reproducibility and clinical validity of the IHC panel itself.

6. Standalone Performance Study (Algorithm Only)

• This question is not applicable as the device is an Immunohistochemistry (IHC) panel, not an algorithm or AI system. Its performance is intrinsically tied to human interpretation (by a pathologist). The studies evaluate the performance of the IHC panel as interpreted by pathologists.

7. Type of Ground Truth Used

• Precision/Reproducibility Studies: The "ground truth" for evaluating these studies was largely based on the expected protein expression status of the selected CRC tissue cases (e.g., "5 cases being protein deficient and 5 cases expressing intact protein"). For calculating agreements over repeat measurements, often a "majority score" from multiple reads or comparison to a baseline read was used as the reference.
• Clinical Performance Study: The ground truth was established by a molecular test: a DNA sequencing panel validated for detecting pathogenic mutations likely to affect MMR protein expression in CRC. This provides an objective measure of MMR gene status, which is then correlated with the protein expression detected by the IHC panel.

8. Sample Size for the Training Set

The document describes pre-market testing and performance characterization, not the development or training of an AI algorithm. Therefore, there is no mention of a "training set" sample size for an algorithm. The "Immunoreactivity" section (Table 15-17) shows the testing of the antibodies across a wide variety of normal, neoplastic, and colorectal cancer tissues (dozens to hundreds of cases across various tissue types) to characterize their expected staining patterns, which could be considered part of the development/characterization phase, but not an algorithmic "training set."

9. How the Ground Truth for the Training Set Was Established

As there is no training set for an AI algorithm mentioned in the document (the device is an IHC panel), this question is not applicable. The "Immunoreactivity" section's characterization of normal and tumor tissue staining patterns served as the basis for understanding expected reactivity, likely through known biological knowledge and expert pathological evaluation.

Ask a specific question about this device

The OncoMate™ MSI Dx Analysis System is a qualitative multiplex polymerase chain reaction (PCR) test intended to detect the deletion of mononucleotides in 5 microsatellite loci (BAT-25, NR-21, NR-24 and MONO-27) using matched tumor and normal DNA obtained from formalin fixed, paraffin-embedded (FFPE) colorectal tissue sections. The OncoMate™ MSI Dx Analysis System is for use with the Applied Biosystems® 3500Dx Genetic Analyzer and OncoMate™ MSI Dx Interpretive Software.

The OncoMate™ MSI Dx Analysis System is indicated in patients diagnosed with colorectal cancer (CRC) to detect microsatellite instability (MSI) as an aid in the identification of probable Lynch syndrome to help identify patients that would benefit from additional genetic testing to diagnose Lynch syndrome.

Results from the OncoMate™ MSI Dx Analysis System should be interpreted by healthcare professionals in conjunction with other clinical findings, family history, and other laboratory data.

The clinical performance of this device to guide treatment decision for MSI high patients has not been established.

The OncoMate™ MSI Dx Analysis System assay encompasses a complete workflow for MSI determination, from DNA extraction to data analysis. DNA is extracted from FFPE colorectal tissue samples (normal and tumor from the same patient) using the Maxwell® CSC DNA FFPE Kit and Maxwell® CSC Instrument. Double-stranded DNA (dsDNA) is then quantified using a fluorescence-based dsDNA quantification system of the user's choice. Next, amplification products are generated through multiplex PCR amplification of DNA microsatellite markers using the OncoMate™M MSI Dx Analysis System amplification kit. The PCR products are then mixed with Hi-Di™ Formamide and Size Standard 500 and heat-denatured. The resulting single-stranded DNA fragments are separated by size and detected via fluorescence using an Applied Biosystems® 3500Dx Genetic Analyzer. Following capillary electrophoresis, allele sizes from the CRC tumor DNA and the normal DNA are calculated and compared for each of the microsatellite markers using OncoMate™ MSI Dx Interpretive Software. If the length of two or more of the five mononucleotide-repeat marker alleles is changed by ≥2.75 base pairs (bp), the tumor is classified as MSI-H; if the allele length is changed for only one marker, or if the difference in allele lengths at the five markers is

Here's an analysis of the acceptance criteria and study data provided for the OncoMate™ MSI Dx Analysis System:

Acceptance Criteria and Reported Device Performance for OncoMate™ MSI Dx Analysis System

I. Acceptance Criteria Table:

Unfortunately, the provided text does not explicitly state a predefined set of quantitative acceptance criteria for the clinical performance endpoints (e.g., specific thresholds for PPA, NPA, or OPA). Instead, it presents the results of various analytical studies and then the "Method Comparison Study" against a predicate device and NGS. The conclusion then broadly states that the device is "substantially equivalent" based on these results.

However, we can infer some implicit performance expectations based on the presented results and the comparison to the predicate. The analytical studies demonstrate robust performance and reproducibility, which are foundational for a diagnostic device. The clinical method comparison aims to show comparable performance to the predicate.

Here's a table based on the reported performance in the method comparison study, which serves as the primary evidence for clinical acceptance:

II. Sample Size and Data Provenance for the Test Set (Clinical Method Comparison):

• Sample Size: A total of 154 cases were used in the primary clinical method comparison study.
• Data Provenance: The samples consisted of Formalin-Fixed, Paraffin-Embedded (FFPE) colorectal cancer patient samples.
  • The study design involved a "sequential series" of CRC patient samples enriched with a second set of suspected Lynch syndrome samples. This implies a mixture of prospective (sequential series) and potentially retrospective (enriched set sourced from a registry/archive of suspected Lynch cases) samples, although specific dates or collection methods aren't detailed.
  • No specific country of origin is mentioned for the patient samples, but the context of an FDA submission for a US market suggests they were likely representative of a US population or a population broadly applicable to US clinical practice.

III. Number of Experts and Qualifications for Ground Truth Establishment (Clinical Method Comparison):

The ground truth for the clinical method comparison was primarily established by two reference methods:

1. VENTANA MMR IHC Panel: This is a commercially available immunohistochemistry test, and its interpretation would typically be performed by qualified pathologists. The document states, "Immunohistochemistry was performed on all 154 tumor samples to determine protein expression... per the VENTANA instructions for use and the laboratory's Standard Operating Procedures." While the number of pathologists or their specific years of experience are not explicitly stated, the reference to SOPs suggests adherence to standard professional practice.
2. Next Generation Sequencing (NGS) for DNA mismatch repair genes: This was performed by a "reference laboratory." The ground truth for Lynch syndrome confirmation was based on "Pathogenic or likely pathogenic mutations listed in the ClinVar database (21)." This implies that the interpretation of NGS results relied on an established public database and likely expert geneticists or molecular pathologists interpreting the findings according to recognized clinical significance criteria (e.g., ACMG guidelines). Again, the number of experts or their specific qualifications for interpreting the NGS results directly from this study is not explicitly stated, but the reliance on ClinVar and "reference laboratory" practices implicitly indicates expert interpretation.

IV. Adjudication Method for the Test Set:

The document does not describe an explicit adjudication method (e.g., 2+1, 3+1 consensus with multiple readers) for establishing the ground truth of the clinical samples within the context of the study.

• For the IHC comparison, the IHC results served as one gold standard, without mention of multiple independent IHC reads and subsequent adjudication if there were discrepancies.
• For the NGS comparison, the detection of pathogenic/likely pathogenic mutations in MMR genes (and BRAF exon 15) as recorded in ClinVar served as the ground truth. This is an objective molecular test result criterion rather than a subjective interpretation requiring adjudication.

V. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:

A traditional MRMC comparative effectiveness study, where human readers interpret cases with and without AI assistance to measure effect size, was not performed or described in this document.

The "OncoMate™ MSI Dx Analysis System" explicitly includes "OncoMate™ MSI Dx Interpretive Software" as part of the system. This implies that the software provides an "automated interpretive result, either MSI-H or MSS" (page 9). Therefore, the device itself is the interpretive system, rather than an AI assistance tool for human readers. The clinical study evaluated the performance of this system as a whole (kit + software) against other diagnostic methods (IHC and NGS).

VI. Standalone (Algorithm Only) Performance:

Yes, a standalone (algorithm only) performance study was performed. The entire analytical and clinical performance sections evaluate the OncoMate™ MSI Dx Analysis System, which includes the OncoMate™ MSI Dx Interpretive Software, as a self-contained diagnostic tool. The software provides an "automated interpretive result" (page 9) without requiring further human interpretation of the raw data. The performance metrics (PPA, NPA, OPA, reproducibility, LOD, etc.) are all measures of this standalone system's ability to classify MSI status.

VII. Type of Ground Truth Used:

The ground truth for the clinical method comparison study was established using two different methods:

1. Expert Interpretation of a Predicate Device: The VENTANA MMR IHC Panel results, interpreted to determine MMR Loss or MMR Intact status. This represents an indirect measure of MSI status based on protein expression.
2. NGS Mismatch Repair Gene Mutations and ClinVar Database: Direct molecular evidence of pathogenic or likely pathogenic germline mutations in MMR genes (MLH1, MSH2, MSH6, PMS2) and BRAF exon 15, cross-referenced with the ClinVar database. This is a direct genetic confirmation of Lynch syndrome.

Thus, the ground truth is a combination of established diagnostic methods: protein-level immunohistochemistry and germline genetic sequencing data verified against a public expert-curated database.

VIII. Sample Size for the Training Set:

The document does not explicitly describe a separate training set or its sample size for the OncoMate™ MSI Dx Analysis System. Diagnostic devices based on PCR and fragment analysis, like this one, typically rely on analytical validation (precision, accuracy of sizing, LOD, etc.) and clinical validation (method comparison) rather than traditional machine learning "training" sets in the same way an image-based AI algorithm might.

The development and optimization of the interpretive software's logic (e.g., the ≥2.75bp threshold for instability, the rule for MSI-H/MSS classification) would have been informed by extensive research and analytical studies, but these wouldn't be referred to as a "training set" in the machine learning sense. The provided performance studies are primarily about validation of the finalized system.

IX. How the Ground Truth for the Training Set Was Established:

As there is no explicitly stated "training set" in the context of machine learning, the concept of establishing ground truth for it as a separate phase is not detailed. The design of the system, including the software's interpretive rules, would have been based on established scientific principles of microsatellite instability analysis and potentially internal development data. For example, the definition of "marker instability as a 3bp change (implemented as ≥2.75bp to account for the sizing precision of capillary electrophoresis)" (page 9) is a pre-defined rule, not learned from a dataset. The analytical studies (like "Normal Range and Cutoff") verify these inherent rules and measurement capabilities.

Ask a specific question about this device

The VENTANA MMR IHC Panel is a qualitative immunohistochemistry (IHC) test intended for use in the light microscopic assessment of mismatch repair (MMR) proteins (MLH1, PMS2, MSH2, and MSH6) and BRAF V600E proteins in formalin-fixed, paraffin-embedded colorectal cancer (CRC) tissue sections. The OptiView DAB IHC Detection Kit is used with MLH1, MSH2, MSH6 and BRAF V600E, and the OptiView DAB IHC Detection Kit with OptiView Amplification Kit is used for PMS2 detection. The VENTANA MMR IHC Panel is for use on the VENTANA BenchMark ULTRA instrument. The VENTANA MMR IHC Panel includes VENTANA anti-MLH1 (M1) Mouse Monoclonal Primary Antibody, VENTANA anti-PMS2 (A16-4) Mouse Monoclonal Primary Antibody, VENTANA anti-MSH2 (G219-1129) Mouse Monoclonal Primary Antibody, VENTANA anti-MSH6 (SP93) Rabbit Monoclonal Primary Antibody, and VENTANA anti-BRAF V600E (VE1) Mouse Monoclonal Primary Antibody.

The VENTANA MMR IHC Panel is indicated in patients diagnosed with colorectal cancer (CRC) to detect mismatch repair (MMR) proteins deficiency as an aid in the identification of probable Lynch syndrome and to detect BRAFV600E protein as an aid to differentiate between sporadic CRC and probable Lynch syndrome.

Results from the Ventana MMR IHC Panel should be interpreted by a qualified pathologist in conjunction with histological examination, relevant clinical information, and proper controls.

The clinical performance of this device to guide treatment of MMR deficient patients has not been established.

The Ventana MMR IHC panel is comprised of five primary antibodies used to detect the MMR proteins MLH1, PMS2, MSH2 and MSH6 and mutated BRAF V600E protein in CRC tissue specimens. The primary antibodies are used in combination with individually optimized detection reagents and in conjunction with ancillary reagents common to all immunohistochemistry test systems in order to complete specimen testing. The MMR IHC panel and BRAF V600E are optimized to run on the VENTANA BenchMark Ultra platform with OptiView DAB detection kit or in the case of PSM2 antibody the OptiView DAB detection Kit with the OptiView Amplification Kit. The presence or absence of target proteins is determined by visual examination of the specimen slide under light microscope by a qualified pathologist.

The Ventana MMR IHC panel antibodies are packaged as individual products in single ready to use reagent dispensers. The MMR IHC panel test is run on five separate CRC tissue slides and stained on BenchMark Ultra instrument.

The Ventura MMR IHC Panel is a qualitative immunohistochemistry (IHC) test used to detect mismatch repair (MMR) proteins (MLH1, PMS2, MSH2, and MSH6) and BRAF V600E proteins in formalin-fixed, paraffin-embedded colorectal cancer (CRC) tissue sections. It is intended to aid in the identification of probable Lynch syndrome and differentiate between sporadic CRC and probable Lynch syndrome.

Here's an analysis of the acceptance criteria and the study that proves the device meets those criteria:

1. Acceptance Criteria and Reported Device Performance

The core acceptance criterion for the primary clinical performance study was that both the Positive Percent Agreement (PPA) and Negative Percent Agreement (NPA) rates across all observations must exhibit a lower bound of the 2-sided 95% confidence interval (LBCI) > 85%, when using the modal result for each case as the reference for that case.

This criterion was applied in the reproducibility study and generally implied in the clinical validation study's concordance metrics.

Note on Clinical Performance Acceptance: While the overall PPA LBCI of 54.8% for the clinical study appears to not meet the >85% threshold, the document frequently states that "The study met pre-specified acceptance criteria" for various segments. The overall PPA is heavily influenced by the smaller number of pathogenic mutation cases and the complexities of interpreting clinical performance (where some IHC "loss" may not correlate with a "pathogenic mutation" by sequencing, particularly in MLH1/PMS2 due to sporadic nature). The detailed breakdown by individual markers and stratified cohorts shows varying performance, with some individual marker PPAs and NPAs exceeding the implied threshold. The document's conclusion that "The probable clinical benefits of this device outweigh the potential risks" suggests that the clinical performance, despite some lower PPA values, was deemed acceptable in the overall benefit-risk assessment. For instance, in the Enrichment Cohort, the PPA for MLH1, PMS2, and MSH2 was 100%.

2. Sample Size and Data Provenance

Reproducibility Test Set:

• Sample Size: 40 archival FFPE CRC tissue specimens.
  • For the 4 MMR antibodies: 6 CRC specimens each (3 intact, 3 loss) = 24 cases total.
  • For anti-BRAF V600E antibody: 16 CRC specimens (8 positive, 8 negative).
• Data Provenance: Archival FFPE CRC tissue specimens. The country of origin is not explicitly stated but implies multicenter data given the "3 external clinical sites." The setting is retrospective, using archival samples.

Clinical Performance Test Set:

• Sample Size:
  • Initially, a sequential series of CRC specimens yielding 111 cases.
  • Enriched with a second set of 15 CRC cases with confirmed loss status for MMR proteins by IHC, for a total of 126 specimens.
  • 7 cases excluded due to insufficient viable tumor/misclassification, 1 due to clerical error.
  • 7 additional cases failed DNA sequencing, and 1 failed IHC, leading to 119 evaluable specimens for the main concordance analysis.
• Data Provenance: Not explicitly stated, but "sequential series of CRC specimens" and "enriched with a second set of specimens with known Lynch syndrome variants" suggest a mix of retrospective collection from a clinical setting.

3. Number of Experts and Qualifications for Ground Truth

Reproducibility Study:

• Number of Experts: Two pathologists at each of the 3 sites, leading to a total of 6 pathologists.
• Qualifications: "Qualified pathologist." Specific years of experience are not mentioned.

Clinical Performance Study:

• Number of Experts for IHC Interpretation: "a qualified pathologist" for the device result, and the DNA sequencing results are used as the reference/ground truth. For establishing the ground truth from DNA sequencing, specific expertise is not detailed, but it implies a molecular pathology or genetic testing lab.

4. Adjudication Method

Reproducibility Study (for modal case reference status):

• Adjudication Method: Implicitly, a "modal case reference status for calculation PPA, NPA and OA was derived on the most often observed status of 30 observations." This suggests a majority vote (e.g., if 30 observations were made for a case, whatever classification (intact/loss, positive/negative) occurred most frequently was taken as the reference).

Clinical Performance Study:

• Adjudication Method: Not explicitly stated for the "DNA sequencing results" which served as the ground truth. For discrepancies between IHC and DNA sequencing, the report describes some situations, such as "low allele frequency in sequencing suggesting sporadic CRC" or "somatic mutations," as part of the reconciliation, but a formal adjudication process (e.g., 2+1, 3+1) is not explicitly detailed.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No MRMC comparative effectiveness study, evaluating the effect size of human readers with vs. without AI assistance, was explicitly reported. This device is an IHC panel interpreted by pathologists, not an AI-assisted diagnostic tool.

6. Standalone (Algorithm Only) Performance

Not applicable. This is an immunohistochemistry panel, which is a laboratory assay where interpretation is done by a human pathologist using a light microscope. There is no independent algorithm-only performance to assess.

7. Type of Ground Truth Used

Reproducibility Study:

• Ground Truth Type: "Modal case reference status," derived from the most frequently observed status across 30 observations for each case. This is a form of expert consensus derived from the study's pathologists.

Clinical Performance Study:

• Ground Truth Type: DNA sequencing panel validated for detecting pathogenic Lynch syndrome variants and BRAF V600E mutations. This represents a high-level molecular ground truth, considered the gold standard for genetic mutations.

8. Sample Size for the Training Set

The document describes performance characteristics of the VENTANA MMR IHC Panel, which is a set of antibodies and reagents used on an automated staining instrument, with interpretation by a pathologist. It is not a machine learning or AI-based diagnostic algorithm that typically has a "training set."

The development and optimization of the antibodies and their protocols would have involved various internal studies and sample sets, but these are not referred to as a "training set" in the context of an algorithm. The reproducibility and clinical studies utilize "archival FFPE CRC tissue specimens" or "sequential series of CRC specimens" which function as test sets for this IVD kit.

9. How the Ground Truth for the Training Set Was Established

As noted above, there is no "training set" in the context of an AI/ML algorithm for this device. The development of the IHC assays themselves would have involved extensive R&D and validation steps, where the "ground truth" for antibody specificity and reactivity would be based on established scientific principles, molecular characterization of cell lines and tissues (e.g., known MMR status, presence of BRAF V600E mutation), and expert pathological evaluation.

For the analytical specificity studies (Western Blot and Immunoreactivity), cell lines with known MMR loss or intact status and BRAFV600E containing cell lines (engineered to express moderate and high levels of the V600E protein) were used. This provides a clear, controlled ground truth for analytical validation.

Ask a specific question about this device