(150 days)
The VENTANA MMR IHC Panel is a qualitative immunohistochemistry (IHC) test intended for use in the light microscopic assessment of mismatch repair (MMR) proteins (MLH1, PMS2, MSH2, and MSH6) and BRAF V600E proteins in formalin-fixed, paraffin-embedded colorectal cancer (CRC) tissue sections. The OptiView DAB IHC Detection Kit is used with MLH1, MSH2, MSH6 and BRAF V600E, and the OptiView DAB IHC Detection Kit with OptiView Amplification Kit is used for PMS2 detection. The VENTANA MMR IHC Panel is for use on the VENTANA BenchMark ULTRA instrument. The VENTANA MMR IHC Panel includes VENTANA anti-MLH1 (M1) Mouse Monoclonal Primary Antibody, VENTANA anti-PMS2 (A16-4) Mouse Monoclonal Primary Antibody, VENTANA anti-MSH2 (G219-1129) Mouse Monoclonal Primary Antibody, VENTANA anti-MSH6 (SP93) Rabbit Monoclonal Primary Antibody, and VENTANA anti-BRAF V600E (VE1) Mouse Monoclonal Primary Antibody.
The VENTANA MMR IHC Panel is indicated in patients diagnosed with colorectal cancer (CRC) to detect mismatch repair (MMR) proteins deficiency as an aid in the identification of probable Lynch syndrome and to detect BRAFV600E protein as an aid to differentiate between sporadic CRC and probable Lynch syndrome.
Results from the Ventana MMR IHC Panel should be interpreted by a qualified pathologist in conjunction with histological examination, relevant clinical information, and proper controls.
The clinical performance of this device to guide treatment of MMR deficient patients has not been established.
The Ventana MMR IHC panel is comprised of five primary antibodies used to detect the MMR proteins MLH1, PMS2, MSH2 and MSH6 and mutated BRAF V600E protein in CRC tissue specimens. The primary antibodies are used in combination with individually optimized detection reagents and in conjunction with ancillary reagents common to all immunohistochemistry test systems in order to complete specimen testing. The MMR IHC panel and BRAF V600E are optimized to run on the VENTANA BenchMark Ultra platform with OptiView DAB detection kit or in the case of PSM2 antibody the OptiView DAB detection Kit with the OptiView Amplification Kit. The presence or absence of target proteins is determined by visual examination of the specimen slide under light microscope by a qualified pathologist.
The Ventana MMR IHC panel antibodies are packaged as individual products in single ready to use reagent dispensers. The MMR IHC panel test is run on five separate CRC tissue slides and stained on BenchMark Ultra instrument.
The Ventura MMR IHC Panel is a qualitative immunohistochemistry (IHC) test used to detect mismatch repair (MMR) proteins (MLH1, PMS2, MSH2, and MSH6) and BRAF V600E proteins in formalin-fixed, paraffin-embedded colorectal cancer (CRC) tissue sections. It is intended to aid in the identification of probable Lynch syndrome and differentiate between sporadic CRC and probable Lynch syndrome.
Here's an analysis of the acceptance criteria and the study that proves the device meets those criteria:
1. Acceptance Criteria and Reported Device Performance
The core acceptance criterion for the primary clinical performance study was that both the Positive Percent Agreement (PPA) and Negative Percent Agreement (NPA) rates across all observations must exhibit a lower bound of the 2-sided 95% confidence interval (LBCI) > 85%, when using the modal result for each case as the reference for that case.
This criterion was applied in the reproducibility study and generally implied in the clinical validation study's concordance metrics.
| Acceptance Criteria (Reproducibility Study) | Reported Device Performance (Reproducibility Study - Overall Proteins) |
|---|---|
| PPA LBCI > 85% | 98.7% (598/600) |
| NPA LBCI > 85% | 97.4% (593/600) |
| Acceptance Criteria (Clinical Performance Study - Overall) | Reported Device Performance (Clinical Performance Study - Overall) |
|---|---|
| PPA LBCI > 85% | 54.8% (14/18) |
| NPA LBCI > 85% | 91.6% (98/101) |
Note on Clinical Performance Acceptance: While the overall PPA LBCI of 54.8% for the clinical study appears to not meet the >85% threshold, the document frequently states that "The study met pre-specified acceptance criteria" for various segments. The overall PPA is heavily influenced by the smaller number of pathogenic mutation cases and the complexities of interpreting clinical performance (where some IHC "loss" may not correlate with a "pathogenic mutation" by sequencing, particularly in MLH1/PMS2 due to sporadic nature). The detailed breakdown by individual markers and stratified cohorts shows varying performance, with some individual marker PPAs and NPAs exceeding the implied threshold. The document's conclusion that "The probable clinical benefits of this device outweigh the potential risks" suggests that the clinical performance, despite some lower PPA values, was deemed acceptable in the overall benefit-risk assessment. For instance, in the Enrichment Cohort, the PPA for MLH1, PMS2, and MSH2 was 100%.
2. Sample Size and Data Provenance
Reproducibility Test Set:
- Sample Size: 40 archival FFPE CRC tissue specimens.
- For the 4 MMR antibodies: 6 CRC specimens each (3 intact, 3 loss) = 24 cases total.
- For anti-BRAF V600E antibody: 16 CRC specimens (8 positive, 8 negative).
- Data Provenance: Archival FFPE CRC tissue specimens. The country of origin is not explicitly stated but implies multicenter data given the "3 external clinical sites." The setting is retrospective, using archival samples.
Clinical Performance Test Set:
- Sample Size:
- Initially, a sequential series of CRC specimens yielding 111 cases.
- Enriched with a second set of 15 CRC cases with confirmed loss status for MMR proteins by IHC, for a total of 126 specimens.
- 7 cases excluded due to insufficient viable tumor/misclassification, 1 due to clerical error.
- 7 additional cases failed DNA sequencing, and 1 failed IHC, leading to 119 evaluable specimens for the main concordance analysis.
- Data Provenance: Not explicitly stated, but "sequential series of CRC specimens" and "enriched with a second set of specimens with known Lynch syndrome variants" suggest a mix of retrospective collection from a clinical setting.
3. Number of Experts and Qualifications for Ground Truth
Reproducibility Study:
- Number of Experts: Two pathologists at each of the 3 sites, leading to a total of 6 pathologists.
- Qualifications: "Qualified pathologist." Specific years of experience are not mentioned.
Clinical Performance Study:
- Number of Experts for IHC Interpretation: "a qualified pathologist" for the device result, and the DNA sequencing results are used as the reference/ground truth. For establishing the ground truth from DNA sequencing, specific expertise is not detailed, but it implies a molecular pathology or genetic testing lab.
4. Adjudication Method
Reproducibility Study (for modal case reference status):
- Adjudication Method: Implicitly, a "modal case reference status for calculation PPA, NPA and OA was derived on the most often observed status of 30 observations." This suggests a majority vote (e.g., if 30 observations were made for a case, whatever classification (intact/loss, positive/negative) occurred most frequently was taken as the reference).
Clinical Performance Study:
- Adjudication Method: Not explicitly stated for the "DNA sequencing results" which served as the ground truth. For discrepancies between IHC and DNA sequencing, the report describes some situations, such as "low allele frequency in sequencing suggesting sporadic CRC" or "somatic mutations," as part of the reconciliation, but a formal adjudication process (e.g., 2+1, 3+1) is not explicitly detailed.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No MRMC comparative effectiveness study, evaluating the effect size of human readers with vs. without AI assistance, was explicitly reported. This device is an IHC panel interpreted by pathologists, not an AI-assisted diagnostic tool.
6. Standalone (Algorithm Only) Performance
Not applicable. This is an immunohistochemistry panel, which is a laboratory assay where interpretation is done by a human pathologist using a light microscope. There is no independent algorithm-only performance to assess.
7. Type of Ground Truth Used
Reproducibility Study:
- Ground Truth Type: "Modal case reference status," derived from the most frequently observed status across 30 observations for each case. This is a form of expert consensus derived from the study's pathologists.
Clinical Performance Study:
- Ground Truth Type: DNA sequencing panel validated for detecting pathogenic Lynch syndrome variants and BRAF V600E mutations. This represents a high-level molecular ground truth, considered the gold standard for genetic mutations.
8. Sample Size for the Training Set
The document describes performance characteristics of the VENTANA MMR IHC Panel, which is a set of antibodies and reagents used on an automated staining instrument, with interpretation by a pathologist. It is not a machine learning or AI-based diagnostic algorithm that typically has a "training set."
The development and optimization of the antibodies and their protocols would have involved various internal studies and sample sets, but these are not referred to as a "training set" in the context of an algorithm. The reproducibility and clinical studies utilize "archival FFPE CRC tissue specimens" or "sequential series of CRC specimens" which function as test sets for this IVD kit.
9. How the Ground Truth for the Training Set Was Established
As noted above, there is no "training set" in the context of an AI/ML algorithm for this device. The development of the IHC assays themselves would have involved extensive R&D and validation steps, where the "ground truth" for antibody specificity and reactivity would be based on established scientific principles, molecular characterization of cell lines and tissues (e.g., known MMR status, presence of BRAF V600E mutation), and expert pathological evaluation.
For the analytical specificity studies (Western Blot and Immunoreactivity), cell lines with known MMR loss or intact status and BRAFV600E containing cell lines (engineered to express moderate and high levels of the V600E protein) were used. This provides a clear, controlled ground truth for analytical validation.
§ 864.1866 Lynch syndrome test systems.
(a)
Identification. Lynch syndrome test systems are in vitro diagnostic tests for use with tumor tissue to identify previously diagnosed cancer patients at risk for having Lynch syndrome.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Premarket notification submissions must include the following information, as appropriate:
(i) A detailed description of all test components, including all provided reagents, and required but not provided, ancillary reagents.
(ii) A detailed description of instrumentation and equipment, including illustrations or photographs of non-standard equipment or manuals.
(iii) Detailed documentation of the device software, including, but not limited to, standalone software applications and hardware-based devices that incorporate software.
(iv) A detailed description of quality controls including appropriate positive and negative controls that are recommended or provided.
(v) Detailed specifications for sample collection, processing, and storage.
(vi) A detailed description of methodology and assay procedure.
(vii) A description of the assay cut-off (
i.e., the medical decision point between positive and negative results) or other relevant criteria that distinguishes positive and negative results, or ordinal classes of marker expression, including the rationale for the chosen cut-off or other relevant criteria and results supporting validation of the cut-off.(viii) Detailed specification of the criteria for test result interpretation and reporting.
(ix) Detailed information demonstrating the performance characteristics of the device, including:
(A) Data from an appropriate study demonstrating clinical accuracy using well-characterized clinical specimens representative of the intended use population (
i.e., concordance to Deoxyribonucleic Acid (DNA) sequencing results of the Lynch syndrome associated genes or method comparison to the predicate device using samples with known alterations in genes representative of Lynch syndrome). Pre-specified acceptance criteria must be provided and followed.(B) Appropriate device reproducibility data investigating all sources of variance (
e.g., for distributed tests, data generated using a minimum of three sites, of which at least two sites must be external sites). Each site must perform testing over a minimum of 5 nonconsecutive days evaluating a sample panel that spans the claimed measuring range, and includes the clinical threshold. Pre-specified acceptance criteria must be provided and followed.(C) Data demonstrating reader reproducibility, both within-reader and between-reader, assessed by three readers over 3 nonconsecutive days at each site, including a 2 week washout period between reads, as appropriate.
(D) Device precision data using clinical samples spanning the measuring range and controls to evaluate the within-lot, between-lot, within-run, between run, and total variation.
(E) Analytical specificity studies including as appropriate, western blots, peptide inhibition, testing in normal tissues and neoplastic tissues, interference by endogenous and exogenous substances, and cross-reactivity and cross contamination testing.
(F) Device analytical sensitivity data generated by testing an adequate number of samples from individuals with the target condition such that prevalence of the biomarker in the target population is established.
(G) Device stability data, including real-time stability and in-use stability, and stability evaluating various storage times, temperatures, and freeze-thaw conditions, as appropriate.
(H) The staining performance criteria assessed must include overall staining acceptability, background staining acceptability, and morphology acceptability, as appropriate.
(I) Appropriate training requirements for users, including interpretation manual, as applicable.
(J) Identification of risk mitigation elements used by the device, including a description of all additional procedures, methods, and practices incorporated into the instructions for use that mitigate risks associated with testing.
(2) The device's § 809.10(b) of this chapter compliant labeling must include a detailed description of the protocol, including the information described in paragraphs (b)(1)(i) through (viii) of this section, as appropriate, and a detailed description of the performance studies performed and the summary of the results, including those that relate to paragraph (b)(1)(ix) of this section, as appropriate.