(526 days)
The MammaPrint FFPE NGS kit is a qualitative in vitro diagnostic test for use by clinical laboratories using target enrichment Next Generation Sequencing (NGS) technology for gene expression profiling of the 70-gene MammaPrint Breast Cancer signature on formalin-fixed, paraffin-embedded (FFPE) breast cancer tissue samples. The test is used to assess a patient's risk to develop distant metastasis within 5 years and up to 10 years after diagnosis.
The MammaPrint FFPE NGS kit is performed for breast cancer patients with Stage I or Stage II disease, with tumor size ≤ 5.0 cm and lymph node negative. The test result is indicated for use by physicians as a prognostic marker only, along with other clinicopathological factors.
The MammaPrint FFPE NGS kit is a sequencing-based gene expression analysis of a tumor. The analysis is based on several processes: isolation of RNA from FFPE breast cancer tissue sections; library preparation of RNA resulting in cDNA adapter-ligated sequences; enrichment of the 70 genes (capture step); sequencing of the enriched library in the flow cell and data acquisition; MammaPrint Index calculation of the risk classification in breast cancer patients.
Data analysis is performed according to the MammaPrint FFPE NGS algorithm (resulting in MammaPrint Index or MPI). This algorithm was designed and programmed by Agendia and incorporated into a proprietary software program, which loads the FASTQ data file. The software loads file, performs quality control checks and determines the molecular profile of the sample by calculating the MammaPrint index by determining the correlation of the sample's 70 gene expression profile to the mean expression profiles of tumors with a known good and poor outcome.
The provided text describes the acceptance criteria and the study that proves the MammaPrint FFPE NGS kit meets these criteria, primarily by demonstrating substantial equivalence to its predicate device (MammaPrint FFPE microarray).
Here's a breakdown of the requested information:
1. Table of Acceptance Criteria and Reported Device Performance
The document doesn't explicitly outline a formal "acceptance criteria" table with pre-defined thresholds for performance metrics. Instead, it demonstrates performance through concordance studies and reproducibility assessments against the predicate device, or by showing high agreement for controls. The clinical performance is demonstrated by comparing the survival outcomes of the new device to those of the predicate device on the same patient cohort from the RASTER study.
Acceptance Criteria (Implied by the study design and results presented):
| Performance Metric | Implied Acceptance Criteria (Achieved by the device) | Reported Device Performance |
|---|---|---|
| Method Comparison (Study-1): | ||
| OPA (High vs. Low) | High concordance with predicate device. | 97.42% (95% Cl: 93.55, 98.99) |
| NPA (High vs. Low) | High concordance with predicate device. | 93.85% (95% C1: 85.22, 97.58) |
| PPA (High vs. Low) | High concordance with predicate device. | 100.00% (95% CI: 95.91, 100.00) |
| Method Comparison (Study-2): | ||
| OPA (ranging across sites) | High concordance with predicate device. | 91.09% to 92.41% |
| PPA (ranging across sites) | High concordance with predicate device. | 91.39% to 99.34% |
| NPA (ranging across sites) | High concordance with predicate device. | Lower: 82.89% to 93.42% (noted as impacted by borderline samples) |
| Repeatability of RNA Isolation (Categorical Results): | 100% agreement between repeat isolations. | 100% agreement (95%Cl: High Risk: 86.2-100.0, Low Risk: 81.6-100.0, Borderline: 34.2 –100.0) |
| Reproducibility of Controls (Categorical Results): | 100% agreement for known controls across sites/operators/lots. | 100% agreement for both CTRL-HR and CTRL-LR (Table 1: 94.2%-100% CI for both). |
| Precision/Reproducibility (Categorical Results: Study-1): | High agreement for different risk categories. | PREC-BD (Borderline): 81% (72.0, 88.5) (Table 3) |
| PREC-HR (High Risk): 100% (96.9, 100) (Table 3) | ||
| PREC-LR (Low Risk): 100% (96.9, 100) (Table 3) | ||
| Precision/Reproducibility (Categorical Results: Study-2): | High agreement, especially for samples not near threshold. | Range from 58% to 100% (Table 5). Samples 2 (Low Risk) and 5 (High Risk) had 83% and 58% agreement respectively. |
| Detection Limit (Valid Rates): | Acceptable valid rates at specified RNA input and quality. | DV200 "poor" (35-49%) with ≥100 ng RNA: 80% to 100% valid rates. |
| DV200 "Standard" (35-80%) with ≥100 ng RNA: 80% to 100% valid rates. | ||
| Clinical Performance (DRFI): | Similar survival outcomes to predicate device. | Kaplan-Meier plots suggest significant difference in survival curves among different risk groups for MammaPrint FFPE NGS kit, p=0.001. |
| 5-year DRFI: Low Risk 98.1%, Borderline 92.6%, High Risk 88.2% (Table 6). | ||
| 10-year DRFI: Low Risk 95.2%, Borderline 92.6%, High Risk 82.0% (Table 6). | ||
| The results "indicated that both devices show similar clinical performance" based on RASTER study follow-up. |
2. Sample Sizes and Data Provenance
- Method Comparison Study-1: 155 samples used. Data acquired retrospectively as these samples were "previously processed on MammaPrint FFPE microarray as part of routine diagnostics."
- Method Comparison Study-2: 303 samples used. Data acquired retrospectively from "previously collected patient samples."
- Repeatability of RNA Isolation: Not explicitly stated, but implies multiple FFPE tumor blocks, each sectioned twice.
- Reproducibility of Controls: 2 control samples (CTRL-HR, CTRL-LR), each processed 50 times across 4 external sites, 2 operators per site, 6 NGS runs, and 3 lot numbers. Total of 100 measurements for controls.
- Precision/Reproducibility Assessment-Study-1: 3 samples (High Risk, Borderline, Low Risk). Each processed 96 times (4 external sites x 2 operators x 6 NGS runs x 2 duplicates per run).
- Precision/Reproducibility Assessment-Study-2: 8 samples. Each processed repeatedly (from RNA isolation to sequencing) by 2 operators at 3 sites, with 4 runs per site and 2 replicates per run. This results in 16-24 replicates per sample (table indicates N of 16 or 24).
- Detection Limit: 8 FFPE breast cancer samples with "poor" DV200 quality were evaluated, with various dilutions. Additional unstated number of "Standard" DV200 samples.
- Clinical Validation (RASTER Study):
- Original RASTER enrollment: 427 patients.
- Subset of FFPE samples from RASTER study used for NGS kit validation: 345 samples.
- Samples successfully processed for analysis: 316 samples.
- Data provenance: Multicenter observational study conducted in the Netherlands between 2004 and 2006 (for initial RASTER study). The current analysis uses updated 5- and 10-year follow-up data. This is prospective observational data with the device evaluated retrospectively on archived samples.
3. Number of Experts and Qualifications for Ground Truth
The document does not mention the use of experts to establish ground truth for the analytical or reproducibility test sets. The ground truth for these studies is based on comparisons against the predicate device's results or known control sample values.
For the clinical validation, the ground truth for distant recurrence-free interval (DRFI) and breast cancer specific survival (BCSS) is outcomes data from the RASTER study, defined as distant breast cancer recurrence or death from breast cancer. The establishment of these clinical outcomes likely involved clinical experts (e.g., oncologists, pathologists, and medical records review) over the 10-year follow-up period, but the specific number and qualifications of these experts are not detailed in this submission.
4. Adjudication Method for Test Set
The document does not describe an adjudication method (such as 2+1 or 3+1) for the test sets. For analytical studies, agreement is based on direct comparison of results between NGS and microarray platforms or within the NGS platform itself (repeatability/reproducibility). For the clinical study, outcomes (DRFI, BCSS) are established from patient follow-up data.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No MRMC comparative effectiveness study was performed as this device is a gene expression profiling test system, not an imaging device requiring human reader interpretation. The comparison is between the new NGS-based assay and the predicate microarray-based assay.
6. Standalone Performance
The "standalone" performance shown is the analytical performance of the MammaPrint FFPE NGS kit itself (repeatability, reproducibility, detection limit) and its clinical prognostic ability, which is an algorithm-only output (MammaPrint Index). The clinical validation study (RASTER) assesses the algorithm's ability to stratify patients by risk based on actual clinical outcomes. The device performance (result classification: High Risk, Borderline, Low Risk) is reported directly from the algorithm without a human-in-the-loop component for classification.
7. Type of Ground Truth Used
- Analytical Performance (Method Comparison, Repeatability, Reproducibility): The ground truth is effectively the predicate device's results or the known values of control samples. For repeatability/reproducibility, the ground truth is consistency of the device's own output.
- Clinical Validation: The ground truth is outcomes data (Distant Recurrence-Free Interval - DRFI, and Breast Cancer Specific Survival - BCSS) from prospectively collected clinical follow-up in the RASTER study. This is directly observed patient outcomes over 5 and 10 years.
8. Sample Size for the Training Set
The document focuses on the performance of the MammaPrint FFPE NGS kit, which is stated to use an "unchanged" 70-gene signature and scoring algorithm from the predicate device. The original MammaPrint 70-gene signature was developed much earlier. The description states: "This algorithm was designed and programmed by Agendia and incorporated into a proprietary software program, which loads the FASTQ data file." The specifics of the training set for the original MammaPrint algorithm are not provided in this document. This submission is for a new platform (NGS) of an existing and cleared diagnostic system, demonstrating its equivalence, rather than a new algorithm development.
9. How the Ground Truth for the Training Set was Established
As noted above, the original training of the MammaPrint algorithm is not detailed here. The clinical validation in this document uses the RASTER study data, where the ground truth is clinical outcome data (DRFI, BCSS) established through follow-up. This RASTER data is used for validation of the new NGS platform, not for training a new algorithm.
§ 866.6040 Gene expression profiling test system for breast cancer prognosis.
(a)
Identification. A gene expression profiling test system for breast cancer prognosis is a device that measures the ribonucleic acid (RNA) expression level of multiple genes and combines this information to yield a signature (pattern or classifier or index) to aid in prognosis of previously diagnosed breast cancer.(b)
Classification. Class II (special controls). The special control is FDA's guidance document entitled “Class II Special Controls Guidance Document: Gene Expression Profiling Test System for Breast Cancer Prognosis.” See § 866.1(e) for the availability of this guidance document.