Search Results
Found 8 results
510(k) Data Aggregation
(526 days)
Agendia Inc.
The MammaPrint FFPE NGS kit is a qualitative in vitro diagnostic test for use by clinical laboratories using target enrichment Next Generation Sequencing (NGS) technology for gene expression profiling of the 70-gene MammaPrint Breast Cancer signature on formalin-fixed, paraffin-embedded (FFPE) breast cancer tissue samples. The test is used to assess a patient's risk to develop distant metastasis within 5 years and up to 10 years after diagnosis.
The MammaPrint FFPE NGS kit is performed for breast cancer patients with Stage I or Stage II disease, with tumor size ≤ 5.0 cm and lymph node negative. The test result is indicated for use by physicians as a prognostic marker only, along with other clinicopathological factors.
The MammaPrint FFPE NGS kit is a sequencing-based gene expression analysis of a tumor. The analysis is based on several processes: isolation of RNA from FFPE breast cancer tissue sections; library preparation of RNA resulting in cDNA adapter-ligated sequences; enrichment of the 70 genes (capture step); sequencing of the enriched library in the flow cell and data acquisition; MammaPrint Index calculation of the risk classification in breast cancer patients.
Data analysis is performed according to the MammaPrint FFPE NGS algorithm (resulting in MammaPrint Index or MPI). This algorithm was designed and programmed by Agendia and incorporated into a proprietary software program, which loads the FASTQ data file. The software loads file, performs quality control checks and determines the molecular profile of the sample by calculating the MammaPrint index by determining the correlation of the sample's 70 gene expression profile to the mean expression profiles of tumors with a known good and poor outcome.
The provided text describes the acceptance criteria and the study that proves the MammaPrint FFPE NGS kit meets these criteria, primarily by demonstrating substantial equivalence to its predicate device (MammaPrint FFPE microarray).
Here's a breakdown of the requested information:
1. Table of Acceptance Criteria and Reported Device Performance
The document doesn't explicitly outline a formal "acceptance criteria" table with pre-defined thresholds for performance metrics. Instead, it demonstrates performance through concordance studies and reproducibility assessments against the predicate device, or by showing high agreement for controls. The clinical performance is demonstrated by comparing the survival outcomes of the new device to those of the predicate device on the same patient cohort from the RASTER study.
Acceptance Criteria (Implied by the study design and results presented):
| Performance Metric | Implied Acceptance Criteria (Achieved by the device) | Reported Device Performance |
|---|---|---|
| Method Comparison (Study-1): | ||
| OPA (High vs. Low) | High concordance with predicate device. | 97.42% (95% Cl: 93.55, 98.99) |
| NPA (High vs. Low) | High concordance with predicate device. | 93.85% (95% C1: 85.22, 97.58) |
| PPA (High vs. Low) | High concordance with predicate device. | 100.00% (95% CI: 95.91, 100.00) |
| Method Comparison (Study-2): | ||
| OPA (ranging across sites) | High concordance with predicate device. | 91.09% to 92.41% |
| PPA (ranging across sites) | High concordance with predicate device. | 91.39% to 99.34% |
| NPA (ranging across sites) | High concordance with predicate device. | Lower: 82.89% to 93.42% (noted as impacted by borderline samples) |
| Repeatability of RNA Isolation (Categorical Results): | 100% agreement between repeat isolations. | 100% agreement (95%Cl: High Risk: 86.2-100.0, Low Risk: 81.6-100.0, Borderline: 34.2 –100.0) |
| Reproducibility of Controls (Categorical Results): | 100% agreement for known controls across sites/operators/lots. | 100% agreement for both CTRL-HR and CTRL-LR (Table 1: 94.2%-100% CI for both). |
| Precision/Reproducibility (Categorical Results: Study-1): | High agreement for different risk categories. | PREC-BD (Borderline): 81% (72.0, 88.5) (Table 3) |
| PREC-HR (High Risk): 100% (96.9, 100) (Table 3) | ||
| PREC-LR (Low Risk): 100% (96.9, 100) (Table 3) | ||
| Precision/Reproducibility (Categorical Results: Study-2): | High agreement, especially for samples not near threshold. | Range from 58% to 100% (Table 5). Samples 2 (Low Risk) and 5 (High Risk) had 83% and 58% agreement respectively. |
| Detection Limit (Valid Rates): | Acceptable valid rates at specified RNA input and quality. | DV200 "poor" (35-49%) with ≥100 ng RNA: 80% to 100% valid rates. |
| DV200 "Standard" (35-80%) with ≥100 ng RNA: 80% to 100% valid rates. | ||
| Clinical Performance (DRFI): | Similar survival outcomes to predicate device. | Kaplan-Meier plots suggest significant difference in survival curves among different risk groups for MammaPrint FFPE NGS kit, p=0.001. |
| 5-year DRFI: Low Risk 98.1%, Borderline 92.6%, High Risk 88.2% (Table 6). | ||
| 10-year DRFI: Low Risk 95.2%, Borderline 92.6%, High Risk 82.0% (Table 6). | ||
| The results "indicated that both devices show similar clinical performance" based on RASTER study follow-up. |
2. Sample Sizes and Data Provenance
- Method Comparison Study-1: 155 samples used. Data acquired retrospectively as these samples were "previously processed on MammaPrint FFPE microarray as part of routine diagnostics."
- Method Comparison Study-2: 303 samples used. Data acquired retrospectively from "previously collected patient samples."
- Repeatability of RNA Isolation: Not explicitly stated, but implies multiple FFPE tumor blocks, each sectioned twice.
- Reproducibility of Controls: 2 control samples (CTRL-HR, CTRL-LR), each processed 50 times across 4 external sites, 2 operators per site, 6 NGS runs, and 3 lot numbers. Total of 100 measurements for controls.
- Precision/Reproducibility Assessment-Study-1: 3 samples (High Risk, Borderline, Low Risk). Each processed 96 times (4 external sites x 2 operators x 6 NGS runs x 2 duplicates per run).
- Precision/Reproducibility Assessment-Study-2: 8 samples. Each processed repeatedly (from RNA isolation to sequencing) by 2 operators at 3 sites, with 4 runs per site and 2 replicates per run. This results in 16-24 replicates per sample (table indicates N of 16 or 24).
- Detection Limit: 8 FFPE breast cancer samples with "poor" DV200 quality were evaluated, with various dilutions. Additional unstated number of "Standard" DV200 samples.
- Clinical Validation (RASTER Study):
- Original RASTER enrollment: 427 patients.
- Subset of FFPE samples from RASTER study used for NGS kit validation: 345 samples.
- Samples successfully processed for analysis: 316 samples.
- Data provenance: Multicenter observational study conducted in the Netherlands between 2004 and 2006 (for initial RASTER study). The current analysis uses updated 5- and 10-year follow-up data. This is prospective observational data with the device evaluated retrospectively on archived samples.
3. Number of Experts and Qualifications for Ground Truth
The document does not mention the use of experts to establish ground truth for the analytical or reproducibility test sets. The ground truth for these studies is based on comparisons against the predicate device's results or known control sample values.
For the clinical validation, the ground truth for distant recurrence-free interval (DRFI) and breast cancer specific survival (BCSS) is outcomes data from the RASTER study, defined as distant breast cancer recurrence or death from breast cancer. The establishment of these clinical outcomes likely involved clinical experts (e.g., oncologists, pathologists, and medical records review) over the 10-year follow-up period, but the specific number and qualifications of these experts are not detailed in this submission.
4. Adjudication Method for Test Set
The document does not describe an adjudication method (such as 2+1 or 3+1) for the test sets. For analytical studies, agreement is based on direct comparison of results between NGS and microarray platforms or within the NGS platform itself (repeatability/reproducibility). For the clinical study, outcomes (DRFI, BCSS) are established from patient follow-up data.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No MRMC comparative effectiveness study was performed as this device is a gene expression profiling test system, not an imaging device requiring human reader interpretation. The comparison is between the new NGS-based assay and the predicate microarray-based assay.
6. Standalone Performance
The "standalone" performance shown is the analytical performance of the MammaPrint FFPE NGS kit itself (repeatability, reproducibility, detection limit) and its clinical prognostic ability, which is an algorithm-only output (MammaPrint Index). The clinical validation study (RASTER) assesses the algorithm's ability to stratify patients by risk based on actual clinical outcomes. The device performance (result classification: High Risk, Borderline, Low Risk) is reported directly from the algorithm without a human-in-the-loop component for classification.
7. Type of Ground Truth Used
- Analytical Performance (Method Comparison, Repeatability, Reproducibility): The ground truth is effectively the predicate device's results or the known values of control samples. For repeatability/reproducibility, the ground truth is consistency of the device's own output.
- Clinical Validation: The ground truth is outcomes data (Distant Recurrence-Free Interval - DRFI, and Breast Cancer Specific Survival - BCSS) from prospectively collected clinical follow-up in the RASTER study. This is directly observed patient outcomes over 5 and 10 years.
8. Sample Size for the Training Set
The document focuses on the performance of the MammaPrint FFPE NGS kit, which is stated to use an "unchanged" 70-gene signature and scoring algorithm from the predicate device. The original MammaPrint 70-gene signature was developed much earlier. The description states: "This algorithm was designed and programmed by Agendia and incorporated into a proprietary software program, which loads the FASTQ data file." The specifics of the training set for the original MammaPrint algorithm are not provided in this document. This submission is for a new platform (NGS) of an existing and cleared diagnostic system, demonstrating its equivalence, rather than a new algorithm development.
9. How the Ground Truth for the Training Set was Established
As noted above, the original training of the MammaPrint algorithm is not detailed here. The clinical validation in this document uses the RASTER study data, where the ground truth is clinical outcome data (DRFI, BCSS) established through follow-up. This RASTER data is used for validation of the new NGS platform, not for training a new algorithm.
Ask a specific question about this device
(120 days)
Agendia Inc.
MammaPrint® FFPE is a qualitative in vitro diagnostic test, performed in a central laboratory, using the gene expression profile obtained from formalin-fixed paraffin embedded (FFPE) breast cancer tissue samples to assess a patient's risk for distant metastasis within 5 years.
The test is performed for breast cancer patients, with Stage I disease, with tumor size ≤ 5.0 cm and lymph node negative. The MammaPrint FFPE result is indicated for use by physicians as a prognostic marker only, along with other clinicopathological factors.
The MammaPrint service is a microarray-based gene expression analysis of a tumor. The analysis is based on several processes: isolation of RNA from formalin-fixed paraffin embedded (FFPE) tumor tissue sections, DNase treatment of isolated RNA, amplification DNase treated RNA resulting in cDNA, labeling and purification of amplified cDNA, hybridization of the diagnostic microarray, scanning the MammaPrint microarray and data acquisition (Feature Extraction), calculation and determination of the risk of recurrence in breast cancer patients.
The MammaPrint analysis is designed to determine the gene activity of specific genes in a FFPE tissue sample. The result is an expression profile, or fingerprint, of the sample.
The molecular profile of the sample is determined (Low Risk) by calculating the MammaPrint index (MPI) by determining the correlation of the sample expression profile to the mean expression profiles of risk templates of tumors with a known good and poor outcome.
The provided text describes the acceptance criteria and study for the MammaPrint FFPE device, focusing on the analytical performance related to a change in microarray scanner.
Here's the breakdown of the information requested:
1. A table of acceptance criteria and the reported device performance
| Acceptance Criteria Category | Specific Criteria | Reported Device Performance |
|---|---|---|
| Technical Equivalence (Concordance between Scanners) | Passing and Bablok regression for MammaPrint indices between C-scanners (Agilent G2505) and D-scanners (Agilent SureScan Dx G5761AA) must meet pre-defined acceptance criteria (specific values for slope and intercept not explicitly stated, but implied to be near 1 and 0, respectively). | Scanner SG18309119 vs. C-scanners: y=0.00 +1.00 x (95%Cl – slope: 1.000 – 1.002, 95%Cl intercept: -0.0002 to 0.000). Scanner SG18449122 vs. C-scanners: y=0.0010 +1.00 x (95%Cl – slope: 1.000 – 1.0021, 95%Cl intercept: 0.0014 to 0.001). Both results are stated to be "within the pre-defined acceptance criteria." |
| Overall concordance, Negative Percent Agreement (NPA), and Positive Percent Agreement (PPA) for MammaPrint categorical results (High/Low Risk) between C-scanners and D-scanners must be within pre-defined acceptance criteria. | Scanner SG18309119 vs. C-scanners: Overall concordance: 99.7%. NPA: 100% (95%Cl: 98.1-100). PPA: 99.3% (95%Cl: 96.0 – 99.9). Scanner SG18449122 vs. C-scanners: Overall concordance: 100%. NPA: 100% (95%Cl: 97.1 – 100). PPA: 100% (95%Cl: 97.5 - 100). All reported values are stated to be "within the predefined acceptance criteria." | |
| Precision Assessment | F-test p-values for variability of repeated measurements of control samples between C-scanners and D-scanners should not show a significant difference (i.e., p-value > 0.05). | F-test p-values for all four control samples were "all well below the significance level of 0.05, indicating there is no significant difference in precision between C and D-scanners." (This wording is a bit confusing; "well below 0.05" would typically indicate a significant difference. However, given the context of demonstrating no significant difference, it likely implies that the null hypothesis of equal variance could not be rejected, which means the p-value was above 0.05. Re-reading, "indicating there is no significant difference" following "well below the significance level of 0.05" is a logical contradiction. Assuming the intent was to show no significant difference in precision, the p-values should have been above 0.05. This might be a typo in the FDA document, or the interpretation of the F-test result is inversely stated. Given the conclusion that "there is no significant difference," the F-test result must have been non-significant, meaning p > 0.05. Let's assume the intent was to show non-significant difference.) |
| Clinical Correlation | MammaPrint index should correlate with clinical outcome (distant recurrence risk). | Result distribution: In the RASTER study, Bin 1 (MPI 0.36 to +1) had 0% observed 5-yr DR risk (N=37). Bin 4 (MPI -1 to -0.57) had 13.6% observed 5-yr DR risk (N=66). Cox regression analysis: With each increase in MammaPrint index unit, there is a 0.224 (4.5 folds) decrease in recurrence risk at 5 years (p=0.001; 95% Cl: 0.092-0.543). This demonstrates correlation. |
2. Sample size used for the test set and the data provenance (e.g., country of origin of the data, retrospective or prospective)
- Sample Size for Technical Equivalence (Concordance): 92 8-pack arrays (likely representing 92 individual samples, or potentially 92 unique arrays, each containing multiple spots/analyses).
- Sample Size for Precision Assessment: Four diagnostic control samples (PLEP3, PHHE2, PHTR2, PBCL2), measured repeatedly. The number of repeated measurements is not specified.
- Sample Size for Clinical Correlation (RASTER study): 345 patients.
- Data Provenance: Not explicitly stated for analytical performance (concordance and precision). For the clinical correlation, it references the RASTER study, which was "previously submitted in K141142," but country of origin and retrospective/prospective nature are not detailed here.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g., radiologist with 10 years of experience)
- For Analytical Performance: Ground truth is established by the MammaPrint index and categorical results from the FDA-cleared C-scanners (Agilent G2505), acting as the reference. No human experts are involved in establishing this specific ground truth.
- For Clinical Correlation (RASTER study): The clinical outcome (distant recurrence risk) is the ground truth. While not detailed in this excerpt, clinical outcomes studies typically rely on physician diagnoses, follow-up, and potentially pathology reports, rather than a panel of experts specifically adjudicating "ground truth" for the test results.
4. Adjudication method (e.g., 2+1, 3+1, none) for the test set
- For Analytical Performance: None. The ground truth is the output from the predicate device's scanner (Agilent G2505).
- For Clinical Correlation: Not specified in this document. Clinical outcome data collection methods vary, but typically don't involve an adjudication panel for the outcome itself, but rather established clinical trial protocols and follow-up.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
- This is not an MRMC study. The MammaPrint device is a gene expression profiling test, not an AI imaging or diagnostic assistance tool for human readers. It provides a prognostic marker directly from tissue analysis.
6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done
- Yes, this is a standalone device. The MammaPrint FFPE test generates a result (MammaPrint Index and High/Low Risk classification) directly from the gene expression profile of the tumor tissue. The "algorithm only" performance is what is being evaluated and validated. The "human-in-the-loop" component mentioned is the physician's use of the result "along with other clinicopathological factors" for prognostic assessment.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
- For Analytical Performance (concordance and precision): The ground truth is the MammaPrint index and categorical result generated by the previously FDA-cleared C-scanners (Agilent G2505).
- For Clinical Correlation: The ground truth is 5-year distant recurrence (DR) risk, which is a form of outcomes data.
8. The sample size for the training set
- Not explicitly mentioned for the current submission. The MammaPrint assay itself was developed and validated earlier, and the current submission is for a modification (scanner change) to an already cleared predicate device (K141142). The original training set for the MammaPrint algorithm would have been part of the earlier submissions but is not detailed here.
9. How the ground truth for the training set was established
- Not explicitly mentioned in this document. For the original MammaPrint development, the ground truth for training would have involved gene expression profiles correlated with long-term clinical outcomes (e.g., distant metastasis-free survival) from large cohorts of breast cancer patients, likely using robust clinical data and follow-up.
Ask a specific question about this device
(266 days)
AGENDIA
MammaPrint® FFPE is a qualitative in vitro diagnostic test, performed in a central laboratory, using the gene expression profile obtained from formalin-fixed paraffin embedded (FFPE) breast cancer tissue samples to assess a patient's risk for distant metastasis within 5 years.
The test is performed for breast cancer patients, with Stage II disease, with tumor size ≤ 5.0 cm and lymph node negative. The MammaPrint® FFPE result is indicated for use by physicians as a prognostic marker only, along with other clinico-pathological factors.
The MammaPrint® FFPE test is a microarray based gene expression analysis of a tumor. The analysis is based on several processes: isolation of RNA from FFPE breast cancer tissue sections; elimination of gDNA, reverse transcription of RNA resulting in cDNA; amplification of the cDNA, purification and labeling of cDNA; hybridization of the amplified and labeled cDNA to the diagnostic microarray; washing and scanning the diagnostic microarray and data acquisition (feature extraction); calculation and determination of the risk of recurrence.
The MammaPrint® FFPE analysis is designed to determine the expression of specific genes in a tissue sample. The result is an expression profile, or "fingerprint", of the sample. Using this expression profile, the MammaPrint® FFPE Index is calculated and the molecular prognosis profile of the sample is determined (Low Risk, High Risk).
Below is a summary of the acceptance criteria and study details for the Agendia MammaPrint® FFPE device, based on the provided text.
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria for MammaPrint® FFPE are primarily based on demonstrating equivalence to the MammaPrint® Fresh device (K101454) in terms of concordance and clinical performance.
| Test Category | Acceptance Criteria | Reported Device Performance and Confidence Intervals |
|---|---|---|
| Analytical Performance | ||
| Concordance (FFPE vs. Fresh) - 1st Study (n=122) | High concordance between MammaPrint Fresh and FFPE outcomes. | Overall Concordance: 89.34% |
| True Positive/Negative Concordance (non-borderline): 92.7% | ||
| NPA (95% CI): 88.4% (80.5-95) | ||
| PPA (95% CI): 90.6% (79.8-95.9) | ||
| Concordance (FFPE vs. Fresh) - 2nd Study (n=345) | High concordance between MammaPrint Fresh and FFPE outcomes. | Overall Concordance: 89.28% |
| True Positive/Negative Concordance (non-borderline): 93.5% | ||
| NPA (95% CI): 91.5% (86.7-94.7) | ||
| PPA (95% CI): 86.6% (80.4-91.1) | ||
| Intra-sample Reproducibility (4 isolations) | No significant difference in MammaPrint Indices or Outcome across multiple isolations from the same sample. | MammaPrint Indices: p=0.994 (no significant difference) |
| MammaPrint Outcome: p=0.290 (no significant difference) | ||
| Inter-assay Reproducibility (Control samples over time) | Standard deviations of MammaPrint Indices for control samples to meet predefined criteria. | PHTR=0.045 (n=54), PLEP=0.056 (n=52), PHHE=0.072 (n=52) (all passed predefined criteria) |
| Method Precision (Repeatability & Within-lab) | Repeatability and Method Precision results to meet predefined acceptance criteria. | Repeatability (Within-Run): |
| High Risk (1): 0.036 | ||
| High Risk (2): 0.046 | ||
| Low Risk: 0.042 | ||
| Low Risk-Borderline: 0.049 | ||
| Method Precision (Within-Laboratory): | ||
| High Risk (1): 0.044 | ||
| High Risk (2): 0.057 | ||
| Low Risk: 0.050 | ||
| Low Risk-Borderline: 0.066 | ||
| (All met predefined criteria) | ||
| Inter-laboratory Comparison (Irvine vs. Amsterdam) | High agreement in MammaPrint Index and Outcome between laboratories, passing predefined acceptance criteria. | Irvine: Kappa score = 0.90 |
| Amsterdam: Kappa Score = 0.9 | ||
| Intercept close to zero, slope close to 1 (Passing and Bablok regression) | ||
| Microarray Scanner Comparison (Amsterdam) | High agreement in MammaPrint Index and Outcome between different scanners, passing predefined acceptance criteria. | Pearson correlation = 1.0 |
| Kappa score = 1.0 | ||
| NPA (95% CI): 100% (67.6-100) | ||
| PPA (95% CI): 100% (81.6-99.0) | ||
| Intercept close to zero, slope close to 1 | ||
| Microarray Scanner Comparison (Irvine) | High agreement in MammaPrint Index and Outcome between different scanners, passing predefined acceptance criteria. | Pearson correlation = 1.0 |
| Kappa score = 1.0 | ||
| NPA (95% CI): 100% (77.2-100) | ||
| PPA (95% CI): 100% (78.5-100) | ||
| Intercept close to zero, slope close to 1 | ||
| Minimum Input (Dilution Study) | Very stable results even at low input of cDNA expected. | Results showed very stable results even at low input of cDNA. |
| Clinical Performance | ||
| 5-year DRFI (Distant Recurrence Free Interval) | MammaPrint FFPE performance (low and high risk signatures) to fall within the 95% CI of MammaPrint Fresh. | Low Risk Signature: |
| MammaPrint Fresh 2013: 0.976 (0.952-1.000) | ||
| MammaPrint FFPE: 0.977 (0.955-0.999) | ||
| High Risk Signature: | ||
| MammaPrint Fresh 2013: 0.891 (0.840-0.942) | ||
| MammaPrint FFPE: 0.885 (0.830-0.940) | ||
| (All fall within 95% CI) | ||
| 5-year DM1st (Distant Metastasis as first event) | MammaPrint FFPE performance (low and high risk signatures) to fall within the 95% CI of MammaPrint Fresh. | Low Risk Signature: |
| MammaPrint Fresh 2013: 0.976 (0.952-1.000) | ||
| MammaPrint FFPE: 0.977 (0.955-0.999) | ||
| High Risk Signature: | ||
| MammaPrint Fresh 2013: 0.907 (0.860-0.954) | ||
| MammaPrint FFPE: 0.903 (0.854-0.952) | ||
| (All fall within 95% CI) |
2. Sample Size Used for the Test Set and Data Provenance
- Analytical Performance - Concordance Studies:
- First independent validation: n=122 tumor samples (FFPE vs. Fresh from the same tumor). Data provenance is not explicitly stated beyond being "internal" to Agendia but given the global operations might be multi-country. The study appears to be retrospective as it compares processed samples.
- Second independent validation (Raster study): n=345 samples (FFPE tissue vs. Fresh RNA from the same patients). The Raster study is mentioned as the source, which is a clinical study. The geographical origin is not explicitly stated, but clinical studies often involve multiple centers. It's retrospective in the sense that existing samples with follow-up were used.
- Analytical Performance - Reproducibility & Precision:
- Multiple isolations: 30 FFPE samples. Provenance not specified.
- Multiple labeling/hybridizations (control samples): PHTR (n=54), PLEP (n=52), PHHE (n=52) over time. Provenance not specified.
- Precision and Evaluation (P&E): 4 test samples (representing different risk levels) over 20 days. Provenance not specified.
- Analytical Performance - Inter-laboratory Comparison: 25 FFPE samples. Provenance not specified, but involved samples processed in Amsterdam and Irvine laboratories.
- Analytical Performance - Microarray Scanner Validation: 25 samples (Amsterdam), 27 samples (Irvine). Provenance not specified.
- Analytical Performance - Minimum Input: 3 samples and 3 control samples. Provenance not specified.
- Clinical Performance (Comparison with MammaPrint Fresh):
- Raster study: 345 samples with clinical follow-up of 5 years. This data is from the Raster study, which is a clinical prospective study (although the use here for the FFPE comparison would be retrospective on archived samples).
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The document does not specify the number or qualifications of experts used to establish ground truth for the analytical or clinical validation. The "ground truth" for the analytical studies is the result from the established MammaPrint Fresh assay (K101454) for concordance, or statistical assessments of variability for reproducibility/precision. For clinical outcomes, the ground truth is clinical follow-up data (Distant Recurrence Free Interval and Distant Metastasis as first event) over 5 years. This outcome data is typically collected through clinical study follow-up by medical professionals, but not "established by experts" in the sense of a panel review for each case in this context.
4. Adjudication Method for the Test Set
No explicit adjudication method (like 2+1 or 3+1) is mentioned for the test sets. The ground truth for analytical concordance is the result from the predicate device (MammaPrint Fresh), and for clinical performance, it's the 5-year clinical outcome data (DRFI, DM1st) from the Raster study.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done
No MRMC comparative effectiveness study involving human readers is described. This device is a gene expression profiling test performed in a central laboratory, not an image-based diagnostic read by human readers. Therefore, the concept of human readers improving with AI assistance is not applicable in this context.
6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was Done
Yes, the studies described are standalone performance evaluations of the MammaPrint® FFPE algorithm. The "device performance" and "reported device performance" in the table refer to the algorithm's output (Low Risk/High Risk and MammaPrint Index calculation) based on the tissue sample analysis.
7. The Type of Ground Truth Used
- Analytical Performance:
- Concordance: The results from the predicate device, MammaPrint® Fresh (K101454), using fresh tissue samples of the same tumor.
- Reproducibility, Precision, Inter-laboratory, Scanner Validation, Minimum Input: Statistical measures of consistency and variation against predefined internal criteria, using the device's own output.
- Clinical Performance:
- Outcomes Data: 5-year Distant Recurrence Free Interval (DRFI) and 5-year Distant Metastasis as first event (DM1st) collected during the Raster study.
8. The Sample Size for the Training Set
The document does not explicitly state the sample size for a training set for the MammaPrint® FFPE algorithm. The MammaPrint® technology was likely developed and validated using significant cohorts prior to this specific submission, which focuses on extending its use to FFPE samples and demonstrating equivalence. The "Raster study" is mentioned for clinical evaluation, but it's not explicitly stated as a training set. The descriptions focus on the validation of the FFPE version against the established Fresh version and clinical outcomes.
9. How the Ground Truth for the Training Set Was Established
Since a dedicated training set and its ground truth establishment are not described, this information cannot be provided from the given text. MammaPrint is a gene expression profiling test, and the "ground truth" for its original development (not necessarily a "training set" in the machine learning sense for this submission) would typically involve correlating gene expression profiles with long-term clinical outcomes in large patient cohorts to define the prognostic signature. This submission focuses on validating the FFPE version against the already cleared predicate device (MammaPrint Fresh).
Ask a specific question about this device
(248 days)
AGENDIA
MammaPrint is a qualitative in vitro diagnostic test service, performed in a central laboratory, using the gene expression profile of fresh breast cancer tissue samples to assess a patient's risk for distant metastasis (up to 10 years for patients less than 61 years old, up to 5 years for patients' ≥ 61 years).
The test is performed for breast cancer patients with Stage I or Stage II disease, with tumor size
The MammaPrint service is a microarray based gene expression analysis of a tumor. The analysis is based on several processes: isolation of RNA from frozen tumor tissue sections. DNA'se treatment of isolated RNA, linear amplification and labeling of DNA'se treated RNA, cRNA purification, hybridization of the cRNA to the MammaPrint microarray, scanning the MammaPrint microarray and data acquisition (feature extraction), calculation and determination of the risk of recurrence in breast cancer patients.
The MammaPrint analysis is designed to determine the gene activity of specific genes in a tissue sample compared to a reference standard. The result is an expression profile, or fingerprint, of the sample.
The correlation of the sample expression profile to a template (the mean expression profile of 44 tumors with a known good clinical outcome) is calculated and the molecular profile of the sample is determined (Low Risk, High Risk).
The MammaPrint device is a qualitative in vitro diagnostic test service that assesses a patient's risk for distant metastasis in breast cancer. The study presented focuses on the analytical performance and reproducibility of the device, particularly when new scanners and bio-analyzers are used, and when the service is performed in different central laboratory sites.
Here's a breakdown of the requested information:
1. Acceptance Criteria and Reported Device Performance
The provided document describes analytical performance evaluation rather than a clinical study with specific diagnostic accuracy metrics (like sensitivity, specificity, AUC). The acceptance criteria are based on statistical comparisons to the predicate device's accepted variance and predetermined validation acceptance criteria.
| Acceptance Criteria Category | Specific Acceptance Criteria | Reported Device Performance |
|---|---|---|
| Micro Array Scanners | Difference between mean, median, and standard deviation of MammaPrint Indices (between new and predicate scanners) falls within 1.96 * 0.030 (accepted variance of predicate device). | Differences fall within the accepted variance of the predicate device (1.96 * 0.030). |
| Bio Analyzers (RIN Measurements) | No significant difference in RNA Integrity Number (RIN) measurements between FDA-cleared and new Bio-analyzers (p-value for statistical test should indicate no significant difference). | Wilcoxon signed ranks test showed no significant difference in RIN measurements between FDA cleared and New Bioanalyzers (p=0.46 and p=0.47 respectively). |
| Central Laboratory Site Comparison (RNA Quality/RIN) | No significant difference in RNA quality (RIN measurement) between Amsterdam (L1) and US lab (L2). All results must comply with predefined validation acceptance criteria. | No significant difference in RNA quality of RIN measurement between Amsterdam (L1) and US lab (L2). All results comply with predefined validation acceptance criteria. |
| Central Laboratory Site Comparison (MammaPrint Index/Outcome) | No significant difference in MammaPrint Indices and outcome between European/Dutch (L1) and US/California (L2) lab. All results must comply with predefined validation acceptance criteria. | No significant difference in MammaPrint Indices between European/Dutch (L1) and US/California (L2) lab. All results comply with predefined validation acceptance criteria. |
2. Sample Size and Data Provenance
The study focused on analytical validation rather than a clinical diagnostic accuracy study.
- Micro Array Scanners: Used a selection of 25 slides from which MammaPrint Indices were generated. These slides consisted of approximately 100 samples and 20 control samples (LRC and HRC). The samples were analyzed during regular diagnostics. The provenance (country of origin) is not explicitly stated for these specific samples, but the company is based in the Netherlands with a US lab mentioned later. The samples were likely retrospective as they were previously run for diagnostic purposes.
- Bio Analyzers: A selection of about 60 samples covering the complete RIN measuring range. Data provenance is not explicitly stated.
- Central Laboratory Sites (RNA Isolation): 36 samples selected based on sufficient tissue material. The samples had previously shown acceptable RNA quality at the Amsterdam lab.
- Central Laboratory Sites (Amplification/Labeling and Hybridization): 99 samples were used. All samples had been previously subjected to a diagnostic MammaPrint test at the Amsterdam Lab (Lab 1). The result distribution was 54 high risk, 38 low risk, and 7 borderline.
- Data Provenance: The central laboratories include Amsterdam (Netherlands) and a US laboratory (California). The samples used for comparison were run in both labs. This implies a retrospective analysis of samples previously processed.
3. Number of Experts and Qualifications for Ground Truth
The document does not describe a process where experts established a ground truth for a test set in the traditional sense of diagnostic accuracy studies (e.g., radiologists reviewing images). The ground truth for the analytical validation appears to be the results obtained from the predicate device or a previously established reference standard within Agendia's own validated processes.
4. Adjudication Method
Not applicable for this type of analytical validation study. There was no expert adjudication process for establishing a "ground truth" for a test set. The comparisons were statistical analyses of measurements (MammaPrint Indices, RIN values) between different instrumentations or lab sites.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No, this document does not describe a MRMC comparative effectiveness study involving human readers. This is an analytical validation of the device's technical performance and reproducibility, not a study on how human interpretation improves with AI assistance.
6. Standalone (Algorithm Only) Performance
Yes, the study describes the standalone performance of the MammaPrint algorithm, focusing on the analytical performance characteristics like precision and reproducibility across different hardware and lab sites. It evaluates the consistency of the algorithm's output (MammaPrint Index, RIN) under various conditions.
7. Type of Ground Truth Used
The "ground truth" in this context refers to the established and accepted performance of the predicate device and the existing, validated MammaPrint procedures. For instance:
- For scanner validation, the MammaPrint Indices generated by the FDA-cleared predicate scanners served as the reference.
- For bio-analyzer validation, the RIN measurements from FDA-cleared bio-analyzers served as the reference.
- For central lab site comparison, the results from the Amsterdam lab (L1), where samples had been previously subjected to diagnostic MammaPrint, served as the reference for comparison with the US lab (L2).
This is an internal reference standard/predicate device comparison.
8. Sample Size for the Training Set
The document does not describe a training set for a machine learning algorithm. MammaPrint is a gene expression profiling test, which typically relies on a fixed algorithm/signature derived from earlier research, not an adaptive machine learning model that is "trained" in an ongoing manner. The document focuses on the analytical validation of the test's execution and reproducibility.
9. How the Ground Truth for the Training Set Was Established
Not applicable, as there is no mention of a training set for a machine learning algorithm in the provided text. The MammaPrint test itself is based on a specific gene expression profile correlated to clinical outcome. The methods for establishing this original correlation (which forms the basis of the "template" mentioned in the device description) would be found in the foundational clinical studies for MammaPrint, not in this analytical validation document. The "template" (mean expression profile of 44 tumors with a known good clinical outcome) explicitly mentioned in the device description is effectively the core of their model, derived from prior clinical data.
Ask a specific question about this device
(603 days)
AGENDIA
MammaPrint® is a gualitative in vitro diagnostic test service, performed in a single laboratory, using the gene expression profile of fresh breast cancer tissue samples to assess a patients' risk for distant metastasis (up to 10 years for patients less than 61 years old, up to 5 years for patients ≥ 61 years).
The test is performed for breast cancer patients with Stage I or Stage II disease, with a tumor size of ≤ 5.0 cm and lymph node negative. The MammaPrint result is indicated for use by physicians as a prognostic marker only, along with other clinicopathological factors.
The MammaPrint service is a microarray based gene expression analysis of a tumor. The analysis is based on several processes: isolation of RNA from frozen tumor tissue sections, DNA'se treatment of isolated RNA, linear amplification and labeling of DNA'se treated RNA, cRNA purification, hybridization of the cRNA to the MammaPrint microarray, scanning the MammaPrint microarray and data acquisition (feature extraction), index calculation and determination of the risk of distant recurrence in breast cancer patients.
The MammaPrint analysis is designed to determine the gene activity of specific genes in a tissue sample compared to a reference standard. The result is an expression profile, or fingerprint, of the sample.
The correlation of the sample expression profile to a template (the mean expression profile of 44 tumors with a known good clinical outcome) is calculated and the molecular profile index of the sample is determined (Low Risk, High Risk).
Here's a breakdown of the acceptance criteria and the study information for the MammaPrint® device, based on the provided text:
MammaPrint® Acceptance Criteria and Study Information
1. Acceptance Criteria and Reported Device Performance
The provided document primarily focuses on analytical performance as internally validated and clinical performance as demonstrated through peer-reviewed studies. It doesn't explicitly state "acceptance criteria" in a typical pass/fail numerical sense for clinical performance, but rather presents the performance achieved by the device in various clinical settings.
| Acceptance Criteria Category | Specific Metric/Description | Reported Device Performance/Finding |
|---|---|---|
| Analytical Performance | Accuracy of measurement (based on repeated experiments of control samples) | 98.5% Analytical Accuracy of measurement |
| Accuracy of classifying a sample as High Risk or Low Risk | At least 98.9% (i.e., 1.1% false negative classification) | |
| Percentage of "Borderline Samples" | Less than 5% of analyzed samples are considered "Borderline Samples" | |
| Classification accuracy for "Borderline Samples" | Approximately 90% (i.e., 10% chance of false classification) | |
| Clinical Performance | **Assessment of risk for distant metastasis (up to 10 years for patients |
Ask a specific question about this device
(172 days)
AGENDIA BV
MammaPrint is a qualitative in vitro diagnostic test service, performed in a single laboratory, using the gene expression profile of fresh breast cancer tissue samples to assess a patient's risk for distant metastasis.
The test is performed for breast cancer patients who are less than 61 years old, with Stage I or Stage II disease, with tumor size
The MammaPrint service is a microarray based gene expression analysis of a tumor. The analysis is based on several processes: isolation of RNA from frozen turnor tissue sections, DNA'se treatment of isolated RNA, linear amplification and labeling of DNA'se treated RNA, cRNA purification, hybridization of the cRNA to the MammaPrint microarray, scanning the MammaPrint microarray and data acquisition (feature extraction), calculation and determination of the risk of recurrence in breast cancer patients.
The MammaPrint analysis is designed to determine the gene activity of specific genes in a tissue sample compared to a reference standard. The result is an expression profile, or fingerprint, of the sample.
The correlation of the sample expression profile to a template (the mean expression profile of 44 tumors with a known good clinical outcome) is calculated and the molecular profile of the sample is determined (Low Risk, High Risk).
Here's a breakdown of the acceptance criteria and the study information for the MammaPrint® device, based on the provided text:
MammaPrint® Device Performance Study Analysis
1. Acceptance Criteria and Reported Device Performance
The provided document primarily focuses on demonstrating substantial equivalence to a predicate device (Agendia BV's MammaPrint (K070675)) rather than setting explicit, quantifiable clinical acceptance criteria for de novo approval. However, it does present analytical performance targets relative to the predicate and clinical outcomes from supporting studies.
Here's a table summarizing the implicit acceptance criteria (based on predicate performance) and the reported device performance from the analytical studies:
| Acceptance Criteria / Performance Metric | Target (derived from predicate or equivalency) | Reported Device Performance (New MammaPrint HD) |
|---|---|---|
| Analytical Performance | ||
| Repeatability and Precision (MammaPrint index Standard Deviation) | At least as good as predicate (≤ 0.030) | At least as good as predicate (0.030) |
| Repeatability and Precision (MammaPrint index Variance) | At least as good as predicate (≤ 0.001) | At least as good as predicate (0.001) |
| Concordance in MammaPrint outcome (HD vs. LD microarray) | Within 97.7% technical accuracy of predicate | 98.9% |
| MammaPrint Index difference between scanners (mean, median, standard deviation) | Within predicate's accepted variance (1.96 * 0.030) | Falls within accepted variance |
| Accuracy of classifying as High Risk or Low Risk | Better than predicate (97.7% accuracy, 1.1% false negative) | 98.9% (0.5% false negative classification) |
| Percentage of "Borderline Samples" | Not explicitly stated as a target, but reported for context | Less than 5% |
| "Borderline Samples" Classification Accuracy | Not explicitly stated as a target, but reported for context | Approximately 90% (10% chance of false classification) |
2. Sample Size and Data Provenance for Test Set (Analytical Performance)
- Precision Evaluation (Method Validation):
- Sample Size: 3 samples with different outcome levels (high, low, borderline) were run repeatedly over 20 days. Two replicates per sample level per day. Also included High Risk Control (HRC) and Low Risk Control (LRC). Total 120 (3 samples x 2 replicates x 20 days), plus controls.
- Data Provenance: Not explicitly stated, but likely internal laboratory data from Agendia BV. Retrospective, as these were established samples.
- LD and HD Microarray Comparison:
- Sample Size: 98 historical MammaPrint service samples.
- Data Provenance: Samples from the period 2004 through 2007. Origin country is not explicitly stated in this section, but the company is based in the Netherlands. This is retrospective data.
- Scanner Performance Comparison:
- Sample Size: 26 newly hybridized slides (representing 104 samples). This included three samples with varying risk profiles, repeated, and LRC/HRC controls.
- Data Provenance: Likely internal laboratory data from Agendia BV. Retrospective.
For Clinical Performance, the document refers to four published studies. These studies serve as the clinical evidence for the device's utility, rather than a single "test set" in the context of analytical validation. The details provided are for each study:
- Nature Paper (1): 78 patients
- NEJM Paper (2): 151 patients
- MammaPrint Paper (3): Not a clinical study for patient outcomes, focuses on reproducibility of (1) and (2).
- Transbig Paper (4): 302 patients
The provenance of these clinical studies would generally be international (e.g., European for Transbig). All are retrospective analyses of patient cohorts.
3. Number of Experts and Qualifications for Ground Truth (Test Set)
For the analytical performance studies, "ground truth" refers to the known characteristics of the samples (e.g., whether they were designed to be high, low, or borderline risk, or previously classified by the predicate device). No human experts are used to establish ground truth in these analytical comparisons; instead, it relies on the pre-determined characteristics of the control samples or the results from the predicate device.
For the extensive clinical studies referenced (Nature, NEJM, Transbig), the ground truth for patient outcomes (e.g., metastasis-free survival) would have been established through long-term clinical follow-up by medical professionals, including oncologists and pathologists. The number and specific qualifications of these experts are not detailed in this 510(k) summary, as it refers to published peer-reviewed literature. Typically, these studies involve panels of clinicians (e.g., pathologists, oncologists) determining initial diagnosis and patient outcomes.
4. Adjudication Method for the Test Set
- Analytical Performance: No adjudication method as human interpretation is not involved in comparing array results or scanner outputs. The comparisons are quantitative.
- Clinical Performance (Referenced Studies): The 510(k) summary does not detail the specific adjudication methods used in the referenced clinical studies (e.g., for outcome endpoints like metastasis). Such details would typically be found within the full publications.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No MRMC study is mentioned or implied. MammaPrint is a gene expression profiling test, which is an automated, quantitative assay. Human readers do not directly interpret the "cases" (gene expression profiles) in the same way they would an image. The device provides a quantitative risk score (Low Risk, High Risk) that physicians use to make decisions, but the device itself does not involve human interpretation for its output. Therefore, a study demonstrating how human readers improve with AI vs. without AI assistance is not applicable here.
6. Standalone Performance Study
Yes, the entire submission describes the standalone performance of the MammaPrint device (the algorithm/system itself) in classifying tissue samples as High Risk or Low Risk based on gene expression profiles. The analytical performance metrics (precision, reproducibility, classification accuracy) are all measures of the algorithm's performance in isolation from human interpretation.
7. Type of Ground Truth Used
- Analytical Performance:
- Known outcome levels for control samples (high, low, borderline).
- Results from the predicate device (FDA cleared Low Density microarray).
- Clinical Performance (Referenced Studies):
- Long-term clinical follow-up data: Metastasis-free survival.
- Clinicopathological factors (e.g., tumor size, lymph node status, age).
8. Sample Size for the Training Set
The document mentions: "The correlation of the sample expression profile to a template (the mean expression profile of 44 tumors with a known good clinical outcome) is calculated and the molecular profile of the sample is determined (Low Risk, High Risk)." This "template" of 44 tumors appears to be a core component of the algorithm's development.
The broader clinical studies referenced (e.g., Nature Paper (1) with 78 patients) were likely instrumental in the initial development and refinement (training) of the 70-gene profile itself. The MammaPrint Paper (3) describes the "Development of MammaPrint" and "Converting a breast cancer microarray signature into a high-throughput diagnostic test," implying further training and optimization steps using data from earlier studies.
Therefore, the initial "training set" for defining the gene profile involved patient cohorts (e.g., the 78 patients in the Nature paper, or the 44 specific "good clinical outcome" tumors).
9. How the Ground Truth for the Training Set Was Established
For the "44 tumors with a known good clinical outcome" that form the template, the "known good clinical outcome" would have been established through:
- Long-term clinical follow-up: Patients whose tumors formed this template would have had a prolonged period without recurrence or distant metastasis.
- Clinicopathological data: Standard clinical assessments (e.g., pathology reports, surgical findings) would have confirmed the characteristics of these tumors and the patients' disease stage.
For the broader training of the 70-gene profile, as implied by the referenced papers, the ground truth was established by:
- Clinical outcome data: Patients were followed for several years (e.g., 5-10 years) to determine their metastasis-free survival or overall survival.
- Pathology: Original biopsy/surgical pathology diagnoses, including tumor type, grade, and lymph node status, were used.
- Expert Consensus: While not explicitly stated for individual training cases in this summary, the clinical outcomes and pathological assessments that defined "good clinical outcome" and "poor clinical outcome" would have been made by qualified medical professionals (pathologists, oncologists).
Ask a specific question about this device
(102 days)
AGENDIA BV
MammaPrint® is a qualitative in vitro diagnostic test service, performed in a single laboratory, using the gene expression profile of fresh breast cancer tissue samples to assess a patients' risk for distant metastasis.
The test is performed for breast cancer patients who are less than 61 years old, with Stage I or Stage II disease, with tumor size ≤ 5.0 cm and lymph node negative. The MammaPrint® result is indicated for use by physicians as a prognostic marker only, along with other clinicopathological factors.
The MammaPrint service is a microarray based gene expression analysis of a tumor. The analysis is based on several processes: using fresh tissue stored in RNAlater, isolation of RNA from frozen tumor tissue sections, DNA'se treatment of isolated RNA, linear amplification and labeling of DNA'se treated RNA, cRNA purification, hybridization of the cRNA to the MammaPrint microarray, scanning the MammaPrint microarray and data acquisition (feature extraction), calculation and determination of the risk of recurrence in breast cancer patients.
The MammaPrint analysis is designed to determine the gene activity of specific genes in a tissue sample compared to a reference standard. The result is an expression profile, or fingerprint, of the sample.
The correlation of the sample expression profile to a template (the mean expression profile of 44 tumors with a known good clinical outcome) is calculated and the molecular profile of the sample is determined (Low Risk, High Risk).
This document describes the MammaPrint® device, a gene expression profiling test system for breast cancer prognosis.
Acceptance Criteria and Device Performance
| Criteria | Reported Device Performance |
|---|---|
| Analytical Accuracy | 98.5% (based on 190+ independent analyses of control samples) |
| Classification Accuracy | 97.7% for High Risk/Low Risk classification |
| Borderline Samples | Less than 5% of analyzed samples |
| Repeatability | Considered Precise, Reproducible, Sensitive, Specific, Accurate, and Robust |
Study Details
2. Sample Size and Data Provenance
- Training Set Sample Size:
- Nature Paper (1): 78 patients
- NEJM Paper (2): 151 patients
- Validation Set Sample Size:
- NEJM Paper (2): 151 patients (used for both training and validation)
- Transbig Paper (4): 302 patients
- Data Provenance: Not explicitly stated, but the studies mention "independent European validation" for the Transbig Paper, implying data from Europe. The "comprehensive three-way inter-laboratory comparison study between three independent laboratories in three different countries (Dutch, French and U.S.)" indicates data from these countries for analytical performance, but not specifically for clinical studies. Retrospective or prospective nature is not specified, but the "time frame" for clinical studies (e.g., 2002, 2006) and the reporting of 5-year and 10-year metastasis-free survival strongly suggest retrospective analysis of existing patient cohorts.
3. Number of Experts and Qualifications for Ground Truth
- Not explicitly stated for the clinical studies. The clinical performance is based on outcomes data (metastasis-free survival) rather than expert consensus on individual cases.
4. Adjudication Method for Test Set
- Not applicable as the ground truth is based on clinical outcomes (metastasis-free survival) rather than expert consensus requiring adjudication.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- No, a multi-reader multi-case (MRMC) comparative effectiveness study was not performed or mentioned in the provided text. The device is a diagnostic test, not an AI-assisted interpretation tool for human readers.
6. Standalone Performance Study
- Yes, the performance reported is a standalone (algorithm only) performance. The MammaPrint is a gene expression analysis that directly provides a risk classification (High Risk or Low Risk) based on the gene activity profile. The "Classification performance" states "the accuracy of classifying a sample as High Risk or Low Risk, is 97.7% (i.e., 1.1% false negative classification)," which refers to the algorithm's performance.
7. Type of Ground Truth Used for Clinical Studies
- Outcomes Data: The clinical studies (Nature Paper, NEJM Paper, Transbig Paper) used clinical outcomes data, specifically metastasis-free survival at 5 and 10 years, as the ground truth for validating the prognostic ability of the MammaPrint profile. This is indicated by phrases like "Metastasis-free survival by profile at 10 yrs."
8. Sample Size for the Training Set
- As detailed in Section 2, the training set sizes were:
- Nature Paper (1): 78 patients
- NEJM Paper (2): 151 patients (also used for validation)
9. How the Ground Truth for the Training Set Was Established
- The ground truth for the training set (and validation set) was established using clinical outcomes data, specifically the occurrence of distant metastasis over time. The studies measured "metastasis-free survival" in patient cohorts. This implies that the initial patient samples were analyzed by MammaPrint, and then patients were followed over several years to observe their clinical outcome regarding metastasis.
Ask a specific question about this device
(7 days)
AGENDIA BV
MammaPrint® is a qualitative in vitro diagnostic test service, performed in a single laboratory, using the gene expression profile of fresh frozen breast cancer tissue samples to assess a patients' risk for distant metastasis.
The test is performed for breast cancer patients who are less than 61 years old, with Stage I or Stage II disease, with tumor size ≤ 5.0 cm and who are lymph node negative. The MammaPrint® result is indicated for use by physicians as a prognostic marker only. along with other clinicopathological factors.
The MammaPrint® test is performed and provided as a service by Agendia Laboratory. The test is a microarray based gene expression analysis of RNA extracted from breast tumor tissue. The test is a custom-designed array chip manufactured by Agilent Technologies using the Agilent oligonucleotide microarray platform which assesses the mRNA expression of the 70 genes in triplicate. The MammaPrint® microarray features eight 1900-feature subarrays per glass slide which can each be individually hybridized. Per subarray 232 reporter genes are printed in triplicate, including the 70 genes which make up the MammaPrint® prognostic profile. Each subarray additionally includes 915 normalization genes and 289 spots for hybridization and printing quality control.
The analysis is based on several processes: isolation of RNA from frozen tumor tissue sections, DNAse treatment of isolated RNA, linear amplification and labeling of DNAse treated RNA, cRNA purification, hybridization of the cRNA to the MammaPrint® microaray, scanning the MammaPrint® microarray and data acquisition (feature extraction), calculation and determination of the risk of recurrence in breast cancer patients.
The MammaPrint® analysis is designed to determine the gene activity of specific genes in a tissue sample compared to a reference standard. The result is an expression profile, or fingerprint, of the sample. The correlation of the sample expression profile to a template (the mean expression profile of 44 tumors with a known good clinical outcome) is calculated and the molecular profile of the sample is determined (Low Risk, High Risk, Low Risk Borderline, High Risk Borderline).
Here's a detailed breakdown of the acceptance criteria and the study that proves the MammaPrint® device meets them, based on the provided text:
Acceptance Criteria and Reported Device Performance
| Acceptance Criteria Category | Specific Metric | Reported Device Performance | Study Proving Performance |
|---|---|---|---|
| Prognostic Performance (5-year) | Negative Predictive Value (NPV) for distant metastasis | 0.95 (0.91-0.99) | TRANSBIG Study (Independent European Validation) |
| Prognostic Performance (5-year) | Positive Predictive Value (PPV) for distant metastasis | 0.22 (0.16-0.28) | TRANSBIG Study |
| Prognostic Performance (5-year) | Low Risk Group: Metastasis-free survival probability | 0.95 | TRANSBIG Study (Kaplan-Meier data) |
| Prognostic Performance (5-year) | High Risk Group: Metastasis-free survival probability | 0.78 | TRANSBIG Study (Kaplan-Meier data) |
| Prognostic Performance (10-year) | Negative Predictive Value (NPV) for distant metastasis | 0.90 (0.85-0.96) | TRANSBIG Study |
| Prognostic Performance (10-year) | Positive Predictive Value (PPV) for distant metastasis | 0.29 (0.22-0.35) | TRANSBIG Study |
| Prognostic Performance (10-year) | Low Risk Group: Metastasis-free survival probability | 0.90 | TRANSBIG Study (Kaplan-Meier data) |
| Prognostic Performance (10-year) | High Risk Group: Metastasis-free survival probability | 0.71 | TRANSBIG Study (Kaplan-Meier data) |
| Analytical Reproducibility (RNA pool) | MammaPrint Index SD (LRC pool) | 0.026 | Internal reproducibility study from RNA pool |
| Analytical Reproducibility (RNA pool) | MammaPrint Index SD (HRC pool) | 0.02747 | Internal reproducibility study from RNA pool |
| Analytical Reproducibility (RNA pool) | MammaPrint Index SD (BLS sample) | 0.031 | Internal reproducibility study from RNA pool |
| Repeatability (Individual patients) | Intra-class correlation coefficient (ICC) | 0.9953 | Internal repeatability study (46 patients) |
| Repeatability (Hybridization) | MammaPrint Index SD (one labeled sample, 8 hybridizations) | 0.020 | Internal repeatability study (one labeled sample) |
| Tissue % cut-off | Minimum tumor cell content required | 30% tumor epithelial cells | Internal validation experiment (adapted from original 50% cutoff) |
Study Details for Clinical Performance (TRANSBIG Study)
2. Sample Size Used for the Test Set and Data Provenance:
- Sample Size: 302 patients
- Data Provenance: Retrospective, from five European centers (Villejuif, France; Stockholm, Sweden; Saint-Cloud, France; London, U.K.; Oxford, U.K.). No US patients were included.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications:
The document does not explicitly state the number or specific qualifications of experts used to establish the "ground truth" (i.e., metastasis events or metastasis-free survival) for the TRANSBIG test set. However, it mentions:
- "Paraffin embedded tumor samples from this validation series were independently evaluated." This implies a pathologist, but no specific number or qualifications are given.
- Clinical outcome (metastatic disease within 5 or 10 years) is an objective outcome and generally requires clinical follow-up data, not primary expert interpretation of the initial diagnostic material in the way an imaging study would.
4. Adjudication Method for the Test Set:
The document does not explicitly describe an adjudication method (like 2+1 or 3+1 consensus) for establishing the ground truth outcomes (metastasis events) in the TRANSBIG study. The outcome data appears to be based on observed clinical events over time.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and the Effect Size:
- No, an MRMC comparative effectiveness study was not done in the context of comparing human readers with and without AI assistance.
- The MammaPrint® is a standalone gene expression profiling test, not a tool for human readers to interpret clinical images or data with AI assistance.
- The study did report an unadjusted hazard ratio for time to distant metastases of 2.32 (95% CI: 1.35 to 4.00), which reflects the standalone performance of the gene signature in predicting outcomes, not its assistive effect on human readers. When adjusted for clinical factors (Adjuvant online), the hazard ratio was 2.13 (95% CI: 1.19 to 3.82). This indicates the gene signature provides additional prognostic information beyond standard clinical factors.
6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Was Done:
- Yes, a standalone performance study was done. The entire clinical validation (TRANSBIG study) describes the performance of the MammaPrint® algorithm (the 70-gene signature) without human interpretation in the loop. The device directly outputs a "Low Risk" or "High Risk" classification. The hazard ratios and NPV/PPV values directly reflect this standalone performance.
7. The Type of Ground Truth Used:
- The primary ground truth for the clinical studies (especially TRANSBIG) was outcomes data and pathology. Specifically, the ground truth was the occurrence or non-occurrence of distant metastasis within specific timeframes (5 and 10 years) and the pathology confirmation of breast cancer.
8. The Sample Size for the Training Set:
- Nature Paper (1): 78 patients
- This paper describes the "Development of breast cancer prognosis 70-gene profile."
9. How the Ground Truth for the Training Set Was Established:
- The Nature paper (1) description states it was for "Development of breast cancer prognosis 70-gene profile." While the specific methodology for establishing ground truth isn't detailed in this summary, typically for such studies, it would involve adjudicated clinical outcomes (e.g., metastasis-free survival status) confirmed through patient follow-up and medical records, often correlating with pathology reports for initial diagnosis and tumor characteristics. The mention of "metastasis risk" directly implies reliance on long-term clinical outcome data.
Ask a specific question about this device
Page 1 of 1