(248 days)
MammaPrint is a qualitative in vitro diagnostic test service, performed in a central laboratory, using the gene expression profile of fresh breast cancer tissue samples to assess a patient's risk for distant metastasis (up to 10 years for patients less than 61 years old, up to 5 years for patients' ≥ 61 years).
The test is performed for breast cancer patients with Stage I or Stage II disease, with tumor size
The MammaPrint service is a microarray based gene expression analysis of a tumor. The analysis is based on several processes: isolation of RNA from frozen tumor tissue sections. DNA'se treatment of isolated RNA, linear amplification and labeling of DNA'se treated RNA, cRNA purification, hybridization of the cRNA to the MammaPrint microarray, scanning the MammaPrint microarray and data acquisition (feature extraction), calculation and determination of the risk of recurrence in breast cancer patients.
The MammaPrint analysis is designed to determine the gene activity of specific genes in a tissue sample compared to a reference standard. The result is an expression profile, or fingerprint, of the sample.
The correlation of the sample expression profile to a template (the mean expression profile of 44 tumors with a known good clinical outcome) is calculated and the molecular profile of the sample is determined (Low Risk, High Risk).
The MammaPrint device is a qualitative in vitro diagnostic test service that assesses a patient's risk for distant metastasis in breast cancer. The study presented focuses on the analytical performance and reproducibility of the device, particularly when new scanners and bio-analyzers are used, and when the service is performed in different central laboratory sites.
Here's a breakdown of the requested information:
1. Acceptance Criteria and Reported Device Performance
The provided document describes analytical performance evaluation rather than a clinical study with specific diagnostic accuracy metrics (like sensitivity, specificity, AUC). The acceptance criteria are based on statistical comparisons to the predicate device's accepted variance and predetermined validation acceptance criteria.
| Acceptance Criteria Category | Specific Acceptance Criteria | Reported Device Performance |
|---|---|---|
| Micro Array Scanners | Difference between mean, median, and standard deviation of MammaPrint Indices (between new and predicate scanners) falls within 1.96 * 0.030 (accepted variance of predicate device). | Differences fall within the accepted variance of the predicate device (1.96 * 0.030). |
| Bio Analyzers (RIN Measurements) | No significant difference in RNA Integrity Number (RIN) measurements between FDA-cleared and new Bio-analyzers (p-value for statistical test should indicate no significant difference). | Wilcoxon signed ranks test showed no significant difference in RIN measurements between FDA cleared and New Bioanalyzers (p=0.46 and p=0.47 respectively). |
| Central Laboratory Site Comparison (RNA Quality/RIN) | No significant difference in RNA quality (RIN measurement) between Amsterdam (L1) and US lab (L2). All results must comply with predefined validation acceptance criteria. | No significant difference in RNA quality of RIN measurement between Amsterdam (L1) and US lab (L2). All results comply with predefined validation acceptance criteria. |
| Central Laboratory Site Comparison (MammaPrint Index/Outcome) | No significant difference in MammaPrint Indices and outcome between European/Dutch (L1) and US/California (L2) lab. All results must comply with predefined validation acceptance criteria. | No significant difference in MammaPrint Indices between European/Dutch (L1) and US/California (L2) lab. All results comply with predefined validation acceptance criteria. |
2. Sample Size and Data Provenance
The study focused on analytical validation rather than a clinical diagnostic accuracy study.
- Micro Array Scanners: Used a selection of 25 slides from which MammaPrint Indices were generated. These slides consisted of approximately 100 samples and 20 control samples (LRC and HRC). The samples were analyzed during regular diagnostics. The provenance (country of origin) is not explicitly stated for these specific samples, but the company is based in the Netherlands with a US lab mentioned later. The samples were likely retrospective as they were previously run for diagnostic purposes.
- Bio Analyzers: A selection of about 60 samples covering the complete RIN measuring range. Data provenance is not explicitly stated.
- Central Laboratory Sites (RNA Isolation): 36 samples selected based on sufficient tissue material. The samples had previously shown acceptable RNA quality at the Amsterdam lab.
- Central Laboratory Sites (Amplification/Labeling and Hybridization): 99 samples were used. All samples had been previously subjected to a diagnostic MammaPrint test at the Amsterdam Lab (Lab 1). The result distribution was 54 high risk, 38 low risk, and 7 borderline.
- Data Provenance: The central laboratories include Amsterdam (Netherlands) and a US laboratory (California). The samples used for comparison were run in both labs. This implies a retrospective analysis of samples previously processed.
3. Number of Experts and Qualifications for Ground Truth
The document does not describe a process where experts established a ground truth for a test set in the traditional sense of diagnostic accuracy studies (e.g., radiologists reviewing images). The ground truth for the analytical validation appears to be the results obtained from the predicate device or a previously established reference standard within Agendia's own validated processes.
4. Adjudication Method
Not applicable for this type of analytical validation study. There was no expert adjudication process for establishing a "ground truth" for a test set. The comparisons were statistical analyses of measurements (MammaPrint Indices, RIN values) between different instrumentations or lab sites.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No, this document does not describe a MRMC comparative effectiveness study involving human readers. This is an analytical validation of the device's technical performance and reproducibility, not a study on how human interpretation improves with AI assistance.
6. Standalone (Algorithm Only) Performance
Yes, the study describes the standalone performance of the MammaPrint algorithm, focusing on the analytical performance characteristics like precision and reproducibility across different hardware and lab sites. It evaluates the consistency of the algorithm's output (MammaPrint Index, RIN) under various conditions.
7. Type of Ground Truth Used
The "ground truth" in this context refers to the established and accepted performance of the predicate device and the existing, validated MammaPrint procedures. For instance:
- For scanner validation, the MammaPrint Indices generated by the FDA-cleared predicate scanners served as the reference.
- For bio-analyzer validation, the RIN measurements from FDA-cleared bio-analyzers served as the reference.
- For central lab site comparison, the results from the Amsterdam lab (L1), where samples had been previously subjected to diagnostic MammaPrint, served as the reference for comparison with the US lab (L2).
This is an internal reference standard/predicate device comparison.
8. Sample Size for the Training Set
The document does not describe a training set for a machine learning algorithm. MammaPrint is a gene expression profiling test, which typically relies on a fixed algorithm/signature derived from earlier research, not an adaptive machine learning model that is "trained" in an ongoing manner. The document focuses on the analytical validation of the test's execution and reproducibility.
9. How the Ground Truth for the Training Set Was Established
Not applicable, as there is no mention of a training set for a machine learning algorithm in the provided text. The MammaPrint test itself is based on a specific gene expression profile correlated to clinical outcome. The methods for establishing this original correlation (which forms the basis of the "template" mentioned in the device description) would be found in the foundational clinical studies for MammaPrint, not in this analytical validation document. The "template" (mean expression profile of 44 tumors with a known good clinical outcome) explicitly mentioned in the device description is effectively the core of their model, derived from prior clinical data.
§ 866.6040 Gene expression profiling test system for breast cancer prognosis.
(a)
Identification. A gene expression profiling test system for breast cancer prognosis is a device that measures the ribonucleic acid (RNA) expression level of multiple genes and combines this information to yield a signature (pattern or classifier or index) to aid in prognosis of previously diagnosed breast cancer.(b)
Classification. Class II (special controls). The special control is FDA's guidance document entitled “Class II Special Controls Guidance Document: Gene Expression Profiling Test System for Breast Cancer Prognosis.” See § 866.1(e) for the availability of this guidance document.