Search Results
Found 14 results
510(k) Data Aggregation
(526 days)
Irvine, CA 92618
Re: K210973
Trade/Device Name: MammaPrint FFPE NGS kit Regulation Number: 21 CFR 866.6040
The predicate device, with product code NYI, is regulated under 21CFR 866.6040.
The MammaPrint FFPE NGS kit is a qualitative in vitro diagnostic test for use by clinical laboratories using target enrichment Next Generation Sequencing (NGS) technology for gene expression profiling of the 70-gene MammaPrint Breast Cancer signature on formalin-fixed, paraffin-embedded (FFPE) breast cancer tissue samples. The test is used to assess a patient's risk to develop distant metastasis within 5 years and up to 10 years after diagnosis.
The MammaPrint FFPE NGS kit is performed for breast cancer patients with Stage I or Stage II disease, with tumor size ≤ 5.0 cm and lymph node negative. The test result is indicated for use by physicians as a prognostic marker only, along with other clinicopathological factors.
The MammaPrint FFPE NGS kit is a sequencing-based gene expression analysis of a tumor. The analysis is based on several processes: isolation of RNA from FFPE breast cancer tissue sections; library preparation of RNA resulting in cDNA adapter-ligated sequences; enrichment of the 70 genes (capture step); sequencing of the enriched library in the flow cell and data acquisition; MammaPrint Index calculation of the risk classification in breast cancer patients.
Data analysis is performed according to the MammaPrint FFPE NGS algorithm (resulting in MammaPrint Index or MPI). This algorithm was designed and programmed by Agendia and incorporated into a proprietary software program, which loads the FASTQ data file. The software loads file, performs quality control checks and determines the molecular profile of the sample by calculating the MammaPrint index by determining the correlation of the sample's 70 gene expression profile to the mean expression profiles of tumors with a known good and poor outcome.
The provided text describes the acceptance criteria and the study that proves the MammaPrint FFPE NGS kit meets these criteria, primarily by demonstrating substantial equivalence to its predicate device (MammaPrint FFPE microarray).
Here's a breakdown of the requested information:
1. Table of Acceptance Criteria and Reported Device Performance
The document doesn't explicitly outline a formal "acceptance criteria" table with pre-defined thresholds for performance metrics. Instead, it demonstrates performance through concordance studies and reproducibility assessments against the predicate device, or by showing high agreement for controls. The clinical performance is demonstrated by comparing the survival outcomes of the new device to those of the predicate device on the same patient cohort from the RASTER study.
Acceptance Criteria (Implied by the study design and results presented):
Performance Metric | Implied Acceptance Criteria (Achieved by the device) | Reported Device Performance |
---|---|---|
Method Comparison (Study-1): | ||
OPA (High vs. Low) | High concordance with predicate device. | 97.42% (95% Cl: 93.55, 98.99) |
NPA (High vs. Low) | High concordance with predicate device. | 93.85% (95% C1: 85.22, 97.58) |
PPA (High vs. Low) | High concordance with predicate device. | 100.00% (95% CI: 95.91, 100.00) |
Method Comparison (Study-2): | ||
OPA (ranging across sites) | High concordance with predicate device. | 91.09% to 92.41% |
PPA (ranging across sites) | High concordance with predicate device. | 91.39% to 99.34% |
NPA (ranging across sites) | High concordance with predicate device. | Lower: 82.89% to 93.42% (noted as impacted by borderline samples) |
Repeatability of RNA Isolation (Categorical Results): | 100% agreement between repeat isolations. | 100% agreement (95%Cl: High Risk: 86.2-100.0, Low Risk: 81.6-100.0, Borderline: 34.2 –100.0) |
Reproducibility of Controls (Categorical Results): | 100% agreement for known controls across sites/operators/lots. | 100% agreement for both CTRL-HR and CTRL-LR (Table 1: 94.2%-100% CI for both). |
Precision/Reproducibility (Categorical Results: Study-1): | High agreement for different risk categories. | PREC-BD (Borderline): 81% (72.0, 88.5) (Table 3) |
PREC-HR (High Risk): 100% (96.9, 100) (Table 3) | ||
PREC-LR (Low Risk): 100% (96.9, 100) (Table 3) | ||
Precision/Reproducibility (Categorical Results: Study-2): | High agreement, especially for samples not near threshold. | Range from 58% to 100% (Table 5). Samples 2 (Low Risk) and 5 (High Risk) had 83% and 58% agreement respectively. |
Detection Limit (Valid Rates): | Acceptable valid rates at specified RNA input and quality. | DV200 "poor" (35-49%) with ≥100 ng RNA: 80% to 100% valid rates. |
DV200 "Standard" (35-80%) with ≥100 ng RNA: 80% to 100% valid rates. | ||
Clinical Performance (DRFI): | Similar survival outcomes to predicate device. | Kaplan-Meier plots suggest significant difference in survival curves among different risk groups for MammaPrint FFPE NGS kit, p=0.001. |
5-year DRFI: Low Risk 98.1%, Borderline 92.6%, High Risk 88.2% (Table 6). | ||
10-year DRFI: Low Risk 95.2%, Borderline 92.6%, High Risk 82.0% (Table 6). | ||
The results "indicated that both devices show similar clinical performance" based on RASTER study follow-up. |
2. Sample Sizes and Data Provenance
- Method Comparison Study-1: 155 samples used. Data acquired retrospectively as these samples were "previously processed on MammaPrint FFPE microarray as part of routine diagnostics."
- Method Comparison Study-2: 303 samples used. Data acquired retrospectively from "previously collected patient samples."
- Repeatability of RNA Isolation: Not explicitly stated, but implies multiple FFPE tumor blocks, each sectioned twice.
- Reproducibility of Controls: 2 control samples (CTRL-HR, CTRL-LR), each processed 50 times across 4 external sites, 2 operators per site, 6 NGS runs, and 3 lot numbers. Total of 100 measurements for controls.
- Precision/Reproducibility Assessment-Study-1: 3 samples (High Risk, Borderline, Low Risk). Each processed 96 times (4 external sites x 2 operators x 6 NGS runs x 2 duplicates per run).
- Precision/Reproducibility Assessment-Study-2: 8 samples. Each processed repeatedly (from RNA isolation to sequencing) by 2 operators at 3 sites, with 4 runs per site and 2 replicates per run. This results in 16-24 replicates per sample (table indicates N of 16 or 24).
- Detection Limit: 8 FFPE breast cancer samples with "poor" DV200 quality were evaluated, with various dilutions. Additional unstated number of "Standard" DV200 samples.
- Clinical Validation (RASTER Study):
- Original RASTER enrollment: 427 patients.
- Subset of FFPE samples from RASTER study used for NGS kit validation: 345 samples.
- Samples successfully processed for analysis: 316 samples.
- Data provenance: Multicenter observational study conducted in the Netherlands between 2004 and 2006 (for initial RASTER study). The current analysis uses updated 5- and 10-year follow-up data. This is prospective observational data with the device evaluated retrospectively on archived samples.
3. Number of Experts and Qualifications for Ground Truth
The document does not mention the use of experts to establish ground truth for the analytical or reproducibility test sets. The ground truth for these studies is based on comparisons against the predicate device's results or known control sample values.
For the clinical validation, the ground truth for distant recurrence-free interval (DRFI) and breast cancer specific survival (BCSS) is outcomes data from the RASTER study, defined as distant breast cancer recurrence or death from breast cancer. The establishment of these clinical outcomes likely involved clinical experts (e.g., oncologists, pathologists, and medical records review) over the 10-year follow-up period, but the specific number and qualifications of these experts are not detailed in this submission.
4. Adjudication Method for Test Set
The document does not describe an adjudication method (such as 2+1 or 3+1) for the test sets. For analytical studies, agreement is based on direct comparison of results between NGS and microarray platforms or within the NGS platform itself (repeatability/reproducibility). For the clinical study, outcomes (DRFI, BCSS) are established from patient follow-up data.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No MRMC comparative effectiveness study was performed as this device is a gene expression profiling test system, not an imaging device requiring human reader interpretation. The comparison is between the new NGS-based assay and the predicate microarray-based assay.
6. Standalone Performance
The "standalone" performance shown is the analytical performance of the MammaPrint FFPE NGS kit itself (repeatability, reproducibility, detection limit) and its clinical prognostic ability, which is an algorithm-only output (MammaPrint Index). The clinical validation study (RASTER) assesses the algorithm's ability to stratify patients by risk based on actual clinical outcomes. The device performance (result classification: High Risk, Borderline, Low Risk) is reported directly from the algorithm without a human-in-the-loop component for classification.
7. Type of Ground Truth Used
- Analytical Performance (Method Comparison, Repeatability, Reproducibility): The ground truth is effectively the predicate device's results or the known values of control samples. For repeatability/reproducibility, the ground truth is consistency of the device's own output.
- Clinical Validation: The ground truth is outcomes data (Distant Recurrence-Free Interval - DRFI, and Breast Cancer Specific Survival - BCSS) from prospectively collected clinical follow-up in the RASTER study. This is directly observed patient outcomes over 5 and 10 years.
8. Sample Size for the Training Set
The document focuses on the performance of the MammaPrint FFPE NGS kit, which is stated to use an "unchanged" 70-gene signature and scoring algorithm from the predicate device. The original MammaPrint 70-gene signature was developed much earlier. The description states: "This algorithm was designed and programmed by Agendia and incorporated into a proprietary software program, which loads the FASTQ data file." The specifics of the training set for the original MammaPrint algorithm are not provided in this document. This submission is for a new platform (NGS) of an existing and cleared diagnostic system, demonstrating its equivalence, rather than a new algorithm development.
9. How the Ground Truth for the Training Set was Established
As noted above, the original training of the MammaPrint algorithm is not detailed here. The clinical validation in this document uses the RASTER study data, where the ground truth is clinical outcome data (DRFI, BCSS) established through follow-up. This RASTER data is used for validation of the new NGS platform, not for training a new algorithm.
Ask a specific question about this device
(120 days)
Philadelphia, PA 19103
Re: K201902
Trade/Device Name: MammaPrint® FFPE Regulation Number: 21 CFR 866.6040
MammaPrint® FFPE is a qualitative in vitro diagnostic test, performed in a central laboratory, using the gene expression profile obtained from formalin-fixed paraffin embedded (FFPE) breast cancer tissue samples to assess a patient's risk for distant metastasis within 5 years.
The test is performed for breast cancer patients, with Stage I disease, with tumor size ≤ 5.0 cm and lymph node negative. The MammaPrint FFPE result is indicated for use by physicians as a prognostic marker only, along with other clinicopathological factors.
The MammaPrint service is a microarray-based gene expression analysis of a tumor. The analysis is based on several processes: isolation of RNA from formalin-fixed paraffin embedded (FFPE) tumor tissue sections, DNase treatment of isolated RNA, amplification DNase treated RNA resulting in cDNA, labeling and purification of amplified cDNA, hybridization of the diagnostic microarray, scanning the MammaPrint microarray and data acquisition (Feature Extraction), calculation and determination of the risk of recurrence in breast cancer patients.
The MammaPrint analysis is designed to determine the gene activity of specific genes in a FFPE tissue sample. The result is an expression profile, or fingerprint, of the sample.
The molecular profile of the sample is determined (Low Risk) by calculating the MammaPrint index (MPI) by determining the correlation of the sample expression profile to the mean expression profiles of risk templates of tumors with a known good and poor outcome.
The provided text describes the acceptance criteria and study for the MammaPrint FFPE device, focusing on the analytical performance related to a change in microarray scanner.
Here's the breakdown of the information requested:
1. A table of acceptance criteria and the reported device performance
Acceptance Criteria Category | Specific Criteria | Reported Device Performance |
---|---|---|
Technical Equivalence (Concordance between Scanners) | Passing and Bablok regression for MammaPrint indices between C-scanners (Agilent G2505) and D-scanners (Agilent SureScan Dx G5761AA) must meet pre-defined acceptance criteria (specific values for slope and intercept not explicitly stated, but implied to be near 1 and 0, respectively). | Scanner SG18309119 vs. C-scanners: y=0.00 +1.00 x (95%Cl – slope: 1.000 – 1.002, 95%Cl intercept: -0.0002 to 0.000). Scanner SG18449122 vs. C-scanners: y=0.0010 +1.00 x (95%Cl – slope: 1.000 – 1.0021, 95%Cl intercept: 0.0014 to 0.001). Both results are stated to be "within the pre-defined acceptance criteria." |
Overall concordance, Negative Percent Agreement (NPA), and Positive Percent Agreement (PPA) for MammaPrint categorical results (High/Low Risk) between C-scanners and D-scanners must be within pre-defined acceptance criteria. | Scanner SG18309119 vs. C-scanners: Overall concordance: 99.7%. NPA: 100% (95%Cl: 98.1-100). PPA: 99.3% (95%Cl: 96.0 – 99.9). Scanner SG18449122 vs. C-scanners: Overall concordance: 100%. NPA: 100% (95%Cl: 97.1 – 100). PPA: 100% (95%Cl: 97.5 - 100). All reported values are stated to be "within the predefined acceptance criteria." | |
Precision Assessment | F-test p-values for variability of repeated measurements of control samples between C-scanners and D-scanners should not show a significant difference (i.e., p-value > 0.05). | F-test p-values for all four control samples were "all well below the significance level of 0.05, indicating there is no significant difference in precision between C and D-scanners." (This wording is a bit confusing; "well below 0.05" would typically indicate a significant difference. However, given the context of demonstrating no significant difference, it likely implies that the null hypothesis of equal variance could not be rejected, which means the p-value was above 0.05. Re-reading, "indicating there is no significant difference" following "well below the significance level of 0.05" is a logical contradiction. Assuming the intent was to show no significant difference in precision, the p-values should have been above 0.05. This might be a typo in the FDA document, or the interpretation of the F-test result is inversely stated. Given the conclusion that "there is no significant difference," the F-test result must have been non-significant, meaning p > 0.05. Let's assume the intent was to show non-significant difference.) |
Clinical Correlation | MammaPrint index should correlate with clinical outcome (distant recurrence risk). | Result distribution: In the RASTER study, Bin 1 (MPI 0.36 to +1) had 0% observed 5-yr DR risk (N=37). Bin 4 (MPI -1 to -0.57) had 13.6% observed 5-yr DR risk (N=66). Cox regression analysis: With each increase in MammaPrint index unit, there is a 0.224 (4.5 folds) decrease in recurrence risk at 5 years (p=0.001; 95% Cl: 0.092-0.543). This demonstrates correlation. |
2. Sample size used for the test set and the data provenance (e.g., country of origin of the data, retrospective or prospective)
- Sample Size for Technical Equivalence (Concordance): 92 8-pack arrays (likely representing 92 individual samples, or potentially 92 unique arrays, each containing multiple spots/analyses).
- Sample Size for Precision Assessment: Four diagnostic control samples (PLEP3, PHHE2, PHTR2, PBCL2), measured repeatedly. The number of repeated measurements is not specified.
- Sample Size for Clinical Correlation (RASTER study): 345 patients.
- Data Provenance: Not explicitly stated for analytical performance (concordance and precision). For the clinical correlation, it references the RASTER study, which was "previously submitted in K141142," but country of origin and retrospective/prospective nature are not detailed here.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g., radiologist with 10 years of experience)
- For Analytical Performance: Ground truth is established by the MammaPrint index and categorical results from the FDA-cleared C-scanners (Agilent G2505), acting as the reference. No human experts are involved in establishing this specific ground truth.
- For Clinical Correlation (RASTER study): The clinical outcome (distant recurrence risk) is the ground truth. While not detailed in this excerpt, clinical outcomes studies typically rely on physician diagnoses, follow-up, and potentially pathology reports, rather than a panel of experts specifically adjudicating "ground truth" for the test results.
4. Adjudication method (e.g., 2+1, 3+1, none) for the test set
- For Analytical Performance: None. The ground truth is the output from the predicate device's scanner (Agilent G2505).
- For Clinical Correlation: Not specified in this document. Clinical outcome data collection methods vary, but typically don't involve an adjudication panel for the outcome itself, but rather established clinical trial protocols and follow-up.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
- This is not an MRMC study. The MammaPrint device is a gene expression profiling test, not an AI imaging or diagnostic assistance tool for human readers. It provides a prognostic marker directly from tissue analysis.
6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done
- Yes, this is a standalone device. The MammaPrint FFPE test generates a result (MammaPrint Index and High/Low Risk classification) directly from the gene expression profile of the tumor tissue. The "algorithm only" performance is what is being evaluated and validated. The "human-in-the-loop" component mentioned is the physician's use of the result "along with other clinicopathological factors" for prognostic assessment.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
- For Analytical Performance (concordance and precision): The ground truth is the MammaPrint index and categorical result generated by the previously FDA-cleared C-scanners (Agilent G2505).
- For Clinical Correlation: The ground truth is 5-year distant recurrence (DR) risk, which is a form of outcomes data.
8. The sample size for the training set
- Not explicitly mentioned for the current submission. The MammaPrint assay itself was developed and validated earlier, and the current submission is for a modification (scanner change) to an already cleared predicate device (K141142). The original training set for the MammaPrint algorithm would have been part of the earlier submissions but is not detailed here.
9. How the ground truth for the training set was established
- Not explicitly mentioned in this document. For the original MammaPrint development, the ground truth for training would have involved gene expression profiles correlated with long-term clinical outcomes (e.g., distant metastasis-free survival) from large cohorts of breast cancer patients, likely using robust clinical data and follow-up.
Ask a specific question about this device
(266 days)
Amsterdam, The Netherlands
Re: K141142 Trade/Device Name: MammaPrint® FFPE Regulation Number: 21 CFR §866.6040
prognosis Classification Name: Gene expression profiling test system, for breast cancer prognosis (21 CFR 866.6040
MammaPrint® FFPE is a qualitative in vitro diagnostic test, performed in a central laboratory, using the gene expression profile obtained from formalin-fixed paraffin embedded (FFPE) breast cancer tissue samples to assess a patient's risk for distant metastasis within 5 years.
The test is performed for breast cancer patients, with Stage II disease, with tumor size ≤ 5.0 cm and lymph node negative. The MammaPrint® FFPE result is indicated for use by physicians as a prognostic marker only, along with other clinico-pathological factors.
The MammaPrint® FFPE test is a microarray based gene expression analysis of a tumor. The analysis is based on several processes: isolation of RNA from FFPE breast cancer tissue sections; elimination of gDNA, reverse transcription of RNA resulting in cDNA; amplification of the cDNA, purification and labeling of cDNA; hybridization of the amplified and labeled cDNA to the diagnostic microarray; washing and scanning the diagnostic microarray and data acquisition (feature extraction); calculation and determination of the risk of recurrence.
The MammaPrint® FFPE analysis is designed to determine the expression of specific genes in a tissue sample. The result is an expression profile, or "fingerprint", of the sample. Using this expression profile, the MammaPrint® FFPE Index is calculated and the molecular prognosis profile of the sample is determined (Low Risk, High Risk).
Below is a summary of the acceptance criteria and study details for the Agendia MammaPrint® FFPE device, based on the provided text.
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria for MammaPrint® FFPE are primarily based on demonstrating equivalence to the MammaPrint® Fresh device (K101454) in terms of concordance and clinical performance.
Test Category | Acceptance Criteria | Reported Device Performance and Confidence Intervals |
---|---|---|
Analytical Performance | ||
Concordance (FFPE vs. Fresh) - 1st Study (n=122) | High concordance between MammaPrint Fresh and FFPE outcomes. | Overall Concordance: 89.34% |
True Positive/Negative Concordance (non-borderline): 92.7% | ||
NPA (95% CI): 88.4% (80.5-95) | ||
PPA (95% CI): 90.6% (79.8-95.9) | ||
Concordance (FFPE vs. Fresh) - 2nd Study (n=345) | High concordance between MammaPrint Fresh and FFPE outcomes. | Overall Concordance: 89.28% |
True Positive/Negative Concordance (non-borderline): 93.5% | ||
NPA (95% CI): 91.5% (86.7-94.7) | ||
PPA (95% CI): 86.6% (80.4-91.1) | ||
Intra-sample Reproducibility (4 isolations) | No significant difference in MammaPrint Indices or Outcome across multiple isolations from the same sample. | MammaPrint Indices: p=0.994 (no significant difference) |
MammaPrint Outcome: p=0.290 (no significant difference) | ||
Inter-assay Reproducibility (Control samples over time) | Standard deviations of MammaPrint Indices for control samples to meet predefined criteria. | PHTR=0.045 (n=54), PLEP=0.056 (n=52), PHHE=0.072 (n=52) (all passed predefined criteria) |
Method Precision (Repeatability & Within-lab) | Repeatability and Method Precision results to meet predefined acceptance criteria. | Repeatability (Within-Run): |
High Risk (1): 0.036 | ||
High Risk (2): 0.046 | ||
Low Risk: 0.042 | ||
Low Risk-Borderline: 0.049 | ||
Method Precision (Within-Laboratory): | ||
High Risk (1): 0.044 | ||
High Risk (2): 0.057 | ||
Low Risk: 0.050 | ||
Low Risk-Borderline: 0.066 | ||
(All met predefined criteria) | ||
Inter-laboratory Comparison (Irvine vs. Amsterdam) | High agreement in MammaPrint Index and Outcome between laboratories, passing predefined acceptance criteria. | Irvine: Kappa score = 0.90 |
Amsterdam: Kappa Score = 0.9 | ||
Intercept close to zero, slope close to 1 (Passing and Bablok regression) | ||
Microarray Scanner Comparison (Amsterdam) | High agreement in MammaPrint Index and Outcome between different scanners, passing predefined acceptance criteria. | Pearson correlation = 1.0 |
Kappa score = 1.0 | ||
NPA (95% CI): 100% (67.6-100) | ||
PPA (95% CI): 100% (81.6-99.0) | ||
Intercept close to zero, slope close to 1 | ||
Microarray Scanner Comparison (Irvine) | High agreement in MammaPrint Index and Outcome between different scanners, passing predefined acceptance criteria. | Pearson correlation = 1.0 |
Kappa score = 1.0 | ||
NPA (95% CI): 100% (77.2-100) | ||
PPA (95% CI): 100% (78.5-100) | ||
Intercept close to zero, slope close to 1 | ||
Minimum Input (Dilution Study) | Very stable results even at low input of cDNA expected. | Results showed very stable results even at low input of cDNA. |
Clinical Performance | ||
5-year DRFI (Distant Recurrence Free Interval) | MammaPrint FFPE performance (low and high risk signatures) to fall within the 95% CI of MammaPrint Fresh. | Low Risk Signature: |
MammaPrint Fresh 2013: 0.976 (0.952-1.000) | ||
MammaPrint FFPE: 0.977 (0.955-0.999) | ||
High Risk Signature: | ||
MammaPrint Fresh 2013: 0.891 (0.840-0.942) | ||
MammaPrint FFPE: 0.885 (0.830-0.940) | ||
(All fall within 95% CI) | ||
5-year DM1st (Distant Metastasis as first event) | MammaPrint FFPE performance (low and high risk signatures) to fall within the 95% CI of MammaPrint Fresh. | Low Risk Signature: |
MammaPrint Fresh 2013: 0.976 (0.952-1.000) | ||
MammaPrint FFPE: 0.977 (0.955-0.999) | ||
High Risk Signature: | ||
MammaPrint Fresh 2013: 0.907 (0.860-0.954) | ||
MammaPrint FFPE: 0.903 (0.854-0.952) | ||
(All fall within 95% CI) |
2. Sample Size Used for the Test Set and Data Provenance
- Analytical Performance - Concordance Studies:
- First independent validation: n=122 tumor samples (FFPE vs. Fresh from the same tumor). Data provenance is not explicitly stated beyond being "internal" to Agendia but given the global operations might be multi-country. The study appears to be retrospective as it compares processed samples.
- Second independent validation (Raster study): n=345 samples (FFPE tissue vs. Fresh RNA from the same patients). The Raster study is mentioned as the source, which is a clinical study. The geographical origin is not explicitly stated, but clinical studies often involve multiple centers. It's retrospective in the sense that existing samples with follow-up were used.
- Analytical Performance - Reproducibility & Precision:
- Multiple isolations: 30 FFPE samples. Provenance not specified.
- Multiple labeling/hybridizations (control samples): PHTR (n=54), PLEP (n=52), PHHE (n=52) over time. Provenance not specified.
- Precision and Evaluation (P&E): 4 test samples (representing different risk levels) over 20 days. Provenance not specified.
- Analytical Performance - Inter-laboratory Comparison: 25 FFPE samples. Provenance not specified, but involved samples processed in Amsterdam and Irvine laboratories.
- Analytical Performance - Microarray Scanner Validation: 25 samples (Amsterdam), 27 samples (Irvine). Provenance not specified.
- Analytical Performance - Minimum Input: 3 samples and 3 control samples. Provenance not specified.
- Clinical Performance (Comparison with MammaPrint Fresh):
- Raster study: 345 samples with clinical follow-up of 5 years. This data is from the Raster study, which is a clinical prospective study (although the use here for the FFPE comparison would be retrospective on archived samples).
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The document does not specify the number or qualifications of experts used to establish ground truth for the analytical or clinical validation. The "ground truth" for the analytical studies is the result from the established MammaPrint Fresh assay (K101454) for concordance, or statistical assessments of variability for reproducibility/precision. For clinical outcomes, the ground truth is clinical follow-up data (Distant Recurrence Free Interval and Distant Metastasis as first event) over 5 years. This outcome data is typically collected through clinical study follow-up by medical professionals, but not "established by experts" in the sense of a panel review for each case in this context.
4. Adjudication Method for the Test Set
No explicit adjudication method (like 2+1 or 3+1) is mentioned for the test sets. The ground truth for analytical concordance is the result from the predicate device (MammaPrint Fresh), and for clinical performance, it's the 5-year clinical outcome data (DRFI, DM1st) from the Raster study.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done
No MRMC comparative effectiveness study involving human readers is described. This device is a gene expression profiling test performed in a central laboratory, not an image-based diagnostic read by human readers. Therefore, the concept of human readers improving with AI assistance is not applicable in this context.
6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was Done
Yes, the studies described are standalone performance evaluations of the MammaPrint® FFPE algorithm. The "device performance" and "reported device performance" in the table refer to the algorithm's output (Low Risk/High Risk and MammaPrint Index calculation) based on the tissue sample analysis.
7. The Type of Ground Truth Used
- Analytical Performance:
- Concordance: The results from the predicate device, MammaPrint® Fresh (K101454), using fresh tissue samples of the same tumor.
- Reproducibility, Precision, Inter-laboratory, Scanner Validation, Minimum Input: Statistical measures of consistency and variation against predefined internal criteria, using the device's own output.
- Clinical Performance:
- Outcomes Data: 5-year Distant Recurrence Free Interval (DRFI) and 5-year Distant Metastasis as first event (DM1st) collected during the Raster study.
8. The Sample Size for the Training Set
The document does not explicitly state the sample size for a training set for the MammaPrint® FFPE algorithm. The MammaPrint® technology was likely developed and validated using significant cohorts prior to this specific submission, which focuses on extending its use to FFPE samples and demonstrating equivalence. The "Raster study" is mentioned for clinical evaluation, but it's not explicitly stated as a training set. The descriptions focus on the validation of the FFPE version against the established Fresh version and clinical outcomes.
9. How the Ground Truth for the Training Set Was Established
Since a dedicated training set and its ground truth establishment are not described, this information cannot be provided from the given text. MammaPrint is a gene expression profiling test, and the "ground truth" for its original development (not necessarily a "training set" in the machine learning sense for this submission) would typically involve correlating gene expression profiles with long-term clinical outcomes in large patient cohorts to define the prognostic signature. This submission focuses on validating the FFPE version against the already cleared predicate device (MammaPrint Fresh).
Ask a specific question about this device
(129 days)
Trade/Device Name: Prosignatin Breast Cancer Prognostic Gene Signature Assay Regulation Number: 21 CFR §866.6040
Regulation section:
21 CFR §866.6040 Gene expression profiling test system for breast cancer prognosis
The Prosigna® Breast Cancer Prognostic Gene Signature Assay is an in vitro diagnostic assay which is performed on the NanoString nCounter® Dx Analysis System using FFPE breast tumor tissue previously diagnosed as invasive breast carcinoma. This qualitative assay utilizes gene expression data, weighted together with clinical variables to generate a risk category and numerical score, to assess a patient's risk of distant recurrence of disease.
The Prosigna Breast Cancer Prognostic Gene Signature Assay is indicated in female breast cancer patients who have undergone surgery in conjunction with locoregional treatment consistent with standard of care, either as:
-
A prognostic indicator for distant recurrence-free survival at 10 years in postmenopausal women with Hormone Receptor-Positive (HR+), lymph node-negative, Stage I or II breast cancer to be treated with adjuvant endocrine therapy alone, when used in conjunction with other clinicopathological factors.
-
A prognostic indicator for distant recurrence-free survival at 10 years in postmenopausal women with Hormone Receptor-Positive (HR+), lymph node-positive (1-3 positive nodes), Stage II breast cancer to be treated with adjuvant endocrine therapy alone, when used in conjunction with other clinicopathological factors. The device is not intended for patients with 4 or more positive nodes
The required components for the Prosigna™ Breast Cancer Prognostic Gene Signature Assay include the RNA Isolation kit (manufactured by Roche), Prosigna reagents (Reference Sample, CodeSet, Prep Pack, Cartridge(s) and Prep Plate) and the instruments that comprise the nCounter Dx Analysis System; the Prep Station and Digital Analyzer.
The assay requires microdissection of tumor from FFPE biopsies, isolation of RNA using a Roche RNA isolation kit, transfer of RNA to PCR tubes for hybridization before placing onto the prep station. Two sets of probes specific to each of 58 RNAs are added to the hybridization reaction. These consist of biotin-labeled magnetic probes to purify the RNAs and capture them on the assay cartridge and fluorescent "barcode" probes to detect and quantify individual RNAs. The patient sample and probes are pipetted automatically into the Prosigna test cartridge by the Prep Station. The prep station uses magnetic bead capture and washing to remove excess RNA and un-hybridized probes. The isolated and hybridized RNA species are then bound via biotin on the capture probe randomly to streptavidin on the cartridge. The fluorescent molecules are then aligned on the cartridge by addition of an electric current. The cartridge is then transferred to the Digital Analyzer where the cartridge is scanned and digital analysis software is used to count the number of each RNA species present. The amount of each RNA is then put into a proprietary algorithm to produce a Prosigna score.
The test output is a patient specific report which includes a Prosigna score (0-100) and risk category (low/intermediate/high).
This document describes the FDA's decision to clear the Prosigna™ Breast Cancer Prognostic Gene Signature Assay (K141771) based on its substantial equivalence to a predicate device (K130010). The submission in question for K141771 relates to a modification of the device configuration and software, specifically introducing a 4-sample kit configuration and updating the instrument software. Therefore, the acceptance criteria and study data presented primarily focus on demonstrating that these modifications do not negatively impact the performance established by the predicate device.
1. Table of Acceptance Criteria and Reported Device Performance
For K141771, the primary acceptance criterion related to device performance was to demonstrate that the new 4-sample kit configuration and updated software produced results substantially equivalent to the previously cleared 10-sample kit configuration.
Acceptance Criteria | Reported Device Performance (K141771) |
---|---|
Equivalence of Prosigna Scores between 4-sample and 10-sample kit configurations | Deming Regression: Data was linear over the range of the assay with no outliers between the two methods, indicating substantial equivalence. No value deviated by more than 2 units from the average score when run using either configuration. |
Bland-Altman Analysis: Variations in data using the 4-kit versus the 10-kit configuration did not bias results. | |
No change in risk categorization (Low/Intermediate/High) between 4-sample and 10-sample kit configurations | No changes in risk categorization occurred between the 4-kit and 10-kit configurations. |
Note: The document explicitly states "See Predicate Device K130010 for Analytical Performance Data" and "See Predicate Device K130010 for Clinical Performance Data." This means that the core analytical and clinical performance acceptance criteria and their supporting studies were primarily established during the clearance of the predicate device (K130010). The K141771 submission focused on demonstrating that changes to the device (new kit configuration, software) did not degrade this established performance.
2. Sample Size Used for the Test Set and Data Provenance
For the specific comparison study outlined in K141771 (comparing the 4-sample and 10-sample kits):
- Sample Size: Forty (40) samples of RNA previously extracted from FFPE breast tissue were tested with both configurations.
- Data Provenance: The document does not specify the country of origin of the data. It indicates the samples were "previously extracted from FFPE breast tissue," implying the data was retrospective in nature for this specific comparison.
For the original clinical performance data (referenced as part of K130010):
- Sample Size: "a study with over 1007 patient samples associating Prosigna score with long-term outcome, defined by distance recurrence free survival at 10 years (DRFS) (Table 2)."
- Data Provenance: Not specified in this document for the 1007 patient sample study, but it would have been part of the K130010 submission.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
This information is not directly applicable to the K141771 submission's comparison study, as the ground truth was the Prosigna score itself, as generated by the instrument. The study compared the scores generated by two different configurations of the same device.
For the original clinical validation (referenced from K130010), the ground truth for "distant recurrence-free survival at 10 years (DRFS)" would typically be established based on patient follow-up data, which is objective outcomes data rather than subjective expert consensus.
4. Adjudication Method for the Test Set
Not applicable to the K141771 comparison study as there was no expert review or human adjudication involved in generating the Prosigna scores. The comparison was between two automated outputs of the device.
5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study was done
No, a multi-reader multi-case (MRMC) comparative effectiveness study was not done. The device (Prosigna Assay) is an in vitro diagnostic (IVD) test that provides an automated, objective score. It is not an imaging AI diagnostic system designed to assist human readers.
6. If a Standalone (i.e. algorithm only without human-in-the loop performance) was done
The Prosigna Assay is inherently a standalone algorithm. It takes gene expression data and clinical variables as input and produces a risk category and numerical score as output without human intervention in the scoring process. The performance of this standalone algorithm was presumably established and validated in the clinical studies for the predicate device (K130010), which involved "a study with over 1007 patient samples associating Prosigna score with long-term outcome, defined by distance recurrence free survival at 10 years (DRFS)."
7. The Type of Ground Truth Used
The ground truth for the clinical validation of the Prosigna Assay (for K130010) was outcomes data, specifically "distant recurrence free survival at 10 years (DRFS)". This is a definitive clinical outcome, not expert consensus or pathology review of the assay results themselves.
8. The Sample Size for the Training Set
The document does not specify the sample size for the training set. It mentions "Three risk categories (low, intermediate and high) were defined based on a study with over 1007 patient samples associating Prosigna score with long-term outcome...". This "study" likely encompassed both the development/training and validation of the algorithm, but the specific breakdown is not provided here.
9. How the Ground Truth for the Training Set was Established
The ground truth for the training set (or the 1007 patient study mentioned) was established based on long-term clinical outcomes, specifically "distance recurrence free survival at 10 years (DRFS)". This means that for each patient in the dataset, their disease recurrence status was tracked for 10 years after initial treatment.
Ask a specific question about this device
(246 days)
Trade/Device Name: ProsignaTM Breast Cancer Prognostic Gene Signature Assay Regulation Number: 21 CFR §866.6040
NanoString gene expression profiling test for breast cancer prognosis
Classification:
21 CFR § 866.6040
The Prosigna™ Breast Cancer Prognostic Gene Signature Assay is an in vitro diagnostic assay which is performed on the NanoString nCounter® Dx Analysis System using FFPE breast tumor tissue previously diagnosed as invasive breast carcinoma. This qualitative assay utilizes gene expression data, weighted together with clinical variables to generate a risk category and numerical score, to assess a patient's risk of distant recurrence of disease.
The Prosigna Breast Cancer Prognostic Gene Signature Assay is indicated in female breast cancer patients who have undergone surgery in conjunction with locoregional treatment consistent with standard of care, either as:
-
A prognostic indicator for distant recurrence-free survival at 10 years in post-menopausal women with Hormone Receptor-Positive (HR+), lymph node-negative, Stage I or II breast cancer to be treated with adjuvant endocrine therapy alone, when used in conjunction with other clinicopathological factors.
-
A prognostic indicator for distant recurrence-free survival at 10 years in post-menopausal women with Hormone Receptor-Positive (HR+), lymph node-positive (1-3 positive nodes), Stage II breast cancer to be treated with adjuvant endocrine therapy alone, when used in conjunction with other clinicopathological factors. The device is not intended for patients with 4 or more positive nodes.
Special Conditions for Use: Prosigna is not intended for diagnosis, to predict or detect response to therapy, or to help select the optimal therapy for patients.
Used together, the Prosigna™ Breast Cancer Prognostic Gene Signature Assay and nCounter Dx Analysis System are a nucleic acid hybridization and image analysis system based upon coded probes designed to detect the messenger RNA transcribed from 58 genes. The test input is purified RNA from FFPE breast tumor specimens which are acquired from surgical resection. The Prosigna assay uses gene-specific probe pairs that hybridize directly to the mRNA transcripts in solution. The nCounter Dx Analysis System delivers direct, multiplexed measurements of gene expression through digital readouts of the relative abundance of the mRNA transcripts. Specifications are included as part of the Prosigna Assay to control for sample quality, RNA quality, and process quality. Prosigna simultaneously measures the expression levels of 50 genes used in the PAM50 classification algorithm (Parker et al., 2009), 8 housekeeping genes used for signal normalization, 6 positive controls, and 8 negative controls in a single hybridization reaction, using nucleic acid probes designed specifically to those genes. The Prosigna assay utilizes prototypical expression profiles (centroids) which are associated with and define each of the four PAM50 molecular subtypes of breast cancer. The software algorithm produces a Prosigna Score (referred to as ROR Score or Risk of Recurrence Score in the literature (Dowsett et al., 2013)) based on the similarity of the expression profile to each PAM50 molecular subtype, as well as the gross pathological tumor size and a proliferation score computed from a subset of genes. Three risk categories (low, intermediate and high) were defined based on a study with over 1007 patient samples associating Prosigna score with longterm outcome.
The required components for the Prosigna Assay include the RNA Isolation kit (manufactured by Roche), Prosigna reagents (Reference Sample, CodeSet, Prep Pack, Cartridge(s) and Prep Plate) and the instruments that comprise the nCounter Dx Analysis System; the Prep Station and Digital Analyzer.
The test output is a patient specific report which includes a Prosigna score (0-100) and risk category (low/intermediate/high).
Let's break down the acceptance criteria and the study that proves the device meets those criteria for the Prosigna™ Breast Cancer Prognostic Gene Signature Assay, based on the provided FDA 510(k) summary.
1. Table of Acceptance Criteria and Reported Device Performance
The FDA 510(k) summary for the Prosigna assay demonstrates clinical performance relative to its intended use as a prognostic indicator for distant recurrence-free survival (DRFS) at 10 years. The acceptance criteria are implicit in the statistical significance and magnitude of the prognostic information provided by the Prosigna Score, both as a continuous variable and when categorized into risk groups.
Acceptance Criteria (Implicit) | Reported Device Performance (ABCSG-8 Study) |
---|---|
Primary Clinical Performance: Prosigna Score must add "significant prognostic information" for DRFS over and above existing clinical and treatment variables (CTS). | For DRFS at 10 years: |
- Prosigna Score as a continuous variable: Added significant prognostic information (p 90%. |
| Clinical Utility (Small Score Changes): Small changes in Prosigna Score (5-10 units) should be statistically significant for time to distant recurrence. | C-index analysis showed statistically significant utility for small changes in Prosigna Score (P
Ask a specific question about this device
(510 days)
CA 92011
SEP 2 0 2011
રિભ: 1101182 Trade/Device Name: NADiA® Pros Vue™ Regulation Number: 21 CFR 866.6040
NADiA® ProsVue™ is an in-vitro diagnostic assay for determining rate of change of serum total prostate specific antigen over a period of time (slope, pg/mL per month). The NADiA® ProsVue™ assay is performed for patients having less than 0.1 ng/mL serum total PSA values (determined by standard-of-care assays that are FDA approved/cleared) in the first sample collected more than 6 weeks after radical prostatectomy. ProsVue™ slope is indicated for use as a prognostic marker in conjunction with clinical evaluation as an aid in identifying those patients at reduced risk for recurrence of prostate cancer for the eight year period following prostatectomy.
The NADiA® ProsVue™ assav is not intended for the diagnosis or for the monitoring of prostate cancer.
ProsVue™ is a two-site immunoassay utilizing an assay specific synthetic DNA sequence as a label with a PCR detection method. Calibrators, controls and samples react with a reagent containing a monoclonal PSA-specific antibody labeled with an assay-specific double-stranded DNA sequence. Then a reagent of paramagnetic microparticles coupled to a monoclonal antibody specific for another site on PSA is added and allowed to react to form a specific sandwich complex with PSA. After washing the particles to remove reactants, a reagent containing a heat-stable polymerase, specific primers, nucleotides, and a fluorescent dye is added to the washed microparticles. An Applied Biosystems® (AB) 7500 Fast Dx Real-Time PCR instrument is utilized to detect the presence of the monoclonal PSA-specific antibody labeled with an assay specific DNA sequence indicating the levels of PSA in the samples. PSA values of controls and samples are calculated in pg/mL from a calibrator dose-response plot. ProsVue slope is calculated using ProsVue Software from the calculated PSA concentrations of three patient samples collected between six weeks and 20 months post radical prostatectomy.
The assay is made up of Reporter Antibody Reagent, Target Capture Reagent. PCR Reagent, Calibrators, Directions for Use (DFU) and ProsVue Software. Wash Reagent, Sample Diluent, and a three-level assayed control set are provided separately. ProsVue users initially receive the DFU and ProsVue software, which provide instructions and quality control support for the process of sample scheduling, collection and storage. When samples are ready for testing, the user contacts Iris Molecular Diagnostics (IMD), and IMD ships the required reagents.
Here's a breakdown of the acceptance criteria and study details for the NADiA® ProsVue™ device, based on the provided text:
Acceptance Criteria and Device Performance
The document describes pre-specified performance characteristics and then presents the results of the studies conducted. It does not explicitly state "acceptance criteria" as a separate, quantitative table for clinical performance. Instead, it describes performance with results that demonstrate the device's capability. For analytical performance, the limits are established, and the device is shown to meet them.
Table of Performance Metrics and Reported Values
Performance Metric | Acceptance Criteria (Implied / Stated Limits) | Reported Device Performance |
---|---|---|
Analytical Performance | ||
Measuring Interval | N/A - but defined as the range the device can measure. | 0.65 pg/mL to 100 pg/mL |
Limit of Blank (LOB) | Determined with CLSI EP17-A guideline. | 0.17 pg/mL |
Limit of Detection (LOD) | Determined with CLSI EP17-A guideline. | 0.27 pg/mL |
Limit of Quantitation (LOQ) | Determined with CLSI EP17-A guideline. | 0.65 pg/mL |
Linearity | Deviation from linearity less than 24% from 0.65 pg/mL to 100 pg/mL. | Deviation from linearity less than 24% from 0.65 pg/mL to 100 pg/mL. |
Total Variation (%CV) (Precision) | N/A - but demonstrated across low, intermediate, and high samples. | Low Sample: 15.2% |
Intermediate Sample: 9.4% | ||
High Sample: 10.6% | ||
Between-lot Imprecision | Not more than 4.0% | Not more than 4.0% |
Interference (Blood Const.) | Less than 10% interference in assay results at specified concentrations. | Less than 10% interference at tested concentrations. |
Interference (Drugs) | Less than 10% interference in assay results at specified concentrations. | Less than 10% interference at tested concentrations. |
PSA Stability | Stability projections: 99.6% and 99.7% immunoreactive at 20 years at -70°C. | Met these projections based on accelerated stability study. |
Reagent/Control Shelf Life | Defined by control recovery within limits over time. | Kit, Calibrators, PCR Reagent: 41 days |
Controls: 78 days | ||
Target Capture, Reporter Antibody: 41 days | ||
Wash Solution: 41 days | ||
Sample Diluent: 41 days | ||
Clinical Performance | ||
Kaplan-Meier 8-year Probability of "Stable" (Slope ≤ 2 pg/mL/month) | N/A - but demonstrated to be high. | 95.9% (95% CI: 93.4 - 98.4%) |
Kaplan-Meier 8-year Probability of "Stable" (Slope > 2 pg/mL/month) | N/A - but demonstrated to be low. | 37.3% (95% CI: 25.0 - 49.6%) |
Univariate Cox HR (Slope > 2 vs. ≤ 2) | N/A - but demonstrated to be a significant predictor. | HR = 18.3 (95% CI: 10.6 to 31.8; p 2 vs. ≤ 2) |
Ask a specific question about this device
(789 days)
AUG 2 9 2011
Re: K091960
Trade/Device Name: Vysis EGR1 FISH Probe Kit Regulation Number: 21 CFR §866.6040
The Vysis EGR1 FISH Probe Kit is intended to detect deletion of the LSI EGR1 probe target on chromosome 5q in bone marrow specimens and to be used, in addition to cytogenetics, other biomarkers, morphology and other clinical information, at the time of acute myeloid leukemia (AML) diagnosis as an aid in determining prognosis. Deletion of chromosome 5q has been associated with an unfavorable prognosis in AML patients.
The Vysis EGR1 FISH Probe Kit uses fluorescence in situ hybridization (FISH) DNA probe technology to determine deletion status of the LSI EGR1 (containing early growth response 1 gene; location chromosome 5q31) probe target in AML specimens. The Vysis EGR1 FISH Probe Kit also contains the LSI D5S23, D5S721 probe (location chromosome 5p15.2) and serves as a control. The kit consists of one vial containing two DNA FISH probes and four general purpose reagents sufficient to process 20 specimens.
Here's a summary of the acceptance criteria and the study details for the Vysis EGR1 FISH Probe Kit, based on the provided text:
Vysis EGR1 FISH Probe Kit: Acceptance Criteria and Study Details
1. Table of Acceptance Criteria and Reported Device Performance
Acceptance Criteria | Reported Device Performance |
---|---|
Analytical Specificity: | |
LSI D5S23, D5S721 Probe | 100% (95% CI: 98, 100) |
EGR1 Probe | 100% (95% CI: 98, 100) |
Analytical Sensitivity: | |
LSI EGR1/D5S23, D5S721 | 99.6% (95% CI: 99.4, 99.7) for 2R2G signal pattern |
Normal Cut-off: | |
Max. 1R2G patterns | Specimens exceeding 6% or 12 1R2G patterns per 200 nuclei considered abnormal. No normal specimens produced 1R2G signals at or above 6%. |
Reproducibility (Site-to-Site): | |
Overall Agreement (High Positive) | 100% |
Overall Agreement (Low Positive) | 100% |
Overall Agreement (Normal) | 98% (one discordant result out of 60 observations) |
Reproducibility (Lot-to-Lot): | |
Overall Agreement (High Positive) | 100% |
Overall Agreement (Low Positive) | 100% |
Overall Agreement (Normal) | 100% |
Clinical Utility (Agreement with Karyotype): | |
Overall Agreement | 98.90% (179/181) (95% CI: 96.06%-99.70%) |
Negative Percent Agreement | 100% (171/171) (95% CI: 97.80%-100.00%) |
Positive Percent Agreement | 80% (8/10) (95% CI: 49.02%-94.33%) |
2. Sample Size Used for the Test Set and Data Provenance
- Analytical Specificity: Metaphase chromosomes from 5 karyotypically normal males.
- Analytical Sensitivity: Interphase nuclei from 25 bone marrow specimens (karyotypically normal or 5p15 and 5q31 deletion-free).
- Verification of Normal Cut-off: Interphase nuclei from 25 bone marrow specimens (karyotypically normal or 5p15.2 and 5q31 deletion-free).
- Reproducibility (Site-to-Site): 2 high-positive, 2 low-positive, and 2 normal specimens (bone marrow cells mixed to create positive controls).
- Reproducibility (Lot-to-Lot): Same 2 high-positive, 2 low-positive, and 2 normal specimens.
- Clinical Utility (Agreement with Karyotype): 181 bone marrow specimens from an Eastern Cooperative Oncology Group (ECOG) clinical trial (E1900) for AML diagnosis. This appears to be retrospective data from a specific clinical trial. The country of origin is implied to be the US given the ECOG trial.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
- Analytical Specificity: "one technologist" evaluated 100 metaphase nuclei per sample. No specific qualifications are provided for this technologist beyond being a "technologist."
- Analytical Sensitivity: "two technologists" evaluated 100 nuclei per specimen. No specific qualifications provided.
- Verification of Normal Cut-off: "two technologists" evaluated 100 nuclei per specimen. No specific qualifications provided.
- Clinical Utility (Agreement with Karyotype): The ground truth for this section is based on "karyotype" results. While the text refers to a publication by Vance et al. from an ECOG clinical trial, it does not explicitly state the number or qualifications of the experts establishing the karyotype ground truth for this specific study. However, karyotyping is a specialized cytogenetic analysis performed by trained professionals (cytogeneticists/pathologists).
4. Adjudication Method for the Test Set
- Analytical Specificity, Sensitivity, Normal Cut-off: The text mentions evaluation by one or two technologists. For cases where two technologists evaluated, it doesn't explicitly state an adjudication method if discrepancies occurred. It simply states "Each technologist evaluated..." or "evaluated by two technologists." This suggests independent evaluation, but no formal adjudication process is described for reconciling differences.
- Reproducibility: A mean and standard deviation were calculated across replicates and sites/lots, indicating a statistical comparison of results, not a consensus-based adjudication of individual cases.
- Clinical Utility (Agreement with Karyotype): The comparison is against established karyotype results, which are considered the ground truth. There's no further adjudication stated for these 181 specimens beyond that initial karyotype.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No Multi-Reader Multi-Case (MRMC) comparative effectiveness study was done comparing human readers with AI assistance vs. without AI assistance. This device is a FISH probe kit, not an AI-assisted diagnostic tool.
6. Standalone (Algorithm Only) Performance
This device is a physical FISH probe kit designed for laboratory use with microscopic interpretation by trained personnel. It does not involve a standalone algorithm or AI for interpretation. Performance metrics like analytical specificity, sensitivity, and reproducibility are for the kit's ability to produce expected signal patterns when interpreted by human technologists.
7. Type of Ground Truth Used
- Analytical Specificity: Metaphase chromosomes from karyotypically normal individuals, with "correct locus" hybridization serving as the ground truth.
- Analytical Sensitivity: Expected normal signal pattern (2R2G) in interphase nuclei from karyotypically normal or deletion-free bone marrow specimens.
- Verification of Normal Cut-off: Normal bone marrow specimens that were karyotypically normal or 5p15.2 and 5q31 deletion-free.
- Reproducibility: Known status of specimens (high-positive, low-positive, normal) created by mixing positive and normal bone marrow cells.
- Clinical Utility (Agreement with Karyotype): Cytogenetic results (karyotype) from clinical AML diagnosis ("-5/del5q" vs. normal karyotype).
8. Sample Size for the Training Set
The document does not explicitly describe a separate "training set" in the context of machine learning or algorithm development, as this device is a probe kit. The studies described are for analytical validation and clinical correlation.
9. How the Ground Truth for the Training Set Was Established
Not applicable, as there's no mention of a traditional "training set" for an algorithm in this submission. The validation studies rely on established biological principles and comparisons to conventional cytogenetic methods.
Ask a specific question about this device
(582 days)
In Situ Hybridization (FISH) reagents
Classification
Class II
Regulation Number
21 CFR 866.6040
Regulation Name: Regulatorv Class: Product Code: Dated: Received:
Vysis CLL FISH Probe Kit 21 CFR §866.6040
The Vysis CLL FISH Probe Kit is intended to detect deletion of the LS1 TP53, LSI ATM, and LSI D13S319 probe targets and gain of the D12Z3 sequence in peripheral blood specimens from untreated patients with B-cell chronic lymphocytic leukemia (CLL). The assay may be used to dichotomize CLL (the 13q-, +12, or normal genotype group versus the 11q- or 17p- group) and may be used as an aid in determining disease prognosis in combination with additional biomarkers, morphology, and other clinical information. The Vysis CLL FISH Probe Kit is not intended for use in selection of therapy or in monitoring of residual disease.
The Vysis CLL FISH Probe Kit uses fluorescence in situ hybridization (FISH) DNA probe technology to determine deletions of the locus-specific identifier (LSI) TP53, LSI ATM, and LSI D13S319 probe targets and gain of the D12Z3 sequence.
The Vysis CLL FISH Probe Kit (List No. 4N02-020) consists of two DNA FISH probe sets and three general purpose reagents sufficient to process 20 assays.
- . LSI TP53 SpectrumOrange/ATM SpectrumGreen Probe
- LSI D13S319 SpectrumOrange/13q34 SpectrumAqua/CEP 12 SpectrumGreen Probe .
- DAPI II Counterstain .
- NP-40 .
- 20X SSC Salt .
Here's a summary of the acceptance criteria and the study details for the Vysis CLL FISH Probe Kit, based on the provided 510(k) summary:
Acceptance Criteria and Device Performance
Note: The 510(k) summary for the Vysis CLL FISH Probe Kit primarily focuses on analytical performance (specificity, sensitivity, normal cut-off values, precision, reproducibility) and concordance with an existing clinical assay rather than traditional "acceptance criteria" related to a new AI/CADe device. The provided tables outline the device's technical performance.
Acceptance Criterion / Performance Metric | Reported Device Performance (Mean/Overall) |
---|---|
Analytical Specificity | 100% for all probes (LSI TP53, LSI ATM, LSI D13S319, LSI 13q34, CEP 12) |
Analytical Sensitivity (Expected normal signal pattern: 2 signals/nucleus) | LSI TP53: 97.98% |
LSI ATM: 98.68% | |
LSI D13S319: 98.60% | |
CEP 12: 98.94% | |
Normal Cut-off Values (Percentage of abnormal nuclear FISH patterns) | LSI TP53 (1 signal): 7.0% (14/200 nuclei) |
LSI ATM (1 signal): 6.0% (12/200 nuclei) | |
LSI D13S319 (1 signal): 5.5% (11/200 nuclei) | |
LSI D13S319 (0 signal): 1.5% (3/200 nuclei) | |
CEP 12 (3 signals): 2.5% (5/200 nuclei) | |
Precision (Mean % abnormal cells for various probes/abnormalities) | Ranges from 0.0% to 73.2% across different samples and probes (SDs provided in Tables 4-8) |
Reproducibility (Overall Agreement, Site to Site by Probe) | TP53 (17p-): 100% |
ATM (11q-): 90% | |
CEP 12 (+12): 100% | |
D13S319 1x (13q-): 90% | |
D13S319 2x (13q-): 90% | |
Reproducibility (Prognostic Category, Generalized Kappa) | Kappa = 0.86 (Strength: "Almost Perfect") |
Method Concordance (AMT vs. RFT for Prognostic Category) | Overall Agreement: 97% (62/64) |
Lower bound one-sided 95% CI: 90% |
Study Details
-
Sample sizes used for the test set and the data provenance:
- Analytical Specificity: 5 karyotypically normal male peripheral blood cultures (retrospective, assumed domestic, not explicitly stated).
- Analytical Sensitivity: 25 karyotypically normal patient peripheral blood cultures (retrospective, assumed domestic, not explicitly stated).
- Analytical Characterization of Normal Cut-off Values: 25 karyotypically normal patient peripheral blood cultures (retrospective, assumed domestic, not explicitly stated).
- Precision:
- Precision Study 1: 2 negative peripheral blood specimens (lacking abnormalities) and 8 additional specimens with at least one abnormality. Likely retrospective, but not explicitly stated.
- Precision Study 2: 8 different patient specimens (blinded panel). Likely retrospective, but not explicitly stated.
- Reproducibility: A blinded 20-member slide panel representing five Döhner classifications. Likely retrospective, but not explicitly stated.
- Method Concordance: 64 specimens with pre-established Döhner classifications. Retrospective, as classifications were "based on previous results using the RFT." Provenance not explicitly stated.
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Analytical Specificity: One technologist. Qualifications not specified beyond being a "technologist."
- Analytical Sensitivity: Not explicitly stated, but each technologist evaluated 100 nuclei per specimen, implying multiple technologists were involved. Qualifications not specified.
- Analytical Characterization of Normal Cut-off Values: Not explicitly stated, but implies multiple technologists based on the sensitivity study. Qualifications not specified.
- Precision: Not specified, but involved counting signals, implying trained technologists.
- Reproducibility: Not specified, but involved three different laboratories, implying multiple trained personnel.
- Method Concordance: The "Reference FISH Test (RFT)" was used to establish initial Döhner classifications. The original Shanafelt study (referred to as the RFT) implies multiple pathologists/cytogeneticists. This study used "previous results" from the RFT as its ground truth.
-
Adjudication method for the test set:
- No formal adjudication method (like 2+1 or 3+1) is explicitly described for establishing the ground truth or validating results in the analytical studies. Results were typically enumerated by technologists and aggregated.
- For the Reproducibility study, "overall agreement" between three testing sites was assessed, but a specific adjudication process for discrepancies is not detailed beyond reporting disagreement counts.
- For Method Concordance, the ground truth was the "Reference FISH Test (RFT) used in the Shanafelt study," which effectively served as the gold standard, and the AMT results were compared against it.
-
If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No MRMC comparative effectiveness study involving human readers improving with AI vs. without AI assistance was reported. This device is a FISH probe kit, not an AI-powered diagnostic system. The studies focused on the analytical and clinical validity of the probe kit itself.
-
If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:
- This is not an algorithm-only device. It is a laboratory assay (FISH probe kit) where human technologists perform the analysis by counting fluorescent signals under a microscope. The studies performed were of the laboratory kit's performance, not an automated algorithm.
-
The type of ground truth used:
- Analytical Specificity, Sensitivity, Normal Cut-off, Precision: Based on visual identification and enumeration of FISH signals by trained technologists, comparing observed signal patterns to expected normal or abnormal patterns in karyotypically normal and patient samples. The "expected normal" pattern serves as a form of expert-derived ground truth.
- Reproducibility: Comparison of results across multiple labs/readers for the same samples. The "agreement" between sites implies a consensus or majority rule for comparison, but the ultimate ground truth for a given sample's true Döhner classification is not explicitly detailed but likely derived from expert pathological/cytogenetic review.
- Method Concordance: The ground truth was established by a "Reference FISH Test (RFT) used in the Shanafelt study," which is stated to have its clinical validity documented via peer-reviewed literature. This implies a ground truth based on established clinical and pathological diagnosis as determined by a validated method.
-
The sample size for the training set:
- This device is a diagnostic kit, not an AI/machine learning algorithm, so there is no "training set" in the conventional sense of AI development. The studies described are for analytical validation and clinical concordance.
-
How the ground truth for the training set was established:
- Since there's no AI "training set," this question is not applicable. The device's performance characteristics (specificity, sensitivity, cut-offs) are established through testing on defined sets of samples (e.g., karyotypically normal individuals, patients with confirmed CLL aberrations) where the expected outcome is known or determined by expert review using established cytogenetic methods.
Ask a specific question about this device
(248 days)
Classification
Class II, regulated under 21 CFR 866.6040, product code NYI
9.
The Netherlands
JAN 2 8 2011
Re: K101454
Trade/Device Name: MammaPrint® Regulation Number: 21 CFR§866.6040
MammaPrint is a qualitative in vitro diagnostic test service, performed in a central laboratory, using the gene expression profile of fresh breast cancer tissue samples to assess a patient's risk for distant metastasis (up to 10 years for patients less than 61 years old, up to 5 years for patients' ≥ 61 years).
The test is performed for breast cancer patients with Stage I or Stage II disease, with tumor size
The MammaPrint service is a microarray based gene expression analysis of a tumor. The analysis is based on several processes: isolation of RNA from frozen tumor tissue sections. DNA'se treatment of isolated RNA, linear amplification and labeling of DNA'se treated RNA, cRNA purification, hybridization of the cRNA to the MammaPrint microarray, scanning the MammaPrint microarray and data acquisition (feature extraction), calculation and determination of the risk of recurrence in breast cancer patients.
The MammaPrint analysis is designed to determine the gene activity of specific genes in a tissue sample compared to a reference standard. The result is an expression profile, or fingerprint, of the sample.
The correlation of the sample expression profile to a template (the mean expression profile of 44 tumors with a known good clinical outcome) is calculated and the molecular profile of the sample is determined (Low Risk, High Risk).
The MammaPrint device is a qualitative in vitro diagnostic test service that assesses a patient's risk for distant metastasis in breast cancer. The study presented focuses on the analytical performance and reproducibility of the device, particularly when new scanners and bio-analyzers are used, and when the service is performed in different central laboratory sites.
Here's a breakdown of the requested information:
1. Acceptance Criteria and Reported Device Performance
The provided document describes analytical performance evaluation rather than a clinical study with specific diagnostic accuracy metrics (like sensitivity, specificity, AUC). The acceptance criteria are based on statistical comparisons to the predicate device's accepted variance and predetermined validation acceptance criteria.
Acceptance Criteria Category | Specific Acceptance Criteria | Reported Device Performance |
---|---|---|
Micro Array Scanners | Difference between mean, median, and standard deviation of MammaPrint Indices (between new and predicate scanners) falls within 1.96 * 0.030 (accepted variance of predicate device). | Differences fall within the accepted variance of the predicate device (1.96 * 0.030). |
Bio Analyzers (RIN Measurements) | No significant difference in RNA Integrity Number (RIN) measurements between FDA-cleared and new Bio-analyzers (p-value for statistical test should indicate no significant difference). | Wilcoxon signed ranks test showed no significant difference in RIN measurements between FDA cleared and New Bioanalyzers (p=0.46 and p=0.47 respectively). |
Central Laboratory Site Comparison (RNA Quality/RIN) | No significant difference in RNA quality (RIN measurement) between Amsterdam (L1) and US lab (L2). All results must comply with predefined validation acceptance criteria. | No significant difference in RNA quality of RIN measurement between Amsterdam (L1) and US lab (L2). All results comply with predefined validation acceptance criteria. |
Central Laboratory Site Comparison (MammaPrint Index/Outcome) | No significant difference in MammaPrint Indices and outcome between European/Dutch (L1) and US/California (L2) lab. All results must comply with predefined validation acceptance criteria. | No significant difference in MammaPrint Indices between European/Dutch (L1) and US/California (L2) lab. All results comply with predefined validation acceptance criteria. |
2. Sample Size and Data Provenance
The study focused on analytical validation rather than a clinical diagnostic accuracy study.
- Micro Array Scanners: Used a selection of 25 slides from which MammaPrint Indices were generated. These slides consisted of approximately 100 samples and 20 control samples (LRC and HRC). The samples were analyzed during regular diagnostics. The provenance (country of origin) is not explicitly stated for these specific samples, but the company is based in the Netherlands with a US lab mentioned later. The samples were likely retrospective as they were previously run for diagnostic purposes.
- Bio Analyzers: A selection of about 60 samples covering the complete RIN measuring range. Data provenance is not explicitly stated.
- Central Laboratory Sites (RNA Isolation): 36 samples selected based on sufficient tissue material. The samples had previously shown acceptable RNA quality at the Amsterdam lab.
- Central Laboratory Sites (Amplification/Labeling and Hybridization): 99 samples were used. All samples had been previously subjected to a diagnostic MammaPrint test at the Amsterdam Lab (Lab 1). The result distribution was 54 high risk, 38 low risk, and 7 borderline.
- Data Provenance: The central laboratories include Amsterdam (Netherlands) and a US laboratory (California). The samples used for comparison were run in both labs. This implies a retrospective analysis of samples previously processed.
3. Number of Experts and Qualifications for Ground Truth
The document does not describe a process where experts established a ground truth for a test set in the traditional sense of diagnostic accuracy studies (e.g., radiologists reviewing images). The ground truth for the analytical validation appears to be the results obtained from the predicate device or a previously established reference standard within Agendia's own validated processes.
4. Adjudication Method
Not applicable for this type of analytical validation study. There was no expert adjudication process for establishing a "ground truth" for a test set. The comparisons were statistical analyses of measurements (MammaPrint Indices, RIN values) between different instrumentations or lab sites.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No, this document does not describe a MRMC comparative effectiveness study involving human readers. This is an analytical validation of the device's technical performance and reproducibility, not a study on how human interpretation improves with AI assistance.
6. Standalone (Algorithm Only) Performance
Yes, the study describes the standalone performance of the MammaPrint algorithm, focusing on the analytical performance characteristics like precision and reproducibility across different hardware and lab sites. It evaluates the consistency of the algorithm's output (MammaPrint Index, RIN) under various conditions.
7. Type of Ground Truth Used
The "ground truth" in this context refers to the established and accepted performance of the predicate device and the existing, validated MammaPrint procedures. For instance:
- For scanner validation, the MammaPrint Indices generated by the FDA-cleared predicate scanners served as the reference.
- For bio-analyzer validation, the RIN measurements from FDA-cleared bio-analyzers served as the reference.
- For central lab site comparison, the results from the Amsterdam lab (L1), where samples had been previously subjected to diagnostic MammaPrint, served as the reference for comparison with the US lab (L2).
This is an internal reference standard/predicate device comparison.
8. Sample Size for the Training Set
The document does not describe a training set for a machine learning algorithm. MammaPrint is a gene expression profiling test, which typically relies on a fixed algorithm/signature derived from earlier research, not an adaptive machine learning model that is "trained" in an ongoing manner. The document focuses on the analytical validation of the test's execution and reproducibility.
9. How the Ground Truth for the Training Set Was Established
Not applicable, as there is no mention of a training set for a machine learning algorithm in the provided text. The MammaPrint test itself is based on a specific gene expression profile correlated to clinical outcome. The methods for establishing this original correlation (which forms the basis of the "template" mentioned in the device description) would be found in the foundational clinical studies for MammaPrint, not in this analytical validation document. The "template" (mean expression profile of 44 tumors with a known good clinical outcome) explicitly mentioned in the device description is effectively the core of their model, derived from prior clinical data.
Ask a specific question about this device
(603 days)
Classification
Class II, regulated under 21 CFR 866.6040, product code NYI
9.
Netherlands
DEC 1 1 2009
Re: K081092
Trade/Device Name: MammaPrint® Regulation Number: 21 CFR §866.6040
MammaPrint® is a gualitative in vitro diagnostic test service, performed in a single laboratory, using the gene expression profile of fresh breast cancer tissue samples to assess a patients' risk for distant metastasis (up to 10 years for patients less than 61 years old, up to 5 years for patients ≥ 61 years).
The test is performed for breast cancer patients with Stage I or Stage II disease, with a tumor size of ≤ 5.0 cm and lymph node negative. The MammaPrint result is indicated for use by physicians as a prognostic marker only, along with other clinicopathological factors.
The MammaPrint service is a microarray based gene expression analysis of a tumor. The analysis is based on several processes: isolation of RNA from frozen tumor tissue sections, DNA'se treatment of isolated RNA, linear amplification and labeling of DNA'se treated RNA, cRNA purification, hybridization of the cRNA to the MammaPrint microarray, scanning the MammaPrint microarray and data acquisition (feature extraction), index calculation and determination of the risk of distant recurrence in breast cancer patients.
The MammaPrint analysis is designed to determine the gene activity of specific genes in a tissue sample compared to a reference standard. The result is an expression profile, or fingerprint, of the sample.
The correlation of the sample expression profile to a template (the mean expression profile of 44 tumors with a known good clinical outcome) is calculated and the molecular profile index of the sample is determined (Low Risk, High Risk).
Here's a breakdown of the acceptance criteria and the study information for the MammaPrint® device, based on the provided text:
MammaPrint® Acceptance Criteria and Study Information
1. Acceptance Criteria and Reported Device Performance
The provided document primarily focuses on analytical performance as internally validated and clinical performance as demonstrated through peer-reviewed studies. It doesn't explicitly state "acceptance criteria" in a typical pass/fail numerical sense for clinical performance, but rather presents the performance achieved by the device in various clinical settings.
Acceptance Criteria Category | Specific Metric/Description | Reported Device Performance/Finding |
---|---|---|
Analytical Performance | Accuracy of measurement (based on repeated experiments of control samples) | 98.5% Analytical Accuracy of measurement |
Accuracy of classifying a sample as High Risk or Low Risk | At least 98.9% (i.e., 1.1% false negative classification) | |
Percentage of "Borderline Samples" | Less than 5% of analyzed samples are considered "Borderline Samples" | |
Classification accuracy for "Borderline Samples" | Approximately 90% (i.e., 10% chance of false classification) | |
Clinical Performance | **Assessment of risk for distant metastasis (up to 10 years for patients |
Ask a specific question about this device
Page 1 of 2