(374 days)
The Bayer Immuno 1™ CA 15-3™ Assay is an in vitro, diagnostic, solid phase immunoassay intended to quantitatively measure CA 15-3™ Assay values in human serum on the Bayer Immuno 1 system. When used in conjunction with other clinical and diagnostic procedures, serial testing with the CA 15-3™ Assay is useful for monitoring the course of disease and therapy in metastatic breast cancer patients, and for detection of recurrence in previously treated Stage II, with greater than 2 positive lymph nodes, or Stage III breast cancer patients.
The Bayer Immuno IIM CA 15-3™ Assay is a sandwich immunoassay in which one monoclonal antibody (115D8) is conjugated to fluorescein (R1) and a second monoclonal antibody (DF3) is conjugated to alkaline phosphatase (R2). Immuno 1 magnetic particles coated with antibody; the R1 conjugate, and patient sample, calibrator, or control are mixed simultaneously and incubated at 37℃ on the system. The R2 conjugate is then added, which binds to the immobilized CA 15-3™ to form a sandwich immunocomplex on the solid phase. The magnetic particles complexed with the immunological sandwich are then washed to separate unbound molecules, and a colorimetric substrate is added. The rate of conversion of substrate to a compound with absorbance at 405 and 450 nm is measured; the measured rate is proportional to the concentration of CA 15-3™ assay values in the sample. A cubic-through-zero curve fitting algorithm is used to generate standard curves.
The provided text describes the Bayer Immuno 1™ CA 15-3™ Assay and studies to demonstrate its safety and effectiveness.
Here's a breakdown of the acceptance criteria and study details:
1. Table of Acceptance Criteria and Reported Device Performance
The document provides various performance metrics, but not a single, explicit "acceptance criteria" table with corresponding "reported performance." Instead, individual study results serve as the reported performance, which implicitly met the internal or regulatory acceptance criteria for each test.
Here's a summary derived from the text, focusing on the quantitative results:
| Criteria/Performance Metric | Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|---|
| Specificity (Interference) | Negligible effect on CA 15-3™ recovery by common interferants. | Endogenous Interferants: -0.2% to 1.6% maximum effect for Triglycerides (900 mg/dL), Immunoglobulin (5.3 g/dL), Hemoglobin (1.0 g/dL), Heparin (0.15 mg/mL), Bilirubin (25 mg/dL), Albumin (6.5 g/dL). Anti-neoplastic Drug Interferants: Drug Pool #1 showed -21.3% effect (attributed to Methotrexate). Methotrexate at 300 ug/mL showed ~65% inhibition. Drug Pool #2 and #3 had 0% and 2.0% effect respectively. OTC Drugs/Vitamins: -3.5% to 2.1% maximum effect for various vitamins, caffeine, codeine, acetaminophen, ibuprofen, and aspirin. |
| Heterophilic Antibodies | Linear recovery upon dilution. | Percent recovery for samples with high rheumatoid factor/human anti-mouse antibody titers ranged from 83.9% to 92.6% when diluted, indicating a lack of significant interference. |
| Linearity | Recoveries of intermediate dilutions within acceptable range of expected. | Recoveries of intermediate dilutions (from pools >180 U/mL) consistently between 98.2% and 105.2% of the expected value over the entire calibration range. |
| Hook Effect | No "hooking" back into assay range at high concentrations. | No "hook effect" observed up to 28,500 U/mL. |
| Spiked Recovery | Acceptable percentage recovery of spiked antigen. | Recoveries ranged from 92.9% to 122.9% for antigen spiked at 30 and 120 U/mL. |
| Parallelism | No significant deviation from linearity in diluted patient samples. | Linear regression analysis showed no deviation from linearity. Recoveries ranged from 88.2-116.9%. |
| Minimum Detectable Concentration (MDC) | ≤ 0.2 U/mL (Method Sheet Claim) | 0.13 U/mL (based on 1336 determinations). |
| Imprecision (Within-run CV) | Low CV across various analyte concentrations. | Ranged from 1.3% to 3.4% across controls and calibrator levels (0.1 to 190.9 U/mL). |
| Imprecision (Total CV) | Low CV across various analyte concentrations. | Ranged from 3.0% to 4.0% across controls and calibrator levels (0.1 to 190.9 U/mL). |
| Reagent Shelf-Life Stability | Product performance within stability specifications at 18 months. | Data for four lots indicated product within stability specifications at 18 months. Calibrator sensitivity and control recoveries within acceptable limits at the latest time point tested. |
| Reagent On-System Stability | Stable for 30 days. | On-system stability studies on four lots supported a 30-day recommendation. Specifications for minimum calibrator sensitivity and control recovery were met. |
| Reagent Shipping Stability | No significant change in performance after heat/chill cycles. | No significant change in control recovery after three heat/chill cycles; recoveries remained well within specification. |
| Calibrator Shelf-Life Stability | Stable for 12 months at 2-10°C or below. | Two lots completed 52 weeks, one completed 26 weeks. Recoveries of calibrators stored at elevated temperatures exceeded ±13% specification, but control materials remained within specifications. Interpretation based on the provided text, it seems the control recoveries being within spec is the key, despite calibrator recovery exceeding ±13%. |
| Calibrator Open Vial Stability | Stable for 30 days at 2-8°C. | Control recoveries remained within specifications at all tested timepoints (0, 1, 3, 5 weeks) for two lots. |
| Calibrator Shipping Stability | Stable for up to three freeze/thaw cycles. | Recoveries did not exceed percent difference specification after one, two, or three freeze/thaw cycles for three lots. |
| Monitoring Metastatic BC (Sensitivity to Clinical Change) | Changes in Immuno 1 CA 15-3™ Assay values consistent with clinical condition in ≥ 60% of patients. | 64% (98/152) sensitivity for detecting changes in clinical status. |
| Monitoring Metastatic BC (Positive Predictive Value) | Not explicitly stated, but high PPV for changes in assay values. | 95% (98/103) positive predictive value of a ≥ 25% change in serial assay values. |
| Early Detection of Recurrence (Sensitivity) | Longitudinal increases prior to or in conjunction with clinical evidence in ≥ 40% of patients that recur. | 53% sensitivity (Immuno 1) for breast cancer recurrence (compared to 53% for predicate Biomira assay). |
| Early Detection of Recurrence (Specificity) | Not explicitly stated, but high specificity for remaining disease-free. | 96% specificity (Immuno 1) for remaining disease-free (compared to 91% for predicate Biomira assay). |
| Correlation with Predicate Device (R) | High correlation (e.g., >0.90) | Immuno 1 vs. Biomira: R = 0.93 |
| Immuno 1 vs. Centocor: R = 0.96 | ||
| Normal Range 97.5th Percentile | Consistent with existing literature and predicate device. | 35.9 U/mL (Immuno 1) for total healthy population (199 women). This was consistent with 35.6 U/mL for Centocor RIA and 31.3 U/mL for Biomira RIA. |
2. Sample Size Used for the Test Set and the Data Provenance
- Nonclinical Studies (Interference, Linearity, Hook Effect, Spiked Recovery, Parallelism, MDC, Imprecision):
- MDC: 1336 determinations of the Level 1 calibrator (0 U/mL).
- Imprecision: Varied from 301 to 1340 observations per level, with tests performed over 10 days in two runs per day.
- Other nonclinical studies: Specific sample numbers are not provided for each test (e.g., "Medical Decision Pool," "three clinical sample pools," "seven patients" for spiked recovery), but the overall volume of work suggests comprehensive testing.
- Provenance: Serum-based controls, patient samples, internal serum-based control (Medical Decision Pool), and cultured cell antigen derived from an in vitro culture of ZR75-1 Anchorage Dependent Cells. The testing was performed by Bayer Corporation, Tarrytown, N.Y., and at four US clinical trial sites.
- Method Comparison Studies:
- Test Set Size: 501 human serum samples.
- Data Provenance: Samples obtained from BioClinical Partners, Inc. (Sharon, MA) and Bayer Corporation, Business Group Diagnostics (Tarrytown, NY). The samples included various clinical classifications:
- Normal (Premenopausal): 100
- Normal (Postmenopausal): 100
- Breast Cancer (Single Point): 150
- Lung Cancer: 35
- Ovarian Cancer: 35
- Colorectal Cancer: 35
- Benign Breast Disease: 46
- Unknown disease: 1 (with elevated CA 15-3™)
- Retrospective/Prospective: The method comparison study appears to be retrospective, using existing collected serum samples.
- Multi-site Clinical Studies:
- Monitoring Metastatic Breast Cancer: 158 metastatic breast cancer patients.
- Early Detection of Recurrence: 182 Stage 2 and Stage 3 breast cancer patients (17 recurred, 120 remained disease-free were analyzed for longitudinal trends; 15/17 recurred patients and 103/120 disease-free patients were compared to Biomira).
- Distribution of CA 15-3™ Concentrations:
- Healthy Women: 418 (216 premenopausal, 202 postmenopausal)
- Pregnant Women: 81
- Benign Breast Disease: 114
- Other Benign Diseases: 180
- Breast Cancer (various stages): 318
- Non-breast cancers: 210
- Data Provenance: Performed at four US clinical trial sites (Fairfax Prince William Hematology Oncology Associates, Johns Hopkins Hospital, M .D. Anderson Cancer Center, Memorial Sloan-Kettering Cancer Center, Hartford Hospital).
- Retrospective/Prospective: These studies involved longitudinal monitoring, implying a prospective collection for this specific evaluation, though the initial patient enrollment for monitoring might have started with existing patient cohorts.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts
- For Method Comparison and Distribution Studies: The "ground truth" for the disease status (normal, breast cancer, other cancers, benign conditions) was established by the clinical classification of the patient samples, presumably by the physicians or diagnostic centers providing them (BioClinical Partners, Inc. and Bayer Corporation). The specific number and qualifications of individual experts for classifying these samples are not explicitly stated in the provided text.
- For Monitoring Metastatic Breast Cancer and Early Detection of Recurrence:
- Clinical Status/Disease Recurrence: The ground truth for changes in clinical status and disease recurrence was determined by "objective measurement of recurrence (CT scans, MRI, X-rays, bone scans, liver scans, physical examination)" and "physician evaluations, and diagnostic testing."
- Principal Investigators: Five principal investigators from major US medical centers/hospitals are named, suggesting highly qualified medical professionals were involved in the clinical assessment and determination of patient status: Dr. Roy Beveridge, Dr. Daniel W. Chan, Dr. Herbert A. Fritsche, Dr. Morton K. Schwartz, and Dr. Alan H. B. Wu. Their specific qualifications (e.g., "radiologist with 10 years of experience") are not detailed, but their affiliation with prominent cancer centers implies high expertise. The study states "patient histories were thoroughly documented" implying clinical expert input.
4. Adjudication Method for the Test Set
The document does not explicitly describe a formal "adjudication method" (like 2+1 or 3+1 consensus) for the ground truth of clinical status or disease recurrence. The assessment appears to be based on "objective measurement of recurrence" and "physician evaluations, and diagnostic testing" by the clinical trial sites and their principal investigators. This suggests a multi-faceted clinical assessment rather than a single panel adjudication.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs without AI Assistance
No, an MRMC comparative effectiveness study involving human readers and AI assistance was not conducted. This device is an in vitro diagnostic assay (a laboratory test for measuring a biomarker), not an imaging AI diagnostic tool that assists human readers.
The comparative effectiveness studies performed were:
- Method Concordance: Comparison of the Immuno 1 CA 15-3™ Assay with two predicate devices (Biomira TRUQUANT® BR™ RIA and Centocor CA 15-3™ RIA) using patient serum samples.
- Clinical Utility: Evaluation of the Immuno 1 CA 15-3™ Assay's ability to correlate with clinical status changes and detect recurrence, often with comparisons to the Biomira TRUQUANT® BR™ RIA.
6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) Was Done
Yes, the studies are inherently standalone as the device is an in vitro diagnostic immunoassay. The device itself performs the measurement and provides a quantitative CA 15-3™ value. Its performance is evaluated directly against clinical outcomes or other reference methods, and it is intended to be used by laboratory personnel, with results interpreted by physicians in conjunction with other clinical information. There is no "human-in-the-loop" component in the algorithm's direct performance; the physician makes decisions based on the algorithm's output.
7. The Type of Ground Truth Used
The ground truth used in these studies varied depending on the objective:
- Clinical Classification: For the method comparison and distribution studies, "clinical classification" (e.g., normal, breast cancer, lung cancer, benign breast disease) of the patient samples was used. This would be based on a combination of pathology reports, clinical diagnoses, and patient history.
- Objective Measurement of Recurrence/Clinical Progression: For the management and early detection of recurrence studies, the ground truth was established by "objective measurement of recurrence (CT scans, MRI, X-rays, bone scans, liver scans, physical examination)" and "physician evaluations" of clinical status. This is a form of outcomes data and clinical expert assessment.
8. The Sample Size for the Training Set
The provided document describes studies for validation of the device's performance, not the development or training of an AI algorithm. Therefore, there is no explicit training set sample size mentioned. The "nonclinical studies" and "method comparison studies" serve to characterize the assay, and the "multi-site clinical studies" validate its clinical utility. These are test and validation sets in a regulatory context, not training sets for an algorithm.
9. How the Ground Truth for the Training Set Was Established
As explained in point 8, this document does not describe the development of an AI algorithm with a training set. Instead, it details the validation of a laboratory immunoassay. The "ground truth" for various aspects of the validation (e.g., disease status, clinical progression, recurrence) was established through clinical classifications, objective measurements, and physician evaluations, as described in point 7.
§ 866.6010 Tumor-associated antigen immunological test system.
(a)
Identification. A tumor-associated antigen immunological test system is a device that consists of reagents used to qualitatively or quantitatively measure, by immunochemical techniques, tumor-associated antigens in serum, plasma, urine, or other body fluids. This device is intended as an aid in monitoring patients for disease progress or response to therapy or for the detection of recurrent or residual disease.(b)
Classification. Class II (special controls). Tumor markers must comply with the following special controls: (1) A guidance document entitled “Guidance Document for the Submission of Tumor Associated Antigen Premarket Notifications (510(k)s) to FDA,” and (2) voluntary assay performance standards issued by the National Committee on Clinical Laboratory Standards.