(264 days)
The AMDL-ELISA DR-70® (FDP) immunoassay is designed for IN VITRO DIAGNOSTIC USE ONLY for the quantitative measurement of DR-70® (FDP) in human serum. Serial testing using the AMDL- ELISA DR-70® (FDP) is to be used as an aid in monitoring the disease progression in patients who have been diagnosed previously with colorectal cancer. Results of DR-70® (FDP) testing should be used in conjunction with other clinical modalities that are standard of care for monitoring disease progression in these patients.
The AMDL, Inc. DR-70® (FDP) assay is an ELISA based assay utilizing removable strips in a 96 micro titer plate well format. The wells are coated with affinity purified rabbit anti-DR-70® (FDP) antibodies. The DR-70® (FDP) in diluted sera (1:200) is captured from the sera by these antibodies immobilized on the well of a micro titer plate. After a wash step, anti-DR-70® (FDP) antibodies conjugated to horseradish peroxidase are added to the wells. If the DR-70® (FDP) antigen is present, the anti-human fibrinogen peroxidase complex will bind to the captured tumor marker to form an immunological sandwich with the immobilized antibodies. After a second wash step, the enzyme substrate 3,3',5'-tetramethylbenzidine (TMB) is added to the well. The end point is read in a micro plate reader at 450 nm once the reaction is stopped with 0.1N HCl. The intensity of the color formed is proportional to the amount of DR-70® (FDP) in the serum. The amount is quantified by interpolation from a standard curve using the calibrators provided with the kit.
Here's a summary of the acceptance criteria and the study that proves the device meets them, based on the provided text:
Device: AMDL-ELISA DR-70® (FDP)
Intended Use: For IN VITRO DIAGNOSTIC USE ONLY for the quantitative measurement of DR-70® (FDP) in human serum. Serial testing using the AMDL-ELISA DR-70® (FDP) is to be used as an aid in monitoring the disease progression in patients who have been diagnosed previously with colorectal cancer. Results of DR-70® (FDP) testing should be used in conjunction with other clinical modalities that are standard of care for monitoring disease progression in these patients.
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state "acceptance criteria" in a table format. Instead, it presents various performance characteristics and concludes that the device is "informative" for monitoring disease progression in colorectal cancer and supports a finding of substantial equivalence. For this table, I will use the established performance metrics as the defacto "acceptance criteria" through their demonstration of "informativeness" or adequate performance for the intended use in comparison to a predicate device and clinical status.
| Performance Metric | Acceptance Criteria (Implied / Demonstrated) | Reported Device Performance |
|---|---|---|
| Recovery | Acceptable recovery across a range of spiked concentrations. | % Mean Recovery ranged from 99% to 110% across 5 spike levels (1.5 µg/ml to 10 µg/ml) when tested in patient serum and compared to theoretical values. Concerns of sample matrix effects noted but overall analysis found "kit is a quantitative test." |
| Linearity | Linearity in the assay range. | Concentrations were statistically found to be linearly related for dilutions above 0.625 µg/ml. |
| Precision | Acceptable variability. | Within-run CV: 1.27% to 5.53% |
| Total Variability: 9.91% (at 2.739 µg/ml) to 28.21% (at 0.240 µg/ml) | ||
| Day-to-day Variation: 3.43% to 8.58% | ||
| Run-to-run variation: Max 5.53%. | ||
| Analytical Sensitivity (MDC) | Low minimal detectable concentration. | 0.06 µg/ml |
| Functional Sensitivity | Low functional sensitivity. | Between 0.052 and 0.063 µg/ml (compares well to Analytical Sensitivity) |
| Interference | No significant interference from common substances and drugs. | No interference (recovery outside 10%) from added hemoglobin (up to 500 mg/dl), bilirubin (up to 30 mg/dl), triglycerides (up to 1000 mg/dl), heparin (500 U/ml), and a list of common pharmaceutical agents. |
| Hook Effect | No hook effect observed within a relevant range. | No evidence of a hook effect found up to 250 µg/ml. |
| Clinical Informativeness (for Progression Monitoring - 15% increase as threshold) | Performance greater than clinical diagnoses based on chance alone. | Per Visit Analysis: |
- Sensitivity: 65.19% (SD 2.58)
- Specificity: 67.34% (SD 2.94)
- Sum of Sensitivity and Specificity: 132.53 (SD 3.91)
- PPV: 57.52% (SD 1.63)
- NPV: 74.03% (SD 2.44)
Per Patient Analysis: - Sensitivity: 66.21%
- Specificity: 68.18%
- Sum of Sensitivity and Specificity: 134.39
- PPV: 53.44%
- NPV: 69.58%
Conclusion: "demonstrate that the DR-70 test when taken as a 15% or greater change from the previous visit, yields informative data regarding colon cancer progression." |
2. Sample Size Used for the Test Set and Data Provenance
-
Test Set Description: "retrospective blood samples collected prospectively"
-
Data Provenance: Not explicitly stated, but the submission is to the US FDA, implying clinical studies conducted in a manner acceptable for US regulatory review.
-
Sample Size for Clinical Informativeness Study:
- Patients: 112 colon cancer patients.
- Serial Observations: 446 paired observations (DR-70 reading and disease progression determination).
- Post-baseline paired observations for analysis: 335.
- Visits for Sensitivity Analysis: 135.
- Visits for Specificity Analysis: 198.
- Patients with at least one sensitivity, specificity, or both: 112 patients.
- Estimated per-patient sensitivity: 70 estimates.
- Estimated per-patient specificity: 86 estimates.
-
Sample Sizes for Distribution of DR-70® (FDP) values (used for establishing reference ranges/disease cohorts):
- Normal: 420 (337 65 years)
- Benign: 326 (94 GU Disease, 61 GI Disease, 84 Pancreas, 87 Heart Disease)
- Malignant: 439 (187 Colon, 44 Lung, 44 Liver, 31 Breast, 31 Ovarian, 28 Cervical, 19 Gall Bladder, 28 Pancreas, 27 Gastric/Other)
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
- The ground truth for the clinical study was established based on "clinical impressions of the treating physicians based on Subject interviews, physical examination, laboratory results, X-rays, CAT scans and MRI as they are used in routine clinical practice in managing colorectal cancer Subjects."
- Number of Experts: Not specified. It refers to "treating physicians" (plural), implying multiple physicians were involved across the patient cohort.
- Qualifications of Experts: Assumed to be qualified physicians specializing in the management of colorectal cancer, given they are "treating physicians" and using standard clinical modalities. Specific years of experience or board certifications are not detailed in the provided text.
4. Adjudication Method for the Test Set
- The ground truth relied on "clinical impressions of the treating physicians" using a combination of standard clinical modalities. However, the document does not specify a formal adjudication method (e.g., 2+1, 3+1 consensus). It appears to accept the treating physician's integrated clinical assessment as the ground truth.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and Effect Size
- No, a MRMC comparative effectiveness study was not done in the context of human readers using/not using AI assistance. This device is an in vitro diagnostic (IVD) immunoassay, not an AI imaging device or decision support system for human readers. The study evaluates the performance of the immunoassay itself.
6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Was Done
- Yes, a standalone performance study was done. The entire clinical study described evaluates the AMDL-ELISA DR-70® (FDP) immunoassay as a standalone diagnostic tool. Its results (DR-70® FDP concentration changes) are directly compared to the clinical disease status (the ground truth). The statement "The DR-70® (FDP) immunoassay results must be used in conjunction with standard of care procedures" indicates that it is intended to augment, rather than replace, clinical judgment, but its performance was assessed independently against clinical outcomes.
7. The Type of Ground Truth Used
- Clinical Ground Truth: The ground truth for the clinical informativeness study was derived from "clinical impressions of the treating physicians based on Subject interviews, physical examination, laboratory results, X-rays, CAT scans and MRI as they are used in routine clinical practice in managing colorectal cancer Subjects."
- For the initial cohort distribution studies (Normal, Benign, Malignant), the ground truth was the established diagnosis of the subjects enrolling in those cohorts.
8. The Sample Size for the Training Set
- The document describes a "clinical trial using retrospective blood samples collected prospectively" that served as the test set. It does not explicitly mention a separate "training set" for the immunoassy itself in the context of machine learning, as this is a biochemical assay.
- The "training" (or rather, development) of the assay would involve various analytical performance studies (recovery, linearity, precision, etc.) and the establishment of assay parameters (e.g., standard curve, cut-offs), which are described as "Performance Testing." These involve internal experiments rather than a distinct 'training set' of clinical data for an algorithm.
9. How the Ground Truth for the Training Set Was Established
- As noted above, for an immunoassay, the concept of a "training set" and associated "ground truth" for machine learning is not directly applicable. The "ground truth" for developing and validating the analytical performance of the immunoassay (e.g., calibrator values for recovery, known concentrations for linearity and sensitivity) would be established through laboratory standards, spiked samples with known concentrations, and internal controls according to standard laboratory practices.
- The normal, benign, and malignant disease cohorts in the clinical study helped establish the clinical context and distribution of DR-70 values, which informs clinical interpretation rather than directly "training" an algorithm in the machine learning sense. The ground truth for these cohorts was their clinical diagnosis.
§ 866.6010 Tumor-associated antigen immunological test system.
(a)
Identification. A tumor-associated antigen immunological test system is a device that consists of reagents used to qualitatively or quantitatively measure, by immunochemical techniques, tumor-associated antigens in serum, plasma, urine, or other body fluids. This device is intended as an aid in monitoring patients for disease progress or response to therapy or for the detection of recurrent or residual disease.(b)
Classification. Class II (special controls). Tumor markers must comply with the following special controls: (1) A guidance document entitled “Guidance Document for the Submission of Tumor Associated Antigen Premarket Notifications (510(k)s) to FDA,” and (2) voluntary assay performance standards issued by the National Committee on Clinical Laboratory Standards.