Search Results

For in vitro diagnostic use in the quantitative determination of the HER-2/neu protein in human serum or using the ADVIA Centaur System. HER-2/neu values obtained may be used in the follow-up and monitoring of patients with metastatic breast cancer whose initial serum HER-2/neu level is greater than 15 ng/mL. HER-2/neu values should be used in conjunction with information available from clinical and other diagnostic procedures in the management of breast cancer. The clinical utility of the measurement of HER-2/neu in serum as a prognostic indicator for early recurrence and in the management of patients on immunotherapy has not been fully established. This test should be used by or under the order of a physician. This assay is not intended for use on any other system.

The ADVIA Centaur HER-2/neu assay is a fully automated, two-site sandwich immunoassay using direct, chemiluminescent technology. The Lite Reagent is composed of the monoclonal mouse antibody, TA-1 labeled with acridinium ester. The Fluorescein Conjugate Reagent is composed of the monoclonal mouse antibody, NB-3, labeled with fluorescein. The two monoclonal antibodies are specific for unique epitopes on the extracellular domain of HER-2/neu. The Solid Phase is composed of purified monoclonal mouse capture antibody, which is covalently coupled to paramagnetic particles. The sample is incubated with Fluorescein Conjugate Reagent and Lite Reagent simultaneously for 5.5 minutes. After this incubation, the Solid Phase is added and the mixture is incubated for an additional 2.75 minutes. After this final incubation, the immuno-complex formed is washed. The measured chemiluminescence is directly proportional to the quantity of HER-2/neu antigen in the sample.

The provided text describes the ADVIA Centaur HER-2/neu Immunoassay and its substantial equivalence to a predicate device, the Bayer Immuno-1 HER-2/neu kit. The primary study presented focuses on demonstrating this equivalence rather than establishing new acceptance criteria against a clinical ground truth.

Here's an analysis based on the provided information:

1. Table of Acceptance Criteria and Reported Device Performance

Note: The document directly states that the "data demonstrate substantial equivalence." The acceptance criteria here are implicitly the thresholds for correlation and agreement that the FDA and the manufacturer consider sufficient to declare substantial equivalence to the predicate.

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size: 196 samples
• Data Provenance: Not specified (e.g., country of origin, retrospective/prospective). However, the study involved comparing the new device's results to those of the predicate device on these 196 samples.

3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications of Those Experts

Not applicable. The study's purpose was to demonstrate substantial equivalence to a predicate device, not to establish a new ground truth based on expert consensus for the 196 samples. The "ground truth" in this context is the results obtained from the predicate device.

4. Adjudication Method for the Test Set

Not applicable. The study involved a direct comparison between two assays: the new ADVIA Centaur HER-2/neu assay and the predicate Bayer Immuno-1 HER-2/neu assay. There was no expert adjudication process for discordant results as the goal was method comparison.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done

No. This was a direct comparison study between two in vitro diagnostic (IVD) assays, not a study evaluating human reader performance with or without AI assistance.

6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Was Done

Yes. The study assessed the performance of the ADVIA Centaur HER-2/neu assay as a standalone device by comparing its results to those of another standalone IVD, the Bayer Immuno-1 HER-2/neu assay. Human interpretation of the raw chemiluminescence signal is not part of the device's functional mechanism; the device provides a quantitative result directly.

7. The Type of Ground Truth Used

The "ground truth" for this study was the results obtained from the predicate device (Bayer Immuno-1 HER-2/neu assay). The study aimed to show that the new device produced comparable results to an already FDA-cleared device.

8. The Sample Size for the Training Set

Not applicable. This device is an immunoassay kit, not an AI/machine learning algorithm that requires a training set in the conventional sense. The development of such an assay involves reagent optimization and calibration, but not an "AI training set."

9. How the Ground Truth for the Training Set Was Established

Not applicable, as there is no "training set" in the context of an immunoassay kit in the same way there would be for an AI algorithm. The development of the assay's reagents and calibration would be based on established biochemical and immunological principles, likely utilizing reference materials and internal validation processes.

Ask a specific question about this device

The Bayer Immuno 1™ Her-2/neu Assay is an in vitro, diagnostic device intended for use in the quantitative determination of HER-2/neu in human serum. HER-2/neu values obtained may be used in the follow-up and monitoring of patients with metastatic breast cancer. HER-2/neu values should be used in conjunction with information available from clinical and other diagnostic procedures in the management of breast cancer. The clinical utility of the serum measurement of HER-2/neu as a prognostic indicator for early detection of recurrence and in the management of patients on immunotherapy regimens has not been fully established.

The Bayer Immuno I™ HER-2/neu Assay utilizes a well-established immunoassay technology in which one monoclonal HER-2/neu antibody (NB-3) is conjugated to fluorescein (designated Reagent 1, or R1) and the Fab' fragment of another monoclonal HER-2/neu antibody (TA-1) is conjugated to alkaline phosphatase (Reagent 2, or R2). The R1 and R2 conjugates are reacted with patient sample, calibrator, or control and are incubated at 37℃ on the system. Immuno 1 Magnetic Particles coated with an antifluorescein antibody (mIMP™ Reagent) are then added and a second incubation occurs during which the antibody complex is bound. The magnetic particles complexed with the immunological sandwich are then washed to separate unbound molecules, and a colorimetric substrate is added. The rate of conversion of substrate to a compound with absorbance at 405 and 450 nm is measured is proportional to the concentration of HER-2/neu in the sample. A cubic-through-zero curve-fitting algorithm is used to generate standard curves.

The assay has a range of zero to 250 ng/mL. The Bayer SETpoint™ HER-2/neu Calibrators consist of a set of six calibrator levels at 0, 10, 25, 60, 125, and 250 ng/mL. The Bayer TESTpoint™ HER-2/neu Controls consist of a set of three control levels at approximately 15, 50 and 100 ng/mL.

The provided document describes the Bayer Immuno 1™ HER-2/neu Assay, an in vitro diagnostic device for the quantitative determination of HER-2/neu in human serum for monitoring metastatic breast cancer patients.

Here's an analysis of the acceptance criteria and the study that proves the device meets them, based on the provided text:

Acceptance Criteria and Reported Device Performance

Study Details

1. Sample size used for the test set and the data provenance:

   • Clinical Study Test Set: 60 patients with metastatic breast cancer for longitudinal monitoring. The study used "retrospective serum samples from three clinical sites in the United States."
   • Specificity Test Set (detection limit): 480 replicates of the zero calibrator.
   • Reproducibility Test Set: A human serum pool (approx. 15 ng/mL), and various calibrator and control levels. The number of replicates varied per level, up to 640 replicates for some calibrators tested over 80 runs.
   • Assay Performance Studies (interference, linearity, hook effect, parallelism): Varied sample sizes (e.g., "five individual serum samples" for linearity, "patient samples" and "pools of serum" for interference, "patient samples with HAMA, RF titers, or autoimmune diseases" for heterophilic antibodies). No specific total numbers for these are provided.
   • Data Provenance: Predominantly from the United States (three clinical trial sites). The clinical study was retrospective.

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g., radiologist with 10 years of experience):

   • The document states, "Clinical data were gathered during well controlled investigations conducted by qualified experts." It also mentions "Hospitals, medical centers and other health care organizations" and "qualified investigators." However, it does not specify the exact number or precise qualifications of these experts (e.g., "radiologist with 10 years of experience").
   • For the clinical study, "serial changes in serum HER-2/neu were correlated with changes in clinical status." The "clinical status" is the ground truth, which would have been determined by these "qualified experts" based on "clinical and other diagnostic procedures."

3. Adjudication method (e.g., 2+1, 3+1, none) for the test set:

   • The document does not explicitly describe a formal adjudication method (like 2+1 or 3+1) for establishing the clinical ground truth. It simply states that "clinical status" was used, presumably reflecting the standard diagnostic practice at the three clinical sites.

4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • This is not an AI/imaging device. It is an in vitro diagnostic assay. Therefore, a multi-reader multi-case (MRMC) comparative effectiveness study with human readers assisting AI or vice-versa is not applicable and was not performed. The device's utility is evaluated by correlation with clinical status, not by improving human interpretation of images.

5. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:

   • Yes, the performance characteristics (e.g., reproducibility, sensitivity, linearity) and the clinical utility were evaluated for the "Immuno 1 HER-2/neu Assay" as a standalone device. The "indications for use" state that "HER-2/neu values should be used in conjunction with information available from clinical and other diagnostic procedures," implying it's an aid, but its performance as a measurement tool is standalone.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

   • For the clinical utility study, the ground truth was "changes in clinical status" of metastatic breast cancer patients due to "progression of disease." This would be based on a combination of clinical assessments, potentially including various diagnostic procedures and outcomes data, determined by medical experts. It does not explicitly mention pathology as the primary ground truth for monitoring changes over time, though initial diagnosis would involve pathology.

7. The sample size for the training set:

   • The document does not specify a separate training set size for the algorithms within the device. This assay is a chemiluminescent immunoassay using established technology and a cubic-through-zero curve-fitting algorithm. The "training" for such systems typically involves determining the standard curve and calibrating the system. The calibrators themselves define the reference points for the curve. The document states "The Bayer SETpoint™ HER-2/neu Calibrators consist of a set of six calibrator levels," which are used to generate standard curves.

8. How the ground truth for the training set was established:

   • For the analytical performance (e.g., calibrators), the "ground truth" (i.e., assigned concentrations) is established through rigorous internal validation and standardization methods during the manufacturing and development of the calibrator materials. The document mentions "recombinant 3T3 mouse cell line 3-30" as the source of antigen and "Western blot analysis" for characterization, indicating controlled biochemical processes for establishing the integrity and concentration of the calibrator material concentrations. For the curve-fitting algorithm, the "ground truth" is derived from these established calibrator concentrations.

Ask a specific question about this device

The OSDI HER-2/neu Assay is an in vitro, diagnostic device intended for use in the quantitative determination of serum HER-2/neu in women with metastatic breast cancer who have an initial value of 15 ng/ml or greater. HER-2/neu values obtained may be used in the follow-up and monitoring of patients with metastatic breast cancer. HER-2/neu values should be used in conjunction with information available from clinical and other diagnostic procedures in the management of breast cancer. The clinical utility of the serum measurement of HER-2/neu as a prognostic indicator for early recurrence and in the management of patients on immunotherapy regiments has not been fully established.

The HER-2/neu ELISA is a sandwich enzyme immunoassay, which utilizes two monoclonal antibodies to quantitate the extracellular domain (ECD) of the HER-2/neu protein in serum. The capture antibody (NB-3) has been immobilized on the interior surface of the microtiter plate wells. To perform the assay, an appropriate volume of serum is incubated in the coated well to allow binding of the antigen by the capture antibody. The captured HER-2/neu ECD is then reacted with a different anti-HER-2/neu antibody designated TA-1. The TA-1 antibody is biotinylated. The detection of the ECD of HER-2/neu is with a streptavidin horseradish peroxidase conjugate, which then catalyzes the conversion of the chromogenic substrate o-phenylenediamine into a colored product. The colored reaction product is quantitated by spectrophotometry (read absorbance at 490 mm) and is related to the amount of the HER-2/neu ECD in the serum sample. Six prepared HER-2/neu standards (0, 2.5, 7.5, 15, 25 and 35 ng/ml) allow construction of a standard curve for subsequent quantification of HER-2/neu in serum samples. Also available from OSDI is a set of HER-2/neu controls (10-CVX) designed to be used in conjunction with the ELISA for quality monitoring of assay performance. This control set consists of three (3) control levels at 2.9, 9.1 and 24.0 ng/mL HER-2/neu protein.

The Oncogene Science Diagnostics, Inc. (OSDI) Manual HER-2/neu Microtiter ELISA is an in vitro diagnostic assay intended for the quantitative measurement of the extracellular domain of HER-2/neu protein in human serum. It is used in the follow-up and monitoring of patients with metastatic breast cancer whose initial serum level of HER-2/neu is 15 ng/mL or greater, in conjunction with other clinical and diagnostic procedures.

Here's an analysis of the acceptance criteria and the study proving the device meets those criteria:

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria are not explicitly stated in a quantitative table within the document but can be inferred from the performance metrics described for various aspects of the assay. The "reported device performance" are the results from the non-clinical and clinical studies.

2. Sample size used for the test set and the data provenance

• Test Set Sample Size:
  • Non-clinical studies: The exact number of samples for each non-clinical test (interference, cross-reactivity, linearity, spike and recovery, hook effect, parallelism, end-to-end variation, plate coating variations, reproducibility, sensitivity) is not always explicitly stated as a single total number. For example:
    • Reproducibility: 130-131 runs and 385-390 replicates combined across three sites and three kit lots for 4 controls (Table 4.3-2).
    • Sensitivity (MDD): 232 replicates of Sample Diluent alone.
    • Interference: Not specified, but involved numerous compounds tested.
    • Linearity: Two controls combined to produce five samples.
  • Clinical studies:
    • Longitudinal monitoring: 56 clinically evaluable metastatic breast cancer patients with initial elevated HER-2/neu (≥15 ng/mL). There were 236 visit-by-visit comparisons (Table 4.4-1).
    • Upper Limit of Normals determination: 38 healthy women (for longitudinal variability of 20% criterion).
    • Clinical sensitivity/specificity to establish 15 ng/ml cutoff: "patients with breast cancer," "patients with benign breast diseases," "other non-malignant diseases," and "normal healthy individuals." Specific numbers for each group are not provided for this specific analysis.
• Data Provenance: Retrospective (Page 10, 12).
• Country of Origin: The clinical evaluations were conducted at "two US clinical trial sites" (Page 2, 3), and studies were performed at OSDI, Cambridge, Massachusetts (Page 3).

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

• Number of Experts: Not explicitly stated as a specific number.
• Qualifications of Experts: For the clinical studies, the "Clinical course or status was determined by the physician" (Page 10). It is implied these are qualified medical professionals, but specific qualifications (e.g., "radiologist with 10 years of experience") are not detailed.

4. Adjudication method for the test set

• The document implies that "Clinical course or status was determined by the physician" (Page 10). This suggests individual physician assessment rather than a detailed, multi-reader, adjudicated ground truth process for each patient case, at least as explicitly described for the clinical status.
• For the visit-by-visit analysis (Table 4.4-1), changes greater than or equal to 20% in HER-2/neu values were considered reflective of disease progression, while less than 20% increase reflected lack of progression. This 20% criterion was internally derived from the longitudinal variability in normal patients (six serial samples from 38 healthy women), suggesting an objective, pre-defined threshold rather than expert adjudication for each change.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• No, a multi-reader multi-case (MRMC) comparative effectiveness study was not done. This device is an immunoassay (ELISA), not an AI-powered image analysis tool that would typically involve human readers.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

• Yes, the device was evaluated in a standalone manner. The ELISA assay itself provides quantitative values for HER-2/neu. The non-clinical studies (e.g., interference, linearity, reproducibility, sensitivity) directly assess the performance of the assay independent of human interpretation or intervention beyond performing the laboratory procedure correctly. The clinical studies then correlated the values generated by the ELISA with the clinical course of the disease, but the device itself is a measurement tool, not an interpretive algorithm that a human would then refine.

7. The type of ground truth used

• Non-clinical studies:
  • Defined Antigen: The standard antigen (p105) itself is a characterization of the "ground truth" for what the assay is supposed to detect (Page 3).
  • Spiked Samples/Known Concentrations: For analytical performance like linearity, spike and recovery, and hook effect, samples with known, defined concentrations of HER-2/neu were used as ground truth.
  • Clinical Studies:
    • Clinical Course/Status Determined by Physician: For the metastatic patient serial monitoring, the ground truth for disease progression or lack thereof (stable/responding disease) was the "clinical course or status determined by the physician" (Page 10). This would be based on a combination of medical assessments and other diagnostic procedures.
    • Retrospective Sample Storage: The ground truth relied on patient samples collected and stored in specimen banks, with the assumption that HER-2/neu concentration remained stable during storage (Page 10).
    • Statistical Threshold: The 20% change criterion for correlating HER-2/neu values with clinical status was established based on longitudinal variability in normal patients, acting as a statistical ground truth for what constitutes a "significant" change (Page 12).
    • ROC Analysis: Used to confirm 15 ng/ml as an appropriate cutoff for 95% specificity, likely against a diagnosed patient population as ground truth (Page 12).

8. The sample size for the training set

• The document does not explicitly describe a separate "training set" in the context of typical machine learning models. Instead, the non-clinical studies (which involve extensive characterization of the assay's performance) and the selection of assay parameters (like antibody choice or standard curve concentrations) can be thought of as the "development" phase that would precede a formal clinical validation. The standard development of an ELISA kit does not typically involve a "training set" in the same way an AI/ML model does.

9. How the ground truth for the training set was established

• As a traditional immunoassay, there isn't a "training set" in the machine learning sense. The assay's "ground truth" during its development would have been established through:
  • Characterization of the Antigen and Antibodies: (Pages 3-4) Understanding the specific extracellular domain of HER-2/neu (p105) and developing highly specific monoclonal antibodies (NB-3 and TA-1) that bind to it. This involves biological and biochemical validation.
  • Standard Curve Development: Using prepared HER-2/neu standards of known concentrations (0, 2.5, 7.5, 15, 25, and 35 ng/ml) to establish accurate quantification (Page 2).
  • Analytical Validation: Extensive non-clinical studies to demonstrate performance characteristics (specificity, linearity, sensitivity, reproducibility) using samples with known properties (e.g., spiked samples, controls derived from cell lines or serum pools). (Pages 3-9)

Ask a specific question about this device