(167 days)
The Automated Cellular Imaging System (ACIS) device is intended to detect, count, and classify cells of clinical interest based on recognition of cellular objects of particular color, size, and shape.
In this software application the ACIS is intended for laboratory use as an accessory to the DakoCytomation HercepTest™ to aid in the detection and semi-quantitative measurement of Her2/neu (c-erbB-2) in formalin-fixed, paraffin embedded normal and neoplastic tissue.
The ACIS is capable of detecting and quantifying regions of clinical interest in immunocytochemically stained material that would otherwise be appropriate for manual visualization by conventional microscopy.
When used with the DakoCytomation HercepTest™, it is indicated for use as an aid in the assessment of breast cancer patients for whom HERCEPTIN® (Trastuzumab) treatment is being considered. The pathologist should verify agreement with the ACIS score.
The ACIS system is an adjunctive computer-assisted methodology to assist the reproducibility of a qualified pathologist in the acquisition and measurement of images from microscopic slides of breast cancer specimens stained for the presence of Her2 receptor protein. The accuracy of the test result depends upon the quality of immunohistochemical staining. It is the responsibility of a qualified pathologist to employ appropriate morphological studies and controls as specified in instructions for DakoCytomation HercepTest™ to assure the validity of the ACIS-assisted Her2 score.
NOTE: All of the patients in the HERCEPTIN® clinical trials were selected using an investigational immunohistochemical clinical trial assay (CTA). None of the patients in those trials were selected using the DakoCytomation HercepTest™. The DakoCytomation HercepTest™ was compared to the CTA on an independent set of samples and found to provide acceptably concordant results. The actual correlation of the DakoCytomation HercepTest™ to HERCEPTIN® clinical outcome has not been established.
The ACIS is a highly automated microscope and software system capable of operating continuously with limited human supervision and maintenance. By automating certain steps of the basic microscopic procedure (loading, positioning, focusing, scanning, and cell identification and quantification), the ACIS can significantly improve the consistency, speed, reproducibility, and accuracy of traditional manual or semi-manual laboratory results.
The ACIS can automatically scan and process up to 100 slides while operating unattended. As each slide is processed, the system automatically stores individual fields of view in order to map and integrate this into a single image, a histological reconstruction of the entire tissue section. The proprietary software-based digital color detection technology offers significant sensitivity and specificity to identify targeted cells using common laboratory stains. This technology utilizes advanced imaging concepts referred to as "color spaces" and "color space conversion."
During normal operation, a laboratory technician mounts prepared microscope slides onto a specially designed slide carrier which has the capacity to hold up to four slides. Up to 25 slide carriers can be loaded into the ACIS slide transport system, for a "run" throughput of 100 slides. The scanning process begins with the automatic loading of the first carrier of slides onto the microscope stage. During this stage of the process, the ACIS automatically reads bar codes affixed to the slides to facilitate participant data storage and access. During scanning, the ACIS automatically focuses each field of view, images the object using the CCD camera, and processes the resulting digital image through the image processor. Slides are then scanned at a low optical magnification (typically 4X objective) to find regions of interest based primarily on their color, along with size and shape characteristics. The locations of regions are stored until scanning is completed. Once the region of interest containing the specimen is found, a scan of the specimen commences to build a low power image of the specimen (histological reconstruction).
For the HER2 assay, the ACIS creates the histological reconstruction. The pathologist may then visit any area of the scanned tissue section at low or high magnification. The pathologist electronically selects regions of invasive carcinoma. The scoring is expressed as intensity of brown color within the stained cells in the selected area.
After scanning is completed, the operator is able to view the histological reconstruction of all stored images for interpretation of the detected objects, along with quantitative information, such as the number of objects detected and colorimetric value. Upon completion of this review, a report containing relevant images identified by the pathologist may be printed. Images may be saved on the hard drive or archived to removable media.
The ChromaVision Medical Systems, Inc. ACIS (Automated Cellular Imaging System) for HER2/neu detection involved several studies to establish its performance and substantial equivalence to predicate devices.
1. Acceptance Criteria and Reported Device Performance:
The document doesn't explicitly state quantitative acceptance criteria in a pass/fail format. However, the studies demonstrate the ACIS device's performance in terms of agreement with manual methods and reproducibility. The implicit acceptance criteria appear to be showing comparable or improved performance over manual methods.
| Performance Metric | Acceptance Criteria (Implicit) | Reported Device Performance |
|---|---|---|
| Agreement with Manual Methods | Demonstrate acceptable concordance with manual slide examination. | Overall agreement of 75% between ACIS and manual review methods for HER2/neu scoring (0, 1+, 2+, 3+ categories). |
| Between Pathologist Reproducibility | Show comparable or improved inter-pathologist agreement compared to manual. | Manual readings: 55% to 75% agreement between pathologists. |
| ACIS-assisted data: 93% to 98% pairwise agreement between pathologists. | ||
| Interpretation: ACIS device provides a greater rate of inter-pathologist agreement. | ||
| Within Pathologist Reproducibility | Show comparable or improved intra-pathologist agreement compared to manual. | Overall manual inter-observer agreement: 85.4%. |
| Overall ACIS inter-observer agreement: 95.3% for the 4-category scale. | ||
| Interpretation: ACIS device provided a higher consistency rate. | ||
| Inter-ACIS Reproducibility | Demonstrate no significant difference in scores across different ACIS instruments. | No significant difference between the scores reported by three different ACIS instruments (p=0.897). |
2. Sample Sizes and Data Provenance:
- Test Set for Agreement (ACIS vs. Manual):
- Sample Size: 90 specimens on a Tissue Micro Array (TMA) slide.
- Data Provenance: Not explicitly stated, but clinical specimens. Given the context of a US regulatory submission, it is highly likely to be US-sourced, but this is an inference. Retrospective, as specimens were "previously selected."
- Test Set for Within/Between Reproducibility:
- Sample Size: 60 specimens.
- Data Provenance: Not explicitly stated, but clinical specimens. Given the context of a US regulatory submission, it is highly likely to be US-sourced, but this is an inference. Retrospective.
- Test Set for Inter-ACIS Reproducibility:
- Sample Size: The "same cases specified in the above study" (referring to the within/between reproducibility study, so likely 60 specimens).
- Data Provenance: Not explicitly stated, but clinical specimens. Given the context of a US regulatory submission, it is highly likely to be US-sourced, but this is an inference. Retrospective.
3. Number of Experts and Qualifications for Ground Truth:
- Agreement with Manual Methods:
- Number of Experts: 3 pathologists.
- Qualifications: "Pathologists." No specific years of experience are mentioned, but "qualified pathologist" is referenced in the intended use.
- An "independent pathologist" initially selected the specimens to represent a range of HER2/neu staining intensities.
- Within/Between Reproducibility:
- Number of Experts: 3 pathologists.
- Qualifications: "Pathologists." No specific years of experience are mentioned.
- Inter-ACIS Reproducibility:
- Number of Experts: 1 pathologist.
- Qualifications: "Pathologist." No specific years of experience are mentioned.
4. Adjudication Method for the Test Set:
- The document does not specify an explicit adjudication method for establishing a single "ground truth" score when there was disagreement among pathologists for the manual readings.
- For the agreement study, "raw data was recorded, analyzed, and presented in 4 X 4 tables," suggesting a comparison of individual scores rather than a resolution to a single adjudicated truth.
- For reproducibility studies, agreement percentages are calculated, indicating how often pathologists (or an individual pathologist over multiple readings) matched each other or themselves, rather than a method to reconcile differences into a single "correct" score.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:
- Yes, a form of MRMC study was conducted for the "Between Pathologist Reproducibility Results" and "Within Pathologist Reproducibility Results."
- Effect Size (Improvement with AI vs. Without AI Assistance):
- Between Pathologist Agreement: Manual agreement ranged from 55% to 75%. ACIS-assisted agreement ranged from 93% to 98%. This shows a significant improvement in inter-pathologist agreement ranging from 18% to 43% absolute increase in agreement.
- Within Pathologist Agreement: Manual inter-observer agreement was 85.4%. ACIS inter-observer agreement was 95.3%. This is an absolute increase of 9.9% in intra-pathologist consistency.
6. Standalone Performance (Algorithm Only without Human-in-the-loop performance):
- Implicitly no (or not directly reported as such). The ACIS device is described as an "adjunctive computer-assisted methodology to assist the reproducibility of a qualified pathologist." The studies presented always involve a pathologist interpreting the ACIS output or comparing it to manual interpretation. The statement "The pathologist should verify agreement with the ACIS score" further reinforces the human-in-the-loop nature. While the system automates scanning and image processing, its output (e.g., scoring) is evaluated in the context of a pathologist's review or comparison to manual pathologist scores.
7. Type of Ground Truth Used:
- Expert Consensus (Implicit/Comparison to Expert): The ground truth for the comparison and reproducibility studies was established by the scores of the pathologists themselves, either individually or in comparison to each other. The "manual methods of review" by the pathologists serve as the reference point for assessing the ACIS.
- The initial selection of specimens (0, 1+, 2+, 3+ categories) was done by an "independent pathologist," establishing a pre-classification that guided the selection of cases for the study.
- It does not explicitly state the use of pathology (e.g., gold standard FISH for HER2 amplification) or outcomes data as the primary ground truth for the device's performance evaluation against the 0-3+ scoring. The DakoCytomation HercepTest™ itself is a semi-quantitative IHC test, and the ACIS aids in its interpretation.
8. Sample Size for the Training Set:
- The document does not specify a sample size for a training set. The clinical tests described focus on validation of the device's performance and reproducibility, not on the specific training of the underlying algorithms. It is possible that the algorithms were trained on internal datasets not detailed in this 510(k) summary, or that the system relies on predefined colorimetric parameters rather than a machine learning model that requires a dedicated training set as understood today.
9. How Ground Truth for Training Set Was Established:
- As the training set size is not provided, the method for establishing its ground truth is also not described in this document.
§ 864.1860 Immunohistochemistry reagents and kits.
(a)
Identification. Immunohistochemistry test systems (IHC's) are in vitro diagnostic devices consisting of polyclonal or monoclonal antibodies labeled with directions for use and performance claims, which may be packaged with ancillary reagents in kits. Their intended use is to identify, by immunological techniques, antigens in tissues or cytologic specimens. Similar devices intended for use with flow cytometry devices are not considered IHC's.(b)
Classification of immunohistochemistry devices. (1) Class I (general controls). Except as described in paragraphs (b)(2) and (b)(3) of this section, these devices are exempt from the premarket notification requirements in part 807, subpart E of this chapter. This exemption applies to IHC's that provide the pathologist with adjunctive diagnostic information that may be incorporated into the pathologist's report, but that is not ordinarily reported to the clinician as an independent finding. These IHC's are used after the primary diagnosis of tumor (neoplasm) has been made by conventional histopathology using nonimmunologic histochemical stains, such as hematoxylin and eosin. Examples of class I IHC's are differentiation markers that are used as adjunctive tests to subclassify tumors, such as keratin.(2) Class II (special control, guidance document: “FDA Guidance for Submission of Immunohistochemistry Applications to the FDA,” Center for Devices and Radiologic Health, 1998). These IHC's are intended for the detection and/or measurement of certain target analytes in order to provide prognostic or predictive data that are not directly confirmed by routine histopathologic internal and external control specimens. These IHC's provide the pathologist with information that is ordinarily reported as independent diagnostic information to the ordering clinician, and the claims associated with these data are widely accepted and supported by valid scientific evidence. Examples of class II IHC's are those intended for semiquantitative measurement of an analyte, such as hormone receptors in breast cancer.
(3) Class III (premarket approval). IHC's intended for any use not described in paragraphs (b)(1) or (b)(2) of this section.
(c)
Date of PMA or notice of completion of a PDP is required. As of May 28, 1976, an approval under section 515 of the Federal Food, Drug, and Cosmetic Act is required for any device described in paragraph (b)(3) of this section before this device may be commercially distributed. See § 864.3.