(309 days)
PATHIAM-Assisted Scoring: Intended for clinical laboratory use as an accessory to the DAKO HercepTest to aid in the detection and semi-quantitative measurement of Her2/neu in formalin fixed, paraffin-embedded normal and neoplastic tissue. When used with the DAKO HercepTest, Pathiam Assisted Scoring is indicated for use as an aid in the assessment of breast cancer patients for whom HERCEPTIN (Trastuzumab) treatment is being considered. The pathologist should verify agreement with the PATHIAM System score.
HER2/neu results are indicated for use as an aid in the management, prognosis and predication of therapy outcomes of breast cancer. Note: The actual correlation of the DAKO HercepTest to Herceptin® clinical outcome has not been established.
The PATHIAMIM System is an instrument (iScan) and image analysis software system designed to assist the qualified pathologist in the consistent quantitative assessment of marker expression in immunohistochemically stained histological sections digital images. The sample tissue is breast tissue prepared using the DAKO HercepTest Reagent Kit. The PATHIAM system consists of the BioImagene iScan slide scanner, computer with the PATHIAM Software, monitor, keyboard and mouse.
The PATHIAM System digitizes formalin-fixed, paraffin embedded normal and neoplastic tissue and provides semi-quantitative analysis of extent and intensity of stained tissue, providing the pathologist with an aid to interpretation of the level of expression of HER2/neu in breast cancer tissue. The pathologist is presented with a digital image of the tissue section and a suggested staining score (0 to 3). The pathologist then makes an assessment of the digital image and reports his/her score. Alternately, the pathologist can simply use the digitized image to perform his interpretation of the level of expression, without employing the software.
Hardware: The iScan slide scanning device captures digital images of formalin-fixed, paraffin-embedded tissues that are suitable for storage, viewing and visual analysis. The device includes a digital slide scanner, racks for loading glass slides, an Intel based PC, dual core, dual Xeon processor, PATHIAM Software, and a monitor.
Software: The PATHIAM Software requires competent human intervention at all steps in the analysis process. The system is designed to complement the routine workflow of a qualified pathologist screening the immunohistochemically stained histological slides with additional quantitative data to assist the reproducibility of the slide interpretation. It allows the user to select the area of interest on the breast tissue image. The user marks the area of interest for the analysis. The system software makes no independent interpretations of the data. The software will act as a tool for the user, to improve consistency and data recording. The image produced digitally may also be used independent of the software, by allowing the pathologist to count from the digital image, rather than from the microscope.
The provided document outlines the performance study for the PATHIAM™ System with iScan for HER2/neu, primarily focusing on its agreement with manual HercepTest methods and reproducibility.
Here's an analysis of the acceptance criteria and study details:
1. Table of Acceptance Criteria and Reported Device Performance:
| Acceptance Criteria Category | Acceptance Criteria | Reported Device Performance |
|---|---|---|
| Agreement with Manual HercepTest Method | While explicit numerical acceptance criteria for agreement between the PATHIAM System and manual scores are not stated as "acceptance criteria," the study aimed to demonstrate acceptable levels of concordance. Implicitly, the observed percentage agreements in each site were considered acceptable for the claim of aiding pathologists. | Site 1: 77% Overall Agreement (95% CI: 70% - 83%) |
| Site 2: 81% Overall Agreement (95% CI: 75% - 87%) | ||
| Site 3: 88% Overall Agreement (95% CI: 82% - 92%) | ||
| Reproducibility Between Pathologists and PATHIAM Systems (Inter-site with PATHIAM assistance) | Implicitly, good concordance was desired. The study reported "Concordance for the PATHIAM System values between labs ranged from 89% to 92%." This suggests that the system-assisted readings were highly reproducible across sites. | Site 1 vs. 2: 92% Overall Agreement (95% CI: 87% - 96%) |
| Site 2 vs. 3: 89% Overall Agreement (95% CI: 83% - 93%) | ||
| Site 3 vs. 1: 91% Overall Agreement (95% CI: 86% - 95%) | ||
| Manual HercepTest Reproducibility (Inter-site without PATHIAM assistance) | While not an acceptance criterion for the device itself, this serves as a baseline for comparison. The goal would likely be to show the PATHIAM system-assisted readings are at least as reproducible or better than manual readings. | Site 1 vs. 2: 67% Overall Agreement (95% CI: 60% - 74%) |
| Site 2 vs. 3: 81% Overall Agreement (95% CI: 74% - 86%) | ||
| Site 3 vs. 1: 80% Overall Agreement (95% CI: 73% - 85%) | ||
| iScan Slide Scanner Precision | Not explicitly stated as a numerical acceptance criterion, but "precision was calculated to be 97%." | 97% |
| Inter-run/Inter System Reproducibility (iScan) | Not explicitly stated as a numerical acceptance criterion, but the study implies a high degree of agreement was expected. | 100% agreement between PATHIAM System scores for different scans and for different iScan slide scanners. |
2. Sample size used for the test set and the data provenance:
- Sample Size: 176 stained tissue specimens.
- Data Provenance: The document does not explicitly state the country of origin. The study appears to be prospective in the sense that the pathologists performed readings specifically for this study, first manually and then with the PATHIAM System. There is no indication of retrospective analysis of existing clinical data.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Number of Experts: Three pathologists.
- Qualifications of Experts: The document states "trained pathologists" without specifying years of experience or sub-specialization.
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set:
- Adjudication Method: Not explicitly stated. For the "Comparison with Manual HercepTest method," pathologists observed digital images and the suggested score, then selected an appropriate tissue score. This suggests the pathologist made the final judgment for the PATHIAM System assisted score. For the manual scores, they read the slides manually. There's no mention of a consensus process or arbitration for discrepant scores.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- MRMC Study: Yes, a form of MRMC study was conducted. Three pathologists (multiple readers) assessed 176 cases (multiple cases) both with and without the AI assistance (manual vs. PATHIAM-assisted).
- Effect Size of Improvement: The study demonstrates improved inter-reader reproducibility with AI assistance compared to manual reading.
- Manual Inter-reader Agreement: Ranged from 67% to 81%.
- PATHIAM-assisted Inter-reader Agreement: Ranged from 89% to 92%.
- This represents an improvement in inter-reader agreement of 8% to 22% when using the PATHIAM system compared to manual reading alone (e.g., 92% vs 67% for Site 1 vs 2 comparison, an improvement of 25 percentage points; 89% vs 81% for Site 2 vs 3, an improvement of 8 percentage points; 91% vs 80% for Site 3 vs 1, an improvement of 11 percentage points). The "effect size" here is the increase in percentage agreement between pathologists.
6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done:
- Standalone Performance: No, a standalone performance study of the algorithm without human-in-the-loop was not performed. The device description explicitly states: "The PATHIAM Software requires competent human Software: intervention at all steps in the analysis process. The system is designed to complement the routine workflow of a qualified pathologist..." and "The software will act as a tool for the user, to improve consistency and data recording." The "PATHIAM System values" are derived from the pathologist's review of the digital images and the software's suggested score.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc):
- Ground Truth: The "ground truth" for evaluating the PATHIAM System's performance was the manual assessment by trained pathologists using the DAKO HercepTest package insert. This is implicitly assumed to be the "gold standard" against which the system-assisted readings are compared. The reproducibility study between pathologists also uses their independent assessments as the reference.
8. The sample size for the training set:
- Training Set Sample Size: The document does not provide information on the training set size or how the algorithm was developed. The studies described are performance validation studies.
9. How the ground truth for the training set was established:
- Training Set Ground Truth: As with the training set size, information on how the ground truth for any potential training set was established is not provided in this summary.
§ 864.1860 Immunohistochemistry reagents and kits.
(a)
Identification. Immunohistochemistry test systems (IHC's) are in vitro diagnostic devices consisting of polyclonal or monoclonal antibodies labeled with directions for use and performance claims, which may be packaged with ancillary reagents in kits. Their intended use is to identify, by immunological techniques, antigens in tissues or cytologic specimens. Similar devices intended for use with flow cytometry devices are not considered IHC's.(b)
Classification of immunohistochemistry devices. (1) Class I (general controls). Except as described in paragraphs (b)(2) and (b)(3) of this section, these devices are exempt from the premarket notification requirements in part 807, subpart E of this chapter. This exemption applies to IHC's that provide the pathologist with adjunctive diagnostic information that may be incorporated into the pathologist's report, but that is not ordinarily reported to the clinician as an independent finding. These IHC's are used after the primary diagnosis of tumor (neoplasm) has been made by conventional histopathology using nonimmunologic histochemical stains, such as hematoxylin and eosin. Examples of class I IHC's are differentiation markers that are used as adjunctive tests to subclassify tumors, such as keratin.(2) Class II (special control, guidance document: “FDA Guidance for Submission of Immunohistochemistry Applications to the FDA,” Center for Devices and Radiologic Health, 1998). These IHC's are intended for the detection and/or measurement of certain target analytes in order to provide prognostic or predictive data that are not directly confirmed by routine histopathologic internal and external control specimens. These IHC's provide the pathologist with information that is ordinarily reported as independent diagnostic information to the ordering clinician, and the claims associated with these data are widely accepted and supported by valid scientific evidence. Examples of class II IHC's are those intended for semiquantitative measurement of an analyte, such as hormone receptors in breast cancer.
(3) Class III (premarket approval). IHC's intended for any use not described in paragraphs (b)(1) or (b)(2) of this section.
(c)
Date of PMA or notice of completion of a PDP is required. As of May 28, 1976, an approval under section 515 of the Federal Food, Drug, and Cosmetic Act is required for any device described in paragraph (b)(3) of this section before this device may be commercially distributed. See § 864.3.