(449 days)
The PATHIAM System is intended as an aid to the pathologist to detect, count, and classify cells of clinical interest based on recognition of cellular objects of particular color, size, and shape, using appropriate controls to assure the validity of the scores.
The p53 results provided by the PATHIAM System are indicated for use for the identification of p53 accumulation in human neoplasias when used with IVD reagents marketed for this indication. Interpretation should be made within the context of the patient's clinical history and other diagnostic tests by a qualified pathologist. The pathologist must verify agreement with the PATHIAM score.
Ki-67 results provided by the PATHIAM System are indicated for use to assess proliferative activity when used with in vitro diagnostic reagents marketed for this indication. Interpretation should be made within the context of the patient's clinical history and other diagnostic tests by a qualified pathologist. The pathologist must verify agreement with the PATHIAM score.
The PATHIAM™ System is an instrument and software system designed to assist the qualified pathologist in the consistent quantitative assessment of protein expression in immunohistochemically stained histologic sections from formalin-fixed, paraffin-embedded normal and neoplastic tissues. The system consists of a slide scanner (iScan), computer, monitor, keyboard, mouse, image analysis algorithms for specific immunohistochemical markers, and software with a Windows web browser-based user interface. PATHIAM is a web-based, end-to-end digital pathology software solution that allows pathology labs to acquire, manage, view, analyze, share, and report on digital images of pathology specimens. Using the PATHIAM software, the pathologist can view digital images, add annotations, make measurements, perform image analysis, and generate reports.
The iScan slide scanning device captures digital images of formalin-fixed, paraffin-embedded tissues that are suitable for storage and viewing. The device includes a digital slide scanner, racks for loading glass slides, computer, scanner software, keyboard, mouse and monitor.
The PATHIAM software is designed to complement the routine workflow of a qualified pathologist in the review of immunohistochemically stained histologic slides. It allows the user to select fields of view (FOVs) in the digital image for analysis and provides quantitative data on these FOVs to assist with interpretation. The software makes no independent interpretations of the data and requires competent human intervention at all steps in the analysis process.
Here's a breakdown of the acceptance criteria and study details for the PATHIAM™ System with iScan for p53 and Ki-67, extracting information from the provided document:
Acceptance Criteria and Device Performance
| Criteria Category | Acceptance Criteria (Implicit from Study Conclusion) | Reported Device Performance (Summary from Study Conclusion) |
|---|---|---|
| P53 Scoring | ||
| Manual vs. PATHIAM-assisted Substantial Equivalence Concordance | ≥ 75% concordance | Met by all three pathologists for all three clinical cut-offs (>1%, >5%, >10%). |
1%: 82% - 90%
5%: 77% - 85%
10%: 83% - 89% |
| PATHIAM-assisted vs. PATHIAM-assisted Reproducibility (Inter-Pathologist) | ≥ 75% concordance | Exceeded 75% concordance at all three clinical cut-offs, and was higher than manual microscopy reproducibility.
1%: 88% - 93%
5%: 90% - 93%
10%: 93% - 97% |
| PATHIAM-assisted vs. PATHIAM-assisted Reproducibility (Intra-Pathologist) | ≥ 75% concordance (for 3 scoring events) | Exceeded 75% concordance at all three clinical cut-offs.
1%: 85%
5%: 80%
10%: 80% |
| Ki-67 Scoring | | |
| Manual vs. PATHIAM-assisted Substantial Equivalence Concordance | ≥ 75% concordance | Met by all three pathologists for all three clinical cut-offs (>1%, >5%, >10%).
1%: 88% - 93%
5%: 87% - 93%
10%: 81% - 89% |
| PATHIAM-assisted vs. PATHIAM-assisted Reproducibility (Inter-Pathologist) | ≥ 75% concordance | Exceeded 75% concordance at all three clinical cut-offs, and was higher than manual microscopy reproducibility.
1%: 92% - 94%
5%: 90% - 93%
10%: 88% - 95% |
| PATHIAM-assisted vs. PATHIAM-assisted Reproducibility (Intra-Pathologist) | ≥ 75% concordance (for 3 scoring events) | Exceeded 75% concordance at all three clinical cut-offs.
1%: 80%
5%: 85%
10%: 85% |
| System Precision/Reproducibility | Precision and reproducibility similar to predicate devices | The tables for intra- and inter-system studies "confirm the precision and reproducibility of Ki-67 and p53 scoring within the same system and between different systems." and "showed that PATHIAM System precision and reproducibility is similar to that of the predicate devices" (e.g., %CV values provided in tables 5-12). |
Study Details
-
Sample sizes used for the test set and the data provenance:
- Test Set (for comparative and pathologist reproducibility studies):
- p53 study: 120 de-identified archived breast carcinoma sections in Tissue Micro Array (TMA) form.
- Ki-67 study: 120 de-identified archived breast carcinoma sections in TMA form.
- Data Provenance: The samples were sourced from a single research center, Ohio State University Medical Center (OSU). The data is retrospective, using archived pathological specimens. The country of origin is the United States (Ohio).
- Test Set (for system precision/reproducibility studies):
- 8 pre-selected fields of view (EOVs) from TMA cores for both p53 and Ki-67.
- These 8 samples were among the 120 cases from the clinical studies.
- Test Set (for comparative and pathologist reproducibility studies):
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Number of Experts: Three investigators.
- Qualifications: "Qualified pathologists". Specific experience levels (e.g., "10 years of experience") are not provided, but they are from different clinical labs.
- The ground truth used for comparative studies was manual microscopy scores by these pathologists. The system also performs automated scoring as part of its operation, which is then compared to the human pathologist's assessment.
-
Adjudication method for the test set:
- The study design does not explicitly mention a formal adjudication method (e.g., 2+1, 3+1) for resolving disagreements between pathologists' scores in the inter-pathologist studies.
- Concordance was calculated based on the agreement between individual pathologists' scores, or between a pathologist's score and the PATHIAM-assisted score, or between two pathologists' manual scores. The study focused on assessing agreement rather than establishing a single adjudicated ground truth from multiple readers.
-
If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- Yes, an MRMC-like comparative effectiveness study was conducted ("PATHIAM System Comparison Studies (Inter and Intra Pathologist Studies)").
- Effect Size (Improvement with AI assistance):
- The study reports that Inter-Pathologist reproducibility using the PATHIAM system was higher than Inter-Pathologist reproducibility using manual microscopy at all three clinical cut-offs. This indicates an improvement in consistency among human readers when assisted by the AI.
- For p53:
- PATHIAM-assisted vs. PATHIAM-assisted Reproducibility: 88-97%
- Manual vs. Manual Reproducibility: 78-95%
- (Improvement is visible, but an exact "effect size" (e.g., AUC difference, specific statistical metric) is not quantified beyond showing higher concordance ranges.)
- For Ki-67:
- PATHIAM-assisted vs. PATHIAM-assisted Reproducibility: 88-95%
- Manual vs. Manual Reproducibility: 80-91%
- (Similar indication of improvement in consistency.)
-
If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:
- Yes, standalone performance was assessed in the "PATHIAM System Reproducibility and Precision Study (Inter and Intra System Studies)".
- This study specifically aimed to "assess the consistency and reproducibility of the PATHIAM system (no pathologist)" for p53 and Ki-67 scoring on different systems.
- The PATHIAM system's raw scores were used directly for comparison in these precision studies.
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
- For comparative effectiveness studies: The ground truth for comparative performance (manual vs. PATHIAM-assisted) was derived from individual manual microscopy scores by qualified pathologists.
- For pathologist reproducibility studies: The ground truth for inter-pathologist reproducibility was agreement between the PATHIAM-assisted scores of different pathologists. For intra-pathologist reproducibility, it was agreement of a single pathologist's PATHIAM-assisted scores across multiple reads.
- For system precision/reproducibility studies: The "ground truth" was the PATHIAM system's own raw score for a given field of view, and the studies assessed the consistency of these scores across repeated measurements and different systems. There was no independent external ground truth (like a molecular test or pathology review report) in these specific precision studies; they validated the system's internal consistency.
-
The sample size for the training set:
- The document does not explicitly state the sample size used for training the PATHIAM algorithms. The description focuses on the validation studies.
-
How the ground truth for the training set was established:
- Since the training set size is not provided, the method for establishing its ground truth is also not detailed in this document.
§ 864.1860 Immunohistochemistry reagents and kits.
(a)
Identification. Immunohistochemistry test systems (IHC's) are in vitro diagnostic devices consisting of polyclonal or monoclonal antibodies labeled with directions for use and performance claims, which may be packaged with ancillary reagents in kits. Their intended use is to identify, by immunological techniques, antigens in tissues or cytologic specimens. Similar devices intended for use with flow cytometry devices are not considered IHC's.(b)
Classification of immunohistochemistry devices. (1) Class I (general controls). Except as described in paragraphs (b)(2) and (b)(3) of this section, these devices are exempt from the premarket notification requirements in part 807, subpart E of this chapter. This exemption applies to IHC's that provide the pathologist with adjunctive diagnostic information that may be incorporated into the pathologist's report, but that is not ordinarily reported to the clinician as an independent finding. These IHC's are used after the primary diagnosis of tumor (neoplasm) has been made by conventional histopathology using nonimmunologic histochemical stains, such as hematoxylin and eosin. Examples of class I IHC's are differentiation markers that are used as adjunctive tests to subclassify tumors, such as keratin.(2) Class II (special control, guidance document: “FDA Guidance for Submission of Immunohistochemistry Applications to the FDA,” Center for Devices and Radiologic Health, 1998). These IHC's are intended for the detection and/or measurement of certain target analytes in order to provide prognostic or predictive data that are not directly confirmed by routine histopathologic internal and external control specimens. These IHC's provide the pathologist with information that is ordinarily reported as independent diagnostic information to the ordering clinician, and the claims associated with these data are widely accepted and supported by valid scientific evidence. Examples of class II IHC's are those intended for semiquantitative measurement of an analyte, such as hormone receptors in breast cancer.
(3) Class III (premarket approval). IHC's intended for any use not described in paragraphs (b)(1) or (b)(2) of this section.
(c)
Date of PMA or notice of completion of a PDP is required. As of May 28, 1976, an approval under section 515 of the Federal Food, Drug, and Cosmetic Act is required for any device described in paragraph (b)(3) of this section before this device may be commercially distributed. See § 864.3.