(259 days)
The Philips HER2/neu IHC Digital Manual Read is intended for in vitro diagnostic use as an aid to the pathologist in the display, detection, counting and classification of tissues and cells of clinical interest based on particular color, intensity, size, pattern and shape. The Philips HER2/neu IHC Digital Manual Read is based on the Philips Digital Pathology Solution platform, which is an automated digital slide creation, management, viewing and analysis system.
The Philips HER2/neu IHC Digital Manual Read is intended for use as an accessory to the Dako HercepTest™ to aid in the detection and semi-quantitative measurement of HER2/neu (c-erbB-2) in formalin-fixed, paraffin-embedded neoplastic tissue immunohistochemically stained for HER-2 receptors on a computer monitor. When used with the Dako HercepTest™, it is indicated for use as an aid in the assessment of breast cancer patients from whom HERCEPTIN® (Trastuzumab), PERJETA® (Pertuzumab) or KADCYLA® (Ado-Trastuzumab Emtansine) treatment is being considered. Note: The actual correlation of the Dako HercepTest™ to Herceptin®, Perjeta®, or Kadcyla®, clinical outcome has not been established.
Note: The Philips HER2/neu IHC Digital Manual Read is for evaluation of digital images of immunohistochemically stained slides that would otherwise be appropriate for manual visualization by conventional microscopy. It is the responsibility of a qualified pathologist to employ appropriate morphological studies and controls as specified in the instructions for Dako HercepTest™ to assure the validity of the scores obtained using Philips HER2/neu IHC Digital Manual Read.
The Philips HER2/neu IHC Digital Manual Read is a digital manual read application and an adjunct to primary diagnosis. The application utilizes the Philips Digital Pathology Solution (DPS) platform that includes a Philips Ultra Fast Scanner (UFS) and Philips Image Management System (IMS).
The Philips Digital Pathology Solution is an automated digital slide creation, management, sharing, viewing and analysis system. It is intended for in vitro diagnostic use as an aid to the pathologist in the display, detection, counting and classification of tissues and cells of clinical interest based on particular color, intensity, size, pattern and shape.
The Philips UFS system digitizes slides at high resolution and generates the whole slide images (WSI). The Philips UFS also takes snapshot images of the entire glass slide as well as the glass slide label and decodes the barcode. Based on the macro image of the scanner determines which region on the slide will be scanned. All images, WSI and snapshot images together with information about the decoded barcode are sent to the Philips Image Management System (Philips IMS).
The Phillips IMS comes supplied with the Barco MDCC 2121 monitor and runs on commercially available server and workstation IT hardware which are specified by Philips and purchased by the customer. The server stores and manages the digital slide images and digital slide metadata. The server supports interoperability with other information systems such as the laboratory information systems (LIS) via a HL7 interface. The IMS Web Viewer software provides the User Interface for the pathologist to view and read the digital slides.
Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state formal acceptance criteria with specific threshold values for the performance metrics. Instead, it describes clinical studies aiming to demonstrate substantial equivalence to manual optical reading. The conclusion states that "All of the method comparison and precision clinical studies components met the expected acceptance criteria." This implies that the observed agreements were considered sufficient, even if a numerical threshold wasn't articulated in the summary.
Inferred "Acceptance" (demonstrated performance) and Reported Device Performance:
| Performance Metric | Reported Device Performance (Manual Digital vs. Manual Optical) |
|---|---|
| Method Comparison (Inter-Method Agreement - Trichotomous) | |
| Pathologist 1 (Site 1) - Score 0, 1+ | 91.40% [83.75%, 96.21%] |
| Pathologist 1 (Site 1) - Score 2+ | 71.74% [56.54%, 84.01%] |
| Pathologist 1 (Site 1) - Score 3+ | 97.78% [88.23%, 99.94%] |
| Pathologist 2 (Site 1) - Score 0, 1+ | 68.75% [58.48%, 77.82%] |
| Pathologist 2 (Site 1) - Score 2+ | 88.24% [76.13%, 95.56%] |
| Pathologist 2 (Site 1) - Score 3+ | 100.0% [91.78%, 100.0%] |
| Pathologist 3 (Site 2) - Score 0, 1+ | 88.62% [81.64%, 93.64%] |
| Pathologist 3 (Site 2) - Score 2+ | 70.00% [53.47%, 83.44%] |
| Pathologist 3 (Site 2) - Score 3+ | 96.97% [84.24%, 99.92%] |
| Intra-Pathologist Precision | |
| Manual Optical Agreement (outliers) | 87.50% (12.50%) |
| Manual Digital Agreement (outliers) | 92.50% (7.50%) |
| Inter-Pathologist Precision (Binary) | |
| Manual Digital Overall Agreement | 84.78% [80.80%, 88.77%] |
| Manual Optical Overall Agreement | 88.04% [84.06%, 91.67%] |
| Instrument Precision (3x3 Overall Agreement) | |
| Inter-Day/Intra-System (IDIS) | 92.98% [86.76%, 96.40%] |
| Inter-System (Inter-S) | 88.24% [80.55%, 93.14%] |
2. Sample Size for Test Set and Data Provenance
- Sample Size (Method Comparison Study): Two hundred (200) formalin-fixed, paraffin-embedded breast tissue specimens. Each specimen was scored by 3 pathologists using two different methods, resulting in 6 study reads per specimen. However, the 4x4 tables show totals around 184-196, indicating some slides might have been excluded due to quality issues.
- Sample Size (Intra-Pathologist Precision Study): A target set of 8 HercepTest™ slides, evaluated 5 times for each method (Manual Digital and Manual Optical). To reduce bias, these 8 slides were mixed with an additional set of 12 "wild card" slides for each reading session (total of 20 slides per session).
- Sample Size (Inter-Pathologist Precision Study): This study utilized the data collected from all sites in the method comparison study (i.e., all 200 specimens scored by 3 pathologists).
- Sample Size (Instrument Precision Studies): 40 HercepTest™ stained tissue slides (core slides). These were augmented with 32 "wild card" slides for each reading session.
- Data Provenance: The specimens were obtained from a tissue bank of de-identified human specimens. The country of origin is not specified, but the applicant is based in the Netherlands. The study simulates the use environment (three pathologists from two sites participated). The nature of the historical tissue bank makes it retrospective.
3. Number of Experts and Qualifications for Ground Truth for the Test Set
The concept of a single "ground truth" for the test set, as might be established by pathology or outcomes data, is not directly applicable here. This study primarily compares two methods of interpretation (manual optical vs. manual digital) by human pathologists.
-
Method Comparison Study:
- Number of "Experts": Three (3) pathologists.
- Qualifications: "The pathologists in the study were trained in the use of the investigational device according to the labeling." No further specific qualifications like years of experience or board certification are detailed for these three.
- "Ground Truth": No external "ground truth" was established a priori. The study aims to demonstrate concordance between the two reading methods by the same pathologists. The slides were prescreened by a pathologist (not necessarily one of the study pathologists) to evaluate staining quality and provide an initial score, which was used for selection to ensure a balanced distribution of scores, but this initial score was not treated as the definitive ground truth against which study observations were compared.
-
Intra-Pathologist Precision Study:
- Number of "Experts": One (1) pathologist.
- Qualifications: Not explicitly stated beyond "a pathologist."
- "Ground Truth": The "median values of the scores provided by the pathologist over 5 runs" were used as the reference against which outlying scores were identified for agreement calculation.
-
Inter-Pathologist Precision Study:
- Number of "Experts": Three (3) pathologists.
- Qualifications: Not explicitly stated beyond "pathologists."
- "Ground Truth": No external "ground truth." This study evaluated agreement among the pathologists' readings.
-
Instrument Precision Studies:
- Number of "Experts": Two (2) board-certified pathologists. One for Inter-S and one for IDIS.
- Qualifications: "Two board-certified pathologists."
- "Ground Truth": The specimens were "pre-scored... by a pathologist not involved in the study" to ensure equal distribution of scores, but this pre-score was not used as definitive ground truth for the analyses presented. The study assessed the consistency of interpretation of images generated by different scans/days by the assigned study pathologist.
4. Adjudication Method for the Test Set
For the method comparison and precision studies:
- No explicit adjudication method (like 2+1 or 3+1) was used to establish a definitive ground truth by consensus among multiple experts.
- Instead, the studies focused on concordance between the manual optical reading and the manual digital reading by the same pathologist, and then on agreement among different pathologists' readings for inter-pathologist precision.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- Yes, a form of MRMC study was done in the "Method Comparison (Manual Digital vs. Manual Optical)" section. It compared the performance of human readers (pathologists) using a traditional optical microscope versus using the Philips HER2/neu IHC Digital Manual Read system.
- Effect Size of AI vs. Without AI Assistance:
The device is described as "No image analysis algorithms are applied to these systems (i.e. digital manual read only)." This means the device itself does not include any AI assistance for interpretation; it is a digital display system for manual reading by a human. Therefore, there is no reported effect size of how much human readers improve with AI vs. without AI assistance because the "AI" component is absent from the interpretation process. The study demonstrates the equivalence of manual digital reading to manual optical reading.
6. Standalone (Algorithm Only) Performance
- No standalone (algorithm only) performance study was done because the device is explicitly a "Digital Manual Read" application and "No image analysis algorithms are applied to these systems." The device functions as a display and management system for images that are still manually interpreted by a pathologist.
7. Type of Ground Truth Used
As elaborated in point 3, the studies did not use a single, external "ground truth" (such as unanimous expert consensus, pathology, or outcomes data) against which the device's performance was measured in an absolute sense.
- Method Comparison: The "ground truth" was effectively the pathologist's own reading using the conventional optical microscope, against which their reading using the digital system was compared for agreement.
- Precision Studies: The "ground truth" was either the pathologist's median score (for intra-pathologist) or the agreement among pathologists (for inter-pathologist).
- For the instrument precision studies, the comparison was based on consistency of scores by a single pathologist across different scans/days/instruments.
8. Sample Size for the Training Set
- The document describes a "Digital Manual Read" device, which means it is a display and management system for pathologists to manually interpret whole slide images. It explicitly states, "No image analysis algorithms are applied to these systems."
- Therefore, there is no AI algorithm being "trained," and consequently, there is no "training set" of data for an AI model.
9. How Ground Truth for Training Set Was Established
- As there is no AI algorithm that requires training, there is no training set and no ground truth established for a training set.
§ 864.1860 Immunohistochemistry reagents and kits.
(a)
Identification. Immunohistochemistry test systems (IHC's) are in vitro diagnostic devices consisting of polyclonal or monoclonal antibodies labeled with directions for use and performance claims, which may be packaged with ancillary reagents in kits. Their intended use is to identify, by immunological techniques, antigens in tissues or cytologic specimens. Similar devices intended for use with flow cytometry devices are not considered IHC's.(b)
Classification of immunohistochemistry devices. (1) Class I (general controls). Except as described in paragraphs (b)(2) and (b)(3) of this section, these devices are exempt from the premarket notification requirements in part 807, subpart E of this chapter. This exemption applies to IHC's that provide the pathologist with adjunctive diagnostic information that may be incorporated into the pathologist's report, but that is not ordinarily reported to the clinician as an independent finding. These IHC's are used after the primary diagnosis of tumor (neoplasm) has been made by conventional histopathology using nonimmunologic histochemical stains, such as hematoxylin and eosin. Examples of class I IHC's are differentiation markers that are used as adjunctive tests to subclassify tumors, such as keratin.(2) Class II (special control, guidance document: “FDA Guidance for Submission of Immunohistochemistry Applications to the FDA,” Center for Devices and Radiologic Health, 1998). These IHC's are intended for the detection and/or measurement of certain target analytes in order to provide prognostic or predictive data that are not directly confirmed by routine histopathologic internal and external control specimens. These IHC's provide the pathologist with information that is ordinarily reported as independent diagnostic information to the ordering clinician, and the claims associated with these data are widely accepted and supported by valid scientific evidence. Examples of class II IHC's are those intended for semiquantitative measurement of an analyte, such as hormone receptors in breast cancer.
(3) Class III (premarket approval). IHC's intended for any use not described in paragraphs (b)(1) or (b)(2) of this section.
(c)
Date of PMA or notice of completion of a PDP is required. As of May 28, 1976, an approval under section 515 of the Federal Food, Drug, and Cosmetic Act is required for any device described in paragraph (b)(3) of this section before this device may be commercially distributed. See § 864.3.