(274 days)
The ScanScope® System is an automated digital slide creation, management, viewing and analysis system. It is intended for in vitro diagnostic use as an aid to the pathologist in the display, detection, counting and classification of tissues and cells of clinical interest based on particular color, intensity, size, pattern and shape.
The ScanScope® system is intended for use as an aid to the pathologist in the detection and quantitative measurement of PR (Progesterone Receptor) by manual examination of the digital slide of formalin-fixed, paraffin-embedded normal and neoplastic tissue immunohistochemically stained for PR on a computer monitor.
It is indicated for use as an aid in the management, prognosis, and prediction of therapy outcomes of breast cancer.
The system comprises a ScanScope® XT digital slide scanner instrument and a computer system executing Spectrum " software. The system capabilitics include digitizing microscope slides at diagnostic resolution, storing and managing the resulting digital slide images, retrieving and displaying digital slides, including support for remote access over wide-area networks, providing facilities for annotating digital slides and entering and editing metadata associated with digital slides, and facilities for image analysis of digital slides, including the ability to quantify characteristics useful to Pathologists, such as measuring and scoring immunohistochemical stains applied to histology specimens, such as Dako PR, which reveal the presence of PR (Progesterone Receptor) protein expression, which may be used to determine patient treatment for breast cancer.
The provided document, K080254, describes the Aperio ScanScope® XT System, intended for in vitro diagnostic use as an aid to pathologists in the display, detection, counting, and classification of tissues and cells of clinical interest. Specifically, the document focuses on its use for the detection and quantitative measurement of Progesterone Receptor (PR) by manual examination of digital slides.
Here's an analysis based on the requested information:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria are implicitly based on the agreement between manual microscopy and manual reading of digital slides, and the reproducibility of the digital slide system. The document does not explicitly state pre-defined numerical acceptance criteria (e.g., "agreement must be >X%"). Instead, it reports the range of observed agreements and precision metrics.
| Metric | Acceptance Criteria (Implicit) | Reported Device Performance (ScanScope® XT System) |
|---|---|---|
| Clinical Performance (Pathologist Agreement) | ||
| Inter-Pathologist Agreement (Digital Slides) - % Positive Nuclei | N/A (Comparison study to conventional microscopy for substantial equivalence) | 76.3% - 98.0% |
| Inter-Pathologist Agreement (Manual Microscopy) - % Positive Nuclei | N/A | 83.8% - 99.0% |
| Agreement between Manual Microscopy & Digital Slides - % Positive Nuclei | N/A (Demonstrate substantial equivalence) | 78.8% - 100.0% |
| Inter-Pathologist Agreement (Digital Slides) - Intensity Score | N/A | 58.8% - 78.0% |
| Inter-Pathologist Agreement (Manual Microscopy) - Intensity Score | N/A | 58.8% - 88.0% |
| Agreement between Manual Microscopy & Digital Slides - Intensity Score | N/A | 62.5% - 96.0% |
| Analytical Performance (Reproducibility via Image Analysis Algorithm) | ||
| Intra-system (10 scans) - % Positive Nuclei Standard Deviation | N/A (Demonstrate precision/reproducibility) | Overall SD: 0.54% (max 1.47%), Average Range: 1.06% (max 4.78%) |
| Intra-system (10 scans) - Intensity Values Standard Deviation | N/A | Overall SD: 0.9 (max 1.60), Average Range: 2.48 (max 4.27) |
| Inter-day/Intra-system (20 scans) - % Positive Nuclei Standard Deviation | N/A | Overall SD: 0.54% (max 1.09%), Average Range: 1.52% (max 3.90%) |
| Inter-day/Intra-system (20 scans) - Intensity Values Standard Deviation | N/A | Overall SD: 1.44 (max 2.43), Average Range: 5.29 (max 11.39) |
| Inter-system (3 systems, 10 scans each) - % Positive Nuclei Standard Deviation | N/A | Individual System Average SD: 0.54%, 0.53%, 0.75% (max 1.47%, 1.23%, 2.05%) |
| Combined Systems Overall Average SD: 0.87% (max 1.57%) | ||
| Inter-system (3 systems, 10 scans each) - Intensity Values Standard Deviation | N/A | Individual System Average SD: 0.9%, 1.01%, 0.93% (max 1.60%, 1.48%) |
| Combined Systems Overall Average SD: 1.35% (max 2.03%) | ||
| Intra-pathologist (5 reads) - % Positive Nuclei Standard Deviation (Manual Microscopy) | N/A | Overall Average SD: 6.73% (max 16.73%), Average Range: 9.8% (max 40%) |
| Intra-pathologist (5 reads) - % Positive Nuclei Standard Deviation (Digital Slides) | N/A | Overall Average SD: 11.81% (max 28.72%), Average Range: 16.2% (max 75%) |
| Intra-pathologist (5 reads) - Intensity Score Outliers (Manual Microscopy) | N/A | 8 outliers out of 50 scores (16%) |
| Intra-pathologist (5 reads) - Intensity Score Outliers (Digital Slides) | N/A | 9 outliers out of 50 scores (18%) |
| Inter-pathologist - % Positive Nuclei Standard Deviation (Manual Microscopy) | N/A | Overall Average SD: 13.30% (max 32.15%), Average Range: 17.2% (max 60%) |
| Inter-pathologist - % Positive Nuclei Standard Deviation (Digital Slides) | N/A | Overall Average SD: 11.3% (max 20.82%), Average Range: 16.0% (max 40%) |
| Inter-pathologist - Intensity Score Outliers (Manual Microscopy) | N/A | 7 outliers out of 30 scores (23%) |
| Inter-pathologist - Intensity Score Outliers (Digital Slides) | N/A | 7 outliers out of 30 scores (23%) |
2. Sample Size Used for the Test Set and Data Provenance
- Test Set Sample Size: 180 formalin-fixed, paraffin-embedded breast tissue specimens.
- Clinical Site 1: 80 slides.
- Clinical Site 2: 100 slides.
- Data Provenance: The study was conducted at two Clinical Laboratory Improvement Amendments (CLIA) qualified clinical sites. The specimens were immunohistochemically stained at these clinical sites using Dako IVD FDA cleared reagents. Glass slides were prepared in the sites' clinical laboratories.
- Clinical Site 1: Specimens selected based on existing clinical scores to provide an equal distribution of PR slides across different percentage positive nuclei ranges (0%, 1-9%, 10-49%, 50-100%).
- Clinical Site 2: Routine specimens taken from clinical operations, representing a typical clinical setting.
- Retrospective/Prospective: Not explicitly stated as strictly retrospective or prospective. The specimens were "from both clinical sites" and for Site 1, "selected based on their clinical scores on file," suggesting a retrospective selection of cases. For Site 2, "routine specimens taken from their clinical operation" could imply concurrent collection or a recent retrospective selection. The reading of these slides by pathologists for the study itself was a prospective activity within the study design.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
- Number of Experts: Three different board-certified pathologists at each of the two clinical sites were used. This totals 6 pathologists for the initial manual reads (3 at site 1, 3 at site 2). Later, for the digital reads, the "same three Pathologists at each clinical site" performed the digital read. It's unclear if the same 6 pathologists or 3 total pathologists across both sites participated in all stages. However, for ground truth generation, it was 3 pathologists per site.
- Qualifications of Experts: "Board-certified staff pathologists."
4. Adjudication Method for the Test Set
The "ground truth" for the comparison study was established by the consensus or average of the three pathologists' manual microscopy reads. The statistical analyses were presented for each of the scores (percentage of positive nuclei and intensity scores) and "comparatively between the two methods for the clinical sites with their different three pathologists." This implies a form of expert consensus, where the average/distribution of their reads served as the reference for comparison, rather than a formal adjudication to a single "correct" answer by an independent panel.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- Was an MRMC study done? Yes, a comparative study was conducted where three pathologists at each of two clinical sites read 180 cases by traditional manual microscopy and then later (after a wash-out period and randomization) read the digital slides of the same cases on a computer monitor. This fits the description of a multi-reader, multi-case comparison study.
- Effect size of human readers improve with AI vs without AI assistance: This study does not describe AI assistance for human readers. The ScanScope® XT System is described as a digital slide creation, management, viewing, and analysis system intended as an aid to the pathologist by manual examination of digital slides. The analytical performance section mentions an "image analysis algorithm," but this algorithm was used for precision/reproducibility studies of the system itself, not to assist pathologists in their interpretation of diagnostic cases. The clinical comparison study directly compares manual microscopy performance to human pathologists reading digital slides visually. Therefore, an effect size of human readers with AI vs without AI assistance is not reported because the clinical study did not involve AI assistance for the pathologists.
6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study
- Was a standalone study done? No, a standalone study demonstrating the algorithm's diagnostic performance without human-in-the-loop was not performed or reported for its intended diagnostic use.
- An "image analysis algorithm" was used in the analytical performance (precision/reproducibility) section to quantify cell features and scoring schemes objectively. This algorithm reported percentage of positive nuclei and intensity scores for system assessment, not for diagnostic claims for the algorithm itself. It was used to understand scanning variability, not to replace or assist a pathologist's diagnosis.
7. Type of Ground Truth Used
The ground truth for the comparison study (clinical performance) was established by expert consensus based on the average/distribution of manual microscopy readings from three board-certified pathologists for each slide. The Dako IVD FDA cleared Monoclonal Mouse Anti-Human Progesterone Receptor (Clone PgR 636) (K020023) was used for immunohistochemical staining, providing a standardized basis for the pathologists' assessment.
8. Sample Size for the Training Set
The document does not mention a training set for the ScanScope® XT System itself, as it is a digital slide scanner and management system, with the focus of the clinical study being on the equivalence of manual pathologist interpretation of digital slides compared to glass slides. The "image analysis algorithm" used in the analytical performance section is not presented as a component needing a separate training set for diagnostic purposes described here.
9. How the Ground Truth for the Training Set Was Established
As no training set is described for a diagnostic algorithm, the method for establishing its ground truth is not applicable in this context. The clinical study focuses on establishing the equivalence of the digital viewing method to conventional microscopy, with the human pathologist remaining the primary interpreter.
§ 864.1860 Immunohistochemistry reagents and kits.
(a)
Identification. Immunohistochemistry test systems (IHC's) are in vitro diagnostic devices consisting of polyclonal or monoclonal antibodies labeled with directions for use and performance claims, which may be packaged with ancillary reagents in kits. Their intended use is to identify, by immunological techniques, antigens in tissues or cytologic specimens. Similar devices intended for use with flow cytometry devices are not considered IHC's.(b)
Classification of immunohistochemistry devices. (1) Class I (general controls). Except as described in paragraphs (b)(2) and (b)(3) of this section, these devices are exempt from the premarket notification requirements in part 807, subpart E of this chapter. This exemption applies to IHC's that provide the pathologist with adjunctive diagnostic information that may be incorporated into the pathologist's report, but that is not ordinarily reported to the clinician as an independent finding. These IHC's are used after the primary diagnosis of tumor (neoplasm) has been made by conventional histopathology using nonimmunologic histochemical stains, such as hematoxylin and eosin. Examples of class I IHC's are differentiation markers that are used as adjunctive tests to subclassify tumors, such as keratin.(2) Class II (special control, guidance document: “FDA Guidance for Submission of Immunohistochemistry Applications to the FDA,” Center for Devices and Radiologic Health, 1998). These IHC's are intended for the detection and/or measurement of certain target analytes in order to provide prognostic or predictive data that are not directly confirmed by routine histopathologic internal and external control specimens. These IHC's provide the pathologist with information that is ordinarily reported as independent diagnostic information to the ordering clinician, and the claims associated with these data are widely accepted and supported by valid scientific evidence. Examples of class II IHC's are those intended for semiquantitative measurement of an analyte, such as hormone receptors in breast cancer.
(3) Class III (premarket approval). IHC's intended for any use not described in paragraphs (b)(1) or (b)(2) of this section.
(c)
Date of PMA or notice of completion of a PDP is required. As of May 28, 1976, an approval under section 515 of the Federal Food, Drug, and Cosmetic Act is required for any device described in paragraph (b)(3) of this section before this device may be commercially distributed. See § 864.3.