Search Results

The ScanScope System is an automated digital slide creation, management, viewing and analysis system. It is intended for in vitro diagnostic use as an aid to the pathologist in the display, detection, counting and classification of tissues and cells of clinical interest based on particular color, intensity, size, pattern and shape.

The IHC HER2 Breast Tissue Tunable Image Analysis application is intended for use as an aid to the pathologist in the detection and semi-quantitative measurement of HER2/neu (c-erbB-2) in formalin-fixed, paraffin-embedded normal and neoplastic tissue.

The IHC HER2 Breast Tissue Tunable Image Analysis application is intended for use as an accessory to the Dako HercepTest™ to aid in the detection and semi-quantitative measurement of HER2/neu (c-erbB-2) in formalin-fixed, paraffin-embedded normal and neoplastic tissue. It is indicated for use as an aid in the assessment of breast cancer patients for whom HERCEPTIN® (Trastuzumab) treatment is being considered. Note: The IHC HER2 Breast Tissue Tunable Image Analysis application is an adjunctive computer-assisted methodology to assist the reproducibility of a qualified pathologist in the acquisition and measurement of images from microscope slides of breast cancer specimens stained for the presence of HER2 receptor protein. The accuracy of the test result depends upon the quality of the immunohistochemical staining. It is the responsibility of a qualified pathologist to employ appropriate morphological studies and controls as specified in the instructions for the HER2 reagent/kit used to assure the validity of the IHC HER2 Breast Tissue Tunable Image Analysis application assisted HER2/neu score. The actual correlation of the HER2 reagents/kits to Herceptin® clinical outcome has not been established.

The ScanScope® XT System is an automated digital slide creation, management, viewing and analysis system. The system is comprised of a slide scanner instrument and a computer executing Spectrum™ software. The system capabilities include digitizing microscope slides at diagnostic resolution, storing and managing the resulting digital slide images, retrieving and displaying digital slides, including support for remote access over wide-area networks, providing facilities for annotating digital slides and entering and editing metadata associated with digital slides, and facilities for image analysis of digital slides, including the ability to quantify characteristics useful to Pathologists, such as measuring and scoring immunohistochemical stains applied to histology specimens, such as Dako HerceptTestTM which reveals the presence of proteins such as Human Epidermal growth factor Receptor 2 (HER2), which may be used to determine patient treatment for breast cancer.

Here's an analysis of the acceptance criteria and study detailed in the provided K080564 submission for the Aperio ScanScope® XT System:

1. Table of Acceptance Criteria and Reported Device Performance

The submission focuses on demonstrating substantial equivalence rather than predefined acceptance criteria in the traditional sense of a specific performance target for accuracy or sensitivity. Instead, the "acceptance criteria" are implicitly met by demonstrating comparable performance to manual microscopy and superior inter-pathologist agreement. The primary performance metric presented is Percent Agreement (PA).

2. Sample Size and Data Provenance

• Test Set Sample Size: 180 formalin-fixed, paraffin-embedded breast tissue specimens.
  • Clinical Site 1: 80 specimens.
  • Clinical Site 2: 100 specimens.
• Data Provenance: Retrospective, with specimens from two unnamed clinical sites. The country of origin is not explicitly stated but is implicitly within the scope of FDA approval, suggesting US-based clinical sites.

3. Number of Experts and Qualifications for Ground Truth for the Test Set

• Number of Experts: Three (3) board-certified pathologists at each clinical site (a total of 6 unique pathologists across both sites, although for each site, it's 3 pathologists).
• Qualifications of Experts: "Board-certified pathologists." No further details on years of experience are provided.

4. Adjudication Method for the Test Set

The primary method for establishing the reference HER2 scores for the image analysis comparison was the manual microscopic review by three pathologists. The algorithm's score was then compared against these individual pathologist scores and implicitly against the consensus of pathologists (e.g., the "average HER2 scores from the three pathologists" was used to stratify slides for the algorithm training set).

For the inter-pathologist agreement, each pathologist's manual score was compared against the others (Pathologist 1 vs 2, 1 vs 3, 2 vs 3). Similarly, for the Image Analysis inter-pathologist agreement, the image analysis scores derived from each pathologist's outlined tumor regions were compared.

The initial manual microscopy average HER2 scores from the three pathologists were used to define the HER2 score distribution for the study.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Status and Effect Size

No explicit MRMC comparative effectiveness study, as typically understood for human readers improving with AI vs. without AI assistance, was performed. The study evaluates the agreement between manual microscopy and the image analysis system, and the inter-pathologist agreement for both manual and image analysis methods separately. It doesn't directly measure the improvement in human reader performance when using AI assistance in a diagnostic workflow.

However, the study does compare inter-pathologist agreement between manual microscopy and image analysis.

• Manual Microscopy Inter-Pathologist Agreement: Ranged from 65.0% to 91.3%
• Image Analysis Inter-Pathologist Agreement: Ranged from 85.0% to 94.0%

This suggests that the image analysis system itself results in higher inter-pathologist agreement compared to manual microscopy performed by independent pathologists. The submission also notes, "This study shows a good example how image analysis can help Pathologists with the standardization of the scoring." and "The variations introduced by a single pathologist by outlining different tumor regions from one read to another is 3x to 3.7x smaller than the variations introduced by different pathologists outlining different tumor regions" which supports the idea that the system could improve consistency.

6. Standalone (Algorithm Only) Performance

Yes, a standalone performance was done for the reported device. The Image Analysis (algorithm) was run in "batch processing mode completely separated from the pathologists outlining the tumor regions to avoid influencing the pathologists in their choice of tumor regions." The agreement percentages for "Manual Microscopy vs Image Analysis - same Pathologist - Agreements" and "Image Analysis - Inter-Pathologists - Agreements" inherently describe the standalone performance relative to human input.

7. Type of Ground Truth Used

Expert Consensus (modified): The ground truth was based on the independent scoring of three board-certified pathologists for each slide, using manual microscopy. For the purpose of stratifying the training set, the "average HER2 score provided by three pathologists using manual microscopy" was used. For comparison studies, the algorithm's output was compared pathologist-by-pathologist to their respective manual reads and to the image analysis scores derived from their own outlined regions.

8. Sample Size for the Training Set

• Algorithm Training Set (for the comparison study): 20 HER2 slides (5 slides for each 0, 1+, 2+, and 3+ HER2 class), randomly selected from the available slides.
• Algorithm Training Set (for the separate analytical performance "Algorithm Training Set" section): 20 slides (again, 5 slides from each 0, 1+, 2+, and 3+ HER2 class, chosen via stratified-random selection) from a set of 100 HER2 slides. The remaining 80 slides formed the evaluation dataset for this separate analysis.

9. How Ground Truth for the Training Set Was Established

The ground truth for the training set was established based on the "average HER2 score from the three pathologists" using manual microscopy. These average scores were used to stratify the slides into 0, 1+, 2+, and 3+ classes from which the training slides were then selected. The algorithm was "tuned" using these selected training slides and the procedure outlined later in the submission (though the specific tuning procedure isn't fully detailed in the provided text).

Ask a specific question about this device

The ScanScope® System is an automated digital slide creation, management, viewing and analysis system. It is intended for in vitro diagnostic use as an aid to the pathologist in the display, detection, counting and classification of tissues and cells of clinical interest based on particular color, intensity, size, pattern and shape.

The ScanScope® system is intended for use as an aid to the pathologist in the detection and quantitative measurement of PR (Progesterone Receptor) by manual examination of the digital slide of formalin-fixed, paraffin-embedded normal and neoplastic tissue immunohistochemically stained for PR on a computer monitor.

It is indicated for use as an aid in the management, prognosis, and prediction of therapy outcomes of breast cancer.

The system comprises a ScanScope® XT digital slide scanner instrument and a computer system executing Spectrum " software. The system capabilitics include digitizing microscope slides at diagnostic resolution, storing and managing the resulting digital slide images, retrieving and displaying digital slides, including support for remote access over wide-area networks, providing facilities for annotating digital slides and entering and editing metadata associated with digital slides, and facilities for image analysis of digital slides, including the ability to quantify characteristics useful to Pathologists, such as measuring and scoring immunohistochemical stains applied to histology specimens, such as Dako PR, which reveal the presence of PR (Progesterone Receptor) protein expression, which may be used to determine patient treatment for breast cancer.

The provided document, K080254, describes the Aperio ScanScope® XT System, intended for in vitro diagnostic use as an aid to pathologists in the display, detection, counting, and classification of tissues and cells of clinical interest. Specifically, the document focuses on its use for the detection and quantitative measurement of Progesterone Receptor (PR) by manual examination of digital slides.

Here's an analysis based on the requested information:

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria are implicitly based on the agreement between manual microscopy and manual reading of digital slides, and the reproducibility of the digital slide system. The document does not explicitly state pre-defined numerical acceptance criteria (e.g., "agreement must be >X%"). Instead, it reports the range of observed agreements and precision metrics.

2. Sample Size Used for the Test Set and Data Provenance

• Test Set Sample Size: 180 formalin-fixed, paraffin-embedded breast tissue specimens.
  • Clinical Site 1: 80 slides.
  • Clinical Site 2: 100 slides.
• Data Provenance: The study was conducted at two Clinical Laboratory Improvement Amendments (CLIA) qualified clinical sites. The specimens were immunohistochemically stained at these clinical sites using Dako IVD FDA cleared reagents. Glass slides were prepared in the sites' clinical laboratories.
  • Clinical Site 1: Specimens selected based on existing clinical scores to provide an equal distribution of PR slides across different percentage positive nuclei ranges (0%, 1-9%, 10-49%, 50-100%).
  • Clinical Site 2: Routine specimens taken from clinical operations, representing a typical clinical setting.
• Retrospective/Prospective: Not explicitly stated as strictly retrospective or prospective. The specimens were "from both clinical sites" and for Site 1, "selected based on their clinical scores on file," suggesting a retrospective selection of cases. For Site 2, "routine specimens taken from their clinical operation" could imply concurrent collection or a recent retrospective selection. The reading of these slides by pathologists for the study itself was a prospective activity within the study design.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts

• Number of Experts: Three different board-certified pathologists at each of the two clinical sites were used. This totals 6 pathologists for the initial manual reads (3 at site 1, 3 at site 2). Later, for the digital reads, the "same three Pathologists at each clinical site" performed the digital read. It's unclear if the same 6 pathologists or 3 total pathologists across both sites participated in all stages. However, for ground truth generation, it was 3 pathologists per site.
• Qualifications of Experts: "Board-certified staff pathologists."

4. Adjudication Method for the Test Set

The "ground truth" for the comparison study was established by the consensus or average of the three pathologists' manual microscopy reads. The statistical analyses were presented for each of the scores (percentage of positive nuclei and intensity scores) and "comparatively between the two methods for the clinical sites with their different three pathologists." This implies a form of expert consensus, where the average/distribution of their reads served as the reference for comparison, rather than a formal adjudication to a single "correct" answer by an independent panel.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Was an MRMC study done? Yes, a comparative study was conducted where three pathologists at each of two clinical sites read 180 cases by traditional manual microscopy and then later (after a wash-out period and randomization) read the digital slides of the same cases on a computer monitor. This fits the description of a multi-reader, multi-case comparison study.
• Effect size of human readers improve with AI vs without AI assistance: This study does not describe AI assistance for human readers. The ScanScope® XT System is described as a digital slide creation, management, viewing, and analysis system intended as an aid to the pathologist by manual examination of digital slides. The analytical performance section mentions an "image analysis algorithm," but this algorithm was used for precision/reproducibility studies of the system itself, not to assist pathologists in their interpretation of diagnostic cases. The clinical comparison study directly compares manual microscopy performance to human pathologists reading digital slides visually. Therefore, an effect size of human readers with AI vs without AI assistance is not reported because the clinical study did not involve AI assistance for the pathologists.

6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study

• Was a standalone study done? No, a standalone study demonstrating the algorithm's diagnostic performance without human-in-the-loop was not performed or reported for its intended diagnostic use.
  • An "image analysis algorithm" was used in the analytical performance (precision/reproducibility) section to quantify cell features and scoring schemes objectively. This algorithm reported percentage of positive nuclei and intensity scores for system assessment, not for diagnostic claims for the algorithm itself. It was used to understand scanning variability, not to replace or assist a pathologist's diagnosis.

7. Type of Ground Truth Used

The ground truth for the comparison study (clinical performance) was established by expert consensus based on the average/distribution of manual microscopy readings from three board-certified pathologists for each slide. The Dako IVD FDA cleared Monoclonal Mouse Anti-Human Progesterone Receptor (Clone PgR 636) (K020023) was used for immunohistochemical staining, providing a standardized basis for the pathologists' assessment.

8. Sample Size for the Training Set

The document does not mention a training set for the ScanScope® XT System itself, as it is a digital slide scanner and management system, with the focus of the clinical study being on the equivalence of manual pathologist interpretation of digital slides compared to glass slides. The "image analysis algorithm" used in the analytical performance section is not presented as a component needing a separate training set for diagnostic purposes described here.

9. How the Ground Truth for the Training Set Was Established

As no training set is described for a diagnostic algorithm, the method for establishing its ground truth is not applicable in this context. The clinical study focuses on establishing the equivalence of the digital viewing method to conventional microscopy, with the human pathologist remaining the primary interpreter.

Ask a specific question about this device

The ScanScope® XT System is an automated digital slide creation, management, viewing and analysis system. It is intended for in vitro diagnostic use as an aid to the pathologist in the display, detection, counting and classification of tissues and cells of clinical interest based on particular color, intensity, size, pattern and shape.

The IHC ER Image Analysis application is intended for use as an aid to the pathologist in the detection and quantitative measurement of ER (Estrogen Receptor) in formalin-fixed paraffinembedded normal and neoplastic tissue.

The IHC PR Image Analysis application is intended for use as an aid to the pathologist in the detection and quantitation measurement of PR (Progesterone Receptor) in formalin-fixed. paraffin-embedded normal and neoplastic tissue.

It is indicated for use as an aid in the management, prognosis, and prediction of therapy outcomes of breast cancer.

Note: The IHC ER and PR Image Analysis applications are an adjunctive computer-assisted methodology to assist the reproducibility of a qualified pathologist in the acquisition and measurement of images from microscope slides of breast cancer specimens stained for the presence of estrogen and progesterone receptor proteins. The accuracy of the test result depends upon the quality of the immunohistochemical staining. It is the responsibility of a qualified pathologist to employ appropriate morphological studies and controls as specified in the instructions for the ER and PR reagent/kit used to assure the validity of the IHC ER and PR Image Analysis application assisted scores.

The system comprises a ScanScope® XT digital slide scanner instrument and a computer system executing Spectrum " software. The system capabilities include digitizing microscope slides at diagnostic resolution, storing and managing the resulting digital slide images, retrieving and displaying digital slides, including support for remote access over wide-area networks, providing facilities for annotating digital slides and entering and editing metadata associated with digital slides, and facilities for image analysis of digital slides, including the ability to quantify characteristics useful to Pathologists, such as measuring and scoring immunohistochemical stains applied to histology specimens, such as Dako ER/PR, which reveal the presence of ER (Estrogen Receptor) protein and PR (Progesterone Receptor) protein expression, which may be used to determine patient treatment for breast cancer.

Hardware Operation: The ScanScope XT digital slide scanner creates seamless true color digital slide images of entire glass slides in a matter of minutes. A high numeric aperture 20x, as found on conventional microscopes, is used to produce high-quality images. (When the 2X magnification changer is inserted, the effective magnification of the images is 40X.) The ScanScope XT employs a linear-array scanning technique that generates images free from optical aberrations along the scanning axis. The result is digital slide images that have no tiling artifacts and are seamless.

Software Operation: The Spectrum software is a full-featured digital pathology management system. The software runs on a server computer called a Digital Slide Repository (DSR), which stores digital slide images on disk storage such as a RAID array, and which hosts an SQL database that contains digital slide metadata. Spectrum includes a web application and services which encapsulate database and digital slide image access for other computers. The Spectrum server supports the capability of running a variety of image analysis algorithms on digital slides, and storing the results of analysis into the database. Spectrum also includes support for locally or remotely connected image workstation computers, which run digital slide viewing and analysis software provided as part of Spectrum.

Overview of System Operation: The laboratory technician or operator loads glass microscope slides into a specially designed slide carrier with a capacity of up to 120 slides. The scanning process begins when the operator starts the ScanScope scanner and finishes when the scanner has completed scanning of all loaded slides. As each glass slide is processed, the system automatically stores individual "striped" images of the tissue contained on the glass slide and integrates the striped images into a single digital slide image, which represents a histological reconstruction of the entire tissue section. After scanning is completed, the operator is able to view and perform certain analytical tests on the digital slides.

Here's a summary of the acceptance criteria and the study that proves the device meets them, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria are implied through the results presented, which aim to demonstrate substantial equivalence to manual microscopy. The study primarily focuses on inter-pathologist agreement for both manual microscopy and the device's image analysis, as well as agreement between manual microscopy and the device's image analysis. Precision studies also demonstrate the device's consistency.

Given the document structure, the "acceptance criteria" appear to be defined not as specific numerical thresholds prior to the study, but rather by demonstrating that the device performs comparably to manual microscopy and shows acceptable levels of precision. The reported performance shows the ranges of agreement found.

2. Sample Size for the Test Set and Data Provenance

• ER Study Test Set: 80 formalin-fixed, paraffin-embedded breast tissue specimens.
  • Data Provenance: Retrospective, from a single CLIA-qualified clinical site in the US (implied by CLIA qualification, as it's a US regulatory standard). Specimens were "selected based on their clinical scores on file."
• PR Study Test Set: 180 formalin-fixed, paraffin-embedded breast tissue specimens.
  • Data Provenance: Retrospective, from two CLIA-qualified clinical sites in the US. 80 slides from the first site (selected based on clinical scores) and 100 slides from the second site (routine clinical specimens, representing the target population).

3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications

• Number of Experts: Three board-certified pathologists at each clinical site.
• Qualifications: "Board-certified staff pathologists" at CLIA-qualified clinical sites. (No specific years of experience are mentioned).

4. Adjudication Method for the Test Set

The document describes a form of expert consensus and comparison rather than a strict adjudication to arrive at a single "ground truth" value for the test set.

• For Manual Microscopy: Three different board-certified pathologists at each clinical site performed a blinded manual review of each glass slide. They reported the percentage of positive nuclei and average intensity score. The study then uses the "manual microscopy average percentages of positive nuclei from the three pathologists" and "manual microscopy average intensity scores from the three pathologists" for comparisons. This suggests an averaging approach rather than a specific adjudication rule (e.g., 2-out-of-3 majority; a 3+1 method where a fourth expert adjudicates disagreements is not explicitly stated).
• For Image Analysis: Each of the three pathologists outlined tumor regions on digital slides (blinded from each other and from image analysis results). Image analysis was then performed on each set of outlined regions, resulting in a separate image analysis score for each of the three pathologists. No formal adjudication is described to combine these three algorithm scores into a single "ground truth" for the algorithm; rather, the agreement between the pathologists' manual scores and their respective image analysis scores is evaluated, as well as inter-pathologist agreement for both methods.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, and its effect size:

• This was a type of MRMC study, as multiple readers (pathologists) evaluated multiple cases (slides) both manually and with the assistance of the device (though the AI component was run after and blinded from the pathologists' region selection).
• Effect Size: The document does not report an effect size for how much human readers improve with AI vs. without AI assistance. Instead, it reports agreement percentages between pathologists' manual scores, between pathologists' AI-assisted scores, and between a pathologist's manual score and their corresponding AI-assisted score. The study's focus was on demonstrating substantial equivalence and agreement, not on measuring reader improvement with assistance.

6. If a standalone (i.e., algorithm-only without human-in-the-loop performance) was done:

• Yes, in part. The image analysis algorithm reported the percentage of positive nuclei and average intensity score for each digital slide. However, the input to the algorithm (the tumor regions) was still defined by human pathologists. The critical step of selecting the region of interest was human-in-the-loop, even if the quantitative analysis within that region was standalone.
• The document states: "Image analysis was run in batch processing mode completely separated from the pathologists outlining the tumor regions to avoid influencing the pathologists in their choice of tumor regions." This clarifies that the numerical output of the algorithm was standalone for a given region, but the selection of that region itself was pathologist-driven.

7. The Type of Ground Truth Used

• Expert Consensus (Averaged): For comparing the device performance, the "ground truth" for manual microscopy was established by taking the average percentage of positive nuclei and average intensity scores from three board-certified pathologists. This serves as the reference against which the digital system's performance (also tied to pathologist-defined regions) is compared.
• Not Pathology, but based on Pathology Scores: While pathology slides were used, the ground truth was not solely an independent pathology report in the traditional sense, but rather the statistically combined scores of the evaluating pathologists who were part of the study.
• Not Outcomes Data: The study did not use patient outcomes data.

8. The Sample Size for the Training Set

• The document does not specify the sample size used for the training set for the image analysis algorithm. The provided information focuses entirely on the clinical validation study (test set).

9. How the Ground Truth for the Training Set was Established

• The document does not provide information on how the ground truth for the training set was established, as the training set details are not described.

Ask a specific question about this device

The ScanScope® System is an automated digital slide creation, management, viewing and analysis system. It is intended for in vitro diagnostic use as an aid to the pathologist in the display, detection, counting and classification of tissues and cells of clinical interest based on particular color, intensity, size, pattern and shape.

The IHC HER2 Manual Read of Digital Slides application is intended for use as an aid to the pathologist in the detection and semi-quantitative measurement of HER-2/neu (cerbB-2) by manual examination of the digital slide of formalin-fixed, paraffin-embedded normal and neoplastic tissue immunohistochemically stained for HER-2 receptors on a computer monitor. HER2 results are indicated for use as an aid in the management, prognosis and prediction of therapy outcomes of breast cancer.

The IHC HER2 Manual Read of Digital Slides application is intended for use as an accessory to the Dako HercepTest™ to aid the pathologist in the detection and semi-quantitative measurement of HER-2/neu (cerbB-2) by manual examination of the digital slide of formalin-fixed, paraffin-embedded normal and neoplastic tissue immunohistochemically stained for HER-2 receptors on a computer monitor. When used with the Dako HercepTest™, it is indicated for use as an aid in the assessment of breast cancer patients for whom HERCEPTIN® (Trastuzumab) treatment is being considered. Note: The actual correlation of the Dako HercepTest™ to Herceptin® clinical outcome has not been established.

The ScanScope ® System is an automated digital slide creation, management, viewing and analysis system. The ScanScope® System components consist of an automated digital microscope slide scanner, computer, color monitor, keyboard and digital pathology information management software. The system capabilities include digitizing microscope slides at high resolution, storing and managing the resulting digital slide images, retrieving and displaying digital slides, including support for remote access over wide-area networks, providing facilities for annotating digital slides and entering and cditing metadata associated with digital slides, and facilities for image analysis of digital slides. Image analysis capabilities include the ability to detect and quantify characteristics useful to Pathologists, such as detecting and quantifying certain proteins revealed by immunohistochemical stains applied to histology specimens. The remote digital slide viewing capabilities of the system support reading digital slides on a computer monitor, enabling Pathologists to make clinically relevant decisions analogous to those they make using a conventional microscope. Specifically, the system supports the pathologist in the detection and semi-quantitative measurement of HER-2/neu (cerbB-2) by manual examination of the digital slide of formalin-fixed, paraffin-embedded normal and neoplastic tissue immunohistochemically stained for HER-2 receptors on a computer monitor.

Here's an analysis of the acceptance criteria and the study proving the device meets them, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

The document does not explicitly state pre-defined acceptance criteria (e.g., "The device must achieve X% agreement"). Instead, it focuses on demonstrating "substantial equivalence" through agreement percentages within and between pathologists and methods. The performance is presented as ranges and specific percentages.

2. Sample Size Used for the Test Set and Data Provenance

• Test Set Sample Size:
  • Clinical Comparison Study: 180 formalin-fixed, paraffin-embedded breast tissue specimens.
  • Precision/Reproducibility Study: 8 HER2 slides (with two slides per HER2 score 0, 1+, 2+, and 3+).
• Data Provenance:
  • Clinical Comparison Study: 80 specimens from a "first clinical site" and 100 specimens from a "second clinical site." The text does not specify the country of origin, but it implies a clinical setting within the US given the FDA submission. The study was prospective in the sense that the readings were done as part of the study, rather than re-analyzing existing interpretations. The specimens themselves might have been collected retrospectively from existing tissue banks.
  • Precision/Reproducibility Study: The 8 HER2 slides were "sampled from one of the clinical sites." Similar to the above, the country of origin is not explicitly stated but implied to be within the US.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

• Clinical Comparison Study: Three (3) pathologists at each of the two clinical sites were used, totaling six pathologists for the study (3 per site). The qualifications are stated as "Pathologists," which implies they are medical doctors specialized in pathology. No specific experience level (e.g., 10 years) is given.
• Precision/Reproducibility Study (for initial slide sampling): The "rounded average score of the manual microscopy scores provided by the three pathologists" was used to sample the 8 slides.
• Precision/Reproducibility Study (Intra-Pathologist): One pathologist.
• Precision/Reproducibility Study (Inter-Pathologists): Three pathologists.

4. Adjudication Method for the Test Set

The primary clinical comparison study used a "blinded manual examination" by individual pathologists. The agreement percentages are reported between pathologists and between methods (manual microscopy vs. digital reading for the same pathologist). There is no explicit mention of an adjudication method (like 2+1 or 3+1) to establish a single, definitive ground truth for each case by consensus or a senior expert for the entire test set. Instead, the study design allows for comparing agreement rates.

For the precision studies, "outliers are defined as scores that are different from the median values of the scores provided by the pathologist over 5 runs" (Intra-Pathologist) or "scores that are different from the median values of the scores provided by the three pathologists in this study" (Inter-Pathologist). This is a method for identifying discrepancies, not necessarily for adjudicating a final ground truth for each case for the purpose of primary performance metrics.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and the Effect Size of How Much Human Readers Improve with AI vs. Without AI Assistance

• Was an MRMC study done? Yes, a form of MRMC study was performed in the "Comparison studies: a. Method comparison with predicate device." It involved multiple readers (pathologists) and multiple cases (180 specimens). It compared two reading methods: manual microscopy and manual digital slide reading.

• Effect Size of AI assistance: The device (ScanScope XT System) is described as an "automated digital slide creation, management, viewing and analysis system." However, the "IHC HER2 Manual Read of Digital Slides application" specifically tested in the clinical comparison is for manual examination of digital slides by a pathologist. The document explicitly states: "The IHC HER2 Manual Read of Digital Slides application is intended for use as an aid to the pathologist in the detection and semi-quantitative measurement of HER-2/neu ... by manual examination of the digital slide...".

  Therefore, this study does not measure the effect size of AI assistance on human readers. It rather evaluates the equivalence of reading digital slides on a monitor to reading physical slides under a microscope.

  While there is a mention of "Aperio's IHC HER2 image analysis algorithm" within the precision/reproducibility section, that part of the study was not for clinical comparison or human reader improvement, but to objectively quantify system variability separate from pathologist variability. The clinical comparison specifically focuses on the manual read function of the digital slides by pathologists.

6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Was Done

Yes, a standalone performance assessment was conducted for the device's analytical capabilities in terms of image analysis on HER2 scores. This specifically evaluated "Aperio's IHC HER2 image analysis algorithm" (not the manual reading by pathologists). The precision studies (Intra-System, Inter-Day/Intra-System, Inter-System) were done using this algorithm, with results showing 100% agreement for calculated HER2 scores and very low standard deviations/ranges for cellular percentages. This demonstrates the performance of the algorithm without human intervention for specific analytical tasks, separate from the clinical "manual read" study.

7. The Type of Ground Truth Used

• Clinical Comparison Study (for the comparison itself): The ground truth for the comparison was essentially established by the pathologists' manual microscopic examination of the glass slides. The digital readings were compared against this traditional method, which serves as the established clinical standard. There isn't an "absolute" or "independent" ground truth like pathology reports from excisional biopsies or patient outcomes explicitly stated for each case in the comparison data. The predicate device for substantial equivalence also relies on manual microscopic assessment.
• Precision/Reproducibility Study (for sampling): The 8 slides used for precision studies were sampled based on the "rounded average score of the manual microscopy scores provided by the three pathologists," implying a consensus-based approach derived from expert readings.

8. The Sample Size for the Training Set

The document does not explicitly state a sample size for a "training set." The clinical study described is a comparison study and precision study, not a study for training or validating an AI model for clinical decision-making with direct human-in-the-loop assistance. While the text mentions "Aperio's IHC HER2 image analysis algorithm," it doesn't provide details on how this algorithm was trained or validated.

9. How the Ground Truth for the Training Set Was Established

Since no specific training set and its ground truth establishment are detailed for an AI model for clinical use in this submission, this information is not available in the provided text. The "ground truth" referenced for the analytical precision studies of the algorithm is inherent in its design to quantify cell features and use a scoring scheme mimicking pathologists' manual approaches.

Ask a specific question about this device

The ScanScope System is an automated digital slide creation, management, viewing and analysis system. It is intended for in vitro diagnostic use as an aid to the pathologist in the display, detection, counting and classification of tissues and cells of clinical interest based on particular color, intensity, size, pattern and shape.

The IHC HER2 Image Analysis application is intended for use as an aid to the pathologist in the detection and semi-quantitative measurement of HER2/neu (c-erbB-2) in formalin-fixed, paraffin-embedded normal and neoplastic tissue.

The IHC HER2 Image Analysis application is intended for use as an accessory to the Dako HercepTest™ to aid in the detection and semi-quantitative measurement of HER2/neu (c-erbB-2) in formalin-fixed, paraffin-embedded normal and neoplastic tissue. When used with the Dako HercepTest™, it is indicated for use as an aid in the assessment of breast cancer patients for whom HERCEPTIN® (Trastuzumab) treatment is being considered. Note: The IHC HER2 Image Analysis application is an adjunctive computer-assisted methodology to assist the reproducibility of a qualified pathologist in the acquisition and measurement of images from microscope slides of breast cancer specimens stained for the presence of HER-2 receptor protein. The accuracy of the test result depends upon the quality of the immunohistochemical staining. It is the responsibility of a qualified pathologist to employ appropriate morphological studies and controls as specified in the instructions for the Dako HercepTest™ to assure the validity of the IHC HER2 Image Analysis application assisted HER-2/neu score. The actual correlation of the Dako HercepTest™ to Herceptin® clinical outcome has not been established.

The ScanScope® XT System is an automated digital slide creation, management, viewing and analysis system. The ScanScope® XT System components consist of an automated digital microscope slide scanner, computer, color monitor, keyboard and digital pathology information management software. The system capabilities include digitizing microscope slides at high resolution, storing and managing the resulting digital slide images, retrieving and displaying digital slides, including support for remote access over wide-area networks, providing facilities for annotating digital slides and entering and editing metadata associated with digital slides, and facilities for image analysis of digital slides. Image analysis capabilities include the ability to quantify characteristics useful to Pathologists, such as measuring and scoring immunohistochemical stains applied to histology specimens, such as the Dako HerceptTest"M, which reveal the presence of proteins such as Human Epidermal growth factor Receptor 2 (HER2), which may be used to determine patient treatment for breast cancer.

Here's a breakdown of the acceptance criteria and the study details for the Aperio Technologies ScanScope® XT System, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

The document does not explicitly state pre-defined acceptance criteria in terms of numerical thresholds for comparison between the manual microscopy and the image analysis system. Instead, it aims to demonstrate substantial equivalence by showing that the "agreements between the pathologists' manual microscopy and performed (blinded) image analysis were comparable to the inter-pathologists agreements for manual microscopy." The study design itself serves as the framework for proving this comparability.

Clinical Site 2:
Manual Microscopy Inter-Pathologist Agreements (PA): 84.0% - 90.0%
Image Analysis Inter-Pathologist Agreements (PA): 87.0% - 92.0%
Manual Microscopy vs. Image Analysis (Same Pathologist) Agreements (PA): 79.0% - 90.0%

Conclusion: Inter-pathologist agreements for image analysis (86.3-93.8%) were comparable to manual microscopy (76.3-91.3%). Agreements between manual microscopy and image analysis (77.5-92.5%) were also comparable to inter-pathologist agreements for manual microscopy (76.3-91.3%). |
| Precision (intra-run, inter-run, inter-system) | Intra-run/Intra-system: 100% perfect agreement for calculated HER2 scores across all runs.
Inter-run/Intra-system: 100% perfect agreement for calculated HER2 scores across all runs.
Inter-systems: 100% perfect agreement for calculated HER2 scores across all systems and across all runs. |

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size: 180 formalin-fixed, paraffin-embedded breast tissue specimens.
  • Site 1: 80 specimens (with approximately equal HER2 score distribution)
  • Site 2: 100 routine specimens
• Data Provenance: Retrospective, as the specimens were already stained and presumably collected prior to the study. The study was conducted at two clinical sites, implying a multi-center study within the US (though country of origin is not explicitly stated, "clinical sites" typically refers to healthcare facilities within the country where the submission is filed – in this case, the US FDA).

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts

• Number of Experts: Three pathologists at each of the two clinical sites, totaling 6 pathologists involved in establishing ground truth.
• Qualifications of Experts: The document refers to them as "pathologists," implying they are qualified medical professionals specializing in pathology. No specific years of experience or sub-specialty certification are provided.

4. Adjudication Method for the Test Set

The document describes a comparative study where three pathologists at each site independently performed a blinded read of the glass slides for manual microscopy. For the image analysis, the same three pathologists remotely viewed and outlined tumor regions. The algorithm then reported the HER2 score for each pathologist's outlined regions.

There is no explicit adjudication method (like 2+1 or 3+1 consensus) described for establishing a single "ground truth" for each slide based on expert opinion before comparison. Instead, the study compares inter-pathologist agreement for manual reads, inter-pathologist agreement for image analysis results, and agreement between individual pathologist's manual reads and their corresponding image analysis results. The image analysis algorithm's output serves as a separate measure to be compared against each pathologist's manual assessment.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs. Without AI Assistance

• MRMC Comparative Effectiveness Study: Yes, an MRMC-like study was conducted. It involved multiple readers (pathologists) and multiple cases (180 breast tissue specimens). The comparison was between manual microscopy and image analysis, with pathologists themselves interacting with the image analysis system by outlining regions.
• Effect Size of Improvement with AI Assistance: The document does not quantify the improvement of human readers with AI assistance in terms of an effect size. It focuses on the "comparability" of agreements:
  • Inter-pathologist agreements for the blinded image analysis (PA: 86.3-93.8%) were comparable to inter-pathologist agreements for manual microscopy (PA: 76.3-91.3%).
  • Agreements between the pathologists' manual microscopy and performed (blinded) image analysis (PA: 77.5-92.5%) were comparable to inter-pathologist agreements for manual microscopy (PA: 76.3-91.3%).

This indicates the system performed similarly to human agreement without necessarily making a claim of "improvement" in diagnostic accuracy or efficiency for the human reader while using the AI. The study's goal was to demonstrate substantial equivalence, not superior performance or augmentation.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done

Yes, a standalone component of the algorithm's performance was evaluated. The pathologists outlined representative tumor regions, and then the algorithm was run in "batch mode, blinded from the pathologists" and used "out of the box" to report the HER2 score for those outlined regions. This means the algorithm's output for a defined region was generated independently of further human intervention in the scoring process for that specific region.

7. The Type of Ground Truth Used

The ground truth used for comparison was expert consensus (implied via agreement metrics) or expert opinion for each pathologist's manual read. There wasn't a single, definitive "gold standard" ground truth like pathology or outcomes data established for each slide beforehand. Instead, the study evaluates agreement between different forms of assessment (manual vs. AI-assisted) and among experts. The Dako HercepTest™ staining is mentioned as the method used for preparing the specimens, which is a standardized immunohistochemical stain, but the interpretation of this stain (the HER2 score) is what is being compared.

8. The Sample Size for the Training Set

The document does not provide any information regarding the sample size used for the training set of the IHC HER2 Image Analysis application. It only describes the test set used for validating the device.

9. How the Ground Truth for the Training Set Was Established

The document does not provide any information on how the ground truth for the training set was established. This information is typically proprietary to the developer and not always disclosed in 510(k) summaries, which focus on the validation study.

Ask a specific question about this device