Search Results

The Aperio ePathology eIHC IVD System is an automated digital slide creation, management, viewing and analysis system. It is intended for in vitro diagnostic use as an aid to the pathologist in the display, detection, counting and classification of tissues and cells of clinical interest based on particular color, intensity, size, pattern and shape.

The IHC HER2 Image Analysis application is intended for use as an aid to the pathologist in the detection and semi-quantitative measurement of HER2/neu (c-erbB-2) in formalin-fixed, paraffin-embedded neoplastic tissue.

The IHC HER2 Image Analysis application is intended for use as an accessory to the Dako HercepTest™ to aid in the detection and semi-quantitative measurement of Her2/neu (c-erbB-2) in formalin-fixed, paraffin-embedded neoplastic tissue. When used with the Dako HercepTest™, it is indicated for use as an aid in the assessment of breast cancer patients for whom HERCEPTIN®(Trastuzumab) treatment is being considered.

The IHC ER Image Analysis application is intended for use as an aid to the pathologist in the detection and quantitative measurement of ER (Estrogen Receptor) in formalin-fixed paraffin-embedded neoplastic tissue.

The IHC PR Image Analysis application is intended for use as an aid to the pathologist in the detection and quantitation measurement of PR (Progesterone Receptor) in formalin-fixed, paraffin-embedded neoplastic tissue.

lt is indicated for use as an aid in the management, prognosis, and prediction of therapy outcomes of breast cancer.

The Aperio ePathology eIHC IVD System ("System") is an automated digital slide creation, management, viewing and analysis system. The Aperio ePathology eIHC IVD System components consist of an automated digital microscope slide scanner, computer, color monitor, keyboard and digital pathology information management software. The system capabilities include digitizing microscope slides at high resolution, storing and managing the resulting digital slide images, retrieving and displaying digital slides, including support for remote access over wide-area networks, providing tools for annotating digital slides and entering and editing data associated with digital slides, and tools for image analysis of digital slides. Image analysis capabilities include the ability to quantify characteristics useful to Pathologists, such as measuring and scoring immunohistochemical stains applied to histology specimens, including the Dako HerceptTest™, Dako ER/PR, which reveal the presence of proteins such as Human Epidermal growth factor Receptor 2 (HER2), ER (Estrogen Receptor) protein and PR (Progesterone Receptor) protein expression.

Below summarizes the acceptance criteria and study information for the Aperio ePathology eIHC IVD System, specifically for its HER2, ER, and PR Image Analysis applications.

1. Table of Acceptance Criteria and Reported Device Performance

For IHC HER2 Image Analysis:

For IHC ER and PR Image Analysis:

2. Sample Size and Data Provenance for Test Set

• Sample Size: Not explicitly stated for either HER2 or ER/PR. The document mentions "slides represented the range of HER2 scores (0, 1+, 2+ and 3+)" and "slides represented the range of ER and PR scores that are observed clinically."
• Data Provenance: The samples were "Paraffin embedded breast tissue slides prepared with the appropriate Dako HER2 IHC test kit" and "Paraffin embedded breast tissue slides prepared with the appropriate Dako ER and PR IHC test kits." The origin (e.g., country) is not specified. The studies were internal system performance tests, implying a retrospective nature where pre-existing clinical samples were used.

3. Number of Experts and Qualifications for Ground Truth

• The document states that the HER2, ER, and PR scores obtained from the updated ScanScope systems were "evaluated for concordance with the scores obtained from the predicate device." This suggests the predicate device's scores served as the reference for comparison, rather than human experts directly establishing ground truth for the test set in these specific performance studies.
• No information is provided about the number or qualifications of experts directly establishing ground truth for the test set used in the system performance testing. The predicate device's output was the reference.

4. Adjudication Method

• Not applicable as the comparison was against the predicate device's scores, not human expert consensus requiring adjudication.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• No, an MRMC comparative effectiveness study was not done. The studies described were system-level performance tests evaluating the updated ScanScope instruments against a predicate device.
• Therefore, no effect size of human readers improving with AI vs. without AI assistance is provided. The device is intended as an "adjunctive computer-assisted methodology to assist the reproducibility of a qualified pathologist."

6. Standalone Performance Study

• Yes, a standalone study (algorithm-only performance) was conducted. The "System Performance Testing" described for both HER2 and ER/PR evaluated the "accuracy of the HER2 image analysis algorithm on the updated ScanScope instruments" and "accuracy of the ER and PR image analysis algorithms on the updated ScanScope instruments," respectively, by comparing their output against the predicate device's scores.

7. Type of Ground Truth Used

• For the system performance testing, the ground truth used was the scores obtained from the legally marketed predicate device (ScanScope XT). The updated ScanScope models' results were evaluated for concordance with the predicate device's scores.

8. Sample Size for the Training Set

• The document does not provide information regarding the sample size specifically used for training the image analysis algorithms. The described studies are performance evaluations of the already developed algorithms.

9. How Ground Truth for Training Set was Established

• The document does not provide information on how the ground truth for the training set was established. The focus of this submission is on the direct comparison of the updated devices' performance to the predicate device.

Ask a specific question about this device

The Virtuoso system provides automated digital slide creation, management, analysis, and viewing. It is intended for in vitro diagnostic use as an aid to the pathologist in the display, detection, counting, review and classification of tissues and cells of clinical interest based on particular morphology, color, intensity, size, pattern and shape.

The IHC Ki-67 (30-9) Digital Read and Image Analysis applications are intended for use as an aid to the pathologist in the detection and semi-quantitative measurement of Ki-67 protein in formalin-fixed, paraffin-embedded normal and neoplastic tissue. Ki-67 results are indicated for use in assessing the proliferative activity of normal and neoplastic breast tissue. When used with Ventana Medical Systems, Inc. CONFIRM™ anti-Ki-67 (30-9) Rabbit Monoclonal Primary Antibody Assay, it is indicated for use as an aid in the assessment of Ki-67 status in breast cancer patients (but is not the sole basis for treatment).

The Virtuoso™ System is an instrument-plus-software system designed to assist the qualified pathologist in the consistent assessment of protein expression in immunohistochemically stained histologic sections from formalin-fixed, paraffinembedded normal and neoplastic tissues. The system consists of a slide scanner (iScan), computer, monitor, keyboard, mouse, image analysis algorithms for specific immunohistochemical markers, and software with a Windows web browser-based user interface. Virtuoso is a web-based, end-to-end, digital pathology software solution that allows pathology laboratories to acquire, manage, view, analyze, share, and report digital images of pathology specimens. Using the Virtuoso software, the pathologist can view digital images, add annotations, make measurements, perform image analysis, and generate reports.

Hardware: The iScan slide scanning device captures digital images of formalin-fixed. paraffin-embedded tissues that are suitable for storage and viewing. The device includes a digital slide scanner, racks for loading glass slides, computer, scanner software, keyboard, mouse and monitor.

Software: The Virtuoso software is designed to complement the routine workflow of a qualified pathologist in the review of immunohistochemically stained histologic slides. It allows the user to select fields of view (FOVs) in the digital image for analysis and provides quantitative data on these FOVs to assist with interpretation. The software makes no independent interpretations of the data and requires competent human intervention for all steps in the analysis process.

The Ventana Virtuoso™ System for IHC Ki-67 (30-9) was tested for its performance in both digital reading (DR) and image analysis (IA) of Ki-67 stained slides. The overall acceptance criterion for each measurement was 75% overall percent agreement (OPA) with manual microscopic reads.

1. Table of Acceptance Criteria and Reported Device Performance

2. Sample Size and Data Provenance for the Test Set

• Digital Read (DR) Test Set: 120 cases.
• Image Analysis (IA) Test Set: 120 cases.
• Data Provenance: The document does not explicitly state the country of origin. The studies appear to be prospective concordance studies where new data was generated for validation.

3. Number of Experts and Qualifications for Ground Truth Establishment (Test Set)

• Digital Read (DR): "one pathologist at one site" established the manual score (reference result). No specific qualifications are provided beyond "pathologist".
• Image Analysis (IA): "three pathologists at three sites" established the manual score (reference result). No specific qualifications are provided beyond "pathologists".

4. Adjudication Method for the Test Set

The document describes concordance studies where manual scores were compared to device scores. It does not mention any formal adjudication method beyond establishing the "manual score (reference result)" by one or more pathologists. This implies a single expert's opinion served as the ground truth for each case initially, rather than multiple experts with a specific adjudication process.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No formal MRMC comparative effectiveness study is described where human readers' performance with and without AI assistance is directly compared to quantify an effect size. The studies focus on the agreement between the device alone (digital read or image analysis) and manual reads.

6. Standalone (Algorithm Only) Performance

Yes, standalone performance was done for both Digital Read and Image Analysis:

• Digital Read Performance: This section describes the agreement between the digital image reviewed by a pathologist ("Digital Read") and the manual microscopic read. While a pathologist is involved in the "Digital Read," the intent is to show that replacing the physical microscope with a digital image viewer (which is part of the system) does not negatively impact the pathologist's ability to score the cases.
• Image Analysis Performance: This section directly assesses the agreement between the Virtuoso Image Analysis application's output and the manual microscopic read. This represents the algorithm-only performance in providing a quantitative score before any potential human override.

7. Type of Ground Truth Used

The ground truth used for both the Digital Read and Image Analysis studies was expert consensus (specifically, manual microscopic reads) by qualified pathologists. The manual score was explicitly stated as the "reference result" or "true score."

8. Sample Size for the Training Set

The document does not provide any information regarding the sample size used for the training set of the Virtuoso™ System's image analysis algorithms. This information is typically found in the development and validation sections, which are not detailed in this 510(k) summary.

9. How Ground Truth for the Training Set Was Established

The document does not provide any information on how the ground truth for the training set was established.

Ask a specific question about this device

The Virtuoso system provides automated digital slide creation, management, analysis, and viewing. It is intended for in vitro diagnostic use as an aid to the pathologist in the display, detection, counting, review and classification of tissues and cells of clinical interest based on particular morphology, color, intensity, size, pattern and shape.

The Virtuoso™ System for p53 (DO-7) is for digital read and image analysis applications. This particular Virtuoso system is intended for use as an aid to the pathologist in the detection and semi-quantitative measurement of p53 (DO-7) protein in formalin-fixed, paraffin-embedded normal and neoplastic tissue. This device is an accessory to the Ventana Medical Systems, Inc. CONFIRM™ anti-p53 (DO-7) Mouse Monoclonal Primary Antibody assay. The Ventana Medical Systems, Inc. CONFIRM™ anti-p53 assay is indicated for the assessment of p53 protein where mutations have been linked to tumor proliferation. When used with this assay, the Virtuoso™ System for p53 (DO-7) is indicated for use as an aid in the assessment of p53 status in breast cancer patients (but is not the sole basis for treatment).

The Virtuoso™ System is an instrument-plus-software system designed to assist the qualified pathologist in the consistent assessment of protein expression in immunohistochemically stained histologic sections from formalin-fixed, paraffin-embedded normal and neoplastic tissues. The system consists of a slide scanner (iScan), computer, monitor, keyboard, mouse, image analysis algorithms for specific immunohistochemical markers, and software with a Windows web browser-based user interface. Virtuoso is a web-based, end-to-end, digital pathology software solution that allows pathology laboratories to acquire, manage, view, analyze, share, and report digital images of pathology specimens. Using the Virtuoso software, the pathologist can view digital images, add annotations, make measurements, perform image analysis, and generate reports.

Hardware: The iScan slide scanning device captures digital images of formalin-fixed, paraffin-embedded tissues that are suitable for storage and viewing. The device includes a digital slide scanner, racks for loading glass slides, computer, scanner software, keyboard, mouse and monitor.

Software: The Virtuoso software is designed to complement the routine workflow of a qualified pathologist in the review of immunohistochemically stained histologic slides. It allows the user to select fields of view (FOVs) in the digital image for analysis and provides quantitative data on these FOVs to assist with interpretation. The software makes no independent interpretations of the data and requires competent human intervention for all steps in the analysis process.

Acceptance Criteria and Device Performance Study for Virtuoso™ System for IHC p53 (DO-7)

The information provided describes clinical validation studies for the Virtuoso™ System for IHC p53 (DO-7), first with the Benchmark XT stainer (predicate device's studies essentially) and then with the Benchmark ULTRA stainer (for this specific submission). The acceptance criteria are implicitly derived from the reported performance in these studies, focusing on agreement between the device's digital read (DR) and image analysis (IA) with manual microscopic assessment.

1. Table of Acceptance Criteria and Reported Device Performance

Note: The provided document does not explicitly state predefined "acceptance criteria" but rather reports the observed performance and concludes that the system is "safe and effective for its intended use" based on these results. Therefore, the "Acceptance Criteria" below are inferred from the reported performance figures that led to regulatory clearance.

2. Sample Sizes Used for the Test Set and Data Provenance

For predicate device (Benchmark XT stainer):

• Sample Size (Agreement/Concordance):
  • Site 1: n = 119
  • Site 2: n = 119
  • Site 3: n = 117
  • Site 4: n = 114 (for Digital Read), n = 105 (for Image Analysis)
  • Total across sites: Approximately 469 unique cases for Digital Read, and 460 unique cases for Image Analysis.
• Sample Size (Reproducibility - Intra-Pathologist/Inter-Day and Inter-Pathologist): The document reports agreement percentages but does not explicitly state the number of cases (n) used for these specific reproducibility comparisons in the summary table. However, confusion matrices for intra-pathologist reproducibility show "Session 1 Neg" count for example, which implies a smaller, constant set of cases (e.g., 40 cases if (26+14) for neg and (1+25) for pos in Session 1 for Digital Read).
• Data Provenance: The document states "across four sites". No country of origin is specified. The studies are clinical validation studies, suggesting prospective data collection for the validation phase.

For new device (Benchmark ULTRA stainer):

• Sample Size (Agreement/Concordance): 120 cases
• Data Provenance: "one pathologist at one site". No country of origin is specified. This was a concordance study.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications

For predicate device (Benchmark XT stainer):

• Number of Experts: The agreement/concordance study involved "Each pathologist's Virtuoso digital read results were compared to their manual results." This implies that the ground truth was established by the same pathologists who then used the digital system. The reproducibility studies involved three pathologists.
• Qualifications: "qualified pathologist." No specific years of experience or sub-specialty are explicitly mentioned beyond being "qualified."

For new device (Benchmark ULTRA stainer):

• Number of Experts: "one pathologist at one site."
• Qualifications: "qualified pathologist." No specific years of experience or sub-specialty are explicitly mentioned beyond being "qualified."

4. Adjudication Method for the Test Set

The document does not explicitly describe a formal adjudication method (e.g., 2+1, 3+1). For the agreement studies, the manual microscopic reading by the "qualified pathologist" appears to be the reference standard against which the digital read and image analysis results are compared. This suggests that the pathologist's own manual score served as the definitive ground truth for their comparative analysis, not an independent adjudicated ground truth from multiple experts for each case before comparison.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Yes, a form of MRMC study was done for reproducibility purposes for the predicate device (Benchmark XT stainer).
  • "Inter-Pathologist/Site" study: This evaluated the reproducibility among three pathologists for digital read and image analysis.
  • However, this was a reproducibility study among AI-assisted reads, not a direct comparison of human readers with AI vs. without AI assistance to measure an "effect size" of improvement. The primary concordance studies compared each pathologist's digital/IA read to their own manual read.
• Effect Size of human reader improvement: The document does not report an effect size for how much human readers improve with AI vs. without AI assistance in the format of a comparative effectiveness study showing gain in performance. Instead, it measures the agreement between the AI-assisted read and the manual read, and the reproducibility of the AI-assisted reads among multiple pathologists and over time for a single pathologist.

6. Standalone (Algorithm Only) Performance Study

• Yes, a standalone performance was done for Image Analysis (IA). The "Image Analysis vs Manual Method" sections provide data on the algorithm's performance (standalone, as it generates the quantitative score) compared to the manual method. While a human selects the fields of view, the scoring itself is algorithmic.

7. Type of Ground Truth Used

• Expert Consensus / Pathology (Manual Read): The ground truth for the predicate device studies and the new device study was established by a "qualified pathologist" performing a "manual method" with a traditional microscope. This is described as the "reference manual method" or "manual score (reference result)."

8. Sample Size for the Training Set

The document does not provide information on the sample size used for the training set for the Virtuoso™ system's image analysis algorithms. The studies described are clinical validation studies using test sets, not details about the algorithm development or training data.

9. How the Ground Truth for the Training Set Was Established

As the document does not provide details on the training set, it does not specify how the ground truth for the training set was established.

Ask a specific question about this device

The Virtuoso system provides automated digital slide creation, management, analysis, and viewing. It is intended for in vitro diagnostic use as an aid to the pathologist in the display, detection, counting, review and classification of tissues and cells of clinical interest based on particular morphology, color, intensity, size, pattern and shape.

The Virtuoso™ System for Ki67 (30-9) is for digital read and image analysis applications. This particular Virtuoso system is intended for use as an aid to the pathologist in the detection and semi-quantitative measurement of Ki67 (30-9) protein in formalin-fixed, paraffin-embedded normal and neoplastic tissue. This device is an accessory to the Ventana Medical Systems, Inc. CONFIRM™ anti-Ki67 (30-9) Rabbit Monoclonal Primary Antibody assay. The Ventana Medical Systems, Inc. CONFIRM™ anti-Ki67 (30-9) assay is indicated for use in assessing the proliferative activity of normal and neoplastic breast tissue. When used with this assay, the Virtuoso™ System for Ki67 (30-9) is indicated for use as an aid in the assessment of Ki-67 status in breast cancer patients (but is not the sole basis for treatment).

The Virtuoso™ System is an instrument-plus-software system designed to assist the qualified pathologist in the consistent assessment of protein expression in immunohistochemically stained histologic sections from formalin-fixed, paraffinembedded normal and neoplastic tissues. The system consists of a slide scanner (iScan), computer, monitor, keyboard, mouse, image analysis algorithms for specific immunohistochemical markers, and software with a Windows web browser-based user interface. Virtuoso is a web-based, end-to-end, digital pathology software solution that allows pathology laboratories to acquire, manage, view, analyze, share, and report digital images of pathology specimens. Using the Virtuoso software, the pathologist can view digital images, add annotations, make measurements, perform image analysis, and generate reports.

Hardware: The iScan slide scanning device captures digital images of formalin-fixed, paraffin-embedded tissues that are suitable for storage and viewing. The device includes a digital slide scanner, racks for loading glass slides. computer, scanner software, keyboard, mouse and monitor.

Software: The Virtuoso software is designed to complement the routine workflow of a qualified pathologist in the review of immunohistochemically stained histologic slides. It allows the user to select fields of view (FOVs) in the digital image for analysis and provides quantitative data on these FOVs to assist with interpretation. The software makes no independent interpretations of the data and requires competent human intervention for all steps in the analysis process.

Here's a breakdown of the acceptance criteria and the study details for the VENTANA Virtuoso™ System for IHC Ki-67 (30-9), based on the provided 510(k) summary:

1. Table of Acceptance Criteria and Reported Device Performance

The document does not explicitly state pre-defined "acceptance criteria" in terms of specific thresholds that needed to be met for agreement percentages. Instead, it presents the results of concordance and reproducibility studies. The performance reported below serves as the demonstrated effectiveness of the device.

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size:
  • Concordance studies:
    • Site 1: n = 120
    • Site 2: n = 118 (Digital Read), n = 117 (Image Analysis)
    • Site 3: n = 114
    • Site 4: n = 118 (Digital Read), n = 117 (Image Analysis)
  • Reproducibility studies (intra-pathologist):
    • The confusion matrices show cell counts of 37-38 for positive and 1-8 for negative detections in Session 1, suggesting a smaller sample size per pathologist for this specific part, but the total number of cases for reproducibility is not explicitly stated.
  • Reproducibility studies (inter-pathologist): The confusion matrices show cell counts of ~30-60 for negative and positive detections per site/pathologist comparison, again implying a specific sample size for this portion of the study that is not explicitly stated as a total N.
• Data Provenance: The document does not specify the country of origin of the data. The study was a "primary study" that "evaluated overall system performance across four sites," implying a multi-center study. It does not explicitly state if the data was retrospective or prospective. Given it's a clinical validation study, it would typically be prospective, but this is not confirmed.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

• The ground truth for the concordance studies (Digital Read vs. Manual, Image Analysis vs. Manual) was established by pathologists performing a "manual method (with a traditional microscope)".
• The number of pathologists involved in establishing the "manual method" ground truth for the test set is not explicitly stated, but it implies at least one pathologist per site for the concordance studies (four sites) and three pathologists for the reproducibility studies.
• Qualifications: Referred to as "qualified pathologist[s]". Specific years of experience or sub-specialty are not provided.

4. Adjudication Method for the Test Set

The document does not describe a formal adjudication method (e.g., 2+1 or 3+1 consensus) for establishing the ground truth from the manual method. It simply states the comparison was made against the pathologist's manual results. For the reproducibility studies, it measures agreement between pathologists or within a pathologist over time, not a specific ground truth adjudication process for the test set itself.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Yes, elements of an MRMC study were conducted:
  • The "Agreement/Concordance" section compares the Virtuoso Digital Read and Image Analysis applications against a "Manual Method" performed by pathologists. Each pathologist's Virtuoso digital read results were compared to their manual results. This is a comparison of human performance (manual) against human-assisted performance (digital read and IA).
  • The "Reproducibility" section directly assesses inter-pathologist and intra-pathologist variability with the Virtuoso system.
• Effect Size of Human Readers' Improvement with AI vs. Without AI Assistance: The document does not provide a direct "effect size" in terms of how much human readers improve with AI assistance compared to solely manual reading. Instead, it measures the agreement between the AI-assisted methods (Digital Read, Image Analysis) and the "manual method" (considered the true score). For instance, overall agreements for Digital Read vs. Manual ranged from 81-92%, and for Image Analysis vs. Manual ranged from 81-88%. This indicates a high level of agreement, but not an improvement factor in diagnostic accuracy or efficiency.

6. Standalone Algorithm Performance

• Yes, in part. The "Image Analysis" application results (e.g., in the "Virtuoso Image Analysis vs Manual Method" section) represent the performance of the algorithm itself, providing quantitative scores. However, the device description emphasizes that "The software makes no independent interpretations of the data and requires competent human intervention for all steps in the analysis process." The pathologist selects the fields of view and can accept or override the software's score. Therefore, while the algorithm provides a standalone score, its final clinical performance is inherently linked to human interaction. The "Digital Read" performance also represents an AI-assisted mode where the human reads the digital image.

7. Type of Ground Truth Used

• Expert Consensus / Pathologist Interpretation (Manual Method): The ground truth was established by pathologists performing a "manual method (with a traditional microscope)". This means the manual pathological interpretation served as the reference standard against which the digital read and image analysis results were compared.

8. Sample Size for the Training Set

The document does not provide any information regarding the sample size used for the training set of the Virtuoso system's algorithms.

9. How the Ground Truth for the Training Set Was Established

The document does not provide any information on how the ground truth for the training set was established. It only describes the validation studies for the device.

Ask a specific question about this device

The Virtuoso system provides automated digital slide creation, management, analysis, and viewing. It is intended for in vitro diagnostic use as an aid to the pathologist in the display, detection, counting, review and classification of tissues and cells of clinical interest based on particular morphology, color, intensity, size, pattern and shape.

The Virtuoso™ System for IHC HER2 (4B5) is for digital read and image analysis applications. This particular Virtuoso system is intended for use as an aid to the pathologist in the detection and semi-quantitative measurement of HER2 protein in formalin-fixed, paraffin-embedded normal and neoplastic tissue. This device is an accessory to Ventana Medical Systems, Inc. PATHWAY® anti-HER2/neu (4B5) Rabbit Monoclonal Primary Antibody. The PATHWAY® anti-HER2/neu (4B5) Rabbit Monoclonal Primary Antibody is indicated for use as an aid in the assessment of breast cancer patients for whom HERCEPTIN® (Trastuzumab) treatment is being considered.

NOTE: The IHC HER2 4B5 Digital Read and Image Analysis applications are adjunctive computer-assisted methodologies for the qualified pathologist in the acquisition and measurement of images from microscope glass slides of breast cancer specimens stained for the presence of HER-2/neu receptor protein. The pathologist should verify agreement with the Image Analysis software application score. The accuracy of the test results depends on the quality of the immunohistochemical staining. It is the responsibility of a qualified pathologist to employ appropriate morphological studies and controls as specified in the instructions for the PATHWAY® anti-HER-2/neu (4B5) Rabbit Monoclonal Primary Antibody assay used to assure the validity of the iScan System for IHC HER2 4B5 Digital Read and Image Analysis scores. The actual correlation of PATHWAY® anti-HER-2/neu (4B5) to clinical outcome has not been established.

The Virtuoso™ System is an instrument-plus-software system designed to assist the qualified pathologist in the consistent assessment of protein expression in immunohistochemically stained histologic sections from formalin-fixed, paraffinembedded normal and neoplastic tissues. The system consists of a slide scanner (iScan), computer, monitor, keyboard, mouse, image analysis algorithms for specific immunohistochemical markers, and software with a Windows web browser-based user interface. Virtuoso is a web-based, end-to-end, digital pathology software solution that allows pathology laboratories to acquire, manage, view, analyze, share, and report digital images of pathology specimens. Using the Virtuoso software, the pathologist can view digital images, add annotations, make measurements, perform image analysis, and generate reports.

The iScan slide scanning device captures digital images of Hardware: formalin-fixed, paraffin-embedded tissues that are suitable for storage and viewing. The device includes a digital slide scanner, racks for loading glass slides, computer, scanner software, keyboard, mouse and monitor.

Software: The Virtuoso software is designed to complement the routine workflow of a qualified pathologist in the review of immunohistochemically stained histologic slides. It allows the user to select fields of view (FOVs) in the digital image for analysis and provides quantitative data on these FOVs to assist with interpretation. The software makes no independent interpretations of the data and requires competent human intervention for all steps in the analysis process.

1. Table of Acceptance Criteria and Reported Device Performance:

2. Sample Size and Data Provenance:

• Test Set Sample Size:
  • Agreement (Digital Read vs Manual): Site 1 (n = 119), Site 2 (n = 120), Site 3 (n = 118).
  • Agreement (Image Analysis vs Manual): Site 1 (n = 117), Site 2 (n = 120), Site 3 (n = 120).
  • Intra-Pathologist/Inter-Day Reproducibility: Exact number of cases not explicitly stated for individual sessions, but the confusion matrices show sums of around 40 cases for the comparisons (e.g., 40 cases for Session 1 digital read, 40 cases for Session 1 image analysis).
  • Inter-Pathologist Reproducibility: Exact number of cases not explicitly stated for individual comparisons, but the confusion matrices show sums of around 120 cases for the comparisons (e.g., 120 cases for Site 1 manual, 120 cases for Site 1 digital read, 117 cases for Site 1 image analysis).
  • Scanner Precision: A subset of 40 clinical cases from the primary study.
• Data Provenance: Not explicitly stated (e.g., country of origin). The studies appear to be prospective, as they involved pathologists reviewing cases using the Virtuoso system and comparing them to manual methods, and also involved multiple sessions for reproducibility.

3. Number of Experts and Qualifications for Ground Truth:

• Number of Experts: The study indicates that "each pathologist's Virtuoso digital read results were compared to their manual results," and also refers to "three pathologists." This suggests at least three pathologists were involved in generating the manual ground truth for the primary study.
• Qualifications of Experts: "Qualified pathologist" is mentioned. Specific years of experience or subspecialty training are not provided.

4. Adjudication Method for the Test Set:

• For the "Agreement" studies, the "Manual Method" by each pathologist served as the reference/ground truth. There is no mention of an adjudication process (e.g., 2+1, 3+1) to establish a consensus ground truth among multiple pathologists for the manual scores before comparison with the device. Each pathologist's manual read was compared to their own digital read/image analysis.
• For "Reproducibility" studies, agreement was assessed between different sessions or different pathologists, but no formal adjudication to establish a "true" score for these comparisons is described.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:

• A comparative effectiveness study involving human readers with and without AI assistance is not explicitly described. The study focuses on the agreement of the AI (Virtuoso digital read and image analysis) with the manual method performed by pathologists. It also assesses inter- and intra-pathologist reproducibility of the Virtuoso system, but not a direct comparison of human performance with vs. without AI assistance. Therefore, an effect size of how much human readers improve with AI vs. without AI assistance is not provided.

6. Standalone (Algorithm Only) Performance:

• Yes, the "Image Analysis" component of the Virtuoso system represents the standalone algorithm performance. The results for "Image Analysis vs Manual Method" (Agreements of 92%, 82%, 88%) and "Intra-Pathologist Image Analysis" (100% agreement for reproducibility) and "Inter-Pathologist Image Analysis" (93%-95% agreement) and "Scanner Precision" (in excess of 90%) directly reflect the algorithm's performance. The document states that the "image analysis application is the more sensitive of the two applications, and it generates an instrument-generated HER2 score that is not affected by memory bias as would be the case with human interpretations." This confirms it as an algorithm-only evaluation.

7. Type of Ground Truth Used:

• The ground truth used was expert consensus (manual pathological assessment). The manual interpretation of HER2 protein expression by qualified pathologists using traditional microscopy was considered the "true score" against which the Virtuoso system's performance was measured.

8. Sample Size for the Training Set:

• The document does not provide the sample size for the training set. It focuses solely on the clinical validation studies (test set).

9. How the Ground Truth for the Training Set Was Established:

• The document does not provide information on how the ground truth for any potential training set was established.

Ask a specific question about this device

The ScanScope System is an automated digital slide creation, management, viewing and analysis system. It is intended for in vitro diagnostic use as an aid to the pathologist in the display, detection, counting and classification of tissues and cells of clinical interest based on particular color, intensity, size, pattern and shape.

The IHC HER2 Breast Tissue Tunable Image Analysis application is intended for use as an aid to the pathologist in the detection and semi-quantitative measurement of HER2/neu (c-erbB-2) in formalin-fixed, paraffin-embedded normal and neoplastic tissue.

The IHC HER2 Breast Tissue Tunable Image Analysis application is intended for use as an accessory to the Dako HercepTest™ to aid in the detection and semi-quantitative measurement of HER2/neu (c-erbB-2) in formalin-fixed, paraffin-embedded normal and neoplastic tissue. It is indicated for use as an aid in the assessment of breast cancer patients for whom HERCEPTIN® (Trastuzumab) treatment is being considered. Note: The IHC HER2 Breast Tissue Tunable Image Analysis application is an adjunctive computer-assisted methodology to assist the reproducibility of a qualified pathologist in the acquisition and measurement of images from microscope slides of breast cancer specimens stained for the presence of HER2 receptor protein. The accuracy of the test result depends upon the quality of the immunohistochemical staining. It is the responsibility of a qualified pathologist to employ appropriate morphological studies and controls as specified in the instructions for the HER2 reagent/kit used to assure the validity of the IHC HER2 Breast Tissue Tunable Image Analysis application assisted HER2/neu score. The actual correlation of the HER2 reagents/kits to Herceptin® clinical outcome has not been established.

The ScanScope® XT System is an automated digital slide creation, management, viewing and analysis system. The system is comprised of a slide scanner instrument and a computer executing Spectrum™ software. The system capabilities include digitizing microscope slides at diagnostic resolution, storing and managing the resulting digital slide images, retrieving and displaying digital slides, including support for remote access over wide-area networks, providing facilities for annotating digital slides and entering and editing metadata associated with digital slides, and facilities for image analysis of digital slides, including the ability to quantify characteristics useful to Pathologists, such as measuring and scoring immunohistochemical stains applied to histology specimens, such as Dako HerceptTestTM which reveals the presence of proteins such as Human Epidermal growth factor Receptor 2 (HER2), which may be used to determine patient treatment for breast cancer.

Here's an analysis of the acceptance criteria and study detailed in the provided K080564 submission for the Aperio ScanScope® XT System:

1. Table of Acceptance Criteria and Reported Device Performance

The submission focuses on demonstrating substantial equivalence rather than predefined acceptance criteria in the traditional sense of a specific performance target for accuracy or sensitivity. Instead, the "acceptance criteria" are implicitly met by demonstrating comparable performance to manual microscopy and superior inter-pathologist agreement. The primary performance metric presented is Percent Agreement (PA).

2. Sample Size and Data Provenance

• Test Set Sample Size: 180 formalin-fixed, paraffin-embedded breast tissue specimens.
  • Clinical Site 1: 80 specimens.
  • Clinical Site 2: 100 specimens.
• Data Provenance: Retrospective, with specimens from two unnamed clinical sites. The country of origin is not explicitly stated but is implicitly within the scope of FDA approval, suggesting US-based clinical sites.

3. Number of Experts and Qualifications for Ground Truth for the Test Set

• Number of Experts: Three (3) board-certified pathologists at each clinical site (a total of 6 unique pathologists across both sites, although for each site, it's 3 pathologists).
• Qualifications of Experts: "Board-certified pathologists." No further details on years of experience are provided.

4. Adjudication Method for the Test Set

The primary method for establishing the reference HER2 scores for the image analysis comparison was the manual microscopic review by three pathologists. The algorithm's score was then compared against these individual pathologist scores and implicitly against the consensus of pathologists (e.g., the "average HER2 scores from the three pathologists" was used to stratify slides for the algorithm training set).

For the inter-pathologist agreement, each pathologist's manual score was compared against the others (Pathologist 1 vs 2, 1 vs 3, 2 vs 3). Similarly, for the Image Analysis inter-pathologist agreement, the image analysis scores derived from each pathologist's outlined tumor regions were compared.

The initial manual microscopy average HER2 scores from the three pathologists were used to define the HER2 score distribution for the study.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Status and Effect Size

No explicit MRMC comparative effectiveness study, as typically understood for human readers improving with AI vs. without AI assistance, was performed. The study evaluates the agreement between manual microscopy and the image analysis system, and the inter-pathologist agreement for both manual and image analysis methods separately. It doesn't directly measure the improvement in human reader performance when using AI assistance in a diagnostic workflow.

However, the study does compare inter-pathologist agreement between manual microscopy and image analysis.

• Manual Microscopy Inter-Pathologist Agreement: Ranged from 65.0% to 91.3%
• Image Analysis Inter-Pathologist Agreement: Ranged from 85.0% to 94.0%

This suggests that the image analysis system itself results in higher inter-pathologist agreement compared to manual microscopy performed by independent pathologists. The submission also notes, "This study shows a good example how image analysis can help Pathologists with the standardization of the scoring." and "The variations introduced by a single pathologist by outlining different tumor regions from one read to another is 3x to 3.7x smaller than the variations introduced by different pathologists outlining different tumor regions" which supports the idea that the system could improve consistency.

6. Standalone (Algorithm Only) Performance

Yes, a standalone performance was done for the reported device. The Image Analysis (algorithm) was run in "batch processing mode completely separated from the pathologists outlining the tumor regions to avoid influencing the pathologists in their choice of tumor regions." The agreement percentages for "Manual Microscopy vs Image Analysis - same Pathologist - Agreements" and "Image Analysis - Inter-Pathologists - Agreements" inherently describe the standalone performance relative to human input.

7. Type of Ground Truth Used

Expert Consensus (modified): The ground truth was based on the independent scoring of three board-certified pathologists for each slide, using manual microscopy. For the purpose of stratifying the training set, the "average HER2 score provided by three pathologists using manual microscopy" was used. For comparison studies, the algorithm's output was compared pathologist-by-pathologist to their respective manual reads and to the image analysis scores derived from their own outlined regions.

8. Sample Size for the Training Set

• Algorithm Training Set (for the comparison study): 20 HER2 slides (5 slides for each 0, 1+, 2+, and 3+ HER2 class), randomly selected from the available slides.
• Algorithm Training Set (for the separate analytical performance "Algorithm Training Set" section): 20 slides (again, 5 slides from each 0, 1+, 2+, and 3+ HER2 class, chosen via stratified-random selection) from a set of 100 HER2 slides. The remaining 80 slides formed the evaluation dataset for this separate analysis.

9. How Ground Truth for the Training Set Was Established

The ground truth for the training set was established based on the "average HER2 score from the three pathologists" using manual microscopy. These average scores were used to stratify the slides into 0, 1+, 2+, and 3+ classes from which the training slides were then selected. The algorithm was "tuned" using these selected training slides and the procedure outlined later in the submission (though the specific tuning procedure isn't fully detailed in the provided text).

Ask a specific question about this device

PATHIAM-Assisted Scoring: Intended for clinical laboratory use as an accessory to the DAKO HercepTest to aid in the detection and semi-quantitative measurement of Her2/neu in formalin fixed, paraffin-embedded normal and neoplastic tissue. When used with the DAKO HercepTest, Pathiam Assisted Scoring is indicated for use as an aid in the assessment of breast cancer patients for whom HERCEPTIN (Trastuzumab) treatment is being considered. The pathologist should verify agreement with the PATHIAM System score.

HER2/neu results are indicated for use as an aid in the management, prognosis and predication of therapy outcomes of breast cancer. Note: The actual correlation of the DAKO HercepTest to Herceptin® clinical outcome has not been established.

The PATHIAMIM System is an instrument (iScan) and image analysis software system designed to assist the qualified pathologist in the consistent quantitative assessment of marker expression in immunohistochemically stained histological sections digital images. The sample tissue is breast tissue prepared using the DAKO HercepTest Reagent Kit. The PATHIAM system consists of the BioImagene iScan slide scanner, computer with the PATHIAM Software, monitor, keyboard and mouse.

The PATHIAM System digitizes formalin-fixed, paraffin embedded normal and neoplastic tissue and provides semi-quantitative analysis of extent and intensity of stained tissue, providing the pathologist with an aid to interpretation of the level of expression of HER2/neu in breast cancer tissue. The pathologist is presented with a digital image of the tissue section and a suggested staining score (0 to 3). The pathologist then makes an assessment of the digital image and reports his/her score. Alternately, the pathologist can simply use the digitized image to perform his interpretation of the level of expression, without employing the software.

Hardware: The iScan slide scanning device captures digital images of formalin-fixed, paraffin-embedded tissues that are suitable for storage, viewing and visual analysis. The device includes a digital slide scanner, racks for loading glass slides, an Intel based PC, dual core, dual Xeon processor, PATHIAM Software, and a monitor.

Software: The PATHIAM Software requires competent human intervention at all steps in the analysis process. The system is designed to complement the routine workflow of a qualified pathologist screening the immunohistochemically stained histological slides with additional quantitative data to assist the reproducibility of the slide interpretation. It allows the user to select the area of interest on the breast tissue image. The user marks the area of interest for the analysis. The system software makes no independent interpretations of the data. The software will act as a tool for the user, to improve consistency and data recording. The image produced digitally may also be used independent of the software, by allowing the pathologist to count from the digital image, rather than from the microscope.

The provided document outlines the performance study for the PATHIAM™ System with iScan for HER2/neu, primarily focusing on its agreement with manual HercepTest methods and reproducibility.

Here's an analysis of the acceptance criteria and study details:

1. Table of Acceptance Criteria and Reported Device Performance:

2. Sample size used for the test set and the data provenance:

• Sample Size: 176 stained tissue specimens.
• Data Provenance: The document does not explicitly state the country of origin. The study appears to be prospective in the sense that the pathologists performed readings specifically for this study, first manually and then with the PATHIAM System. There is no indication of retrospective analysis of existing clinical data.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

• Number of Experts: Three pathologists.
• Qualifications of Experts: The document states "trained pathologists" without specifying years of experience or sub-specialization.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set:

• Adjudication Method: Not explicitly stated. For the "Comparison with Manual HercepTest method," pathologists observed digital images and the suggested score, then selected an appropriate tissue score. This suggests the pathologist made the final judgment for the PATHIAM System assisted score. For the manual scores, they read the slides manually. There's no mention of a consensus process or arbitration for discrepant scores.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

• MRMC Study: Yes, a form of MRMC study was conducted. Three pathologists (multiple readers) assessed 176 cases (multiple cases) both with and without the AI assistance (manual vs. PATHIAM-assisted).
• Effect Size of Improvement: The study demonstrates improved inter-reader reproducibility with AI assistance compared to manual reading.
  • Manual Inter-reader Agreement: Ranged from 67% to 81%.
  • PATHIAM-assisted Inter-reader Agreement: Ranged from 89% to 92%.
  • This represents an improvement in inter-reader agreement of 8% to 22% when using the PATHIAM system compared to manual reading alone (e.g., 92% vs 67% for Site 1 vs 2 comparison, an improvement of 25 percentage points; 89% vs 81% for Site 2 vs 3, an improvement of 8 percentage points; 91% vs 80% for Site 3 vs 1, an improvement of 11 percentage points). The "effect size" here is the increase in percentage agreement between pathologists.

6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done:

• Standalone Performance: No, a standalone performance study of the algorithm without human-in-the-loop was not performed. The device description explicitly states: "The PATHIAM Software requires competent human Software: intervention at all steps in the analysis process. The system is designed to complement the routine workflow of a qualified pathologist..." and "The software will act as a tool for the user, to improve consistency and data recording." The "PATHIAM System values" are derived from the pathologist's review of the digital images and the software's suggested score.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc):

• Ground Truth: The "ground truth" for evaluating the PATHIAM System's performance was the manual assessment by trained pathologists using the DAKO HercepTest package insert. This is implicitly assumed to be the "gold standard" against which the system-assisted readings are compared. The reproducibility study between pathologists also uses their independent assessments as the reference.

8. The sample size for the training set:

• Training Set Sample Size: The document does not provide information on the training set size or how the algorithm was developed. The studies described are performance validation studies.

9. How the ground truth for the training set was established:

• Training Set Ground Truth: As with the training set size, information on how the ground truth for any potential training set was established is not provided in this summary.

Ask a specific question about this device

The ScanScope System is an automated digital slide creation, management, viewing and analysis system. It is intended for in vitro diagnostic use as an aid to the pathologist in the display, detection, counting and classification of tissues and cells of clinical interest based on particular color, intensity, size, pattern and shape.

The IHC HER2 Image Analysis application is intended for use as an aid to the pathologist in the detection and semi-quantitative measurement of HER2/neu (c-erbB-2) in formalin-fixed, paraffin-embedded normal and neoplastic tissue.

The IHC HER2 Image Analysis application is intended for use as an accessory to the Dako HercepTest™ to aid in the detection and semi-quantitative measurement of HER2/neu (c-erbB-2) in formalin-fixed, paraffin-embedded normal and neoplastic tissue. When used with the Dako HercepTest™, it is indicated for use as an aid in the assessment of breast cancer patients for whom HERCEPTIN® (Trastuzumab) treatment is being considered. Note: The IHC HER2 Image Analysis application is an adjunctive computer-assisted methodology to assist the reproducibility of a qualified pathologist in the acquisition and measurement of images from microscope slides of breast cancer specimens stained for the presence of HER-2 receptor protein. The accuracy of the test result depends upon the quality of the immunohistochemical staining. It is the responsibility of a qualified pathologist to employ appropriate morphological studies and controls as specified in the instructions for the Dako HercepTest™ to assure the validity of the IHC HER2 Image Analysis application assisted HER-2/neu score. The actual correlation of the Dako HercepTest™ to Herceptin® clinical outcome has not been established.

The ScanScope® XT System is an automated digital slide creation, management, viewing and analysis system. The ScanScope® XT System components consist of an automated digital microscope slide scanner, computer, color monitor, keyboard and digital pathology information management software. The system capabilities include digitizing microscope slides at high resolution, storing and managing the resulting digital slide images, retrieving and displaying digital slides, including support for remote access over wide-area networks, providing facilities for annotating digital slides and entering and editing metadata associated with digital slides, and facilities for image analysis of digital slides. Image analysis capabilities include the ability to quantify characteristics useful to Pathologists, such as measuring and scoring immunohistochemical stains applied to histology specimens, such as the Dako HerceptTest"M, which reveal the presence of proteins such as Human Epidermal growth factor Receptor 2 (HER2), which may be used to determine patient treatment for breast cancer.

Here's a breakdown of the acceptance criteria and the study details for the Aperio Technologies ScanScope® XT System, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

The document does not explicitly state pre-defined acceptance criteria in terms of numerical thresholds for comparison between the manual microscopy and the image analysis system. Instead, it aims to demonstrate substantial equivalence by showing that the "agreements between the pathologists' manual microscopy and performed (blinded) image analysis were comparable to the inter-pathologists agreements for manual microscopy." The study design itself serves as the framework for proving this comparability.

Clinical Site 2:
Manual Microscopy Inter-Pathologist Agreements (PA): 84.0% - 90.0%
Image Analysis Inter-Pathologist Agreements (PA): 87.0% - 92.0%
Manual Microscopy vs. Image Analysis (Same Pathologist) Agreements (PA): 79.0% - 90.0%

Conclusion: Inter-pathologist agreements for image analysis (86.3-93.8%) were comparable to manual microscopy (76.3-91.3%). Agreements between manual microscopy and image analysis (77.5-92.5%) were also comparable to inter-pathologist agreements for manual microscopy (76.3-91.3%). |
| Precision (intra-run, inter-run, inter-system) | Intra-run/Intra-system: 100% perfect agreement for calculated HER2 scores across all runs.
Inter-run/Intra-system: 100% perfect agreement for calculated HER2 scores across all runs.
Inter-systems: 100% perfect agreement for calculated HER2 scores across all systems and across all runs. |

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size: 180 formalin-fixed, paraffin-embedded breast tissue specimens.
  • Site 1: 80 specimens (with approximately equal HER2 score distribution)
  • Site 2: 100 routine specimens
• Data Provenance: Retrospective, as the specimens were already stained and presumably collected prior to the study. The study was conducted at two clinical sites, implying a multi-center study within the US (though country of origin is not explicitly stated, "clinical sites" typically refers to healthcare facilities within the country where the submission is filed – in this case, the US FDA).

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts

• Number of Experts: Three pathologists at each of the two clinical sites, totaling 6 pathologists involved in establishing ground truth.
• Qualifications of Experts: The document refers to them as "pathologists," implying they are qualified medical professionals specializing in pathology. No specific years of experience or sub-specialty certification are provided.

4. Adjudication Method for the Test Set

The document describes a comparative study where three pathologists at each site independently performed a blinded read of the glass slides for manual microscopy. For the image analysis, the same three pathologists remotely viewed and outlined tumor regions. The algorithm then reported the HER2 score for each pathologist's outlined regions.

There is no explicit adjudication method (like 2+1 or 3+1 consensus) described for establishing a single "ground truth" for each slide based on expert opinion before comparison. Instead, the study compares inter-pathologist agreement for manual reads, inter-pathologist agreement for image analysis results, and agreement between individual pathologist's manual reads and their corresponding image analysis results. The image analysis algorithm's output serves as a separate measure to be compared against each pathologist's manual assessment.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs. Without AI Assistance

• MRMC Comparative Effectiveness Study: Yes, an MRMC-like study was conducted. It involved multiple readers (pathologists) and multiple cases (180 breast tissue specimens). The comparison was between manual microscopy and image analysis, with pathologists themselves interacting with the image analysis system by outlining regions.
• Effect Size of Improvement with AI Assistance: The document does not quantify the improvement of human readers with AI assistance in terms of an effect size. It focuses on the "comparability" of agreements:
  • Inter-pathologist agreements for the blinded image analysis (PA: 86.3-93.8%) were comparable to inter-pathologist agreements for manual microscopy (PA: 76.3-91.3%).
  • Agreements between the pathologists' manual microscopy and performed (blinded) image analysis (PA: 77.5-92.5%) were comparable to inter-pathologist agreements for manual microscopy (PA: 76.3-91.3%).

This indicates the system performed similarly to human agreement without necessarily making a claim of "improvement" in diagnostic accuracy or efficiency for the human reader while using the AI. The study's goal was to demonstrate substantial equivalence, not superior performance or augmentation.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done

Yes, a standalone component of the algorithm's performance was evaluated. The pathologists outlined representative tumor regions, and then the algorithm was run in "batch mode, blinded from the pathologists" and used "out of the box" to report the HER2 score for those outlined regions. This means the algorithm's output for a defined region was generated independently of further human intervention in the scoring process for that specific region.

7. The Type of Ground Truth Used

The ground truth used for comparison was expert consensus (implied via agreement metrics) or expert opinion for each pathologist's manual read. There wasn't a single, definitive "gold standard" ground truth like pathology or outcomes data established for each slide beforehand. Instead, the study evaluates agreement between different forms of assessment (manual vs. AI-assisted) and among experts. The Dako HercepTest™ staining is mentioned as the method used for preparing the specimens, which is a standardized immunohistochemical stain, but the interpretation of this stain (the HER2 score) is what is being compared.

8. The Sample Size for the Training Set

The document does not provide any information regarding the sample size used for the training set of the IHC HER2 Image Analysis application. It only describes the test set used for validating the device.

9. How the Ground Truth for the Training Set Was Established

The document does not provide any information on how the ground truth for the training set was established. This information is typically proprietary to the developer and not always disclosed in 510(k) summaries, which focus on the validation study.

Ask a specific question about this device

Biolmagene PATHIAM is intended for use as an accessory to the Dako HercepTest® to aid a pathologist in semi-quantitative measurement of HER2/neu (c-erbB-2) in Formalin-fixed, paraffin-embedded breast cancer tissue. When used with the Dako HercepTest it is indicated as an aid in the assessment of breast cancer patients for whom Herceptin® (Trastuzumab) treatment is being considered.

The imaging software is intended to detect and classify cells of clinical interest by analyzing digitized images of microscope slides based on recognition of cellular objects of particular color, size and shape. The software can be used with a computer and image digitizer with features specified in the labeling.

PATHIAM software is a standalone software application that will work on a system with the following features required but not provided:

Computer

• Processor: 2.4 GHz, Pentium IV equivalent
• Memory: 512 MB RAM
• Operating System: Windows 2000 or later
• Hard Drive: minimum 100MB for software installation, 20GB for image storage
• LAN connectivity, minimum 100 MBPS (recommended), support for USB interface, support for HTTP, TCP/IP protocols (using the Operating system)
• High Speed Graphic Accelerator Card (1024 X 768)
• 17" High resolution display monitor
• 24 bit color depth
• Font Setting: Small font (DPI setting: 96 DPI)

Digitizing Equipment: Camera

• Resolution: at least 2048 x 1536 pixels
• Frame rate: 20 fps@1200 x 768 resolution (6 fps @ 2048 x 1536 resolution)
• Sampling Frequency of 6.26 square/um
• Compression format: JPEG 2000, BMP, TIFF, JPEG
• Color: 24-bit (R. G. B)
• Connection to computer

Digitizing Equipment: Digital Side Scanner

• Input Format: 25X75mm microscope slides
• Resolution: 54,000 pixel/inch with 20X objective
• Method: Line-scanning
• File Format: TIFF/JPEG2000; compliant with TIFF 6.0 standard.
• Color: 24-bit (R.G. B)
• Connectivity: 100/1000 MBPS Ethernet
• Compression format: JPEG 2000, BMP, TIFF, JPEG

The software allows both archiving of the digital image, and semi quantitative analysis of extent and intensity of stained tissue, providing the pathologist with an aid to interpretation of level of expression of Her2/neu in breast cancer tissue. The pathologist is presented with a digital image of the tissue section and a suggested staining score (0 to 3). The pathologist then makes an assessment of the digital image and reports his/her score.

Here's a breakdown of the acceptance criteria and study details for the BioImagene PATHIAM Image Analysis Software for Her2/neu, based on the provided 510(k) summary:

Acceptance Criteria and Reported Device Performance

The acceptance criteria for the PATHIAM software are implicitly defined by comparison to the established predicate device (ACIS Her2/neu software component) and through performance studies demonstrating inter-laboratory and inter-reader agreement. While explicit numerical acceptance criteria for overall agreement are not stated as "target thresholds," the demonstrated high levels of agreement strongly support the device's performance.

Study Details

1. Sample sizes used for the test set and the data provenance:

   • Test Set Sample Size: 176 stained breast cancer tissue specimens.
   • Data Provenance: The study was conducted in the US at three different sites. There is no explicit mention of the data being retrospective or prospective, but the description of "analyzed images of the same set of 176 stained breast cancer tissue specimens" suggests a retrospective analysis of existing samples.

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • Number of Experts: Three different pathologists.
   • Qualifications: They are referred to as "trained pathologists," but no specific experience levels (e.g., years of experience, board certifications) are provided.

3. Adjudication method (e.g. 2+1, 3+1, none) for the test set:

   • The study involved multiple pathologists providing scores, and comparisons were made between their scores. However, there is no explicit mention of an adjudication method to establish a single, definitive ground truth score for each case through consensus or a tie-breaker. Instead, the study focuses on agreement between the individual scores of the device, manual readers, and device-assisted readers across different labs.

4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • Yes, implicitly, an MRMC comparative effectiveness study was performed in the sense that multiple readers (pathologists) evaluated multiple cases with and without the PATHIAM assistance, and across different laboratory setups.
   • Effect Size of Improvement:
     • Manual Scores (Inter-lab agreement): 76% - 97% overall agreement (Table 3).
     • Pathologist Assisted by PATHIAM Scores (Inter-lab agreement): 81% - 96% overall agreement (Table 2).
     • The summary states: "Consistency is improved when the PATHIAM score assists the pathologist in their interpretation (Table 2)."
     • While not quantified as a single "effect size" number, comparing Table 3 (Manual Scores) to Table 2 (PATHIAM-Assisted Scores) suggests an improvement in agreement. For instance, Lab 1 vs Lab 3 manual agreement was 76%, while with PATHIAM assistance, it was 81%. Lab 2 vs Lab 3 manual agreement was 78%, and with PATHIAM assistance, it was also 81%. This indicates a trend towards improved inter-reader consistency with AI assistance.

5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

   • Yes, a standalone performance assessment was conducted. This is evidenced by the "Between-Lab Agreement for Raw PATHIAM- Scores" (Table 1), which shows the consistency of the algorithm's output across different laboratories without direct pathologist modification of the software's initial score. The study also compared "PATHIAM raw scores and Manual Scores" (Table 5), which is a comparison of the algorithm's output against human interpretation.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

   • The primary "ground truth" or reference for comparison in this study is the pathologist's interpretation, both manual (microscopic assessment) and PATHIAM-assisted. The study focuses on agreement between these interpretations rather than against an external, independent gold standard like pathology results or clinical outcomes. The device is an aid to a pathologist, implying its performance is assessed by how well it aligns with or improves human expert interpretation.

7. The sample size for the training set:

   • The document does not provide information on the sample size used for the training set. The performance data presented relates exclusively to the test set used for validation.

8. How the ground truth for the training set was established:

   • Since the training set size and details are not provided, it is also not stated how the ground truth for any potential training set was established.

Ask a specific question about this device

The Ventana Image Analysis System (VIAS) is an adjunctive computer-assisted image analysis system functionally connected to an interactive microscope. It is intended for use as an aid to the pathologist in the detection and counting of cells of interest based on marker intensity. size and shape using appropriate controls to assure the validity of the VIAS scores.

In this application, the VIAS is intended to aid a qualified pathologist for the semi-quantitative detection of c-erbB-2 (HER-2/neu) in formalin-fixed, paraffin embedded normal and neoplastic tissue specimens immunohistochemically stained for the presence of HER-2/neu proteins using Ventana's HER-2/neu reagents as well as Ventana's DAB copper chromogen and nuclear hematoxylin.

This particular application is an accessory to the Ventana PATHWAY™ Her2 (clone CB11) (Ventana Medical Systems, Inc., Tucson, Arizona) and the Ventana PATHWAY™ Her2 is indicated as an aid in the assessment of breast cancer patients for whom Herceptin® treatment is considered.

The VIAS is an adjunctive computer-assisted methodology to assist the reproducibility of a qualified pathologist in the acquisition and measurement of images from microscope slides of breast cancer specimens stained for the presence of HER2 receptor protein. The accuracy of the test result depends upon the quality of the immunohistochemical staining. It is the responsibility of a qualified pathologist to employ appropriate morphological studies and controls as specified in the instructions for the Ventana PATHWAY™ Her2 to assure the validity of the VIAS-assisted HER2 score.

The Ventana Image Analysis System (VIAS) is an adjunctive computer-assisted image analysis system functionally connected to an interactive microscope.

The provided document is a 510(k) premarket notification letter from the FDA for the Ventana Image Analysis System – Her2/neu. It mentions the device's indications for use but does not contain any information regarding acceptance criteria, study details, or performance data.

Therefore, I cannot fulfill your request for:

• A table of acceptance criteria and reported device performance.
• Sample size used for the test set or data provenance.
• Number of experts used to establish ground truth or their qualifications.
• Adjudication method.
• Information on a multi-reader multi-case (MRMC) comparative effectiveness study or its effect size.
• Information on a standalone performance study.
• Type of ground truth used.
• Sample size for the training set.
• How the ground truth for the training set was established.

The document only states that the device is "substantially equivalent" to legally marketed predicate devices. To obtain the information you're looking for, you would typically need to refer to the full 510(k) submission summary or a separate clinical study report, which is not included in this text.

Ask a specific question about this device