Search Results

The Virtuoso system provides automated digital slide creation, management, analysis, and viewing. It is intended for in vitro diagnostic use as an aid to the pathologist in the display, detection, counting, review and classification of tissues and cells of clinical interest based on particular morphology, color, intensity, size, pattern and shape.

The Virtuoso™ System for p53 (DO-7) is for digital read and image analysis applications. This particular Virtuoso system is intended for use as an aid to the pathologist in the detection and semi-quantitative measurement of p53 (DO-7) protein in formalin-fixed, paraffin-embedded normal and neoplastic tissue. This device is an accessory to the Ventana Medical Systems, Inc. CONFIRM™ anti-p53 (DO-7) Mouse Monoclonal Primary Antibody assay. The Ventana Medical Systems, Inc. CONFIRM™ anti-p53 assay is indicated for the assessment of p53 protein where mutations have been linked to tumor proliferation. When used with this assay, the Virtuoso™ System for p53 (DO-7) is indicated for use as an aid in the assessment of p53 status in breast cancer patients (but is not the sole basis for treatment).

Note: The IHC p53 (D0-7) Digital Read and Image Analysis applications are adjunctive computer-assisted methodologies for the qualified pathologist in the acquisition and measurement of images from microscope glass slides of breast cancer specimens stained for the presence of p53 protein. The pathologist should verify agreement with the Image Analysis software application score. The accuracy of the test results depends on the quality of the immunohistochemical staining. It is the responsibility of a qualified pathologist to employ appropriate morphological studies and controls as specified in the instructions for the CONFIRMTM anti-p53 (DO-7) Mouse Monoclonal Primary Antibody assay used to assure the validity of the Virtuoso System for IHC p53 Digital Read and Image Analysis scores. The actual correlation of CONFIRM™ anti-p53 (DO-7) Mouse Monoclonal Primary Antibody to clinical outcome has not been established.

The Virtuoso™ System is an instrument-plus-software system designed to assist the qualified pathologist in the consistent assessment of protein expression in immunohistochemically stained histologic sections from formalin-fixed, paraffinembedded normal and neoplastic tissues. The system consists of a slide scanner (iScan), computer, monitor, keyboard, mouse, image analysis algorithms for specific immunohistochemical markers, and software with a Windows web browser-based user interface. Virtuoso is a web-based, end-to-end, digital pathology software solution that allows pathology laboratories to acquire, manage, view, analyze, share, and report digital images of pathology specimens. Using the Virtuoso software, the pathologist can view digital images, add annotations, make measurements, perform image analysis, and generate reports.

Hardware: The iScan slide scanning device captures digital images of formalin-fixed, paraffin-embedded tissues that are suitable for storage and viewing. The device includes a digital slide scanner, racks for loading glass slides, computer, scanner software, keyboard, mouse and monitor.

Software: The Virtuoso software is designed to complement the routine workflow of a qualified pathologist in the review of immunohistochemically stained histologic slides. It allows the user to select fields of view (FOVs) in the digital image for analysis and provides quantitative data on these FOVs to assist with interpretation. The software makes no independent interpretations of the data and requires competent human intervention for all steps in the analysis process.

Here's an analysis of the acceptance criteria and study details for the Virtuoso™ System for IHC p53 (DO-7), based on the provided document:

Acceptance Criteria and Reported Device Performance

The document does not explicitly state pre-defined acceptance criteria with specific thresholds for agreement percentages. Instead, it presents the results of several agreement and reproducibility studies. For the purpose of this summary, the reported performance values themselves are considered against an implicit expectation of high agreement.

Study Details

The evaluation involved two studies: a primary study for overall system performance and a secondary study for scanner precision.

1. Sample size used for the test set and the data provenance:

   • Agreement Studies (Virtuoso vs Manual): The sample sizes for the primary agreement study were 119 cases for Site 1, 119 cases for Site 2, 117 cases for Site 3, and 114 cases for Site 4 for the Digital Read analysis. For Image Analysis, Site 4 had 105 cases.
   • Reproducibility Studies: The sample sizes for reproducibility studies are not explicitly stated as total number of cases. However, the confusion matrices for intra-pathologist reproducibility (pages 5 & 6) suggest working with sample sizes in the range of 30-40 cases across sessions (e.g., for Digital Read: 27 Neg, 13 Pos in Session 1, implying around 40 cases). For inter-pathologist reproducibility (pages 6 & 7), the sample sizes range from 119 to ~120 (e.g., Site 1 vs Site 2 had counts summing to 119).
   • Scanner Precision Study: A subset of 40 clinical cases from the primary study was used.
   • Data Provenance: Not explicitly stated, but the mention of "four sites" suggests data was collected from multiple clinical institutions. It is a clinical validation study and, given the nature of regulatory submissions, is almost certainly retrospective analysis of previously collected and stained slides. No country of origin is specified, but the submission is to the US FDA, implying US (and potentially international) clinical data.

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • Number of Experts: For the primary study, at least three pathologists were involved for inter-pathologist reproducibility, and each of them performed manual readings for comparison. For the agreement with the manual method, "Each pathologist's Virtuoso digital read results were compared to their manual results." This implies that the manual reading by the same pathologist served as a direct comparison for their digital (DR or IA) reading.
   • Qualifications of Experts: The document states the device is "designed to assist the qualified pathologist" and requires "competent human intervention." While specific years of experience are not mentioned, it is implicit that these were qualified pathologists using traditional diagnostic methods.

3. Adjudication method (e.g., 2+1, 3+1, none) for the test set:

   • The primary study compared the Virtuoso system's readings (Digital Read and Image Analysis) against the manual method findings of the same pathologist. This means the pathologist's own manual interpretation served as the reference for their digital assessment, rather than an independent adjudicated ground truth from multiple experts.
   • The inter-pathologist agreement section compares readings among three pathologists but does not describe an explicit adjudication process (like 2+1 majority vote) to establish a single "ground truth" across them for the reproducibility analysis itself. Instead, it measures the agreement between them.

4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • This document does not describe a MRMC comparative effectiveness study that measures the improvement of human readers with AI assistance vs. without AI assistance.
   • Instead, it evaluates the agreement of the AI system's outputs (Digital Read and Image Analysis) with human manual readings, and the reproducibility of human readings, both with and without the digital system (manual vs. digital read, IA vs. manual). It's more of a validation of the digital system as an equivalent tool, rather than an AI-assisted improvement study.

5. If a standalone (i.e. algorithm only, without human-in-the-loop performance) was done:

   • The "Image Analysis" portion of the Virtuoso system represents the algorithm's standalone performance, albeit within a workflow where a pathologist selects the fields of view. The software "produces a quantitative score for the FOV and an aggregate score over all the FOVs for the whole slide," and "the pathologist has the choice of accepting the result or overriding with his/her own score."
   • The agreement of "Virtuoso Image Analysis vs Manual Method" (page 5) effectively serves as an evaluation of the algorithm's standalone performance (after FOV selection), compared to a human's manual assessment.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

   • The ground truth for comparison was established by the manual interpretation of qualified pathologists using traditional microscopy. This is termed the "reference manual method."
   • The p53 classifications were "less than or equal to 10% to describe negative, and greater than 10% to describe positive." This threshold-based binary classification was applied by the pathologists.

7. The sample size for the training set:

   • The document does not provide any information about the sample size used for the training set of the Virtuoso system's image analysis algorithms.

8. How the ground truth for the training set was established:

   • The document does not provide any information on how the ground truth for the training set was established.

Ask a specific question about this device