Search Results
Found 2 results
510(k) Data Aggregation
(519 days)
VIRTUOSO SYSTEM FOR IHC KI-67 (30-9)
The Virtuoso system provides automated digital slide creation, management, analysis, and viewing. It is intended for in vitro diagnostic use as an aid to the pathologist in the display, detection, counting, review and classification of tissues and cells of clinical interest based on particular morphology, color, intensity, size, pattern and shape.
The IHC Ki-67 (30-9) Digital Read and Image Analysis applications are intended for use as an aid to the pathologist in the detection and semi-quantitative measurement of Ki-67 protein in formalin-fixed, paraffin-embedded normal and neoplastic tissue. Ki-67 results are indicated for use in assessing the proliferative activity of normal and neoplastic breast tissue. When used with Ventana Medical Systems, Inc. CONFIRM™ anti-Ki-67 (30-9) Rabbit Monoclonal Primary Antibody Assay, it is indicated for use as an aid in the assessment of Ki-67 status in breast cancer patients (but is not the sole basis for treatment).
The Virtuoso™ System is an instrument-plus-software system designed to assist the qualified pathologist in the consistent assessment of protein expression in immunohistochemically stained histologic sections from formalin-fixed, paraffinembedded normal and neoplastic tissues. The system consists of a slide scanner (iScan), computer, monitor, keyboard, mouse, image analysis algorithms for specific immunohistochemical markers, and software with a Windows web browser-based user interface. Virtuoso is a web-based, end-to-end, digital pathology software solution that allows pathology laboratories to acquire, manage, view, analyze, share, and report digital images of pathology specimens. Using the Virtuoso software, the pathologist can view digital images, add annotations, make measurements, perform image analysis, and generate reports.
Hardware: The iScan slide scanning device captures digital images of formalin-fixed. paraffin-embedded tissues that are suitable for storage and viewing. The device includes a digital slide scanner, racks for loading glass slides, computer, scanner software, keyboard, mouse and monitor.
Software: The Virtuoso software is designed to complement the routine workflow of a qualified pathologist in the review of immunohistochemically stained histologic slides. It allows the user to select fields of view (FOVs) in the digital image for analysis and provides quantitative data on these FOVs to assist with interpretation. The software makes no independent interpretations of the data and requires competent human intervention for all steps in the analysis process.
The Ventana Virtuoso™ System for IHC Ki-67 (30-9) was tested for its performance in both digital reading (DR) and image analysis (IA) of Ki-67 stained slides. The overall acceptance criterion for each measurement was 75% overall percent agreement (OPA) with manual microscopic reads.
1. Table of Acceptance Criteria and Reported Device Performance
| Performance Metric | Acceptance Criteria (Overall Percent Agreement) | Reported Device Performance (Overall Percent Agreement) |
|---|---|---|
| Digital Read (DR) vs. Manual Read | ≥ 75% | 90.0% (95% CI: 83.3-94.2) |
| Image Analysis (IA) vs. Manual Read (Overall) | ≥ 75% | 78.0% (95% CI: 69.7-84.5) |
| Image Analysis (IA) vs. Manual Read (Site 1) | ≥ 75% | 86.6% (95% CI: 79.1-91.7) |
| Image Analysis (IA) vs. Manual Read (Site 2) | ≥ 75% | 90.8% (95% CI: 84.2-94.8) |
2. Sample Size and Data Provenance for the Test Set
- Digital Read (DR) Test Set: 120 cases.
- Image Analysis (IA) Test Set: 120 cases.
- Data Provenance: The document does not explicitly state the country of origin. The studies appear to be prospective concordance studies where new data was generated for validation.
3. Number of Experts and Qualifications for Ground Truth Establishment (Test Set)
- Digital Read (DR): "one pathologist at one site" established the manual score (reference result). No specific qualifications are provided beyond "pathologist".
- Image Analysis (IA): "three pathologists at three sites" established the manual score (reference result). No specific qualifications are provided beyond "pathologists".
4. Adjudication Method for the Test Set
The document describes concordance studies where manual scores were compared to device scores. It does not mention any formal adjudication method beyond establishing the "manual score (reference result)" by one or more pathologists. This implies a single expert's opinion served as the ground truth for each case initially, rather than multiple experts with a specific adjudication process.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No formal MRMC comparative effectiveness study is described where human readers' performance with and without AI assistance is directly compared to quantify an effect size. The studies focus on the agreement between the device alone (digital read or image analysis) and manual reads.
6. Standalone (Algorithm Only) Performance
Yes, standalone performance was done for both Digital Read and Image Analysis:
- Digital Read Performance: This section describes the agreement between the digital image reviewed by a pathologist ("Digital Read") and the manual microscopic read. While a pathologist is involved in the "Digital Read," the intent is to show that replacing the physical microscope with a digital image viewer (which is part of the system) does not negatively impact the pathologist's ability to score the cases.
- Image Analysis Performance: This section directly assesses the agreement between the Virtuoso Image Analysis application's output and the manual microscopic read. This represents the algorithm-only performance in providing a quantitative score before any potential human override.
7. Type of Ground Truth Used
The ground truth used for both the Digital Read and Image Analysis studies was expert consensus (specifically, manual microscopic reads) by qualified pathologists. The manual score was explicitly stated as the "reference result" or "true score."
8. Sample Size for the Training Set
The document does not provide any information regarding the sample size used for the training set of the Virtuoso™ System's image analysis algorithms. This information is typically found in the development and validation sections, which are not detailed in this 510(k) summary.
9. How Ground Truth for the Training Set Was Established
The document does not provide any information on how the ground truth for the training set was established.
Ask a specific question about this device
(245 days)
VIRTUOSO SYSTEM FOR IHC KI-67 (30-9)
The Virtuoso system provides automated digital slide creation, management, analysis, and viewing. It is intended for in vitro diagnostic use as an aid to the pathologist in the display, detection, counting, review and classification of tissues and cells of clinical interest based on particular morphology, color, intensity, size, pattern and shape.
The Virtuoso™ System for Ki67 (30-9) is for digital read and image analysis applications. This particular Virtuoso system is intended for use as an aid to the pathologist in the detection and semi-quantitative measurement of Ki67 (30-9) protein in formalin-fixed, paraffin-embedded normal and neoplastic tissue. This device is an accessory to the Ventana Medical Systems, Inc. CONFIRM™ anti-Ki67 (30-9) Rabbit Monoclonal Primary Antibody assay. The Ventana Medical Systems, Inc. CONFIRM™ anti-Ki67 (30-9) assay is indicated for use in assessing the proliferative activity of normal and neoplastic breast tissue. When used with this assay, the Virtuoso™ System for Ki67 (30-9) is indicated for use as an aid in the assessment of Ki-67 status in breast cancer patients (but is not the sole basis for treatment).
The Virtuoso™ System is an instrument-plus-software system designed to assist the qualified pathologist in the consistent assessment of protein expression in immunohistochemically stained histologic sections from formalin-fixed, paraffinembedded normal and neoplastic tissues. The system consists of a slide scanner (iScan), computer, monitor, keyboard, mouse, image analysis algorithms for specific immunohistochemical markers, and software with a Windows web browser-based user interface. Virtuoso is a web-based, end-to-end, digital pathology software solution that allows pathology laboratories to acquire, manage, view, analyze, share, and report digital images of pathology specimens. Using the Virtuoso software, the pathologist can view digital images, add annotations, make measurements, perform image analysis, and generate reports.
Hardware: The iScan slide scanning device captures digital images of formalin-fixed, paraffin-embedded tissues that are suitable for storage and viewing. The device includes a digital slide scanner, racks for loading glass slides. computer, scanner software, keyboard, mouse and monitor.
Software: The Virtuoso software is designed to complement the routine workflow of a qualified pathologist in the review of immunohistochemically stained histologic slides. It allows the user to select fields of view (FOVs) in the digital image for analysis and provides quantitative data on these FOVs to assist with interpretation. The software makes no independent interpretations of the data and requires competent human intervention for all steps in the analysis process.
Here's a breakdown of the acceptance criteria and the study details for the VENTANA Virtuoso™ System for IHC Ki-67 (30-9), based on the provided 510(k) summary:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state pre-defined "acceptance criteria" in terms of specific thresholds that needed to be met for agreement percentages. Instead, it presents the results of concordance and reproducibility studies. The performance reported below serves as the demonstrated effectiveness of the device.
| Performance Metric | Acceptance Criteria (Implied by reported results) | Reported Device Performance |
|---|---|---|
| Concordance - Digital Read vs. Manual | High agreement between digital read and manual method for Ki-67 classification (≤10% neg, >10% pos). | Overall agreements: 92%, 81%, 86%, 87% across four sites. |
| Negative % Agreement (Digital Read vs. Manual) | 79% (Site 1), 100% (Site 2), 94% (Site 3), 92% (Site 4) | |
| Positive % Agreement (Digital Read vs. Manual) | 100% (Site 1), 69% (Site 2), 80% (Site 3), 81% (Site 4) | |
| Concordance - Image Analysis vs. Manual | High agreement between image analysis and manual method for Ki-67 classification. | Overall agreements: 88%, 86%, 86%, 81% across four sites. |
| Negative % Agreement (Image Analysis vs. Manual) | 79% (Site 1), 95% (Site 2), 86% (Site 3), 65% (Site 4) | |
| Positive % Agreement (Image Analysis vs. Manual) | 93% (Site 1), 81% (Site 2), 86% (Site 3), 100% (Site 4) | |
| Reproducibility - Intra-Pathologist/Inter-Day (Digital Read) | High agreement between repeated digital readings by the same pathologist on different days. | Total agreements ranged from 98% to 100% (across 3 session comparisons). |
| Reproducibility - Intra-Pathologist/Inter-Day (Image Analysis) | High agreement between repeated image analyses by the same pathologist on different days. | Total agreements ranged from 89% to 97% (across 3 session comparisons). |
| Reproducibility - Inter-Pathologist (Digital Read) | Acceptable agreement between digital readings by different pathologists. | Total agreements ranged from 71% to 85%. |
| Reproducibility - Inter-Pathologist (Image Analysis) | Acceptable agreement between image analysis interpretations by different pathologists. | Total agreements ranged from 84% to 93%. |
| Scanner Precision | High percentage agreement for image analysis fields of views across inter-site and intra-site/inter-day scans. | Approximately 90% for every comparison. |
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size:
- Concordance studies:
- Site 1: n = 120
- Site 2: n = 118 (Digital Read), n = 117 (Image Analysis)
- Site 3: n = 114
- Site 4: n = 118 (Digital Read), n = 117 (Image Analysis)
- Reproducibility studies (intra-pathologist):
- The confusion matrices show cell counts of 37-38 for positive and 1-8 for negative detections in Session 1, suggesting a smaller sample size per pathologist for this specific part, but the total number of cases for reproducibility is not explicitly stated.
- Reproducibility studies (inter-pathologist): The confusion matrices show cell counts of ~30-60 for negative and positive detections per site/pathologist comparison, again implying a specific sample size for this portion of the study that is not explicitly stated as a total N.
- Concordance studies:
- Data Provenance: The document does not specify the country of origin of the data. The study was a "primary study" that "evaluated overall system performance across four sites," implying a multi-center study. It does not explicitly state if the data was retrospective or prospective. Given it's a clinical validation study, it would typically be prospective, but this is not confirmed.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
- The ground truth for the concordance studies (Digital Read vs. Manual, Image Analysis vs. Manual) was established by pathologists performing a "manual method (with a traditional microscope)".
- The number of pathologists involved in establishing the "manual method" ground truth for the test set is not explicitly stated, but it implies at least one pathologist per site for the concordance studies (four sites) and three pathologists for the reproducibility studies.
- Qualifications: Referred to as "qualified pathologist[s]". Specific years of experience or sub-specialty are not provided.
4. Adjudication Method for the Test Set
The document does not describe a formal adjudication method (e.g., 2+1 or 3+1 consensus) for establishing the ground truth from the manual method. It simply states the comparison was made against the pathologist's manual results. For the reproducibility studies, it measures agreement between pathologists or within a pathologist over time, not a specific ground truth adjudication process for the test set itself.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- Yes, elements of an MRMC study were conducted:
- The "Agreement/Concordance" section compares the Virtuoso Digital Read and Image Analysis applications against a "Manual Method" performed by pathologists. Each pathologist's Virtuoso digital read results were compared to their manual results. This is a comparison of human performance (manual) against human-assisted performance (digital read and IA).
- The "Reproducibility" section directly assesses inter-pathologist and intra-pathologist variability with the Virtuoso system.
- Effect Size of Human Readers' Improvement with AI vs. Without AI Assistance: The document does not provide a direct "effect size" in terms of how much human readers improve with AI assistance compared to solely manual reading. Instead, it measures the agreement between the AI-assisted methods (Digital Read, Image Analysis) and the "manual method" (considered the true score). For instance, overall agreements for Digital Read vs. Manual ranged from 81-92%, and for Image Analysis vs. Manual ranged from 81-88%. This indicates a high level of agreement, but not an improvement factor in diagnostic accuracy or efficiency.
6. Standalone Algorithm Performance
- Yes, in part. The "Image Analysis" application results (e.g., in the "Virtuoso Image Analysis vs Manual Method" section) represent the performance of the algorithm itself, providing quantitative scores. However, the device description emphasizes that "The software makes no independent interpretations of the data and requires competent human intervention for all steps in the analysis process." The pathologist selects the fields of view and can accept or override the software's score. Therefore, while the algorithm provides a standalone score, its final clinical performance is inherently linked to human interaction. The "Digital Read" performance also represents an AI-assisted mode where the human reads the digital image.
7. Type of Ground Truth Used
- Expert Consensus / Pathologist Interpretation (Manual Method): The ground truth was established by pathologists performing a "manual method (with a traditional microscope)". This means the manual pathological interpretation served as the reference standard against which the digital read and image analysis results were compared.
8. Sample Size for the Training Set
The document does not provide any information regarding the sample size used for the training set of the Virtuoso system's algorithms.
9. How the Ground Truth for the Training Set Was Established
The document does not provide any information on how the ground truth for the training set was established. It only describes the validation studies for the device.
Ask a specific question about this device
Page 1 of 1