Search Results

Virtual Nodule Clinic (VNC) is a software device used in the tracking, assessment and characterization of incidentally detected pulmonary nodules.

VNC includes a computer-aided diagnosis (CADx) function, available only to pulmonologists and radiologists. This automatically analyzes user-selected regions of interest (RO) within lung CT data to provide volumetric and computer analysis based on morphological characteristics. Using only imaging features extracted from the CT image data, an artificial intelligence algorithm calculates a single value, the LCP-CNN score, which is displayed to the user. The LCP-CNN score is analyzed relative to LCP-CNN scores generated on a database of cases with known ground-truth using a histogram display format. The LCP-CNN score may be useful in the characterization of pulmonary nodules during image interpretation and may be used as one input to clinical decision making when following published clinical guidelines.

VNC's LCP-CNN score is indicated for the evaluation of incidentally detected solid and semi-solid pulmonary nodules of diameter 5-30mm in patients aged 35 years or above. In cases where multiple abnormalities are present, VNC's LCP-CNN score can be used to assess each abnormality independently.

Note that LCP-CNN is not indicated for lung cancer screening nor is it indicated for nodules of pure ground glass opacity. In addition, high contrast CT images were not used in clinical validation (as measured as >300HU median attenuation in the aortic arch) and the validation data also excluded CT images with only calcified nodules (since these are typically considered to be benign), with implants, motion artifacts, or cases with greater than 5 nodules. Finally, the validation data excluded patients with history of cancer of less than 5 years to avoid the presence of metastatic lesions.

Users other than radiologists and pulmonologists, e.g. clinicians, nurse practitioners and navigators, may use VNC to view CT images and reports, organize patient management workflow, track patients, record management decisions and organize nodule clinics. For these users, the LCP-CNN score is unavailable.

VNC is a software only device which consists of two main components: a web application accessed via standard desktop web browsers and the LCP-CNN machine learning model.

Virtual Nodule Clinic (VNC) is a software application designed for trained medical professionals in the clinical management of patients with pulmonary nodules. VNC includes a computer-aided diagnosis (CADx) function to assist pulmonologists and radiologists in the assessment and characterization of incidentally detected pulmonary nodules using CT image data.

VNC has two main functions:

• A CADx function to assist radiologists and pulmonologists in clinical decision making by providing a score using machine learning (namely the LCP-CNN algorithm, Lung Cancer Prediction Convolutional Network). Also note that the output of the CADx function is referred to synonymously in this document as either "Optellum LCP Score" or "LCP-CNN score", depending on context.
• . A management function to allow users to easily track patients that need to be under management follow-up for indeterminate pulmonary nodules (IPNs).

VNC consists of two main software components:

• . A web application deployed on a virtual server within a hospital datacenter;
• The LCP-CNN module deployed on a GPU-equipped server on hospital premises or in the cloud.

VNC is connected to two other IT systems in the hospital: a DICOM-compatible Picture Archiving and Communication System (PACS) for accessing images and to the Radiology Information System (RIS) or reporting system for accessing the clinical reports.

Acceptance Criteria and Study for Optellum™ Virtual Nodule Clinic

1. Table of Acceptance Criteria and Reported Device Performance

2. Sample size used for the test set and the data provenance

• Test Set Sample Size:

  • CADx Standalone Testing: This information is not explicitly stated as a separate "test set" for the standalone performance, but the ROC curve itself is based on the performance of the LCP-CNN model. The "150 out of 300" on the ROC graph likely refers to the number of malignant cases correctly identified out of the total malignant cases. The LCP-CNN model was evaluated on a dataset consisting of malignant (10%) and benign nodules (90%) that were not used during model training. The exact size of this dataset isn't explicitly given in this section, but the MRMC study used 300 subjects.
  • MRMC Clinical Study: 300 subjects (cases), with 150 malignant and 150 benign nodules. Each subject involved a single nodule of interest.

• Data Provenance:

  • CADx Standalone Testing Data: The document mentions the dataset used for mapping the continuous LCP-CNN output to an integer score (which is effectively what is being evaluated in standalone testing) consisted of malignant (10%) and benign (90%) nodules. No explicit country of origin or retrospective/prospective information is given for this specific mapping dataset.
  • MRMC Clinical Study Data: Retrospectively collected from 9 academic and community hospitals.
    • Country of Origin: 174 cases from the USA and 126 cases from the EU. The document notes that while the device is indicated for incidentally detected nodules, some community screening-detected nodules were included due to data scarcity.
    • Retrospective/Prospective: Retrospective. Patient consent was waived for all datasets (other than NLST data, which is mentioned as a general note but not explicitly stated as being part of this study's validation data).

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

• Number of Experts: Not explicitly stated as a specific number of experts directly establishing ground truth for the retrospective clinical validation dataset.
• Qualifications of Experts: The ground truth for the nodules in the clinical validation dataset was established either by:
  • Biopsy
  • Resection
  • 2-year follow-up (for benign nodules, or nodule disappearance)
  • The document also states that the diagnosis of all nodules used in the study, whether screening or incidental, was established using the "same common protocol that was used for the training dataset." The training dataset's ground truth was based on "nodules that were confidently matched to a definitive diagnosis, as provided with the data." This suggests that the ground truth was based on definitive clinical outcomes/histopathology, which implicitly involves medical experts.

4. Adjudication method (e.g., 2+1, 3+1, none) for the test set

The document does not explicitly describe an adjudication method like 2+1 or 3+1 for establishing the ground truth of the test set cases. Instead, it states that the ground truth was determined by definitive clinical outcomes (biopsy, resection, or 2-year follow-up). This implies a direct, objective determination rather than an expert consensus process that would typically require adjudication.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• Yes, a MRMC comparative effectiveness study was done.
• Effect Size of Improvement: The mean performance of human readers (radiologists and pulmonologists) improved by 6.85 AUC points (95% CI [4.29, 9.41], p < .001) when assisted by the Optellum LCP-CNN score compared to reading without the AI assistance. The average AUC for solo clinician performance was 81.9%, and with AI assistance, it was 88.8%.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

• Yes, a standalone performance test was done. The LCP-CNN model (the CADx function) was subjected to standalone testing. This testing measured the model's performance in discriminating between benign and malignant nodules before its incorporation into the device for further testing.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

• For the MRMC Study and underlying validation dataset: Ground truth was established by definitive clinical outcomes: biopsy, resection, or 2-year follow-up (for benign nodules, or nodule disappearance). This is a strong form of outcomes data.
• For the training set: Ground truth was established based on "nodules that were confidently matched to a definitive diagnosis, as provided with the data." This is consistent with definitive clinical outcomes.

8. The sample size for the training set

The training data for the LCP-CNN network consisted of "solid and semi-solid nodules of at least 5mm in diameter... 8% malignant nodules and 92% benign nodules." The total number of nodules in the training set is not explicitly stated in the provided text.

9. How the ground truth for the training set was established

The ground truth for the training set was established for "nodules that were confidently matched to a definitive diagnosis, as provided with the data." This suggests the use of definitive clinical outcomes such as biopsy, resection, or long-term follow-up and clinical records provided with the datasets acquired for training.

Ask a specific question about this device