Search Results

NinesMeasure is a semi-automatic tool indicated for use by trained radiologists to aid in the analysis and review of adult thoracic CT images. NinesMeasure provides quantitative information about pulmonary nodule size on a single study or over the time course of several thoracic studies by providing long and short axis diameter measurements in the axial plane.

Based on analysis of DICOM images and provided input from a radiologist, indicating the location of the pulmonary nodule, the device uses artificial intelligence algorithms to automatically perform the measurements, and allows the axial measurements to be displayed and reviewed. NinesMeasure is limited for use on solid pulmonary nodules.

The device is intended to be used as a measurement tool by a trained radiologist and is limited to analysis of imaging data and should not be used in-lieu of full patient evaluation or relied upon to make or confirm a diagnosis. The device does not alter the original medical image.

NinesMeasure is a semi-automatic, diagnostic patient imaging tool used to measure the size of selected pulmonary nodules in a radiological image. The software system is comprised of a set of software modules for performing image analysis at a specified image location to calculate measurements of pulmonary nodules on adult thoracic CT images. The system operates over a standard network interface and receives the DICOM images and coordinates of the pulmonary nodule to measure. The system then returns the measurements for the long and short axis diameters for review by a trained radiologist.

NinesMeasure is designed to be used with a standard PACS, where the user can indicate a location of the pulmonary nodule to measure, and then review and edit the measurements on the DICOM image.

The image analysis uses Artificial Intelligence (Al) technology to analyze chest CT images for computing the measurements. Specifically, the device utilizes a machine learning (ML) algorithm to compute segmentations of nodules, from which the long and short axis measurements are then calculated.

The provided document describes the NinesMeasure device, a semi-automatic tool for measuring pulmonary nodule size on adult thoracic CT images. The document includes information about the device's performance testing.

Here's a breakdown of the acceptance criteria and the study that proves the device meets them:

1. Table of Acceptance Criteria and Reported Device Performance

The document presents primary endpoints related to the normalized error in measuring nodule diameters. While explicit "acceptance criteria" (thresholds the device must meet to be considered effective) are not stated numerically, the narrative confirms that "The performance goals for the primary endpoints were met." This implies internal acceptance criteria for these normalized error values.

Stratified by Nodule Size:

2. Sample Size and Data Provenance for the Test Set

• Sample Size for Test Set:
  • All Nodules: The total number of nodules can be inferred by summing the stratified categories: 100 (3-6mm) + 63 (6-8mm) + 46 (8-10mm) = 209 nodules.
  • The study used cases that were "diverse, and included 3 different major scanner manufacturers, 7 different scanner models, 11 different clinical sites."
• Data Provenance: The study was a retrospective, multi-center image comparison study. The document does not specify the country of origin for the data.

3. Number of Experts and Qualifications for Ground Truth

The document does not explicitly state the "number of experts used to establish the ground truth" or their specific qualifications (e.g., "Radiologist with 10 years of experience"). However, the device "is limited for use by trained radiologists" and relies on "provided input from a radiologist, indicating the location of the pulmonary nodule." This strongly suggests that ground truth measurements would likely be derived from expert radiologist annotations or measurements.

4. Adjudication Method for the Test Set

The document does not describe the specific adjudication method (e.g., 2+1, 3+1) used to establish the ground truth measurements for the test set.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

An MRMC study was not explicitly mentioned or detailed. The study described ("retrospective, multi-center image comparison study") evaluates the device's performance in measuring nodules, but it doesn't compare human readers with and without AI assistance to quantify improvement. The device is a "semi-automatic tool" and requires "provided input from a radiologist," implying a human-in-the-loop design, but the study focuses on the device's measurement accuracy after radiologist input, not the radiologist's performance change.

6. Standalone Performance (Algorithm Only)

The study describes the performance of the NinesMeasure device, which "uses artificial intelligence algorithms to automatically perform the measurements, and allows the axial measurements to be displayed and reviewed."
Given the description "semi-automatic tool" and "provided input from a radiologist, indicating the location of the pulmonary nodule," the reported performance metrics likely represent the algorithm's performance after the initial human input (locating the nodule). It does not appear to be a purely standalone (algorithm-only, end-to-end detection and measurement) performance evaluation. The algorithm performs the measurement after the radiologist pinpoints the nodule.

7. Type of Ground Truth Used

Based on the context of measuring pulmonary nodule size, the ground truth would most likely be expert consensus measurements (manual measurements performed by radiologists or other imaging experts) of the nodule diameters. Pathology or outcomes data are not typically used for direct measurement accuracy ground truth in this context.

8. Sample Size for the Training Set

The document does not specify the sample size used for the training set. It only describes the validation (test) set.

9. How the Ground Truth for the Training Set Was Established

The document does not provide details on how the ground truth for the training set was established. Given the nature of a machine learning (ML) algorithm for segmentation and measurement, it would typically involve expertly annotated images (manual segmentations or measurements) on a separate dataset from the test set.

Ask a specific question about this device

NinesAl is a parallel workflow tool indicated for use by hospital networks and trained clinicians to identify images of specific patients to a radiologist, independent of standard of care workflow, to aid in prioritizing and performing the radiological review. NinesAl uses artificial intelligence algorithms to analyze head CT images for findings suggestive of a pre-specified emergent clinical condition.

The software automatically analyzes Digital Imaging and Communications in Medicine (DICOM) images as they arrive in the Picture Archive and Communication System (PACS) using machine learning algorithms. Identification of suspected findings is not for diagnostic use beyond notification. Specifically, the software analyzes head CT images of the brain to assess the suspected presence of intracranial hemorrhage and/or mass effect and identifies images with potential emergent findings in a radiologist's worklist.

NinesAl is intended to be used as a triage tool limited to analysis of imaging data and should not be used in-lieu of full patient evaluation or relied upon to make or confirm a diagnosis. Additionally, preview images displayed to the radiologist outside of the DICOM viewer are non-diagnostic quality and should only be used for informational purposes.

NinesAl notifies a radiologist of the presence of a suspected critical abnormality in a radiological image. The software system is a complete package comprised of image analysis software and a workstation module that is used to alert the radiologist. The image analysis can also be configured to send HL7 messages and DICOM secondary series.

The image analysis uses Artificial Intelligence (AI) technology to analyze non contrast CT Head scans for the presence of Intracranial Hemorrhage and/or Mass Effect. More specifically, the device utilizes two machine learning (ML) algorithms to detect each finding respectively.

NinesAl is a software device and does not come into contact with patients. All radiological studies are still reviewed by trained radiologists. NinesAl is meant to be used as an aid for case prioritization.

Here's a summary of the acceptance criteria and study details for the NinesAI device, based on the provided text:

Acceptance Criteria and Device Performance

The acceptance criteria are derived from the observed performance of the predicate device (Aidoc's BriefCase) and a baseline of 0.80 for both sensitivity and specificity for general emergent findings.

Time Benefit Analysis:

Study Details

1. Sample Size and Data Provenance (Test Set):

   • Sample Size: Not explicitly stated as a single number, but the text mentions "Head CT studies included in each of the test datasets were obtained from over 20 clinical sites."
   • Data Provenance: Retrospective. The studies were obtained from "over 20 clinical sites" and included "a minimum of 3 scanner manufacturers and over 20 scanner models, and also reflected broad patient demographics," suggesting a diverse dataset. The country of origin for the data is not specified.

2. Number of Experts and Qualifications (Ground Truth for Test Set):

   • Number of Experts: Not explicitly stated. The text mentions "agreement rate between labelers who determined ground truth for the test dataset studies." This implies multiple experts were involved in establishing the ground truth.
   • Qualifications of Experts: Not explicitly stated, but the term "labelers" typically refers to trained medical professionals who are qualified to interpret medical images, such as radiologists.

3. Adjudication Method (Test Set):

   • Not explicitly stated. The mention of "agreement rate between labelers who determined ground truth" suggests some form of consensus or agreement process, but the specific method (e.g., 2+1, 3+1) is not detailed.

4. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:

   • No, a specific MRMC comparative effectiveness study is not explicitly mentioned. The study focuses on the standalone performance of the AI algorithm and a time benefit analysis, which compares AI notification time to standard-of-care time-to-open-dictation, rather than comparing human reader performance with and without AI assistance.

5. Standalone Performance Study:

   • Yes, a standalone (algorithm only) performance study was conducted. The algorithms were evaluated independently, and primary endpoints like sensitivity and specificity were measured for each algorithm.

6. Type of Ground Truth Used (Test Set):

   • Expert Consensus: The text states, "agreement rate between labelers who determined ground truth for the test dataset studies." This indicates that human expert consensus was used to establish the ground truth.

7. Sample Size for Training Set:

   • Not explicitly stated in the provided text. The text mentions, "The algorithms are trained using a database of radiological images," but does not give a specific number for the training set size.

8. How Ground Truth for Training Set was Established:

   • Not explicitly stated in the provided text. It is generally inferred that similar expert labeling methods would be used for training data, but the document does not detail this.

Ask a specific question about this device