(196 days)
ScreenDx is a software-only device that receives and analyzes lung computed tomography (CT) imaging data in order to assess for interstitial lung findings compatible with interstitial lung disease. The device supplements the standard-of-care workflow by providing a qualitative output of imaging findings based on pattern recognition, in order to provide adjunctive information as part of a referral pathway to an appropriately qualified clinician. Patients with positively identified patterns may undergo assessment for lung fibrosis, but ScreenDx does not replace the current standard of care methods for diagnosis of lung fibrosis and the results of the device are not intended to rule-out or rule-in lung fibrosis. The results of ScreenDx are intended to be used only by clinicians qualified in the care of lung disease, in conjunction with the patient's clinical history, symptoms, and other diagnostic tests, as well as the clinician's professional judgment.
The input to ScreenDx is a DICOM-compliant lung CT scan. Clinical case eligibility includes the following criteria:
Age > 22 years old.
ScreenDx is a computer-assisted analysis software device. The software analyzes lung computed tomography (CT) imaging data to provide a qualitative output assessing for interstitial lung findings compatible with interstitial lung disease. The software system is based on a software algorithm component and connection Application Programing Interface (API) to enable image transfer and notifications. The device consists of the following 3 components:
- (1) Image Receiver API for image acquisition;
- (2) Ingestion Pipeline and Analysis System for image processing; and
- (3) Output API for notification transmission.
- (1) The Image Receiver API is accessed via any technologically compliant system (e.g., DICOM, PACS). The case is submitted to the device through the API directly. The API passes the data to the Ingestion Pipeline and Analysis System.
- (2) The Ingestion Pipeline and Analysis System accepts the images, selects cases appropriate for processing, processes the images for analyses, analyzes the images, and stores the images. This system includes the analysis algorithm that identifies lung abnormalities in the case. No diagnostic information is generated from the software.
- (3) The Output API transmits the result of interstitial lung findings compatible with interstitial lung disease to an assigned set of users in the hospital or clinic, specialists who will then review the case. The Output API is integrated into the hospital or clinic notification software (e.g., EHR, messaging system).
The Analysis System is composed of a 3-D deep learning algorithm trained to identify interstitial lung findings compatible with interstitial lung disease. The training dataset included >3,000 lung CT cases from five different data sources from numerous clinical facilities. The algorithm takes in the ingested CT scan, runs it through the locked model, and classifies whether interstitial lung findings compatible with interstitial lung disease appear to be present. The average patient age was 63 years with male and females representing 51.5% and 48.5% of the patient population respectively. All major CT manufacturers were included, and prevalence of positive cases was 24%.
The software output is a binary Positive (Suggestive of ILD)/Negative result for interstitial lung findings compatible with interstitial lung disease. The output is stored for all cases run. Workflow for managing the output is customizable and under the control of the hospital or clinic making use of the device. For example, one workflow can include configuring the output to list Positive cases in a worklist for clinician review (e.g. a dedicated clinician within the pulmonary clinic environment). Another workflow may include integration with dedicated 3rd party software for workflow management of Positive cases. Regardless of method for case list management, cases with a Positive result will be reviewed for consideration of whether additional work-up is clinically indicated.
No analyzed images or other visually assessed features are output by the device. No regions of interest are either input or provided as an output. Additionally, the software does not provide localization information and there is no filtering, post processing, or annotations. The device is designed to not interrupt standard workflows and operates only in parallel, identifying patients who may benefit from additional follow-up for possible lung fibrosis, based on interstitial lung findings compatible with interstitial lung disease.
The Imvaria ScreenDx device is a software-only device designed to analyze lung CT imaging data for interstitial lung findings compatible with interstitial lung disease (ILD). It provides a qualitative output (Positive/Negative) to supplement standard-of-care workflow as part of a referral pathway to a qualified clinician.
1. Table of Acceptance Criteria and Reported Device Performance
| Metric | Acceptance Criteria | Reported Device Performance (Pivotal Study) |
|---|---|---|
| Sensitivity | 80% | 91.4% [CI: 89.0-93.3%] |
| Specificity | 80% | 95.2% [CI: 94.3-96.1%] |
2. Sample Size Used for the Test Set and Data Provenance
- Pivotal Study Test Set: 3,018 unique patients.
- Data Provenance: Multiple datasets from multiple clinical sites. The report does not specify the country of origin but states it was a retrospective, multicenter study.
- Additional Independent Validation Study Test Set: 2,482 cases.
- Data Provenance: Collected prospectively by an independent organization. The report does not specify the country of origin.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
The ground truth for the test set was established via clinical diagnosis derived directly from the data sources. These methodologies for clinical diagnosis were via combined clinical, radiological, laboratory, and/or pathological assessments. The document does not specify the number of experts or their specific qualifications (e.g., "radiologist with 10 years of experience") but implies that the diagnoses were established by qualified clinicians.
4. Adjudication Method for the Test Set
The document states that diagnostic information had been recorded independently for each case via combined clinical, radiological, laboratory, and/or pathological assessments. It does not mention a specific adjudication method like "2+1" or "3+1" for expert review of the test set by the study authors. The ground truth was based on pre-existing clinical diagnoses from various sources.
5. Multi-Reader-Multi-Case (MRMC) Comparative Effectiveness Study
No MRMC comparative effectiveness study was performed or reported. The study focuses on the standalone performance of the algorithm. Therefore, there is no reported effect size of how much human readers improve with AI vs. without AI assistance.
6. Standalone Performance
Yes, a standalone performance study was done. The pivotal study and the additional independent validation study evaluated the algorithm's performance without a human-in-the-loop, reporting sensitivity and specificity metrics.
7. Type of Ground Truth Used
The ground truth used was expert consensus clinical diagnosis (derived from combined clinical, radiological, laboratory, and/or pathological assessments). The presence or absence of the pattern assessed by ScreenDx was intended to correlate with a clinical diagnosis that included lung fibrosis (e.g., IPF, fibrotic NSIP, and other related diagnoses).
8. Sample Size for the Training Set
The algorithm was trained on a dataset that included >3,000 lung CT cases from five different data sources from numerous clinical facilities. Data checks were completed to ensure no overlap between training and test sets.
9. How the Ground Truth for the Training Set Was Established
The document states that the training dataset included cases used to identify interstitial lung findings compatible with interstitial lung disease. While specific details on how the ground truth for the training set was established are not explicitly provided, it is highly likely that similar methodologies as for the test set (clinical diagnosis via combined clinical, radiological, laboratory, and/or pathological assessments) were employed, as this is standard practice for medical imaging AI development based on clinical outcomes. However, the document does not definitively state this for the training set.
N/A