(234 days)
RevealAI-Lung Software is a computer aided diagnostic (CADx) software application intended for the characterization of incidentally-detected lung nodules on computed tomography (CT) scans. When a nodule is identified, the Software automatically compares the nodule characteristics with a clinically established database of lung nodules and provides a similarity score to assist clinicians' assessment of patients' cancer risk.
The mSI score is indicated for the evaluation of incidentally-detected pulmonary nodules of diameter 6-15mm in patients aged 18 years or above. In cases where multiple abnormalities are present, the mSI score can be used to assess each abnormality independently. Risk should be interpreted on an individual patient level and mSI is a relative risk score, not a percentage cancer risk.
Note that mSI is not indicated for lung cancer screening. The validation data excluded CT images with missing slices.
The RevealAI-Lung device is a post-processing software program that analyzes patient lung computed tomography (CT) images and is designed to provide computer-aided diagnostic (CADx) information about lung nodules to radiologists.
The user opens the patient's lung CT image from a third-party acquisition device in an existing medical device viewing system and scrolls through the image slices as in their normal workflow. The user identifies a lung nodule on the CT image, and evaluates that nodule for cancer risk and the potential need for follow-up using existing known risk factors, clinical management guidelines and the Reveal-AI-Lung provided mSI score. In cases where multiple nodules are present, RevealAI-Lung can be used to assess each nodule independently.
Here's a breakdown of the acceptance criteria and the study proving RevealAI-Lung meets them, based on the provided FDA 510(k) Clearance Letter:
1. Table of Acceptance Criteria and Reported Device Performance
| Acceptance Criterion | Reported Device Performance |
|---|---|
| Primary Endpoint (Multi-Reader Multi-Case (MRMC) Study): Improvement in radiologists' ability to discriminate between malignant and benign pulmonary nodules from CT images with and without the aid of the mSI. Measured as the difference in Area Under the Receiver Operating Characteristic Curve (AUC). | Average AUC improvement: 0.181 (from 0.538 unassisted to 0.719 with RevealAI-Lung assistance). This difference was statistically significant (p < 0.0001). |
| Consistency of Performance Across Readers: Every radiologist must improve their performance when using RevealAI-Lung. | Achieved: Every radiologist (10/10) improved their performance when using RevealAI-Lung. Individual AUC improvements ranged from 0.106 to 0.258. |
| Sensitivity Improvement (at 5% malignancy likelihood threshold): Increase in sensitivity when using RevealAI-Lung. | Increased sensitivity by 14 points (from 0.68 ± 0.039 to 0.82 ± 0.036). |
| Specificity Improvement (at 5% malignancy likelihood threshold): Increase in specificity when using RevealAI-Lung. | Increased specificity by 12 points (from 0.344 ± 0.041 to 0.467 ± 0.043). |
| Standalone Performance: Ability of RevealAI-Lung to discriminate between benign and malignant nodules. | Achieved: Standalone testing of RevealAI-Lung demonstrated it performed as expected in discriminating between benign and malignant nodules. (Specific quantitative metrics for standalone AUC are not explicitly provided, but "performed as expected" is stated.) |
| Validation on External Populations: Consistent device performance across additional incidental nodule populations. | Achieved: Tested on three additional populations (US, Canada, UK). Each study produced performance with an AUC > 0.8, and demonstrated follow-up decisions would be improved compared to clinical guidelines. |
| Consistency Across Subgroups: Performance improvements consistent across patient, nodule, and technical parameters. | Achieved: Results were independent of radiologist experience, patient demographics (age, sex, race/ethnicity), scan characteristics (contrast, scan date, manufacturer), and nodule parameters (size, lobe, opacity). Range of improvement in subgroups: 0.12 - 0.30. |
| Software Quality System Compliance: Adherence to FDA guidance for software in medical devices, 21 CFR §892.2060 special controls, human factors, usability, and cybersecurity. | Achieved: Design, validation, and verification were planned, executed, and documented according to FDA guidance. Assessed as Moderate Level of Concern. Usability evaluations confirmed safety and effectiveness. Cybersecurity activities and risk management were performed. |
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size for Clinical Performance Testing (MRMC Study): 108 cases (patients) with incidental lung nodules. The cases included size-matched benign and malignant nodules.
- Sample Size for Validation Testing on External Populations: 675 patients with incidental lung nodules (276 with cancer).
- Data Provenance:
- MRMC Study: Sourced from 3 US sites and 1 in Canada.
- External Validation Studies: One each from the US, Canada, and the UK.
- Retrospective or Prospective: Both the MRMC study and the external validation studies appear to be based on retrospective data, as they used "CT series... from patients in routine practice where lung nodules had been noted incidentally on the original radiology report" and involved "following the patients for at least 5 years" for ground truth (where pathology was not available).
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The document specifies the ground truth for the test sets (both MRMC and external validation) was established with "strict requirement for diagnostic certainty (either pathologic confirmation or two-years radiologic monitoring to confirm benign nodules)."
While it doesn't explicitly state the number of experts who established the ground truth, the involvement of "pathologic confirmation" or "two-years radiologic monitoring" implies the standard clinical practice involving pathologists and/or radiologists in the diagnostic process. The MRMC study itself involved 10 radiologists reading the cases, and while they were assessing malignancy likelihood, the ground truth for those cases was pre-established based on the methods described.
4. Adjudication Method for the Test Set
The adjudication method for establishing the ground truth (pathologic confirmation or two-year radiological monitoring) is not explicitly detailed in terms of expert consensus (e.g., 2+1, 3+1). However, the "strict requirement for diagnostic certainty" implies a high standard of clinical diagnosis.
For the MRMC study's reader evaluations, there was no direct adjudication of reader disagreement against each other. Instead, each reader's interpretation (with and without AI) was compared against the pre-established ground truth for each case. Each case was read twice by each reader, separated by a 28-day washout period, with AI use randomized for the second read.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, What Was the Effect Size of How Much Human Readers Improve with AI vs. Without AI Assistance
Yes, an MRMC comparative effectiveness study was done.
- Reader Improvement: Radiologists improved their accuracy for the diagnosis of pulmonary nodules by an average of 18 points (0.181 AUC).
- Average AUC without the device: 0.538
- Average AUC with the device: 0.719
- Statistical Significance: This difference was statistically significant (p < 0.0001; Dorfman-Berbaum-Metz ANOVA random-reader random-case (RRRC) with jackknife (Wilcoxon)).
- Consistent Improvement: Every radiologist (10 out of 10) improved their performance when using RevealAI-Lung, with individual improvements ranging from 0.11 to 0.26 AUC points.
6. If a Standalone (i.e., Algorithm Only Without Human-in-the-Loop Performance) Was Done
Yes, standalone testing was done.
- Performance: "Standalone testing of RevealAI-Lung demonstrated that it performed as expected in discriminating between benign and malignant nodules."
- Additional Validation: "Validation of RevealAI-Lung was performed to determine device performance against the ground truth using pre-established acceptance criteria. The device was subsequently tested on incidental nodules from three additional populations (one each US, Canada, and the UK). Each of these studies produced performance with an AUC > 0.8, and demonstrated follow-up decisions would be improved compared to clinical guidelines." This indicates strong standalone performance on external datasets.
7. The Type of Ground Truth Used (Expert Consensus, Pathology, Outcomes Data, etc.)
The ground truth for both training and validation sets was established with "strict requirement for diagnostic certainty":
- Pathologic Confirmation: For malignant nodules, this would typically involve biopsy results.
- Two-Years Radiologic Monitoring: For benign nodules, this means stable appearance over two years of follow-up CT scans, indicating a non-cancerous nature.
- Outcome Data: The phrase "following the patients for at least 5 years" for confidently matched diagnoses used in training, and implied in validation, points to long-term outcomes data to confirm the definitive diagnosis.
8. The Sample Size for the Training Set
- Training Dataset: RevealAI-Lung was trained on "radiologist-identified lung nodules from 4-30mm in diameter."
- Specific Sample Size: The exact number of cases or nodules in the training set is not explicitly stated in the provided document, beyond the characteristics of the subjects (median age 63, 43% female).
9. How the Ground Truth for the Training Set Was Established
- Method: "Only nodules that were confidently matched to a definitive diagnosis were used for training, including following the patients for at least 5 years."
- This implies a combination of pathology (for malignant cases) and long-term radiologic stability/outcomes (for benign cases) to ensure diagnostic certainty, similar to the method described for the test sets. The mention of "radiologist-identified lung nodules" for the training set likely refers to how the nodules were initially marked or selected, while the "confidently matched to a definitive diagnosis" over 5 years is how their ground truth was ultimately confirmed.
FDA 510(k) Clearance Letter - RevealAI-Lung
Page 1
U.S. Food & Drug Administration
10903 New Hampshire Avenue
Silver Spring, MD 20993
www.fda.gov
Doc ID # 04017.08.02
January 30, 2026
Precision Medical Ventures, Inc. dba RevealDx
℅ Michael Calhoun
Chief Technology Officer
1800 Westlake Ave. N., Ste 140
SEATTLE, WA 98109
Re: K251769
Trade/Device Name: RevealAI-Lung
Regulation Number: 21 CFR 892.2060
Regulation Name: Radiological Computer-Assisted Diagnostic Software For Lesions Suspicious Of Cancer
Regulatory Class: Class II
Product Code: POK
Dated: June 10, 2025
Received: January 2, 2026
Dear Michael Calhoun:
We have reviewed your section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (the Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Page 2
K251769 - Michael Calhoun Page 2
Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled "Deciding When to Submit a 510(k) for a Change to an Existing Device" (https://www.fda.gov/media/99812/download) and "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" (https://www.fda.gov/media/99785/download).
Your device is also subject to, among other requirements, the Quality System (QS) regulation (21 CFR Part 820), which includes, but is not limited to, 21 CFR 820.30, Design controls; 21 CFR 820.90, Nonconforming product; and 21 CFR 820.100, Corrective and preventive action. Please note that regardless of whether a change requires premarket review, the QS regulation requires device manufacturers to review and approve changes to device design and production (21 CFR 820.30 and 21 CFR 820.70) and document changes and approvals in the device master record (21 CFR 820.181).
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting (reporting of medical device-related adverse events) (21 CFR Part 803) for devices or postmarketing safety reporting (21 CFR Part 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reporting-combination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR Part 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR Parts 1000-1050.
All medical devices, including Class I and unclassified devices and combination product device constituent parts are required to be in compliance with the final Unique Device Identification System rule ("UDI Rule"). The UDI Rule requires, among other things, that a device bear a unique device identifier (UDI) on its label and package (21 CFR 801.20(a)) unless an exception or alternative applies (21 CFR 801.20(b)) and that the dates on the device label be formatted in accordance with 21 CFR 801.18. The UDI Rule (21 CFR 830.300(a) and 830.320(b)) also requires that certain information be submitted to the Global Unique Device Identification Database (GUDID) (21 CFR Part 830 Subpart E). For additional information on these requirements, please see the UDI System webpage at https://www.fda.gov/medical-devices/device-advice-comprehensive-regulatory-assistance/unique-device-identification-system-udi-system.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-devices/medical-device-safety/medical-device-reporting-mdr-how-report-medical-device-problems.
For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medical-devices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-devices/device-advice-comprehensive-regulatory-
Page 3
K251769 - Michael Calhoun Page 3
assistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely,
Lu Jiang
Lu Jiang, Ph.D.
Assistant Director
Diagnostic X-Ray Systems Team
DHT8B: Division of Radiological Imaging
Devices and Electronic Products
OHT8: Office of Radiological Health
Office of Product Evaluation and Quality
Center for Devices and Radiological Health
Enclosure
Page 4
DEPARTMENT OF HEALTH AND HUMAN SERVICES
Food and Drug Administration
Indications for Use
Form Approved: OMB No. 0910-0120
Expiration Date: 07/31/2026
See PRA Statement below.
510(k) Number (if known): K251769
Device Name: RevealAI-Lung
Indications for Use (Describe)
RevealAI-Lung Software is a computer aided diagnostic (CADx) software application intended for the characterization of incidentally-detected lung nodules on computed tomography (CT) scans. When a nodule is identified, the Software automatically compares the nodule characteristics with a clinically established database of lung nodules and provides a similarity score to assist clinicians' assessment of patients' cancer risk.
The mSI score is indicated for the evaluation of incidentally-detected pulmonary nodules of diameter 6-15mm in patients aged 18 years or above. In cases where multiple abnormalities are present, the mSI score can be used to assess each abnormality independently. Risk should be interpreted on an individual patient level and mSI is a relative risk score, not a percentage cancer risk.
Note that mSI is not indicated for lung cancer screening. The validation data excluded CT images with missing slices.
Type of Use (Select one or both, as applicable)
☒ Prescription Use (Part 21 CFR 801 Subpart D)
☐ Over-The-Counter Use (21 CFR 801 Subpart C)
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services
Food and Drug Administration
Office of Chief Information Officer
Paperwork Reduction Act (PRA) Staff
PRAStaff@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
FORM FDA 3881 (8/23) Page 1 of 1 PSC Publishing Services (301) 443-6740 EF
Page 5
510(k) Summary
This 510(k) summary of safety and effectiveness information is submitted as part of the Pre-Market Notification in accordance with the requirements of 21 CFR Part 807, Subpart E, Section 807.92.
Date prepared: January 28, 2026
Submitter organization: Precision Medical Ventures, Inc. dba RevealDx
Submitter address: 1800 Westlake Ave. N., Ste 104
Seattle, WA 98109
United States
Submitter telephone: +1 425.409.9579
Correspondence contact: Michael Calhoun, PhD
michael@reveal-dx.com
510(k) number: K251769
Device trade name: RevealAI-Lung
Device common name: Computer-aided diagnosis (CADx) software
Regulation name: Computer-Assisted Diagnostic Software For Lesions Suspicious For Cancer
Regulatory class: Class II
Regulation number: 21 CFR 892.2060
Product code: POK
1. Predicate Device
This section identifies the legally marketed device (predicate) to which Precision Medical Ventures claims equivalence.
Trade name: Optellum™ Software
Manufacturer: Optellum Ltd.
510(k) number: K202300
Device common name: Computer-aided diagnosis (CADx) software
Regulation name: Computer-Assisted Diagnostic Software For Lesions Suspicious For Cancer
Regulatory class: Class II
Regulation number: 21 CFR 892.2060
Product code: POK
Page 6
2. Device description
2.1. Device characteristics
The RevealAI-Lung device is a post-processing software program that analyzes patient lung computed tomography (CT) images and is designed to provide computer-aided diagnostic (CADx) information about lung nodules to radiologists.
The user opens the patient's lung CT image from a third-party acquisition device in an existing medical device viewing system and scrolls through the image slices as in their normal workflow. The user identifies a lung nodule on the CT image, and evaluates that nodule for cancer risk and the potential need for follow-up using existing known risk factors, clinical management guidelines and the Reveal-AI-Lung provided mSI score. In cases where multiple nodules are present, RevealAI-Lung can be used to assess each nodule independently.
2.2. Use environment
The Software is used in hospitals and healthcare provider office settings.
2.3. Device features and usage
RevealAI-Lung compares features from the selected nodule to a reference image database derived from National Lung Cancer Screening Trial (NLST) patient cases. Once the analysis is complete, the software opens a separate integrated frame in the MIMPS viewer which displays analytics for the selected nodule.
The software provides a similarity score (malignancy similarity index, or mSI) for each user-selected nodule based on extracted image characteristics from the nodule region and the degree of similarity between the user-selected nodule and known lung nodules from the NLST image database. The mSI value is expressed as a numeric value from 0 to 1 based on a machine learning algorithm trained on features calculated from nodules in the NLST database where diagnoses were confirmed.
RevealAI-Lung provides comparative analysis for the mSI to nodules with known ground truth using a histogram display (showing mSI score and the number of truthed cases in the NLST database that are benign vs. malignant at a particular score level). A user experienced with the significance of such data will be able to view and interpret this additional information during interpretation of lung nodules.
The mSI score, in combination with other information, may be used in the characterization of incidentally detected pulmonary nodules and may be used as one input to clinical decision making when following published clinical guidelines.
The RevealAI-Lung user interface includes the mSI score and a histogram of case distributions for benign and malignant lung nodules within the training dataset. Information on the intended use, regulatory background information and manufacturer labeling is available within the interface.
Page 7
2.4. Materials of use
- RevealAI-Lung is a software only device.
2.5. Key performance characteristics
RevealAI-Lung is a software only device, so design, validation and verification were planned, executed and documented according to FDA guidance, including the following:
- FDA's Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices.
- 21 CFR §892.2060 special controls.
- FDA's Guidance for Applying Human Factors and Usability Engineering to Medical Devices.
- FDA's Guidance for Content of Premarket Submissions for Management of Cybersecurity in Medical Devices.
3. Intended use
This section describes RevealAI-Lung's intended use, including a general description of the diseases or conditions that the device will diagnose and of the patient population for which the device is intended. It also compares RevealAI-Lung's intended use to that of the predicate device.
3.1. Indications for Use statement
RevealAI-Lung Software is a computer aided diagnostic (CADx) software application intended for the characterization of incidentally-detected lung nodules on computed tomography (CT) scans. When a nodule is identified, the Software automatically compares the nodule characteristics with a clinically established database of lung nodules and provides a similarity score to assist clinicians' assessment of patients' cancer risk.
The mSI score is indicated for the evaluation of incidentally-detected pulmonary nodules of diameter 6-15mm in patients aged 18 years or above. In cases where multiple abnormalities are present, the mSI score can be used to assess each abnormality independently. Risk should be interpreted on an individual patient level and mSI is a relative risk score, not a percentage cancer risk.
Note that mSI is not indicated for lung cancer screening. The validation data excluded CT images with missing slices.
3.2. Intended patient population
The targeted population inclusion criteria are based on those where lung nodules are found on CT scans in clinical practice, and where product validation has occurred, specifically where lung nodules have been identified incidentally on CT.
3.3. Contraindications
- Not indicated for use on modalities other than lung CT images
- Not indicated for use on non-lung-nodule image locations
- Not indicated for unassisted reads (device output requires practitioner review)
Page 8
- Not indicated for use where image quality or the presence of surgical or implanted artifacts obscures the lung nodule
- Not indicated for use when CT images were acquired at a slice spacing exceeding 5mm
3.4. Intended users
RevealAI-Lung Software is designed to be used by radiologists who are licensed to make diagnostic or other follow-up decisions using radiological imagery and/or radiology reports. The information provided by RevealAI-Lung should only be used in conjunction with other clinically accepted information to guide medical decision-making at the discretion of the clinician.
3.5. Comparison with predicate device
This subsection explains the similarities with and differences between RevealAI-Lung and the predicate device in relation to intended use, including their respective Indications for Use, and explains why the differences do not raise new types of questions related to device safety or effectiveness.
3.5.1. Similarities
Both RevealAI-Lung and Optellum Software are computer aided diagnostic (CADx) software devices used in the assessment of pulmonary nodules visible on computed tomography (CT) scans, as a subset of the broader POK product code for "Computer-Assisted Diagnostic Software for Lesions Suspicious for Cancer".
Both RevealAI-Lung and Optellum Software are computer-aided diagnosis (CADx) software devices used to assist clinicians in the assessment and characterization of abnormalities using image data. Both devices extract image data from user-confirmed lung nodules to provide volumetric and computer analysis based on morphological characteristics. Both devices use an artificial intelligence algorithm to synthesize these imaging (or radiomic) features into a single value represented as a score which is analyzed relative to scores generated on a database of known cases.
Optellum Software patient population is limited to patients presenting with between one and five incidentally detected solid and/or semi-solid pulmonary nodules. RevealAI-Lung includes use on these same types of nodules.
For both devices, device failure could lead to an absence of results, delay in scan interpretation, or incorrect risk scores.
Similar to the RevealAI-Lung device, efficacy of Optellum Software is determined following the same special controls for design, validation, verification and labeling; both use reader studies to validate the impact on reader performance.
Both devices provide the user with information to aid in the interpretation of medical images and both devices provide a machine learning based score derived from the patient's image to aid the reader in interpreting the image and making recommendations and patient decisions. The RevealAI-Lung device provides an mSI score whereas the Optellum Software device provides an
Page 9
LCP-CNN score. In both cases, the score is not a probability of malignancy but provides a likelihood function, to be interpreted using a display of information derived from a database of cases with known ground-truth.
In cases where multiple abnormalities are present, both devices can be used to assess each abnormality independently.
Both devices provide a histogram of the score distribution over malignant and benign cases from a database where the ground-truth is known. In both devices, the histogram is provided as a reference to aid in the use of the score in image interpretation.
In both devices, the user makes the final determination and decision on patient management.
3.5.2. Differences
The following features of the Optellum Software are not included with RevealAI-Lung (note that these are non-CADx functions):
- View CT images and reports
- Organize patient management workflow
- Track patients
- Record management decisions
- Organize nodule clinics
RevealAI-Lung is not indicated for use by pulmonologists, and does not have the above-described features of the Optellum Software intended for other practitioners (e.g., clinicians, nurses). RevealAI-Lung is only intended for use by radiologists.
RevealAI-Lung does not include screen-detected nodules in the intended use.
Compared to the predicate, RevealAI-Lung is intended for use with validation demonstrated for:
- Contrast-enhanced CT scans,
- Pure ground glass opacities, and,
- Nodules 6 - 15mm in diameter.
Patients with recent cancer diagnoses were not excluded from RevealAI-Lung validation data.
RevealAI-Lung does not display DICOM images. RevealAI-Lung does not provide information regarding the patient demographics.
3.5.3. Comparison of Key Features of RevealAI-Lung and the Predicate Device
Clinical aspects of the RevealDx RevealAI-Lung device were assessed and compared to the Optellum predicate device. The table below summarizes and compares each device.
Page 10
Table 1 Predicate Comparison Chart - Clinical Aspects
| Clinical Aspect | Submitted DeviceRevealDxRevealAI-Lung(This Submission) | Primary Predicate DeviceOptellum Ltd.Optellum™ Software(K202300) |
|---|---|---|
| Intended Use | § 892.2060 Radiological computer-assisted diagnostic software for lesions suspicious of cancer.A radiological computer-assisted diagnostic software for lesions suspicious of cancer is an image processing prescription device intended to aid in the characterization of lesions as suspicious for cancer identified on acquired medical images such as magnetic resonance, mammography, radiography, or computed tomography. The device characterizes lesions based on features or information extracted from the images and provides information about the lesion(s) to the user. Diagnostic and patient management decisions are made by the clinical user. | § 892.2060 Radiological computer-assisted diagnostic software for lesions suspicious of cancer.A radiological computer-assisted diagnostic software for lesions suspicious of cancer is an image processing prescription device intended to aid in the characterization of lesions as suspicious for cancer identified on acquired medical images such as magnetic resonance, mammography, radiography, or computed tomography. The device characterizes lesions based on features or information extracted from the images and provides information about the lesion(s) to the user. Diagnostic and patient management decisions are made by the clinical user. |
| Indications for Use | RevealAI-Lung Software is a computer aided diagnostic (CADx) software application intended for the characterization of incidentally-detected lung nodules on computed tomography (CT) scans. When a nodule is identified, the Software automatically compares the nodule characteristics with a clinically established database of lung nodules and provides a similarity score to assist clinicians' assessment of patients' cancer risk.The mSI score is indicated for the evaluation of incidentally-detected pulmonary nodules of diameter 6-15mm in patients aged 18 years or above. In cases where multiple abnormalities are present, the mSI score can be used to assess each abnormality independently. Risk should be interpreted on an individual patient level and mSI is a | Virtual Nodule Clinic (VNC) is a software device used in the tracking, assessment and characterization of incidentally detected pulmonary nodules.VNC includes a computer-aided diagnosis (CADx) function, available only to pulmonologists and radiologists. This automatically analyzes user-selected regions of interest (ROI) within lung CT data to provide volumetric and computer analysis based on morphological characteristics. Using only imaging features extracted from the CT image data, an artificial intelligence algorithm calculates a single value, the LCP-CNN score, which is displayed to the user. The LCP-CNN score is analyzed relative to LCP-CNN scores generated on a database of cases |
Page 11
| Clinical Aspect | Submitted DeviceRevealDxRevealAI-Lung(This Submission) | Primary Predicate DeviceOptellum Ltd.Optellum™ Software(K202300) |
|---|---|---|
| relative risk score, not a percentage cancer risk.Note that mSI is not indicated for lung cancer screening. The validation data excluded CT images with missing slices. | with known ground-truth using a histogram display format. The LCP-CNN score may be useful in the characterization of pulmonary nodules during image interpretation and may be used as one input to clinical decision making when following published clinical guidelines.VNC's LCP-CNN score is indicated for the evaluation of incidentally detected solid and semi-solid pulmonary nodules of diameter 5-30mm in patients aged 35 years or above. In cases where multiple abnormalities are present, VNC's LCP-CNN score can be used to assess each abnormality independently.Note that LCP-CNN is not indicated for lung cancer screening nor is it indicated for nodules of pure ground glass opacity. In addition, high contrast CT images were not used in clinical validation (as measured as >300HU median attenuation in the aortic arch) and the validation data also excluded CT images with only calcified nodules (since these are typically considered to be benign), with implants, motion artifacts, missing slices, or cases with greater than 5 nodules. Finally, the validation data excluded patients with history of cancer of less than 5 years to avoid the presence of metastatic lesions.Users other than radiologists and pulmonologists, e.g. clinicians, nurses, nurse practitioners and navigators, may use VNC to view CT images and reports, organize patient management workflow, track patients, record management |
Page 12
| Clinical Aspect | Submitted DeviceRevealDxRevealAI-Lung(This Submission) | Primary Predicate DeviceOptellum Ltd.Optellum™ Software(K202300) |
|---|---|---|
| decisions and organize nodule clinics. For these users, the LCP-CNN score is unavailable. | ||
| Intended Patient Population | The targeted population inclusion criteria are based on those where lung nodules are found on CT scans in clinical practice, and where product validation has occurred, specifically where lung nodules have been identified incidentally on CT. | Optellum Software is a software device used in the tracking, assessment, and characterization of incidentally detected pulmonary nodules. |
| Contraindications | • Not indicated for use on modalities other than lung CT images• Not indicated for use on non-lung-nodule image locations• Not indicated for unassisted reads (device output requires practitioner review)• Not indicated for use where image quality or the presence of surgical or implanted artifacts obscures the lung nodule• Not indicated for use when CT images were acquired at a slice spacing exceeding 5mm | • The Optellum LCP Score is not indicated for pure Ground-Glass Opacities (GGO) or for calcified nodules.• The Optellum LCP Score is not indicated for patients with a history of cancer less than 5 years. The Optellum LCP Score is not indicated for nodules detected by lung cancer screening studies. The Optellum LCP Score is not indicated for patients with more than five pulmonary nodules.• The Optellum LCP Score is not indicated for patients with thoracic implants that impact the image appearance of the nodule.• The Optellum LCP Score is not indicated for patients younger than 35 years. |
| Intended Users | RevealAI-Lung Software is designed to be used by radiologists who are licensed to make diagnostic or other follow-up decisions using radiological imagery and/or radiology reports. The information provided by RevealAI-Lung should only be used in conjunction with other clinically accepted information to guide medical decision-making at the discretion of the clinician. | A nodule clinic is typically led by a pulmonologist and includes a multi-disciplinary team of radiologists, thoracic surgeon and sometimes pathologist and oncologist. |
| Intended Use Environment | The Software is used in hospitals and healthcare provider office settings. | Virtual Nodule Clinic is intended for use in hospitals to support these clinics. |
Page 13
Technical characteristics for the RevealDx RevealAI-Lung device were assessed and compared to the Optellum predicate device. The table below summarizes the assessment made.
Table 2 Predicate Comparison Chart - Technical Features
| Technical Features | Submitted DeviceRevealDxRevealAI-Lung(This Submission) | Primary Predicate DeviceOptellum Ltd.Optellum™ Software(K202300) |
|---|---|---|
| Regulatory Classification | Regulation: 21 CFR 892.2060Product Code: POKDevice Classification Name: Computer-Assisted Diagnostic Software for Lesions Suspicious for Cancer | Regulation: 21 CFR 892.2060Product Code: POKDevice Classification Name: Computer-Assisted Diagnostic Software for Lesions Suspicious for Cancer |
| Product Features | Assists in malignancy assessmentReduces variations between individual physiciansImproves clinical decision makingHas been independently clinically validated in multi-center studies | Increases the accuracy of malignancy assessmentReduces variations between individual physiciansImproves clinical decision makingHas been independently clinically validated in multi-center studies |
| Diagnostic Workflow | Patient scan: The CT data and any prior scans are automatically uploaded into the RevealAI-Lung software.Identify nodule: Easily review any available CT and mark the nodule(s) of interest.RevealAI-Lung mSI score: Within seconds, RevealAI-Lung analyzes the CT image data to compute the score.Optimal clinical decisions: With the support of the RevealAI-Lung mSI score, make the optimal clinical management decision for the patient. | Patient scan: The CT and any prior scans are automatically uploaded into the Optellum Software.Identify nodule: Easily review any available CT and mark the nodule(s) of interest.Optellum Lung Cancer Prediction score: Within seconds, the Optellum Lung Cancer Prediction analyzes the 3D image region around the nodule to compute the score.Optimal clinical decisions: With the support of the Optellum LCP score, make the optimal clinical management decision for the patient. |
| Patient Contact Materials | None. No patient contact | None. No patient contact |
| Device Characteristics | Provides a computer-aided diagnosis (CADx) function to assist radiologists who are familiar with lung nodule management in the assessment and characterization of | Provides a computer-aided diagnosis (CADx) function to assist pulmonologists and radiologists in the assessment and characterization of incidentally detected pulmonary nodules using CT image data. |
Page 14
| Technical Features | Submitted DeviceRevealDxRevealAI-Lung(This Submission) | Primary Predicate DeviceOptellum Ltd.Optellum™ Software(K202300) |
|---|---|---|
| pulmonary nodules using CT image data.Provides a score using machine learning referred to as mSI or malignancy similarity index.The RevealAI-Lung software is deployed on a server on-premise at the healthcare facility or within cloud-based systems securely connected to the facility network.Connected to two other IT systems in the hospital: a DICOM-compatible Picture Archiving and Communication System (MIMPS) for accessing images and to the Radiology Information System (RIS) or reporting system for accessing the clinical reports. | Provides a score using machine learning referred to as LCP-CNN algorithm, or Lung Cancer Prediction Convolutional Neural Network.The LCP-CNN module is deployed on a GPU-equipped server on hospital premises or in the cloud.Connected to two other IT systems in the hospital: a DICOM-compatible Picture Archiving and Communication System (MIMPS) for accessing images and to the Radiology Information System (RIS) or reporting system for accessing the clinical reports. | |
| Use Environment | RevealAI-Lung is used in hospitals, imaging centers and other radiologist reader settings where diagnostic evaluation of CT images occur. | Pulmonary nodule clinics are typically set up in hospitals, so that nodules can be followed up or clinical investigations can be prescribed to confirm the diagnosis associated with the presence of pulmonary nodules. A nodule clinic is typically led by a pulmonologist and includes a multi-disciplinary team of radiologists, thoracic surgeon and sometimes pathologist and oncologist. Virtual Nodule Clinic is intended for use in hospitals to support these clinics. |
| Computer-Aided Diagnosis | The CADx function automatically analyzes user-selected lung nodules from lung CT data. This function extracts image data from the nodule to provide 3D analysis and computer analytics based on morphological characteristics. These imaging (or radiomic) features are computed solely from the image extract and then synthesized by an artificial intelligence algorithm into a single value, the mSI score, which is displayed to the user. The score displayed is an integer between 0 and 1 where 0 means very likely benign and 1 means very likely | The CADx function automatically analyzes user-selected regions of interest (ROI) from lung CT data. This function extracts image data from the ROI to provide 3D analysis and computer analytics based on morphological characteristics. These imaging (or radiomic) features are computed solely from the image extract and then synthesized by an artificial intelligence algorithm into a single value, the LCP-CNN score, which is displayed to the user. The score displayed is an integer between 1 and 10 where 1 means very likely benign and 10 means very likely |
Page 15
| Technical Features | Submitted DeviceRevealDxRevealAI-Lung(This Submission) | Primary Predicate DeviceOptellum Ltd.Optellum™ Software(K202300) |
|---|---|---|
| malignant. The mSI score is not a probability of malignancy but is intended to be analyzed relative to mSI scores generated on a database of cases with known ground-truth, using a histogram display format.The mSI score, in combination with other information may be used in the characterization of pulmonary nodules and may be used as one input to clinical decision making when following published clinical guidelines. | malignant. The LCP-CNN score is not a probability of malignancy but is intended to be analyzed relative to LCP-CNN scores generated on a database of cases with known ground-truth, using a histogram display format.The LCP-CNN score, in combination with other information may be used in the characterization of incidentally detected pulmonary nodules and may be used as one input to clinical decision making when following published clinical guidelines. | |
| Principles of Operation | RevealAI-Lung is accessed via integration with a MIMPS system for radiologist use. Once generated, results may be incorporated in reporting at the discretion of the radiologist. | Optellum Software is accessed via a web browser on any PC or Apple workstation in the hospital. The software is validated with web browsers Google Chrome, Mozilla Firefox, Microsoft Edge, Microsoft Internet Explorer and Apple Safari. |
| Image Analysis Algorithm: Input | Input to RevealAI-Lung is a DICOM CT image and a spatial coordinate centered on a lung nodule.Internally RevealAI-Lung creates 3D views of the nodule location and whole CT slice, and extracts thousands of machine vision features including layers of transformations into frequency and other signal processing domains. Machine learning relates these features to confirmed diagnoses of input nodules. | CNN and input: The LCP-CNN system is based on the Dense Convolutional Network, a widely used type of deep learning CNN architecture that was designed for computer vision tasks. LCP-CNN's ensemble of convolutional neural network models ends with a fully connected binary classification layer (malignant or benign). Input to LCP-CNN is a 3-dimensional crop of a CT image, centered on a lung nodule. |
| Image Analysis Algorithm: Training Dataset | Training dataset: RevealAI-Lung was trained on radiologist-identified lung nodules from 4-30mm in diameter, with data augmentation to vary the spatial coordinate. The median age of the subjects was 63 (55-74), with 43% of the subjects female.Only nodules that were confidently matched to a definitive diagnosis were used for training, including | Training dataset: The LCP-CNN network was trained using solid and semi-solid nodules of at least 5mm in diameter. The training data consisted of 8% malignant nodules and 92% benign nodules and included both screening (95%) and incidentally detected (5%) nodules. The median age of the subjects was 62 (20-90) and included a mix of US (95%) and EU data (5%). The data included females (38%) and males (62%). Only |
Page 16
| Technical Features | Submitted DeviceRevealDxRevealAI-Lung(This Submission) | Primary Predicate DeviceOptellum Ltd.Optellum™ Software(K202300) |
|---|---|---|
| following the patients for at least 5 years. | nodules that were confidently matched to a definitive diagnosis, as provided with the data, were used for training. | |
| Image Analysis Algorithm: Output | RevealAI-Lung provides an mSI score for relative malignancy risk with a scale of 0 to 1, with 0 representing lowest risk and 1 highest risk. The score is shown with contextual information to relate mSI to the training population's confirmed nodule diagnoses and risk levels across the scale. | CNN output: LCP-CNN's output is a continuous value between 0 and 1; this is mapped to an integer score between 1 and 10. This mapping was constructed by computing the raw LCP score on a dataset consisting of malignant (10%) and benign nodules (90%) but which were not used during the model training. The mapped integer score is shown to the user alongside a plot detailing the cancer prevalence in each of 10 bins for a population with a 30% cancer prevalence. |
| Design Verification & Validation: Nonclinical Performance Testing | Standalone testing of RevealAI-Lung demonstrated that it performed as expected in discriminating between benign and malignant nodules. Technical validation is also described for variation in nodule coordinate and acquisition dose. | Standalone testing of the LCP-CNN model demonstrated that it performed as expected in discriminating between benign and malignant nodules. |
| Design Verification & Validation: Clinical Performance Testing | Concurrent use of RevealAI-Lung improved radiologists accuracy for the diagnosis of pulmonary nodules by an average of 18 points, from 0.54 to 0.72 (p < 0.001)Every radiologist improved their performance when using RevealAI-Lung, and performance was consistent across patient, nodule and technical parameters. | • Concurrent use of the LCP-CNN feature in Optellum Virtual Nodule Clinic software to read CT exams improves radiologists' and pulmonologists' accuracy for the diagnosis of pulmonary nodules by an average of 6.85 AUC points (p < .001) (from 81.9 to 88.8 AUC)• Every radiologist and pulmonologist improved their accuracy when using the Optellum LCP-CNN feature to assist their read. |
4. Technological Characteristics
This section summarizes RevealAI-Lung's technological characteristics and explains the similarities and differences with those of the predicate device, and explains why the differences do not raise new types of questions related to device safety or effectiveness.
Page 17
4.1. Principles of operation
Both devices are software only (software as a medical device). Both devices have a graphical user interface. Both can import DICOM images from connected third party DICOM-compatible devices.
RevealAI-Lung interfaces with an existing MIMPS system (Medical Image Processing and Management System) to provide the lung nodule characterization in a familiar, secure environment characteristic of radiology diagnostic workflows.
While the predicate states that it is "accessed via a web browser on any PC or Apple workstation in the hospital", RevealAI-Lung is accessed only via connections to established MIMPS.
RevealAI-Lung is installed on server(s) that are common in a variety of healthcare settings, while Optellum Software, as described, requires servers with specialty GPU capabilities. Both use Linux-based operating systems with a high level of security, safety and functionality supporting device installation and use.
4.2. Image analysis algorithm
Image analysis algorithms behind RevealAI-Lung's CADx function, are described in detail in this submission, according to 21 CFR §892.2060 special control 1(i), and summarized here:
Inputs: Input to RevealAI-Lung is a DICOM CT image and a spatial coordinate centered on a lung nodule.
Internally RevealAI-Lung extracts thousands of machine vision features from the CT, and machine learning techniques were used to relate these features to confirmed diagnoses within the reference dataset.
Training dataset: RevealAI-Lung was trained on radiologist-identified lung nodules from 4-30mm in diameter. The only augmentation to this dataset was to vary the spatial coordinate corresponding to the nodule center. The median age of the subjects was 63 (55-74), with 43% of the subjects female.
Only nodules that were confidently matched to a definitive diagnosis were used for training, including following the patients for at least 5 years.
Outputs: RevealAI-Lung provides an mSI score for relative malignancy risk with a scale of 0 to 1, with 0 representing lowest risk and 1 highest risk. The score is shown with contextual information to relate mSI to the training population's confirmed nodule diagnoses and risk levels across the scale.
4.3. MIMPS connectivity
Integration with MIMPS allows the RevealAI-Lung mSI results and user interface described within to be displayed in the typical radiology workflow when provided with a CT series and nodule coordinate.
4.4. Reporting archive connectivity
The predicate's software includes reporting capabilities not provided with RevealAI-Lung. Reporting of mSI results is at the discretion of the reading radiologist using existing systems and methods based on their reading environment.
Page 18
5. Nonclinical performance
This section includes discussion of the nonclinical tests used to establish substantial equivalence of RevealAI-Lung to the predicate device.
5.1. Design, verification and validation
Like the predicate device, RevealAI-Lung is a software only device, so design, validation and verification were planned, executed and documented according to the following, and summarized in the rest of this subsection:
-
FDA's Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices. Based on this guidance, RevealAI-Lung was assessed to represent a Moderate Level of Concern, the same as the predicate device.
-
21 CFR §892.2060 special control 1(v) Appropriate software documentation.
As a software-only device, all requirements are software requirements. User requirements (those relating directly to the intended users) and product requirements (e.g. environmental and regulatory requirements) were identified without reference to design; they are design inputs.
Device hazard analysis and risk management, based on ISO 14971:2019 Application of Risk Management to Medical Devices, was used to identify and control hazards associated with the device. This consists of:
- Device Characteristics Analysis, including intended use, reasonably foreseeable misuse, identification of characteristics related to safety.
- Risk analysis, including identification of hazards and estimation of the associated risks.
- Risk control, including identification and analysis of risk control measures (design specifications and architectural design), implementation and verification of risk control measures; traceability; residual risk analysis to determine acceptability.
- Risk review, including the overall residual risk compared to benefits of using the device.
Implementation of requirements is documented in detailed architectural design and design specifications, including acceptance criteria.
5.2. Planned and pre-defined testing included:
- Validation that RevealAI-Lung's user and product requirements are met (acceptance testing) – test results are included in this submission.
- Verification of correct implementation of design specifications, including risk control measures (system testing) – test results are included in this submission.
- Verification of correct implementation of architectural and detailed software design (unit and integration testing).
A series of traceability tables link enumerated requirements, risks, design specifications and tests. Unresolved software anomalies known at the time of manufacture are listed in this submission.
No remaining risks affect or create any identified hazards, nor do they affect the device's safety and effectiveness.
Page 19
In summary, RevealAI-Lung is successfully validated against user and product requirements and successfully verified against design specifications. All risk control measures, including those related to usability, are verified to be performing as designed.
5.3. Usability evaluation
Usability evaluations were performed as part of validation and verification, using FDA guidance Applying Human Factors and Usability Engineering to Medical Devices. This evaluation confirmed that RevealAI-Lung is safe and effective for the intended users, uses and use environments.
5.4. Cybersecurity
Cybersecurity activities were performed using FDA's "Cybersecurity in Medical Devices: Quality System Considerations and Content of Premarket Submissions; Guidance for Industry and Food and Drug Administration Staff" (Document issued on September 27, 2023)
The following activities were performed:
- Identification of assets, threats, and vulnerabilities
- For each threat and vulnerability, assessment of:
- the impact on device functionality and end users/patients
- the likelihood of a threat and of a vulnerability being exploited
- risk levels and suitable mitigation strategies
- residual risk and risk acceptance criteria.
- Planning activities for post-market surveillance of cybersecurity.
Identified cybersecurity risks, their control measures and verifications were managed along with other device risks. RevealAI-Lung's technical manual includes description of requirements and activities related to cybersecurity at installation locations.
5.5. Validation testing on external populations
Validation of RevealAI-Lung was performed to determine device performance against the ground truth using pre-established acceptance criteria. The device was subsequently tested on incidental nodules from three additional populations (one each US, Canada, and the UK). Each of these studies produced performance with an AUC > 0.8, and demonstrated follow-up decisions would be improved compared to clinical guidelines. Together these studies validated performance on 675 patients with incidental lung nodules (276 with cancer).
All of these studies were performed on a population basis without exclusion criteria other than a strict requirement for diagnostic certainty (either pathologic confirmation or two-years radiologic monitoring to confirm benign nodules). They included broad patient demographics (age, sex, race/ethnicity), scan characteristics (contrast, scan date, manufacturer), and nodule parameters (size, lobe, opacity).
Page 20
6. Clinical performance
6.1. Summary
Like the predicate device, clinical performance of RevealAI-Lung's CADx function was assessed and documented according to 21 CFR §892.2060 special controls 1(ii) and 1(iii).
Clinical performance of the mSI feature in the RevealAI-Lung software was evaluated in a fully-crossed, multiple-reader multiple-case (MRMC) study. Radiologists were asked to evaluate the malignancy risk of incidental lung nodules identified on CT. These CT were displayed in a familiar radiologist reading environment, and lung nodules were evaluated without RevealAI-Lung or with concurrent use of RevealAI-Lung, alternating between these case types. Each case was read twice separated by a 28-day washout period, with use of RevealAI-Lung randomized between the two reads for each reader. The study focused on size-matched benign and malignant nodules with 10 readers, 108 cases.
This study demonstrated the safety and effectiveness of RevealAI-Lung. The size-matching removed nodule size as a covariate from the analysis, allowing comparison of the challenge facing radiologists when assessing similar nodules.
6.2. Clinical endpoints
6.2.1. Primary:
The primary endpoint compares the ability of the readers to discriminate between malignant and benign pulmonary nodules from CT images only, with and without the aid of the mSI. For a given reader, discrimination was measured using the area under the receiver operating characteristic curve (AUC) across all cases. The effect size of the mSI intervention was measured as the difference in AUC before and after consulting the malignancy score provided by the mSI (i.e. unaided and aided reads). Statistics used the stringent random-reader, random-case paradigm to account for potential variance with intended users and patients.
Sub-analyses:
- Patient sex, race, age
- Scan contrast, year, manufacturer
- Nodule size, lobe, attenuation (solid, non-solid)
- Radiologist training and experience levels
- Radiologist accuracy in indicating nodule center
6.2.2. Secondary:
Secondary endpoints were also evaluated including consistency of device performance and reader interpretation, and sensitivity and specificity measures.
6.3. Test methods
6.3.1. Reader selection
Ten radiologists from a variety of experience levels and practice backgrounds participated in the
Page 21
reader studies. All were board certified with an active medical license.
6.3.2. Reader training
Prior to performing study reads, readers were trained on RevealAI-Lung operating principles and the clinical background (literature review), then asked to perform a small set of training cases (n=18). After these reads, the reader was provided feedback based on the literature for malignancy likelihood of nodules with that type/size. While malignancy likelihood estimation is specific to the clinical evaluation, the training was typical for radiologists before clinical use of the device.
6.3.3. Study Workflow
Readers used a radiology viewer to load CT series containing the lung nodule. Readers were free to evaluate the case as in practice for unassisted reads. For assisted reads, only the image data and the mSI score were used to assess the study. For each of the cases, the reader clicked on the nodule center, and was presented with the RevealAI-Lung output with a user-interface requesting entry of the malignancy likelihood.
6.3.4. Datasets Used
CT Series were used from patients in routine practice where lung nodules had been noted incidentally on the original radiology report. The data was sourced from 3 US sites and 1 in Canada, and did not filter by reason for scan, the use of contrast material, or any other patient, clinical or technical characteristics. This produced sample sets representing the intended use population.
For statistical purposes, the samples were enriched to have matching numbers of patients with malignant vs. benign nodules. Sampling used an algorithm to produce size-paired nodules such that radiologists would be 'challenged' in distinguishing nodules that would be treated similarly based on this variable.
The following are the summary characteristics of the patients, scans and nodules used:
Page 22
The above parameters were broadly similar in the control versus cancer groups. Note that Ethnicity data was only available for US-based subjects (Not Applicable means not disclosed).
Nodule sizes were closely matched between malignant (mean: 9.2mm) and benign nodules (9.1mm).
6.3.5. Clinical Study Results
The analysis for reader LOM, comparing results with/without the device using AUC are shown in Figure 1.
Page 23
Figure 1. AUC across all readers without the device (Unassisted, blue) or with the device (RevealAI-Lung, green), with the ROC curve shown with 95% confidence intervals across readers (left), and the reader means (right; with median and IQR (Tukey plot)).
Note that baseline AUC statistics confirm the challenging nature of this size-matched dataset with low baseline radiologist performance (chance is 0.5 AUC).
Every reader improved when using the device (range 0.11 - 0.26):
| Reader # | AUC without | AUC with | Δ |
|---|---|---|---|
| 1 | 0.639 | 0.773 | 0.134 |
| 2 | 0.437 | 0.695 | 0.258 |
| 3 | 0.502 | 0.714 | 0.212 |
| 4 | 0.612 | 0.717 | 0.106 |
| 5 | 0.601 | 0.736 | 0.136 |
| 6 | 0.477 | 0.709 | 0.232 |
| 7 | 0.553 | 0.697 | 0.143 |
| 8 | 0.549 | 0.682 | 0.134 |
| 9 | 0.491 | 0.719 | 0.229 |
| 10 | 0.517 | 0.745 | 0.228 |
| Mean: | 0.538 | 0.719 | 0.181 |
This difference was statistically significant (p < 0.0001; Dorfman-Berbaum-Metz ANOVA random-reader random-case (RRRC) with jackknife (Wilcoxon)).
Sensitivity/Specificity
Page 24
We pre-specified an example 5% likelihood of malignancy threshold for this sample. Use of RevealAI-Lung increased sensitivity by 14 points (0.68 ± 0.039 to 0.82 ± 0.036) and specificity by 12 points (0.344 ± 0.041 to 0.467 ± 0.043).
Note: These values should not be interpreted as population statistics as they are dependent on the included sample.
6.3.6. Summary of sub-analyses
Results were independent of radiologist experience, patient demographics (age, sex, race/ethnicity), scan characteristics (contrast, scan date, manufacturer), and nodule parameters (size, lobe, opacity). Each subgroup for all of the categories had an improved AUC when using the device (range of improvement in subgroups: 0.12 - 0.30).
6.4. Conclusions
6.4.1. Nonclinical performance testing
RevealAI-Lung was successfully validated against user and product requirements and successfully verified against design specifications. All risk control measures, including those related to usability, were verified to be performing as designed.
Results across populations demonstrated consistent device performance and met requirements for characterizing benign versus malignant lung nodules. These results were independent of patient population, and differences in clinical care and scan technical characteristics.
6.4.2. Clinical performance testing
The described clinical performance studies validated:
- Usability: consistent device performance with radiologist use
- Generalizability: consistent device performance across populations, nodule & scan types
- Safety: Performance improvements consistent across readers
- Clinical benefit: Readers appropriately increase/decrease malignancy likelihood
6.4.3. Benefit-risk assessment
Risk management based on ISO 14971:2019-12 was performed in the design and development of this device. These assessments have determined that based upon the residual risk acceptability criteria and the residual risk documented, the overall residual risk and risk/benefit of RevealAI-Lung is as low as possible and acceptable.
The studies summarized herein are part of the Clinical Evaluation Procedure that includes formal assessment of the device risks and benefits. As described above, when used as intended by radiologists on patients accurately representing the intended use population, the device has significant clinical benefit.
Together, the reduction of all categories of risks documented in these procedures, and the clinical benefits described herein, present an acceptable risk/benefit profile. The intended use, indications for use, risk assessment, instructions, claims that are contained herein and supported by the clinical study outcomes, all align with the safety and efficacy of the product.
Page 25
6.5. Substantial Equivalence Statement
RevealAI-Lung has the same or similar indications for use, intended use and technological characteristics to the predicate device. The non-clinical and clinical performance described herein demonstrate that RevealAI-Lung performs at least as safely and effectively as the predicate device in the context of its intended use and with the proposed labeling. RevealAI-Lung meets the requirements for safety and effectiveness and does not introduce any new potential safety risks.
The information provided in this submission supports our claim that RevealAI-Lung is as safe, as effective, and performs as well as or better than the legally marketed predicate device.
§ 892.2060 Radiological computer-assisted diagnostic software for lesions suspicious of cancer.
(a)
Identification. A radiological computer-assisted diagnostic software for lesions suspicious of cancer is an image processing prescription device intended to aid in the characterization of lesions as suspicious for cancer identified on acquired medical images such as magnetic resonance, mammography, radiography, or computed tomography. The device characterizes lesions based on features or information extracted from the images and provides information about the lesion(s) to the user. Diagnostic and patient management decisions are made by the clinical user.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Design verification and validation must include:
(i) A detailed description of the image analysis algorithms including, but not limited to, a detailed description of the algorithm inputs and outputs, each major component or block, and algorithm limitations.
(ii) A detailed description of pre-specified performance testing protocols and dataset(s) used to assess whether the device will improve reader performance as intended.
(iii) Results from performance testing protocols that demonstrate that the device improves reader performance in the intended use population when used in accordance with the instructions for use. The performance assessment must be based on appropriate diagnostic accuracy measures (
e.g., receiver operator characteristic plot, sensitivity, specificity, predictive value, and diagnostic likelihood ratio). The test dataset must contain sufficient numbers of cases from important cohorts (e.g., subsets defined by clinically relevant confounders, effect modifiers, concomitant diseases, and subsets defined by image acquisition characteristics) such that the performance estimates and confidence intervals of the device for these individual subsets can be characterized for the intended use population and imaging equipment.(iv) Standalone performance testing protocols and results of the device.
(v) Appropriate software documentation (
e.g., device hazard analysis; software requirements specification document; software design specification document; traceability analysis; and description of verification and validation activities including system level test protocol, pass/fail criteria, results, and cybersecurity).(2) Labeling must include:
(i) A detailed description of the patient population for which the device is indicated for use.
(ii) A detailed description of the intended reading protocol.
(iii) A detailed description of the intended user and recommended user training.
(iv) A detailed description of the device inputs and outputs.
(v) A detailed description of compatible imaging hardware and imaging protocols.
(vi) Warnings, precautions, and limitations, including situations in which the device may fail or may not operate at its expected performance level (
e.g., poor image quality or for certain subpopulations), as applicable.(vii) Detailed instructions for use.
(viii) A detailed summary of the performance testing, including: Test methods, dataset characteristics, results, and a summary of sub-analyses on case distributions stratified by relevant confounders (
e.g., lesion and organ characteristics, disease stages, and imaging equipment).