(143 days)
Canon CR-2 Plus AF,Canon CR-2 AF
Yes
The device description explicitly states that the EyeArt Analysis Computation Engine "uses artificial intelligence algorithms to analyze the fundus images" and "consists of an ensemble of clinically aligned machine learning (deep learning) algorithms."
No.
The device is indicated for use in detecting diabetic retinopathy, not for treating any condition.
Yes
The device, EyeArt, is indicated for use by healthcare providers to automatically detect more than mild diabetic retinopathy and vision-threatening diabetic retinopathy. This function directly aligns with the definition of a diagnostic device, which is used to identify or determine the presence of a disease or condition.
Yes
The device description explicitly states "EyeArt is a software as a medical device" and details its components as Client, Server, and Analysis Computation Engine, all of which are software-based. While it interacts with a fundus camera, the camera is described as a separate, indicated device, not part of the EyeArt system itself.
No, this device is not an IVD (In Vitro Diagnostic).
Here's why:
- IVD Definition: In Vitro Diagnostics are medical devices intended for use in vitro (outside the body) for the examination of specimens, including blood, tissue, and urine, derived from the human body, to provide information for diagnostic, monitoring or compatibility purposes.
- EyeArt's Function: EyeArt analyzes images of the retina captured by a fundus camera. It does not analyze biological specimens taken from the patient's body.
- Intended Use: The intended use is to automatically detect diabetic retinopathy based on the analysis of these images, not through the examination of biological samples.
Therefore, based on the provided information, EyeArt falls under the category of a medical device that uses imaging and software analysis, but it does not meet the definition of an In Vitro Diagnostic.
No
The letter does not explicitly state that the FDA has reviewed and approved or cleared a PCCP for this specific device.
Intended Use / Indications for Use
EyeArt is indicated for use by healthcare providers to automatically detect more than mild diabetic retinopathy and visionthreatening diabetic retinopathy (severe non-proliferative diabetic retinopathy or proliferative diabetic retinopathy and/or diabetic macular edema) in eyes of adults diagnosed with diabetes who have not been previously diagnosed with more than mild diabetic retinopathy. EyeArt is indicated for use with Canon CR-2 Plus AF cameras in both primary care and eye care settings.
Product codes
PIB
Device Description
EyeArt is a software as a medical device that consists of several components - Client, Server, and Analysis Computation Engine. A retinal fundus camera, used to capture retinal fundus images of the patient, is connected to a computer where the EyeArt Client software is installed. The EyeArt Client software provides a graphical user interface (GUI) that allows the EyeArt operator to transfer the appropriate fundus images to and receive results from the remote EyeArt Analysis Computation Engine through the EyeArt Server. The EyeArt Analysis Computation Engine is installed on remote computer(s) in a secure data center and uses artificial intelligence algorithms to analyze the fundus images and return results. EyeArt is intended to be used with color fundus images of resolution 1.69 megapixels or higher captured using one of the indicated color fundus cameras (Canon CR-2 AF and Canon CR-2 Plus AF) with 45 degrees field of view. EyeArt is specified for use with two color fundus images per eye: optic nerve head (ONH) centered and macula centered.
For each patient eye, the EyeArt results separately indicate whether more than mild diabetic retinopathy (mtmDR) and vision-threatening diabetic retinopathy (vtDR) are detected. More than mild diabetic retinopathy is defined as the presence of moderate non-proliferative diabetic retinopathy or worse on the International Clinical Diabetic Retinopathy (ICDR) severity scale and/or the presence of diabetic macular edema. Vision-threatening diabetic retinopathy is defined as the presence of severe non-proliferative diabetic retinopathy or proliferative diabetic retinopathy on the ICDR severity scale and/or the presence of diabetic macular edema.
- EyeArt Client: This component is installed on the computer used by the EyeArt operator (working under supervision of a healthcare provider). It allows the operator to transfer images to the EyeArt Analysis Computation Engine and receive results. Its functioning requires an internet connection. If images from a patient encounter cannot be analyzed, due to poor image quality or due to lack of all required image fields, feedback is provided to the operator to help successfully obtain results upon resubmission.
- EyeArt Server: This component provides an interface that securely handles incoming requests and securely stores user information including images and results. It enables the EyeArt Client to use the EyeArt Analysis Computation Engine through an application programming interface (API).
- EyeArt Analysis Computation Engine: This component analyzes the images to determine exam quality and detect mtmDR and vtDR. It consists of an ensemble of clinically aligned machine learning (deep learning) algorithms.
Mentions image processing
Yes
Mentions AI, DNN, or ML
Yes
Input Imaging Modality
Retinal fundus images (color fundus images)
Anatomical Site
Eyes / Retinal fundus
Indicated Patient Age Range
Adults aged 22 years and older
Intended User / Care Setting
Healthcare providers in both primary care and eye care settings.
Description of the training set, sample size, data source, and annotation protocol
Not Found
Description of the test set, sample size, data source, and annotation protocol
The test set for the pivotal clinical trial consisted of 655 participants (after exclusions) enrolled at 11 US study sites including primary care and general ophthalmology centers. Participants were divided into two cohorts: sequential enrollment (235 subjects) and enrichment-permitted (420 subjects).
During the study visit, subjects underwent 2-field retinal photography for EyeArt processing. They then underwent dilated 4-wide field stereo retinal photography by an FPRC certified operator for the clinical reference standard. The dilated four wide-field stereo-image sets were independently reviewed and graded by certified graders at the Fundus Photography Reading Center (FPRC), to determine the severity of diabetic retinopathy (DR) and diabetic macular edema (DME) according to the ETDRS severity scale. Each subject's images were graded independently by 2 experienced and certified graders, and in case of significant differences, a more experienced adjudication grader graded the same images. Graders and study staff at the FPRC were masked to patient history and EyeArt results.
The FPRC grading was used to determine the clinical reference standard for subject eyes as follows:
- Clinical reference standard for more than mild DR (mtmDR): positive if ETDRS level was 35 or greater (but not equal to 90) or clinically significant macular edema [CSME] grade was CSME present; negative if ETDRS levels were 10-20 and CSME grade was CSME absent; ungradable if ETDRS level was 90, or CSME grade was cannot grade or questionable with ETDRS level 10-20.
- Clinical reference standard for vision-threatening DR (vtDR): positive if ETDRS level was 53 or greater (but not equal to 90) or CSME grade was CSME present; negative if ETDRS levels were 10-47 and CSME grade was CSME absent; ungradable if ETDRS level was 90, or CSME grade was cannot grade or questionable with ETDRS level 10-47.
Summary of Performance Studies (study type, sample size, AUC, MRMC, standalone performance, key results)
Study Type: Prospective, multi-center pivotal clinical trial (ClinicalTrials.gov ID NCT03112005) and precision (repeatability and reproducibility) study.
Sample Size:
- Pivotal Clinical Trial: Initially 942 subjects consented, 915 met eligibility, 655 participants included in analysis after excluding subjects not meeting additional prespecified criteria (e.g., age
§ 886.1100 Retinal diagnostic software device.
(a)
Identification. A retinal diagnostic software device is a prescription software device that incorporates an adaptive algorithm to evaluate ophthalmic images for diagnostic screening to identify retinal diseases or conditions.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Software verification and validation documentation, based on a comprehensive hazard analysis, must fulfill the following:
(i) Software documentation must provide a full characterization of technical parameters of the software, including algorithm(s).
(ii) Software documentation must describe the expected impact of applicable image acquisition hardware characteristics on performance and associated minimum specifications.
(iii) Software documentation must include a cybersecurity vulnerability and management process to assure software functionality.
(iv) Software documentation must include mitigation measures to manage failure of any subsystem components with respect to incorrect patient reports and operator failures.
(2) Clinical performance data supporting the indications for use must be provided, including the following:
(i) Clinical performance testing must evaluate sensitivity, specificity, positive predictive value, and negative predictive value for each endpoint reported for the indicated disease or condition across the range of available device outcomes.
(ii) Clinical performance testing must evaluate performance under anticipated conditions of use.
(iii) Statistical methods must include the following:
(A) Where multiple samples from the same patient are used, statistical analysis must not assume statistical independence without adequate justification.
(B) Statistical analysis must provide confidence intervals for each performance metric.
(iv) Clinical data must evaluate the variability in output performance due to both the user and the image acquisition device used.
(3) A training program with instructions on how to acquire and process quality images must be provided.
(4) Human factors validation testing that evaluates the effect of the training program on user performance must be provided.
(5) A protocol must be developed that describes the level of change in device technical specifications that could significantly affect the safety or effectiveness of the device.
(6) Labeling must include:
(i) Instructions for use, including a description of how to obtain quality images and how device performance is affected by user interaction and user training;
(ii) The type of imaging data used, what the device outputs to the user, and whether the output is qualitative or quantitative;
(iii) Warnings regarding image acquisition factors that affect image quality;
(iv) Warnings regarding interpretation of the provided outcomes, including:
(A) A warning that the device is not to be used to screen for the presence of diseases or conditions beyond its indicated uses;
(B) A warning that the device provides a screening diagnosis only and that it is critical that the patient be advised to receive followup care; and
(C) A warning that the device does not treat the screened disease;
(v) A summary of the clinical performance of the device for each output, with confidence intervals; and
(vi) A summary of the clinical performance testing conducted with the device, including a description of the patient population and clinical environment under which it was evaluated.
0
August 3, 2020
Image /page/0/Picture/1 description: The image shows the logo of the U.S. Food and Drug Administration (FDA). On the left is the Department of Health & Human Services logo. To the right of that is a blue square with the letters "FDA" in white. To the right of the blue square is the text "U.S. FOOD & DRUG ADMINISTRATION" in blue.
Eyenuk, Inc Kaushal Solanki, Ph.D., Chief Executive Officer 5850 Canoga Ave. Suite 250 Los Angeles, California 91367
Re: K200667
Trade/Device Name: EyeArt Regulation Number: 21 CFR 886.1100 Regulation Name: Retinal diagnostic software device Regulatory Class: Class II Product Code: PIB Dated: June 30, 2020 Received: June 30, 2020
Dear Dr. Solanki:
We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database located at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part
1
801); medical device reporting of medical device-related adverse events) (21 CFR 803) for devices or postmarketing safety reporting (21 CFR 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR 4. Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR 1000-1050.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.
For comprehensive regulatory information about mediation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely,
Elvin Ng Assistant Director DHT1A: Division of Ophthalmic Devices OHT1: Office of Ophthalmic, Anesthesia, Respiratory, ENT and Dental Devices Office of Product Evaluation and Quality Center for Devices and Radiological Health
Enclosure
2
Indications for Use
510(k) Number (if known) K200667
Device Name EyeArt
Indications for Use (Describe)
EyeArt is indicated for use by healthcare providers to automatically detect more than mild diabetic retinopathy and visionthreatening diabetic retinopathy (severe non-proliferative diabetic retinopathy or proliferative diabetic retinopathy and/or diabetic macular edema) in eyes of adults diagnosed with diabetes who have not been previously diagnosed with more than mild diabetic retinopathy. EyeArt is indicated for use with Canon CR-2 Plus AF cameras in both primary care and eye care settings.
| Type of Use (Select one or both, as applicable) | |
|---|---|
| ☒ Prescription Use (Part 21 CFR 801 Subpart D) | ☐ Over-The-Counter Use (21 CFR 801 Subpart C) |
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
3
510(k) SUMMARY
Eyenuk's EyeArt
Submitter
Eyenuk, Inc. 5850 Canoga Ave., Suite 250 Los Angeles, CA, 91367
Phone: +1 (818) 835-3585
Contact Person: Kaushal Solanki
Date Prepared: August 3, 2020
Name of Device: EyeArt
Classification Name: Retinal diagnostic software device
Regulatory Class: Class II
Regulation: 21 CFR 886.1100
Product Code: PIB
Legally Marketed Predicate Device
Trade name of the device: IDx-DR
Manufacturer's Name: IDx LLC
De Novo Number: DEN180001
A. Intended Use / Indications for Use
EyeArt is indicated for use by healthcare providers to automatically detect more than mild diabetic retinopathy and vision-threatening diabetic retinopathy (severe nonproliferative diabetic retinopathy or proliferative diabetic retinopathy and/or diabetic macular edema) in eyes of adults diagnosed with diabetes who have not been previously diagnosed with more than mild diabetic retinopathy. EyeArt is indicated for use with Canon CR-2 AF and Canon CR-2 Plus AF cameras in both primary care and eye care settings.
4
B. Device Description
EyeArt is a software as a medical device that consists of several components - Client, Server, and Analysis Computation Engine - as presented in Figure 1 below.
Image /page/4/Figure/3 description: This image shows a diagram of the EyeArt system. The diagram is divided into two sections: "Local to the user" and "Remote". On the left side, a patient is shown with a camera that captures fundus images. The images are then sent to the EyeArt Client on the user's computer. The EyeArt Client then sends the images to the EyeArt Server on a secure virtual private network, where the EyeArt Analysis Computation Engine processes the images and sends the results back to the EyeArt Client.
Figure 1: EyeArt components: Client, Server, and Analysis Compute Engine.
A retinal fundus camera, used to capture retinal fundus images of the patient, is connected to a computer where the EyeArt Client software is installed. The EyeArt Client software provides a graphical user interface (GUI) that allows the EyeArt operator to transfer the appropriate fundus images to and receive results from the remote EyeArt Analysis Computation Engine through the EyeArt Server. The EyeArt Analysis Computation Engine is installed on remote computer(s) in a secure data center and uses artificial intelligence algorithms to analyze the fundus images and return results. EyeArt is intended to be used with color fundus images of resolution 1.69 megapixels or higher captured using one of the indicated color fundus cameras (Canon CR-2 AF and Canon CR-2 Plus AF) with 45 degrees field of view. EyeArt is specified for use with two color fundus images per eye: optic nerve head (ONH) centered and macula centered.
For each patient eye, the EyeArt results separately indicate whether more than mild diabetic retinopathy (mtmDR) and vision-threatening diabetic retinopathy (vtDR) are detected. More than mild diabetic retinopathy is defined as the presence of moderate non-proliferative diabetic retinopathy or worse on the International Clinical Diabetic Retinopathy (ICDR) severity scale and/or the presence of diabetic macular edema. Vision-threatening diabetic retinopathy is defined as the presence of severe non-proliferative diabetic retinopathy or proliferative diabetic retinopathy on the ICDR severity scale and/or the presence of diabetic macular edema. Description of EyeArt components is provided below.
- . EyeArt Client: This component is installed on the computer used by the EyeArt operator (working under supervision of a healthcare provider). It allows the operator to transfer images to the EyeArt Analysis Computation Engine and receive results. Its functioning requires an internet connection. If images from a patient encounter cannot be analyzed,
5
due to poor image quality or due to lack of all required image fields, feedback is provided to the operator to help successfully obtain results upon resubmission.
- EyeArt Server: This component provides an interface that securely handles incoming . requests and securely stores user information including images and results. It enables the EyeArt Client to use the EyeArt Analysis Computation Engine through an application programming interface (API).
- EyeArt Analysis Computation Engine: This component analyzes the images to determine . exam quality and detect mtmDR and vtDR. It consists of an ensemble of clinically aligned machine learning (deep learning) algorithms.
C. Non-clinical Testing
EyeArt (software version v2.1.0) was identified as having a major level of concern as defined in the FDA guidance document "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices."
Verification and validation activities at unit, integration, and system level were performed. In all instances, EyeArt functioned as intended and results observed were as expected (i.e. all specifications were met).
Comprehensive risk analysis has been conducted for EyeArt with identification and detailed characterization of the hazards including their causes and severity. Adequate risk control measures have been designed and implemented to mitigate all identified hazards to acceptable levels.
EyeArt also implements comprehensive cybersecurity measures for data confidentiality, data integrity, and data and service availability. Designed to meet industry standard cybersecurity best practices, EyeArt ensures that data remains secure (with encryption during transit and at rest) and private (with authentication and authorization protocols enabling access).
EyeArt has been designed to provide results that are aligned with the clinical practice recommendations for the ophthalmic care of patients with diabetes and has been developed in a clinically aligned framework.
A change protocol was also submitted, to allow for updates and improvements to EyeArt while ensuring that the changes do not introduce risks that adversely affect the safety and effectiveness of the device for its intended use.
D. Clinical Testing
1. Overview
EyeArt was validated in a prospective, multi-center pivotal clinical trial (ClinicalTrials.gov ID NCT03112005). A total of 942 subjects were consented of which 915 subjects met study
6
eligibility criteria. The study was designed to support a De Novo submission since a predicate device did not exist when the EyeArt clinical trial was launched. To better align the analysis population with the proposed intended use population, analyses were presented on 655 participants after excluding subjects who did not meet certain additional prespecified criteria (e.g., subjects 21 years old or younger). The 655 participants were enrolled in the study at 11 US study sites that included primary care centers and general ophthalmology centers. The participants were further divided into two cohorts: one for subjects enrolled during a period of sequential enrollment only (235 subjects, with 45 enrolled at primary care sites and 190 enrolled at ophthalmology sites) and the second enrolled during a period when procedures allowed sequential as well as enriched enrollment (420 subjects, with 335 enrolled at primary care sites and 85 enrolled at ophthalmology sites). The performance of EyeArt was evaluated against a reference standard determined by experienced and certified graders at the Fundus Photography Reading Center (FPRC) per the Early Treatment for Diabetic Retinopathy Study severity (ETDRS) scale on dilated 4-wide field stereo fundus imaging by FPRC certified photographers. The subject disposition is shown in Figure 2 below.
Image /page/6/Figure/2 description: This image shows a flow chart of patient enrollment in a study. The study started with 942 patients, and then 27 failed to meet pre-specified eligibility criteria, 24 were younger than 22 years, and 122 were enrolled at retina sites. 96 had a recorded history of DR, and 18 failed to complete study protocols, resulting in 655 patients in the intent to screen cohort. The cohort was then divided into sequential enrollment (235) and enrichment permitted (420) cohorts, with each cohort further divided into primary care and ophthalmology groups.
Image /page/6/Figure/3 description: The image is a title for a figure. The title reads "Figure 2: Subject disposition and cohorts used for analyses to support substantial equivalence."
2. Study Design and Methods
The target population of the EyeArt pivotal clinical trial analysis was asymptomatic persons aged 22 years and older who were diagnosed with diabetes and not diagnosed with more than mild diabetic retinopathy. The enrollment was conducted sequentially for the first several months of the study ("sequential enrollment" period). In order to increase the likelihood of enrolling patients with more severe levels of diabetic retinopathy (DR), for a portion of the study, sites were permitted to preferentially enroll subjects who met specific eligibility criteria based on
7
duration of diabetes, insulin dependence (for subjects with type-2 diabetes), or hemoglobin A1C (HbA1c) levels. During this period ("enrichment-permitted" period), sites could also enroll scheduled subjects sequentially. In other words, during this period, enrichment was permitted but not required.
During the study visit, subjects underwent 2-field retinal photography by a trained operator for processing by EyeArt. Subjects then underwent dilated 4-wide field stereo retinal photography by an FPRC certified operator for the clinical reference standard. The dilated four wide-field stereo-image sets were independently reviewed and graded by certified graders at FPRC, to determine the severity of diabetic retinopathy (DR) and diabetic macular edema (DME) according to the ETDRS severity scale. Each subject's images were graded independently by 2 experienced and certified graders and in case of significant differences (determined using prespecified significance levels) in the 2 independent gradings, a more experienced adjudication grader graded the same images. Throughout the study, the graders and study staff at the FPRC were masked to the patient history and EyeArt results. The FPRC grading was used to determine the clinical reference standard for subject eyes as follows for the analyses:
- Clinical reference standard for more than mild DR (mtmDR) ●
- o positive if ETDRS level was 35 or greater (but not equal to 90) or clinically significant macular edema [CSME] grade was CSME present
- negative if ETDRS levels were 10-20 and CSME grade was CSME absent o
- ungradable if ETDRS level was 90, or CSME grade was cannot grade or O questionable with ETDRS level 10-20
- Clinical reference standard for vision-threatening DR (vtDR) ●
- o positive if ETDRS level was 53 or greater (but not equal to 90) or CSME grade was CSME present
- negative if ETDRS levels were 10-47 and CSME grade was CSME absent o
- ungradable if ETDRS level was 90, or CSME grade was cannot grade or O questionable with ETDRS level 10-47
EyeArt results for each eye included detection results for more than mild DR (mtmDR) and vision-threatening DR (vtDR) and each of the mtmDR and vtDR results were negative or positive, or when sufficient-quality images were not present, ungradable. The EyeArt mtmDR and vtDR results for each eye were compared to the mtmDR and vtDR clinical reference standard based on the FPRC grading.
3. Study Population and Demographics
In Table 1, demographic and diabetes characteristics of the analysis population are presented stratified by the 4 subgroups defined in Figure 2.
8
| Characteristics | Sequential enrollment cohort | Enrichment permitted cohort | |||
|---|---|---|---|---|---|
| Primary care | |||||
| N=45 subjects | Ophthalmology | ||||
| N=190 subjects | Primary care | ||||
| N=335 subjects | Ophthalmology | ||||
| N=85 subjects | |||||
| Age in | mean (Std. Dev) | 51.9 (10.0) | 60.5 (11.0) | 51.5 (16.1) | 60.0 (10.3) |
| years | median | 52 | 61.5 | 54 | 60 |
| range | 27.0 - 75.0 | 27.0 - 88.0 | 22.0 - 86.0 | 33.0 - 83.0 | |
| HbA1c | mean (Std. Dev) | 9.2 (2.2) | 7.0 (1.5) | 7.8 (1.7) | 7.5 (1.6) |
| median | 9.1 | 6.8 | 7.5 | 7.2 | |
| range | 6.1 - 13.1 | 5.0 - 14.0 | 5.1 - 15.3 | 5.7 - 13.0 | |
| Diabetes | mean (Std. Dev) | 7.9 (7.6) | 11.3 (10.4) | 14.4 (10.9) | 15.7 (8.9) |
| duration | |||||
| in years | median | 5 | 8 | 13 | 13 |
| range | 0.0 - 34.0 | 0.0 - 50.0 | 0.0 - 51.0 | 0.0 - 48.0 | |
| Female % (count) | 44.4% (20) | 61.6% (117) | 44.5% (149) | 70.6% (60) | |
| Race % | |||||
| (count) | American Indian | ||||
| or Alaska Native | 0.0% (0) | 0.0% (0) | 0.3% (1) | 2.4% (2) | |
| Asian | 8.9% (4) | 2.6% (5) | 1.2% (4) | 0.0% (0) | |
| Black or African- | |||||
| American | 4.4% (2) | 20.0% (38) | 14.9% (50) | 14.1% (12) | |
| Native Hawaiian | |||||
| or Pacific Islander | 0.0% (0) | 0.5% (1) | 0.0% (0) | 0.0% (0) | |
| Other | 0.0% (0) | 6.3% (12) | 1.5% (5) | 8.2% (7) | |
| White | 86.7% (39) | 70.5% (134) | 82.1% (275) | 75.3% (64) | |
| Ethnicity | |||||
| % | Non- | ||||
| Hispanic/Latino | 17.8% (8) | 80.0% (152) | 86.6% (290) | 78.8% (67) | |
| (count) | Hispanic/Latino | 82.2% (37) | 20.0% (38) | 13.4% (45) | 21.2% (18) |
| Diabetes | |||||
| type % | Type 1 | 13.3% (6) | 3.7% (7) | 36.7% (123) | 3.5% (3) |
| (count) | Type 2 | 86.7% (39) | 96.3% (183) | 63.3% (212) | 96.5% (82) |
Table 1: Demographic and diabetes characteristics of the study population.
4. Summary of Clinical Study Results
The primary outcomes were the sensitivity and specificity of EyeArt for detecting mtmDR and vtDR in subject eyes. The performance measures are separately reported for each cohort.
a) EyeArt performance for detecting more than mild DR (mtmDR) and visionthreatening DR (vtDR) in the sequentially enrolled cohort
In Table 2, the key performance measures are summarized for the sequentially enrolled cohorts at primary care and ophthalmology sites. Contingency tables and additional performance measures for sequential enrollment cohorts at primary care sites and ophthalmology sites are
9
presented in the Appendix in Table 6 and Table 7 for EyeArt mtmDR and vtDR outputs respectively.
| Sequentially enrolled cohort | ||||
|---|---|---|---|---|
| EyeArt mtmDR output | EyeArt vtDR output | |||
| Primary care | ||||
| (N=90 eyes) | Ophthalmology | |||
| (N=380 eyes) | Primary care | |||
| (N=90 eyes) | Ophthalmology | |||
| (N=380 eyes) | ||||
| Sensitivity | 100.0% | |||
| [74.1% - 100%]† | ||||
| (11/11) | 94.9% | |||
| [86.4% - 100.0%] | ||||
| (37/39) | 100.0% | |||
| [51.0% - 100%]† | ||||
| (4/4) | 88.9% | |||
| NA | ||||
| (8/9) | ||||
| Specificity | 92.0% | |||
| [85.1% - 97.5%] | ||||
| (69/75) | 86.7% | |||
| [82.1% - 90.7%] | ||||
| (281/324) | 97.5% | |||
| [93.4% - 100.0%] | ||||
| (77/79) | 93.8% | |||
| [90.4% - 96.6%] | ||||
| (331/353) | ||||
| Imageability | 96.6% | |||
| [90.9% - 100.0%] | ||||
| (86/89) | 98.6% | |||
| [97.0% - 99.7%] | ||||
| (363/368) | 96.5% | |||
| [90.6% - 100.0%] | ||||
| (83/86) | 98.6% | |||
| [97.0% - 99.7%] | ||||
| (362/367) | ||||
| Positive | ||||
| predictive value | ||||
| (PPV) | 64.7% | |||
| [40.0% - 85.7%] | ||||
| (11/17) | 46.2% | |||
| [32.2% - 59.0%] | ||||
| (37/80) | 66.7% | |||
| NA | ||||
| (4/6) | 26.7% | |||
| [11.1% - 44.4%] | ||||
| (8/30) | ||||
| Negative | ||||
| predictive value | ||||
| (NPV) | 100.0% | |||
| [94.7% - 100%]† | ||||
| (69/69) | 99.3% | |||
| [98.2% - 100.0%] | ||||
| (281/283) | 100.0% | |||
| [95.2% - 100%]† | ||||
| (77/77) | 99.7% | |||
| [99.1% - 100.0%] | ||||
| (331/332) | ||||
| Disease | ||||
| prevalence | 12.2% | |||
| [4.4% - 20.0%] | ||||
| (11/90) | 10.5% | |||
| [6.6% - 15.0%] | ||||
| (40/380) | 4.4% | |||
| [0.0% - 11.1%] | ||||
| (4/90) | 2.4% | |||
| [1.0% - 4.2%] | ||||
| (9/380) | ||||
| All the 95% confidence intervals (CIs) are computed using the clustered bootstrap method that takes into consideration the |
Table 2: Summary of EyeArt performance for sequentially enrolled cohorts at primary care and ophthalmology sites.
correlation between eyes of the same subject. "NA" indicates instances when this clue to small sample sizes. TFor cases with proportion of 100%, the 95% confidence intervals using clustered bootstrap are [100% ], hence the Wilson method is used, which however is not designed to consider eye correlation.
EyeArt performance for detecting more than mild DR (mtmDR) and visionb) threatening DR (vtDR) in the enrichment-permitted cohort
InTable , the key performance measures are summarized for the enrichment-permitted cohorts at primary care and ophthalmology sites. Contingency tables and additional performance measures for enrichment-permitted cohorts at primary care sites and ophthalmology sites are presented in the Appendix in Table 8 and Table 9 respectively.
10
| Enrichment-permitted cohort | |||||
|---|---|---|---|---|---|
| EyeArt mtmDR output | EyeArt vtDR output | ||||
| Primary care | Ophthalmology | Primary care | Ophthalmology | ||
| (N=670 eyes) | (N=170 eyes) | (N=670 eyes) | (N=170 eyes) | ||
| Sensitivity | 92.9% | 96.6% | 91.7% | 100.0% | |
| [87.1% - 97.5%] | [87.5% - 100.0%] | [80.0% - 100.0%] | [51.0% - 100%]† | ||
| (92/99) | (28/29) | (22/24) | (4/4) | ||
| Specificity | 85.6% | 85.2% | 92.2% | 89.8% | |
| [82.2% - 89.1%] | [78.1% - 91.5%] | [89.6% - 94.6%] | [83.9% - 95.4%] | ||
| (457/534) | (115/135) | (553/600) | (141/157) | ||
| Imageability | 96.8% | 96.5% | 96.7% | 97.0% | |
| [94.8% - 98.5%] | [91.8% - 100.0%] | [94.8% - 98.5%] | [92.9% - 100.0%] | ||
| (633/654) | (164/170) | (624/645) | (161/166) | ||
| Positive | 54.4% | 58.3% | 31.9% | 20.0% | |
| predictive value | [45.3% - 63.6%] | [40.3% - 74.5%] | [19.7% - 44.4%] | [0.0% - 42.1%] | |
| (PPV) | (92/169) | (28/48) | (22/69) | (4/20) | |
| Negative | 98.5% | 99.1% | 99.6% | 100.0% | |
| predictive value | [97.3% - 99.5%] | [97.2% - 100.0%] | [99.1% - 100.0%] | [97.3% - 100%]† | |
| (NPV) | (457/464) | (115/116) | (553/555) | (141/141) | |
| Disease | |||||
| prevalence | 15.5% | ||||
| [12.1% - 19.3%] | |||||
| (104/670) | 19.4% | ||||
| [11.8% - 27.6%] | |||||
| (33/170) | 4.2% | ||||
| [2.4% - 6.3%] | |||||
| (28/670) | 2.4% | ||||
| [0.0% - 5.9%] | |||||
| (4/170) |
Table 3: Summary of EyeArt performance at primary care and ophthalmology cohorts when enrichment was permitted.
All the 95% confidence intervals (Cls) are computed using the clusterap method that takes into consideration the correlation between eyes of the same subject. "NA" indicates instances when this CI method fails due to small sample sizes. TFor cases with proportion of 100%, the 95% confidence intervals using clustered bootstrap are [100% - 100%], hence the Wilson method is used, which however is not designed to consider eye correlation.
Imageability c)
EyeArt disease results (positive or negative) were obtained for a vast majority of eyes (imageability of 96.5% or greater) that received a completed FPRC grading.
- Sequential enrollment cohort at primary care sites had an imageability of 96.5% or greater, where gradable results were obtained in the first attempt without dilation in (81/86) 94.2% eves.
- Sequential enrollment cohort at ophthalmology sites had an imageability of 98.6%, where gradable results were obtained in the first attempt without dilation in (300/367) 81.7% eyes.
- Enrichment-permitted cohort at primary care sites had an imageability of 96.7% or greater, ● where gradable results were obtained in the first attempt without dilation in (582/654) 89.0% eyes.
11
- . Enrichment-permitted cohort at ophthalmology sites had an imageability of 96.5% or greater, where gradable results were obtained in the first attempt without dilation in (142/170) 83.5% eyes.
5. Precision Study
Eyenuk conducted a separate precision (repeatability and reproducibility) study on 62 subjects at 2 US primary care sites (31 subjects each) that evaluated the EyeArt results when retinal photography was repeated using the Canon CR-2 AF camera with different operator-camera pairs. Repeatability (intra-operator variability) and reproducibility (inter-operator variability) were specifically analyzed using data from Cohorts P1 and P2 from the study as described in the following sections. All the 95% Cls reported here are computed using the clustered bootstrap method taking into consideration the correlation between eyes.
a) Repeatability (Intra-operator variability) analysis
For subjects at the first site, Cohort P1, there were 3 operator-camera pairings consisting of 3 different operators using 2 different camera units of the same model (each operator operated a given camera unit). Each eye of each subject was to undergo imaging by each operator-camera pair twice for a total of 186 pairs of images (where one “image” = a disc-centered image and a macula-centered image for one eye). The order of camera and operator pairings was randomized. Data for 6 eyes of 3 subjects were missing, because the first images of each eye were not obtained according to the protocol (dilate and repeat imaging, if the initial result was ungradable). For subjects at the second site. Cohort P2, each eye of each subject was to undergo imaging 3 times by a single operator using a single camera.
The observed intra-operator overall agreement (OA) for Cohort P1 was 93.9% [95% CI: 89.7% -97.7%] for the EyeArt mtmDR output and 98.9% [95% CI: 96.5% - 100.0%] for the EyeArt vtDR output. The observed intra-operator OA for Cohort P2 was 93.5% [95% CI: 87.1% -98.4%] for the EyeArt mtmDR output and 96.8% [95% CI: 91.9% - 100.0%] for the EyeArt vtDR output. Contingency tables and additional performance measures are reported in Table 10 -Table 12 in the Appendix.
b) Reproducibility (Inter-operator variability) analysis
The reproducibility was measured in Cohort P1 by analyzing the agreement among the EyeArt results for each eye from the first of two Canon CR-2 AF images obtained by each of the 3 operators. Because the first images of 6 eyes of 3 subjects were missing due to protocol deviations, these were replaced with the second image in each case. For the EyeArt mtmDR output, an inter-operator OA of 90.3% [95% CI: 82.3% - 96.8%] was achieved and for the EyeArt vtDR output, an inter-operator OA of 96.8% [95% CI: 90.3% - 100.0%] was achieved. Contingency tables and additional performance measures are reported in the Appendix in Table 13 for the mtmDR output and in Table 14 for the vtDR output.
12
6. Human Factors Validation Testing
The human factors data support the safety and effectiveness of the camera operation and use of the EyeArt Client user interface. The human factors report contained information from a formative study and a validation study using EyeArt v2.1.0 rev005. A review of detailed human factors engineering processes, including use-related risk impact assessment of the device modification referred to as EyeArt v2.1.0 rev006, indicated that no new critical tasks were introduced or existing critical tasks in v2.1.0 rev005 were impacted by the device modification. The formative study was conducted to confirm the data in the task analysis and risk assessment and to uncover any additional potential use errors. The critical task for using EyeArt is the ability to capture four images of sufficient quality to produce EyeArt gradable results. The results of the simulated-use human factors validation study for using EyeArt with the Canon CR-2 AF camera indicate that camera operators with no prior retinal photography experience can be trained to capture four medical-grade retinal images from a vast majority of subjects.
E. Substantial Equivalence
EyeArt has the same intended use and similar indications for use (IFU) as the predicate IDx-DR device. Table 4 provides a comparison between the IFU of EyeArt and that of the predicate device. The differences in the IFU do not alter the intended use.
| EyeArt (subject device, K200667) | Predicate Device (IDx-DR DEN180001) | Discussion |
|---|---|---|
| EyeArt is indicated for use by healthcare providers to automatically detect more than mild diabetic retinopathy and vision-threatening diabetic retinopathy (severe non-proliferative diabetic retinopathy or proliferative diabetic retinopathy and/or diabetic macular edema) | IDx-DR is intended for use by health care providers to automatically detect more than mild diabetic retinopathy (mtmDR) | Substantially equivalent |
| in eyes of adults diagnosed with diabetes who have not been previously diagnosed with more than mild diabetic retinopathy. | in adults diagnosed with diabetes who have not been previously diagnosed with diabetic retinopathy. | |
| EyeArt is indicated for use with Canon CR-2 AF and Canon CR-2 Plus AF cameras in both primary care and eye care settings. | IDx-DR is indicated for use with the Topcon NW400. |
Table 4: Comparison of indications for use of the EyeArt device and the predicate device
As summarized in Table 5, the subject and predicate devices have similar technological characteristics. Although the specific algorithms differ, these differences do not raise new issues of safety and effectiveness as compared to the predicate and are supported by the clinical performance data.
13
| | EyeArt
(subject device,
K200667) | Predicate device
(IDx-DR,
DEN180001) | Discussion |
|----------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Technological
principle | Artificial Intelligence
software as a medical
device | Artificial Intelligence
software as a medical
device | Equivalent |
| Inputs | Macula and disc
centered color
fundus images with
45° field of view, 2
per eye | Macula and disc
centered color
fundus images with
45° field of view, 2
per eye | Equivalent |
| Outputs | For each eye:
More than mild
diabetic retinopathy
(mtmDR): one of
negative for mtmDR,
mtmDR detected, or
ungradable.
Vision-threatening
diabetic retinopathy
(vtDR): one of
negative for vtDR,
vtDR detected, or
ungradable. | For the patient:
More than mild
diabetic retinopathy
(mtmDR): one of
mtmDR not detected,
mtmDR detected, or
insufficient quality | Equivalent mtmDR outputs, since this
refers to the International Clinical
Diabetic Retinopathy (ICDR) and
Diabetic Macular Edema Disease
Severity Scales' definitions of
moderate non-proliferative diabetic
retinopathy or worst and/or the
presence of diabetic macular edema
(DME).
EyeArt eye-level output (rather than
patient-level output by the predicate)
does not alter intended use and is
supported by clinical performance
data.
Additional vision-threatening DR
detection does not alter intended use
and is supported by clinical
performance data. |
| Architecture | Client software (user
facing) transfers
images to and
receives results from
Analysis
Computation Engine
through Server. | Client software (user
facing) transfers
images to and
receives results from
Analysis through
Web Server. | Equivalent |
| Indicated
Cameras | Canon CR-2 AF and
Canon CR-2 Plus AF | Topcon NW400
camera | The legally marketed cameras
specified for use are being used to
capture macula and disc centered
retinal images with 45° field of view
(2 per eye) for both EyeArt and
predicate device. The clinical
performance data support the use of
EyeArt with the indicated cameras. |
Table 5: Comparison of the technological elements of the EyeArt device and the predicate device.
14
F. Conclusions
EyeArt is substantially equivalent to the predicate device, IDx-DR. EyeArt has the same intended use and equivalent indications for use. The technological characteristics are similar. The technological differences between EyeArt and its predicate device raise no new issues of safety or effectiveness. Performance data support the substantial equivalence of EyeArt to the predicate device.
15
G. Appendix
1. Detailed results from the EyeArt pivotal study
In this section we present detailed results from the pivotal study. We present contingency tables along with other quantities for the following cohorts: (1) sequentially enrolled cohort at primary care sites (mtmDR - Table 6; vtDR - Table 7), (2) sequentially enrolled cohort at ophthalmology sites (mtmDR - Table 6; vtDR - Table 7), (3) enrichment-permitted cohort at primary care sites (mtmDR - Table 8; vtDR - Table 9), and (4) enrichment-permitted cohort at ophthalmology sites (mtmDR - Table 8; vtDR - Table 9). The parameters that are presented in Table 6 to Table 9 are defined below:
| Reading center reference standard | ||||
|---|---|---|---|---|
| Positive | Negative | Ungradable | Total | |
| EyeArt Positive | PP | PN | PU | $DP=PP+PN+PU$ |
| EyeArt Negative | NP | NN | NU | $DN=NP+NN+NU$ |
| EyeArt Ungradable | UP | UN | UU | $DU=UP+UN+UU$ |
| Total | $TP = PP+NP+UP$ | $TN = PN+NN+UN$ | $TU=PU+NU+UN$ | $N=TP+TN+TU$ |
Sensitivity: PP/(PP+NP)
Specificity: NN/(NN+PN)
Imageability: (PP+NN+PN+NP)/(TP+TN)
Positive predictive value (PPV): PP/(PP+PN)
Negative predictive value (NPV): NN/(NN+NP)
Positive likelihood ratio: (PP/TP)/(PN/TN)
Negative likelihood ratio: (NP/TP)/(NN/TN)
Sensitivity considering ungradables: PP/(PP+NP+UP)
Specificity considering ungradables: NN/(PN+NN+UN)
PPV considering ungradables: PP/(PP+PN+PU)
NPV considering ungradables: NN/(NN+NP+NU)
Disease prevalence: TP/N, Percentage of eyes identified as positive per reference standard.
16
| Sequential enrollment cohort | | | Primary care
(45 subjects, 90 eyes) | | Ophthalmology
(190 subjects, 380 eyes) | | | |
|--------------------------------------------------|--------------------------------------------|----------------------------------------|--------------------------------------------|-------------------------------------------|--------------------------------------------|-----|-----------------------------------|-----|
| mtmDR | mtmDR Reading center
reference standard | | | | mtmDR Reading center
reference standard | | | |
| | Pos | Neg | Ung | Tot | Pos | Neg | Ung | Tot |
| EyeArt Positive | 11 | 6 | 0 | 17 | 37 | 43 | 1 | 81 |
| EyeArt Negative | 0 | 69 | 0 | 69 | 2 | 281 | 8 | 291 |
| EyeArt Ungradable | 0 | 3 | 1 | 4 | 1 | 4 | 3 | 8 |
| Total | 11 | 78 | 1 | 90 | 40 | 328 | 12 | 380 |
| Sensitivity | | | 100.0% [74.1% - 100%]†
(11/11) | | 94.9% [86.4% - 100.0%]
(37/39) | | | |
| Specificity | | | 92.0% [85.1% - 97.5%]
(69/75) | | 86.7% [82.1% - 90.7%]
(281/324) | | | |
| Imageability | | | 96.6% [90.9% - 100.0%]
(86/89) | | 98.6% [97.0% - 99.7%]
(363/368) | | | |
| Positive Predictive Value (PPV) | | | 64.7% [40.0% - 85.7%]
(11/17) | | 46.2% [32.2% - 59.0%]
(37/80) | | | |
| Negative Predictive Value (NPV) | | | 100.0% [94.7% - 100%]†
(69/69) | | 99.3% [98.2% - 100.0%]
(281/283) | | | |
| Positive Likelihood Ratio | | | 13 [7.000 - 41.012] | | 7.056 [5.118 - 10.184] | | | |
| Negative Likelihood Ratio | | | 0 [0.0 - 0.0] | | 0.058 [0.000 - 0.155] | | | |
| Sensitivity considering
ungradables | | | 100% [74.1% - 100%]†
(11/11) | | 92.5% [82.6% - 100.0%]
(37/40) | | | |
| Specificity considering
ungradables | | | 88.5% [80.0% - 95.8%]
(69/78) | | 85.7% [80.9% - 89.7%]
(281/328) | | | |
| PPV considering ungradables | | | 64.7% [40.0% - 85.7%]
(11/17) | | 45.7% [31.8% - 58.3%]
(37/81) | | | |
| NPV considering ungradables | | | 100.0% [94.7% - 100.0%]†
(69/69) | | 96.6% [94.1% - 98.6%]
(281/291) | | | |
| Disease (mtmDR) prevalence
in this population | | | 12.2% [4.4% - 20.0%]
(11/90) | | 10.5% [6.6% - 15.0%]
(40/380) | | | |
| Sequential enrollment cohort | | Primary care
(45 subjects, 90 eyes) | | Ophthalmology
(190 subjects, 380 eyes) | | | | |
| vtDR | vtDR Reading center
reference standard | | | | vtDR Reading center
reference standard | | | |
| | Pos | Neg | Ung | Tot | Pos | Neg | Ung | Tot |
| EyeArt Positive | 4 | 2 | 3 | 9 | 8 | 22 | 1 | 31 |
| EyeArt Negative | 0 | 77 | 0 | 77 | 1 | 331 | 9 | 341 |
| EyeArt Ungradable | 0 | 3 | 1 | 4 | 0 | 5 | 3 | 8 |
| Total | 4 | 82 | 4 | 90 | 9 | 358 | 13 | 380 |
| Sensitivity | | 100.0% [51.0% - 100%]†
(4/4) | | | 88.9% NA
(8/9) | | | |
| Specificity | 97.5% [93.4% - 100.0%]
(77/79) | | | | 93.8% [90.4% - 96.6%]
(331/353) | | | |
| Imageability | 96.5% [90.6% - 100.0%]
(83/86) | | | | 98.6% [97.0% - 99.7%]
(362/367) | | | |
| Positive Predictive Value (PPV) | 66.7% NA
(4/6) | | | | 26.7% [11.1% - 44.4%]
(8/30) | | | |
| Negative Predictive Value (NPV) | | 100.0% [95.2% - 100%]†
(77/77) | | | 99.7% [99.1% - 100.0%]
(331/332) | | | |
| Positive Likelihood Ratio | | 41.0 NA | | | 14.465 [8.595 - 27.426] | | | |
| Negative Likelihood Ratio | | 0.0 [0.000 - 0.000] | | | 0.12 [0.000 - 0.426] | | | |
| Sensitivity considering
ungradables | | 100.0% [51.0% - 100%]†
(4/4) | | | 88.9% [60.0% - 100.0%]
(8/9) | | | |
| Specificity considering
ungradables | | 93.9% [86.5% - 98.8%]
(77/82) | | | 92.5% [88.7% - 95.6%]
(331/358) | | | |
| PPV considering ungradables | | 44.4% NA
(4/9) | | | 25.8% [10.8% - 42.3%]
(8/31) | | | |
| NPV considering ungradables | | 100.0% [95.2% - 100.0%]†
(77/77) | | | 97.1% [95.0% - 98.8%]
(331/341) | | | |
| Disease (vtDR) prevalence
in this population | | 4.4% [0.0% - 11.1%]
(4/90) | | | 2.4% [1.0% - 4.2%]
(9/380) | | | |
| | | | Primary care
(335 subjects, 670 eyes) | | Ophthalmology
(85 subjects, 170 eyes) | | | |
| Enrichment-permitted cohort
mtmDR | | | mtmDR Reading center
reference standard | | mtmDR Reading center
reference standard | | | |
| | Pos | Neg | Ung | Tot | Pos | Neg | Ung | Tot |
| EyeArt Positive | 92 | 77 | 2 | 171 | 28 | 20 | 0 | 48 |
| EyeArt Negative | 7 | 457 | 9 | 473 | 1 | 115 | 0 | 116 |
| EyeArt Ungradable | 5 | 16 | 5 | 26 | 4 | 2 | 0 | 6 |
| Total | 104 | 550 | 16 | 670 | 33 | 137 | 0 | 170 |
| Sensitivity | | | 92.9% [87.1% - 97.5%]
(92/99) | | | | 96.6% [87.5% - 100.0%]
(28/29) | |
| Specificity | 85.6% [82.2% - 89.1%]
(457/534) | | | | 85.2% [78.1% - 91.5%]
(115/135) | | | |
| Imageability | 96.8% [94.8% - 98.5%]
(633/654) | | | | 96.5% [91.8% - 100.0%]
(164/170) | | | |
| Positive Predictive Value (PPV) | 54.4% [45.3% - 63.6%]
(92/169) | | | | 58.3% [40.3% - 74.5%]
(28/48) | | | |
| Negative Predictive Value (NPV) | | | 98.5% [97.3% - 99.5%]
(457/464) | | 99.1% [97.2% - 100.0%]
(115/116) | | | |
| Positive Likelihood Ratio | | | 6.319 [4.988 - 8.473] | | 5.812 [3.736 - 10.182] | | | |
| Negative Likelihood Ratio | | | 0.081 [0.029 - 0.145] | | | | 0.036 [0.000 - 0.131] | |
| Sensitivity considering
ungradables | | | 88.5% [80.5% - 94.9%]
(92/104) | | 84.9% [65.5% - 100.0%]
(28/33) | | | |
| Specificity considering
ungradables | 83.1% [79.4% - 86.8%]
(457/550) | | | | 83.9% [76.6% - 90.6%]
(115/137) | | | |
| PPV considering ungradables | 53.8% [44.6% - 63.1%]
(92/171) | | | | 58.3% [40.3% - 74.5%]
(28/48) | | | |
| NPV considering ungradables | 96.6% [94.5% - 98.4%]
(457/473) | | | | 99.1% [97.2% - 100.0%]
(115/116) | | | |
| Disease (mtmDR) prevalence
in this population | | | 15.5% [12.1% - 19.3%]
(104/670) | | | | 19.4% [11.8% - 27.6%]
(33/170) | |
Table 6: EyeArt eye-level mtmDR analysis for subjects enrolled during sequential enrollment.
All the 95% confidence intervals (CIs) are computed using the clusterap method that takes into consideration the correlation between eyes of the same subject. "NA" indicates instances when this CI method fails due to small sample sizes. f For cases with proportion of 100%, the 95% confidence intervals using clustered bootstrap are [100% - 100%], hence the Wilson method is used, which however is not designed to consider eye correlation.
17
Table 7: EyeArt eye-level vtDR analysis for subjects enrolled during sequential enrollment.
All the 95% confidence intervals (Cls) are computed using the clustered bootstrap method that takes into consideration the correlation between eyes of the same subject. "NA" indicates instances when this CI method fails due to small sample sizes. *For cases with proportion of 100%, the 95% confidence intervals using clustered bootstrap are [100% - 100%], hence the Wilson method is used, which however is not designed to consider eye correlation.
18
Table 8: EyeArt eye-level mtmDR analysis for subjects enrolled when enrichment was permitted.
correlation between eyes of the same subject. "NA" indicates instances when this Cl method fails and small sizes.
19
| Enrichment permitted cohort | | | Primary care
(335 subjects, 670 eyes) | | | | Ophthalmology
(85 subjects, 170 eyes) | | | |
|-------------------------------------------------|------------------------------------|-----|-------------------------------------------|-----|-------------------------------------|-----|-------------------------------------------|-----|--|--|
| vtDR | | | vtDR Reading center
reference standard | | | | vtDR Reading center
reference standard | | | |
| | Pos | Neg | Ung | Tot | Pos | Neg | Ung | Tot | | |
| EyeArt Positive | 22 | 47 | 9 | 78 | 4 | 16 | 1 | 21 | | |
| EyeArt Negative | 2 | 553 | 11 | 566 | 0 | 141 | 2 | 143 | | |
| EyeArt Ungradable | 4 | 17 | 5 | 26 | 0 | 5 | 1 | 6 | | |
| Total | 28 | 617 | 25 | 670 | 4 | 162 | 4 | 170 | | |
| Sensitivity | | | 91.7% [80.0% - 100.0%]
(22/24) | | 100.0% [51.0% - 100%]†
(4/4) | | | | | |
| Specificity | 92.2% [89.6% - 94.6%]
(553/600) | | | | 89.8% [83.9% - 95.4%]
(141/157) | | | | | |
| Imageability | 96.7% [94.8% - 98.5%]
(624/645) | | | | 97.0% [92.9% - 100.0%]
(161/166) | | | | | |
| Positive Predictive Value (PPV) | | | 31.9% [19.7% - 44.4%]
(22/69) | | 20.0% [0.0% - 42.1%]
(4/20) | | | | | |
| Negative Predictive Value (NPV) | | | 99.6% [99.1% - 100.0%]
(553/555) | | 100.0% [97.3% - 100%]†
(141/141) | | | | | |
| Positive Likelihood Ratio | | | 10.315 [6.965 - 15.704] | | 10.125 [6.403 - 22.571] | | | | | |
| Negative Likelihood Ratio | | | 0.08 [0.000 - 0.195] | | 0 [0.0 - 0.0] | | | | | |
| Sensitivity considering
ungradables | 78.6% [58.8% - 95.8%]
(22/28) | | | | 100.0% [51.0% - 100%]†
(4/4) | | | | | |
| Specificity considering
ungradables | | | 89.6% [86.6% - 92.4%]
(553/617) | | 87.0% [80.1% - 93.1%]
(141/162) | | | | | |
| PPV considering ungradables | 28.2% [17.4% - 38.9%]
(22/78) | | | | 19.0% [0.0% - 38.5%]
(4/21) | | | | | |
| NPV considering ungradables | 97.7% [96.0% - 99.1%]
(553/566) | | | | 98.6% [95.7% - 100.0%]
(141/143) | | | | | |
| Disease (vtDR) prevalence
in this population | | | 4.2% [2.4% - 6.3%]
(28/670) | | 2.4% [0.0% - 5.9%]
(4/170) | | | | | |
Table 9: EyeArt eye-level vtDR analysis for subjects enrolled when enrichment was allowed.
All the 95% confidence intervals (Cls) are computed using the clustered bootstrap method that takes into consideration the correlation between eyes of the same subject. "NA" indicates instances when this clue to small sample sizes. *For cases with proportion of 100%, the 95% confidence intervals using clustered bootstrap are [100% - 100%], hence the Wilson method is used, which however is not designed to consider eye correlation.
20
2. Detailed results from the EyeArt precision study
In this section, we present detailed results of the EyeArt precision study including contingency tables and the following parameters, defined below.
| EyeArt result 2
EyeArt result 1 | EyeArt Negative | EyeArt Positive | EyeArt
Ungradable | Total |
|------------------------------------|-----------------|-----------------|----------------------|-------------|
| EyeArt Negative | NN | NP | NU | NT=NN+NP+NU |
| EyeArt Positive | PN | PP | PU | PT=PN+PP+PU |
| EyeArt Ungradable | UN | UP | UU | UT=UN+UP+UU |
| Total | TN = NN+PN+UN | TP = NP+PP+UP | TU=NU+PU+UN | N=TN+TP+TU |
For 3x3 contingency tables, the definitions are as follows:
Overall agreement (OA): (PP+NN+UU) / N
Average Positive Agreement (APA): $(2*PP) / (TP + PT)$
Average Negative Agreement (ANA): $(2*NN) / (TN + NT)$
Average Ungradable Agreement (AUA): $(2*UU) / (TU + UT)$
For 3-dimensional (3x3x3) contingency tables, the definitions are as follows:
Overall agreement (OA) for the 3x3x3 contingency tables is computed as follows, where $x_{ii}$ is the count of triplets of operations with outcomes $(i, j, k)$ where each outcome is one of $n$ (negative), p (positive), and u (ungradable):
$$OA = \left(\chi_{nm} + \chi_{ppp} + \chi_{unu}\right) / \left(\sum_{i,j,k=n,p,u} \chi_{ijk}\right)$$
The percent agreements for the 3x3x3 contingency tables are evaluated as follows, where $R_1, R_2$, and R3 stand for EyeArt output for the three operations 1, 2, and 3 respectively that are used to build the 3x3x3 contingency table:
Average positive agreement (APA) is computed as:
$$APA = \frac{\sum_{1 \le i