Search Results

The Alleye is a mobile medical software application indicated for the detection and characterization of metamorphopsia, a visual distortion, in patients with age-related macular degeneration (AMD) and as an aid in the monitoring of the progression of this condition in respect of metamorphopsia. It is intended to be used by patients who have the capability to regularly perform a simple self-test at home.

Alleye is a digital technology visual function test, consisting of two different items: a mobile app for patients and a web interface for eye care professionals. Alleye implements an alignment hyperacuity task that helps patients with age-related macular degeneration (AMD) to assess their vision at home. This allows the timely detection of significant changes in vision function, enabling the regular monitoring of the disease progression and/or monitoring the visual function associated with ongoing treatments. Two elements provide feedback about the Alleye test: a score and a colored circle. The score value reflects the visual performance in the dots alignment, whereas the color indicates whether the performance has worsened considerably (red) or remained stable or improved (green) compared to the previous test.

The provided text describes the Alleye device, a mobile medical software application indicated for the detection and characterization of metamorphopsia in patients with age-related macular degeneration (AMD) and as an aid in monitoring the progression of this condition.

Here's an analysis of the acceptance criteria and the studies conducted:

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria for Alleye are not explicitly stated as quantitative targets in the provided document. Instead, the document focuses on demonstrating substantial equivalence to a predicate device (myVisionTrack Model 0005) through various clinical and non-clinical studies. The device performance is described in terms of its ability to detect worsening visual function and its reliability.

2. Sample Size Used for the Test Set and Data Provenance

• Test-Retest Study:
  • Sample Size: 26 healthy subjects and 60 subjects with age-related macular degeneration (AMD).
  • Data Provenance: Not explicitly stated, but the study was conducted to evaluate reliability. It implies clinical, prospective data.
• Clinical Evaluation – Longitudinal Study:
  • Sample Size: 60 patients diagnosed with wet AMD who were undergoing pro re nata intravitreal injection (IVI) of anti-VEGF for treatment. They completed at least a 3-month follow-up, providing 1506 Alleye measurements over 150 follow-up periods.
  • Data Provenance: Not explicitly stated, but implies prospective clinical data from patients undergoing treatment.
• Usability Study:
  • Sample Size: Elderly subjects with AMD (specific number not provided, but at least two clinical studies were conducted).
  • Data Provenance: Implies prospective data collection through user interaction and feedback.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts

The document does not explicitly state the number or qualifications of experts used to establish ground truth for the test sets.

• In the Longitudinal Study, the "need of an injection at the next follow-up visit" served as a clinical outcome, which would typically be determined by an ophthalmologist or retina specialist based on their clinical assessment (e.g., OCT scans, visual acuity changes, signs of active disease). However, the specific methodology for establishing this "ground truth" (i.e., whether an injection was truly needed) and the role of experts in that determination is not detailed.
• For the Test-Retest Study, the "reliability of the metamorphopsia testing" itself is the outcome, suggesting a measure of internal consistency within the device's readings rather than comparison against an external expert-determined ground truth for metamorphopsia.

4. Adjudication Method for the Test Set

The document does not describe any specific adjudication method (e.g., 2+1, 3+1) for establishing ground truth within the described studies. Clinical decisions in the longitudinal study would likely follow standard medical practice by the treating ophthalmologists.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance

No MRMC comparative effectiveness study involving human readers with and without AI assistance is described in the provided text. The Alleye device is positioned as a patient self-testing tool for monitoring, not as an AI-assisted diagnostic tool for human readers.

6. If a Standalone (i.e., Algorithm Only Without Human-in-the-Loop Performance) Was Done

Yes, the studies described investigate the standalone performance of the Alleye application (algorithm only) as a patient self-testing tool.

• The Test-Retest Study evaluated the reliability of the Alleye's metamorphopsia testing.
• The Longitudinal Study evaluated the Alleye's ability to detect worsening visual function.
• The Usability Study evaluated the app's user-friendliness.

In all these cases, the "performance" refers to the Alleye device's capabilities without direct human interpretive input during the testing process itself. The interpretation of the Alleye results by eye care professionals is an intended use, but the studies focus on the patient-facing component.

7. The Type of Ground Truth Used (Expert Consensus, Pathology, Outcomes Data, etc.)

• Test-Retest Study: The ground truth for this study was the internal consistency/reproducibility of the Alleye's own measurements over time in the same individuals. It's about how reliably the device detects metamorphopsia, not necessarily how accurately it matches an external ground truth for metamorphopsia presence.
• Clinical Evaluation – Longitudinal Study: The ground truth appears to be clinical outcomes data related to the "need for an injection at the next follow-up visit," which would be a physician's clinical decision based on a comprehensive assessment (e.g., visual acuity, OCT, physical exam). The study shows Alleye's ability to predict or indicate this need, implying these clinical decisions serve as the reference.
• Usability Study: The ground truth for usability was user feedback (oral feedback and System Usability Scale scores) from the target patient population.

8. The Sample Size for the Training Set

The document does not provide information about the sample size used for the training set of the Alleye algorithm. As a Predicate device (myVisionTrack) is mentioned and the focus is on substantial equivalence, it's possible that the "algorithm" for hyperacuity testing is based on established principles rather than requiring extensive de novo machine learning training data in the context of this submission. However, if machine learning was used, the training data details are not disclosed here.

9. How the Ground Truth for the Training Set Was Established

Since information about a specific training set and its sample size is not provided, the method for establishing its ground truth is also not detailed in this document.

Ask a specific question about this device

The my VisionTrack® Model 0005 is intended for the detection of central 3 degrees metamorphopsia (visual distortion) in patients with maculopathy, including age-related macular degeneration and diabetic retinopathy, and as an aid in monitoring progression of disease factors causing metamorphopsia. It is intended to be used by patients who have the capability to regularly perform a simple self-test at home. The myVisionTrack® Model 0005 is not intended to diagnose; diagnosis is the responsibility of the prescribing eye-care professional.

The myVisionTrack® Model 0005 is a vision function test provided as a downloadable app on to the user's supplied cell phone or tablet. The myVisionTrack® Model 0005 implements a shape discrimination hyperacuity (SDH) vision test which allows patients to perform their own vision test at home. If a significant worsening of vision function is detected the physician will be notified and provided access to the vision self-test results so that they can decide whether the patient needs to be seen sooner than their next already scheduled appointment.

Here's a breakdown of the acceptance criteria and study information for the myVisionTrack® Model 0005, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

The provided document primarily focuses on demonstrating substantial equivalence to a predicate device (myVisionTrack® Model 0003) rather than defining explicit, quantitative acceptance criteria for the new device. However, based on the comparative study, we can infer the performance expectation.

Ask a specific question about this device

The my Vision TrackTM is intended for the detection of central 3 degrees metamorphopsia (visual distortion) in patients with maculopathy, including agerelated macular degeneration and diabetic retinopathy, and as an aid in monitoring progression of disease factors causing metamorphopsia. It is intended to be used by patients who have the capability to regularly perform a simple self-test at home. The myVisionTrackTM is not intended to diagnose; diagnosis is the responsibility of the prescribing eye-care professional.

The myVisionTrack™ is a vision function test provided on a commercially available cell phone. The myVisionTrack™ implements a shape discrimination hyperacuity (SDH) vision test which allows patients to perform their own vision test at home. This enables regular monitoring of disease progression, and for timely detection of significant changes in vision function. If a significant worsening of vision function is detected the physician will be notified and provided access to the vision self-test results so that they can decide whether the patient needs to be seen sooner than their next already scheduled appointment.

The provided document describes the myVisionTrack™ Model 0003, a device intended for the detection and characterization of central 3 degrees metamorphopsia in patients with maculopathy, including age-related macular degeneration and diabetic retinopathy, and as an aid in monitoring progression of disease factors causing metamorphopsia.

However, the document does not explicitly state specific acceptance criteria (e.g., sensitivity, specificity thresholds) for the device's performance. Instead, it focuses on demonstrating substantial equivalence to predicate devices and verifying that patients can effectively use the device for self-monitoring.

Here's an analysis based on the information provided, highlighting what is available and what is not:

1. Table of Acceptance Criteria and Reported Device Performance

As noted above, no explicit acceptance criteria thresholds (like specific sensitivity or specificity values) are provided in the document. The study's conclusion is that the device is "as safe, as effective and performs at least as safely and effectively as the predicate devices."

The document mentions that the study "did show a significant difference between those patients with mild-to-moderate non-proliferative DR (NPDR) and those with very severe NPDR or proliferative DR (PDR), whereas traditional clinic-based visual acuity and contrast sensitivity tests were not able to detect a significant difference." This implies a performance benefit in detecting disease severity, but no specific metrics or acceptance criteria are given for this.

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size for Test Set: 36 individuals.
• Data Provenance: The study was a "6-month Clinical Study" and "6-month longitudinal study" performed by VAS (Vital Art and Science Incorporated). The document does not explicitly state the country of origin but implies it was conducted by the submitter (Vital Art and Science Incorporated, Richardson, TX, USA). It was a prospective longitudinal study.

3. Number of Experts Used to Establish Ground Truth for the Test Set and Their Qualifications

• Number of Experts: Not explicitly stated.
• Qualifications of Experts: Not explicitly stated. The ground truth was based on "clinical judgment" and "traditional clinic-based visual acuity and contrast sensitivity tests" for assessing disease condition (NPDR vs. PDR). It is reasonable to assume these judgments were made by qualified ophthalmologists or eye care professionals, but specific numbers and qualifications are not detailed.

4. Adjudication Method for the Test Set

• Adjudication Method: Not explicitly stated. The document mentions "clinical judgment" for assessing the disease condition, but how multiple experts (if any) arrived at a consensus is not described.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• MRMC Study: No, a multi-reader multi-case (MRMC) comparative effectiveness study was not explicitly described or conducted in the context of human readers improving with or without AI assistance. The study focused on the device's ability to monitor changes and patient compliance.

6. Standalone Performance (Algorithm Only without Human-in-the-loop Performance)

• Standalone Performance: Yes, the described study appears to be a standalone performance study. The myVisionTrack™ device "implements a shape discrimination hyperacuity (SDH) vision test which allows patients to perform their own vision test at home." The results are then analyzed by the device's algorithm, and "If a significant worsening of vision function is detected the physician will be notified." The study tested "effective self-monitoring... using myVisionTrack™" and demonstrated its ability to detect differences in disease states and generate notifications based on a "0.2 logMAR notification rule." This rule is an algorithmic threshold applied to the device's output.

7. Type of Ground Truth Used

• Type of Ground Truth: The ground truth for defining "significant change of disease condition" was based on clinical judgment by medical professionals and comparisons to traditional clinic-based visual acuity and contrast sensitivity tests.

8. Sample Size for the Training Set

• Sample Size for Training Set: The document does not specify a separate training set or its sample size. The description of the clinical study appears to relate to the validation of the device's performance, not necessarily the training of its algorithm (which is a "shape discrimination hyperacuity test"). The algorithms are described as using an "adaptive staircase algorithm," which is a testing methodology, not typically a machine learning training process that requires a training set in the conventional sense.

9. How the Ground Truth for the Training Set Was Established

• Establishment of Ground Truth for Training Set: Not applicable, as a distinct training set for a machine learning algorithm is not described. The "shape discrimination hyperacuity test" is a known psychophysical testing method. The "adaptive staircase algorithm" is a method for efficiently finding thresholds. The document states "Numerous published studies have shown that patients with AMD and other forms of maculopathy have significantly poorer results as compared to normal subjects on the shape discrimination test," indicating that the underlying principle is well-established in scientific literature.

Ask a specific question about this device

The Notal Vision Macular Computerized Psychophysical Test (MCPT) is indicated for The early detection of central and paracentral irregularities (vision abnormalities) in the visual field, most commonly associated with macular degeneration.

The device is a computerized interactive software device that provides the user with a series of images oriented in vertical and horizontal planes on a imaging screen that is designed to identify irregularities in the central and paracentral macular field of the human visual system. The device analyzes the image presentation in the process of identifying visual field abnormalities and stores the analysis in a computer server. The device is housed in a PC computer.

The Notal Vision Macular Computerized Psychophysical Test (MCPT) is indicated for the early detection of central and paracentral irregularities (vision abnormalities) in the visual field, most commonly associated with macular degeneration.

1. Table of Acceptance Criteria and Reported Device Performance:

The document doesn't explicitly state quantitative "acceptance criteria" for the MCPT in terms of specific sensitivity or specificity targets. Instead, the study's primary objective was to demonstrate that MCPT is equal to or better than the Amsler grid in detecting AMD-related retinal abnormalities. The reported performance compared the MCPT directly against the Amsler Grid.

Note: While the overall P-value for sensitivity strongly favors MCPT, the P-value for Category 1 (healthy subjects) shows MCPT having significantly lower specificity compared to the Amsler Grid.

2. Sample Size Used for the Test Set and Data Provenance:

• Test Set Sample Size: The document does not explicitly state a total sample size for the test set. However, a breakdown of categories is provided:
  • Category 1 (Healthy): 33 subjects
  • Category 2 (AMD): 51 subjects
  • Category 3 (AMD): 20 subjects
  • Category 4 (AMD): 27 subjects
  • Category 5 (AMD): 19 subjects
  • Total subjects across all categories = 150 subjects.
• Data Provenance: The study was a "prospective single blinded randomized clinical investigation conducted at multiple investigational sites." The document does not specify the country of origin of the data.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts:

The document does not provide information on the number of experts used to establish ground truth or their qualifications. The categories are described as "individuals with defined age-related macular degeneration (AMD) at different stages, and the control subjects were those with normal healthy eyes," implying a clinical diagnosis as ground truth, but not the specific experts or their process.

4. Adjudication Method for the Test Set:

The document does not describe any specific adjudication method for the test set. The categories (AMD at different stages and healthy controls) were "defined," suggesting a pre-established clinical diagnosis, but the process of reaching that definition is not detailed.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:

Yes, a comparative effectiveness study was done. The study compared the MCPT to the Amsler Grid.

• Effect Size of Human Readers Improve with AI vs. without AI assistance: This study compares an automated device (MCPT) against a traditional manual test (Amsler grid), not the improvement of human readers with AI assistance versus without AI assistance. Therefore, an effect size of human improvement cannot be determined from this document as it's not a reader study with AI assistance. The study focuses on the standalone performance of the MCPT and the Amsler grid themselves.

6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study:

Yes, a standalone study was performed. The "MCPT" (Macular Computerized Psychophysical Test) is described as an "interactive software device" that "analyzes the image presentation in the process of identifying visual field abnormalities." The results presented for MCPT are its direct performance, acting as an algorithm-only assessment, which is then compared against the Amsler grid.

7. Type of Ground Truth Used:

The ground truth used was clinical diagnosis/classification of "defined age-related macular degeneration (AMD) at different stages" and "normal healthy eyes." This falls under clinical diagnosis rather than pathology or outcomes data specifically.

8. Sample Size for the Training Set:

The document does not mention a separate training set or its sample size. The clinical investigation described appears to be for validation/testing.

9. How the Ground Truth for the Training Set Was Established:

Since no training set is described, information on how its ground truth was established is not available in the provided document.

Ask a specific question about this device