(83 days)
The Alleye is a mobile medical software application indicated for the detection and characterization of metamorphopsia, a visual distortion, in patients with age-related macular degeneration (AMD) and as an aid in the monitoring of the progression of this condition in respect of metamorphopsia. It is intended to be used by patients who have the capability to regularly perform a simple self-test at home.
Alleye is a digital technology visual function test, consisting of two different items: a mobile app for patients and a web interface for eye care professionals. Alleye implements an alignment hyperacuity task that helps patients with age-related macular degeneration (AMD) to assess their vision at home. This allows the timely detection of significant changes in vision function, enabling the regular monitoring of the disease progression and/or monitoring the visual function associated with ongoing treatments. Two elements provide feedback about the Alleye test: a score and a colored circle. The score value reflects the visual performance in the dots alignment, whereas the color indicates whether the performance has worsened considerably (red) or remained stable or improved (green) compared to the previous test.
The provided text describes the Alleye device, a mobile medical software application indicated for the detection and characterization of metamorphopsia in patients with age-related macular degeneration (AMD) and as an aid in monitoring the progression of this condition.
Here's an analysis of the acceptance criteria and the studies conducted:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria for Alleye are not explicitly stated as quantitative targets in the provided document. Instead, the document focuses on demonstrating substantial equivalence to a predicate device (myVisionTrack Model 0005) through various clinical and non-clinical studies. The device performance is described in terms of its ability to detect worsening visual function and its reliability.
| Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|
| Preamble: Substantially equivalent to predicate device. | Preamble: Alleye is deemed substantially equivalent to K143211 (myVisionTrack) based on similar intended use and technological characteristics (mobile app for hyperacuity self-testing). |
| Functional: Reliability in metamorphopsia testing. | Test-Retest Study: Evaluated the reliability of metamorphopsia testing with Alleye. |
| Clinical Efficacy: Aid in monitoring disease progression/need for injection. | Longitudinal Study: Demonstrated that patient self-testing with Alleye allows the detection of worsening visual function prior to follow-up visits, aiding in monitoring for the need of an injection. |
| Usability: User-friendliness for elderly AMD patients. | Usability Study: Oral feedback and System Usability Scale (SUS) results confirmed user-friendliness for elderly subjects with AMD. |
| Software Quality: Compliance with medical device software standards. | Software Verification: Complied with IEC 62304 Medical Device Software - Software Life Cycle Processes. |
| Safety and Effectiveness: Does not raise new issues. | Conclusion: Based on validation and clinical evaluation, the device does not raise new issues of safety or effectiveness compared to the predicate. It is considered safe and effective for its intended use. |
2. Sample Size Used for the Test Set and Data Provenance
- Test-Retest Study:
- Sample Size: 26 healthy subjects and 60 subjects with age-related macular degeneration (AMD).
- Data Provenance: Not explicitly stated, but the study was conducted to evaluate reliability. It implies clinical, prospective data.
- Clinical Evaluation – Longitudinal Study:
- Sample Size: 60 patients diagnosed with wet AMD who were undergoing pro re nata intravitreal injection (IVI) of anti-VEGF for treatment. They completed at least a 3-month follow-up, providing 1506 Alleye measurements over 150 follow-up periods.
- Data Provenance: Not explicitly stated, but implies prospective clinical data from patients undergoing treatment.
- Usability Study:
- Sample Size: Elderly subjects with AMD (specific number not provided, but at least two clinical studies were conducted).
- Data Provenance: Implies prospective data collection through user interaction and feedback.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The document does not explicitly state the number or qualifications of experts used to establish ground truth for the test sets.
- In the Longitudinal Study, the "need of an injection at the next follow-up visit" served as a clinical outcome, which would typically be determined by an ophthalmologist or retina specialist based on their clinical assessment (e.g., OCT scans, visual acuity changes, signs of active disease). However, the specific methodology for establishing this "ground truth" (i.e., whether an injection was truly needed) and the role of experts in that determination is not detailed.
- For the Test-Retest Study, the "reliability of the metamorphopsia testing" itself is the outcome, suggesting a measure of internal consistency within the device's readings rather than comparison against an external expert-determined ground truth for metamorphopsia.
4. Adjudication Method for the Test Set
The document does not describe any specific adjudication method (e.g., 2+1, 3+1) for establishing ground truth within the described studies. Clinical decisions in the longitudinal study would likely follow standard medical practice by the treating ophthalmologists.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance
No MRMC comparative effectiveness study involving human readers with and without AI assistance is described in the provided text. The Alleye device is positioned as a patient self-testing tool for monitoring, not as an AI-assisted diagnostic tool for human readers.
6. If a Standalone (i.e., Algorithm Only Without Human-in-the-Loop Performance) Was Done
Yes, the studies described investigate the standalone performance of the Alleye application (algorithm only) as a patient self-testing tool.
- The Test-Retest Study evaluated the reliability of the Alleye's metamorphopsia testing.
- The Longitudinal Study evaluated the Alleye's ability to detect worsening visual function.
- The Usability Study evaluated the app's user-friendliness.
In all these cases, the "performance" refers to the Alleye device's capabilities without direct human interpretive input during the testing process itself. The interpretation of the Alleye results by eye care professionals is an intended use, but the studies focus on the patient-facing component.
7. The Type of Ground Truth Used (Expert Consensus, Pathology, Outcomes Data, etc.)
- Test-Retest Study: The ground truth for this study was the internal consistency/reproducibility of the Alleye's own measurements over time in the same individuals. It's about how reliably the device detects metamorphopsia, not necessarily how accurately it matches an external ground truth for metamorphopsia presence.
- Clinical Evaluation – Longitudinal Study: The ground truth appears to be clinical outcomes data related to the "need for an injection at the next follow-up visit," which would be a physician's clinical decision based on a comprehensive assessment (e.g., visual acuity, OCT, physical exam). The study shows Alleye's ability to predict or indicate this need, implying these clinical decisions serve as the reference.
- Usability Study: The ground truth for usability was user feedback (oral feedback and System Usability Scale scores) from the target patient population.
8. The Sample Size for the Training Set
The document does not provide information about the sample size used for the training set of the Alleye algorithm. As a Predicate device (myVisionTrack) is mentioned and the focus is on substantial equivalence, it's possible that the "algorithm" for hyperacuity testing is based on established principles rather than requiring extensive de novo machine learning training data in the context of this submission. However, if machine learning was used, the training data details are not disclosed here.
9. How the Ground Truth for the Training Set Was Established
Since information about a specific training set and its sample size is not provided, the method for establishing its ground truth is also not detailed in this document.
§ 886.1330 Amsler grid.
(a)
Identification. An Amsler grid is a device that is a series of charts with grids of different sizes that are held at 30 centimeters distance from the patient and intended to rapidly detect central and paracentral irregularities in the visual field.(b)
Classification. Class I (general controls). The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter, subject to the limitations in § 886.9. The device is also exempt from the current good manufacturing practice requirements of the quality system regulation in part 820 of this chapter, with the exception of § 820.180, with respect to general requirements concerning records, and § 820.198, with respect to complaint files.