(142 days)
The EarliPoint System is indicated for use in specialized developmental disabilities centers as a tool to aid clinicians in the diagnosis and assessment of ASD patients ages 16 months through 30 months.
The device is a more compact version of the predicate device but otherwise has similar functions and features. The system uses an eye tracker to capture the patient's looking behavior while viewing a series of videos. The system then remotely analyzes the looking behavior data using software and outputs a diagnosis of the patient's ASD status and assesses the symptoms associated with ASD.
The system has two modules:
EarliPoint System consists of the following:
- Eye-tracking module and a separate Operator Module that can control the Eye-tracking module remotely. The patient sits on a chair and the Eye-tracking module is adjusted by the operator such that the patient's eyes are within the specification of the eye tracking window
- -Eye-tracking module captures the patient visual response to social information provided in the form of a series of age-appropriate videos
- Operator's module is used to initiate and monitors the session remotely -
- -WebPortal securely stores all patient information, analyzes the eye tracking data, and outputs the results. Users can retrieve the results directly from the web-portal.
- -Artificial intelligence software analyzes the eye-tracking data and provides a diagnosis for ASD. In addition, it also outputs 3 indices (called EarliPoint Severity Indices) that proxy the ADOS-2 and Mullen validated ASD instruments
- o Social Disability Index correlates and proxies ADOS-2
- O Verbal Ability Index correlates and proxies the age equivalent Mullen Verbal Ability score
- O Non-verbal Ability Index correlates and proxies the age equivalent non-verbal Mullen Ability score
The eye-tracker used in the EarliPoint device has similar capability as the eye tracker used in the predicate device.
The provided text describes a 510(k) submission for a modified EarliPoint System, which is an aid for diagnosing and assessing Autism Spectrum Disorder (ASD). The submission details performance testing and a pivotal clinical study.
Here's an analysis of the acceptance criteria and the study proving the device meets them:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria in this submission are derived from the clinical study's performance metrics against a reference standard (expert clinical diagnosis for ASD).
| Performance Metric | Acceptance Criteria (from study results) | Reported Device Performance (EarliPoint mITD) | Reported Device Performance (EarliPoint CertainDx) |
|---|---|---|---|
| Sensitivity | Not explicitly stated as a pre-defined acceptance criteria, but observed performance | 71% (95% CI: 64.6% - 76.9%) | 78.0% (95% CI: 70.5% - 84.3%) |
| Specificity | Not explicitly stated as a pre-defined acceptance criteria, but observed performance | 80.7% (95% CI: 75.3% - 85.4%) | 85.4% (95% CI: 79.5% - 90.2%) |
| Safety | No serious adverse events related to device use | No reported serious adverse events | No reported serious adverse events |
| Correlation with ADOS-2 | Positive correlation with EarliPoint Social Disability Index | Correlates and proxies ADOS-2 | Correlates and proxies ADOS-2 |
| Correlation with Mullen Verbal Ability | Positive correlation with EarliPoint Verbal Ability Index | Correlates and proxies age equivalent Mullen Verbal Ability score | Correlates and proxies age equivalent Mullen Verbal Ability score |
| Correlation with Mullen Non-verbal Ability | Positive correlation with EarliPoint Non-verbal Ability Index | Correlates and proxies age equivalent non-verbal Mullen Ability score | Correlates and proxies age equivalent non-verbal Mullen Ability score |
Note: The document does not explicitly state pre-defined numerical acceptance criteria for sensitivity and specificity. The reported performance is the outcome of the study, and its acceptance implies these values were deemed sufficient by the FDA for substantial equivalence.
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size: 475 evaluable patients for primary and secondary endpoint analysis. 25 patients had missing data for either the device or control diagnosis, bringing the total enrolled to 500.
- Data Provenance:
- Country of Origin: United States
- Retrospective or Prospective: Prospective
- Multi-center: Six sites
3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications
- Number of Experts: Not specified. The text mentions "expert clinicians" but does not quantify the number of individual experts or how many were involved per case.
- Qualifications of Experts: Referred to as "expert clinicians" and their diagnosis is considered the "current best practice for diagnosis of ASD." No further specific qualifications like years of experience or board certifications are provided in the document.
4. Adjudication Method for the Test Set
- Adjudication Method: Not explicitly stated. The text mentions "expert clinician diagnosis" as the reference standard but does not detail how consensus was reached if multiple experts were involved or if there was a single expert per case.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and Effect Size
- MRMC Study: No, a multi-reader multi-case (MRMC) comparative effectiveness study was not conducted in the traditional sense of comparing human readers with AI assistance versus human readers without AI assistance.
- Effect Size: Therefore, no effect size for human reader improvement with AI assistance is reported. The study's design was a within-subject comparison where all patients were evaluated by both the EarliPoint system and expert clinicians, comparing the device's diagnostic output against the expert diagnosis.
6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Was Done
- Standalone Performance: Yes, the study evaluated the "EarliPoint System diagnosis" relative to the expert clinical diagnosis. The device's output (ASD diagnosis and severity indices) is generated by "Artificial intelligence software [that] analyzes the eye-tracking data and provides a diagnosis for ASD." This indicates a standalone performance evaluation of the algorithm. The "EarliPoint CertainDx" analysis further refined this by focusing on cases where clinicians were certain of the diagnosis, still representing the device's standalone output being compared to this subset of expert diagnoses.
7. The Type of Ground Truth Used
- Type of Ground Truth: "Expert clinician diagnosis (current best practice for diagnosis of ASD)." This is a form of expert consensus or clinical standard of care. It is further supported by correlation with validated ASD instruments (ADOS-2 and Mullen).
8. The Sample Size for the Training Set
- The document does not explicitly state the sample size for the training set. The clinical study described (the pivotal study) is an evaluation study (test set) for the device's performance, not the dataset used to train the AI algorithm.
9. How the Ground Truth for the Training Set Was Established
- The document does not explicitly state how the ground truth for the training set was established. While it details the ground truth for the test set (expert clinician diagnosis), it does not provide information on the training data.
§ 882.1491 Pediatric Autism Spectrum Disorder diagnosis aid.
(a)
Identification. A pediatric Autism Spectrum Disorder diagnosis aid is a prescription device that is intended for use as an aid in the diagnosis of Autism Spectrum Disorder in pediatric patients.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Clinical performance testing must demonstrate that the device performs as intended under anticipated conditions of use, including an evaluation of sensitivity, specificity, positive predictive value, and negative predictive value using a reference method of diagnosis and assessment of patient behavioral symptomology.
(2) Software verification, validation, and hazard analysis must be provided. Software documentation must include a detailed, technical description of the algorithm(s) used to generate device output(s), and a cybersecurity assessment of the impact of threats and vulnerabilities on device functionality and user(s).
(3) Usability assessment must demonstrate that the intended user(s) can safely and correctly use the device.
(4) Labeling must include:
(i) Instructions for use, including a detailed description of the device, compatibility information, and information to facilitate clinical interpretation of all device outputs; and
(ii) A summary of any clinical testing conducted to demonstrate how the device functions as an interpretation of patient behavioral symptomology associated with Autism Spectrum Disorder. The summary must include the following:
(A) A description of each device output and clinical interpretation;
(B) Any performance measures, including sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV);
(C) A description of how the cutoff values used for categorical classification of diagnoses were determined; and
(D) Any expected or observed adverse events and complications.
(iii) A statement that the device is not intended for use as a stand-alone diagnostic.