(144 days)
Canvas Dx is intended for use by healthcare providers as an aid in the diagnosis of Autism Spectrum Disorder (ASD) for patients ages 18 months through 72 months who are at risk for developmental delay based on concerns of a parent, caregiver, or healthcare provider.
The device is not intended for use as a stand-alone diagnostic device but as an adjunct to the diagnostic process.
Canvas Dx is a prescription diagnostic aid for healthcare professionals (HCP) considering the diagnosis of Autism Spectrum Disorder (ASD) in patients 18 months through 72 months of age at risk for developmental delay. The subject device is identical to the Cognoa ASD Diagnosis Aid which was authorized under DEN200069 and was renamed Canvas Dx shortly thereafter. Canvas Dx consists of Software as a Medical Device (SaMD) together with several medical device data system (MDDS) components. The SaMD components consist of the following:
- Device inputs:
- Device Input 1: The answers to the Caregiver Questionnaire
- Device Input 2: Patient Video Analysis
- Device Input 3: The answers to the Healthcare Provider Questionnaire
- A machine learning (ML) algorithm ('Algorithm') modeled after standard medical evaluation methodologies and drives the device outputs.
- Device outputs:
- 'Positive for autism'
- 'Negative for autism'
- 'Indeterminate'
The MDDS components that are compatible with the SaMD components include the following:
- A caregiver facing mobile application, which provides Device Input 1;
- A video analyst system, which provides Device Input 2;
- A healthcare provider portal, which provides Device Input 3;
- Several supporting software and backend services and infrastructure, including privacy and security encryption and infrastructure in compliance with HIPAA and other best practices.
The subject of this submission is the inclusion of a Predetermined Change Control Plan (PCCP) that allows updates to the Canvas Dx model and performance thresholds.
The provided FDA 510(k) clearance letter and summary for Canvas Dx primarily describe the Predetermined Change Control Plan (PCCP) for the device, rather than a new standalone clinical study proving the device meets acceptance criteria. The information regarding device performance and clinical validation directly points to the predicate device, Cognoa ASD Diagnosis Aid (DEN200069), stating that Canvas Dx is identical to it.
Therefore, the acceptance criteria and study details provided are those for the original Cognoa ASD Diagnosis Aid (DEN200069) clearance.
Here's an analysis based on the provided text:
Acceptance Criteria and Reported Device Performance
The acceptance criteria are implied by the reported performance metrics of the predicate device (Cognoa ASD Diagnosis Aid, DEN200069), which Canvas Dx claims to replicate.
| Metric | Acceptance Criteria (Implied by Predicate Performance) | Reported Device Performance (from DEN200069) |
|---|---|---|
| Positive Predictive Value (PPV) | Not explicitly stated as a minimum, but established by predicate. | 80.77% (CI: 70.27-88.82%) |
| Negative Predictive Value (NPV) | Not explicitly stated as a minimum, but established by predicate. | 98.25% (CI: 90.61-99.96%) |
| Determinate Rate | Not explicitly stated as a minimum, but established by predicate. | 31.76% (CI: 63.58%-87.67%) - Mistake in document, 31.76% is not in CI 63.58-87.67%. Likely meant 71.76% or similar within CI range. Using the CI lower bound as the more conservative. Assuming it meant Percentage of results that are determinate. |
| Sensitivity | Not explicitly stated as a minimum, but established by predicate. | 98.44% (CI: 91.6%-99.96%) |
| Specificity | Not explicitly stated as a minimum, but established by predicate. | 78.87% (CI: 67.56%-87.67%) |
Note on Determinate Rate: There appears to be a typo in the provided document regarding the Determinate Rate: "31.76% (CI: 63.58%-87.67%)". A value of 31.76% cannot be within a confidence interval of 63.58%-87.67%. It is likely that the "31.76%" is a typo, and the actual determinate rate is within the reported CI, or the CI is for a different metric. For the purpose of this table, I've noted the discrepancy.
Study Proving Device Meets Acceptance Criteria
The details provided refer to the original clinical validation study for the Cognoa ASD Diagnosis Aid (DEN200069), as no new clinical testing was performed for the Canvas Dx submission (K243558) itself, which focuses on a Predetermined Change Control Plan (PCCP).
-
Sample size used for the test set and the data provenance:
- Sample Size: Not explicitly stated as an exact number of patients in the provided text. The study was described as a "prospective, double-blinded, single-arm" study conducted at "14 sites".
- Data Provenance: The document does not specify the country of origin of the data. It was a prospective study.
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Number of Experts: "3 clinical specialists."
- Qualifications: The specific qualifications (e.g., number of years of experience, specific sub-specialties beyond "clinical specialists") are not provided in the text.
-
Adjudication method for the test set:
- The document states "Based on review by 3 clinical specialists" for establishing ground truth. It does not specify the adjudication method (e.g., 2+1, 3+1, majority vote, consensus meeting) used by these three specialists.
-
If a multi-reader multi-case (MRMC) comparative effectiveness study was done, if so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No information about an MRMC study or the effect size of human reader improvement with AI assistance is provided. The device is described as an "aid in the diagnosis" and "adjunct to the diagnostic process," implying a human-in-the-loop, but the clinical study described is a direct comparison to a "clinical reference standard," not a human-AI team comparison.
-
If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:
- Yes, the performance metrics (PPV, NPV, Sensitivity, Specificity, Determinate Rate) presented are representative of the algorithm's standalone performance in comparison to the clinical reference standard. The device inputs are collected from caregivers and healthcare providers, but the algorithm itself generates the "Positive for autism," "Negative for autism," or "Indeterminate" output. The text explicitly states, "The device is not intended for use as a stand-alone diagnostic device but as an adjunct to the diagnostic process," which refers to its clinical use case, but the performance values provided relate to the algorithm's direct output on the test set.
-
The type of ground truth used:
- "Clinical reference standard." The exact components of this clinical reference standard (e.g., ADOS, ADI-R, expert clinical diagnosis, combination) are not specified but implied to be a robust, recognized method for ASD diagnosis.
-
The sample size for the training set:
- The sample size for the training set is not provided in the document.
-
How the ground truth for the training set was established:
- This information is not provided in the document.
§ 882.1491 Pediatric Autism Spectrum Disorder diagnosis aid.
(a)
Identification. A pediatric Autism Spectrum Disorder diagnosis aid is a prescription device that is intended for use as an aid in the diagnosis of Autism Spectrum Disorder in pediatric patients.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Clinical performance testing must demonstrate that the device performs as intended under anticipated conditions of use, including an evaluation of sensitivity, specificity, positive predictive value, and negative predictive value using a reference method of diagnosis and assessment of patient behavioral symptomology.
(2) Software verification, validation, and hazard analysis must be provided. Software documentation must include a detailed, technical description of the algorithm(s) used to generate device output(s), and a cybersecurity assessment of the impact of threats and vulnerabilities on device functionality and user(s).
(3) Usability assessment must demonstrate that the intended user(s) can safely and correctly use the device.
(4) Labeling must include:
(i) Instructions for use, including a detailed description of the device, compatibility information, and information to facilitate clinical interpretation of all device outputs; and
(ii) A summary of any clinical testing conducted to demonstrate how the device functions as an interpretation of patient behavioral symptomology associated with Autism Spectrum Disorder. The summary must include the following:
(A) A description of each device output and clinical interpretation;
(B) Any performance measures, including sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV);
(C) A description of how the cutoff values used for categorical classification of diagnoses were determined; and
(D) Any expected or observed adverse events and complications.
(iii) A statement that the device is not intended for use as a stand-alone diagnostic.