Search Results

Canvas Dx is intended for use by healthcare providers as an aid in the diagnosis of Autism Spectrum Disorder (ASD) for patients ages 18 months through 72 months who are at risk for developmental delay based on concerns of a parent, caregiver, or healthcare provider.

The device is not intended for use as a stand-alone diagnostic device but as an adjunct to the diagnostic process.

Canvas Dx is a prescription diagnostic aid for healthcare professionals (HCP) considering the diagnosis of Autism Spectrum Disorder (ASD) in patients 18 months through 72 months of age at risk for developmental delay. The subject device is identical to the Cognoa ASD Diagnosis Aid which was authorized under DEN200069 and was renamed Canvas Dx shortly thereafter. Canvas Dx consists of Software as a Medical Device (SaMD) together with several medical device data system (MDDS) components. The SaMD components consist of the following:

• Device inputs:
  • Device Input 1: The answers to the Caregiver Questionnaire
  • Device Input 2: Patient Video Analysis
  • Device Input 3: The answers to the Healthcare Provider Questionnaire
• A machine learning (ML) algorithm ('Algorithm') modeled after standard medical evaluation methodologies and drives the device outputs.
• Device outputs:
  • 'Positive for autism'
  • 'Negative for autism'
  • 'Indeterminate'

The MDDS components that are compatible with the SaMD components include the following:

• A caregiver facing mobile application, which provides Device Input 1;
• A video analyst system, which provides Device Input 2;
• A healthcare provider portal, which provides Device Input 3;
• Several supporting software and backend services and infrastructure, including privacy and security encryption and infrastructure in compliance with HIPAA and other best practices.

The subject of this submission is the inclusion of a Predetermined Change Control Plan (PCCP) that allows updates to the Canvas Dx model and performance thresholds.

The provided FDA 510(k) clearance letter and summary for Canvas Dx primarily describe the Predetermined Change Control Plan (PCCP) for the device, rather than a new standalone clinical study proving the device meets acceptance criteria. The information regarding device performance and clinical validation directly points to the predicate device, Cognoa ASD Diagnosis Aid (DEN200069), stating that Canvas Dx is identical to it.

Therefore, the acceptance criteria and study details provided are those for the original Cognoa ASD Diagnosis Aid (DEN200069) clearance.

Here's an analysis based on the provided text:

Acceptance Criteria and Reported Device Performance

The acceptance criteria are implied by the reported performance metrics of the predicate device (Cognoa ASD Diagnosis Aid, DEN200069), which Canvas Dx claims to replicate.

Note on Determinate Rate: There appears to be a typo in the provided document regarding the Determinate Rate: "31.76% (CI: 63.58%-87.67%)". A value of 31.76% cannot be within a confidence interval of 63.58%-87.67%. It is likely that the "31.76%" is a typo, and the actual determinate rate is within the reported CI, or the CI is for a different metric. For the purpose of this table, I've noted the discrepancy.

Study Proving Device Meets Acceptance Criteria

The details provided refer to the original clinical validation study for the Cognoa ASD Diagnosis Aid (DEN200069), as no new clinical testing was performed for the Canvas Dx submission (K243558) itself, which focuses on a Predetermined Change Control Plan (PCCP).

1. Sample size used for the test set and the data provenance:

   • Sample Size: Not explicitly stated as an exact number of patients in the provided text. The study was described as a "prospective, double-blinded, single-arm" study conducted at "14 sites".
   • Data Provenance: The document does not specify the country of origin of the data. It was a prospective study.

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • Number of Experts: "3 clinical specialists."
   • Qualifications: The specific qualifications (e.g., number of years of experience, specific sub-specialties beyond "clinical specialists") are not provided in the text.

3. Adjudication method for the test set:

   • The document states "Based on review by 3 clinical specialists" for establishing ground truth. It does not specify the adjudication method (e.g., 2+1, 3+1, majority vote, consensus meeting) used by these three specialists.

4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, if so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No information about an MRMC study or the effect size of human reader improvement with AI assistance is provided. The device is described as an "aid in the diagnosis" and "adjunct to the diagnostic process," implying a human-in-the-loop, but the clinical study described is a direct comparison to a "clinical reference standard," not a human-AI team comparison.

5. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:

   • Yes, the performance metrics (PPV, NPV, Sensitivity, Specificity, Determinate Rate) presented are representative of the algorithm's standalone performance in comparison to the clinical reference standard. The device inputs are collected from caregivers and healthcare providers, but the algorithm itself generates the "Positive for autism," "Negative for autism," or "Indeterminate" output. The text explicitly states, "The device is not intended for use as a stand-alone diagnostic device but as an adjunct to the diagnostic process," which refers to its clinical use case, but the performance values provided relate to the algorithm's direct output on the test set.

6. The type of ground truth used:

   • "Clinical reference standard." The exact components of this clinical reference standard (e.g., ADOS, ADI-R, expert clinical diagnosis, combination) are not specified but implied to be a robust, recognized method for ASD diagnosis.

7. The sample size for the training set:

   • The sample size for the training set is not provided in the document.

8. How the ground truth for the training set was established:

   • This information is not provided in the document.

Ask a specific question about this device

The Cognoa ASD Diagnosis Aid is intended for use by healthcare providers as an aid in the diagnosis of Autism Spectrum Disorder (ASD) for patients ages 18 months through 72 months who are at risk for developmental delay based on concerns of a parent, caregiver, or healthcare provider.

The device is not intended for use as a stand-alone diagnostic device but as an adjunct to the diagnostic process.

The Cognoa ASD Diagnosis Aid is a software as a medical device (SaMD) that utilizes a machine-learning algorithm that receives independent information from caregivers or parents, trained analysts, and healthcare professionals (HCPs) to aid in the diagnosis of ASD. It consists of multiple software applications and hardware platforms. Input data is acquired via a Mobile App, a Video Analyst Portal, and a HCP Portal.

• . Mobile App: User interface (UI) for the caregiver or parent to upload videos of the patient via Wi-Fi connection and answer questions about key developmental behaviors. Interfaces with Application Programming Interface (API) server for transmission and management of patient data. Compatible with both iOS (versions 12 and 13) and Android platforms (versions 9 and 10).
• Video Analyst Portal: UI for trained analysts to review uploaded patient videos . remotely and answer questions about the patients' behaviors observed in the videos.
• . HCP Portal: UI for the HCP to answer questions about key developmental behaviors for the patient's age group, view device output and access the interactive dashboard to view all patient results, patient videos, answers to questionnaires administered and device performance data. Compatible with computer operating systems macOS (Catalina or Mojave) and Windows 10, and browsers Safari (versions 12 or 13) and Chrome (versions 84 or 85).
  Following analysis of the input data, the Cognoa ASD Diagnosis Aid machine-learning algorithm produces a single scalar value between (1) and (6) which is then compared to preset thresholds to determine the classification. If the value is greater than the upper threshold, then the device output is 'Positive for ASD.' If the value is less than the lower threshold, then the device output is 'Negative for ASD.' If the available information does not allow the algorithm to render a reliable result, the device output is 'No Result.'

Here's a breakdown of the acceptance criteria and the study proving the Cognoa ASD Diagnosis Aid meets them, based on the provided text:

Acceptance Criteria and Device Performance

1. Table of Acceptance Criteria and Reported Device Performance

Conclusion on Acceptance: The device successfully met both the primary effectiveness objectives criteria for PPV (80.77% > 65%) and NPV (98.25% > 85%).

2. Sample Size and Data Provenance for the Test Set

• Sample Size:
  • Test Set for Analysis: 425 subjects successfully completed both the device assessment and the specialist assessment (clinical reference standard).
  • Subjects with Device Output (Positive/Negative for ASD): Of the 425 completers, 135 subjects received a definitive "Positive for ASD" or "Negative for ASD" output from the device. This subset was used to calculate the performance metrics (PPV, NPV, sensitivity, specificity).
• Data Provenance:
  • Country of Origin: United States.
  • Retrospective or Prospective: Prospective. The study was designed and conducted specifically to evaluate the device.

3. Number of Experts and Their Qualifications for Ground Truth

• Number of Experts: Up to three specialists were involved in establishing the clinical reference standard (ground truth) for each patient. This included a site-specific specialist and one or two central specialist clinicians.
• Qualifications of Experts: The text states they were "specialists" and "specialist clinicians" using the DSM-5 criteria, implying they were qualified healthcare professionals with expertise in diagnosing ASD. While specific years of experience are not provided, their role in making a clinical diagnosis using established criteria suggests appropriate qualifications.

4. Adjudication Method for the Test Set

The adjudication method for establishing the clinical reference standard was a multi-expert consensus approach:

• Initial Diagnosis: A site-specific specialist made an initial diagnosis using DSM-5 criteria.
• First Review: A central off-site reviewing specialist clinician reviewed the case (standardized medical history, physical form, and a video of the diagnostic encounter).
• Agreement: If the first central reviewer agreed with the site diagnosing clinician, the diagnosis was considered validated.
• Disagreement/Second Review: If the first central reviewer disagreed, the case was referred to a second reviewing specialist clinician.
• Resolution: "Majority rule was used to resolve discrepancies between the two central reviewers and the site diagnosing specialist who all evaluated the same subjects." This can be characterized as a 2+1 consensus model (2 central reviewers + 1 site specialist).

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Was an MRMC study done? No, a traditional MRMC comparative effectiveness study was not explicitly described in terms of comparing human readers with AI vs. without AI assistance to measure an "effect size" of improvement.
• Type of Study: The clinical validation study was a prospective, double-blinded, single-arm study evaluating the device's performance against a clinical reference standard. It focused on the standalone performance characteristics of the device as an aid, not directly on the improvement of human readers when assisted by the AI. The human factors study involved HCPs interacting with the device interface and interpreting its outputs, but it wasn't designed as an MRMC to quantify diagnostic improvement with AI.

6. Standalone (Algorithm Only) Performance

• Was a standalone performance study done? Yes, the core clinical validation study effectively evaluated a form of standalone performance of the algorithm's output (Positive/Negative/No Result) against a clinical reference standard. While the algorithm receives inputs from caregivers, trained analysts, and HCPs, the evaluation of PPV, NPV, sensitivity, and specificity is a measure of the algorithm's diagnostic classification performance on the test data. The output classification is solely based on the algorithm's processing of these inputs, without further human modification of the classification itself.

7. Type of Ground Truth Used

• Type of Ground Truth: Expert Consensus (Clinical Reference Standard). This involved the determination of clinical diagnosis based on the majority assessment of up to three specialists using the DSM-5 criteria.

8. Sample Size for the Training Set

• The document does not explicitly state the sample size used for the training set.
• However, it does mention an exclusion criterion for the clinical study: "Subjects whose medical records had been included in any internal Cognoa training or validation sets." This confirms that separate training and validation sets were used, adhering to good machine learning practices, but the specific size of the training set is not provided in this regulatory summary.

9. How Ground Truth for the Training Set Was Established

• The document does not explicitly describe how the ground truth for the training set was established.
• Given the nature of the device and the rigorous establishment of ground truth for the test set (expert consensus using DSM-5), it is highly probable that a similar, robust method involving clinical experts and diagnostic criteria would have been used for the training set ground truth, but the details are not provided in this specific excerpt.

Ask a specific question about this device