DreaMed Advisor Pro

• K Number: DEN170043
• Device Name: DreaMed Advisor Pro
• Manufacturer: DreaMed Diabetes, Ltd.
• Date Cleared: 2018-06-12 ( 299 days)
• Product Code: QCC , OCC
• Regulation Number: 862.1358
• Type: Direct
• Panel: CH
• Reference & Predicate Devices: N/A

Intended Use

DreaMed Advisor Pro is a decision-support software intended for assisting healthcare professionals in the management of patients with Type 1 diabetes who:

• use insulin pumps as their insulin delivery therapy: .
• . monitor their glucose levels using either of the following:
  • CGM, or O
  • CGM and self-management blood glucose meter O
• . are above the age of 6 and under 65 years old; and
• . use rapid acting U-100 insulin analogs in their pump

DreaMed Advisor Pro is indicated for use by healthcare professionals when analyzing continuous glucose monitoring (CGM), self-monitoring blood glucose (SMBG) and pump data to generate recommendations for optimizing a patient's insulin pump settings for basal rate, carbohydrate ratio (CR), and correction factor (CF); without considering the full clinical status of a particular patient. DreaMed Advisor Pro does not replace clinical judgment.

Device Description

DreaMed Advisor Pro is a software device that is designed to provide insulin therapy adjustment recommendations to physicians to assist in the management of diabetes for patients with Type 1 diabetes using an insulin pump and a continuous glucose monitoring (CGM) system.

The DreaMed Advisor Pro gathers and analyzes information inputted through qualified 3rd party Diabetes Management Systems (DMS), which collects biological input information from various diabetes devices. Diabetes device information required and used by DreaMed Advisor Pro includes glucose readings (CGM sensor readings with the option for capillary blood glucose measurements), insulin dosing logs, and meal data during daily routine care.

Following data collection and analysis, the DreaMed Advisor Pro generates results containing summary data and recommendations for adjustments to the patient's insulin therapy parameters, including: basal insulin delivery rate(s), insulin to carbohydrate ratio, and correction factor (insulin sensitivity)(b) (4) DreaMed Advisor Pro may also advise behavioral changes. Results are sent to a qualified 3th party Diabetes Management Systems, which displays results to physicians and a report provided by DreaMed Diabetes. The physician can approve, reject or change the recommendations and issue the updated treatment plan to the patient.

AI/ML Overview

Acceptance Criteria and Study for DreaMed Advisor Pro

The DreaMed Advisor Pro is a software device that provides insulin therapy adjustment recommendations to physicians for managing Type 1 diabetes in patients using insulin pumps and Continuous Glucose Monitoring (CGM) systems. The device was evaluated for automatic Class III designation.

1. Table of Acceptance Criteria and Reported Device Performance

The provided document does not explicitly present a table of quantitative acceptance criteria and corresponding reported device performance with numerical thresholds for metrics like accuracy, precision, or sensitivity/specificity for the device's recommendations compared to a gold standard. Instead, the "Performance Characteristics" section details two clinical studies that aim to demonstrate clinical validity and safety. The acceptance of the device seems to be based on the safety and a general alignment of its recommendations with expert opinion, rather than meeting specific quantitative performance metrics against a defined ground truth for individual recommendations.

However, based on the findings from the clinical studies described, we can infer the qualitative performance and the safety-focused acceptance:

Acceptance Criteria (Inferred from clinical studies and regulatory decision)	Reported Device Performance (as described in the document)
Safety and Non-Inferiority (Clinical Outcome): Recommendations do not lead to serious adverse events and do not worsen glycemic control compared to expert physician guidance.	- "No serious adverse events occurred in either group of the study."

• Percentage of CGM time within 70-180 mg/dL increased and percentage below 70 mg/dL reduced in Advisor Pro group compared to control group in one study. |
  | Clinical Validity (Alignment with Expert Opinion): Insulin dosing recommendations (basal rate, carb ratio, correction factor) are generally similar to expert physicians. | - Recommendations for basal rate and carbohydrate ratio were "generally similar" to expert physicians.
• Difference noted for correction factor recommendations between DreaMed Advisor Pro and physicians. |
  | Usability and Comprehension: Users (HCPs and patients/caregivers) can understand and appropriately interpret recommendations. | - Human factors testing performed with 48 lay users and 15 HCPs. "The design and results of the study were reviewed and found acceptable." |
  | Mitigation of Risks: Identified risks to health (erroneous recommendations, incorrect interpretation, inappropriate use, data corruption, insecure transmission) are mitigated to an acceptable level. | - Special controls and general controls are deemed sufficient to mitigate identified risks, with the healthcare professional acting as an intermediary. |

2. Sample Size Used for the Test Set and Data Provenance

• Study 1 (Expert Physician Survey):

  • Test Set Sample Size: 15 patients. Each patient's data was presented as an anonymized PDF file.
  • Data Provenance: Retrospective data of 3 weeks of CGM, SMBG, and insulin pump data from children and adolescents with Type 1 Diabetes using an insulin pump. Collected from 3 centers in Germany, Israel, and Slovenia.

• Study 2 (Clinical Intervention Study):

  • Test Set Sample Size:
    • Intervention group (guided by Advisor Pro): 6 subjects.
    • Control group (guided by expert physician): 7 subjects.
  • Data Provenance: Prospective, single-center study conducted at a pediatric hospital in Israel. Subjects used the algorithm for two iterations of treatment recommendations over a 6-week study period, with data collected during this period.

3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications

• Study 1 (Expert Physician Survey):

  • Number of Experts: 9 experts.
  • Qualifications: "Expert endocrinologists" with "expert physicians" from 3 centers (3 physicians from each center) participated. No specific years of experience or sub-specialty mentioned beyond "expert endocrinologists."

• Study 2 (Clinical Intervention Study):

  • Number of Experts: Unspecified number of "diabetes healthcare professionals" or "expert physician" (singular). The control group was guided by "the expert physician." This implies at least one expert managed the control group.
  • Qualifications: "Diabetes healthcare professionals" or "expert physician." No specific qualifications mentioned.

4. Adjudication Method for the Test Set

• Study 1 (Expert Physician Survey):

  • Each of the 9 experts independently provided proposed recommended changes.
  • The recommendations from these physicians were then compared among themselves and to the Advisor Pro automated recommendations. The document states "The recommendations from the physicians at each site were also compared." This suggests a form of comparison or analysis of agreement/disagreement among experts, but not a formal consensus-based adjudication to establish a single ground truth before comparison to the device. Diversity in opinions was noted ("diversity between experts").

• Study 2 (Clinical Intervention Study):

  • This study compared outcomes between two groups: one guided by Advisor Pro and one guided by a physician. There was no explicit adjudication method described for establishing a single "ground truth" for insulin recommendations within this comparative effectiveness study, as the physician's adjustments in the control group served as the "standard of care" for comparison.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

An MRMC study was effectively conducted in Study 1 (Expert Physician Survey).

• Study Design: 9 expert endocrinologists (multi-readers) independently reviewed data from 15 patients (multi-cases) and provided recommendations. These recommendations were then compared to the DreaMed Advisor Pro's recommendations.
• Effect Size of Human Readers with AI vs. without AI assistance: The study compared the Advisor Pro recommendations to expert physician recommendations. It did not involve human readers using the AI and then comparing their performance to human readers not using the AI. Therefore, it does not provide an effect size of how much human readers improve with AI assistance. Instead, it suggests a degree of similarity between AI and expert recommendations:
  • "the recommendations of the DreaMed Advisor Pro were generally similar to the recommendations of expert physicians with respect to the basal rate as well as the carbohydrate ratio (CR)."
  • "there was a difference between the physicians compared to DreaMed Advisor Pro and physician recommendations for the correction factor (CF)."
  • It also noted "diversity between experts with regards to recommendations to the same data."

6. Standalone (Algorithm Only) Performance

Yes, a standalone performance assessment was conducted for the DreaMed Advisor Pro implicit in Study 1 (Expert Physician Survey). The study compared the "Advisor Pro automated recommendations" directly against the recommendations of expert physicians. This indicates the algorithm's output was evaluated without human mediation in terms of generating the recommendation itself.

7. Type of Ground Truth Used

• Study 1 (Expert Physician Survey): Expert consensus/comparison. The ground truth for evaluating the DreaMed Advisor Pro's recommendations was the recommendations provided by the panel of 9 expert endocrinologists. However, as noted, there was "diversity between experts," so it wasn't a single "consensus" in a strict sense, but rather a comparison against a spectrum of expert opinions.
• Study 2 (Clinical Intervention Study): Clinical outcomes data (glycemic control metrics from CGM) and expert physician guidance as a control. The "ground truth" here was less about individual recommendation correctness and more about the impact on patient outcomes. The control group, guided by an expert physician, served as the comparative standard of care.

8. Sample Size for the Training Set

The document does not explicitly state the sample size used for the training set of the DreaMed Advisor Pro algorithm. It only describes the evaluation (test) studies.

9. How the Ground Truth for the Training Set Was Established

The document does not provide information on how the ground truth for the training set was established. This information would typically be part of the software development and validation process, but it is not detailed in the provided "DECISION SUMMARY."

Regulation Number and Section

§ 862.1358 Insulin therapy adjustment device.

(a)
Identification. An insulin therapy adjustment device is a device intended to incorporate biological inputs, including glucose measurement data from a continuous glucose monitor, to recommend insulin therapy adjustments as an aid in optimizing insulin therapy regimens for patients with diabetes mellitus.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Design verification and validation must include the following:
(i) A complete description of the required data inputs, including timeframe over which data inputs must be collected and number of data points required for accurate recommendations;
(ii) A complete description of the types of device outputs and insulin therapy adjustment recommendations, including how the recommendations are generated;
(iii) Robust data demonstrating the clinical validity of the device outputs and insulin therapy recommendations;
(iv) A robust assessment of all input data specifications, including accuracy requirements for continuous glucose monitors and other devices generating data inputs, to ensure accurate and reliable therapy adjustment recommendations. This assessment must include adequate clinical justification for each specification;
(v) A detailed strategy to ensure secure and reliable means of data transmission to and from the device, including data integrity checks, accuracy checks, reliability checks, and security measures;
(vi) Robust data demonstrating that users can understand and appropriately interpret recommendations generated by the device; and
(vii) An appropriate mitigation strategy to minimize the occurrence of dosing recommendation errors, and to mitigate the risk to patients of any residual dosing recommendation errors to a clinically acceptable level.
(2) The device must not be intended for use in implementing automated insulin dosing.
(3) Your 21 CFR 809.10(b) labeling must include:
(i) The identification of specific insulin formulations that have been demonstrated to be compatible with use of the device;
(ii) A detailed description of the specifications of compatible devices that provide acceptable input data (e.g., continuous glucose monitors, insulin pumps) used to provide accurate and reliable therapy adjustment recommendations;
(iii) A detailed description of all types of required data (inputs) and dosing recommendations (outputs) that are provided by the device; and
(iv) A description of device limitations, and instructions to prevent possible disruption of accurate therapy adjustment recommendations (e.g., time zone changes due to travel).