(266 days)
DEEPECHO is intended to analyze fetal ultrasound images and clips using machine learning techniques to detect standard biometry views. Upon views detection, DEEPECHO assists in measurements computation of fetal biometric parameters (i.e., head circumference, abdominal circumference, femur length, and bi-parietal diameter).
The device is intended for use by qualified and appropriately trained healthcare professionals as a concurrent reading and measuring aid during the acquisition and interpretation of fetal ultrasound images of patients aged 18 years or older done between the 14th and the 41st weeks of pregnancy.
DEEPECHO is a cloud-based standalone software as a medical device (SaMD) that helps qualified healthcare professionals in the assessment of obstetrical images.
The device is intended for use by qualified and appropriately trained healthcare professionals, including but not limited to: radiologists, obstetricians, sonographers, maternal and fetal medicine specialists, obstetricians and gynecologists, as well as fetal surgeons. The device's application is intended for pregnant patients aged 18 years and older between the 14th and the 41st weeks of pregnancy.
DEEPECHO takes as an input either Digital Imaging and Communications in Medicine (DICOM) images or a live fetal ultrasound image streaming from ultrasound scanners. When DEEPECHO is in use, it allows healthcare professionals to browse fetal ultrasound images, identify views (cephalic, abdominal, and femoral) and suggest a placement for calipers on fetal ultrasound images of the identified views and compute biometrical measurements of the latter (HC (Head Circumference), BPD (Bi-parietal Diameter), FL (Femur Length), AC (Abdominal Circumference), GA (Gestational Age), EFW (Estimated Fetal Weight)).
When DEEPECHO is used during real time (i.e., during the examination) the device receives real-time image streaming from an ultrasound machine and can be used to identify views (cephalic, abdominal, and femoral), and suggests a placement for calipers. These Ultrasound images are acquired using an HDMI cable that is plugged into the local device (e.g., computer, tablet) running DEEPECHO, through an HDMI to USB Video Capture. Note that the device cannot compute measurements in real time as the real time data received from the ultrasound machine is a live stream video and not in a DICOM format. DICOM data is necessary to automatically compute biometrical measurements.
Additionally, DEEPECHO handles patient and exam management by allowing healthcare professionals to create, update, and archive records. When DICOM files are uploaded post-examination, the software either links the exam to an existing patient record or automatically generates a new one from the DICOM files' metadata. It enables healthcare professionals to track a patient's history, including exams, reports, and all information directly inputted to the platform.
Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided FDA clearance letter:
Acceptance Criteria and Device Performance
1. Table of Acceptance Criteria & Reported Device Performance
The clearance letter does not explicitly state "acceptance criteria" in a separate section with specific numerical thresholds for each metric. However, the "Summary Test Results" section effectively serves as the performance criteria that the device had to meet to demonstrate substantial equivalence. The reported performance is directly from the tables provided.
| Metric | Acceptance Criteria (Implicit from Study Results) | Reported Device Performance |
|---|---|---|
| Femur Length (FL) | Slope close to 1, intercept close to 0 (Deming regression) | Intercept: 0.003 (95% CI: -0.020, 0.025) |
| Slope: 0.969 (95% CI: 0.966, 0.973) | ||
| Head Circumference (HC) | Slope close to 1, intercept close to 0 (Deming regression) | Intercept: -0.360 (95% CI: -0.462, -0.258) |
| Slope: 1.026 (95% CI: 1.022, 1.031) | ||
| Abdominal Circumference (AC) | Slope close to 1, intercept close to 0 (Deming regression) | Intercept: -0.017 (95% CI: -0.101, 0.065) |
| Slope: 1.017 (95% CI: 1.013, 1.021) | ||
| Biparietal Diameter (BPD) | Slope close to 1, intercept close to 0 (Deming regression) | Intercept: -0.165 (95% CI: -0.203, -0.125) |
| Slope: 1.020 (95% CI: 1.015, 1.025) | ||
| Abdominal View Sensitivity | High sensitivity (specific threshold not stated, but demonstrated high performance) | 86.9% (83.8% - 89.7%) |
| Abdominal View Specificity | High specificity (specific threshold not stated, but demonstrated high performance) | 96.8% (96.4% - 97.2%) |
| Cephalic View Sensitivity | High sensitivity (specific threshold not stated, but demonstrated high performance) | 98.2% (97.4% - 99%) |
| Cephalic View Specificity | High specificity (specific threshold not stated, but demonstrated high performance) | 94.8% (94.2% - 95.3%) |
| Femoral View Sensitivity | High sensitivity (specific threshold not stated, but demonstrated high performance) | 91.8% (89% - 94.2%) |
| Femoral View Specificity | High specificity (specific threshold not stated, but demonstrated high performance) | 97.4% (97% - 97.8%) |
Study Details
2. Sample size used for the test set and the data provenance
- Sample Size (Test Set):
- Studies: 417 ultrasound studies initially, with 397 subjects remaining after exclusion criteria.
- Images: 23,544 de-identified 2D grayscale ultrasound images.
- For specific primary endpoints:
- Femur Length: N=431
- Head Circumference: N=858
- Abdominal Circumference: N=499
- Biparietal Diameter: N=858
- Data Provenance:
- Country of Origin: United States, Mexico, and Morocco.
- Retrospective or Prospective: Not explicitly stated, but the description of collecting "de-identified" images and the "truthing process" suggests a retrospective collection of existing ultrasound data.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
- Number of Experts: Three independent experts.
- Qualifications: ARDMS-certified sonographers with a minimum of five years of clinical ultrasound experience.
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set
- Adjudication Method:
- For continuous measurement endpoints (FL, HC, AC, BPD): The ground truth was calculated as the arithmetic mean of three independent caliper placements. This implies a form of consensus/averaging, not a 2+1 or 3+1 style adjudication in case of disagreement, as an average will always be computed.
- For classification endpoints (view identification): Ground truth was based on unanimous agreement across reviewers. If there wasn't unanimous agreement, those cases would presumably not have a ground truth or would be excluded, but the text explicitly states "unanimous agreement."
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
- MRMC Study: No, a multi-reader multi-case (MRMC) comparative effectiveness study with human readers (with vs. without AI assistance) was not reported in this summary. The study focused on the standalone performance of the DEEPECHO software compared to expert-derived ground truth.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
- Standalone Study: Yes, a standalone assessment comparing the performance of the DEEPECHO software to ground truth was performed. The letter explicitly states, "DeepEcho performed a stand-alone assessment comparing the performance of the DEEPECHO software to a ground truth of annotations by qualified experts."
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)
- Type of Ground Truth: Expert consensus/annotations. Specifically, for continuous measurements, it was the arithmetic mean of three independent caliper placements by experts. For classification, it was unanimous agreement among three experts.
8. The sample size for the training set
- Training Set Sample Size: Not explicitly quantified in terms of number of images or studies. The text states where the training data was obtained: "Training data were obtained from two Roshan MFM clinics in New York City and selected sites in Morocco."
9. How the ground truth for the training set was established
- Training Set Ground Truth Establishment: The document does not explicitly detail how the ground truth for the training set was established. It only describes the process for the test set. However, given the context, it's highly probable that a similar expert-based annotation process was used for the training data to ensure consistency and quality.
§ 892.1550 Ultrasonic pulsed doppler imaging system.
(a)
Identification. An ultrasonic pulsed doppler imaging system is a device that combines the features of continuous wave doppler-effect technology with pulsed-echo effect technology and is intended to determine stationary body tissue characteristics, such as depth or location of tissue interfaces or dynamic tissue characteristics such as velocity of blood or tissue motion. This generic type of device may include signal analysis and display equipment, patient and equipment supports, component parts, and accessories.(b)
Classification. Class II.