Search Results

DEEPECHO is intended to analyze fetal ultrasound images and clips using machine learning techniques to detect standard biometry views. Upon views detection, DEEPECHO assists in measurements computation of fetal biometric parameters (i.e., head circumference, abdominal circumference, femur length, and bi-parietal diameter).

The device is intended for use by qualified and appropriately trained healthcare professionals as a concurrent reading and measuring aid during the acquisition and interpretation of fetal ultrasound images of patients aged 18 years or older done between the 14th and the 41st weeks of pregnancy.

DEEPECHO is a cloud-based standalone software as a medical device (SaMD) that helps qualified healthcare professionals in the assessment of obstetrical images.

The device is intended for use by qualified and appropriately trained healthcare professionals, including but not limited to: radiologists, obstetricians, sonographers, maternal and fetal medicine specialists, obstetricians and gynecologists, as well as fetal surgeons. The device's application is intended for pregnant patients aged 18 years and older between the 14th and the 41st weeks of pregnancy.

DEEPECHO takes as an input either Digital Imaging and Communications in Medicine (DICOM) images or a live fetal ultrasound image streaming from ultrasound scanners. When DEEPECHO is in use, it allows healthcare professionals to browse fetal ultrasound images, identify views (cephalic, abdominal, and femoral) and suggest a placement for calipers on fetal ultrasound images of the identified views and compute biometrical measurements of the latter (HC (Head Circumference), BPD (Bi-parietal Diameter), FL (Femur Length), AC (Abdominal Circumference), GA (Gestational Age), EFW (Estimated Fetal Weight)).

When DEEPECHO is used during real time (i.e., during the examination) the device receives real-time image streaming from an ultrasound machine and can be used to identify views (cephalic, abdominal, and femoral), and suggests a placement for calipers. These Ultrasound images are acquired using an HDMI cable that is plugged into the local device (e.g., computer, tablet) running DEEPECHO, through an HDMI to USB Video Capture. Note that the device cannot compute measurements in real time as the real time data received from the ultrasound machine is a live stream video and not in a DICOM format. DICOM data is necessary to automatically compute biometrical measurements.

Additionally, DEEPECHO handles patient and exam management by allowing healthcare professionals to create, update, and archive records. When DICOM files are uploaded post-examination, the software either links the exam to an existing patient record or automatically generates a new one from the DICOM files' metadata. It enables healthcare professionals to track a patient's history, including exams, reports, and all information directly inputted to the platform.

Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided FDA clearance letter:

Acceptance Criteria and Device Performance

1. Table of Acceptance Criteria & Reported Device Performance

The clearance letter does not explicitly state "acceptance criteria" in a separate section with specific numerical thresholds for each metric. However, the "Summary Test Results" section effectively serves as the performance criteria that the device had to meet to demonstrate substantial equivalence. The reported performance is directly from the tables provided.

Study Details

2. Sample size used for the test set and the data provenance

• Sample Size (Test Set):
  • Studies: 417 ultrasound studies initially, with 397 subjects remaining after exclusion criteria.
  • Images: 23,544 de-identified 2D grayscale ultrasound images.
  • For specific primary endpoints:
    • Femur Length: N=431
    • Head Circumference: N=858
    • Abdominal Circumference: N=499
    • Biparietal Diameter: N=858
• Data Provenance:
  • Country of Origin: United States, Mexico, and Morocco.
  • Retrospective or Prospective: Not explicitly stated, but the description of collecting "de-identified" images and the "truthing process" suggests a retrospective collection of existing ultrasound data.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

• Number of Experts: Three independent experts.
• Qualifications: ARDMS-certified sonographers with a minimum of five years of clinical ultrasound experience.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

• Adjudication Method:
  • For continuous measurement endpoints (FL, HC, AC, BPD): The ground truth was calculated as the arithmetic mean of three independent caliper placements. This implies a form of consensus/averaging, not a 2+1 or 3+1 style adjudication in case of disagreement, as an average will always be computed.
  • For classification endpoints (view identification): Ground truth was based on unanimous agreement across reviewers. If there wasn't unanimous agreement, those cases would presumably not have a ground truth or would be excluded, but the text explicitly states "unanimous agreement."

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• MRMC Study: No, a multi-reader multi-case (MRMC) comparative effectiveness study with human readers (with vs. without AI assistance) was not reported in this summary. The study focused on the standalone performance of the DEEPECHO software compared to expert-derived ground truth.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

• Standalone Study: Yes, a standalone assessment comparing the performance of the DEEPECHO software to ground truth was performed. The letter explicitly states, "DeepEcho performed a stand-alone assessment comparing the performance of the DEEPECHO software to a ground truth of annotations by qualified experts."

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)

• Type of Ground Truth: Expert consensus/annotations. Specifically, for continuous measurements, it was the arithmetic mean of three independent caliper placements by experts. For classification, it was unanimous agreement among three experts.

8. The sample size for the training set

• Training Set Sample Size: Not explicitly quantified in terms of number of images or studies. The text states where the training data was obtained: "Training data were obtained from two Roshan MFM clinics in New York City and selected sites in Morocco."

9. How the ground truth for the training set was established

• Training Set Ground Truth Establishment: The document does not explicitly detail how the ground truth for the training set was established. It only describes the process for the test set. However, given the context, it's highly probable that a similar expert-based annotation process was used for the training data to ensure consistency and quality.

Ask a specific question about this device

The BrightHeart View Classifier device is intended to analyze fetal 2D ultrasound images and video clips using machine learning techniques to automatically detect standard views during fetal heart scanning.

The BrightHeart View Classifier device is intended to be used as an adjunct to the acquisition and interpretation of fetal anatomic ultrasound examinations at the second or third trimester of pregnancy performed with transabdominal probes.

BrightHeart View Classifier is a cloud-based software-only device which uses artificial intelligence (AI) to detect standard views during fetal heart scanning in fetal ultrasound images and video clips.

BrightHeart View Classifier is intended to be used by qualified, trained healthcare professional personnel in a professional prenatal ultrasound (US) imaging environment (this includes sonographers, MFMs, OB/GYN, and Fetal surgeons), to help fetal ultrasound examination acquisition and interpretation of 2D grayscale ultrasound by providing automatic classification of video clips and images into standard views, by automatically extracting example frames of standard views from video clips, and by automatically assessing whether the documentation of each standard view in video clips and images satisfies an acquisition protocol defined by the center. Annotated DICOM files generated by the device cannot be modified by the user.

Here's a breakdown of the acceptance criteria and the study details for the BrightHeart View Classifier, based on the provided FDA 510(k) Clearance Letter:

1. Table of Acceptance Criteria and Reported Device Performance

The FDA letter does not explicitly state pre-defined acceptance criteria values that the device needed to meet. Instead, it reports the device's performance metrics directly from the validation study. However, based on the performance report, we can infer the achieved performance and understand that these values were deemed sufficient for clearance.

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size: 2290 clinically acquired images and frames from video clips.
• Number of Fetal Ultrasound Examinations: 579
• Country of Origin of Data: U.S.A. and France.
• Retrospective or Prospective: The document implies retrospective data ("clinically acquired images and frames").

3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications

• Number of Experts: Two experts: "a sonographer and an MFM specialist".
• Qualifications: "with experience in fetal echocardiography". Specific years of experience are not mentioned.

4. Adjudication Method for the Test Set

• Adjudication Method: Independence was maintained in the ground truth establishment. "The reference standard was derived from the dataset through a truthing process in which a sonographer and an MFM specialist with experience in fetal echocardiography determined the presence or absence of standard views on fetal ultrasound images. The truthing process was conducted independently of the BrightHeart View Classifier device." This indicates a consensus or independent review process, but not a specific 2+1 or 3+1 adjudication as those usually imply a tie-breaker. It seems like both experts independently determined the ground truth.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not explicitly described. The study evaluated the standalone performance of the AI device.

6. Standalone (Algorithm Only Without Human-in-the-Loop) Performance

• Yes, a standalone performance study was conducted. The reported sensitivity and specificity values are for the BrightHeart View Classifier identifying standard views on its own.

7. Type of Ground Truth Used

• Type of Ground Truth: Expert consensus (from a sonographer and an MFM specialist with experience in fetal echocardiography).

8. Sample Size for the Training Set

• The sample size for the training set is not explicitly stated in the provided document. The document only mentions that "The ultrasound examinations used for training and validation are entirely distinct from the examinations used in performance testing."

9. How the Ground Truth for the Training Set Was Established

• The document does not explicitly state how the ground truth for the training set was established. However, given the nature of the device and the ground truth method for the test set, it is highly probable that a similar expert review and annotation process was used for the training data.

Ask a specific question about this device

The diagnostic ultrasound system and probes are designed to obtain ultrasound images and analyze body fluids.

The clinical applications include: Fetal/Obstetrics, Abdominal, Gynecology, Pediatric, Small Organ, Neonatal Cephalic, Adult Cephalic, Trans-rectal, Muscular-Skeletal (Conventional, Superficial), Urology, Cardiac Adult, Cardiac Pediatric, Trans-esophageal (Cardiac), Peripheral vessel, Lung and Dermatology.

It is intended for use by, or by the order of, and under the supervision of, an appropriately trained healthcare professional who is qualified for direct use of medical devices. It can be used in hospitals (includes emergency room), private practices, clinics and similar care environment for clinical diagnosis of patients.

Modes of Operation: 2D mode, Color Doppler mode, Power Doppler (PD) mode, M mode, Pulsed Wave (PW) Doppler mode, Continuous Wave (CW) Doppler mode, Tissue Doppler Imaging (TDI) mode, Tissue Doppler Wave (TDW) mode, ElastoScan Mode, Combined modes, Multi-Image mode(Dual, Quad), 3D/4D mode.

The HM70 EVO is a general purpose, mobile, software controlled, diagnostic ultrasound system. Its function is to acquire ultrasound data and to display the data as 2D mode. M mode, Color Doppler imaging, Power Doppler imaging (including Directional Power Doppler mode; S-Flow), PW Spectral Doppler mode, CW Spectral Doppler mode, Harmonic imaging(S-Harmonic), Tissue Doppler imaging, Tissue Doppler Wave, Panoramic Imaging, Freehand 3D, 3D imaging mode (real-time 4D imaging mode), Elastoscan Mode or as a combination of these modes. The HM70 EVO also gives the operator the ability to measure anatomical structures and offers analysis packages that provide information that is used to make a diagnosis by competent health care professionals. The HM70 EVO has real time acoustic output display with two basic indices, a mechanical index and a thermal index. which are both automatically displayed.

The provided text describes two AI-based features of the HM70 EVO Diagnostic Ultrasound System: UterineAssist and NerveTrack. The acceptance criteria and performance studies for each are detailed below.

UterineAssist

1. Table of Acceptance Criteria and Reported Device Performance

For UterineAssist, the document details performance for three areas: image segmentation, feature points extraction, and size measurement. While explicit acceptance criteria values (like a minimum percentage or maximum error) are not stated in a direct acceptance criteria table, the reported device performance serves as the evidence of meeting internal acceptance.

2. Sample Sizes Used for the Test Set and Data Provenance

• Segmentation Test:

  • Sample Size: 450 sagittal uterus images and 150 transverse uterus images (total 600 images).
  • Data Provenance: Collected at three hospitals. Mix of retrospective and prospective data collection in clinical practice.
  • Country of Origin: All Koreans (implies South Korea).

• Feature Points Extraction Test & Size Measurement Test:

  • Sample Size: 45 sagittal and 41 transverse plane images of uterus (total 86 images).
  • Data Provenance: Collected at three hospitals. Mix of retrospective and prospective data collection in clinical practice.
  • Country of Origin: All Koreans (implies South Korea).

3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications

• Number of Experts: Three participating OB/GYN experts.
• Qualifications: Each had more than 10 years' experience.

4. Adjudication Method for the Test Set

• Method: The set of images (uterus and endometrium) were divided into 3 subsets. Each of the three OB/GYN experts drew the ground truths for one of the subsets. The ground truths drawn by one expert were then cross-checked by the other two experts. Any images not meeting inclusion/exclusion criteria were excluded. This can be described as a 1+2 cross-check adjudication method.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done

• No MRMC comparative effectiveness study is reported for UterineAssist, as this section only describes the standalone performance metrics.

6. If a Standalone (algorithm only without human-in-the-loop performance) was done

• Yes, the reported performance metrics (Dice-score, error measurements) reflect the "standalone" performance of the algorithm.

7. The Type of Ground Truth Used

• The ground truth for both segmentation and feature points/size measurements was established by expert consensus/adjudication from three experienced OB/GYN experts.

8. The Sample Size for the Training Set

• The training data details (specific sample size) are not provided, but it is stated that it is independent of the test data.

9. How the Ground Truth for the Training Set was Established

• Not explicitly stated, but implicitly, similar expert labeling or other reliable methods would have been used, consistent with the independent test data approach. It is only mentioned that the training and test data sets are "completely separated" and there is "no overlap."

NerveTrack

1. Table of Acceptance Criteria and Reported Device Performance

2. Sample Sizes Used for the Test Set and Data Provenance

• Number of Subjects: 18 (13 Females, 5 Males)
• Number of Images: 2,146
• Age Range: 22-68 years
• BMI Range: 16-31.5
• Data Provenance: Not explicitly stated as retrospective or prospective, but the description of gathering scan data and expert involvement suggests a prospective collection or a specifically designed retrospective collection process for validation.
• Country of Origin: All Koreans (implies South Korea).

3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications

• Number of Experts: Ten anesthesiologists and five sonographers (total 15 experts) for review and confirmation.
• Qualifications: All had more than 10 years of experience.
• Initial Ground Truth Drawing: One anesthesiologist who scanned the ultrasound directly drew the GT.

4. Adjudication Method for the Test Set

• Method: One anesthesiologist who directly scanned the ultrasound drew the initial ground truth (GT) for the nerve location. Then, "two or more other anesthesiologists and sonographers reviewed and confirmed that it was correct." If any mistake was identified during the review, it was revised. This indicates a 1 + (2 or more) consensus/adjudication method.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done

• No, the document describes "standalone performance" validation of the NerveTrack algorithm, specifically focusing on accuracy and speed. It does not mention any MRMC study comparing human readers with and without AI assistance.

6. If a Standalone (algorithm only without human-in-the-loop performance) was done

• Yes, the validation clearly states, "The standalone performance of NerveTrack was evaluated..." and provides performance metrics (accuracy and speed) for the algorithm itself.

7. The Type of Ground Truth Used

• The ground truth for the location of 10 different kinds of nerves was established by expert consensus/adjudication involving anesthesiologists and sonographers with significant experience.

8. The Sample Size for the Training Set

• The training data details (specific sample size) are not provided, but it is stated that it is independent of the test data.

9. How the Ground Truth for the Training Set was Established

• Not explicitly stated, but it is mentioned that the "training data used for the training of the NerveTrack algorithm are independent of the data used to test the NerveTrack algorithm." This implies a separate, established ground truth for the training set, likely using similar expert labeling methods.

Ask a specific question about this device