(118 days)
Bladder Al uses machine-learning techniques to aid in the quantification of bladder volume from ultrasound images. The device is intended to be used on images of patients aged two years or older.
Bladder Al is a standalone software as a medical device (SaMD) that helps qualified users with image-based assessment of bladder ultrasound images in patients aged 2 or older. It is designed to simplify workflow by helping trained healthcare providers evaluate, quantify, and generate reports for bladder ultrasound images.
Bladder Al takes as an input imported Digital Imaging and Communications in Medicine (DICOM) images from ultrasound scanners and allows users to measure bladder volumes of a single frame and multi-frame ultrasound images, as well as create and finalize examination reports. It provides users with a specific toolset for viewing ultrasound images of the bladder, placing landmarks, and creating reports.
Key features of the software are
- ML-based semi-automatic landmark placements
- Bladder dimension and volume measurements
- Report generation
Here's a summary of the acceptance criteria and the study proving the device meets them, based on the provided text:
Bladder AI (AIBV01) Performance Study Summary
1. Acceptance Criteria and Reported Device Performance:
The document doesn't explicitly state "acceptance criteria" with numerical thresholds directly. However, it demonstrates performance by reporting accuracy and reliability metrics that would implicitly serve as the criteria for clearance. The study's conclusion that "the algorithm performance is reliable and accurate compared to expert clinician" and that "the results support the generalizability of the Bladder Al across the intended patient population" suggests these metrics met the internal or regulatory expectations.
| Metric Type | Acceptance Criteria (Implicit) | Reported Device Performance (Bladder AI) | Note |
|---|---|---|---|
| Accuracy | Acceptable Mean Volume Difference | ||
| Dual-View Bladder Volume | 2 mL (LoA: -42 to 46) | Mean volume difference compared to expert consensus. LoA: Limits of Agreement. | |
| Single-View Bladder Volume | 3 mL (LoA: -49 to 55) | Mean volume difference compared to expert consensus. LoA: Limits of Agreement. | |
| Reliability | Acceptable Intraclass Correlation Coefficient (ICC) | ||
| Dual-View Bladder Volume | 0.98 | ICC measures consistency or agreement between measurements. A higher value (closer to 1) indicates better reliability. | |
| Single-View Bladder Volume | 0.97 | ICC measures consistency or agreement between measurements. A higher value (closer to 1) indicates better reliability. |
2. Sample Size and Data Provenance for the Test Set:
- Sample Size: 122 subjects.
- Data Provenance: A diverse collection of clinical sites in metropolitan cities, chosen to provide a broad range of demographic variables (ethnicity, gender, age 2 to 95 years old). The document does not specify the country of origin, but "metropolitan cities" implies a broad geographic reach within a developed context. The test data was retrospective as it consists of "images acquired from cart-based and portable ultrasound devices."
3. Number of Experts and Qualifications for Ground Truth Establishment (Test Set):
- Number of Experts: Three expert clinicians.
- Qualifications: The document states they were "expert clinicians." Specific experience levels (e.g., "radiologist with 10 years of experience") are not provided.
4. Adjudication Method for the Test Set:
- Adjudication Method: The ground truth for bladder volume was obtained as the average bladder volume measurement among three expert clinicians. This suggests a form of consensus or averaging method, rather than a 2+1 or 3+1 rule for disagreement.
5. MRMC Comparative Effectiveness Study:
- Was an MRMC study done? No. The study assessed the standalone performance of the Bladder AI compared to expert consensus. There is no mention of human readers using the AI for assistance and comparing their performance with and without AI.
- Effect Size of Human Readers Improvement with AI vs. without AI assistance: Not applicable, as no MRMC study was conducted.
6. Standalone Performance:
- Was standalone (algorithm only without human-in-the-loop performance) done? Yes. The performance metrics (Mean volume difference, ICC) are for the Bladder AI's measurements compared directly to the expert-established ground truth value.
7. Type of Ground Truth Used:
- Type of Ground Truth: Expert consensus. Specifically, the "average bladder volume measurement among three expert clinicians."
8. Sample Size for the Training Set:
- The document implies a training set was used, stating "Training and validation datasets have been selected and maintained to be appropriately independent of one another." However, the specific sample size for the training set is not provided in the excerpt.
9. How the Ground Truth for the Training Set was Established:
- The document states that the "Training and validation datasets have been selected and maintained to be appropriately independent of one another." Similar to the training set sample size, the method for establishing ground truth for the training set is not explicitly detailed in the provided text. It is reasonable to infer it would involve expert review, but the specifics are not elucidated.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).