Search Results
Found 2 results
510(k) Data Aggregation
(110 days)
The diagnostic ultrasound system and probes are designed to obtain ultrasound images and analyze body fluids.
The clinical applications include: Fetal/Obstetrics, Abdominal, Gynecology, Intraoperative, Pediatric, Small Organ, Neonatal Cephalic, Adult Cephalic, Trans vaginal, Muscular Skeletal (Conventional, Superficial), Urology, Cardiac Adult, Cardiac Pediatric, Trans esophageal(Cardiac) and Peripheral vessel.
It is intended for use by, or by the order of, and under the supervision of, an appropriately trained healthcare professional who is qualified for direct use of medical devices. It can be used in hospitals, private practices, clinics and similar care environment for clinical diagnosis of patients.
Modes of Operation: 2D mode, Color Doppler mode, Pulsed Wave (PW) Doppler mode, Continuous Wave (CW) Doppler mode, Tissue Doppler Imaging (TDI) mode, Tissue Doppler Wave (TDW) mode, Power Doppler (PD) mode, ElastoScan™ Mode, Multi Image mode(Dual, Quad), Combined modes. 3D/4D mode.
The RS85 is a general purpose, mobile, software controlled, diagnostic ultrasound system. Its function is to acquire ultrasound data and to display the data as 2D mode, Color Doppler imaging, Power Doppler imaging (including Directional Power Doppler mode; S-Flow), PW Spectral Doppler mode, CW Spectral Doppler mode, Harmonic imaging, Tissue Dopler imaging, Tissue Doppler Wave, 3D imaging mode (real-time 4D imaging mode), Elastoscan* Mode, MV-Flow Mode or as a combination of these modes. The RS85 also gives the operator the ability to measure anatomical structures and offers analysis packages that provide information that may aid in making a diagnosis by competent health care professionals. the RS85 has real time acoustic output display with two basic indices, a mechanical index, which are both automatically displayed.
This document describes the validation of AI-based features for the Samsung Medison RS85 Diagnostic Ultrasound System, specifically focusing on NerveTrack's detection, segmentation, and EzNerveMeasure functionalities, as well as SonoSync.
Here's a breakdown of the requested information:
Acceptance Criteria and Reported Device Performance
| Feature | Acceptance Criteria | Reported Device Performance |
|---|---|---|
| NerveTrack (Detection) | Not explicitly stated as a numerical acceptance criterion, but the validation study aimed to demonstrate a high accuracy and reasonable speed for nerve detection. A detection is considered correct if the Dice coefficient is 0.5 or more. | Average accuracy from 10 image sequences: 89.6% (95% Confidence Interval: 86.41, 92.79) Average speed (fps): 14.49 (95% CI: 13.86, 15.11) |
| NerveTrack (Segmentation) | Not explicitly stated as a numerical acceptance criterion, but the validation study aimed to demonstrate high accuracy, Dice similarity coefficient, and low Hausdorff distance for nerve segmentation. A segmentation is considered correct if the Dice coefficient is 0.5 or more. | Average accuracy from nine image sequences: 99.78% (95% Confidence Interval: 99.34, 100) Average speed (fps): 10.44 (95% CI: 10.03, 10.85) Average Dice similarity coefficient: 90.44% (95% Confidence Interval: 86.19, 94.69) 95% Hausdorff distance (excluding bone): 18.69 pixels (95% Confidence Interval: 9.21, 28.17) Hausdorff distance (for bone): 93.51 pixels (95% Confidence Interval: 27.31, 159.72) |
| NerveTrack (EzNerveMeasure - FR) | Not explicitly stated as a numerical acceptance criterion, but the validation study aimed to demonstrate a low error rate for Flattening Ratio (FR) measurements. | Average error rate of FR: 6.11% (95% Confidence Interval: 5.10, 7.12) |
| NerveTrack (EzNerveMeasure - CSA) | Not explicitly stated as a numerical acceptance criterion, but the validation study aimed to demonstrate a low error rate for Cross-sectional Area (CSA) measurements. | Average error rate of CSA: 9.75% (95% Confidence Interval: 8.54, 10.96) |
| SonoSync | Pre-determined criteria were utilized to assess whether remote viewing and reviewing matched the performance of local ultrasound systems. (Specific numerical criteria not provided in the document). | Labeling materials are provided to inform users about the necessary specifications for safely and effectively conducting remote diagnostic reviews and viewing. The document states that validation tests assessed if performance matched local systems. |
Study Details
1. Sample Sizes and Data Provenance
| Feature | Test Set Sample Size | Data Provenance | Retrospective/Prospective |
|---|---|---|---|
| NerveTrack (Detection) | 3,999 nerve images | Eight hospitals in Korea (Ethnicity: Koreans) | Prospective |
| NerveTrack (Segmentation) | 1,753 nerve images | Ten hospitals in Korea (Ethnicity: Koreans) | Prospective |
| NerveTrack (EzNerveMeasure) | 50 median nerve images | A hospital in Korea (Ethnicity: Koreans) | Prospective |
2. Number of Experts and Qualifications for Ground Truth
- Number of Experts: Three participating experts.
- Qualifications of Experts:
- One anesthesiologist with more than 10 years of experience in pain management performed the initial manual drawing of nerve areas.
- Other doctors with more than 10 years of experience performed verification of the ground truth.
3. Adjudication Method for the Test Set
The adjudication method appears to be a 2+1 process:
- Initial ground truth (GT) data was manually drawn by one anesthesiologist.
- Other doctors (presumably at least two distinct individuals to form a consensus if needed, though the document only states "other doctors") with more than 10 years of experience checked every frame.
- If discrepancies ("did not agree on nerve locations/contours") arose, necessary corrections were made to finalize the GT. This indicates a consensus-based approach for disagreement resolution, but the specific number for resolving disagreements is not explicitly 2+1; it just states "other doctors."
4. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No, a multi-reader multi-case (MRMC) comparative effectiveness study was not performed according to the provided text. The studies focus on the standalone performance of the AI algorithms.
5. Standalone Performance Study
Yes, standalone (algorithm only without human-in-the-loop) performance studies were done for NerveTrack's detection, segmentation, and EzNerveMeasure functionalities. The reported results (accuracy, speed, Dice coefficient, Hausdorff distance, and error rates) are direct measurements of the algorithm's performance against established ground truth.
6. Type of Ground Truth Used
The ground truth used was expert consensus. It was established by manual delineation (drawing of nerve areas/contours) by an anesthesiologist with over 10 years of experience, followed by verification and correction (if needed) by other doctors also with over 10 years of experience.
7. Sample Size for the Training Set
The document explicitly states: "Data used for training, tuning and validation purpose are completely separated from the ones during training process, and there is no overlap among the three." However, the exact sample size for the training set is not provided in this document. It only gives the test set sizes.
8. How Ground Truth for the Training Set Was Established
While the exact size of the training set is not given, the method for establishing its ground truth is described similarly to the validation set:
- The GT data for training, tuning, and validation were built by three participating experts.
- Nerve areas were manually drawn by an anesthesiologist with more than 10 years of experience in pain management.
- The doctors who scanned the ultrasound were directly involved in the construction of GT data.
- For verification, other doctors with more than 10 years of experience checked every frame.
- If they did not agree on nerve locations/contours, necessary corrections were made to create the final GT.
This suggests that the ground truth for the training set was also established through expert consensus and manual delineation, following the same "Truthing" process as the test set.
Ask a specific question about this device
(109 days)
The diagnostic ultrasound system and transducers are designed to obtain ultrasound images and analyze body fluids.
The clinical applications include: Fetal/Obstetrics, Abdominal, Gynecology, Intra-operative, Small Organ, Neonatal Cephalic, Adult Cephalic, Trans-vaginal, Muscular-Skeletal (Conventional, Superficial), Urology, Cardiac Adult, Cardiac Pediatric, Thoracic, Trans-esophageal (Cardiac) and Peripheral vessel.
It is intended for use by, or by the order of, and under the supervision of, an appropriately trained healthcare professional who is qualified for direct use of medical devices. It can be used in hospitals, private practices, clinics and similar care environment for clinical diagnosis of patients.
Modes of Operation: 2D mode, Color Doppler mode, Power Doppler (PD) mode, M mode, Pulsed Wave (PW) Doppler mode, Continuous Wave (CW) Doppler mode, Tissue Doppler Imaging (TDI) mode, Tissue Doppler Wave (TDW) mode, ElastoScan Mode, Combined modes, Multi-Image mode(Dual, Quad), 3D/4D mode
The V8 / V7 / V6 / H8 / H7 / H6 are a general purpose, mobile, software controlled, diagnostic ultrasound system. Their function is to acquire ultrasound data and to display the data as 2D mode, Color Doppler mode, Power Doppler (PD) mode, M mode, Pulsed Wave (PW) Doppler mode, Continuous Wave (CW) Doppler mode, Tissue Doppler Imaging (TDI) mode, Tissue Doppler Wave (TDW) mode, ElastoScan Mode, Combined modes, Multi-Image mode(Dual, Quad), 3D/4D mode. The V8 / V7 / V6 / H8 / H7 / H6 also give the operator the ability to measure anatomical structures and offer analysis packages that provide information that is used to make a diagnosis by competent health care professionals. The V8 / V7 / V6 / H8 / H7 / H6 have a real time acoustic output display with two basic indices, a mechanical index and a thermal index, which are both automatically displayed.
Here's an analysis of the acceptance criteria and the study proving the device meets those criteria, based on the provided document:
Device: Samsung Medison V8/H8, V7/H7, V6/H6 Diagnostic Ultrasound System with NerveTrack AI
1. Table of Acceptance Criteria and Reported Device Performance
| Validation Type | Definition | Acceptance Criteria | Reported Device Performance (Average) | Standard Deviation | 95% CI |
|---|---|---|---|---|---|
| Nerve Detection | |||||
| Accuracy (%) | Number of correctly detected frames / Total number of frames with nerve × 100 | ≥ 80% | 90.3% | 4.8 | 88.6 to 92.0 |
| Speed (FPS) | 1000 / Average latency time of each frame (msec) | ≥ 2 FPS | 3.61 | 0.25 | 3.43 to 3.78 |
| Nerve Segmentation | |||||
| Accuracy (%) | Number of correctly segmented frames / Total number of frames with nerve × 100 | ≥ 80% | 98.69% | 0.64 | 96.31 to 100 |
| Speed (FPS) | 1000 / Average latency time of each frame (msec) | ≥ 2 FPS | 3.62 | 0.36 | 3.49 to 3.75 |
2. Sample Size Used for the Test Set and Data Provenance
-
Nerve Detection Test Set:
- Number of Subjects: 18 (13 females, 5 males)
- Number of Images/Frames: 2,146
- Data Provenance: All Koreans. The document does not explicitly state if the data was retrospective or prospective. However, the description of data collection (sliding transducer at specific speeds) suggests it was collected for the purpose of this study, indicating a prospective component or at least an intentionally collected dataset.
-
Nerve Segmentation Test Set:
- Number of Subjects: 11 (8 females, 3 males)
- Number of Images/Frames: 3,836
- Data Provenance: All Koreans. Similar to the detection dataset, the provenance is Korean, and the collection method description points towards intentionally collected data.
3. Number of Experts Used to Establish Ground Truth for the Test Set and Their Qualifications
- Number of Experts: 15 experts were involved (10 anesthesiologists and 5 sonographers).
- Qualifications of Experts: All experts had "more than 10 years of experience."
4. Adjudication Method for the Test Set
The ground truth establishment method was as follows:
- One anesthesiologist who scanned the ultrasound directly drew the initial ground truth (GT) of the nerve location.
- "Two or more other anesthesiologists and sonographers reviewed and confirmed that it was correct."
- "If there was any mistake during the review, it was revised again."
This describes a form of consensus-based adjudication with an initial ground truth creator and subsequent confirmation/revision by multiple independent experts. It's not a strict N+M or sequential read, but rather a collaborative review and refinement process.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done
No, the document does not indicate that a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was done to compare human readers with vs. without AI assistance. The study focuses solely on the standalone performance of the AI algorithm (NerveTrack).
6. If a Standalone (algorithm only without human-in-the-loop performance) was done
Yes, the document explicitly states: "The standalone performance of NerveTrack was evaluated..." The "Summary Performance data" tables provided are for the algorithm's performance without a human in the loop.
7. The Type of Ground Truth Used
The ground truth used was expert consensus. It was established by a team of experienced anesthesiologists and sonographers who reviewed and confirmed the actual nerve locations in the ultrasound images.
8. The Sample Size for the Training Set
The document states: "The training data used for the training of the NerveTrack algorithm are independent of the data used to test the NerveTrack algorithm." However, the exact sample size for the training set is not provided in the given text.
9. How the Ground Truth for the Training Set was Established
The document mentions that the training data is independent of the test data. While it does not explicitly detail the ground truth establishment method for the training set, it is highly probable that a similar expert-based annotation process (as described for the test set) was used to establish the ground truth for the training data. This is a common practice in AI development to ensure consistency in data labeling.
Ask a specific question about this device
Page 1 of 1