(59 days)
Indications for use of TOMTEC-ARENA software are quantification and reporting of cardiovascular, fetal, and abdominal structures and function of patients with suspected disease to support the physician in the diagnosis.
TOMTEC-ARENA is a clinical software package for reviewing, quantifying and reporting digital medical data. The software can be integrated into third party platforms. Platforms enhance the workflow by providing the database, import, export and other services. All analyzed data and images will be transferred to the platform for archiving, reporting and statistical quantification purposes.
TTA2 consists of the following optional modules:
- IMAGE-COM
- I REPORTING
- AutoStrain LV / SAX / RV / LA I
- 2D CPA
- FETAL 2D CPA ■
- 4D LV-ANALYSIS
- . 4D RV-FUNCTION
- I 4D CARDIO-VIEW
- I 4D MV-ASSESSMENT
- I 4D SONO-SCAN
- TOMTEC DATACENTER (incl. STUDY LIST, DATA MAINTENANCE, WEB ■ REVIEW)
The purpose of this traditional 510(k) pre-market notification is to introduce semi-automated cardiac measurements based on an artificial intelligence and machine learning (AI/ML) algorithm. The Al/ML algorithm is a Convolutional Network (CNN) developed using a Supervised Learning approach. This Al/ML algorithm enables TOMTEC-ARENA to produce semi-automated and editable echocardiographic measurements on BMODE and DOPPLER datasets. The algorithm was developed using a controlled internal process that defines activities from the inspection of input data to the training and deployment of the algorithm: The training process begins with the model observing, and optimizing its parameters based on the training pool data. The model's prediction and performance are then evaluated against the test pool. The test pool data is set aside at the beginning of the project. During the training process, the Al/ML algorithm learned to predict measurements by being presented with a large number of echocardiographic data manually generated by qualified healthcare professionals. The echocardiographic studies were randomly assigned to be either used for training (approx. 2,800 studies) or testing (approx. 500 studies). A semi-automated measurement consists of a cascade of detection steps. It starts with a rough geometric estimate, which is subsequently refined more and more: The user selects a frame on which the semi-automated measurements shall be performed in TOMTEC-ARENA. Image- & metadata, e.g. pixel spacing, are transferred to the semi-automated measurement detector. The semi-automated measurement detector predicts the position of start and end caliper in the pixel coordinate system. These co-coordinates are transferred back to the CalcEngine, which converts the received data back into real world coordinates (e.g. mm) and creates the graphical overlay. This superimposed line can be edited by the user. The end user can edit, accept, or reject the measurement(s). This feature does not introduce any new measurements, but allows the end user to perform semi-automated measurements. The end user can also still perform manual measurements and it is not mandatory to use the semi-automated measurements. The semi-automated measurements are licensed separately.
Here's an analysis of the acceptance criteria and study details for the TOMTEC-ARENA device, based on the provided FDA 510(k) summary:
The 510(k) summary describes the TOMTEC-ARENA software, which introduces semi-automated cardiac measurements based on an AI/ML algorithm. The primary focus of the non-clinical performance data is on software verification, risk analysis, and usability evaluation, as no clinical testing was conducted.
1. Table of Acceptance Criteria and Reported Device Performance
The provided document does not explicitly list quantitative acceptance criteria for the AI/ML algorithm's performance in terms of accuracy or precision of the semi-automated measurements. Instead, it states that "Completion of all verification activities demonstrated that the subject device meets all design and performance requirements." and "Testing performed demonstrated that the proposed TOMTEC-ARENA (TTA2.50) meets defined requirements and performance claims." These are general statements rather than specific, measurable performance metrics.
Similarly, there are no reported quantitative device performance metrics (e.g., accuracy, sensitivity, specificity, or error rates) for the AI/ML algorithm's measurements mentioned in this summary. The summary focuses on the functional equivalence and safety of the AI-powered feature compared to existing manual measurements and predicate devices.
However, the document does imply a core "acceptance criterion":
| Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|
| Functional Equivalence/Accuracy: The semi-automated measurements (BMODE and DOPPLER) should provide measurement suggestions that are comparable in principle/technology to those included in the reference device and can be edited, accepted, or rejected by the user. | "Support of additional semi-automated measurements compared to reference device. Additional measurements rely on same principle/technology (e.g. line detection, single-point) as those included in reference device." |
| "The measurement suggestion can be edited. Manual measurements as with TTA2.40.00 are still possible." | |
| Safety and Effectiveness: The introduction of semi-automated measurements should not adversely affect the safety and effectiveness of the device. | "No impact to the safety or effectiveness of the device." |
| "Verification activities performed confirmed that the differences in the design did not adversely affect the safety and effectiveness of the subject device." | |
| Usability: The device is safe and effective for intended users, uses, and environments. | "TOMTEC-ARENA has been found to be safe and effective for the intended users, uses, and use environments." |
| Compliance: Adherence to relevant standards (IEC 62304, IEC 62366-1) and internal processes. | "Software verification was performed according to the standard IEC 62304..." |
| "A Summative Usability Evaluation was performed... according to the standard IEC 62366-1..." | |
| "The proposed modifications were tested in accordance with TOMTEC's internal processes." |
Without specific quantitative metrics for the AI's measurement accuracy, it's challenging to provide a detailed performance table. The provided information focuses on the design validation process rather than specific benchmark results for the AI's performance.
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size for Test Set: Approximately 500 studies.
- Data Provenance: The document does not specify the country of origin of the data. It states, "The echocardiographic studies were randomly assigned to be either used for training (approx. 2,800 studies) or testing (approx. 500 studies)." It does not explicitly state if the data was retrospective or prospective. Given that these are "studies" used for training and testing an algorithm, it is highly probable that they are retrospective data sets, collected prior to the algorithm's deployment.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
- Number of Experts: Not specified. The document states "a large number of echocardiographic data manually generated by qualified healthcare professionals." This implies multiple professionals but does not quantify them.
- Qualifications of Experts: "qualified healthcare professionals." Specific qualifications (e.g., radiologist with X years of experience, sonographer, cardiologist) are not provided.
4. Adjudication Method for the Test Set
- Adjudication Method: Not specified. The ground truth was "manually generated by qualified healthcare professionals," but the process for resolving discrepancies among multiple professionals (if multiple were involved per case) is not described.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- Was an MRMC study done? No. The summary explicitly states: "No clinical testing conducted in support of substantial equivalence when compared to the predicate devices." The nature of the AI algorithm as providing semi-automated, editable measurements, rather than a diagnostic output, likely informed this decision. The user is always in the loop and can accept, edit, or reject the AI's suggestions.
- Effect size of human readers improvement with AI vs. without AI assistance: Not applicable, as no MRMC study was performed.
6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study
- Was a standalone study done? Not explicitly detailed in terms of quantitative performance metrics. While the algorithm "predicts the position of start and end caliper in the pixel coordinate system" and this prediction is mentioned as being evaluated against the test pool, the results are not presented as a standalone performance metric. The nature of the device, where the user can "edit, accept, or reject the measurement(s)", strongly implies that standalone performance is not the primary focus for regulatory purposes, as it is always intended to be used with human oversight. The comparison is generally with the predicate device's manual measurement workflow and a reference device's semi-automated features.
7. Type of Ground Truth Used
- Type of Ground Truth: "manually generated by qualified healthcare professionals." This suggests expert consensus or expert-derived measurements serving as the reference standard for the algorithm's training and testing.
8. Sample Size for the Training Set
- Sample Size for Training Set: Approximately 2,800 studies.
9. How the Ground Truth for the Training Set Was Established
- How Ground Truth Was Established: "The Al/ML algorithm learned to predict measurements by being presented with a large number of echocardiographic data manually generated by qualified healthcare professionals." This indicates that human experts manually performed the measurements on the training data, and these manual measurements served as the ground truth for the supervised learning model.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).