(188 days)
The D2P software is intended for use as a software interface and image segmentation system for the transfer of imaging information from a medical scanner such as a CT scanner to an output file. It is also intended as pre-operative software for surgical planning.
3D printed models generated from the output file are meant for visual, non-diagnostic use.
The D2P software is a stand-alone modular software package that allows easy to use and quick digital 3D model preparation for printing or use by third party applications. The software is aimed at usage by medical staff, technicians, nurses, researchers or lab technicians that wish to create patient specific digital anatomical models for variety of uses such as training, education, and pre-operative surgical planning. The patient specific digital anatomical models may be further used as an input to a 3D printer to create physical models for visual, non-diagnostic use. This modular package includes, but is not limited to the following functions:
- DICOM viewer and analysis
- Automated segmentation
- Editing and pre-printing .
- Seamless integration with 3D Systems printers .
- Seamless integration with 3D Systems software packages .
The provided documentation, K161841 for the D2P software, does not contain detailed information regarding the specific acceptance criteria and the comprehensive study proof requested in the prompt. The document primarily focuses on the regulatory submission process, demonstrating substantial equivalence to a predicate device (Mimics, Materialise N.V., K073468).
The "Performance Data" section mentions several studies (Software Verification and Validation, Phantom Study, Usability Study - System Measurements, Usability Study – Segmentation, Segmentation Study) and states that "all measurements fell within the set acceptance criteria" or "showed similarity in all models." However, it does not explicitly list the acceptance criteria or provide the raw performance metrics to prove they were met.
Therefore, I cannot fully complete the requested table and answer all questions based solely on the provided text. I will, however, extract all available information related to performance and study design.
Here's a breakdown of what can be extracted and what information is missing:
Information NOT available in the provided text:
- Explicit Acceptance Criteria Values: The exact numerical values for the acceptance criteria for any of the studies (e.g., specific error margins for measurements, quantitative metrics for segmentation similarity).
- Reported Device Performance Values: The specific numerical performance metrics achieved by the D2P software in any of the studies (e.g., actual measurement deviations, Dice coefficients for segmentation).
- Sample Size for the Test Set: While studies are mentioned, the number of cases or subjects in the test sets for the Phantom, Usability, or Segmentation studies is not specified.
- Data Provenance (Country of Origin, Retrospective/Prospective): This information is not provided for any of the studies.
- Number of Experts and Qualifications for Ground Truth: No details are given about how many experts were involved in establishing ground truth (if applicable) or their qualifications.
- Adjudication Method: Not mentioned.
- Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study: The document doesn't describe an MRMC study comparing human readers with and without AI assistance, nor does it provide an effect size if one were done. The studies mentioned focus on the device's technical performance and user variability.
- Standalone (Algorithm-only) Performance: While the D2P software is a "stand-alone modular software package," the details of the performance studies don't explicitly differentiate between algorithm-only performance and human-in-the-loop performance. The Usability Studies do involve users, suggesting human interaction.
- Type of Ground Truth Used (Pathology, Outcomes Data, etc.): For the Phantom Study, the ground truth is the "physical phantom model." For segmentation and usability studies, it appears to be based on comparisons between the subject device, predicate device, and/or inter/intra-user variability, but the ultimate "ground truth" (e.g., expert consensus on clinical cases, pathological confirmation) is not specified.
- Sample Size for the Training Set: No information is provided about the training set or how the algorithms within D2P were trained.
- Ground Truth Establishment for Training Set: No information is provided about how ground truth for a training set (if one existed) was established.
Information available or inferable from the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
| Performance Metric/Study | Acceptance Criteria (Stated as met) | Reported Device Performance (Stated as met) |
|---|---|---|
| Phantom Study | Not explicitly quantified (e.g., "all measurements fell within the set acceptance criteria") | Not explicitly quantified (e.g., "all measurements fell within the set acceptance criteria") |
| Usability Study – System Measurements (Inter/Intra-user variability) | Not explicitly quantified (e.g., "all measurements fell within the set acceptance criteria") | Not explicitly quantified (e.g., "all measurements fell within the set acceptance criteria") |
| Usability Study – Segmentation | Not explicitly quantified (e.g., "showed similarity in all models") | Not explicitly quantified (e.g., "showed similarity in all models") |
| Segmentation Study | Not explicitly quantified (e.g., "showed similarity in all models") | Not explicitly quantified (e.g., "showed similarity in all models") |
2. Sample size used for the test set and the data provenance:
- Sample Size for Test Set: Not specified for any of the studies (Phantom, Usability, Segmentation).
- Data Provenance: Not specified (e.g., country of origin, retrospective/prospective). The phantom study used a physical phantom model. For patient data in segmentation/usability studies, the provenance is not mentioned.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Number of Experts: Not specified.
- Qualifications of Experts: Not specified.
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set:
- Not specified.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, if so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No evidence of an MRMC comparative effectiveness study of human readers with vs. without AI assistance is detailed in this document. The Usability Studies assessed inter/intra-user variability of measurements and segmentation similarity, indicating human interaction with the device, but not a comparative study demonstrating improvement in reader performance due to the AI.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- The D2P software is described as a "stand-alone modular software package." The "Software Verification and Validation Testing" and "Segmentation Study" imply assessment of the software's inherent capabilities. However, the presence of "Usability Studies" involving human users suggests that human-in-the-loop performance was also part of the evaluation, but it's not explicitly segmented as "algorithm only" vs. "human-in-the-loop with AI assistance." The document doesn't provide distinct results for an "algorithm only" performance metric.
7. The type of ground truth used:
- Phantom Study: The ground truth was the "physical phantom model." Comparisons were made between segmentations created by the subject and predicate device from a CT scan of this physical phantom.
- Usability Study – System Measurements: Ground truth appears to be based on comparing inter and intra-user variability in measurements taken within the subject device. The reference for what constitutes "ground truth" for these measurements (e.g., true anatomical measures) is not explicitly stated beyond comparing user consistency.
- Usability Study – Segmentation / Segmentation Study: Ground truth for these studies is implied by "comparison showed similarity in all models" or comparison between subject and predicate devices. This suggests a relative ground truth (e.g., consistency across methods/users) rather than an absolute ground truth like pathology.
8. The sample size for the training set:
- Not specified. The document does not describe the specific training of machine learning algorithms, only the software's intended use and performance validation.
9. How the ground truth for the training set was established:
- Not specified, as information about a training set is not provided.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).