Search Results

The KLS Martin Individual Patient Solutions (IPS) Planning System is intended for use as a software system and image segmentation system for the transfer of imaging information from a computerized tomography (CT) medical scan. The input data file is processed by the IPS Planning System and the result is an output data file that may then be provided as digital models or used as input to a rapid prototyping portion of the system that produces physical outputs including anatomical models, guides, and case reports for use in the marking and cutting of cranial bone in cranial surgery. The IPS Planning System is also intended as a pre-operative software tool for simulating / evaluating surgical treatment options. Information provided by the software and device output is not intended to eliminate, replace, or substitute, in whole or in part, the healthcare provider's judgment and analysis of the patient's condition.

The KLS Martin Individual Patient Solutions (IPS) Planning System is a collection of software and associated additive manufacturing (rapid prototyping) equipment intended to provide a variety of outputs to support reconstructive cranial surgeries. The system uses electronic medical images of the patients' anatomy (CT data) with input from the physician, to manipulate original patient images for planning and executing surgery. The system processes the medical images and produces a variety of patient specific physical and/or digital output devices which include anatomical models, guides, and case reports for use in the marking and cutting of cranial bone in cranial surgery.

The provided text is a 510(k) summary for the KLS Martin Individual Patient Solutions (IPS) Planning System. It details the device, its intended use, and comparisons to predicate and reference devices. However, it does not describe specific acceptance criteria and a study dedicated to proving the device meets those criteria in the typical format of a diagnostic AI/ML device submission.

Instead, the document primarily focuses on demonstrating substantial equivalence to a predicate device (K182889) and leveraging existing data from that predicate, as well as two reference devices (K182789 and K190229). The "performance data" sections describe traditional medical device testing (tensile, biocompatibility, sterilization, software V&V) and a simulated design validation testing and human factors and usability testing rather than a clinical study evaluating the accuracy of an AI/ML algorithm's output against a ground truth.

Specifically, there is no mention of:

• Acceptance criteria for an AI/ML model's performance (e.g., sensitivity, specificity, AUC).
• A test set with sample size, data provenance, or ground truth establishment details for AI/ML performance evaluation.
• Expert adjudication methods, MRMC studies, or standalone algorithm performance.

The "Simulated Design Validation Testing" and "Human Factors and Usability Testing" are the closest sections to a performance study for the IPS Planning System, but they are not framed as an AI/ML performance study as requested in the prompt.

Given this, I will extract and synthesize the information available regarding the described testing and attempt to structure it to address your questions, while explicitly noting where the requested information is not present in the provided document.

Acceptance Criteria and Device Performance (as inferred from the document)

The document primarily states that the device passes "all acceptance criteria" for various tests, but the specific numerical acceptance criteria (e.g., minimum tensile strength, maximum endotoxin levels) and reported performance values are generally not explicitly quantified in a table format. The closest to "performance" is the statement that "additively manufactured titanium devices are equivalent or better than titanium devices manufactured using traditional (subtractive) methods."

Since the document doesn't provide a table of acceptance criteria and reported numerical performance for an AI/ML model's accuracy, I will present the acceptance criteria and performance as described for the tests performed:

Detailed Study Information (Based on available text):

1. Sample size used for the test set and the data provenance:

   • Test Set for Simulated Design Validation Testing: A "representative cranial case" was "extrapolated to six (6) distinct age ranges for input data (CT scan) equals output data validation." This implies 6 simulated cases were tested, but no further details on the number of actual CT scans or patients are provided.
   • Test Set for Human Factors and Usability Testing: "Eighteen (18) cases were analyzed" (6 distinct age ranges, with outputs sent to 3 clinical experts, meaning 6 (age ranges) x 3 (experts) = 18 cases analyzed in total by the experts).
   • Data Provenance: Not specified for the "representative cranial case" in simulated design validation. For human factors, it implicitly used outputs derived from the "six (6) distinct age ranges" based on the system's processing. The document does not specify if the data was retrospective or prospective, or the country of origin.

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • Simulated Design Validation Testing: Not explicitly stated that experts established ground truth for this test. It seems to be a technical validation against the design specifications.
   • Human Factors and Usability Testing: "Three separate clinical experts" were used to review the outputs. Their qualifications are not specified beyond being "clinical experts." Their role was to analyze for potential use problems and make recommendations, and confirm applicability to real-life situations. This is not the establishment of ground truth in the sense of a diagnostic classification.

3. Adjudication method (e.g. 2+1, 3+1, none) for the test set:

   • No adjudication method is described for either the simulated design validation or human factors/usability testing. The human factors testing involved reviews by multiple experts, but no process for reconciling disagreements or establishing a consensus "ground truth" among them is mentioned; they each provided independent feedback.

4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • An MRMC comparative effectiveness study was not conducted according or described in this document. The device is not presented as an AI-assisted diagnostic tool that improves human reader performance in the traditional sense. It's a pre-operative planning system that processes CT data to create physical/digital outputs. The "Human Factors and Usability Testing" involved multiple readers (clinical experts) and multiple cases, but it was for usability assessment rather than a comparative effectiveness study of AI assistance.

5. If a standalone (i.e. algorithm only without human-in-the loop performance) was done:

   • The document describes "Software Verification and Validation" which is a form of standalone testing for the software applications used. It states that "all software requirements and specifications were implemented correctly and completely." However, this is a validation of the software's functionality and adherence to specifications, not a performance study of an AI/ML algorithm's accuracy in a diagnostic context. The system is explicitly described as requiring "trained employees/engineers who utilize the software applications to manipulate data and work with the physician to create the virtual planning session," indicating a human-in-the-loop process for generating the final outputs.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc):

   • Simulated Design Validation Testing: The "ground truth" appears to be the "initial input data (.STL)" and the design specifications; the test verifies that "output data (CT scan) equals output data validation" (likely intended to mean input equals output, or input from CT leads to correct output).
   • Human Factors and Usability Testing: The "ground truth" is effectively the "expert opinion" of the three clinical experts regarding the usability and applicability of the outputs, rather than a definitive medical diagnosis.

7. The sample size for the training set:

   • The document describes the KLS Martin IPS Planning System as using "commercially off-the-shelf (COTS) software applications" (Materialise Mimics and Geomagic® Freeform PlusTM) for image segmentation and manipulation. This implies that the core algorithms were pre-existing and not developed by KLS Martin as a novel AI/ML model that would require a distinct training set outlined in this submission. Therefore, no information on a training set size is provided for the device.

8. How the ground truth for the training set was established:

   • Not applicable, as no training set for a novel AI/ML model by KLS Martin is described. The COTS software validation would have been performed by their respective developers.

Ask a specific question about this device