(257 days)
The Medical Modeling VSP® System is intended for use as a software system and image segmentation system for the transfer of imaging information from a medical scanner such as a CT based system. The input data file is processed by the VSP® System and the result is an output data file that may then be provided as digital models or used as input to a rapid prototyping portion of the system that produces physical outputs including anatomical models, templates, and surgical guides for use in maxillofacial surgery. The VSP® System is also intended as a pre-operative software tool for simulating / evaluating surgical treatment options.
The Medical Modeling VSP® System is a collection of software and associated additive manufacturing (rapid prototyping) equipment intended to provide a variety of outputs to support reconstructive surgery. The system uses electronic medical images of the patients' anatomy with input from the physician, to manipulate original patient images for planning and executing surgery. The system produces a variety of patient specific outputs including; anatomical models (physical and digital), surgical templates / guides, splints, and patient specific case reports.
Following the Medical Modeling Quality System and specific Work Instructions, trained employees utilize a combination of Commercial Off-The-Shelf (COTS) and custom software to manipulate 3-D medical scan images which can include Computed Tomography (CT), Cone Beam CT (CBCT), and/or 3-D scan images from patient physical models (stone models of the patient's teeth) to create patient-specific physical and digital outputs. The process requires clinical input and review from the physician during planning and prior to delivery of the final outputs. While the process and dataflow vary somewhat based on the particular requirements of a given patient and physician, the following description outlines the functions of key sub-components of the system, and how they interact to produce the defined system outputs. It should be noted that the system is operated only by trained Medical Modeling employees, and the physician does not directly input information. The physician provides input for model manipulation and interactive feedback through viewing of digital models of system outputs that are modified by the engineer during the planning session.
The VSP® System is made up of 13 individual pieces of software and 9 pieces of manufacturing equipment integrated to provide a range of anatomical models (physical and digital), surgical guides, and patient specific planning reports for reconstructive surgery in the maxillofacial region. Surgical guides for graft bone harvesting which can include the fibula and hip are also produced by the system.
The VSP® System requires an input 3-D image file from medical imaging systems (i.e. CT). This input is then used, with support from the prescribing physician to provide the following potential outputs to support reconstructive surgery. Each system output is designed with physician input, and reviewed by the physician prior to finalization. All outputs are used only with direct physician involvement reducing the criticality of the outputs.
System Outputs:
Anatomical Models
Surgical Positioning Templates / Guides
Osteotomy Templates / Guides
Plate Bending Templates/Guides
Patient Specific Case Reports
The VSP® System also contains a set of Stainless Steel Cutting and Drill Inserts (VSP® System Accessories) which are intended to be used by the physician to guide cutting and drilling activities during the surgical procedure. The inserts fit into a standard slot/hole in the cutting/drill guides, and can be used across all the VSP® System guides and templates.
The Medical Modeling VSP® System is a collection of software and associated additive manufacturing (rapid prototyping) equipment intended to provide a variety of outputs to support reconstructive surgery. The system uses electronic medical images of the patients' anatomy with input from the physician, to manipulate original patient images for planning and executing surgery.
Here's an analysis of the acceptance criteria and the study that proves the device meets them, based on the provided text:
Acceptance Criteria and Reported Device Performance
The provided document describes a 510(k) submission for a medical device and outlines various verification and validation testing performed. However, it does not explicitly state a table of specific acceptance criteria with quantifiable metrics (e.g., "accuracy must be > 95%") alongside reported device performance for the VSP® System.
Instead, the document details a comprehensive set of engineering and process validation activities. The general acceptance criterion is implicit: "All, Design, Process, and Other Verification and Validation Testing, which was required as a result of risk analysis and design impact assessments, showed conformity with pre-established specifications and acceptance criteria." This implies that individual test plans for each category would have contained the specific acceptance criteria.
The study proves the device meets the acceptance criteria through a series of "Verification and Validation testing" categories. The overarching goal was to demonstrate that "the manufacturing process could correctly translate 3 dimensional electronic models of patient anatomies, templates and guides into matching physical models within the allowed tolerance ranges from the process specifications."
Given the lack of a direct table in the provided text, I will infer the categories of performance testing as "acceptance criteria" through successful completion, and "reported device performance" as the successful demonstration against these criteria.
| Acceptance Criteria Category (from 9.3) | Reported Device Performance (Implied from 9.4) |
|---|---|
| Installation Qualification (IQ) | Showed conformity with pre-established specifications and acceptance criteria. |
| Operational Qualification (OQ) | Showed conformity with pre-established specifications and acceptance criteria. |
| Performance Qualification (PQ) | Showed conformity with pre-established specifications and acceptance criteria. |
| Software Validation | Showed conformity with pre-established specifications and acceptance criteria; Verified independent subsystems, interfaces, and validated the fully integrated system. |
| Simulated Use (Planning Session) | Showed conformity with pre-established specifications and acceptance criteria. |
| Labeling Validation | Showed conformity with pre-established specifications and acceptance criteria. |
| Cleaning Validation | Showed conformity with pre-established specifications and acceptance criteria. |
| Sterilization Validation | Showed conformity with pre-established specifications and acceptance criteria. |
| Biocompatibility | Showed conformity with pre-established specifications and acceptance criteria. |
| Packaging Validation | Showed conformity with pre-established specifications and acceptance criteria. |
Additional Information on the Study:
-
Sample size used for the test set and the data provenance:
The document does not specify a distinct "test set" sample size in terms of patient cases or imaging data. The testing described is focused on the manufacturing process and the system's ability to translate 3D electronic models into physical models within tolerance. Therefore, the "sample" for testing would likely refer to the number of physical models, guides, or templates produced and evaluated during OQ, PQ, and Simulated Use. These details are not provided in the summary.
Data provenance (e.g., country of origin, retrospective/prospective) is not mentioned, as the focus is on system functionality rather than clinical performance on a patient cohort for this type of device (a system for creating models and guides). -
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
The document states that "The process requires clinical input and review from the physician during planning and prior to delivery of the final outputs." and "All outputs are used only with direct physician involvement reducing the criticality of the outputs." This indicates that physicians are key domain experts providing input and review. However, the specific number of experts used for establishing "ground truth" during validation testing (e.g., for evaluating the accuracy of physical models against digital plans) and their qualifications are not detailed. It can be inferred that "trained Medical Modeling employees" (engineers/CAD specialists) operate the system, and physicians provide clinical feedback. -
Adjudication method (e.g., 2+1, 3+1, none) for the test set:
The document does not specify any formal adjudication method like "2+1" or "3+1" for establishing ground truth or evaluating the test set results. The process involves physician input and review, but a formal adjudication process for test results is not detailed in this summary. -
If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
No MRMC study was conducted or reported. The VSP® System is a tool for planning and manufacturing patient-specific devices, not an AI-assisted diagnostic or interpretation system that would involve "human readers" in the typical sense of an MRMC study. There is no mention of "AI assistance" in this context. -
If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
The system is explicitly described as having "human-in-the-loop" performance. "The system is operated only by trained Medical Modeling employees, and the physician does not directly input information. The physician provides input for model manipulation and interactive feedback through viewing of digital models of system outputs that are modified by the engineer during the planning session." Therefore, no standalone (algorithm only) performance was presented or is applicable to the intended use. -
The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
For mechanical and dimensional testing (OQ, PQ), the ground truth for physical models would be established by comparing them to the digital models/plans generated by the system, referencing "allowed tolerance ranges from the process specifications." This essentially uses the digitally designed model as the 'truth' against which the manufactured physical output is measured. For "Simulated Use (Planning Session)," the ground truth would likely be based on expert (physician) review and acceptance of the surgical plan and resulting outputs as clinically appropriate and accurate based on the patient's anatomy. -
The sample size for the training set:
The document does not describe a "training set" in the context of machine learning or AI, as this system is not presented as an AI-driven device. It is a software system for image transfer, segmentation, and pre-operative planning, utilizing COTS and custom software operated by trained personnel. Therefore, the concept of a "training set" for an algorithm is not applicable here. -
How the ground truth for the training set was established:
As there is no mention of an algorithm training set, this question is not applicable. The software within the VSP® system undergoes standard software validation (9.3.4), where its performance against defined requirements and interfaces (likely functional specifications) would be the "ground truth."
§ 872.4120 Bone cutting instrument and accessories.
(a)
Identification. A bone cutting instrument and accessories is a metal device intended for use in reconstructive oral surgery to drill or cut into the upper or lower jaw and may be used to prepare bone to insert a wire, pin, or screw. The device includes the manual bone drill and wire driver, powered bone drill, rotary bone cutting handpiece, and AC-powered bone saw.(b)
Classification. Class II.