(197 days)
The VSP® System is intended for use as a software system and image segmentation system for the transfer of imaging information from a medical scanner such as a CT based system. The is processed by the VSP® System and the result is an output data file that may then be provided as digital models or used as input to a rapid prototyping portion of the system that produces physical outputs including anatomical models, templates, and surgical guides for use in maxillofacial surgery. The VSP® System is also intended as a pre-operative software tool for simulating / evaluating surgical treatment options.
The VSP System utilizes a combination of Commercial Off-The-Shelf (COTS) and custom software to manipulate 3D medical images (CT based systems) to create virtual and physical anatomical models, templates, surgical guides, and surgical plans for reconstructive surgical procedures.
The provided text describes the VSP® System, a software and image segmentation system for maxillofacial surgery, and its 510(k) submission (K192192) to the FDA. The submission focuses on demonstrating substantial equivalence to a predicate device (VSP® System, K133907), with modifications primarily related to the expansion of materials used for guides (Polyamide and Titanium Alloy).
However, the provided document does not contain the specific details required to fully address all parts of your request regarding acceptance criteria and the study that proves the device meets them, especially in the context of AI/algorithm performance.
The document primarily discusses design validation and process performance qualification, which are standard engineering and manufacturing tests, rather than clinical performance studies of an AI algorithm's diagnostic or predictive capabilities. It explicitly states: "Clinical testing was not necessary for the determination of substantial equivalence." This indicates that the regulatory pathway for this device did not require a human-in-the-loop or standalone AI performance study against a clinical ground truth as might be expected for an AI-driven diagnostic device.
Here's an attempt to answer your questions based on the available information, with clear indications where the information is not provided in the text:
1. A table of acceptance criteria and the reported device performance
The document mentions that acceptance criteria were established and met for various tests. However, it does not provide a specific table or quantified acceptance criteria and reported performance values for the device's accuracy in image segmentation or surgical planning. Instead, it describes general conclusions for each test:
| Acceptance Criteria Category | Reported Device Performance |
|---|---|
| Design Validation | "All acceptance criteria were met." specifically for "accuracy of cutting location/trajectory, drilling location/trajectory, and bony segment length utilizing subject device guides on bone models." |
| Process Performance Qualification | "All test method acceptance criteria were met," verifying "Both digital and physical outputs from all manufacturing processes... against design specifications." |
| Cleaning Validation | "All test method acceptance criteria were met." |
| Sterilization Validation | "All test method acceptance criteria were met." |
| Biocompatibility Validation | "Within the pre-defined acceptance criteria," adequately addressing biocompatibility. |
| Mechanical Performance Validation | "Subject devices perform equivalent or better than the predicate device" in flexural deformation testing. |
| Packaging Validation | "All test method acceptance criteria were met." |
2. Sample size used for the test set and the data provenance
- Test Set Sample Size: The document does not specify the sample size for any of the validation tests.
- Data Provenance: The document generally refers to "bone models" for design validation cases and "representative of orthognathic and reconstruction procedures" for process performance qualification. It does not specify the country of origin of the data or whether it was retrospective or prospective. Given the nature of the device (surgical planning and guide creation from medical scans), the "data" would likely be patient CT scans, but the details of their origin are not provided.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
The document does not specify the number or qualifications of experts used to establish ground truth for the test set. It mentions "user needs and intended use of supporting maxillofacial surgeries" for design validation, implying involvement of surgeons, but no specific details on their role in ground truth establishment.
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set
The document does not mention any adjudication method used for establishing ground truth or evaluating the test set results.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
The document explicitly states: "Clinical testing was not necessary for the determination of substantial equivalence." Therefore, no MRMC comparative effectiveness study was conducted to evaluate human reader improvement with or without AI assistance. The regulatory pathway for this device, a software system for surgical planning and guide production, did not require such a clinical study to demonstrate safety and effectiveness for substantial equivalence.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
A formal standalone performance study of the algorithm's diagnostic or predictive capabilities (e.g., measuring segmentation accuracy against ground truth in a clinical context) is not specifically described in the document as a separate study. The "Design Validation" and "Process Performance Qualification" likely involved evaluating the algorithm's output (digital models, physical guides) against design specifications, which could be considered a form of standalone testing of the system's output creation accuracy, but not a clinical diagnostic performance study.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)
The document primarily refers to "design specifications" as the basis for validation. For "Design Validation," it assessed "accuracy of cutting location/trajectory, drilling location/trajectory, and bony segment length utilizing subject device guides on bone models." This implies the ground truth for these tests was pre-defined ideal anatomical measurements or surgical plans on synthetic or cadaveric models, rather than clinical pathology or outcomes data from human patients.
8. The sample size for the training set
The document does not mention a training set or any details about machine learning model development. The description of the VSP System implies it uses "custom software" and "image transfer and manipulation," but does not explicitly state it employs machine learning or AI that requires a distinct "training set." It's possible the system uses rule-based algorithms or traditional image processing techniques rather than data-driven machine learning models.
9. How the ground truth for the training set was established
Since a training set is not mentioned, the method for establishing its ground truth is not provided.
§ 872.4120 Bone cutting instrument and accessories.
(a)
Identification. A bone cutting instrument and accessories is a metal device intended for use in reconstructive oral surgery to drill or cut into the upper or lower jaw and may be used to prepare bone to insert a wire, pin, or screw. The device includes the manual bone drill and wire driver, powered bone drill, rotary bone cutting handpiece, and AC-powered bone saw.(b)
Classification. Class II.