(135 days)
ENT EM is intended as an image-guided planning and navigation system to enable ENT procedures. The device is indicated for any medical condition in which a reference to a rigid anatomical structure can be identified relative to images (CT, CTA, X-Ray, MR, MRA and ultrasound) of the anatomy, such as:
- Intranasal structures and Paranasal Sinus Surgery
- Functional endoscopic sinus surgery (FESS)
- Intranasal structures and paranasal sinus surgery, including revision and distorted anatomy
- Anterior skull base procedures
The Subject Device ENT EM is an image guided planning and navigation system to enable navigated surgery during ENT procedures. It offers guidance for setting up the EM equipment, different patient image registration methods and instrument selection and calibration to allow surgical navigation by using electromagnetic tracking (EM) technology. The device provides different workflows guiding the user through preoperative and intraoperative steps. To fulfill this purpose, it links patient anatomy (using a patient reference) and instruments in the real world or "patient space" to patient scan data or "image space". This allows for the continuous localization of medical instruments and patient anatomy for medical interventions in ENT procedures. The software is installed on a mobile Image Guided Surgery (IGS) platform (Kick 2 Navigation Station or Curve Navigation 17700) to support the surgeon in clinical procedures by displaying tracked instruments in patient's image data. The IGS platforms consist of a mobile Monitor Cart and an EM tracking unit for image guided surgery purposes. ENT EM consists of: Several software modules for registration, instrument handling, navigation and infrastructure tasks, IGS platforms and surgical instruments for navigation, patient referencing and registration.
The provided text describes the acceptance criteria and the study that proves the device meets those criteria for the Brainlab ENT EM system, which incorporates an AI/ML-based function for pre-registration in surface matching.
Here's the breakdown of the information requested:
1. Table of Acceptance Criteria and Reported Device Performance
| Parameter/Characteristic | Acceptance Criteria | Reported Device Performance |
|---|---|---|
| System Accuracy | ||
| Mean Positional Error | ≤ 2 mm | Achieves the same accuracy performance (mean location error ≤ 2 mm) as both predicate and reference device. |
| Mean Angular Error | ≤ 2º | Achieves the same accuracy performance (mean trajectory angle error ≤ 2 degrees) as both predicate and reference device. |
| AI/ML Landmark Detection | Equivalent performance to conventional method | Performance testing comparing conventional to machine learning based landmark detection were performed showing equivalent performance as in the reference device. |
| Usability | Safe and effective for intended user group | Summative usability evaluation in a simulated clinical environment showed ENT EM is safe and effective for use by the intended user group. |
| Electrical Safety & EMC | Compliance with standards | Compliance to IEC 60601-1, AIM 7351731, and IEC 60601-1-2. Tests showed the subject device performs as intended. |
| Instrument Biocompatibility | Biologically safe | Biocompatibility assessment considering different endpoints provided. |
| Instrument Reprocessing | Appropriateness of cleaning/disinfection/sterilization | Cleaning and disinfection evaluation/reprocessing validation provided. |
| Instrument Mechanical Properties | Withstand typical torsional strengths/torques | Evaluated considering typical torsional strengths, torques, and conditions instruments can be subject to during use. |
2. Sample Size Used for the Test Set and Data Provenance
The document does not explicitly state the numerical sample size for the test set used for the AI/ML algorithm's performance evaluation. It mentions that "The model's prediction and performance are then evaluated against the test pool. The test pool data is set aside at the beginning of the project."
The data provenance is not explicitly stated regarding country of origin or specific patient demographics. However, it indicates a "controlled internal process" for development and evaluation. It's a static algorithm (locked), suggesting it's developed and tested once rather than continuously learning. The context implies it's retrospective as data was "set aside."
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
This information is not provided in the text. The document refers to "landmarks delivered by a ML based calculation" and compares its performance to a "conventional" method in the reference device, but it doesn't detail how the ground truth for these landmarks was established for testing.
4. Adjudication Method for the Test Set
The adjudication method for establishing ground truth for the test set is not explicitly described.
5. Multi Reader Multi Case (MRMC) Comparative Effectiveness Study
There is no mention of a Multi Reader Multi Case (MRMC) comparative effectiveness study being performed with human readers to assess improvement with AI vs. without AI assistance. The testing primarily focuses on the AI/ML algorithm's performance equivalence to the predicate/reference device's conventional method, and overall system accuracy.
6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study
Yes, a form of standalone performance evaluation was done for the AI/ML algorithm. The text states: "Performance testing comparing conventional to machine learning based landmark detection were performed showing equivalent performance as in the reference device." This implies an evaluation of the algorithm's output (landmark detection) without a human reader in the interpretation loop, by comparing its results directly to a "conventional" method.
7. The Type of Ground Truth Used for Performance Testing
The type of ground truth for the AI/ML landmark detection is implicitly based on the "conventional" landmark detection method used in the reference device. The document states "Performance testing comparing conventional to machine learning based landmark detection were performed showing equivalent performance as in the reference device." This suggests the conventional method's output serves as the reference ground truth, or there's an established "true" landmark position that both are compared against. For system accuracy, the ground truth is established through physical measurements of "Mean Positional Error" and "Mean Angular Error" against a known configuration.
8. The Sample Size for the Training Set
The document does not provide the numerical sample size for the training set. It mentions the algorithm was developed using a "Supervised Learning approach" and that "the training process begins with the model observing, learning, and optimizing its parameters based on the training pool data."
9. How the Ground Truth for the Training Set Was Established
The method for establishing ground truth for the training set is not explicitly detailed. It only states that the algorithm was developed using a "Supervised Learning approach" and a "controlled internal process" that defines activities from "inspection of input data to the training and verification." This implies that the training data included true labels or targets for the landmarks that the AI/ML algorithm was trained to detect, but the source or method of obtaining these true labels (e.g., expert annotation, manual registration results) is not specified.
§ 882.4560 Stereotaxic instrument.
(a)
Identification. A stereotaxic instrument is a device consisting of a rigid frame with a calibrated guide mechanism for precisely positioning probes or other devices within a patient's brain, spinal cord, or other part of the nervous system.(b)
Classification. Class II (performance standards).