(213 days)
The GI Genius System is a computer-assisted reading tool designed to aid endoscopists in detecting colonic mucosal lesions (such as polyps and adenomas) in real time during standard white-light endoscopy examinations of patients undergoing screening and surveillance endoscopic mucosal evaluations. The GI Genius computer-assisted detection device is limited for use with standard white-light endoscopy imaging only. This device is not intended to replace clinical decision making.
The GI GENIUS™ is an artificial intelligence/machine learning (AI/ML) device system comprised of software, hardware, and accessories that is intended for polyp detection during standard white-light colonoscopy. The device system generates a video on the main endoscopy display that contains the original live video together with superimposed markers (in the form of green boxes) that appear when a lesion is detected.
The GI Genius takes the Serial Digital Interface (SDI) output stream from the video endoscope processor as an input and then generates an SDI output stream to the existing monitor/display system containing the original video stream with additional markers superimposed on it. In essence, the system is inserted into the video stream just prior to it being displayed to the user/operator.
Here's a breakdown of the acceptance criteria and the study that proves the GI Genius device meets them, based on the provided text:
Acceptance Criteria and Device Performance
| Acceptance Criteria Category | Specific Acceptance Criteria | Reported Device Performance |
|---|---|---|
| I. Nonclinical/Bench Studies | ||
| Video Delay | Time $\leq$ 5.75 milliseconds | (b)(4) microseconds ((b)(4)%) |
| Annotation Delay | Time $\leq$ 120 milliseconds | (b)(4) or (b)(4) milliseconds ((b)(4) frames) |
| Video Quality Integrity | No degradation in image quality (identical pixels in 3 color channels, excluding marker overlays) | Test data show no pixel-level discrepancies except for pixels overlaid with markers |
| II. Standalone Performance | ||
| Activation Time | Device detects lesion faster than endoscopist reaction time | GI Genius detected polyp 1270 ms (95% CI: 857 ms, 1684 ms) before average endoscopist |
| Object-Level Sensitivity (persistence > 0ms) | No specific threshold given; presented as trade-off with FP rate. The device detects polyps before endoscopist detection time. | 81.96% (95% CI: 77.35%; 85.97%) |
| Object-Level False Positives (persistence > 0ms) | No specific threshold given; presented as trade-off with sensitivity. | 156.31 (95% CI: 135.61; 177.00) FP Objects/Patient |
| Frame-Level False Positive Rate (FPR / FRAME) | Expected: 4.85% or lower (based on estimation from video database) | Logistic Regression Mixed Model: 1.44% (95% CI: 1.27%; 1.63%) |
| Non-parametric Cluster Bootstrap: 2.02% (95% CI: 1.72%; 2.35%) | ||
| Frame-Level True Positive Rate (TPR / FRAME) | No specific threshold given; presented. | Logistic Regression Mixed Model: 47.46% (95% CI: 42.51%; 52.45%) |
| Non-parametric Cluster Bootstrap: 49.57% (95% CI: 45.24%; 54.06%) | ||
| Standalone Performance (Overall) | Sufficient to fulfill indications for use; adequate benchmark for improved lesion detection. | Met pre-defined performance criteria and found adequate for benchmarking. |
| III. Clinical Performance (AID Study) | ||
| Primary Endpoint: Adenoma Detection Rate* (ADR*) | Non-inferiority (10% margin) to standard colonoscopy; then superiority if non-inferiority met. | Superiority Demonstrated: GI Genius ADR* 55.1% (95% CI: 44.0% to 65.8%) vs. Standard 42.0% (95% CI: 31.3% to 53.4%). Difference 13.1% (95% CI: 0.09; 23.3), p \ 3. |
| Qualitative Questions | No usability errors; no changes to colonoscopy workflow. | No usability errors reported; no changes to workflow. |
| V. EMC & Electrical Safety | Passes acceptance criteria of ANSI/AAMI/IEC 60601-1-2:2014 and IEC 60601-1:2005 + A1:2012 (Ed. 3.1). | Test results pass acceptance criteria. |
| VI. Software/Cybersecurity | Identified as moderate level of concern; submitted all required documentation (hazard analysis, V&V, threat model, etc.). | Documentation included. Acceptable verification and validation activities at unit, integration, and system level. Cybersecurity documentation complete. |
Study Details
2. Sample Sizes and Data Provenance
- Test Set (Standalone Performance):
- Sample Size: 150 colonoscopy videos.
- 105 videos included 338 excised polyps with histology confirmation.
- 45 videos did not include polyps or lesions.
- Data Provenance: Originally from a study titled, "The Safety and Efficacy of Methylene Blue MMX® Modified Release Tablets Administered to Subjects Undergoing Screening or Surveillance Colonoscopy" [NCT01694966]. This study was a multi-arm study conducted without specific country information, but the clinical study (AID study) was performed in Italy. Given the standalone performance data was derived from the MMX trial, it is retrospective. The 150 videos for the Holdout Test Set were specifically without methylene blue.
- Sample Size: 150 colonoscopy videos.
- Clinical Study (AID Study):
- Sample Size (mITT Population): 263 patients.
- 136 patients randomized to GI Genius+colonoscopy.
- 127 patients randomized to standard colonoscopy.
- Data Provenance: Randomized, prospective, multicenter, controlled clinical investigation performed in Italy (Humanitas Research Hospital [Milan], Nuovo Regina Margherita Hospital [Rome], and Valduce Hospital [Como]).
- Sample Size (mITT Population): 263 patients.
3. Number of Experts and Qualifications for Test Set Ground Truth
- Standalone Performance Test Set:
- Experts: A panel of five expert endoscopists reviewed video clips to establish the critical time frame for lesion detection and to define the endoscopists' first detection time for object-level performance.
- Qualifications: Not explicitly stated beyond "expert endoscopists."
- Clinical Study Test Set (AID Study):
- Experts: Six endoscopists were involved in conducting the colonoscopies and establishing ground truth through biopsy and histological confirmation.
- Qualifications: Endoscopists with moderate endoscopy expertise, defined as an Adenoma Detection Rate (ADR) between 25% to 40%.
4. Adjudication Method for Test Set
- Standalone Performance Test Set:
- The reference standard for true positives, true negatives, false positives, and false negatives was established by having endoscopists review video clips around histologically confirmed polyps and placing an annotation box around the polyps visible in each frame. The device's markers were then assessed for overlap with these annotations using an Intersection over Union (IoU) criterion. This implies expert consensus/annotation was the primary method, followed by a quantitative comparison. No explicit adjudication for disagreements among experts is described, rather, the experts created the ground truth.
- Clinical Study Test Set:
- Ground truth for polyps in the clinical study was established by histological confirmation of excised lesions. The study design does not explicitly describe a multi-reader, multi-case (MRMC) adjudication process for detection, but rather the gold standard for lesion presence/type relied on pathology.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- Yes, a form of MRMC comparative effectiveness study was done, in the clinical study (AID Study).
- The study design involved six endoscopists, performing colonoscopies both with AI assistance (GI Genius) and without (standard colonoscopy). This is a direct comparison of human performance with and without AI assistance.
- It was a randomized, prospective, multicenter, controlled clinical investigation.
- Effect Size of Human Reader Improvement:
- For the primary endpoint, Adenoma Detection Rate (ADR*): The GI Genius + colonoscopy group had an adjusted ADR* of 55.1%, while the standard colonoscopy group had an ADR* of 42.0%. This represents an absolute improvement of 13.1 percentage points with AI assistance (p
§ 876.1520 Gastrointestinal lesion software detection system.
(a)
Identification. A gastrointestinal lesion software detection system is a computer-assisted detection device used in conjunction with endoscopy for the detection of abnormal lesions in the gastrointestinal tract. This device with advanced software algorithms brings attention to images to aid in the detection of lesions. The device may contain hardware to support interfacing with an endoscope.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Clinical performance testing must demonstrate that the device performs as intended under anticipated conditions of use, including detection of gastrointestinal lesions and evaluation of all adverse events.
(2) Non-clinical performance testing must demonstrate that the device performs as intended under anticipated conditions of use. Testing must include:
(i) Standalone algorithm performance testing;
(ii) Pixel-level comparison of degradation of image quality due to the device;
(iii) Assessment of video delay due to marker annotation; and
(iv) Assessment of real-time endoscopic video delay due to the device.
(3) Usability assessment must demonstrate that the intended user(s) can safely and correctly use the device.
(4) Performance data must demonstrate electromagnetic compatibility and electrical safety, mechanical safety, and thermal safety testing for any hardware components of the device.
(5) Software verification, validation, and hazard analysis must be provided. Software description must include a detailed, technical description including the impact of any software and hardware on the device's functions, the associated capabilities and limitations of each part, the associated inputs and outputs, mapping of the software architecture, and a description of the video signal pipeline.
(6) Labeling must include:
(i) Instructions for use, including a detailed description of the device and compatibility information;
(ii) Warnings to avoid overreliance on the device, that the device is not intended to be used for diagnosis or characterization of lesions, and that the device does not replace clinical decision making;
(iii) A summary of the clinical performance testing conducted with the device, including detailed definitions of the study endpoints and statistical confidence intervals; and
(iv) A summary of the standalone performance testing and associated statistical analysis.