(263 days)
The SKOUT system is a software device designed to detect potential colorectal polyps in real time during colonoscopy examinations. It is indicated as a computer-aided detection tool providing colorectal polyps location information to assist qualified and trained gastroenterologists in identifying potential colorectal polyps during colonoscopy examinations in adult patients undergoing colorectal cancer screening or surveillance.
The SKOUT system is only intended to assist the gastroenterologist in identifying suspected colorectal polyps and the gastroenterologist is responsible for reviewing SKOUT suspected polyp areas and confirming the presence of a polyp based on their own medical judgment. SKOUT is not intended to replace a full patient evaluation, nor is it intended to be relied upon to make a primary interpretation of endoscopic procedures, medical diagnosis, or recommendations of treatment/course of action for patients. SKOUT is indicated for white light colonoscopy only.
The SKOUT™ system is a software-based computer aided detection (CADe) system for the analysis of high-definition endoscopic video during colonoscopy procedures. The SKOUT™ system is intended to aid gastroenterologists with the detection of potential colorectal polyps during colonoscopy by providing an informational visual aid on the endoscopic monitor using trained software that processes the endoscopic video in real time.
Users will primarily interact with the SKOUT™ system by observing the software display, including the polyp detection box and device status indicator signal.
Acceptance Criteria and Device Performance for SKOUT™ System
1. Table of Acceptance Criteria and Reported Device Performance
The provided document doesn't explicitly state "acceptance criteria" in a separate section. However, it presents the results of "Non-clinical performance testing" and "Clinical Testing" with specific metrics and confidence intervals, which serve as the performance benchmarks the device has met.
Based on the information provided, here's a table summarizing the implicit acceptance criteria (as demonstrated by the positive study outcomes) and the reported device performance:
| Acceptance Criterion (Implicit) | Reported Device Performance | Study Type |
|---|---|---|
| Non-Clinical Performance: | ||
| Object Level True Positive Rate (TPR): High proportion of actual polyps detected by the device. | 97.87% (95% CI: 94.96%, 100.0%) | Standalone Algorithm Performance Testing |
| Object Level False Positives (FP): Low number of non-polyps flagged as polyps per procedure. | 22.55 objects per 15-minute interval (95% CI: 18.954, 26.148) | Standalone Algorithm Performance Testing |
| Frame Level True Positive Rate (TPR): High proportion of frames with confirmed polyps where the device bounds the polyp. | Mean: 55.66% (95% CI: 55.50%, 55.83%) | Standalone Algorithm Performance Testing |
| Frame Level False Positive Rate (FPR): Low proportion of frames where the device bounds an object not detected by the gastroenterologist. | Mean: 2.31% (95% CI: 2.16%–2.44%) | Standalone Algorithm Performance Testing |
| Marker Overlap (IOGT): High proportion of ground truth polyp area engulfed by the SKOUT bounding box to ensure appropriate visibility for the physician. (Implied acceptance of IOGT = 1.0 as a positive indicator given the purpose of bounding box enlargement). | Median IOGT of signal: 1.0 (meaning on a median basis, all polyps were engulfed by a SKOUT bounding box). Mean IOU: 0.299 (95% CI: 0.289, 0.309) which is expected to be low due to artificial bounding box enlargement. | Standalone Algorithm Performance Testing |
| Image Quality Degradation: No visually detectable degradation due to the device. | No visually detectable differences between images found with the introduction of the SKOUT™ system. | Special Control Testing |
| Video Delay due to Marker Annotation: Minimal delay introduced by the device. | SDI: 56.00ms (95% CI: 50.54, 61.46) and 3.25 frame delay (95% CI: 2.93, 3.56). DVI: 62.33ms (95% CI: 60.76, 63.90) and 3.74 frame delay (95% CI: 3.65, 3.83). | Special Control Testing |
| Real-time Endoscopic Video Delay: Minimal delay in real-time video feed. | SDI: 56.67ms (95% CI: 51.01, 62.33) and 3.28 frame delay (95% CI: 2.96, 3.62). DVI: 60.67ms (95% CI: 57.72, 63.61) and 3.64 frame delay (95% CI: 3.46, 3.81). | Special Control Testing |
| Clinical Performance: | ||
| Adenomas Per Colonoscopy (APC): Significant improvement in the number of adenomas detected per colonoscopy with AI assistance. | Treatment (AI-aided arm): 1.054 Control (Standard colonoscopy): 0.830 Difference (Treatment - Control): 0.224 (95% CI: 0.060, 0.382), p-value: 0.002 (Statistically significant increase) | Multi-center, prospective, randomized controlled trial (Clinical Study) |
| Positive Percent Agreement (PPA) / Positive Predictive Value (PPV): Non-inferiority or comparable safety profile to standard colonoscopy. (PPA/PPV used as a safety endpoint representing the fraction of resected lesions that were indeed polyps). | PPA (mITT cohort): Control: 75.7% Treatment: 70.9% Difference: -4.8% (95% CI: -9.5% to 0.3%) and p-value: <0.001. The results "fell statistically within the prespecified noninferiority margin". | Multi-center, prospective, randomized controlled trial (Clinical Study) |
| Safety: No increase in adverse events or complications. | No adverse events or complications were reported during the study. | Multi-center, prospective, randomized controlled trial (Clinical Study) |
2. Sample Sizes and Data Provenance
-
Standalone Algorithm Performance Testing:
- Test Set Sample Size: 79 HD videos containing 94 polyps.
- Data Provenance: Not specified regarding country of origin. The document states "a predefined set of 79 HD videos". It's implied to be retrospective as it's a "set of videos" used for evaluation.
-
Clinical Testing (MCRCT):
- Test Set Sample Size: A total of 1,359 patients were included in the modified Intention To Treat (mITT) population.
- AI-aided arm (SKOUT™ system): 682 patients
- Control arm (Standard colonoscopy): 677 patients
- Data Provenance: Multi-center, prospective, randomized controlled trial conducted in the United States.
- Test Set Sample Size: A total of 1,359 patients were included in the modified Intention To Treat (mITT) population.
3. Number of Experts and Qualifications for Ground Truth Establishment
-
Standalone Algorithm Performance Testing:
- Number of Experts: "Expert gastroenterologists" (plural, but specific number not provided).
- Qualifications: "Expert gastroenterologists" who reviewed and either validated, rejected, or created new labels. No specific details like years of experience are given, but "expert" implies significant experience.
-
Clinical Testing:
- Number of Experts: The endoscopists participating in the trial served as the "human experts" whose findings formed the basis of the clinical ground truth. The eligibility requirements for these providers were: a) United States board-certified gastroenterologist, b) Has performed at least 1,000 colonoscopies, and c) Has an ADR (Adenoma Detection Rate) greater than or equal to 25%.
- Qualifications: Board-certified gastroenterologists with extensive experience (minimum 1,000 colonoscopies and a good ADR).
4. Adjudication Method for the Test Set
-
Standalone Algorithm Performance Testing: "Ground truth was defined as data reviewed and either validated or created by expert gastroenterologists through a process referred to as gastroenterologist review. During gastroenterologist review, experts reviewed and either validated, rejected new labels post primary annotation." This implies a consensus process, but the exact number of experts involved in each review/adjudication step (e.g., 2+1, 3+1) is not explicitly stated. It mentions a "team of trained annotators" initially, then "expert gastroenterologists" for validation/adjudication.
-
Clinical Testing: For the clinical trial, the ground truth was based on the findings confirmed by the gastroenterologist during the colonoscopy and subsequent pathology findings. As this was a clinical trial involving real-time diagnosis and procedure (polyp removal), standard clinical practice implicitly serves as the "adjudication" for polyp presence, with pathology confirming the nature of the resected tissue. There wasn't a separate, explicit "adjudication method" in the sense of multiple independent readers reviewing the same data retrospectively; rather, it was a prospective clinical study using the treating physician's findings as the clinical gold standard, confirmed by pathology when resected.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- Yes, a clinical study was performed that can be considered a form of comparative effectiveness study, although not explicitly labeled as an MRMC study in the traditional sense of multiple readers evaluating the same cases. It was a randomized controlled trial comparing two groups of patients (and thus, two groups of cases and different physicians): one with AI assistance and one without.
- Effect Size of Human Readers Improvement with AI vs. without AI assistance:
- The primary performance endpoint was Adenomas Per Colonoscopy (APC).
- Without AI (Control Arm): Mean APC = 0.830
- With AI Assistance (Treatment Arm): Mean APC = 1.054
- Effect Size / Improvement: The difference in APC was +0.224 (Treatment - Control), with a p-value of 0.002. This indicates a statistically significant increase in the number of adenomas detected per colonoscopy when the SKOUT™ system was used. This is a direct measure of how much human readers (gastroenterologists) improved their adenoma detection rate when assisted by the AI.
6. Standalone (Algorithm Only) Performance Study
- Yes, a standalone algorithm performance testing was done.
- Details: "The Standalone Performance Assessment was performed to assess trends in the detection and classification of the SKOUT™ system and determine its ability to discriminate between normal mucosa and polyp tissue." This included analysis of 79 HD videos. Performance metrics such as Object Level TPR, Object Level FP, Frame Level TPR, and Frame Level FPR were evaluated.
7. Type of Ground Truth Used
-
Standalone Algorithm Performance Testing:
- "Gastroenterologist review" followed by pathology findings for Object Level TPR (polyps confirmed by pathology findings).
- For Frame Level metrics, the ground truth for polyp detection frames was established by expert gastroenterologists, with "normal mucosa" as the negative ground truth.
-
Clinical Testing:
- Clinical Diagnosis from Trained Gastroenterologist + Pathology: For the APC endpoint, adenomas were confirmed through histopathology after resection. For PPA/PPV, the ground truth was based on resected polyps confirmed by pathology.
8. Sample Size for the Training Set
- The document does not explicitly state the sample size used for the training set of the SKOUT™ system's AI algorithm. It only details the test set for standalone performance and the patient cohort for the clinical trial.
9. How the Ground Truth for the Training Set Was Established
- The document does not explicitly detail how the ground truth for the training set was established. It mentions that the SKOUT™ system "utilizes an artificial intelligence-based algorithm to perform the polyp detection function" and refers to "trained software," implying a training phase. However, the specific process for ground truth establishment for the training data is not provided in this document, only for the evaluation (test) datasets.
{0}------------------------------------------------
Image /page/0/Picture/0 description: The image contains the logo of the U.S. Food and Drug Administration (FDA). On the left is the Department of Health & Human Services logo. To the right of that is the FDA logo, which is a blue square with the letters "FDA" in white. To the right of the blue square is the text "U.S. FOOD & DRUG ADMINISTRATION" in blue.
August 12, 2022
Iterative Scopes Inc. Dennis Francoeur Director of Regulatory Affairs 14 Arrow St. 3rd Floor Cambridge, MA 02138
Re: K213686 Trade/Device Name: SKOUT Software Regulation Number: 21 CFR 876.1520 Regulation Name: Gastrointestinal Lesion Software Detection System Regulatory Class: Class II Product Code: QNP Dated: November 22, 2021 Received: November 22, 2021
Dear Dennis Francoeur:
We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database located at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the
{1}------------------------------------------------
Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR 803) for devices or postmarketing safety reporting (21 CFR 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safetyreporting-combination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR 4. Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR 1000-1050.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-devices/medical-device-safety/medicaldevice-reporting-mdr-how-report-medical-device-problems.
For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medical-device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-devices/device-advicecomprehensive-regulatory-assistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely,
Shanil P. Haugen, Ph.D. Assistant Director DHT3A: Division of Renal, Gastrointestinal, Obesity and Transplant Devices OHT3: Office of GastroRenal, ObGyn, General Hospital and Urology Devices Office of Product Evaluation and Quality Center for Devices and Radiological Health
Enclosure
{2}------------------------------------------------
Indications for Use
510(k) Number (if known) K213686
Device Name SKOUT System
Indications for Use (Describe)
The SKOUT system is a software device designed to detect potential colorectal polyps in real time during colonoscopy examinations. It is indicated as a computer-aided detection tool providing colorectal polyps location information to assist qualified and trained gastroenterologists in identifying potential colorectal polyps during colonoscopy examinations in adult patients undergoing colorectal cancer screening or surveillance.
The SKOUT system is only intended to assist the gastroenterologist in identifying suspected colorectal polyps and the gastroenterologist is responsible for reviewing SKOUT suspected polyp areas and confirming the presence of a polyp based on their own medical judgment. SKOUT is not intended to replace a full patient evaluation, nor is it intended to be relied upon to make a primary interpretation of endoscopic procedures, medical diagnosis, or recommendations of treatment/course of action for patients. SKOUT is indicated for white light colonoscopy only.
| Type of Use (Select one or both, as applicable) | |
|---|---|
| ------------------------------------------------- | -- |
X Prescription Use (Part 21 CFR 801 Subpart D)
Over-The-Counter Use (21 CFR 801 Subpart C)
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
{3}------------------------------------------------
K213686 Page 1 of 16
510(k) SUMMARY
SKOUT™ system
Submitter:
Iterative Scopes, Inc.
14 Arrow St. 3rd Floor
Cambridge MA 02138
Phone: (603) 819-8387
Contact Person: Dennis Francoeur, Director of Requlatory Affairs
Date Prepared: August 11, 2022
Name of Device: SKOUTTM svstem
Classification Name: Gastrointestinal Lesion Software Detection System
Classification Panel: Gastroenterology and Urology
Regulation Number: 876.1520
Product Code: QNP
Predicate Device: GI Genius, Cosmo Artificial Intelligence - AI, LTD, DEN200055
Device Description
The SKOUT™ system is a software-based computer aided detection (CADe) system for the analysis of high-definition endoscopic video during colonoscopy procedures. The SKOUT™ system is intended to aid gastroenterologists with the detection of potential colorectal polyps during colonoscopy by providing an informational visual aid on the endoscopic monitor using trained software that processes the endoscopic video in real time.
Users will primarily interact with the SKOUT™ system by observing the software display, including the polyp detection box and device status indicator signal.
Polyp Detection Notification
The SKOUT™ system has a main graphical user interface (GU) feature of the polyp detection notification. The polyp detection is a two-dimensional blue rectangular outline generated around any suspected polyps on the endoscopic video feed. Display of this notification is deactivated if / when a surgical tool enters the frame or if the polyp is no longer being detected.
The polyp detection notification enables users to:
- Detect potential colorectal polyps during colonoscopy examinations in adult patients . undergoing a colorectal cancer screening or surveillance procedure.
- . Utilize a tool that provides additional information for endoscopic observation.
{4}------------------------------------------------
Device Status Indicator
The SKOUT™ system has an additional GUI feature that notifies users of the current device status (active or error). The device status indicator signal displays as a two-dimensional solid green box in the left-hand corner of the display if the device is powered on and actively processing the input video and as a red X if there is a video processing error.
Intended Use / Indications for Use
The SKOUT™ system is a software device designed to detect potential colorectal polyps in real time during colonoscopy examinations. It is indicated as a computer-aided detection tool providing colorectal polyps location information to assist qualified and trained gastroenterologists in identifying potential colorectal polyps during colonoscopy examinations in adult patients undergoing colorectal cancer screening or surveillance.
The SKOUT™ system is only intended to assist the gastroenterologist in identifying suspected colorectal polyps and the gastroenterologist is responsible for reviewing SKOUT™ suspected polyp areas and confirming the presence or absence of a polyp based on their own medical judgment. SKOUT™ is not intended to replace a full patient evaluation, nor is it intended to be relied upon to make a primary interpretation of endoscopic procedures, medical diagnosis, or recommendations of treatment/course of action for patients. SKOUT™ is indicated for white light colonoscopy only.
Summary of Technological Characteristics
Both the subject and the predicate devices are Computer-Assisted Detection (CADe) devices used in conjunction with endoscopy for the detection of abnormal lesions in the gastrointestinal tract. At a high level, the subject and predicate devices are based on the following same technological elements. A table comparing the key features of the subject and predicate devices is provided below:
| SKOUT™ System | GI Genius | |
|---|---|---|
| Intended Use | A gastrointestinal lesion softwaredetection system is acomputer-assisted detection deviceused in conjunction with endoscopy forthe detection of abnormal lesions inthe gastrointestinal tract. This devicewith advanced software algorithmsbrings attention to images to aid in thedetection of lesions. The device hashardware components to supportinterfacing with an endoscope. | A gastrointestinal lesion softwaredetection system is acomputer-assisted detection deviceused in conjunction with endoscopyfor the detection of abnormal lesionsin the gastrointestinal tract. Thisdevice with advanced softwarealgorithms brings attention toimages to aid in the detection oflesions. The device may containhardware to support interfacing withan endoscope. |
| Indications forUse | The SKOUT™ system is a softwaredevice designed to detect potentialcolorectal polyps in real time duringcolonoscopy examinations. It isindicated as a computer-aideddetection tool providing colorectal | The GI Genius System is acomputer-assisted reading tooldesigned to aid endoscopists indetecting colonic mucosal lesions(such as polyps and adenomas) inreal time during standard white-light |
| SKOUT™ System | GI Genius | |
| polys location information to assistqualified and trainedgastroenterologists in identifyingpotential colorectal polyps duringcolonoscopy examinations in adultpatients undergoing colorectal cancerscreening or surveillance. | endoscopy examinations of patientsundergoing screening andsurveillance endoscopic mucosalevaluations. The GI Geniuscomputer-assisted detection deviceis limited for use with standard whitelight endoscopy imaging only. This | |
| The SKOUT™ system is only intendedto assist the gastroenterologist inidentifying suspected colorectal polypsand the gastroenterologist isresponsible for reviewing SKOUT™suspected polyp areas and confirmingthe presence or absence of a polypbased on their own medical judgment.SKOUT™ is not intended to replace afull patient evaluation, nor is itintended to be relied upon to make aprimary interpretation of endoscopicprocedures, medical diagnosis, orrecommendations of treatment/courseof action for patients. SKOUT™ isindicated for white light colonoscopyonly. | device is not intended to replaceclinical decision making. | |
| UserPopulation | Adult patients undergoing colorectalcancer screening or surveillancecolonoscopy. | Adult patients undergoing screeningand surveillance endoscopicmucosal evaluations. |
| TechnologicalCharacteristics | The SKOUT™ system is composed ofhardware and software designed tohighlight portions of the colon wherethe device detects potential colorectalpolyps. | The GI Genius is composed ofhardware and software designed tohighlight portions of the colon wherethe device detects a potential lesion. |
| SoftwareAlgorithm | The SKOUT™ system utilizes anartificial intelligence-based algorithm toperform the polyp detection function. | The GI Genius system utilizes anartificial intelligence-based algorithmto perform the polyp detectionfunction. |
| Power Source | Hospital mains power | Hospital mains power |
| Safety Features | The Video Display Switch allows forinstantaneous toggling between theSKOUT™ video feed and the standardvideo feed in the event of softwareerror that affects video quality.The polyp detection marker is disabledif a biopsy tool enters the field of viewto prevent obstruction of the area ofinterest during intervention. | Unknown |
| SKOUT™ System | GI Genius | |
| SKOUT™ system GUI also has adevice status indicator, a greensquare, located in the top left corner ofthe SKOUT™ video feed. This GUIfeature is an additional provide acheck to the user that the SKOUT™system is on and in use, even whenpolyp detection notifications are not onthe screen to prevent undesired use ofthe Al. | ||
| Device Output | SKOUT™ system generates markersin the form of blue rectanglessuperimposed on the endoscopicvideo when potential colorectal polypsare identified. SKOUT™ markers arenot accompanied by a sound.The polyp detection marker is disabledif a biopsy tool enters the field of viewto prevent obstruction of the area ofinterest during intervention. | During a colonoscopy, the GIGenius system generates markers,which look like green squares andare accompanied by a short,low-volume sound, andsuperimposes them on the videofrom the endoscope camera when itidentifies a potential lesion. |
Table 1: Technological Characteristics Comparison
{5}------------------------------------------------
{6}------------------------------------------------
The SKOUT™ system and GI Genius have the same intended use, similar indications for use, comparable user population, hardware and software characteristics. Both devices provide as an output polyp detection markers that are superimposed onto endoscopic videos. Though there are minor differences between the two devices, such as the Video Display Switch and the low-volume sound, these differences do not raise different questions of safety and effectiveness as demonstrated by the non-clinical and clinical performance evaluation results.
Performance Data
The following testing was conducted for the SKOUT™ system with data included in the 510(k) document.
Software Verification and Validation
Software verification and validation was conducted for the SKOUT™ software to validate it for its intended use per the design documentation in line with recommendations outlined in General Principles of Software Validation, Guidance for Industry and FDA Staff. The SKOUT™ software demonstrated passing results on all applicable unit, integration, and requirements testing.
Electrical Safety/Electromagnetic Compatibility
The SKOUT™ system was evaluated for compliance to the following FDA-Recognized Consensus Standards:
- . IEC 60601-1:2005, AMD 1:2012 - Medical electrical equipment - Part 1: General requirements for basic requirements for basic safety and essential performance
{7}------------------------------------------------
- . IEC 60601-1-2: 2014 - Medical electrical equipment - Part 1-2: General requirements for basic requirements for basic safety and essential performance - Collateral standard: Electromagnetic compatibility - Requirements and tests
- IEC 60601-2-18: 2009 - Medical electrical equipment - Part 2-24: Particular requirements for the basic safety and essential performance of endoscopic equipment
Non-clinical performance testing
The following non-clinical performance testing areas, and corresponding results, were conducted:
- Standalone algorithm performance testing; The Standalone Performance Assessment was . performed to assess trends in the detection and classification of the SKOUT™ system and determine its ability to discriminate between normal mucosa and polyp tissue. This includes analysis of a predefined set of 79 HD videos with 94 polyps of relevant non-clinical performance metrics to evaluate this performance. These metrics comprehensively encompass standalone SKOUT™ system performance in relation to the algorithm's ability to detect and highlight potential polyps with a bounding box on colonoscopy videos. Of the 94 polyps included in this analysis, SKOUT™ detected 92 polyps.
- · Annotation methods:
Annotation was performed by a team of trained annotators who were assessed on their ability to successfully identify and correctly label polyps.
Ground truth was defined as data reviewed and either validated or created by expert gastroenterologists through a process referred to as gastroenterologist review. During gastroenterologist review, experts reviewed and either validated, rejected new labels post primary annotation. Gastroenterologist labels serve as the ground truth for frame level True Positive Rate (TPR), False Positive Rate(FPR) and object level TPR, FPR.
- . Primary Endpoints
-
- Object Level TPR: The proportion of suspected polyps that were detected by the device in the evaluation dataset and confirmed to be polyps using pathology findings. This metric demonstrates object/polyp level performance of the device algorithm.
-
- Object Level FPs: The number of suspected polyps that the device bounds per procedure which are not confirmed to be polyps by a) resection and b) pathology findings.
- Secondary Endpoints .
-
- Frame Level TPR: The proportion of all the frames with confirmed polyps in which the device bounds the polyp in the evaluation dataset.
-
- Frame Level FPR: The proportion of frames in which the device bounds an object that is not detected by the gastroenterologist in a colonoscopy during normal use (does not include frames when a surgical tool or NBI is detected).
{8}------------------------------------------------
- . Demographics
| Table 2: Demographic and Baseline InformationDemographics (Total no. of videos = 79) | ||
|---|---|---|
| Sex | No. of Videos | |
| Male | 34 | |
| Female | 45 | |
| Race | ||
| White | 72 | |
| Asian | 3 | |
| Black or African American | 2 | |
| Not Reported - Declined | 2 | |
| Ethnicity | ||
| Hispanic or Latino Heritage | 0 | |
| Non-Hispanic or Latin Heritage | 78 | |
| Declined | 1 | |
| Endoscopic Processor | ||
| EVIS EXERA III processor CV-190 | 79 |
Table 2: Demographic and Baseline Information
- Results ●
Table 3: Primary and Secondary Endpoints Results
| Object Level True PositiveRate (TPR) | 97.87 (95% Cl: 94.96, 100.0) |
|---|---|
| Object Level False Positive(FP) | 22.55 objects per a 15 minuteinterval (95% Cl: 18.954, 26.148) |
| Frame level performance | (79 videos / 94 polyps)True positive: 193,861True negative: 3,459,211False positive: 81,930False negative: 154,429 |
| True positive rate per frame | Mean: 55.66 % (95% CI: 55.50,55.83) |
| False positive rate perframe | Mean: 2.31% (95% CI:2.16 -2.44) |
- Subgroup analysis of endpoints ●
Table 4: Subgroups analysis of endpoints
Total Videos = 79, Total Polyps = 94, Total Detected Polyps = 92
{9}------------------------------------------------
| Polyp Detected | Total Polyps | Object TPR | 95 % Confidence Interval for Object TPR [low, high] | mean Object FP (15 min interval) | 95 % Confidence Interval for mean Object FP (15-min interval) [low, high] | ||
|---|---|---|---|---|---|---|---|
| Sex | |||||||
| Male | 45 | 46 | 97.83% | [93.61, 100.0] | 25.66 | [19.387, 31.927] | |
| Female | 47 | 48 | 97.92% | [93.88, 100.0] | 20.2 | [15.934, 24.4746] | |
| Age | |||||||
| 40 years to 64 years | 60 | 61 | 98.36% | [95.17, 100.0] | 21.54 | [17.002, 26.081] | |
| 65 years and up | 32 | 33 | 96.97% | [91.12, 100.0] | 24.61 | [18.429, 30.788] | |
| Size(mm) | |||||||
| 0 - 4 | 35 | 36 | 98.11% | [94.45, 100.0] | N/A | N/A | |
| 5 - 9 | 47 | 48 | 98.13% | [95.09, 100.0] | N/A | N/A | |
| 10+ | 10 | 10 | 100.00% | [100.0, 100.0] | N/A | N/A | |
| Histology | |||||||
| Adenoma | 50 | 50 | 100.00% | [100.0, 100.0] | N/A | N/A | |
| SSLs | 16 | 16 | 100.00% | [100.0, 100.0] | N/A | N/A | |
| Hyperplastic Polyp | 16 | 17 | 96.43% | [89.55, 100.0] | N/A | N/A | |
| Inflammatory Polyp | 2 | 2 | 100.00% | [100.0, 100.0] | N/A | N/A | |
| Not Histologically a Polyp | 5 | 6 | 90.91% | [79.79, 100.0] | N/A | N/A | |
| Unknown | 3 | 3 | 100.00% | [100.0, 100.0] | N/A | N/A | |
| Race | |||||||
| Declined | 2 | 2 | 100.00% | [100.0,100.0] | 37.72 | [30.512,47.189] | |
| White | 78 | 80 | 97.50% | [94.08,100.0] | 21.91 | [18.364,25.46] | |
| Asian | 11 | 11 | 100.00% | [100.0,100.0] | 34.69 | [9.012,78.4] | |
| Black orAfricanAmerican | 1 | 1 | 100.00% | [100.0,100.0] | 12.18 | [5.269,19.622] |
{10}------------------------------------------------
- Marker Persistence is defined as the continuous uninterrupted detection of a target in time. Alternatively, it can be explained as the time (in milliseconds) from the first appearance of a marker on a polyp until the first disappearance of the marker for the same polyp.
A sensitivity analysis of marker persistence is performed with all 79 videos denoting the object-level True Positive Rate and False Positive event count (mean per 15 minute session) as a function of marker persistence.
The following graph demonstrates that markers with higher persistence have a higher likelihood to be true positives.
Image /page/10/Figure/5 description: The image is a plot titled "Polyp TPR based on marker persistence". The x-axis is labeled "Marker persistence less than (ms)" and ranges from 0 to 8000. The y-axis is labeled "Polyp TPR" and ranges from 0.0 to 0.8. The plot shows a curve that increases rapidly from 0 to around 0.3 between 0 and 500 on the x-axis, then gradually increases to around 0.9 between 500 and 5000 on the x-axis, and then plateaus.
Figure 1: Polyp TPR based on Marker Persistence
The plot below shows FP events (per 15 min) as a function of marker persistence. The graph demonstrates that the majority of FP events occur with markers that persist for 1 second or less.
{11}------------------------------------------------
Polyp FP Event count (mean per 15 min session) based on marker persistence
Image /page/11/Figure/2 description: The image is a plot of "Polyp FP Event count (mean per 15 min session)" versus "Marker persistence less than (ms)". The x-axis ranges from 0 to 8000, while the y-axis ranges from 0 to 20. The plot shows a curve that increases rapidly from 0 to around 2000 on the x-axis, and then gradually flattens out as the x-axis increases.
Figure 2: Polyp FP Event count (mean per 15 min session) based on Marker Persistence
- Marker Overlap: In order to not obstruct gastroenterologists' view of polyps, ● SKOUT™ artificially increases the size of the boxes output by its Al algorithm.
Various metrics are used to understand SKOUT™'s overlap performance with the ground truth reference standards.
Intersection over Union (IOU) is the ratio of area of intersection of the SKOUT™ signal with just the intersection of the two boxes over the total area of both boxes (Figure 03). This calculation means that when comparing SKOUT's performance (artificially increased bounding box), the IOU figure should be low.
Image /page/11/Figure/7 description: The image shows two sets of overlapping rectangles. In the top set, a blue rectangle overlaps a green rectangle, and a yellow rectangle overlaps both. In the bottom set, a blue rectangle overlaps a yellow rectangle, and the yellow rectangle overlaps a green rectangle.
Figure 03: IOU
Intersection over Ground Truth (IOGT) is the ratio of area of intersection of SKOUT™ signal with ground truth bounding box over the area of the ground truth box (Figure 04). An IOGT value of 1 indicates that the ground truth box is fully
{12}------------------------------------------------
engulfed by the SKOUT™ signal.
Image /page/12/Picture/2 description: The image contains two rectangles, one above a horizontal line and one below. The rectangle above the line is nested inside another rectangle. The outer rectangle is blue, and the inner rectangle has a green border and a yellow fill. The rectangle below the line has a green border and a yellow fill.
Figure 04: IOGT
Our results show that our IOU was 0.299. More importantly, our IOGT value was 1.0 which indicates that on a median basis, all polyps were engulfed by a SKOUT bounding box.
Table 05: Signal overlap analysis
| Mean, 95% confidence interval if IOU ofsignal on 411 samples | 0.299, [0.289, 0.309] |
|---|---|
| Median IOGT of signal | 1.0 |
- Special Control Testing
- Pixel-level comparison of degradation of image quality due to the device: No visually . detectable differences between images were found with the introduction of the SKOUT™ system.
- Assessment of video delay due to marker annotation: 56.00ms (95% Cl: 50.54, . 61.46) and 3.25 (95% Cl: 2.93, 3.56) frame delay for Serial Digital Interface (SDI) and 62.33ms (95% Cl: 60.76, 63.90) and 3.74 (95% Cl: 3.65, 3.83) frame delay for Digital Visual Interface (DVI).
- Assessment of real-time endoscopic video delay due to the device; 56.67ms (95% . CI: 51.01, 62.33) and 3.28 (95% CI: 2.96, 3.62) frame delay for SDI and 60.67ms (95% Cl: 57.72, 63.61) and 3.64 (95% Cl: 3.46, 3.81) frame delay for DVI.
Human Factors
Human factors validation was performed following the FDA Guidance document Applying Human Factors and Usability Engineering to Medical Devices. Guidance for Industry and FDA Staff recommendations. The human factors validation demonstrated that the device functioned as intended, use-related risk has been mitigated, and the SKOUT™ system is safe for its intended use.
{13}------------------------------------------------
Clinical Testing
A multicenter, prospective, randomized controlled trial of the SKOUT™ system was conducted in the United States in order to evaluate the safety and efficacy of this device. The study design involved 2 arms in which adult patients undergoing either screening or surveillance colonoscopy procedures were randomized to either an Al-aided arm (standard colonoscopy with the use of the SKOUT™ system), or the control arm (standard colonoscopy without the use of the SKOUT™ system).
The aim of this study is to evaluate the clinical benefit and safety of using a computer-aided detection (CADe) device, the SKOUT™ system, in colonoscopy procedures with the indication of screening or surveillance.
Inclusion Criteria:
- Undergoing colonoscopy with screening or surveillance. ●
- Whose endoscopist is a participating provider. ●
- Who have given informed consent.
Exclusion Criteria:
- Have a history of inflammatory bowel disease.
- Have a history of familial adenomatous polyposis.
- Are under the age of 40.
- Have had a colonoscopy within the previous three (3) years.
- . Patients undergoing diagnostic colonoscopy with high risk indications including iron deficiency anemia, abnormal CT imaging, unexplained weight loss, Lynch Syndrome, blood in stool or FIT positive test.
- . Use anti-platelet agents or anticoagulants that preclude the removal of polyps during the procedure.
- Entered with poor bowel preparation (inadeguate for procedure as assessed by the Investigator).
Provider's Eligibility requirements:
- United States board-certified gastroenterologist.
- Has performed at least 1,000 colonoscopies. ●
- Has an ADR greater than or equal to 25%.
The co-primary endpoints of the study included a performance endpoint (adenomas per colonoscopy - APC) and a safety endpoint (positive percent agreement - PPA).
- APC: The total number of adenomas detected divided by the total number of colonoscopies. .
- PPA**: PPA is the fraction of adenomas, sessile serrated lesions, and hyperplastic polyps . of the proximal colon (caecum, ascending colon, hepatic flexure, and transverse colon) out of the total number of resections.
- PPA (or PPV): It is the fraction of adenomas, sessile serrated lesions, and large ● (>10mm) hyperplastic polyps of the proximal colon (caecum, ascending colon, hepatic flexure, and transverse colon) out of total number of resections.
Table 06:Demographics and Baseline Information
{14}------------------------------------------------
| Control (N=677) | Treatment (N=682) | p-value | |
|---|---|---|---|
| Sex | 0.612 | ||
| Male | 355 (52.4%) | 368 (54%) | |
| Female | 322 (47.6%) | 314 (46%) | |
| Age - Continuous | 0.159 | ||
| Mean (SD) | 59.9 (8.8) | 60.6 (8.9) | |
| Age - Categorical | 0.21 | ||
| 40 <= years < 50 | 54 (8%) | 53 (7.8%) | |
| 50 <= years < 65 | 415 (61.3%) | 389 (57%) | |
| >= 65 years | 208 (30.7%) | 240 (35.2%) | |
| Race | 0.194* | ||
| American Indian or Alaska Native | 2 (0.3%) | 0 (0%) | |
| Asian | 21 (3.1%) | 18 (2.6%) | |
| Black or African American | 36 (5.3%) | 47 (6.9%) | |
| Native Hawaiian or OtherPacific-Islander | 3 (0.4%) | 0 (0%) | |
| White | 563 (83.2%) | 567 (83.3%) | |
| More Than One Race | 5 (0.7%) | 2 (0.3%) | |
| Don't Know | 11 (1.6%) | 18 (2.6%) | |
| Refused | 36 (5.3%) | 29 (4.3%) | |
| Ethnicity | 0.596 | ||
| Not Hispanic or Latino | 627 (92.6%) | 627 (91.9%) | |
| Hispanic or Latino | 31 (4.6%) | 29 (4.3%) | |
| Don't Know | 7 (1%) | 13 (1.9%) | |
| Refused | 12 (1.8%) | 13 (1.9%) | |
| Site of Procedure | >0.99 | ||
| Concord Endoscopy Center | 206 (30.4%) | 210 (30.8%) | |
| Mount Auburn Hospital | 168 (24.8%) | 163 (23.9%) | |
| Massachusetts General Hospital | 59 (8.7%) | 62 (9.1%) | |
| MNGI Digestive Health | 160 (23.6%) | 161 (23.6%) | |
| Boston Medical Center | 84 (12.4%) | 86 (12.6%) | |
| Years Since Last Colonscopy | >0.99 | ||
| 3 <= years < 5 | 71 (10.5%) | 72 (10.6%) | |
| 5 <= years < 10 | 261 (38.6%) | 268 (39.3%) | |
| >= 10 years | 129 (19.1%) | 127 (18.6%) | |
| No Previous Colonoscopy | 216 (31.9%) | 215 (31.5%) |
- Fisher's Exact Test Used Instead of Chi-Square Test
A total of 1,359 patients were included in the modified Intention To Treat (mlTT) population for primary analysis, including 682 who received a colonoscopy with the SKOUT™ system and 677 who received a standard colonoscopy. The evaluation of our primary endpoints and secondary endpoint for the mITT population is summarized below:
Table 07: Primary and Secondary Endpoints results for mITT population
{15}------------------------------------------------
| Control(N=677) | Treatment(N=682) | Difference(Treatment-Control) | 95% CI forDifference | p-value | |
|---|---|---|---|---|---|
| Adenomas PerColonoscopy* | 0.830 | 1.054 | 0.224 | (0.060, 0.382) | 0.002 |
| Positive PredictiveValue | 0.717 | 0.674 | -0.043 | (-0.094, 0.010) | <0.001 |
| Adenoma DetectionRate* | 0.439 | 0.478 | 0.039 | (-0.012, 0.097) | 0.065 |
| Average Number ofSessile SerratedLesions | 0.284 | 0.199 | -0.084 | (0.003, 0.165) | 0.042 |
| Mean SurveillanceInterval for NextColonoscopy | 6.307 | 6.275 | -0.032 | (-0.275, 0.338) | 0.839 |
APC and PPV are Co-Primary Endpoints
- No Adenocarcinomas Found
In the mITT cohort, PPA** was found to be 75.7% in the control arm (n=677) and 70.9% in the treatment arm (n=682), a difference of -4.8%. The 95% Cl for the difference is (- 9.5% to 0.3%) and the p-value is <0.001.
No adverse events or complications were reported during the study.
{16}------------------------------------------------
| Standard(N=1129) | SkoutColonoscopy(N=1334) | P value | |
|---|---|---|---|
| Polyp Location (Endoscopy Report)Rectum | 116 (10.7%) | 141 (10.9%) | 0.641 |
| Rectosigmoid | 27 (2.5%) | 39 (3%) | |
| Sigmoid | 169 (15.6%) | 193 (14.9%) | |
| Descending Colon | 111 (10.2%) | 132 (10.2%) | |
| Splenic Flexure | 12 (1.1%) | 18 (1.4%) | |
| Transverse Colon | 270 (24.9%) | 293 (22.6%) | |
| Hepatic Flexure | 41 (3.8%) | 69 (5.3%) | |
| Ascending Colon | 250 (23%) | 295 (22.7%) | |
| Cecum | 90 (8.3%) | 119 (9.2%) | |
| Polyp Size (Via Endoscopy Report)Mean (SD) | 5 (3) | 4.8 (3.1) | 0.125 |
| Polyp Size (via Endoscopy Report)<5 mm | 607 (53.8%) | 720 (54%) | <0.001* |
| 5 <= mm < 10 | 410 (36.3%) | 537 (40.3%) | |
| >= 10 mm | 112 (9.9%) | 77 (5.8%) | |
| Size Unmatched | 0 (0%) | 0 (0%) | |
| Polyp MorphologySessile | 937 (86.4%) | 1123 (86.6%) | 0.185 |
| Pedunculated | 87 (8%) | 82 (6.3%) | |
| Flat | 48 (4.4%) | 76 (5.9%) | |
| Not Available | 12 (1.1%) | 16 (1.2%) | |
| Polyp HistologyHyperplastic | 205 (18.9%) | 280 (21.6%) | <0.001* |
| Adenoma | 557 (51.4%) | 716 (55.2%) | |
| Adenoma with High-Grade Dysplasia | 2 (0.2%) | 2 (0.2%) | |
| Tubulovillous Adenoma | 22 (2%) | 30 (2.3%) | |
| Tubulovillous Adenoma with High-Grade Dysplasia | 3 (0.3%) | 0 (0%) | |
| Villous Adenoma | 0 (0%) | 0 (0%) | |
| Villous Adenoma with High-Grade Dysplasia | 0 (0%) | 0 (0%) | |
| Sessile Serrated Adenoma | 188 (17.3%) | 141 (10.9%) | |
| Sessile Serrated Adenoma with High-Grade Dysplasia | 0 (0%) | 0 (0%) | |
| Traditional Serrated Adenoma | 8 (0.7%) | 0 (0%) | |
| Traditional Serrated Adenoma with High-Grade Dysplasia | 0 (0%) | 0 (0%) | |
| Adenocarinoma | 0 (0%) | 0 (0%) | |
| Carninoid Tumor | 0 (0%) | 0 (0%) | |
| Inflammatory Polyp | 9 (0.8%) | 16 (1.2%) | |
| Unknown/Unavailable | 12 (1.1%) | 16 (1.2%) | |
| Not Histologically A Polyp | 78 (7.2%) | 96 (7.4%) | |
| Polyp Match | 0.007 | ||
| One-to-One Match | 892 (81.8%) | 1118 (85.9%) | |
| Match Cannot Be Determined | 199 (18.2%) | 183 (14.1%) | |
| Standard (N=1129) | Skout Colonoscopy (N=1334) | P value | |
| Distal ColonMean (SD) | 4.8 (2.7) | 4.6 (2.7) | 0.282 |
| Distal Colon | 0.1* | ||
| <5 mm | 234 (53.8%) | 308 (58.9%) | |
| 5 <= mm < 10 | 170 (39.1%) | 192 (36.7%) | |
| >= 10 mm | 31 (7.1%) | 23 (4.4%) | |
| Size Unmatched | 0 (0%) | 0 (0%) | |
| Proximal ColonMean (SD) | 5.2 (3.2) | 5 (3.2) | 0.27 |
| Proximal Colon | <0.001* | ||
| <5 mm | 346 (53.1%) | 389 (50.1%) | |
| 5 <= mm < 10 | 230 (35.3%) | 335 (43.2%) | |
| >= 10 mm | 75 (11.5%) | 52 (6.7%) | |
| Size Unmatched | 0 (0%) | 0 (0%) |
Table 09: Relationship between size of resected polyps and their location
{17}------------------------------------------------
Table 10: Relationship between size of resected polyps across histology:
| Control (N=986) | Treatment (N=1168) | p-value | |
|---|---|---|---|
| Adenoma | |||
| Size: < 5 mm | 301 (30.527%) | 376 (32.192%) | 0.434 |
| Size: 5 <= mm < 10 | 221 (22.414%) | 327 (27.997%) | 0.004 |
| Size: >= 10 mm | 63 (6.389%) | 44 (3.767%) | 0.007 |
| Size: All Sizes | 585 (59.331%) | 747 (63.955%) | 0.031 |
| SSL | |||
| Size: < 5 mm | 70 (7.099%) | 36 (3.082%) | <0.001 |
| Size: 5 <= mm < 10 | 90 (9.128%) | 79 (6.764%) | 0.051 |
| Size: >= 10 mm | 36 (3.651%) | 26 (2.226%) | 0.066 |
| Size: All Sizes | 196 (19.878%) | 141 (12.072%) | <0.001 |
| Hyperplastic | |||
| Size: < 5 mm | 138 (13.996%) | 200 (17.123%) | 0.054 |
| Size: 5 <= mm < 10 | 64 (6.491%) | 77 (6.592%) | >0.99 |
| Size: >= 10 mm | 3 (0.304%) | 3 (0.257%) | >0.99 |
| Size: All Sizes | 205 (20.791%) | 280 (23.973%) | 0.087 |
The results of the clinical performance as documented in the pivotal clinical study show a statistically significant increase in APC, and PPA fell statistically within the prespecified noninferiority margin , demonstrating that the performance of the SKOUT™ system achieved benchmark expectations and a safety and effectiveness profile comparable to the predicate device.
Conclusions
{18}------------------------------------------------
The SKOUT™ system has the same intended uses and similar indications, technological characteristics, and principles of operation as its predicate device. The minor differences in indications do not alter the intended use of the device and do not affect its safety and effectiveness when used as labeled. In addition, the minor technological differences between the SKOUT™ system and its predicate device do not raise different issues of safety or effectiveness. Performance data from this study demonstrate that the SKOUT™ system is as safe and effective as the predicate device. Thus, the SKOUT™ system can be considered substantially equivalent.
§ 876.1520 Gastrointestinal lesion software detection system.
(a)
Identification. A gastrointestinal lesion software detection system is a computer-assisted detection device used in conjunction with endoscopy for the detection of abnormal lesions in the gastrointestinal tract. This device with advanced software algorithms brings attention to images to aid in the detection of lesions. The device may contain hardware to support interfacing with an endoscope.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Clinical performance testing must demonstrate that the device performs as intended under anticipated conditions of use, including detection of gastrointestinal lesions and evaluation of all adverse events.
(2) Non-clinical performance testing must demonstrate that the device performs as intended under anticipated conditions of use. Testing must include:
(i) Standalone algorithm performance testing;
(ii) Pixel-level comparison of degradation of image quality due to the device;
(iii) Assessment of video delay due to marker annotation; and
(iv) Assessment of real-time endoscopic video delay due to the device.
(3) Usability assessment must demonstrate that the intended user(s) can safely and correctly use the device.
(4) Performance data must demonstrate electromagnetic compatibility and electrical safety, mechanical safety, and thermal safety testing for any hardware components of the device.
(5) Software verification, validation, and hazard analysis must be provided. Software description must include a detailed, technical description including the impact of any software and hardware on the device's functions, the associated capabilities and limitations of each part, the associated inputs and outputs, mapping of the software architecture, and a description of the video signal pipeline.
(6) Labeling must include:
(i) Instructions for use, including a detailed description of the device and compatibility information;
(ii) Warnings to avoid overreliance on the device, that the device is not intended to be used for diagnosis or characterization of lesions, and that the device does not replace clinical decision making;
(iii) A summary of the clinical performance testing conducted with the device, including detailed definitions of the study endpoints and statistical confidence intervals; and
(iv) A summary of the standalone performance testing and associated statistical analysis.