(263 days)
Not Found
Yes
The summary explicitly states, "The SKOUT™ system utilizes an artificial intelligence-based algorithm to perform the polyp detection function."
No.
The device is a computer-aided detection tool that assists gastroenterologists in identifying potential colorectal polyps; it is explicitly stated that it is "not intended to replace a full patient evaluation, nor is it intended to be relied upon to make a primary interpretation of endoscopic procedures, medical diagnosis, or recommendations of treatment/course of action for patients." Therefore, it is not a therapeutic device.
Yes
The device aids in the identification of potential colorectal polyps, which are indicators of disease, thus performing a diagnostic function by assisting in the detection of abnormalities.
Yes
The device is explicitly described as a "software device" and a "software-based computer aided detection (CADe) system" that processes video data. There is no mention of accompanying hardware components included with the device itself.
Based on the provided information, the SKOUT system is not an In Vitro Diagnostic (IVD) device.
Here's why:
- Intended Use: The primary intended use of the SKOUT system is to detect potential colorectal polyps in real time during colonoscopy examinations and provide location information to assist gastroenterologists. It is explicitly stated that it is not intended to make a primary interpretation of endoscopic procedures, medical diagnosis, or recommendations of treatment/course of action for patients. IVD devices are typically used to perform tests on samples taken from the human body (like blood, urine, tissue) to provide information for diagnosis, monitoring, or screening. The SKOUT system analyzes live video feed, not biological samples.
- Device Description: The device processes endoscopic video in real time to provide a visual aid. This is distinct from the analysis of biological samples characteristic of IVD devices.
- Input: The input is high-definition endoscopic video, not biological specimens.
- Performance Studies: The performance studies focus on the system's ability to detect polyps within the video feed and its impact on clinical outcomes during colonoscopy. While these studies are important for demonstrating the device's effectiveness, they are not the types of analytical and clinical performance studies typically conducted for IVD devices (e.g., studies evaluating the accuracy of measuring a specific analyte in a sample).
In summary, the SKOUT system is a software-based medical device that uses AI to assist in the visual detection of potential polyps during a colonoscopy procedure. It does not perform tests on biological samples and is not intended for primary diagnosis, which are key characteristics of IVD devices.
No
The letter does not state that the FDA has reviewed and approved or cleared a Predetermined Change Control Plan (PCCP) for this specific device.
Intended Use / Indications for Use
The SKOUT system is a software device designed to detect potential colorectal polyps in real time during colonoscopy examinations. It is indicated as a computer-aided detection tool providing colorectal polyps location information to assist qualified and trained gastroenterologists in identifying potential colorectal polyps during colonoscopy examinations in adult patients undergoing colorectal cancer screening or surveillance.
The SKOUT system is only intended to assist the gastroenterologist in identifying suspected colorectal polyps and the gastroenterologist is responsible for reviewing SKOUT suspected polyp areas and confirming the presence of a polyp based on their own medical judgment. SKOUT is not intended to replace a full patient evaluation, nor is it intended to be relied upon to make a primary interpretation of endoscopic procedures, medical diagnosis, or recommendations of treatment/course of action for patients. SKOUT is indicated for white light colonoscopy only.
Product codes
QNP
Device Description
The SKOUT™ system is a software-based computer aided detection (CADe) system for the analysis of high-definition endoscopic video during colonoscopy procedures. The SKOUT™ system is intended to aid gastroenterologists with the detection of potential colorectal polyps during colonoscopy by providing an informational visual aid on the endoscopic monitor using trained software that processes the endoscopic video in real time.
Users will primarily interact with the SKOUT™ system by observing the software display, including the polyp detection box and device status indicator signal.
Mentions image processing
The SKOUT™ system is a software-based computer aided detection (CADe) system for the analysis of high-definition endoscopic video during colonoscopy procedures. ... processes the endoscopic video in real time.
Mentions AI, DNN, or ML
The SKOUT™ system utilizes an artificial intelligence-based algorithm to perform the polyp detection function.
Input Imaging Modality
High-definition endoscopic video; white light colonoscopy
Anatomical Site
Colorectal / colon
Indicated Patient Age Range
Adult patients / 40 years to 64 years; 65 years and up
Intended User / Care Setting
Qualified and trained gastroenterologists; Hospital
Description of the training set, sample size, data source, and annotation protocol
Not Found
Description of the test set, sample size, data source, and annotation protocol
Standalone algorithm performance testing; The Standalone Performance Assessment was performed to assess trends in the detection and classification of the SKOUT™ system and determine its ability to discriminate between normal mucosa and polyp tissue. This includes analysis of a predefined set of 79 HD videos with 94 polyps of relevant non-clinical performance metrics to evaluate this performance. These metrics comprehensively encompass standalone SKOUT™ system performance in relation to the algorithm's ability to detect and highlight potential polyps with a bounding box on colonoscopy videos. Annotation was performed by a team of trained annotators who were assessed on their ability to successfully identify and correctly label polyps. Ground truth was defined as data reviewed and either validated or created by expert gastroenterologists through a process referred to as gastroenterologist review. During gastroenterologist review, experts reviewed and either validated, rejected new labels post primary annotation. Gastroenterologist labels serve as the ground truth for frame level True Positive Rate (TPR), False Positive Rate(FPR) and object level TPR, FPR.
Summary of Performance Studies (study type, sample size, AUC, MRMC, standalone performance, key results)
Standalone algorithm performance testing:
- Study type: Standalone algorithm performance testing to assess trends in detection and classification and ability to discriminate between normal mucosa and polyp tissue.
- Sample size: 79 HD videos with 94 polyps.
- Standalone performance: Of the 94 polyps included in this analysis, SKOUT™ detected 92 polyps.
- Primary Endpoints:
- Object Level TPR: 97.87 (95% Cl: 94.96, 100.0)
- Object Level FPs: 22.55 objects per a 15 minute interval (95% Cl: 18.954, 26.148)
- Secondary Endpoints:
- Frame level performance (79 videos / 94 polyps):
- True positive: 193,861
- True negative: 3,459,211
- False positive: 81,930
- False negative: 154,429
- True positive rate per frame: Mean: 55.66 % (95% CI: 55.50, 55.83)
- False positive rate per frame: Mean: 2.31% (95% CI:2.16 - 2.44)
- Frame level performance (79 videos / 94 polyps):
- Key results: Markers with higher persistence have a higher likelihood to be true positives. The majority of FP events occur with markers that persist for 1 second or less. IOU was 0.299. Median IOGT value was 1.0, indicating that on a median basis, all polyps were engulfed by a SKOUT bounding box.
Clinical Testing:
- Study type: A multicenter, prospective, randomized controlled trial.
- Sample size: 1,359 patients included in the modified Intention To Treat (mITT) population for primary analysis (682 in AI-aided arm, 677 in control arm).
- Co-primary endpoints: Adenomas per colonoscopy (APC) and positive percent agreement (PPA/PPV).
- Key results:
- APC: Control arm: 0.830, Treatment arm: 1.054. Difference (Treatment-Control): 0.224 (95% CI: 0.060, 0.382), p-value: 0.002.
- Positive Predictive Value (PPV): Control arm: 0.717, Treatment arm: 0.674. Difference (Treatment-Control): -0.043 (95% CI: -0.094, 0.010), p-value:
§ 876.1520 Gastrointestinal lesion software detection system.
(a)
Identification. A gastrointestinal lesion software detection system is a computer-assisted detection device used in conjunction with endoscopy for the detection of abnormal lesions in the gastrointestinal tract. This device with advanced software algorithms brings attention to images to aid in the detection of lesions. The device may contain hardware to support interfacing with an endoscope.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Clinical performance testing must demonstrate that the device performs as intended under anticipated conditions of use, including detection of gastrointestinal lesions and evaluation of all adverse events.
(2) Non-clinical performance testing must demonstrate that the device performs as intended under anticipated conditions of use. Testing must include:
(i) Standalone algorithm performance testing;
(ii) Pixel-level comparison of degradation of image quality due to the device;
(iii) Assessment of video delay due to marker annotation; and
(iv) Assessment of real-time endoscopic video delay due to the device.
(3) Usability assessment must demonstrate that the intended user(s) can safely and correctly use the device.
(4) Performance data must demonstrate electromagnetic compatibility and electrical safety, mechanical safety, and thermal safety testing for any hardware components of the device.
(5) Software verification, validation, and hazard analysis must be provided. Software description must include a detailed, technical description including the impact of any software and hardware on the device's functions, the associated capabilities and limitations of each part, the associated inputs and outputs, mapping of the software architecture, and a description of the video signal pipeline.
(6) Labeling must include:
(i) Instructions for use, including a detailed description of the device and compatibility information;
(ii) Warnings to avoid overreliance on the device, that the device is not intended to be used for diagnosis or characterization of lesions, and that the device does not replace clinical decision making;
(iii) A summary of the clinical performance testing conducted with the device, including detailed definitions of the study endpoints and statistical confidence intervals; and
(iv) A summary of the standalone performance testing and associated statistical analysis.
0
Image /page/0/Picture/0 description: The image contains the logo of the U.S. Food and Drug Administration (FDA). On the left is the Department of Health & Human Services logo. To the right of that is the FDA logo, which is a blue square with the letters "FDA" in white. To the right of the blue square is the text "U.S. FOOD & DRUG ADMINISTRATION" in blue.
August 12, 2022
Iterative Scopes Inc. Dennis Francoeur Director of Regulatory Affairs 14 Arrow St. 3rd Floor Cambridge, MA 02138
Re: K213686 Trade/Device Name: SKOUT Software Regulation Number: 21 CFR 876.1520 Regulation Name: Gastrointestinal Lesion Software Detection System Regulatory Class: Class II Product Code: QNP Dated: November 22, 2021 Received: November 22, 2021
Dear Dennis Francoeur:
We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database located at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the
1
Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR 803) for devices or postmarketing safety reporting (21 CFR 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safetyreporting-combination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR 4. Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR 1000-1050.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-devices/medical-device-safety/medicaldevice-reporting-mdr-how-report-medical-device-problems.
For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medical-device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-devices/device-advicecomprehensive-regulatory-assistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely,
Shanil P. Haugen, Ph.D. Assistant Director DHT3A: Division of Renal, Gastrointestinal, Obesity and Transplant Devices OHT3: Office of GastroRenal, ObGyn, General Hospital and Urology Devices Office of Product Evaluation and Quality Center for Devices and Radiological Health
Enclosure
2
Indications for Use
510(k) Number (if known) K213686
Device Name SKOUT System
Indications for Use (Describe)
The SKOUT system is a software device designed to detect potential colorectal polyps in real time during colonoscopy examinations. It is indicated as a computer-aided detection tool providing colorectal polyps location information to assist qualified and trained gastroenterologists in identifying potential colorectal polyps during colonoscopy examinations in adult patients undergoing colorectal cancer screening or surveillance.
The SKOUT system is only intended to assist the gastroenterologist in identifying suspected colorectal polyps and the gastroenterologist is responsible for reviewing SKOUT suspected polyp areas and confirming the presence of a polyp based on their own medical judgment. SKOUT is not intended to replace a full patient evaluation, nor is it intended to be relied upon to make a primary interpretation of endoscopic procedures, medical diagnosis, or recommendations of treatment/course of action for patients. SKOUT is indicated for white light colonoscopy only.
Type of Use (Select one or both, as applicable) | |
---|---|
------------------------------------------------- | -- |
X Prescription Use (Part 21 CFR 801 Subpart D)
Over-The-Counter Use (21 CFR 801 Subpart C)
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
3
K213686 Page 1 of 16
510(k) SUMMARY
SKOUT™ system
Submitter:
Iterative Scopes, Inc.
14 Arrow St. 3rd Floor
Cambridge MA 02138
Phone: (603) 819-8387
Contact Person: Dennis Francoeur, Director of Requlatory Affairs
Date Prepared: August 11, 2022
Name of Device: SKOUTTM svstem
Classification Name: Gastrointestinal Lesion Software Detection System
Classification Panel: Gastroenterology and Urology
Regulation Number: 876.1520
Product Code: QNP
Predicate Device: GI Genius, Cosmo Artificial Intelligence - AI, LTD, DEN200055
Device Description
The SKOUT™ system is a software-based computer aided detection (CADe) system for the analysis of high-definition endoscopic video during colonoscopy procedures. The SKOUT™ system is intended to aid gastroenterologists with the detection of potential colorectal polyps during colonoscopy by providing an informational visual aid on the endoscopic monitor using trained software that processes the endoscopic video in real time.
Users will primarily interact with the SKOUT™ system by observing the software display, including the polyp detection box and device status indicator signal.
Polyp Detection Notification
The SKOUT™ system has a main graphical user interface (GU) feature of the polyp detection notification. The polyp detection is a two-dimensional blue rectangular outline generated around any suspected polyps on the endoscopic video feed. Display of this notification is deactivated if / when a surgical tool enters the frame or if the polyp is no longer being detected.
The polyp detection notification enables users to:
- Detect potential colorectal polyps during colonoscopy examinations in adult patients . undergoing a colorectal cancer screening or surveillance procedure.
- . Utilize a tool that provides additional information for endoscopic observation.
4
Device Status Indicator
The SKOUT™ system has an additional GUI feature that notifies users of the current device status (active or error). The device status indicator signal displays as a two-dimensional solid green box in the left-hand corner of the display if the device is powered on and actively processing the input video and as a red X if there is a video processing error.
Intended Use / Indications for Use
The SKOUT™ system is a software device designed to detect potential colorectal polyps in real time during colonoscopy examinations. It is indicated as a computer-aided detection tool providing colorectal polyps location information to assist qualified and trained gastroenterologists in identifying potential colorectal polyps during colonoscopy examinations in adult patients undergoing colorectal cancer screening or surveillance.
The SKOUT™ system is only intended to assist the gastroenterologist in identifying suspected colorectal polyps and the gastroenterologist is responsible for reviewing SKOUT™ suspected polyp areas and confirming the presence or absence of a polyp based on their own medical judgment. SKOUT™ is not intended to replace a full patient evaluation, nor is it intended to be relied upon to make a primary interpretation of endoscopic procedures, medical diagnosis, or recommendations of treatment/course of action for patients. SKOUT™ is indicated for white light colonoscopy only.
Summary of Technological Characteristics
Both the subject and the predicate devices are Computer-Assisted Detection (CADe) devices used in conjunction with endoscopy for the detection of abnormal lesions in the gastrointestinal tract. At a high level, the subject and predicate devices are based on the following same technological elements. A table comparing the key features of the subject and predicate devices is provided below:
SKOUT™ System | GI Genius | |
---|---|---|
Intended Use | A gastrointestinal lesion software | |
detection system is a | ||
computer-assisted detection device | ||
used in conjunction with endoscopy for | ||
the detection of abnormal lesions in | ||
the gastrointestinal tract. This device | ||
with advanced software algorithms | ||
brings attention to images to aid in the | ||
detection of lesions. The device has | ||
hardware components to support | ||
interfacing with an endoscope. | A gastrointestinal lesion software | |
detection system is a | ||
computer-assisted detection device | ||
used in conjunction with endoscopy | ||
for the detection of abnormal lesions | ||
in the gastrointestinal tract. This | ||
device with advanced software | ||
algorithms brings attention to | ||
images to aid in the detection of | ||
lesions. The device may contain | ||
hardware to support interfacing with | ||
an endoscope. | ||
Indications for | ||
Use | The SKOUT™ system is a software | |
device designed to detect potential | ||
colorectal polyps in real time during | ||
colonoscopy examinations. It is | ||
indicated as a computer-aided | ||
detection tool providing colorectal | The GI Genius System is a | |
computer-assisted reading tool | ||
designed to aid endoscopists in | ||
detecting colonic mucosal lesions | ||
(such as polyps and adenomas) in | ||
real time during standard white-light | ||
SKOUT™ System | GI Genius | |
polys location information to assist | ||
qualified and trained | ||
gastroenterologists in identifying | ||
potential colorectal polyps during | ||
colonoscopy examinations in adult | ||
patients undergoing colorectal cancer | ||
screening or surveillance. | endoscopy examinations of patients | |
undergoing screening and | ||
surveillance endoscopic mucosal | ||
evaluations. The GI Genius | ||
computer-assisted detection device | ||
is limited for use with standard white | ||
light endoscopy imaging only. This | ||
The SKOUT™ system is only intended | ||
to assist the gastroenterologist in | ||
identifying suspected colorectal polyps | ||
and the gastroenterologist is | ||
responsible for reviewing SKOUT™ | ||
suspected polyp areas and confirming | ||
the presence or absence of a polyp | ||
based on their own medical judgment. | ||
SKOUT™ is not intended to replace a | ||
full patient evaluation, nor is it | ||
intended to be relied upon to make a | ||
primary interpretation of endoscopic | ||
procedures, medical diagnosis, or | ||
recommendations of treatment/course | ||
of action for patients. SKOUT™ is | ||
indicated for white light colonoscopy | ||
only. | device is not intended to replace | |
clinical decision making. | ||
User | ||
Population | Adult patients undergoing colorectal | |
cancer screening or surveillance | ||
colonoscopy. | Adult patients undergoing screening | |
and surveillance endoscopic | ||
mucosal evaluations. | ||
Technological | ||
Characteristics | The SKOUT™ system is composed of | |
hardware and software designed to | ||
highlight portions of the colon where | ||
the device detects potential colorectal | ||
polyps. | The GI Genius is composed of | |
hardware and software designed to | ||
highlight portions of the colon where | ||
the device detects a potential lesion. | ||
Software | ||
Algorithm | The SKOUT™ system utilizes an | |
artificial intelligence-based algorithm to | ||
perform the polyp detection function. | The GI Genius system utilizes an | |
artificial intelligence-based algorithm | ||
to perform the polyp detection | ||
function. | ||
Power Source | Hospital mains power | Hospital mains power |
Safety Features | The Video Display Switch allows for | |
instantaneous toggling between the | ||
SKOUT™ video feed and the standard | ||
video feed in the event of software | ||
error that affects video quality. |
The polyp detection marker is disabled
if a biopsy tool enters the field of view
to prevent obstruction of the area of
interest during intervention. | Unknown |
| | SKOUT™ System | GI Genius |
| | SKOUT™ system GUI also has a
device status indicator, a green
square, located in the top left corner of
the SKOUT™ video feed. This GUI
feature is an additional provide a
check to the user that the SKOUT™
system is on and in use, even when
polyp detection notifications are not on
the screen to prevent undesired use of
the Al. | |
| Device Output | SKOUT™ system generates markers
in the form of blue rectangles
superimposed on the endoscopic
video when potential colorectal polyps
are identified. SKOUT™ markers are
not accompanied by a sound.
The polyp detection marker is disabled
if a biopsy tool enters the field of view
to prevent obstruction of the area of
interest during intervention. | During a colonoscopy, the GI
Genius system generates markers,
which look like green squares and
are accompanied by a short,
low-volume sound, and
superimposes them on the video
from the endoscope camera when it
identifies a potential lesion. |
Table 1: Technological Characteristics Comparison
5
6
The SKOUT™ system and GI Genius have the same intended use, similar indications for use, comparable user population, hardware and software characteristics. Both devices provide as an output polyp detection markers that are superimposed onto endoscopic videos. Though there are minor differences between the two devices, such as the Video Display Switch and the low-volume sound, these differences do not raise different questions of safety and effectiveness as demonstrated by the non-clinical and clinical performance evaluation results.
Performance Data
The following testing was conducted for the SKOUT™ system with data included in the 510(k) document.
Software Verification and Validation
Software verification and validation was conducted for the SKOUT™ software to validate it for its intended use per the design documentation in line with recommendations outlined in General Principles of Software Validation, Guidance for Industry and FDA Staff. The SKOUT™ software demonstrated passing results on all applicable unit, integration, and requirements testing.
Electrical Safety/Electromagnetic Compatibility
The SKOUT™ system was evaluated for compliance to the following FDA-Recognized Consensus Standards:
- . IEC 60601-1:2005, AMD 1:2012 - Medical electrical equipment - Part 1: General requirements for basic requirements for basic safety and essential performance
7
- . IEC 60601-1-2: 2014 - Medical electrical equipment - Part 1-2: General requirements for basic requirements for basic safety and essential performance - Collateral standard: Electromagnetic compatibility - Requirements and tests
- IEC 60601-2-18: 2009 - Medical electrical equipment - Part 2-24: Particular requirements for the basic safety and essential performance of endoscopic equipment
Non-clinical performance testing
The following non-clinical performance testing areas, and corresponding results, were conducted:
- Standalone algorithm performance testing; The Standalone Performance Assessment was . performed to assess trends in the detection and classification of the SKOUT™ system and determine its ability to discriminate between normal mucosa and polyp tissue. This includes analysis of a predefined set of 79 HD videos with 94 polyps of relevant non-clinical performance metrics to evaluate this performance. These metrics comprehensively encompass standalone SKOUT™ system performance in relation to the algorithm's ability to detect and highlight potential polyps with a bounding box on colonoscopy videos. Of the 94 polyps included in this analysis, SKOUT™ detected 92 polyps.
- · Annotation methods:
Annotation was performed by a team of trained annotators who were assessed on their ability to successfully identify and correctly label polyps.
Ground truth was defined as data reviewed and either validated or created by expert gastroenterologists through a process referred to as gastroenterologist review. During gastroenterologist review, experts reviewed and either validated, rejected new labels post primary annotation. Gastroenterologist labels serve as the ground truth for frame level True Positive Rate (TPR), False Positive Rate(FPR) and object level TPR, FPR.
- . Primary Endpoints
-
- Object Level TPR: The proportion of suspected polyps that were detected by the device in the evaluation dataset and confirmed to be polyps using pathology findings. This metric demonstrates object/polyp level performance of the device algorithm.
-
- Object Level FPs: The number of suspected polyps that the device bounds per procedure which are not confirmed to be polyps by a) resection and b) pathology findings.
- Secondary Endpoints .
-
- Frame Level TPR: The proportion of all the frames with confirmed polyps in which the device bounds the polyp in the evaluation dataset.
-
- Frame Level FPR: The proportion of frames in which the device bounds an object that is not detected by the gastroenterologist in a colonoscopy during normal use (does not include frames when a surgical tool or NBI is detected).
8
- . Demographics
| Table 2: Demographic and Baseline Information
Demographics (Total no. of videos = 79) | ||
---|---|---|
Sex | No. of Videos | |
Male | 34 | |
Female | 45 | |
Race | ||
White | 72 | |
Asian | 3 | |
Black or African American | 2 | |
Not Reported - Declined | 2 | |
Ethnicity | ||
Hispanic or Latino Heritage | 0 | |
Non-Hispanic or Latin Heritage | 78 | |
Declined | 1 | |
Endoscopic Processor | ||
EVIS EXERA III processor CV-190 | 79 |
Table 2: Demographic and Baseline Information
- Results ●
Table 3: Primary and Secondary Endpoints Results
| Object Level True Positive
Rate (TPR) | 97.87 (95% Cl: 94.96, 100.0) |
---|---|
Object Level False Positive | |
(FP) | 22.55 objects per a 15 minute |
interval (95% Cl: 18.954, 26.148) | |
Frame level performance | (79 videos / 94 polyps) |
True positive: 193,861 | |
True negative: 3,459,211 | |
False positive: 81,930 | |
False negative: 154,429 | |
True positive rate per frame | Mean: 55.66 % (95% CI: 55.50, |
55.83) | |
False positive rate per | |
frame | Mean: 2.31% (95% CI:2.16 - |
2.44) |
- Subgroup analysis of endpoints ●
Table 4: Subgroups analysis of endpoints
Total Videos = 79, Total Polyps = 94, Total Detected Polyps = 92
9
Polyp Detected | Total Polyps | Object TPR | 95 % Confidence Interval for Object TPR [low, high] | mean Object FP (15 min interval) | 95 % Confidence Interval for mean Object FP (15-min interval) [low, high] | ||
---|---|---|---|---|---|---|---|
Sex | |||||||
Male | 45 | 46 | 97.83% | [93.61, 100.0] | 25.66 | [19.387, 31.927] | |
Female | 47 | 48 | 97.92% | [93.88, 100.0] | 20.2 | [15.934, 24.4746] | |
Age | |||||||
40 years to 64 years | 60 | 61 | 98.36% | [95.17, 100.0] | 21.54 | [17.002, 26.081] | |
65 years and up | 32 | 33 | 96.97% | [91.12, 100.0] | 24.61 | [18.429, 30.788] | |
Size(mm) | |||||||
0 - 4 | 35 | 36 | 98.11% | [94.45, 100.0] | N/A | N/A | |
5 - 9 | 47 | 48 | 98.13% | [95.09, 100.0] | N/A | N/A | |
10+ | 10 | 10 | 100.00% | [100.0, 100.0] | N/A | N/A | |
Histology | |||||||
Adenoma | 50 | 50 | 100.00% | [100.0, 100.0] | N/A | N/A | |
SSLs | 16 | 16 | 100.00% | [100.0, 100.0] | N/A | N/A | |
Hyperplastic Polyp | 16 | 17 | 96.43% | [89.55, 100.0] | N/A | N/A | |
Inflammatory Polyp | 2 | 2 | 100.00% | [100.0, 100.0] | N/A | N/A | |
Not Histologically a Polyp | 5 | 6 | 90.91% | [79.79, 100.0] | N/A | N/A | |
Unknown | 3 | 3 | 100.00% | [100.0, 100.0] | N/A | N/A | |
Race | |||||||
Declined | 2 | 2 | 100.00% | [100.0, | |||
100.0] | 37.72 | [30.512, | |||||
47.189] | |||||||
White | 78 | 80 | 97.50% | [94.08, | |||
100.0] | 21.91 | [18.364, | |||||
25.46] | |||||||
Asian | 11 | 11 | 100.00% | [100.0, | |||
100.0] | 34.69 | [9.012, | |||||
78.4] | |||||||
Black or | |||||||
African | |||||||
American | 1 | 1 | 100.00% | [100.0, | |||
100.0] | 12.18 | [5.269, | |||||
19.622] |
10
- Marker Persistence is defined as the continuous uninterrupted detection of a target in time. Alternatively, it can be explained as the time (in milliseconds) from the first appearance of a marker on a polyp until the first disappearance of the marker for the same polyp.
A sensitivity analysis of marker persistence is performed with all 79 videos denoting the object-level True Positive Rate and False Positive event count (mean per 15 minute session) as a function of marker persistence.
The following graph demonstrates that markers with higher persistence have a higher likelihood to be true positives.
Image /page/10/Figure/5 description: The image is a plot titled "Polyp TPR based on marker persistence". The x-axis is labeled "Marker persistence less than (ms)" and ranges from 0 to 8000. The y-axis is labeled "Polyp TPR" and ranges from 0.0 to 0.8. The plot shows a curve that increases rapidly from 0 to around 0.3 between 0 and 500 on the x-axis, then gradually increases to around 0.9 between 500 and 5000 on the x-axis, and then plateaus.
Figure 1: Polyp TPR based on Marker Persistence
The plot below shows FP events (per 15 min) as a function of marker persistence. The graph demonstrates that the majority of FP events occur with markers that persist for 1 second or less.
11
Polyp FP Event count (mean per 15 min session) based on marker persistence
Image /page/11/Figure/2 description: The image is a plot of "Polyp FP Event count (mean per 15 min session)" versus "Marker persistence less than (ms)". The x-axis ranges from 0 to 8000, while the y-axis ranges from 0 to 20. The plot shows a curve that increases rapidly from 0 to around 2000 on the x-axis, and then gradually flattens out as the x-axis increases.
Figure 2: Polyp FP Event count (mean per 15 min session) based on Marker Persistence
- Marker Overlap: In order to not obstruct gastroenterologists' view of polyps, ● SKOUT™ artificially increases the size of the boxes output by its Al algorithm.
Various metrics are used to understand SKOUT™'s overlap performance with the ground truth reference standards.
Intersection over Union (IOU) is the ratio of area of intersection of the SKOUT™ signal with just the intersection of the two boxes over the total area of both boxes (Figure 03). This calculation means that when comparing SKOUT's performance (artificially increased bounding box), the IOU figure should be low.
Image /page/11/Figure/7 description: The image shows two sets of overlapping rectangles. In the top set, a blue rectangle overlaps a green rectangle, and a yellow rectangle overlaps both. In the bottom set, a blue rectangle overlaps a yellow rectangle, and the yellow rectangle overlaps a green rectangle.
Figure 03: IOU
Intersection over Ground Truth (IOGT) is the ratio of area of intersection of SKOUT™ signal with ground truth bounding box over the area of the ground truth box (Figure 04). An IOGT value of 1 indicates that the ground truth box is fully
12
engulfed by the SKOUT™ signal.
Image /page/12/Picture/2 description: The image contains two rectangles, one above a horizontal line and one below. The rectangle above the line is nested inside another rectangle. The outer rectangle is blue, and the inner rectangle has a green border and a yellow fill. The rectangle below the line has a green border and a yellow fill.
Figure 04: IOGT
Our results show that our IOU was 0.299. More importantly, our IOGT value was 1.0 which indicates that on a median basis, all polyps were engulfed by a SKOUT bounding box.
Table 05: Signal overlap analysis
| Mean, 95% confidence interval if IOU of
signal on 411 samples | 0.299, [0.289, 0.309] |
---|---|
Median IOGT of signal | 1.0 |
- Special Control Testing
- Pixel-level comparison of degradation of image quality due to the device: No visually . detectable differences between images were found with the introduction of the SKOUT™ system.
- Assessment of video delay due to marker annotation: 56.00ms (95% Cl: 50.54, . 61.46) and 3.25 (95% Cl: 2.93, 3.56) frame delay for Serial Digital Interface (SDI) and 62.33ms (95% Cl: 60.76, 63.90) and 3.74 (95% Cl: 3.65, 3.83) frame delay for Digital Visual Interface (DVI).
- Assessment of real-time endoscopic video delay due to the device; 56.67ms (95% . CI: 51.01, 62.33) and 3.28 (95% CI: 2.96, 3.62) frame delay for SDI and 60.67ms (95% Cl: 57.72, 63.61) and 3.64 (95% Cl: 3.46, 3.81) frame delay for DVI.
Human Factors
Human factors validation was performed following the FDA Guidance document Applying Human Factors and Usability Engineering to Medical Devices. Guidance for Industry and FDA Staff recommendations. The human factors validation demonstrated that the device functioned as intended, use-related risk has been mitigated, and the SKOUT™ system is safe for its intended use.
13
Clinical Testing
A multicenter, prospective, randomized controlled trial of the SKOUT™ system was conducted in the United States in order to evaluate the safety and efficacy of this device. The study design involved 2 arms in which adult patients undergoing either screening or surveillance colonoscopy procedures were randomized to either an Al-aided arm (standard colonoscopy with the use of the SKOUT™ system), or the control arm (standard colonoscopy without the use of the SKOUT™ system).
The aim of this study is to evaluate the clinical benefit and safety of using a computer-aided detection (CADe) device, the SKOUT™ system, in colonoscopy procedures with the indication of screening or surveillance.
Inclusion Criteria:
- Undergoing colonoscopy with screening or surveillance. ●
- Whose endoscopist is a participating provider. ●
- Who have given informed consent.
Exclusion Criteria:
- Have a history of inflammatory bowel disease.
- Have a history of familial adenomatous polyposis.
- Are under the age of 40.
- Have had a colonoscopy within the previous three (3) years.
- . Patients undergoing diagnostic colonoscopy with high risk indications including iron deficiency anemia, abnormal CT imaging, unexplained weight loss, Lynch Syndrome, blood in stool or FIT positive test.
- . Use anti-platelet agents or anticoagulants that preclude the removal of polyps during the procedure.
- Entered with poor bowel preparation (inadeguate for procedure as assessed by the Investigator).
Provider's Eligibility requirements:
- United States board-certified gastroenterologist.
- Has performed at least 1,000 colonoscopies. ●
- Has an ADR greater than or equal to 25%.
The co-primary endpoints of the study included a performance endpoint (adenomas per colonoscopy - APC) and a safety endpoint (positive percent agreement - PPA).
- APC: The total number of adenomas detected divided by the total number of colonoscopies. .
- PPA**: PPA is the fraction of adenomas, sessile serrated lesions, and hyperplastic polyps . of the proximal colon (caecum, ascending colon, hepatic flexure, and transverse colon) out of the total number of resections.
- PPA (or PPV): It is the fraction of adenomas, sessile serrated lesions, and large ● (>10mm) hyperplastic polyps of the proximal colon (caecum, ascending colon, hepatic flexure, and transverse colon) out of total number of resections.
Table 06:Demographics and Baseline Information
14
Control (N=677) | Treatment (N=682) | p-value | |
---|---|---|---|
Sex | 0.612 | ||
Male | 355 (52.4%) | 368 (54%) | |
Female | 322 (47.6%) | 314 (46%) | |
Age - Continuous | 0.159 | ||
Mean (SD) | 59.9 (8.8) | 60.6 (8.9) | |
Age - Categorical | 0.21 | ||
40 = 65 years | 208 (30.7%) | 240 (35.2%) | |
Race | 0.194* | ||
American Indian or Alaska Native | 2 (0.3%) | 0 (0%) | |
Asian | 21 (3.1%) | 18 (2.6%) | |
Black or African American | 36 (5.3%) | 47 (6.9%) | |
Native Hawaiian or Other | |||
Pacific-Islander | 3 (0.4%) | 0 (0%) | |
White | 563 (83.2%) | 567 (83.3%) | |
More Than One Race | 5 (0.7%) | 2 (0.3%) | |
Don't Know | 11 (1.6%) | 18 (2.6%) | |
Refused | 36 (5.3%) | 29 (4.3%) | |
Ethnicity | 0.596 | ||
Not Hispanic or Latino | 627 (92.6%) | 627 (91.9%) | |
Hispanic or Latino | 31 (4.6%) | 29 (4.3%) | |
Don't Know | 7 (1%) | 13 (1.9%) | |
Refused | 12 (1.8%) | 13 (1.9%) | |
Site of Procedure | >0.99 | ||
Concord Endoscopy Center | 206 (30.4%) | 210 (30.8%) | |
Mount Auburn Hospital | 168 (24.8%) | 163 (23.9%) | |
Massachusetts General Hospital | 59 (8.7%) | 62 (9.1%) | |
MNGI Digestive Health | 160 (23.6%) | 161 (23.6%) | |
Boston Medical Center | 84 (12.4%) | 86 (12.6%) | |
Years Since Last Colonscopy | >0.99 | ||
3 = 10 years | 129 (19.1%) | 127 (18.6%) | |
No Previous Colonoscopy | 216 (31.9%) | 215 (31.5%) |
- Fisher's Exact Test Used Instead of Chi-Square Test
A total of 1,359 patients were included in the modified Intention To Treat (mlTT) population for primary analysis, including 682 who received a colonoscopy with the SKOUT™ system and 677 who received a standard colonoscopy. The evaluation of our primary endpoints and secondary endpoint for the mITT population is summarized below:
Table 07: Primary and Secondary Endpoints results for mITT population
15
| | Control
(N=677) | Treatment
(N=682) | Difference
(Treatment-
Control) | 95% CI for
Difference | p-value |
|-------------------------------------------------------|--------------------|----------------------|---------------------------------------|--------------------------|---------|
| Adenomas Per
Colonoscopy* | 0.830 | 1.054 | 0.224 | (0.060, 0.382) | 0.002 |
| Positive Predictive
Value | 0.717 | 0.674 | -0.043 | (-0.094, 0.010) | = 10 mm | 112 (9.9%) | 77 (5.8%) | |
| Size Unmatched | 0 (0%) | 0 (0%) | |
| Polyp Morphology
Sessile | 937 (86.4%) | 1123 (86.6%) | 0.185 |
| Pedunculated | 87 (8%) | 82 (6.3%) | |
| Flat | 48 (4.4%) | 76 (5.9%) | |
| Not Available | 12 (1.1%) | 16 (1.2%) | |
| Polyp Histology
Hyperplastic | 205 (18.9%) | 280 (21.6%) | = 10 mm | 31 (7.1%) | 23 (4.4%) | |
| Size Unmatched | 0 (0%) | 0 (0%) | |
| Proximal Colon
Mean (SD) | 5.2 (3.2) | 5 (3.2) | 0.27 |
| Proximal Colon | | | = 10 mm | 75 (11.5%) | 52 (6.7%) | |
| Size Unmatched | 0 (0%) | 0 (0%) | |
Table 09: Relationship between size of resected polyps and their location
17
Table 10: Relationship between size of resected polyps across histology:
Control (N=986) | Treatment (N=1168) | p-value | |
---|---|---|---|
Adenoma | |||
Size: = 10 mm | 63 (6.389%) | 44 (3.767%) | 0.007 |
Size: All Sizes | 585 (59.331%) | 747 (63.955%) | 0.031 |
SSL | |||
Size: = 10 mm | 36 (3.651%) | 26 (2.226%) | 0.066 |
Size: All Sizes | 196 (19.878%) | 141 (12.072%) | 0.99 |
Size: >= 10 mm | 3 (0.304%) | 3 (0.257%) | >0.99 |
Size: All Sizes | 205 (20.791%) | 280 (23.973%) | 0.087 |
The results of the clinical performance as documented in the pivotal clinical study show a statistically significant increase in APC, and PPA fell statistically within the prespecified noninferiority margin , demonstrating that the performance of the SKOUT™ system achieved benchmark expectations and a safety and effectiveness profile comparable to the predicate device.
Conclusions
18
The SKOUT™ system has the same intended uses and similar indications, technological characteristics, and principles of operation as its predicate device. The minor differences in indications do not alter the intended use of the device and do not affect its safety and effectiveness when used as labeled. In addition, the minor technological differences between the SKOUT™ system and its predicate device do not raise different issues of safety or effectiveness. Performance data from this study demonstrate that the SKOUT™ system is as safe and effective as the predicate device. Thus, the SKOUT™ system can be considered substantially equivalent.