Search Results

Deep Capsule® is an artificial intelligence (AI) assisted reading tool designed to aid small bowel capsule endoscopy reviewers in decreasing the time to review capsule endoscopy images for adult patients in whom the capsule endoscopy images were obtained for suspected small bowel bleeding. The clinician is responsible for conducting their own assessment of the findings of the AI-assisted reading through review of the entire video, as clinically appropriate. This tool is not intended to replace clinical decision-making.

Deep Capsule® is an artificial intelligence (AI) assisted reading tool designed to aid small bowel capsule endoscopy reviewers (SBCapER) in decreasing the time to review capsule endoscopy images for adult patients in whom the capsule endoscopy images were obtained for suspected small bowel bleeding.

Deep Capsule® is capable of detecting small bowel lesions, without differentiating them. The detection of lesions by Deep Capsule® is insufficient to achieve a direct diagnosis, which is dependent on the clinical integration of the different findings. Deep Capsule® should be integrated into this multifactorial and complex context, both in the diagnostic workup or in the patient follow-up.

In summary, although the Deep Capsule® detects small bowel lesions, it acts only as a support to the clinical decision. The ultimate diagnosis will always be given by the small bowel capsule endoscopy reviewers ("human in the loop").

It is important to note that Deep Capsule® also provides in the findings count as a non-AI software output, and user inputs that are inserted and edited by SBCapERs, including CapE device model, patient exam priority category, procedure date, responsible physician, clinical indication and medical notes.

Deep Capsule® software includes a main algorithm as illustrated in Figure 1:

• Small bowel lesion detection, which includes the small bowel lesion image analysis algorithm designed to automatically identify and localize potential small bowel lesions in capsule endoscopy images. Suspected findings are selected automatically from the video frames as illustrated in Figure 1 below, with the active video displayed on the left side of the user interface. On the right side, AI-selected frames are presented as a structured gallery of thumbnails, allowing the reviewer to efficiently navigate through suspected findings. The interface also provides exam details, frame indexing, clinical indication, findings count, and medical notes to support structured review.

Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided FDA 510(k) clearance letter for Deep Capsule®:

1. Table of Acceptance Criteria and Reported Device Performance

The FDA clearance letter does not explicitly define "acceptance criteria" for the device's performance in a table format. However, it presents the results of its performance studies against benchmarks like "Expert Reading" (standalone algorithm testing) and "Expert Board (Ground Truth)" (clinical validation).

Based on the provided data, we can infer the performance metrics deemed acceptable:

2. Sample Size for the Test Set and Data Provenance

• Standalone Algorithm Testing Test Set:

  • Patients: 272 patients (from a total dataset of 1,133 patients).
  • Images: 101,802 images (from a total dataset of 321,357 images).
  • Data Provenance: Retrospectively collected from two clinical institutions in Portugal (Centro Hospitalar Universitário São João, Porto, Portugal; and ManopH – Laboratório de Endoscopia e Motilidade Digestiva, Lda., Vila Nova de Gaia, Portugal).

• Clinical Validation Study Test Set:

  • Patients: 330 patients.
  • Data Provenance: Retrospectively collected between January 2021 and April 2024 from seven independent clinical centers across four countries: Portugal, Spain, Brazil, and the United States. No overlap existed between training and clinical validation datasets, and clinical validation sites were independent from the institutions used for model development.

3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications

• Standalone Algorithm Testing (Internal Test Set): The document states that images were "expert-labeled images." It does not specify the exact number of experts, their qualifications, or the adjudication method for this initial labeling of the training/internal test dataset.
• Clinical Validation Study (Ground Truth):
  • Adjudicators: 5 independent expert adjudicators.
  • Qualifications: "board-certified gastroenterologists with extensive capsule endoscopy experience."
  • Clinical Readers: 15 board-certified gastroenterologists.
  • Qualifications mentioned for Clinical Readers: "certified specialists in gastroenterology and/or digestive endoscopy with a minimum of 5 years of post-fellowship clinical experience." Readers were located across the United States, Portugal, Spain, and Brazil. There was no overlap between clinical readers and adjudicators.
  • Image-level ground truth: "established by expert reviewers blinded to AI outputs." (Implies the 5 expert adjudicators).

4. Adjudication Method for the Test Set

• Standalone Algorithm Testing (for initial expert-labeled images - training/internal test): Not explicitly stated, but implies expert consensus or single expert labeling for the "expert-labeled images."
• Clinical Validation Study (for Expert Board Ground Truth): "Ground truth was established through independent manual review of full capsule endoscopy videos by two experienced gastroenterologists, with adjudication by an expert board in cases of disagreement." This is a 2+1 adjudication model.

5. Multi Reader Multi Case (MRMC) Comparative Effectiveness Study

Yes, an MRMC comparative effectiveness study was done.

• Effect Size of Human Readers Improving with AI vs. without AI assistance:
  • Diagnostic Yield: AI-aided CapE showed a Diagnostic Yield (DY) of 0.961 compared to Standard-of-Care (SoC) at 0.761. The difference in DY between AI-aided and SoC was 0.200 (0.149 - 0.251), with non-inferiority established (p < 0.001). This indicates a substantial improvement in diagnostic yield when readers are assisted by AI.
  • Sensitivity (Per-Patient): AI-aided CapE had a sensitivity of 0.972 compared to SoC at 0.762. The improvement in sensitivity for AI-aided reading over unaided (SoC) was 21.0% (13.9 - 27.6%).
  • Specificity (Per-Patient): AI-aided CapE had a specificity of 0.125 compared to SoC at 0.250. The difference in specificity was -12.5% (-34.7 - 11.9%). It decreased with AI assistance (due to the AI prioritizing sensitivity and detecting more potential lesions, leading to more "false positives" that clinicians must then review for true positivity).
  • Mean Reading Time: "mean reading time was significantly reduced with AI+Physician reading, as compared to standard reading." (Specific quantification of reduction not provided in the excerpt).

6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study

Yes, a standalone algorithm performance study was done. This is referred to as "Standalone Algorithm Testing" or "PERFORMANCE TESTING – BENCH – STANDALONE PERFORMANCE."

• Patient-Level Performance: Sensitivity 95.8%, Specificity 75%.
• Image-Level Performance: Sensitivity 92.1%, Specificity 88.0%.

7. Type of Ground Truth Used

• Standalone Algorithm Testing: "Expert-labeled images."
• Clinical Validation Study: Expert board reference standard involving "independent manual review of full capsule endoscopy videos by two experienced gastroenterologists, with adjudication by an expert board in cases of disagreement."

8. The Sample Size for the Training Set

• Patients: 861 patients (from a total of 1,133 patients).
• Images: 219,555 images (from a total of 321,357 images).

9. How the Ground Truth for the Training Set Was Established

The document states: "Images included in the dataset [which includes the training set] were labeled by expert readers following a structured annotation process." It does not provide further details on the number of experts, their qualifications, or the exact adjudication method during the training data labeling phase.

Ask a specific question about this device

NaviCam ProScan is an artificial intelligence (AI) assisted reading tool designed to aid small bowel capsule endoscopy reviewers in decreasing the time to review capsule endoscopy images for adult patients in whom the capsule endoscopy images were obtained for suspected small bowel bleeding. The clinician is responsible for conducting their own assessment of the findings of the AI-assisted reading through review of the entire video, as clinically appropriate. ProScan also assists small bowel capsule endoscopy reviewers in identifying the digestive tract location (oral cavity and beyond, esophagus, stomach, small bowel) of the image in adults. This tool is not intended to replace clinical decision making.

The NaviCam ProScan is artificial intelligence software that has been trained to process capsule endoscopy images of the small bowel acquired by the NaviCam Small Bowel Capsule Endoscopy System to recognize the various sections of the digestive tract and to recognize and mark images containing suspected abnormal lesions.

NaviCam ProScan is intended to be used as an adjunct to the ESView software of the NaviCam Small Bowel Capsule Endoscopy System (both cleared in K221590) and is not intended to replace gastroenterologist assessment or histopathological sampling.

NaviCam ProScan does not make any modification or alteration to the original capsule endoscopy video. It only overlays graphical markers and includes an option to only display these identified images. The whole small bowel capsule endoscopy video and highlighted regions still must be independently assessed by the clinician and appropriate actions taken according to standard clinical practice.

The NaviCam ProScan software includes two main algorithms, as illustrated in Figure 1 below:

• Digestive tract site recognition, which includes an image analysis algorithm and site segmentation algorithm to determine: oral and beyond, esophagus, stomach, and small bowel. Tract site is displayed as a color code on the video timeline with descriptions on the indicators at the bottom of the software user interface.
• Small bowel lesion recognition, which includes the small bowel lesion image analysis algorithm with lesion region localization. Potential lesions are marked with a bounding box as illustrated in Figure 2 below, with the active video played at the top section of the figure, and ProScan-identified images in the lower section, which includes images with suspected lesions and individual images marking the transition in the digestive tract. The algorithm is functional only on those sections of the GI tract that were identified as "small bowel" by the digestive tract site recognition software function.

Here's a detailed breakdown of the acceptance criteria and the studies proving the device meets them, based on the provided text:

Acceptance Criteria and Device Performance

Lesion Detection - Standalone Algorithm Performance (Image-Level)

Tract Site Recognition - Standalone Algorithm Performance (Image-Level)

Clinical Performance (AI+Physician vs. Standard Reading)

Study Details and Provenance

2. Sample Sizes and Data Provenance

Standalone Algorithm Testing (Lesion Detection)

• Test Set Sample Size: 218 patients
• Data Provenance: Obtained from 8 clinical institutions in China. The study was retrospective.

Standalone Algorithm Testing (Tract Site Recognition)

• Test Set Sample Size: 424 patients
• Data Provenance: Obtained from 8 clinical institutions in China. The study was retrospective.

Clinical Study (ARTIC Study)

• Test Set Sample Size: 133 patients (from an initial enrollment of 137).
• Data Provenance: Patients enrolled prospectively from 7 European centers (Italy, France, Germany, Hungary, Spain, Sweden, and UK) from February 2021 to January 2022.

3. Number of Experts and Qualifications for Ground Truth

Standalone Algorithm Testing (Lesion Detection & Tract Site Recognition)

• Number of Experts: Initially three gastroenterologists for pre-annotation, followed by two arbitration experts for review and modification. A total of five experts were involved in establishing the ground truth when including the arbitration experts.
• Qualifications: "Gastroenterologists" are explicitly stated. No specific experience level (e.g., years of experience) is provided for these experts in the available text.

Clinical Study (ARTIC Study)

• Number of Experts: An expert board consisting of 5 of the original 22 clinician readers was used to establish ground truth.
• Qualifications: The original 22 clinician readers "had capsule endoscopy experience of over 500 readings." It can be inferred that the 5 experts on the expert board had similar or higher qualifications.

4. Adjudication Method

Standalone Algorithm Testing (Lesion Detection & Tract Site Recognition)

• Method: Initial annotations by three gastroenterologists. "The computer automatically determines consistency and merges the classification results while preserving differing opinions." If consistency was less than a cutoff value (specifically "less than 3" for lesion detection, implying inconsistency among the 3 initial annotators), two arbitration experts independently review and modify the results. In difficult cases, "collective discussion and confirmation" were conducted by the adjudication experts. This aligns with a 3+2 adjudication model or a similar consensus-based approach with arbitration.

Clinical Study (ARTIC Study)

• Method: An expert board was used to "adjudicate the findings in case of disagreement" between standard readings and AI+Physician readings. Discordant cases were "re-evaluated and eventually reclassified during the adjudication phase." This suggests a consensus-based adjudication by the expert board. The exact protocol (e.g., how disagreements within the expert board were resolved) is not explicitly detailed, but it functions as the final ground truth determination.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Yes, an MRMC comparative effectiveness study was conducted (the ARTIC study).
• Effect Size of Human Readers' Improvement with AI vs. without AI Assistance:
  • Diagnostic Yield: AI-assisted reading (AI+Physician) achieved a diagnostic yield of 73.7% compared to 62.4% for standard reading (without AI), showing an absolute improvement of 11.3 percentage points. This improvement was statistically significant (p=0.015).
  • Reading Time: Mean reading time with AI assistance was 3 minutes 50 seconds, significantly faster than 33 minutes 42 seconds for standard reading. This represents a reduction of approximately 88.5% in reading time.

6. Standalone Performance (Algorithm Only without Human-in-the-Loop)

• Yes, standalone performance was done for both the lesion detection function and the tract site recognition function.

  • Lesion Detection (Standalone):
    • Patient-level sensitivity: 98%
    • Patient-level specificity: 37%
    • Image-level sensitivity: 95.05%
    • Image-level specificity: 97.54%
  • Tract Site Recognition (Standalone):
    • Sensitivity and specificity values for each anatomical site were all above 98%.

• Important Caveat: The regulatory information states, "In the clinical study of the device, performance (sensitivity and specificity) of the device in the absence of clinician input was not evaluated. Therefore, the AI standalone performance in the clinical study of NaviCam ProScan has not been established." This highlights a distinction between the "standalone algorithm testing" reported in detail and the performance within the clinical use context (i.e., the AI output before a clinician potentially overrides it). The clinical study, ARTIC, primarily evaluates "AI+Physician" performance. The document explicitly notes that the number of false positive predictions from the AI software (in the absence of physician input) in the ARTIC study is unknown.

7. Type of Ground Truth Used

• Standalone Algorithm Testing: Expert consensus (multiple gastroenterologists with arbitration) on individual images and patient cases.
• Clinical Study (ARTIC Study): Expert board reading and adjudication (5 experienced readers) of videos. This essentially serves as an expert consensus ground truth for the clinical effectiveness study.

8. Sample Size for the Training Set

Lesion Detection Function:

• Training Set Sample Size: 1,476 patients (from a dataset of 2,642 patients).

Tract Site Recognition Function:

• Training Set Sample Size: 1,386 patients (from a dataset of 2,642 patients).

9. How Ground Truth for the Training Set Was Established

The ground truth for the training set was established using a multi-expert annotation process:

Lesion Detection Function:

• Pre-Annotation: Full videos were randomly assigned to three gastroenterologists who annotated positive and negative lesion image segments.
• Annotation (Truthing): The sampled image dataset was annotated by the same three gastroenterologists using software. The computer checked for consistency and merged results. For inconsistencies (cutoff value < 3 for consistency), two arbitration experts independently reviewed and modified the classifications, correcting missed diagnoses/misdiagnoses. Difficult questions were resolved through collective discussion and confirmation by the arbitration experts.

Tract Site Recognition Function:

• Pre-Annotation: Full video data was randomly assigned to three experts who marked the boundary positions of each site (Oral Cavity and beyond, Esophagus, Stomach, Small Bowel) within each video.
• Annotation (Truthing): The sampled image dataset was annotated by three gastroenterologists in a blinded manner to classify the four sites. The computer determined consistency and merged results. For inconsistencies (cutoff value < 3 for consistency), two adjudication experts independently reviewed and modified the classifications. Difficult cases involved collective discussions and confirmations by the adjudication experts.

Ask a specific question about this device