Search Results

Overjet Caries Assist-Pediatric (OCA-Ped) is a radiological, automated, concurrent-read, computer-assisted detection (CADe) software intended to aid in the detection and segmentation of caries on bitewing and periapical radiographs. The device provides additional information for the clinician to use in their diagnosis of a tooth surface suspected of being carious. The device is not intended as a replacement for a complete dentist's review or their clinical judgment that takes into account other relevant information from the image, patient history, or actual in vivo clinical assessment.

The intended patient population of the device is patients aged 4-11 years old that have primary or permanent teeth (primary or mixed dentition) and are indicated for dental radiographs.

Overjet Caries Assist-Pediatric (OCA-Ped) is a radiological, automated, concurrent-read, computer-assisted detection (CADe) software intended to aid in the detection and segmentation of caries on bitewing and periapical radiographs. The device provides additional information for the dentist to use in their diagnosis of a tooth surface suspected of being carious. The device is not intended as a replacement for a complete dentist's review or their clinical judgment that takes into account other relevant information from the image, patient history, or actual in vivo clinical assessment.

OCA-Ped is a software-only device which operates in three layers: a Network Layer, a Presentation Layer, and a Decision Layer. Images are pulled in from a clinic/dental office, and the Machine Learning model creates predictions in the Decision Laver and results are pushed to the dashboard, which are in the Presentation Layer.

The provided document describes the Overjet Caries Assist-Pediatric (OCA-Ped) device, a computer-assisted detection (CADe) software intended to aid in the detection and segmentation of caries on bitewing and periapical radiographs for patients aged 4-11 years old.

Here's the breakdown of the acceptance criteria and study details:

1. Table of Acceptance Criteria and Reported Device Performance

The document does not explicitly state "acceptance criteria" in a separate section with specific numerical thresholds. However, it presents the performance metrics from the MRMC reader study and standalone testing, which can be interpreted as demonstrating the device's acceptable performance.

2. Sample Size Used for the Test Set and Data Provenance

• MRMC Test Set: 636 images, each from a unique patient.
• Standalone Test Set: 1190 bitewing and periapical images.
• Data Provenance: "Images were obtained from male and female patients aged 4-11 years." For the Standalone testing, "images were obtained from male and female patients, from a range of distinctly different geographic regions." The document does not specify if the data was retrospective or prospective, nor does it explicitly mention the country of origin, beyond "US licensed dentists" participating in the MRMC study. Given that "US licensed dentists" participated and "distinctly different geographic regions" are mentioned for standalone, it's implied to be US-centric.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts

• Number of Experts: 3
• Qualifications of Experts: General dentists. No specific experience level (e.g., "10 years of experience") is provided, only "3 general dentists."

4. Adjudication Method for the Test Set

• Adjudication Method: Consensus ground truth established by 3 general dentists. The exact process of reaching consensus (e.g., silent read, discussion, majority vote) is not detailed, but it implies agreement among the three experts.

5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study Was Done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• Yes, an MRMC comparative effectiveness study was done.
• Effect Size of Improvement with AI Assistance:
  • AUC of wAFROC: Averaged across all readers, there was a 7.5% improvement (95% CI: 0.062, 0.088) in assisted readers compared to unassisted readers.
  • Tooth-level Sensitivity: Averaged across all readers, sensitivity increased by 11.8% (95% CI: 0.102, 0.137) when compared to unassisted readers.
  • Tooth-level Specificity: A slight decrease of -0.011 (95% CI: -0.015, -0.008) between assisted and unassisted readers.

6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) Was Done

• Yes, standalone performance testing was done.
  • Tooth-level standalone sensitivity was 83.9% (95% CI: 0.816, 0.860).
  • Tooth-level standalone specificity was 97.5% (95% CI: 0.971, 0.979).
  • Standalone Dice was a mean of 79.0% (95% CI: 0.784, 0.797).

7. The Type of Ground Truth Used

• Ground Truth Type: Expert consensus. Specifically, "consensus reference standard established by 3 general dentists."

8. The Sample Size for the Training Set

• The document does not provide the sample size for the training set. It only describes the test sets used for MRMC and standalone performance evaluation.

9. How the Ground Truth for the Training Set Was Established

• The document does not describe how the ground truth for the training set was established. It only details the ground truth establishment for the test sets.

Ask a specific question about this device

Overjet Caries Assist (OCA) is a radiological, automated, concurrent-read, computer-assisted detection (CADe) software intended to aid in the detection and segmentation of caries on bitewing and periapical radiographs. The device provides additional information for the dentist to use in their diagnosis of a tooth surface suspected of being carious. The device is not intended as a replacement for a complete dentist's review or their clinical judgment that takes into account other relevant information from the image, patient history, or actual in vivo clinical assessment.

Overjet Caries Assist (OCA) is a radiological, automated, concurrent-read, computer-assisted detection (CADe) software intended to aid in the detection and segmentation of caries on bitewing and periapical radiographs. The device provides additional information for the dentist to use in their diagnosis of a tooth surface suspected of being carious. The device is not intended as a replacement for a complete dentist's review or their clinical judgment that takes into account other relevant information from the image, patient history, or actual in vivo clinical assessment.

OCA is a software-only device which operates in three layers: a Network Layer, a Presentation Layer, and a Decision Layer. Images are pulled in from a clinic/dental office, and the Machine Learning model creates predictions in the Decision Layer and results are pushed to the dashboard, which are in the Presentation Layer.

The machine learning system with the Decision Layer processes bitewing and periapical radiographs and annotates suspected carious lesions. It is comprised of four modules:

• Image Preprocessor Module
• Tooth Number Assignment Module
• Caries Module
• Post Processing

This document describes the Overjet Caries Assist (OCA) device, a computer-assisted detection (CADe) software intended to aid dentists in the detection and segmentation of caries on bitewing and periapical radiographs.

Here's an analysis of the acceptance criteria and the study that proves the device meets them:

1. Table of Acceptance Criteria and Reported Device Performance:

The document doesn't explicitly state numerical acceptance criteria for sensitivity or specificity in a "table" format as initial goals. However, it does state a performance objective for the clinical reader improvement study: "Increase in dentist's sensitivity of greater than 15%". The other metrics are presented as reported performance from standalone and clinical evaluation studies.

2. Sample Size for Test Set and Data Provenance:

• Standalone Testing:

  • Sample Size: 1,293 Bitewing images (27,920 surfaces) and 1,314 Periapical images (16,254 surfaces).
  • Data Provenance: Not explicitly stated regarding country of origin. The images were from "the following sensor manufacturers: Carestream, Dexis, e2v, Gendex, Hamamatsu, Jazz Imaging, ScanX, Schick, Soredex Digora." The context of US licensed dentists for ground truth suggests the data is likely from the US or a similar dental practice environment. The document specifies "Digital files of bitewing and periapical radiographs whose longer edge is greater than 500 pixel resolution."
  • Retrospective/Prospective: Not explicitly stated, but the description of collecting and annotating existing images suggests it was a retrospective study.

• Clinical Evaluation (Reader Improvement Study):

  • Sample Size: 330 bitewing images (94 containing caries / 236 without caries) and 330 periapical images (94 containing caries / 236 without caries).
  • Data Provenance: Not explicitly stated regarding country of origin, but the "US licensed dentists" implies the study was conducted within the US.
  • Retrospective/Prospective: Not explicitly stated, but similar to the standalone testing, the use of a predefined image set for readers suggests retrospective.

3. Number of Experts and Qualifications for Ground Truth:

• Standalone Testing: Ground truth established by consensus of three US licensed dentists. Non-consensus labels were adjudicated by an oral radiologist. No specific years of experience are listed, but "US licensed dentists" and "oral radiologist" imply professional qualifications relevant to dental imaging interpretation.
• Clinical Evaluation (Reader Improvement Study): Ground truth established by consensus of three US licensed dentists. Non-consensus labels were adjudicated by an oral radiologist. (Same as standalone testing).

4. Adjudication Method for the Test Set:

• Standalone Testing & Clinical Evaluation: The method described is consensus of three experts, with a fourth expert (oral radiologist) adjudicating non-consensus cases. This can be described as a "3+1 adjudication" method, where the "1" is specifically for tie-breaking or resolving disagreements among the initial three.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:

• Yes, a MRMC study was done. This is detailed under "Clinical Evaluation - Reader Improvement."
• Effect Size / Reader Improvement:
  • Bitewing Images:
    • Overall reader sensitivity improved from 64.6% (unassisted) to 78.5% (assisted). This is an absolute increase of 13.9 percentage points.
    • Weighted AFROC AUC increased from 0.729 (unassisted) to 0.785 (assisted), an increase of 0.055. This increase was statistically significant (p < 0.001).
    • Mean Dice scores increased from 0.67 (unassisted) to 0.76 (assisted) for primary caries. Similar increases were observed for other caries types.
  • Periapical Images:
    • Overall reader sensitivity improved from 65.6% (unassisted) to 79.0% (assisted). This is an absolute increase of 13.4 percentage points.
    • Weighted AFROC AUC increased from 0.799 (unassisted) to 0.848 (assisted), an increase of 0.050. This increase was statistically significant (p < 0.001).
    • Mean Dice scores increased from 0.73 (unassisted) to 0.80 (assisted) for primary caries. Similar increases were observed for other caries types.

The document concludes that the increase in wAFROC numbers clearly demonstrates improvement in caries detection by dentists when aided by Overjet Caries Assist, aligned with increases in sensitivity and minimal decrease in specificity.

6. Standalone (Algorithm Only) Performance Study:

• Yes, a standalone performance study was done. This is detailed under "Standalone Testing." The results for the algorithm's sensitivity, specificity, and Dice scores are provided for both bitewing and periapical images.

7. Type of Ground Truth Used:

• Expert Consensus. For both standalone and clinical evaluation studies, the ground truth was "established by consensus of labels of three US licensed dentists, and non-consensus labels were adjudicated by an oral radiologist."

8. Sample Size for the Training Set:

• The document does not explicitly state the sample size for the training set. It describes the "Machine Learning model" and its components but does not provide details about the data used for training.

9. How the Ground Truth for the Training Set Was Established:

• The document does not explicitly state how the ground truth for the training set was established. It only details the ground truth establishment for the test sets used in standalone and clinical performance evaluations.

Ask a specific question about this device

The Overjet Caries Assist (OCA) is a radiological, automated, concurrent read, computer-assisted detection software intended to aid in the detection and segmentation of caries on bitewing radiographs. The device provides additional information for the dentist to use in their diagnosis of a tooth surface suspected of being carious. The device is not intended as a replacement for a complete dentist's review or that takes into account other relevant information from the image, patient history, and actual in vivo clinical assessment.

Overjet Caries Assist (OCA) is a radiological automated concurrent read computer-assisted detection (CAD) software intended to aid in the detection and segmentation of caries on bitewing radiographs. The device provides additional information for the clinician to use in their diagnosis of a tooth surface suspected of being carious. The device is not intended as a replacement for a complete clinician's review or their clinical judgment that takes into account other relevant information from the image or patient history.

OCA is a software-only device which operates in three layers - a Network Layer, a Presentation Layer, and a Decision Layer (as shown in the data flow diagram below). Images are pulled in from a clinic/dental office, and the Machine Learning model creates predictions in the Decision Layer and results are pushed to the dashboard, which are in the Presentation Layer.

The Machine Learning System within the Decision Layer processes bitewing radiographs and annotates suspected carious lesions. It is comprised of four modules:

• Image Classifier The model evaluates the incoming radiograph and predicts the ● image type between Bitewing and Periapical Radiograph. This classification is used to support the data flow of the incoming radiograph. As part of the classification of the image type any non-radiographs are classified as "junk" and not processed. These include patient charting information, or other non-bitewing or periapical radiographs. OCA shares classifier and Tooth Number modules with the Overjet Dental Assist product cleared under K210187.
• . Tooth Number Assignment module - This module analyzes the processed image and determines what tooth numbers are present and provides a pixel wise segmentation mask for each tooth number.
• Caries module - This module outputs a pixel wise segmentation mask of all carious lesions using an ensemble of 3 U-Net based models. The shape and location of every carious lesion is contained in this mask as the carious lesions' predictions.
• Post Processing The overlap of tooth masks from the Tooth Number . Assignment Module and carious lesions from the Caries Module is used to assign specific carious lesions to a specific tooth. The Image Post Processor module annotates the original radiograph with the carious lesions' predictions.

Acceptance Criteria and Device Performance for Overjet Caries Assist

The Overjet Caries Assist (OCA) is a radiological, automated, concurrent read, computer-assisted detection software intended to aid in the detection and segmentation of caries on bitewing radiographs. The device's performance was evaluated through standalone testing of the AI algorithm and a clinical reader improvement study.

1. Table of Acceptance Criteria and Reported Device Performance

2. Sample Size and Data Provenance for the Test Set

• Sample Size for Test Set: 352 bitewing radiographs (104 containing caries / 248 without caries).
• Data Provenance: Not explicitly stated in the provided text (e.g., country of origin). However, given the context of U.S. FDA clearance and the use of US-licensed dentists, it is likely that the data is either from the US or representative of populations seen in the US. The type of data is retrospective, as existing radiographs were used.

3. Number of Experts and Qualifications for Ground Truth

• Number of Experts: Three US-licensed dentists for initial consensus, and one Dental Radiologist for adjudication of non-consensus labels.
• Qualifications of Experts: All experts were US-licensed dentists. The adjudicating expert was specifically a Dental Radiologist. No further details on years of experience were provided.

4. Adjudication Method for the Test Set

The adjudication method used was a "3+1" approach. Ground truth was initially established by the consensus labels of three US-licensed dentists. Any non-consensus labels were then adjudicated by a Dental Radiologist.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

Yes, an MRMC comparative effectiveness study was conducted.

• Effect Size of Human Readers Improvement with AI vs. without AI assistance:
  • Sensitivity: Overall reader sensitivity improved by 18.3 percentage points (from 57.9% unassisted to 76.2% assisted).
    • For primary caries, sensitivity improved by 18.9 percentage points (60.5% unassisted to 79.4% assisted).
    • For secondary caries, sensitivity improved by 13.2 percentage points (49.8% unassisted to 63.0% assisted).
  • Specificity: Overall reader specificity decreased slightly by 0.9 percentage points (from 99.3% unassisted to 98.4% assisted).
  • AFROC AUC: The average AUC for all readers increased by 0.057 (from 0.593 unassisted to 0.649 assisted). This increase was statistically significant (p < 0.001).
  • Average Dice Scores for Segmentation:
    • Primary caries: Mean Dice scores improved from 0.67 unassisted to 0.69 assisted.
    • Secondary caries: Mean Dice scores improved from 0.65 unassisted to 0.74 assisted. (Note: These segmentation improvements were not statistically significant).

6. Standalone (Algorithm Only) Performance

Yes, standalone performance (algorithm only without human-in-the-loop) was conducted.

• Overall standalone sensitivity: 72.0% (95% CI: 62.9%, 81.1%)
• Overall standalone specificity: 98.1% (95% CI: 97.7%, 98.5%)
• Lesion Segmentation (Dice Score):
  • Primary caries: Mean Dice score of 0.69
  • Secondary caries: Mean Dice score of 0.75

7. Type of Ground Truth Used

The ground truth used was expert consensus complemented by expert adjudication. Specifically, a consensus of three US-licensed dentists, with non-consensus cases adjudicated by a Dental Radiologist.

8. Sample Size for the Training Set

The sample size for the training set is not provided in the excerpt. The document only mentions "training data" in the context of the algorithm's capability to learn during its operation, but not a specific size for its initial training.

9. How the Ground Truth for the Training Set was Established

The method for establishing ground truth for the training set is not explicitly detailed in the provided text. It generally states that the algorithm "has been trained," but does not provide information on how the ground truth for that training was established.

Ask a specific question about this device