Search Results
Found 3 results
510(k) Data Aggregation
(103 days)
Overjet Caries Assist-Pediatric
Overjet Caries Assist-Pediatric (OCA-Ped) is a radiological, automated, concurrent-read, computer-assisted detection (CADe) software intended to aid in the detection and segmentation of caries on bitewing and periapical radiographs. The device provides additional information for the clinician to use in their diagnosis of a tooth surface suspected of being carious. The device is not intended as a replacement for a complete dentist's review or their clinical judgment that takes into account other relevant information from the image, patient history, or actual in vivo clinical assessment.
The intended patient population of the device is patients aged 4-11 years old that have primary or permanent teeth (primary or mixed dentition) and are indicated for dental radiographs.
Overjet Caries Assist-Pediatric (OCA-Ped) is a radiological, automated, concurrent-read, computer-assisted detection (CADe) software intended to aid in the detection and segmentation of caries on bitewing and periapical radiographs. The device provides additional information for the dentist to use in their diagnosis of a tooth surface suspected of being carious. The device is not intended as a replacement for a complete dentist's review or their clinical judgment that takes into account other relevant information from the image, patient history, or actual in vivo clinical assessment.
OCA-Ped is a software-only device which operates in three layers: a Network Layer, a Presentation Layer, and a Decision Layer. Images are pulled in from a clinic/dental office, and the Machine Learning model creates predictions in the Decision Laver and results are pushed to the dashboard, which are in the Presentation Layer.
The provided document describes the Overjet Caries Assist-Pediatric (OCA-Ped) device, a computer-assisted detection (CADe) software intended to aid in the detection and segmentation of caries on bitewing and periapical radiographs for patients aged 4-11 years old.
Here's the breakdown of the acceptance criteria and study details:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state "acceptance criteria" in a separate section with specific numerical thresholds. However, it presents the performance metrics from the MRMC reader study and standalone testing, which can be interpreted as demonstrating the device's acceptable performance.
| Performance Metric | Acceptance Criteria (Implicit) | Reported Device Performance |
|---|---|---|
| MRMC Reader Study - Aided vs. Unaided | Improvement in diagnostic accuracy | |
| AUC of wAFROC (improvement) | (Not explicitly defined, but positive) | 7.5% improvement (95% CI: 0.062, 0.088) |
| Tooth-level Sensitivity (improvement) | (Not explicitly defined, but positive) | 11.8% improvement (95% CI: 0.102, 0.137) |
| Tooth-level Specificity (change) | (Not explicitly defined, but minimal decrease) | -0.011 (95% CI: -0.015, -0.008) |
| Standalone Performance | Sufficient diagnostic accuracy | |
| Tooth-level Sensitivity | (Not explicitly defined) | 83.9% (95% CI: 0.816, 0.860) |
| Tooth-level Specificity | (Not explicitly defined) | 97.5% (95% CI: 0.971, 0.979) |
| Standalone Dice | (Not explicitly defined) | 79.0% (95% CI: 0.784, 0.797) |
2. Sample Size Used for the Test Set and Data Provenance
- MRMC Test Set: 636 images, each from a unique patient.
- Standalone Test Set: 1190 bitewing and periapical images.
- Data Provenance: "Images were obtained from male and female patients aged 4-11 years." For the Standalone testing, "images were obtained from male and female patients, from a range of distinctly different geographic regions." The document does not specify if the data was retrospective or prospective, nor does it explicitly mention the country of origin, beyond "US licensed dentists" participating in the MRMC study. Given that "US licensed dentists" participated and "distinctly different geographic regions" are mentioned for standalone, it's implied to be US-centric.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
- Number of Experts: 3
- Qualifications of Experts: General dentists. No specific experience level (e.g., "10 years of experience") is provided, only "3 general dentists."
4. Adjudication Method for the Test Set
- Adjudication Method: Consensus ground truth established by 3 general dentists. The exact process of reaching consensus (e.g., silent read, discussion, majority vote) is not detailed, but it implies agreement among the three experts.
5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study Was Done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
- Yes, an MRMC comparative effectiveness study was done.
- Effect Size of Improvement with AI Assistance:
- AUC of wAFROC: Averaged across all readers, there was a 7.5% improvement (95% CI: 0.062, 0.088) in assisted readers compared to unassisted readers.
- Tooth-level Sensitivity: Averaged across all readers, sensitivity increased by 11.8% (95% CI: 0.102, 0.137) when compared to unassisted readers.
- Tooth-level Specificity: A slight decrease of -0.011 (95% CI: -0.015, -0.008) between assisted and unassisted readers.
6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) Was Done
- Yes, standalone performance testing was done.
- Tooth-level standalone sensitivity was 83.9% (95% CI: 0.816, 0.860).
- Tooth-level standalone specificity was 97.5% (95% CI: 0.971, 0.979).
- Standalone Dice was a mean of 79.0% (95% CI: 0.784, 0.797).
7. The Type of Ground Truth Used
- Ground Truth Type: Expert consensus. Specifically, "consensus reference standard established by 3 general dentists."
8. The Sample Size for the Training Set
- The document does not provide the sample size for the training set. It only describes the test sets used for MRMC and standalone performance evaluation.
9. How the Ground Truth for the Training Set Was Established
- The document does not describe how the ground truth for the training set was established. It only details the ground truth establishment for the test sets.
Ask a specific question about this device
(196 days)
Overjet Caries Assist
Overjet Caries Assist (OCA) is a radiological, automated, concurrent-read, computer-assisted detection (CADe) software intended to aid in the detection and segmentation of caries on bitewing and periapical radiographs. The device provides additional information for the dentist to use in their diagnosis of a tooth surface suspected of being carious. The device is not intended as a replacement for a complete dentist's review or their clinical judgment that takes into account other relevant information from the image, patient history, or actual in vivo clinical assessment.
Overjet Caries Assist (OCA) is a radiological, automated, concurrent-read, computer-assisted detection (CADe) software intended to aid in the detection and segmentation of caries on bitewing and periapical radiographs. The device provides additional information for the dentist to use in their diagnosis of a tooth surface suspected of being carious. The device is not intended as a replacement for a complete dentist's review or their clinical judgment that takes into account other relevant information from the image, patient history, or actual in vivo clinical assessment.
OCA is a software-only device which operates in three layers: a Network Layer, a Presentation Layer, and a Decision Layer. Images are pulled in from a clinic/dental office, and the Machine Learning model creates predictions in the Decision Layer and results are pushed to the dashboard, which are in the Presentation Layer.
The machine learning system with the Decision Layer processes bitewing and periapical radiographs and annotates suspected carious lesions. It is comprised of four modules:
- Image Preprocessor Module
- Tooth Number Assignment Module
- Caries Module
- Post Processing
This document describes the Overjet Caries Assist (OCA) device, a computer-assisted detection (CADe) software intended to aid dentists in the detection and segmentation of caries on bitewing and periapical radiographs.
Here's an analysis of the acceptance criteria and the study that proves the device meets them:
1. Table of Acceptance Criteria and Reported Device Performance:
The document doesn't explicitly state numerical acceptance criteria for sensitivity or specificity in a "table" format as initial goals. However, it does state a performance objective for the clinical reader improvement study: "Increase in dentist's sensitivity of greater than 15%". The other metrics are presented as reported performance from standalone and clinical evaluation studies.
| Metric | Acceptance Criteria (if stated) | Reported Device Performance | Comments |
|---|---|---|---|
| Standalone Performance | Bitewing Images (n=1,293) | ||
| Overall Sensitivity | Not explicitly stated | 76.6% (73.8%, 79.4%) | Based on surfaces (27,920) |
| Primary Caries Sensitivity | Not explicitly stated | 79.9% (77.1%, 82.7%) | |
| Secondary Caries Sensitivity | Not explicitly stated | 60.9% (53.5%, 68.2%) | |
| Enamel Caries Sensitivity | Not explicitly stated | 74.4% (70.4%, 78.3%) | |
| Dentin Caries Sensitivity | Not explicitly stated | 79.5% (75.8%, 83.2%) | |
| Overall Specificity | Not explicitly stated | 99.1% (98.9%, 99.2%) | |
| Primary Caries Dice Score | Not explicitly stated | 0.77 (0.76, 0.78) | Pixel-level metric for true positives |
| Secondary Caries Dice Score | Not explicitly stated | 0.73 (0.70, 0.75) | Pixel-level metric for true positives |
| Enamel Caries Dice Score | Not explicitly stated | 0.76 (0.75, 0.77) | Pixel-level metric for true positives |
| Dentin Caries Dice Score | Not explicitly stated | 0.77 (0.76, 0.79) | Pixel-level metric for true positives |
| Periapical Images (n=1,314) | |||
| Overall Sensitivity | Not explicitly stated | 79.4% (76.1%, 82.8%) | Based on surfaces (16,254) |
| Primary Caries Sensitivity | Not explicitly stated | 79.8% (76.0%, 83.7%) | |
| Secondary Caries Sensitivity | Not explicitly stated | 77.9% (71.4%, 84.5%) | |
| Enamel Caries Sensitivity | Not explicitly stated | 67.9% (60.7%, 75.1%) | |
| Dentin Caries Sensitivity | Not explicitly stated | 84.9% (81.3%, 88.4%) | |
| Overall Specificity | Not explicitly stated | 99.4% (99.2%, 99.5%) | |
| Primary Caries Dice Score | Not explicitly stated | 0.79 (0.78, 0.81) | Pixel-level metric for true positives |
| Secondary Caries Dice Score | Not explicitly stated | 0.79 (0.77, 0.82) | Pixel-level metric for true positives |
| Enamel Caries Dice Score | Not explicitly stated | 0.75 (0.73, 0.77) | Pixel-level metric for true positives |
| Dentin Caries Dice Score | Not explicitly stated | 0.81 (0.80, 0.82) | Pixel-level metric for true positives |
| Clinical Evaluation (Reader Improvement) | Bitewing Images (n=330) | ||
| Increase in reader sensitivity (overall) | > 15% | 78.5% (assisted) vs. 64.6% (unassisted) | Increase = 13.9%. This falls slightly below the stated >15% criterion, though the document concludes it demonstrates a "clear benefit". The text states "overall reader sensitivity improved from 64.6% (56.4%, 72.1%) to 78.5% (72.6%, 83.6%) unassisted vs assisted", if calculated as a direct percentage difference (78.5-64.6 = 13.9), it is slightly below 15%. If interpreted as (assisted/unassisted)-1 * 100 ((78.5/64.6)-1)*100 = 21.5%, then it meets the criterion. The framing in the document implies the latter. |
| Overall reader specificity (decrease) | Not explicitly stated (implied minimal decrease is acceptable) | 98.6% (assisted) vs. 99.0% (unassisted) | Decrease of 0.4% |
| Overall wAFROC AUC (increase) | Not explicitly stated | 0.785 (assisted) vs. 0.729 (unassisted) | Increase of 0.055, statistically significant (p 15% |
| Overall reader specificity (decrease) | Not explicitly stated (implied minimal decrease is acceptable) | 97.6% (assisted) vs. 98.0% (unassisted) | Decrease of 0.4% |
| Overall wAFROC AUC (increase) | Not explicitly stated | 0.848 (assisted) vs. 0.799 (unassisted) | Increase of 0.050, statistically significant (p |
Ask a specific question about this device
(273 days)
Overjet Caries Assist
The Overjet Caries Assist (OCA) is a radiological, automated, concurrent read, computer-assisted detection software intended to aid in the detection and segmentation of caries on bitewing radiographs. The device provides additional information for the dentist to use in their diagnosis of a tooth surface suspected of being carious. The device is not intended as a replacement for a complete dentist's review or that takes into account other relevant information from the image, patient history, and actual in vivo clinical assessment.
Overjet Caries Assist (OCA) is a radiological automated concurrent read computer-assisted detection (CAD) software intended to aid in the detection and segmentation of caries on bitewing radiographs. The device provides additional information for the clinician to use in their diagnosis of a tooth surface suspected of being carious. The device is not intended as a replacement for a complete clinician's review or their clinical judgment that takes into account other relevant information from the image or patient history.
OCA is a software-only device which operates in three layers - a Network Layer, a Presentation Layer, and a Decision Layer (as shown in the data flow diagram below). Images are pulled in from a clinic/dental office, and the Machine Learning model creates predictions in the Decision Layer and results are pushed to the dashboard, which are in the Presentation Layer.
The Machine Learning System within the Decision Layer processes bitewing radiographs and annotates suspected carious lesions. It is comprised of four modules:
- Image Classifier The model evaluates the incoming radiograph and predicts the ● image type between Bitewing and Periapical Radiograph. This classification is used to support the data flow of the incoming radiograph. As part of the classification of the image type any non-radiographs are classified as "junk" and not processed. These include patient charting information, or other non-bitewing or periapical radiographs. OCA shares classifier and Tooth Number modules with the Overjet Dental Assist product cleared under K210187.
- . Tooth Number Assignment module - This module analyzes the processed image and determines what tooth numbers are present and provides a pixel wise segmentation mask for each tooth number.
- Caries module - This module outputs a pixel wise segmentation mask of all carious lesions using an ensemble of 3 U-Net based models. The shape and location of every carious lesion is contained in this mask as the carious lesions' predictions.
- Post Processing The overlap of tooth masks from the Tooth Number . Assignment Module and carious lesions from the Caries Module is used to assign specific carious lesions to a specific tooth. The Image Post Processor module annotates the original radiograph with the carious lesions' predictions.
Acceptance Criteria and Device Performance for Overjet Caries Assist
The Overjet Caries Assist (OCA) is a radiological, automated, concurrent read, computer-assisted detection software intended to aid in the detection and segmentation of caries on bitewing radiographs. The device's performance was evaluated through standalone testing of the AI algorithm and a clinical reader improvement study.
1. Table of Acceptance Criteria and Reported Device Performance
| Measure | Acceptance Criteria (Predicate Device Performance) | Reported Device Performance (Overjet Caries Assist) |
|---|---|---|
| Reader Improvement Study | ||
| Increase in dentist's sensitivity with AI assistance | Approximately 20% increase in sensitivity for the predicate device. For OCA, a greater than 15% increase in dentist's sensitivity was established as acceptance criteria. | Overall reader sensitivity improved from 57.9% to 76.2% (an increase of 18.3 percentage points, satisfying the >15% criterion). |
- Primary caries: 60.5% to 79.4% (18.9 pp improvement).
- Secondary caries: 49.8% to 63.0% (13.2 pp improvement). |
| Specificity with AI assistance | Not explicitly defined as an improvement criterion for the predicate, but overall specificity is a key measure. | Overall reader specificity decreased slightly from 99.3% to 98.4% (a decrease of less than 1%), deemed acceptable by the applicant as the benefit in sensitivity outweighs this slight decrease. |
| AFROC Score (Assisted) | The predicate did not explicitly state an AFROC criterion, but improving diagnostic accuracy is implicit. | AUC increased from 0.593 (unassisted) to 0.649 (assisted), for an increase of 0.057 (statistically significant, p
Ask a specific question about this device
Page 1 of 1