Search Results

Videa Dental AI is a computer-assisted detection (CADe) device that analyzes intraoral radiographs to identify and localize the following features. Videa Dental AI is indicated for the review of bitewing, periapical, and panoramic radiographs acquired from patients aged 3 years or older.

Suspected Dental Findings:

• Caries
• Attrition
• Broken/Chipped Tooth
• Restorative Imperfection
• Pulp Stones
• Dens Invaginatus
• Periapical Radiolucency
• Widened Periodontal Ligament
• Furcation
• Calculus

Historical Treatments:

• Crown
• Filling
• Bridge
• Post and Core
• Root Canal
• Endosteal Implant
• Implant Abutment
• Bonded Orthodontic Retainer
• Braces

Normal Anatomy:

• Maxillary Sinus
• Maxillary Tuberosity
• Mental Foramen
• Mandibular Canal
• Inferior Border of the Mandible
• Mandibular Tori
• Mandibular Condyle
• Developing Tooth
• Erupting Teeth
• Non-matured Erupted Teeth
• Exfoliating Teeth
• Impacted Teeth
• Crowding Teeth

Videa Dental AI (VDA) software is a cloud-based AI-powered medical device for the automatic detection of the features listed in the Indications For Use statement in dental radiographs. The device itself is available as a service via an API (Application Programming Interface) behind a firewalled network. Provided proper authentication and an eligible bitewing, periapical or panoramic image, the device returns a set of bounding boxes and/or segmentation outlines depending on the indication representing the suspect dental finding, historical treatment or normal anatomy detected.

VDA is accessed by the dental practitioner through their dental image viewer. From within the dental viewer the user can upload a radiograph to VDA and then review the results. The device outputs a binary indication to identify the presence or absence of findings for each indication. If findings are present the device outputs the number of findings by finding type and the coordinates of the bounding boxes/segmentation outlines for each finding. If no findings are present the device outputs a clear indication that there are no findings identified for each indication. The device output will show all findings from one radiograph regardless of the number of teeth present.

The intended users of Videa Dental AI are trained dental professionals such as dentists and dental hygienists. For the suspect dental findings indications specifically, VDA is intended to be used as an adjunct tool and should not replace a dentist's review of the image. Only dentists that are performing diagnostic activities shall use the suspect dental finding indications.

VDA should not be used in-lieu of full patient evaluation or solely relied upon to make or confirm a diagnosis. The system is to be used by trained dental professionals including, but not limited to, dentists and dental hygienists.

Depending on the specific VDA indication for use, the intended patients of Videa Dental AI are patients 3 years of age and older above with primary, mixed and/or permanent dentition undergoing routine dental visits or suspected of one of the suspected dental findings listed in the VDA indications for use statement above. VDA may be used on eligible bitewing, periapical or panoramic radiographs depending on the indication.

See Table 1 below for the specific patient age group and image modality that each VDA indication for use is designed and tested to meet. VDA uses the images metadata to only show the indications for the patient age and image modalities in scope as shown in Table 1. VDA will not show any findings output for an indication for use that is outside of the patient age and radiographic view scope.

Here's a summary of the acceptance criteria and study details for Videa Dental AI, based on the provided FDA 510(k) Clearance Letter:

1. Table of Acceptance Criteria and Reported Device Performance:

The document doesn't explicitly state numeric acceptance criteria thresholds for all indications. However, it implicitly states that Videa Dental AI meets its performance requirements by demonstrating statistically significant improvement in detection performance for clinicians when aided by the device compared to unaided performance in the clinical study for certain indications. For standalone performance, DICE scores are provided for caries, calculus, and normal tooth anatomy segmentations.

2. Sample Size Used for the Test Set and Data Provenance:

• Standalone Performance Test Set:
  • Sample Size: 1,445 radiographs
  • Data Provenance: Collected from more than 35 US sites (retrospective, implied, as it's for ground-truthing/benchmarking).
• Clinical Performance (MRMC) Test Set:
  • Sample Size: 378 radiographs
  • Data Provenance: Collected from over 25 US locations spread across the country (retrospective, implied, as it's for ground-truthing/benchmarking).

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications:

• Standalone Performance Test Set:
  • Number of Experts: Three
  • Qualifications: US board-certified dentists.
• Clinical Performance (MRMC) Test Set:
  • Number of Experts: Not explicitly stated for the initial labeling, but a single US licensed dentist adjudicated the labels to establish the reference standard.
  • Qualifications: US licensed dentists labeled the data, and a US licensed dentist adjudicated those labels.

4. Adjudication Method for the Test Set:

• Standalone Performance Test Set: Ground-truthed by three US board-certified dentists. The specific adjudication method (e.g., consensus, majority) is not explicitly detailed beyond "ground-truthed by three...".
• Clinical Performance (MRMC) Test Set: US licensed dentists labeled the data, and a US licensed dentist adjudicated those labels to establish a reference standard. This implies a consensus or expert-review model, possibly 2+1 or similar, where initial labels were reviewed and finalized by a single adjudicator.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done, What was the Effect Size of How Much Human Readers Improve with AI vs. Without AI Assistance:

• Yes, an MRMC comparative effectiveness study was done.
• Hypothesis Tested:
  • H₀: AFROC FOMₐᵢdₑd - AFROC FOMᵤₙₐᵢdₑd ≤ 0
  • H₁: AFROC FOMₐᵢdₑd - AFROC FOMᵤₙₐᵢdₑd > 0
• Effect Size:
  • Across 8 Videa Dental AI Suspect Dental Finding indications in the clinical study, the average amount of aided improvement over unaided performance was 0.002%.
  • For the caries and periapical radiolucency VDA indications (with a second operating point), clinicians had statistically significant improvement in detection performance regardless of the operating point used. The specific AFROC FOM delta is not provided for these, only that it was statistically significant.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was Done:

• Yes, a standalone performance assessment was conducted.
• It measured and reported the performance of Videa Dental AI by itself, in the absence of any interaction with a dental professional in identifying regions of interest for all suspect dental finding, historical treatment, and normal anatomy VDA indications.

7. The Type of Ground Truth Used:

• Expert Consensus/Review: The ground truth for both standalone and clinical studies was established by US board-certified or licensed dentists who labeled and/or adjudicated the findings on the radiographs.

8. The Sample Size for the Training Set:

• The document does not explicitly state the sample size for the training set. It mentions the AI algorithms were "trained with that patient population" and "trained with bitewing, periapical and panoramic radiographs," but gives no specific number of images or patients for the training dataset.

9. How the Ground Truth for the Training Set Was Established:

• The document does not explicitly state how the ground truth for the training set was established. It only broadly states that the AI algorithms were trained with a specific patient population and image types. Given the general practice for medical AI, it can be inferred that expert labeling similar to the test set would have been used, but this is not confirmed in the provided text.

Ask a specific question about this device

VELMENI for DENTISTS (V4D) is a concurrent-read, computer-assisted detection software intended to assist dentists in the clinical detection of dental caries, fillings/restorations, fixed prostheses, and implants in digital bitewing, periapical, and panoramic radiographs of permanent teeth in patients 15 years of age or older. This device provides additional information for dentists in examining radiographs of patients' teeth. This device is not intended as a replacement for a complete examination by the dentist or their clinical judgment that considers other relevant information from the image, patient history, or actual in vivo clinical assessment. Final diagnoses and patient treatment plans are the responsibility of the dentist.

This device includes a Predetermined Change Control Plan (PCCP).

V4D software medical device comprises of the following key components:

• Web Application Interface delivers front-end capabilities and is the point of interaction between the device and the user.
• Machine Learning (ML) Engine delivers V4D's core ML capabilities through the radiograph type classifier, condition detection module, tooth numbering module, and merging module.
• Backend API allows interaction between all the components, as defined in this section, in order to fulfill the user's requests on the web application interface.
• Queue receives and stores messages from Backend API to send to AI-Worker.
• AI-Worker accepts radiograph analysis requests from Backend API via the Queue, passes gray scale radiographs to the ML Engine in the supported extensions (jpeg and png), and returns the ML analysis results to the Backend API.
• Database and File Storage store critical information related to the application, including user data, patient profiles, analysis results, radiographs, and associated data.

The following non-medical interfaces are also available with VELMENI for DENTISTS (V4D):

• VELMENI BRIDGE (VB) acts as a conduit enabling data and information exchange between Backend API and third-party software like Patient Management or Imaging Software
• Rejection Review (RR) module captures the ML-detected conditions rejected by dental professionals to aid in future product development and to be evaluated in accordance with VELMENIs post-market surveillance procedure.

This device includes a Predetermined Change Control Plan (PCCP).

This 510(k) clearance letter for VELMENI for DENTISTS (V4D) states that the proposed device is unchanged from its predicate (VELMENI for Dentists cleared under K240003), except for the inclusion of a Predetermined Change Control Plan (PCCP). Therefore, all performance data refers back to the original K240003 clearance. The provided document does not contain the specific performance study details directly, but it references their applicability from the predicate device.

Based on the provided text, the response will extract what details are available and note where specific information is not included in this document, but referred to as existing from the predicate device's clearance.

1. Table of Acceptance Criteria and Reported Device Performance

The provided document refers to the acceptance criteria and performance data existing from the predicate device (K240003). It also mentions that the PCCP updates the acceptance criteria for Sensitivity, Specificity, and Average False Positives to match the lower bounds of the confidence interval demonstrated by the originally cleared models' standalone results. However, the specific values for these criteria and the reported performance are not explicitly stated in this document.

Note: The document only states that MRMC results concluded the effectiveness of the V4D software in assisting readers to identify more caries and identify more fixed prostheses, implants, and restorations correctly. Specific quantitative performance metrics (e.g., Sensitivity, Specificity, AUC, FROC, etc.) are not provided in this document.

2. Sample Size Used for the Test Set and Data Provenance

The document states:

• "The new models will be evaluated on a combined test dataset with balanced ratio of historical and new data for validation to avoid overfitting historical data from repeated use."
• "The new test data is fully independent on a site-level from training/tuning data, and the test dataset remains at least 50% US data."

Specific sample size for the test set is not provided in this document.
Data Provenance: At least 50% US data, including both historical and new data. It is a retrospective dataset for testing as it uses both historical and new data collected implicitly beforehand.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

The document does not specify the number of experts used and their qualifications for establishing ground truth for the test set.

4. Adjudication Method for the Test Set

The document does not specify the adjudication method used for the test set (e.g., 2+1, 3+1, none).

5. Multi Reader Multi Case (MRMC) Comparative Effectiveness Study

Yes, an MRMC comparative effectiveness study was done.

The document states: "MRMC results concluded the effectiveness of the V4D software in assisting readers to identify more caries and identify more fixed prostheses, implants, and restorations correctly."

Effect Size: The document does not provide a specific quantitative effect size of how much human readers improve with AI vs. without AI assistance. It only makes a qualitative statement about improved identification of conditions.

6. Standalone Performance Study

Yes, a standalone (algorithm only without human-in-the-loop performance) study was done.

The document states: "The acceptance criteria for Sensitivity, Specificity and Average False Positives have been updated to match the lower bounds of confidence interval demonstrated by the originally cleared models' standalone results." This implies that standalone performance metrics were evaluated for the original clearance.

7. Type of Ground Truth Used

The document does not explicitly state the type of ground truth used (e.g., expert consensus, pathology, outcomes data). However, for a dental imaging device assisting dentists, it is highly likely that expert consensus from dental professionals (dentists or dental radiologists) would have been used for establishing ground truth. The mention of "dental professionals" rejecting ML-detected conditions in the "Rejection Review (RR)" module also hints at expert review for ground truth establishment.

8. Sample Size for the Training Set

The document does not specify the sample size for the training set. It mentions "new and existing training and tuning data" for re-training.

9. How the Ground Truth for the Training Set Was Established

The document does not explicitly state how the ground truth for the training set was established. However, given the context of a medical device aiding dentists in clinical detection, it is highly probable that ground truth would have been established through expert annotations or consensus from qualified dental professionals.

Ask a specific question about this device

Second Opinion® CS is a computer aided detection ("CADe") software to aid in the detection and segmentation of caries in periapical radiographs.

It is designed to aid dental health professionals to review periapical radiographs of permanent teeth in patients 12 years of age or older as a second reader.

Second Opinion CS detects suspected carious lesions and presents them as an overlay of segmented contours. The software highlights detected caries with an overlay and provides a detailed analysis of the lesion's overlap with dentine and enamel, presented as a percentage. The output of Second Opinion CS is a visual overlay of regions of the input radiograph which have been detected as potentially containing caries. The user can hover over the caries detection to see the segmentation analysis.

Second Opinion PC consists of three parts:

• Application Programing Interface ("API")
• Machine Learning Modules ("ML Modules")
• Client User Interface ("Client")

The processing sequence for an image is as follows:

1. Images are sent for processing via the API
2. The API routes images to the ML modules
3. The ML modules produce detection output
4. The UI renders the detection output

The API serves as a conduit for passing imagery and metadata between the user interface and the machine learning modules. The API sends imagery to the machine learning modules for processing and subsequently receives metadata generated by the machine learning modules which is passed to the interface for rendering.

Second Opinion CS uses machine learning to detect and segment caries. Images received by the ML modules are processed yielding detections which are represented as metadata. The final output is made accessible to the API for the purpose of sending to the UI for visualization. Detected carious lesions are displayed as overlays atop the original radiograph which indicate to the practitioner which teeth contain which detected carious lesions that may require clinical review. The clinician can toggle over the image to highlight a potential condition for viewing. Further, the clinician can hover over the detected caries to show a hover information box containing the segmentation of the caries in the form of percentages.

Here's a detailed breakdown of the acceptance criteria and the study proving the device meets them, based on the provided FDA 510(k) clearance letter for Second Opinion® CS:

Acceptance Criteria and Reported Device Performance

Study Details

1. Sample Size for Test Set and Data Provenance

• Standalone Test Set: 1250 periapical radiograph images containing 404 overall caries lesions on 286 abnormal images.
  • Provenance: Retrospective. Data was collected from various geographical regions within the United States: Northwest (11.0%), Northeast (18.8%), South (29.2%), West (15.6%), and Midwest (25.5%).
  • Demographics: Includes radiographs from females (50.1%), males (44.6%), and unknown gender (5.3%). Age distribution: 12-18 (12.3%), 18-75 (81.7%), and 75+ (6.0%).
  • Imaging Devices: Carestream-Trophy KodakRVG6100 (25.7%), Carestream-Trophy RVG5200 (3.2%), Carestream-Trophy RVG6200 (27.0%), DEXIS Platinum (19.2%), DEXIS Titanium (18.8%), KodakTrophy KodakRVG6100 (0.8%), and unknown devices (5.3%).
• MRMC Test Set: 330 radiographs with 508 caries lesions across 179 abnormal images.
  • Provenance: Not explicitly stated but inferred to be retrospective, similar to the standalone study, given the focus on existing image characteristics.

2. Number of Experts and Qualifications for Test Set Ground Truth

• Standalone Study: Not explicitly stated for the standalone study. However, the MRMC study description clarifies the method for ground truth establishment, which likely applies to the test set used for standalone evaluation as well.
• MRMC Study: Ground truth was determined by four experienced dentists.
  • Qualifications: "U.S.-certified dentists" and "experienced dentists."

3. Adjudication Method for Test Set

• Standalone Study: Not explicitly stated, but implies expert consensus was used to establish ground truth.
• MRMC Study: Consensus was achieved when the Jaccard index was ≥0.4 amongst the four experienced dentists. This indicates a form of consensus-based adjudication where a certain level of agreement on lesion boundaries was required.

4. MRMC Comparative Effectiveness Study

• Yes, a fully crossed multi-reader multi-case (MRMC) study was done.
• Effect Size (Improvement of human readers with AI vs. without AI assistance):
  • Overall Caries Detection (wAFROC-FOM): Aided readers showed a significant improvement of 0.05 (95% CI: 0.01–0.09) in wAFROC-FOM compared to unaided readers.
  • Lesion-level Sensitivity: Aided readers showed a significant increase of 0.20 (95% CI: 0.16–0.24) in lesion-level sensitivity.
  • False Positives Per Image (FPPI): FPPI increased slightly by 0.16 (95% CI: -0.03–0.36).
  • Surface-level Specificity: Decreased marginally by -0.02 (95% CI: -0.04–0.00).

5. Standalone (Algorithm Only) Performance Study

• Yes, a standalone performance assessment was done to validate the inclusion of a new caries lesion anatomical segmentation.
• Key Results:
  • Sensitivity was > 70%, with an estimated lesion level sensitivity of 0.88 (95% CI), which was statistically significant (p < 0.0001).
  • Caries subgroup sensitivity for enamel was 0.95 (95% CI: 0.92, 0.97) and for dentin was 0.86 (95% CI: 0.81, 0.90).
  • Caries subgroup FPPI for enamel was 0.76 (95% CI: 0.70, 0.83) and for dentin was 0.48 (95% CI: 0.43, 0.52).
  • The standalone performance of the CAD for overall caries outperformed unaided readers in terms of wAFROC-FOM and sensitivity, albeit with higher FPPI.

6. Type of Ground Truth Used

• Expert Consensus: For the MRMC study, the ground truth was established by four experienced dentists achieving consensus (Jaccard index ≥0.4). This expert consensus is implied for the standalone study's ground truth as well.

7. Sample Size for Training Set

• The document does not explicitly state the sample size for the training set. It mentions the device uses "computer vision neural network algorithms, developed from open-source models using supervised machine learning techniques," implying a training phase, but the size of the training data is not provided.

8. How Ground Truth for Training Set Was Established

• The document states that the software uses "supervised machine learning techniques," which inherently requires a labeled dataset for training. However, it does not explicitly describe the method by which the ground truth for this training set was established (e.g., number of experts, their qualifications, or adjudication methods). It can be inferred that a similar expert labeling process was used, but details are not provided in this clearance letter.

Ask a specific question about this device

Smile Dx® is a computer-assisted detection (CADe) software designed to aid dentists in the review of digital files of bitewing and periapical radiographs of permanent teeth. It is intended to aid in the detection and segmentation of suspected dental findings which include: caries, periapical radiolucencies (PARL), restorations, and dental anatomy.

Smile Dx® is also intended to aid dentists in the measurement (in millimeter and percentage measurements) of mesial and distal bone levels associated with each tooth.

The device is not intended as a replacement for a complete dentist's review or their clinical judgment that takes into account other relevant information from the image, patient history, and actual in vivo clinical assessment.

Smile Dx® supports both digital and phosphor sensors.

Smile Dx® is a computer assisted detection (CADe) device indicated for use by licensed dentists as an aid in their assessment of bitewing and periapical radiographs of secondary dentition in adult patients. Smile Dx® utilizes machine learning to produce annotations for the following findings:

• Caries
• Periapical radiolucencies
• Bone level measurements (mesial and distal)
• Normal anatomy (enamel, dentin, pulp, and bone)
• Restorations

The provided FDA 510(k) Clearance Letter for Smile Dx® outlines the device's acceptance criteria and the studies conducted to prove it meets those criteria.

Acceptance Criteria and Device Performance

The acceptance criteria are implicitly defined by the performance metrics reported in the "Performance Testing" section. The device's performance is reported in terms of various metrics for both standalone and human-in-the-loop (MRMC) evaluations.

Here's a table summarizing the reported device performance against the implied acceptance criteria:

Table 1: Acceptance Criteria and Reported Device Performance

Study Details for Device Performance Proof:

1. Sample Sizes for the Test Set and Data Provenance

• Standalone Testing:
  • Caries and Periapical Radiolucency Detection: 867 cases.
  • Bone Level Detection and Bone Loss Measurement: 352 cases.
  • Normal Anatomy and Restorations: 200 cases.
• MRMC Clinical Evaluation: 352 cases.
• Data Provenance: All test sets were collected from "multiple U.S. sites." The data is retrospective, as it's used in a "retrospective study" for the MRMC evaluation. Sub-group analysis also included "imaging hardware" and "patient demographics (i.e., age, sex, race)," indicating diversity in data.

2. Number of Experts and Qualifications for Ground Truth

• Standalone Testing (Implicit): Not explicitly stated how ground truth for standalone testing was established, but it is likely derived from expert consensus, similar to the MRMC study.
• MRMC Clinical Evaluation: Ground truth was established by the "consensus labels of four US licensed dentists."

3. Adjudication Method for the Test Set

• MRMC Clinical Evaluation: The ground truth for the MRMC study was established by the "consensus labels of four US licensed dentists." This implies a form of consensus adjudication, likely where all four experts reviewed and reached agreement on the findings. The specific method (e.g., majority vote, 2+1, 3+1) is not explicitly detailed beyond "consensus labels." For standalone testing, the adjudication method for ground truth is not specified.

4. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Yes, an MRMC comparative effectiveness study was done.
• Effect Size of Human Readers' Improvement with AI vs. Without AI Assistance:
  • Caries Detection:
    • wAFROC Δθ: +0.127 [0.081, 0.172] (p < 0.001)
    • Sensitivity Improvement: 19.6% increase (from 64.3% without device to 83.9% with device).
    • Specificity Improvement: 16.7% increase (from 73.6% without device to 90.2% with device).
  • Periapical Radiolucency (PARL) Detection:
    • wAFROC Δθ: +0.098 [0.061, 0.135] (p < 0.001)
    • Sensitivity Improvement: 19.1% increase (from 70.7% without device to 89.8% with device).
    • Specificity Improvement: 4.7% increase (from 92.6% without device to 97.3% with device).
• The study design was a "fully-crossed, multiple-reader multiple-case (MRMC) evaluation method" with "at least 13 US licensed dentists (Smile Dx® had 14 readers)." Half of the data set contained unannotated images, and the second half contained radiographs that had been processed through the CADe device. Radiographs were presented to readers in alternating groups throughout two different sessions, separated by a washout period.

5. Standalone Performance Study (Algorithm Only)

• Yes, a standalone performance study was done.
• It evaluated the algorithm's performance for:
  • Caries and Periapical Radiolucency Detection (Dice, Sensitivity)
  • Bone Level Detection and Bone Loss Measurement (Sensitivity, Specificity, Mean Absolute Error)
  • Normal Anatomy and Restorations Detection (Dice, Pixel-level Sensitivity, Contour-level Sensitivity, Contour-level Specificity)

6. Type of Ground Truth Used

• Explicitly for MRMC Study: "Consensus labels of four US licensed dentists." This indicates expert consensus was used for ground truth for the human-in-the-loop evaluation and likely for the standalone evaluation's ground truth as well. There is no mention of pathology or outcomes data.

7. Sample Size for the Training Set

• The document does not explicitly state the sample size for the training set. It only mentions the test set sizes.

8. How Ground Truth for Training Set Was Established

• The document does not explicitly state how ground truth for the training set was established. It mentions the model "utilizes machine learning to produce annotations" and "training data" is used, but provides no details on its annotation process. It's highly probable that expert annotation was also used for the training data, similar to the test set, but this is not confirmed in the provided text.

Ask a specific question about this device

Second Opinion® Pediatric is a computer aided detection ("CADe") software to aid in the detection of caries in bitewing and periapical radiographs.

The intended patient population of the device is patients aged 4 years and older that have primary or permanent teeth (primary or mixed dentition) and are indicated for dental radiographs.

Second Opinion Pediatric is a radiological, automated, computer-assisted detection (CADe) software intended to aid in the detection and segmentation of caries on bitewing and periapical radiographs. The device is not intended as a replacement for a complete dentist's review or their clinical judgment which considers other relevant information from the image, patient history, or actual in vivo clinical assessment.

Second Opinion Pediatric consists of three parts:

• Application Programing Interface ("API")
• Machine Learning Modules ("ML Modules")
• Client User Interface (UI) ("Client")

The processing sequence for an image is as follows:

1. Images are sent for processing via the API
2. The API routes images to the ML modules
3. The ML modules produce detection output
4. The UI renders the detection output

The API serves as a conduit for passing imagery and metadata between the user interface and the machine learning modules. The API sends imagery to the machine learning modules for processing and subsequently receives metadata generated by the machine learning modules which is passed to the interface for rendering.

Second Opinion® Pediatric uses machine learning to detect caries. Images received by the ML modules are processed yielding detections which are represented as metadata. The final output is made accessible to the API for the purpose of sending to the UI for visualization. Detected caries are displayed as polygonal overlays atop the original radiograph which indicate to the practitioner which teeth contain detected caries that may require clinical review. The clinician can toggle over the image to highlight a potential condition for viewing. Further, the clinician can hover over the detected caries to show a hover information box containing the segmentation of the caries in the form of percentages.

Here's a breakdown of the acceptance criteria and study details for the Second Opinion® Pediatric device, based on the provided FDA 510(k) clearance letter:

Acceptance Criteria and Reported Device Performance

Study Details

1. Sample sizes used for the test set and the data provenance:

   • Test Set Sample Size: 1182 radiographic images, containing 1085 caries lesions on 549 abnormal images.
   • Data Provenance: Not specified in the provided document (e.g., country of origin, retrospective or prospective). However, it states it was a "standalone retrospective study."

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • Number of Experts: Not explicitly stated.
   • Qualifications of Experts: Not specified. The document only mentions "Ground Truth," but details on the experts who established it are absent.

3. Adjudication method for the test set:

   • Adjudication Method: Not explicitly stated. The document refers to "Ground Truth" but does not detail how potential disagreements among experts (if multiple were used) were resolved. It previously mentions "consensus truthing method" for the predicate device's study, which might imply a similar approach, but it is not confirmed for the subject device.

4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • MRMC Study: No, an MRMC comparative effectiveness study was not performed for the Second Opinion® Pediatric device (the subject device). The provided text states, "The effectiveness of Second Opinion® Pediatric was evaluated in a standalone performance assessment to validate the CAD." The predicate device description mentions its purpose is to "aid dental health professionals... as a second reader," which implies an assistive role, but no MRMC data on human reader improvement with AI assistance is provided for either the subject or predicate device.

5. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:

   • Standalone Study: Yes, a standalone performance assessment was explicitly conducted for the Second Opinion® Pediatric device. The study "assessed the sensitivity of caries detection of Second Opinion® Pediatric compared to the Ground Truth."

6. The type of ground truth used:

   • Ground Truth Type: Expert consensus is implied, as the study compared the device's performance against "Ground Truth" typically established by expert review. For the predicate, it explicitly mentions "consensus truthing method." It does not specify if pathology or outcomes data were used.

7. The sample size for the training set:

   • Training Set Sample Size: Not specified in the provided document. The document focuses on the validation study.

8. How the ground truth for the training set was established:

   • Training Set Ground Truth Establishment: Not specified in the provided document.

Ask a specific question about this device

Lung AI software device is a Computer-Aided Detection (CADe) tool designed to assist in the detection of consolidation/atelectasis and pleural effusion during the review of lung ultrasound scans.

The software is an adjunctive tool to alert users to the presence of regions of interest (ROI) with consolidation/atelectasis and pleural effusion within the analyzed lung ultrasound cine clip.

Lung AI is intended to be used on images collected from the PLAPS point, in accordance with the BLUE protocol.

The intended users are healthcare professionals who are trained and qualified in performing lung ultrasound and routinely perform lung ultrasounds as part of their current practice in a point-of-care environment—namely Emergency Departments (EDs). The device was not designed and tested with use environments representing EMTs and military medics.

Lung AI is not intended for clinical diagnosis and does not replace the healthcare provider's judgment or other diagnostic tests in the standard care for lung ultrasound findings. All cases where a Chest CT scan and/or Chest X-ray is part of the standard of care should undergo these imaging procedures, irrespective of the device output.

The software is indicated for adults only.

Lung AI is a Computer-Aided Detection (CADe) tool designed to assist in the analysis of lung ultrasound images by suggesting the presence of consolidation/atelectasis and pleural effusion in a scan. This adjunctive tool is intended to aid users to detect the presence of regions of interest (ROI) with consolidation/atelectasis and pleural effusion. However, the device does not provide a diagnosis for any disease nor replace any diagnostic testing in the standard of care.

The lung AI module processes Ultrasound cine clips and flags any evidence of pleural effusion and/or consolidation/atelectasis present without aggregating data across regions or making any patient-level decisions. For positive cases, a single ROI per clip from a frame with the largest pleural effusion (or consolidation/atelectasis) is generated as part of the device output. Moreover, the ROI output is for visualization only and should not be relied on for precise anatomical localization. The final decision regarding the overall assessment of the information from all regions/clips remains the responsibility of the user. Lung AI is intended to be used on clips collected only from the PLAPS point, in accordance with the BLUE protocol.

Lung AI is developed as a module to be integrated by another computer programmer into their legally marketed ultrasound imaging device. The software integrates with third-party ultrasound imaging devices and functions as a post-processing tool. The software does not include a built-in viewer; instead, it works within the existing third-party device interface.

Lung AI is validated to meet applicable safety and efficacy requirements and to be generalizable to image data sourced from ultrasound transducers of a specific frequency range.

The device is intended to be used on images of adult patients undergoing point-of-care (POC) lung ultrasound scans in the emergency departments due to suspicion of pleural effusion and/or consolidation/atelectasis. It is important to note that patient management decisions should not be made solely on the results of the Lung AI analysis.

Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided FDA 510(k) clearance letter for Lung AI (LAI001).

Acceptance Criteria and Device Performance for Lung AI (LAI001)

The Lung AI (LAI001) device underwent both standalone performance evaluation and a multi-reader, multi-case (MRMC) study to demonstrate its safety and effectiveness.

1. Table of Acceptance Criteria and Reported Device Performance

The document specifies performance metrics based on the standalone evaluation (sensitivity and specificity for detection and localization) and the MRMC study (AUC, sensitivity, and specificity for human reader performance with and without AI assistance). The acceptance criteria for the MRMC study are explicitly stated as an improvement of at least 2% in overall reader performance (AUC-ROC).

Standalone Performance Metrics (Derived from "Summary of Lung AI performance" and "Summary of Lung AI localization performance")

Note: Specific numerical acceptance criteria for standalone performance are not explicitly stated in the document, but the reported values demonstrated meeting the required performance for FDA clearance.

MRMC Study Acceptance Criteria and Reported Device Performance

2. Sample Size and Data Provenance for Test Set

• Sample Size for Standalone Test Set: 465 lung scans from 359 unique patients.

• Data Provenance: Retrospectively collected from 6 imaging centers in the U.S. and Canada, with more than 50% of the data coming from U.S. centers. The dataset was enriched with abnormal cases (at least 30% abnormal per center) and included diverse demographic variables (gender, age 21-96, ethnicity).

• Sample Size for MRMC Test Set: 322 unique patients (cases). Each of the 6 readers analyzed 748 cases per reading period, for a total of 4488 cases overall.

3. Number of Experts and Qualifications for Ground Truth Establishment (Test Set)

• Number of Experts: Two US board-certified experts initially, with a third expert for adjudication.
• Qualifications of Experts: Experienced in point-of-care ultrasound, reading lung ultrasound scans, and diagnostic radiology.

4. Adjudication Method for Test Set

• Method: In cases of disagreement between the first two experts, a third expert provided adjudication. This is a "2+1" (primary readers + adjudicator) method.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Was it done?: Yes, an MRMC study was conducted.
• Effect Size of Improvement:
  • Pleural Effusion:
    • AUC improved by 0.035 (ΔAUC-PLEFF = 0.035) when aided by the device.
    • Sensitivity improved by 0.18 (ΔSe-PLEFF = 0.18) when aided by the device.
    • Specificity slightly decreased by -0.03 when aided by the device.
  • Consolidation/Atelectasis:
    • AUC improved by 0.028 (ΔAUC-CONS = 0.028) when aided by the device.
    • Sensitivity improved by 0.16 (ΔSp-CONS = 0.16) when aided by the device.
    • Specificity slightly decreased by -0.008 when aided by the device.

6. Standalone (Algorithm Only) Performance Study

• Was it done?: Yes, the "Bench Testing" section describes a standalone performance evaluation.

7. Type of Ground Truth Used

• Type of Ground Truth: Expert consensus (established by two US board-certified experts with a third adjudicator) for the presence/absence of consolidation/atelectasis and pleural effusion per cine clip. They also provided bounding box annotations for localization ground truth.

8. Sample Size for Training Set

• Sample Size: 3,453 ultrasound cine clips from 1,036 patients.

9. How Ground Truth for Training Set Was Established

• The document states that the underlying deep learning models were "trained on a diverse dataset of 3,453 ultrasound cine clips from 1,036 patients." While it doesn't explicitly detail the process for establishing ground truth for the training set, it can be inferred that a similar expert review process, likely involving radiologists or expert sonographers, was used, as is standard practice for supervised deep learning in medical imaging. The clinical confounders mentioned (Pneumonia, Pulmonary Embolism, CHF, Tamponade, Covid19, ARDS, COPD) suggest a robust labeling process to differentiate findings.

Ask a specific question about this device

Second Opinion PC is a computer aided detection ("CADe") software to aid dentists in the detection of periapical radiolucencies by drawing bounding polygons to highlight the suspected region of interest.

It is designed to aid dental health professionals to review periapical radiographs of permanent teeth in patients 12 years of age or older as a second reader.

Second Opinion PC (Periapical Radiolucency Contouring) is a radiological, automated, computer-assisted detection (CADe) software intended to aid in the detection of periapical radiolucencies on periapical radiographs using polygonal contours. The device is not intended as a replacement for a complete dentist's review or their clinical judgment which considers other relevant information from the image, patient history, or actual in vivo clinical assessment.

Second Opinion PC consists of three parts:

• Application Programing Interface ("API")
• Machine Learning Modules ("ML Modules")
• Client User Interface ("Client")

The processing sequence for an image is as follows:

1. Images are sent for processing via the API
2. The API routes images to the ML modules
3. The ML modules produce detection output
4. The UI renders the detection output

The API serves as a conduit for passing imagery and metadata between the user interface and the machine learning modules. The API sends imagery to the machine learning modules for processing and subsequently receives metadata generated by the machine learning modules which is passed to the interface for rendering.

Second Opinion PC uses machine learning to detect periapical radiolucencies. Images received by the ML modules are processed yielding detections which are represented as metadata. The final output is made accessible to the API for the purpose of sending to the UI for visualization. Detected periapical radiolucencies are displayed as polygonal overlays atop the original radiograph which indicate to the practitioner which teeth contain which detected periapical radiolucencies that may require clinical review. The clinician can toggle over the image to highlight a potential condition for viewing.

Here's a detailed breakdown of the acceptance criteria and the study proving the device meets them, based on the provided FDA 510(k) clearance letter:

Acceptance Criteria and Device Performance Study

The Pear Inc. "Second Opinion Periapical Radiolucency Contours" (Second Opinion PC) device aims to aid dentists in detecting periapical radiolucencies using polygonal contours, functioning as a second reader. The device's performance was evaluated through a standalone clinical study demonstrating non-inferiority to its predicate device, which used bounding boxes.

1. Table of Acceptance Criteria and Reported Device Performance

The submission document primarily focuses on demonstrating non-inferiority to the predicate device rather than explicitly stating pre-defined acceptance criteria with specific thresholds for "passing." However, the implicit acceptance criteria are that the device is non-inferior to its predicate (Second Opinion K210365) in detecting periapical radiolucencies when using polygonal contours.

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size: 500 unique unannotated periapical radiographs.
• Data Provenance: The dataset is characterized by a representative distribution across:
  • Geographical Regions (within the United States):
    • Northwest: 116 radiographs (23.2%)
    • Southwest: 46 radiographs (9.2%)
    • South: 141 radiographs (28.2%)
    • East: 84 radiographs (16.8%)
    • Midwest: 113 radiographs (22.6%)
  • Patient Cohorts (Age Distribution):
    • 12-18 years: 4 radiographs (0.8%)
    • 18-75 years: 209 radiographs (41.8%)
    • 75+ years: 8 radiographs (1.6%)
    • Unknown age: 279 radiographs (55.8%)
  • Imaging Devices: A variety of devices were used, including Carestream-Trophy (RVG6100, RVG5200, RVG6200), DEXIS (DEXIS, DEXIS Platinum, KaVo Dental Technologies DEXIS Titanium), Kodak-Trophy KodakRVG6100, XDR EV71JU213, and unknown devices.
• Retrospective or Prospective: Not explicitly stated, but the description of "representative distribution" and diverse origins suggests a retrospective collection of existing images.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

• Number of Experts: Four expert readers.
• Qualifications of Experts: Not explicitly stated beyond "expert readers."

4. Adjudication Method for the Test Set

• Adjudication Method: Consensus approach based on agreement among at least three out of four expert readers (3+1 adjudication).

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done

• Was an MRMC study done? No, a traditional MRMC comparative effectiveness study was not performed for the subject device (Second Opinion PC).
• Effect Size of Human Readers with AI vs. without AI: Not applicable for this specific study of Second Opinion PC. The predicate device (Second Opinion K210365) did undergo MRMC studies, demonstrating statistically significant improvement in aided reader performance for that device. The current study focuses on the standalone non-inferiority of Second Opinion PC compared to its predicate.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done

• Was a standalone study done? Yes, a standalone clinical study was performed. The study compared the performance of Second Opinion PC (polygonal localization) directly with Second Opinion (bounding box localization) in detecting periapical radiolucencies.
• Metrics: wAFROC-FOM and HR-ROC-AUC were used.
• Key Finding: Second Opinion PC was found to be non-inferior to Second Opinion.

7. The Type of Ground Truth Used

• Type of Ground Truth: Expert consensus. The ground truth (GT) was established by the consensus of at least three out of four expert readers who independently marked periapical radiolucencies using the smallest possible polygonal contour.

8. The Sample Size for the Training Set

• Sample Size for Training Set: Not explicitly mentioned in the provided text. The document focuses on the clinical validation (test set).

9. How the Ground Truth for the Training Set Was Established

• Ground Truth for Training Set: Not explicitly mentioned in the provided text. It is implied that the device was "developed using machine learning techniques" from "open-source models using supervised machine learning," which typically requires a labeled training set, but specifics on its establishment are absent.

Ask a specific question about this device

Rayvolve LN is a computer-aided detection software device to identify and mark regions in relation to suspected pulmonary nodules from 6 to 30mm size. It is designed to aid radiologists in reviewing the frontal (AP/PA) chest radiographs of patients of 18 years of age or older acquired on digital radiographic systems as a second reader and be used with any DICOM Node server. Rayvolve LN provides adjunctive information only and is not a substitute for the original chest radiographic image.

The medical device is called Rayvolve LN. Rayvolve LN is one of the verticals of the Rayvolve product line. It is a standalone software that uses deep learning techniques to detect and localize pulmonary nodules on chest X-rays. Rayvolve LN is intended to be used as an aided-diagnosis device and does not operate autonomously.

Rayvolve LN has been developed to use the current edition of the DICOM image standard. DICOM is the international standard for transmitting, storing, retrieving, printing, processing, and displaying medical imaging.

Using the DICOM standard allows Rayvolve LN to interact with existing DICOM Node servers (eg.: PACS) and clinical-grade image viewers. The device is designed for running on-premise, cloud platform, connected to the radiology center local network, and can interact with the DICOM Node server.

When remotely connected to a medical center DICOM Node server, Rayvolve LN directly interacts with the DICOM files to output the prediction (potential presence of pulmonary nodules) the original image appears first, followed by the image processed by Rayvolve.

Rayvolve LN does not intend to replace medical doctors. The instructions for use are strictly and systematically transmitted to each user and used to train them on Rayvolve LN's use.

Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided text:

Acceptance Criteria and Device Performance

The document doesn't explicitly list a table of acceptance criteria in the sense of predefined thresholds for performance metrics. Instead, it describes a comparative study where the acceptance criterion is superiority to unaided readers and comparability to a predicate device. The reported device performance is then presented as the outcomes of these studies.

However, we can infer the performance metrics used for evaluation.

Inferred Acceptance Criteria & Reported Device Performance:

Note: The direct "acceptance criteria" are implied by the study's primary and secondary objectives (i.e., improvement over unaided reading and comparability to a predicate device). The tables above synthesize the key performance metrics reported.

Study Details:

1. Sample Sizes and Data Provenance:

• Test Set (Standalone Performance): 2181 radiographs. The data provenance is not explicitly stated in terms of country of origin, nor whether it was retrospective or prospective. It is described as "all the study types and views in the indication for use."
• Test Set (Clinical Data - MRMC Study): 400 cases. These cases were "randomly sampled from the validation dataset used for the standalone performance study," implying they are a subset of the 2181 radiographs mentioned above.
• Training Set: The sample size for the training set is not provided in the document.

2. Number of Experts for Ground Truth & Qualifications:

• Number of Experts: The document does not explicitly state the number of experts used to establish the ground truth for the test set. It mentions "ground truth binary labeling indicating the presence or absence of pulmonary nodules" for the MRMC study but doesn't detail how this ground truth was derived.
• Qualifications of Experts: Not specified.

3. Adjudication Method for the Test Set:

• The adjudication method for establishing ground truth is not explicitly detailed. It merely states "ground truth binary labeling indicating the presence or absence of pulmonary nodules."

4. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:

• Yes, an MRMC study was done.
• Effect Size of Improvement:
  • Reader AUC: Improved from 0.8071 (unaided) to 0.8583 (aided), a difference of 0.0511. (95% CI: 0.0501; 0.0518)
  • Reader Sensitivity (per image): Improved from 0.7975 (unaided) to 0.8935 (aided), a difference of 0.096.
  • Reader Specificity (per image): Improved from 0.8235 (unaided) to 0.8510 (aided), a difference of 0.0275.

5. Standalone Performance (Algorithm Only):

• Yes, a standalone performance assessment was done.
• Reported Metrics:
  • Sensitivity: 0.8847 (95% CI: 0.8638; 0.9028)
  • Specificity: 0.8294 (95% CI: 0.8066; 0.9028)
  • AUC: 0.8408 (95% Bootstrap CI: 0.8272; 0.8548)

6. Type of Ground Truth Used:

• The ground truth for both the standalone and MRMC studies was described as "ground truth binary labeling indicating the presence or absence of pulmonary nodules." It does not specify if this was expert consensus, pathology, or outcomes data. However, the context of detecting nodules on chest radiographs for radiologists implies expert consensus as the most probable method.

7. Sample Size for the Training Set:

• Not provided in the document.

8. How Ground Truth for the Training Set was Established:

• Not provided in the document. The document only mentions that the device uses "deep learning techniques" and "supervised Deep learning," which implies labeled training data was used, but details on its establishment are absent.

Ask a specific question about this device

Better Diagnostics Caries Assist is a radiological, automated, concurrent read, CADe software intended to identify and localize carious lesions on bitewings and periapical radiographs acquired from patients aged 18 years or older. Better Diagnostics Caries assist is indicated for use by board licensed dentists. The device is not intended as a replacement for a complete dentist's review or their clinical judgment that takes into account other relevant information from the image, patient history or actual in vivo clinical assessment.

Better Diagnostics Caries Assist (BDCA) Version 1.0 is a computer-aided detection (CADe) software designed for the automated detection of carious lesions in Bitewings and periapical dental radiographs. This software offers supplementary information to assist clinicians in their diagnosis of potentially carious tooth surfaces. It is important to note that BDCA v1.0 is not meant to replace a comprehensive clinical evaluation by a clinician, which should consider other pertinent information from the image, the patient's medical history and clinical examination. This software is intended for use in identifying carious lesions in permanent teeth of patients who are 18 years or older.

BDCA v1.0 does not make treatment recommendations or provide a diagnosis. Dentists should review images annotated by BDCA v1.0 concurrently with original, unannotated images before making the final diagnosis on a case. BDCA v1.0 is an adjunct tool and does not replace the role of the dentist. The CAD generated output should not be used as the primary interpretation by the dentists. BDCA v1.0 is not designed to detect conditions other than the following: Caries.

BDCA v1.0 comprises four main components:

• Presentation Layer: This component includes "AI Results Screen" a web-based interface (user interface) that allows users to view AI marked annotations. This is a custom code provided by Better Diagnostics AI Corp to Dental PMS customers. User Interface uses Angular.js and node.js technology to show images on the "AI Results Screen". System can process PNG, BMP and JPG format images. All images are converted into JPEG format for processing. Computer Vision Models returns AI annotations and co-ordinates to the business layer. Business layer sends coordinates to the presentation layer and bounding boxes are drawn on the image using custom code written in Angular.js and node.js. Dentists can view, accept or reject the annotations based on his evaluation. Better Diagnostics AI provides UI code to customers e.g. dental practice management software and imaging firms for utilization of BCDA v1.0 software.
• Application Programming Interface (API): APIs are a set of definitions and protocols for building and integrating application software. It's sometimes referred to as a contract between an information provider and an information user. BDCA v1.0 APIs connect the Dental PMS with the business layer. API receives images input from Dental PMS and passes it to the business layer. It also receives annotations and co-ordinates from the business layer and passes it to the presentation layer hosted by Dental PMS.
• Business Layer: Receives image from the API Gateway and passes it to computer vision models. It also receives the bounding boxes coordinates from the model and retrieves images from the cloud storage. It sends all information to the "AI Results screen" to display rectangle bounding boxes.
• Computer Vision Models (CV Models): These models are hosted on a cloud computing platform and are responsible for image processing. They provide a binary indication to determine the presence of carious findings. If carious findings are detected, the software will output the coordinates of the bounding boxes for each finding. If no carious lesions are found, the output will not contain any bounding boxes and will have a message stating "No Suspected: Caries Detected"

AI models have three parts:

• Pre-Processing Module: Standardization of image to specific height and width to maintain consistency for AI model. Finds out the type of image including IOPA, Bitewings or other types. BDCA v 1.0 can only process Bitewings and IOPA images for patients over age 18. All other types of images will be rejected.
• Core Module: This module provides carious lesion annotations and co-ordinates to draw bounding boxes.
• Post-Processing Module: includes cleanup process to remove outliers/incorrect annotations from the images.

Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided FDA 510(k) summary for Better Diagnostics Caries Assist (BDCA) Version 1.0.

1. Acceptance Criteria and Reported Device Performance

The acceptance criteria are not explicitly stated as a pass/fail threshold in a dedicated table, but rather embedded in the "predefined performance goals" mentioned in the standalone testing section. The reported device performance is presented against these implicit goals.

Note: For image-level sensitivity and specificity, the report provides both "Conservative" and "Optimistic" definitions as per FDA guidance. The "Optimistic" sensitivity for IOPA images had a specific goal. For other image-level metrics and conservative sensitivities, while performance is reported as "robust" and exceeding "performance thresholds," a specific numerical goal isn't explicitly listed in the text. However, the reported values consistently exceed the surface-level goals, implying sufficient performance.

2. Sample Size Used for the Test Set and Data Provenance

From the Standalone Testing section:

• BW Images: 614 images (310 with cavities, 304 without). Within these, 15,687 surfaces were examined (585 positively identified with cavities, 15,102 negative).
• IOPA Images: 684 images (367 with cavities, 317 without). Within these, 9,253 surfaces were examined (618 showed positive results for cavities, 8,635 negative).
• Data Provenance: The document does not explicitly state the country of origin. It indicates that "A patient had at most one BW and one IOPA image included in the analysis dataset." The study states that "Twenty-nine United States (US) licensed dentists" participated in the MRMC study, which may suggest the data is from the US, but this is not definitively stated for the standalone test set. The study type is retrospective as existing images were analyzed.

From the Clinical Evaluation-Reader Improvement (MRMC) Study section:

• Total Radiographs: 328 (108 BW and 220 IOPA).
• BW Images with Cavities: 72 out of 108.
• IOPA Images with Cavities: 91 out of 220.
• BW Surfaces with Cavities: 221 out of 2716.
• IOPA Surfaces with Cavities: 160 out of 2967.
• Data Provenance: The document does not explicitly state the country of origin of the data itself, only that US licensed dentists interpreted them. The study setup implies a retrospective analysis of existing radiographs.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

• Number of Experts: Three experienced, licensed dentists.
• Qualifications: Each with over 10 years of professional experience.

4. Adjudication Method for the Test Set

• Adjudication Method: Consensus of two out of three experts (2+1).
  • "Ground truth was determined through the consensus of two out of three experienced, licensed dentists... agreeing on the final labels for analysis when at least two dentists identified a surface as carious."

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Was an MRMC study done? Yes, a multi-reader, multi-case (MRMC) study was conducted.

• Effect Size of Human Readers Improvement with AI vs. Without AI Assistance:

  • AFROC Score (AUC):

    • BW Images: Aided AUC = 0.848, Unaided AUC = 0.806. (Improvement = +0.042)
    • IOPA Images: Aided AUC = 0.845, Unaided AUC = 0.807. (Improvement = +0.038)
    • The differences were statistically significant (p-values < 0.001).

  • Sensitivity at Image Level (Optimistic Definition):

    • BW Images: Aided Sensitivity = 0.890, Unaided Sensitivity = 0.857. (Improvement = +0.033)
    • IOPA Images: Aided Sensitivity = 0.809, Unaided Sensitivity = 0.761. (Improvement = +0.048)

  • Sensitivity at Image Level (Conservative Definition):

    • BW Images: Aided Sensitivity = 0.509, Unaided Sensitivity = 0.444. (Improvement = +0.065)
    • IOPA Images: Aided Sensitivity = 0.619, Unaided Sensitivity = 0.564. (Improvement = +0.055)

  • Specificity at Image Level:

    • BW Images: Aided Specificity = 0.682, Unaided Specificity = 0.634. (Improvement = +0.048)
    • IOPA Images: Aided Specificity = 0.844, Unaided Specificity = 0.813. (Improvement = +0.031). This difference was statistically significant.

  • Sensitivity at Surface Level:

    • BW Images: Aided Sensitivity = 0.763, Unaided Sensitivity = 0.707. (Mean increase = +0.056 - statistically significant).
    • IOPA Images: Aided Sensitivity = 0.746, Unaided Sensitivity = 0.691. (Mean increase = +0.055 - statistically significant).

  • Specificity at Surface Level:

    • BW Images: Aided Specificity = 0.980, Unaided Specificity = 0.974. (Slight but significant improvement).
    • IOPA Images: Aided Specificity = 0.983, Unaided Specificity = 0.979. (Slightly improved).

6. Standalone Performance (Algorithm Only without Human-in-the-Loop)

• Was a standalone study done? Yes, standalone testing was performed.
• Performance Metrics (from Section 8.a.i "Standalone Testing"):
  • BW Surface Level: Sensitivity = 89.2%, Specificity = 99.5%
  • IOPA Surface Level: Sensitivity = 88.2%, Specificity = 99.1%
  • BW Image Level (Conservative): Sensitivity = 81.0%, Specificity = 98.4%
  • BW Image Level (Optimistic): Sensitivity = 91.9%, Specificity = 98.4%
  • IOPA Image Level (Conservative): Sensitivity = 83.1%, Specificity = 98.4%
  • IOPA Image Level (Optimistic): Sensitivity = 91.8%, Specificity = 98.4%

7. Type of Ground Truth Used

• Type of Ground Truth: Expert consensus. Specifically, the consensus of two out of three experienced, licensed dentists.

8. Sample Size for the Training Set

The document does not explicitly state the sample size for the training set. It only discusses the test sets for standalone and clinical performance evaluations.

9. How the Ground Truth for the Training Set Was Established

The document does not provide details on how the ground truth for the training set was established. It only describes the ground truth establishment for the test sets used in the standalone and MRMC studies.

Ask a specific question about this device

ChestView US is a radiological Computer-Assisted Detection (CADe) software device that analyzes frontal and lateral chest radiographs of patients presenting with symptoms (e.g. dyspnea, cough, pain) or suspected for findings related to regions of interest (ROIs) in the lungs, airways, mediastinum/hila and pleural space. The device uses machine learning techniques to identify and produces boxes around the ROIs. The boxes are labeled with one of the following radiographic findings: Nodule, Pleural space abnormality, Mediastinum/Hila abnormality, and Consolidation.

ChestView US is intended for use as a concurrent reading aid for radiologists and emergency medicine physicians. It does not replace the role of radiologists and emergency medicine physicians or of other diagnostic testing in the standard of care. ChestView US is for prescription use only and is indicated for adults only.

ChestView US is a radiological Computer-Assisted Detection (CADe) software device intended to analyze frontal and lateral chest radiographs for suspicious regions of interest (ROIs): Nodule, Consolidation, Pleural Space Abnormality and Mediastinum/Hila Abnormality.

The nodule ROI category was developed from images with focal nonlinear opacity with a generally spherical shape situated in the pulmonary interstitium.

The consolidation ROI category was developed from images with area of increased attenuation of lung parenchyma due to the replacement of air in the alveoli.

The pleural space abnormality ROI category was developed from images with:

• Pleural Effusion that is an abnormal presence of fluid in the pleural space
• Pneumothorax that is an abnormal presence of air or gas in the pleural space that separates the parietal and the visceral pleura

The mediastinum/hila abnormality ROI category was developed from images with enlargement of the mediastinum or the hilar region with a deformation of its contours.

ChestView US can be deployed on cloud and be connected to several computing platforms and X-ray imaging platforms such as radiographic systems, or PACS. More precisely, ChestView US can be deployed in the cloud connected to a DICOM Source/Destination with a DICOM Viewer, i.e. a PACS.

After the acquisition of the radiographs on the patient and their storage in the DICOM Source, the radiographs are automatically received by ChestView US from the user's DICOM Source through intermediate DICOM node(s) (for example, a specific Gateway, or a dedicated API). The DICOM Source can be the user's image storage system (for example, the Picture Archiving and Communication System, or PACS), or other radiological equipment (for example X-ray systems).

Once received by ChestView US, the radiographs are automatically processed by the AI algorithm to identify regions of interest. Based on the processing result, ChestView US generates result files in DICOM format. These result files consist of annotated images with boxes drawn around the regions of interest on a copy of all images (as an overlay). ChestView US does not alter the original images, nor does it change the order of original images or delete any image from the DICOM Source.

Once available, the result files are sent by ChestView US to the DICOM Destination through the same intermediate DICOM node(s). Similar to the DICOM Source, the DICOM Destination can be the user's image storage system (for example, the Picture Archiving and Communication System, or PACS), or other radiological equipment (for example X-ray systems). The DICOM Source and the DICOM Destination are not necessarily identical.

The DICOM Destination can be used to visualize the result files provided by ChestView US or to transfer the results to another DICOM host for visualization. The users are them as a concurrent reading aid to provide their diagnosis.

For each exam analyzed by ChestView US, a DICOM Secondary Capture is generated.

If any ROI is detected by ChestView US, the output DICOM image includes a copy of the original images of the study and the following information:

• Above the images, a header with the text "CHESTVIEW ROI" and the list of the findings detected in the image.
• Around the ROI(s), a bounding box with a solid or dotted line depending on the confidence of the algorithm and the type of ROI written above the box:
  • Dotted-line Bounding Box: Identified region of interest when the confidence degree of the AI algorithm associated with the possible finding is above "high-sensitivity operating point" and below "high specificity operating point" displayed as a dotted bounding box around the area of interest.
  • Solid-line Bounding Box: Identified region of interest when the confidence degree of the AI algorithm associated with the finding is above "high-specificity operating point" displayed as a solid bounding box around the area of interest.
• Below the images, a footer with:
  • The scope of ChestView US to allow the user to always have available the list of ROI type that are in the indications for use of the device and avoid any risk of confusion or misinterpretation of the types of ROI detected by ChestView US.
  • The total number of regions of interest identified by ChestView US on the exam (sum of solid-line and dotted-line bounding boxes)

If no ROI is detected by ChestView US, the output DICOM image includes a copy of the original images of the study and the text "NO CHESTVIEW ROI" with the scope of ChestView US to allow the user to always have available the list of ROI type that are in the indications for use of the device and avoid any risk of confusion or misinterpretation of the types of ROI detected by ChestView US. Finally, if the processing of the exam by ChestView US is not possible because it is outside the indications for use of the device or some information is missing to allow the processing, the output DICOM image includes a copy of the original images of the study and, in a header, the text "OUT OF SCOPE" and a caution message explaining the reason why no result was provided by the device.

Here's a breakdown of the acceptance criteria and study details for ChestView US:

1. Table of Acceptance Criteria and Reported Device Performance

Standalone Performance (ChestView US)

MRMC Study Acceptance Criteria and Reported Performance (Improvement with AI Aid)

The acceptance criteria for the standalone performance are implied by the presentation of high AUC, sensitivity, and specificity metrics, suggesting that these values met an internal performance threshold deemed acceptable by the manufacturer and the FDA for market clearance. The MRMC study explicitly states that "Reader AUC estimates for both specialties significantly improved for all four categories (p-values < 0.001)," which serves as the acceptance criterion for the human-in-the-loop performance.

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size for Standalone Test Set: 3,884 chest radiograph cases.
• Data Provenance (Standalone and MRMC): "representative of the intended use population." While the document does not explicitly state the country of origin or whether the data was retrospective or prospective, most such studies use retrospective data from diverse patient populations to represent real-world clinical scenarios. The use of "U.S. board-certified radiologists" for ground truth suggests U.S. data sources are likely.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications

• Number of Experts: A "panel of U.S. board-certified radiologists" was used. The exact number is not specified.
• Qualifications of Experts: U.S. board-certified radiologists. No specific experience levels (e.g., "10 years of experience") are mentioned.

4. Adjudication Method for the Test Set

The document does not explicitly state the adjudication method (e.g., 2+1, 3+1). It only mentions that a "panel of U.S. board-certified radiologists" assessed the presence or absence of ROIs. This typically implies a consensus-based approach, but the specific mechanics (e.g., how disagreements were resolved) are not provided.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Yes, an MRMC comparative effectiveness study was done.
• Effect Size of Human Readers Improvement with AI vs. without AI Assistance (Difference in AUC):
  • Nodule Detection:
    • Emergency Medicine Physicians: 0.136 (95% CI [0.107, 0.17])
    • Radiologists: 0.038 (95% CI [0.026, 0.052])
  • Mediastinum/Hila Abnormality Detection:
    • Emergency Medicine Physicians: 0.158 (95% CI [0.14, 0.178])
    • Radiologists: 0.057 (95% CI [0.039, 0.077])
  • Consolidation Detection:
    • Emergency Medicine Physicians: 0.099 (95% CI [0.083, 0.116])
    • Radiologists: 0.059 (95% CI [0.038, 0.079])
  • Pleural Space Abnormality Detection:
    • Emergency Medicine Physicians: 0.127 (95% CI [0.078, 0.18])
    • Radiologists: 0.034 (95% CI [0.019, 0.049])

6. Standalone (Algorithm Only without Human-in-the-Loop Performance)

• Yes, a standalone clinical performance study was done. The results are presented in Table 2 (AUC) and Table 3 (Specificity/Sensitivity).

7. Type of Ground Truth Used

• Expert Consensus: The ground truth for both the standalone and MRMC studies was established by a "panel of U.S. board-certified radiologists" who assessed the presence or absence of ROIs.

8. Sample Size for the Training Set

The document does not specify the sample size used for the training set. It only mentions the "standalone clinical performance study on 3,884 chest radiograph cases representative of the intended use population" for testing.

9. How the Ground Truth for the Training Set Was Established

The document does not provide details on how the ground truth for the training set was established. It only describes the establishment of ground truth for the test set by a panel of U.S. board-certified radiologists.

Ask a specific question about this device