Search Results

Tyto Insights for Eardrum Bulging Detection is an over-the-counter web-based AI-enabled software analytics device intended to automatically detect the bulging of the eardrum in Otoscopic video recordings acquired by the compatible electronic otoscope, as identified in the product labeling, in patients aged 6 months and older. The device is not intended for diagnosis. A healthcare provider's advice is required to understand the meaning of the Tyto Insights for Eardrum Bulging Detection result. Healthcare providers should consider the device result in conjunction with otoscopic recordings and other relevant patient data. The device is not intended to detect other abnormal otoscopic findings.

Tyto Insights for Eardrum Bulging Detection is a web-based AI enabled software analytics device intended to automatically detect the bulging of the Eardrum in Otoscopic video recordings acquired by the compatible Tyto Otoscope in pediatrics and adults. The device is intended to process recordings only from the compatible Tyto Otoscope (Class I medical device, exempt from premarket notification procedures), manufactured by TytoCare Ltd. The acquisition of the video otoscopy data (recordings) is carried out by a professional user in a clinical environment or by a lay user in a non-clinical environment.

Bulging is a medical characteristic of the tympanic membrane that is commonly used for the diagnosis of Acute Otitis Media, but is also observed in ear trauma, barotrauma, bullous myringitis, etc. Therefore, the device can provide an aid to diagnosis when used in conjunction with otoscopic ear examinations and when considering all patient symptoms and medical history.

The Tyto Insights for Eardrum Bulging Detection system is a processing service and has an interface with a Telehealth system which is not part of this product. The Tyto Insights for Eardrum Bulging Detection device is responsible for the data transfer, store and read-only display of device output (otoscopic video analysis results).

Here's a breakdown of the acceptance criteria and the study that demonstrates the device's adherence to those criteria, based on the provided document:

Acceptance Criteria and Device Performance Study for Tyto Insights for Eardrum Bulging Detection

1. Table of Acceptance Criteria and Reported Device Performance

The primary acceptance criteria for the Tyto Insights for Eardrum Bulging Detection device were defined in a comparative effectiveness study, comparing the device's performance against a panel of non-ENT clinical readers. The criteria focused on demonstrating superiority in both sensitivity and specificity.

Acceptance Criteria for Primary Endpoint (Comparative Clinical Performance):

Standalone Performance (Against Expert GT)

While not explicitly stated as acceptance criteria in the same manner as the comparative study, the standalone performance against the expert ground truth provides the baseline for the device's accuracy.

Note: PPV and NPV are calculated based on Bayes theorem to account for eardrum prevalence of 8.06%.

Secondary Endpoint (Eardrum Quality Filter Performance):

No success criteria was applied to the secondary endpoint.

2. Sample Size and Data Provenance for Test Set

• Sample Size for Test Set: 857 otoscopic recordings (857 patients).
• Data Provenance: Retrospective validation dataset sourced from real-world use of the Tyto Otoscope recordings. While specific countries are not mentioned for the test set, the training data originates from the TytoCare RWD (Real-World Data) Database, implying a similar origin of real-world telehealth use. The comparative human performance study specifically states the non-ENT clinicians were U.S.-licensed, suggesting the data may represent typical U.S. patient populations.

3. Number of Experts and Qualifications for Test Set Ground Truth

• Number of Experts: For the standalone performance (bench study) against expert ground truth, the document states the external ground truth was established by "a panel of ENTs." For the comparative clinical performance study (human reader comparison), the ground truth for the device's standalone performance within that study was also established by a panel of ENTs. The exact number of ENTs for the test set ground truth is not explicitly stated, but the training set ground truth process provides insight (see point 9).
• Qualifications of Experts: ENT (Ear Nose Throat) specialists (otolaryngologists) for establishing the primary ground truth. For the comparative study, the human reader panel consisted of two pediatricians and one general practitioner (U.S.-licensed), who were chosen to represent the intended clinical users of the device in U.S. primary care settings.

4. Adjudication Method for the Test Set

The document does not explicitly state the adjudication method (e.g., 2+1, 3+1) for establishing the ground truth of the test set. However, for the training set, it describes a process where 3 ENTs annotated data, and the "final ground truth is determined by majority vote among the three truthers," suggesting a 3+1 (majority vote) approach was likely used for expert ground truth establishment for the test set as well, or at least a consensus-based approach among a panel.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Was it done? Yes, a comparative effectiveness study was performed, comparing the device's standalone performance against a panel of human readers (two pediatricians and one general practitioner). This implicitly functions similar to an "AI vs. human" comparative study, although the AI was standalone and not assistive.

• Effect Size (Device vs. Human Readers):

  • Sensitivity Improvement: The device had a sensitivity improvement of 0.1989 (LCI: 0.1691, UCI: 0.2315) compared to the human readers. (Device Sensitivity: 0.9231 vs. Readers Sensitivity: 0.7242)
  • Specificity Improvement: The device had a specificity improvement of 0.2087 (LCI: 0.1756, UCI: 0.2442) compared to the human readers. (Device Specificity: 0.9463 vs. Readers Specificity: 0.7325)
  • PPV Improvement (Prevalence-adjusted): 0.4038 (0.3161 – 0.5070) in favor of the device.
  • NPV Improvement (Prevalence-adjusted): 0.0250 (0.0215 – 0.0288) in favor of the device.

  The study demonstrated that the device exhibited "higher sensitivity and specificity" compared to the non-ENT reader panel. This "effect size" highlights how much better the AI performed alone compared to the human readers alone for the detection task. It's not an AI-assisted human performance metric.

6. Standalone Performance (Algorithm Only)

• Was it done? Yes. The "Performance Testing - Bench" section (Page 8) explicitly details the standalone performance and accuracy (Sensitivity, Specificity, PPV, NPV) of the Tyto Insights for Eardrum Bulging Detection device.

7. Type of Ground Truth Used

• Expert Consensus: For both training and validation (test) sets, the ground truth for eardrum bulging detection was established by a panel of ENT (Ear Nose Throat) specialists (otolaryngologists) through a consensus or majority vote process.

8. Sample Size for the Training Set

• Total Training Dataset: 3,102 otoscopic recordings.
  • Bulging Positive: 606 recordings
  • Bulging Negative: 1,510 recordings
  • Tympanic Membrane Position Not Sufficiently Captured: 986 recordings

9. How the Ground Truth for the Training Set Was Established

The ground truth for the training set was established through a multi-step process involving multiple ENT specialists:

• Step 1 (Initial Screening): Recordings were systematically annotated by one ENT specialist into "Ear Abnormal Recording" and "Ear Normal Recording." This step aimed to address the low prevalence of bulging in real-world data by enriching the "abnormal" pool.
• Step 2 (Detailed Annotation & Majority Vote): All data marked "Abnormal" from Step 1, plus a randomly selected portion of "Normal" data, were then annotated by three ENTs. They categorized recordings as "Bulging Positive," "Bulging Negative," or "Tympanic Membrane Position was not sufficiently Captured." The final ground truth for these recordings was determined by a majority vote among the three ENTs.
• Step 3 (Frame-level Annotation): Randomly selected recordings from Step 2 were further annotated on a frame level by an ENT. This highly granular annotation was used to train both the Eardrum Bulging Detection and Quality Filter Algorithms.

This comprehensive approach leveraged multiple expert opinions and a tiered annotation process to establish robust ground truth labels for training the AI algorithms.

Ask a specific question about this device