(138 days)
The Tyto Insights for Wheeze Detection is an over the-counter artificial intelligence (Al) enabled decision support software system used in the evaluation of lung sounds in adults and pediatrics (2 years and older). It automatically analyzes the acoustic signal of the lung as recorded by the FDA cleared compatible Tyto Stethoscope and identifies recordings where a specific abnormal lung sound suggestive of "Wheeze" is suspected. It is not intended to detect other abnormal or normal lung sounds. A licensed health care professional's advice is required to understand the meaning of the Tyto Insights for Wheeze Detection result. Healthcare providers should consider the device result in conjunction with recording and other relevant patient data.
The Tyto Insights for Wheeze Detection is a web-based (AI) enabled software system designed to aid in the clinical assessment of lungs auscultation sound data by analyzing recorded lung sounds to determine whether a Wheeze is detected within the recorded sound data.
The Tyto Insights for Wheeze Detection Software is intended to process recordings from the FDA-cleared compatible Tyto Stethoscope (Tyto Stethoscope, K181612). The acquisition of the acoustic data (recordings) is carried out by a professional user in a clinical environment or by a lay- user in a non-medical environment, in compliance with the labeling of the Tyto Stethoscope. The system is composed of the following sub-systems:
-
- The Tyto Insights for Wheeze Detection Application Server (APS) communicates with the Tyto Insights for Wheeze Detection Algorithm Server (ALS) and implements an application programming interface (API) for communication with the telehealth server.
-
- The Tyto Insights for Wheeze Detection Algorithm Server (ALS) receives an audio file as input and returns an analysis result of positive or negative regarding whether a wheeze was detected as output.
- The Tyto Insights for Wheeze Detection Web Server (WBS) provides a graphic 3. indication whether a wheeze is detected in the recording. It can be utilized both in patient and clinician side.
All the software subsystems (servers and storage) are hosted in the cloud and communicate through IP network.
Here's a breakdown of the acceptance criteria and study details for the "Tyto Insights for Wheeze Detection" device, based on the provided FDA 510(k) summary:
1. Table of Acceptance Criteria and Reported Device Performance
The document doesn't explicitly state "acceptance criteria" as a pass/fail threshold, but rather focuses on comparing the device's performance to its predicate and demonstrating non-inferiority. The key performance metrics are sensitivity, specificity, and Area Under the Curve (AUC).
| Performance Metric | Acceptance Criterion (Implicit) | Reported Device Performance (Tyto Insights for Wheeze Detection) |
|---|---|---|
| Primary Endpoint | Non-inferiority to the predicate device (TytoCare Lung Sounds Analyzer K221614) based on AUC. Specifically, the lower bound of the 95% two-sided CI for the difference in AUCs (Insights - Analyzer) must be higher than a non-inferiority margin of -0.05. | Insights - Analyzer: 0.0570 (LCI: 0.0289, UCI: 0.0917) |
| Standalone Sensitivity | Not explicitly defined as a pass/fail criterion in the document, but provided as a secondary endpoint. | 0.54 (95% CI: 0.43 - 0.65) |
| Standalone Specificity | Not explicitly defined as a pass/fail criterion in the document, but provided as a secondary endpoint. | 0.98 (95% CI: 0.97 - 0.99) |
| Positive Predictive Value (PPV) | Not explicitly defined as a pass/fail criterion. | 0.72 (95% CI: 0.48 - 0.89) |
| Negative Predictive Value (NPV) | Not explicitly defined as a pass/fail criterion. | 0.97 (95% CI: 0.97 - 0.98) |
| Overall Diagnostic Accuracy | The document states "The overall diagnostic performance show overall good diagnostic accuracy." This is a qualitative statement summarizing the quantitative metrics. | Demonstrated by the above metrics and non-inferiority of AUC. |
Conclusion regarding acceptance criteria: The device met the primary endpoint of non-inferiority to the predicate device based on AUC, as the lower bound of the 95% CI for the difference in AUCs (0.0289) was well above the non-inferiority margin of -0.05.
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size for Test Set: 371 recordings, corresponding to 359 patients. (86 recordings were Wheeze positive and 285 negative).
- Data Provenance: Retrospective validation dataset sourced from real-world use of the FDA cleared compatible Tyto Stethoscope. The country of origin is not explicitly stated, but Tyto Care Ltd. is based in Israel.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
- Number of Experts: Three blinded experienced Pulmonologists.
- Qualifications of Experts: Described as "experienced Pulmonologists." Specific details like years of experience are not provided.
4. Adjudication Method for the Test Set
- Adjudication Method: Binary ground truth was determined by a majority vote of the three blinded Pulmonologists. This implies a "3+1" approach where if at least two out of three experts agreed on the presence or absence of wheeze, that was considered the ground truth.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done
- There is no indication of a Multi-Reader Multi-Case (MRMC) comparative effectiveness study being performed with human readers with and without AI assistance. The study focused on the stand-alone performance of the AI algorithm and its non-inferiority to a predicate device's algorithm, not on the improvement of human readers with AI assistance.
6. If a Standalone Study (Algorithm Only Without Human-in-the-Loop Performance) was Done
- Yes, a standalone study was done. The performance evaluation section explicitly states: "For the characterization of the stand-alone accuracy, the automated binary result of the software has been compared to ground truth and specificity and sensitivity were calculated." and "The primary end point was area under the Curve (AUC) for the detection of wheezes by the proposed device compared to the TytoCare Lung Sounds Analyzer (K221614)".
7. The Type of Ground Truth Used
- Type of Ground Truth: Expert consensus. Specifically, the binary ground truth was determined by a majority vote of three blinded experienced Pulmonologists.
8. The Sample Size for the Training Set
- The sample size for the training set is not explicitly provided in the document. The text states: "The AI Algorithm was trained with recordings acquired by the real-world use of the compatible Tyto Stethoscope."
9. How the Ground Truth for the Training Set Was Established
- The document states that the AI algorithm was "trained with recordings acquired by the real-world use of the compatible Tyto Stethoscope." However, it does not explicitly detail how the ground truth for this training set was established. It can be inferred that a similar expert labeling process might have been used, but this is not confirmed in the provided text.
§ 868.1900 Diagnostic pulmonary-function interpretation calculator.
(a)
Identification. A diagnostic pulmonary-function interpretation calculator is a device that interprets pulmonary study data to determine clinical significance of pulmonary-function values.(b)
Classification. Class II (performance standards).