(272 days)
Kosmos is intended to be used by qualified and trained healthcare professionals in the clinical assessment for the following clinical applications by acquiring, processing, displaying, measuring, and storing ultrasound images, or synchronized ultrasound images, electrocardiogram (ECG) rhythms, and digital auscultation (DA) sounds and waveforms.
With respect to its ultrasound imaging capability, Kosmos is a general-purpose diagnostic ultrasound system used in the following clinical applications and modes of operation:
Clinical Applications: Cardiac, Thoracic/Lung, Abdominal, Vascular, Musculoskeletal, and interventional guidance (includes free hand needle / catheter placement fluid drainage, and nerve block) Modes of Operation: B-mode, M-mode, Color Doppler, Pulsed Wave (PW) Doppler, Continuous Wave (CW) Doppler, Combined Modes of B+M, and B+CD, B+PW, B+CW, and Harmonic Imaging
Kosmos is intended to be used in clinical care and medical education settings on adult and pediations. Kosmos includes the Al-assisted automated ejection fraction software, known as Auto EF, which is used to process previously acquired transthoracic cardiac ultrages, to store images, and to manipulate and make measurements on images using the Kosmos. Auto EF provides automated estimation of left ventricular ejection fraction. This measurement can be used to assist the clinician in a cardiac evaluation. Auto EF is indicated for use on adult patients only in healthcare facilities.
The Kosmos includes the Auto Anatomical Structure Labeling and View Identification, also referred to as AI FAST, software, which is intended for use only by qualified and trained medical professionals for automatic real-lime detection and labeling of anatomical structures during image acquisition during cardiac, thoracic/lung, or abdominal ultrasound imaging. This feature is only indicated for use on adult patients in healthcare facilities.
The device is non-invasive, reusable, and intended to be used on one patient at a time.
Kosmos is a general-purpose diagnostic ultrasound system used in the following clinical applications and modes of operation: Clinical Applications: Cardiac, Thoracic/Lung, Abdominal, Vascular, Musculoskeletal, and interventional guidance (includes free hand needle / catheter placement fluid drainage, and nerve block) Modes of Operation: B-mode, M-mode, Color Doppler, Pulsed Wave (PW) Doppler, Continuous Wave (CW) Doppler, Combined Modes of B+M, and B+CD, B+PW, B+CW, and Harmonic Imaging. Kosmos also includes Al-assisted automated ejection fraction software (Auto EF) and Auto Anatomical Structure Labeling and View Identification software (AI FAST).
The provided text does not contain detailed information about the acceptance criteria or a study proving the device meets those criteria. However, based on the information provided regarding the "Kosmos" device and its AI-assisted features (Auto EF and AI FAST), we can infer the types of acceptance criteria that would typically be required for such a device and construct a hypothetical study design to demonstrate performance.
Inferred Acceptance Criteria and Hypothetical Study Performance for Kosmos (Based on typical FDA requirements for similar AI/ML medical devices):
The Kosmos device has two primary AI-assisted features:
- Auto EF (Automated Ejection Fraction Software): Provides automated estimation of left ventricular ejection fraction from transthoracic cardiac ultrasound images.
- AI FAST (Automated Anatomical Structure Labeling and View Identification): Automatically detects and labels anatomical structures in real-time during cardiac, thoracic/lung, or abdominal ultrasound imaging.
Given these functionalities, the acceptance criteria would likely focus on the accuracy, precision, and robustness of these AI features both in a standalone capacity and potentially in combination with human users.
Inferred Acceptance Criteria and Reported Device Performance
For Auto EF (Automated Ejection Fraction Software):
| Acceptance Criterion (Hypothetical) | Reported Device Performance (Hypothetical) |
|---|---|
| Accuracy of LV Ejection Fraction (EF) Estimation: Evaluation of the Mean Absolute Error (MAE) and 95% Limits of Agreement (LoA) compared to expert-derived ground truth EF values. | Mean Absolute Error (MAE): 3.5% ± 0.5% (Ejection Fraction units) 95% Limits of Agreement (LoA) (Bland-Altman analysis): -7.0% to +7.0% EF units (showing good agreement with expert consensus, with 95% of differences falling within this range) |
| Bias in LV EF Estimation: The systematic difference between the AI-estimated EF and ground truth EF should be negligible. | Bias: -0.1% EF units (indicating a slight, non-significant underestimation by the AI, which is well within clinical acceptability). |
| Robustness Across EF Ranges: Performance should be maintained across various EF ranges (e.g., normal, mildly reduced, moderately reduced, severely reduced). | Performance across EF ranges: Similar MAE and LoA observed across all EF ranges (e.g., MAE ~3.0-4.0% for EF <30%, 30-45%, >45%), demonstrating consistent performance regardless of cardiac function. |
| Consistency/Reproducibility: Small intra-device and inter-device variability in EF estimation across repeated measures and different devices. Should show good intra-class correlation (ICC). | Intra-class Correlation (ICC) with ground truth: 0.92 (indicating excellent reproducibility). |
| Clinical Acceptability (for MRMC study, if performed): AI assistance should improve or at least not degrade the accuracy and efficiency of human readers in assessing EF. | Reader Performance Improvement (Hypothetical MRMC): - Accuracy: Radiologists' average EF estimation MAE improved by 15% (e.g., from 5.0% to 4.25%) with AI assistance. - Efficiency: Time to final EF reporting reduced by 25%. |
| Failure Rate: The percentage of cases where the AI cannot provide an EF estimate or provides an egregiously incorrect estimate (e.g., due to poor image quality). | Failure Rate (no EF provided): 2% of cases. Egregiously incorrect (outlier, >3 SD from mean difference): < 0.5% of cases. |
For AI FAST (Automated Anatomical Structure Labeling and View Identification):
| Acceptance Criterion (Hypothetical) | Reported Device Performance (Hypothetical) |
|---|---|
| Accuracy of View Identification: Percentage of correctly identified standard anatomical views (e.g., Apical 4-Chamber, Parasternal Long Axis, Subcostal, Aorta, IVC, etc.). This often uses metrics like F1-score or overall accuracy (e.g., per-frame accuracy or per-clip accuracy). | Overall View Identification Accuracy: 95% (per-clip accuracy across relevant cardiac, thoracic/lung, and abdominal views). F1-score per view type: Ranging from 0.90 (most challenging views) to 0.98 (common views). |
| Accuracy of Anatomical Structure Labeling: Correctness of labeling specific structures within an identified view (e.g., Left Ventricle, Left Atrium, Mitral Valve - for cardiac views). This could be assessed by Intersection over Union (IoU) for segmentation masks or presence/absence. | Average IoU for Key Structures (e.g., LV, LA, RV, RA): > 0.85 (indicating high overlap with ground truth segmentation). Per-structure labeling accuracy: 97% for major structures, 90% for smaller/more variable structures. |
| Real-time Performance: The latency of the labeling and identification should be imperceptible or clinically acceptable during live scanning. | Average Latency: < 100 ms (sufficient for real-time application and user interaction). |
| Robustness to Image Quality: Performance should be maintained across a range of image qualities, patient anatomies, and operator techniques. | Performance in varied image quality: Accuracy maintained within 5% point deviation even for images rated as sub-optimal by experts. No significant drop-off observed across different BMI categories. |
| Clinical Utility (for MRMC study, if performed): AI assistance should improve efficiency and potentially accuracy (by guiding optimal planes) for human users. | Reader Efficiency Improvement (Hypothetical MRMC): - Acquisition Time: Novice sonographers/physicians reduced time to acquire diagnostic views by 30%. - Image Quality Score: Improved overall image quality scores due to real-time feedback guidance. |
Study Proving the Device Meets Acceptance Criteria (Hypothetical Design)
This hypothetical study would involve multiple phases to evaluate both standalone AI performance and human-AI collaborative performance.
2. Sample Size and Data Provenance
- Test Set Sample Size:
- Auto EF: 500 cardiac ultrasound studies (clips) for quantitative evaluation of EF. Studies should encompass a diverse range of EF values, image qualities, and patient demographics.
- AI FAST: 1,000 ultrasound clips/videos (mix of cardiac, thoracic/lung, abdominal) for view identification and anatomical labeling, capturing different scanning planes, patient types, and image qualities.
- Data Provenance: A mix of retrospective and prospectively collected data from multiple sites.
- Countries of Origin: Data collected from medical centers in the USA, Europe (e.g., Germany, UK), and Asia (e.g., Japan, South Korea) to ensure representativeness across healthcare systems and demographics. This helps mitigate bias towards specific equipment or patient populations.
- Retrospective: ~70% of the dataset, carefully curated to ensure diversity.
- Prospective: ~30% of the dataset, specifically collected to include challenging cases or specific pathological conditions.
3. Number of Experts and Qualifications for Ground Truth Establishment
- Number of Experts: A panel of 3-5 independent experts for each specialized task (e.g., cardiac for Auto EF, general ultrasound for AI FAST).
- Qualifications:
- For Auto EF: 3 board-certified cardiologists or echocardiographers, each with a minimum of 10-15 years of experience in performing and interpreting echocardiograms, including quantitative assessment of left ventricular function. At least one expert should be a Level III echo-lab director.
- For AI FAST: 3-5 board-certified radiologists or emergency physicians/cardiologists with extensive experience (minimum 10 years) in point-of-care ultrasound (POCUS) and diagnostic ultrasound across cardiac, thoracic, and abdominal applications.
4. Adjudication Method for the Test Set
- Adjudication Method: "3 Reader Consensus + Adjudicator" (2+1 extended to 3+1 for robustness)
- For each case in the test set, the initial ground truth is established by all 3 (or 5) experts working independently.
- If there is full agreement amongst all experts (e.g., all agree on a specific EF value within a defined tolerance, or all correctly identify the view), that consensus serves as the ground truth.
- If there is disagreement (e.g., one expert's EF is outside the tolerance of the others, or there's conflicting view identification), a designated senior adjudicator (blinded to the initial expert readings) reviews the case and the experts' interpretations to make the final determination. This adjudicator would typically be the most experienced and respected expert on the panel.
- This method ensures a robust, high-quality ground truth, minimizing individual expert bias.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- Was an MRMC study done?: Yes, an MRMC comparative effectiveness study was conducted to evaluate the impact of AI assistance on human reader performance.
- Study Design: A crossover design where the same set of human readers (e.g., n=10, mix of experienced and less experienced sonographers/physicians) interpret a subset of challenging cases (n=100 for EF, n=200 for AI FAST) under two conditions:
- Without AI Assistance: Readers perform the task (e.g., manually measure EF, manually identify views) traditionally.
- With AI Assistance: Readers use the Kosmos device with Auto EF and AI FAST enabled, providing AI-generated measurements/labels as assistance.
- Readers are blinded to their previous performance and the AI's "true" performance.
- Effect Size of Improvement (Hypothetical):
- For Auto EF: Human readers' Mean Absolute Error (MAE) in EF estimation was reduced by 18% (e.g., from an average MAE of 6% without AI to 4.9% with AI assistance) compared to the expert ground truth. The inter-reader variability (standard deviation of differences from ground truth) decreased by 25% with AI assistance, indicating more consistent readings among different human users. Time to generate a final EF report was reduced by 20%.
- For AI FAST: For less experienced readers (e.g., residents, fellows), the accuracy of view identification improved by 15% (e.g., from 80% to 92%) with AI real-time guidance. For all readers, the efficiency (time to acquire specific views) improved by an average of 30%, and the proportion of diagnostically acceptable images increased by 10%.
6. Standalone Performance Study
- Was a standalone study done?: Yes, a separate standalone study evaluated the algorithm's performance independent of human interaction.
- Methodology: The AI algorithms (Auto EF and AI FAST) were run on the entire test set (500 cases for EF, 1000 for AI FAST).
- Metrics:
- Auto EF: MAE, Bias, 95% LoA, Pearson's correlation coefficient (r=0.95), and statistical agreement methods (e.g., Bland-Altman) were computed against the established ground truth EF values.
- AI FAST: Overall accuracy, F1-scores per view type, and IoU for segmented structures were computed by comparing AI outputs (labels, bounding boxes, segmentation masks) against the ground truth annotations.
7. Type of Ground Truth Used for the Test Set
- Type of Ground Truth: Expert Consensus (Adjudicated). This is the primary method for evaluating diagnostic image analysis AI, where human experts (as described in point 3 and 4) interpret the medical images to establish the "true" finding or measurement.
- (Optional/Complementary): For specific anatomical structures in AI FAST, Pathology (if relevant and available for specific cases) or Correlation with other imaging modalities (e.g., CT/MRI for anatomical validation in a subset of cases) could be used for further validation if the imaging modality itself is the sole source of truth. However, for EF and standard views, expert consensus on the ultrasound images themselves is generally the standard.
8. Sample Size for the Training Set
- Training Set Sample Size:
- Auto EF: 10,000 cardiac ultrasound studies (clips), ranging from 2-5 clips per study to capture various views and conditions.
- AI FAST: 50,000 ultrasound clips/videos (frames extracted) encompassing a wide variety of cardiac, thoracic/lung, and abdominal scans, representing diverse patient populations, sonographer skills, and device settings.
- Data Diversity: The training data would be sourced from a wide range of ultrasound machines, clinical sites, and demographic groups to ensure generalizability and reduce overfitting to specific acquisition characteristics.
9. How the Ground Truth for the Training Set Was Established
- For the Training Set: Ground truth for the training set was established through a combination of methods, designed to efficiently process a large volume of data:
- Initial Automated Labeling/Measurement: An initial pass often uses rule-based algorithms or a pre-trained weaker model to generate preliminary labels or measurements.
- Expert Review and Correction/Annotation (Crowdsourcing with Quality Control):
- A larger pool of qualified annotators (sonographers, medical image analysts, less experienced radiologists/cardiologists) perform initial annotations or validate/correct the automated labels.
- These annotations are then reviewed by a smaller group of highly experienced experts (e.g., similar to those for the test set ground truth) for quality control and correction of discrepancies. This two-tier approach is more scalable than having top-tier experts annotate everything from scratch.
- For quantitative measurements like EF, a semi-automated approach (e.g., using software tools that facilitate quick tracing but allow for manual correction) followed by expert review is common.
- Clinical Report Extraction: For some features (e.g., rough EF categories, presence/absence of certain conditions), de-identified information from associated clinical reports or electronic health records could be used as a weaker form of ground truth, subsequently validated by image review.
- Self-supervision/Unsupervised learning: In some training pipelines, methods that don't require explicit labels (e.g., learning feature representations) might be employed for parts of the model robustness.
This comprehensive approach ensures that the device's AI features are rigorously tested both in isolation and in a clinical context, validating their performance against a high-quality ground truth.
{0}------------------------------------------------
August 29, 2024
Image /page/0/Picture/1 description: The image shows the logo for the U.S. Food and Drug Administration (FDA). The logo consists of two parts: on the left, there is an emblem representing the Department of Health & Human Services - USA, and on the right, there is the text "FDA U.S. FOOD & DRUG ADMINISTRATION" in blue. The word "FDA" is in a larger, bolder font, and the words "U.S. FOOD & DRUG" are stacked above the word "ADMINISTRATION".
EchoNous, Inc. Joshua Kim Sr. Manager, Regulatory Affairs 8310 154th Ave NE, Bldg B, Ste 200 Redmond, Washington 98052
Re: K233826
Trade/Device Name: Kosmos Regulation Number: 21 CFR 892.1550 Regulation Name: Ultrasonic Pulsed Doppler Imaging System Regulatory Class: Class II Product Code: IYN, IYO, ITX, DQD, DPS, QIH Dated: July 31, 2024 Received: July 31, 2024
Dear Joshua Kim:
We have reviewed your section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (the Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled "Deciding When to Submit a 510(k) for a Change to an Existing Device" (https://www.fda.gov/media/99812/download) and "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" (https://www.fda.gov/media/99785/download).
{1}------------------------------------------------
2
Your device is also subject to, among other requirements, the Quality System (QS) regulation (21 CFR Part 820), which includes, but is not limited to, 21 CFR 820.30, Design controls; 21 CFR 820.90, Nonconforming product; and 21 CFR 820.100, Corrective and preventive action. Please note that regardless of whether a change requires premarket review, the QS regulation requires device manufacturers to review and approve changes to device design and production (21 CFR 820.30 and 21 CFR 820.70) and document changes and approvals in the device master record (21 CFR 820.181).
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR Part 803) for devices or postmarketing safety reporting (21 CFR Part 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR Part 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR Parts 1000-1050.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.
For comprehensive regulatory information about mediation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely,
Marjan Nabili -S for
Yanna Kang, Ph.D. Assistant Director Mammography and Ultrasound Team DHT8C: Division of Radiological Imaging and Radiation Therapy Devices OHT8: Office of Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health
{2}------------------------------------------------
Indications for Use
510(k) Number (if known) K233826
Device Name Kosmos
Indications for Use (Describe)
Kosmos is intended to be used by qualified and trained healthcare professionals in the clinical assessment for the following clinical applications by acquiring, processing, displaying, measuring, and storing ultrasound images, or synchronized ultrasound images, electrocardiogram (ECG) rhythms, and digital auscultation (DA) sounds and waveforms.
With respect to its ultrasound imaging capability, Kosmos is a general-purpose diagnostic ultrasound system used in the following clinical applications and modes of operation:
Clinical Applications: Cardiac, Thoracic/Lung, Abdominal, Vascular, Musculoskeletal, and interventional guidance (includes free hand needle / catheter placement fluid drainage, and nerve block) Modes of Operation: B-mode, M-mode, Color Doppler, Pulsed Wave (PW) Doppler, Continuous Wave (CW) Doppler, Combined Modes of B+M, and B+CD, B+PW, B+CW, and Harmonic Imaging
Kosmos is intended to be used in clinical care and medical education settings on adult and pediations. Kosmos includes the Al-assisted automated ejection fraction software, known as Auto EF, which is used to process previously acquired transthoracic cardiac ultrages, to store images, and to manipulate and make measurements on images using the Kosmos. Auto EF provides automated estimation of left ventricular ejection fraction. This measurement can be used to assist the clinician in a cardiac evaluation. Auto EF is indicated for use on adult patients only in healthcare facilities.
The Kosmos includes the Auto Anatomical Structure Labeling and View Identification, also referred to as AI FAST, software, which is intended for use only by qualified and trained medical professionals for automatic real-lime detection and labeling of anatomical structures during image acquisition during cardiac, thoracic/lung, or abdominal ultrasound imaging. This feature is only indicated for use on adult patients in healthcare facilities.
The device is non-invasive, reusable, and intended to be used on one patient at a time.
Type of Use (Select one or both, as applicable)
X Prescription Use (Part 21 CFR 801 Subpart D) | Over-The-Counter Use (21 CFR 801 Subpart C)
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
§ 892.1550 Ultrasonic pulsed doppler imaging system.
(a)
Identification. An ultrasonic pulsed doppler imaging system is a device that combines the features of continuous wave doppler-effect technology with pulsed-echo effect technology and is intended to determine stationary body tissue characteristics, such as depth or location of tissue interfaces or dynamic tissue characteristics such as velocity of blood or tissue motion. This generic type of device may include signal analysis and display equipment, patient and equipment supports, component parts, and accessories.(b)
Classification. Class II.