(63 days)
The Caption Guidance software is intended to assist medical professionals in the acquisition of cardiac ultrasound images. The Caption Guidance software is an accessory to compatible general purpose diagnostic ultrasound systems.
The Caption Guidance software is indicated for use in two-dimensional transthoracic echocardiography (2D-TTE) for adult patients, specifically in the acquisition of the following standard views: Parasternal Long-Axis (PLAX), Parasternal Short-Axis at the Aortic Valve (PSAX-AV), Parasternal Short-Axis at the Mitral Valve (PSAX-MV), Parasternal Short- Axis at the Papillary Muscle (PSAX-PM), Apical 4-Chamber (AP4), Apical 5-Chamber (AP5), Apical 2-Chamber (AP2), Apical 3-Chamber (AP3), Subcostal 4-Chamber (SubC4), and Subcostal Inferior Vena Cava (SC-IVC).
The Caption Guidance software is a radiological computer assisted acquisition guidance system that provides real-time user quidance during acquisition of echocardiography to assist the user in obtaining anatomically correct images that represent standard 2D echocardiographic diagnostic views and orientations. Caption Guidance is a software-only device that uses artificial intelligence to emulate the expertise of sonographers.
Caption Guidance is comprised of several different features that, combined, provide expert quidance to the user. These include:
- Quality Meter: The real-time feedback from the Quality Meter advises the user on the expected diagnostic quality of the resulting clip, such that the user can make decisions to further optimize the quality, for example by following the prescriptive guidance feature below.
- Prescriptive Guidance: The prescriptive guidance feature in Caption Guidance . provides direction to the user to emulate how a sonographer would manipulate the transducer to acquire the optimal view.
- Auto-Capture: The Caption Guidance Auto-Capture feature triggers an automatic . capture of a clip when the quality is predicted to be diagnostic, emulating the way in which a sonographer knows when an image is of sufficient quality to be diagnostic and records it.
- Save Best Clip: This feature continually assesses clip quality while the user is scanning . and, in the event that the user is not able to obtain a clip sufficient for Auto-Capture, the software allows the user to retrospectively record the highest quality clip obtained so far, mimicking the choice a sonographer might make when recording an exam.
Here's a breakdown of the acceptance criteria and the study details for the Caption Guidance software, based on the provided text.
Note: The provided document is a 510(k) summary for a modification to an already cleared device (K201992, which is predicated on K200755). Therefore, the document primarily focuses on demonstrating substantial equivalence to the previous version of the device and outlining a modification to a predetermined change control plan (PCCP). It does not detail a new clinical study to prove initial performance against acceptance criteria for the entire device, but rather refers to the established equivalence and the PCCP.
However, based on what's typically expected for such devices, and inferring from the description of the device's capabilities, I will construct a plausible set of acceptance criteria and discuss what can be gleaned about the study from the provided text, while acknowledging its limitations for providing full study details.
Acceptance Criteria and Reported Device Performance
Given the device's function (assisting with cardiac ultrasound image acquisition by guiding users to obtain specific standard views and optimizing quality), the acceptance criteria would likely revolve around the accuracy of its guidance, the quality of the "auto-captured" views, and its ability to help users acquire diagnostically relevant images.
Since this document is a 510(k) for a modification and states "The current iteration of the Caption Guidance software is as safe and effective as the previous iteration of such software," and "The Caption Guidance software has the same intended use, indications for use, technological characteristics, and principles of operation as its predicate device," the specific performance metrics from the original predicate device's clearance are not explicitly stated here.
However, a hypothetical table of common acceptance criteria for such a device and inferred performance (based on the device being cleared and performing "as safe and effective as the previous iteration") would look something like this:
| Acceptance Criteria | Reported Device Performance (Inferred/Implicitly Met) |
|---|---|
| View Classification Accuracy: The software should correctly identify and guide the user towards the specified standard cardiac ultrasound views (PLAX, PSAX-AV, PSAX-MV, PSAX-PM, AP4, AP5, AP2, AP3, SubC4, SC-IVC) with high accuracy. | Implicitly met, as the device is cleared for this function and states it's "as safe and effective as the previous iteration" which performed this. The previous clearance would have established a threshold (e.g., >90% or 95% accuracy in guiding to correct view). |
| Quality Assessment Accuracy: The "Quality Meter" should accurately reflect the diagnostic quality of the scan in real-time, enabling users to optimize the image. | Implicitly met. Performance would likely have been measured as correlation between the AI's quality score and expert-rated image quality, or improvement in image quality metrics in AI-assisted scans. |
| Auto-Capture Performance: The "Auto-Capture" feature should reliably capture clips when the quality is predicted to be diagnostic, minimizing non-diagnostic captures and maximizing diagnostic ones. | Implicitly met. Metrics would include precision and recall for capturing diagnostic clips, or the rate of correctly auto-captured diagnostic clips. |
| Prescriptive Guidance Effectiveness: The "Prescriptive Guidance" should effectively direct users to manipulate the transducer to acquire optimal views, leading to an increase in the proportion of quality images. | Implicitly met. This would likely be measured by the rate of successful view acquisition and/or time to acquire optimal views with and without guidance. |
| Clinical Equivalence/Non-Inferiority: The overall use of the Caption Guidance software should lead to the acquisition of cardiac ultrasound images that are non-inferior (or superior) in diagnostic quality and completeness compared to standard methods. | Implicitly met via substantial equivalence to predicate. The original predicate study would have demonstrated that images acquired with the system were diagnostically useful. |
Study Details (Based on available information in the document)
1. Sample size used for the test set and the data provenance:
- Sample Size: Not explicitly stated in this 510(k) summary. Given this is for a PCCP modification, new clinical test data for this submission is not provided, but rather relies on the predicate's performance.
- Data Provenance: Not explicitly stated for either the training or test sets in this document. It's common for such data to come from multiple sites and locations to ensure generalizability, but the document doesn't specify. The document refers to the previous iteration's performance, which would have had this data.
2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Not explicitly stated in this 510(k) summary. This information would be present in the original 510(k) for the predicate device (K200755). Typically, a panel of board-certified radiologists or cardiologists with expertise in echocardiography would be used.
3. Adjudication method (e.g. 2+1, 3+1, none) for the test set:
- Not explicitly stated. This detail would be found in the original 510(k) submission for the device that established its initial substantial equivalence. Common methods include majority rule (e.g., 2 out of 3 or 3 out of 5 experts agreeing), or a senior expert adjudicating disagreements.
4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- The document implies that the device "emulates the expertise of sonographers" and "provides real-time user guidance." This strongly suggests that the original predicate submission would have included a study (likely an MRMC-type study or a study comparing guided vs. unguided acquisition) to demonstrate that the system assists users in acquiring better images.
- Effect Size: Not provided in this summary. Such a study would likely show improvements in metrics like:
- Percentage of standard views successfully acquired.
- Time taken to acquire optimal views.
- Image quality scores (e.g., higher proportion of "diagnostic quality" images).
- Reduction in inter-user variability for image acquisition.
- Potentially, images acquired were more similar to those obtained by expert sonographers.
5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- Yes, implicitly. The very nature of the "Quality Meter," "Auto-Capture," and "Prescriptive Guidance" features means the AI must perform standalone assessments (e.g., classifying views, assessing quality) to provide its guidance. The performance metrics listed under acceptance criteria would have standalone components (e.g., accuracy of the AI's view classification vs. ground truth).
6. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
- Expert Consensus/Expert Review: For a device guiding image acquisition, ground truth for view classification and image quality would almost certainly be established by expert review (e.g., highly experienced sonographers, cardiologists, or radiologists reviewing the acquired images and assigning view labels and quality scores). This is standard for image guidance systems.
7. The sample size for the training set:
- Not explicitly stated. This information is typically proprietary and part of the design and development details, but would have been documented for the original clearance.
8. How the ground truth for the training set was established:
- Not explicitly stated, but it would align with the method for the test set ground truth: Expert Consensus/Expert Review. The training data (images and associated metadata) would be meticulously labeled by qualified experts (e.g., specifying which view each image represents, and potentially assigning quality scores) to enable the AI to learn.
892.2100 Radiological acquisition and/or optimization guidance system.
892.2100 Radiological acquisition and/or optimization guidance system.
(a)
Identification. A radiological acquisition and/or optimization guidance system is a device that is intended to aid in the acquisition and/or optimization of images and/or diagnostic signals. The device interfaces with the acquisition system, analyzes its output, and provides guidance and/or feedback to the operator for improving image and/or signal quality.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Design verification and validation must include:
(i) A detailed, technical device description, including a detailed description of the impact of any software and hardware on the device's functions, the associated capabilities and limitations of each part, and the associated inputs and outputs.
(ii) A detailed, technical report on the non-clinical performance testing of the subject device in the intended use environments, using relevant consensus standards when applicable.
(iii) A detailed report on the clinical performance testing, obtained from either clinical testing, accepted virtual/physical systems designed to capture clinical variability, comparison to a closely-related device with established clinical performance, or other sources that are justified appropriately. The choice of the method must be justified given the risk of the device and the general acceptance of the test methods. The report must include the following:
(A) A thorough description of the testing protocol(s).
(B) A thorough, quantitative evaluation of the diagnostic utility and quality of images/data acquired, or optimized, using the device.
(C) A thorough, quantitative evaluation of the performance in a representative user population and patient population, under anticipated conditions and environments of use.
(D) A thorough discussion on the generalizability of the clinical performance testing results.
(E) A thorough discussion on use-related risk analysis/human factors data.
(iv) A detailed protocol that describes, in the event of a future change, the level of change in the device technical specifications or indications for use at which the change or changes could significantly affect the safety or effectiveness of the device and the risks posed by these changes. The assessment metrics, acceptance criteria, and analytical methods used for the performance testing of changes that are within the scope of the protocol must be included.
(v) Documentation of an appropriate training program, including instructions on how to acquire and process quality images and video clips, and a report on usability testing demonstrating the effectiveness of that training program on user performance, including acquiring and processing quality images.
(2) The labeling required under § 801.109(c) of this chapter must include:
(i) A detailed description of the device, including information on all required and/or compatible parts.
(ii) A detailed description of the patient population for which the device is indicated for use.
(iii) A detailed description of the intended user population, and the recommended user training.
(iv) Detailed instructions for use, including the information provided in the training program used to meet the requirements of paragraph (b)(1)(iv) of this section.
(v) A warning that the images and data acquired using the device are to be interpreted only by qualified medical professionals.
(vi) A detailed summary of the reports required under paragraphs (b)(1)(ii) and (iii) of this section.
(vii) A statement on upholding the As Low As Reasonably Achievable (ALARA) principle with a discussion on the associated device controls/options.