(197 days)
FlightPlan for Liver is a post processing software package that helps the analysis of 3D X-ray images of the liver arterial tree. Its output is intended as an adjunct means to help identify arteries leading to the vicinity of hypervascular lesions in the liver. This adjunct information may be used by physicians to aid them in their evaluation of hepatic arterial anatomy during embolization procedures.
FlightPlan for Liver is a post-processing software application for use with interventional fluoroscopy procedures, using 3D rotational angiography images as input. It operates on the AW VolumeShare 4 [K052995] and AW VolumeShare 5 [K110834] platform. It is an extension to the Volume Viewer application [K041521] utilizing the rich set of the 3D processing features of Volume Viewer. FlightPlan for Liver delivers post-processing features that will aid physicians in their analysis of 3D X-ray images of the liver arterial tree. Additionally FlightPlan for Liver includes an algorithm to highlight the potential vessel(s) in the vicinity of a target.
Here's an analysis of the provided text regarding the acceptance criteria and study for the FlightPlan for Liver device:
Acceptance Criteria and Device Performance
There is no explicit table of acceptance criteria or reported device performance metrics (e.g., sensitivity, specificity, AUC) in the provided document. The submission focuses on demonstrating substantial equivalence to a predicate device and confirming that the software functions as required and fulfills user needs.
The "Performance testing" mentioned is described as "computing time of algorithm on several data," implying it's a speed or efficiency metric rather than a diagnostic performance metric. The "Verification confirms that the Design Output meets the Design Input (Product Specifications) requirements" and "Validation confirms that the product fulfills the user needs and the intended use under simulated use conditions," but specific, quantifiable acceptance criteria are not detailed.
The "Summary of Clinical Tests" states that the study "demonstrate[d] the safety and effectiveness of FlightPlan for Liver" and compared its output "to a reference reading established by two senior interventional oncologists." However, the exact metrics used for comparison and the "acceptance criteria" for those metrics are not provided. The key takeaway is that the clinical data was not intended to support a claim of improved clinical outcomes.
Study Details
Here's what can be extracted about the study that proves the device meets the (unspecified quantitative) acceptance criteria:
1. A table of acceptance criteria and the reported device performance
| Acceptance Criteria Category | Specific Criteria (Implicit/General) | Reported Device Performance |
|---|---|---|
| Functional Verification | Application works as required; Risk mitigations correctly implemented. | "Verification tests... performed to check whether the application works as required and whether the risk mitigations have been correctly implemented." |
| Performance Testing | Algorithm computing time (specific targets not provided). | "Performance testing consists of computing time of algorithm on several data." |
| Design Validation | Product fulfills user needs and intended use under simulated use conditions. | "Validation tests consist of typical use case scenario described by the sequence of operator actions. The Design Validation confirms that the product fulfills the user needs and the intended use under simulated use conditions." |
| Clinical Effectiveness | Output provides adjunct information to aid physicians in evaluating hepatic arterial anatomy; output compared to reference reading. | Output was compared to a reference reading established by two senior interventional oncologists. No specific quantitative performance metrics (e.g., accuracy, precision) are provided, nor are numerical results of this comparison. |
| Substantial Equivalence | Functionality, safety, and effectiveness are comparable to the predicate device. | "GE Healthcare considers the FlightPlan for Liver application to be as safe and as effective as its predicate device, and its performance is substantially equivalent to the predicate device." |
2. Sample size used for the test set and the data provenance
- Test Set Size: 44 subjects, representing a total of 66 tumors.
- Data Provenance: Retrospective study. The country of origin is not explicitly stated, but given the submitter's address (Buc, FRANCE) and the GE Healthcare global nature, it could be either European or multinational, but this is speculative.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
- Number of Experts: Two.
- Qualifications: "Senior interventional oncologists." Specific experience (e.g., years) is not provided.
4. Adjudication method for the test set
- The ground truth was established by a "reference reading established by two senior interventional oncologists." While it states the two established the reference, it doesn't specify if this was by consensus, independent reads with adjudication, or another method. The phrasing "a reference reading established by two" suggests a single, agreed-upon ground truth, likely consensus or 2-reader agreement if initial reads differed.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
- No, a multi-reader, multi-case (MRMC) comparative effectiveness study comparing human readers with AI assistance vs. without AI assistance was not done. The study specifically states that the clinical data "was not designed nor intended to support a claim of an improvement in clinical outcomes of such procedures, and no such claim is being made." The study focused on comparing the device's output to an expert reference, not on human performance improvement.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
- Yes, the clinical study appears to evaluate the algorithm's standalone performance compared to a "reference reading." The "output of FlightPlan for Liver was compared to a reference reading," indicating the algorithm's direct output was assessed. No mention is made of human interaction or interpretation of the algorithm's output as part of this comparison.
7. The type of ground truth used
- Type of Ground Truth: Expert consensus/reference reading. Specifically, "a reference reading established by two senior interventional oncologists."
8. The sample size for the training set
- The document does not mention the sample size for the training set. It only describes the clinical study as a "retrospective study" used for verification and validation, implying it was a test set. There's no information about the data used to train the "algorithm to highlight the potential vessel(s)."
9. How the ground truth for the training set was established
- This information is not provided since the document does not detail the training set or its ground truth establishment.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).