(447 days)
The Auto Lung Nodule Detection is computer-aided detection software to identify and mark regions in relation to suspected pulmonary nodules from 10 to 30 mm in size. It is designed to aid the physician to review the PA chest radiographs of adults as a second reader and be used as part of S-Station, which is operation software installed on Samsung Digital X-ray Imaging systems. Auto Lung Nodule Detection cannot be used on the patients who have lung lesions other than abnormal nodules.
Auto Lung Nodule Detection is a Computer-Aided Detection (CADe) device that is designed to perform CAD processing in Chest X-ray images for indication of locations for high nodule probability, which has an effective detection sizes from 10 mm to 30 mm.
Auto Lung Nodule Detection receives images acquired with SAMSUNG Digital X-ray Imaging Systems as an input and identifies suspected nodules, and then sends information of suspected nodules to the visualization part of S-Station, which is installed on all kinds of SAMSUNG Digital X-ray Imaging Systems, to generate output images with circular marks. The CAD performed images, are displayed on the screen by S-Station without defeat of original images and used as a second reader only after the initial read is completed.
Here's a breakdown of the acceptance criteria and study details for the Auto Lung Nodule Detection device, based on the provided FDA 510(k) Premarket Notification:
1. Table of Acceptance Criteria and Reported Device Performance
The document describes the clinical study's objective as demonstrating that the device improves human readers' nodule detection performance. While specific numerical acceptance criteria (e.g., minimum sensitivity, maximum FPPI) are not explicitly called out in a "table of acceptance criteria," the clinical performance testing section clearly states that the results have demonstrated that all readers' nodule detection performances using the proposed device have increased with statistical significance. This implicitly defines the acceptance criteria: a statistically significant improvement in nodule detection performance metrics.
| Metric (Implicit Acceptance Criteria) | Reported Device Performance |
|---|---|
| Sensitivity | Increased with statistical significance (with ALND assistance) |
| False Positives per Image (FPPI) | Improved (implicitly, as performance increased) |
| JAFROC Figure of Merit (FOM) | Increased with statistical significance (with ALND assistance) |
2. Sample Size and Data Provenance
- Test Set Sample Size: Not explicitly stated. The document mentions "a test dataset containing both normal and diseased images."
- Data Provenance: Not explicitly stated (e.g., country of origin). The study is described as a "clinical evaluation," implying real patient data. It is not specified if the data was retrospective or prospective.
3. Number of Experts and Qualifications for Ground Truth
- Number of Experts: Not explicitly stated.
- Qualifications of Experts: Not explicitly stated. However, given the context of a CADe device for radiological use, these would typically be board-certified radiologists or pulmonary specialists.
4. Adjudication Method for the Test Set
The document mentions that "Readers were asked to mark their region of nodule suspicion on the images while also providing confidence scores on each decision they have made." It then states, "After independent reading, readers were allowed to adjust their confidence scores after reviewing the ALND's detection results." This describes a workflow, but does not explicitly describe the ground truth adjudication method (e.g., consensus reading by multiple experts, pathology confirmation). The term "independent reading" before ALND review suggests individual reads prior to any potential adjudication or consensus if it occurred.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
Yes, an MRMC comparative effectiveness study was conducted.
- Effect Size of Improvement: The document states, "The results have demonstrated that all readers' nodule detection performances using the proposed device have increased with statistical significance." While specific numerical effect sizes (e.g., percentage increase in sensitivity, reduction in FPPI, or change in JAFROC FOM) are not provided, the key finding is the statistical significance of the improvement when radiologists used ALND as an assistant tool.
6. Standalone (Algorithm Only) Performance
The document does not explicitly report standalone performance of the algorithm without human-in-the-loop. The clinical evaluation focuses on the human-in-the-loop performance (with ALND assistance vs. without).
7. Type of Ground Truth Used
The type of ground truth used is not explicitly stated. It can be inferred that it involved expert consensus or a gold standard based on follow-up, but the document does not specify if it was pathology, follow-up imaging, or expert consensus.
8. Sample Size for the Training Set
The sample size for the training set is not provided in this document.
9. How Ground Truth for the Training Set was Established
The method for establishing ground truth for the training set is not provided in this document.
§ 892.2070 Medical image analyzer.
(a)
Identification. Medical image analyzers, including computer-assisted/aided detection (CADe) devices for mammography breast cancer, ultrasound breast lesions, radiograph lung nodules, and radiograph dental caries detection, is a prescription device that is intended to identify, mark, highlight, or in any other manner direct the clinicians' attention to portions of a radiology image that may reveal abnormalities during interpretation of patient radiology images by the clinicians. This device incorporates pattern recognition and data analysis capabilities and operates on previously acquired medical images. This device is not intended to replace the review by a qualified radiologist, and is not intended to be used for triage, or to recommend diagnosis.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Design verification and validation must include:
(i) A detailed description of the image analysis algorithms including a description of the algorithm inputs and outputs, each major component or block, and algorithm limitations.
(ii) A detailed description of pre-specified performance testing methods and dataset(s) used to assess whether the device will improve reader performance as intended and to characterize the standalone device performance. Performance testing includes one or more standalone tests, side-by-side comparisons, or a reader study, as applicable.
(iii) Results from performance testing that demonstrate that the device improves reader performance in the intended use population when used in accordance with the instructions for use. The performance assessment must be based on appropriate diagnostic accuracy measures (
e.g., receiver operator characteristic plot, sensitivity, specificity, predictive value, and diagnostic likelihood ratio). The test dataset must contain a sufficient number of cases from important cohorts (e.g., subsets defined by clinically relevant confounders, effect modifiers, concomitant diseases, and subsets defined by image acquisition characteristics) such that the performance estimates and confidence intervals of the device for these individual subsets can be characterized for the intended use population and imaging equipment.(iv) Appropriate software documentation (
e.g., device hazard analysis; software requirements specification document; software design specification document; traceability analysis; description of verification and validation activities including system level test protocol, pass/fail criteria, and results; and cybersecurity).(2) Labeling must include the following:
(i) A detailed description of the patient population for which the device is indicated for use.
(ii) A detailed description of the intended reading protocol.
(iii) A detailed description of the intended user and user training that addresses appropriate reading protocols for the device.
(iv) A detailed description of the device inputs and outputs.
(v) A detailed description of compatible imaging hardware and imaging protocols.
(vi) Discussion of warnings, precautions, and limitations must include situations in which the device may fail or may not operate at its expected performance level (
e.g., poor image quality or for certain subpopulations), as applicable.(vii) Device operating instructions.
(viii) A detailed summary of the performance testing, including: test methods, dataset characteristics, results, and a summary of sub-analyses on case distributions stratified by relevant confounders, such as lesion and organ characteristics, disease stages, and imaging equipment.