(215 days)
Genius AI Detection is a computer-aided detection and diagnosis (CADe/CADx) software device intended to be used with compatible digital breast tomosynthesis (DBT) systems to identify and mark regions of interest including soft tissue densities (masses, architectural distortions and asymmetries) and calcifications in DBT exams from compatible DBT systems and provide confidence scores that offer assessment for Certainty of Findings and a Case Score. The device intends to aid in the interpretation of digital breast tomosynthesis exams in a concurrent fashion, where the interpreting physician confirms or dismisses the findings during the reading of the exam.
Genius Al Detection is a software device intended to identify potential abnormalities in breast tomosynthesis images. Genius Al Detection analyzes each standard mammographic view in a digital breast tomosynthesis examination using deep learning networks. For each detected lesion, Genius Al Detection produces CAD results that include the location of the lesion, an outline of the lesion and a confidence score for that lesion. Genius Al Detection also produces a case score for the entire tomosynthesis exam.
Genius Al Detection packages all CAD findings derived from the corresponding analysis of a tomosynthesis exam into a DICOM Mammography CAD SR object and distributes it for display on DICOM compliant review workstations. The interpreting physician will have access to the CAD findings concurrently to the reading of the tomosynthesis exam. In addition, a combination of peripheral information such as number of marks and case scores may be used on the review workstation to enhance the interpreting physician's workflow by offering a better organization of the patient worklist.
The acceptance criteria for the Genius AI Detection device and the study that proves it meets these criteria can be summarized as follows:
1. Table of Acceptance Criteria and Reported Device Performance:
| Acceptance Criteria | Reported Device Performance (with CAD vs. without CAD) |
|---|---|
| Improved diagnostic performance (AUC) | Average observed AUC: +0.031 (95% Cl: 0.012, 0.051) |
| Improved reader sensitivity for cancer cases | Average observed reader sensitivity: +9.0% (99% Cl: 6.0%, 12.1%) |
| Manageable (or improved) recall rate for non-cancer cases | Average observed recall rate: +2.4% (99% Cl: 0.7%, 4.2%) |
| Optimized workflow/read-time | Average observed case read-time difference: +5.7s (95% Cl: 4.9s to 6.4s) |
| Comparable performance across different Hologic tomosynthesis acquisition modes (standard vs. high-resolution) | fROC analysis showed comparable detection performance. No significant differences in the number and type of cancers detected. Stratified fROC analysis by lesion type and breast density also showed comparable performance. |
2. Sample Size for Test Set and Data Provenance:
- Sample Size: 764 cases for standalone testing, including 106 cancers and 658 non-cancer cases. For the MRMC study, 390 cases were used (106 cancers and 284 negative cases).
- Data Provenance: Not explicitly stated in the provided text, but it's implied that the data is from Digital Breast Tomosynthesis (DBT) exams. No country of origin or whether it was retrospective or prospective is mentioned.
3. Number of Experts and Qualifications for Ground Truth:
- Number of Experts: 17 readers participated in the MRMC study.
- Qualifications of Experts: Described as "MQSA-Qualified Interpreting Physicians and Radiologists." No specific years of experience or subspecialty focus are detailed.
4. Adjudication Method for Test Set:
- The document implies that the ground truth for cases with cancer was established prior to the reader study (e.g., "106 cancers"). However, the specific method of adjudication (e.g., 2+1, 3+1 expert consensus, pathology, follow-up) for establishing the definitive ground truth for the test set (both cancer and non-cancer cases) is not explicitly described in the provided text. It only states that the MRMC study utilized "106 cancers, and 284 negative cases."
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:
- Yes, an MRMC comparative effectiveness study was conducted.
- Effect Size of Human Readers with AI vs. without AI Assistance:
- AUC Improvement: The average observed AUC increased by +0.031 (95% Cl: 0.012, 0.051) with CAD.
- Sensitivity Improvement: The average observed reader sensitivity for cancer cases increased by +9.0% (99% Cl: 6.0%, 12.1%) with CAD.
- Recall Rate Change: The average observed recall rate for non-cancer cases increased by +2.4% (99% Cl: 0.7%, 4.2%) with CAD.
- Read-time Change: The average observed case read-time increased by 5.7s (95% Cl: 4.9s to 6.4s) with CAD.
6. Standalone (Algorithm Only) Performance Study:
- Yes, a standalone study was conducted.
- Purpose: To establish equivalence of Genius AI Detection performance on Hologic's standard resolution tomosynthesis images (~100um) compared to its performance on high-resolution tomosynthesis images (70μm).
- Methodology: The standalone study was conducted on paired high-resolution and standard-resolution 3D data sets, both acquired from a single exposure under the same compression.
- Results: The study confirmed "comparable detection performance" as observed by fROC analysis, with "no significant differences... in either acquisition mode," including when stratified by lesion type and breast density.
7. Type of Ground Truth Used:
- The text frequently refers to "cancers" and "non-cancer cases." This implies the ground truth for cancer diagnoses was established through pathology (biopsy), which is the definitive method for cancer. For "non-cancer cases" it would likely be a combination of imaging findings confirmed by follow-up or expert consensus, though this is not explicitly detailed.
8. Sample Size for Training Set:
- The sample size for the training set is not provided in the given document.
9. How Ground Truth for Training Set was Established:
- How the ground truth for the training set was established is not provided in the given document.
{0}------------------------------------------------
November 18, 2020
Image /page/0/Picture/1 description: The image contains two logos. On the left is the Department of Health & Human Services logo, which features a stylized human figure. To the right is the FDA (U.S. Food & Drug Administration) logo, with the letters "FDA" in a blue square and the words "U.S. FOOD & DRUG ADMINISTRATION" in blue text to the right of the square.
Hologic, Inc. % Ms. Deborah Thomas Regulatory Affairs Manager 250 Campus Drive MARLBOROUGH MA 01752
Re: K201019
Trade/Device Name: Genius AI Detection Regulation Number: 21 CFR 892.2090 Regulation Name: Radiological computer assisted detection and diagnosis software Regulatory Class: Class II Product Code: QDQ Dated: October 16, 2020 Received: October 19, 2020
Dear Ms. Thomas:
We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database located at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmp/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR 803) for devices or postmarketing safety reporting (21 CFR 4, Subpart B) for combination products (see
{1}------------------------------------------------
https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR 1000-1050.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.
For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely,
For
Thalia T. Mills, Ph.D. Director Division of Radiological Health OHT7: Office of In Vitro Diagnostics and Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health
Enclosure
{2}------------------------------------------------
Indications for Use
510(k) Number (if known) K201019
Device Name Genius AI Detection
Indications for Use (Describe)
Genius AI Detection is a computer-aided detection and diagnosis (CADe/CADx) software device intended to be used with compatible digital breast tomosynthesis (DBT) systems to identify and mark regions of interest including soft tissue densities (masses, architectural distortions and asymmetries) and calcifications in DBT exams from compatible DBT systems and provide confidence scores that offer assessment for Certainty of Findings and a Case Score. The device intends to aid in the interpretation of digital breast tomosynthesis exams in a concurrent fashion, where the interpreting physician confirms or dismisses the findings during the reading of the exam.
| Type of Use (Select one or both, as applicable) |
|---|
| ☒ Prescription Use (Part 21 CFR 801 Subpart D) |
| ☐ Over-The-Counter Use (21 CFR 801 Subpart C) |
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
{3}------------------------------------------------
Traditional 510(k) Summary K201019
This 510(k) Summary is submitted in accordance with the requirements of 21 CFR Part 807.92
| Date Prepared: | November 13, 2020 |
|---|---|
| Manufacturer: | Hologic, Inc. |
| 36 Apple Ridge Road | |
| Danbury, CT 06810 USA | |
| Establishment Registration #: | 1220984 |
| Contact Person: | Deborah Thomas |
| Regulatory Affairs Manager | |
| P: 508.210.6107 | |
| Identification of the Device: | |
| Proprietary/Trade Name: | Genius Al Detection |
| Classification Name: | Radiological Computer Assisted Detection/DiagnosisSoftware for Lesions Suspicious For Cancer |
| Regulatory Number: | 21 CFR 892.2090 |
| Product Code: | QDQ |
| Device Class: | Class II |
| Review Panel: | Radiology |
| Identification of the Legally Marketed Predicate Device: | |
| Trade Name: | PowerLook Tomo Detection V2 Software |
| Classification Name: | Radiological Computer Assisted Detection/DiagnosisSoftware for Lesions Suspicious For Cancer |
| Regulatory Number: | 21 CFR 892.2090 |
| Product Code: | QDQ |
| Device Class: | Class II |
| Review Panel: | Radiology |
| Submitter/510(k) Holder: | iCAD |
| Clearance: | K182373 (cleared December 6, 2018) |
Device Description:
Genius Al Detection is a software device intended to identify potential abnormalities in breast tomosynthesis images. Genius Al Detection analyzes each standard mammographic view in a digital breast tomosynthesis examination using deep learning networks. For each detected lesion, Genius Al Detection produces CAD results that include the location of the lesion, an outline of the lesion and a confidence score for that lesion. Genius Al Detection also produces a case score for the entire tomosynthesis exam.
{4}------------------------------------------------
Genius Al Detection packages all CAD findings derived from the corresponding analysis of a tomosynthesis exam into a DICOM Mammography CAD SR object and distributes it for display on DICOM compliant review workstations. The interpreting physician will have access to the CAD findings concurrently to the reading of the tomosynthesis exam. In addition, a combination of peripheral information such as number of marks and case scores may be used on the review workstation to enhance the interpreting physician's workflow by offering a better organization of the patient worklist.
Indications for Use:
Genius Al Detection is a computer-aided detection and diagnosis (CADe/CADx) software device intended to be used with compatible digital breast tomosynthesis (DBT) systems to identify and mark regions of interest including soft tissue densities (masses, architectural distortions and asymmetries) and calcifications in DBT exams from compatible DBT systems and provide confidence scores that offer assessment for Certainty of Findings and a Case Score. The device intends to aid in the interpretation of digital breast tomosynthesis exams in a concurrent fashion, where the interpreting physician confirms or dismisses the findings during the reading of the exam.
Standards:
- IEC 62304: 2015 – Medical device software – Software Life Cycle Processes (#13-79)
- ISO 14971: 2012 – Medical devices – Application of Risk Management to Medical Devices
- . DEN180005 Evaluation of automatic class III designation for OsteoDetect – Decision summary with special controls.
FDA Guidance Documents:
- Guidance for Industry and FDA Staff Guidance for the Content of Premarket Submissions for ● Software Contained in Medical Devices (Issued on May 11, 2005)
- . Computer-Assisted Detection Devices Applied to Radiology Images and Radiology Device Data – Premarket Notification [510(k)] Submissions (Issued on July 3, 2012)
- . Guidance for Industry and FDA Staff - Clinical Performance Assessment: Considerations for Computer-Assisted Detection Devices Applied to Radiology Images and Radiology Device Data -Premarket Approval (PMA) and Premarket Notification [510(k)] Submissions (Issued on January 22, 2020)
- . "Content of Premarket Submissions for Management of Cybersecurity in Medical Devices," issued on October 2, 2014
- "Off-the-Shelf Software Use in Medical Devices," issued on September 9, 1999
{5}------------------------------------------------
Summary of Substantial Equivalence:
| Features andCharacteristics | Subject Device | Predicate Device | Difference andcomments |
|---|---|---|---|
| Hologic, Inc.Genius AI Detection | iCAD Inc.PowerLook®Tomo | ||
| RegulationNumber/Name | 21 CFR 892.2090 /Radiological Computer AssistedDetection and Diagnosis Software | Same | N/A |
| Product Code | QDQ | Same | N/A |
| RegulationDescription | A radiological computer assisted detectionand diagnostic software is an imageprocessing device intended to aid in thedetection, localization, andcharacterization of fracture, lesions, orother disease specific findings on acquiredmedical images (e.g. radiography, MR, CT).The device detects, identifies andcharacterizes findings based on featuresor information extracted from images, andprovides information about the presence,location, and characteristics of thefindings to the user. The analysis isintended to inform the primary diagnosticand patient management decisions thatare made by the clinical user. The deviceis not intended as a replacement for acomplete clinician's review or theirclinical judgment that takes into accountother relevant information from theimage or patient history. | Same | N/A |
| Indications forUse | Genius Al Detection is a computer-aideddetection and diagnosis (CADe/CADx)software device intended to be used withcompatible digital breast tomosynthesis(DBT) systems to identify and markregions of interest including soft tissuedensities (masses, architecturaldistortions and asymmetries) andcalcifications in DBT exams fromcompatible DBT systems and provideconfidence scores that offer assessmentfor Certainty of Findings and a Case Score.The device intends to aid in theinterpretation of digital breasttomosynthesis exams in a concurrentfashion, where the interpreting physicianconfirms or dismisses the findings duringthe reading of the exam. | PowerLook® TomoDetection V2 softwareis a computer-assisted detection anddiagnosis (CAD)software deviceintended to be usedconcurrently byinterpreting physicianswhile reading digitalbreast tomosynthesis(DBT) exams fromcompatible DBTsystems. The systemdetects soft tissuedensities (masses,architecturaldistortions andasymmetries) andcalcifications in the 3DDBT slices. Thedetections andCertainty of Findingand Case Scores assistinterpreting physiciansin identifying softtissue densities and | |
| CompatibleDBT Systems | Hologic Selenia DimensionsHologic 3DimensionsSupports both models in the followingmodes:standard resolution 1-mm slices high resolution 1-mm slices(Clarity HD), high resolution 6-mm SmartSlices(3DQuorum) | tissue densities andHologic SeleniaDimensions (standardresolution, 1-mmslices)GE Pristina | The subjectdevice andpredicate deviceare compatiblewith differentsystems asnoted. |
| Type of CADSoftware | Radiological computer assisteddetection and diagnostic software. | same | N/A |
| Mode of Action | Image processing device utilizingmachine learning to aid in thedetection, localization, andcharacterization of soft tissue densities(masses, architectural distortions andasymmetries) and calcifications in the1-mm 3D DBT slices. Findings are co-registered to 6-mm SmartSlices. | Image processingdevice utilizingmachine learningto aid in thedetection,localization, andcharacterization of softtissue densities(masses, architecturaldistortions andasymmetries) and | Co-registration offindings to 6-mmSmartSlices. |
| Clinical Output | To inform the primary diagnostic andpatient management decisions that aremade by the clinical user. | same | N/A |
| PatientPopulation | Symptomatic and asymptomaticwomen undergoing mammography | same | N/A |
| End Users | MQSA-Qualified InterpretingPhysicians and Radiologists | same | N/A |
| Image SourceModalities | Digital breast tomosynthesis slices | same | N/A |
| Output Device | Softcopy Workstation | same | N/A |
| Deployment | Stand-alone computer | same | N/A |
| VisualizationFeatures | Places mark within suspicious lesion bydefault (Emphasize™; RightOn™) andreports confidence of finding next to eachidentified lesion in the image. CAD displaymay be toggled on/off. Option toautomatically zoom into or contour thesuspicious region of interest (PeerView™). | Contours suspiciouslesions by default anddisplays confidence offinding next to eachidentified lesion in theimage. CAD display maybe toggled on/off. Noother marks. | extra displayfunctions of marks |
| Method Of Use | Concurrent read | Concurrent read. FFDMand 2D synthetic viewswhen available are tobe reviewed before | N/A |
| Supported Views | CC and MLO | same | N/A |
{6}------------------------------------------------
{7}------------------------------------------------
{8}------------------------------------------------
Comparison with Predicate Device:
The Summary of Substantial Equivalence Table above details the similarities and differences between the Genius Al Detection device and its predicate device, PowerLook® Tomo Detection V2. Both devices aid in the detection, localization, and characterization of disease specific findings on acquired medical images.
Genius Al Detection is the same technology as the predicate per 21 CFR 892.2090; both devices are radiological computer assisted detection and diagnostic software, intended to aid in the detection, localization, and characterization of disease specific findings on acquired medical images. The outputs of both devices serve to augment the interpretation of digital breast tomosynthesis exams as a concurrent reading tool. The output is used to inform and assist the interpreting physician, supplementing their clinical expertise and judgment. In the case of any differences that may occur between the subject device and the predicate, rationale for safety and effectiveness is provided above along with special controls established for Radiological Computer Assisted Detection and Diagnosis Software that are in place to further mitigate any risks in these differences.
Compatible DBT Systems
The following image types have been tested and are compatible with Genius Al Detection:
- Hologic standard resolution tomosynthesis slices (1 mm)
- Hologic high resolution tomosynthesis slices (1 mm) (Clarity HD)
- Hologic high resolution SmartSlices (6 mm) (3DQuorum)
The CAD marks generated by Genius Al Detection for the above image types can also be projected on their corresponding synthesized 2D images, providing that the diagnostic review workstation supports such a feature.
Performance Testing - Reader Study Results
The study was successfully executed with all 17 readers; the results below represent a per-protocol analysis of the 390 cases (106 cancers, and 284 negative cases) included in the MRMC where both rounds of reading were completed.
Based on analyses that do not control type I error and therefore cannot be generalized to specific comparisons outside this particular study, in this study:
- The average observed AUC was 0.825 (95% Cl: 0.783, 0.867) with CAD and 0.794 (95% Cl: 0.748, 0.840) without CAD. The difference in observed AUC was +0.031 (95% Cl: 0.012, 0.051).
{9}------------------------------------------------
- The average observed reader sensitivity for cancer cases was 75.9% with CAD and 66.8% without CAD. The difference in observed sensitivity was +9.0% (99% Cl: 6.0%, 12.1%).
- . The average observed recall rate for non-cancer cases was 25.8% with CAD and 23.4% without CAD. The observed difference in negative recall rate was +2.4% (99% Cl: 0.7%, 4.2%).
- . The average observed case read-time was 52.0s with CAD and 46.3s without CAD. The observed difference in read-time was 5.7s (95% Cl: 4.9s to 6.4s).
Standalone Testing:
Hologic conducted an MRMC reader study to assess the safety and efficacy of Genius Al Detection on Hologic's high resolution tomosynthesis images, where 3D data sets are reconstructed at 70μm. Parallel to the MRMC study, a standalone study was conducted to establish equivalence of Genius Al Detection performance on Hologic's standard resolution tomosynthesis images, where 3D data sets are reconstructed at ~100um compared to the performance on the high resolution tomosynthesis images. The standalone study was conducted on paired high resolution and standard resolution 3D data sets, where each high-resolution reconstructed 3D volume had a counterpart standard resolution 3D volume, both acquired from a single exposure and under the same compression.
All the results of the standalone study confirmed that Genius Al Detection when operating under the Hologic's standard tomosynthesis acquisition mode performs comparably to when operating under the high-resolution mode.
Using an overall data set of 764 cases including 106 cancers and 658 non-cancer cases, Genius AI 1. Detection demonstrated comparable detection performance on both Hologic's standard and highresolution acquisition modes as observed by fROC analysis.
No significant differences were observed in the number and type of cancers detected by Genius Al 2. Detection in either acquisition mode.
- Stratified fROC analysis using lesion type as well as breast density also showed comparable performance of Genius Al Detection operating in either acquisition mode
Assessment of Benefit-Risk, Safety and Effectiveness, and Substantial Equivalence:
As a part of the submission, we have demonstrated the probable benefits of the device through clinical study including reader accuracy as assessed by AUC (i.e. diagnostic performance and improved assistedread detection from the sensitivity analysis. In totality, Hologic finds that the proposed device has a positive balance in terms of probable benefits vs probable risks and thus may be considered safe and effective based on the special controls established in DEN180005 such that "the device will provide improved assisted-read detection and diagnostic performance."
Conclusion:
Based on the information submitted in this premarket notification, the Genius Al Detection device and its predicate device, PowerLook® Tomo Detection V2 both have a similar intended use and are devices which aid in the detection, localization, and characterization of disease specific findings on acquired medical images. The differences discussed are not significant to the technology and clinical application of the device. The proposed Genius Al Detection device has been found to be substantially equivalent to the predicate PowerLook® Tomo Detection V2 device (K182373).
§ 892.2090 Radiological computer-assisted detection and diagnosis software.
(a)
Identification. A radiological computer-assisted detection and diagnostic software is an image processing device intended to aid in the detection, localization, and characterization of fracture, lesions, or other disease-specific findings on acquired medical images (e.g., radiography, magnetic resonance, computed tomography). The device detects, identifies, and characterizes findings based on features or information extracted from images, and provides information about the presence, location, and characteristics of the findings to the user. The analysis is intended to inform the primary diagnostic and patient management decisions that are made by the clinical user. The device is not intended as a replacement for a complete clinician's review or their clinical judgment that takes into account other relevant information from the image or patient history.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Design verification and validation must include:
(i) A detailed description of the image analysis algorithm, including a description of the algorithm inputs and outputs, each major component or block, how the algorithm and output affects or relates to clinical practice or patient care, and any algorithm limitations.
(ii) A detailed description of pre-specified performance testing protocols and dataset(s) used to assess whether the device will provide improved assisted-read detection and diagnostic performance as intended in the indicated user population(s), and to characterize the standalone device performance for labeling. Performance testing includes standalone test(s), side-by-side comparison(s), and/or a reader study, as applicable.
(iii) Results from standalone performance testing used to characterize the independent performance of the device separate from aided user performance. The performance assessment must be based on appropriate diagnostic accuracy measures (
e.g., receiver operator characteristic plot, sensitivity, specificity, positive and negative predictive values, and diagnostic likelihood ratio). Devices with localization output must include localization accuracy testing as a component of standalone testing. The test dataset must be representative of the typical patient population with enrichment made only to ensure that the test dataset contains a sufficient number of cases from important cohorts (e.g., subsets defined by clinically relevant confounders, effect modifiers, concomitant disease, and subsets defined by image acquisition characteristics) such that the performance estimates and confidence intervals of the device for these individual subsets can be characterized for the intended use population and imaging equipment.(iv) Results from performance testing that demonstrate that the device provides improved assisted-read detection and/or diagnostic performance as intended in the indicated user population(s) when used in accordance with the instructions for use. The reader population must be comprised of the intended user population in terms of clinical training, certification, and years of experience. The performance assessment must be based on appropriate diagnostic accuracy measures (
e.g., receiver operator characteristic plot, sensitivity, specificity, positive and negative predictive values, and diagnostic likelihood ratio). Test datasets must meet the requirements described in paragraph (b)(1)(iii) of this section.(v) Appropriate software documentation, including device hazard analysis, software requirements specification document, software design specification document, traceability analysis, system level test protocol, pass/fail criteria, testing results, and cybersecurity measures.
(2) Labeling must include the following:
(i) A detailed description of the patient population for which the device is indicated for use.
(ii) A detailed description of the device instructions for use, including the intended reading protocol and how the user should interpret the device output.
(iii) A detailed description of the intended user, and any user training materials or programs that address appropriate reading protocols for the device, to ensure that the end user is fully aware of how to interpret and apply the device output.
(iv) A detailed description of the device inputs and outputs.
(v) A detailed description of compatible imaging hardware and imaging protocols.
(vi) Warnings, precautions, and limitations must include situations in which the device may fail or may not operate at its expected performance level (
e.g., poor image quality or for certain subpopulations), as applicable.(vii) A detailed summary of the performance testing, including test methods, dataset characteristics, results, and a summary of sub-analyses on case distributions stratified by relevant confounders, such as anatomical characteristics, patient demographics and medical history, user experience, and imaging equipment.