(102 days)
Not Found
Yes
The intended use explicitly states that cmTriage uses an "artificial intelligence algorithm" to analyze mammograms.
No.
The device is described as a "prioritization-only, parallel-workflow software tool" and "does not provide any diagnostic information beyond triage and prioritization". Its function is to flag suspicious findings to help radiologists prioritize their worklist, not to treat or diagnose.
No
The text explicitly states: "cm Triage is limited to the categorization of exams, does not provide any diagnostic information beyond triage and prioritization, does not remove images from the radiologist's worklist, and should not be used in lieu of full patient evaluation, or relied upon to make or confirm diagnosis." and "The device does not alter the original medical image and is not intended to be used as a diagnostic device."
Yes
The device description explicitly states that cmTriage is a "radiological computer-assisted triage and notification software device" and details its function as software interacting with existing hardware (FFDM systems, PACS) without including any hardware components itself.
Based on the provided information, this device is not an In Vitro Diagnostic (IVD).
Here's why:
- IVDs analyze biological specimens: In Vitro Diagnostics are designed to examine samples taken from the human body, such as blood, urine, tissue, etc., to provide information about a person's health.
- This device analyzes medical images: cmTriage analyzes digital mammograms (medical images), not biological specimens.
- The intended use is for image prioritization: The primary function of cmTriage is to prioritize medical images within a radiologist's worklist based on the analysis of those images. It does not perform any analysis on biological samples.
Therefore, while it is a medical device that uses AI and image processing, its function and intended use fall outside the definition of an In Vitro Diagnostic.
No
The letter does not explicitly state that the FDA has reviewed and approved or cleared a PCCP for this specific device. The section "Control Plan Authorized (PCCP) and relevant text" explicitly states "Not Found".
Intended Use / Indications for Use
cmTriage is a passive notification for prioritization-only, parallel-workflow software tool used by radiologists to prioritize specific patients within the standard-of-care image worklist for 2D FFDM screening mammograms, cmTriage uses an artificial intelligence algorithm to analyze 2D FFDM screening mammograms and flags those that are suggestive of the presence of at least one suspicious finding at the exam level. These flags are viewed by the radiologist via their Picture Archiving and Communication System (PACS) worklist. The decision to use cmTriage codes and how to use on Triage codes is ultimately up to the radiologist. cmTriage does not send a proactive alert directly to the radiologist.
Radiologists are responsible for reviewing each exam on a diagnostic viewer according to the current standard of care.
cmTriage is limited to the categorization of exams, does not provide any diagnostic information beyond triage and prioritization, does not remove images from the radiologist's worklist, and should not be used in lieu of full patient evaluation, or relied upon to make or confirm diagnosis.
cmTriage is for prescription use only.
Product codes (comma separated list FDA assigned to the subject device)
OFM
Device Description
CureMetrix's cmTriage is a radiological computer-assisted triage and notification software device. Digital two-dimensional (2D) mammograms are captured by a Full-Field Digital Mammography (FFDM) system and deposited on the PACS. The CureMetrix image forwarding software, acting as a PACS listener, receives a copy of the mammography DICOM image(s), creates a local copy of the image(s), de-identifies the local copy, transmits the local copy, transmits the local copy to the CureMetrix cloud, and then deletes the local copy.
Within the CureMetrix cloud, the cmTriage service receives the DICOM image(s), groups them by study, analyzes the image(s) within the study, and produces a result for each study. The results are produced in the form of DICOM Structured Report (SR) file with a cmTriage result.
The result file is encrypted and transmitted from the CureMetrix cloud back to cmEdge where it is decrypted and reassociated with the original study. The DICOM SR is then routed to the PACS.
Once the PACS receives the DICOM SR, the file is opened, the cmTriage code ("Impression Description") is extracted for the exam, and the worklist column for the exam is updated. The cmTriage code will either indicate "Suspicious" or "" (blank).
Within the PACS worklist the cmTriage code can be displayed in a separate column. Each PACS may have different features and functionality depending on the manufacturer which are outside of the scope and control of CureMetrix and cmTriage. However, in general, at a minimum, the user is able to sort their worklist based on values in columns. This sorting functionality (if present) would allow the radiologist to group Suspicious exams together. The device does not alter the original medical image and is not intended to be used as a diagnostic device.
The standard of care for breast cancer screening in the US is quickly becoming one in which both FFDM and digital breast tomosynthesis (DBT) are acquired during the exam. However, cmTriage only operates on 2D images. If a site does not use FFDM and 2D but instead only uses 3D and DBT, they will not be able to use this current device.
In summary, the cmTriage device is intended to provide a passive notification through the PACS to the radiologist indicating the existence of a case that may potentially benefit from that radiologist's prioritization.
Mentions image processing
Yes
Mentions AI, DNN, or ML
Mentions AI.
Input Imaging Modality
2D FFDM screening mammograms
Anatomical Site
Breast
Indicated Patient Age Range
Not Found
Intended User / Care Setting
Radiologist / Hospital or Clinic
Description of the training set, sample size, data source, and annotation protocol
Not Found
Description of the test set, sample size, data source, and annotation protocol
The quarantined test data were obtained from multiple clinical sites. The dataset consists of 1255 mammographic studies: 400 biopsy-proven cancer studies of soft-tissue density and 122 studies of microcalcifications) and 855 normal studies (BIRADS 1 and 2 with two-year follow-up of negative diagnosis). Each mammographic study in the quarantined test dataset consists of at most 4 standard-view screening mammograms: (LCC), (LMLO), (RCC), (RMLO). By limiting each study to at most 4 standard-view screening mammograms, we emulate the workflow in standard clinical patient care where 4 screening mammographic images are acquired for each patient.
The quarantined test set was constructed to ensure that confounding factors present in the population were addressed in the data and consistent with the population of women undergoing breast cancer screening. Confounding factors that were considered include: 1) Lesion Type; 2) Lesion Size for Soft-Tissue Densities; 3 Age; and 4) Breast Density. To establish a population baseline for comparison, CureMetrix used the Breast Cancer Surveillance Consortium for age and lesion type and used an NIH study for density.
Summary of Performance Studies (study type, sample size, AUC, MRMC, standalone performance, key results)
Study Type: Retrospective, blinded, multi-center study.
Sample Size: 1255 mammographic studies (400 biopsy-proven cancer studies and 855 normal studies).
Standalone Performance and Key Results:
- Sensitivity: At the default sensitivity of 93%, cmTriage will incorrectly mark 16.1% of the normal cases as suspicious in this enriched test set. Adjusting for a 0.5% cancer rate, at 93% sensitivity, CureMetrix would have indicated that a total of 24% of the exams are suspicious.
- Sensitivity (Primary Endpoint): Observed mean of 86.9% with a 95% confidence interval (CI) of 83.6% to 90.2%.
- Specificity (Primary Endpoint): Observed mean of 88.5% with a 95% CI of 86.4% to 90.7%.
- When cmTriage software is set to a sensitivity of 84.4%, it has a standalone specificity of 94.0%.
- A population-adjusted mark rate of 6.37% at 84.4% sensitivity was well below the target recall rate of 9.60% of radiologists at the same sensitivity.
- Timing Performance (Secondary Endpoint): Average of 3.35 minutes at a network speed of 10Mbits/s upload and 37Mbits/s download.
AUC:
Overall AUC of 0.951 with a 95% CI of 0.937 to 0.964 on this dataset.
- Density 1: AUC 0.964 with a 95% CI of 0.934 to 0.994
- Density 2: AUC 0.964 with a 95% CI of 0.946 to 0.981
- Density 3: AUC 0.940 with a 95% CI of 0.917 to 0.963
- Density 4: AUC 0.958 with a 95% CI of 0.92 to 0.995
- Lesion Type - Mass: AUC 0.941 with a 95% CI of 0.923 to 0.959
- Lesion Type - Calcifications: AUC 0.972 with a 95% CI of 0.958 to 0.985
Key Metrics (Sensitivity, Specificity, PPV, NPV, etc.)
Sensitivity, Specificity
Predicate Device(s): If the device was cleared using the 510(k) pathway, identify the Predicate Device(s) K/DEN number used to claim substantial equivalence and list them here in a comma separated list exactly as they appear in the text. List the primary predicate first in the list.
Reference Device(s): Identify the Reference Device(s) K/DEN number and list them here in a comma separated list exactly as they appear in the text.
Not Found
Predetermined Change Control Plan (PCCP) - All Relevant Information for the subject device only (e.g. presence / absence, what scope was granted / cleared under the PCCP, any restrictions, etc).
Not Found
§ 892.2080 Radiological computer aided triage and notification software.
(a)
Identification. Radiological computer aided triage and notification software is an image processing prescription device intended to aid in prioritization and triage of radiological medical images. The device notifies a designated list of clinicians of the availability of time sensitive radiological medical images for review based on computer aided image analysis of those images performed by the device. The device does not mark, highlight, or direct users' attention to a specific location in the original image. The device does not remove cases from a reading queue. The device operates in parallel with the standard of care, which remains the default option for all cases.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Design verification and validation must include:
(i) A detailed description of the notification and triage algorithms and all underlying image analysis algorithms including, but not limited to, a detailed description of the algorithm inputs and outputs, each major component or block, how the algorithm affects or relates to clinical practice or patient care, and any algorithm limitations.
(ii) A detailed description of pre-specified performance testing protocols and dataset(s) used to assess whether the device will provide effective triage (
e.g., improved time to review of prioritized images for pre-specified clinicians).(iii) Results from performance testing that demonstrate that the device will provide effective triage. The performance assessment must be based on an appropriate measure to estimate the clinical effectiveness. The test dataset must contain sufficient numbers of cases from important cohorts (
e.g., subsets defined by clinically relevant confounders, effect modifiers, associated diseases, and subsets defined by image acquisition characteristics) such that the performance estimates and confidence intervals for these individual subsets can be characterized with the device for the intended use population and imaging equipment.(iv) Stand-alone performance testing protocols and results of the device.
(v) Appropriate software documentation (
e.g., device hazard analysis; software requirements specification document; software design specification document; traceability analysis; description of verification and validation activities including system level test protocol, pass/fail criteria, and results).(2) Labeling must include the following:
(i) A detailed description of the patient population for which the device is indicated for use;
(ii) A detailed description of the intended user and user training that addresses appropriate use protocols for the device;
(iii) Discussion of warnings, precautions, and limitations must include situations in which the device may fail or may not operate at its expected performance level (
e.g., poor image quality for certain subpopulations), as applicable;(iv) A detailed description of compatible imaging hardware, imaging protocols, and requirements for input images;
(v) Device operating instructions; and
(vi) A detailed summary of the performance testing, including: test methods, dataset characteristics, triage effectiveness (
e.g., improved time to review of prioritized images for pre-specified clinicians), diagnostic accuracy of algorithms informing triage decision, and results with associated statistical uncertainty (e.g., confidence intervals), including a summary of subanalyses on case distributions stratified by relevant confounders, such as lesion and organ characteristics, disease stages, and imaging equipment.
0
Image /page/0/Picture/0 description: The image contains the logo of the U.S. Food and Drug Administration (FDA). On the left is the seal of the Department of Health & Human Services. To the right of that is the FDA logo, which consists of the letters "FDA" in a blue square, followed by the words "U.S. FOOD & DRUG" in blue, and then the word "ADMINISTRATION" in a smaller font size below that.
CureMetrix, Inc. % Mr. Kevin Harris CEO 9404 Genesee Avenue, Suite 330 LA JOLLA CA 92037
March 8, 2019
Re: K183285
Trade/Device Name: cmTriage Regulation Number: 21 CFR 892.2080 Regulation Name: Radiological computer aided triage and notification software Regulatory Class: Class II Product Code: OFM Dated: February 4, 2019 Received: February 4, 2019
Dear Mr. Harris:
We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database located at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR 803) for
1
devices or postmarketing safety reporting (21 CFR 4, Subpart B) for combination products (see https://www.fda.gov/CombinationProducts/GuidanceRegulatoryInformation/ucm597488.html; good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR 1000-1050.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to http://www.fda.gov/MedicalDevices/Safety/ReportaProblem/default.htm.
For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/) and CDRH Learn (http://www.fda.gov/Training/CDRHLearn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (http://www.fda.gov/DICE) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely,
Michael D. O'Hara For
Thalia Mills, Ph.D. Director Division of Radiological Health Office of In Vitro Diagnostics and Radiological Health Center for Devices and Radiological Health
Enclosure
2
Indications for Use
510(k) Number (if known) K183285
Device Name
cm Triage
Indications for Use (Describe)
cm Triage is a passive notification for prioritization-only, parallel-workflow software tool used by radiologists to prioritize specific patients within the standard-of-care image worklist for 2D FFDM screening mammograms, cmTriage uses an artificial intelligence algorithm to analyze 2D FFDM screening mammograms and flags those that are suggestive of the presence of at least one suspicious finding at the exam level. These flags are viewed by the radiologist via their Picture Archiving and Communication System (PACS) worklist. The decision to use cmTriage codes and how to use on Triage codes is ultimately up to the radiologist. cmTriage does not send a proactive alert directly to the radiologist.
Radiologists are responsible for reviewing each exam on a diagnostic viewer according to the current standard of care.
cm Triage is limited to the categorization of exams, does not provide any diagnostic information beyond triage and prioritization, does not remove images from the radiologist's worklist, and should not be used in lieu of full patient evaluation, or relied upon to make or confirm diagnosis.
cmTriage is for prescription use only.
Type of Use (Select one or both, as applicable)
Prescription Use (Part 21 CFR 801 Subpart D)
Over-The-Counter Use (21 CFR 801 Subpart C)
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
*DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW."
The burden time for this collection of information is estimated to average 79 hours per response. including the time to review instructions, search existing data sources, gather and maintain the data needed and comblete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@ fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
3
510(k) Summary
| Submitter | CureMetrix, Inc. |
|---|---|
| Contact Person | Kevin Harris |
| 9404 Genesee Ave | |
| Suite 330 | |
| La Jolla, CA 92037 | |
| 619-339-9889 | |
| Kevin@CureMetrix.com | |
| Common Name | Radiological Computer-Assisted Triage and Notification Software |
| Trade Name | cmTriage™ |
| Classification | |
| Name: | Radiological Computer-Assisted Prioritization Software |
| Regulation: | 21 CFR 892.2080 |
| Code: | QFM |
| Class: | Class II |
| Predicate Device | |
| Name: | ContaCT |
| Company: | Viz.AI |
| Code: | QAS |
| 510(k): | DEN170073 |
| 510(k) Submission Number: | K183285 |
Device Description
CureMetrix's cmTriage is a radiological computer-assisted triage and notification software device. Digital two-dimensional (2D) mammograms are captured by a Full-Field Digital Mammography (FFDM) system and deposited on the PACS. The CureMetrix image forwarding software, acting as a PACS listener, receives a copy of the mammography DICOM image(s), creates a local copy of the image(s), de-identifies the local copy, transmits the local copy, transmits the local copy to the CureMetrix cloud, and then deletes the local copy.
Within the CureMetrix cloud, the cmTriage service receives the DICOM image(s), groups them by study, analyzes the image(s) within the study, and produces a result for each study. The results are produced in the form of DICOM Structured Report (SR) file with a cmTriage result.
The result file is encrypted and transmitted from the CureMetrix cloud back to cmEdge where it is decrypted and reassociated with the original study. The DICOM SR is then routed to the PACS.
Once the PACS receives the DICOM SR, the file is opened, the cmTriage code ("Impression Description") is extracted for the exam, and the worklist column for the exam is updated. The cmTriage code will either indicate "Suspicious" or "" (blank).
Within the PACS worklist the cmTriage code can be displayed in a separate column. Each PACS may have different features and functionality depending on the manufacturer which are outside of the scope and control of CureMetrix and cmTriage. However, in general, at a minimum, the user is able to sort their worklist based on values in columns. This sorting functionality (if present) would allow the radiologist to group Suspicious exams together. The device does not alter the original medical image and is not intended to be used as a diagnostic device.
The standard of care for breast cancer screening in the US is quickly becoming one in which both FFDM and digital breast tomosynthesis (DBT) are acquired during the exam. However, cmTriage only operates on 2D images. If a site does not use FFDM and 2D but instead only uses 3D and DBT, they will not be able to use this current device.
In summary, the cmTriage device is intended to provide a passive notification through the PACS to the radiologist indicating the existence of a case that may potentially benefit from that radiologist's prioritization.
4
Intended Use/Indications for Use
cmTriage is a passive notification-only, parallel-workflow software tool used by radiologists to prioritize specific patients within the standard-of-care image worklist for 2D FFDM screening mammograms. cmTrificial intelligence algorithm to analyze 2D FFDM screening mammograms and flags those that are suggestive of the presence of at least one suspicious finding at the exam level. These flags are viewed by the radiologist via their Picture Archiving and Communication System (PACS) worklist. The decision to use cmTriage codes and how to use cmTriage codes is ultimately up to the radiologist. cmTriage does not send a proactive alert directly to the radiologist.
Radiologists are responsible for reviewing each exam on a diagnostic viewer according to the current standard of care.
cmTriage is limited to the categorization of exams, does not provide any diagnostic information beyond triage and prioritization, does not remove images from the radiologist's worklist, and should not be used in lieu of full patient evaluation, or relied upon to make or confirm diagnosis.
cmTriage is for prescription use only.
Technological Characteristics
cmTriage is a software only device that can be segmented into three components: (1) Image Forwarding and Result File Receiving Software, (2) Image Processing and Analysis Software, and (3) Notification Result File.
The Image Forwarding and Result File Receiving Software is configured by the hospital to interact with a healthcare facility's PACS or local DICOM router and is responsible for automatically transmitting a copy of DICOM files from the local router through a secured channel to the Image Processing and Analysis Software based on DICOM metadata.
The Image Processing and Analysis Software component of the device is hosted in the cloud and is reseiving, assembling, processing, and storing DICOM images. This software component includes the software algorithm that is responsible for identifying and quantifying image characteristics that are consistent with a suspicious region of interest in a screening mammogram. When the software algorithm detects imaging characteristics suggestive of a suspicious region of interest, the Image Processing and Analysis Software component sets a flag in the Notification Result File indicating that the exam is "Suspicious" and sends the file back to the Image Forwarding and Result File Receiving Software. While the software algorithm informs the notification process, no other diagnostic information is generated from the algorithm or available to the user beyond the notification.
The Notification Result File is a DICOM SR file that can be used by the PACS or viewing workstation to update the worklist with an indication of that reflects the cmTriage notification of either "Suspicious" or "" (blank).
| Characteristic | Predicate: Viz.AI ContaCT | cmTriage | Similarities or
Differences |
|-----------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| FDA Clearance | DEN170073 | TBD | |
| Clearance Date | 2/13/18 | TBD | |
| Product Code | QAS | QFR | |
| Class | II | II | Same |
| Regulation | 892.2080 | 892.2080 | Same |
| OTC vs. PUO | PUO | PUO | Same |
| | | | |
| Intended
Use.
Indications
for
Use | ContaCT is a notification-only, parallel workflow
tool for use by hospital networks and radiologists
to identify and communicate images of specific
patients to a radiologist, independent of standard
of care workflow.
ContaCT uses an artificial intelligence algorithm to
analyze images for findings suggestive of a pre-
specified clinical condition and to notify an
appropriate medical radiologist of these findings
in parallel to standard of care image
interpretation. Identification of suspected findings
is not for diagnostic use beyond notification.
Specifically, the device analyzes CT angiogram
images of the brain acquired in the acute setting,
and sends notifications to neurovascular
radiologist that a suspected large vessel occlusion
has been identified and recommends review of
those images. Images can be previewed through a
mobile application.
Images that are previewed through the mobile
application are compressed and are for
informational purposes only and not intended for
diagnostic use beyond notification. Notified
radiologists are responsible for viewing non-
compressed images on a diagnostic viewer and
engaging in appropriate patient evaluation and
relevant discussion with a treating physician
before making care-related decisions or requests.
ContaCT is limited to analysis of imaging data and
should not be used in-lieu of full patient evaluation
or relied upon to make or confirm diagnosis. | cmTriage is a passive notification for
prioritization-only, parallel-workflow software
tool used by radiologists to prioritize specific
patients within the standard-of-care image
worklist for 2D FFDM screening mammograms.
cmTriage uses an artificial intelligence algorithm
to analyze 2D FFDM screening mammograms and
flags those that are suggestive of the presence of at
least one suspicious finding at the exam level.
These flags are viewed by the radiologist via their
Picture Archiving and Communication System
(PACS) worklist. The decision to use cmTriage
codes and how to use cmTriage codes is ultimately
up to the radiologist. cmTriage does not send a
proactive alert directly to the radiologist.
Radiologists are responsible for reviewing each
exam on a diagnostic viewer according to the
current standard of care.
cmTriage is limited to the categorization of exams,
does not provide any diagnostic information
beyond triage and prioritization, does not remove
images from the radiologist's worklist, and should
not be used in lieu of full patient evaluation, or
relied upon to make or confirm diagnosis.
cmTriage is for prescription use only. | Similarities
• Used by radiologists
• Identifies specific
patients
• Operates in parallel
to standard of care
workflow
• Uses artificial
intelligence to
analyze images
• Looking for findings
suggestive of a pre-
specified clinical
condition
• Not intended for
diagnostic use
• Radiologists are
responsible for
viewing images on a
diagnostic viewer
• Limited to analysis
of imaging data and
should not be used
in-lieu of full patient
evaluation or relied
upon to make or
confirm diagnosis
Noted difference:
cmTriage notification
is passive vs. proactive
and cmTriage codes
are accessed by
radiologist through
PACS worklist and
used for prioritization. |
| Technical
Method | The device provides triage or notification that is
informed by machine learning, artificial
intelligence or other image analysis algorithms | The device provides triage or notification that is
informed by machine learning, artificial
intelligence or other image analysis algorithms | Same |
| Target Area | The device operates on radiological images of the
human body. | The device operates on radiological images of the
human body. | Same |
| Anatomical Site | Head | Breast | Different anatomic site |
| Where Used | Hospital or Clinic | Hospital or Clinic | Same |
| User Population | Radiologist | Radiologist | Same |
| Software | Device is software only | Device is software only | Same |
| Software Level
of Concern | Moderate | Moderate | Same |
| Communication
with Patient | Communicates images of patients to a radiologist. | Does not communicate images of patients.
cmTriage passively notifies the radiologist via the
PACS worklist, whereas ContaCT notifies a pre-
identified specialist via direct message. | Noted difference: No
images are
communicated from
cmTriage to
radiologist. Only
cmTriage codes are
returned for viewing
within the PACS
worklist. |
| Notification/
Priority | Yes | Yes | cmTriage prioritizes
based on suspicious
findings; ContaCT
connects a specialist to
a patient that may be
in need of urgent
treatment for stroke.
ContaCT proactively
notifies radiologist.
cmTriage passively
notifies and thus
radiologist is
responsible for
accessing cmTriage
codes via PACS
worklist. |
| Preview Images | Presentation of notification and preview of the
study for initial assessment not meant for
diagnostic purposes.
The device operates in parallel with the standard
of care, which remains the default option for all
cases. | Presentation of notification and preview of the
study for initial assessment not meant for
diagnostic purposes.
The device operates in parallel with the standard
of care, which remains the default option for all
cases. | Same |
| Alteration of
original image | No | No | Same |
| Image Viewing | Images previewed through a mobile app | Images viewed on existing viewing workstation | cmTriage codes are
viewed through PACS
worklist |
Summary of Comparison to Predicate Device
5
6
Image /page/6/Picture/0 description: The image shows the logo for CureMetrix. The logo features a blue and green design with a stylized "M" that incorporates a heartbeat waveform. The text "CureMetrix" is written in green, with the "C" connected to the "M" symbol.
Performance Data
CureMetrix conducted a retrospective, blinded, multi-center study of the cmTriage software. A statistical analysis was conducted to estimate to the number of cancer and normal studies required to achieve the stated primary endpoint 95% confidence interval (Cl) and to ensure that our detailed statistical analyses are consistent with a priori estimates.
Sensitivity and Specificity targets were set based on the BCSC study. A primary endpoint goal established to validate that cmTriage operates at a 95% Cl for both sensitivity above the 80% Cl reported in the BCSC. Further the goal was refined to validate that at a sensitivity of 86.9% corresponding to the median sensitivity reported by the BCSC, the specificity would be comparable to the median specificity reported by the bottom of the 80% Cl. For cmTriage a sensitivity of 86.9% corresponds to a specificity of 88.5% -- comparable to the median specificity of 88.9% of the BCSC.
A secondary endpoint was established to determine time performance of cmTriage to ensure that mammograms can be processed, and notification results returned for use by radiologists within minutes which is clinically acceptable.
The population of cases (exams) was selected to representative cross-section of patients based on age, density, and lesion type mix typically seen in a screening population was enriched with biopsy confirmed cancers to more accurately measure the efficacy of the cmTriage software.
The quarantined test data were obtained from multiple clinical sites . The dataset consists of 1255 mammographic studies: 400 biopsy-proven cancer studies of soft-tissue density and 122 studies of microcalcifications) and 855 normal studies (BIRADS 1 and 2 with two-year follow-up of negative diagnosis). Each mammographic study in the quarantined test dataset consists of at most 4 standard-view screening mammograms: (LCC), (LMLO), (RCC), (RMLO). By limiting each study to at most 4 standard-view screening mammograms, we emulate the workflow in standard clinical patient care where 4 screening mammographic images are acquired for each patient.
7
The quarantined test set was constructed to ensure that confounding factors present in the population were addressed in the data and consistent with the population of women undergoing breast cancer screening. Confounding factors that were considered include: 1) Lesion Type; 2) Lesion Size for Soft-Tissue Densities; 3 Age; and 4) Breast Density. To establish a population baseline for comparison, CureMetrix used the Breast Cancer Surveillance Consortium for age and lesion type and used an NIH study for density.
When the cmTriage software is set to a sensitivity of 84.4%, in direct comparison to the BCSC results, cmTriage has a standalone specificity of 94.0% and would indicate that only 4.1% of the total number of cases are incorrectly marked as "suspicious" on the test set. However, this is an enriched data set. Adiusting for a 0.5% cancer rate, at 84.4% sensitivity. CureMetrix would have indicated that a total of 6.4% of the exams are suspicious. Thus, in stand-alone mode CureMetrix is operating at higher specificity than radiologists. At the default sensitivity of 93% (specificity = 76.3%), cmTriage will incorrectly mark 16.1% of the normal cases as suspicious in this enriched test set. Adjusting for a 0.5% cancer rate, at 93% sensitivity, CureMetrix would have indicated that a total of 24% of the exams are suspicious.
Based on the results demonstrated above, the primary endpoints of sensitivity and specificity exceeded the targets. Sensitivity was observed at mean of 86.9% with a 95% confidence interval (CI) of 83.6% to 90.2%. Specificity was observed at a mean of 88.5% with a 95% Cl of 86.4% to 90.7%. The low end of our 95% Cl for sensitivity and specificity, 83.6% and 86.4% respectively, exceeded BCSC's low end of their 80% Cl (80.7% and 82.6% for sensitivity and specificity, respectively).
A population-adjusted mark rate of 6.37% at 84.4% sensitivity was well below the target recall rate of 9.60% of radiologists at the same sensitivity.
Overall cmTriage was able to demonstrate an area under the curve (AUC) of 0.951 with a 95% Cl of 0.937 to 0.964 on this dataset. This AUC held across density 1: AUC 0.964 with a 95% Cl of 0.934 to 0.994; Density 2: AUC 0.964 with a 95% Cl of 0.946 to 0.981; Density 3: AUC 0.940 with a 95% Cl of 0.917 to 0.963; and Density 4: AUC 0.958 with a 95% Cl of 0.92 to 0.995. This AUC also held across lesion types: Mass: AUC 0.941 with a 95% Cl of 0.923 to 0.959; Calcifications: AUC 0.972 with a 95% CI of 0.958 to 0.985.
Lastly the timing performance for the device was shown to be an average of 3.35 minutes at a network speed of 10Mbits/s upload and 37Mbits/s download. This is well within the clinical operations of breast cancer screening.
These data in combination establish effective triage analysis algorithm and notification functionality of the cmTriage device as compared to the standard of care.
Conclusion
The subject cmTriage and the ContaCT predicate devices are both software only devices intended to aid in triage of radiological images of specific patients to a radiologist, independent of care workflow. The labeling of both devices are limited to the categorization of exams and are not to be used in-lieu of full patient evaluation or relied upon to make or confirm diagnosis.
Both devices operate in parallel to the standard of care workflow in the sense that they do not change the original image, do not provide any marking on the output preview, and do not remove images from the standard of care "First in First out" (FIFO) queue, thus not disturbing standard interpretation of the images by trained clinicians. The minor differences between the subject device and the predicate raise no new issues of safety or effectiveness. In addition, performance testing demonstrates that the cmTriage performs as intended.
The cmTriage device is thus substantially equivalent to the ContaCT predicate.