Search Results

syngo.CT CaScoring is an image analysis software package for evaluating CT data sets.

The software is designed to support the physician in evaluating and documenting calcified coronary lesions, using standard or low-dose spiral or sequential CT scanning data sets. After loading noncontrasted cardiac CT images, syngo. CT CaScoring can be used to mark calcified coronary lesions and to allocate each lesion to one of several coronary arteries, that is, the right coronary artery (RCA), the left main coronary artery (LM), the left anterior descending artery (LAD), and the left circumflex artery (CX).

syngo. CT CaScoring calculates the Agatston equivalent score, the mass score and the volume score of each coronary artery as well as the corresponding total scores all coronary arteries. syngo.CT CaScoring allows the user to create a paper report including the calcium scoring data, any user-documented images, cited literature and additional relevant information.

The post-processing application syngo.CT CaScoring SOMARIS/8 VB50 is designed to support the physician in evaluating and documenting calcified coronary lesions. After loading non-contrasted cardiac CT images, syngo.CT CaScoring can be used to interactively mark calcified coronary lessions and to allocate each lesion to one of several coronary arteries, that is, the right coronary artery (RCA), the left main coronary artery (LM), the left anterior descending artery (LAD), and the left circumflex artery (CX). syngo.CT CaScoring calculates the Agatston-equivalent score, the mass score and the volume score of each coronary artery as well as the corresponding total scores across all coronary arteries. syngo.CT CaScoring allows the user to create a paper report including the calcium scoring data, any user-documented images, cited literature and additional relevant information.

For the current software version SOMARIS/8 VB50 one major and one minor change have been implemented:

• . Since the last 510(k) clearance of the predicate device (syngo.CT Calcium Scoring SOMARIS/8 VB40, K192763, clearance date 12/17/2019) the algorithm to precompute the calcium score has been enhanced and extended. In the subject device, the CaScoring algorithm was extended to label coronary calcifications as belonging to either the left main, left anterior descending, left circumflex or right coronary artery.
• . This version contains UI (user-interface) modifications.

The provided text outlines the performance evaluation of the syngo.CT CaScoring algorithm. Although it mentions a "bench test" and a "reader study," the details regarding the acceptance criteria and the studies are somewhat general. Based on the information available, here's a structured description:

Acceptance Criteria and Device Performance Study

The syngo.CT CaScoring device aims to automatically score and assign coronary calcifications to specific coronary arteries. The enhanced algorithm's performance was evaluated through a "bench test" and a "reader study."

1. Table of Acceptance Criteria and Device Performance

2. Sample Size and Data Provenance

• Test Set Sample Size:
  • Bench Test: Not explicitly stated, but described as "considerably more diverse" than the reader study population, implying a larger and/or more varied dataset.
  • Reader Study: Not explicitly stated, but mentioned to be "from a single, modern scanner."
• Data Provenance: The document does not specify the country of origin of the data. It also does not explicitly state whether the data was retrospective or prospective. However, for a bench test and reader study evaluating an enhanced algorithm, it is common to use retrospective data.

3. Number of Experts and Qualifications for Ground Truth

• Number of Experts: The reader study involved "three individual readers."
• Qualifications of Experts: Not explicitly stated within the provided text. It is generally assumed that "readers" in a medical imaging context are qualified medical professionals like radiologists or cardiologists.

4. Adjudication Method for the Test Set

• The reader study mentions comparing the performance of "the three individual readers compared to their consensus." This strongly implies a consensus-based adjudication method for establishing the ground truth. A common consensus method is a (N-1) rule or a majority vote among multiple experts (e.g., 2 out of 3, 3 out of 4). The text doesn't specify the exact method (e.g., 2+1 or 3+1), but the use of "their consensus" indicates that the three readers arrived at an agreed-upon ground truth.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Yes, a multi-reader multi-case (MRMC) study was implicitly part of the "reader study." The study involved "three individual readers" and compared their performance (as part of a consensus group) against the algorithm.
• Effect Size: The text states: "No statistically relevant difference between the performance of the three individual readers compared to their consensus, and the algorithm compared to the consensus was found." This suggests that the AI assistance (or the algorithm's standalone performance compared to human consensus) did not statistically significantly improve or worsen the performance for the specific metrics evaluated, particularly for vessel-specific assignments where the task was similarly difficult for humans and the algorithm. It does not provide a specific quantitative effect size (e.g., AUC increase, accuracy percentage point change) for human reader improvement with versus without AI assistance. The focus seems to be on whether the algorithm performs comparably to expert consensus.

6. Standalone (Algorithm Only) Performance

• Yes, a standalone performance evaluation was done. The "bench test" assessed the "adequate and acceptable performance of the automatic scoring algorithm" independently. The "reader study" also compared the algorithm's performance directly against the human consensus, indicating an assessment of its standalone capabilities.

7. Type of Ground Truth Used

• The ground truth for the test sets (both bench test and reader study) was established by expert consensus annotations. The text explicitly mentions "consensus annotations" for comparing automatically assigned LM scores and for comparing reader performance.

8. Sample Size for the Training Set

• The sample size for the training set is not specified in the provided text. It only mentions that the "automatic scoring algorithm was retrained on re-annotated data as part of the vessel assignment extension."

9. How Ground Truth for Training Set was Established

• The ground truth for the training set was established through re-annotation. The text states, "the automatic scoring algorithm was retrained on re-annotated data." While the specifics of who performed the re-annotation (e.g., experts, number of experts, adjudication) are not detailed, it implies data that underwent a human review and labeling process to serve as the training truth.

Ask a specific question about this device

This computed tomography system is intended to generate and process cross-sectional images of patients by computer reconstruction of x-ray transmission data.

The images delivered by the system can be used by a trained physician as an aid in diagnosis. The images delivered by the system can be used by trained staff as an aid in diagnosis, treatment preparation and radiation therapy planning.

This CT system can be used for low dose lung cancer screening in high risk populations *

• As defined by professional medical societies. Please refer to clinical literature, including the results of the National Lung Screening Trial (N Engl J Med 2011: 365:395-409) and subsequent literature, for further information.

Scan&GO:
The in-room scan application is a planning and information system designed to perform the necessary functions required for planning and controlling scans of supported Siemens CT scanners. It allows users to work in close proximity to the scanner.

The in-room scan application runs on standard information technology hardware and software, utilizing the standard information technology operating systems and user interface. Communication and data exchange are done using special protocols.

Siemens intends to update software version, SOMARIS/10 syngo CT VA30 (Update) for Siemens SOMATOM Computed Tomography (CT) Scanner Systems with mobile workflow and 3D Camera options.

This update includes support of additional hardware for the go. platform and includes reuse of optional postprocessing applications for Recon&GO for all scanners subject of this submission.

The SOMATOM CT Scanner Systems that support the same software platform update include:

• . SOMATOM go.Up
• . SOMATOM go.Now
• SOMATOM go.Top .
• SOMATOM go.All .
• . SOMATOM ao.Sim
• . SOMATOM go.Open Pro
• SOMATOM X.cite
• Scan&GO Mobile Medical Application (optional mobile workflow component) .

The subject device SOMATOM go. platform and SOMATOM X.cite with SOMARIS/10 syngo CT VA30 (update) are Computed Tomography X-ray Systems which feature one continuously rotating tubedetector system and function according to the fan beam principle. The SOMATOM go. platform and SOMATOM X.cite with software SOMARIS/10 syngo CT VA30 (update) produces CT images in DICOM format. These images can be used by trained staff for post-processing applications commercially distributed by Siemens Healthcare and other vendors. These images aid in diagnosis, treatment preparation and therapy planning support (including, but not limited to, Brachytherapy, Particle including Proton Therapy, External Beam Radiation Therapy, Surgery), The computer system delivered with the CT scanner is able to run optional post processing applications.

The Scan&GO mobile workflow is an optional planning and information software designed to perform the necessary functions required for planning and controlling of the SOMATOM X.cite and SOMATOM go. platform CT scanners. Scan&GO can be operated on a Siemens provided tablet or personal computer that meets certain minimum technical requirements. It allows users to work in close proximity to the scanner and the patient.

The software version for the SQMATOM go, platform and SOMATOM X.cite, syngo CT VA30 (update) (SOMARIS/10 syngo CT VA30 (update)), is a command-based program used for patient management, data management, X-ray scan control, image reconstruction, and image archive/evaluation. The software platform SOMARIS/10 syngo CT VA30 (update) is designed to support a software plugin interface to reuse a subset of stand-alone, cleared processing software applications.

The Siemens Medical Solutions USA, Inc. K200524 submission describes an update to the SOMATOM X.cite and SOMATOM Go Platform CT Scanners (software version SOMARIS/10 syngo CT VA30). The submission focuses on demonstrating substantial equivalence to previously cleared devices through non-clinical testing.

Here's an analysis of the acceptance criteria and the study that proves the device meets them, based on the provided text:

1. Table of Acceptance Criteria and the Reported Device Performance:

The document primarily discusses verification and validation testing, with acceptance criteria tied to the fulfillment of requirements and comparable performance to predicate devices. Specific quantitative acceptance criteria are not explicitly detailed in a separate table format within the provided text, but rather described in the "Testing Performed" column of Table S5-06.

2. Sample Sizes Used for the Test Set and the Data Provenance:

• FAST Integrated Workflow (3D Camera): "Clinical data based software validation" was conducted. The specific number of cases or patients is not quantified in the provided text. Data provenance is not explicitly stated beyond "clinical data based software validation," implying it's likely from a clinical setting, but country of origin or retrospective/prospective nature is not specified.
• Contrast Media Protocol: The evaluation was based on factory protocols and comparison to approved drug labeling. This does not involve a "test set" in the traditional sense of patient data.
• Scan&GO Supported Hardware: "Bench test." The sample size for this is not specified. Data provenance is a bench test, presumably conducted by the manufacturer.
• Wireless Coexistence Testing: No specific sample size (number of wireless devices or test scenarios) is mentioned.
• Customer Use Testing:
  • Internal Clinical Use Test: "The CT scanner customer environment is simulated in Siemens Test Cabins. For such a test, customers with clinical expertise are typically invited to perform tests." The number of "customers with clinical expertise" or individual test cases is not quantified.
  • External Clinical Use Test: "The CT scanner is tested in the environment of the clinic/hospital. Typically we perform these tests with selected customer before rollout of the CT scanner." The number of "selected customer" sites or test cases is not quantified. Data provenance is clinical environments.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts:

• For the FAST Integrated Workflow, the ground truth for "isocenter deviation" and "landmark boundaries" was established presumably by comparison to a reference or manual measurements, but the document does not specify the number or qualifications of experts involved in establishing this ground truth.
• For the Contrast Media Protocol, the ground truth is established by the "approved labeling of Ultravist® or Visipaque®." No external experts were involved in establishing new ground truth for this test.
• For Customer Use Testing, "customers with clinical expertise" were invited for internal tests, and "selected customer" (presumably clinical staff) performed external tests. The exact number and specific qualifications (e.g., "radiologist with 10 years of experience") are not provided.

4. Adjudication Method for the Test Set:

The document does not describe any explicit adjudication method (like 2+1 or 3+1) for any of the described tests. Performance for FAST Integrated Workflow appears to be based on direct measurement comparison. For customer use tests, it's implied that feedback from "customers with clinical expertise" determined meeting acceptance criteria, but no formal adjudication process is detailed.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done:

No, the document does not describe a Multi-Reader Multi-Case (MRMC) comparative effectiveness study to measure the effect size of how much human readers improve with AI vs. without AI assistance. The study focuses on the technical performance of the device's features.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done:

Yes, the testing described appears to be primarily focused on standalone algorithm/device performance for the modifications. For example:

• FAST Isocentering and FAST Range: The measurement of deviation is a direct assessment of the algorithm's accuracy in proposing an isocenter or landmark, independent of a human reader's interpretation improvement.
• Recon&GO features are noted as "post-processing algorithms" or "inline results methods" that appear to be integrated for improved workflow, implying their standalone function in generating these results.
• Software Specifications and Wireless Coexistence testing inherently evaluate the algorithm and system performance without explicit human-in-the-loop assessment as the primary outcome.

7. The Type of Ground Truth Used:

• FAST Integrated Workflow: The ground truth for isocenter deviation and landmark boundaries seems to be based on a reference standard or manual measurements to which the device's output is compared. This is an implicit form of expert consensus or highly accurate measurement.
• Contrast Media Protocol: The ground truth is the approved labeling of Ultravist® or Visipaque®, which serves as a regulatory and clinical standard.
• Scan&GO Supported Hardware: The ground truth is the information shown on tablets, serving as a reference for comparison of the in-room monitor's display.
• Software Specifications: Ground truth is defined by the software requirements/specifications themselves.
• Wireless Coexistence Testing: Ground truth is adherence to technical standards and successful communication parameters.
• Customer Use Testing: Ground truth seems to be based on expert opinion/feedback from "customers with clinical expertise" or "selected customer" in clinical environments, confirming the safety and effectiveness of the intended use.
• National Lung Screening Trial (NLST): This is referenced as supportive data for lung cancer screening indications. The ground truth for this external study (NLST) would have been clinical outcomes data (e.g., biopsy-confirmed cancer, mortality). However, it's important to note this is not the ground truth created for the current device's primary testing but rather cited clinical evidence supporting an indication for use.

8. The Sample Size for the Training Set:

The document does not specify any sample sizes for training sets. The submission describes updates to existing CT scanner systems and software, and the testing focuses on the verification and validation of these updates against predicate devices and defined requirements. This implies the core algorithms were likely developed and trained prior to this specific update.

9. How the Ground Truth for the Training Set Was Established:

Since no training set is mentioned, the method for establishing its ground truth is not described in this document.

Ask a specific question about this device