Search Results

The Monaco system is used to make treatment plans for patients with prescriptions for external beam radiation therapy. The system calculates dose for photon treatment plans and displays, on-screen and in hard-copy, two- or three-dimensional radiation dose distributions inside patients for given treatment plan set-ups.

The Monaco product line is intended for use in radiation treatment planning. It uses generally accepted methods for:

• contouring
• image manipulation
• simulation
• image fusion
• plan optimization
• QA and plan review

Monaco is a radiation treatment planning system that first received FDA clearance in 2007 (K071938). The modified system received clearance in 2009, when Volumetric Modulated Arc Therapy (VMAT) planning capability was added (K091179). The Monaco system accepts patient diagnostic imaging data and "source" dosimetry data from a linear accelerator. The system then permits the user to display and define (contour) the target volume to be treated and critical structures which must not receive above a certain level of radiation on these diagnostic images.

Based on the prescribed dose, the user, a Dosimetrist or Medical Physicist, can create multiple treatment scenarios involving the number, position(s) and energy of radiation beams and the use of a multileaf collimator (MLC) between the source of radiation and the patient to shape the beam. The Monaco system then produces a display of radiation dose distribution within the patient, indicating doses to the target volume and surrounding structures. The "best" plan satisfying the prescription is then selected, one that maximizes dose to the target volume while minimizing dose to surrounding healthy volumes.

The Monaco RTP System is a radiation treatment planning system. Here's a breakdown of its acceptance criteria and the supporting study:

1. Table of Acceptance Criteria and Reported Device Performance

The provided summary does not explicitly list distinct, quantifiable acceptance criteria with corresponding performance metrics in a readily extractable table format for dose calculation or planning accuracy. Instead, it states that verification tests were "written and executed to ensure that the system is working as designed" and that "Pass/fail requirements and results of this testing can be found in section 18 of this submission." However, Section 18 is not included in the provided text.

Based on the available information, the general performance criteria can be inferred as:

2. Sample Size Used for the Test Set and the Data Provenance

The summary states that "Clinical trials were not performed as part of the development of this product." Instead, "Algorithm testing was performed to compare calculated against measured doses," and "clinically oriented validation test cases were written and executed in-house by CMS customer support personnel."

Therefore:

• Test Set Sample Size: Not specified in terms of number of patient cases. The testing involved "algorithm testing" (comparing calculated vs. measured doses) and an unspecified number of "clinically oriented validation test cases."
• Data Provenance: Not explicitly stated regarding origin (e.g., country). However, the testing was "in-house" by the manufacturer (Computerized Medical Systems, Inc., USA). This implies the data used for the algorithm and validation tests would be internally generated or sourced. The context suggests it was not patient data from clinical settings.
• Retrospective/Prospective: The testing appears to be retrospective in the sense that it did not involve prospective human subjects but rather validation against pre-existing data (measured doses) or simulated/representative cases for the "clinically oriented validation test cases."

3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts

• Number of Experts: Not explicitly stated. The "clinically oriented validation test cases" were "written and executed in-house by CMS customer support personnel."
• Qualifications of Experts: The personnel were "CMS customer support personnel." While they handled "clinically oriented" test cases, their specific clinical qualifications (e.g., medical physicist, dosimetrist, or specific years of experience) are not provided. The summary also notes that plans are "reviewed and approved by qualified clinicians" in a clinical setting, but this refers to post-approval clinical use, not the ground truth establishment for the premarket testing.

4. Adjudication Method for the Test Set

The document does not describe an adjudication method for establishing ground truth for the test set. Since the testing involved "algorithm testing" comparing calculated against measured doses, and "clinically oriented validation test cases" executed in-house, it is unlikely a multi-expert adjudication method was employed in the traditional sense. The "ground truth" for algorithmic accuracy would be established by the physical measurements, and for validation cases, by adherence to predefined clinical expectations or specifications.

5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study Was Done, If so, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance?

No, a Multi Reader Multi Case (MRMC) comparative effectiveness study was not done. The device is a radiation treatment planning system, not an AI-assisted diagnostic tool for human readers. Its primary function is to calculate dose and aid in plan creation, not to improve human reader performance in interpreting images or making diagnoses.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done

Yes, a form of standalone performance assessment was done. "Algorithm testing was performed to compare calculated against measured doses to ensure dose calculation accuracy." This directly evaluates the algorithm's output (calculated dose) against an objective standard (measured dose) without a human-in-the-loop decision-making process. The "clinically oriented validation test cases" also assessed the system's ability to produce acceptable plans based on defined criteria.

7. The Type of Ground Truth Used

• For Algorithm Testing: The ground truth was measured doses. The summary states "Algorithm testing was performed to compare calculated against measured doses." This implies physical measurements were used as the gold standard.
• For "Clinically Oriented Validation Test Cases": The ground truth was based on predefined clinical expectations/specifications or internal standards established by the CMS customer support personnel who wrote and executed these cases.

8. The Sample Size for the Training Set

The document does not specify a separate "training set" sample size. The Monaco system is a radiation treatment planning system that calculates dose and optimizes plans based on established physics models and algorithms. It does not appear to be a machine learning model that requires a distinct "training set" in the common understanding of AI devices. Its development would involve calibration, verification, and validation, rather than a training process on a large dataset of patient images or outcomes.

9. How the Ground Truth for the Training Set Was Established

Since a "training set" in the context of machine learning is not mentioned or implied for this device, the method for establishing its ground truth is not applicable/not provided. The system's foundational accuracy would stem from its underlying physical models and their calibration, which would involve experimental data and established physics principles, rather than a labeled training dataset.

Ask a specific question about this device

The XiO RTP System is used to create treatment plans for any cancer patient for whom external beam radiation therapy or brachytherapy has been prescribed. The system will calculate and display, both on-screen and in hard-copy, either two- or three-dimensional radiation dose distributions within a patient for a given treatment plan set-up.

The XiO Radiation Treatment Planning system accepts a) patient diagnostic imaging data from CT and MR scans, or from films, and b) "source" dosimetry data, typically from a linear accelerator. The system then permits the user to display and define (contour) the target volume, which is the structure to be treated, and critical structures, or organs-at-risk, to which radiation dose must be limited. Based on the dose prescribed, the user, typically a Dosimetrist or Medical Physicist, can then create multiple treatment scenarios involving the type, number, position(s) and energy of radiation beams and the use of treatment aids between the source of radiation and the patient (wedges, blocks, ports, etc.). The XiO system produces a display of radiation dose distribution within the patient, indicating doses to the target volume and critical structures. Appropriate clinical personnel select the plan that they believe most effectively maximizes dose to the target volume while minimizing dose to critical structures. The parameters of the plan are output in hard-copy format for later reference placed in the patient file. This Premarket Notification addresses the addition of the Proton Spot Scanning. XiO provides the user with the ability to choose between multiple dose calculation algorithms, selecting the algorithm most appropriate for the given clinical scenario.

The provided K102216 submission for the XiO RTP System with Proton Spot Scanning focuses on the safety and effectiveness of a radiation treatment planning system. Therefore, the "acceptance criteria" and "device performance" in this context refer to the accuracy of the dose calculation algorithm and the successful execution of verification tests, rather than typical clinical performance metrics like sensitivity, specificity, or AUC which are common for diagnostic AI devices.

Here's an analysis of the acceptance criteria and study information provided:

1. Table of Acceptance Criteria and Reported Device Performance

2. Sample Size Used for the Test Set and Data Provenance

The document states:

• "Algorithm testing was performed to compare calculated against measured doses to ensure dose calculation accuracy."
• "Clinically oriented validation test cases were written and executed in-house by CMS customer support personnel."

This indicates the test sets were synthetically created or derived from experimental measurements in a lab setting (for algorithm performance) and internal validation cases rather than patient data.

• Sample Size: Not explicitly stated for either the algorithm testing or the clinically oriented validation test cases. It is implied there were sufficient cases to validate the algorithms and system functionality.
• Data Provenance: The data for algorithm testing would likely be from physical measurements in a lab (e.g., phantom studies) against which the calculated doses are compared. The "clinically oriented validation test cases" were "written and executed in-house" by the manufacturer (CMS customer support personnel), suggesting simulated clinical scenarios or predefined test inputs mirroring real-world conditions, rather than primary patient data.
• Retrospective or Prospective: Both types of testing (algorithm and validation test cases) are described as retrospective analyses or internal validation exercises on predefined scenarios/data, not prospective studies on real patients.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

• Number of Experts: Not explicitly stated.
• Qualifications of Experts: The ground truth for dose calculation accuracy would be established by dosimetrists or medical physicists who perform the physical measurements of radiation dose distributions in a lab setting. The "clinically oriented validation test cases" were executed by "CMS customer support personnel," which might include individuals with dosimetric or clinical application knowledge, but their specific qualifications are not detailed beyond "customer support personnel." Given the "Major Level of Concern" for this device, a qualified medical physicist would likely have overseen or been involved in the interpretation of algorithm accuracy.

4. Adjudication Method for the Test Set

Not applicable in the conventional sense for a typical AI diagnostic device. The "ground truth" for this device is the measured physical dose or the correct output based on system specifications for verification tests. Discrepancies would be resolved by re-measurement, re-analysis, or debugging, not by expert consensus adjudication of human interpretation.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and the Effect Size

No. An MRMC study is not relevant for this type of device (a radiation treatment planning system). The device assists human readers (dosimetrists/medical physicists) in planning treatments but does not present images for interpretation in a diagnostic context. Its primary function is calculation and display of dose distributions.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done

Yes, in essence. The "algorithm testing" performed to compare calculated against measured doses is a standalone performance assessment of the core dose calculation engine. This evaluates the algorithm's accuracy independent of a human user's input or interpretation after the calculation.

7. The Type of Ground Truth Used

• Algorithm Testing: Measured physical dose distributions (e.g., from phantom studies, ion chamber measurements, film dosimetry). This is a form of empirical measurement/experimental data.
• Verification Tests: The expected correct system behavior and output as defined by the system's design specifications. This can be considered definitive system specification ground truth.
• Clinically Oriented Validation Test Cases: Predefined correct treatment plans or expected outcomes based on established clinical practice and physics principles. This combines elements of expert consensus (on what constitutes a correct plan) and physics-based ground truth.

8. The Sample Size for the Training Set

Not applicable. This document describes a traditional software upgrade to a radiation treatment planning system, not a machine learning or AI algorithm that requires a "training set" in the common sense. The "training" of such a system involves the development and calibration of physics-based dose calculation algorithms against physical models and experimental data, not supervised learning from a dataset of labeled clinical cases.

9. How the Ground Truth for the Training Set Was Established

Not applicable. As noted above, this is not an AI/ML device that uses a "training set" in the context of supervised learning. The underlying physics models and algorithms are developed based on established scientific principles, physical measurements, and mathematical formulations, which constitute their "ground truth" or foundational knowledge.

Ask a specific question about this device

The XiO RTP System is used to create treatment plans for any cancer patient for whom external beam radiation therapy or brachytherapy has been prescribed. The system will calculate and display, both on-screen and in hard-copy, either two- or three-dimensional radiation dose distributions within a patient for a given treatment plan set-up.

The XiO Radiation Treatment Planning system accepts a) patient diagnostic imaging data from CT and MR scans, or from films, and b) "source" dosimetry data, typically from a linear accelerator. The system then permits the user to display and define (contour) the target volume, which is the structure to be treated, and critical structures, or organs-atrisk, to which radiation dose must be limited.

Based on the dose prescribed, the user, typically a Dosimetrist or Medical Physicist, can then create multiple treatment scenarios involving the type, number, position(s) and energy of radiation beams and the use of treatment aids between the source of radiation and the patient (wedges, blocks, ports, etc.). The XiO system produces a display of radiation dose distribution within the patient, indicating doses to the target volume and critical structures. Appropriate clinical personnel select the plan that they believe most effectively maximizes dose to the target volume while minimizing dose to critical structures. The parameters of the plan are output in hard-copy format for later reference placed in the patient file.

This Premarket Notification addresses the addition of the Electron Monte Carlo dose calculation algorithm. XiO provides the user with the ability to choose between multiple dose calculation algorithms, selecting the algorithm most appropriate for the given clinical scenario. More accurate dose computation increases the probability that disease will be effectively treated and decreases the probability of undesirable side effects. No algorithm produces a perfectly accurate description of dose distribution; all algorithms have limitations, which are generally well understood and documented in scientific literature.

The addition of the Monte Carlo dose calculation algorithm gives users a new option for electron treatment plans. The algorithm represents the state of the art in radiation treatment planning and is widely recognized as the most accurate method currently available for computing the dose delivered by a beam of high-energy electrons.

The provided text describes the XiO RTP System with the addition of an Electron Monte Carlo dose calculation algorithm. While it mentions verification testing and algorithmic accuracy, it does not explicitly define acceptance criteria in a quantitative table or detail a study that directly proves the device meets such criteria through a clinical or reader study.

Here's a breakdown of the available information against your requested points:

1. Table of Acceptance Criteria and Reported Device Performance

The document does not provide a formal table of acceptance criteria or specific quantitative performance metrics beyond stating that XiO "successfully passed verification testing" and that algorithmic testing was performed to "ensure dose calculation accuracy." No specific numerical targets for accuracy or precision are given.

2. Sample Size Used for the Test Set and Data Provenance

The document states:

• "Algorithm testing was performed to compare calculated against measured doses to ensure dose calculation accuracy."
• "In addition, clinically oriented validation test cases were written and executed in-house by CMS customer support personnel."

The specific sample size for this "algorithm testing" or these "validation test cases" is not mentioned.
The data provenance (e.g., country of origin, retrospective or prospective) for this testing is not specified. It's implied to be internal data ("in-house by CMS customer support personnel") rather than real-world patient data from specific countries.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications

• Number of Experts: The document does not specify a number of experts involved in establishing ground truth for the test set. It mentions "qualified clinicians" review plans and "CMS customer support personnel" executed validation test cases, but it doesn't detail their role in establishing ground truth for an independent test set.
• Qualifications of Experts: It refers to "qualified clinicians" and "Dosimetrist or Medical Physicist" as typical users of the system. However, it does not specify the qualifications (e.g., years of experience, board certification) of individuals who might have established ground truth for testing.

4. Adjudication Method for the Test Set

The document does not describe an adjudication method for a test set. The testing mentioned appears to be primarily algorithm verification against measured doses and internal validation cases, not a process involving multiple human reviewers resolving discrepancies.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No MRMC comparative effectiveness study was done. The document explicitly states: "Clinical trials were not performed as part of the development of this product." It focuses on algorithm accuracy and internal validation.

6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study

Yes, a standalone study was done, but not in a formal clinical sense. The "Algorithm testing was performed to compare calculated against measured doses to ensure dose calculation accuracy" is a standalone evaluation of the algorithm's performance against a physical standard. The "clinically oriented validation test cases" also represent an evaluation of the algorithm's output without direct human-in-the-loop performance measurement that would typically be seen in a reader study.

7. Type of Ground Truth Used

The primary type of "ground truth" implied for the algorithm testing is measured doses. This means physical measurements of radiation dose distribution, likely obtained from phantoms or experimental setups, were used as the reference standard against which the Monte Carlo algorithm's calculated doses were compared. The "clinically oriented validation test cases" likely involved scenarios with expected or known outcomes based on physics principles or established clinical practices, rather than pathology or patient outcomes data.

8. Sample Size for the Training Set

The document does not mention a discrete training set sample size. The Monte Carlo algorithm is a physics-based simulation method rather than a machine learning model that would typically have a "training set" in the conventional sense. Its development would involve calibrating physical parameters and validating the underlying physics models, rather than training on a dataset of examples.

9. How the Ground Truth for the Training Set Was Established

As the Monte Carlo algorithm is not a typical machine learning model trained on a "training set" of data, the concept of establishing ground truth for a training set in this context is not applicable or described. The "ground truth" for the development of such an algorithm would be based on fundamental physics principles, experimentally validated cross-sections, and particle interaction data, rather than annotated clinical cases.

Ask a specific question about this device

The Monaco system is used to create treatment plans for any cancer patient for whom external beam intensity modulated radiation therapy (IMRT) has been prescribed. The system will calculate and display, both on-screen and in hard-copy, either two- or threedimensional radiation dose distributions within a patient for a given treatment plan setup.

The Monaco product line is intended for use in radiation treatment planning using generally accepted methods for contouring, image manipulation, simulation, image fusion, plan optimization and QA and plan review.

Monaco uses local biological measures for optimization to create intensity modulated radiation therapy (IMRT) plans using Multileaf Collimators. With the addition of VMAT planning capability, Monaco also allows users to creatment plans in which the devices that aim and shape the beam are in motion while the beam is on.

The provided text describes the Monaco RTP System - VMAT Option, a software used for radiation treatment planning. It states that "Clinical trials were not performed as part of the development of this product." Instead, "Clinically oriented validation test cases were written and executed in-house by Customer Support personnel in a simulated clinical environment. Algorithm testing was also performed by qualified Medical Physicists using measured data from clinical facilities."

However, the document does not provide:

• A table of acceptance criteria and reported device performance.
• Sample sizes used for the test set or data provenance.
• Number or qualifications of experts used to establish ground truth.
• Adjudication methods.
• Information on multi-reader multi-case (MRMC) comparative effectiveness studies.
• Specific details on standalone algorithm performance.
• Type of ground truth used (beyond "measured data").
• Sample size for the training set.
• How ground truth for the training set was established.

Therefore,Based on the provided text, a comprehensive description of acceptance criteria and the study proving the device meets them cannot be fully constructed for many of the requested categories. The document explicitly states that "Clinical trials were not performed."

Here's what can be extracted and what is missing:

Acceptance Criteria and Study Details:

Reported Device Performance: The text states, "Monaco with the VMAT option successfully passed both testing efforts and was deemed fit for clinical use." No quantitative performance metrics (e.g., accuracy, precision, dose deviation thresholds) are provided in this summary. |
| 2. Sample size for test set and data provenance | Sample Size: Not specified. The document mentions "Clinically oriented validation test cases" and "measured data from clinical facilities" but does not give a number of cases or patients.

Data Provenance: "measured data from clinical facilities." Specific countries or retrospective/prospective nature are not mentioned. |
| 3. Number and qualifications of experts for ground truth | Number of Experts: Not specified.

Qualifications: "qualified Medical Physicists" performed algorithm testing. For clinically oriented validation, "Customer Support personnel in a simulated clinical environment" executed tests. No specific experience or board certifications are provided for either group. |
| 4. Adjudication method for the test set | Not documented. |
| 5. MRMC comparative effectiveness study | No. The document explicitly states: "Clinical trials were not performed as part of the development of this product." |
| 6. Standalone (algorithm only) performance study | Yes, to some extent. "Algorithm testing was also performed by qualified Medical Physicists using measured data from clinical facilities." This suggests an assessment of the algorithm's output against known measured data, which is a form of standalone performance evaluation. However, detailed results or metrics are not provided in this summary. |
| 7. Type of ground truth used | "Measured data from clinical facilities." This implies physical measurements of radiation dose distributions that the system's calculations were compared against. |
| 8. Sample size for the training set | Not applicable/Not specified. This is a radiation treatment planning system that calculates dose distributions, not a machine learning model that would typically have a "training set" in the conventional sense. The "algorithm" was developed, and then tested. |
| 9. How ground truth for the training set was established | Not applicable/Not specified (see point 8). |

Summary of Study (as described in the document):

The Monaco RTP System - VMAT Option underwent two main types of testing:

1. Clinically oriented validation test cases: These were executed in-house by Customer Support personnel within a simulated clinical environment. The specific nature of these "test cases" (e.g., number of plans, complexity) is not detailed.
2. Algorithm testing: This was performed by qualified Medical Physicists. The testing involved comparing the algorithm's outputs against "measured data from clinical facilities." The report states that "Test reports are included in section 20 of this submission," but these details are not provided in the summary.

Both testing efforts were successfully passed, leading the device to be deemed "fit for clinical use." Verification tests, with associated pass/fail requirements, were also conducted to ensure the system functioned as designed, and these were also successfully passed.

Crucially, no clinical trials involving human subjects were performed or deemed necessary for this premarket notification, as the manufacturer argued that "testing can be performed such that no human subjects are exposed to risk" and that clinical testing was "not advantageous in demonstrating substantial equivalence or safety and effectiveness." The device's safety and effectiveness were established through comparison to predicate devices and the non-clinical testing described above.

Ask a specific question about this device

Atlas-Based Autosegmentation is a standalone software application that produces estimates of anatomy boundary contours needed for the creation of a radiotherapy treatment plan.

Contouring of radiation therapy targets and surrounding anatomical structures (also known as image segmentation) is a critical part of radiation treatment planning that can be extremely time consuming. Atlas-Based Autosegmentation (ABAS) is a software application that automates the contouring process using atlas-based autosegmentation. This method uses an already-segmented image set (atlas) to segment a set of new, user-input images using deformable registration algorithms. The contours ABAS generates are not usable for treatment as-is; they must be exported to a treatment planning system for editing. However, Atlas-based Autosegmentation provides a good starting point from which minimal editing is required, enabling the clinician to create a high quality treatment plan more efficiently.

Here's a summary of the acceptance criteria and study information for the Atlas-Based Autosegmentation (ABAS) device, based on the provided text:

Important Note: The provided document is a 510(k) summary, which often focuses on demonstrating substantial equivalence rather than detailed clinical performance studies. As such, information regarding specific quantitative acceptance criteria or detailed clinical trial results is limited. The document explicitly states: "Clinical trials were not performed as part of the development of this product. Clinical testing is not advantageous in demonstrating substantial equivalence or safety and effectiveness of the device since testing can be performed such that no human subjects are exposed to risk. Clinically oriented validation test cases were written and executed in-house by CMS customer support personnel. ABAS was deemed fit for clinical use."

Therefore, many of the requested sections below will reflect the absence of such clinical studies.

Acceptance Criteria and Device Performance

Study Details

1. Sample size used for the test set and the data provenance:

   • Sample Size: Not specified in the provided document. The text mentions "clinically oriented validation test cases" but does not quantify the number of cases.
   • Data Provenance: The document does not specify the country of origin of the data. The "clinically oriented validation test cases" were executed "in-house by CMS customer support personnel," implying they were likely internal or simulated datasets, not from external clinical sites. The data was retrospective as clinical trials were not performed.

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • This information is not provided. The phrase "clinically oriented validation test cases" suggests some form of clinical relevance, but the method and personnel for establishing ground truth are not described.

3. Adjudication method for the test set:

   • Not specified. Given the testing was "in-house by CMS customer support personnel" and clinical trials were not performed, a formal adjudication process akin to clinical studies is unlikely to have occurred.

4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No, an MRMC comparative effectiveness study was not done. The document explicitly states: "Clinical trials were not performed as part of the development of this product." The device's primary function is described as providing an "initial contouring function" that requires further editing by clinicians in a treatment planning system. Therefore, a study on human reader improvement with AI assistance (i.e., human-in-the-loop performance) was not presented.

5. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:

   • Yes, in a sense, standalone performance was assessed through non-clinical verification and internal "clinically oriented validation test cases." The device is described as a "standalone software application that produces estimates of anatomy boundary contours." The "Summary of Non-Clinical Testing" indicates "Verification tests were written and executed to ensure that the system is working as designed," and the "clinically oriented validation test cases" were used to deem ABAS "fit for clinical use." However, quantitative metrics of accuracy, precision, etc., for this standalone performance against a defined ground truth, are not provided in this summary. It's also important to note the disclaimer that "The contours ABAS generates are not usable for treatment as-is; they must be exported to a treatment planning system for editing."

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

   • Not explicitly stated. For the "clinically oriented validation test cases," the nature of the ground truth is not detailed beyond "already-segmented image set (atlas)" being used in the process. It's reasonable to infer that the "atlas" itself serves as a form of expert-derived ground truth for the autosegmentation process.

7. The sample size for the training set:

   • Not specified. The device uses an "atlas-based autosegmentation" method, which implies a training set or an "atlas" library. However, the size or composition of this atlas is not mentioned. It states that users can "expand its library of atlases," suggesting a flexible and potentially user-managed training-like data source.

8. How the ground truth for the training set was established:

   • The document implies that the ground truth for the model's operation is derived from "already-segmented image set (atlas)." How these initial atlas segmentations were created (e.g., by experts, manually) is not detailed in this summary.

Ask a specific question about this device

The Monaco system is used to create treatment plans for any cancer patient for whom external beam intensity modulated radiation therapy (IMRT) has been prescribed. The system will calculate and display, both on-screen and in hard-copy, either two- or three-dimensional radiation dose distributions within a patient for a given treatment plan set-up.

The Monaco product line is intended for use in radiation treatment planning using generally accepted methods for contouring, image manipulation, simulation, image fusion, plan optimization and QA and plan review.

Monaco uses local biological measures for optimization to create intensity modulated radiation therapy (IMRT) plans using Multileaf Collimators.

The provided text describes the Monaco RTP System, a radiation treatment planning system, and its non-clinical testing for substantial equivalence. It does not contain information about clinical trials or the specific acceptance criteria and performance metrics typically found in studies involving AI performance for diagnosis or image interpretation.

Based on the provided text, here is an analysis of the acceptance criteria and study information:

1. Table of Acceptance Criteria and Reported Device Performance:

The document does not provide a table with specific quantitative acceptance criteria or detailed performance metrics. Instead, it states:

2. Sample Sizes Used for Test Set and Data Provenance:

The document explicitly states that clinical trials were not performed. Therefore, there isn't a "test set" in the traditional sense involving patient data for an effectiveness study.

• Test Set Sample Size: Not applicable, as no clinical test set was used for effectiveness.
• Data Provenance: Not applicable for effectiveness testing. However, for "On-site validation testing," it mentions "a small group of customers, using actual patient data." The origin of this patient data (country, retrospective/prospective) is not specified. This validation was not a formal clinical trial for effectiveness.

3. Number of Experts and Qualifications for Ground Truth:

• Number of Experts: Not specified.
• Qualifications of Experts: For the "on-site validation testing," it mentions "qualified clinicians" who review and approve plans and "a small group of customers." Specific qualifications (e.g., radiologist with X years of experience) are not provided. The document highlights that "qualified clinicians" review plans and may subject them to quality assurance before treatment, implying their role in validating the plans generated by the system.

4. Adjudication Method for Test Set:

Not applicable, as no formal clinical test set or adjudication process for a diagnostic outcome was performed. The document mentions "qualified clinicians" review and approve plans, but this isn't an adjudication method for a test set in the context of an effectiveness study.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:

No. The document explicitly states: "Clinical trials were not performed as part of the development of this feature." Therefore, an MRMC comparative effectiveness study was not conducted.

6. Standalone (Algorithm Only) Performance Study:

Yes, in a non-clinical context. The document refers to "Verification tests, including algorithm test cases," and "algorithm accuracy requirements." It states that "Monaco successfully passed both verification and algorithm testing," suggesting a standalone evaluation of the algorithms.

7. Type of Ground Truth Used:

For the non-clinical "verification tests" and "algorithm test cases," the ground truth would likely be established through:

• Pre-defined Pass/Fail Criteria: These would be based on engineering specifications and expected output from the algorithms.
• Comparison to Known Analytical Solutions/Benchmarks: For specific algorithm calculations (e.g., dose distribution), "ground truth" would be established by comparing the software's output to mathematically derived solutions or outputs from trusted reference calculators.
• Outputs from Predicate Devices: The text mentions "the same algorithm accuracy requirements as those used to evaluate the XiO RTP System and the Focal Workstation," implying that the performance of predicate devices might serve as a benchmark or a form of 'ground truth' for comparison during testing.

For the "on-site validation testing using actual patient data," the "ground truth" for clinical fitness would likely involve:

• Clinical Acceptability: Expert opinion from the "small group of customers" (clinicians) on whether the generated plans were clinically appropriate and safe.
• Quality Assurance (QA) Practices: Comparison against established QA standards for radiation oncology treatment plans.

8. Sample Size for the Training Set:

Not specified. The document does not provide details on the training methodology or the datasets used to develop the algorithms within the Monaco RTP System.

9. How Ground Truth for Training Set Was Established:

Not specified. Since details about the training set are not provided, how its ground truth was established is also not mentioned.

Ask a specific question about this device

The XiO RTP System is used to create treatment plans for any cancer patient for whom external beam radiation therapy or brachytherapy has been prescribed. The system will calculate and display, both on-screen and in hard-copy, either two- or three-dimensional radiation dose distributions within a patient for a given treatment plan set-up.

Optionally, the user may elect to generate plans using Dynamic Conformal Arc Therapy capability. Dynamic Conformal Arc Therapy is a treatment modality in which the gantry rotates in an arc (or multiple arcs) over user-specified angles while the leaves of a multileaf collimator (MLC) continually reshape the beam to conform to the targer.

The XiO Radiation Treatment Planning system accepts a) patient diagnostic imaging data from CT and MR scans, or from films, and b) "source" dosimetry data, typically from a linear accelerator. The system then permits the user to display and define (contour) a) the target volume to be treated and b) critical structures which must not receive above a certain level of radiation, on these diagnostic images.

Based on the prescribed dose, the user, typically a Dosimetrist or Medical Physicist can then create multiple treatment scenarios involving the type, number, position(s) and energy of radiation beams and the use of treatment aids between the source of radiation and the patient (wedges, blocks, ports, etc.). The XiO system then produces a display of radiation dose distribution within the patient, indicating not only doses to the target volume but to surrounding tissue and structures. The "best" plan satisfying the prescription is then selected, one that maximizes dose to the target volume while minimizing dose to surrounding healthy volume. The parameters of the plan are output in hard-copy format for later reference and for placement in the patient file.

This Premarket Notification addresses the addition of support for Dynamic Conformal Therapy. Dynamic Conformal is a treatment modality in which radiation beams are continuously shaped to conform to a target while the gantry rotates and the beam is on. In XiO, the user chooses the target, defines structures to avoid, optional margin(s), and treatment angles. XiO then plans a treatment over a specified arc and with specified beam increments, with the leaves of the multi-leaf collimator (MLC) continually reshaping the beam to conform to the target. The target receives a homogenous dose while the structures designated as "avoidance structures" are avoided absolutely.

Dose calculation is performed using existing, validated algorithms within XiO. Determination of dose at the specified angles is calculated in the same way as conventional and asymmetric arc beams; the calculated dynamic beam dose distribution is determined as the sum of multiple fixed beam dose distributions across the specified arc.

This submission describes the XiO Radiation Treatment Planning System with the added functionality of Dynamic Conformal Therapy. The information provided heavily emphasizes the regulatory process and comparisons to predicate devices rather than detailed performance studies with acceptance criteria.

Here's an analysis based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

The provided text does not explicitly state specific acceptance criteria in terms of numerical thresholds for dose calculation accuracy or clinical outcomes. Instead, it focuses on the device's ability to calculate dose and conform to targets.

2. Sample Size Used for the Test Set and Data Provenance

• Test Set Sample Size: Not explicitly stated. The document mentions "Algorithm test cases were written and executed." The number of these test cases is not specified.
• Data Provenance: The testing was "executed in-house by CMS customer support personnel." This suggests simulated or internally generated data rather than real patient data. The country of origin is implicitly the US, where Computerized Medical Systems, Inc. is located. It was a retrospective evaluation against defined test cases.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications

• Number of Experts: Not specified. The "Algorithm test cases were written and executed" and "Clinically oriented validation test cases were written and executed" by "CMS customer support personnel." It's unclear if these personnel are qualified medical experts (e.g., medical physicists, dosimetrists) or if external experts were involved in establishing the ground truth for these test cases.
• Qualifications of Experts: Not explicitly stated. While the system's users are described as "typically a Dosimetrist or Medical Physicist," it doesn't confirm if these roles or similarly qualified individuals generated the test cases or the ground truth for them.

4. Adjudication Method for the Test Set

• Adjudication Method: Not specified. Since the testing involved "algorithm test cases" and "clinically oriented validation test cases," it's likely a comparison against predetermined correct outputs for those cases rather than an adjudication process involving multiple human observers reviewing complex real-world interpretations.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done

• No, an MRMC comparative effectiveness study was not done. The text explicitly states: "Actual testing in a clinic was not performed as part of the development of this feature." It also adds that such testing "is not required to demonstrate substantial equivalence or safety and effectiveness of the device." Therefore, there is no effect size reported for human readers improving with AI vs. without AI assistance.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done

• Yes, a standalone study was done, as far as the algorithm's performance is concerned. The text highlights "Algorithm test cases were written and executed to ensure that the system is calculating dose correctly for Dynamic Conformal beams." This "algorithm test" would represent a standalone evaluation of the core calculation engine's accuracy against predefined inputs and expected outputs, without direct human intervention in each calculation step.

7. The Type of Ground Truth Used

• Type of Ground Truth: The ground truth for the "Algorithm test cases" would likely be pre-calculated, theoretical, or established results from known physics principles or established algorithms for radiation dose calculation under specific, controlled parameters. For "Clinically oriented validation test cases," the ground truth would similarly be pre-defined correct treatment plans or dose distributions established by medical physicists/dosimetrists for those specific clinical scenarios. It is not pathology, expert consensus on patient images, or outcomes data.

8. The Sample Size for the Training Set

• Training Set Sample Size: Not applicable/Not provided. Radiation Treatment Planning (RTP) systems like XiO, particularly from this era (2003), typically rely on deterministic physical algorithms for dose calculation rather than machine learning models that require training data sets in the conventional sense. The "algorithms" mentioned are likely physics-based models rather than AI models.

9. How the Ground Truth for the Training Set Was Established

• Ground Truth for Training Set: Not applicable. As mentioned above, this system likely operates on deterministic algorithms rather than machine learning, so there isn't a "training set" or corresponding ground truth in the AI context. The algorithms themselves would have been developed and validated against established physics principles and possibly empirical measurements (though those details are not in this summary).

Ask a specific question about this device

I-Beam is a self-contained mobile patient positioning system that uses real time ultrasound images of patient target organs or tumors while the patient is positioned on the couch of the linear accelerator to confirm the location of these patient target organs or tumors prior to delivery of external beam radiation therapy.

The I-Beam Patient Positioning System provides a method for a hospital/clinic to accurately position a patient prior to delivery of external beam radiation therapy each day such that the patient tumor volume on the day of therapy coincides with the tumor volume from the treatment plan. Patient positioning may be necessary because of day-to-day movement of the soft tissue target within the patient. Closer alignment assures proper tumor coverage and a minimum of dose to healthy tissue and structures surrounding the target.

I-Beam achieves this alignment by having a camera provide a signal to I-Beam. The camera is attached to an ultrasound transducer body by use of a custom "clamshell" in an orientation that points the camera in a direction 180 degrees from the plane of the transducer output. As the transducer ultrasound system (not part of I-Beam) images the patient target, the camera is looking up at a targeting grid located on the shadow tray of the linear accelerator. Using pattern recognition techniques, the I-Beam system is able to determine the position of the isocenter of the target volume relative to the isocenter of the linear accelerator. The user then superimposes the ultrasound tumor volume data obtained from this scan on that from the original treatment plan data input to I-Beam earlier. This treatment plan is based on a CT study set, and the user aligns the two tumor volumes by "moving" one relative to the other. I-Beam senses this relative movement of the ultrasound tumor volume to that from the treatment plan and converts that into a three axis "translation" figure which gives the operator the amount and direction the patient must be moved in each axis to achieve alignment with the treatment plan and thus the linear accelerator. Correct re-positioning of the patient is verified by performing a second ultrasound scan of the patient and overlaying that with the original CT-based treatment plan tumor volume information. The translation figures will advise the user of any remaining misalignment should there have been a misinterpretation of the translation date and/or patient positioning.

The provided documentation for the I-Beam Patient Positioning System (K032100) does not contain specific acceptance criteria or a study detailing device performance against such criteria. The submission states that "Actual testing in a clinic was not performed as part of the development" and "Clinical testing is not required to demonstrate substantial equivalence or safety and effectiveness of the device."

Instead, the submission relies on "Non-Clinical Testing" which involved:

• Verification and validation test cases: These were "written and executed to assure the system is correctly measuring patient tumor volume shift as well as outputting correct patient 'translation' to achieve alignment of planned versus actual patient tumor volume."
• Verification of correct operation: This included the "optical alignment target and the camera and the ability of I-Beam to convert this information into clinically correct patient repositioning information."

Given the information provided, it is impossible to create the table or answer most of the questions as the details simply aren't present in the document. The document describes the device's function and how it was designed to achieve its purpose but does not provide quantitative performance metrics or a formal study report.

Here's a breakdown of what can be said based on the provided text, and what cannot:

1. Table of Acceptance Criteria and Reported Device Performance:

• Acceptance Criteria: Not explicitly stated in quantitative terms. The text mentions ensuring "correctly measuring patient tumor volume shift" and "outputting correct patient 'translation'" and "clinically correct patient repositioning information." These are qualitative goals, not quantifiable acceptance criteria.
• Reported Device Performance: No quantitative performance metrics are provided. The document confirms that non-clinical testing was performed to "assure the system is correctly measuring" and "outputting correct" information, but no specific values or ranges are given.

2. Sample size used for the test set and the data provenance:

• Sample Size: Not specified. The document only mentions "both verification and validation test cases were written and executed."
• Data Provenance: Not specified.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

• Not specified. The ground truth method itself is not explicitly defined in terms of expert involvement for the test set.

4. Adjudication method for the test set:

• Not specified.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

• No, an MRMC study was not done. The document explicitly states, "Actual testing in a clinic was not performed." This device is a patient positioning system, not an AI diagnostic tool involving human readers in the traditional sense of an MRMC study.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

• The non-clinical testing appears to have evaluated the algorithm's performance in terms of its ability to measure shift and output translation. However, it's not a "standalone" performance evaluation in the context of typical AI device submissions without any human involvement, as the system provides "translation figures" for an "operator" to move the patient. The verification process described involves the system converting information into "clinically correct patient repositioning information," implying an assessment of its output for clinical use. No quantitative performance metrics are provided for this.

7. The type of ground truth used:

• The ground truth for the non-clinical testing was implied to be established against the "planned versus actual patient tumor volume" and the ability to convert information into "clinically correct patient repositioning information." This suggests a comparison against known or computed correct values within the simulated or test environment, rather than expert consensus, pathology, or direct outcomes data from real patients.

8. The sample size for the training set:

• Not applicable as this is a patient positioning system, and the description does not indicate the use of a "training set" in the context of machine learning model development.

9. How the ground truth for the training set was established:

• Not applicable.

In summary: The provided 510(k) summary for the I-Beam Patient Positioning System focuses on demonstrating substantial equivalence to predicate devices and describes the non-clinical verification and validation activities conducted. However, it does not include the detailed quantitative acceptance criteria or performance study results that would typically be found for a device requiring such data, especially in the context of AI/ML-driven devices. The submission clearly states clinical testing was not performed or deemed necessary for this particular device and its 2003 submission context.

Ask a specific question about this device

The intended use of FOCUS RTP System is to provide radiation treatment planning capability, for both external beam and brachytherapy sources, to satisfy the prescription of a Radiation Oncologist. The resultant treatment plan is to be evaluated, modified as necessary, approved and delivered by qualified medical personnel. Operation of the system is identical to FOCUS systems cleared under previous Premarket Notifications with the exception the user can now select a third type of external beam particle for therapy (protons) in addition to the earlier two particles (electrons and photons)..

The FOCUS Radiation Treatment Planning System accepts a) patient diagnostic imaging data from CT and MR scans, or from films, and b) "source" dosimetry data, typically from a linear accelerator. The system then permits the user to display and define (contour) a) the target volume to be treated and b) critical structures which must not receive above a certain level of radiation, on these diagnostic images. Based on the prescribed dose, the user, typically a Dosimetrist or Medical Physicist, can then create multiple treatment scenarios involving the type, number, position(s) and energy of radiation beams and the use of treatment aids between the source of radiation and the patient (wedges, blocks, ports, etc.). The FOCUS system then produces a display of radiation dose distribution within the patient, indicating not only doses to the target volume but to surrounding tissue and structures. The "best" plan satisfying the prescription is then selected, one which maximizes dose to the target volume while minimizing dose to surrounding healthy volume. The parameters of the plan are output in hard-copy format for later reference and for placement in the patient file. Previously, for situations where external beam therapy was to be used, either Electron and/or Photon radiation beams could be selected.. These were delivered by a linear accelerator whose output characteristics are input to the treatment planning system prior to beginning planning. This Premarket Notification addresses the addition of a third type of radiation beam -Proton. The algorithm for calculating dose was provided by the Massachusetts General Hospital (MGH), based on their years of experience at the Harvard Cyclotron Lab. In addition to providing the algorithm, MGH also worked with CMS in its implementation. Software developers at MGH were trained on the CMS software development process to permit them to create code directly for use in FOCUS. As the final step, MGH provided the verification testing to assure the algorithm had been implemented correctly, measuring calculated dose against measures. A FOCUS RTP System with proton planning capability is now in clinical use at the Northeast Proton Therapy Center.

The provided text describes the K020027 FOCUS Radiation Treatment Planning System with Proton Planning Capability. It details the device's function, its new proton planning feature, and the non-clinical testing performed.

Here's an analysis of the acceptance criteria and study that proves the device meets them, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size: Not explicitly stated in terms of a specific number of cases or plans. The text refers to "Algorithm test cases" without detailing their quantity.
• Data Provenance: The testing was a "non-clinical" study. The algorithm was developed based on "years of experience at the Harvard Cyclotron Lab" at Massachusetts General Hospital (MGH). The verification testing involved measuring "calculated dose against measures," implying an experimental setup or phantom measurements rather than patient data. The study, "Validation of the CMS Proton Treatment Planning System for Treatments In Large Field Beam Line at the Harvard Cyclotron Laboratory (HCL)," suggests a laboratory-based, prospective evaluation of the algorithm's performance against physical measurements or highly controlled simulated scenarios.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts

• Number of Experts: Not explicitly stated. However, the algorithm itself was provided by the Massachusetts General Hospital (MGH), implying the expertise of their staff at the Harvard Cyclotron Lab (HCL). MGH also performed the verification testing.
• Qualifications of Experts: The text states the algorithm was based on "years of experience at the Harvard Cyclotron Lab." This implies the involvement of highly experienced medical physicists and other specialists familiar with proton therapy dose calculation. "Skip Rosenthal of MGH" authored the validation document, indicating his expertise in this area.

4. Adjudication Method for the Test Set

• Adjudication Method: Not applicable or not described. The validation appears to be a direct comparison of calculated dose values to measured dose values, rather than an expert consensus process.

5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance

• MRMC Study: No, an MRMC comparative effectiveness study was not done. The device is a radiation treatment planning system algorithm, not a diagnostic AI intended to assist human readers in interpreting images or making a diagnosis. Its function is to calculate dose distributions based on user input for treatment planning. The study focused on the accuracy of the dose calculation algorithm itself.

6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) Was Done

• Standalone Performance Study: Yes, in essence, the "non-clinical testing" described is a standalone performance study. The "Algorithm test cases were written and executed to assure the system is calculating dose correctly for proton treatment plans." This focused solely on the algorithm's output (calculated dose) compared to a reference (measured dose), without evaluating user interaction or clinical outcomes in humans. The current device is meant to be used by "Dosimetrist or Medical Physicist" where they would evaluate, modify and approve the plans. However, the specific testing described in the summary is of the algorithm's accuracy in isolation.

7. The Type of Ground Truth Used

• Type of Ground Truth: The ground truth used was measured dose values. The text explicitly states "measuring calculated dose against measures" in the verification testing. This indicates that the algorithm's computed dose distributions were compared against physical measurements obtained from a phantom or experimental setup at the Harvard Cyclotron Laboratory.

8. The Sample Size for the Training Set

• Sample Size for Training Set: Not explicitly stated. The algorithm itself was provided by MGH, based on their "years of experience at the Harvard Cyclotron Lab." This implies that it was developed and implicitly "trained" or refined over a long period from extensive data and knowledge accumulated at HCL, but there's no defined "training set" in the context of supervised machine learning with a specific sample size.

9. How the Ground Truth for the Training Set Was Established

• Ground Truth for Training Set: The ground truth for the underlying algorithm's development (or its "training" in a broader sense) was established through extensive experience and data collection at the Harvard Cyclotron Lab (HCL) over many years. This would likely involve:
  • Clinical experience and outcomes: Understanding how protons interact with tissue and the effects of different dose distributions.
  • Physics measurements: Extensive experimental data on proton beam characteristics, interactions, and dose deposition in phantoms and biological samples.
  • Computational modeling and simulations: Development and validation of physics models used in dose calculation.
    The algorithm provided by MGH represents the culmination of this accumulated knowledge and empirical validation.

Ask a specific question about this device

The FOCAL workstation software is a computer software package intended to be used as an accessory to a radiation treatment planning system.

FOCAL Sim is intended to permit CT simulation to be performed on the FOCAL workstation. The CT scan is read into the radiation treatment planning system and then sent to the FOCAL workstation. On FOCAL, the user is able to identify patient isocenters, place treatment beams and identify beam modifiers (blocks, wedges, etc.). This information is then passed back to the radiation treatment planning system for storage and documentation of the resultant treatment plan and calculation of patient dose based on this information. The resultant plan is to be evaluated, modified as necessary, approved and delivered by qualified medical personnel.

The FOCAL Workstation was initially cleared for marketing under K981535. The initial release of the product had, as its intended use, the remote contouring of patient outlines, structures and tumors as part of radiation therapy planning. The FOCAL Workstation was designed to work with our FOCUS Radiation Treatment Planning (RTP) System. The task of contouring, typically performed by the Radiation Oncologist, is the most time consuming task in radiation therapy planning and requires a minimum of the most recently cleared FOCUS RTP Computer Capacity. It was our goal to free up the higher powered UNIX-based RTP Workstation for performing the calculation-intensive activities of treatment planning by moving the contouring to a remote device. This remote device was a Personal Computer loaded with the FOCAL contouring software running on that PC. While contouring was complete, the information was returned to the RTP System to continue the treatment planning process.

The first release of FOCAL contained only manual contouring capability and was given the trade name of "FOCAL Ease". The second release enhanced the users ability to view CT and MR images as well as providing an autosegmentation capability. This added functionality was given the trade name "FOCAL Fusion".

A later release of FOCAL software provided the capability to view the results of treatment planning performed earlier on the FOCUS RTP System. This included the ability to view isodose contours as well as Dose Volume Histograms (DVH's) and Digitally Reconstructed Radiations (DRR's). This was given the name "FOCAL Vue". This provided the Radiation Oncologist with a remote capability to view and compare alternate treatment plans and select the one which best satisfied her/his prescription.

The subject of this Premarket Notification is the addition of the ability to perform CTV simulation, a feature we call "FOCAL Sim". This addition moves the FOCAL Workstation from merely contouring of patient targets or viewing of treatment planning results into a more active role in the treatment planning process.

The provided document is a 510(k) Summary of Safety and Effectiveness for the FOCAL CT Simulation (FOCAL SIM) device. It describes the device, its intended use, and a comparison to predicate devices. However, it explicitly states that clinical testing was not performed as part of the development of this feature, and that non-clinical testing results can be found in Section 9 of the submittal (which is not provided in the extracted text).

Therefore, based solely on the provided text, I cannot answer most of your questions as the requested information is absent.

Here's what can be extracted based on your questions, with the understanding that key information regarding acceptance criteria and performance data from a study is missing:

1. A table of acceptance criteria and the reported device performance

• Acceptance Criteria: Not explicitly stated in the provided text.
• Reported Device Performance: Not explicitly stated or measured in the provided text's "Summary of Clinical Testing" or "Summary of Non-Clinical Testing."

2. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

• Sample Size for Test Set: Not applicable. Clinical testing was explicitly not performed.
• Data Provenance: Not applicable. No clinical data was used for testing purposes.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

• Not applicable. No clinical testing was performed, thus no ground truth was established by experts for a test set in this context.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

• Not applicable. No clinical testing was performed.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• MRMC Study: No. The document states, "Clinical testing was not performed as part of the development of this feature." The device is a "CT Simulation" software module, not explicitly described as an AI-assisted diagnostic tool as you might find in an MRMC study context.
• Effect Size: Not applicable.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

• The document implies that the "FOCAL Sim" module allows "CT simulation to be performed on the FOCAL workstation" and that the user "is able to identify patient isocenters, place treatment beams and identify beam modifiers," suggesting a human-in-the-loop system. However, no specific "standalone" performance study is mentioned in the provided text.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)

• Not applicable. No clinical testing was performed. The device's "substantial equivalence" was based on comparison to predicate devices and non-clinical testing (details of which are not in the provided text).

8. The sample size for the training set

• Not applicable. The document does not describe the use of machine learning or AI models that would typically require a training set. The "FOCAL Sim" is described as a software module that "merely takes some of the tasks previously performed on the RTP System and allows them to be performed on the FOCAL Workstation."

9. How the ground truth for the training set was established

• Not applicable. No training set is mentioned.

In summary, based only on the provided text, the 510(k) summary explicitly states that clinical testing was not performed for the FOCAL CT Simulation (FOCAL SIM) feature. The regulatory approval was based on demonstrating "substantial equivalence" to predicate devices and non-clinical testing (details of which are not included in this extract). Therefore, the detailed information about acceptance criteria, performance data, sample sizes, ground truth establishment, and specific study types (like MRMC or standalone) is not present in the provided document.

Ask a specific question about this device