(147 days)
imaging human organs through use of injected radiopharmaceuticals, which are separately regulated as diagnostic drugs. The device will have a wide variety of applications in planar single organ and whole body imaging.
The Protec-1024 is an imaging device utilized to determine the distribution of gammaemitting radioactive tracers within the human body. The device consists of a radiation detection head, containing a multiwire proportional detector, coupled to a mobile part, containing electronics utilized to process images from the detector, and a computer utilized to control image acquisition and to provide image processing and display.
The Protec-1024 Mobile Gamma Camera System (also referred to as Multiwire Gamma Camera) is intended to determine the distribution of gamma-emitting radioactive tracers within the human body. The device is non-invasive and is positioned near the surface of the chest to acquire images based on the detection of radiation emitted from the body.
Here's an analysis of the provided information regarding acceptance criteria and the supporting studies:
1. Table of Acceptance Criteria and Reported Device Performance
The provided document does not explicitly present a formal table of acceptance criteria with corresponding performance metrics. Instead, it relies on demonstrating substantial equivalence to existing, approved devices and on published clinical and bench performance data.
However, based on the text, we can infer some key performance aspects and their reported outcomes:
| Performance Aspect | Acceptance Criteria (Inferred) | Reported Device Performance |
|---|---|---|
| Equivalence to Prior Devices | Substantially equivalent to existing scintillation gamma camera systems (e.g., SIM-400) and the Xenos Mobile Cam. | The Protec-1024 is stated to be "substantially equivalent" to scintillation gamma cameras and a previous multiwire camera (Xenos Mobile Cam). The detector of the Protec-1024 is "essentially identical in characteristics" to the Xenos device. Assessment of provided performance data "establishes the equivalence of this device to other similar systems." |
| Energy Range | Comparable to other gamma cameras, particularly for blood pool imaging applications. | 30 keV to 90 keV (compared to 30 keV to 300 keV for conventional scintillation gamma cameras). This difference is noted, but the device is still considered useful, particularly in blood pool imaging where the energy range is sufficient. |
| Event Rates | Higher event rates are desirable for certain applications. | Operates at "much higher event rates" than single crystal scintillation cameras. |
| Human Imaging Usefulness | Demonstrated utility in human imaging applications, especially for blood pool imaging. | "Extensively evaluated in human imaging applications and has been shown to be useful, particularly in blood pool imaging." |
| General Indication for Use | Effective for imaging human organs using injected radiopharmaceuticals. | Stated to be for "imaging human organs through use of injected radiopharmaceuticals." Intended for "planar single organ and whole body imaging." |
| Clinical Performance (Cardiology) | Comparable to approved devices in assessing cardiac function (e.g., left/right ventricular function, transient coronary occlusion). | A study comparing the MWGC system to the approved Baird multicrystal device was reported. Studies reported results and measurements of left and right ventricular function following balloon angioplasty intervention. Clinical validation data from 53 patient studies by Dr. Ami Iskandrian covered "imaging performance and functionality." |
| Software Performance (Image Acquisition) | Proper and accurate control of image data acquisition. | A "Software validation data" report critically evaluated the "static and dynamic image acquisition system," verifying "proper, accurate performance of the software in controlling acquisition of image data." |
| Inter/Intraobserver Variability | Low and acceptable variability in key measurements (e.g., ejection fraction). | "A comprehensive study of the interobserver and intraobserver variability in the ejection fraction measurements was reported in the aforementioned 1992 Journal of the American College of Cardiology paper evaluating left ventricular function." (Specific metrics not provided in the text). |
2. Sample Size Used for the Test Set and Data Provenance
- Clinical Validation (Philadelphia Heart Institute):
- Sample Size: 53 patient studies.
- Data Provenance: Retrospective, conducted at The Philadelphia Heart Institute. The location implies data from the United States.
- Other Clinical Studies (Verani et al.):
- The document mentions multiple publications without specific test set sizes directly in the text excerpt, but refers to human imaging applications. One study "compared the performance of the MWGC system to the approved Baird multicrystal device." Another focused on "left and right ventricular function following balloon angioplasty intervention." These are prospective clinical studies.
- Bench Performance Data (Lacy et al.): These are laboratory-based studies, not involving human subjects as a "test set" in the clinical sense.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
- Clinical Validation (Philadelphia Heart Institute):
- Number of Experts: At least one, Dr. Ami Iskandrian, 1995 president of the American Society of Nuclear Cardiology. It's implied that his evaluation contributed to the ground truth.
- Qualifications: "Respected nuclear cardiologists," specifically identifying Dr. Ami Iskandrian, M.D., 1995 president of the American Society of Nuclear Cardiology. This indicates a highly qualified expert in nuclear medicine and cardiology.
- Other Clinical Publications:
- The studies mentioned (Lacy JL, Leblanc AD, et al.; Lacy JL, Verani MS, et al.; Verani MS, Lacy JL, et al.; Verani MS, Guidry GW, et al.) were published by "reputed cardiology investigators" and "some of today's most respected nuclear cardiologists." While specific numbers aren't given, it implies involvement of multiple highly qualified medical professionals in establishing the clinical findings and ground truth for those studies.
4. Adjudication Method for the Test Set
The document does not explicitly describe a formal "adjudication method" like 2+1 or 3+1 for establishing ground truth from multiple readers.
- For the Philadelphia Heart Institute clinical validation, it appears Dr. Iskandrian's evaluation was central. It's not stated whether other experts independently reviewed cases to reach a consensus for ground truth.
- For the published clinical studies, the standard peer-review process for scientific publications implies a level of expert review and consensus on methodology and findings, but not a specific adjudication process for individual cases in establishing ground truth.
- The "interobserver and intraobserver variability" study suggests that multiple readers were involved at some point to assess consistency, but this is a measurement of variability, not necessarily a method for establishing a single adjudicated ground truth for the entire test set.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
- No, an MRMC comparative effectiveness study was not explicitly done in the context of AI assistance.
- The device is a gamma camera, not an AI algorithm designed to assist human readers.
- The document primarily focuses on the standalone performance of the camera itself and its equivalence to established technologies.
- There is a comparison of the MWGC system to the approved Baird multicrystal device, which is a comparison of devices, not human-AI interaction.
- The "interobserver and intraobserver variability" study assesses human variability but not in the context of AI assistance.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done
- Yes, in essence, standalone performance was the primary focus. The Protec-1024 Mobile Gamma Camera System is a standalone imaging device. The various studies (bench, clinical, software validation) describe the performance of the device and its components (detector, software) operating independently to produce images and measurements.
- The software validation specifically describes "algorithm only" performance in validating “the proper, accurate performance of the software in controlling acquisition of image data."
7. The Type of Ground Truth Used
- Clinical Ground Truth:
- Expert Consensus/Clinical Diagnoses: The clinical studies (e.g., assessing left and right ventricular function, transient coronary occlusion) implicitly rely on established clinical diagnostic criteria, physiological measurements, and expert interpretation by cardiologists. The comparison to the "approved Baird multicrystal device" suggests the Baird device's output served as a reference (a form of established clinical truth).
- Physiological Measurements/Outcomes Data: The studies on "left and right ventricular function following balloon angioplasty intervention" relate device measurements to the physiological state and post-procedure outcomes, which serve as ground truth for evaluating functional changes.
- Bench Ground Truth:
- Physical Measurements/Standards: The bench performance data (collimator and detector sensitivity measurements, articles on camera design) used controlled physical parameters as ground truth.
- Software Ground Truth:
- Expected Software Behavior/Specifications: The software validation used predefined expected behaviors and specifications for image acquisition as the ground truth against which the software's performance was measured.
8. The Sample Size for the Training Set
The provided text does not contain information about a "training set" in the context of machine learning or AI. The Protec-1024 is described as a hardware imaging device with associated software, not a machine learning model that requires a specific training dataset. The development and validation relied on traditional engineering, physics, and clinical evaluation methods.
9. How the Ground Truth for the Training Set was Established
As there is no mention of a "training set" for a machine learning model, this question is not applicable to the information provided. The "ground truth" for the device's development and validation was established through scientific principles, engineering tests, and expert clinical evaluation as described above.
§ 892.1200 Emission computed tomography system.
(a)
Identification. An emission computed tomography system is a device intended to detect the location and distribution of gamma ray- and positron-emitting radionuclides in the body and produce cross-sectional images through computer reconstruction of the data. This generic type of device may include signal analysis and display equipment, patient and equipment supports, radionuclide anatomical markers, component parts, and accessories.(b)
Classification. Class II.