Search Results

The VisAR System is intended as an aid for precisely locating anatomical structures in either open or percutaneous spine procedures. VisAR is indicated for any medical condition in which the use of stereotaxic surgery may be appropriate, and where reference to at least one rigid anatomical structure, such as the spine or iliac crests, can be identified relative to CT imagery of the anatomy. This can include guidance for procedures, such as Posterior Pedicle Screw Placement in the thoracic and sacro-lumbar region.

VisAR displays a virtual screen for stereoscopic 3D images acquired from CT sources. It is intended to enable users to segment previously acquired 3D datasets, overlay, and register these 3D segmented datasets with the anatomy of the patient in order to support intraoperative analysis and guidance.

The virtual screen is indicated for displaying the virtual instrument location to the virtual anatomy to assist in visualization and trajectory planning for both open and percutaneous surgeries.

The VisAR system is an image-guided navigation system that is designed to assist surgeons in placing pedicle screws accurately, during open or percutaneous spinal surgery. The system consists of Novarad's immersive augmented reality software running on the Microsoft Hololens 2 headset, image visible ARTags (AprilTags), a pre-operative planning workstation and the Novarad PACS server. It uses optical tracking technology to co-localize the virtual 3D image datasets to the patient and displays to the surgeon the location of pre-operatively planned operative tracks and the tracked surgical instruments relative to the acquired intraoperative patient's scan, onto the surgical field. The 3D scanned image, along with tracking information, are projected to the surgeons' retina using a transparent near-eye display stereoscopic headset, allowing the surgeon to both look at the navigation data at the same time.

Here's a summary of the acceptance criteria and the study proving the device meets them, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

Ask a specific question about this device

OpenSight is intended to enable users to display, manipulate, and evaluate 2D, 3D, and 4D digital images acquired from CR, DX, CT, MR, and PT sources. It is intended to visualize 3D imaging holograms of the patient, for preoperative localization and pre-operative planning of surgical options. OpenSight is designed for use only with performance-tested hardware specified in the user documentation.

OpenSight is intended to enable users to segment previously acquired 3D datasets, overlay, and register these 3D segmented datasets with the same anatomy of the patient in order to support pre-operative analysis.

OpenSight is not intended for intraoperative use. It is not to be used for stereotactic procedures.

OpenSight is intended for use by trained healthcare professionals, including surgeons, radiologists, chiropractors, physicians, cardiologists, technologists, and medical educators. The device assists doctors to better understand anatomy and pathology of patient.

OpenSight is the combination of Microsoft HoloLens and Novarad's medical imaging software to create threedimensional holograms of scanned images from different modalities including CR, DX, CT, MR, and PT. This combination of augmented reality glasses and imaging software allows the user to see and manipulate hologram images with the swipe of a finger.

OpenSight uses the HoloLens technology to register scanned images over the patient when user has OpenSight headset on and in use. This allows the user to both see the patient and through them, with dynamic holograms of the patient's internal anatomy. OpenSight tools/features include window level, segmentation and rendering, registration, motion correction, virtual tools, alignment, and the capability to measure distance and image intensity values, such as standardized uptake value. OpenSight displays measurement lines, and regions of interest. 3D images include but not limited to tumors, masses, appendices, heart, kidney, bladder, stomach, blood vessels, arteries, and nerves.

The OpenSight Augmented Reality system uses the Microsoft HoloLens hardware and the Microsoft 10 Operating System as the platform on which this system runs. The OpenSight technology is written specifically for this hardware. NovaPACS contributes to the process by creating annotations and providing the preoperative analysis of images that are fed to the OpenSight device.

The 3D holograms are created by a refractory system in the OpenSight device, using a combination of the Microsoft HoloLens hardware and the OpenSight technology for 3D image display and rendering. Images are actual visible rendered of the object in the OpenSight device. Images are streamed in a 2D format from the Novarad server via wireless communication. The communication is encrypted with 256 encryption.

Registration of the patient (reality) to another image data set such as MRI or CT (augmented reality) are performed by the OpenSight device which contains infrared ranging cameras which can map the surface geometry of an object creating a mesh of triangles conforming to whatever the object is. This can include the patient, the surrounding room, the table, etc. The resolution of the mesh is controlled by the device. For mapping a large object such as a room, a larger mesh would be utilized. Surface geometry mapping of a patient's anatomy utilizes the maximum resolution of the device while the user may walk around the object in a 360° circle mapping the object from many views in order to obtain the best localization in space.

The camera device from the OpenSight headset has ranging and localizing technology, which maps the surrounding environment, including the patient. It knows where objects are and mesh surface maps of these objects are created for determination of their 3D positioning. The 3D radiologic images are then rendered and surface shells of the patient's skin are matched to the augmented reality device when user has OpenSight headset on and in use. The advantage of this is if the patient moves this can be compensated for. The registration does not require expensive infrared tracking devices or other fiducials in order to perform registration. The anatomy and the correct patient will only register if there is a match of the data, thus diminishing the potentive use on the wrong patient with the wrong images.

The patient's anatomy can be displayed in 2D, 3D, or 4D mode. The OpenSight technology allows for virtual screens in space, which are manipulated by finger movement or from voice commands. These images are superimposed on the patient's anatomy and one can either scroll through the images or rotate three dimensionally. Because the holographic system has mapped the space of the room and patient, it "knows" where this is and therefore as one rotates around the patient or the anatomy in question, the images are automatically rotated with the device.

The actual visible rendering of the object in the OpenSight device (i.e. how fast can the hologram be updated as ones position relative to the patient changes) has no discernible time lag with the object rendering is in excess of 30 frames per second for standard image rendering). If one turns on advanced lighting and shadowing, cubic spine interpolation of the image and utilizes a large image dataset (in excess of 200 images) then there is a visible time lag between the holographic rendering and the projection on to the patient. See attached video (Motion.MOV) that demonstrates this. It is still less than a second under the worst-case scenario.

The rendering tools are derived from technology created in the NovaPACS system for allowing 3D tools, including simple image manipulation such as window/leveling as well as more advanced technologies of segmentation, rendering, registration and motion correction. Virtual tools as well as 3D annotations can be created and displayed in the holographic image. These might include lines, distance measurements, etc. They could also be volumetric measurements or outlines of tumors, anatomic structures, etc. The operating principles of these tools are similar to those with other 3D PACS devices, including technology that has already undergone 510K approval by Novarad Corporation.

The OpenSight Augmented Reality system is a device that allows the user to more quickly and more accurately define both anatomy and pathology by using mixed reality. One can see through this device the actual patient but also superimposed on this are holographic images of the patient's anatomy, which have been previously taken through MRI, CT, or other imaging techniques.

The following is a description of pre-operative use cases for OpenSight:

• . Ability to mark the appropriate entrance point, or angle, trajectory, and location for placement of a needle into the body, to extract a foreign body such as a piece of glass, to place a pedicle screw, etc. Being able to preoperatively identify the anatomy and expected trajectory for device insertion, could greatly aid in facilitating the speed and safety of procedures. Provided are images from three different preoperative interventions; a Percutaneous Discectomy, a Facet injection, and a Sacroiliac Joint. In each case, the OpenSight facilitates the positioning of the best trajectory for entrance into one of these structures.

• Ability to aid the operating physician to localize anatomy prior to intervention. This can be used as an aid to . augment, and correlate with the location of a patient's injury. For example, rib fractures can be difficult to localize in the operating room and frequently incisions will be larger than needed in order to plate a displaced rib fracture. Virtually all patients with acute appendicitis in the United States receive a CT scan prior to operative intervention for diagnostic purposes. With this technology, the location of the appendix could be identified and the surgeon would be able to see variations in the anatomy prior to making an incision in an area that may or may not have the appendix. Another example would be the location of masses, lymph nodes, or tumors that may be difficult to find due to body habitus or location. For example, the abilize a disc or vertebral body prior to operative intervention would save valuable surgical time and fluoroscopy.

• Ability to superimpose an anatomic atlas upon the patients' anatomy, allowing one to more readily identify structures that would either need to be treated or need to be avoided for a surgical procedure. This could be invaluable for example for a neurosurgeon to understand preoperatively, the best approach for cranial surgery. It could allow a head and neck surgeon to have a better understanding of the skull base in threedimensional detail. This internal visualization can be achieved without the surgeon ever making an incision on the patient. He/she of course can be guided by their best judgment, experience and training as to the ultimate approach and performance of any given procedure. OpenSight is intended simply as a guide.

• Ability for surgical trainees to visualize both the internal anatomy from cross sectional imaging such as CT, ● MRI, or PET scanning super imposed on a patient prior to actual operation providing invaluable 3-D understanding of a surgical approach. Such rendering can be performed just prior to the surgery allowing them to see the anatomy and orientation that would be encountered during the surgery. It is much less expensive and complicated than trying to print a 3-D model, which often is not available onsite and can take days to achieve. It also allows the trainee to interrogate in a virtual manner the anatomy of a given area and understand the structural relationships, critical structures that may complicate or interfere with surgery, as well as the unique size/position/orientation of a given patient's anatomy.

• Some operations are exceedingly complex and require a much greater depth of understanding in order to ● execute. Such is the case with congenital heart malformations where complex three-dimensional vascular anatomy makes surgical treatment difficult at best. Users are able to visualize this anatomy preoperatively in OpenSight before surgically opening the patient's chest and could potentially speed the operation and allow the surgeons to be better equipped to perform the procedure. Currently these types of procedures are performed after a surgeon has done complex and time consuming 3-D printing of models in order to better understand the anatomy. OpenSight allows one to render this in 2-D, 3-D and 4-D. In this use case, the images do not need to be on the patient. The doctor can rotate and magnify the anatomy free of the patient to get a better visual picture.

• As part of the preoperative experience, the target organs can be colored, outlined, or annotated in the medical images using the Novarad 3-D viewer. The annotated-holographic images can be shown to the patient or family superimposed on the patient. This would make the interpretation of the images much clearer. This will improve a patients understanding of the risks and the complexities of a surgical procedure.

• Surgeons in general, do not have the same degree of training in imaging and image processing as radiologists and it is often difficult for them to take 2-dimensional anatomy and apply this to their 3-dimensional world. OpenSight will allow Surgeons to better understand complex anatomy and disease processes by taking the data rich information, which they already have, and providing this in a more accessible format through holographic imaging. The value of the OpenSight is that it not only allows one to see the 3-dimensional data sets but also it can be co-localized to the patient and gives the 3-dimensional understanding of what he is attempting to do. Holographic augmented reality allows one to see with better understanding because the images are co-localized to the patient. The system with its mapping cameras, maps both the patient and the surrounding environment; from above, to the side, behind or even underneath the patient.

One possible example scenario of using OpenSight for preoperative planning is described in Appendix D.

OpenSight is not designed as a primary tool for disease detection or diagnosis.

OpenSight integrates with NovaPACS software.

OpenSight contains wireless technology using Wi-Fi 802.11ac networking standard. The wreless technology is used to stream images in a 2D format from a Novarad server onto the OpenSight headset. Images are actual visible rendered of the object in the OpenSight device with reliable and accurate information. The wireless information transfer is encrypted with 256 encryption for data security.

Here's an analysis of the acceptance criteria and study detailed in the provided text:

Acceptance Criteria and Device Performance

• Highest average: 1.67 mm
• Lowest average: 0 mm
  Mean offsets (by angle and distance):
• 0 degrees, 6 inches: 0.8596 mm (SD 1.7189 mm)
• 0 degrees, 1 foot: 0.9861 mm (SD 1.1305 mm)
• 90 degrees, 6 inches: 1.5213 mm (SD 2.0604 mm)
• 90 degrees, 1 foot: 0.8050 mm (SD 1.1602 mm)
• 135 degrees, 6 inches: 2.0825 mm (SD 1.8636 mm)
• 135 degrees, 1 foot: 1.6913 mm (SD 2.3016 mm)
  The angle with the lowest mean offset was 90 degrees and one foot away (0.8050 mm). The least amount of deviation was at 0 degrees and one foot away (SD 1.1305 mm). The confidence interval for 0 degrees and 6 inches away contained zero, suggesting no significant offset. |
  | Graphic Rendering Frame Rate | - Standard image rendering: In excess of 30 frames per second (fps).
• Advanced lighting/shadowing, cubic spine interpolation, large dataset: Visible time lag, less than a second.
• Images displayed: 60 fps (normal mode 30 fps).
• Volume mode geometry recomputing: 6 fps.
• Alignment mode geometry recomputing: 13-15 fps.
• Slice mode geometry recomputing: 40-50 fps. |
  | Surface Geometry Mapping (registration) | The device uses infrared ranging cameras to map surface geometry, creating a mesh of triangles. It can compensate for patient movement and does not require fiducials. Registration only occurs if there is a match of data, preventing use of wrong images on the wrong patient. |
  | Environmental Robustness | Ambient light levels (3743, 407.8, & 157.9 LUX) in various room settings (including OR and office) did not affect the field of view or object appearance. |

Study Details

1. Sample sizes used for the test set and data provenance:

   • Sphere Test: One MRI calibration sphere. Scanned by CT modality.
   • Box and BB testing: One box with copper BBs. Scanned by CT modality.
   • Data Provenance: Not explicitly stated, but the objects ("Riverwood's imaging") seem to be test phantoms, suggesting internally generated data rather than patient data. The study describes physical objects and their manipulation, implying prospective testing in a controlled environment directly with the device.

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • Sphere Test: The "ground truth" (physical circumference/diameter) was established by direct physical measurement using a sewing measuring tape. No human experts were involved in establishing this physical ground truth beyond the person performing the measurement.
   • Box and BB testing: The "ground truth" (physical BB locations) was established by direct physical measurement using an H&H 6' Dial Caliper. No human experts were involved in establishing this physical ground truth.
   • Qualifications: Not applicable for establishing the physical ground truth measurements.

3. Adjudication method for the test set:

   • No adjudication method was described as the ground truth was based on direct physical measurements of the test objects.

4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No MRMC comparative effectiveness study was conducted or described in this submission. The testing focused on the device's accuracy in rendering and registration, not on its impact on human reader performance.

5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

   • Yes, performance tests such as the "Sphere Test," "Box and BB testing," "Frame Rates," and "Surface Geometry Mapping" primarily evaluated the OpenSight software and hardware in a standalone manner, assessing its ability to render, register, and display images accurately against physical ground truth. While a user operates the device, the metrics measured are characteristics of the device's output rather than the user's interpretive performance.

6. The type of ground truth used:

   • Physical Measurements/Phantom Ground Truth: For the "Sphere Test" and "Box and BB testing," the ground truth was established by direct physical measurements of the test objects (sphere and box with BBs).
   • Device Specifications: For frame rates and image quality, the ground truth is implicitly defined by expected or acceptable performance specifications for the device's rendering capabilities.

7. The sample size for the training set:

   • The document does not explicitly mention a "training set" size for the OpenSight device. This submission primarily focuses on verification and validation testing of the commercialized product, not on the development or training of AI/machine learning models. The device's operation involves rendering existing medical images, not learning from a new dataset in the typical ML sense.

8. How the ground truth for the training set was established:

   • As no "training set" in the context of AI/ML was discussed, this question is not applicable. The device processes pre-acquired medical images (CR, DX, CT, MR, PT) and renders them as holograms based on its programmed algorithms, not through a learning phase in the context of this submission.

Ask a specific question about this device

NovaPACS is intended for the viewing, archiving, analysis, annotation, distribution, editing, fusion, and processing of digital medical images and data acquired from diagnostic imaging devices and all DICOM devices, including mammography.

NovaPACS is intended for use by trained healthcare professionals, including radiologists, physicians, technologists, clinicians, and nurses. NovaPACS allows the end user to display, manipulate, archive, and evaluate images.

Mobile devices are not intended to replace a full workstation and should be used only when there is no access to a workstation. They are not to be used for mammography or fMRI. Mobile devices are used for diagnosis of medical images from different modalities including CT, MR, US, CR/DX, NM, PT, and XA. For a list of compatible mobile platforms see NovaPACS Diagnostic Viewer User Manual.

While NovaPACS full workstation provides tools to assist the healthcare professional determine diagnostic viability, it is the user's responsibility to ensure quality, display contrast, ambient light conditions, and to confirm image compression ratios are consistent with the generally accepted standards of the clinical application.

NovaPACS is intended for providing analysis and visualization of functional MRI data of the human brain, presenting derived properties and parameters in a clinically useful context.

NovaPACS is a picture archiving and communication system software that retrieves, archives, and displays images and data from all common modalities. NovaPACS uses a variety of workstations, including a Technologist Workstation, Enterprise Radiologist Workstation, Cardio Viewer and Workstation, NovaMG Workstation, and NovaWeb Web Viewer.

The NovaPACS software makes images and data available in digital format from all common modalities. The images are viewed on a computer monitor or portable device. NovaPACS tools/features include the following: window, level, zoom, pan, digital subtraction, ejection, cross localization, note-taking ability, voice dictation, and other similar tools. It includes the capability to measure distance and image intensity values, such as standardized uptake value. NovaPACS displays measurement lines, annotations, regions of interest, and fusion blending control functionality. Advanced features include 3D image rendering, virtual fly-through, time domain imaging, vessel analysis, and blood oxygen level dependent (BOLD) fMRI.

BOLD fMRI analysis is used to highlight small susceptibility changes in the human brain in areas with altered bloodflow resulting from neuronal stimulation. The Functional Processing Software includes features such as scalp stripping, 3D motion correction, smoothing, coregistration, normalize images to MNI templates, and warping.

Images and data are stored on a digital archive with multiple redundances; images and data are available on-site and off-site. Novarad provides all software, including third party software (i.e. Windows® OS). NovaPACS software resides on third party hardware, which may vary depending on the client's PACS needs. All hardware is connected to the radiology department local area network.

NovaPACS integrates with NovaRIS and may integrate with any other third party RIS software that has HL7 interface capabilities.

NovaPACS integrates with Novarad Mobile Rad application and web viewers to display data on 3rd party mobile platforms. Mobile devices are not intended to replace full workstation and should be used only when there is no access to a workstation. They are not to be used for mammography or fMRI.

The provided text describes NovaPACS, a picture archiving and communication system (PACS) software, and its substantial equivalence to predicate devices. However, the document does not contain a detailed study with specific acceptance criteria and reported device performance in the format requested. It mentions performance testing for the fMRI software features but lacks quantitative data for a table.

Here's a breakdown of the information that can be extracted, and where there are gaps:

1. Table of Acceptance Criteria and Reported Device Performance

This information is not provided in the document in a quantifiable manner that allows for a table comparing acceptance criteria against specific performance metrics (e.g., sensitivity, specificity, accuracy, or other benchmarked values). The document states:

• "Nineteen test cases were run on the NovaPACS to fulfill the fMRI requirements. All 19 test cases passed."
• "NovaPACS software passes all performance requirements and meets all specifications prior to release, including:
  • All requirements in the iteration have a test case and the test case has run and passed.
  • All Acceptance tests have passed
  • All Current tests have passed
  • All high-impact bugs have been corrected and verified by Quality Assurance
  • Any unresolved anomalies have been assessed in a risk meeting, and it has been found that they do not pose a safety risk to the end user (or their patients) and do not substantially affect the performance of NovaPACS software."

These are general statements about successful internal testing and meeting requirements, but they do not disclose the specific "acceptance criteria" (e.g., a specific numerical threshold for an imaging performance metric) or the "reported device performance" against those criteria.

2. Sample size used for the test set and the data provenance

• Sample size for test set: The document mentions "Nineteen test cases were run on the NovaPACS to fulfill the fMRI requirements." It does not specify the number of cases (e.g., patient studies or images) within these test cases, nor does it detail the specific characteristics or provenance of the data used in these tests.
• Data provenance: Not specified (e.g., country of origin, retrospective or prospective).

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

This information is not provided. The document does not mention the involvement of experts for establishing ground truth during the performance testing.

4. Adjudication method for the test set

This information is not provided.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, and the effect size of how much human readers improve with AI vs without AI assistance

• MRMC study: No, an MRMC comparative effectiveness study was not done. The document explicitly states: "There are no clinical tests to compare NovaPACS and predicate devices, as they are software products that send and store images and information."
• Effect size of human reader improvement: Not applicable, as no MRMC study was performed.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

• Standalone performance: The testing described focuses on the functionality and safety of the NovaPACS fMRI software features ("Nineteen test cases were run on the NovaPACS to fulfill the fMRI requirements. All 19 test cases passed."). This implies standalone testing of the algorithm's performance against predefined requirements, as no human-in-the-loop study is mentioned. However, specific metrics of this standalone performance are not shared beyond "all passed."

7. The type of ground truth used

The document does not specify the type of ground truth used for performance testing (e.g., expert consensus, pathology, outcomes data). It broadly refers to "fMRI requirements," suggesting that the tests validated the software's ability to correctly process and display fMRI data according to established functional MRI analysis methodologies, rather than a diagnostic ground truth.

8. The sample size for the training set

This information is not provided. The document describes NovaPACS as "a picture archiving and communication system software," and its fMRI analysis component. It does not explicitly state that the fMRI feature relies on machine learning or AI models that would require a "training set" in the conventional sense. The focus appears to be on general software functionality and adherence to fMRI processing standards.

9. How the ground truth for the training set was established

This information is not provided, as a training set is not explicitly mentioned or implied for an AI/ML model.

Ask a specific question about this device

NovaPACS is intended for the viewing, analysis, annotation, registration, distribution, editing, fusion, and processing of digital medical images and data acquired from diagnostic imaging devices and all DICOM devices, etc.

NovaPACS is intended for use by trained healthcare professionals, including radiologists, physicians, technologists, clinicians, and nurses. NovaPACS allows the end user to display, manipulate, archive, and evaluate images.

Mobile devices are not intended to replace a full workstation and should be used only when there is no access to a workstation. They are not to be used for mammography. Mobile devices are used for diagnosis of medical images from different modalities including CT, MR, US, CR/DX, NM, PT, and XA. For a list of compatible mobile platforms see NovaPACS Diagnostic Viewer User Manual.

While NovaPACS full workstation provides tools to assist the healthcare professional determine diagnostic viability, it is the user's responsibility to ensure quality, display contrast, ambient light conditions, and to confirm image compression ratios are consistent with the generally accepted standards of the clinical application.

NovaPACS is a picture archiving and communication system software that retrieves, archives, and displays images and data from all common modalities. NovaPACS uses a variety of workstations, including a Technologist Workstation, Enterprise Radiologist Workstation, Cardio Viewer and Workstation, NovaMG Workstation, and NovaWeb Web Viewer. NovaPACS uses a vatiety of mobile platforms and browers including iPad 2 (Safari Browser), Nexus 7 (Chrome Browser), and iPad mobileRAD (Native Application). For a list of possible browser choices for one platform that are valid for diagnostic viewing see NovaPACS Diagnostic Viewer User Manual.

The NovaPACS software makes images and data available in digital format from all common modalities. The images are viewed on a computer monitor or portable device. NovaPACS tools/features include the following: window, level, zoom, pan, digital subtraction, ejection, cross localization, note-taking ability, voice dictation, and other similar tools. It includes the capability to measure distance and image intensity values, such as standardized uptake value. NovaPACS displays measurement lines, annotations, regions of interest, and fusion blending control functionality. Advanced features include 3D image rendering, virtual fly-through, time domain imaging, and vessel analysis.

Images and data are stored on a digital archive with multiple redundancies; images and data are available on-site and off-site. Novarad provides all software, including third party software (i.e. Windows® OS). NovaPACS software resides on third party hardware, which may vary depending on the client's PACS needs. All hardware is connected to the radiology department local area network.

NovaPACS integrates with NovaRIS and may integrate with any other third party RIS software that has HL7 anterface capabilities.

NovaPACS integrates with Novarad Mobile Rad application and web viewers to display data on 3rd party mobile platforms. Mobile devices are not intended to replace full workstation and should be used only when there is no access to a workstation. They are not to be used for mammography.

Here's a breakdown of the acceptance criteria and study details for the NovaPACS device, extracted from the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria are not explicitly listed in a quantitative table format with specific thresholds. Instead, they are described qualitatively through the results of the clinical and performance testing.

2. Sample Size Used for the Test Set and Data Provenance

• Test Set Sample Size: For the clinical testing, the radiologists evaluated six typical cases for each imaging modality, across seven modalities (CT, MR, US, CR/DX, NM, PT, and XA). This means 6 cases/modality * 7 modalities = 42 cases were used in the clinical evaluation. These cases were evaluated on multiple mobile device platforms (Native App (mobileRad), iOS, Android, and Windows mobile device platforms).
• Data Provenance: The clinical testing was conducted by a panel of board-certified radiologists in the United States. The description refers to "typical cases," suggesting these were retrospective cases, although it's not explicitly stated as retrospective or prospective.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications

• Number of Experts: Three board-certified radiologists were used.
• Qualifications: They were described as "board certified radiologists in the United States." No specific years of experience are mentioned.

4. Adjudication Method for the Test Set

• The text states: "For each study the radiologist individually rated the contrast, sharpness, artifact and overall quality..." and "Each radiologist agreed that the Native App (mobileRad), IOS, Android, and Windows mobile devices were comparable..." and "Each radiologist agreed that the same diagnosis would be made..."
• This indicates a consensus-based approach among the three radiologists rather than a formal pre-defined adjudication method like 2+1 or 3+1. It appears they reached a unanimous agreement on the comparability and diagnostic quality.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and Effect Size

• Yes, a form of MRMC comparative effectiveness study was done. The clinical testing involved multiple readers (3 radiologists) evaluating multiple cases across different mobile platforms against a predicate workstation.
• Effect Size: The study's primary conclusion regarding effect size is qualitative: the radiologists agreed that the mobile devices were "comparable," "adequate quality for clinical use," and "equivalent" to the predicate workstation for diagnostic purposes. A quantitative effect size (e.g., specific metrics like AUC difference or sensitivity/specificity improvement) is not provided in this summary, as the study focused on demonstrating non-inferiority/equivalence qualitatively for diagnostic performance.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done

• No, a standalone (algorithm only) performance study was not done. NovaPACS is a PACS system designed for human use (viewing, analysis, annotation) and its evaluation focused on its utility with a human user, not as an autonomous diagnostic algorithm. While there's "performance testing" of the software itself (99% passed), this is system-level functional testing, not a diagnostic accuracy assessment in a standalone manner.

7. The Type of Ground Truth Used

• The ground truth for the clinical study was based on expert consensus (the agreement of the three board-certified radiologists) regarding the image quality, diagnostic assurance, and the ability to make the same diagnosis compared to the predicate workstation. The predicate NovaPACS workstation itself serves as the de facto "ground truth" or reference standard for comparison in this 510(k) submission, as the goal is to show substantial equivalence.
• There's no mention of pathology or outcomes data being used as ground truth.

8. The Sample Size for the Training Set

• The document does not specify a sample size for a training set. This is because NovaPACS is a Picture Archiving and Communications System (PACS), not an AI/ML-driven diagnostic algorithm that typically relies on a distinct training phase with labeled data. Its "training" would primarily involve software development, bug fixing, and internal quality assurance, rather than machine learning model training on specific image datasets.

9. How the Ground Truth for the Training Set Was Established

• As NovaPACS is not an AI/ML diagnostic algorithm, the concept of a "training set" with established ground truth for diagnostic purposes (e.g., presence/absence of disease) is not applicable in the context of this 510(k) summary. The "ground truth" for its development would be its functional specifications, software requirements, and the expected behavior of a PACS system relative to established standards and predicate devices.

Ask a specific question about this device

NovaPACS is intended for the viewing, archiving, analysis, annotation, distribution, editing, fusion, and processing of digital medical images and data acquired from diagnostic imaging devices and all DICOM devices, etc.

NovaPACS is intended for use by trained healthcare professionals, including radiologists, physicians, technologists, clinicians, and nurses. NovaPACS allows the end user to display, manipulate, archive, and evaluate images.

NovaPACS is not intended for diagnostic image review on a mobile platform.

While NovaPACS provides tools to assist the healthcare professional determine diagnostic viability, it is the user's responsibility to ensure quality, display contrast, ambient light confirm image compression ratios are consistent with the generally accepted standards of the clinical application.

NovaPACS is a picture archiving and communication system software that retrieves, archives, and displays images and data from all common modalities. NovaPACS uses a variety of workstations, including a Technologist Workstation, Enterprise Radiologist Workstation, Cardio Viewer and Workstation, NovaMG Workstation, and NovaWeb Web Viewer.

The NovaPACS software makes images and data available in digital format from all common modalities. The images are viewed on a computer monitor or portable device. NovaPACS tools/features include the following: window, level, zoom, pan, digital subtraction, ejection, cross localization, note-taking ability, voice dictation, and other similar tools. It includes the capability to measure distance and image intensity values, such as standardized uptake value. NovaPACS displays measurement lines, annotations, regions of interest, and fusion blending control functionality. Advanced features include 3D image rendering, virtual colonoscopy, and vessel analysis.

Images and data are stored on a digital archive with multiple redundancies; images and data are site and off-site. Novarad provides all software, including third party software (i.e. Windows® OS). NovaPACS software resides on third party hardware, which may vary depending on the client's PACS needs. All hardware is connected to the radiology department local area network.

NovaPACS integrates with NovaRIS and may integrate with any other third party RIS software that has HL7 interface capabilities.

NovaPACS integrates with Novarad Mobile Rad application to display data on 3rd party mobile platforms. Mobile Rad is not intended to replace a full diagnostic workstation.

This K132853 510(k) summary for NovaPACS does not contain the specific information required to complete the request thoroughly. The document describes the device, its intended use, and substantial equivalence to predicate devices, but lacks detailed performance data and study specifics.

Here's a breakdown of what can and cannot be extracted from the provided text regarding acceptance criteria and a study proving fulfillment:

1. A table of acceptance criteria and the reported device performance:

The document mentions "performance requirements" and "specifications" but does not provide a specific table of acceptance criteria with corresponding device performance values. It states:

• "NovaPACS software passes all performance requirements and meets all specifications prior to release, including:"
  • "All requirements in the iteration have a test case and the test case has run and passed a."
  • "All Acceptance tests have passed b."
  • "All Current tests have passed C."
  • "d. All high-impact bugs have been corrected and verified by Quality Assurance"
  • "e. Any unresolved anomalies have been assessed in a risk meeting, and it has been found that they do not pose a safety risk to the end user (or their patients) and do not substantially affect the performance of NovaPACS software."

It also states: "Performance testing results show that the software features of NovaPACS operate correctly and safely and meet equivalent objectives and perform equivalent functions as those represented in the predicate devices."

And more broadly: "Thorough software testing has been performed for NovaPACS to safety and efficacy of the device. Of over 1200 test cases run on the NovaPACS Software, 99% passed, 1% failed, and 0% were blocked on their latest run. Of the failed tests, the majority represent minor user interface errors."

Lacking Information: Specific quantitative acceptance criteria (e.g., minimum accuracy, sensitivity, specificity, processing speed, image quality metrics) are not provided. The phrase "All Acceptance tests have passed" confirms the existence of such tests and their successful completion, but not the content of those tests or their specific targets.

2. Sample size used for the test set and the data provenance:

• Test set sample size: The document mentions "over 1200 test cases run on the NovaPACS Software." It does not specify if these test cases correspond to a specific number of patient cases or images, nor does it detail the nature of the data used in these tests. It seems to refer more to functional and system-level tests rather than a clinical dataset.
• Data provenance: Not specified. It's likely internal testing data related to software function.
• Retrospective or prospective: Not specified, but given the nature of the "test cases," it's likely part of a software development and validation process rather than a traditional clinical study with patient data collected retrospectively or prospectively.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

• Number of experts: Not specified.
• Qualifications of experts: Not specified.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set:

• Not specified. The document primarily discusses software testing for functionality and bug resolution, not expert review of medical images.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

• Not performed. The document explicitly states: "There are no clinical tests to compare NovaPACS and predicate devices, as they are software products that send and store images and information." This implies no MRMC study or study comparing human performance with or without AI assistance was conducted or deemed necessary for this type of PACS device. NovaPACS is described as a system for viewing, archiving, and processing, not as an AI diagnostic aid.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

• Not applicable in the typical AI sense. While the software has "features" like 3D rendering and vessel analysis, it's presented as a tool for "trained healthcare professionals" to "display, manipulate, archive, and evaluate images." It's not an autonomous diagnostic algorithm. The performance described ("99% passed" of 1200 test cases) relates to the correct functioning of the software itself, not its diagnostic accuracy in a standalone mode.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc):

• Not explicitly stated in terms of diagnostic ground truth. For the stated "test cases," the ground truth would be the expected functional behavior of the software feature (e.g., whether an image loads correctly, a measurement tool accurately calculates distance, or a 3D rendering displays as designed). It's not referring to ground truth for clinical diagnoses on patient images.

8. The sample size for the training set:

• Not applicable/Not provided. This document describes a PACS software system, which typically does not involve a "training set" in the machine learning sense for pattern recognition or diagnostic AI. The software's capabilities are based on coded algorithms and functionalities, not on learning from a dataset.

9. How the ground truth for the training set was established:

• Not applicable/Not provided. (See point 8).

In summary, the provided K132853 document focuses on demonstrating the functional equivalence and safety of the NovaPACS software as a Picture Archiving and Communication System (PACS) through software testing, rather than reporting on a clinical study of diagnostic performance using AI, which would typically involve detailed acceptance criteria, ground truth, and reader studies.

Ask a specific question about this device

The product being submitted is going to be used as a PACS system and provide viewing tools for radiologists, technologists, other physicians, cardiologists, and patients. It will also provide archiving of images and data. The product will be sold to hospitals, imaging centers, and clinics. It is a software product sold with accompanying off-the-shelf hardware, It is substantially equivalent to other products now being sold as PACS. Further it is for using DICOM for presentation of Mammography studies, not for image processing. Will include the standard features for Mammo of window/level/zoom, as well as volumetric measurements and other Mammo tools. Mammo will only be interpreted on 510K approved 5 Mp LCD monitors. The Mammo images will be stored lossless (non-lossy compressed). Digitized conventional screen film mammograms will not be used in soft copy format for primary diagnosis.

The PACS product provides a system where images and accompanying data are sent from the modalities to the radiologists and technologists in digital format. The images are then viewed on a computer with such available tools a window, level, zoom, pan, roi, digital subtraction, ejection fraction, cross localize, and many other similar tools and with the ability to make notes and dictate a report. The images and report are then stored long term on a digital archive with multiple redundancy. All the information is also made available as a web based product where referring physicians or radiologists can access the information from anywhere using a secure system. The software is provided by NovaRad along with some 31d party software, principally from windows, and resides on off-the-shelf hardware hooked up to the radiology department local area network.

Here's an analysis of the provided text regarding the acceptance criteria and study for the NovaRad PACS (K063153):

This 510(k) summary explicitly states that no clinical tests were performed to compare the device to a predicate. The justification provided is that the device is "merely software products that send and store images and information. Since the images are not changed or analyzed by the product, there is no safety issue, except possible related to the safety of the archived images."

Therefore, the following table and subsequent sections will reflect this lack of clinical study data.

Acceptance Criteria and Reported Device Performance

Study Details (or lack thereof)

1. Sample size used for the test set and the data provenance:

   • Not applicable. No clinical test set was used, as no clinical study comparing the devices was conducted. The equivalence was based on functional comparison and safety considerations regarding image storage.

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • Not applicable. No clinical test set or ground truth establishment by experts was performed for comparison between the predicate and the NovaRad PACS. The submission primarily relies on the functional equivalency of a PACS system and its ability to store and display images without alteration.

3. Adjudication method (e.g. 2+1, 3+1, none) for the test set:

   • Not applicable. No clinical test set was used, therefore no adjudication method was employed.

4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No. An MRMC comparative effectiveness study was not performed. The device is a Picture Archiving and Communication System (PACS) and does not involve AI assistance for image interpretation.

5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

   • No. This device is a PACS for image management and viewing, not an AI algorithm. Therefore, a standalone performance study as understood for AI devices was not conducted.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

   • Not applicable. No clinical ground truth was established or used for the purpose of demonstrating substantial equivalence. The claim of substantial equivalence is based on the system's ability to perform the same functions as the predicate PACS without altering image data.

7. The sample size for the training set:

   • Not applicable. As this is a PACS system designed for image storage and viewing, and not an AI or image analysis device, there is no "training set" in the context of machine learning.

8. How the ground truth for the training set was established:

   • Not applicable. As there is no training set, there was no ground truth to establish for it.

Ask a specific question about this device

Picture Archiving and Communications for presenting images and information from the modalities to radiologists or others for reading. Includes standard viewing tools made to help the radiologist or cardiologist or others organize the images and read them in an efficient and easy manner.

The PACS product provides a system where images and accompanying data are sent from the modalities to the radiologists and technologists in digital format. The images are then viewed on a computer with such available tools a window, level, zoom, pan, roi, digital subtraction, ejection fraction, cross localize, and many other similar tools and with the ability to make notes and dictate a report. The images and report are then stored long term on a digital archive with multiple redundancy. All the information is also made available as a web based product where referring physicians or radiologists can access the information from anywhere using a secure system. The software is provided by NovaRad along with some 3 party software, principally from windows, and resides on off-the-shelf hardware hooked up to the radiology department local area network.

The provided text is a 510(k) summary for the NovaRad Corporation's PACS product (NovaPACS). It explicitly states that "There are no clinical tests to compare the two as they are merely software products that send and store images and information. Since the images are not changed or analyzed by the product there is no safety issue, except possible related to the safety of the archived images."

Therefore, based on the provided document, the following answers apply:

1. A table of acceptance criteria and the reported device performance
No acceptance criteria or reported device performance from a study are provided in the document. The device is a Picture Archiving and Communication System (PACS) and is not intended to change or analyze medical images, but rather to present and store them. Substantial equivalence was claimed based on functional similarity to a predicate device, not clinical performance metrics.

2. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)
Not applicable. No clinical or performance studies using a test set were conducted or mentioned.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)
Not applicable. No clinical or performance studies using a test set were conducted or mentioned, hence no ground truth was established by experts for such a purpose.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set
Not applicable. No clinical or performance studies using a test set were conducted or mentioned.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
Not applicable. No MRMC study was conducted. The device is a PACS for image viewing and storage, not an AI-powered diagnostic or assistance tool.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
Not applicable. The device is a PACS for image viewing and storage and does not involve a standalone algorithm for diagnostic performance.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
Not applicable. No clinical or performance studies requiring ground truth were conducted or mentioned.

8. The sample size for the training set
Not applicable. No machine learning or AI components requiring a training set are described for this PACS device.

9. How the ground truth for the training set was established
Not applicable. No machine learning or AI components requiring a training set are described for this PACS device, therefore no ground truth for a training set was established.

Ask a specific question about this device