|Kosin Kalarat||School of Informatics, Walailak University, 222 Thaiburi, Thasala District, Nakhon Si Thammarat 80161, Thailand|
|Phanit Koomhin||School of Medicine, Walailak University, 222 Thaiburi, Thasala District, Nakhon Si Thammarat 80161, Thailand|
Volume visualization using Direct Volume Rendering (DVR) techniques is used to view information inside 3D volumetric data. Data is classified using a transfer function to emphasize or filter some parts of volumetric information, such as that from Computed Tomography (CT) or Magnetic Resonance Imaging (MRI). In this paper, we introduced an application for real-time volume rendering interaction with 1D transfer functions using Virtual Reality (VR) technology based on the Oculus Rift headset and Oculus Touch controllers. Resulting images were visualized stereoscopically at 60 frames per second using a ray-casting shader, which works based on Graphics Processing Unit (GPU). To evaluate the system, 20 participants interacted with the application to complete three tasks, including a free viewpoint scan, clipping planes renderer, and an editable transfer function in the virtual environment. Then, a survey was carried out using a questionnaire to gather data. Findings showed that the average usability score for the application was 87.54, which suggested that it was highly usable.
Direct volume rendering; Ray casting; Virtual environment; Virtual reality
Volume rendering is an important visualization method in scientific visualization and computer graphics. In medical visualization, Direct Volume Rendering (DVR) has been used to project 3D information from medical data. For example, data from Computed Tomography (CT) or Magnetic Resonance Imaging (MRI) can be reconstructed using reconstruction algorithms, such as back projection or ART or SART algorithms (Kalarat, 2005) to increase diagnostic capabilities and operation planning. To make a medical diagnosis, doctors or radiologists must assess patients’ bones, organs, or tumors from a program application display on a personal computer and use a mouse, keyboard, and monitor to control their viewpoint. However, such investigations are limited by the use of 2D screens.
A few years ago, Oculus VR developed and manufactured Oculus Rift, a Virtual Reality (VR) headset with stereoscopic imaging and head tracking. This technology made VR technology popular again due to its effectiveness and low cost. Many large companies have been interested in this technology and have developed and brought to market their own VR headsets, such as High Tech Computer (HTC), Samsung, Sony, Lenovo, and so on, creating a red ocean market. It is a good time for customers and VR developers to obtain these devices, which enable a low cost VR experience. VR technology provides a new way of interacting with virtual information, especially virtual 3D data.
Applications for VR have been found in a diverse range of fields, such as construction, education, entertainment, the military, medicine and manufacturing (Bahar et al., 2014).
Our team has been interested in the medical applications of DVR in VR. We used the effective head-mounted displays (HMDs) of VR technology currently available to develop an application that increased diagnostic viewpoints when visualizing 3D medical data via volume rendering in an immersive virtual environment. Unlike volume rendering in augmented realities (Kutter et al., 2008) which are in the real environment, in a virtual environment, users can interact with 3D medical data in real-time by rotating it or changing viewpoints. Our VR application was developed to run on an Oculus Rift headset and Oculus Touch controller, which was used to interact with clinical data. Unity3D, real-time development platform, was used to manage the interaction, render pipeline, and program the relevant shader to parallelize the time-consuming ray-casting algorithm. For this research, we developed a VR application to visualize medical data using a DVR technique on VR technology focusing on system interaction. Three main features, which included a free viewpoint scan, clipping plane renderer, and an editable transfer function, were used to evaluate the performance and usability of the VR application.
Figure 1 Virtual reality using an Oculus Rift headset and Oculus Touch controller
In this paper, we have presented the new way interaction with volume rendering in immersive Virtual Environment using Oculus Rift and its controller applying for medical volumetric data from MRI and CT-Scan. The interaction includes 3 features which are free viewpoint scan, clipping planes and editable transfer function.
The application evaluation result for the performance and usability shows that the VR application is able to provide the volumetric data smoothly and intuitively with real-time interaction. Because of sufficient frame rate, users have no motion sickness from the stereoscopic rendering of volume data and they have the freedom to see every part of data with the free viewpoint in real-time. Moreover, the application allows the user to use the Oculus’s controller to interact with volumetric data easily, even for inexperienced users. However, this research has been in the early stage of interactive evaluation as the usability tests were tested by the participants, who are not involved in radiological technology. Therefore, this application will be evaluated by the professional radiologist or person who concerned about this field.
In the future, we would evaluate the usability of these features for the VR application with the specific group of users such as radiological technologists or medical students who are studying in the subject of radiology for the aspect of the interactive diagnostic radiology in Virtual Reality.
This research is supported by VR Inventors project from DEPA (Digital Economy Promotion Agency), Brain-science and Artificial intelligence research unit, and School of Informatics, Walailak University.
Bahar, Y.N., Landrieu, J., Pere, C., Nicolle, C., 2014. Simulation and Visualization of Thermal Metaphor in a Virtual Environment for Thermal Building Assessment. International Journal of Technology, Volume 5(1), pp. 3–13
Brooke, J., 1996. SUS-A Quick and Dirty Usability Scale. Usability Evaluation in Industry, Volume 189(194), pp. 4–7
Drebin, A.R., Carpenter L., Hanrahan P., 1988. Volume Rendering. ACM Siggraph Computer Graphics, Volume 22(4), pp. 65–74
Fons, J., Monclús Lahoya, E., Vázquez Alcocer, P., Navazo Álvaro, I., 2018. Rendering and Interacting with Volume Models in Immersive Environments. In: CEIG 18: XXVIII Spanish Computer Graphics Conference: Madrid, Spain, June 27-29, 2018, European Association for Computer Graphics (Eurographics), pp. 47–50
Heng, Y., Gu, L., 2006. GPU-based Volume Rendering for Medical Image Visualization. In: 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference, pp. 5145–5148
Kalarat, K., 2014. Relief Mapping on Facade of Sino Portuguese Architecture in Virtual Reality. In: 2014 Fourth International Conference on Digital Information and Communication Technology and its Applications (DICTAP), pp. 333–336
Kalarat, K., Narkbuakaew, W., Pintavirooj, C., Sangworasil, M., 2005. Rapid Simultaneous Algebraic Reconstruction Technique (SART) for Cone-beam Geometry on Clustering System. In: TENCON 2005-2005 IEEE Region 10 Conference, pp. 1–4
Kruger, J., Westermann, R., 2003. Acceleration Techniques for GPU-based Volume Rendering. In: Proceedings of the 14th IEEE Visualization 2003 (VIS'03), pp. 287–292
Kutter, O., Aichert, A., Bichlmeier, C., Traub, J., Heining, S.M., Ockert, B., Euler E., Navab, N., 2008. Real-time Volume Rendering for High Quality Visualization in Augmented Reality. In: International Workshop on Augmented environments for Medical Imaging including Augmented Reality in Computer-aided Surgery (AMI-ARCS 2008), New York, USA, pp. 104–113
Scharsach, H., Hadwiger, M., Neubauer, A., Wolfsberger, S., Bühler, K., 2006. Perspective Isosurface and Direct Volume Rendering for Virtual Endoscopy Applications. In: Eurovis, Volume 6, pp. 315–322
Sherman, W.R., Craig, A.B., 2003. Understanding Virtual Reality: Interface, Application, and Design. Morgan Kaufmann. USA, pp. 4–16