Real-time Volume Rendering Interaction in Virtual Reality
Published at : 29 Nov 2019
Volume : IJtech
Vol 10, No 7 (2019)
DOI : https://doi.org/10.14716/ijtech.v10i7.3259
Kalarat, K., Koomhin, P., 2019. Real-time Volume Rendering Interaction in Virtual Reality. International Journal of Technology. Volume 10(7), pp. 1307-1314
| Kosin Kalarat | School of Informatics, Walailak University, 222 Thaiburi, Thasala District, Nakhon Si Thammarat 80161, Thailand |
| Phanit Koomhin | School of Medicine, Walailak University, 222 Thaiburi, Thasala District, Nakhon Si Thammarat 80161, Thailand |
Volume visualization using Direct Volume Rendering
(DVR) techniques is used to view information inside 3D volumetric data. Data is
classified using a transfer function to emphasize or filter some parts of
volumetric information, such as that from Computed Tomography (CT) or Magnetic
Resonance Imaging (MRI). In this paper, we introduced an application for
real-time volume rendering interaction with 1D transfer functions using Virtual
Reality (VR) technology based on the Oculus Rift headset and Oculus Touch
controllers. Resulting images were visualized stereoscopically at 60 frames per
second using a ray-casting shader, which works based on Graphics Processing
Unit (GPU). To evaluate the system, 20 participants interacted with the
application to complete three tasks, including a free viewpoint scan, clipping
planes renderer, and an editable transfer function in the virtual environment.
Then, a survey was carried out using a questionnaire to gather data. Findings
showed that the average usability score for the application was 87.54, which
suggested that it was highly usable.
Direct volume rendering; Ray casting; Virtual environment; Virtual reality
Volume
rendering is an important visualization method in scientific visualization and
computer graphics. In medical visualization, Direct Volume Rendering (DVR) has
been used to project 3D information from medical data. For example, data from
Computed Tomography (CT) or Magnetic Resonance Imaging (MRI) can be
reconstructed using reconstruction algorithms, such as back projection or ART
or SART algorithms (Kalarat, 2005) to increase diagnostic capabilities and
operation planning. To make a medical diagnosis, doctors or radiologists must
assess patients’ bones, organs, or tumors from a program application display on
a personal computer and use a mouse, keyboard, and monitor to control their
viewpoint. However, such investigations are limited by the use of 2D screens.
A few years ago, Oculus VR developed and manufactured
Oculus Rift, a Virtual Reality (VR) headset with stereoscopic imaging and head
tracking. This technology made VR technology popular again due to its
effectiveness and low cost. Many large companies have been interested in this
technology and have developed and brought to market their own VR headsets, such
as High Tech Computer (HTC), Samsung, Sony, Lenovo, and so on, creating a red
ocean market. It is a good time for customers and VR developers to obtain these
devices, which enable a low cost VR experience. VR technology provides a new
way of interacting with virtual information,
Applications for VR have been found in a diverse range
of fields, such as construction, education, entertainment, the military,
medicine and manufacturing (Bahar et al., 2014).
Our team has been interested in the medical applications of DVR in VR. We used the effective head-mounted displays (HMDs) of VR technology currently available to develop an application that increased diagnostic viewpoints when visualizing 3D medical data via volume rendering in an immersive virtual environment. Unlike volume rendering in augmented realities (Kutter et al., 2008) which are in the real environment, in a virtual environment, users can interact with 3D medical data in real-time by rotating it or changing viewpoints. Our VR application was developed to run on an Oculus Rift headset and Oculus Touch controller, which was used to interact with clinical data. Unity3D, real-time development platform, was used to manage the interaction, render pipeline, and program the relevant shader to parallelize the time-consuming ray-casting algorithm. For this research, we developed a VR application to visualize medical data using a DVR technique on VR technology focusing on system interaction. Three main features, which included a free viewpoint scan, clipping plane renderer, and an editable transfer function, were used to evaluate the performance and usability of the VR application.
Figure 1 Virtual reality using an Oculus Rift headset
and Oculus Touch controller
In this
paper, we have presented the new way interaction with volume rendering in
immersive Virtual Environment using Oculus Rift and its controller applying for
medical volumetric data from MRI and CT-Scan. The interaction includes 3
features which are free viewpoint scan, clipping planes and editable transfer
function.
The application evaluation result for the performance and
usability shows that the VR application is able to provide the volumetric data
smoothly and intuitively with real-time interaction. Because of sufficient
frame rate, users have no motion sickness from the stereoscopic rendering of
volume data and they have the freedom to see every part of data with the free
viewpoint in real-time. Moreover, the application allows the user to use the
Oculus’s controller to interact with volumetric data easily, even for
inexperienced users. However, this research has been in the early stage of
interactive evaluation as the usability tests were tested by the participants,
who are not involved in radiological technology. Therefore, this application
will be evaluated by the professional radiologist or person who concerned about
this field.
In the future, we would evaluate the usability of these features
for the VR application with the specific group of users such as radiological
technologists or medical students who are studying in the subject of radiology
for the aspect of the interactive diagnostic radiology in Virtual Reality.
This research is supported by VR Inventors project from DEPA (Digital Economy Promotion Agency), Brain-science and Artificial intelligence research unit, and School of Informatics, Walailak University.
Bahar, Y.N., Landrieu, J., Pere, C.,
Nicolle, C., 2014. Simulation and Visualization of Thermal Metaphor in a
Virtual Environment for Thermal Building Assessment. International Journal
of Technology, Volume 5(1), pp. 3–13
Brooke, J., 1996. SUS-A Quick and Dirty
Usability Scale. Usability Evaluation in Industry, Volume 189(194), pp.
4–7
Drebin, A.R., Carpenter L., Hanrahan P.,
1988. Volume Rendering. ACM Siggraph Computer Graphics, Volume 22(4),
pp. 65–74
Fons, J., Monclús Lahoya, E., Vázquez
Alcocer, P., Navazo Álvaro, I., 2018. Rendering and Interacting with Volume
Models in Immersive Environments. In:
CEIG 18: XXVIII Spanish Computer
Graphics Conference: Madrid, Spain, June 27-29, 2018, European
Association for Computer Graphics (Eurographics), pp. 47–50
Heng, Y., Gu, L., 2006. GPU-based Volume
Rendering for Medical Image Visualization. In:
2005 IEEE Engineering in Medicine and
Biology 27th Annual Conference, pp. 5145–5148
Kalarat, K., 2014. Relief Mapping on
Facade of Sino Portuguese Architecture in Virtual Reality. In: 2014 Fourth International
Conference on Digital Information and Communication Technology and its
Applications (DICTAP), pp. 333–336
Kalarat, K., Narkbuakaew, W., Pintavirooj,
C., Sangworasil, M., 2005. Rapid Simultaneous Algebraic Reconstruction
Technique (SART) for Cone-beam Geometry on Clustering System. In: TENCON 2005-2005 IEEE Region 10 Conference, pp. 1–4
Kruger, J., Westermann, R., 2003.
Acceleration Techniques for GPU-based Volume Rendering. In: Proceedings of the 14th
IEEE Visualization 2003 (VIS'03), pp. 287–292
Kutter, O., Aichert, A., Bichlmeier, C.,
Traub, J., Heining, S.M., Ockert, B., Euler E., Navab, N., 2008. Real-time
Volume Rendering for High Quality Visualization in Augmented Reality. In: International Workshop on Augmented environments for Medical Imaging including
Augmented Reality in Computer-aided Surgery (AMI-ARCS 2008), New York, USA,
pp. 104–113
Scharsach, H., Hadwiger, M., Neubauer, A.,
Wolfsberger, S., Bühler, K., 2006. Perspective Isosurface and Direct Volume
Rendering for Virtual Endoscopy Applications. In: Eurovis, Volume 6,
pp. 315–322
Sherman, W.R., Craig, A.B., 2003.
Understanding Virtual Reality: Interface, Application, and Design. Morgan Kaufmann. USA, pp. 4–16