Stefan Marks

Evaluation of game engines for simulated surgical training

The increasing complexity and costs of surgical training and the constant development of new surgical procedures has made virtual surgical training an essential tool in medical education. Unfortunately, commercial tools are very expensive and have a small support base. Game engines offer unique advantages for the creation of highly interactive and collaborative environments. This paper examines the suitability of currently available game engines for developing applications for medical education and simulated surgical training. We formally evaluate a list of available game engines for stability, availability, the possibility of custom content creation and the interaction of multiple users via a network. Based on these criteria, three of the highest ranked engines are used for further case studies. We found that in general it is possible to easily create scenarios with custom medical models that can be cooperatively viewed and interacted with. Limitations in physical simulation capabilities make some engines unsuitable for fully interactive applications, but they can be used in combination with predefined animations. We show that overall game engines represent a good foundation for low cost virtual surgery applications and we discuss technologies which can be used to further extend their physical simulation capabilities.

Towards the Holodeck: Fully Immersive Virtual Reality Visualisation of Scientific and Engineering Data

In this paper, we describe the development and operating principles of an immersive virtual reality (VR) visualisation environment that is designed around the use of consumer VR headsets in an existing wide area motion capture suite. We present two case studies in the application areas of visualisation of scientific and engineering data. Each of these case studies utilise a different render engine, namely a custom engine for one case and a commercial game engine for the other. The advantages and appropriateness of each approach are discussed along with suggestions for future work.

An Intuitive Tangible Game Controller

This paper outlines the development of a sensory feedback device providing a tangible interface for controlling digital environments, in this example a flight simulator, where the intention for the device is that it is relatively low cost, versatile and intuitive. Gesture based input allows for a more immersive experience, so rather than making the user feel like they are controlling an aircraft the intuitive interface allows the user to become the aircraft that is controlled by the movements of the users hand. The movements are designed to allow a sense of immersion that would be difficult to achieve with an alternative interface. A vibrotactile based haptic feedback is incorporated in the device to further enhance the connection between the user and the game environment by providing immediate confirmation of game events. When used for navigating an aircraft simulator, this device invites playful action and thrill. It bridges new territory on portable, low cost solutions for haptic devices in gaming contexts.

Creating Creative Technologists: Playing With(in) Education

The Bachelor of Creative Technologies (BCT) degree is offered by Colab, a unique academic unit at Auckland University of Technology. The unit is a research-teaching nexus or ‘collaboratory’ at the intersection of four existing schools (Art and Design, Communications and Media Studies, Computer and Mathematical Sciences and Engineering) in the Faculty of Design and Creative Technologies. The goal of Colab is to develop new experimental alliances, research collaborations and learning experiences across these overlapping disciplines. Its researchers, students and stakeholders are encouraged to imagine, construct and navigate rapidly changing social, economic, technological and career environments.

An investigation of meshless deformation for fast soft tissue simulation in virtual surgery applications

B. Tolsdorff, A. Petersik, B. Pflesser, A. Pommert, U. Tiede, R. Leuwer, K. H. Höhne Department of ENT, University Medical Center Hamburg–Eppendorf, Hamburg, Germany VOXEL-MAN Group, University Medical Center Hamburg– Eppendorf, Hamburg, Germany Department of ENT, Krefeld General Hospital, Krefeld, Germany Abstract Virtual simulation of bone drilling interventions in middle ear surgery has reached a level close to reality. The aim of our study was to evaluate the quality of patient specific models as well as the benefit of preoperative simulation for the surgical procedure to follow. Models based on 0.4 mm CT slice distances or better were found to be excellent in handling, acceptable in showing mastoidal organs and helpful in giving an impression of the individual surgical situation, although landmarks are easier detected in the vital mastoid. Depending on the patient’s complexity level, the quality of preoperative rehearsal of bone drilling interventions in middle ear surgery is substantially improved by individual preoperative virtual simulation of the procedure.

Evaluation of the effectiveness of head tracking for view and avatar control in virtual environments

Virtual environments (VE) are gaining in popularity and are increasingly used for teamwork training purposes, e.g., for medical teams. We have identified two shortcomings of modern VEs: First, nonverbal communication channels are essential for teamwork but are not supported well. Second, view control in VEs is usually done manually, requiring the user to learn the controls before being able to effectively use them. We address those two shortcomings by using an inexpensive webcam to track the users head. The rotational movement is used to control the head movement of the users avatar, thereby conveying head gestures and adding a nonverbal communication channel. The translational movement is used to control the view of the VE in an intuitive way. Our paper presents the results of a user study designed to investigate how well users were able to use our systems advantages.

Getting up your nose: a virtual reality education tool for nasal cavity anatomy

This article explores the application of virtual reality (VR) to the area of anatomical education, specifically the shape of and the airflow through the human nasal cavity. We argue the benefits of VR technology in this specific domain, and describe the creation of the VR application which is intended to be used in future courses. Through two preliminary case studies, we describe our experiences, and discuss advantages and disadvantages of the use of VR in this area.

Immersive visualisation of 3-dimensional spiking neural networks

Recent development in artificial neural networks has led to an increase in performance, but also in complexity and size. This poses a significant challenge for the exploration and analysis of the spatial structure and temporal behaviour of such networks. Several projects for the 3D visualisation of neural networks exist, but they focus largely on the exploration of the spatial structure alone, and are using standard 2D screens as output and mouse and keyboard as input devices. In this article, we present NeuVis, a framework for an intuitive and immersive 3D visualisation of spiking neural networks in virtual reality, allowing for a larger variety of input and output devices. We apply NeuVis to NeuCube, a 3-dimensional spiking neural network learning framework, significantly improving the user’s abilities to explore, analyse, and also debug the network. Finally, we discuss further venues of development and alternative render methods that are currently under development and will increase the visual accuracy and realism of the visualisation, as well as further extending its analysis and exploration capabilities.

Optimisation and comparison framework for monocular camera-based face tracking

Tracking the position and orientation of the human face with respect to a camera has valuable applications in human computer interaction (HCI). Examples are navigating through a virtual environment, controlling objects using head gestures, and enabling avatars in a virtual environment to reflect the users behaviour. Tracking performance can be heavily influenced by environmental parameters. Developers and users of face tracking plugins without computer vision experience need guidelines how to optimise face tracking performance in real world set-ups and they need measures how environmental parameters influence the results. In this paper we develop a qualitative framework for determining ideal working conditions of face tracking algorithms. We apply our framework to a commercially available face tracking solution and present the results of this analysis.

From von Neumann Architecture and Atanasoffs ABC to Neuro-Morphic Computation and Kasabov’s NeuCube: Principles and Implementations

During the 1940s John Atanasoff with the help of one of his students Clifford E. Berry, at Iowa State College, created the ABC (Atanasoff-Berry Computer) that was the first electronic digital computer. The ABC computer was not a general-purpose one, but still, it was the first to implement three of the most important ideas used in computers nowadays: binary data representation; using electronics instead of mechanical switches and wheels; using a von Neumann architecture, where the memory and the computations are separated. A new computational paradigm, named as Neuromorphic, utilises the above two principles, but instead of the von Neumann principle, it integrates the memory and the computation in a single module a spiking neural network structure. This chapter first reviews the principles of the earlier published work by the team on neuromorphic computational architecture NeuCube. NeuCube is not a general purpose machine but is still the first neuromorphic spatio/spectro-temporal data machine for learning, pattern recognition and understanding of spatio/spectro-temporal data. The chapter further presents the software/hardware implementation of the NeuCube as a development system for efficient applications on temporal or spatio/spectro-temporal across domain areas, including: brain data (EEG, fMRI), brain computer interfaces, robot control, multi-sensory data modelling, seismic stream data modelling and earthquake prediction, financial time series forecasting, climate data modelling and personalised, on-line risk of stroke prediction, and others. A limited version of the NeuCube software implementation is available from http://www.kedri.aut.ac.nz/neucube/.