Aparajithan Sivanathan

Brain-computer interface game applications for combined neurofeedback and biofeedback treatment for children on the autism spectrum

Individuals with autism spectrum disorder (ASD) show deficits in social and communicative skills, including imitation, empathy, and shared attention, as well as restricted interests and repetitive patterns of behaviors. Evidence for and against the idea that dysfunctions in the mirror neuron system are involved in imitation and could be one underlying cause for ASD is discussed in this review. Neurofeedback interventions have reduced symptoms in children with ASD by self-regulation of brain rhythms. However, cortical deficiencies are not the only cause of these symptoms. Peripheral physiological activity, such as the heart rate and its variability, is closely linked to neurophysiological signals and associated with social engagement. Therefore, a combined approach targeting the interplay between brain, body, and behavior could be more effective. Brain–computer interface applications for combined neurofeedback and biofeedback treatment for children with ASD are currently nonexistent. To facilitate their use, we have designed an innovative game that includes social interactions and provides neural- and body-based feedback that corresponds directly to the underlying significance of the trained signals as well as to the behavior that is reinforced.

Musculoskeletal disorders in construction: A review and a novel system for activity tracking with body area network

Human body motions have been analysed for decades with a view on enhancing occupational well-being and performance of workers. On-going progresses in miniaturised wearable sensors are set to revolutionise biomechanical analysis by providing accurate and real-time quantitative motion data. The construction industry has a poor record of occupational health, in particular with regard to work-related musculoskeletal disorders (WMSDs). In this article, we therefore focus on the study of human body motions that could cause WMSDs in construction-related activities. We first present an in-depth review of existing assessment frameworks used in practice for the evaluation of human body motion. Subsequently different methods for measuring working postures and motions are reviewed and compared, pointing out the technological developments, limitations and gaps; Inertial Measurement Units (IMUs) are particularly investigated. Finally, we introduce a new system to detect and characterise unsafe postures of construction workers based on the measurement of motion data from wearable wireless IMUs integrated in a body area network. The potential of this system is demonstrated through experiments conducts in a laboratory as well as in a college with actual construction trade trainees.

The application of ubiquitous multimodal synchronous data capture in CAD

Design is an interactive and iterative process where the designers skills and knowledge are fused with emotive rationales aided by design tools. A design solution is thus influenced by the designers creativity, experience and emotional perception. Consequently, there is a need within computer aided design (CAD) research for ubiquitous tools to capture the affective states of engineers during design activities to further understand the product design process.This paper proposes a generic framework for ubiquitous multimodal synchronous data capture, based around the capture of CAD system activities, to monitor and log a variety of inputs, interactions, biophysical data and design solutions with a view to providing meta and chronological performance data for post design task analysis. The framework has been employed in two use cases namely, a CAD station activity and a collaborative design review. The results of these trials validated the architecture and use of the ubiquitous data capture approach demonstrating the practical application of time-phased data capture, analysis and the subsequent output of metadata in CAD environments providing a new perspective on, and a new way of investigating CAD-based design activities.This research also extrapolates the frameworks usefulness into future CAD and PLM systems by arguing why and how they need to adopt such ubiquitous platforms. It also subjectively points to potential opportunities and issues that might arise when implementing the ubiquitous multimodal metadata architecture in a real-life environment. A generic ubiquitous data capture framework is demonstrated via two case studies.Variety of inputs, interactions, biophysical data and design solutions are captured.Tight temporal synchronisation with commodity data logging tools is achieved.Demonstrates engineering knowledge capture linking CAD and PLM via generated metadata.The frameworks use in future CAD and PLM systems is extrapolated.

Temporal Synchronisation of Data Logging in Racing Gameplay

Abstract Contemporary game play is a complex environment where user interacts with many elements. Data logging in a game play is a common practice to study a players in-game behaviour. Identified behavioural knowledge is then applied to enhance the game elements. Analyzing the user behaviour is a multivariate process and therefore requires more information than mere logging of game context. Multi modal data channels such as biofeedback signals are increasingly used to study the game play so that more detailed understanding of the user behaviour can be established. Precise synchronisation in capturing of multiple streams is essential to produce accurate and meaningful information. This paper presents a generic technique to accurately synchronize multiple data streams captured during a gaming session. It is demonstrated by applying it to a driving game Simulation. Observable correlation between the in-game data and psycho-physiological signals are presented to demonstrate the accuracy and granularity of the synchronisation.

Temporal multimodal data synchronisation for the analysis of a game driving task using EEG

Abstract Multimodal data channels such as bio-physiological signals are increasingly used in game-play studies to better understand players’ behaviours and their motivations. It is however difficult to perform any sort of conclusive analysis solely based on bio-physiological signals due to the complex nature of epistemic, semiotic and ergotic activities surrounding in-game activities and the artefacts facilitating player immersion. Thus a combined analysis of multiple data streams including in-game data and bio-physiological signals is indispensable to produce contextualised information from which a deep analysis of game mechanics and their effects can be performed. Precise synchronisation in capturing multiple streams is required to generate valid inter-stream correlations and meaningful information. Typically there are no automatic mechanisms built in the game architecture or in commercial data logging systems for multimodal data synchronisation and data fusion. This paper presents a novel and generic technique based on inducing identifiable signature pulses in data channels to accurately synchronise multiple temporal data streams. This technique is applied and its capabilities are exhibited using a driving game simulation as an exemplar. In this example, driver’s in-game behavioural data is synchronised and correlated with their temporal brain activity. The concept of simplex method borrowed from linear programming is used to correlate between the driving patterns and brain activity in this initial study is provided so as to allow studying/investigating user behaviour in relation to learning of the driving track.

Towards a Cyber-Physical Gaming System for Training in the Construction and Engineering Industry

Introducing serious gaming systems (SGS) has the potential to enhance trainee experience and performance across the construction industry and its supply chain, such as mechanical engineering services. SGS as an ‘enabler’ in architectural engineering has received limited research in its role to assess and enhance the performance of its workforce. In a personnel high-risk environment, improving training standards to eliminate or reduce health and safety risks, in addition to providing an understanding of workers’ ergonomics, ensures sustainability of both the project and its workforce.This paper presents an activity tracking and feedback system that captures the physical activity of a construction worker climbing a ladder. Climbing is captured with a 3D motion capture system and processed in real-time to identify potential areas of underperformance. A simple and representative scoring method was established as a reporting method (game statistics) for giving feedback about the correctness of the activity. It can nonetheless be tuned to characterise and adjust to various complexity levels in-line with the required training standards. Furthermore, the motion data and feedback information are fed into a virtual gaming environment enabling the real-time visualisation of the trainee’s motion and experiential learning of the performance through visual and audio feedback. The gaming concepts are employed here with multiple purposes, particularly for accelerating and facilitating the learning process of the trainee. In addition to the 3D motion capturing system, this paper outlines and tests a proposed serious cyber-physical gaming system that incorporates wearable technologies that has the potential to support both construction training and practice.Copyright

THE EVALUATION OF A VIRTUAL-AIDED DESIGN ENGINEERING REVIEW (VADER) SYSTEM FOR AUTOMATED KNOWLEDGE CAPTURE AND REUSE

Conducting knowledge capture and embedding it into a products’ through lifecycle remains a key issue in engineering industries; particularly with regard to rationale associated knowledge emanating during formal design reviews. Manual, and often interruptive, methods with associated costly overheads, exacerbate the already time consuming process. As well as these disadvantages, manual methods can potentially capture the wrong data due to human error or not fullycapturing all the pertinent information and associated relationships. Consequently, industries are seeking automated engineering knowledge capture and rationale that adds value to product and processes, potentially reaping the benefits of time and cost. Previous work by the authors proved how userlogging in virtual environments aid unobtrusive capture of engineering knowledge and rationale in design tasks. This paper advances the work further through a Virtual Aided Design Engineering Review (VADER) system developed to automatically and unobtrusively capture both multimodal human-computer and human-human interactivity during design reviews via the synchronous time-phased logging of software interactions, product models, audio, video and input devices. By processing the captured data review reports and records can be automatically generated as well as allowing fast knowledge retrieval. The backbone of VADER is a multimodal device and data fusion architecture to capture and synchronise structured and unstructured data in realtime. Visualisation is through a 3D virtual environment. In addition to allowing engineers to visualise and annotate 3D design models, the system provides a timeline interface to search and visualise the captured decisions from a design review. The VADER system has been put through its initial industrial trial and reported herein. Objective and subjective analysis indicate the VADER system is intuitive to use and can lead to savings in both time and cost with regard to project reviews.

A novel design engineering review system with searchable content: knowledge engineering via real-time multimodal recording

ABSTRACT Cradle to grave product support has been a key issue in the engineering sector over many years, particularly because product engineering legacy knowledge is often lost during the product development process unless rigorously captured in some way. This is particularly the case during formal design reviews at any point during a products lifecycle where engineering changes are not fully documented or where salient but important aspects of decision making are difficult to document explicitly. Though many software systems are available to support design reviews, they have not necessarily met the expectations of industry. Consequently, traditional knowledge capture methods tend to be time-consuming, costly and disruptive leading to many companies simply giving up on this crucial aspect of product development. This paper presents research carried out with regard to prototyping and testing a potential knowledge engineering capture and reuse solution, demonstrating real-time user-logging using virtual design environments focused on team-based design ‘reviews’. Called the Virtual Aided Design Engineering Review (VADER) system, it provides millisecond precision time-phased knowledge capture in an automatic and unobtrusive manner. Both structured and unstructured data are synthesised via a ubiquitous integration and temporal synchronisation (UbiITS) framework that enables interactive information mapping, retrieval and mining. VADERs frontend includes a virtual reality based 3D model view display as a multiuser collaborative interface and an auxiliary web interface for concurrent access by multiple distributed users during product design discussions. Feedback from engineers using the system demonstrated that this concept is one which believe would substantially enhance their engineering task knowledge capture, rapid retrieval and reuse capability. It was also surmised that, if required, such a system can be extended throughout the whole product development process capturing individual and team-based engineers’ inputs across the whole cradle-to-grave product life cycle. Also, due to its generic nature, this approach is not limited only to engineering applications or virtual environments but can potentially be used in other sectors using computer-based technologies of any kind.

Lecture Notes in Computer Science: Beyond simulators, Using F1 Games to Predict Driver Performance, Learning and Potential

Formula One (F1) drivers are amongst the most highly skilled drivers in the world, but not every F1 driver is destined to be a F1 World Champion. Discovering new talent or refreshing strategies are long-term investments for all competitive F1 teams. The F1 world and teams invest vast amounts in developing high-fidelity simulators; however, driving games have seldom been associated with uncovering certain natural abilities. Beyond nature and nurture to attain success at the top level, certain motor-cognitive aspects are paramount for proficiency. One method of potentially finding talent is studying the behavioral and cognitive patterns associated with learning. Here, an F1 simulation game was used to demonstrate how learning had taken place. The indicative change of interest is from cognitive to motor via more skilled autonomous driving style –a skill synonymous with expert driving and ultimately winning races. Our data show clear patterns of how this skill develops.