Jeff K. T. Tang

A Virtual Reality Dance Training System Using Motion Capture Technology

In this paper, a new dance training system based on the motion capture and virtual reality (VR) technologies is proposed. Our system is inspired by the traditional way to learn new movements-imitating the teachers movements and listening to the teachers feedback. A prototype of our proposed system is implemented, in which a student can imitate the motion demonstrated by a virtual teacher projected on the wall screen. Meanwhile, the students motions will be captured and analyzed by the system based on which feedback is given back to them. The result of user studies showed that our system can successfully guide students to improve their skills. The subjects agreed that the system is interesting and can motivate them to learn.

Retrieval of logically relevant 3D human motions by Adaptive Feature Selection with Graded Relevance Feedback

A system that can retrieve logically relevant 3D captured motions is useful in game and animation production. We presented a robust logical relevance metric based on the relative distances among the joints. Existing methods select a universal subset of features for all kinds of queries which may not well characterize the variations in different queries. To break through this limitation we proposed an Adaptive Feature Selection (AFS) method that abstracts the characteristics of the query by a Linear Regression Model, and different feature subsets can be selected according to the properties of the specific query. With a Graded Relevance Feedback (GRF) algorithm, we refined the feature subset that enhances the retrieval performance according to the graded relevance of the feedback samples. With an ontology that predefines the logical relevance between motion classes in terms of graded relevance, the performance of the proposed AFS-GRF algorithm is evaluated and shown to outperform other class-specific feature selection and motion retrieval methods.

Interactive dancing game with real-time recognition of continuous dance moves from 3D human motion capture

We have implemented an interactive dancing game using optical 3D motion capture technology. We propose a Progressive Block Matching algorithm to recognize the dance moves performed by the player in real-time. This makes a virtual partner be able to recognize and respond to the players movement without a noticeable delay. The completion progress of a move is tracked progressively and the virtual partners move is rendered in synchronization with the players current action. Our interactive dancing game contains moves with various difficulty levels that suits for both novices and skillful players. Through animating the virtual partner in response to the players movements, the player gets immersed into the virtual environment. A user test is performed to have a subjective evaluation of our game and the feedbacks from the subjects are positive.

Interaction Retrieval by Spacetime Proximity Graphs

In this paper, we propose a new method to index and retrieve animation scenes in which multiple characters closely interact with one another. Such a technique can be an important tool for animators when they want to automatically extract the desired scene from a large database of animation sequence. Existing methods for single character movements do not scale well for multiple characters as they do not take into account the interaction of different body parts. In this paper, we propose a new distance function that computes the similarity of two‐character interations using the spatial relationship of the body parts. For each interaction, we produce a time‐varying graph structure based on the proximity of different joints, and compute the similarity of interactions by comparing the topology and Laplacian coordinates of the time‐varying graph. Experimental results show that the proposed method outperforms previous methods which are based on the kinematics of individual characters. The top retrieved samples are found similar in high level semantics while containing style variations.

A Big Data Framework for Early Identification of Dropout Students in MOOC

Massive Open Online Courses (MOOC) became popular and they posted great impact to education. Students could enroll and attend any MOOC anytime and anywhere according to their interest, schedule and learning pace. However, the dropout rate of MOOC was known to be very high in practice. It is desirable to discover students who have high chance to dropout in MOOC in early stage, and the course leader could impose intervention immediately in order to reduce the dropout rate. In this paper, we proposed a framework that applies big data methods to identify the students who are likely to dropout in MOOC. Real-world data were collected for the evaluation of our proposed framework. The results demonstrated that our framework is effective and helpful.

AR interior designer: Automatic furniture arrangement using spatial and functional relationships

In big cities the population is increasing that leads to smaller flats. It is hard to pick a furniture that perfectly suits to the available space at home. Therefore it requires people to measure the available space by hands, which is inefficient. In this paper, we propose an Augmented Reality (AR) interior design tool. The furniture reseller can make virtual 3D furniture available, and then via AR the virtual furniture are rendered in the real scene of our home space. A depth camera is used to measure the size of available space (in terms of supporting surfaces) for new furniture, such as the floor and desk/table surfaces. By the meantime, the obstacles are detected and marked unavailable for furniture placement. In a glance, the user can quickly determine whether that furniture is suitable or fit enough to the available space. Sometimes, the user may buy two or more furniture. Our another contribution is recommending an optimum arrangement of the furniture by determining the spatial and functional relationship between different pieces of furniture. Some challenging cases of furniture placement are shown.

Emerging Human-Toy Interaction Techniques with Augmented and Mixed Reality

In this book chapter, we will review the emerging technologies that promote extensive interactions between toys and their players. Cutting-edge display technologies have taken a major role in human-toy interaction. In particular, reality-virtuality technologies such as Augmented Reality (AR) and Mixed Reality (MR) have been adopted in digital entertainment and physical toys. With advances of portable and wearable devices, reality-virtuality technologies have become more popular and immersed into our daily life. Various interaction techniques are identified such as depth sensors and haptic devices. We will introduce example technologies, devices and products of the above technologies. However, displayed/projected virtual objects cannot give the user a sense of touch. Additional apparatus such as haptic styli are necessary to interact with the virtual objects. We foresee that in the near future, more kinds of virtual senses (such as taste) could be simulated and become part of the toy.

Synthesizing Two-character Interactions by Merging Captured Interaction Samples with their Spacetime Relationships

Existing synthesis methods for closely interacting virtual characters relied on user-specified constraints such as the reaching positions and the distance between body parts. In this paper, we present a novel method for synthesizing new interacting motion by composing two existing interacting motion samples without the need to specify the constraints manually. Our method automatically detects the type of interactions contained in the inputs and determines a suitable timing for the interaction composition by analyzing the spacetime relationships of the input characters. To preserve the features of the inputs in the synthesized interaction, the two inputs will be aligned and normalized according to the relative distance and orientation of the characters from the inputs. With a linear optimization method, the output is the optimal solution to preserve the close interaction of two characters and the local details of individual character behavior. The output animations demonstrated that our method is able to create interactions of new styles that combine the characteristics of the original inputs.

Enterprise Content Management System for IT Training

Information Technology IT training is undergoing rapid changes around the world, creating inter-institutional partnerships between IT training centers as well as software and hardware vendors to promote new products and training to users. Online training has resulted in an increasing enrolment of a wider spectrum of learners with more diverse technical skills and needs that must be served. Enterprise content management ECM enables IT training centers to deliver education beyond company walls, supporting a range of tools to improve information dissemination, communication, and creating collaborative virtual classrooms. In this article, the authors propose a learning platform with both Web 2.0 and mobile interfaces that supports IT training for nourishing busy workers with the latest IT technology knowledge and skills. Alerts are used as a mechanism to facilitate communications among students, instructors, and administrators. It also reminds students to keep track on their learning progress. The content flow management such as content editing, content approval, use of content, perform training exercise, practical tests, and examination as well as external access control for the Web portal can be easily controlled through the e-platform. Besides, the authors also demonstrated how to systematically specify document classifications and tagging of training materials by using the modular design of an integrated IT Training Enterprise Content Management System ITTECMS together with ontologies from the Semantic Web. Watermarking technology will be deployed for further protection of the training materials, both online and offline. It also allows the flexibility to create match-making of training content according to user interests. Not only users can retrieve matched contents effectively, vendors can also obtain multi-dimensional analysis of user problems and comments to form a comprehensive knowledgebase and achieve cross-sale of education products to target customers.

The Development of a Final Year Project Management System for Information Technology Programmes

This project tries to develop an online platform which facilitates the final year projects (FYP) process implemented by our information technology programme. The whole FYP is a year-long process involving groups of students and their supervisors to accomplish a theme based project. It is very necessary to employ the latest technologies in order to allow different parties to contribute and communicate more efficiently. Therefore, we have designed and developed a comprehensive web-based system to better support the three kinds of users; they are the FYP programme organizer (PO), project supervisors and the project group members.