Young Sang Choi

Towards unobtrusive emotion recognition for affective social communication

Awareness of the emotion of those who communicate with others is a fundamental challenge in building affective intelligent systems. Emotion is a complex state of the mind influenced by external events, physiological changes, or relationships with others. Because emotions can represent a users internal context or intention, researchers suggested various methods to measure the users emotions from analysis of physiological signals, facial expressions, or voice. However, existing methods have practical limitations to be used with consumer devices, such as smartphones; they may cause inconvenience to users and require special equipment such as a skin conductance sensor. Our approach is to recognize emotions of the user by inconspicuously collecting and analyzing user-generated data from different types of sensors on the smartphone. To achieve this, we adopted a machine learning approach to gather, analyze and classify device usage patterns, and developed a social network service client for Android smartphones which unobtrusively find various behavioral patterns and the current context of users. Also, we conducted a pilot study to gather real-world data which imply various behaviors and situations of a participant in her/his everyday life. From these data, we extracted 10 features and applied them to build a Bayesian Network classifier for emotion recognition. Experimental results show that our system can classify user emotions into 7 classes such as happiness, surprise, anger, disgust, sadness, fear, and neutral with a surprisingly high accuracy. The proposed system applied to a smartphone demonstrated the feasibility of an unobtrusive emotion recognition approach and a user scenario for emotion-oriented social communication between users.

An adaptive user interface based on spatiotemporal structure learning

We developed a user interface prototype for the Android smartphone which recommends a number of applications to best match the users context. To consider the users context of use, we utilized 5 prototypical variables; time, location, weather, emotion, and activities. The developed system derives the best three recommended applications based on the results of supervised learning from such data sets. To consider the history of past context information, in addition to the current one, we developed a novel and effective probabilistic learning and inference algorithm named “Spatiotemporal Structure Learning”. By extending Naïve Bayesian Classifier, the spatiotemporal structure learning can create a probability model which represents relationship between time-series contextual variables. We implemented a prototype system which shows the current context and the inferred recommendation of applications. For the prototype system, we developed an Android widget application for the user interface and a Java-based server application which learns structure from training data and provides inference results in real-time. To gather training data and evaluate proposed system, we conducted a pilot study which showed 69% of accuracy in predicting the users application usage. The prototype demonstrated the feasibility of an adaptive user interface applied to a state of the art smartphone. We also believe that the suggested spatiotemporal structure learning can be applied to number of application areas including healthcare or energy problems.

Smart pose: mobile posture-aware system for lowering physical health risk of smartphone users

With the widespread use of smartphones, users tend to use their smartphones for a long period of time with unhealthy postures, bending forward their upper body including the neck. If users keep such an unhealthy posture for a long time, their neck and back muscles get chronically strained, which might cause diseases such as cervical myalgia. To prevent these diseases, we propose a new methodology to monitor the posture of smartphone users with built-in sensors. The proposed mechanism estimates a value representing user postures like head/neck tilt angle by analyzing sensor data from a front-faced camera, 3-axis accelerometer, and orientation sensor. It then informs the user if the estimated value is maintained within the abnormal range over a pre-defined time.

A new posture monitoring system for preventing physical illness of smartphone users

With the widespread use of smartphones, users tend to use their smartphone for a long period of time in unhealthy postures; bending forward the neck and watching the relatively small screen closely with concentration. If users keep such unhealthy postures for a long time, they are susceptible to musculoskeletal disorders and eye problems such as cervical disc and myopia, respectively. To prevent users from having these diseases, we propose a new methodology to monitor the posture of smartphone users with built-in sensors. The proposed mechanism estimates various values representing user postures like the tilt angle of the neck, viewing distance, and gaze condition of the user, by analyzing sensor data from a front-faced camera, 3-axis accelerometer, orientation sensor, or any combination thereof, and warns the user if estimated values are maintained within the abnormal range over the pre-defined time. As a proof of concept, we developed an Android application named Smart Pose which estimates the users neck tilt angle from analysis of the facial image, shakiness and tilt angle of the smartphone, and then notifies the user when her/his neck tilt angle is maintained in an unusual range during the smartphone operation. Also, we validated the result of our system by the comparison with measurements from 3D posture imaging equipment in our research facility. Via the proposed mechanism, a participant was able to be aware of his unhealthy postures, and then try to correct them.

Mobile posture monitoring system to prevent physical health risk of smartphone users

With the widespread use of a smartphone, users tend to use their smartphone for a long period of time in unhealthy postures; bending forward the neck and watching the relatively small screen closely with concentration. If users keep such unhealthy postures for a long time, they are susceptible to musculoskeletal disorders and eye problems such as cervical disc and myopia, respectively. To prevent users from having these diseases, we propose a new methodology to monitor the posture of smartphone users with built-in sensors. The proposed mechanism estimates various values representing user postures like the tilt angle of the neck, viewing distance, and gaze condition of the user, by analyzing sensor data from a front-faced camera, 3-axis accelerometer, orientation sensor, or any combination thereof, and warns the user if estimated values are maintained within the abnormal range over the allowed time. Via the proposed mechanism, users are able to be aware of their unhealthy postures, and then try to correct their postures.

Eating activity recognition for health and wellness: A case study on Asian eating style

In this paper, we propose an unobtrusive a method of daily life monitoring with a triaxial accelerometer embedded in a wrist band. The method recognizes each operation of the activities such as picking up food with chopsticks or eating steamed rice with a spoon and classifies the activities. We believe this method will contribute to the field of innovative health-promoting consumer electronics, which enables consumers to easily and accurately manage their daily life, especially eating activities, and consequently encourage to maintain healthier lifestyle.