Sunjae Lee

Towards unobtrusive emotion recognition for affective social communication

Awareness of the emotion of those who communicate with others is a fundamental challenge in building affective intelligent systems. Emotion is a complex state of the mind influenced by external events, physiological changes, or relationships with others. Because emotions can represent a users internal context or intention, researchers suggested various methods to measure the users emotions from analysis of physiological signals, facial expressions, or voice. However, existing methods have practical limitations to be used with consumer devices, such as smartphones; they may cause inconvenience to users and require special equipment such as a skin conductance sensor. Our approach is to recognize emotions of the user by inconspicuously collecting and analyzing user-generated data from different types of sensors on the smartphone. To achieve this, we adopted a machine learning approach to gather, analyze and classify device usage patterns, and developed a social network service client for Android smartphones which unobtrusively find various behavioral patterns and the current context of users. Also, we conducted a pilot study to gather real-world data which imply various behaviors and situations of a participant in her/his everyday life. From these data, we extracted 10 features and applied them to build a Bayesian Network classifier for emotion recognition. Experimental results show that our system can classify user emotions into 7 classes such as happiness, surprise, anger, disgust, sadness, fear, and neutral with a surprisingly high accuracy. The proposed system applied to a smartphone demonstrated the feasibility of an unobtrusive emotion recognition approach and a user scenario for emotion-oriented social communication between users.

An ontology-based reasoning approach towards energy-aware smart homes

We present an ontology-based reasoning approach for saving energy in a smart home setting where a mobile phone can serve as a generic sensor which can collect the inhabitants contextual data. The paper details an ontology that describes the smart home domain and a prototype to test the system. Finally, we conclude with lessons learned from our work in developing an energy-aware smart home prototype and suggestions for future work.

Smart pose: mobile posture-aware system for lowering physical health risk of smartphone users

With the widespread use of smartphones, users tend to use their smartphones for a long period of time with unhealthy postures, bending forward their upper body including the neck. If users keep such an unhealthy posture for a long time, their neck and back muscles get chronically strained, which might cause diseases such as cervical myalgia. To prevent these diseases, we propose a new methodology to monitor the posture of smartphone users with built-in sensors. The proposed mechanism estimates a value representing user postures like head/neck tilt angle by analyzing sensor data from a front-faced camera, 3-axis accelerometer, and orientation sensor. It then informs the user if the estimated value is maintained within the abnormal range over a pre-defined time.

A new posture monitoring system for preventing physical illness of smartphone users

With the widespread use of smartphones, users tend to use their smartphone for a long period of time in unhealthy postures; bending forward the neck and watching the relatively small screen closely with concentration. If users keep such unhealthy postures for a long time, they are susceptible to musculoskeletal disorders and eye problems such as cervical disc and myopia, respectively. To prevent users from having these diseases, we propose a new methodology to monitor the posture of smartphone users with built-in sensors. The proposed mechanism estimates various values representing user postures like the tilt angle of the neck, viewing distance, and gaze condition of the user, by analyzing sensor data from a front-faced camera, 3-axis accelerometer, orientation sensor, or any combination thereof, and warns the user if estimated values are maintained within the abnormal range over the pre-defined time. As a proof of concept, we developed an Android application named Smart Pose which estimates the users neck tilt angle from analysis of the facial image, shakiness and tilt angle of the smartphone, and then notifies the user when her/his neck tilt angle is maintained in an unusual range during the smartphone operation. Also, we validated the result of our system by the comparison with measurements from 3D posture imaging equipment in our research facility. Via the proposed mechanism, a participant was able to be aware of his unhealthy postures, and then try to correct them.

Mobile posture monitoring system to prevent physical health risk of smartphone users

With the widespread use of a smartphone, users tend to use their smartphone for a long period of time in unhealthy postures; bending forward the neck and watching the relatively small screen closely with concentration. If users keep such unhealthy postures for a long time, they are susceptible to musculoskeletal disorders and eye problems such as cervical disc and myopia, respectively. To prevent users from having these diseases, we propose a new methodology to monitor the posture of smartphone users with built-in sensors. The proposed mechanism estimates various values representing user postures like the tilt angle of the neck, viewing distance, and gaze condition of the user, by analyzing sensor data from a front-faced camera, 3-axis accelerometer, orientation sensor, or any combination thereof, and warns the user if estimated values are maintained within the abnormal range over the allowed time. Via the proposed mechanism, users are able to be aware of their unhealthy postures, and then try to correct their postures.