Youngjun Cho

A new subminiature impact actuator for mobile devices

This paper presents a new subminiature impact type actuator which creates the haptic sensation in a mobile device. Recently, linear resonance actuators (LRA) are widely used. One of the key differences between an LRA and a traditional eccentric motor is that the LRA generates the vibrotactile sensation through resonance for minimizing response time. The strategy of operation near the resonant frequency, however, brought a new issue for creating vibrotactile sensation which can be strong enough to feel in arbitrary frequency. In order to achieve this issue, we adopt an unstable structure to amplify impact force and to reduce response rate. Due to the impact with fast rising and falling time in the proposed impact actuator, the available strong impact vibration is created over wide frequency range from 0Hz to 100Hz. The impact vibration generated from the proposed actuator is suitable for reproducing realistic button sensation and creating various vibration patterns in mobile devices.

Mechatronics Technology in Mobile Devices

Recently, haptics research has grown into an interdisciplinary field, covering perception, psychophysics, virtual reality, mechanism design, and control. Haptics research is considered to have originated from teleoperator systems. In these initial explorations, increasing the transparency level of the mechanical master/slave manipulator system was the main issue. As computer graphics technology has emerged to realize a wide range of virtual reality applications, the development of control technologies and haptic master device designs have adapted to focus mainly on interaction with virtual environments. Numerous breakthroughs in visual, sound, and haptic modeling technologies enable the real-time display of contact with virtual objects, including capabilities such as shape deformation and reactive force. There have been attempts to model and display the fine details of touched surfaces to enhance virtual presence. Research in neuroscience and psychophysics has led to discoveries in the human perceptual processes underlying haptic sensations. Drawing on this understanding, researchers have begun to examine efficient methods of building tactile display units that are capable of rendering feelings of roughness, softness, and temperature.

Robust tracking of respiratory rate in high-dynamic range scenes using mobile thermal imaging

The ability to monitor the respiratory rate, one of the vital signs, is extremely important for the medical treatment, healthcare and fitness sectors. In many situations, mobile methods, which allow users to undertake everyday activities, are required. However, current monitoring systems can be obtrusive, requiring users to wear respiration belts or nasal probes. Alternatively, contactless digital image sensor based remote-photoplethysmography (PPG) can be used. However, remote PPG requires an ambient source of light, and does not work properly in dark places or under varying lighting conditions. Recent advances in thermographic systems have shrunk their size, weight and cost, to the point where it is possible to create smart-phone based respiration rate monitoring devices that are not affected by lighting conditions. However, mobile thermal imaging is challenged in scenes with high thermal dynamic ranges (e.g. due to the different environmental temperature distributions indoors and outdoors). This challenge is further amplified by general problems such as motion artifacts and low spatial resolution, leading to unreliable breathing signals. In this paper, we propose a novel and robust approach for respiration tracking which compensates for the negative effects of variations in the ambient temperature and motion artifacts and can accurately extract breathing rates in highly dynamic thermal scenes. The approach is based on tracking the nostril of the user and using local temperature variations to infer inhalation and exhalation cycles. It has three main contributions. The first is a novel Optimal Quantization technique which adaptively constructs a color mapping of absolute temperature to improve segmentation, classification and tracking. The second is the Thermal Gradient Flow method that computes thermal gradient magnitude maps to enhance the accuracy of the nostril region tracking. Finally, we introduce the Thermal Voxel method to increase the reliability of the captured respiration signals compared to the traditional averaging method. We demonstrate the extreme robustness of our system to track the nostril-region and measure the respiratory rate by evaluating it during controlled respiration exercises in high thermal dynamic scenes (e.g. strong correlation (r = 0.9987) with the ground truth from the respiration-belt sensor). We also demonstrate how our algorithm outperformed standard algorithms in settings with different amounts of environmental thermal changes and human motion. We open the tracked ROI sequences of the datasets collected for these studies (i.e. under both controlled and unconstrained real-world settings) to the community to foster work in this area.

Deep Thermal Imaging: Proximate Material Type Recognition in the Wild through Deep Learning of Spatial Surface Temperature Patterns

We introduce Deep Thermal Imaging, a new approach for close-range automatic recognition of materials to enhance the understanding of people and ubiquitous technologies of their proximal environment. Our approach uses a low-cost mobile thermal camera integrated into a smartphone to capture thermal textures. A deep neural network classifies these textures into material types. This approach works effectively without the need for ambient light sources or direct contact with materials. Furthermore, the use of a deep learning network removes the need to handcraft the set of features for different materials. We evaluated the performance of the system by training it to recognize 32 material types in both indoor and outdoor environments. Our approach produced recognition accuracies above 98% in 14,860 images of 15 indoor materials and above 89% in 26,584 images of 17 outdoor materials. We conclude by discussing its potentials for real-time use in HCI applications and future directions.

Instant Automated Inference of Perceived Mental Stress through Smartphone PPG and Thermal Imaging

Background A smartphone is a promising tool for daily cardiovascular measurement and mental stress monitoring. Photoplethysmography (PPG) and low-cost thermography can be used to create cheap, convenient and mobile systems. However, to achieve robustness, a person has to remain still for several minutes while a measurement is being taken. This is very cumbersome, and limits the usage in applications such producing instant measurements of stress. Objective We propose to use smartphone-based mobile PPG and thermal imaging to provide a fast binary measure of stress responses to an event using dynamical physiological changes which occur within 20 seconds of the event finishing. Methods We propose a system that uses a smartphone and its physiological sensors to reliably and continuously measure over a short window of time a person’s blood volume pulse, the time interval between heartbeats (R-R interval) and the 1D thermal signature of the nose tip. 17 healthy participants, involved in a series of stress-inducing mental activities, measured their physiological response to stress in the 20 second-window immediately following each activity. A 10-cm Visual Analogue Scale was used by them to self-report their level of mental stress. As a main labeling strategy, normalized K-means clustering is used to better treat interpersonal differences in ratings. By taking an array of the R-R intervals and thermal directionality as a low-level feature input, we mainly use an artificial neural network to enable the automatic feature learning and the machine learning inference process. To compare the automated inference performance, we also extracted widely used high level features from HRV (e.g., LF/HF ratio) and the thermal signature and input them to a k-nearest neighbor to infer perceived stress levels. Results First, we tested the physiological measurement reliability. The measured cardiac signals were considered highly reliable (signal goodness probability used, Mean=0.9584, SD=0.0151). The proposed 1D thermal signal processing algorithm effectively minimized the effect of respiratory cycles on detecting the apparent temperature of the nose tip (respiratory signal goodness probability Mean=0.8998 to Mean=0). Second, we tested the 20 seconds instant perceived stress inference performance. The best results were obtained by using automatic feature learning and classification using artificial neural networks rather than using pre-crafted features. The combination of both modalities produced higher accuracy on the binary classification task using 17-fold leave-one-subject-out (LOSO) cross-validation (accuracy: HRV+Thermal: 76.96%; HRV: 60.29%; Thermal: 61.37%). The results are comparable with the state of the art automatic stress recognition methods requiring long term measurements (a minimum of 2 minutes for up to around 80% accuracy from LOSO). Lastly, we explored the impact of different data labeling strategies used in the field on the sensitivity of our inference methods and the need for normalization within individual. Conclusions Results demonstrate the capability of smartphone biomedical imaging in instant mental stress recognition. Given that this approach does not require long measurements requiring attention and reduced mobility, it is more feasible for mobile mental healthcare solution in the wild.