Yomna Abdelrahman

Exploiting thermal reflection for interactive systems

Thermal cameras have recently drawn the attention of HCI researchers as a new sensory system enabling novel interactive systems. They are robust to illumination changes and make it easy to separate human bodies from the image background. Far-infrared radiation, however, has another characteristic that distinguishes thermal cameras from their RGB or depth counterparts, namely thermal reflection. Common surfaces reflect thermal radiation differently than visual light and can be perfect thermal mirrors. In this paper, we show that through thermal reflection, thermal cameras can sense the space beyond their direct field-of-view. A thermal camera can sense areas besides and even behind its field-of-view through thermal reflection. We investigate how thermal reflection can increase the interaction space of projected surfaces using camera-projection systems. We moreover discuss the reflection characteristics of common surfaces in our vicinity in both the visual and thermal radiation bands. Using a proof-of-concept prototype, we demonstrate the increased interaction space for hand-held camera-projection system. Furthermore, we depict a number of promising application examples that can benefit from the thermal reflection characteristics of surfaces.

Stay Cool! Understanding Thermal Attacks on Mobile-based User Authentication

PINs and patterns remain among the most widely used knowledge-based authentication schemes. As thermal cameras become ubiquitous and affordable, we foresee a new form of threat to user privacy on mobile devices. Thermal cameras allow performing thermal attacks, where heat traces, resulting from authentication, can be used to reconstruct passwords. In this work we investigate in details the viability of exploiting thermal imaging to infer PINs and patterns on mobile devices. We present a study (N=18) where we evaluated how properties of PINs and patterns influence their thermal attacks resistance. We found that thermal attacks are indeed viable on mobile devices; overlapping patterns significantly decrease successful thermal attack rate from 100% to 16.67%, while PINs remain vulnerable (>72% success rate) even with duplicate digits. We conclude by recommendations for users and designers of authentication schemes on how to resist thermal attacks.

Investigation of Material Properties for Thermal Imaging-Based Interaction

Recent work demonstrated the exciting opportunities that thermal imaging offers for the development of interactive systems. It was shown that a thermal camera can sense when a user touches a surface, performs gestures in the cameras direct field of view and, in addition, performs gestures outside the cameras direct field of view through thermal reflection. In this paper, we investigate the material properties that should be considered for detecting interaction using thermal imaging considering both in- and outdoor settings. We conducted a study to analyze the recognition performance for different gestures and different surfaces. Using the results, we derive guidelines on material properties of surfaces for detecting on-surface as well as mid-air interaction using a thermal camera. We discuss the constrains that should be taken into account using thermal imaging as the sensing technology. Finally, we present a material space based on our findings. The space depicts surfaces and the required properties that enable the different interaction techniques.

Cognitive Heat: Exploring the Usage of Thermal Imaging to Unobtrusively Estimate Cognitive Load

Current digital systems are largely blind to users’ cognitive states. Systems that adapt to users’ states show great potential for augmenting cognition and for creating novel user experiences. However, most approaches for sensing cognitive states, and cognitive load specifically, involve obtrusive technologies, such as physiological sensors attached to users’ bodies. This paper present an unobtrusive indicator of the users’ cognitive load based on thermal imaging that is applicable in real-world. We use a commercial thermal camera to monitor a person’s forehead and nose temperature changes to estimate their cognitive load. To assess the effect of different levels of cognitive load on facial temperature we conducted a user study with 12 participants. The study showed that different levels of the Stroop test and the complexity of reading texts affect facial temperature patterns, thereby giving a measure of cognitive load. To validate the feasibility for real-time assessments of cognitive load, we conducted a second study with 24 participants, we analyzed the temporal latency of temperature changes. Our system detected temperature changes with an average latency of 0.7 seconds after users were exposed to a stimulus, outperforming latency in related work that used other thermal imaging techniques. We provide empirical evidence showing how to unobtrusively detect changes in cognitive load in real-time. Our exploration of exposing users to different content types gives rise to thermal-based activity tracking, which facilitates new applications in the field of cognition-aware computing.

Effects of camera position and media type on lifelogging images

With an increasing number of new camera devices entering the market, lifelogging has turned into a viable everyday practice. The promise of comprehensively capturing our lifes happenings has caused adoption rates to grow, but approaches to do so greatly differ. In this paper we evaluate existing visual lifelogging capture approaches through a user study with two main capture dimensions: (1) comparing the body position where a lifelogging camera is worn: head versus chest (2) comparing the media captures: video versus stills. We equipped 30 participants with cameras on their heads and chests. That data was evaluated by subjective user ratings as well as by objective image processing analysis. Our findings indicate that (1) chest-worn devices are more stable and contain less motion blur through which feature detection by image processing algorithms works better than from head-worn cameras; 2) head-worn video cameras, however, seem to be the better choice for lifelogging as they capture more important autobiographical cues than chest-worn devices, e.g., faces that have been shown to be most relevant for recall.

Snake view: exploring thermal imaging as a vision extender in mountains

Humans vision can only operate in the limited visible band of the electromagnetic spectrum. Using commercially available imaging sensors can be beneficial to extend humans visual perception in different environments. Typically, these environments include challenging conditions, for instance smoky views during a fire or occluded, foggy, cloudy and windy view in mountain environments. Recently, thermal imaging became more commercially available, which makes utilizing it to extend the humans visual perception affordable and deployable. In this paper, we propose the usage of thermal imaging as a vision extension tool. Two initial prototypes are presented depicting the different form factors of thermal cameras attachment to Head Mounted Displays. Finally, we discuss potential use case of extending the humans vision to cover the thermal spectrum during mountains activities.

One size does not fit all: challenges of providing interactive worker assistance in industrial settings

Teaching new assembly instructions at manual assembly workplaces has evolved from human supervision to digitized automatic assistance. Assistive systems provide dynamic support, adapt to the user needs, and alleviate perceived workload from expert workers supporting freshman workers. New assembly instructions can be implemented at a fast pace. These assistive systems decrease the cognitive workload of workers as they need to memorize new assembly instructions with each change of product lines. However, the design of assistive systems for the industry is a challenging task. Once deployed, people have to work with such systems for full workdays. From experiences made during our past project motionEAP, we report on design challenges for interactive worker assistance at manual assembly workplaces as well as challenges encountered when deploying interactive assistive systems for diverse user populations.

See through the fire: evaluating the augmentation of visual perception of firefighters using depth and thermal cameras

Our visual perception is limited to the abilities of our eyes, where we only perceive visible light. This limitation might influence how we perceive and react to our surroundings, however, this limitation might endanger us in certain scenarios e.g. firefighting. In this paper, we explore the potential of augmenting the visual sensing of the firefighters using depth and thermal imaging to increase their awareness about the environment. Additionally, we built and evaluated two form factors, hand held and head mounted display. To evaluate our built prototypes, we conducted two user studies in a simulated fire environment with real firefighters. In this workshop paper, we present our findings from the evaluation of the concept and prototypes with real firefighters.

Proxemic zones of exhibits and their manipulation using floor projection

We investigate intimate proxemic zones of exhibits, which are zones that visitor will respect and hence not enter. In a first experiment we show that exhibits have, like humans, an intimate proxemic zone that is respected by exhibition visitors and which is 27cm. A zone around an exhibit that visitors will not enter can be increased through lines projected on the floor. In a second experiment we showed that for distances larger than 95cm, static lines will not protect the intimate zone of the exhibit anymore, but animated or color changing lines will do and allow for dynamically changing the zone size. This paper provides fundamental insight in intimate proxemic zones of exhibits and about how we can use interactive floor projections to manipulate these zones in the context of exhibitions. That might be used, for example, to increase the safety zone of exhibits if the room is crowded.

Super Alpha: Arabic Alphabet Learning Serious Game for Children with Learning Disabilities

Children with learning disabilities need dedicated attention to cope with their peers in class. One way to help is using technological assistive learning means as an extra gate to learn different concepts. This study describes the findings of comparative experimental research that questions the effect of using serious games on these children’s learning process. The main focus is the design, implementation, and evaluation of an educational platform with different games that are picked based on existing special teaching strategies. It is created to question whether serious games can facilitate the work of the professionals or not by comparing it with a normal instructional method, in the context of an educational activity, which is learning the Arabic alphabet.