Rainhard Dieter Findling

Towards face unlock: on the difficulty of reliably detecting faces on mobile phones

Currently, reliable face detection and recognition are becoming more important on mobile devices -- e.g. to unlock the screen. However, using only frontal face images for authentication purposes can no longer be considered secure under the assumption of easy availability of frontal snapshots of the respective device owners from social networks or other media. In most current implementations, a sufficiently high-resolution face image displayed on another mobile device will be enough to circumvent security measures. In this paper, we analyze current methods to face detection and recognition regarding their usability in the mobile domain, and then propose an approach to a Face Unlock system on a smart phone intended to be more secure than current approaches while still being convenient to use: we use both frontal and profile face information available during a pan shot around the users head, by combining camera images and movement sensor data. Current results to face detection are promising, but reliable face recognition needs further research.

Diversity in locked and unlocked mobile device usage

We analyze locked and unlocked mobile device usage of 1 960 Android smartphones. Based on approximately 10TB of mobile device data logs collected by the Device Analyzer project, we derive 6.9 million usage sessions using a screen power state machine based approach. From these session we examine the number of interactions per day, the average interaction duration as well as the total daily device usage time. Findings indicate that on average users interact with their devices 117 minutes a day, separated over 57 interactions -- while unlocking their device only 43% of the time (e. g. to check for notifications).

Mobile Device Usage Characteristics: The Effect of Context and Form Factor on Locked and Unlocked Usage

Smartphones and tablets are an indispensable part of modern communication and people spend considerable time interacting with their devices every day. While substantial research has been conducted concerning smartphone usage, little is known about how tablets are used. This paper studies mobile device usage characteristics like session length, interaction frequency, and daily usage in locked and unlocked state with respect to location context. Based on logs from 1,585 Android devices (470 years of total usage time), we derive and analyze 23 million usage sessions. We found that devices remain locked for 60% of the interactions and usage at home occurs twice as frequent as at work. With an average of 58 interactions per day, smartphones are used twice as often as tablets, while tablet sessions are 2.5 times longer, resulting in almost equal aggregated daily usage. We conclude that usage session characteristics differ considerably between tablets and smartphones.

ShakeUnlock: Securely Unlock Mobile Devices by Shaking them Together

The inherent weakness of typical mobile device unlocking approaches (PIN, password, graphic pattern) is that they demand time and attention, leading a majority of end users to disable them, effectively lowering device security. We propose a method for unlocking mobile devices by shaking them together, implicitly passing the unlocked state from one device to another. One obvious use case includes a locked mobile phone and a wrist watch, which remains unlocked as long as strapped to the users wrist. Shaking both devices together generates a one-time unlocking event for the phone without the user interacting with the screen. We explicitly analyze the usability critical impact of shaking duration with respect to the level of security. Results indicate that unlocking is possible with a true match rate of 0.795 and true non match rate of 0.867 for a shaking duration as short as two seconds.

Towards pan shot face unlock: Using biometric face information from different perspectives to unlock mobile devices

– Personal mobile devices currently have access to a significant portion of their users private sensitive data and are increasingly used for processing mobile payments. Consequently, securing access to these mobile devices is a requirement for securing access to the sensitive data and potentially costly services. The authors propose and evaluate a first version of a pan shot face unlock method: a mobile device unlock mechanism using all information available from a 180° pan shot of the device around the users head – utilizing biometric face information as well as sensor data of built‐in sensors of the device. The paper aims to discuss these issues., – This approach uses grayscale 2D images, on which the authors perform frontal and profile face detection. For face recognition, the authors evaluate different support vector machines and neural networks. To reproducibly evaluate this pan shot face unlock toolchain, the authors assembled the 2013 Hagenberg stereo vision pan shot face database, which the authors describe in detail in this article., – Current results indicate that the approach to face recognition is sufficient for further usage in this research. However, face detection is still error prone for the mobile use case, which consequently decreases the face recognition performance as well., – The contributions of this paper include: introducing pan shot face unlock as an approach to increase security and usability during mobile device authentication; introducing the 2013 Hagenberg stereo vision pan shot face database; evaluating this current pan shot face unlock toolchain using the newly created face database.

A Large-Scale, Long-Term Analysis of Mobile Device Usage Characteristics

Today, mobile devices like smartphones and tablets have become an indispensable part of peoples lives, posing many new questions e.g., in terms of interaction methods, but also security. In this paper, we conduct a large scale, long term analysis of mobile device usage characteristics like session length, interaction frequency, and daily usage in locked and unlocked state with respect to location context and diurnal pattern. Based on detailed logs from 29,279 mobile phones and tablets representing a total of 5,811 years of usage time, we identify and analyze 52.2 million usage sessions with some participants providing data for more than four years. Our results show that context has a highly significant effect on both frequency and extent of mobile device usage, with mobile phones being used twice as much at home compared to in the office. Interestingly, devices are unlocked for only 46 % of the interactions. We found that with an average of 60 interactions per day, smartphones are used almost thrice as often as tablet devices (23), while usage sessions on tablets are three times longer, hence are used almost for an equal amount of time throughout the day. We conclude that usage session characteristics differ considerably between tablets and smartphones. These results inform future approaches to mobile interaction as well as security.

Towards Secure Personal Device Unlock Using Stereo Camera Pan Shots

Personal mobile devices hold sensitive data and can be used to access services with associated cost. For security reasons, most mo- bile platforms hence implement automatic device locking after a period of inactivity. Unlocking them using approaches like PIN, password or an unlock pattern is both problematic in terms of usability and poten- tially insecure, as it is prone to the shoulder surfing attack: an attacker watching the display during user authentication. Therefore, face unlock - using biometric face information for authentication - was developed as a more secure as well as more usable personal device unlock. Unfortu- nately, when using frontal face information only, authentication can still be circumvented by a photo attack: presenting a photo/video of the au- thorized person to the camera. We propose a variant of face unlock which is harder to circumvent by using all face information that is available dur- ing a 180 ◦ pan shot around the users head. Based on stereo vision, 2D and range images of the users head are recorded and classified along with sensor data of the device movement. We evaluate different classi- fiers for both grayscale 2D and range images and present our current results based on a new stereo vision face database.

Comparing the Placement of Two Arm-Worn Devices for Recognizing Dynamic Hand Gestures

Dynamic hand gestures have become increasingly popular as an input modality for interactive systems. There exists a variety of arm-worn devices for the recognition of hand gestures, which differ not only in their capabilities, but also in the arm positions where they are worn. The aim of this paper is to investigate the effect of placement of such devices on the accuracy for recognizing dynamic hand gestures (e.g. waving the hand). This is relevant as different devices require different positions and thus differ in the achievable recognition accuracy. We have chosen two positions on the forearm: on the wrist and right below the elbow. These positions are interesing as smartwatches are usually worn on the wrist and devices using EMG sensors for the detection of static hand gestures (e.g. spreading the fingers) have to be worn right below the elbow. We used an LG G Watch worn on the wrist and a Myo armband from Thalmic Labs worn below the elbow. Both are equipped with three-axis accelerometers, which we used for gesture recognition. Our hypothesis was that the wrist-worn device would have a better recognition accuracy, as dynamic hand gestures have a bigger action radius on the wrist and therefore lead to bigger acceleration values. We conducted a comparative study with nine participants that performed eight simple, dynamic gestures on both devices. We tested the 4320 gesture samples with different classifiers and feature sets. Although the recognition results for the wrist-worn device were higher, the difference was not significant due to the substantial variation across participants.

Towards device-to-user authentication: protecting against phishing hardware by ensuring mobile device authenticity using vibration patterns

Users usually authenticate to mobile devices before using them (e.g. PIN, password), but devices do not do the same to users. Revealing the authentication secret to a non-authenticated device potentially enables attackers to obtain the secret, by replacing the device with an identical-looking malicious device. The revealed authentication secret could be transmitted to the attackers immediately, who then conveniently authenticate to the real device. Addressing this attack scenario, we analyze different approaches towards mobile device-to-user (D2U) authentication, for which we provide an overview of advantages/drawbacks, potential risks and device authentication data bandwidth estimations. We further analyze vibration as one D2U feedback channel that is unobtrusive and hard to eavesdrop, including a user study to estimate vibration pattern recognition using a setup of ~7 bits per second (b/s). Study findings indicate that users are able to distinguish vibration patterns with median correctness of 97.5% (without taking training effects into account) -- which indicates that vibration could act as authentication feedback channel and should be investigated further in future research.

Confidence and Risk Estimation Plugins for Multi-Modal Authentication on Mobile Devices using CORMORANT

Mobile devices, ubiquitous in modern lifestyle, embody and provide convenient access to our digital lives. Being small and mobile, they are easily lost or stole, therefore require strong authentication to mitigate the risk of unauthorized access. Common knowledge-based mechanism like PIN or pattern, however, fail to scale with the high frequency but short duration of device interactions and ever increasing number of mobile devices carried simultaneously. To overcome these limitations, we present CORMORANT, an extensible framework for risk-aware multi-modal biometric authentication across multiple mobile devices that offers increased security and requires less user interaction.