Juho Hella

User expectations and user experience with different modalities in a mobile phone controlled home entertainment system

Home environment is an exciting application domain for multimodal mobile interfaces. Instead of multiple remote controls, personal mobile devices could be used to operate home entertainment systems. This paper reports a subjective evaluation of multimodal inputs and outputs for controlling a home media center using a mobile phone. A within-subject evaluation with 26 participants revealed significant differences on user expectations on and experiences with different modalities. Speech input was received extremely well, even surpassing expectations in some cases, while gestures and haptic feedback were almost failing to meet the lowest expectations. The results can be applied for designing similar multimodal applications in home environments.

Multimodal interaction with speech and physical touch interface in a media center application

We present a multimodal media center interface based on a novel combination of new modalities. The application is based on a combination of a large high-definition display and a mobile phone. Users can interact with the system using speech input (speech recognition), physical touch (touching physical icons with the mobile phone), and gestures. We present the key results from a laboratory experiment where user expectations and actual usage experiences are compared.

Accessible Multimodal Media Center Application for Blind and Partially Sighted People

We present a multimodal media center interface designed for blind and partially sighted people. It features a zooming focus-plus-context graphical user interface coupled with speech output and haptic feedback. A multimodal combination of gestures, key input, and speech input is utilized to interact with the interface. The interface has been developed and evaluated in close cooperation with representatives from the target user groups. We discuss the results from longitudinal evaluations that took place in participants’ homes, and compare the results to other pilot and laboratory studies carried out previously with physically disabled and nondisabled users.

“Together at school” - a school-based intervention program to promote socio-emotional skills and mental health in children: study protocol for a cluster randomized controlled trial

BackgroundSchools provide a natural context to promote children’s mental health. However, there is a need for more evidence-based, high quality school intervention programs combined with an accurate evaluation of their general effectiveness and effectiveness of specific intervention methods. The aim of this paper is to present a study protocol of a cluster randomized controlled trial evaluating the “Together at School” intervention program. The intervention program is designed to promote social-emotional skills and mental health by utilizing whole-school approach and focuses on classroom curriculum, work environment of school staff, and parent-teacher collaboration methods.Methods/DesignThe evaluation study examines the effects of the intervention on children’s socio-emotional skills and mental health in a cluster randomized controlled trial design with 1) an intervention group and 2) an active control group. Altogether 79 primary school participated at baseline. A multi-informant setting involves the children themselves, their parents, and teachers. The primary outcomes are measured using parent and teacher ratings of children’s socio-emotional skills and psychological problems measured by the Strengths and Difficulties Questionnaire and the Multisource Assessment of Social Competence Scale. Secondary outcomes for the children include emotional understanding, altruistic behavior, and executive functions (e.g. working memory, planning, and inhibition). Secondary outcomes for the teachers include ratings of e.g. school environment, teaching style and well-being. Secondary outcomes for both teachers and parents include e.g. emotional self-efficacy, child rearing practices, and teacher-parent collaboration. The data was collected at baseline (autumn 2013), 6 months after baseline, and will be collected also 18 months after baseline from the same participants.DiscussionThis study protocol outlines a trial which aims to add to the current state of intervention programs by presenting and studying a contextually developed and carefully tested intervention program which is tailored to fit a national school system. Identification of effective intervention elements to promote children’s mental health in early school years is crucial for optimal later development.Trial registrationClinicalTrials.gov register: NCT02178332.

Multimodal Interaction with Speech, Gestures and Haptic Feedback in a Media Center Application

We demonstrate interaction with a multimodal media center application. Mobile phone-based interface includes speech and gesture input and haptic feedback. The setup resembles our long-term public pilot study, where a living room environment containing the application was constructed inside a local media museum allowing visitors to freely test the system.

Multimodal Media Center Interface Based on Speech, Gestures and Haptic Feedback

We present a multimodal media center interface based on speech input, gestures, and haptic feedback (hapticons). In addition, the application includes a zoomable context + focus GUI in tight combination with speech output. The resulting interface is designed for and evaluated with different user groups, including visually and physically impaired users. Finally, we present the key results from its user evaluation and public pilot studies.

Enhancing the Conference Experience with a Multi-Device, Multimodal, Multi-User Program Guide

Large-scale public events, such as conferences are becoming increasingly interactive with tools to support Web-based and mobile access to the event program. Multimodal interoperable devices can further improve the event experience. This paper introduces a multi-device, multimodal and multi-user conference program guide (M3PG), which provides access to the conference program through shared public displays and mobile devices. The service offers additional functionality such as multimedia content, voting, and event feedback. Its interaction methods include spoken language, gestures and haptic feedback. The service has been used in two conferences where it received very positive feedback. Together with pre-commercialization studies, the results indicate that the service is useful for conferences both from the user and organizer perspectives.

Multimodal multi-device program guide for smart conferences

We demonstrate a multimodal, multi-user, and multi-device conference program guide for conference participants. Its functionality includes access to the conference program with additional multimedia content, voting, feedback, and communication with the other participants. People can interact with the system in a multimodal way using spoken language, gestures and haptic feedback with mobile phones and shared public displays.

User experience of speech controlled media center for physically disabled users

In this paper, we present results from a long-term user pilot study of speech controlled media center. The pilot users in this case were physically disabled and the system was installed in their apartment for six weeks. We designed a multimodal media center interface based on speech. Full speech control is provided with a hands-free speech recognition input method for people with physical disabilities. In addition, the application includes a zoomable context + focus GUI in tight combination with speech output. The resulting interface was designed following human-centered principles. Finally, the results of user experience studies are presented.

Accessible speech-based and multimodal media center interface for users with physical disabilities

We present a multimodal media center user interface with a hands-free speech recognition input method for users with physical disabilities. In addition to speech input, the application features a zoomable context + focus graphical user interface and several other modalities, including speech output, haptic feedback, and gesture input. These features have been developed in co-operation with representatives from the target user groups. In this article, we focus on the speech input interface and its evaluations. We discuss the user interface design and results from a long-term pilot study taking place in homes of physically disabled users, and compare the results to a public pilot study and laboratory studies carried out with non-disabled users.