Mikko Pyykkönen

Multimodal interaction with speech and physical touch interface in a media center application

We present a multimodal media center interface based on a novel combination of new modalities. The application is based on a combination of a large high-definition display and a mobile phone. Users can interact with the system using speech input (speech recognition), physical touch (touching physical icons with the mobile phone), and gestures. We present the key results from a laboratory experiment where user expectations and actual usage experiences are compared.

NFC-Based User Interfaces

Here, we discuss the potential of user interfaces built from NFC phones and NFC tags. We present these NFC-based user interfaces as components of interactive spaces, environments equipped with rich user interfaces and offering a large variety of services for users. Moreover, we specify an interaction model for NFC-based user interfaces and a graphical language for advertising NFC tags. In the model, interaction is decomposed into discovery, composition and usage stages and described using users, tokens, resources, and services. The language provides graphical elements for advertising a large set of different services and commands in a uniform manner. Attention, interaction, technology, and action elements are the most important ones and additional information can be given with context and instruction elements.

Touch & Compose: Physical User Interface for Application Composition in Smart Environments

In this paper, we sketch a model for interaction between smart environment and their inhabitants. We also introduce Touch & Compose, a concept for composing applications which utilizes this interaction model. The basic idea of Touch & Compose is to assemble applications from the resources that the user has selected manually by touching them with her mobile terminal. Resources (devices, services, files, etc.) are represented with icons attached to real environment objects. RFID tags are placed under the icons; they contain data identifying the resources. The mobile terminal is equipped with an RFID reader. The touched icons are collected in the mobile terminals resource stack; an application is composed from the resources in the stack either automatically or when requested by the user. Some resources collected from the environment can be stored permanently in the mobile terminal. The stack allows also sharing resources with other users at the environment. We also present software architecture for implementing the Touch & Compose concept and suggest a GUI for the mobile terminal.

Universal Remote Control for the Smart World

In this paper, we discuss how to build user friendly user interfaces to the smart world. We present the REACHeS architecture for controlling Internet services through physical user interfaces, using a mobile terminal and icons placed in the environment. An icon advertises a service that can be started by touching the icon with a mobile terminal. This service activation configures the mobile terminal as a remote control for the service. We have implemented this architecture and designed an icon set. The physical user interface is based on RFID technology: the terminals are equipped with RFID readers and RFID tags are placed under the icons. We present the first prototype applications and the first usability tests that we have carried out.

Designing tangible user interfaces for NFC phones

The increasing amount of NFC phones is attracting application developers to utilize NFC functionality. We can hence soon expect a large amount of mobile applications that users command by touching NFC tags in their environment with their NFC phones. The communication technology and the data formats have been standardized by the NFC Forum, but there are no conventions for advertising to the users NFC tags and the functionality touching the tags triggers. Only individual graphical symbols have been suggested when guidelines for advertising a rich variety of functionality are called for. In this paper, we identify themain challenges and present our proposal, a set of design guidelines based on more than twenty application prototypes we have built. We hope to initiate discussion and research resulting in uniform user interfaces for NFC-based services.

Remote control for pervasive services

In this paper, we present the REACHeS architecture for controlling pervasive services through physical user interfaces, using a mobile terminal and icons placed in the environment. An icon advertises a service that can be started by touching the icon with a mobile terminal. This service activation configures the mobile terminal as a remote control for the service. We have implemented this architecture and designed an icon set. The physical user interface is based on RFID technology: the terminals are equipped with RFID readers and RFID tags are placed under the icons. We present the first prototype applications and the first usability tests that we have carried out.

Activity pad: teaching tool combining tangible interaction and affordance of paper

In this paper, we describe the design process and early experiences of the Activity Pad, an interactive digital artifact for active learning environments. The pad combines a 4x6 grid of programmable NFC readers together with printed sheets of A4-sized paper to allow teacher-driven creation of interactive learning applications featuring application-specific tangibles. We describe iterative design process for this teaching tool, including mock-up prototypes, focus group discussions with teachers and the first complete prototype together with two example applications. Teachers were eager to innovate applications for the Activity Pad, and the feedback indicates the potential of this kind of teaching tool in diverse learning environments.

iCompose: context-aware physical user interface for application composition

Ubiquitous computing environments advocate creating applications by plugging together various resources (mobile devices, displays, augmented everyday objects, and so on) and Web Services to support the user’s everyday activities and needs. This approach is referred to as application composition, and such applications are called composite. Due to the dynamic nature of ubiquitous environments, application composition has to be supported at runtime, so that the applications are able to adapt to the situation within the environment and other contexts. Application composition is usually performed by autonomous context-aware mechanisms that provide limited or no control for users. Still, users need to be aware of their environments and be able to control and configure applications when they are composed and executed. Towards this goal we present in this article a context-aware application composition system based on the iCompose interface for composing and controlling applications at runtime. Users compose applications by simply touching the resources in the environment with their mobile devices, while the iCompose interface provides feedback and assists users by suggesting them possible further actions. The interface relies on a rule-based reasoner and utilizes various context sources to support users dynamically, according to the situation in which they compose applications. We present a complete implementation of the system and report the results of a user evaluation study conducted with 21 participants. This study assesses the issues of control, usability, feasibility and user acceptance of the iCompose interface for context-aware application composition and the prototype as a whole.

Human and Technological Dimensions of Making in FabLab

Abstract In this research, we studied the human dimensions of experience and knowledge, confidence, motivation, and fun with regard to four technological dimensions referring to a FabLab environment: 2D and 3D design, tools and machines, prototyping with electronics, and programming. An intensive, two-week training period for high school students in digital fabrication and design was utilized as a testbed to evaluate how the participants modified their perception of the four human dimensions during the training. We identified that prototyping with electronics and programming were the most significant obstacles. In addition, the perception of acquired knowledge and confidence had increased considerably after training except for the programming domain. FabLab trainers can utilize the trainees’ perceptions on different dimensions to emphasize the specific design aspects of the activity in order to achieve the training goals. We also expect that a detailed description of the experiment setup can be useful to other researchers and practitioners while organizing activities at FabLab.

Experiences with smart city traffic pilot

The infrastructure built in the City of Oulu provides rich information about the city environment and objects moving in it. We utilize this infrastructure in building an IoT system for data-intensive smart city services; by collecting data from real city environment and developing analysis methods for these data. We are building Smart City Traffic Pilot on top of the infrastructure to provide the functionality to collect the data and perform the analysis. Based on this experience, we present in this article requirements for data-intensive smart city services. Moreover, we describe four implemented use cases for utilizing rich data sources available in the smart city: situational picture, driving coach, real time reasoning, and mobile code. A lively collaboration between a large number of different actors is essential in realizing these use cases. Finally, we discuss how the use cases fulfill the requirements and the lessons we have learnt.