Jari Varsaluoma

Collecting cross-cultural user data with internationalized storyboard survey

Globalization and the search for experiential aspects of technology products and services have increased the demand for cross-cultural user feedback. Remote methods would suit agile global data collection, but only few common practices yet exist. Thus, the goal of the present study was to determine ways in which common visual stimulus material (internationalized storyboards) are perceived similarly and differently by cross-cultural respondents. An internationalized remote online storyboard survey was designed to collect cross-cultural user data of 252 respondents, from the USA, Brazil, India, Italy and Finland -- around the topic of mobile content sharing concepts. It was found that, for the majority of situations and details, storyboards supported a similar interpretation by users from different cultural backgrounds; and internationalized pictures assisted respondents in providing rich answers to a long survey because of a sound understanding of the intended situations and ease of imagining themselves in different usage situations.

The Fuzzy Front End of Experience Design: Eliciting and Communicating Experience Goals

When starting an experience design process, designers should first determine the experience to aim for. In the fuzzy front end of the experience design process, there are often several alternative sources for gaining insight and inspiration in defining this experience. In this paper, we describe our findings from two surveys about experience goal setting and approaches to communicate about these goals with stakeholders. The results from researchers working on 9 different experience design cases suggest that empathic understanding of the users world is the most used source of insight and inspiration in defining experience goals. As an end result, we propose the model for Experience Goal Elicitation Process to clarify the fuzzy front end of experience design and instructions to support designers in defining and evaluating experience goals.

Scenarios in the Heuristic Evaluation of Mobile Devices: Emphasizing the Context of Use

Varying contexts of use make the usability studies of mobile devices difficult. The existing evaluation methods, such as Heuristic Evaluation (HE), must be redesigned in order to create more awareness of the mobile context. Through the reworking of existing heuristics and use of written use scenarios, there have already been some promising results. In this study the context of use of mobile devices was examined with written scenarios. The main target was to improve the reliability of HE by increasing the number of right predictions and reducing the number of false positives produced by the evaluators. The results seem to differ from those of a previously conducted study as the scenarios did not improve the HE regarding the numbers of false positives or accurate predictions. There is a need for more research regarding the possible benefits of different scenarios and other factors that affect the outcomes of HE.

Axe UX: Exploring long-term user experience with iScale and AttrakDiff

Positive user experience (UX) is an important goal in product design. Positive long-term UX is believed to improve customer loyalty, therefore being vital for continuous commercial success. Most UX research investigates momentary or short-term UX although the relationship between the user and the product evolves over time. There is a need to develop methods for measuring long-term UX and evaluate their feasibility in different product contexts. In this explorative study, 18 customers reported their experiences during the first three months of use of a non-interactive design tool, an axe. UX was evaluated with retrospective iScale tool and monthly repeated AttrakDiff questionnaire. iScale demonstrated the long-term trend of the attractiveness of the product well and provided information about the causes of the change in the experience. The AttrakDiff questionnaire was a good indicator of attractiveness during a longer period of time and is also well suited to remote studies.

DrawUX: web-based research tool for long-term user experience evaluation

DrawUX is a web-based research tool for retrospective long-term user experience evaluation especially in remote studies. Users sketch a curve and add text comments to report how their experience on a product or service has changed during the time of use. DrawUX provides both quantitative and qualitative data about users experiences during a long-term use of a product or service.

Information Visualization Heuristics in Practical Expert Evaluation

While traditional HCI heuristics can be used to find usability issues also from information visualization systems, specialized heuristics tailored for the information visualization (InfoViz) domain can be more effective and focus on the special characteristics of these systems. In this study, we describe the application of ten information visualization heuristics from prior research and their testing in practical heuristic evaluation. We found that the selected heuristics were useful with good coverage in our application case. However, based on our observations, we argue that interaction, veracity, and aesthetics related heuristics should be added to the previously used set. The lack of domain knowledge made the evaluators somewhat uneasy with their capability to carry out the investigation in-depth. We suggest to train domain experts with understanding of the data and application domain to carry out the evaluation to get insightful feedback beyond usability issues.

Usefulness of long-term user experience evaluation to product development: practitioners' views from three case studies

Understanding the temporal aspects of user experience (UX) has received increasing attention in the HCI community. However, little empirical evidence is available on how practitioners in product development companies evaluate the usefulness or actually use long-term UX evaluation data in their work. In this study, we explore how practitioners (e.g., managers, designers and UX specialists) evaluate the usefulness of long-term UX evaluation results to their own work. Three case studies were conducted with longitudinal and retrospective methods in a company developing interactive sports products. Our findings suggest that long-term UX evaluation provides results that are perceived as interesting, relevant and useful by practitioners. Potential uses for the results were e.g., verifying practitioners expectations, planning future work, understanding changes in UX, the development of future products, and updating current software products. Future research should focus on how to provide long-term UX evaluation results in more efficient manner to benefit product development.

Comparing the effectiveness of electronic diary and UX curve methods in multi-component product study

Due to the complex nature of multi-component products, methods for their user experience (UX) evaluation are critical. In this study, we evaluated the long-term UX of a multi-component product from the sports domain. We present a comparison of two remote UX evaluation methods: a weekly filled-in electronic diary and the UX Curve method for retrospective evaluation. The cost-effectiveness of the methods was evaluated using two factors: the amount of time required by 1) the researchers in employing the methods and 2) the participants to report their experiences with the method. Also, to determine the quality of the reported experiences, their content was compared between the methods. The results show that the electronic diary was more effective in revealing the experiences related to utility, missing functionalities, technical faults, dissatisfaction, and effectiveness. However, the UX Curve method was more effective in capturing the experiences related to stimulation.

Guidelines for Development and Evaluation of Usage Data Analytics Tools for Human-Machine Interactions with Industrial Manufacturing Systems

We present the lessons learned during the development and evaluation process for UX-sensors, a visual data analytics tool for inspecting logged usage data from flexible manufacturing systems (FMS). Based on the experiences during a collaborative development process with practitioners from one FMS supplier company, we propose guidelines to support other developers of visual data analytics tools for usage data logging in context of complex industrial systems. For instance, involving stakeholders with different roles can help to identify user requirements and generate valuable development ideas. Tool developers should confirm early access to real usage data from customers systems and familiarize themselves with the log data structure. We argue that combining expert evaluations with field study methods can provide a more diverse set of usability issues to address. For future research, we encourage studies on insights emerging from usage data analytics and their impact on the viewpoints of the supplier and customer.

Exploring motivational aspects and user experience of mobile mathematics learning service in south africa

The rapid rise in the number of mobile phone users in developing countries has created an opportunity to research and develop new mobile learning services. However, designing mobile learning services that are both motivating and provide positive user experiences is crucial for being successful in mobile learning markets. In this paper, we study the motivational factors for studying mathematics and using mobile mathematics learning service Microsoft Math in South Africa. The relationship of learners motivational factors, UX, math skills and behavioral intentions are studied. No significant relationships were found between UX and motivational factors. As a result, we provide 1) the summaries of the identified motivational factors for studying mathematics and for using mobile mathematics learning services, 2) UX goals for designing mobile learning services for developing countries, and 3) implications for conducting remote mobile surveys in developing countries.