Ville Harkke

Busting myths of electronic word of mouth: the relationship between customer ratings and the sales of mobile applications

Business and academic research frequently highlights the power of electronic word of mouth, relying on the knowledge that online customer ratings and reviews influence consumer decision making. Numerous studies in different disciplines have been conducted to examine the effectiveness of electronic word of mouth communication. Previously, typically small sample studies suggest that positive electronic word of mouth increases sales and that the effects depend on the volume and valence of reviews and ratings. This studys contribution lies in testing the relationship between electronic word of mouth and the sales of applications in a mobile application ecosystem (Google Play) with an extensive dataset (over 260 million customer ratings; 18 months). The results show that higher values of valence of customer ratings correlate statistically significantly with higher sales. The volume of ratings correlates positively with sales in the long term but negatively in the short term. Furthermore, the relationship between electronic word of mouth and sales seems to be more important when the price of the application increases. The findings also underline the importance of the choice of a measurement period in studies.

Back to User-Centered Usability Testing

Usability testing is a widely used evaluation method for product design during and after the development. Conventional usability testing applies short and discrete test tasks and task scenarios that are based on the tasks the product is designed to support. Thus, conventional test task design relies heavily on the representations of the specified context of use and the specified user requirements of the proposed design solution. However, a premature commitment to the specified context, requirements and proposed solutions may limit the scope of usability testing in a manner that hinders its capability to elicit and validate new user requirements, which is one of the objectives of the evaluation phase in the iterative user-centered design process. In this paper, we introduce a user-centered task design approach, which allows test participants to follow their natural work flow and freely express their needs during a test session. The main idea of this open-ended task approach is to break the tight link between the produced design solutions and the tasks used in the usability test and in this way increase the probability that novel user needs can emerge during a test session. Empirical results from a case study are used to depict the approach and its prerequisites, strengths, and limitations are discussed.

Evaluation for Evaluation: Usability Work during Tendering Process

In large-scale IT implementations the iterative system development and system tailoring to organization-specific needs begins after a tendering process is over. In the tendering process a procuring organization defines usability requirements, evaluation procedures and selection criteria, while IT vendors try and build and propose solutions that best fulfill these requirements. Existing usability research has instructed the procuring organizations on how usability should be included in the request for proposals, in order to ensure the usability of the selected system. However, the IT vendor perspective on request for proposals and related usability work during the tendering process remain unexplored. This paper grasps the issue with an empirical study on large-scale IT procurement. The preliminary findings give practical advice to vendors planning usability work and buyers writing request for proposals.

Testing the Unknown – Value of Usability Testing for Complex Professional Systems Development

To make an impact on the design in usability testing, the test tasks are essential ingredients for the early system development process. Complex design problems are not solved by focusing on the details of a prototype and setting the scope on what is already known by the design team. Instead, the design value of usability testing is increased by deliberately relinquishing the assumptions made and implemented into a design. In the development of complex systems, usability testing with extended scope and open-ended structure, as presented in this paper with three empirical cases, delivers not only specific knowledge about the user interactions with the system, but reveals issues that, despite rigorous user research efforts, have been overlooked in the preceding phases of system development. Therefore, we suggest applying open-ended usability test tasks for testing systems in complex settings such as in the development of health care systems.

Are We Testing Utility? Analysis of Usability Problem Types

Usability problems and related redesign recommendations are the main outcome of usability tests although both are questioned in terms of impact in the design process. Problem classifications aim to provide better feedback for designers by improving usability problem identification, analysis and reporting. However, within the classifications, quite little is discussed about the types and the contents of usability problems as well as the types of required design efforts. We address this problem by scrutinizing the findings of three empirical usability tests conducted in software development projects. As a result, 173 problems were classified into 11 categories. Specific focus was placed on the distinction between the utility and usability types of problems, in order to define the correct development phase and method to fix the problem. The number of utility problems varied from 51 % to 74 %, which shows that early usability testing with a think-aloud protocol and an open task structure measure both utility and usability equally well.