Philip Lew

Quality, quality in use, actual usability and user experience as key drivers for web application evaluation

Due to the increasing interest in Web quality, usability and user experience, quality models and frameworks have become a prominent research area as a first step in evaluating them. The ISO 25010/25012 standards were recently issued which specify and evaluate software and data quality requirements. In this work we propose extending the ISO 25010 standard to incorporate new characteristics and concepts into a flexible modeling framework. Particularly, we focus on including information quality, and learnability in use characteristics, and actual usability and user experience concepts into the modeling framework. The resulting models and framework contribute towards a flexible, integrated approach to evaluate Web applications. The operability and particularly the learnability of a real Web application are evaluated using the framework.

An integrated strategy to systematically understand and manage quality in use for web applications

The main goal in evaluating software quality is to ultimately improve its quality. In this work, we discuss SIQinU (Strategy for Improving Quality in Use), a six-phased evaluation-driven strategy for understanding and improving software quality requirements in a systematic way. Starting with quality in use (QinU), we design specific user tasks and context of use, and through identifying problems in QinU, we determine external quality (EQ) attributes that could be related to these QinU weakly performing indicators. Then, after deriving EQ attributes related to the QinU problems, we evaluate EQ and derive a benchmark to be used as a basis to make improvements. Once improvement recommendations are made based on poorly performing EQ indicators, a new version of the software application is completed and evaluated again for its EQ to establish a delta from the initial benchmark. Then, we re-evaluate QinU to determine the improvements resulting in QinU from the improvements made at the EQ level, thus leading to a cyclic strategy for improvement and development of relationships. SIQinU is a repeatable and consistent strategy which relies on: a conceptual framework (with ontological base), a process, and specific methods. In order to illustrate SIQinU, a real case study is conducted.

Specifying process views for a measurement, evaluation, and improvement strategy

Any organization that develops software strives to improve the quality of its products. To do this first requires an understanding of the quality of the current product version. Then, by iteratively making changes, the software can be improved with subsequent versions. But this must be done in a systematic and methodical way, and, for this purpose, we have developed a specific strategy called SIQinU (Strategy for understanding and Improving Quality in Use). SIQinU recognizes problems of quality in use through evaluation of a real system-in-use situation and proposes product improvements by understanding and making changes to the products attributes. Then, reevaluating quality in use of the new version, improvement gains can be gauged along with the changes that led to those improvements. SIQinU aligns with GOCAME (Goal-Oriented Context-Aware Measurement and Evaluation), a multipurpose generic strategy previously developed formeasurement and evaluation, which utilizes a conceptual framework (with ontological base), a process, and methods and tools. Since defining SIQinU relies on numerous phase and activity definitions, in this paper, we model different process views, for example, taking into account activities, interdependencies, artifacts, and roles, while illustrating them with excerpts from a real-case study.

Evaluating Mobileapp Usability: A Holistic Quality Approach

As newer-generation smartphones enhance functionalities, interactions and services become more complex, leading to usability issues that are increasingly critical and challenging. Also mobile apps have several particular features that pose challenges evaluating their usability using current quality models, usability views, and their relations with target and context entities. With respect to the current literature, usability, actual usability, and user experience are poorly related to target entities (e.g. system and system in use) and context entities, to quality views (e.g. external quality and quality in use), in addition to measurement and evaluation building blocks. In this paper, we propose a holistic quality approach for evaluating usability and user experience of mobile apps. Practical use of our strategy is demonstrated through evaluation for the Facebook mobile app from the system usability viewpoint. Ultimately, a usability evaluation strategy should help designers to understand usability problems effectively and produce better design solutions so we analyze in the context of the framework’s applicability toward this goal.

For User-Driven Software Evolution: Requirements Elicitation Derived from Mining Online Reviews

Online reviews that manifest user feedback have become an available resource for eliciting requirements to design future releases. However, due to complex and diverse opinion expressions, it is challenging to utilize automated analysis for deriving constructive feedback from these reviews. What’s more, determining important changes in requirements based on user feedback is also challenging. To address these two problems, this paper proposes a systematic approach for transforming online reviews to evolutionary requirements. According to the characteristics of reviews, we first adapt opinion mining techniques to automatically extract opinion expressions about common software features. To provide meaningful feedback, we then present an optimized method of clustering opinion expressions in terms of a macro network topology. Based on this feedback, we finally combine user satisfaction analysis with the inherent economic attributes associated with the software’s revenue to determine evolutionary requirements. Experimental results show that our approach achieves good performance for obtaining constructive feedback even with large amounts of review data, and furthermore discovers the evolutionary requirements that tend to be ignored by developers from a technology perspective.

Using Web Quality Models and Questionnaires for Web Applications Understanding and Evaluation

As web applications (WebApps) continue to gain popularity, evaluating new generation WebApps poses challenges regarding use of current quality models and their included (sub-) characteristics. For instance, communicability and sense of community are characteristics often neglected in quality modeling. Considering the recently issued ISO/IEC 25010/25012 standards, we utilize our previously developed and recently revised quality models and framework, 2Q2U (Quality, Quality in use, actual Usability and User experience) in combination with SIQinU (Strategy to Improve Quality in Use), a purpose oriented improvement strategy. We show SIQinUs flexible approach through using questionnaires to measure and evaluate quality in use for WebApps. To illustrate this approach a practical case study for evaluating pragmatic and hedonic user perceptions for two online radio WebApps is conducted.

Relating User Experience with MobileApp Quality Evaluation and Design

Designing quality into web applications (WebApps) and evaluating WebApp quality and usability have been the subject of abundant research. However, both the design and evaluation of traditional WebApps cannot account for the particular features and usage contexts of mobile applications (MobileApps). MobileApps have several characteristics that pose challenges in their design and evaluation regarding current quality models and their included characteristics and sub-characteristics. For instance, the operability of a user interface has a much different and greater influence when evaluating MobileApp usability and user experience due to the context of the user. Characteristics such as multi-touch gestures, button size, and widget usage have a magnified impact on task completion rates. We propose in this paper utilizing our previously developed ISO 25010-based quality models and framework so-called 2Q2U (Quality, Quality in use, actual Usability and User experience) for MobileApps. Specific MobileApp task screens and attributes are illustrated in order to show our evaluation approach applicability.

Strategy to improve quality for software applications: a process view

Each organization devoted to developing software/web applications should have as one of its ultimate goals to improve the quality in use of its products. In order to accomplish this, first it has to understand the quality of the current product version and then make appropriate changes to increase the quality of the new version if improvement actions were needed. For this purpose, we have developed a specific strategy called SIQinU (Strategy for understanding and Improving Quality in Use), which allows recognizing problems of quality in use through evaluation and proposes product improvements by understanding and making changes on product attributes. Hence by re-evaluating quality in use of the new version, improvement gains can be gauged. SIQinU is in alignment with GOCAME (Goal-Oriented Context-Aware Measurement and Evaluation), a multi-purpose generic strategy previously developed for measurement and evaluation which relies on: a conceptual framework (with ontological base), a process, and methods and tools. Since the process aspect is paramount in defining SIQinU - given the amount of phases and activities - in this paper we model the functional and behavioral process views illustrating them with excerpts of a real case study.

Evaluating software learnability: A learnability attributes model

Learnability has been regarded as a fundamental usability attribute and plays a significant role for the success of software applications. Good learnability not only leads to a better productivity quickly but also plays a vital role in initial adoption or rejection of a technology. To evaluate and hence improve learnability, understanding the factors affecting learnability is necessary. In this paper we develop a comprehensive model of learnability by identifying and then abstracting learnability attributes logically to lower level. We carry out a case study to prove adequacy of the proposed model. This learnability attribute model can set a baseline for future learnability understanding and evaluation.

Integrating quality, quality in use, actual usability and user experience

Selecting and using a quality model is usually a first step in evaluating software quality. In this work we propose employing previous work incorporating two new characteristics into recently issued the ISO 25010 standard combined with concepts of actual usability and user experience in a flexible framework. The resulting models and framework can be instantiated to evaluate software and Web applications. Finally, a strategy and process for non-intrusively evaluating and improving quality in use based on real usage data is outlined. The application of this strategy is our current endeavor.