Luis Olsina

Measuring Web application quality with WebQEM

This article discusses using WebQEM, a quantitative evaluation strategy to assess Web site and application quality. Defining and measuring quality indicators can help stakeholders understand and improve Web products. An e-commerce case study illustrates the methodologys utility in systematically assessing attributes that influence product quality.

Specifying Quality Characteristics and Attributes for Websites

In this work, we outline more than a hundred characteristics and attributes for the domain of academic sites in order to analyze the quality requirement tree and a way to specify them. These elements are used in a quantitative methodology for assessment, comparison, and ranking processes. The proposed Web-site Quality Evaluation Methodology (QEM) can be a useful approach to assess the quality in different phases of a Web product life cycle. In the academic study, we have observed three different evaluation audiences regarding visitor users: current and prospective students, academic personnel, and research sponsors. Besides, the aim of this work is to show a hierarchical and descriptive specification framework for characteristics, subcharacteristics and attributes. This framework is a key underlying piece in the construction of a hyperdocumented evaluation tool. Finally, some results are presented and concluding remarks are discussed.

How to Measure and Evaluate Web Applications in a Consistent Way

In the present chapter we analyze the rationale to measure and evaluate software and web applications or products from different perspectives. First, we focus on the process for measurement and evaluation, i.e. the main technical activities that might be planned. Second, we discuss a measurement and evaluation framework that relies on a sound conceptual base. This framework so-called INCAMI (Information Need, Concept model, Attribute, Metric and Indicator) is based on our ontological proposal of metrics and indicators. The underlying hypothesis is that without appropriate recorded metadata of information needs, attributes, metrics and indicators it is difficult to ensure measure and indicator values are repeatable and comparable among organization’s projects. Moreover, analyses and comparisons could be performed in an inconsistent way. Third, in order to illustrate the measurement and evaluation process and framework with specific models and procedures an inspection methodology named WebQEM (Web Quality Evaluation Method) is used. In addition, a case study on the external quality of the shopping cart component of a typical e-commerce website is carried out. Finally, additional discussions about the flexibility of the measurement and evaluation framework as well as concluding remarks are drawn.

Assessing the Quality of Academic Websites: a Case Study

Abstract In this paper, a quantitative evaluation approach to assess the quality of sites called Website Quality Evaluation Method (QEM) is proposed. This prescriptive and descriptive approach might be useful to evaluate and compare quality characteristics and attributes in different phases of a Web product lifecycle. Particularly, to discuss this methodology, we evaluate the level of accomplishment of required quality characteristics (like usability, functionality, reliability, efficiency, and derived subcharacteristics) in six typical academic sites. At the end of the evaluation process, a ranking for each selected site is obtained. Specifically, the evaluation process generates elemental, partial, and global indicators or quality preferences that can be easily analyzed, backward and forward traced, justified, and efficiently employed in decision-making activities. Hence, conclusions about the state-of-the-art of the quality in the operative phase of these sites can be drawn. In addition, recommendation...

Towards an ontology for software metrics and indicators as the foundation for a cataloging Web system

So that the software and Web measurement field can become a more robust engineering discipline, it is mandatory to start reaching a common agreement between researchers and other stakeholders about primitive concepts such as attribute, metric, measure, measurement and calculation method, scale, elementary and global indicator, calculable concept, among others. There are various useful recently-issued ISO standards related to software quality models, measurement and evaluation processes, however, we observe sometimes a lack of a sound consensus among the same terms in different documents or sometimes absent terms. We present an ontology for software metrics and indicators-based as much as possible on the concepts of those standards, which can be useful to support different assurance processes, methods and tools in addition to being the foundation for our cataloging Web system [Olsina, L et al., (2003)]. Without sound and consensuated definitions it is difficult to assure metadata consistency and, ultimately, data values are comparable on the same basis.

E-commerce Site Evaluation: a Case Study

Recent surveys on e-commerce sites confirm the increased use of the Web for shopping. The reasons of these trends have been attributed to different factors such as convenience, saving time, absence of sales pressure, among others. However, an essential site characteristic that should also be taken into account is the quality in use. In this paper, we present a case study on five e-bookstores in order to assess characteristics and attributes that influence quality in use utilizing for such an end the Web-site Quality Evaluation Methodology (QEM). The main goal of this work is to show the level of accomplishment of required quality characteristic like usability, functionality, reliability, efficiency and derived sub-characteristics and attributes, regarding the user standpoint. In addition, we focus on sub-characteristics and attributes concerning e-commerce site functionality.

Quality, quality in use, actual usability and user experience as key drivers for web application evaluation

Due to the increasing interest in Web quality, usability and user experience, quality models and frameworks have become a prominent research area as a first step in evaluating them. The ISO 25010/25012 standards were recently issued which specify and evaluate software and data quality requirements. In this work we propose extending the ISO 25010 standard to incorporate new characteristics and concepts into a flexible modeling framework. Particularly, we focus on including information quality, and learnability in use characteristics, and actual usability and user experience concepts into the modeling framework. The resulting models and framework contribute towards a flexible, integrated approach to evaluate Web applications. The operability and particularly the learnability of a real Web application are evaluated using the framework.

Assessing quality in use in a consistent way

In the present paper we discuss a systematic approach in order to specify, measure, and evaluate quality in use, i.e., the perceived quality of software or web applications by actual users in real con-texts of use. The methodological proposal is grounded on our measurement and evaluation framework which in turn is based on the metrics and indicators ontology. In order to illustrate the pro-posal a case study for assessing the quality in use of an e-Learning application was conducted. The final results are analyzed in the light of both the consistency of the methodological support and the challenges of assessing quality in use with actual users.

Defining and validating metrics for navigational models

Nowadays, several approaches for developing Web applications have been proposed in the literature. Most of them extend existing object-oriented conceptual modeling methods, incorporating new constructors in order to model the navigational structure and the content of Web applications. Such new constructors are commonly represented in a navigational model. While navigational models constitute the backbone of Web application design, their quality has a great impact on the quality of the final product, which is actually implemented and delivered. We discuss a set of metrics for navigational models that has been proposed for analyzing the quality of Web applications in terms of size and structural complexity. These metrics were defined and validated using a formal framework (DISTANCE) for software measure construction that satisfies the measurement needs of empirical software engineering research. Some experimental studies have shown that complexity affects the ability to understand and maintain conceptual models. In order to prove this, we also made a controlled experiment to observe how the proposed metrics can be used as early maintainability indicators.

An integrated strategy to systematically understand and manage quality in use for web applications

The main goal in evaluating software quality is to ultimately improve its quality. In this work, we discuss SIQinU (Strategy for Improving Quality in Use), a six-phased evaluation-driven strategy for understanding and improving software quality requirements in a systematic way. Starting with quality in use (QinU), we design specific user tasks and context of use, and through identifying problems in QinU, we determine external quality (EQ) attributes that could be related to these QinU weakly performing indicators. Then, after deriving EQ attributes related to the QinU problems, we evaluate EQ and derive a benchmark to be used as a basis to make improvements. Once improvement recommendations are made based on poorly performing EQ indicators, a new version of the software application is completed and evaluated again for its EQ to establish a delta from the initial benchmark. Then, we re-evaluate QinU to determine the improvements resulting in QinU from the improvements made at the EQ level, thus leading to a cyclic strategy for improvement and development of relationships. SIQinU is a repeatable and consistent strategy which relies on: a conceptual framework (with ontological base), a process, and specific methods. In order to illustrate SIQinU, a real case study is conducted.