Ashok Sivaji

Usability Testing Methodology: Effectiveness of Heuristic Evaluation in E-Government Website Development

Software development organizations consist of marketing, project management, development, design, and quality assurance team. It is important for the various teams within the organization to understand the benefits and limitation of incorporating various usability testing methods within the software development life cycle. Some of the reasons for poor usability include effort prioritization conflicts from project management, development and design team. The role of the usability engineer is to get involved as the heuristic evaluator and facilitate the development and design efforts are based on usability principles and at the same time adhering to the project time line. Two of the common usability inspection methods consist of user experience testing and expert review or more commonly known as Heuristic Evaluation (HE). This paper focuses on understanding the effectiveness of HE as a methodology for defect detection. The results show the effectiveness of the HE as a usability testing methodology in capturing defects and prioritizing development and design efforts. The results also reinforce the need for integrating traditional heuristics with modified heuristics customized to the domain or field of the project being tested such as E-Government.

A Comparison of Usability Testing Methods for an E-Commerce Website: A Case Study on a Malaysia Online Gift Shop

Customers perceived trust towards an e-commerce Web site is crucial for the success of online business. Effective design of Web interfaces increases perceived trust of customers. Given many associated usability issues when performing tasks on a Web site, it is important for technopreneurs embarking on online business to understand issues related to usability problems of an e-commerce Web site and the techniques to identify these issues. In this study, usability evaluation was performed on an online gift shop with a group of potential consumers with age range of 18-22. Four different evaluation methods were used: Feedback Capture after Task (FCAT), Retrospective Think Aloud (RTA), Retrospective Think Aloud with Eye Movement (RTE) and observation. This study reveals that the major defects found were language and content, user guidance and support, flexibility and control, and visual clarity. Therefore, this study suggest that applying good user interface design could provide better user experience and thus increase perceived trust and user satisfaction towards the Web site.

Website user experience (UX) testing tool development using Open Source Software (OSS)

Website usability and user experience (UX) are crucial measures of website quality. The challenges faced in manual usability testing gave rise to the growth of usability assessment tools in the market. As these tools may be costly, it is important for usability practitioners to justify the benefits and cost of these tools. Hence, the objective of this study was to first gather the requirements for website UX testing. This is followed by developing an Open Source Software (OSS) called Ultimate Reliable and Native Usability System (URANUS) to support most of the techniques used for usability test. The final objective was to benchmark URANUS against existing tools. The methodologies employed during requirements gathering included literature review and interviews with target audience. This was followed by development of URANUS using the Rapid Application Development (RAD). Based on the LBUT and RUT comparisons URANUS turned out to score relative high in terms of benefits with low cost of ownership.

Hybrid Usability Methodology: Integrating Heuristic Evaluation with Laboratory Testing across the Software Development Lifecycle

The reviews of literature have shown that many website usability problems are found late in the Software Development Life Cycle (SDLC). This is due to the usage of traditional usability testing techniques which are not sufficient and suitable with the growing complexity of websites and constraints faced by usability practitioners. For instance, the Lab Based Usability Testing (LBUT) is expensive and has lesser coverage than Exploratory Heuristics Evaluation (EHE) while the EHE is subjected to false alarms. A hybrid usability methodology (HUM) comprising of LBUT and EHE is proposed. Six experiments involving EHE and LBUT were performed at the early, intermediate and advanced stages of the SDLC of websites, during which the optimal relative performance of each method were measured using the dependent variables followed by the design of a HUM. To validate the HUM, four case studies were conducted, during which significant improvements were observed in website effectiveness and efficiency. Based on the findings, HUM is a feasible approach for usability practitioners and also provides stakeholders a validated situational decision making framework for usability testing strategies taking into account real world constraints.

Enhancing the Effectiveness of Usability Evaluation by Automated Heuristic Evaluation System

Usability defects test escapee can have a negative impact on the success of software. It is quite common for projects to have a tight timeline. For these projects, it is crucial to ensure there are effective processes in place. One way to ensure project success is to improve the manual processes of the usability inspection via automation. An automated usability tool will enable the evaluator to reduce manual processes and focus on capturing more defects in a shorter period of time. Thus improving the effectiveness of the usability inspection and minimizing defects escapee. There exist many usability testing and inspection methods. The scope of this paper is on the Heuristic Evaluation (HE) procedures automation. The Usability Management System (UMS) was developed to automate as many manual steps as possible throughout the software development life cycle (SDLC). It is important for the various teams within the organization to understand the benefits of automation. The results show that with the help of automation more usability defects can be detected. Hence, enhancing the effectiveness of usability evaluation by an automated Heuristic Evaluation System is feasible.

Measuring Malaysian M-commerce user behaviour

M-Commerce is expected to play an even larger role, with over half a billion customers following the trend to shop via mobile devices by 2016. According to the Malaysian Ministry of Tourism and Culture, tourism has been identified as one of the twelve National Key Economic Area to drive the countrys growth. This gave rise to this research objective which is to determine the attention ranking of Malaysians while booking a hotel using a mobile devise. Lab Based Usability Testing with eye tracking technology was carried out with Malaysians to achieve this objective. Results revealed that Malaysian are highly attracted to both the “booking” and “price” information, followed by the “ratings”, “introduction” and lastly the “title” of the hotel, which is in contrast to other countries.

Measuring public value UX-based on ISO/IEC 25010 quality attributes: Case study on e-Government website

e-Government can be simplified as the use of electronic technology by government to provide information and services to the citizens. In theory, it shall be one of the effective ways for government to reach and interact with the citizens. In line with these, the usability and user experience of e-government website should meet or exceed citizens need. This first objective of this study is to measure user experience of citizen using a jobs-seeking website managed by the Malaysian government. The next objective is to determine suitable UX methodologies to measure the same. The ISO/IEC 25010 quality characteristics have been applied as the guidelines to measure the usability and user experience of the local jobs portal. This paper is written to address the user experiences issues of the registration process which is one of the common tasks to be completed as an attempt to obtain account in a job-seeking website service provided by the government. User experience test conducted with undergraduate users have shown various usability issues. Thus, recommendations obtain and analysis of the results shall be useful to guide the improvement of current version of jobs portal to better serve the good cause of its existence.

Combining eye gaze and hand tracking for pointer control in HCI: Developing a more robust and accurate interaction system for pointer positioning and clicking

This paper describes a method to combine the use of eye gaze and hand tracking to provide a more robust and accurate way to control a pointer in a windows, icons, menus, and pointer (WIMP) graphic user interfaces (GUI). In the proposed interaction system, the eye gaze is used to move a rectangular overlay called ‘area of interaction’ (AOI), on a computer screen. The AOI defines the boundary for pointer positioning. Hand fingertip detection is used to position the pointer. The presence of a second hand then triggers a click event. We tested the performance of the method against a standalone hand tracking interaction. Our system of combining the two types of tracking was found to be more accurate and experienced less error than using hand tracking method alone. The system is able to utilize the speed of eye gaze while overcoming its lack of accuracy by switching to hand tracking for more accurate pointing.

Eyes on OSH Usability Testing with Eye-Tracking and User Preference Rating

Occupational Safety and Health (OSH) is significant for all personnel engaged in work or employment. OSH websites is as well important as it should be relevant to educate the workers to be aware of any hazard or inconvenience act that may jeopardize their health and safety in workplace. This study is conducted to evaluate three OSH websites which are DOSH, NIOSH and OSHA using eye-tracker equipment to collect usability data. This paper is an extended version of our previous submitted paper which reported the details of methodology and results of usability testing conducted for these three websites. This paper reports the satisfaction results obtained from users subjective preferences. The satisfaction data is represented into rating percentage and word clouds. This paper also concludes the results of OSH website usability in terms of user satisfaction and preferences with recommendations on further research on usability for OSH domain.

A Textual Feedback Tool for Empowering Participants in Usability and UX Evaluations

ABSTRACT The usability movement has historically always sought to empower end-users of computers so that they understand what is happening and can control the outcome. In this article, we develop and evaluate a “Textual Feedback” tool for usability and user experience (UX) evaluation that can be used to empower well-educated but low-status users in UX evaluations in countries and contexts with high power distances. The proposed tool contributes to the Human–Computer Interaction (HCI) community’s pool of localized UX evaluation tools. We evaluate the tool with 40 users from two socio-economic groups in real-life UX usability evaluations settings in Malaysia. The results indicate that the Textual Feedback tool may help participants to give their thoughts in UX evaluation in high power distance contexts. In particular, the Textual Feedback tool helps high status females and low status males express more UX problems than they can with traditional concurrent think aloud (CTA) alone. We found that classic concurrent think aloud UX evaluation works fine in high power contexts, but only with the addition of Textual Feedback to mitigate the effects of socio-economic status in certain user groups. We suggest that future research on UX evaluation look more into how to empower certain user groups, such as low status female users, in UX evaluations done in high power distance contexts.