Christos Katsanos

Perceived usability evaluation of learning management systems: Empirical evaluation of the System Usability Scale

Perceived usability affects greatly student’s learning effectiveness and overall learning experience, and thus is an important requirement of educational software. The System Usability Scale (SUS) is a well-researched and widely used questionnaire for perceived usability evaluation. However, surprisingly few studies have used SUS to evaluate the perceived usability of learning management systems (LMSs). This paper presents an empirical evaluation of the SUS questionnaire in the context of LMSs’ perceived usability evaluation. Eleven studies involving 769 students were conducted, in which participants evaluated the usability of two LMSs (eClass and Moodle) used within courses of their curriculum. It was found that the perceived usability of the evaluated LMSs is at a satisfactory level (mean SUS score 76.27). Analysis of the results also demonstrated the validity and reliability of SUS for LMSs’ evaluation, and that it remains robust even for small sample sizes. Moreover, the following SUS attributes were investigated in the context of LMSs evaluation: gender, age, prior experience with the LMS, Internet self-efficacy, attitude towards the Internet and usage frequency of the LMS.

Automated semantic elaboration of web site information architecture

Structuring of the content is an important step in web site design, affecting greatly navigability and the overall user experience. Automated support of this task is the object of this paper. AutoCardSorter, a computational tool that supports clustering of the web pages of a site, is introduced. The proposed tool-based methodology uses semantic similarity measures, such as latent semantic analysis, and hierarchical clustering algorithms, in order to suggest suitable information navigation schemes. In the paper, after introducing AutoCardSorter design and functionality, three independent studies are discussed. The studies, that were conducted in order to validate the proposal, compared the proposed method with the established card-sorting approach, in different domains. It was found that substantial gain in effectiveness was achieved without expense in the quality of results, therefore, reducing the required time and human resources.

AutoCardSorter: designing the information architecture of a web site using latent semantic analysis

In this paper, we describe an innovative tool that supports the design and evaluation of the information architecture of a Web site. The tool uses Latent Semantic Analysis and hierarchical clustering algorithms to provide optimal information navigation schemes in an automated manner. The proposed, tool-based, approach addresses the problem of reasonable content structuring, which established techniques such as card sorting also address. A real world case study depicted substantial effectiveness gain, without expense in the quality of results. We argue that such an approach could facilitate information-rich applications design, like most Web sites, by reducing time and resources required.

Perceived Usability Evaluation of Learning Management Systems: A First Step towards Standardization of the System Usability Scale in Greek

Perceived usability is an important requirement of educational software, affecting greatly students learning effectiveness and overall learning experience. Although the System Usability Scale (SUS) questionnaire is considered as a de facto standard for perceived usability evaluation, surprisingly, few studies are reporting results from evaluations related to Learning Management Systems (LMSs). In addition, given that the questionnaire comprises 10 questions presented in English, its applicability in studies involving speakers of other languages is questionable. In this paper, a first step towards standardization of a Greek version of SUS in the context of LMSs perceived usability evaluation is reported. To this end, three studies involving 280 university students in both blended and distance learning education were conducted. Analysis of the results demonstrated the validity and reliability of the Greek version of SUS.

Learning about web accessibility: A project based tool-mediated approach

Many websites remain inaccessible for people with disabilities, despite the availability of relevant guidelines and tools. This is mainly due to lack of appropriate training of Web designers on accessibility technology. In this paper, a project based learning activity designed to instruct Web accessibility guidelines and good design practices is presented. The activity is mediated by a web-based learning environment, which presents real-world examples of accessibility impasses that arise when certain, established guidelines are violated, and then provides advice on how to avoid or resolve them. The learning material contained in the tool is offered through a faceted browsing approach, thus enabling active exploration by the learner. A within-subjects case study compared the learning effectiveness of traditional academic instruction (pre-condition) with the proposed project based activity (post-condition) in the context of a University course. A significant improvement in students’ academic performance and perceived learning was found.

A hybrid adaptive routing protocol for ad hoc wireless networks

Ad hoc networks are characterized by dynamic topology caused by node mobility, multihop wireless connectivity and channel non-deterministic behavior (interference, multipath, hidden and exposed node problem make the wireless channel very difficult to predict). The behavior of ad hoc networks must be analyzed in detail as a result of the pairing of the selected MAC and routing protocols. We focus our studies in the routing layer while closely observing the developments in MAC layer. We present and examine the characteristics of a new hybrid adaptive routing protocol based on the ZRP protocol.

Usability Guidelines for the Design of Robot Teleoperation: A Taxonomy

This paper presents a taxonomy of design guidelines for robot teleoperation developed from a focused literature review of robot teleoperation. A list of user interface design guidelines was assembled, open card sorting and a focus group were used to classify them, and closed card sorting was employed to validate and further refine the proposed taxonomy. The initially obtained set of 70 guidelines is grouped into eight categories: platform architecture and scalability, error prevention and recovery, visual design, information presentation, robot state awareness, interaction effectiveness and efficiency, robot environment/surroundings awareness, and cognitive factors. Agricultural robots were used as an application case study for implementation and field evaluation. The proposed guideline taxonomy was used heuristically to evaluate the usability of an existing user interface of a teleoperated agricultural robot.

Evaluating User's Emotional Experience in HCI: the PhysiOBS Approach

As computing is changing parameters, apart from effectiveness and efficiency in human-computer interaction, such as emotion have become more relevant than before. In this paper, a new tool-based evaluation approach of user’s emotional experience during human-computer interaction is presented. The proposed approach combines user’s physiological signals, observation data and self-reported data in an innovative tool (PhysiOBS) that allows continuous and multiple emotional states analysis. To the best of our knowledge, such an approach that effectively combines all these user-generated data in the context of user’s emotional experience evaluation does not exist. Results from a preliminary evaluation study of the tool were rather encouraging revealing that the proposed approach can provide valuable insights to user experience practitioners.

KLM Form Analyzer: Automated Evaluation of Web Form Filling Tasks Using Human Performance Models

Filling forms is a common and frequent task in web interaction. Therefore, designing web forms that enhance users’ efficiency is an important task. This paper presents a tool entitled KLM Form Analyzer (KLM-FA) that enables effortless predictions of execution times of web form filling tasks. To this end, the tool employs established models of human performance, namely the Keystroke Level Model and optionally the Fitts’ law. KLM-FA can support various evaluation scenarios, both in a formative and summative context, and according to different interaction strategies or modeled users’ characteristics. A study investigated the accuracy of KLM-FA predictions by comparing them to participants’ execution times for six form filling tasks in popular social networking websites. The tool produced highly accurate predictions (89.1% agreement with user data) in an efficient manner.

Stress in interactive applications: analysis of the valence-arousal space based on physiological signals and self-reported data

Measuring users’ emotional reaction to interactive multimedia and hypermedia is important. One particularly popular self-reported method for emotion assessment is the Valence-Arousal (VA) Scale: a 9 × 9 affective grid. This paper aims to identify specific stress region(s) in the VA space by combining self-reported ratings (pairs of VA) and physiological signals (skin conductance). To this end, 31 healthy volunteers participated in an experiment by performing five stressful interaction tasks while their skin conductance was monitored. The selected interaction tasks were most frequently listed as stressful by a separate group of 15 interviewees. After each task, participants expressed their perceived emotional experience using the VA rating space. Our findings show which regions in the VA rating space may reliably indicate self-reported stress that is in alignment with one’s measured skin conductance while using interactive applications. One additional important contribution of this work is the proposed approach for the empirical identification of affect regions in the VA space based on physiological signals.