Claudia Ziegler Acemyan

The Relationship Between Trust and Usability in Systems

There is a large body of literature on trust, which focuses on interpersonal relationships. In recent years the scope of this research has expanded, as some studies have begun to explore trust between people and systems. Trust in these systems is critical, because if someone does not trust a system they will not use it. Another factor that heavily influences system use is usability, which has been extensively studied in human factors research. The goal of the research presented here was to expand the current understanding of usability by exploring its relationship to trust in two contexts: (1) popular consumer products that people can choose to use and (2) voting systems that citizens must use if they participate in an election. In the first context the relationship between usability and trust was explored across fourteen popular consumer products. In the second, the manner in which trust changed across three paper-ballot voting methods, each characterized by a different level of usability, was examined. In both studies it was found that the more usable systems were also the more trusted systems. At the same time, the relationship between the two factors differed in each context, implying other variables likely influenced it.

The Impact of Geographic Location on the Subjective Assessment of System Usability

ABSTRACT Marketing researchers use geography to identify specific user groups for studies to more effectively describe their potential customer base. Since usability professionals often recruit users employing similar selection criteria as their marketing peers, the use of geographic information might also be relevant when selecting usability test participants. In total, 3,168 participants from across the United States rated the usability of different hardware, software, and web-based products using the System Usability Scale (SUS). SUS scores were compared across geographic divisions to determine if usability assessments differ by location. SUS scores were also compared across rural and urban areas to determine if usability assessment scores change with population density. There was a lack of evidence to support significant differences in usability scores across both US geographic areas and zones of population density. The findings suggest that people make similar system usability assessments regardless of the area of the United States in which they live.

Does the Polling Station Environment Matter? The Relation Between Voting Machine Layouts Within Polling Stations And Anticipated System Usability:

This study sought to understand if voting equipment layouts impacted anticipated voting system usability. Thirty-five participants viewed photorealistic rendered images of twelve different voting system configurations within a polling station. After viewing each condition, participants completed a survey to assess anticipated system usability and related measures. Results showed that spatial configurations of voting machines inside a polling station impacted all subjective ratings. Anticipated usability was highest when the voting systems had dividers around the interface, there was space between units, and polling booths were arranged in the room such that voters did not face one another. This study demonstrates that environmental variables might need to be considered when configuring polling stations in order to maximize subjective assessments, like anticipated usability, that could impact future system use.

Do Voters Really Fail to Detect Changes to Their Ballots? An Investigation of Ballot Type on Voter Error Detection

Data from previous research that investigates whether or not voters fail to notice changes to their electronic ballot selections indicate that high percentages of people do not notice if their votes were altered, even after inspecting a final review screen. This paper questions if voters really do not notice changes as the previous studies found, or if the findings are an artifact of the types of races and propositions that they vote for in the experimental mock elections. Specifically, previous research studies used ballots that were comprised of fabricated candidates and propositions, which do not reflect the types of real-world issues presented on a ballot during an actual election. To investigate the issue, this study replicated the prior research, but introduced into the experiment a new type of ballot that was based on actual people and issues that the participant population was familiar with and felt was relevant to their current circumstances. Our findings reveal that the type of races presented to participants on a ballot in a mock election does not impact the ability of users to notice altered selections on a vote verification review screen. This suggests that the review screens currently implemented in electronic voting technologies might not be effective since user mistakes and potential fraud cannot be reliably identified by the voter before a vote is cast and then counted in the election tally.

Using the SUS: Lessons Learned & Forward Looking Research

This panel will discuss the System Usability Scale. Panelists all have extensive experience using the SUS within a broad range of contexts: diverse people (e.g., abilities, languages); different types of products; and different testing scenarios. Members of the audience will have the opportunity to ask questions about new research on the validity of the SUS in different environments as well as about lessons learned from practitioners using it to evaluate commercial products. Topics of specific interest to the authors are detailed within this paper.

2FA Might Be Secure, But It’s Not Usable: A Summative Usability Assessment of Google’s Two-factor Authentication (2FA) Methods

Computer security experts recommend that people use two-factor authentication (2FA) on password protected systems to help keep hackers out. Providing two pieces of information to verify a person’s identity adds extra security to an account. However, it is not clear if the added security and procedures impact system usability. This paper aims to answer this question by assessing per ISO 9241-11’s suggested measurements the usability of Google’s optional 2FA methods. We found few differences across four different 2FA methods when comparing efficiency, effectiveness and satisfaction measures—illustrating that one method is not necessarily more or less usable then another. Overall, the measures indicated that the systems’ usability needed to be improved, especially with regard to the initial setup of 2FA. In conclusion, developers need to focus more attention on making 2FA easier and faster to use, especially since it is often optional for password users, yet makes accounts significantly more secure.

Assessing the Usability of the Hart InterCivic eSlate During the 2016 Presidential Election

For the first time, the system usability of the eSlate–one of the most ubiquitous electronic voting machines used in US elections–was assessed in the field on election day 2016 using ISO 9241-11 metrics. Effectiveness and satisfaction measures were collected immediately after voters used the eSlate to vote at their assigned polling location. The findings showed that the perceived usability of the system in the polling station was judged to be quite good, despite previous research that indicated usability difficulties with the voting system. It is hypothesized that voters were making retrospective assessments of not just the eSlate, but also the supporting systems and environment in which they used it on election day. In particular, voters were likely taking into account their entire experience—including the help of poll workers and circumstances like short lines—versus just their experience with the machine itself. Practitioners and researchers should be aware that a system might not be completely usable but still be subjectively rated as usable due in part to other supporting systems and environmental variables.

Can Voters Tell When Their Voting Method Is Secure? Effects of End-to-end Security and Security Theater on Perceptions of Voting Systems:

The aim of this study was to understand how voters’ perceptions of voting system security might be impacted by the addition of actual security features and fake security features (i.e., security theater) to a voting method. In the context of voting, positive perceptions of security might be more important than security itself because voters must feel voting systems are safe enough to keep their votes secure and anonymous, or else they might choose to avoid participating in an election—potentially impacting election outcomes. To investigate this problem, 90 Rice University undergraduate students participated in a mock election using one of three systems, each associated with differing levels of security: a standard paper ballot, a paper ballot enhanced by end-to-end (e2e) security mechanisms, and a paper ballot that included fake security mechanisms to give the feeling of enhanced security without actually making the system safer. After voting, each participant completed a survey that included questions about voting system security to understand if one system was perceived to be more secure than another. In this study, participants rated the paper voting method with advanced e2e security to be the most secure; both the traditional paper ballot and security theater ballot were found to be the least secure. In summary, the advanced security features positively impacted voters’ perceptions of system security—indicating that people accurately assessed the safety of the provided voting method. This finding is promising and suggests that during real elections, there is the potential for voters to assess accurately a system’s security and then act-on and make decisions based on this information.

The Relationship Between User Mouse-based Performance And Subjective Usability Assessments

Objective metrics, such as effectiveness and efficiency, are often considered to be the best website usability measurements. User performance metrics that can be collected remotely, such as mouse clicks and the distance the mouse has traveled show particular promise. However, no studies have demonstrated a direct relationship between subjective usability measures and these two objective user performance metrics. In this paper, thirty participants completed five different tasks of varying difficulty on a commercial website. Mouse clicks, the distance the mouse moved, success rates, and System Usability Scale (SUS) scores were collected for each task. Results showed that participants made fewer mouse clicks on tasks at which they were successful than on tasks they failed. Participants moved the mouse over twice as far on failed tasks as compared to successful tasks. The correlations between SUS scores and the two mouse-based measurements were remarkably strong.

To Verify or Not to Verify, That is The Question: Voters' use and expectations of voting verification methods

Recently developed secure, end-to-end voting systems are designed to allow voters to confidentially verify on the internet that their ballot selections are cast and counted correctly. In response, this research paper characterizes both voters’ desire to actually use vote verification methods and the format of verification that users expected. 767 registered voters participated in a 2012 pre-election poll. Respondents were queried about their confidence in current voting systems, if they would want to be able to verify the accuracy of their ballot after voting in an election, and the preferred verification format to be used in future voting methods. The findings demonstrate voters’ confidence in current voting systems could be improved, possibly by being able to verify that their votes were cast and counted accurately. Voters indicated that they would want to be able to check on their votes in future elections. In addition, there was not a single form of verification that was preferred by most respondents—meaning a verification form that fulfills all users’ expectations would need to be developed. In order to increase voter use and acceptance of secure, voter verifiable voting methods, these findings should be considered when developing the next generation of systems.