Ulf-Dietrich Reips

Interval-level measurement with visual analogue scales in Internet-based research: VAS Generator

The present article describes VAS Generator (www.vasgenerator.net), a free Web service for creating a wide range of visual analogue scales that can be used as measurement devices in Web surveys and Web experimentation, as well as for local computerized assessment. A step-by-step example for creating and implementing a visual analogue scale with visual feedback is given. VAS Generator and the scales it generates work independently of platforms and use the underlying languages HTML and JavaScript. Results from a validation study with 355 participants are reported and show that the scales generated with VAS Generator approximate an interval-scale level. In light of previous research on visual analogue versus categorical (e.g., radio button) scales in Internet-based research, we conclude that categorical scales only reach ordinal-scale level, and thus visual analogue scales are to be preferred whenever possible.

Chapter 4 – The Web Experiment Method: Advantages, Disadvantages, and Solutions

Publisher Summary This chapter discusses the pros and cons of the experimenting methods of e-research in order to understand why conducting Web experiments are an opportunity for research science. To analyze the contemporary methods it is important to know the traditional approaches and their flaws. However, the current researches have an advantage of generalizability, volunteer bias, detectablity of motivational confounding, and other advantages like cost. Openness is one of the fundamental principles of science and this can be achieved in a much better way through Web experiments than in laboratory experiments. Traditional experiments may contain features that are not described in the method section that may turn out to be important but public Web experiments are openly accessible and can remain indefinitely on the World Wide Web for documentation purposes. Therefore, the Web experimental method opens the door to research areas that were almost inaccessible for established methods.

A Brief History of Web Experimenting

Publisher Summary This chapter discusses the history of Web-based research experiments. The computerized experimenting was first introduced in 1970s. It revolutionized traditional laboratory research with its attractive new features such as standardized and controlled presentation of stimuli, item-branching capabilities, immediacy of data entry, elimination of missing responses, elimination of transcription costs and errors, and accurate measurements of response times. Today, most human experimental research in psychology is aided by computer automation. Usage of extending computerized experimenting beyond single PCs, local computer networks is collecting data rapidly and programs are being written in high-level languages such as C ++ , Pascal, or Delphi, or with program packages such as Super-Lab, Psy-Scope and MEL to optimize the data collecting tools in order to get quality data. Although, at the moment, the number of Web experiments is still small, a rapid growth can be predicted on the basis of the present result.

Personalization, authentication and self-disclosure in self-administered Internet surveys

Abstract Two studies are presented that examine disclosure of sensitive information and personalization in Internet-based surveys. In the first study, the impact of a personalized salutation on two forms of non-disclosure to a sensitive personal question (salary level) is tested. The results revealed that a personalized salutation tends to increase levels of active non-disclosure (measured through use of an ‘I prefer not to answer’ option), but not passive non-disclosure (where the respondent selects no option). In the second study, participants are directed to the study via either a personalized URL (which incorporated an encoded identifier not obvious as such to the responder) or via a secure log-on page that required the user to type in identifying information. Non-disclosure to a sensitive question (salary) was significantly higher when participants went through a log-on procedure. We suggest that this pattern of non-disclosure to sensitive questions reflects increases in identifiability when a personalized salutation or log-on procedure is used. We further suggest that the provision of an active non-disclosure option to a sensitive question is particularly appropriate in contexts in which anonymity may be compromised, since it enables participants to both protect their privacy and respond appropriately to the survey.

The Web Experimental Psychology Lab: Five years of data collection on the Internet

In fall 1995, the worldwide-accessible Web Experimental Psychology Lab (http://www.genpsylab. unizh.ch) opened its doors to Web surfers and Web experimenters. It offers a frequently visited place at which to conduct true experiments over the Internet. Data from 5 years of laboratory running time are presented, along with recommendations for setting up and maintaining a virtual laboratory, including sections on the history of the Web laboratory and of Web experimenting, the laboratory’s structure and design, visitor demographics, the Kids’ Experimental Psychology Lab, access statistics, administration, software and hardware, marketing, other Web laboratories, data security, and data quality. It is concluded that experimental data collection via the Internet has proven to be an enrichment to science. Consequently, the Web Experimental Psychology Lab will continue and extend its services to the scientific community.

Personalized salutation, power of sender and response rates to Web-based surveys

Three studies were conducted to examine the effect of personalized salutation and sender power on signing up to an online survey panel, and subsequent survey response rates. In the first study, significantly more people joined a panel if addressed by a personalized salutation. In Study 2, this effect was replicated using an invitation to leave a second panel. In the final study, a significant salutation effect was found when power of the sender was high, and not when power of the sender was neutral. It is argued that for this sample, power of audience and participant identifiability linked to create a compliance-based motivation to join and maintain membership of an online panel. Implications for the maintenance of online panels, survey response rates, and the collection of sensitive personal information, are discussed.

WEXTOR: A Web-based tool for generating and visualizing experimental designs and procedures

WEXTOR is a Javascript-based experiment generator and teaching tool on the World-Wide Web that can be used to design laboratory and Web experiments in a guided step-by-step process. It dynamically creates the customized Web pages and Javascripts needed for the experimental procedure and provides experimenters with a print-ready visual display of their experimental design. WEXTOR flexibly supportscomplete and incomplete factorial designs withbetween-subjects, within-subjects, andquasi-experimental factors, as well as mixed designs. The software implements client-side response time measurement and contains a content wizard for creating interactive materials, as well as dependent measures (graphical scales, multiple-choice items, etc.), on the experiment pages. However, it does not aim to replace a full-fledged HTML editor. Several methodological features specifically needed in Web experimental design have been implemented in the Web-based tool and are described in this paper. WEXTOR is platform independent. The created Web pages can be uploaded to any type of Web server in which data may be recorded in logfiles or via a database. The current version of WEXTOR is freely available for educational and noncommercial purposes. Its Web address is http://www.genpsylab.unizh.ch/wextor/index.html.

What are participants doing while filling in an online questionnaire: A paradata collection tool and an empirical study

The use of online questionnaires is rapidly increasing. Contrary to manifold advantages, not much is known about user behavior that can be measured outside the boundaries set by standard web technologies like HTML form elements. To show how the lack of knowledge about the user setting in web studies can be accounted for, we present a tool called UserActionTracer, with which it is possible to collect more behavior information than with any other paradata gathering tool, in order to (1) gather additional data unobtrusively from the process of answering questions and (2) to visualize individual user behavior on web pages. In an empirical study on a large web sample (N=1046) we observed and analysed online behaviors (e.g., clicking through). We found that only 10.5% of participants showed more than five single behaviors with highly negative influence on data quality in the whole online questionnaire (out of 132 possible single behavior judgments). Furthermore, results were validated by comparison with data from online address books. With the UserActionTracer it is possible to gain further insight into the process of answering online questionnaires.

Why Semantic Differentials in Web-Based Research Should be Made From Visual Analogue Scales and Not From 5-Point Scales

In a web experiment, participants were randomly assigned to two semantic differentials either made from discrete 5-point ordinal rating scales or from continuous visual analogue scales (VASs) with 250 gradations. Respondents adjusted their ratings with VASs more often to maximize the precision of answers, which had a beneficial effect on data quality. No side effects like differences in means, higher dropout, more nonresponse, or higher response times were observed. Overall, the combination of semantic differentials and VASs results in a number of advantages. Potential for further research is discussed.

TheWeb Experiment List : a web service for the recruitment of participants and archiving of Internet-based experiments

TheWeb Experiment List (http://genpsylab-wexlist.unizh.ch/), a free Web-based service for the recruitment of participants in Internet-based experiments, is presented. The Web Experiment List also serves as a searchable archive for the research community. It lists more than 250 links to and descriptions of current and past Web experiments. Searches can be conducted by area of research, language, type of study, date, and status (active vs. archived). Data from log file analyses reveal an increasing use of the Web Experiment List and provide a picture of the distribution of the use of the Web experiment method across disciplines. On a general theoretical note, Web services are discussed as a viable software alternative to the traditional program format.