Robert S. Schulman

An empirical study of the object-oriented paradigm and software reuse

Little or no empirical validation exists for many of software engineerings basic assumptions. While some of these assumptions are intuitive, the need for scientific experimentation remains clear. Several assumptions are made about the factors affecting software reuse, and in particular, the role of the object-oriented paradigm. This paper describes the preliminary results of a controlled experiment designed to evaluate the impact of the object-oriented paradigm on software reuse. The experiment concludes that (1) the object-oriented paradigm substantially improves productivity, although a significant part of this improvement is due to the effect of reuse, (2) reuse without regard to language paradigm improves productivity, (3) language differences are far more important when programmers reuse than when they do not, and (4) the object-oriented paradigm has a particular affinity to the reuse process.

Human-computer interface development tools: a methodology for their evaluation

A comprehensive checklist-based methodology produces quantifiable criteria for evaluating and comparing human-computer interface development tools along two dimensions: functionality and usability. An empirical evaluation shows that the methodology which is in use in several corporate interface development environments, produces reliable (consistent) results

The Usability Problem Taxonomy: A Framework for Classificationand Analysis

Although much can be gained by analyzing usability problems, there is no overall framework in which large sets of usability problems can be easily classified, compared, and analyzed. Current approaches to problem analysis that focus on identifying specific problem characteristics (such as severity or cost-to-fix) do provide additional information to the developer; however, they do not adequately support high-level (global) analysis. High-level approaches to problem analysis depend on the developer / evaluators ability to group problems, yet commonly used techniques for organizing usability problems are incomplete and / or provide inadequate information for problem correction. This paper presents the Usability Problem Taxonomy (UPT), a taxonomic model in which usability problems detected in graphical user interfaces with textual components are classified from both an artifact and a task perspective. The UPT was built empirically using over 400 usability problem descriptions collected on real-world development projects. The UPT has two components and contains 28 categories: 19 are in the artifact component and nine are in the task component. A study was conducted showing that problems can be classified reliably using the UPT. Techniques for high-level problem analysis are explored using UPT classification of a set of usability problems detected during an evaluation of a CASE tool. In addition, ways to augment or complement existing problem analysis strategies using UPT analysis are suggested. A summary of reports from two developers who have used the UPT in the workplace provides anecdotal evidence indicating that UPT classification has improved problem identification, reporting, analysis, and prioritization prior to correction.

Experiments in social data mining: The TopicShop system

Social data mining systems enable people to share opinions and benefit from each others experience. They do this by mining and redistributing information from computational records of social activity such as Usenet messages, system usage history, citations, or hyperlinks. Some general questions for evaluating such systems are: (1) is the extracted information valuable? and (2) do interfaces based on the information improve user task performance? We report here on TopicShop, a system that mines information from the structure and content of Web pages and provides an exploratory information workspace interface. We carried out experiments that yielded positive answers to both evaluation questions. First, a number of automatically computable features about Web sites do a good job of predicting expert quality judgments about sites. Second, compared to popular Web search interfaces, the TopicShop interface to this information lets users select significantly more high-quality sites, in less time and with less effort, and to organize the sites they select into personally meaningful collections more quickly and easily. We conclude by discussing how our results may be applied and considering how they touch on general issues concerning quality, expertise, and consensus.

Ordinal Data: An Alternative Distribution.

To date, virtually all techniques appropriate for ordinal data are based on the uniform probability distribution over the permutations. In this paper we introduce and examine an alternative probability model for the distribution of ordinal data. Preliminary to deriving the expectations of Spearmans rho and Kendalls tau under this model, we show how to compute certain conditional expectations of rho and tau under the uniform distribution. The alternative probability model is then applied to ordinal test theory, and the calculation of true scores and test reliability are discussed.

A Test Theory Model for Ordinal Measurements.

A model is proposed for the description of ordinal test scores based on the definition of true score as expected rank. Derivations from the model are compared with results from clasiscal test theory as developed by Lord and Novick, in particular with respect to parallel tests and composites. An unbiased estimator of population true score from sample data is derived and its variance is shown to decrease with increasing sample size. Population reliability is shown to be analytically related to expected sample reliability, and methods of reliability estimation are discussed.

A Geometric Model of Rank Correlation

Abstract A plot of each ranking of N objects in N-dimensional space is shown to provide geometric interpretations of Kendalls tau and Spearmans rho and also of the relationship of rho to a sum of inversion weights. The computation of rho from a sum of inversion weights is shown to allow sequential calculation of rho.

Correlation and prediction in ordinal test theory

Based on the test theory model for ordinal measurements proposed by Schulman and Haden, the present paper considers correlations between tests, attenuation, regressions involving true and observed scores, and prediction of test reliability.The population correlation between tests is shown to be related to the expected sample correlation for samples of sizen1 andn2. Errors of estimation, measurement and prediction are found to be similar to their counterparts in interval test theory, while attenuation is identical to its counterpart. The bias in estimating population reliability from sample data is compared for Kendalls tau and Spearmans rho.

A Nested Analysis for Data Collected from Groups Making Crowding Research more Efficient

Traditionally, most crowding research involves the observation of subjects in groups formed either by manipulation in the laboratory or by natural factors in field studies. The conventional analysis of such data employs the group mean (or median) as the experimental unit and results in large numbers of individuals being observed to yield comparatively few degrees of freedom for the statistical tests of treatment effects. This paper examines the difference between dependent and independent responses and suggests that, for the case of independence, a nested analysis of variance model is appropriate. The advantages of this analytic approach are explained, and conditions are discussed under which more powerful test of treatment effects may be obtained.

Development and evaluation of a taxonomical model of behavioral representation techniques

A user-centered approach to interactive system development requires a way to represent the behavior of a user interacting with an interface. While a number of behavioral representation techniques exist, not all provide the capabilities necessary to support the interaction development process. Based on observations of existing representation techniques and comments from users of the User Action Notation (UAN), a userand task-centered behavioral representation technique, we have developed a taxonomical model of behavioral representation techniques. Our model is an epistemological framework for discussing, analyzing, extending, and comparing existing behavioral representation techniques, as well as being a springboard for developing and evaluating new techniques. We present the model and results of our evaluation demonstrating the model’s reliability y and utility within the context of behavioral representation techniques. PROBLEM STATEMENT AND CONTEXT It has been shown repeatedly that traditional software engineering methods do not necessarily lead to high usability when applied to the development of user interfaces (e.g. 1,2, 3). This is reasonable since the focus of these methods is the software, not the user. To achieve high usability, the interface development process should be user-centered; i.e., it should focus on the user’s tasks, needs, and behavior while interacting with the system. This view, referred to as the behavioral view (4), has led to a variety of techniques for representing the design of a user interface in terms of the behavior of the user, independently of user interface software and hardware considerations. One such technique, the User Action Notation (UAN) (4, 5), is a userand task-oriented notation that describes the behavior of a user and an interface during their cooperative performance of a task. Other behavioral representation Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, raquires a fee and/or specific permission. CH194-4/94 Boston, Massachusetts USA a 1994 ACM 0-89791 -650 -619410159 . ..