Barbara D. Klein

User evaluations of IS as surrogates for objective performance

User evaluations of information systems are frequently used as measures of MIS success, since it is extremely difficult to get objective measures of system performance. However, user evaluations have been appropriately criticized as lacking a clearly articulated theoretical basis for linking them to systems effectiveness, and almost no research has been found that explicitly tests the link between user evaluations of systems and objectively measured performance. In this paper, we focus on user evaluations of task-technology fit for mandatory use systems and develop theoretical arguments for the link to individual performance. This is then empirically tested in a controlled experiment with objective performance measures and carefully validated user evaluations. Statistically significant support for the link is found for one measure of performance but not for a second. These findings are consistent with others which found that users are not necessarily accurate reporters of key constructs related to use of IS, specifically that self reporting is a poor measure of actual utilization. The possibility that user evaluations have a stronger link to performance when users receive feedback on their performance is proposed. Implications are discussed.

Can humans detect errors in data? Impact of base rates, incentives, and goals

There is strong evidence that data items stored in organizational databases have a significant rate of errors. If undetected in use, those errors in stored data may significantly affect business outcomes. Published research suggests that users of information systems ~Robert Zmud was the accepting senior editor for this paper. ISRL Categories: AD05, BG03, HC0201

Data quality in neural network models: effect of error rate and magnitude of error on predictive accuracy

Neural networks have been applied in a wide variety of business domains. Although databases used in many organizations have been found to contain errors, little is known about the effect of these errors on predictions made by neural network models. The article uses a real-world example, the prediction of the net asset values of mutual funds, to investigate the effect of data quality on neural network models. The results of two experiments are reported. The first experiment shows that the error rate (ranging from 25 to 100%) and magnitude of error (5 and 10%) in data used in model prediction affect the predictive accuracy of neural networks. The second experiment shows that the error rate (ranging from 5 to 20%) and the magnitude of error (5 and 10%) in data used to build the model affect the predictive accuracy of neural networks. The findings have managerial implications for users and builders of neural networks working with databases containing errors.

New complexity measures for the facility layout problem: an empirical study using traditional and neural network analysis

Abstract The most frequent application of the quadratic assignment problem has been in facility layout, which is concerned with locating the activities or departments of an organization in a fixed set of locations such that those activities with the strongest interrelationships are closest to each other. Early attempts to explain the varying performance of optimal and heuristic solution procedures on different layout problems focused on a univariate complexity measure such as the coefficient of variation. Recently, a multivariate measure for assessing the complexity of the quadratic assignment problem (QAP) formulation of the facility layout problem has been proposed. This paper introduces a new thresholding procedure for capturing critical relationship information, expands the eight components of the multivariate complexity measure by adding two new components, demonstrates a new solution value bound and experimentally determines the effect on solution quality of using either one of two heuristic solution procedures versus an optimal procedure. This result is important because it enables layout planners to determine in advance the likelihood of obtaining an optimal solution. It also tells them which problem characteristics are important and their relative weights in determining the solution outcome.

An Examination of the Effects of Flow on Learning in a Graduate-Level Introductory Operations Management Course.

The authors investigated the effects of flow on learning outcomes in a graduate-level operations management course. Flow was assessed through an overall flow score, four dimensions of flow, and three characteristics of flow activities. Learning outcomes were measured objectively through multiple-choice quiz scores and subjectively using measures of students’ perceived learning of the subject matter, students’ perceived skill development, and student satisfaction. The findings show that flow affected students’ perceived learning of the subject matter and student satisfaction but did not affect learning performance as measured through multiple-choice quizzes. Partial support is found for an effect of flow on students’ perceived skill development.

Detecting errors in data: clarification of the impact of base rate expectations and incentives

Organizational databases have a significant rate of data errors and detecting and correcting these errors can be problematic. This paper builds on a stream of research demonstrating that users of these databases can detect data errors under certain circumstances. A theory of error detection and research on the effect of base rate expectations in probabilistic judgement tasks are applied to the development of two propositions about error detection. It is argued that expectations about the base rate of errors in data affect error detection performance when they are developed through direct experience and that incentives affect error detection performance. The two research propositions are tested in a laboratory experiment. Experience-based expectations about the base rate of errors and incentives are found to affect error detection performance.

The detection of data errors in computer information systems: field interviews with municipal bond analysts

There is strong evidence that data stored in organizational databases have a significant rate of errors. As computerized databases continue to proliferate, the number of errors in stored data and the organizational impact of these errors are likely to increase. The impact of data errors on business processes and decision making can be lessened if users of information systems are able and willing to detect and correct data errors. However, some published research suggests that users of information systems do not detect data errors. This paper reports the results of a study showing that municipal bond analysts detect data errors. The results provide insight into the conditions under which users in organizational settings detect data errors. Guidelines for improving error detection are also discussed

Data quality in linear regression models: Effect of errors in test data and errors in training data on predictive accuracy

Introduction There is strong evidence (e.g., Laudon, 1986; Morey, 1982; Redman, 1992, 1995, 1996) that data stored in organizational databases have a significant rate of errors. The effect of data errors on the outputs of computer-based models has been investigated by a number of researchers (e.g., Ballou and Pazer, 1985; Ballou et al., 1987; Bansal et al., 1993). This investigation builds on this prior research by examining the effect of data quality on linear regression models. A financial application of a linear regression model is used to examine this question. Data errors may affect the predictive accuracy of linear regression models in two ways. First, the training data used to build the model may contain errors. Second, even if training data are free of errors, once a linear regression model is used for forecasting a user may input test data containing errors to the model. In general, when claims about the predictive accuracy of linear regression models are made, it is assumed that data used to train the models and data input to make predictions are free of errors. In this study we relax this assumption by asking two questions: (1) What is the effect of errors in test data on predictions made using linear regression models? and (2) What is the effect of errors in training data on predictions made using linear regression models? The first question is focused on the effect of data errors when the model is used for forecasting. The second question is focused on the effect of data errors during model construction. An understanding of the effect of data errors on linear regression models is particularly important because the availability of inexpensive software packages for personal computers makes the development and use of linear regression models by end-users feasible. Researchers have argued that end-user computing has increased the potential for data errors in computer applications (Boockholdt, 1989). As end users develop applications, it is possible that fewer data validation methods such as logic tests and control totals will be in place and it is likely that less rigorous testing will occur before applications are used in production (Corman, 1988; Davis, 1984; Davis et al., 1983; Panko, 1998). The remaining sections of this paper present (1) a review of relevant prior research on data quality, (2) a brief explanation of linear regression models, (3) a description of the linear regression models constructed in the study, (4) a discussion of the methodology of two experiments, (5) the results of two experiments and (6) conclusions. Background Data quality is generally recognized as a multidimensional concept (Wand and Wang, 1996; Wang and Strong, 1996). While no single definition of data quality has been accepted by researchers working in this area, there is agreement that data accuracy, currency, completeness, and consistency are important areas of concern (Agmon and Ahituv, 1987; Ballou and Pazer, 1985; Davis and Olson, 1985; Fox et al., 1993; Huh et al., 1990; Madnick and Wang, 1992; Wand and Wang, 1996; Wang and Strong, 1996; Zmud, 1978). This investigation adopts the conceptualization of data quality proposed by Ballou and Pazer (1985) that includes four dimensions: accuracy, timeliness, completeness, and consistency. This study is primarily concerned with data accuracy, defined as conformity between a recorded data value and the corresponding actual data value. Prior research has found that organizational databases are not in general free of errors (e.g., Laudon, 1986; Morey, 1982; Redman, 1992, 1995). Between one and twenty percent of data items in critical organizational databases are estimated to be inaccurate (Laudon, 1986; Madnick and Wang, 1992; Morey, 1982; Redman, 1992). Data quality problems have been found to affect the accuracy and timeliness of economic data published by the United States government (Hershey, 1995; Morgenstern, 1963). …

The case of flow and learning revisited

ABSTRACT Many business schools are criticized for being ineffective in helping students learn proper management skills and knowledge. Flow theory has been cited as being helpful in many learning environments in that flow experience can enhance student learning. The authors conducted a study of 315 students in an undergraduate operations management (OM) class to assess learning outcomes and flow experience. Results show that student learning performance and flow are related. Implications and suggestions for further research are provided.

Analysis of structure importance of compensation capacitor in jointless track circuit

This paper models the input signal amplitude of the main track and the small track of the adjacent jointless track circuit (JTC) when JTC is idle and the track circuit reader(TCR) received signal amplitude when JTC is occupied, based on the work mechanism of JTC and TCR. Based on the models, the relative impact of compensation capacitor on signal amplitude is obtained by simulation. The paper further proposes a calculation method for structure importance of compensation capacitors. Experimental results indicate that the rankings of structure importance are not affected by ballast resistance of JTC in this method. The results also show that the compensation capacitors closer to the receiving end are more important than those closer to the sending end. In addition, C2, C6, and C3 closer to receiving end are the most important and should be paid close attention during maintenance. The second, the first and the fifth capacitor from the sending end, have less impact on the JTC and TCR signal. This paper is helpful to determine the maintenance priority of each capacitor, optimize the maintenance strategy, and make better use of JTC.