Iris Vessey

Cognitive Fit: An Empirical Study of Information Acquisition

From a broad perspective, our research can be viewed as investigating the fit of technology to task, the users view of the fit between technology and task, and the relative importance of each to problem-solving or decision-making performance. The technology investigated in this research is the mode of information presentation. Although there has been a considerable amount of research into problem solving using graphs and tables, until recently the circumstances in which each is more effective have been largely unresolved. Recent research has suggested that performance benefits accrue when cognitive fit occurs, i.e., when factors such as the problem representation and problem solving tools match the characteristics of the task. In this paper, we investigate the effects of the basic paradigm of cognitive fit and extensions to the paradigm in a laboratory experiment that examined the nature of subjects mental representations as well as problem-solving performance. The experiment, using 128 MBA students in ...

The effect of user involvement on system success: a contingency approach

Despite the fact that commercial computer systems have been in existence for almost three decades, many systems in the process of being implemented may be classed as failures. One of the factors frequently cited as important to successful system development is involving users in the design and implementation process. This paper reports the results of a field study, conducted on data from forty-two systems, that investigates the role of user involvement and factors affecting the employment of user involvement on the success of system development. Path analysis was used to investigate both the direct effects of the contingent variables on system success and the effect of user involvement as a mediating variable between the contingent variables and system success. The results show that high system complexity and constraints on the resources available for system development are associated with less successful systems.

Requirements specification: learning object, process, and data methodologies

pccifying information requir~mcnts-dcttr~i”i~g and documenting the requirements for an information system, is arguably the key to developing successful information systems (IS). Since this is the first step in the systems development process, it is clear that not doing it effectively will have significant impact on both the effectiveness and the efficiency of the resultant system. Not getting the correct final system requirements initially is largely responsible for the cost and schedule overruns that are still fairly prevalent in IS development. Specifying information requirements is not only the most important step in developing IS, it is probably also the most difficult. The crucial aspect of the process is to develop a mental model ofthe system (i.e., to determine what the system needs to do) [6]. Once the information requirements ofthe new IS have been specified, the development process consists of successive transformations ofthat initial model until, finally, a computerbased solution to the problem is achieved. Specifying requirements is a complex task that is diff&lt for systems analysts to address, due to human prohlrrn-solving limitations [5]. Short-term memory is limited in capacity; transfer from short-term to long-term memory requires a relatively long time, approximately 5 to 10 seconds; storage and retrieval mechanisms are fallible. Short-term memory capacity is perhaps the most devastating aspect of these limitations. Classical estimates of short-term memory capacity suggested that problem solvers can effectively handle 7 ? 2 chunks of information [14], but this is an upper limit that may be drastically reduced by interference. To overcome these limitations, human problem solvers resort to various ways of effectively increasing the capacity of short-term memory as well as using external memory devices and/or techniques to support working memory. The aid most commonly used to support working memory in information requirements specification is a systems development methodology and the associated representation technique. By methodology, we mean a systematic approach to the task of systems development (see [Xl). Early methodologies provided guidelines mostly in the form ofa set of standardized activities and standardized forms to be completed; little effort was devoted to handling complexity explicitly. Subsequently, the major thrust ofmethodologies has been the desire to address complexity directly [6, 8, lo]. Despite the significance of information requirements specification to the successful design of systems and the large body of descriptive and conceptual literature on methodologies, there have been surprisingly few empirical studies conducted on the methodologies themselves. In fact, review of the literature revealed only one such study, by Yadav et al. [24], who investigated the effectiveness of data flow diagramming [8] and a variant of the systems analysis and design technique (SALT) [ZO] in specifying information requirements. They report that although data flow diagrams are easier to learn and to UT, neither produced significantly better specifications. Given the importance of methodologies to IS development and the dearth ofempirical evidence regarding their use, we conducted an exploratory study to investigate the performance ol three methodologies in aiding novice systems analysts who were learning to specify information requirements. The methodologies investigated were the structured techniques [6, 8, 251, Jackson System Development USD) [lo], and the object-oriented approach [3]. We investigated the structured techniques and JSD because they arc currently the most widely used struttured analysis methodologies (see Necco et al.) [15]. Since object-oriented approaches to systems development are receiving considerable attention in the software engineering arena and since some firms, such as Arthur Andersen Consulting, are moving toward replacing their current methods with object-oriented methods, it seemed appropriate to investigate the relative merits of an object-oriented approach in a business setting.

The effect of information presentation on decision making: a cost-benefit analysis

Cost-benefit theory is presented as a cohesive way of organizing knowledge about information presentation. This theory suggests that decision makers change strategy so that they minimize the joint cost of effort and error in making a decision. Here we apply two aspects of the theory. First, the theory of cognitive fit applies to decision making on fairly simple tasks of information acquisition and simple evaluation. Certain strategies (in this case problem-solving “processes”) will dominate alternative ones when the problem representation matches the nature of the decision-making task. The processes are effectively controlled, resulting in maximum accuracy at minimum cost. Second, the more traditional view is applied to decision making on more complex tasks, where a number of appropriate strategies may be available, with trade-offs between error and effort. Both aspects of the theory are tested by evaluating results of published graphs versus tables studies. The theory presented successfully explains the results of the majority of these studies. Further, the paper examines the implications of the cost-benefit framework for the design of decision support systems (DSS) and the notion of DSS restrictiveness. Rather than addressing the role of strategy alone in investigating the effectiveness of different display formats, researchers in behavioral decision making are encouraged to also address the role of problem-solving processes.

Learning to specify information requirements: the relationship between application and methodology

The systems development process involves establishing the information requirements of an application and successively transforming those requirements into a computer-based model of the application. Attention iS usually focused almost exclusively on the method of transformation, however, with little recognition of the role of the application. As a first step in examining the relevance of knowledge of the application to the systems development process, this study addresses whether there are synergistic effects of application and methodology knowledge in specifying information requirements. This was achieved via a repeated-measures protocol analysis study that manipulated both experience with the application and knowledge of the methodology. The results show that in learning to specify information requirements, novice analysts: • performed more effectively over time when trained to use a methodology applied the methodology more effectively when familiar with the application; • performed more effectively when they used procedural methodology knowledge rather than declarative methodology knowledge alone; • improved the effectiveness of their problem solving over time only when they used procedural methodology knowledge; • produced idiosyncratic results based on the application. Based on the findings of this research, it appears that research into the nature of applications, as well as methodologies, is warranted. From the viewpoint of the practitioner, since application knowledge is idiosyncratic, it may be necessary to include more than one application-knowledgable person on a systems development team.

Strong vs. weak approaches to systems development

S ystems development is, fundamentally, a problem-solving activity. A problem in an application domain is transformed by the systems development process into a solution in the computer’s implementation domain. What can systems development specialists borrow from the traditional literature on problem solving? Are there ideas in this older discipline that can be reused? There is one particularly vital notion that has important implications for both the theory and practice of systems development. The problem-solving literature distinguishes between “strong” and “weak” problemsolving approaches. Strong methods are those designed to address a specific type of problem, while weak methods are general approaches that may be applied to many types of problems [7]. The words “strong” and “weak” are deliberately chosen. A strong method, like a specific size of wrench, is designed to fit and do an optimal job on one kind of problem; a weak method, like a monkey wrench, is designed to adjust to a multiplicity of problems, but solve none of them optimally. It is the purpose of this article to pursue this notion of “strong” and “weak” problem-solving approaches more thoroughly from a systems development point of view. Are we making use of this notion in today’s theory and in today’s practice? If not, how can we change what we are doing to make use of strong problem-solving approaches?

Cognitive fit: an empirical study of recursion and iteration

A laboratory experiment was conducted to assess the basic theory and extensions to the theory for recursive tasks across programming languages. The experiment used 34 LISP and 48 PASCAL computer science students in two repeated measures designs. Findings of the study are reported and analyzed. The results strongly suggest that investigation of programming constructs should take place in the context of specific programming languages. Since a number of languages provide similar kinds of programming constructs, it is difficult for programmers to choose those implementations that best suit their needs. One way of encouraging the use of desirable constructs would be to develop languages adapted to certain types of tasks. Such an approach would inherently lead to cognitive fit and the attendant performance benefits would be realized. >

The relevance of application domain knowledge: characterizing the computer program comprehension process

Recent research using professional programmers suggests that knowledge of the application domain plays a major role in the cognitive processes they use to understand computer programs. In general, programmers use a more top-down comprehension process when working in familiar application domains, and a more bottom-up process in unfamiliar domains. The present study builds on that research by further characterizing comprehension processes. The findings show that: (1) certain programmers use different types of comprehension processes depending on their familiarity with the application domain (flexible approach), while others do not (top-down and bottom-up approaches); (2) familiarity with the application domain and the use of a particular comprehension process have marked effects on references programmers make to both application and programming domain knowledge; and (3) programmers who use a flexible comprehension process achieved the highest levels of comprehension. The present research also examines some cognitive determinants of the comprehension process. The findings highlight the need to consider application, as well as programming, domain knowledge as areas of computer programming expertise, to investigate factors influencing use of specific comprehension processes, and to develop tools to support flexible comprehension processes.

Programmer and analyst time/cost estimation

The ability to estimate the personnel time and costs required for the completion of programming and systems projects is an important managerial tool for the information systems department. This article presents a survey of the estimation techniques found in the literature by describing each technique and discussing its strengths and weaknesses. Some empirical evidenc3e on how the various program and programmer/analyst characteristics affect project time and cost are also reported.

Software tasks: intellectual or clerical?

Abstract There are two conflicting views of the complexity of software development: ‘anyone can do it’, or ‘it is the most complex activity the human mind has ever undertaken’. We address this difference empirically in two exploratory studies that examined the intellectual (non-routine) and clerical (routine) nature of software tasks. The first study sought to determine the proportion of software tasks that can be regarded as intellectual or clerical in nature. Taxonomics of software tasks were classified based on the assessment of highly experienced raters. The second study examined the length of time novice systems analysts spent in carrying out tasks during the information requirements specification phase of the systems development life cycle. The experiment used protocol and videotape analysis. Results show that the numbers of intellectual tasks in software development, and the time spent on those tasks, both predominate over clerical tasks by 4 to 1. These initial results suggest that even simple tasks are more intellectual than the ‘anyone can do it’ or the ‘software development can be automated’ viewpoints frequently expressed in the literature.