John Rohrbaugh

Improving the quality of group judgment: Social judgment analysis and the Delphi technique

Abstract Previous research findings suggest that group judgment is superior to individual judgment, although groups fail to reach their full potential because of problems associated with the interaction process. Thus, groups perform at a level generally better than the competence of their average members but rarely as well as their most proficient members. The current study explores two methods of group judgment making which have been developed to reduce the discrepancy between potential and actual group performance: social judgment analysis and the Delphi technique. These two methods are compared in a controlled experimental setting with regard to their potential both to significantly reduce group disagreement and to provide accurate judgments. The two methods were found to be equal in the quality of judgments produced. Social judgment analysis, however, was a significantly better method of reducing disagreement than the Delphi technique.

Model-building for group decision support: Issues and alternatives in knowledge elicitation

Abstract System dynamics models are often created using multiple streams of information including quantitative data, written records, and information contained in the mental models of both individuals and groups. While qualitative sources of information are widely recognized as important in all stages of the model-building process, little systematic research has been completed on how best to elicit and map this knowledge from groups of experts. In this article, we survey the existing literature on mapping and eliciting knowledge for system dynamics modeling and also explore the literature in the broader fields of cognitive psychology and small group processes. Special attention is paid to new software advances to support these processes. Two case studies illustrate how these knowledge-eliciting techniques can be used to support the construction of computer simulation models.

Improving the quality of group judgment: Social judgment analysis and the nominal group technique

Abstract Previous research findings suggest that group judgment is superior to individual judgment, although groups fail to reach their full potential because of problems associated with the interaction process. Thus, uninstructed interacting groups perform at a level generally better than the competence of their average members but rarely as well as their most proficient members. The current study explores two methods of group judgment making which have been developed to reduce the discrepancy between potential and actual group performance: Social Judgment Analysis and the Nominal Group Technique. These two methods are compared in a controlled experimental setting with regard to their potential both to significantly reduce group disagreement and to provide accurate judgments. The two methods were found to be equally strong in the quality of judgments produced; both sets of groups performed at a level of accuracy equivalent to that of the most proficient member. Social Judgment Analysis, however, was found to be a significantly better method of developing consensus among group members than the Nominal Group Technique.

Evaluating group decision support system effectiveness: A performance study of decision conferencing

Abstract Effectiveness of decision making with decision conferences was explored by examining groups using the Decision Techtronics Group. Quantitative data were gathered through a self-administered survey of participants in 14 conferences. Perceived conference effectiveness was assessed with respect to a global outcome measure and 8 scales measuring the decision process effectiveness, based on the Competing Values Approach to organizational analysis. Differences in perceived conference success were related to (1) the proportion of participants who believed the conference resulted in a decision, and (2) the level of benefits derived from: full support of the structure or preference technology; the opportunity for full, extended discussion; development of an action plan; and expected resolution of the problem by the conference end.

Interdisciplinary Perspectives on Small Groups

After many years of developing in small islands scattered around different disciplines, small group research has reached a point where interdisciplinary scholarship has the potential to foster major progress. The goal of this special issue on interdisciplinary perspectives is to capitalize on the theoretical advances made over the last 50 years by synthesizing and integrating models and theories on small groups proposed by various disciplines into a set of general theoretical perspectives. In this introduction, the authors identify nine general theoretical perspectives from which small groups have been examined: the psychodynamic, functional, temporal, conflict-power-status, symbolic-interpretive, social identity, social-evolutionary, social network, and feminist perspectives. This article summarizes each theoretical perspective briefly and then offers some observations about the perspectives as a whole. Articles describing three of these interdisciplinary perspectives appear in this special issue, and four other perspectives will be introduced in the next issue.

Group Decision Process Effectiveness A Competing Values Approach

Four standards are proposed to assess instruments designed to evaluate group decision process effectiveness. These standards were applied to a multicriteria evaluative instrument based on the Competing Values Approach to decision process effectiveness. In a study using nine groups and three types of raters (facilitators, participants, and observers) in a field setting, the instrument was found to be appropriately insensitive to role differences between raters within groups and appropriately sensitive to differences across groups. Ratings of facilitators and participants within groups were the most consistent, although their ratings diverged on four of the effectiveness scales.

Decision conferencing for systems planning

Abstract Although managerial support and involvement is critical to the success of information systems planning efforts, managers typically have limited time to devote to such efforts, do not understand their role, and do not make the necessary commitment. This paper describes the successful use of decision conferencing as a corporate tool to increase managerial involvement and support and to improve the efficiency and effectiveness of systems planning. In a two-day, computer-supported meeting, executives identify critical issues, anticipated decisions, and related information needs. Using a computer-based decision model, these needs are organized in an issue-based framework, evaluated according to their organizational value, and prioritized with respect to available system development resources. A case study involving the New York State Insurance Department illustrates the process.

Eliciting Group Knowledge for Model-Building

System dynamics models are typically created using multiple streams of information including quantitative data, written records, and information contained in the mental models of both individuals and groups. While qualitative sources of information are widely recognized as important in all stages of the model building process, little systematic research has been completed on how best to elicit and map this knowledge. In this paper, we survey the existing literature on mapping and eliciting knowledge for system dynamics modeling and also explore the literature in the broader fields of cognitive psychology and small group processes. Special attention is paid to new software advances to support these processes.2

Negotiation and design: Supporting resource allocation decisions through analytical mediation

The common element of all negotiations is change. Design is the key to directing and managing change, and resource allocation is the most critical component of design. Negotiations about change are, therefore, fundamentally, negotiations about design and resource allocation. Negotiations vary along a continuum, from those in which negotiators have consonant interests (share objectives) to discordant ones (disagree about appropriate objectives). The joint distribution of all possible payoffs defines the structure of the negotiation problem—the opportunities the problem affords and constraints it imposes on negotiators. The analytical mediation approach supports the activities of an impartial, neutral third party who attempts to assist the disputants to reach a mutually satisfactory agreement. It makes use of different types of techniques to support negotiations, depending on their location along the negotiation continuum. Two case studies involving analytical mediation are reported. One case study involves a budgeting exercise, in which the negotiators’ interests were essentially consonant. The second case study involves a labor-management contract problem, in which the negotiators’ interests were highly discordant.

Some conceptual and technical problems in integrating models of judgment with simulation models

Recent research has focused on how policy makers learn from and make better use of policy-oriented simulation models. It is proposed that those who construct simulation models make simplifying assumptions concerning the wide range of judgment tasks that surround both model building and decision making in real policy contexts. Linking formal judgment models with formal simulation models has been suggested as an approach to dealing with those simplifications. A demonstration experiment that illustrates how simulation models and models of human judgment explicitly can be linked within a unified framework is presented. In creating this linkage six important conceptual and technical problems are encountered and discussed. While many of the technical problems are believed to result from the specific approaches used in the study, the broader conceptual problems are believed to persist, even if differing methods are used. >