Vicky Arnold

The differential use and effect of knowledgebased system explanations in novice and expert judgment decisions

Explanation facilities are considered essential in facilitating user interaction with knowledge-based systems (KBS). Research on explanation provision and the impact on KBS users has shown that the domain expertise affects the type of explanations selected by the user and the basis for seeking such explanations. The prior literature has been limited, however, by the use of simulated KBS that generally provide only feedback explanations (i.e., ex post to the recommendation of the KBS being presented to the user). The purpose of this study is to examine the way users with varying levels of expertise use alternative types of KBS explanations and the impact of that use on decision making. A total of 64 partner/ manager-level and 82 senior/staff-level insolvency professionals participated in an experiment involving the use of a fully functioning KBS to complete a complex judgment task. In addition to feedback explanations, the KBS also provided feedforward explanations (i.e., general explanations during user input about the relationships between information cues in the KBS) and included definition type explanations (i.e., declarative-level knowledge). The results show that users were more likely to adhere to recommendations of the KBS when an explanation facility was available. Choice patterns in using explanations indicated that novices used feedforward explanations more than experts did, while experts were more likely than novices to use feedback explanations. Novices also used more declarative knowledge and initial problem solving type explanations, while experts used more procedural knowledge explanations. Finally, use of feedback explanations led to greater adherence to the KBS recommendation by experts-a condition that was even more prevalent as the use of feedback explanations increased. The results have several implications for the design and use of KBS in a professional decision-making environment.

Behavioral research opportunities: Understanding the impact of enterprise systems

Abstract Over the past ten years, many organizations, both large and small, have implemented enterprise resource planning systems or enterprise systems. The motivation behind these investments is to improve organizational efficiency, effectiveness, and ultimately performance. Underlying the expectation of performance increases is the belief that information can be captured and disseminated throughout the organization more quickly to help individuals do their jobs better by making better decisions. While the return on these investments is certainly expected, understanding how both the organization and individuals within these organizations are affected and how the benefits will actually materialize from an interorganizational perspective has received little attention. The purpose of this discussion is to highlight the largely ignored behavioral impacts of enterprise systems implementation and integration and to suggest ways to improve the validity and generalizability of field research through method triangulation. This discussion advocates that the contemporary enterprise systems research could learn much from the management accounting research discipline as it searches for greater meaning and rigor in its research approaches. Specifically, this paper suggests that future research on enterprise systems should include (1) experimentation that focuses primarily on judgment and decision making at the individual level and improvements in organizational performance, and (2) triangulation methods that integrate case research, surveys, and cross-sectional field studies.

The effect of experience and complexity on order and recency bias in decision making by professional accountants

Recently, questions have been raised regarding the impact of experience on the susceptibility of professional accountants to judgment bias-particularly order and recency biases as predicted in the Belief-Adjustment Model. The Belief-Adjustment Model predicts recency effects will always exist in complex decision domains (regardless of experience). Complexity is defined within the model as a function of task familiarity and information load. The prior studies on experience and bias in accounting domains have focused on varying task familiarity and have found that task familiarity can mitigate order/recency bias. In this study, complexity is operationalised through heavy information load (a condition more consistent with professional accounting environments) while maintaining a high level of task familiarity. Two experiments were conducted. The first experiment utilised a going concern decision using highly experienced partners and managers. The second experiment was conducted in the insolvency domain and used 87 experienced insolvency practitioners. The results indicate that experience does not mitigate order/recency bias under conditions of heavy information load.

Competing pressures of risk and absorptive capacity potential on commitment and information sharing in global supply chains

Organizations’ competitiveness and success are no longer dependent solely on their own performance, but rather are dependent on the competitiveness of the supply chains in which they participate. Increasingly, these supply chains are globally distributed introducing the possibility of greater benefits, as well as greater risk. This study examines the countervailing impact of a global supply chain partners business-to-business e-commerce business risk and absorptive capacity on an organizations willingness to commit to and share information with that supply chain partner. We survey 207 organizations on their perceptions of specific offshore outsourcing and supply chain partners across dimensions of risk, absorptive capacity, commitment, and information sharing. The results support the theorized relationships indicating that a supply chain partners increased levels of perceived risk has a strong negative effect on an organizations commitment and information sharing; conjointly, increases in a supply chain partners absorptive capacity has a strong positive effect on commitment and information sharing. For both risk and absorptive capacity, commitment partially mediates the relationship with information sharing. Testing for systemic effects from geographical/cultural location on the relationship factors provides no evidence of a regional effect on measured items.

Focus group methods: Using interactive and nominal groups to explore emerging technology-driven phenomena in accounting and information systems

This paper puts forth arguments for why focus group methods should be used more frequently in accounting and information technology research in order to address emerging technology-driven phenomena in accounting. In this overview of focus group methods and their application to studying accounting and information technology phenomena, we focus on what alternative types of focus group methods may be applied, when such methods are applicable, and what methodological challenges the researcher must address. In examining the methodological challenges that face researchers when using these methods, we demonstrate how these challenges have been addressed in our own research as a way of exemplifying the challenges researchers may face and what compensating strategies researchers might use. Our objective is to highlight when focus groups may be the best method for accounting information systems researchers and demonstrate how they can be used when they are applicable.

Explanation Provision and Use in an Intelligent Decision Aid

Intelligent decision aids have been widely adopted by organizations in an effort to capture, retain and disseminate the knowledge of individuals within the organization. To date, these efforts have met with mixed results. Two primary limitations have been the usability of systems (users either cannot or do not wish to use systems) and the inability to transfer knowledge from such systems to less expert decision makers. Explanation capabilities have been perceived as a potential means for improving the effectiveness of such systems in terms of both usability and knowledge transfer. In this study, four advancements are put forth. First, a working prototype system for highly complex decision making is used as the foundation for implementing a full-scale, embedded explanation facility. Second, the development of the explanation facility is carefully grounded in the extant prescriptive literature on explanation types and delivery processes—including feedforward and feedback. Third, a development strategy is formulated and demonstrated as a generalized process of explanation development and integration. Fourth, high-level professional decision makers use the system and provide evaluative feedback on the usability and value of the explanation facility. Overall, the results of the development efforts and subsequent evaluation are very positive and provide a solid foundation for future research and practice efforts related to explanation provision within the context of intelligent systems. Copyright

An analysis of the research productivity of AIS faculty

Abstract During the past 25 years, a number of articles have examined the accounting research productivity of individual faculty members and institutions. Many articles have focused on the quantity of publications, with a few incorporating a quality measurement component. Concurrently, other research efforts have examined the quality of the journals that are considered to be potential outlets for a broad, cross-section of accounting academics. These studies, taken as a whole, have provided valuable insights into the research productivity across the domain of accounting academics. The purpose of this paper is to examine research in the subdiscipline of AIS, from both a quantity and quality perspective and evaluate both institutional productivity and individual faculty productivity. Information from all AIS research published from 1982 through 1998 in 45 accounting and information systems journals was collected and analyzed. Using this data, we show which journals are considered the highest quality outlets for AIS publications, which faculty have published the most AIS research, which employing institutions are rated highest in AIS research productivity and which doctoral granting institutions are rated the highest for AIS research productivity. The findings of this study contribute to the understanding of the AIS research domain and to accounting by providing valuable insights into the quality of AIS research.

An Analysis of the Market for Systems Reliability Assurance Services

As the accounting profession enters new assurance markets, many decisions as to the best manner in which such services can be provided must be addressed. For instance, the traditional audit/attest‐reporting model may not provide adequate information to justify the cost of assurance services in some situations. Alternatively, potential clients may turn to other assurance providers who are willing to provide better quality services and/or more informative assurance reporting. In this study, we examine the impact of a two‐tier reporting model on the market demand for assurance services on software product reliability. One tier uses the traditional binary report common with audit/attest services where a standard report notes whether a software product achieves a minimum standard, while the other tier of reporting provides for a graded assessment of multiple levels of quality. The results indicate that when a graded report that differentiates high‐quality vendors from moderateor low‐quality vendors is availabl...

An Analysis of the Group Dynamics Surrounding Internal Control Assessment in Information Systems Audit and Assurance Domains

Both the evolution toward online continuous auditing and new assurance services for information systems reliability have helped fuel changes in the audit/attest process. These changes have already been of concern in dealing with large organizations using complex information systems to process their accounting and business information. As a result, these changes have necessitated a change in focus from traditional accounting control processes to increasingly complex information‐systems‐based control processes for advanced technology applications. With the resulting increased complexity in the internal control assessment process, the move toward group decision making in the major accountancy firms is expected to accelerate—particularly for the control‐assessment process. The research documented in this paper focuses on the impact of group decision making on decision quality within the internal control‐assessment process for information systems environments. The results indicate that improved decision qualit...

The Sarbanes-Oxley Act and the changing role of the CIO and IT function

The focus of this study is on the changing role of the Chief Information Officer (CIO). At a time when there is an increasing focus on corporate governance and enterprise risk management, the competencies required of CIOs are evolving causing a shift to a third era of the CIO. This shift is being accelerated and influenced by the passage of the Sarbanes-Oxley Act of 2002 in the USA. This Act mandates detailed reporting of internal controls over financial and related systems, and has consumed the energy and the budgets of the IT function in the past few years. In the coming years, the Act will likely impact the success of CIOs and the competencies desired by organisations hiring CIOs. These issues are explored in detail. The article concludes with an agenda for research on CIOs in this new era.