Steve Smithson

Information systems evaluation in practice : a case study of organizational change

Information systems (IS) evaluation is a difficult problem in both theory and practice. While theoreticians are moving away from a positivistic approach towards a more interpretive approach, organizations are adopting a more entrepreneurial approach. We examine such a move through a case study of a large utility company in the UK which was relatively unsuccessful in its attempts at improving the evaluation of its IS. This paper uses an interpretive approach to explain the lack of success in introducing the new entrepreneurial evaluation procedures. However, we argue that the failure was not so much due to a lack of understanding of evaluation but more due to a failure to appreciate and support the necessary organizational changes.

Information systems evaluation as an organizational institution – experience from a case study

Abstract.  Information systems evaluation has strong social and organizational dimensions while existing research focuses primarily on the formal and positivistic characteristics. We investigate the organizational roles of the key stakeholders within the particular evaluation context (orientation). We focus on four evaluation orientations: control, sense‐making, learning and exploratory. Experience from a case study is used to analyse these concepts and relations. The case organization attempted to move from a state of an ad‐hoc evaluation practice to a more business‐driven one. New tools, processes, roles and responsibilities were developed during the attempt. We examine the behavioural and organizational integration aspects of evaluation during that journey.

Rethinking the approaches to information systems investment evaluation

Numerous information system evaluation methodologies have been proposed in the literature. However, most of them suffer from a number of inherited disabilities originating in the nature of their fundamental principles as well as the nature of information systems evaluation. Uses evidence from the evaluation literature and two case studies to address the multiple dimensions for evaluation practices. We focus on the context, content and process of information systems evaluation as a source of organisational change. We found a noticeable gap between the recent theoretical work on IS evaluation and the practices within the case study organisations. This was not due to any lack of knowledge but was attributed to contextual variables such as the organisational culture and the power of important stakeholder groups.

Information retrieval evaluation in practice: a case study approach

The evaluation of information retrieval systems is an important yet difficult operation. This paper describes an exploratory evaluation study that takes an interpretive approach to evaluation. The longitudinal study examines evaluation through the information-seeking behaviour of 22 case studies of ‘real’ users. The eclectic approach to data collection produced behavioral data that is compared with relevance judgments and satisfaction ratings. The study demonstrates considerable variation among the cases, among different evaluation measures within the same case, and among the same measures at different stages within a single case. It is argued that those involved in evaluation should be aware of the difficulties, and base any evaluation on a good understanding of the cases in question.

An exploratory study of information systems in support of employee empowerment

The notion of employee empowerment in todays organizations is becoming increasingly important with the growing degree of responsibility individual employees hold. This paper presents the findings of a longitudinal study which examined the relationship between computer-based information systems (IS) and empowerment in detail drawing heavily on empirical work carried out in the British manufacturing industry. The paper elaborates and discusses the conceptual issues which are relevant to the idea of empowerment and its relationship with IS, before determining empowerment to be in essence the decentralization of decision-making authority to lower levels of the organization. The lack of any research on this topic led us to carry out a combination of empirical research. A large-scale postal survey of 450 UK manufacturing companies followed by a series of 20 in-depth interviews in 18 top UK manufacturing organizations formed the basis for this paper. The findings confirmed that empowerment is indeed pursued by many UK manufacturing organizations within their various efforts at improving their organization of work. The research participants viewed IS as an important enabling tool for empowerment offering many opportunities, but clearly noted that the role of IS in this is supportive rather than initiating; IS do not lead to employees becoming empowered. However, the interviews revealed that manufacturing organizations are facing a wide range of problems and difficulties with their existing IS raising a number of constraints. The paper concludes with a number of practical suggestions for using IS as a supportive means for empowering employees.

The management of change for information systems evaluation practice: Experience from a case study

Most of todays cost-driven, project evaluation methodologies and accountancy frameworks fail to take into account the intangible benefits and associated risks, and cannot reflect the infrastructural nature of modern information system (IS) This paper argues that an interpretivist framework is initially needed to understand and study the IS evaluation process. The emphasis here is on describing and analysing processes of change regarding information technology (IT) appraisal practices in context, illustrating why and how their content and the strategies for introducing them can be constrained and/or enabled by features of the organizational context. In these terms, this paper analyses the case of a UK insurance organization where the need for a rigourous IT appraisal methodology initiated a 12-month project to design and develop such a method and a series of supporting tools. The paper examines the use the evolution of the methodology over the past two years.

E-business and organizational change: a structurational approach

Abstract.  Although e‐business is a familiar part of the organizational landscape, its implementation remains a problem for large traditional organizations. This paper argues that the pervasiveness of e‐business calls for the adoption of an organizational change perspective to study its implementation. Using structuration theory, the paper analyzes a detailed case study of the implementation of a major e‐business initiative in a traditional automotive manufacturer. It shows how a combination of structural contradictions and unexpected consequences derailed the initiative.

Role of information systems on the business network formation process: an empirical analysis of the automotive sector

The conditions of change and complexity that characterise the new environment are forcing companies to adopt more flexible organizational structures that allow a quicker adaptation to these continuous changes. States that there are very few companies with enough resources to form their own value chain. Opines this is one of the reasons why the establishment of collaborative agreements among companies seems to be one of the most interesting alternatives in order to complement one anothers deficiencies. The formation process of a cooperative business network is analysed, identifying two principal stages: trial and integration. Through an analysis of the network formation process in a case study concerning the Spanish automotive sector, particular issues are identified that characterise the two stages of the process. Shows clearly that the role of information technology and information systems in this situation is very much one of a facilitator.

Online visibility as a source of competitive advantage for small- and medium-sized tourism accommodation enterprises

This study examines the importance of the Internet as a distribution channel for medium-sized and small accommodation enterprises, identifying the characteristics that could make the Internet a source of competitive advantage. From the resource-based view, online visibility is considered a differentiating factor in the accommodation business, able to produce superior organisational performance through the capture of new clients and by increasing the occupancy rate. The hypotheses are evaluated using regression models with data from small- and medium-sized hotels in Spain available in the public database SABI. The results show a positive effect of online visibility on organisational performance, although explaining only 10% of performance variance. However, online visibility is a determinant for the percentage of reservations made via the Internet.

Analyzing research data

Many information systems projects have a significant empirical element and, for this reason, students spend considerable time and effort collecting data by means such as postal questionnaires, interviews or involvement in case studies. Having collected large amounts of data, much of which may seem interesting, confusing or downright obvious, the student then has to decide how to use this data to develop and support their arguments. This analysis task is the subject of this chapter. Our experience suggests that, for a variety of reasons, data analysis is often carried out rather badly with the result that much of the valuable data collected with great effort goes to waste. Either students shy away from any analysis, leaving the reader wondering frustratedly what could have been achieved or, at the other extreme, students bombard the reader with endless tables, long quotes and sophisticated statistics so that the key points of the argument are swamped.