Sérgio Pereira dos Santos

Adding value to performance measurement by using system dynamics and multicriteria analysis

The design, implementation and use of adequate performance measurement and management frameworks can play an important role if organisations are to succeed in an increasingly complex, interdependent and changing world. Yet, despite widespread recognition of the importance of performance assessment, there are some issues that require further study if measurement systems are to be effective in the process of management through monitoring and decision making. This article proposes and seeks to illustrate that the use of system dynamics and multicriteria decision analysis, as part of a broader framework for performance measurement, can address some of the issues identified. The use of these approaches, independently and in an integrated manner, provides a means of exploring the dynamic complexity of organisations, making explicit trade‐offs between performance measures, and assessing the impacts of initiatives to improve performance, thereby enhancing understanding and ultimately contributing to the improvement of organisational performance.

Adding Value to Bank Branch Performance Evaluation Using Cognitive Maps and MCDA: A Case Study

Bank branch performance evaluation is a difficult endeavour. Some of the main reasons for this difficulty are the complexity inherent in the variety of aspects considered in the evaluation, and the multiple and conflicting interests of the different stakeholders involved. In this paper, we show how cognitive mapping and measuring attractiveness by a categorical-based evaluation technique can be used to support the evaluation of bank branches through the development of multidimensional performance evaluation systems, and to deal explicitly with the trade-offs between the different dimensions of performance and interests of different stakeholders. A case study is discussed where these techniques are used in a constructive way, making the learning activity easier and introducing transparency in the decision-making process. The strengths and weaknesses of the integrated use of these two operational research techniques in this context are also discussed.

Challenges for performance assessment and improvement in primary health care: the case of the Portuguese health centres.

Primary care is currently at the heart of health service policy, both in Portugal and internationally. At the same time, interest in the measurement of efficiency of health care delivery has increased considerably. However, very few studies have focused specifically on performance assessment in primary health care provision. In this paper we aim to contribute to a discussion regarding appropriate ways to compare the performance of primary health care providers. Following a review of the literature, a conceptual framework for performance assessment of health centres is developed and tested using the Portuguese case. Our empirical analysis uses data from 2004 and 2005 for all the Portuguese health centres and compares the geographical equity of access to services, technical efficiency and quality of services across district health authorities. Data envelopment analysis is the technique used to measure technical efficiency. From our empirical analysis, we conclude that there is evidence of large variation in equity of access to services, in technical efficiency and quality of services across district health authorities. Our findings suggest that a better use of resources could lead to more and improved services. Whilst formative evaluation could contribute to this, the challenge is to engage health care professionals in the evaluation exercise.

A Multiple Criteria Framework to Evaluate Bank Branch Potential Attractiveness

Remarkable progress has occurred over the years in the performance evaluation of bank branches. Even though financial measures are usually considered the most important in assessing branch viability, we posit that insufficient attention has been given to other factors that affect the branches’ potential profitability and attractiveness, such as: location features, trade area characteristics and facilities management. Based on the integrated use of cognitive maps and multiple criteria decision analysis, we propose a framework that adds value to the way that potential attractiveness criteria to assess bank branches are selected and to the way that the trade-offs among those criteria are obtained. This framework is the result of a process involving several directors from the five largest banks operating in Portugal, and follows a constructivist approach. Our findings suggest that the use of cognitive maps systematically identifies previously omitted criteria that may assess potential attractiveness. The use of multiple criteria techniques clarify and add transparency to the way trade-offs are dealt with. Advantages and disadvantages of the proposed framework are also discussed.

Enhanced performance measurement using OR: a case study

The case study described in this paper aims to illustrate how qualitative and quantitative system dynamics modelling and multicriteria analysis can be used in an integrated way to enhance the process of performance measurement and management in the radiotherapy department of a major UK cancer treatment centre. The complexity of the radiotherapy process and its significance for patients present particular challenges for performance measurement and management. The paper discusses the benefits arising and the practical difficulties faced in the study.

A Multiple Criteria Information System for Pedagogical Evaluation and Professional Development of Teachers

Information systems for the evaluation of pedagogical practices aim to improve teaching activity and encourage the pedagogical development of teachers. The implementation of these systems has been interpreted in different ways, depending on each school’s administration board and/or the person responsible for the evaluation process. This ambiguity (and inherent subjectivity) is a barrier to the principles of transparency, equity and fairness in the pedagogical performance evaluation of teachers. From this premise, the main objective of this paper consists in proposing a multiple criteria information system to support the evaluation of pedagogical practices. By combining cognitive mapping and the measuring attractiveness by a categorical based evaluation technique (MACBETH), our framework allows for the conception of a more transparent evaluation system, bringing new insights to the pedagogical appraisal and professional development of teachers. The strengths and weaknesses of our framework are also discussed.

Evaluating retail banking service quality and convenience with MCDA techniques: a case study at the bank branch level

The intangibility of banking services makes the evaluation of service quality and customer convenience difficult to measure. This paper aims to construct an integrated evaluation system for retail banking service quality and convenience at the bank branch level by combining cognitive mapping with measuring attractiveness by a categorical based evaluation technique. We strive to introduce transparency in the decision making process and add to the performance literature in retail banking. Strengths, weaknesses and practical applications of our multiple criteria evaluation system are also discussed.

Is TQM more difficult to implement than other transformational strategies

This paper aims to assess whether Total Quality Management (TQM) is harder to implement than other business strategies. In order to accomplish this objective we have adopted a two-stage methodology. Firstly, we carried out an extensive review of the literature to identify the rates of TQM implementation failure estimated by researchers and practitioners. Secondly, we compared these rates with those estimated for other organisation-wide transformational efforts and explored the extent to which the obstacles to TQM implementation and to the implementation of other business strategies differ. Based on the literature reviewed and on the comparative analyses performed, we conclude that while it is widely acknowledged that the implementation of TQM can be a difficult task, significant uncertainty remains as to what the exact rate of failure is. Furthermore, the analyses suggest that the rates of failure and obstacles to TQM implementation are similar to those presented by other business strategies, and therefore, there seems to be no grounds to assume that TQM is more difficult to implement than other business strategies. These findings have implications for researchers and practitioners and open up several avenues for further research, which are also discussed in this paper.

Assessing credit risk of mortgage lending using MACBETH: a methodological framework

– Considered the largest investment for most households, buying a house requires careful and transparent analysis by all parties involved in the transaction. The aim of this paper is to propose a methodological framework allowing for the readjustment of trade-offs among risk evaluation criteria, considered of extreme importance in the lending decision process of mortgage loans. , – Multiple criteria decision analysis (MCDA) has proved over the years to be effective and versatile in handling compensations among criteria. Measuring attractiveness is applied by a categorical based evaluation technique (MACBETH) to a pre-established structure of credit-scoring criteria for mortgage lending risk evaluation. This pre-established structure is currently used by one of the largest banks in Portugal. , – The framework allowed the authors to provide the credit experts who participated in the study with a more informed, transparent and accurate mortgage-lending risk-evaluation system. The sensitivity and robustness analyses carried out also helped in promoting discussion and supporting the readjustments made. , – The study shows the usefulness of using the MACBETH approach to assist credit analysts in making better informed decisions, and opens avenues for further research. However, due to the dependence on the participants involved, extrapolations without proper caution are discouraged. , – The credit analysts who participated in this study considered the framework more discerning in terms of Basel directives. , – The integration of MACBETH and credit-scoring mechanisms holds great potential for risk assessment and decision support. No prior work reporting the application of MACBETH in terms of mortgage-lending risk-evaluation is known.

Using Data Envelopment Analysis to support the design of process improvement interventions in electricity distribution

A significant number of studies have documented the use of Data Envelopment Analysis (DEA) for efficiency measurement in the context of electricity distribution, particularly at the level of the distribution utilities. However, their aim has been predominantly descriptive and classificatory, without any attempt to ‘open’ the black box of the transformation process. In contrast, our aim is to explore the potential of DEA to contribute to the design of effective process improvement interventions within a distribution utility.