Cláudia S. Sarrico

Pitfalls and protocols in DEA

Abstract The practical application of data envelopment analysis (DEA) presents a range of procedural issues to be examined and resolved including those relating to the homogeneity of the units under assessment, the input/output set selected, the measurement of those selected variables and the weights attributed to them. Each of these issues can present difficulties in practice. The purpose of this paper is to highlight some of the pitfalls that have been identified in application papers under each of these headings and to suggest protocols to avoid the pitfalls and guide the application of the methodology.

Restricting virtual weights in data envelopment analysis

Abstract The consequences of the use of absolute weights restrictions (i.e. restricting the multipliers) on the efficiency score and targets of a DEA model have been explored elsewhere, the same is not true for the use of restrictions on the virtuals (i.e. the product of the input/output factor by its multiplier). In this paper, a reflection on the uses of virtual weights restrictions is presented. The reasons for using virtual weights restrictions instead of absolute weights restrictions, in particular cases, are explained. Following a critique of Wong and Beasleys [J. Oper. Res. Soc. 41 (1990) 829] first proposed method for constraining the virtuals in DEA, a new classification scheme for virtual weights restrictions is presented, which brings the concept of assurance regions into virtual weights restrictions. It is shown that the use of simple virtual restrictions and virtual assurance regions are preferable to the use of the more generally advocated WBs proportional virtual weights restrictions. In recognition of levels of decision making at the unit, and external to the unit, the use of the terms unit of assessment (UOA) and controller is proposed. It is concluded that the use of virtual assurance regions applying to the target UOA can be a natural representation of preference structures and translate established patterns between the input–output divide. Also, the meaning of the efficiency score and targets in this approach most approximate traditional DEA. Alternatives to using virtual weights restrictions are considered, namely using absolute weights restrictions with a virtual meaning. Finally, an empirical example is offered.

Using DEA for planning in UK universities—an institutional perspective

This paper is focused on the process of performance measurement undertaken by different stakeholders in the UK higher education sector, focusing on the institutional perspective. Different classes of stakeholders have different motivations to measure performance. Institutions will be affected on the one hand by the state evaluation of them, and on the other by the applicants. The contribution of Data Envelopment Analysis (DEA) to inform management is explored and illustrated in an application to the University of Warwick, using concepts from a technique to support strategic option formulation, the Boston Consulting Group (BCG) matrix.

The Influence of Performance Management Systems on Key Actors in Universities

Abstract The aim of this article is to understand to what extent the introduction of performance management systems has affected the roles and influences of the key actors in the governance of universities, especially the role of academics, and whether or not the introduction of these systems has altered accountability regimes within universities. Results from a high performing English university show that, in spite of a substantial increase in the measurement of performance in most areas, there seems to be a lack of action, especially regarding individual performance. In relation to the key actors in the governance of the university, it is clear that they are now held more accountable, especially in a managerial way, mainly resulting from pressures coming both from the State and the market, and their roles have changed.

On Performance in Higher Education: Towards performance governance

The paper has a conceptual nature, and will depart from the literatures on performance management in general and public management in particular and confront them with the literature on higher education studies. Higher education mirrors the developments of other sectors, such as the increasing interest in performance measurement and the continuous enlargement of the basis for performance measurement, from financial, operational, and stakeholders’ towards developmental perspectives. The paper states some developments in performance management in higher education, and argues that it might be moving towards performance governance, as observed with other public services. The originality of the paper rests on an analysis of the developments in higher education in light of developments in organisational performance in general, and in the public sector in particular. It thus presents a hypothesis for further empirical testing.

Implementing Quality Management Systems in Higher Education Institutions

In the last decades, several factors have contributed to raising public concern over higher education institutions’ quality, leading to the emergence of quality measurement and improvement devices such as performance indicators, accreditation, programme and institutional assessment and quality audits, and there have been attempts to import models from the private sector into higher education systems and institutions (Sarrico, Rosa, Teixeira and Cardoso, 2010). This has led to the emergence of a debate on the applicability of quality management principles, methodologies and tools to the higher education sector. As reported in the literature on higher education, several voices have been heard about the non-applicability at all of those management theories, especially because they derived from industry and had nothing to do with the higher education ethos (Harvey, 1995; Maassy, 2003; Birnbaum, 2000; Kells, 1995; Pratasavitskaya and Stensaker, 2010). Other authors gave a more nuanced view on the subject, claiming that although higher education institutions were not companies some of the basic principles and tools could be applied as long as they were instruments at the service of institutions and their governance and management boards, subject to the institutions academic mission, goals and strategies (Williams, 1993; Harvey, 1995; Dill, 1995). Although the debate is old, no firm conclusions have been reached so far. It seems, nevertheless, that in Europe, due to the developments on quality assurance that followed the Bologna Declaration, higher education institutions are now being “forced” to implement internal quality assurance systems based on the European Standards and Guidelines (ESG). Apparently the way these systems should be organised and function is not that specified, apart from the need to comply with the seven standards established in the ESG Part I, being up to each institution to define and implement its own quality assurance system in accordance with its mission, goals and institutional culture (Santos, 2011). So maybe this is the time to look again at quality management principles, tools and frameworks and see if they can be of some help to the development of these quality assurance – or management – systems.

Academics’ perceptions on the purposes of quality assessment

The accountability versus improvement debate is an old one. Although being traditionally considered dichotomous purposes of higher education quality assessment, some authors defend the need of balancing both in quality assessment systems. This article goes a step further and contends that not only they should be balanced but also that other purposes can be devised for assessing quality in higher education. Five different purposes are proposed: communication, motivation, control, improvement and innovation, derived both from the higher education and the organisational performance literatures. Then the answers given to a set of questions related to these five intended purposes are analysed. The answers were collected through a questionnaire designed to investigate Portuguese academics’ perceptions on higher education quality assessment. Overall the analysis performed reveals a certain degree of support for all the purposes, albeit higher in the case of the improvement and communication purposes and lower for control and motivation. Since an adequate implementation of quality assessment systems needs the support of academics, this article can inform the design of systems integrating academics’ views on the subject.

Student satisfaction with Portuguese higher education institutions: the view of different types of students

The purpose of this paper is to discuss student satisfaction with Portuguese higher education institutions, and to analyse how this varies for different types of students. A survey instrument was used to collect data on students’ perceptions and expectations regarding different aspects of service provision. Based on the gap model of satisfaction, satisfaction indices were calculated for all items considered and were checked for their statistical significance. Although, genetically speaking, students’ perceptions and expectations are high, the satisfaction indices are all negative and statistically different from zero. Furthermore, significant differences of satisfaction regarding some aspects of service provision were found between different groups of students. This study may contribute to institutions improving the quality of services they provide. Furthermore, this study will contribute to improving the quality of higher education institutions through correcting mistakes and designing better processes, provided that the information collected through student satisfaction surveys has been treated adequately.

Quality, Evaluation and Accreditation: from Steering, Through Compliance, on to Enhancement and Innovation?

The chapter analyses the evolution of quality mechanisms in Portuguese higher education (HE). The first section reviews the legislative intent behind the ‘quality’ agenda. The second section covers the years 1994–2005. It examines two phases in the move to evaluate HE at first degree level, which ended however with the initiative being abruptly terminated. The third section examines recent developments from 2006 to the present. This third phase saw the publication of two reports: an OECD review of Portuguese HE (OECD, Reviews of national policies for education: tertiary education in Portugal, OECD, Paris, 2007) and another on quality assurance (ENQA, ENQA occasional paper 10, ENQA, Helsinki 2006), which coincided with closing down of the previous evaluation system. Major changes in the interests represented were introduced. Previously, the evaluation system lay in the hands of groups representing institutions of HE. Its successor, which included accreditation, was vested in an independent agency, with both government and higher education representatives. The fourth section muses on the possible outcomes the new evaluative regime may engender. Will its influence be more positive? Or will it, on the contrary, become inquisitorial?

The integration of quality management in higher education institutions: a systematic literature review

This paper presents the results of a systematic literature review showing how the literature on quality management (QM) in higher education (HE) has evolved. As a first contribution, this work presents a systematic breakdown of research in the field of HE quality management. Its second and most innovative contribution is its coverage of the process of introducing quality management into institutions’ global management systems. Theoretically, we believe this second point to be a general trend in the evolution of the quality management literature, and empirically it represents a trend for quality management principles and practices in governance and management systems of higher education institutions (HEIs). The literature was analysed by distinguishing three main levels in HEIs: a process level, an organisational level, and, since our focus is specifically on the field of quality management, a quality management principles level. Overall, this paper concludes that integration at the three levels of analysis is strong. Moreover, we were able to identify a trend towards the development of holistic and comprehensive quality management approaches both in conceptual and empirical research studies.