Nicky Rogge

To publish or not to publish? On the aggregation and drivers of research performance

This paper presents a methodology to aggregate multidimensional research output. Using a tailored version of the non-parametric Data Envelopment Analysis model, we account for the large heterogeneity in research output and the individual researcher preferences by endogenously weighting the various output dimensions. The approach offers three important advantages compared to the traditional approaches: (1) flexibility in the aggregation of different research outputs into an overall evaluation score; (2) a reduction of the impact of measurement errors and a-typical observations; and (3) a correction for the influences of a wide variety of factors outside the evaluated researcher’s control. As a result, research evaluations are more effective representations of actual research performance. The methodology is illustrated on a data set of all faculty members at a large polytechnic university in Belgium. The sample includes questionnaire items on the motivation and perception of the researcher. This allows us to explore whether motivation and background characteristics (such as age, gender, retention, etc.,) of the researchers explain variations in measured research performance.

Wasteful waste-reducing policies? The impact of waste reduction policy instruments on collection and processing costs of municipal solid waste.

We study the impact of some local policies aimed at municipal solid waste (MSW) reduction on the cost efficiency of MSW collection and disposal. We explicitly account for differences between municipalities in background conditions by using a bootstrapped version of the Data Envelopment Analysis methodology in combination with a matching technique. Using data on 299 municipalities in Flanders, Belgium, for the year 2003, our results indicate that municipalities that are member of a waste collection joint venture, or that subscribe to a voluntary agreement to reduce MSW at the highest ambition level, collect and process MSW more efficiently than other municipalities. Weekly instead of two-weekly waste collection, or using a weight-based pricing system appears to have no impact on efficiency. Our results show that aiming at MSW reduction does not lead to lower efficiency of public service provision, even on the contrary.

An environment-adjusted evaluation of citizen satisfaction with local police effectiveness: Evidence from a conditional Data Envelopment Analysis approach

Hard data alone are not sufficient to evaluate local police effectiveness in the new age of community policing. Citizens can provide useful feedback regarding strengths and weaknesses of police operations. However, citizen satisfaction indicators typically fail to accurately convey the multidimensional nature of local policing and account for characteristics that are non-controllable for the local police departments. In this paper, we construct a measure of perceived effectiveness of community oriented police forces that accounts for both multidimensional aspects of local policing and exogenous influences. In specific, this paper suggests the use of a multivariate conditional, robust order-m version of a non-parametric Data Envelopment Analysis approach with no inputs. We show the potentiality of the method by constructing and analyzing perceived effectiveness indicators of local police forces in Belgium. The findings suggest that perceived police effectiveness is significantly conditioned by the demographic and socioeconomic environment.

Does ICT matter for effectiveness and efficiency in mathematics education

Abstract ICT infrastructure investments in educational institutions have been one of the key priorities of education policy during the last decade. Despite the attention, research on the effectiveness and efficiency of ICT is inconclusive. This is mainly due to small-scale research with weak identification strategies which lack a proper control group. Using the 2011 ‘Trends in International Mathematics and Science Study’ (TIMSS) data, we define by a Mahalanobis matching a control group with similar student, teacher, school and regional characteristics. The results indicate that accounting or not accounting for these characteristics, may considerably alter the estimated impact of ICT. This suggests that a correction for characteristics related to the student population, teaching staff, administrative personnel and school management is warranted in the evaluation of the impact of ICT.

Geometric mean quantity index numbers with Benefit-of-the-Doubt weights

Geometric mean index numbers are a multiplicative aggregation of (price or quantity) ratios with their importance exponents/weights derived from one or more observed budget shares. In the specific context of composite indicator construction, we propose to use the budget shares as naturally generated by the linear Benefit-of-the-Doubt model. This approach is directly inspired by the literature on index number theory. Our basic model is easily extended to provide transitive composite indicator orderings in a multilateral setting. Also, a multi-factor decomposition is proposed to explain the intertemporal evolution of a single entity. We illustrate our results with social inclusion data for the EU-countries.

Constructing a Knowledge Economy Composite Indicator with Imprecise Data

This paper focuses on the construction of a composite indicator for the knowledge based economy using imprecise data. Specifically, for some indicators we only have information on the bounds of the interval within which the true value is believed to lie. The proposed approach is based on a recent offspring in the Data Envelopment Analysis literature. Given the setting of evaluating countries, this paper discerns a ‘strong country in weak environment’ and ‘weak country in strong environment’ scenario resulting in respectively an upper and lower bound on countries’ performance. Accordingly, we derive a classification of ‘benchmark countries’, ‘potential benchmark countries’, and ‘countries open to improvement’.

Waste pricing policies and cost efficiency in municipal waste services: the case of Flanders

Major changes in municipal solid waste (MSW)-related services, such as the collection system, often require important initial investments, while the benefits on the output side typically occur over a longer time perspective. However, when analyzing the association between such policy choices and cost efficiency of MSW collection and disposal, the time horizon is often not taken into account. This article advocates the use of the Malmquist Productivity Index approach to measure and analyze cost efficiency changes over time after the introduction of the relative new weight-based waste collection and billing system in Flanders. The results and robustness checks indicate that introducing weight-based pricing does not lead to lower cost efficiency of MSW service provision.

A composite index of citizen satisfaction with local police services

This paper proposes a non-parametric methodology to construct composite scores of citizen satisfaction with local police corpses. More precisely, the paper advocates a custom made version of the popular Data Envelopment Analysis approach, also referred to as the ‘Benefit-of-the-Doubt’-model. The key advantage of this approach is that it weights the citizen satisfaction rates with the multiple local police functions and tasks into the composite score in an endogenous manner, thereby allowing for different values and interpretations of ‘good local policing’ among police corpses. The methodology is illustrated with citizen satisfaction data on a sample of Belgian local police corpses.

Accounting for Economies of Scope in Performance Evaluations of University Professors

Teaching and research are widely regarded as the two key activities of academics. We propose a tailored version of the popular Data Envelopment Analysis methodology to evaluate the overall performance of university faculty. The methodology enables accounting for the potential presence of economies of scope between the teaching and research activities. It is illustrated with a dataset of professors working at a Business and Administration department of a university college. The estimation results reveal that overall the performance scores of faculty decrease if we allow for spillovers from research to teaching and vice-versa.

On aggregating Benefit of the Doubt composite indicators

Abstract In this short communication, I propose a procedure to aggregate individual composite indicators (CIs) into a group CI (e.g., aggregate CI for a group of countries) that is based on the work of Fare and Zelenyuk (2003) that was originally presented in the efficiency measurement literature. It is shown that the implementation of the aggregation procedure into the CI-context is rather straightforward. To illustrate the procedure in the CI-setting, an aggregate CI for human development for the global EU-region is computed using the EU Member States’ HDI sub-indicators.