The Expediting Effect of Monitoring on Infrastructural Works. A Regression-Discontinuity Approach with Multiple Assignment Variables
TThe expediting effect of monitoring on infrastructural works. Aregression-discontinuity approach with multiple assignmentvariables
Giuseppe Francesco Gori, Patrizia Lattarulo, Marco Mariani
IRPET – Regional Institute for the Economic Planning of Tuscany
Abstract
Decentralised government levels are often entrusted with the management of public works andrequired to ensure well-timed infrastructure delivery to their communities. We investigatewhether monitoring the activity of local procuring authorities during the execution phaseof the works they manage may expedite the infrastructure delivery process. Focussing onan Italian regional law which imposes monitoring by the regional government on ”strategic”works carried out by local buyers, we draw causal claims using a regression-discontinuityapproach, made unusual by the presence of multiple assignment variables. Estimation isperformed through discrete-time survival analysis techniques. Results show that monitoringdoes expedite infrastructure delivery.
JEL classification: R53, H54, C21Keywords: Quality of Government, Procurement, Public investment, Regressiondiscontinuity design a r X i v : . [ ec on . GN ] F e b Introduction
Public infrastructural investment is widely acknowledged as a major driver of growth and com-petitiveness. However, the strength of the infrastructure-growth nexus crucially depends onmany context factors. In fact, the type, localisation and cost of infrastructures are key factorsin order to trigger relevant economic effects both in the short-run and over a longer time horizon(Crescenzi and Rodr´ıguez-Pose, 2012; Elburz et al., 2017; V¨alil¨a, 2020; Bogart, 2020). It is,moreover, increasingly recognised in the literature that the Governments’ ability to understandlocal needs and invest in pursuing such needs - bearing adequate costs and in a timely manner-is of paramount importance to ensure returns from public resources at the central and locallevels (Castells and Sol´e-Oll´e, 2005; Dabla-Norris et al., 2012; IMF, 2014; Acemoglu et al., 2015;Brueckner and Picard, 2015; Crescenzi et al., 2016). Unfortunately, such quality of governmentcannot be taken for granted, and the impact of infrastructure can be undermined by inefficienciesarising in all phases, from planning to execution. Public works may suffer, during their execu-tion, from cost increases and long durations. This paper focuses on the issue of work durations:public works taking a long time to complete may imply the dissatisfaction of collective needsand also lead to the delivery of manufactures at a time when they no longer serve the purposethey were planned for (Lewis and Bajari, 2011).Both time and cost inefficiencies have been widely investigated by the public procurementliterature. A very established view in this literature sees poor execution performances of publicworks mainly originating from imperfect regulatory frameworks that may prevent the appro-priate enforcement of procurement contracts. To this regard, attention has been extensivelydevoted to auction formats (e.g., Dimitri et al., 2006; Bajari et al., 2009; Bucciol et al., 2013;Decarolis, 2018; Coviello et al., 2018; De Chiara, 2020), to the degree of contract definition (e.g.,Krahmer and Strausz, 2011; D’Alpaos et al., 2013; Lewis and Bajari, 2014; Arve and Martimort,2016) and, more recently, to the role of systemic/institutional features such as the ”rule of law”(Coviello et al., 2018; Estache and Foucart, 2018). A less beaten track of research looks at thespecific role played by the characteristics of the action of the contracting administrations duringthe entire contract’s life cycle. According to this view, inefficiencies can originate from the lackof know-how and insufficient experience of some procuring authorities, especially small munic-ipalities (Brown and Potoski, 2003; Bandiera et al., 2009; Guccio et al., 2014; Warren, 2014;Saussier and Tirole, 2015; Baldi and Vannoni, 2017; Gori et al., 2017; Decarolis et al., 2019).Therefore, monitoring the action of such procuring authorities may prove useful to guarantee2he quality of their action and to ensure economic impact from infrastructures (e.g., Grigoli andMills, 2013; Finan et al., 2017; Lattarulo, 2020). To the best of our knowledge, no empiricalstudy to date has tried to understand to what extent monitoring the action of local procuringauthorities may improve the situation. This is a gap that the current paper starts to fill, bylooking at whether monitoring the action of local governments during the works’ execution phaseleads to faster infrastructure delivery. In a public administration perspective, monitoring theaction of a procuring authority is aimed at encouraging the proper management of public funds(Finan et al., 2017), including those – such as the European Regional Development Fund – thatare also devoted to infrastructure investment at the regional level (Farole et al., 2011).The opportunity to investigate the effect of monitoring on the execution times of publicworks managed by local buyers is provided by an Italian regional law, requiring municipalitiesto perform careful checks on the execution of works they procured out and report to the regionaladministration, provided these works have certain characteristics . In particular, the regionalparliament of Tuscany, in central Italy, passed Law 35 in 2011, stating that this monitoringregime must be enforced on all projects that are deemed strategic, because they meet a financialsize and benefit from co-financing by the regional government above certain thresholds. Thislaw was specifically designed to expedite the time performance of local public works.To estimate the causal effect of monitoring on the execution duration of public works man-aged by local authorities, we exploit the particular assignment mechanism established by Law35, and use a sharp regression discontinuity approach. Since works’ assignment to monitoringis determined by the simultaneous fulfillment of multiple quantitative criteria, the usual regres-sion discontinuity methodologies may not apply to our case without appropriate extensions. Tothe best of our knowledge, only the recent contribution by Choi and Lee (2018) extends sharpregression discontinuity designs to situations, like ours, where all assignment variables have tobe above certain thresholds to determine treatment receipt. Therefore, we use this approach.However, a further complication in our empirical analysis is due to the fact that, at the end ofthe observation period, the execution of the works in our dataset may be either completed orstill ongoing. This circumstance requires to define a causal effect as the contrast, at the cutoffvalue of both assignment variables, between the hazard of work completion with and withoutmonitoring. Building on the ideas developed by Caliendo et al. (2013) with respect to the singleassignment variable context, the estimation of such hazards of completion is accomplished using”local” discrete-time duration models, adapted to our multiple assignment variables setting.We find that the causal effect of monitoring on time-to-completion is positive at the threshold3alues of the two assignment variables. In particular, our results suggest that monitoring maysoon provide a boost to the work’s execution, but that such effect diminishes later in time.The rest of the paper is organised as follows. Section 2 describes the monitoring policy underinvestigation and the data used for the analysis. Section 3 discusses the methodology. Section4 presents the results of the analysis; and Section 5 is devoted to sensitivity checks. Section 6concludes.
As mentioned in the Introduction, the policy under investigation consists of the monitoringactivity exerted by the regional government on particular types of local infrastructural projectsmanaged by lower-level contracting authorities. This kind of regional monitoring is expectedto speed up the execution of works by incentivising higher attention and more careful controlby such authorities during works’ execution. In line with the programme evaluation literature,such policy will be referred to as the treatment (Imbens and Wooldridge, 2009).Public procurement in Italy sees the presence of different buyers at different governmentlevels. Local governments, and in particular municipalities among them, account for the highestshare of the overall works (Decarolis and Giorgiantonio, 2015; Gori et al., 2017), which makes thepoint of this paper particularly relevant. In general, the activity that any local procurer shouldexert during the work’s execution consists of checks on the work site, aimed at ascertainingwhether the speed and the quality of execution proceeds as scheduled. These checks are usuallyperformed by the technical personnel of the procuring authority. Their success, however, cannotbe taken for granted, especially if a procurer wavers in enforcing the contract.To expedite public works managed by lower-level procurers, the regional parliament of Tus-cany passed law No. 35, in 2011, mandating regional monitoring for local works that, at thesame time, meet a financial size of at least 500,000 Euros and benefit from co-finanancing fromthe regional government of at least 50% of their value. The monitoring regime introduced by thelaw requires contracting authorities to perform thorough checks on the execution schedule, andshare them with the regional offices. In the event of execution slowdowns, the regional officescan thus urge specific actions by the contracting authorities. Only in case the latter are facingparticularly complex issues, the former may provide ad hoc technical, case-specific assistance.Law 35 does not require additional red tape. Instead, it creates an environment where attentionon public works from peripheral contracting authorities is substantively incentivised.4e use an administrative dataset collecting detailed information on all public contracts inItaly: the database of Italian public contracts BDNCP (
Banca Dati Nazionale Contratti Pub-blici ). Such dataset reports information on the buyers and on contract characteristics, includingauction formats, expected costs and durations. Once the contract’s execution is completed, in-formation on actual costs and durations is added. In order to carry out our analysis on the effectsof Tuscany’s regional law 35/2011 we consider the section of the BDNCP dataset comprisingall public works procedures started by Tuscan municipalities from 2011 to 2017, including 5,559projects. Of these, 219 are the projects subject to the regional monitoring regime mandated bylaw 35 (Table 1). Table 1: Descriptive statistics.Not Monitored Monitored OverallShare of regional co-financing 14.2% 79.9% 16.8%Total project cost (Euros) 254,456 1,696,169 311,263Actual duration of completed projects (Months) 8 16.9 11.9Share of right-censored projects 22% 21.9% 22%No. of works under investigation 5,340 219 5,559The paper aims at evaluating whether the application of monitoring causes a desirable re-duction in the execution times of public works. The actual duration of work executions is knownonly for project – the vast majority – that reach their completion before the end of the obser-vation period, i.e. before the 31st December, 2017. Of the remaining projects, we only knowthat they will reach completion at some unknown time point outside of the observation period.These right-censored projects amount to 22% of all projects (Table 1). Their presence moti-vates the usage of survival analysis techniques in the following analysis based on the regressiondiscontinuity design methodology.
Let i = { , ..., } denote the project, S (1) i its financial size, and S (2) i its share of regionalco-financing. Also, let M i = { , } be the treatment indicator for project i , where M i = 05tands for no monitoring (control status), and M i = 1 stands for monitoring (treatment status).Finally, let t = { , ..., T } denote a certain time point after the start of the project’s execution.We adopt the potential outcomes framework. Under the assumption that the potentialoutcomes of one unit do not depend on the treatment status of other units (Rubin, 1986), eachproject i has two potential outcomes at each t : Y it (0) if the project is originally assigned to thecontrol status, and Y it (1) if it is originally assigned to the treatment status. In this application, Y it (0) is an indicator for completion occurring at time t in case project i received no regionalmonitoring, Y it (1) is an inidicator for completion occurring at time t in case project i receivedsuch monitoring.In theory, the project-level effect of monitoring is defined as the contrast – at each time t – between the two previous potential outcomes, Y it (1) − Y it (0). In practice, these potentialoutcomes cannot be observed simultaneously for a same project. Therefore, attention shiftstowards estimands based on the contrast of the hazard of completion under different treatmentconditions, such as h t (1) − h t (0), i.e. on the differential probability that a work at the cutoffvalues reaches completion in t , given that completion was not reached earlier, if it was originallyassigned to the treatment rather than to the control status. Attention also shifts towards theassumptions that are required to interpret the previous contrast as a causal effect, in light ofthe particular mechanism that determines the assignment of projects to monitoring.In this study, each project i is assigned to treatment as a deterministic function of its financialsize ( S (1) i ) and of its share of regional co-financing ( S (2) i ). In particular, monitoring is mandatedif project i simultaneously satisfies the following conditions, S (1) i ≥ c (1) , with c (1) = 500 , S (2) i ≥ c (2) , with c (2) = 0 .
5. On the other hand, project i receives no regionalmonitoring in the following three situations: ( S (1) i < c (1) , S (2) i ≥ c (2) ); ( S (1) i ≥ c (1) , S (2) i < c (2) );and ( S (1) i < c (1) , S (2) i < c (2) ). It may be useful, for the remainder of this paper, to define twofurther indicators. The first one, A (1) i , takes on the value of one if project i is equal or abovethe cutoff value in terms of financial size, i.e. if S (1) i ≥ c (1) , while it takes on the value of zerootherwise. Similarly, the second indicator, A (2) i , takes on the value of one if project i is equal orabove the cutoff value in terms of regional co-financing, i.e. if S (2) i ≥ c (2) , while it takes on thevalue of zero otherwise. 6 .2 A regression discontinuity approach The particular assignment mechanism previously described allows to adopt a sharp regres-sion discontinuity approach. Regression discontinuity designs are highly popular with appliedeconomists and other social scientists because of the possibility they guarantee to recover causaleffects in observational settings without invoking particularly strong assumptions (Imbens andLemieux, 2008; Lee and Lemieux, 2010; Cattaneo and Escanciano, 2017; Choi and Lee, 2017).In the traditional econometric approach, the assignment variable is considered a pretreatmentcovariate and, under relatively mild continuity (also termed smoothness) assumptions, inferencerelies on some form of extrapolation at a cutoff point. A handful of recent methodologicalpapers have explored the possibility to view regression discontinuty designs as locally randomisedexperiments in a region around the cutoff, requiring assumptions other than smoothness foridentification (e.g., Mattei and Mealli, 2016; Cattaneo et al., 2015). In this study, we will followthe more established econometric strand of the methodological literature, as it already providessome extensions to cases, like ours, where the assignment variables are more than one.Much has been written on regression discontinuity designs in settings where there is a sin-gle variable determining treatment assignment, and a single cutoff. Some recent papers haveextended such methodology to cases with multiple cutoffs and/or rankings related to such assign-ment variable (e.g., Cerqua and Pellegrini, 2014; Cattaneo et al., 2016). Designs with multiple,different assignment variables have also been explored in a number of methodological papers,dealing with the situation where only one of these variables has to be above a certain cutoff todetermine treatment assignment (e.g., Jacob and Lefgren, 2004; Matsudaira, 2008; Imbens andZajonc, 2011; Papay et al., 2011; Reardon and Robinson, 2012; Wong et al., 2013). Instead, thepaper by Choi and Lee (2018) provides methodological guidance that is particularly suited forcases, like the one analysed in this study, where treatment receipt depends on multiple assign-ment variables simultaneously being greater or equal to certain cutoff values, rather than ononly one out of multiple assignment variables being greater or equal to a cutoff value.However, the approach by Choi and Lee (2018), originally formalised for continuous out-comes, requires to be adapted to cases, such the one in this study, where unit-level outcomesare not so, and the estimand of interest is the contrast – at each time t – between the hazardof completion under the treatment and the control conditions in correspondence of the cutoffvalues c (1) and c (2) : τ t = h t (1) | c (1) , c (2) − h t (0) | c (1) , c (2) . (1)7owever, it is worth noting that h t (0) | c (1) , c (2) involves potential outcomes that can be associatedto three types of original control statuses: Y it (0) A (1) i =0 ,A (2) i =1 | c (1) , c (2) ; Y it (0) A (1) i =1 ,A (2) i =0 | c (1) , c (2) ;and Y it (0) A (1) i =0 ,A (2) i =0 | c (1) , c (2) ). In a similar fashion, the potential outcome associated underthe treatment status can be viewed as Y it (1) A (1) i =1 ,A (2) i =1 | c (1) , c (2) .The approach proposed by Choi and Lee (2018) allows for the possibility that being aboveone of the two cutoff values may systematically influence the potential outcome, whatever theposition with respect of the other cutoff value. Such influence is viewed as a fixed quantityirrespective of treatment assignment and has to be neutralised. To this end, the estimand ofinterest at each time t can be written as follows: τ t = h t (1) A (1) i =1 ,A (2) i =1 | c (1) , c (2) − h t (0) A (1) i =0 ,A (2) i =0 | c (1) , c (2) − (cid:104) h t (0) A (1) i =1 ,A (2) i =0 | c (1) , c (2) − h t (0) A (1) i =0 ,A (2) i =0 | c (1) , c (2) (cid:105) − (cid:104) h t (0) A (1) i =0 ,A (2) i =1 | c (1) , c (2) − h t (0) A (1) i =0 ,A (2) i =0 | c (1) , c (2) (cid:105) . (2)As recalled at the beginning of this Section, identification in the regression disconinuityapproach usually relies on relatively mild assumptions regarding the continuity, at the cutoff, ofthe average potential outcomes under the control condition (e.g., Imbens and Lemieux 2008).In a multiple assignment variable setting like the one investigated in this study, the followingcontinuity assumptions must be invoked to guarantee that the causal effect in (2) is unbiased(Choi and Lee, 2018):(a) : h t (0) A (1) i =0 ,A (2) i =0 | c (1) , c (2) ) = h t (0) A (1) i =1 ,A (2) i =1 | c (1) , c (2) );(b) : h t (0) A (1) i =1 ,A (2) i =0 | c (1) , c (2) ) = h t (0) A (1) i =1 ,A (2) i =1 | c (1) , c (2) );(c) : h t (0) A (1) i =0 ,A (2) i =1 | c (1) , c (2) ) = h t (0) A (1) i =1 ,A (2) i =1 | c (1) , c (2) ).Under assumptions (a), (b) and (c), the estimand in (2) can be re-written as follows, usingobserved quantities rather than potential outcomes: τ t = lim ( S (1) ,S (2) ) → ( c (1)+ ,c (2)+ ) h t − lim ( S (1) ,S (2) ) → ( c (1) − ,c (2) − ) h t − (cid:34) lim ( S (1) ,S (2) ) → ( c (1)+ ,c (2) − ) h t − lim ( S (1) ,S (2) ) → ( c (1) − ,c (2) − ) h t (cid:35) − (cid:34) lim ( S (1) ,S (2) ) → ( c (1) − ,c (2)+ ) h t − lim ( S (1) ,S (2) ) → ( c (1) − ,c (2) − ) h t (cid:35) . (3)8he previous identification assumptions cannot be directly tested, as they involve quantities –namely the potential outcome of treated projects had they not been treated – which can neverbe observed in the data. However, their plausibility will be extensively assessed in an indirectfashion in Section 5. The presence of right-censored works’ execution spells suggests the use of an approach basedon survival analysis techniques. Although the works’ execution durations are expressed in daysin the original dataset, the use of a discrete-time approach ensures gains in terms of flexibility,especially when – as in the case of this study – the assumption of proportional hazards is notinvoked. This advantage comes at the acceptable price of the hazard being defined on someaggregate time periods, rather than on a daily basis. After a thorough inspection of the works’execution durations in the dataset at hand, the chosen aggregate time periods are as follows: upto 6 months; 6–11 months; 12 months or longer.Building on Caliendo et al. (2013), the estimation of the potential outcomes that are con-trasted in τ t is performed through a discrete-time hazard model, which actually takes the formof a pooled generalised linear model with a logit link (Kalbfleisch and Prentice, 2011). Suchmodel is specified in a way that returns estimates at the boundary point, which is defined – inthis sudy – by the two cutoff values, and runs on dataset where each work is repeated as manytime periods as its execution lasts. Unlike Caliendo et al. (2013), the analysis conducted hereallows for non-proportional odds, which entails that the size of the estimated discontinuity mayvary depending on the specific post-treatment time period, as the coefficients estimated in themodel for the variables M i , A (1) i and A (2) i are time-specific.Let h c (1) ,c (2) t , t = { , , } , denote the hazard of completion at time t for a work at the cutoffvalues c (1) , c (2) . Also, let P ( t ) i denote three indicators for each of the three time periods; Q ( k ) i , k = { , ..., } , denote four indicators for the quadrants defined by the combined values of A (1) i and A (2) i ; and j = { , } index the assignment variables S (1) and S (2) . Finally, let D ( j ) i = S ( j ) i − c ( j ) be the distance separating work i from the cutoff value of the assignment variable j = { , } .Such distance is what makes it possible to interpret the coefficients associated with the remainingvariables as boundary point estimates. Taking the logit of h c (1) ,c (2) t as an outcome, the following9odel can be estimated: logit ( h c (1) ,c (2) t ) = (cid:88) t =1 β ( t )0 P ( t ) i + (cid:88) t =1 β ( t )1 M i P ( t ) i + (cid:88) j =1 3 (cid:88) t =1 β ( jt )2 A ( j ) i P ( t ) i + (cid:88) k =1 2 (cid:88) j =1 β ( kj )3 Q ( k ) i D ( j ) i (4)where all the coefficients have the form of log-odds. Then, such coefficients can be used to predictthe potential outcomes at the cutoffs that are involved in the estimation of τ t (see Equation 2).In particular, exp ( β ( t )0 ) / [1 + exp ( β ( t )0 )] predicts the potential outcome h t (0) A (1) i =0 ,A (2) i =0 , while exp ( β ( t )0 + β ( t )1 ) / [1 + exp ( β ( t )0 + β ( t )1 )] directly predicts the quantity given by h t (1) A (1) i =1 ,A (2) i =1 − (cid:104) h t (0) A (1) i =1 ,A (2) i =0 − h t (0) A (1) i =0 ,A (2) i =0 (cid:105) − (cid:104) h t (0) A (1) i =0 ,A (2) i =1 − h t (0) A (1) i =0 ,A (2) i =0 (cid:105) . Therefore, τ t is ultimately estimated as: τ t = exp ( β ( t )0 + β ( t )1 )1 + exp ( β ( t )0 + β ( t )1 ) − exp ( β ( t )0 )1 + exp ( β ( t )0 ) (5)for t = {
6- months; 6-11 months; 12+ months } .The methodological literature devoted to regression discontinuity designs recommends totrim observations too far away from the cutoff values prior to the start of any local estimationprocedure (Imbens and Lemieux, 2008). In this study, works are discarded from the analysisif one of the following conditions is verified: S (1) i < S (1) i > S (2) i < . S (2) i > .
95. After such trim, the number of works under investigation falls from 5,559 to331 (Table 2). Then, within the remaining region, a selection procedure of a bandwidth h can be implemented to reach a reasonable compromise between the opposing needs of biasreduction on the one hand, and precision enhancement (through smaller variance) on the other.In particular, a very narrow h around the cutoff values is expected to guarantee little bias, atthe price of a high variance of estimates. A very wide h is expected to lead to the oppositeresult. Another advantage from focussing on a subset of units that are relatively close to thecutoffs is that the estimation can be performed using models with relatively simple specifications,instead of resorting to complex polynomial structures (Imbens and Lemieux, 2009; Gelmanand Imbens, 2014). A number of bandwidth selection procedures have been proposed by theliterature (Imbens and Lemieux, 2008; Imbens and Kalyanaraman, 2012; Calonico et al., 2014).However, all the cited bandwidth selectors were conceived for settings with a single assignmentvariable. Therefore, their use in the case study under investigation here is not straightforward,as these approaches require to be generalised to the multiple assignment variable setting. To thisend, the simple leave-one-out cross-validation procedure proposed by Imbens and Lemieux (2008)10s the only one that can be generalised as part as a non-technical paper like this one. In its orginalversion, the goal of such cross-validation procedure is to find a bandwidth h guaranteeing thatthe chosen model specification minimises, at each side of the cutoff value, the root mean squarederror (RMSE). The resulting bandwidth may be symmetrical or asymmetrical with respect tothe cutoff value, with the choice often depending on the number of observations available ateach side (Imbens and Lemieux, 2008). In the presence of a continuous outcome variable, theprocedure can be extended to the multiple assignment variable setting by comparing, separatelyin each quadrant, the RMSE from the model associated with all possible pairs of values of theassignment variables, ultimately selecting the pair of values that exhibit the lowest RSME. Since,in this study, the outcome is binary rather than continuous, the RMSE is replaced by the Brierscore, which is the mean squared error of the probability forecast.Table 2: Results of the bandwidth selection process Quadrant Original Data After Trimming Optimal Bandwidth S (1) i /1000 S (2) i Projects S (1) i /1000 S (2) i Projects S (1) i /1000 S (2) i Projects A (1) i = A (2) i = 1 [500; 9595] [0.5; 1] 219 [500; 1000] [0.5; 0.950] 65 [500; 929] [0.5; 0.846] 49 A (1) i = 1, A (2) i = 0 [500; 14100] [0; 0.5) 486 [500; 1000] [0.05; 0.5) 48 [500; 1000] [0.166; 0.5) 45 A (1) i = A (2) i = 0 [42; 500) [0; 0.5) 4255 [150; 500) [0.05; 0,5) 108 [356; 500) [0.065; 0.5) 23 A (1) i = 0, A (2) i = 1 [41; 500) [0.5; 1] 599 [150; 500) [0.5; 0.95] 110 [356; 500) [0.5; 0.82] 26Total 5559 331 143 The bandwidths resulting from this procedure are shown in Table 2. The number of worksfalling in the selected bandwidth is 143, corresponding to 340 work*period observations.
The estimates of all coefficients of the model described in (4), accompanied by their standarderrors and 90% confidence intervals, are shown in Table 3. Standard errors are estimated ac-counting for observations, i.e. a work i in time period t , being clustered at the level of work i . Some of the estimated coefficients, namely ˆ β ( t )0 and ˆ β ( t )1 , will be involved in predicting thetwo hazards required – as in Equation 5 – to estimate the causal effect τ t . For the moment, it isworth noting that the estimated value of coefficients β ( t )0 increases over time. This fact revealsthat the duration dependence of a work at the cutoffs but not meeting any of the two assignmentcriteria is naturally positive (i.e. the hazard of completion increases when the actual execution11ime gets longer and longer). Also, from the inspection of the estimated coefficients ˆ β ( t )1 andtheir confidence intervals, we get a first intuition that, if such a work had instead been assignedto monitoring, this would have yielded some benefits.When predicting the hazards in Equation 5, the estimated coefficients ˆ β ( jt )2 and β ( kj )3 will beneutralised by fixing their associated covariate at the value of zero. However, before implement-ing such neutralisation, it may be interesting to look at ˆ β ( jt )2 and at their confidence intervals.Both reveal whether – in this specific application – being above one of the two cutoff valuessystematically affects the outcome, whatever the position with respect of the other cutoff. FromTable 3, it definitely seems that being above each one of the two cutoffs may decrease the hazardof completion of works, especially in the earlier time period(s). If so, it was worthwhile to havethese coefficients in the model. Table 3: Estimated coefficientsCoeff. t Estimate Standard Error 90% C.I.1 [= -6 months] -0.558 0.718 -1.739 0.623 β ( t )0 β ( t )1 β (1 t )2 β (2 t )2 Note: The eight coefficients β ( kj )3 associated with distance to the cutoff values are justtechnical components of this particular type of local estimation procedure. Therefore, theirestimates are not reported in the Table. Table 4 reports the estimated causal effect ˆ τ t , in the three time periods chosen, accompaniedby cluster-robust standard errors, 90% confidence intervals, and the p -values associated with thenull hypothesis that the estimated effect is not positive (right-tailed test). The discontinuity in12able 4: Estimated causal effects at the cutoffsTime period ˆ τ t Standard Error p -value 90% C.I.-6 months 0.634 0.164 0.000 0.365 0.9046-12 months 0.240 0.137 0.039 0.015 0.46512+ months 0.105 0.083 0.102 -0.031 0.241the hazard of completion is estimated at 63.4% up to 6 months since the work’s execution begun,then it decreases to 24% in the second execution semester, and finally falls to 10.5% after oneyear. The probability that the estimated effect might not be positive is always very low. However,in the last period, such probability is not low enough to satisfy the conventional requirementsfor statistical significance. These results indicate that a positive effect of monitoring does exist,although it is decreasing over time. In fact, it can be appreciated especially in the early stage ofthe work’s execution, where its 90% interval prediction ranges from 36.5% to 90.4%. Later on,the interval prediction shifts to lower – but still positive – values, and ends up covering zero inthe last period.A likely interpretation of the previous result is that the analysed monitoring scheme maybe useful to address relatively minor issues arising during the work’s execution, which couldnotwithstanding lead to some time escalations. Under these circumstances, monitoring may infact incentivise procurers to do what is needed and feasible to stick to the original schedule.If, however, the work’s execution lasts too long (i.e. one year or longer), a positive effect ofmonitoring is surrounded by a considerable amount of uncertainty, possibly due to the fact that,by that time, the hazard of completion would be quite high even without monitoring. As forprojects not reaching completion, it cannot be ruled out that their time escalation is causedby major issues that monitoring alone is not able to solve. Such major issues may includeunexpected events of different kinds, often requiring the design and implemetation of ad hoc technical and legal solutions.Conducting, in the current three-dimensional setting, the graphical analyses that are usualin two-dimensional regression-discontinuity studies may be not straightforward. However, tographically explore discontinuities at the cutoffs, we resort, in the Appendix, to the ”centeringapproach” (Wong et al., 2013; Cheng, 2016), which allows to investigate such discontinuities inthe simpler two-dimensional space. 13 Supplementary Analyses
The possibility to identify and estimate causal effects relies on the continuity assumptions in-voked in Section 3.2. Such assumptions state that, in the absence of any intervention, the hazardof completion at the cutoffs would have been the same whatever the quadrant in which worksare located. In other words, they rule out that, at least on average, there is any systematicsorting of the observations into the quadrants, which would lead to a discontinuity that cannotbe abscribed to the intervention.The continuity assumptions are not directly testable. An indirect way to address the issue isto evaluate and test if there is any discontinuity at the cutoffs in the level of covariates, i.e. onpredetermined variables that – by definition – cannot be affected by the intervention (Imbensand Lemieux, 2008). Here, covariates act as pseudo-outcomes on which a pseudo-effect of theintervention has to be estimated. If a pseudo-effect is found statistically different from zero, thisraises suspects on the plausibility of the continuity assumptions that were invoked.To link this assessment to the three identification conditions in Section 3.2, we select threedifferent sets of control units depending on their position in terms of the two assignment variables S (1) , S (2) . Then, we estimate the pseudo-effect, each time using the set of treated units joinedwith an alternative set of controls. For a continuous pseudo-outcome (covariate), the model thatcan be used to predict its average, within each alternative treated-control set, is E ( Y i ) c (1) ,c (2) = γ M i + M i · (cid:88) j =1 γ ( j )2 D ( j ) i + (1 − M i ) · (cid:88) j =1 γ ( j )3 D ( j ) i , (6)where the parameter γ directly quantifies the pseudo-effect. If the pseudo outcome is binary,a logit based on a similar linear predictor can be used to forecast the relevant probabilities andcheck for any discontinuity. The pseudo-outcomes considered are three covariates, drawn fromthe field literature (e.g., Bajari et al., 2009; D’Alpaos et al., 2013; Decarolis and Palumbo,2015): Expected duration (number of months of execution set by the contract);
Auction (= 1if the tendering procedure is an open or restricted auction, = 0 if the tendering procedure is anegotiation); and
Lowest-bid (=1 if the award criterion is lowest-bid; = 0 if the award criterion ismost economically advantageous tender). When estimating each of the previous pseudo-effects,a case-specific cross-validation procedure similar to the one described in Section 3.3 is used todetermine the optimal bandwidth. 14able 5 reports the estimated pseudo-effects, their standard errors and the p -value for thenull hypothesis that such pseudo-effects are equal to zero (two-tailed test), which is exactly whatwe hope for. Since the null hypthesis is never rejected, there are no reasons to suspect that theidentification assumptions stated in Section 3.2 are implausible.Table 5: Covariates as pseudo-outcomes Pseudo-outcome Control set Pseudo-effect Standard Error p -valueExpected duration (cont.) A (1) = 1 , A (2) = 0 -4.931 6.188 0.437 A (1) = A (2) = 0 -10.242 7.404 0.197 A (1) = 0 , A (2) = 1 -9.166 6.861 0.190Auction (1/0) A (1) = 1 , A (2) = 0 -0.272 0.519 0.519 A (1) = A (2) = 0 -0.291 0.516 0.573 A (1) = 0 , A (2) = 1 -0.294 0.534 0.581Lowest bid (1/0) A (1) = 1 , A (2) = 0 0.504 0.450 0.263 A (1) = A (2) = 0 0.534 0.511 0.296 A (1) = 0 , A (2) = 1 0.515 0.450 0.276 Note: All pseudo-effects are estimated using the set of treated units, A (1) = A (2) = 1, each timejoined with an alternative set of control units. Additional insights on the plausibility of the identification assumptions may come fromsearching for discontinuities at the cutoffs in the assignment variables density functions (Imbensand Lemieux, 2008; Imbens and Wooldridge, 2009). Such discontinuity is usually interpretedas the result of a manipulation that occurred in the background of the assignment stage, po-tentially (but not necessarily) leading to sorting. This approach, very fashionable in regression-discontinuity design applications, makes sense only if one strongly believes that the hypotheticalmanipulation occurs only in one direction, while it is rather inconclusive otherwise (for furtherlimits of manipulation testing see Choi and Lee (2020)). The methodological literature to dateprovides solutions applicable only to the standard setting with a single assignment variable(McCrary, 2008; Cattaneo et al., 2018). The generalisation of such approach to a two assign-ment variable setting, that is, in a three-dimensional space, is still a challenge ahead for themethodological literature, which can by no means be faced by the current paper. However, toget an intuition on possible density discontinuities at the cutoffs, we may resort, again, to the”centering approach” (Wong et al., 2013; Cheng, 2016). Such analysis, explained and conductedin the Appendix, provides no support to the idea that manipulation constitutes a serious threat15o identification in our analysis.
It is recommended by the methodological literature to assess whether the main results of theanalysis are to be found only in the presence of the selected bandwidth. If so, such results maylose some of their credibility (Imbens and Lemieux, 2008). Since the selection of an optimalbandwidth is desirable in that it strikes a balance between bias and precision, it may be par-ticularly reasonable to check for the stability of results when the selected bandwidth undergoessmall changes that should not alter too much either the amount of accepted bias or the precisionof estimates. The small changes analysed here correspond to one-percent increase or decreaseof each of the eight ”external” limits of the optimal bandwidth (Table 2). It must be stressedthat, in this application, a one-percent change in the bandwidth limit very often entails a morethan proportionate change in the number of projects falling in the resulting new bandwidth.For instance, a 5% decrease of all the limits of the bandwidth diminishes the number of worksby -13%; while a 5% widening of the limits increases the number of works by 12%.Figure 1:
Sensitivity to bandwidth choice. Estimated τ t and 90% confidence intervals for different levelsof the bandwidth ± Expediting the final delivery of infrastructural works is a priority of the political agenda ofmany countries, since economic impact can be undermined by long work durations. Many arethe factors that can affect works’ completion. Among these, a relevant one - which has been sofar neglected by the empirical literature - regards the opportunity to monitor local contractingauthorities during the execution of the works they commissioned. An adequate level of effort,by local authorities, to stick to the original schedule, could ensure that works proceed at a fasterpace, as unexpected hitches can be addressed promptly.This paper explores the effects of a regional monitoring policy aimed at encouraging suchefforts by local authorities, in the attempt to expedite the execution of public works. The policyunder investigation is targeted at a subset of public works deemed strategic, whose assignmentto monitoring by the regional government is a deterministic function of both their financialsize and the share of co-financing by the regional government itself. This peculiar assignmentmechanism calls for the adoption of a regression-discontinuity approach based on multiple scoresto draw causal claims. Since the outcome of interest is the length of execution, estimation isperformed through an appropriately specified, local discrete-time duration model for the hazardof completion of works.The results suggest that, at the cutoff values of the two assignment variables, the causaleffect of monitoring on time-to-completion is positive, especially during the earlier stages ofexecution. This kind of result is new in the empirical field literature. Further research is neededto formulate general policy recommendations, including an analysis of the effects of monitoringon final infrastructure costs and, an investigation of its effect away from the cutoffs (Angristand Rokkanen, 2015), under assumptions appropriately generalised to the multiple assignmentvariable setting. However, the first results highlighted here are promising. Increased monitoringcould contribute to ensure higher returns from public investment. Given that local infrastructureis often co-financed by regional, national or EU funds, it could also help ensure that strategiesformulated at these governance levels are not severely undermined at the moment of their finalimplementation on the territories. 17 ppendix
Conducting the usual graphical analyses on the outcome discontinuity at the cutoff, as well asmanipulation tests, is particularly demanding with two assignment variables, resulting into amulti-dimensional space. Such a challange is outside the scope of the current paper. To keepthings as easy as possible, we may use the ”centering approach” (Wong et al., 2013; Cheng, 2016).Such procedure collapses the two assignment variables S (1) i and S (2) i into a single one, therebymaking it possible to conduct the analyses of interest in the two-dimensional space. Since ourassignment variables have different measurements units, we first standardise them with respectto their cutoffs values. Thus we obtain the new V (1) i and V (2) i , which are comparable and takeon the value of zero at the cutoffs c (1) and c (2) respectively. To collapse these standardisedvariables into a single one, we define a new assignment score as the Euclidean distance betweeneach unit’s position ( V (1) i , V (2) i ) and the origin (0 , Z i = (cid:113) ( V (1) i − + ( V (2) i − . The newassignment score Z i is centered on its cutoff c ( Z ) and, by construction, such cutoff takes on thevalue of zero. Graphical Analysis
To conduct a two-dimensional space graphical analysis of the discontinuity at the cutoff we focuson the cumulative hazard of completion at the end of the observation period, H ( Z ) | T , whichamounts to the probability that a project is completed at some time earlier or equal to time T .The quantities of interest are as follows:(A) H ( Z ) A (1) i = A (2) i =1 | T represents the cumulative hazard of completion observed for treatedprojects. Notice that this gross quantity may be affected by treatment but also by the factof merely being above either cutoff values of the assignemnt variables;(B) H ( Z ) A (1) i =1 ,A (2) i =0 | T is the observed cumulative hazard for untreated projects that lie abovethe cutoff of the financial size assignment variable but below the cutoff of the co-financingassignment variable;(C) H ( Z ) A (1) i = A (2) i =0 | T is the observed cumulative hazard for untreated projects that lie belowboth cutoff values of the assignemnt variables;(D) H ( Z ) A (1) i =0 ,A (2) i =1 | T is the observed cumulative hazard for untreated projects that lie belowthe cutoff of the financial size assignment variable but above the cutoff of the co-financingassignment variable; 18E) ˜ H ( Z ) A (1) i = A (2) i =1 | T represents the cumulative hazard of completion for treated projectsnet of the influence exerted by the fact of merely being above either cutoff values of theassignemnt variables. This quantity is not directly observed in the data. However, buildingon Section 3.2 (equation 2 in particular), it can be reconstructed as :˜ H ( Z ) A (1) i = A (2) i =1 | T =( H ( Z ) A (1) i = A (2) i =1 − H ( Z ) A (1) i =0 ,A (2) i =1 − H ( Z ) A (1) i =1 ,A (2) i =0 + 2 · H ( Z ) A (1) i = A (2) i =0 ) | T. (7)The discontinuity of major graphical interest is the one at the cutoff c ( Z ) , corresponding tothe value of zero, of the assignment score Z , between the quantities ( E ) and ( C ). However, itmay be worthwhile to look also at the other discontinuities at c ( Z ) , in order to get insights onhow surpassing either thresholds affects the cumulative hazard of completion H ( Z ) | T . Figure2 suggests that the discontinuity between ( E ) and ( C ) is positive and amounts to 0 .
18 points.This graphical result is in line with the main outcomes of the causal analysis conducted in thispaper. Moreover, the Figure shows how just surpassing either cutoffs entails downward jump of H ( Z ) | T which is also in line with the results yielded earlier in this paper. Manipulation testing
The goal is to assess the null hypothesis that density f ( . ) is equal at either side of the cutoff c ( Z ) of the single assignment score, that is, H : lim Z → c ( Z ) − f ( Z ) = lim Z → c ( Z )+ f ( Z ). We conduct thisanalysis using the robust bias-corrected statistic and, more in general, the approach developedby Cattaneo et al. (2018), which in turn builds on McCrary (2008). To make this assessment as informative as possible, we can select different sets of control unitsdepending on their position in terms of the two original assignment variables S (1) , S (2) . Test[1] in Table 6 involves all controls whatever their original position. In such global test we areactually contrasting the density at the cutoff of treated units on one hand, and that of untreatedunits taken as a whole. Instead, in tests [2] , [3] and [4], the density of treated units is contrastedwith that of meaningful subsets of controls, which allows to conduct density discontinuity testson a more local scale that is logically connected to the three identification conditions in Section Note that, the net cumulative hazard ˜ H ( T ) A (1) i = A (2) i =1 can be seen as the sum of H ( T ) A (1) i = A (2) i =0 (quantity(C)) and the discontinuity. Moreover, following 2, the latter discontinuity can be written as H ( T ) A (1) i = A (2) i =1 − H ( T ) A (1) i =0 ,A (2) i =1 − H ( T ) A (0) i =0 ,A (2) i =1 + H ( T ) A (1) i = A (2) i =0 , yielding the formula in 7. The anlysis is conducted using the Stata package rddensity (Cattaneo et al., 2018).
Graphical analysis of the discontinuities in the cumulative hazard of completion at the end ofthe observation period
Note: lines (A)-(D) are obtained by plotting the smoothed values from kernel-weighted local polynomial re-gressions. A triangular kernel polynomial function with degree one is used. Line (E) is then obtained as( A ) − ( B ) − ( D ) + 2 · ( C ), following equation 7. A (1) i = 1 if project i is equal or above the cutoff value in terms of financial size and zero otherwise,and A (2) i = 1 if project i is equal or above the cutoff value in terms of regional co-financing andzero otherwise.Table 6: Tests for the null hypothesis of no density discontinuity atcutoff c ( Z ) under different sets of control units. Z i < c ( Z ) Z > c ( Z ) Test statistic p -value[1] All units A (1) = 1 , A (2) = 1 1.104 0.269[2] A (1) = 1 , A (2) = 0 A (1) = 1 , A (2) = 1 1.506 0.132[3] A (1) = 0 , A (2) = 0 A (1) = 1 , A (2) = 1 0.592 0.554[4] A (1) = 0 , A (2) = 1 A (1) = 1 , A (2) = 1 -1.165 0.244 Note: Robust bias-corrected statistic from local polynomial density estimation de-veloped in Calonico et al. (2019). A triangular kernel polynomial function of secondorder is used to construct the density point estimator.
References
Acemoglu, D., C. Garc´ıa-Jimeno, and J. A. Robinson (2015). State capacity and economicdevelopment: A network approach.
American Economic Review 105 (8), 2364–2409.Angrist, J. D. and M. Rokkanen (2015). Wanna get away? regression discontinuity estimationof exam school effects away from the cutoff.
Journal of the American Statistical Associa-tion 110 (512), 1331–1344.Arve, M. and D. Martimort (2016). Dynamic procurement under uncertainty: Optimal designand implications for incomplete contracts.
American Economic Review 106 (11), 3238–3274.Bajari, P., R. McMillan, and S. Tadelis (2009). Auctions versus negotiations in procurement:An empirical analysis.
Journal of Law, Economics, and Organization 25 (2), 372–399.Baldi, S. and D. Vannoni (2017). The impact of centralization on pharmaceutical procurementprices: The role of institutional quality and corruption.
Regional Studies 51 (3), 426–438.Bandiera, O., A. Prat, and T. Valletti (2009). Active and passive waste in government spending:Evidence from a policy experiment.
American Economic Review 99 (4), 1278–1308.Bogart, D. (2020). Infrastructure and institutions: Lessons from history.
Regional Science andUrban Economics (103626).Brown, T. L. and M. Potoski (2003). Contract-management capacity in municipal and countygovernments.
Public Administration Review 63 (2), 153–164.Brueckner, J. K. and P. M. Picard (2015). Where and when to invest in infrastructure.
RegionalScience and Urban Economics 53 , 123–134.Bucciol, A., O. Chillemi, and G. Palazzi (2013). Cost overrun and auction format in small sizepublic works.
European Journal of Political Economy 30 , 35–42.21aliendo, M., K. Tatsiramos, and A. Uhlendorff (2013). Benefit duration, unemployment dura-tion and job match quality: A regression discontinuity approach.
Journal of Applied Econo-metrics 28 (4), 604–627.Calonico, S., M. D. Cattaneo, and M. H. Farrell (2019). Optimal bandwidth choice for robustbias-corrected inference in regression discontinuity designs.
The Econometrics Journal 23 (2),192–210.Calonico, S., M. D. Cattaneo, and R. Titiunik (2014). Robust nonparametric confidence intervalsfor regression-discontinuity designs.
Econometrica 82 (6), 2295–2326.Castells, A. and A. Sol´e-Oll´e (2005). The regional allocation of infrastructure investment: Therole of equity, efficiency and political factors.
European Economic Review 49 (5), 1165–1205.Cattaneo, M. D. and J. C. Escanciano (Eds.) (2017).
Regression Discontinuity Designs: Theoryand Applications . Bingley: Emerald Group Publishing.Cattaneo, M. D., B. R. Frandsen, and R. Titiunik (2015). Randomization inference in theregression discontinuity design: An application to party advantages in the u.s. senate.
Journalof Causal Inference 3 (1), 1–24.Cattaneo, M. D., M. Jansson, and X. Ma (2018). Manipulation testing based on density discon-tinuity.
The Stata Journal 18 (1), 234–261.Cattaneo, M. D., L. Keele, R. Titiunik, and G. Vazquez-Bare (2016). Interpreting regressiondiscontinuity designs with multiple cutoffs.
The Journal of Politics 78 (4), 1229–1248.Cerqua, A. and G. Pellegrini (2014). Do subsidies to private capital boost firms’ growth? amultiple regression discontinuity design approach.
Journal of Public Economics 109 , 114–126.Cheng, Y. A. (2016). Regression discontinuity designs with multiple assignment variables.
Ph.DThesis, University of California at Berkeley .Choi, J.-y. and M.-j. Lee (2017). Regression discontinuity: Review with extensions.
StatisticalPapers 58 (4), 1217–1246.Choi, J.-y. and M.-j. Lee (2018). Regression discontinuity with multiple running variables al-lowing partial effects.
Political Analysis 26 (03), 258–274.22hoi, J.-y. and M.-j. Lee (2020). Detecting manipulation in regression discontinuity is neitheralways necessary nor informative.
SSRN Electronic Journal 3389774 .Coviello, D., A. Guglielmo, and G. Spagnolo (2018). The effect of discretion on procurementperformance.
Management Science 64 (2), 715–738.Coviello, D., L. Moretti, G. Spagnolo, and P. Valbonesi (2018). Court efficiency and procurementperformance.
The Scandinavian Journal of Economics 120 (3), 826–858.Crescenzi, R., M. Di Cataldo, and A. Rodr´ıguez-Pose (2016). Government quality and the eco-nomic returns of transport infrastructure investment in european regions.
Journal of RegionalScience 56 (4), 555–582.Crescenzi, R. and A. Rodr´ıguez-Pose (2012). Infrastructure and regional growth in the europeanunion.
Papers in Regional Science 91 (3), 487–513.Dabla-Norris, E., J. Brumby, A. Kyobe, Z. Mills, and C. Papageorgiou (2012). Investing in publicinvestment: An index of public investment efficiency.
Journal of Economic Growth 17 (3),235–266.D’Alpaos, C., M. Moretto, P. Valbonesi, and S. Vergalli (2013). Time overruns as opportunisticbehavior in public procurement.
Journal of Economics 110 (1), 25–43.De Chiara, A. (2020). Precontractual investment and modes of procurement.
European EconomicReview 124 (103404).Decarolis, F. (2018). Comparing public procurement auctions.
International Economic Re-view 59 (2), 391–419.Decarolis, F. and C. Giorgiantonio (2015). Local public procurement regulations: The case ofitaly.
International Review of Law and Economics 43 , 209–226.Decarolis, F., L. M. Giuffrida, E. Iossa, V. Mollisi, and G. Spagnolo (2019). Bureaucraticcompetence and procurement outcomes.
National Bureau of Economic Research WorkingPaper No. 24201 .Decarolis, F. and G. Palumbo (2015). Renegotiation of public contracts: An empirical analysis.
Economics Letters 132 , 77–81. 23imitri, N., G. Piga, and G. Spagnolo (2006).
Handbook of Procurement . Cambridge: CambridgeUniversity Press.Elburz, Z., P. Nijkamp, and E. Pels (2017). Public infrastructure and regional growth: Lessonsfrom meta-analysis.
Journal of Transport Geography 58 , 1–8.Estache, A. and R. Foucart (2018). The scope and limits of accounting and judicial courtsintervention in inefficient public procurement.
Journal of Public Economics 157 , 95–106.Farole, T., A. Rodriguez-Pose, and M. Storper (2011). Cohesion policy in the european union:Growth, geography, institutions.
Journal of Common Market Studies 49 (5), 1089–1111.Finan, F., B. Olken, and R. Pande (2017). The personnel economics of the developing state. In
Handbook of Economic Field Experiments , pp. 467–514. Elsevier.Gori, G. F., P. Lattarulo, and M. Mariani (2017). Understanding the procurement performanceof local governments: A duration analysis of public works.
Environment and Planning C:Politics and Space 35 (5), 809–827.Grigoli, F. and Z. Mills (2013). Institutions and public investment: an empirical analysis.
Economics of Governance 15 (2), 131–153.Guccio, C., G. Pignataro, and I. Rizzo (2014). Do local governments do it better? Analysis oftime performance in the execution of public works.
European Journal of Political Economy 34 ,237–252.Imbens, G. and K. Kalyanaraman (2012). Optimal bandwidth choice for the regression discon-tinuity estimator.
The Review of Economic Studies 79 (3), 933–959.Imbens, G. and T. Zajonc (2011). Regression discontinuity design with multiple forcing variables.
Harvard University, Report 972 .Imbens, G. W. and T. Lemieux (2008). Regression discontinuity designs: A guide to practice.
Journal of Econometrics 142 (2), 615–635.Imbens, G. W. and J. M. Wooldridge (2009). Recent developments in the econometrics ofprogram evaluation.
Journal of Economic Literature 47 (1), 5–86.IMF (2014).
World Economic Outlook . International Monetary Fund Washington, DC.24acob, B. A. and L. Lefgren (2004). Remedial education and student achievement: A regression-discontinuity analysis.
Review of Economics and Statistics 86 (1), 226–244.Kalbfleisch, J. D. and R. L. Prentice (2011).
The Statistical Analysis of Failure Time Data .Hoboken: John Wiley & Sons, Inc.Krahmer, D. and R. Strausz (2011). Optimal procurement contracts with pre-project planning.
The Review of Economic Studies 78 (3), 1015–1041.Lattarulo, P. (2020). An overview of open reform issues in local government.
Scienze Regionali- Italian Journal of Regional Science 19 (2).Lee, D. S. and T. Lemieux (2010). Regression discontinuity designs in economics.
Journal ofEconomic Literature 48 (2), 281–355.Lewis, G. and P. Bajari (2011). Procurement contracting with time incentives: Theory andevidence.
The Quarterly Journal of Economics 126 (3), 1173–1211.Lewis, G. and P. Bajari (2014). Moral hazard, incentive contracts, and risk: Evidence fromprocurement.
The Review of Economic Studies 81 (3), 1201–1228.Matsudaira, J. D. (2008). Mandatory summer school and student achievement.
Journal ofEconometrics 142 (2), 829–850.Mattei, A. and F. Mealli (2016). Regression discontinuity designs as local randomized experi-ments.
Observational Studies 2 , 156–173.McCrary, J. (2008). Manipulation of the running variable in the regression discontinuity design:A density test.
Journal of Econometrics 142 (2), 698–714.Papay, J. P., J. B. Willett, and R. J. Murnane (2011). Extending the regression-discontinuityapproach to multiple assignment variables.
Journal of Econometrics 161 (2), 203–207.Reardon, S. F. and J. P. Robinson (2012). Regression discontinuity designs with multiple rating-score variables.
Journal of Research on Educational Effectiveness 5 (1), 83–104.Saussier, S. and J. Tirole (2015). Strengthening the efficiency of public procurement.
Notes duConseil d’Analyse Economique 22 (3), 1.V¨alil¨a, T. (2020). Infrastructure and growth: A survey of macro-econometric research.
StructuralChange and Economic Dynamics 53 , 39–49.25arren, P. L. (2014). Contracting officer workload, incomplete contracting, and contractualterms.
The RAND Journal of Economics 45 (2), 395–421.Wong, V. C., P. M. Steiner, and T. D. Cook (2013). Analyzing regression-discontinuity designswith multiple assignment variables.