A New Approach to Electricity Market Clearing With Uniform Purchase Price and Curtailable Block Orders
Iacopo Savelli, Bertrand Cornélusse, Antonio Giannitrapani, Simone Paoletti, Antonio Vicino
aa r X i v : . [ q -f i n . P R ] J un A New Approach to Electricity Market Clearing With Uniform Purchase Price andCurtailable Block Orders
Iacopo Savelli a, ∗ , Bertrand Corn´elusse b , Antonio Giannitrapani a , Simone Paoletti a , Antonio Vicino a a Dipartimento di Ingegneria dell’Informazione e Scienze Matematiche, Universit`a di Siena, Siena 53100, Italy b Department of Electrical Engineering and Computer Science, University of Li`ege, Belgium
Abstract
The European market clearing problem is characterized by a set of heterogeneous orders and rules that force the imple-mentation of heuristic and iterative solving methods. In particular, curtailable block orders and the uniform purchase pricepose serious difficulties. A block order spans over multiple hours, and can be either fully accepted or fully rejected. Theuniform purchase price prescribes that all consumers pay a common price in all the zones, while producers receive zonalprices, which can differ from one zone to another.The market clearing problem in the presence of both the uniform purchase price and block orders is a major open issuein the European context. The uniform purchase price scheme leads to a non-linear optimization problem involving bothprimal and dual variables, whereas block orders introduce multi-temporal constraints and binary variables into the problem.As a consequence, the market clearing problem in the presence of both block orders and the uniform purchase price can beregarded as a non-linear integer programming problem involving both primal and dual variables with complementary andmulti-temporal constraints.The aim of this paper is to present a non-iterative and heuristic-free approach for solving the market clearing problemin the presence of both curtailable block orders and the uniform purchase price scheme. The solution is exact, with noapproximation up to the level of resolution of current market data. By resorting to an equivalent uniform purchase priceformulation, the proposed approach results in a mixed-integer linear program, which is built starting from a non-linearinteger bilevel programming problem. Numerical results using real market data are reported to show the effectiveness of theproposed approach. The model has been implemented in Python, and the code is freely available on a public repository.
Keywords: bilevel programming, curtailable block orders, European market clearing, mixed-integer linear programming,power system economics, uniform purchase price. ∗ Corresponding author
Email addresses: [email protected] (Iacopo Savelli), [email protected] (Bertrand Corn´elusse), [email protected] (Antonio Giannitrapani), [email protected] (Simone Paoletti), [email protected] (Antonio Vicino)
Preprint submitted to Elsevier June 7, 2018 omenclature
A. Sets and Indices i Index of market zones, i ∈ Z . K πti Set of consumers paying the UPP π t in zone i ∈ Z ,with t ∈ T . K πt Set of all consumers paying the UPP, i.e., K πt = ∪ i K πti , with t ∈ T . K ζti Set of consumers paying the zonal price ζ ti in zone i ∈ Z , with t ∈ T . K ζt Set of all consumers paying zonal prices, i.e., K ζt = ∪ i K ζti , with t ∈ T . K ti Set of all consumers in zone i ∈ Z , i.e., K ti = K πti ∪ K ζti , with t ∈ T . K t Set of all consumers, i.e., K t = ∪ i K ti , with t ∈ T . P ti Set of producers submitting simple stepwise orderin zone i ∈ Z , with t ∈ T . P t Set of all producers submitting simple stepwise or-der, i.e., P t = ∪ i P ti , t ∈ T . P Bi Set of producers submitting curtailable profile blockorders in zone i ∈ Z . P B Set of all producers submitting curtailable profileblock orders, i.e., P B = ∪ i P Bi . T Set of the 24 daily hours. T p Set of block order p timespan, with p ∈ P B , and T p ⊆ T . Z π Set of zones enforcing the UPP π t . Z ζ Set of zones without the UPP, all consumers payzonal prices ζ ti . Z Set of all zones, Z = Z π ∪ Z ζ . B. Constants D maxtk Maximum hourly quantity demanded by consumer k ∈ K t at time t ∈ T . F maxtij Maximum flow capacity from zone i to zone j with t ∈ T . M ( . ) M π , M Bp , M Ftij , and M Dth are big-M parameters. O mtk Merit order for consumer k ∈ K πt , lower values meanhigher priority. P dtk Hourly order price submitted by consumer k ∈ K t with t ∈ T . P stp Hourly order price submitted by producer p ∈ P t with t ∈ T . P Bp Block order price submitted by producer p ∈ P B . R minp Minimum acceptance ratio for curtailable block or-der with p ∈ P B . S maxtp Maximum hourly quantity offered by producer p ∈P t at time t ∈ T . S B,maxtp
Profile block order maximum hourly quantity of-fered by producer p ∈ P B with t ∈ T p .
1. Introduction
Electricity markets are experiencing significant changesdue to different factors, as the modification of generation mix[1], the increasing presence of demand response [2] and energystorage systems [3], the growth of renewable energy [4], therequest for both flexibility [5, 6] and security of supply [7],
C. Variables b tji Binary variable used in the binary expansion to con-vert a positive integer number in binary form, where i ∈ Z π , t ∈ T , and j ∈ { , . . . } . d ζtk Executed demand quantity for consumer k ∈ K ζt ,with t ∈ T . d wtk Executed demand quantity for consumer k ∈ K πt if u wtk = 1, with t ∈ T . d dtk Executed demand quantity for consumer k ∈ K πt if u dtk = 1, with t ∈ T . d πtk Executed demand quantity for consumer k ∈ K πt ,where d πtk = u gtk D maxtk + d wtk + d dtk , with k ∈ K πt and t ∈ T . f tij Flow from zone i to zone j with t ∈ T . r p Block order acceptance ratio with p ∈ P B . s tp Executed supply quantity for producer p ∈ P t , with t ∈ T . u ftij Binary variable with i, j ∈ Z π and t ∈ T , where u ftij = 1 if and only if the transmission line from i to j is congested, i.e., f tij = F maxtij . u Bp Binary variable representing the block order accep-tance status with p ∈ P B , where u Bp = 1 meansaccepted, and u Bp = 0 rejected. u gtk Binary variable with k ∈ K πt and t ∈ T , where u gtk = 1 ⇐⇒ P dtk > π , and zero otherwise. u etk Binary variable with k ∈ K πt and t ∈ T , where if u etk = 1 then P dtk = π . u wtk Binary variable with k ∈ K πt and t ∈ T , where if u wtk = 1 then the demand order is at-the-money andit is partially cleared according to a social welfareapproach. u dtk Binary variable with k ∈ K πt and t ∈ T , where if u dtk = 1 then the demand order is at-the-money andit is partially cleared according to an economic dis-patch approach. δ maxtij Dual variable of constraint f tij ≤ F maxtij . ζ ti Zonal price in zone i ∈ Z with t ∈ T . η tij Dual variable of constraint f tij + f tji = 0. κ t Error tolerance in the uniform purchase price defi-nition, currently κ t ∈ [ −
1; 5]. ϕ ζtk Dual variable of constraint d ζtk ≤ D maxtk . ϕ wtk Dual variable of constraint d wtk ≤ D maxtk u wtk . ϕ w,lotk Dual variable of constraint d wtk ≥ ϕ stp Dual variable of constraint s tp ≤ S maxtp . ϕ B,maxp
Dual variable of constraint r p ≤ u Bp . ϕ B,minp
Dual variable of constraint r p ≥ u Bp R minp . π t Uniform purchase price at time t ∈ T . D. Auxiliary Variables y gπtk Auxiliary variable, it replaces the product u gtk π . y gζtki Auxiliary variable, it replaces the product u gtk ζ ti . y eπtk Auxiliary variable, it replaces the product u etk π t . y wϕtk Auxiliary variable, it replaces the product u wtk ϕ wtk . y Bϕ,maxp
Auxiliary variable, it replaces the product u Bp ϕ B,maxp . y Bϕ,minp
Auxiliary variable, it replaces the product u Bp ϕ B,minp . y bζtij Auxiliary variable, it replaces the product b tji ζ ti .c (cid:13) Prezzo Unico Nazionale (PUN).
The UPP scheme requires that all consumers pay a uniqueprice, termed the UPP, in all the zones, while producers re-ceive zonal prices, which can differ from one zone to another.The UPP π t at time t is defined as the average of the zonalprices ζ ti , weighted by the consumers’ cleared quantities d πtk .Formally: π t = P i ∈Z π P k ∈K πti ζ ti d πtk P k ∈K πt d πtk , (1)where Z π is the set of zones enforcing the UPP, K πti is the setof consumers paying the UPP in the zone i at time t ∈ T ,and K πt = ∪ i K πti .Given the UPP definition (1), it is possible to specify thefollowing UPP clearing rule : • demand orders with a submitted price P dtk strictlygreater than π t , that is, in-the-money (ITM) demandorders, must be fully executed, i.e., d πtk = D maxtk ; • demand orders with a submitted price P dtk exactly equalto π t , that is, at-the-money (ATM) demand orders, maybe partially cleared, i.e., 0 ≤ d πtk ≤ D maxtk ; • demand orders with a submitted price P dtk strictly lowerthan π t , that is, out-of-the-money (OTM) demand or-ders, must be fully rejected, i.e., d πtk = 0.In addition, demand orders subject to the UPP scheme areranked by a parameter termed merit order , that determinesa strict total ordering between the UPP orders. The meritorder is assigned by the market operator before the day-aheadauction. Lower merit order O mtk implies a higher priority inexecution. In particular, this ranking coincides with the priceranking for the orders with different submitted prices, i.e., if P dth > P dtk then O mth < O mtk , with k, h ∈ K πt . For the orderswith the same price, the merit order is assigned according Q uan t i t y ( M W h ) Hour
Figure 1: The figure shows a curtailable profile block order (gray +blue area), which is partially executed (blue area), with acceptance ratio r p = 0 . to a set of non-discriminatory rules, as for example the timestamp of submission.Traditionally, pumping units belonging to hydroelectricproduction plants are excluded from the UPP rule. Theseunits buy electricity to refill their reservoirs usually duringthe night, whereas they generate energy during the remaininghours. To harmonize the buying and selling price, demandorders from these units pay zonal prices, and not the UPP. Asa consequence, the set K ζt of consumers paying zonal pricesis usually non-empty in the set of UPP zones Z π , because itis populated by the pumping units. By contrast, the set K πt of consumers paying the UPP is always empty in the zones Z ζ non-enforcing the UPP.Currently, the implementation of the UPP pricing schemeon the Italian market allows an error tolerance κ t ∈ [ −
1; 5].Therefore, the Italian PUN is actually defined as: π t X k ∈K πt d πtk = X i ∈Z π X k ∈K πti ζ ti d πtk + κ t . (2) A block order p submitted by a producer is an order thatspans over multiple hours, and can be either fully acceptedor fully rejected [18]. Moreover, the block order submittedprice P Bp must be the same over the whole timespan. Themost general form of a single block order is the profile blockorder, which allows to submit different quantities for eachhour. Furthermore, an additional feature called minimumacceptance ratio (MAR) has been introduced in the Nordiccountries, which allows to partially execute a block order [17].In this case, the hourly quantities involved can be partiallycleared, and the profile block order is uniformly scaled overthe whole timespan, as depicted in Figure 1. The fraction ofthe quantity executed for each hour is termed the acceptanceratio r p . The acceptance ratio is independent of t , and satis-fies the constraint, R minp ≤ r p ≤
1, where R minp is the MARfor the block order p . For this reason, block orders with MARare called curtailable .Block orders can be classified according to their degree ofmoneyness , that is, a block order submitted by a producer istermed: • in-the-money (ITM), if the submitted price P Bp is smallerthan the average of the zonal prices ζ ti weighted bythe hourly offered quantities S B,maxtp , or equivalently,if the block order has a strictly positive surplus, i.e., P t ∈T p S B,maxtp ( ζ ti − P Bp ) > • at-the-money (ATM), if the submitted price P Bp is equalto the average of the zonal prices ζ ti weighted by thehourly offered quantities S B,maxtp , or equivalently, if thec (cid:13) P t ∈T p S B,maxtp ( ζ ti − P Bp ) = 0; • out-of-the-money (OTM), if the submitted price P Bp isgreater than the average of the zonal prices ζ ti weightedby the hourly offered quantities S B,maxtp , or equivalently,if the block order has a strictly negative surplus, i.e., P t ∈T p S B,maxtp ( ζ ti − P Bp ) < T p ⊆ T is the timespan of block order p . We recallthat the acceptance ratio r p does not depend on time. Forthis reason, in the definitions of the block order surplus only S B,maxtp is considered instead of r p S B,maxtp . ITM block ordersshould be fully executed. ATM block orders can be partiallycleared. OTM block orders must be always rejected. No-tice that, the indivisible nature of regular block orders mayprevent the existence of an optimal market equilibrium, i.e.,the clearing problem may result infeasible [19, 20]. For thisreason, current market rules allow to reject ITM block or-ders [13]. Rejected ITM block orders are termed paradoxi-cally rejected block orders (PRBs). By contrast, OTM blockorders that are accepted are termed paradoxically acceptedblock orders (PABs). PABs are not allowed in the Europeanmarkets. The reasoning behind this different treatment isstraightforward. Under perfect competition, prices submit-ted by producers are the marginal costs [21, 22]. Therefore,a PAB would cause a monetary loss to the producer, whereasa PRB leads to a missed trading opportunity. Notice that,in some US markets [23] and in Turkey [24], it is possibleto compensate producers with side payments [25], but this isnot allowed in the European markets.
In the literature, algorithms to solve the market clearingproblem in the presence of block orders are based on dif-ferent techniques. Reference [26] formulates a mixed-integerlinear program (MILP) to clear the market with an addi-tional iterative process to handle PRBs. Reference [27] pro-poses a primal-dual formulation of the market clearing prob-lem, where an improved Benders-like decomposition methodis further introduced to strengthen the classical Benders cuts,which is extended in [28]. In [19] a clearing method to min-imize the impact of PRBs on the final solution is proposed.Reference [29] introduces a bilevel approach to handle regu-lar block orders in a single-zone market where block ordersurpluses are explicitly considered. The official Europeanalgorithm for market coupling, termed EUPHEMIA [13], isbased on a mixed-integer quadratic programming formula-tion with additional sequential subproblems and modules.It is partially derived from the COSMOS [30] model, orig-inally employed in the central-western European electricitymarkets. Both algorithms implement a branch-and-boundmethod for solving a European social welfare maximizationproblem, where appropriate cuts are introduced until an opti-mal solution fulfilling all the market requirements is achieved.Reference [31] reports an interesting scenario analysis, whichinvestigates the effects of different size, number and type ofblock orders on the computation time required to solve themarket clearing problem. Reference [32] formulates a mixed-integer quadratic program to mimic the complete Europeanday-ahead market, with iterative processes to handle PRBs,complex orders, and the UPP.With respect to the UPP scheme, reference [33] proposes acomplementarity approach to solve a clearing problem un- der mixed pricing rules, which is further extended to re-serves in [34], and block orders in [35]. In the latter case,an iterative process is implemented which involves an ini-tial MILP problem to handle block orders, followed by amixed-complementarity problem to deal with the UPP. Refer-ence [36] uses a complementarity approach to clear a marketwith block and complex orders, which is extended to UPPin [37], where in both cases a heuristic process is used tohandle paradoxical orders. Reference [38] proposes a bilevelmodel to clear a UPP market with simple stepwise orders,where the objective function maximize the surplus of the con-sumers. Reference [39] proposes an income based approach toovercome the non-linearities of the UPP, however this modelcannot be extended to curtailable block orders. Originally,the method to clear the Italian market was based on [40],whereas the current approach implemented by EUPHEMIAis described in [13]. In the first case, the UPP is sequentiallyselected among each possible price in the aggregate marketdemand curve until the whole curve is explored. Then, theoptimal solution is chosen among the feasible candidates thatclear the market and yield the greatest social welfare. Sim-ilarly, EUPHEMIA explores the aggregate market demandcurve until a feasible solution is found that clears the marketwhile satisfying the UPP definition (2) within the error tol-erance κ t . Notice that the UPP scheme differs substantiallyfrom both the consumers payment minimization scheme [41](where the objective is to minimize the total consumers’ pay-ments), and from the clearing rule used in some US markets,such as [42], where the common price paid by consumers iscomputed by using an ex-post iteration. Finally, reference[43] reports an interesting analysis of the European marketcoupling impact on the UPP, where the positive effect of theincreased market liquidity has been assessed. Block orders and the UPP rule pose a considerable bur-den on the European market clearing problem. In partic-ular, block orders introduce at least two kinds of relevantissues. Firstly, the indivisible nature of these orders forcesthe introduction of binary variables. Secondly, block ordersspan over multiple trading hours, and impose multi-temporalconstraints. On the other hand, under the UPP scheme itis not possible to use directly the traditional social welfaremaximization method to clear the market, due to the pos-sible difference in the price paid by consumers (the UPP)and the price received by producers (the zonal price) withinthe same zone. Moreover, the UPP scheme requires to clearconsumers and producers simultaneously, because both haveprice-elastic curves, i.e., the demanded and offered quantitiesdepend on the actual market prices. Furthermore, under themarginal pricing framework [44, 45], market prices are definedas the dual variables of the power balance constraints. Thismeans that the problem formulation must involve the dualvariables. Finally, the UPP definition (1) implies the pres-ence of bilinear terms involving both quantities and prices,i.e., primal and dual variables, which make the problem non-linear and non-convex. As a consequence, the European mar-ket clearing problem with both block orders and the UPP canbe regarded as a non-linear integer program, involving bothprimal and dual variables with complementary and multi-temporal constraints.c (cid:13) .5. Paper contribution
The problem of finding a computationally tractable and ex-act social welfare maximization formulation, for solving themarket clearing problem in the presence of both block ordersand the UPP scheme, is an important open issue in the Euro-pean context. The purpose of this paper is to present a non-iterative solution to this problem, which results in a MILPmodel, that can be solved with off-the-shelf solvers. Thismodel is obtained starting from a non-linear integer bilevelproblem, which is transformed into an equivalent single levelmodel by using primal-dual relations and properties. Then,all the non-linearities are removed by using both standardinteger algebra and an equivalent reformulation of the UPPdefinition. We remark that, this approach is homogeneousin spite of the different traded instruments and market rules.That is, the proposed framework deals with both block or-ders and the UPP by using the same comprehensive modelunder the exact European social welfare maximization objec-tive, with no iterative process or subproblems. Furthermore,by construction, market prices are guaranteed to fulfill themarginal pricing scheme [44] as required by the European reg-ulatory framework [46] and coherently with standard marketpractices [13, 14]. The solution is exact, with no approxima-tion at least up to the level of resolution of current marketdata. Finally, the MILP formulation allows to prove the op-timality of the solution. To summarize, the main noveltiespresented in this work are:1. the exact formulation of the market clearing problem inthe presence of both curtailable profile block orders andthe UPP as a non-linear integer bilevel problem, whichis then transformed into an equivalent MILP model;2. the use of complementary relations and integer methodsto linearize the UPP definition;3. the non-iterative approach maximizing the exact socialwelfare in the presence of both the UPP and curtailableblock orders.The aim of this work is to show that the UPP scheme (i.e.,a non-linear program), and block orders (which involve bi-nary variables), that are currently handled heuristically, canbe recast as a single, all-encompassing MILP problem, fulfill-ing all the European regulatory requirements. This flexibleapproach allows one to gain knowledge on the overall clearingproblem. By providing insights into the problem structure,the model can be used by transmission system operators,policy makers and stakeholders to evaluate the physical andeconomic impacts of both grid expansion plans and modifi-cations to market policies and rules, by carrying out what-if analysis on specific elements reflected in the problem objec-tive and constraints. In this respect, we freely provide theopen-source Python code of the proposed model, in order tobridge the gap between modeling and implementation, andto offer a ready-to-use tool to the interested user. Finally,we recall that the MILP formulation allows one to certify theoptimality of the obtained solution.The remaining part of this paper is organized as follows.Section 2 highlights some of the clearing differences betweenthe UPP scheme and a traditional market. Section 3 presentsa formulation of the non-linear integer bilevel model, andshows how the final MILP is built. Section 4 illustrates someoptional modeling features to detect market splits. Section5 describes the tests performed, and reports the numericalresults. Finally, Section 6 outlines some conclusions. Thecomplete MILP model is reported in Appendix A.
Demand Supply
A B
Figure 2: Market clearing without the UPP rule. The intersection ofthe demand and supply curves determines both the zonal price and thecleared quantities. The demand order labeled by A is an at-the-moneyorder, and it is partially cleared. By contrast, the demand order labeledby B is an out-of-the-money order, and must be rejected.
Demand Supply B π t Quantity (MWh) A Figure 3: Market clearing with the UPP rule. The UPP is assumedto be 5 Euro/MWh. The demand order labeled by A has a price of15 Euro/MWh, whereas demand order labeled by B has a price of 5Euro/Mwh. According to the UPP rule, the order A is in-the-moneyand must be fully executed. By contrast, the order B is at-the-moneyand can be partially cleared.
With respect to [38], curtailable profile block orders arenow considered (both in UPP and non-UPP zones), a noveland equivalent UPP formulation is proposed, and the objec-tive function represents the exact social welfare.
2. Market clearing differences between the UPPscheme and a traditional European market
This section provides a few examples to highlight someof the clearing differences between the UPP scheme and atraditional European market. For ease of reading, this sectionconsiders only stepwise orders.In a traditional European market, i.e., a market cleared ac-cording to a social welfare approach with no UPP involved,the intersection of the supply and the demand curves deter-mines both the quantity executed and the zonal price, asdepicted in Figure 2. In this case, the demand and supplyorders are cleared at the same price, i.e., the zonal price.In particular, a demand order is in-the-money if its price isstrictly greater than the zonal price, whereas it is at-the-money if its price is exactly equal to the zonal price, andit is out-of-the-money if its price is strictly lower than thezonal price. In Figure 2, the demand order labeled by A isintersected by the supply curve, the intersection determinesthe quantity partially executed, and the price of the order Asets the zonal price in the zone. [21, 22]. The order A is anat-the-money order. By contrast, the demand order labeledby B is out-of-the-money, because its price is strictly lowerthan the zonal price, and it must be fully rejected. This isnot necessarily true for a UPP demand order, because it iscleared at the UPP and not at the zonal price.c (cid:13) A Demand Supply ζ ti π t d dtk Quantity (MWh)
Figure 4: Solution of the market clearing problem depicted in Figure 3.The UPP is assumed to be 5 Euro/MWh. The total demanded quantityincludes the quantity d dtk partially cleared. The zonal price ζ ti , collectedby the producers, is determined as the price required to dispatch thedemanded quantity. Figure 3 shows the same demand and supply curves as inFigure 2. However, in this example the UPP rule is enforced,and the UPP is assumed equal to 5 Euro/MWh. Here, allthe demand orders are cleared at the UPP and not at thezonal price. Therefore, the demand order labeled by A isin-the-money and must be fully executed, whereas the de-mand order labeled by B is at-the-money and can be partiallycleared. Notice that, the order labeled by B can be partiallyexecuted regardless of the zonal price in the zone. This is afundamental difference with respect to a traditional market.In the case depicted in Figure 3, the in-the-money demandorders and the quantity partially cleared of the order B mustbe executed. Therefore, the problem boils down to finding theexact executed quantity and the zonal price, given the UPP.We recall that producers collect zonal prices. Therefore, theproblem to solve is to find the price required by producers tomatch the demanded quantity. This problem is an economicdispatch of a potentially variable demand (which in turn de-pends on the UPP). In a dispatch problem, the demand isa constant, and the demand curve is considered as if it wereinelastic, i.e., a vertical line, and the intersection with thesupply curve determines the price required by producers, asdepicted in Figure 4. In this case, the demand includes thequantity partially cleared d dtk . The zonal price ζ ti is the pricesrequired by the producers to match the demanded quantity.An important consequence of the UPP pricing scheme is thatit is possible to have, in the same zone, an at-the-money UPPdemand order and a supply order with two different prices,both partially cleared, as in Figure 4. This is not possible ina traditional market as the one depicted in Figure 2, and itis an additional issue of the UPP pricing scheme.In particular cases, the market equilibrium given by theintersection of the demand and supply curves satisfies alsothe UPP rule, as shown in Figure 5. In this figure, the UPPis assumed to be 15 Euro/MWh. The demand order labeledby A has a price of 15 Euro/MWh, it is therefore at-the-money, and can be partially executed. This order is inter-sected by the supply curve. The quantity partially clearedand the zonal price determined by the intersection, as in atraditional market, fulfills also the UPP rule, because all thein-the-money UPP orders are fully executed, whereas the or-der partially cleared A is at-the-money. As a consequence, inthis particular case, the zonal price, the UPP and the price ofthe demand order A coincide, i.e., P dtk = π t = ζ ti . Both thecases depicted in Figure 4, and Figure 5 will be considered.The first case allows a UPP demand order to be partiallyexecuted regardless of the zonal price, by relying on a dis-patch approach. The second one is a special case, where we Demand Supply B π t A Quantity (MWh)
Figure 5: Market clearing with the UPP rule. The UPP is assumed tobe 15 Euro/MWh. This figure depicts a special case where the marketequilibrium given by the intersection of the supply and demand curves,as in a traditional market, respects also the UPP rule. The demandorder labeled by A is partially executed, whereas the order B must berejected. will exploit the elasticity of the demand curve and the tra-ditional social welfare approach to deal efficiently with theseat-the-money UPP orders, as shown in Section 3.4.
3. The Model
This section presents a formalization of the market clearingproblem in the presence of both the UPP and curtailable pro-file block orders as a non-linear integer bilevel model. Then,it shows how the bilevel model can be transformed into anequivalent MILP problem.
A bilevel model can be regarded as two nested optimiza-tion problems, termed upper and lower level problems [47].Formally, a bilevel model is defined as:max u ∈U F ( u, x ∗ ) (3)s.t. x ∗ = arg max x ∈X f ( x ; u ) , (4)where F and f are the upper and lower level objective func-tions, respectively. The main feature of a bilevel programis that the upper level decision variables, labeled u in (3)-(4), enter the lower level as fixed parameters. The variables x ∗ represent the optimal solution of the lower level problem,which depends on the upper level variables u , i.e., x ∗ = x ∗ ( u ).However, for ease of reading this dependence is usually notformally expressed. Historically, the bilevel approach wasused in the field of game theory to describe non-cooperativeStackelberg games [47]. In a Stackelberg game the upperlevel problem represents a leader that acts before a follower,that is represented by the lower level problem. However, inpower system economics, the bilevel method is typically usedto access the dual variables, i.e., the market prices, and notto actually build a game. Therefore, the upper and the lowerlevel objective functions, i.e., F and f , are usually equiva-lent. The interested reader is referred to [47, 48, 49] for addi-tional information on bilevel programs and their applicationsin power system economics. In the proposed approach, the upper level problem han-dles the UPP and verifies the degree of moneyness of blockorders, whereas the lower level actually clears the market byusing a social welfare maximization approach while properlydispatching the UPP orders, as outlined in Fig.6.c (cid:13) egree ofmoneyness clearedquantities,market pricesLower Level • ObjectiveMax social welfare • PurposeMarket clearing anddispatch of UPP ordersUpper Level • ObjectiveMax social welfare • PurposeCheck of degree of moneynessEnforcing merit ordersUPP computation
Figure 6: An overview of the proposed bilevel model. The upper levelhandles the UPP and determines the orders’ degree of moneyness. Thelower level clears the market while dispatching UPP orders dependingon the UPP rule.
This section describes the upper level problem, which isdefined as follows:max u gtk ,u etk ,u wtk ,u dtk ,u Bp ,π t ,d πtk ,d dtk ,κ t . X t ∈T X k ∈K ζt P dtk d ζtk ∗ + X t ∈T X k ∈K πt P dtk d πtk − X t ∈T X p ∈P t P stp s ∗ tp − X p ∈P B X t ∈T p P Bp r ∗ p S B,maxtp (5)subject to: π t X k ∈K πt d πtk = X i ∈Z π X k ∈K πti ζ ∗ ti d πtk + κ t ∀ t ∈ T (6) P dtk − π t ≤ M π u gtk ∀ t ∈ T , ∀ k ∈ K πt (7) P dtk − π t ≥ ǫ − M π (1 − u gtk ) ∀ t ∈ T , ∀ k ∈ K πt (8) u etk ( P dtk − π t ) = 0 ∀ t ∈ T , ∀ k ∈ K πt (9) u gth ≥ u gtk ∀ t ∈ T , ∀ h, k ∈ K πt : O mth < O mtk (10) d πtk = u gtk D maxtk + d wtk ∗ + d dtk ∀ t ∈ T , ∀ k ∈ K πt (11) X t ∈T p S B,maxtp ( ζ ∗ ti − P Bp ) ≥ − M Bp (1 − u Bp ) ∀ i ∈ Z , ∀ p ∈ P Bi (12) u wtk + u dtk ≤ u etk ∀ t ∈ T , ∀ k ∈ K πt (13) d dtk ≤ D maxtk u dtk ∀ t ∈ T , ∀ k ∈ K πt (14) u gtk ∈ { , } , u etk ∈ { , } ∀ t ∈ T , ∀ k ∈ K πt (15) u wtk ∈ { , } , u dtk ∈ { , } ∀ t ∈ T , ∀ k ∈ K πt (16) u Bp ∈ { , } ∀ p ∈ P B , (17)with d πtk ≥ d dtk ≥ κ t ∈ [ −
1; 5], and π t ∈ R . The term ǫ is a sufficiently small positive parameter, whereas M π , and M Bp are appropriate large constants, and a discussion on the selection of these parameters is given in Appendix A. Noticethat, the starred variables d ζtk ∗ , d wtk ∗ , s ∗ tp , ζ ∗ ti , and r ∗ p , are theoptimal values of the lower level variables, as sketched in (3)-(4). The upper level is a social welfare maximization prob-lem where the first two terms in the objective function (5)represent the demand orders, the third term represents pro-ducers submitting simple stepwise orders, and the last termrepresents producers submitting curtailable profile block or-ders. Constraint (6) is the UPP definition stated in (2). Con-straints (7)-(8) imply that the binary variable u gtk is equal toone if and only if the submitted price P dtk is strictly greaterthan the UPP π t . Whereas, constraint (9) implies that thebinary variable u etk can be equal to one only if the submittedprice P dtk is exactly equal to the UPP π t . Notice that, u gtk and u etk cannot be equal to one at the same time. Constraint(10) enforces the priority due to the merit orders, i.e., it de-termines the sequential execution of the in-the-money UPPdemand orders within the UPP zones, with a significant re-duction in the search space of the binary variables. In thisconstraint, h and k are indices representing all the consumerspaying the UPP, that is h, k ∈ K πt . If the order of consumer h has a smaller merit order than the order of consumer k (i.e., O mth < O mtk ), then the order of consumer h must be executedbefore the order of consumer k . Notice that merit orders areinputs, therefore constraint (10) is linear. Equation (11) de-fines the auxiliary variables d πtk , which are used to recap theexecuted quantities for UPP demand orders into single vari-ables. Constraint (12) verifies the degree of moneyness forblock orders, and implies that the binary variables u Bp can beequal to one only if the block order has a non-negative sur-plus. That is, if the block order is accepted, i.e., u Bp = 1, thenthe block order must be either ITM or ATM. By contrast,a block order can be rejected, i.e., u Bp = 0, regardless of thesurplus. Therefore, this formulation excludes any PAB, i.e.,an OTM block order which is accepted, but it allows PRBs,i.e., ITM block orders which are rejected, consistently withthe European market requirements, as described in Section1.2. The constraint (13) defines the binary variables u wtk and u dtk . These variables can differ from zero only if u etk = 1,i.e., if the UPP demand order is at-the-money, which is therequirement for having a UPP order partially executed. Thevariables u wtk handle the case of partial execution according toa traditional social welfare approach, as depicted in Figure 5,whereas the variables u dtk handle the case of partial executionaccording to an economic dispatch approach, as depicted inFigure 4. The constraint (13) prevents the double executionof the same order. Finally, the constraint (14) sets the limiton the maximum dispatchable quantity d dtk . Given the up-per level decision variables u gtk , u wtk , d dtk , and u Bp the marketclearing is actually performed by the lower level problem. This section describes the lower level problem, which isdefined as follows:c (cid:13) d ζtk ∗ , d wtk ∗ , s ∗ tp , r ∗ p , f ∗ tij , [ ζ ∗ ti ] (cid:17) =arg max d ζtk ,d wtk ,s tp ,r p ,f tij . X t ∈T X k ∈K ζt P dtk d ζtk + X t ∈T X k ∈K πt P dtk d wtk − X t ∈T X p ∈P t P stp s tp − X p ∈P B X t ∈T p P Bp r p S B,maxtp (18)subject to: d wtk ≤ u wtk D maxtk [ ϕ wtk ≥ ∀ t ∈ T , ∀ k ∈ K πt (19) − d wtk ≤ ϕ w,lotk ≥ ∀ t ∈ T , ∀ k ∈ K πt (20) d ζtk ≤ D maxtk [ ϕ ζtk ≥ ∀ t ∈ T , ∀ k ∈ K ζt (21) s tp ≤ S maxtp [ ϕ stp ≥ ∀ t ∈ T , ∀ p ∈ P t (22) f tij ≤ F maxtij [ δ maxtij ≥ ∀ t ∈ T , ∀ i, j ∈ Z (23) f tij + f tji = 0 [ η tij ∈ R ] ∀ t ∈ T , ∀ i, j ∈ Z (24) r p ≤ u Bp [ ϕ B,maxp ≥ ∀ p ∈ P B (25) − r p ≤ − u Bp R minp [ ϕ B,minp ≥ ∀ p ∈ P B (26) X k ∈K ζti d ζtk + X k ∈K πti d wtk − X p ∈P ti s tp + X j ∈Z f tij − X p ∈P Bi r p S B,maxtp = − X k ∈K πti u gtk D maxtk − X k ∈K πti d dtk [ ζ ti ∈ R ] ∀ t ∈ T , ∀ i ∈ Z , (27)with d ζtk ≥ d wtk ∈ R , s tp ≥ r p ∈ R , and f tij ∈ R . Dualvariables are enclosed in square brackets. Given the upperlevel variables u gtk , u wtk , d dtk , and u Bp , the lower level problemactually clears the market while dispatching the UPP orderaccording to their degree of moneyness. We recall that theupper level variables enter the lower level as parameters,therefore the lower level problem is a linear program. Noticethat, the lower level objective function (18) is equivalentto the upper level objective function (5). Indeed, if wesubstitute (11) into (5) and considering that the terms P dtk u gtk D maxtk and P dtk d dtk are constants into the lower levelproblem, and that any constant term can be removedfrom an objective function without altering the optimalsolution, we obtain the equivalent lower level objectivefunction (18). Therefore, given the upper level variables, thelower level clears the market according to an exact socialwelfare maximization problem. Constraints (19)-(22) imposebounds on the demanded and offered quantities. Noticethat, the constraints (20) explicitly sets the lower boundfor the demanded quantities d wtk . This formulation willbe exploited in Section 3.4. Constraints (23)-(24) imposebounds on the inter-zonal flows. Constraints (25)-(26)set the MAR conditions for block orders by enforcing therelation R minp ≤ r p ≤
1. The binary variables u Bp are usedto exclude any out-of-the-money block orders as determinedby (12). We recall that, the acceptance ratio r p must bethe same during all the hours t ∈ T p . This means thatthe day-ahead clearing problem in the presence of blockorders cannot be split in independent hourly subproblems.Finally, equation (27) defines the power balance constraintfor each zone i ∈ Z = Z π ∪ Z ζ . The right-hand-side of(27) specifies the quantities that must be dispatched. Inparticular, the terms u gtk D maxtk represent the in-the-moneyorders that must be fully executed and dispatched accordingto the UPP clearing rule (see Section 1.1). By contrast, theterms d dtk represent the at-the-money UPP orders partiallyexecuted and to dispatch (as in case depicted in Figure 4), where the quantity d dtk is determined by the upper level.Furthermore, notice the presence of d wtk in the left-hand-sideof (27). The variable d wtk is a lower level decision variable,which determines the quantity partially cleared for anat-the-money UPP order by using a social welfare approach,as in the special cases depicted in Figure 5. The constraint(13) prevents the double clearing of the same order. Noticefurther that the set K πti , i.e., the consumers paying theUPP, is empty in the zones non-enforcing the UPP, that is, K πti = Ø if i ∈ Z ζ .The starred variables d ζtk ∗ , d wtk ∗ , s ∗ tp , r ∗ p , f ∗ tij , and ζ ∗ ti rep-resent the optimal values of the lower level variables. Thezonal prices ζ ti are defined as the dual variables of the powerbalance constraints (27), as required by the marginal pricingframework [44, 45]. The zonal prices are used within the up-per level problem to compute the UPP in equation (6) andto verify the degree of moneyness for block orders in the con-straint (12). In the following section, the bilevel program isreduced to a single level optimization problem. In order to access the dual variables ζ ti , i.e., the zonalprices, the bilevel model formalized in Section 3.2 is refor-mulated as an equivalent single level optimization problem.The objective function of the single level problem is thesame of the objective function of the upper level (5), i.e.:max u gtk ,u etk ,u wtk ,u dtk ,u Bp ,π t ,d πtk ,d ζtk ,d wtk ,d dtk ,s tp ,κ t ,r p ,f tij ,ζ ti ,ϕ ζtk ,ϕ wtk ,ϕ w,lotk ,ϕ stp ,η tij ,ϕ B,maxp ,ϕ B,minp ,δ maxtij . X t ∈T X k ∈K ζt P dtk d ζtk + X t ∈T X k ∈K πt P dtk d πtk − X t ∈T X p ∈P t P stp s tp − X p ∈P B X t ∈T p P Bp r p S B,maxtp . (28)The single level problem is a unique optimization pro-gram, and there is no distinction between upper andlower parts. Hence, all the decision variables of both theproblems are present in (28). Furthermore, we recall thatthe lower level problem is a linear program, because allthe upper level variables enter the lower level as parame-ters. As a linear program, the lower level is equivalent toits necessary and sufficient Karush-Kuhn-Tucker (KKT)conditions. Moreover, in a linear program, the KKTcomplementary slackness is equivalent to the strong dualityproperty [48, 50, 51]. As a consequence, the lower levelproblem can be introduced into the single level prob-lem by adding the following constraints to the single level:c (cid:13) t ∈T X k ∈K ζt P dtk d ζtk + X t ∈T X k ∈K πt P dtk d wtk − X k ∈K t X p ∈P t P stp s tp − X p ∈P B X t ∈T p P Bp r p S B,maxtp = X t ∈T X k ∈K πt u wtk ϕ wtk D maxtk + X t ∈T X k ∈K ζt ϕ ζtk D maxtk + X t ∈T X p ∈P t ϕ stp S maxtp + X t ∈T X i ∈Z X j ∈Z δ maxtij F maxtij − X t ∈T X i ∈Z π ζ ti X k ∈K πti ( u gtk D maxtk + d dtk )+ X p ∈P B ϕ B,maxp u Bp − X p ∈P B ϕ B,minp u Bp R minp (29) ϕ wtk − ϕ w,lotk + ζ ti = P dtk ∀ t ∈ T , ∀ i ∈ Z , ∀ k ∈ K πti (30) ϕ ζtk + ζ ti ≥ P dtk ∀ t ∈ T , ∀ i ∈ Z , ∀ k ∈ K ζti (31) ϕ stp − ζ ti ≥ − P stp ∀ t ∈ T , ∀ i ∈ Z , ∀ p ∈ P ti (32) δ maxtij + η tij + η tji + ζ ti = 0 ∀ t ∈ T , ∀ i, j ∈ Z (33) ϕ B,maxp − ϕ B,minp − X t ∈T p ζ ti S B,maxtp = − X t ∈T p P Bp S B,maxtp ∀ i ∈ Z , ∀ p ∈ P Bi (34)(19)-(27) , (35)where (29) is the strong duality property, which requiresthe equivalence between the objective functions’ values inboth the primal problem and the dual problem. Conditions(30)-(34) are the constraints of the dual problem, i.e., the dual feasibility , whereas (35) refers to the original constraintsof the lower level problem, i.e., the primal feasibility .To summarize, the single level optimization problem,equivalent to the bilevel model presented in section 3.2, iscomposed by the following three parts:1. the objective function (28);2. the constraints of the upper level (6)-(17);3. the conditions representing the lower level problem (29)-(35). The single level optimization problem presented in Section3.3 is a non-linear integer program. To obtain the final equiv-alent MILP model, all the non-linearities must be removed.There are three kinds of non-linearities in the single level:1. the products of a binary variable and a continuousbounded variable, as u etk π t in (9);2. the product π t d πtk in the UPP definition (6);3. the product ζ ti P k ∈K πti d dtk in the strong duality (29), andin (6) due to (11).The non-linearities due to the product of a binary and acontinuous bounded variable can be removed by using stan-dard integer algebra. As an example, the product ux of thebinary variable u , and the continuous variable x with bounds ± M , can be replaced by an auxiliary continuous boundedvariable y defined as: − M u ≤ y ≤ + M u (36) − M (1 − u ) ≤ x − y ≤ + M (1 − u ) . (37)The Appendix A reports all the auxiliary variables actuallyused, with a discussion on the selection of the big-M’s values. To handle the UPP definition (6), we propose a novel andequivalent formulation. Firstly, by using (11), the UPP defi-nition (6) can be written as: X k ∈K πt π t u gtk D maxtk + X k ∈K πt π t d wtk + X k ∈K πt π t d dtk = X i ∈Z π X k ∈K πti ζ ti u gtk D maxtk + X i ∈Z π X k ∈K πti ζ ti d wtk + X i ∈Z π X k ∈K πti ζ ti d dtk + κ t ∀ t ∈ T . (38)In (38), the terms π t u gtk D maxtk and ζ ti u gtk D maxtk involve theproducts of a binary variable and a continuous variable, andcan be handled as shown in (36)-(37). Furthermore, due to(9) and (13), the terms d wtk and d dtk refers to at-the-moneyUPP orders, where π t = P dtk , by definition. As a consequence,the terms π t d wtk and π t d dtk in (38) can be replaced by P dtk d wtk and P dtk d dtk , respectively.The term ζ ti d wtk in (38) is handled as follows. Firstly, byusing (30) the zonal prices is recast as: ζ ti = P dtk − ϕ wtk + ϕ w,lotk , (39)therefore, ζ ti d wtk becomes: d wtk P dtk − d wtk ϕ wtk + d wtk ϕ w,lotk . (40)Then, we recall that to obtain the single level problem inSection 3.3, the lower level has been recast by using a set ofequivalent necessary and sufficient conditions, and to avoidthe KKT complementary slackness, the strong duality prop-erty has been used. This is desirable because the KKT com-plementary slackness would introduce further non-linearities.However, the strong duality guarantees that all the KKTcomplementary slackness conditions hold [52, 53]. Therefore,we can use any subset of them for our purpose. In particular,the complementary slackness associated to the constraints(19)-(20) are defined as follows:( d wtk − u wtk D maxtk ) ϕ wtk = 0 ⇐⇒ d wtk ϕ wtk = u wtk D maxtk ϕ wtk (41) − d wtk ϕ w,lotk = 0 . (42)Hence, by using (40)-(42) the following relation can be ob-tained: ζ ti d wtk = d wtk P dtk − u wtk D maxtk ϕ wtk , (43)which involves only the product of a binary and a continuousvariable, and can be handled as showed in (36)-(37).Finally, the only remaining non-linearity to handle is theterm ζ ti P k ∈K πti d dtk which is present not only in (38) but alsoin (29). To deal with this term we use a binary expansion.The basic idea of a binary expansion is to convert an integernumber in binary form by using binary variables [54, 55, 56].In particular, the binary expansion is utilized to convert thequantity P k ∈K πti d dtk in binary form, as follows: J X j =0 b tji j = 10 c X k ∈K πti d dtk ∀ t ∈ T , ∀ i ∈ Z π , (44)where b tji ∈ { , } , J is an appropriate parameter dependingon the market data, and c is the number of decimal digits al-lowed in the considered market. In the Italian market c = 3.The right-hand-side in (44) represents the term under con-version, where the value P k ∈K πti d dtk is multiplied by 10 c inc (cid:13) c , which depends on the market specifications, thediscretization performed in (44) is exact. As a consequencethe following relation holds: ζ ti X k ∈K πti d dtk = 10 − c ζ ti J X j =0 b tji j . (45)Therefore, substituting the terms described, and simpli-fying the common terms, the UPP definition (38), can beequivalently recast as: X k ∈K πt π t u gtk D maxtk + X k ∈K πt P dtk d dtk = X i ∈Z π X k ∈K πti ζ ti u gtk D maxtk − X k ∈K πt u wtk D maxtk ϕ wtk + 10 − c X i ∈Z π ζ ti J X j =0 b tji j + κ t ∀ t ∈ T , (46)which involves only the products of a binary and a continuousvariable that can be handled as showed in (36)-(37). Thedefinition (46) is exact, with no approximation provided thatthe parameter c , introduced in (44), is selected properly.Starting from the single level model described in Section3.3, by using (46) in place of (6), substituting (45) in (29),and after removing all the non-linearities due to the productsof a binary and a continuous variable as outlined in (36)-(37),we obtain the final MILP model reported in Appendix A.The MILP model solves the market clearing problem in thepresence of both the UPP and curtailable profile block or-ders by using an exact social welfare maximization approachwithout any heuristic or iterative methods.
4. Implementation details
Under the UPP pricing scheme, all the UPP orders havea merit order, i.e., a parameter that determines a strict to-tal ordering among the orders, as described in Section 1.1.All the UPP orders must be executed sequentially, accord-ing to the priority established by the merit order. However,the current implementation of the UPP pricing scheme inthe European market, strictly enforces the merit order onlyfor the in-the-money UPP orders, see (10). By contrast, themerit order for the ATM UPP orders is enforced only as longas there is enough transmission capacity between the zonesinvolved, i.e., if there is no market split. For this reason, themerit order for ATM orders is currently enforced ex-post,given the market solution. In this section, we propose a setof constraints to detect whether the connecting lines are con-gested or not. In particular, we introduce a set of conditionsto detect market splits, and to enforce the merit order forthe ATM UPP orders directly within the optimization prob-lem. These constraints are upper level constraints. However,they are not strictly required by the current European mar-ket rules. For this reason, they are described in this sectionand not in Section 3.2. Notice that, these constraints allowone to fix a significant part of the ATM quantities actuallyexecuted, with a significant reduction of the search space ofthe binary expansion (44). Specifically, for all t ∈ T , i ∈ Z π ,and h, k ∈ K πti , such that O mth < O mtk , and P dth = P dtk , thefollowing constraint is added: d wth + d dth ≥ D maxth u etk . (47) Furthermore, for all t ∈ T , i, j ∈ Z π , h ∈ K πti , and k ∈ K πtj ,with i = j , such that O mth < O mtk , P dth = P dtk , and F maxtij > f tij ≤ F maxtij − ǫ f + u ftij (48) f tij ≥ F maxtij − M Ftij (1 − u ftij ) (49) u ftij ∈ { , } (50) d wth + d dth ≥ D maxth u etk − M Dth u ftij − M Dth u ftji . (51)Notice that merit orders, prices and maximum flow capac-ities are inputs, therefore the above constraints are linear.The term ǫ f is a sufficiently small parameter, whereas M Ftij and M Dth are appropriate large constants, that are definedin Appendix A. Constraint (47) enforces the merit order forATM orders within the same zone. Furthermore, from (23)and (48)-(50), u ftij = 1 if and only if f tij = F maxtij , i.e., theline is congested. Constraint (51) enforces the merit orderfor ATM orders in zones directly connected. If the line isnot saturated, then u ftij = 0, and the constraint is enforced,otherwise there is market split and the constraint is deacti-vated. Constraint (51) can be further generalized to zonesconnected through a path involving multiple lines. In thiscase, to enforce the constraint, all the variables u ftij must bezero, i.e., the lines connecting the zones i and j must not besaturated, otherwise there is a market split along the path,and the merit order must not be enforced. As a remark, no-tice that a loose formulation of u ftij can be implemented byusing: u ftij ≤ f tij + F maxtji F maxtij + F maxtji (52)instead of (48)-(49). In this case if f tij < F maxtij , then u ftij = 0,but the converse is not true. This approach appears to bepreferable for dealing with solvers or modeling languages thatdo not allow to prioritize the binary variables. Indeed, themerit order for ATM orders can be regarded as a secondaryrequirement, and it should be enforced by the solver only atthe end of the Branch-and-Bound process. This can be per-formed efficiently by setting the lowest priority to the binaryvariables u ftij .
5. Numerical Results
This section describes the numerical results obtained bytesting the MILP model introduced in Section 3.4, and re-ported in Appendix A.
The data used to test the proposed MILP model was down-loaded from the website of the Italian market operator [15].It refers to the day-ahead market, and covers 31 days, rangingfrom January 1 st to January 31 st , 2018. Each day involveson average 20,307 demand orders and 37,810 offer orders.These orders are distributed over 22 zones. Six zones arethe Italian physical zones, which enforce the UPP scheme,whereas the remaining zones do not enforce the UPP. Werecall that the Italian UPP is termed PUN. Artificial orderswere randomly added to test curtailable profile block orders.Specifically, for a block order p , the maximum hourly quan-tity offered S B,maxtp was sampled from a uniform distributionranging from 1 MWh to 75 MWh, whereas the block orderprice P Bp was generated by using a normal distribution withc (cid:13) ourly Problem, January 1 st to 31 st N u m be r o f B i na r y V a r i ab l e s Binary VariablesCplex Time C p l e x T i m e ( s e c ond s ) Figure 7: Test of the proposed MILP model using real data of the Italianday-ahead market, from January 1 st to January 31 st , 2018. The testinvolves 744 independent hourly problems. For each instance of theproblem, the figure reports the number of binary variables (bar chart,left axis), and the computation time (line chart, right axis) to reach theoptimal solution. mean 50 e /MWh and standard deviation of 10 e /MWh. TheMAR is set to 10%. The MILP model was implemented inPython 2.7 with Pyomo 5.2 [57], and solved with CPLEX 12.5[58] on a 8-core 2.40GHz Intel(R) Xeon(R) CPU E5-2630 v3,with 32 GB of RAM. The Python code is freely accessibleon GitHub [59], and the documentation of the main func-tions is available at [60]. The documentation describes alsohow to obtain the Italian data from [15]. Orders without aprice limit, listed in the Italian market with a price P dtk = 3000Euro/MWh, are assumed to be fully executed. The value ofthe parameter ǫ in (8) can be set arbitrarily small. Since theItalian market limits the resolution of the PUN to six digits,any value not greater than 10 − is acceptable. The tuningof the big-M parameters in (7)-(8), and (12) is discussed inAppendix A. The first test takes into account only the real data of theItalian day-ahead market, which ranges from January 1 st toJanuary 31 st , 2018, and includes 1,801,652 market orders.The purpose is to verify the effectiveness of the proposed ap-proach in solving the UPP clearing scheme. The test involves744 hourly PUN problems. On average, each instance of theproblem contains 985 binary variables, and is solved to opti-mality in 8 .
74 seconds. In the performed tests, all the clearedquantities match the real quantities executed on the Italianmarket. The binary expansion (44) is actually utilized onlyin 9 cases out of 744 (1.21% of the cases). That is, in thesecases the solution contains at least one order with u dtk = 1.Figure 7 reports for each hourly PUN problem the numberof binary variables involved, and the time to reach the op-timal solution. The largest spikes in the computation timecorrespond to the instances where the binary expansion actu-ally takes place. The maximum time is 74.60 seconds, whichcorresponds to the 20th hour of January 24th. The second test involves the data of the Italian market op-erator referring to January 1 st , with the addition of 50 cur-tailable profile block orders randomly generated as describedin Section 5.1. Each block order spans from the 9th to the20th hour. To test the effectiveness of the proposed MILPmodel, the block orders are evenly distributed between an PUN zone (Sicily) and a non-PUN zone (Swiss). The pres-ence of block orders requires to solve a single MILP problemspanning the whole considered day. The clearing probleminvolves 19,246 PUN demand orders, 414 non-UPP demandorders, 34,927 supply orders, 50 curtailable profile block or-ders, and is solved in 848 .
79 seconds.Table 1 reports the real Italian PUN (second column), andthe PUN obtained from the proposed model (third column).The shadowed rows correspond to the hours where the blockorders have been added. As can be observed, the differencebetween the real and the modeled PUN is zero up to thefourth decimal place in the hours where the block orders arenot present. This discrepancy is due to the tolerance param-eter κ t in (6), which can lead to slight differences in the PUN,despite the same matched quantities. By contrast, from the9th to the 20th hour, the presence of block orders leads toa decrease in the PUN, which is caused by the additionalquantity supplied by the block orders. Table 1: Test with real Italian market data and 50 curtailable profileblock orders over 12 hours
Hour PUN (Real) PUN with block orders Difference( e /MWh) ( e /MWh) ( e /MWh)h1 45.822320 45.822277 -0.000043h2 44.160000 44.159954 -0.000046h3 42.240000 42.239952 -0.000048h4 39.290000 39.289950 -0.000050h5 36.000000 35.999949 -0.000051h6 41.990000 41.989950 -0.000050h7 42.250000 42.249953 -0.000047h8 44.970000 44.969955 -0.000045h9 45.000000 43.239955 -1.760045h10 44.940000 43.819958 -1.120042h11 45.024790 44.643418 -0.381372h12 45.709640 45.709599 -0.000041h13 46.700810 45.412933 -1.287877h14 43.980140 43.652657 -0.327483h15 44.961410 41.827956 -3.133454h16 47.532150 45.170423 -2.361727h17 49.906020 49.199960 -0.706060h18 54.300000 52.699966 -1.600034h19 51.910000 50.329967 -1.580033h20 51.380000 50.069968 -1.310032h21 49.200000 49.199967 -0.000033h22 45.730000 45.729965 -0.000035h23 44.840000 44.839962 -0.000038h24 38.110000 38.109958 -0.000042Table 2 reports the surplus of each block order and theacceptance ratio r p . All the block orders with a positivesurplus, i.e., the ITM block orders, are fully cleared ( r p = 1).The block orders with a negative surplus, i.e., the OTM blockorders, are correctly rejected ( r p = 0). Furthermore, noticethat block order 15 in the PUN zone has zero surplus (seethe corresponding row in the left part of Table 2). That is,it is an ATM block order, and it is partially cleared with r p = 0 . (cid:13) able 2: Surplus and acceptance ratio of the 50 curtailable profile blockorders over 12 hours PUN zone non-PUN zoneBlock order r p Surplus Block order r p Surplus
6. Conclusions
The coupling of all the European electricity markets is anongoing process which faces several difficulties. In particular,the presence of heterogeneous orders and rules, such as blockorders and the uniform purchase price, rises several issues.The proposed mixed-integer linear program allows one tosolve the market clearing problem in the presence of both cur-tailable profile block orders and the uniform purchase pricescheme. In particular, it harmonizes within a unique opti-mization program two classes of heterogeneous orders andrules. An exact social welfare maximization problem is for-mulated and solved, as required by European guidelines. Theproposed approach is non-iterative, heuristic-free, and thesolution is exact, with no approximation up to the level ofresolution of current market data. In addition, the solutionis obtained coherently with the marginal pricing framework.Finally, the mixed-integer linear program formulation allowsone to prove the optimality of the obtained solution.Ongoing work aims at introducing linked block orders,piecewise linear orders, and complex orders (as the Iberianminimum income condition) in the proposed framework. Itis expected that the income condition could be modeled in asimilar way as the surplus of the block order. By contrast,piecewise linear orders will pose additional issues, due to thepresence of quadratic terms in the objective function.
Acknowledgment
The authors would like to thank the anonymous Reviewersfor their valuable and constructive comments.
Appendix A. The final MILP model
The objective function of the final MILP model is definedin (28), whereas the problem’s constraints are: X k ∈K πt y gπtk D maxtk + X k ∈K πt P dtk d dtk = X i ∈Z π X k ∈K πti y gζtki D maxtk − X k ∈K πt y wϕtk D maxtk + 10 − c X i ∈Z π J X j =0 y bζtij j + κ t ∀ t ∈ T (A.1) X t ∈T X k ∈K ζt P dtk d ζtk + X t ∈T X k ∈K πt P dtk d wtk − X k ∈K t X p ∈P t P stp s tp − X p ∈P B X t ∈T p P Bp r p S B,maxtp = X t ∈T X k ∈K πt y wϕtk D maxtk + X t ∈T X k ∈K ζt ϕ ζtk D maxtk + X t ∈T X p ∈P t ϕ stp S maxtp + X t ∈T X i ∈Z X j ∈Z δ maxtij F maxtij − X t ∈T X i ∈Z π X k ∈K πti y gζtki D maxtk − − c X t ∈T X i ∈Z π J X j =0 y bζtij j + X p ∈P B y Bϕ,maxp − X p ∈P B y Bϕ,minp R minp (A.2)(7)-(8) (A.3) u etk P dtk − y eπtk = 0 ∀ t ∈ T , ∀ k ∈ K πt (A.4)(10)-(17) (A.5)(19)-(27) (A.6)(30)-(34) (A.7)(44) , (A.8)where the auxiliary variables are defined as follows: − M π u gtk ≤ y gπtk ≤ M π u gtk (A.9) − M π (1 − u gtk ) ≤ π − y gπtk ≤ M π (1 − u gtk ) (A.10) − M π u etk ≤ y eπtk ≤ M π u etk (A.11) − M π (1 − u etk ) ≤ π − y eπtk ≤ M π (1 − u etk ) (A.12)0 ≤ y wϕtk ≤ M π u wtk (A.13)0 ≤ ϕ wtk − y wϕtk ≤ M π (1 − u wtk ) , (A.14)with t ∈ T , k ∈ K πt ; − M π u gtk ≤ y gζtki ≤ M π u gtk (A.15) − M π (1 − u gtk ) ≤ ζ ti − y gζtki ≤ M π (1 − u gtk ) , (A.16)with t ∈ T , i ∈ Z π , k ∈ K πti ; − M π b tji ≤ y bζtij ≤ M π b tji (A.17) − M π (1 − b tji ) ≤ ζ ti − y bζtij ≤ M π (1 − b tji ) , (A.18)with t ∈ T , i ∈ Z π , j ∈ { , . . . , J } , where J is an appropriateparameter depending on market data;0 ≤ y Bϕ,maxp ≤ M Bp u Bp (A.19)0 ≤ ϕ B,maxp − y Bϕ,maxp ≤ M Bp (1 − u Bp ) (A.20)0 ≤ y Bϕ,minp ≤ M Bp u Bp (A.21)0 ≤ ϕ B,minp − y Bϕ,minp ≤ M Bp (1 − u Bp ) , (A.22)c (cid:13) p ∈ P B .In (8) the value of ǫ is 10 − , whereas in (48) the valueof ǫ f is 10 − . In the Italian market, the maximum price M π , used in (7)-(8) and (A.9)-(A.18), is 3000 Euro/MWh.Moreover, considering that block orders span over multiplehours, the value of M Bp in (12) and (A.19)-(A.22) is set to M π P t ∈T p S B,maxtp . In (49) the parameter M Ftij is defined as M Ftij = F maxtij + F maxtji , whereas in (51) the parameter M Dth isdefined as M Dth = D maxth .Furthermore, in order to reduce significantly the searchspace of the upper level binary variables, the following con-straint can be implemented: u etk ≤ u gth − u gtk ∀ t ∈ T , ∀ h, k ∈ K πt , (A.23)such that P dth > P dtk . ReferencesReferences [1] P. Zou, Q. Chen, Y. Yu, Q. Xia, C. Kang, Elec-tricity markets evolution with the changing gen-eration mix: An empirical analysis based onchina 2050 high renewable energy penetrationroadmap, Applied Energy 185 (2017) 56 – 67. doi:https://doi.org/10.1016/j.apenergy.2016.10.061 .[2] J. Mrkle-Hu, S. Feuerriegel, D. Neumann,Large-scale demand response and its implica-tions for spot prices, load and policies: In-sights from the german-austrian electricity mar-ket, Applied Energy 210 (2018) 1290 – 1298. doi:https://doi.org/10.1016/j.apenergy.2017.08.039 .[3] A. S. M. Khan, R. A. Verzijlbergh, O. C. Sak-inci, L. J. D. Vries, How do demand response andelectrical energy storage affect (the need for) a ca-pacity market?, Applied Energy 214 (2018) 39 – 62. doi:https://doi.org/10.1016/j.apenergy.2018.01.057 .[4] K. K. Iychettira, R. A. Hakvoort, P. Linares,R. de Jeu, Towards a comprehensive policy forelectricity from renewable energy: Designing forsocial welfare, Applied Energy 187 (2017) 228 – 242. doi:https://doi.org/10.1016/j.apenergy.2016.11.035 .[5] P. Olivella-Rosell, E. Bullich-Massagu, M. Arags-Pealba, A. Sumper, S. degaard Ottesen, J.-A. Vidal-Clos, R. Villaffila-Robles, Optimization problemfor meeting distribution system operator requestsin local flexibility markets with distributed en-ergy resources, Applied Energy 210 (2018) 881 – 895. doi:https://doi.org/10.1016/j.apenergy.2017.08.136 .[6] X. Ayn, J. Gruber, B. Hayes, J. Usaola, M. Pro-danovi, An optimal day-ahead load schedulingapproach based on the flexibility of aggregatedemands, Applied Energy 198 (2017) 1 – 11. doi:https://doi.org/10.1016/j.apenergy.2017.04.038 .[7] P. Duenas, A. Ramos, K. Tapia-Ahumada, L. Ol-mos, M. Rivier, J.-I. Prez-Arriaga, Security of supplyin a carbon-free electric power system: The caseof iceland, Applied Energy 212 (2018) 443 – 454. doi:https://doi.org/10.1016/j.apenergy.2017.12.028 .[8] B. Aluisio, M. Dicorato, G. Forte, M. Trovato, A. Sallati,C. Gadaleta, C. Vergine, F. Ciasca, The application of aflow-based methodology for yearly network analysis ac-cording to market data, in: 2017 14th International Con-ference on the European Energy Market (EEM), 2017,pp. 1–6. doi:10.1109/EEM.2017.7981918 .[9] J.-M. Glachant, Mapping the course of the EU ‘PowerTarget Model’ on its own terms, Robert Schuman Centrefor Advanced Studies Research Paper No. RSCAS 23.[10] D. Newbery, G. Strbac, I. Viehoff, The benefits of inte-grating European electricity markets, Energy Policy 94(2016) 253–263.[11] T. Brijs, C. D. Jonghe, B. F. Hobbs, R. Belmans,Interactions between the design of short-term elec-tricity markets in the CWE region and power sys-tem flexibility, Applied Energy 195 (2017) 36 – 51. doi:doi.org/10.1016/j.apenergy.2017.03.026 .[12] H. Farahmand, G. Doorman, Balancing market in-tegration in the Northern European continent, Ap-plied Energy 96 (Supplement C) (2012) 316 – 326. doi:doi.org/10.1016/j.apenergy.2011.11.041 .[13] PCR PXs, EUPHEMIA Public Description PCR Market Coupling Algorithm,Version 1.5.URL [14] Polo espa˜nol S.A., Daily and intraday electricity market operating rules.URL [15] Gestore dei Mercati Energetici S.p.A. [link].URL [16] EPEX SPOT SE. [link].URL [17] Nord Pool AS. [link].URL [18] P. N. Biskas, D. I. Chatzigiannis, A. G. Bakirtzis, Euro-pean electricity market integration with mixed marketdesigns. Part I: Formulation, IEEE Trans. Power Syst.29 (1) (2014) 458–465.[19] M. Madani, M. Van Vyve, Minimizing opportunity costsof paradoxically rejected block orders in European day-ahead electricity markets, in: 11th International Con-ference on the European Energy Market (EEM), 2014,IEEE, 2014, pp. 1–6.[20] M. Madani, M. V. Vyve, A. Marien, M. Maen-houdt, P. Luickx, A. Tirez, Non-convexities in Euro-pean day-ahead electricity markets: Belgium as a casestudy, in: 13th International Conference on the Eu-ropean Energy Market (EEM), 2016, 2016, pp. 1–5. doi:10.1109/EEM.2016.7521190 .[21] D. S. Kirschen, G. Strbac, Fundamentals of power sys-tem economics, John Wiley & Sons, 2004.[22] D. Rubinfeld, R. Pindyck, Microeconomics, Pearson,2013.[23] F. E. R. C. Staff, Uplift in RTO and ISO markets(2014).URL c (cid:13) [25] D. Huppmann, S. Siddiqui, An exact solution methodfor binary equilibrium problems with compensationand the power market uplift problem, European Jour-nal of Operational Research 266 (2) (2018) 622 – 638. doi:https://doi.org/10.1016/j.ejor.2017.09.032 .[26] D. I. Chatzigiannis, G. A. Dourbois, P. N. Biskas, A. G.Bakirtzis, European day-ahead electricity market clear-ing model, Electric Power Systems Research 140 (2016)225–239.[27] M. Madani, M. Van Vyve, Computationally efficientMIP formulation and algorithms for European day-ahead electricity market auctions, European Journal ofOperational Research 242 (2) (2015) 580–593.[28] M. Madani, M. V. Vyve, Revisiting minimumprofit conditions in uniform price day-ahead elec-tricity auctions, European Journal of OperationalResearch doi:https://doi.org/10.1016/j.ejor.2017.10.024 .[29] E. J. Zak, S. Ammari, K. W. Cheung, Mod-eling price-based decisions in advanced electricitymarkets, in: 2012 9th International Conferenceon the European Energy Market, 2012, pp. 1–6. doi:10.1109/EEM.2012.6254813 .[30] APX, BELPEX and EPEX SPOT,COSMOS description - CWE Market Coupling algorithm,Version 1.1.URL http://static.epexspot.com/document/20015/COSMOS_public_description.pdf [31] L. Meeus, K. Verhaegen, R. Belmans, Blockorder restrictions in combinatorial electric en-ergy auctions, European Journal of Opera-tional Research 196 (3) (2009) 1202 – 1206. doi:https://doi.org/10.1016/j.ejor.2008.04.031 .[32] L. H. Lam, V. Ilea, C. Bovo, Europeanday-ahead electricity market coupling: Dis-cussion, modeling, and case study, ElectricPower Systems Research 155 (2018) 80 – 92. doi:https://doi.org/10.1016/j.epsr.2017.10.003 .[33] A. G. Vlachos, P. N. Biskas, Balancing supply and de-mand under mixed pricing rules in multi-area electricitymarkets, IEEE Trans. Power Syst. 26 (3) (2011) 1444–1453.[34] A. G. Vlachos, P. N. Biskas, Simultaneous clearing ofenergy and reserves in multi-area markets under mixedpricing rules, IEEE Trans. Power Syst. 26 (4) (2011)2460–2471.[35] D. I. Chatzigiannis, P. N. Biskas, G. A. Dourbois,European-type electricity market clearing model incor-porating PUN orders, IEEE Trans. Power Syst. 32 (1)(2017) 261–273. doi:10.1109/TPWRS.2016.2542823 .[36] G. A. Dourbois, P. N. Biskas, European market couplingalgorithm incorporating clearing conditions of block andcomplex orders, in: PowerTech, 2015 IEEE Eindhoven,IEEE, 2015, pp. 1–6. [37] G. A. Dourbois, P. N. Biskas, A novel method for theclearing of a day-ahead electricity market with mixedpricing rules, in: 2017 IEEE Manchester PowerTech,2017, pp. 1–6. doi:10.1109/PTC.2017.7981100 .[38] I. Savelli, A. Giannitrapani, S. Paoletti, A. Vi-cino, An optimization model for the electricitymarket clearing problem with uniform purchaseprice and zonal selling prices, IEEE Trans. PowerSyst. doi:10.1109/TPWRS.2017.2751258 .[39] ´A. Sleisz, D. Raisz, Integrated mathematical model foruniform purchase prices on multi-zonal power exchanges,Electric Power Systems Research 147 (2017) 10–21.[40] Tabors Caramanis and Associates Inc.,UPPO Auction Module User Manual (2002).URL [41] R. Fern´andez-Blanco, J. M. Arroyo, N. Alguacil, Con-sumer payment minimization under uniform pricing: Amixed-integer linear programming approach, AppliedEnergy 114 (2014) 676–686.[42] California Independent System Operator,Technical Bulletin - Market optimization details(2009).URL [43] L. H. Lam, V. Ilea, C. Bovo, Impact of the price couplingof regions project on the day-ahead electricity market initaly, in: 2017 IEEE Manchester PowerTech, 2017, pp.1–6. doi:10.1109/PTC.2017.7981215 .[44] F. Schweppe, M. Caraminis, R. Tabors, R. Bohn, Spotpricing of electricity, Kluwer Academic Publishers, Nor-well, MA, 1988.[45] M. C. Caramanis, R. E. Bohn, F. C. Schweppe, Optimalspot pricing: Practice and theory, IEEE Trans. PowerApp. Syst. PAS-101 (1982) 3234–3245.[46] Commission of European Union,Commission regulation (EU) 2015/1222 of 24 july 2015 establishing a guideline on capacity allocation and congestion management(2015).URL data.europa.eu/eli/reg/2015/1222/oj [47] J. Bard, Practical bilevel optimization: applications andalgorithms, Kluwer Academic Press Dordrecht, Nether-lands, 1998.[48] R. Fern´andez-Blanco, J. M. Arroyo, N. Alguacil, A uni-fied bilevel programming framework for price-based mar-ket clearing under marginal pricing, IEEE Trans. PowerSyst. 27 (1) (2012) 517–525.[49] S. A. Gabriel, A. J. Conejo, J. D. Fuller, B. F. Hobbs,C. Ruiz, Complementarity modeling in energy markets,Vol. 180, Springer Science & Business Media, 2012.[50] S. Bradley, A. Hax, T. Magnanti, Applied mathematicalprogramming, Addison Wesley, 1977.[51] A. J. Conejo, M. Carri´on, J. M. Morales, Decision mak-ing under uncertainty in electricity markets, Vol. 1,Springer, 2010.c (cid:13) doi:10.1137/110836183 .[55] M. V. Pereira, S. Granville, M. H. Fampa, R. Dix, L. A.Barroso, Strategic bidding under uncertainty: a binaryexpansion approach, IEEE Trans. Power Syst. 20 (1)(2005) 180–188.[56] J.-S. Roy, binarize and project to generate cuts for gen-eral mixed-integer programs, Algorithmic OperationsResearch 2 (1).[57] W. E. Hart, C. D. Laird, J.-P. Watson, D. L. Woodruff,G. A. Hackebeil, B. L. Nicholson, J. D. Siirola, Pyomo–optimization modeling in python, 2nd Edition, Vol. 67,Springer Science & Business Media, 2017.[58] IBM-ILOG, CPLEX User’s Manual - Version 12 Release7 (2016).[59] B. Corn´elusse, I. Savelli, A. Giannitrapani, S. Paoletti,A. Vicino, Open Day-Ahead Market - Implementation,Version 1.0 (2018).URL https://github.com/bcornelusse/openDAM [60] B. Corn´elusse, I. Savelli, A. Giannitrapani, S. Paoletti,A. Vicino, Open Day-Ahead Market - Documentation(2018).URL http://opendam.readthedocs.io c (cid:13)(cid:13)