A Novel Graphic Bending Transformation on Benchmark
aa r X i v : . [ c s . N E ] A p r A Novel Graphic Bending Transformation onBenchmark
Fengyang Sun, Qingrui Ni, Lin Wang ∗ , Bo Yang, and Chunxiuzi Liu Shandong Provincial Key Laboratory of Network based Intelligent Computing,University of Jinan, Jinan 250022, China ∗ Corresponding author. E-mail: [email protected]
Abstract.
Classical benchmark problems utilize multiple transforma-tion techniques to increase optimization difficulty, e.g., shift for anti cen-tering effect and rotation for anti dimension sensitivity. Despite testingthe transformation invariance, however, such operations do not reallychange the landscape’s “shape”, but rather than change the “view point”.For instance, after rotated, ill conditional problems are turned aroundin terms of orientation but still keep proportional components, which, tosome extent, does not create much obstacle in optimization. In this paper,inspired from image processing, we investigate a novel graphic conformalmapping transformation on benchmark problems to deform the functionshape. The bending operation does not alter the function basic proper-ties, e.g., a unimodal function can almost maintain its unimodality afterbent, but can modify the shape of interested area in the search space.Experiments indicate the same optimizer spends more search budget andencounter more failures on the conformal bent functions than the rotatedversion. Several parameters of the proposed function are also analyzedto reveal performance sensitivity of the evolutionary algorithms.
Keywords:
Benchmark Problem · Graphic Transformation · Evolution-ary Computation · Conformal Mapping · Ill Conditional Problem.
The ill conditional property of an optimization problem, generally concerns thatthe sensitivity of function value changes in terms of different variables or searchdirections is distinct [6]. In other word, an ill conditional function is difficultto be optimized because it has such properties: the change in some variables ordirections does slight influence on function value, but a minute change in othervariables or directions can instill a drastically substantial change in functionvalue [7,8]. The previous literature has empirically shown that covariance matrixadaptation evolution strategy (CMA-ES) is efficient in solving such problems [5].We can observe from Fig. 1 that the optimizer spent some time on exploring thebest region over the general search space (most areas of which are undesirable) inthe early stage. However, once it positioned the best region, which is the “bandedlong narrow valley”, the superior local search capability of CMA-ES can guide
F. Sun et al.
Fig. 1.
Visualization for search trace of CMA-ES on a rotated ill conditional function(the bent cigar function). The trace is drawn with black line and the optimum isdenoted with black ∗ . the search agent to rapidly approach the optimum with negligible demand offurther tuning search directions [5].The intuitive inference could be that, the “straight banded valley” providessome convenience for optimization. Since the principal search direction of thisarea is a unique straight line, a strategy similar to line search can gain substantialbenefits after it decides the right search direction around the area. Therefore,the question arises here is, what if the valley is winding instead of straight? Classical benchmark constructing approaches test optimizer generalizationand invariance in some aspects by orthogonal transformations such as rotation.Nevertheless, such transformations does not really change the function “shape”,but rather than achieve certain adjustments in the sense of “view point”. Forinstance, in two dimensional scenario, an ill conditional problem simply changesthe eye focus horizontally or vertically after translated, and changes the eye fo-cus rotationally after rotated (see Fig. 3). However, these transformations bothkeep the straight line shape of raw function. By contrast, the conformal map-ping in topology maintain some basic properties such as local angles and notnecessarily maintain others such as lengths when processing images [9]. Thesecharacteristics enable conformal transformation altering the general appearanceof an ill conditional function but basically still keep its unimodality [10].In this paper, we introduce a novel bending function operation with conformalmapping to test the search behavior changes of evolutionary algorithms. Weconstruct a conformal ill conditional function in two dimensional case. The CMA-ES and particle swarm optimization (PSO) [1] algorithms are evaluated and
Novel Graphic Bending Transformation on Benchmark 3
Original Conformal
Fig. 2.
A grid image is transformed by conformal mapping. The blue horizontal lineand the yellow line have been bent as circles. considerable performance changes are observed based on the behavior statistics.In addition, we demonstrate the involved optimizers are sensitive to positions ofthe optimum and starting search points by tuning the problem parameters.
Conformal mapping is a branch of complex variable function theory. It studiescomplex variable functions from a geometric point of view, which maps a re-gion to another region through an analytic function. Conformal transformationmaintains the angle and the shape of infinitesimal objects. Unlike the tradi-tional translation mapping or rotation, it does not necessarily maintain theirsize. Conformal mapping has two important properties: rotation angle invari-ance and scaling invariance. The reader needs to distinct them from the similardescriptions mentioned in [6].Let the function f ( z ) be parsed in the region Ω , z ∈ Ω , and f ( z ) ′ = 0.Rotation angle invariance means that the angle between two curves passing apoint z in the area Ω and the angle between the two curves obtained aftermapping remain unchanged in size and direction. Scaling invariance means thatthe scaling rate of any curve passing through z is (cid:12)(cid:12) f ( z ) ′ (cid:12)(cid:12) , regardless of its shapeand direction. If the above conditions are met, we call w = f ( z ) a conformalmapping at z [9]. Linear fractional transformation is a commonly used mapping, the basic form ofwhich is w = f ( z ) = az + bcz + d , where z and w are complex variables and a, b, c, d, ∈ R .Let w take a derivative w.r.t. z , and the outcome is ∂w∂z = ad − bc ( cz + d ) . Accordingto the definition of conformal mapping, we can draw a conclusion that f ( z ) is aform of conformal mapping if ad − bc = 0. w = az + bcz + d = (cid:26) ad ( z + ba ) , c = 0 ac + bc − adc ( cz + d ) , c = 0 (1) F. Sun et al.
As shown in Eq. 1, any fractional linear map can be decomposed into threebasic maps: translation mapping w = z + b , similarity mapping w = az , andinversion mapping w = z . Linear fractional transformation has the properties ofone-to-one correspondence and circularity-preserving on the extended complexplane, that is, the circle before the mapping (a straight line is regarded as acircle passing through infinity) is still a circle after mapping.We use the inversion mapping to process the image, and find that the in-version transform distorts the image content (Fig. 2), which is difficult to beimplemented by traditional orthogonal transformations such as translation orrotation. In this way, we can infer that if the search path is inversely transformed,the distortion degree of the path can be changed, resulting in the problem withvarying search difficulty. The conformal version of an ill conditional function is constructed by mappingthe input position vector with w = 1 /z before computing the function value.The entire mapping process consists of three steps, forward box transformationto shrink the original search space, conformal mapping to change the functionshape, inverse box transformation to recover the search space. The input positionvector is generally formed with x = ( x , x , . . . , x D ), where D is the problemdimensionality. In this paper, we specifically focus on two dimensional problem,which makes x = ( x , x ). The following contents of this section explain theimplementation details of the three steps. We use the constructed scale factor s forward and shift factor o forward to scaleand translate the input vector, respectively. For each input vector x ∈ Ω , afterforward box transformation, we get a new form of x ′ in the domain Ω ′ . Thedomain Ω ′ is usually much smaller than the original domain Ω . The computationof x ′ are shown in Eqs. 2, 3, and 4. s forward = ( 2 ξL , − ψL ) , (2) o forward = ( ξ, ψ ) , (3) x ′ = x ◦ s forward + o forward , (4)where ξ and ψ are 2 hyper parameters controlling function shape after deformed. L denotes the scale level of the space, empirically set equal to the length of eachdimension of the hypercube. Also note that we set o forward = (0 ,
0) for simplicityin this paper.
Novel Graphic Bending Transformation on Benchmark 5
Fig. 3.
Visualization for different versions of the bent cigar function. The black ∗ denotes position of the optimum. At this step, we put the vector x ′ in the complex plane and conduct conformalinversion mapping to obtain the mapped vector x ′′ . The computation of x ′′ areshown in Eqs. 5, 6, and 7. z = complex( x ′ ) = x ′ + x ′ i , (5) w = 1 z = 1 x ′ + x ′ i = x ′ x ′ + x ′ − x ′ x ′ + x ′ i , (6) x ′′ = decomplex( w ) = ( x ′ x ′ + x ′ , − x ′ x ′ + x ′ ) , (7)where the function complex transforms a two dimensional vector into a com-plex number and function decomplex transforms a complex number into a twodimensional vector. After conformal mapping, we get the new vector x ′′ in the domain Ω ′ . In order toget the new form of x mapped in the original domain Ω , we use the constructedinverse scaling factor s inverse and the inverse translation factor o inverse to transferit inversely to Ω , noted x ′′′ . Then x ′′′ can be used to calculate the real functionvalue. The computation of x ′′′ are shown in Eqs. 8, 9, and 10. s inverse = ( 2 υL , − ̟L ) , (8) o inverse = ( υ, ̟ ) , (9) x ′′′ = ( x ′′ − o inverse ) ⊘ s inverse , (10)where υ and ̟ are 2 hyper parameters controlling size of the “valley” andposition of the optimum after deformed. For avoiding any confusion, we hereuse ◦ and ⊘ to represent Hadamard element-wise multiplication and division ofvectors, respectively. F. Sun et al.
Trials F ES CMA-ES for Conformal Bent Cigar
Trials F ES CMA-ES for Rotated Bent Cigar
Trials F ES PSO for Conformal Bent Cigar
Trials F ES PSO for Rotated Bent Cigar
Rotated Bent Cigar M ean o f B e s t F i t ne ss -10 -5 PSOCMA-ES × M ean o f B e s t F i t ne ss -10 -5 Conformal Bent Cigar
PSOCMA-ES × Fig. 4.
The convergence distinction and
F ES distribution for CMA-ES and PSOon both rotated and conformal bent cigar functions over 100 independent repeatedtrials. The black horizontal solid line in the right subfigure represents average numberof fitness evaluations used to find the optimum. The black dots represents
F ES usedfor each trial and the red dots for the records that the algorithm failed. The smaller
F ES and lower fitness value indicate better performance.
Fig. 3 demonstrates the visual differences among the raw, rotated and con-formal bent cigar functions. The function is bent as a ring shape by conformalmapping as expected. In this scenario, the optimizer could be forced to carefullyadjust the direction all the time in the winding path and present poor efficiency.
In this section, we explore how the winding shape of the ill conditional functionimpacts search. The results show that the optimizers are extremely hard to findthe optimum after the ill conditional function is bent by conformal mapping. Inaddition, narrower valley around the optimum contributes more difficulty.Since multi restart CMA-ES [4] is empirically regarded efficient on ill con-ditional problems and PSO [2] on the contrary, we here focus on them to testthe performance variation with respect to function shape [6]. The reader is re-ferred to the corresponding citations for the detailed parameter settings andmain source codes of these optimization methods.The ill conditional function bent cigar [3] is used as the basic function to befurther transformed. The default parameter configurations are ξ , ψ , υ , ̟ = 1, L = 10. We choose the expected running time (ERT) [3] and mean best fitnessto measure the performances of the involved algorithms. The calculation formulaof the ERT is shown in Eq. 11. ERT = RT s + 1 − p s p s RT us , (11)where RT s and RT us denote the average number of function evaluations forsuccessful and unsuccessful runs, respectively. p s is the fraction of successful Novel Graphic Bending Transformation on Benchmark 7
Fig. 5.
Contour maps for visualizing two dimensional conform bent cigar functionswith various hyper parameters. The black ∗ denotes position of the optimum. Whenone parameter is changing, the others keep fixed, which are the default configurations ξ , ψ , υ , ̟ = 1. runs. “Successful” means that the algorithm reaches the predefined accuracylevel (10 − ) on the test function, which, to some extent, corresponds to the bestsolution. F ES denotes the number of function evaluations, and maximum of
F ES is set as 10 . Unless specified, the configurations of all the experimentsremain the same. Fig. 4 illustrates the distinct convergence behavior of the involved algorithmsbetween rotated and conformal ill conditional functions. The rotated version ofbent cigar function is relatively easy to be optimized for both CMA-ES andPSO. However, the difficulty is dramatically increasing after the function istransformed by conform mapping. The fitness values are continuously decreasingacross the whole search process but in an unbearable slow speed, especially inthe late phase of optimization. The larger performance variations also reveal theinstability and sensitivity of the optimizers on the conform problem, since theonly difference over various trials is the starting search point.Albeit CMA-ES outperformed PSO in terms of mean fitness values on bothproblems, the performance gap was narrowed on the conform one. Despite more
F. Sun et al.
CMA-ES P e r f o r m an c e D i s t r i bu t i on ξ PSO P e r f o r m an c e D i s t r i bu t i on ξ CMA-ES P e r f o r m an c e D i s t r i bu t i on ψ PSO P e r f o r m an c e D i s t r i bu t i on ψ Fig. 6.
The
F ES and ERT over 100 trials versus increasing hyper parameters ξ and ψ . The other parameters keep unchanged when testing one parameter. The raw valuesare divided by the minimum value and logarithmically scaled to enhance the contrast. trials that find the optimum with smaller F ES , CMA-ES presented largerperformance variation and two times more failed trials than PSO.
The parameter analysis of the conform operation are summarized in visualiza-tions (Fig. 5) and numerical results (Fig. 6 and 7). From Fig. 5 top two rows, theparameters ξ and ψ primarily impact the ring shape by tuning the proportionbetween the “long axis” and “short axis” of the ring. As ξ increases, the ring be-comes lanky. Although the curvature of the search direction around the optimumis getting smaller, the width of the ring is also narrowing with increasing ξ . Asshown in Fig. 6, this case does not bother PSO much, but produce considerabledifficulty for CMA-ES.As ψ increases, the ring becomes prolate. Since the width of the ring is notnarrowing severely compared to ξ , and the ring area far from the optimum isalso becoming “soft winding” (smaller curvature), the search agent is able toapproach the optimum along the ring. Therefore, CMA-ES was less influencedby ψ compared to ξ and performed better as ψ increases (Fig. 6). From Fig. 5 bottom two rows, the parameters υ and ̟ primarily impact the ringsize and optimum position of the conformal bent cigar function. As υ increases, Novel Graphic Bending Transformation on Benchmark 9
2 4 6 8
CMA-ES P e r f o r m an c e D i s t r i bu t i on υ PSO P e r f o r m an c e D i s t r i bu t i on υ CMA-ES P e r f o r m an c e D i s t r i bu t i on ϖ PSO P e r f o r m an c e D i s t r i bu t i on ϖ Fig. 7.
The
F ES and ERT over 100 trials versus increasing hyper parameters υ and ̟ . The other parameters keep unchanged when testing one parameter. The raw valuesare divided by the minimum value and logarithmically scaled to enhance the contrast.Part of the ERT curves are not shown since the corresponding ERT values are infinite,which indicates all of the trials under the same parameter value failed. the basic function shape is not changing but the optimum position is beingmoved to the “thinner” area, which is closed to highly undesirable red points. Inthis case, the search agent is easier to make mistakes around the subtle ring dotlike walking high wire. The CMA-ES and PSO both present terrible performanceunder larger υ value, as shown in Fig. 7. When the value of υ is larger than 2,the involved optimizers can not find position the optimum in any trial withinpreset number of fitness evaluations.As ̟ is increasing, the ring size is becoming smaller. Although the width ofthe ring is also narrowing, the entire area of best region is shrinking due to thecontracted perimeter, which shortens the time finding the optimum, regardlesswhere the starting point is. Fig. 7 has validated the explanation. As ̟ increases,CMA-ES and PSO both present better performance. The shape of local valley in the ill conditional landscape can clearly impact theoptimizer performance. After bent using conformal transformation, it createssubstantial barrier in the search process for both CMA-ES and PSO algorithms.The optimizers spend 100 times more search budget on solving it and evenstill fail to position the optimum for many trials, which is a quite rare case in the optimization of the rotated version of bent cigar function. CMA-ES presentshigher sensitivity on valley shape change caused by hyper parameters than PSO.In addition, the optimum position of the conformal bent cigar function canconsiderably influence both CMA-ES and PSO. When the optimum position isset around the thinner area, CMA-ES and PSO are hardly able to find it withthe preset number of fitness evaluations. We believe these results generalize toother ill conditional problems and other evolutionary algorithms.One of the most important implications from the results could be the ap-propriate use of restart strategy. In general, the current restart mechanism inCMA-ES is activated only when the search agent gets stuck somewhere or thefitness value has no improvement for some time. However, it does not workwell in the conformal bent cigar function since the fitness value is continuouslydecreasing and population mean is also slowly moving. Furthermore, the per-formance variation on the same function indicates the starting search point hasconsiderable influence on optimization of the function. In this case, it could bea better choice to restart the search rather than follow one way to the end. Butthen again, it is a tradeoff between keeping the current promising but slow trendand restarting a possible faster trial.Another consideration can be adopting more intelligent strategy to aid theoptimizer in recognizing the general environment. Obviously, the winding valleyis ring-shaped and has clear regularity. We can deduce that a certain additionalmachine learning could be helpful to sufficiently utilize the previous search ex-perience.
Acknowledgments
This work was supported by National Natural Science Foundation of China underGrant No. 61872419, No. 61573166, No. 61572230. Shandong Provincial KeyR&D Program under Grant No. 2019GGX101041. Taishan Scholars Program ofShandong Province, China, under Grant No. tsqn201812077.
References
1. Eberhart, R., Kennedy, J.: A new optimizer using particle swarm theory. In: MicroMachine and Human Science, 1995. MHS’95., Proceedings of the Sixth Interna-tional Symposium on. pp. 39–43. IEEE (1995)2. El-Abd, M., Kamel, M.S.: Black-box optimization benchmarking for noiseless func-tion testbed using particle swarm optimization. In: Proceedings of the 11th AnnualConference Companion on Genetic and Evolutionary Computation Conference:Late Breaking Papers. pp. 2269–2274. GECCO ’09, ACM (2009)3. Hansen, N., Auger, A., Finck, S., Ros, R.: Real-parameter black-box optimizationbenchmarking 2009: Experimental setup. Tech. Rep. RR-6828, INRIA (2009)4. Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolutionstrategies. Evolutionary Computation (2), 159–195 (2001)5. Hansen, N., Ros, R., Mauny, N., Schoenauer, M., Auger, A.: PSO facing non-separable and ill-conditioned problems. Tech. Rep. RR-6447, INRIA (2008) Novel Graphic Bending Transformation on Benchmark 116. Hansen, N., Ros, R., Mauny, N., Schoenauer, M., Auger, A.: Impacts of invari-ance in search: When CMA-ES and PSO face ill-conditioned and non-separableproblems. Applied Soft Computing (8), 5755–5769 (2011)7. Hariya, Y., Shindo, T., Jin’no, K.: A novel particle swarm optimization algorithmfor non-separable and ill-conditioned problems. In: 2016 IEEE International Con-ference on Systems, Man, and Cybernetics (SMC). pp. 2110–2115. IEEE (2016)8. Saarinen, S., Bramley, R., Cybenko, G.: Ill-conditioning in neural network trainingproblems. SIAM Journal on Scientific Computing (3), 693–714 (1993)9. Schinzinger, R., Laura, P.A.: Conformal mapping: methods and applications.Courier Corporation (2012)10. Sharon, E., Mumford, D.: 2d-shape analysis using conformal mapping. Interna-tional Journal of Computer Vision70