Using the quaternion's representation of individuals in swarm intelligence and evolutionary computation
aa r X i v : . [ c s . N E ] J un Using the quaternion’s representation of individuals in swarmintelligence and evolutionary computation
Iztok Fister ∗ and Iztok Fister Jr. † Abstract
This paper introduces a novel idea for representation of individuals using quaternions in swarmintelligence and evolutionary algorithms. Quaternions are a number system, which extends com-plex numbers. They are successfully applied to problems of theoretical physics and to those areasneeding fast rotation calculations. We propose the application of quaternions in optimization,more precisely, we have been using quaternions for representation of individuals in Bat algorithm.The preliminary results of our experiments when optimizing a test-suite consisting of ten stan-dard functions showed that this new algorithm significantly improved the results of the originalBat algorithm. Moreover, the obtained results are comparable with other swarm intelligence andevolutionary algorithms, like the artificial bees colony, and differential evolution. We believe thatthis representation could also be successfully applied to other swarm intelligence and evolutionaryalgorithms.
To cite paper as follows: I. Fister, I. Fister Jr.: Using the quaternion’s representation of individ-uals in swarm intelligence and evolutionary computation,
Technical Report on Faculty of ElectricalEngineering and Computer Science , Maribor, Slovenia, 2013. ∗ University of Maribor, Faculty of Electrical Engineering and Computer Science Smetanova 17, 2000 Mari-bor; Electronic address: iztok.fi[email protected] † University of Maribor, Faculty of Electrical Engineering and Computer Science Smetanova 17, 2000 Mari-bor; Electronic address: iztok.fi[email protected] . INTRODUCTION Human desire has always been to build an automatic problem solver that would be ableto solve any kind of problems within mathematics, computer science and engineering. Inorder to fulfil this desire, humans have often resorted to solutions from Nature that areeternal sources of inspiration. Let us mention only two the more powerful nature-inspiredproblem solvers nowadays: • Evolutionary Algorithms (EA) inspired by the Darwinian evolutionary process, wherein nature the fittest individuals have the greater possibilities for survival and pass-ontheir characteristics to their offspring during a process of reproduction [1]. • Swarm Intelligence (SI) inspired by the collective behaviour within self-organized anddecentralized natural systems, e.g., ant colony, bees, flocks of birds or shoals of fish [2].Evolutionary computation (EC) is a contemporary term that denotes the whole field ofcomputing inspired by Darwinian principles of natural evolution. As a result, evolution-ary algorithms which are involved in this field are divided into the following disciplines:genetic algorithms (GA) [3, 4], evolution strategies (ES) [5], differential evolution (DE) [6–8], evolutionary programming (EP) [9], and genetic programming (GP) [10]. Although allthese disciplines were developed independently, they share similar characteristics when solv-ing problems. Evolutionary algorithms have been applied in wide-areas of optimization,modeling, and simulation.Swarm intelligence is an artificial intelligence (AI) discipline concerned with the design ofintelligent multi-agent systems that was inspired by the collective behaviour of social insectslike ants, termites, bees, and wasps, as well as from other animal societies like flocks of bird orshoals of fish [2]. Algorithms from this field have been applicable primarily for optimizationproblems, and the control of robots. The more notable swarm intelligence disciplines are asfollows: Ant Colony Optimization (ACO) [11, 12], Particle Swarm Optimization (PSO) [13],Artificial Bees Colony optimization (ABC) [14, 15], Firefly Algorithm (FA) [16], CuckooSearch (CS) [17], Bat Algorithm (BA) [18], etc.This paper focuses mainly on optimization problems. Typically, optimization algorithmsdo not search for solutions within the original problem context because these representthe problems on computers by means of data structures. For instance, genetic algorithms2se binary representation of individuals, evolution strategy and differential evolution realnumbers, genetic programming programs in Lisp, and evolutionary programming finite stateautomata. In general, algorithms in swarm intelligence represent solutions to problem asbinary or real-valued vectors.When solving the optimization problem using genetic algorithms, some mapping between problem context and binary represented problem-solving space needs to be performed. That isthat in the original problem context, the candidate solution determines so-named phenotypespace , i.e., set of points that form the space of possible solutions. On the other hand, thesolution in the problem-solving space determines the set of points that forms the so-named genotype space . By means of mapping from the genotype to the phenotype, the candidatesolution is encoded as a binary vector. In contrast, when the mapping from the phenotypeto the genotype is taken into consideration, the candidate solution is decoded from thebinary representation of individuals. Swarm intelligence and evolutionary algorithms holdthe representation of possible solutions within a population . The population is a set ofmultiple copies of candidate solutions.The variation operators are applied in order to create new individuals from old ones. Fromthe search perspective, the swarm intelligence and the evolutionary algorithms act accordingthe ’generate-and-test’ principle, where the variation operators perform the generate phase.However, variation operators operate differently in swarm intelligence and evolutionary al-gorithms. In general, algorithms in swarm intelligence support a move variation operatorthat obtains a new position for a individual from the old one in a specific manner. Forexample, in particle swarm optimization (PSO) [13] a new position for an individual de-pends on the global best and local best position of individuals within the swarm (i.e., inpopulation). Typically, evolutionary algorithms support two variation operators that mimicoperations in nature, i.e., mutation and crossover . Evolutionary algorithms simulate theprocess of natural selection using selection operator. Actually, two selection operators exist.The former selects the parents for the reproduction (also parent selection ), whilst the latterdetermines the surviving offspring (also survivor selection ).The representation plays an important role in the performance and solution quality ofthe swarm intelligence or evolutionary methods. Therefore, the tasks of designing suchprograms are to find a proper representation for the problem, and to develop appropriatesearch operators [19]. The proper representation needs to encode all possible solutions of3he optimization problem. On the other hand, the appropriate search operator should beapplicable to the proper representation. In general, no theoretical methods exist nowadaysfor describing the effects of representation on the performance. The proper representation ofa specific problem mainly depends on the intuition of the swarm intelligence or evolutionarydesigner. Therefore, developing a new representation is often a result of the repeated ’trial-and-error’ principle.This paper proposes a new representation for individuals using quaternions. In math-ematics, the quaternions extend complex numbers. Quaternion algebra is connected withspecial features of the geometry of the appropriate Euclidean spaces. The idea of quater-nions occurred to William Rowan Hamilton in 1845 [22] whilst he was walking along theRoyal Canal to a meeting of the Royal Irish Academy in Dublin. Interestingly, he carvedthe fundamental formula of quaternion algebra, that is: i = j = k = ijk = − , (1)into the stone of the Brougham Bridge.Quaternions are especially appropriate within those areas where it is necessary to com-pose rotations with minimal computation, e.g., programming the video games or controllersof spacecraft [23]. For instance, 3-dimensional rotation can be specified by a single quater-nion, whilst a pair of quaternions are needed for 4-dimensional rotation. The quaternioncalculus is introduced in several physical applications, like: crystallography, the kinematicsof rigid body motion, the Thomas precession, the special theory of relativity, and classicalelectromagnetism [21]. A step forward in the popularization of quaternions was achievedby Joachim Lambek in 1995 [20], who stated that quaternions can provide a shortcut forpure mathematicians who wish to familiarize themselves with certain aspects of theoreticalphysics.As far as we know, quaternions have never been used in optimization. Each optimiza-tion search process depends on balancing between two major components: exploration andexploitation [24]. Both terms were defined implicitly and are affected by the algorithms’control parameters. For optimization algorithms, the exploration denotes the process ofdiscovering diverse solutions within the search space, whilst exploitation means focusing thesearch process within the vicinities of the best solutions, thus, exploiting the information4iscovered so far. As a result, too much exploration can lead to inefficient search, whilsttoo much exploitation can cause the premature convergence of a search algorithm where thesearch process, usually due to reducing the population diversity, can be trapped into a localoptimum [26]. In place of premature convergence, a phenomenon of stagnation is typicalfor swarm intelligence, which can occur when the search algorithm cannot improve the bestperformance (also fitness) although the diversity is still high [25].In order to avoid the stagnation in swarm intelligence as well as the premature convergencein evolutionary algorithms, we propose the representation of individuals using quaternions.Rather than using the real-valued elements of individuals, the elements of the solution arerepresented by quaternions. In this manner, each an 1-dimensional element of solutionis placed into a 4-dimensional quaternion space, where the search process acts using theoperators of quaternion algebra. However, the quality of solution is evaluated in phenotypespace, therefore, each 4-dimensional quaternion is mapped back into a 1-dimensional elementusing the quaternion’s normalization function. During this mapping some information canbe lost, but on the other hand, it is expected that the fitness landscape as described byquaternions will replace the flat areas and plateaus of the original fitness landscape withpeaks and valleys that are more appropriated for exploration.Our intention in the future is to apply the quaternion’s representation to some swarmintelligence algorithms, like BA, FA and PSO, and some evolutionary algorithms, like DEand ES, in order to show that this representation could have beneficial effects on the swarmintelligence and evolutionary search processes, especially by avoiding the stagnation andpremature convergence. [1] Darwin, C., The origin of species.
John Murray, London, UK (1859).[2] Blum, C. and Li, X.,
Swarm Intelligence in Optimization.
In: C. Blum and D. Merkle (eds.)Swarm Intelligence: Introduction and Applications, pp. 43-86. Springer Verlag, Berlin (2008).[3] Goldberg, D.,
Genetic Algorithms in Search, Optimization, and Machine Learning.
Addison-Wesley, Massachusetts, US (1989).[4] Holland, J.H.,
Adaptation in Natural and Artificial Systems: An Introductory Analysis withApplications to Biology, Control and Artificial Intelligence.
MIT Press, Cambridge, MA, US Evolutionary algorithms in theory and practice - evolution strategies, evolutionaryprogramming, genetic algorithms.
Oxford University Press, Oxford, UK (1996).[6] Storn, R. and Price, K.,
Differential evolution–a simple and efficient heuristic for global opti-mization over continuous spaces.
Journal of global optimization, 4(11):341-359 (1997).[7] Brest, J. and Greiner, S. and Boˇskovi´c, B. and Mernik, M. and ˇZumer, V.,
Self-adapting controlparameters in differential evolution: A comparative study on numerical benchmark problems.
IEEE Transactions on Evolutionary Computation, 10(6):646-657 (2006).[8] Das, S. and Suganthan, P.N.,
Differential evolution: A survey of the state-of-the-art.
IEEETransactions on Evolutionary Computation, 15(1):4-31 (2011).[9] Fogel, L. J. and Owens, A. J. and Walsh, M. J.,
Artificial Intelligence through SimulatedEvolution.
John Wiley, New York, US (1966).[10] Koza, J.R.,
Genetic programming 2 - automatic discovery of reusable programs.
MIT Press,Massachusetts, US (1994).[11] Dorigo, M. and Di Caro, G,
The Ant Colony Optimization meta-heuristic.
In: D. Corne andM. Dorigo and F. Glover (eds.) New Ideas in Optimization, pp. 11-32. McGraw Hill, London,UK (1999).[12] Peter Koroˇsec and Jurij ˇSilc and Bogdan Filipiˇc,
The differential ant-stigmergy algorithm.
Information Sciences, 192(0):82-97 (2012).[13] Kennedy, J. and Eberhart, R.,
Particle swarm optimization.
In: Proceedings of IEEE Inter-national Conference on Neural Networks, pp. 1942-1948. (1995).[14] Karaboga, D. and Basturk, B.,
A powerful and efficient algorithm for numerical function op-timization: artificial bee colony (ABC) algorithm.
Journal of Global Optimization, 39(3):459-471 (2007).[15] Fister, I. and Fister, I.Jr. and Brest, J. and ˇZumer, V.,
Memetic artificial bee colony algo-rithm for large-scale global optimization.
In: Proceedings of IEEE Congress on EvolutionaryComputation, pp. 1-8. (2012).[16] Yang, X.S.,
Firefly algorithm.
Nature-Inspired Metaheuristic Algorithms, Wiley Online Li-brary, pp. 79-90 (2008).[17] Yang, X.S. and Deb, S.,
Cuckoo search via Levy flights.
In: World Congress on Nature &Biologically Inspired Computing (NaBIC 2009), pp. 210-214. (2009).
18] Yang, X.S.,
A New Metaheuristic Bat-Inspired Algorithm.
In: C. Cruz and J.R. Gonzlezand N. Krasnogor, D.A. Pelta, G. Terrazas (eds.) Nature Inspired Cooperative Strategies forOptimization (NISCO 2010), pp. 65-74. Springer Verlag, Berlin (2010).[19] Rothlauf, F.,
Representations for genetic and evolutionary algorithms.
Springer Verlag, Berlin(2006).[20] Lambek, J.,
If Hamilton had prevailed: quaternions in physics.
The Mathematical Intelli-gencer, 17(4):7-15 (1995).[21] Girard, P.R.,
The quaternion group and modern physics.
European Journal of Physics, Euro-pean Physical Society, Northern Ireland 5:25-32 (1984).[22] Hamilton, W.R.,
Elements of quaternions.
Longmans, Green and Co. (1899).[23] Conway, J.H. and Smith, D.A.,
On Quaternions and Octonions: Their Geometry, Arithmetic,and Symmetry.
A K Reters, Wellesley, Massachusetts (2003).[24] ˇCrepinˇsek, M. and Mernik, M. and Liu, S.,
Analysis of exploration and exploitations in evo-lutionary algorithms by ancestry trees.
International Journal of Innovative Computing andApplications, Inderscience 3(1):11-19 (2011).[25] Neri, F.,
Diversity Management in Memetic Algorithms.
In: F. Neri and C. Cotta and P.Moscato (eds.) Handbook of Memetic Algorithms, 379:153-165. Springer Verlag, Berlin (2012).[26] Eiben, A.E. and Smith, J.E.,
Introduction to Evolutionary Computing.