Bram Vanschoenwinkel

An Evolutionary Dynamical Analysis of Multi-Agent Learning in Iterated Games

In this paper, we investigate Reinforcement learning (RL) in multi-agent systems (MAS) from an evolutionary dynamical perspective. Typical for a MAS is that the environment is not stationary and the Markov property is not valid. This requires agents to be adaptive. RL is a natural approach to model the learning of individual agents. These Learning algorithms are however known to be sensitive to the correct choice of parameter settings for single agent systems. This issue is more prevalent in the MAS case due to the changing interactions amongst the agents. It is largely an open question for a developer of MAS of how to design the individual agents such that, through learning, the agents as a collective arrive at good solutions. We will show that modeling RL in MAS, by taking an evolutionary game theoretic point of view, is a new and potentially successful way to guide learning agents to the most suitable solution for their task at hand. We show how evolutionary dynamics (ED) from Evolutionary Game Theory can help the developer of a MAS in good choices of parameter settings of the used RL algorithms. The ED essentially predict the equilibriums outcomes of the MAS where the agents use individual RL algorithms. More specifically, we show how the ED predict the learning trajectories of Q-Learners for iterated games. Moreover, we apply our results to (an extension of) the COllective INtelligence framework (COIN). COIN is a proved engineering approach for learning of cooperative tasks in MASs. The utilities of the agents are re-engineered to contribute to the global utility. We show how the improved results for MAS RL in COIN, and a developed extension, are predicted by the ED.

Species sorting in space and time—the impact of disturbance regime on community assembly in a temporary pool metacommunity

Abstract Ecosystems generally are not stable over long periods of time and are subject to disturbances of different frequencies and intensities. As a result, natural communities usually are not in equilibrium, and temporal snapshots can show different stages of community assembly. Nonequilibrium is especially common in temporary aquatic systems where desiccation continuously resets the development of communities and hydroperiod is an important determinant of species richness and community structure. We investigated effects of disturbance regime on community assembly of aquatic invertebrates in a cluster of 36 temporary rock pools in central South Africa. Pools were assigned to 4 categories based on their long-term disturbance regime and were sampled 4 times during 1 inundation. Disturbance modulated the relative importance of deterministic and stochastic processes driving community assembly and reduced both α and β diversity. Premature desiccation truncated community development. Initial communities were dominated by resident large branchiopods (fairy shrimp and clam shrimp). Arrival of flying colonists, including many predators, caused a drastic change in community composition and initiated a new successional phase dominated by cladocerans. For residents, the link between local environment and community structure was strongest early during inundation and decreased during community assembly. For flying colonists, variation in arrival time and habitat selection led to a better match between communities and environment later during inundation. We conclude that the combination of colonization–extinction dynamics and niche partitioning in space and time determines metacommunity structure in this extreme type of aquatic habitat.

Disturbance regime alters the impact of dispersal on alpha and beta diversity in a natural metacommunity

Disturbance and dispersal are two fundamental ecological processes that shape diversity patterns, yet their interaction and the underlying mechanisms are still poorly understood, and evidence from natural systems is particularly lacking. Using an invertebrate rock pool metacommunity in South Africa as a natural model system, we studied potential interactive effects of disturbance regime and patch isolation on diversity patterns of species with contrasting dispersal modes (passive vs. active dispersal). Isolation and disturbance regime had negative synergistic effects on alpha diversity: both directly, by excluding late-successional species from isolated patches; and indirectly, by modulating establishment success of generalist predators in well-connected patches. Unimodal relationships between isolation and alpha diversity, as predicted by mass effects, were only detected for passive dispersers in frequently disturbed patches and not in active dispersers. For passive dispersers, indications for a positive effect of isolation and a negative effect of disturbance on beta diversity were found, presumably due to differences in deterministic succession and stochastic colonization-extinction dynamics among different patch types. Our findings illustrate that interactions between dispersal rates and disturbance regime are important when explaining species diversity patterns in metacommunities and support the idea that diversity in frequently disturbed habitats is more sensitive to effects of dispersal-based processes.

Hatching phenology, life history and egg bank size of fairy shrimp Branchipodopsis spp. (Branchiopoda, Crustacea) in relation to the ephemerality of their rock pool habitat

In temporary aquatic habitats, permanence and the severe disturbance associated with desiccation are strong selective agents expected to lead to differentiation in life history strategies in populations experiencing different disturbance regimes. Besides optimal timing of hatching of dormant life stages, maturation and reproduction, pool inhabitants also benefit from the acquisition of reliable cues for the quality of the ambient environment. We investigated whether hatching patterns, life history characteristics and egg bank size of Branchipodopsis fairy shrimp (Branchiopoda, Anostraca) inhabiting a cluster of temporary rock pools in South Africa reflect variation in habitat stability and hatching cues. Long-term hydrological variation was used to select pools along a gradient of habitat stability. Initial conductivity was a good indicator for the length of inundations. No hatching occurred under elevated conductivities, which may present a mechanism to avoid abortive hatching. Egg bank size was unaffected by habitat size or habitat stability but instead was related to cover by a protective sheet of dry aquatic vegetation, which presumably counteracts egg bank erosion by wind when pools are dry. Life history but not hatching phenology reflected some aspects of habitat stability. Fairy shrimp populations in ephemeral pools started reproduction earlier than populations in more stable habitats. Additional common garden or transplant experiments, however, will be required to assess the relative importance of environmental and genetic components in explaining the observed variation and acquire more insight into the trade-offs that lie at the base of the evolution of life history strategies along the pond permanence gradient.

Quantifying the Hydroregime of a Temporary Pool Habitat: A Modelling Approach for Ephemeral Rock Pools in SE Botswana

Ecological and evolutionary processes in temporary rock pools operate within constraints imposed by their hydrologic regimes. These shallow pools flood when seasonal rains accumulate on impermeable substrates. Despite the ecological importance of hydrologic conditions for these ecosystems, we typically lack tools and empirical data required to understand the implications of hydrologic variability and climate change for biotic populations and communities in these habitats. In this study, we developed a hydrologic model to simulate rock pool hydrologic regimes based on rainfall, evapotranspiration, and basin geometry. The model was used to investigate long-term patterns of seasonal and inter-annual variation in hydroregime. In addition, hydrologic conditions associated with potential climate change scenarios were simulated and evaluated with respect to the biological requirements of the anostracan Branchipodopsis wolfi. The model’s output for daily inundation matched with field observations with an overall accuracy of 85% and correctly estimated complete hydroperiods with an overall accuracy of 70%. Simulations indicate large variation in individual hydroperiods (76–115%) as well as in the number of hydroperiods per year (19–23%). Furthermore, this study suggests that climate change may significantly alter the rock pool hydroregime. These findings confirm the hydrologic sensitivity of these ephemeral habitats to precipitation patterns, and their potential sensitivity to future climate change. Modelling indicates that the suitability of average inundation conditions for B. wolfi deteriorates significantly under future climate predictions. High levels of spatial and temporal variation in hydrologic conditions are dominant features of these habitats and an essential consideration for understanding population and community-level ecological processes.

Toward a Global Phylogeny of the “Living Fossil" Crustacean Order of the Notostraca

Tadpole shrimp (Crustacea, Notostraca) are iconic inhabitants of temporary aquatic habitats worldwide. Often cited as prime examples of evolutionary stasis, surviving representatives closely resemble fossils older than 200 mya, suggestive of an ancient origin. Despite significant interest in the group as ‘living fossils’ the taxonomy of surviving taxa is still under debate and both the phylogenetic relationships among different lineages and the timing of diversification remain unclear. We constructed a molecular phylogeny of the Notostraca using model based phylogenetic methods. Our analyses supported the monophyly of the two genera Triops and Lepidurus, although for Triops support was weak. Results also revealed high levels of cryptic diversity as well as a peculiar biogeographic link between Australia and North America presumably mediated by historic long distance dispersal. We concluded that, although some present day tadpole shrimp species closely resemble fossil specimens as old as 250 mya, no molecular support was found for an ancient (pre) Mesozoic radiation. Instead, living tadpole shrimp are most likely the result of a relatively recent radiation in the Cenozoic era and close resemblances between recent and fossil taxa are probably the result of the highly conserved general morphology in this group and of homoplasy.

Anostracan monopolisation of early successional phases in temporary waters

The length of the hydrocycle restrains the development of communities in temporary aquatic habitats. Little information is available on the importance of biotic interactions in shaping the communities in these temporary habitats. A common observation in the freshwater rock pool community assembly process is the clearly separated density peaks over time of fairy shrimp (Anostraca) and cladocerans (Anomopoda). Fairy shrimp (Anostraca), the fl agship group and dominant fi lter feeders of temporary waters hatch early after inundation. This contrasts with the occurrence pattern of coexisting cladocerans, which despite higher chances of abortive hatching occur later during the hydroperiod. To study this apparent succession pattern and test the possible presence of a biotic interaction, we set up enclosure experiments in freshwater rock pools in Western Australia and monitored cladoceran (Macrothrix hardingii) densities in the presence of variable densities or absence of fairy shrimp (Branchinella longirostris). The anomopod cladoceran populations decreased with increasing anostracan densities revealing a negative biotic interaction. The competitive dominance of anostracans most probably was mainly due to exploitation competition, with possibly elements of interference competition or predation included. Early hatching and competitive superi- ority may help anostracans in temporary pools to monopolize the habitat with respect to small and/or other fi lter feeding invertebrates. As such they infl uence succession dynamics and invertebrate community structure, making anostracans keystone species of temporary pools.

Long Distance Dispersal of Zooplankton Endemic to Isolated Mountaintops - an Example of an Ecological Process Operating on an Evolutionary Time Scale

Recent findings suggest a convergence of time scales between ecological and evolutionary processes which is usually explained in terms of rapid micro evolution resulting in evolution on ecological time scales. A similar convergence, however, can also emerge when slow ecological processes take place on evolutionary time scales. A good example of such a slow ecological process is the colonization of remote aquatic habitats by passively dispersed zooplankton. Using variation at the protein coding mitochondrial COI gene, we investigated the balance between mutation and migration as drivers of genetic diversity in two Branchipodopsis fairy shrimp species (Crustacea, Anostraca) endemic to remote temporary rock pool clusters at the summit of isolated mountaintops in central South Africa. We showed that both species colonized the region almost simultaneously c. 0.8 My ago, but exhibit contrasting patterns of regional genetic diversity and demographic history. The haplotype network of the common B. cf. wolfi showed clear evidence of 11 long distance dispersal events (up to 140 km) with five haplotypes that are shared among distant inselbergs, as well as some more spatially isolated derivates. Similar patterns were not observed for B. drakensbergensis presumably since this rarer species experienced a genetic bottleneck. We conclude that the observed genetic patterns reflect rare historic colonization events rather than frequent ongoing gene flow. Moreover, the high regional haplotype diversity combined with a high degree of haplotype endemicity indicates that evolutionary- (mutation) and ecological (migration) processes in this system operate on similar time scales.

Mechanistic and evolutionary aspects of light-induced dormancy termination in a temporary pond crustacean

Abstract. Most freshwater zooplankton species produce dormant eggs or other long-lived resistant life stages to bridge periods that are unsuitable for growth and reproduction. One of the principal cues that determine activation of these dormant stages is light exposure. Contrary to the analogous process of seed germination in plants, relatively little is known about the mechanism of light activation and its potential adaptive value in freshwater zooplankton. We made use of a temporary pool model to investigate the hatching response of resting eggs of the fairy shrimp Branchipodopsis wolfi to a specific set of selected light regimes and to evaluate the importance of egg pigmentation for mediating susceptibility to light cues. Our results suggest the presence of a critical light threshold for activation leading to maximal hatching. Moreover, the process of light activation does not appear to be purely cumulative because light administered in a single continuous pulse resulted in much higher hatching than the same amount of light administered in several 2-h pulses. Last, darkly pigmented eggs were less sensitive than lightly pigmented eggs to light exposure, resulting in delayed hatching of darker eggs during an experimental inundation. This result indicates that natural variation in egg pigmentation could be a risk-spreading mechanism that trades off the risks of mortality caused by kin competition during the nutrient-limited conditions that typically prevail early during inundations on one hand and the risk of mortality from premature pond drying on the other hand. Overall, our results indicate that light-energy activation thresholds and variation in pigmentation of the eggshell can be important traits that may contribute to variation in hatching phenology among zooplankton lineages.

Mitochondrial gene trees support persistence of cold tolerant fairy shrimp throughout the Pleistocene glaciations in both southern and more northerly refugia

Fairy shrimp (Crustacea: Anostraca) are specialist inhabitants of temporary aquatic habitats. In many parts of the world and particularly in Western Europe, however, populations are declining while the development of adequate conservation strategies is impeded by a poor knowledge of the genetic structure and taxonomic status of remaining lineages. We reconstructed a phylogeography of the species Chirocephalus diaphanus Prévost, 1803 using partial sequences of the mitochondrial COI gene and discuss the importance of different Pleistocene refugia to explain current diversity patterns. In addition to 20 C. diaphanus populations, we also included populations of six presumably closely related chirocephalids to evaluate their taxonomic status. Based on molecular data, the Eastern European subspecies C. diaphanus romanicus deserves species status while the species status of two Italian chirocephalids, C. salinus and C. ruffoi is questionable. Results indicate European C. diaphanus lineages diverged well before the last glacial maximum and survived the Pleistocene glaciations in multiple (sub)refugia along the Iberian, Italian, and Balkan peninsula. Northern Europe was subsequently recolonized from Southern France, resulting in high levels of cryptic diversity around glacial refugia but also in more widespread haplotypes in mainland Europe.