René Doursat

Neural networks and the bias/variance dilemma

Feedforward neural networks trained by error backpropagation are examples of nonparametric regression estimators. We present a tutorial on nonparametric inference and its relation to neural networks, and we use the statistical viewpoint to highlight strengths and weaknesses of neural models. We illustrate the main points with some recognition experiments involving artificial data as well as handwritten numerals. In way of conclusion, we suggest that current-generation feedforward neural networks are largely inadequate for difficult problems in machine perception and machine learning, regardless of parallel-versus-serial hardware or other implementation issues. Furthermore, we suggest that the fundamental challenges in neural modeling are about representation rather than learning per se. This last point is supported by additional experiments with handwritten numerals.

Organically Grown Architectures: Creating Decentralized, Autonomous Systems by Embryomorphic Engineering

Exploding growth growth in computational systems forces us to gradually replace rigid design and control with decentralization and autonomy. Information technologies will progress, instead, by“meta-designing” mechanisms of system self-assembly, self-regulation and evolution. Nature offers a great variety of efficient complex systems, in which numerous small elements form large-scale, adaptive patterns. The new engineering challenge is to recreate this self-organization and let it freely generate innovative designs under guidance. This article presents an original model of artificial system growth inspired by embryogenesis. A virtual organism is a lattice of cells that proliferate, migrate and self-pattern into differentiated domains. Each cell’s fate is controlled by an internal gene regulatory network network. Embryomorphic engineering emphasizes hyperdistributed architectures, and their development as a prerequisite of evolutionary design.

Isolation-by-Distance and Outbreeding Depression Are Sufficient to Drive Parapatric Speciation in the Absence of Environmental Influences

A commonly held view in evolutionary biology is that speciation (the emergence of genetically distinct and reproductively incompatible subpopulations) is driven by external environmental constraints, such as localized barriers to dispersal or habitat-based variation in selection pressures. We have developed a spatially explicit model of a biological population to study the emergence of spatial and temporal patterns of genetic diversity in the absence of predetermined subpopulation boundaries. We propose a 2-D cellular automata model showing that an initially homogeneous population might spontaneously subdivide into reproductively incompatible species through sheer isolation-by-distance when the viability of offspring decreases as the genomes of parental gametes become increasingly different. This simple implementation of the Dobzhansky-Muller model provides the basis for assessing the process and completion of speciation, which is deemed to occur when there is complete postzygotic isolation between two subpopulations. The model shows an inherent tendency toward spatial self-organization, as has been the case with other spatially explicit models of evolution. A well-mixed version of the model exhibits a relatively stable and unimodal distribution of genetic differences as has been shown with previous models. A much more interesting pattern of temporal waves, however, emerges when the dispersal of individuals is limited to short distances. Each wave represents a subset of comparisons between members of emergent subpopulations diverging from one another, and a subset of these divergences proceeds to the point of speciation. The long-term persistence of diverging subpopulations is the essence of speciation in biological populations, so the rhythmic diversity waves that we have observed suggest an inherent disposition for a population experiencing isolation-by-distance to generate new species.

A workflow to process 3D+time microscopy images of developing organisms and reconstruct their cell lineage.

The quantitative and systematic analysis of embryonic cell dynamics from in vivo 3D+time image data sets is a major challenge at the forefront of developmental biology. Despite recent breakthroughs in the microscopy imaging of living systems, producing an accurate cell lineage tree for any developing organism remains a difficult task. We present here the BioEmergences workflow integrating all reconstruction steps from image acquisition and processing to the interactive visualization of reconstructed data. Original mathematical methods and algorithms underlie image filtering, nucleus centre detection, nucleus and membrane segmentation, and cell tracking. They are demonstrated on zebrafish, ascidian and sea urchin embryos with stained nuclei and membranes. Subsequent validation and annotations are carried out using Mov-IT, a custom-made graphical interface. Compared with eight other software tools, our workflow achieved the best lineage score. Delivered in standalone or web service mode, BioEmergences and Mov-IT offer a unique set of tools for in silico experimental embryology.

Embryomorphic Engineering: Emergent innovation through evolutionary development

Embryomorphic Engineering, a particular instance of Morphogenetic Engineering, takes its inspiration directly from biological development to create new robotic, software or network architectures by decentralized self-assembly of elementary agents. At its core, it combines three key principles of multicellular embryogenesis: chemical gradient diffusion (providing positional information to the agents), gene regulatory networks (triggering their differentiation into types, thus patterning), and cell division or aggregation (creating structural constraints, thus reshaping). This chapter illustrates the potential of Embryomorphic Engineering in different spaces: 2D/3D physical swarms, which can find applications in collective robotics, synthetic biology or nanotechnology; and \(n\)D graph topologies, which can find applications in distributed software and peer-to-peer techno-social networks. In all cases, the specific genotype shared by all the agents makes the phenotype’s complex architecture and function modular, programmable and reproducible.

A digital framework to build, visualize and analyze a gene expression atlas with cellular resolution in zebrafish early embryogenesis.

A gene expression atlas is an essential resource to quantify and understand the multiscale processes of embryogenesis in time and space. The automated reconstruction of a prototypic 4D atlas for vertebrate early embryos, using multicolor fluorescence in situ hybridization with nuclear counterstain, requires dedicated computational strategies. To this goal, we designed an original methodological framework implemented in a software tool called Match-IT. With only minimal human supervision, our system is able to gather gene expression patterns observed in different analyzed embryos with phenotypic variability and map them onto a series of common 3D templates over time, creating a 4D atlas. This framework was used to construct an atlas composed of 6 gene expression templates from a cohort of zebrafish early embryos spanning 6 developmental stages from 4 to 6.3 hpf (hours post fertilization). They included 53 specimens, 181,415 detected cell nuclei and the segmentation of 98 gene expression patterns observed in 3D for 9 different genes. In addition, an interactive visualization software, Atlas-IT, was developed to inspect, supervise and analyze the atlas. Match-IT and Atlas-IT, including user manuals, representative datasets and video tutorials, are publicly and freely available online. We also propose computational methods and tools for the quantitative assessment of the gene expression templates at the cellular scale, with the identification, visualization and analysis of coexpression patterns, synexpression groups and their dynamics through developmental stages.

Morphogenetic Engineering: Reconciling Self-Organization and Architecture

Generally, phenomena of spontaneous pattern formation are random and repetitive, whereas elaborate devices are the deterministic product of human design. Yet, biological organisms and collective insect constructions are exceptional examples of complex systems that are both architectured and self-organized. Can we understand their precise self-formation capabilities and integrate them with technological planning? Can physical systems be endowed with information, or informational systems be embedded in physics, to create autonomous morphologies and functions? This book is the first initiative of its kind toward establishing a new field of research, Morphogenetic Engineering, to explore the modeling and implementation of “self-architecturing” systems. Particular emphasis is set on the programmability and computational abilities of self-organization, properties that are often underappreciated in complex systems science—while, conversely, the benefits of self-organization are often underappreciated in engineering methodologies.

Defining and simulating open-ended novelty: requirements, guidelines, and challenges

The open-endedness of a system is often defined as a continual production of novelty. Here we pin down this concept more fully by defining several types of novelty that a system may exhibit, classified as variation, innovation, and emergence. We then provide a meta-model for including levels of structure in a system’s model. From there, we define an architecture suitable for building simulations of open-ended novelty-generating systems and discuss how previously proposed systems fit into this framework. We discuss the design principles applicable to those systems and close with some challenges for the community.

Brainless Bodies: Controlling the Development and Behavior of Multicellular Animats by Gene Regulation and Diffusive Signals

We present a model of parallel co-evolution of development and motion control in soft-bodied, multicellular animats without neural networks. Development is guided by an artificial gene regulatory network (GRN), with real-valued expression levels, contained in every cell. Embryos develop within a simulated physics environment and are converted into animat structures by connecting neighboring cells through elastic springs. Outer cells, which form the external envelope, are affected by drag forces in a fluid-like environment. Both the developmental program and locomotion controller are encoded into a single genomic sequence, which consists of regulatory regions and genes expressed into transcription factors and morphogens. We apply a genetic algorithm to evolve individuals able to swim in the simulated fluid, where the fitness depends on distance traveled during the evaluation phase. We obtain various emergent morphologies and types of locomotion, some of them showing the use of rudimentary appendages. An analysis of the selected evolved controllers is provided.

A shape-recognition model using dynamical links

A shape-recognition method is proposed, inspired from the dynamic-link theory of von der Malsburg (1981). The quality of a match between two images is assessed through an elastic cost functional; the minimal value reached by the cost over a suitably-defined space of maps is viewed as a distance between these two images. Experiments on nearest-neighbour classification of handwritten numerals are presented, using a computationally effective procedure for finding a reliable estimate of the matching distance.