Perspective: Purposeful Failure in Artificial Life and Artificial Intelligence
PPerspective: Purposeful Failure in Artificial Life and Artificial Intelligence
Lana Sinapayen
Sony Computer Science Laboratories, Kyoto Laboratory, 13-1 Hontorocho, Shimogyo Ward, Kyoto 6008086 [email protected]
Abstract
Complex systems fail. I argue that failures can be a blueprintcharacterizing living organisms and biological intelligence,a control mechanism to increase complexity in evolutionarysimulations, and an alternative to classical fitness optimiza-tion. Imitating biological successes in Artificial Life and Arti-ficial Intelligence can be misleading; imitating failures offersa path towards understanding and emulating life it in artificialsystems.
Failure is Knowledge, Knowledge is Power
You are handed a mysterious box containing the most com-plex object in the universe, and must find how the objectworks. Where do you start?“The human brain is the most complex object in the uni-verse” is a well worn cliche (Constable (1918)). While theclaim might not be true, the human brain is definitely verycomplex. In neuroscience and psychology, one of the mostcompelling ways to understand how the brain works is tostudy how it fails. Brain damage, irrational decisions, sen-sory illusions: internal or external changes that make thebrain fail are how we find how the brain succeeds. Failure isused to understand complex systems beyond neuroscience:reverse-engineering computer software, understanding ani-mal behavior, identifying solid materials... Failure even de-fines Science itself. an hypothesis is considered scientificif and only if it is “falsifiable”: if it can reproducibly fail(Popper (1934)).Why default to observing failures when we don’t knowwhat is going on? Because the success-failure boundary isfull of information. Let me define “failure” in the context ofthis discussion. Imagine being an ant dropped somewhereon top of Fig.1-(a). What is the fastest way to map your sur-roundings? Rather than walking every inch of the surface,find boundaries. When you are investigating a complex sys-tem that is working as expected, you are an ant dropped onMount Success. To find the boundary, you have to push thesystem into failure mode. Staying inside the success spacecan inform you about the robustness of the system to pertur-bations (at best the system recovered from the perturbation, at worst your perturbation was irrelevant), but it is not ex-planatory. Neither is going from failure to failure. You canonly investigate causes and effects if your intervention ac-tually changes something: the failure boundary is not justmore informative, it is a different kind of information alto-gether.Boundaries and failures are not exactly the same. If youare observing a function of the system that does not changewhen it crosses the failure boundary, you will not notice thetransition. If you are observing the right function, you mightsee the system performance on that function become betteror worse. Let us call “failure points” abrupt transitions from“some performance” to 0: they are the most salient of transi-tions. Going back to Fig.1, the ant might not notice the tran-sition from a gentle slope to a flat terrain, but a cliff will benoticeable. Ideally, you would want to map the entire failureboundary; in practice, you will focus on failure points. ? Mount SuccessFailure Bog(a) (b)
Figure 1: (a) The fastest way to map this bog-and-mountterrain is to find the boundary and walk along it.
Simi-larly, a complex system can be characterized by its boundarybetween failure and success. Soft transitions can be hard tonotice, so we are better off finding a few remarkable cliffsoff Mount Success, abrupt “failure points” transitions fromsuccess to failure, and from there extrapolating to the wholefailure boundary. (b) The green system and the red systemshare many successes, but only 2 points of their failureboundary. a r X i v : . [ c s . A I] F e b ailure as a Fingerprint of Complex Systems The re-discovery of adversarial images (Szegedy et al.(2014), Fig.2) was met with dismay in the Deep Learningcommunity, who had welcomed claims of “super-human”performance at image recognition with comparatively littleskepticism (Stallkamp et al. (2012); Cires¸an et al. (2011)).Examining what the networks are doing “better” than hu-mans reveals that the comparison makes little sense: is ahuman wrong to label an image of 3 people in a golf cart as“people”, when the expected label is ”kid”? Is a picture of awall covered in pictures and papers best described by the of-ficial label ”study”? The artificial networks are better tunedto the dataset’s idiosyncracies than to the structure of realworld data (Beyer et al. (2020)). The failures themselves arenot what matters: the problem is the egregious difference be-tween the artificial and biological system’s failures, despiteinitially overlapping successes that prompted some singu-laritarians to claim that “the brain achieves deep learning”(The Kurzweil Library (2018)) and some AI researchers towonder whether deep learning can be used to model brainprocesses (Bengio et al. (2016)). Some versions of this po-sition might be true, but they are weakened by the fact thatthe failure boundaries of the two systems are so different.Yamins & DiCarlo (2016) propose deep hierarchical neu-ral networks as a path to “produce quantitatively accuratecomputational models of sensory systems”, yet also concedethat adversarial images could be the sign of “a fundamentalarchitectural flaw in [deep hierarchical neural networks] asbrain models”.Two systems can succeed at a the same task yet succeedin completely different ways: planes are good for flying, buttheir flight is very different from bird flight. Conversely, twosystems that fail differently at the same task are unlikely tobe working in the same way, as is now recognized by AIresearchers. For a given task, the failure boundary can beused as a fingerprint to identify systems that are genuinelysimilar in the way they perform (see Fig. 1-(b)), even if thetwo systems are made of different “stuff”. The fingerprint issubstrate-agnostic.Both engineering and natural sciences have embraced pur-poseful failure as a tool to deepen their understanding ofcomplex systems. ALife and AI, as synthetic approaches,have the opportunity to use failure for even more than under-standing existing biological systems: they can authenticatethe models they build against the failure boundary of a bio-logical target, to build (even better: evolve!) faithful modelsof those systems.AI in particular strongly rejects failure, overwhelmingly Examples found in ImageNet (Deng et al. (2009)). Imagescould not be added to this manuscript for reasons of copyright andconsent. Believers in the “singularity” claim that humans are about tofuse with a “superintelligence”; Ray Kurzweil is the most famoussingularity activist. ε * Figure 2:
Artificial Neural Networks’ mistakes look noth-ing like the mistakes of biological networks.
Non-randomnoise added to a picture of the author’s dog (confusinglynamed “Cookie”) fools the network into misclassifyingthe image. Examples generated from TensorFlow Authors(2019) based on Goodfellow et al. (2015)focusing on what it perceives as glorifiable biological suc-cesses, especially successes rooted in anthropodenial. “[in-sert task here] is what distinguishes Man from the animal!”...and the bulk of AI work to focus on chess, or natural lan-guage processing, or fuzzy logic. Beyond human exception-alism, another powerful drive in AI is the reality of practicalapplications, where failure is very much undesirable . Thisresults in an unbalanced availability of funding for shortterm applications, overshadowing theoretical work.Recent work on visual illusions, including my own, pro-vides examples going in the opposite, “failure fingerprint”direction. A model of network trained to predict videoframes was found to be sensitive to the same visual illusionsthat trick humans, while still treating regular images as non-illusions (Watanabe et al. (2018)). Illusions are a fascinatingexample of sensory failure; the fact that an artificial systemcan be fooled the same way, and even create its own illu-sions that fool humans (Sinapayen (2021)), is a good exam-ple of shared failure point. Despite the black-box nature ofthe networks, this research tells us that interactions betweenour visual environment and our prediction abilities could bethe root cause of visual illusions.While AI is at the moment dominated by engineering suc-cesses, for better or for worse such is not the case for AL- For example, the imitation of various types of bigotry by imageor text processing networks is both undesirable and disappointing;it is also an unsurprising result, and therefore of low informativevalue.
Failure for complexity
Stanley et al. (2017) called Open Endedness “the last grandchallenge you’ve never heard of”. Open Ended Evolutionis one of ALife’s greatest ambitions. Contemporary OpenEnded Evolution research focuses on obtaining the contin-ual production of “novelty” in artificial worlds: it is some-times described as nontrivial, ever-increasing complexity.Desirable models are usually defined by lists of expectedsuccesses. Is it possible to replace them with lists of ex-pected boundaries from success to failure? In that regard,Open Ended Evolution poses a challenge, as its definition isa list of boundless successes: boundless complexity, bound-less novelty, boundless emergence...Emergence, the process by which a group of units be-comes more than the sum of its parts, is believed to beone of the mechanisms by which the nontrivial complex-ity of Open Endedness might be attained (Banzhaf et al.(2016); Stepney & Beslon (2016) ). When the “parts” areliving organisms such as swarms of birds, coral reefs, orsocial networks, repressing egoistical behavior (even to thepoint of giving up self reproduction), is a prerequisite forsuccessful cooperation (Szathm´ary (2015); Michod & Roze(2001)). For cooperation to persist, the group must havecontrol mechanisms over egoistical individuals, imposing acost on cheaters greater than the potential rewards. In otherwords, cooperation (and with it, emergence) requires theability for a group to induce failure in any individual be-longing to the group. This could be the ultimate exampleof top-down causation (Ellis (2008); Campbell (1974)): aterminal control mechanism from an emergent scale over itsindividual components .Take death, the “failure of life”. In multicellular organ-isms, close cooperation between cells is occasionally brokenby rogue mutant cells, that reproduce too much and refuseto die (Z¨ornig et al. (2001)): cancer cells. Apoptosis (alsocalled programmed cell death or cellular suicide), especiallywhen provoked by external signals, is a mechanism for theorganism to control cells that are abnormal or no longerneeded (Carson & Ribeiro (1993)). Werfel et al. (2015) set out to find whether death could offer evolutionary benefits,and therefore evolve in a population of originally immortalorganisms. Their conclusion was that death is not workingagainst living systems, but offers a way to preserve ecosys-tems’ long term survival through environmental feedback,preventing over-exploitation of the resources of the environ-ment. The ecosystem comprised of a spatial environmentand its resident species self-regulates through death.Over different spatiotemporal scales, failure offers a con-trol mechanism that mediates what Szathm´ary (2015) called“major transitions”, of which they say that “evolution atthe lower level must be somehow constrained by the higherlevel”. Failure boundaries, and in particular the boundarybetween life and death, can be engineered as control mech-anisms. The more tightly knit a group becomes, the moreevolutionary selection changes from being applied to indi-viduals to being applied to the group, and the more inter-nal control is favored. Cell destroys own proteins, organismkills own cells, group exiles organism, ecosystem destroysspecies. Open Ended Evolution’s endless creativity requiresprominent and accessible kill switches at every level, so thatevery unit of selection has a the power to timely get rid of itscomponents.Beyond emulating the failure fingerprint of a simple lifeform, making it go through evolutionary transitions will re-quire keeping its failure points accessible to others agents inthe environment.
Failure and Self-Directed Evolution “We need to invoke the capacity of organisms to pursuegoals in order to explain the origin of adaptive novelties”,said Walsh (2015), referring to the fact that a mutation needsto match the goals of an organism to be exploited by thatorganism. We can go further: failed behavior selects, be-fore they appear, which mutations will be maintained in apopulation. It does more than just weeding out deleteriousmutations after the fact.Imagine a species of flies entirely guided by innate behav-ior in an environment with both food and poison. Throughevolution, the species explores its behavioral space, record-ing what to eat and what to avoid as instinct in its DNA.Having sorted everything as either food or poison, it reachesthe limits of successful behavior. Any deviation leads to fail-ure. Now sometime along its evolution, a mutation arisesthat makes one fly able to digest one type of red poisonas harmless food. Unfortunately, if no fly ever tries to eatthe red crumb, the new mutation might well disappear fromthe population without ever being used. A way around thismissed opportunity is to keep a degree of adventurousnessand failure in the behavior of individuals. If 1% of all fliessometimes decide to eat the red poison (colorblindness? cu-riosity?), when the useful mutation appears it has a higherchance to spread in the population. Without the until-thenfailing behavior, that mutation was effectively neutral. To3ffer a benefit, it would have to appear right at the same timeas a behavioral eat-red-poison mutation, or spread neutrallywaiting for the right behavior.So far it is simply a problem of evolutionary explorationversus exploitation. Now if from the beginning, before themutation, a great percentage of flies eat red poison, behaviormaintained maybe because the poison’s effects are mild, orit is the same shade of red as a red food, the mutation willspread faster when it arises, and the flies will have effectivelyforced the direction of their own evolution.Are random mutations more likely to spread in a popula-tion, or are mutations more likely to be selected in advanceby the behavior of the species? Take another example: an-tibiotic resistance in bacteria. Some bacteria produce an-tibiotics to protect their resources from competing species.Theoretically, these competitors can mutate and acquire theability to survive antibiotics. Yet widespread resistance toantibiotics is a new phenomenon, tied to humans (over)useof antibiotics. In a paper accompanied by a striking video,Baym et al. (2016) showed how highly resistant mutant bac-teria lineages can die out while to less resistant lineages pro-liferate. The originally less resistant bacteria reached theentry of the antibiotic-enhanced space first, and got the op-portunity to fail and try until they found the right mutationto cross over; the more resistant bacteria did not reach themore toxic environment fast enough to exploit their advan-tage. “evolution is not always led by the most resistant mu-tants.” (Baym et al. (2016) ) We are in the era of prolifera-tion of resistant bacteria because resistance genes now con-fer a critical advantage in a human environment. How oftendo those “reasons for mutations to stick around” originatein the species behavior itself rather than in extrinsic factors?Even with our use of antibiotics, bacteria would not exploittheir resistance mutations if they did not “try” to infect hu-mans.In terms of learning, i.e. optimisation at the lifetime scalerather than the at evolutionary scale, failure is a good indi-cator of whether a system is still learning or whether if it hassettled inside its success space and risks missing learningopportunities. A good example of using the failure bound-ary to tailor learning opportunities is the highly successfulGenerative Adversarial Networks (GANs, Goodfellow et al.(2014)). GANs are composed of two networks that learn to-gether by alternatively trying to beat each other. A generatornetwork creates data, trying to pass this data as being part ofan initial real dataset. A discriminator network tries to de-termine which data belongs to the initial dataset and whichdata was created by the generator. When training goes well,the discriminator and generator improve together: as the dis-criminator becomes better at telling fake from genuine, thegenerator becomes better at creating convincing fakes. Thethreshold for failure is effectively negotiated between thenetworks. From this point of view, GANs are a form of cur-riculum learning, adapting the difficulty of the task to the ability of the student.This suggests several possible avenues for AI and SoftALife. For example, how about optimizing for performancevariance in novelty search algorithms? Quality-diversity al-gorithms, used to find a variety of high performance waysto solve a task, require a measure of performance ) and ameasure of diversity. It is notoriously difficult to define “di-versity”. Performance variance could be used as a fitnessfunction to sidestep the need for a definition. Performancedifferences reflect a diversity of ways to fail, i.e. the ac-tual consequences of different behaviors rather than the ob-jective difference between behaviors. Optimizing for per-formance variance would means optimizing for more di-verse behavioral consequences, including not failing at all(as in classical performance optimization) and the opposite,low-performance behaviors, that might become useful if theagent or the environment change. Keeping as many values ofthese measures as possible in the population helps it stretchright to its failure boundary. Conclusion
Focusing on imitating successes has the advantage ofleaving room for many degrees and ways of succeeding.It also means that imitating a system’s successes cannotalways tell us how it works. Can focusing on failures lead tomore faithful models without sacrificing general propertiesof biological systems to idiosyncratic details? It is unlikelythat failure will become the new success, but I believe thatit can at least open new directions of fundamental research,and even lead to improvement in mainstream applications.You open the mysterious box, take the object out, andstart removing pieces until something breaks.
References
Banzhaf, W., Baumgaertner, B., Beslon, G., Doursat, R., Fos-ter, J. A., McMullin, B., De Melo, V. V., Miconi, T., Spec-tor, L., Stepney, S., et al. (2016). Defining and simulat-ing open-ended novelty: requirements, guidelines, and chal-lenges.
Theory in Biosciences , (3), 131–161.Baym, M., Lieberman, T. D., Kelsic, E. D., Chait, R., Gross,R., Yelin, I., & Kishony, R. (2016). Spatiotemporal micro-bial evolution on antibiotic landscapes. Science , (6304),1147–1151.Bengio, Y., Lee, D.-H., Bornschein, J., Mesnard, T., & Lin, Z.(2016). Towards biologically plausible deep learning.Beyer, L., H´enaff, O. J., Kolesnikov, A., Zhai, X., & Oord, A.v. d. (2020). Are we done with imagenet? arXiv preprintarXiv:2006.07159 .Breuer, M., Earnest, T. M., Merryman, C., Wise, K. S., Sun, L.,Lynott, M. R., Hutchison, C. A., Smith, H. O., Lapek, J. D.,Gonzalez, D. J., et al. (2019). Essential metabolism for aminimal cell. Elife , , e36842. Note that fitness in evolutionary algorithms is used as a cause,e.g. the measure used to decide the number of offspring of an in-dividual, while in biology fitness is a consequence, e.g. the repro-ductive success of the individual. ampbell, D. T. (1974). ‘downward causation’in hierarchicallyorganised biological systems. In Studies in the Philosophyof Biology , (pp. 179–186). Springer.Carson, D. A., & Ribeiro, J. M. (1993). Apoptosis and disease.
TheLancet , (8855), 1251–1254.Cires¸an, D., Meier, U., Masci, J., & Schmidhuber, J. (2011). Acommittee of neural networks for traffic sign classification. In The 2011 international joint conference on neural networks ,(pp. 1918–1921). IEEE.Constable, F. C. (1918).
Myself and dreams . New York, Dodd,Mead. P. 86.Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L.(2009). Imagenet: A large-scale hierarchical image database.In , (pp. 248–255). Ieee.Ellis, G. F. (2008). On the nature of causation in complex systems.
Transactions of the Royal Society of South Africa , (1), 69–84.Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014).Generative adversarial networks.Goodfellow, I. J., Shlens, J., & Szegedy, C. (2015). Explaining andharnessing adversarial examples.Jean-Christophe, L., S´ebastien, R., & Palsson, B. O. (2019). Mini-mal cells, maximal knowledge. eLife , .Michod, R. E., & Roze, D. (2001). Cooperation and conflict in theevolution of multicellularity. Heredity , (1), 1–7.Popper, K. R. (1934). The Logic of Scientific Discovery . Hutchin-son.Sinapayen, L. (2021). Eigen: Evolutionary illusion generator.URL https://github.com/LanaSina/evolutionary_illusion_generator
Stallkamp, J., Schlipsing, M., Salmen, J., & Igel, C. (2012). Manvs. computer: Benchmarking machine learning algorithmsfor traffic sign recognition.
Neural networks , , 323–332.Stanley, K. O., Lehman, J., & Soros, L. (2017). Open-endedness:The last grand challenge you’ve never heard of.URL Stepney, S., & Beslon, G. (2016). Open-endedness: Definitionsand shortcuts. In .Szathm´ary, E. (2015). Toward major evolutionary transitions the-ory 2.0.
Proceedings of the National Academy of Sciences , (33), 10104–10111.Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D.,Goodfellow, I., & Fergus, R. (2014). Intriguing propertiesof neural networks.TensorFlow Authors (2019). Adversarial example using fgsm.URL https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/generative/adversarial_fgsm.ipynb The Kurzweil Library (2018). Do our brains use the same kind ofdeep-learning algorithms used in ai?URL
Walsh, D. M. (2015).
Organisms, agency, and evolution . Cam-bridge University Press.Watanabe, E., Kitaoka, A., Sakamoto, K., Yasugi, M., & Tanaka,K. (2018). Illusory motion reproduced by deep neural net-works trained for prediction.
Frontiers in psychology , , 345.Werfel, J., Ingber, D. E., & Bar-Yam, Y. (2015). Programed deathis favored by natural selection in spatial systems. Phys. Rev.Lett. , , 238103.URL https://link.aps.org/doi/10.1103/PhysRevLett.114.238103 Yamins, D. L., & DiCarlo, J. J. (2016). Using goal-driven deeplearning models to understand sensory cortex.
Nature neuro-science , (3), 356–365.Z¨ornig, M., Hueber, A.-O., Baum, W., & Evan, G. (2001). Apop-tosis regulators and their role in tumorigenesis. Biochimicaet Biophysica Acta (BBA)-Reviews on Cancer , (2), F1–F37.(2), F1–F37.