Shahar Avin

The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation

The following organisations are named on the report: Future of Humanity Institute, University of Oxford, Centre for the Study of Existential Risk, University of Cambridge, Center for a New American Security, Electronic Frontier Foundation, OpenAI. The Future of Life Institute is acknowledged as a funder.

Beyond Brain Size

Despite prolonged interest in comparing brain size and behavioral proxies of ‘intelligence’ across taxa, the adaptive and cognitive significance of brain size variation remains elusive. Central to this problem is the continued focus on hominid cognition as a benchmark, and the assumption that behavioral complexity has a simple relationship with brain size. Although comparative studies of brain size have been criticized for not reflecting how evolution actually operates, and for producing spurious, inconsistent results, the causes of these limitations have received little discussion. We show how these issues arise from implicit assumptions about what brain size measures and how it correlates with behavioral and cognitive traits. We explore how inconsistencies can arise through heterogeneity in evolutionary trajectories and selection pressures on neuroanatomy or neurophysiology across taxa. We examine how interference from ecological and life history variables complicates interpretations of brain-behavior correlations, and point out how this problem is exacerbated by the limitations of brain and cognitive measures. These considerations, and the diversity of brain morphologies and behavioral capacities, suggest that comparative brain-behavior research can make greater progress by focusing on specific neuroanatomical and behavioral traits within relevant ecological and evolutionary contexts. We suggest that a synergistic combination of the ‘bottom up’ approach of classical neuroethology and the ‘top down’ approach of comparative biology/psychology within closely related but behaviorally diverse clades can limit the effects of heterogeneity, interference, and noise. We argue this shift away from broad-scale analyses of superficial phenotypes will provide deeper, more robust insights into brain evolution.

Funding Science by Lottery

Motivated by recent criticisms of the low reliability and high costs of science funding allocation by grant peer review, the paper investigates the alternative of funding science by lottery, and more generally the possible introduction of a formal random element in the funding process. At first it may seem that randomness will lower expected efficiency, by allocating funds to less meritorious projects. By focusing on the notion that we want funded research projects to ultimately make our lives better, and the observation that the causal effect of research projects is subject to change over time, the paper argues that the introduction of randomness can counteract a bias towards the familiar present in grant peer review, and thus increase the overall efficiency of science funding. The time-dependant nature of scientific merit is exemplified by the historical processes leading to the discovery of the structure of DNA. The argument regarding the relative effectiveness of random allocation is supported by a computer simulation of different funding mechanisms on a hypothetical dynamic epistemic landscape.

Breaking the grant cycle : on the rational allocation of public resources to scientific research projects

The thesis presents a reformative criticism of science funding by peer review. The criticism is based on epistemological scepticism, regarding the ability of scientific peers, or any other agent, to have access to sufficient information regarding the potential of proposed projects at the time of funding. The scepticism is based on the complexity of factors contributing to the merit of scientific projects, and the rate at which the parameters of this complex system change their values. By constructing models of different science funding mechanisms, a construction supported by historical evidence, computational simulations show that in a significant subset of cases it would be better to select research projects by a lottery mechanism than by selection based on peer review. This last result is used to create a template for an alternative funding mechanism that combines the merits of peer review with the benefits of random allocation, while noting that this alternative is not so far removed from current practice as may first appear.

Centralised Funding and Epistemic Exploration

Computer simulation of an epistemic landscape model, modified to include explicit representation of a centralised funding body, show the method of funding allocation has significant effects on communal trade-off between exploration and exploitation, with consequences for the communitys ability to generate significant truths. The results show this effect is contextual, and depends on the size of the landscape being explored, with funding that includes explicit random allocation performing significantly better than peer-review on large landscapes. The paper proposes a way of incorporating external institutional factors in formal social epistemology, and offers a way of bringing such investigations to bear on current research policy questions.

Mapping Intelligence: Requirements and Possibilities

New types of artificial intelligence (AI), from cognitive assistants to social robots, are challenging meaningful comparison with other kinds of intelligence. How can such intelligent systems be catalogued, evaluated, and contrasted, with representations and projections that offer meaningful insights? To catalyse the research in AI and the future of cognition, we present the motivation, requirements and possibilities for an atlas of intelligence: an integrated framework and collaborative open repository for collecting and exhibiting information of all kinds of intelligence, including humans, non-human animals, AI systems, hybrids and collectives thereof. After presenting this initiative, we review related efforts and present the requirements of such a framework. We survey existing visualisations and representations, and discuss which criteria of inclusion should be used to configure an atlas of intelligence.