John J. Grefenstette

Optimization of Control Parameters for Genetic Algorithms

The task of optimizing a complex system presents at least two levels of problems for the system designer. First, a class of optimization algorithms must be chosen that is suitable for application to the system. Second, various parameters of the optimization algorithm need to be tuned for efficiency. A class of adaptive search procedures called genetic algorithms (GA) has been used to optimize a wide variety of complex systems. GAs are applied to the second level task of identifying efficient GAs for a set of numerical optimization problems. The results are validated on an image registration problem. GAs are shown to be effective for both levels of the systems optimization problem.

Credit assignment in rule discovery systems based on genetic algorithms

In rule discovery systems, learning often proceeds by first assessing the quality of the systems current rules and then modifying rules based on that assessment. This paper addresses the credit assignment problem that arises when long sequences of rules fire between successive external rewards. The focus is on the kinds of rule assessment schemes which have been proposed for rule discovery systems that use genetic algorithms as the primary rule modification strategy. Two distinct approaches to rule learning with genetic algorithms have been previously reported, each approach offering a useful solution to a different level of the credit assignment problem. We describe a system, called RUDI, that exploits both approaches. We present analytic and experimental results that support the hypothesis that multiple levels of credit assignment can improve the performance of rule learning systems based on genetic algorithms.

Genetic Algorithms in Noisy Environments

Genetic algorithms are adaptive search techniques which have been used to learn high-performance knowledge structures in reactive environments that provide information in the form of payoff. In general, payoff can be viewed as a noisy function of the structure being evaluated, and the learning task can be viewed as an optimization problem in a noisy environment. Previous studies have shown that genetic algorithms can perform effectively in the presence of noise. This work explores in detail the tradeoffs between the amount of effort spent on evaluating each structure and the number of structures evaluated during a given iteration of the genetic algorithm. Theoretical analysis shows that, in some cases, more efficient search results from less accurate evaluations. Further evidence is provided by a case study in which genetic algorithms are used to obtain good registrations of digital images.

Learning Sequential Decision Rules Using Simulation Models and Competition

The problem of learning decision rules for sequential tasks is addressed, focusing on the problem of learning tactical decision rules from a simple flight simulator. The learning method relies on the notion of competition and employs genetic algorithms to search the space of decision policies. Several experiments are presented that address issues arising from differences between the simulation model on which learning occurs and the target environment on which the decision rules are ultimately tested.

Evolutionary algorithms for reinforcement learning

There are two distinct approaches to solving reinforcement learning problems, namely, searching in value function space and searching in policy space. Temporal difference methods and evolutionary algorithms are well-known examples of these approaches. Kaelbling, Littman and Moore recently provided an informative survey of temporal difference methods. This article focuses on the application of evolutionary algorithms to the reinforcement learning problem, emphasizing alternative policy representations, credit assignment methods, and problem-specific genetic operators. Strengths and weaknesses of the evolutionary approach to reinforcement learning are presented, along with a survey of representative applications.

Contagious Diseases in the United States from 1888 to the Present

Using data from digitized weekly surveillance reports of notifiable diseases for U.S. cities and states for 1888 through 2011, the authors derived a quantitative history of disease reduction in the United States, focusing particularly on the effects of vaccination programs.

Simulating School Closure Strategies to Mitigate an Influenza Epidemic

BACKGROUND There remains substantial debate over the impact of school closure as a mitigation strategy during an influenza pandemic. The ongoing 2009 H1N1 influenza pandemic has provided an unparalleled opportunity to test interventions with the most up-to-date simulations. METHODS To assist the Allegheny County Health Department during the 2009 H1N1 influenza pandemic, the University of Pittsburgh Models of Infectious Disease Agents Study group employed an agent-based computer simulation model (ABM) of Allegheny County, Pennsylvania, to explore the effects of various school closure strategies on mitigating influenza epidemics of different reproductive rates (R0). RESULTS Entire school system closures were not more effective than individual school closures. Any type of school closure may need to be maintained throughout most of the epidemic (ie, at least 8 weeks) to have any significant effect on the overall serologic attack rate. In fact, relatively short school closures (ie, 2 weeks or less) may actually slightly increase the overall attack rate by returning susceptible students back into schools in the middle of the epidemic. Varying the illness threshold at which school closures are triggered did not seem to have substantial impact on the effectiveness of school closures, suggesting that short delays in closing schools should not cause concern. CONCLUSIONS School closures alone may not be able to quell an epidemic but, when maintained for at least 8 weeks, could delay the epidemic peak for up to a week, providing additional time to implement a second more effective intervention such as vaccination.

High-resolution haplotype block structure in the cattle genome

BackgroundThe Bovine HapMap Consortium has generated assay panels to genotype ~30,000 single nucleotide polymorphisms (SNPs) from 501 animals sampled from 19 worldwide taurine and indicine breeds, plus two outgroup species (Anoa and Water Buffalo). Within the larger set of SNPs we targeted 101 high density regions spanning up to 7.6 Mb with an average density of approximately one SNP per 4 kb, and characterized the linkage disequilibrium (LD) and haplotype block structure within individual breeds and groups of breeds in relation to their geographic origin and use.ResultsFrom the 101 targeted high-density regions on bovine chromosomes 6, 14, and 25, between 57 and 95% of the SNPs were informative in the individual breeds. The regions of high LD extend up to ~100 kb and the size of haplotype blocks ranges between 30 bases and 75 kb (10.3 kb average). On the scale from 1–100 kb the extent of LD and haplotype block structure in cattle has high similarity to humans. The estimation of effective population sizes over the previous 10,000 generations conforms to two main events in cattle history: the initiation of cattle domestication (~12,000 years ago), and the intensification of population isolation and current population bottleneck that breeds have experienced worldwide within the last ~700 years. Haplotype block density correlation, block boundary discordances, and haplotype sharing analyses were consistent in revealing unexpected similarities between some beef and dairy breeds, making them non-differentiable. Clustering techniques permitted grouping of breeds into different clades given their similarities and dissimilarities in genetic structure.ConclusionThis work presents the first high-resolution analysis of haplotype block structure in worldwide cattle samples. Several novel results were obtained. First, cattle and human share a high similarity in LD and haplotype block structure on the scale of 1–100 kb. Second, unexpected similarities in haplotype block structure between dairy and beef breeds make them non-differentiable. Finally, our findings suggest that ~30,000 uniformly distributed SNPs would be necessary to construct a complete genome LD map in Bos taurus breeds, and ~580,000 SNPs would be necessary to characterize the haplotype block structure across the complete cattle genome.

Evolvability in dynamic fitness landscapes: a genetic algorithm approach

Evolvability refers to the adaptation of a populations genetic operator set over time. In traditional genetic algorithms, the genetic operator set, consisting of mutation operators, crossover operators, and their associated rates, is usually fixed. We explore the effects of allowing these operators and rates to vary under the influence of selection. The paper focuses on the suitability of alternative mutation models in dynamic landscapes. The mutation models include both traditional models in which all members of the population are subject to the same level of mutation and models in which mutation rates are genetically controlled.

A computer simulation of vaccine prioritization, allocation, and rationing during the 2009 H1N1 influenza pandemic

In the fall 2009, the University of Pittsburgh Models of Infectious Disease Agent Study (MIDAS) team employed an agent-based computer simulation model (ABM) of the greater Washington, DC, metropolitan region to assist the Office of the Assistant Secretary of Public Preparedness and Response, Department of Health and Human Services, to address several key questions regarding vaccine allocation during the 2009 H1N1 influenza pandemic, including comparing a vaccinating children (i.e., highest transmitters)-first policy versus the Advisory Committee on Immunization Practices (ACIP)-recommended vaccinating at-risk individuals-first policy. Our study supported adherence to the ACIP (instead of a children-first policy) prioritization recommendations for the H1N1 influenza vaccine when vaccine is in limited supply and that within the ACIP groups, children should receive highest priority.