Daniel Sheldon

LambdaMerge: merging the results of query reformulations

Search engines can automatically reformulate user queries in a variety of ways, often leading to multiple queries that are candidates to replace the original. However, selecting a replacement can be risky: a reformulation may be more effective than the original or significantly worse, depending on the nature of the query, the source of reformulation candidates, and the corpus. In this paper, we explore methods to mitigate this risk by issuing several versions of the query (including the original) and merging their results. We focus on reformulations generated by random walks on the click graph, a method that can produce very good reformulations but is also variable and prone to topic drift. Our primary contribution is λ-Merge, a supervised merging method that is trained to directly optimize a retrieval metric (such as NDCG or MAP) using features that describe both the reformulations and the documents they return. In experiments on Bing data and GOV2, λ-Merge outperforms the original query and several unsupervised merging methods. λ-Merge also outperforms a supervised method to predict and select the best single formulation, and is competitive with an oracle that always selects the best formulation.

Manipulation-resistant reputations using hitting time

Popular reputation systems for linked networks can be manipulated by spammers who strategically place links. The reputation of node v is interpreted as the worlds opinion of vs importance. In PageRank [4], vs own opinion can be seen to have considerable influence on her reputation, where v expresses a high opinion of herself by participating in short directed cycles. In contrast, we show that expected hitting time -- the time to reach v in a random walk -- measures essentially the same quantity as PageRank, but excludes vs opinion. We make these notions precise, and show that a reputation system based on hitting time resists tampering by individuals or groups who strategically place outlinks. We also present an algorithm to efficiently compute hitting time for all nodes in a massive graph; conventional algorithms do not scale adequately.

A hidden Markov model for reconstructing animal paths from solar geolocation loggers using templates for light intensity

BackgroundSolar archival tags (henceforth called geolocators) are tracking devices deployed on animals to reconstruct their long-distance movements on the basis of locations inferred post hoc with reference to the geographical and seasonal variations in the timing and speeds of sunrise and sunset. The increased use of geolocators has created a need for analytical tools to produce accurate and objective estimates of migration routes that are explicit in their uncertainty about the position estimates.ResultsWe developed a hidden Markov chain model for the analysis of geolocator data. This model estimates tracks for animals with complex migratory behaviour by combining: (1) a shading-insensitive, template-fit physical model, (2) an uncorrelated random walk movement model that includes migratory and sedentary behavioural states, and (3) spatially explicit behavioural masks.The model is implemented in a specially developed open source R package FLightR. We used the particle filter (PF) algorithm to provide relatively fast model posterior computation. We illustrate our modelling approach with analysis of simulated data for stationary tags and of real tracks of both a tree swallow Tachycineta bicolor migrating along the east and a golden-crowned sparrow Zonotrichia atricapilla migrating along the west coast of North America.ConclusionsWe provide a model that increases accuracy in analyses of noisy data and movements of animals with complicated migration behaviour. It provides posterior distributions for the positions of animals, their behavioural states (e.g., migrating or sedentary), and distance and direction of movement.Our approach allows biologists to estimate locations of animals with complex migratory behaviour based on raw light data. This model advances the current methods for estimating migration tracks from solar geolocation, and will benefit a fast-growing number of tracking studies with this technology.

Migration timing and its determinants for nocturnal migratory birds during autumn migration

1. Migration is a common strategy used by birds that breed in seasonal environments, and multiple environmental and biological factors determine the timing of migration. How these factors operate in combination during autumn migration, which is considered to be under weaker time constraints relative to spring migration, is not clear. 2. Here, we examine the patterns and determinants of migration timing for nocturnal migrants during autumn migration in the north-eastern USA using nocturnal reflectivity data from 12 weather surveillance radar stations and modelled diurnal probability of occurrence for 142 species of nocturnal migrants. We first model the capacity of seasonal atmospheric conditions (wind and precipitation) and ecological productivity (vegetation greenness) to predict autumn migration intensity. We then test predictions, formulated under optimal migration theory, on how migration timing should be related to assemblage-level estimates of body size and total migration distance within the context of dietary guild (insectivore and omnivore) and level of dietary plasticity during autumn migration. 3. Our results indicate seasonal declines in ecological productivity delineate the beginning and end of peak migration, whose intensity is best predicted by the velocity of winds at migration altitudes. Insectivorous migrants departed earlier in the season and, consistent with our predictions, large-bodied and long-distance insectivorous migrants departed the earliest. Contrary to our predictions, large-bodied and some long-distance omnivorous migrants departed later in the season, patterns that were replicated in part by insectivorous migrants that displayed dietary plasticity during autumn migration. 4. Our findings indicate migration timing in the region is dictated by optimality strategies, modified based on the breadth and flexibility of migrants foraging diets, with declining ecological productivity defining possible resource thresholds during which migration occurs when winds at migration altitudes are mild. These observations provide the basis to assess how avian migration strategies may be affected by adjustments in seasonal patterns of atmospheric circulation and ecological productivity that may occur under global climate change.

Autumn morning flights of migrant songbirds in the northeastern United States are linked to nocturnal migration and winds aloft

ABSTRACT Many passerines that typically migrate at night also engage in migratory flights just after sunrise. These widely observed “morning flights” often involve birds flying in directions other than those aimed toward their ultimate destinations, especially in coastal areas. Morning flights have received little formal investigation, and their study may improve our understanding of how birds orient themselves during and after nocturnal movements and how they use stopover habitat. We studied autumn morning flights in the northeastern United States to identify associations between the number of birds undertaking morning flights and the magnitude of nocturnal migratory movements, nocturnal winds, and local topography. Our analyses included observations of more than 15,000 passerines at 7 locations. We found positive relationships between morning flight size and nocturnal migration density and winds aloft: Significantly more birds flew following larger nocturnal movements, quantified from weather surveillance radar and recordings of nocturnal flight calls, and after stronger nocturnal crosswinds. We also found consistent differences in morning flight size and direction among sites. These patterns are consistent with migrants engaging in morning flight as a corrective measure following displacement by nocturnal winds and to search for suitable stopover habitat.

GREEN'S FUNCTIONS ON FRACTALS

For a regular harmonic structure on a post-critically finite (p.c.f.) self-similar fractal, the Dirichlet problem for the Laplacian can be solved by integrating against an explicitly given Greens function. We give a recursive formula for computing the values of the Greens function near the diagonal, and use it to give sharp estimates for the decay of the Greens function near the boundary. We present data from computer experiments searching for the absolute maximum of the Greens function for two different examples, and we formulate two radically different conjectures for where the maximum occurs. We also investigate a local Greens function that can be used to solve an initial value problem for the Laplacian, giving an explicit formula for the case of the Sierpinski gasket. The local Greens function turns out to be unbounded, and in fact not even integrable, but because of cancelation, it is still possible to form a singular integral to solve the initial value problem if the given function satisfies a Holder condition.

Seasonal changes in the altitudinal distribution of nocturnally migrating birds during autumn migration

Wind plays a significant role in the flight altitudes selected by nocturnally migrating birds. At mid-latitudes in the Northern Hemisphere, atmospheric conditions are dictated by the polar-front jet stream, whose amplitude increases in the autumn. One consequence for migratory birds is that the region’s prevailing westerly winds become progressively stronger at higher migration altitudes. We expect this seasonality in wind speed to result in migrants occupying progressively lower flight altitudes, which we test using density estimates of nocturnal migrants at 100 m altitudinal intervals from 12 weather surveillance radar stations located in the northeastern USA. Contrary to our expectations, median migration altitudes deviated little across the season, and the variance was lower during the middle of the season and higher during the beginning and especially the end of the season. Early-season migrants included small- to intermediate-sized long-distance migrants in the orders Charadriiformes and Passeriformes, and late-season migrants included large-bodied and intermediate-distance migrants in the order Anseriformes. Therefore, seasonality in the composition of migratory species, and related variation in migration strategies and behaviours, resulted in a convex–concave bounded distribution of migration altitudes. Our results provide a basis for assessing the implications for migratory bird populations of changes in mid-latitude atmospheric conditions probably occurring under global climate change.

An Optimization Framework for Merging Multiple Result Lists

Developing effective methods for fusing multiple ranked lists of documents is crucial to many applications. Federated web search, for instance, has become a common practice where a query is issued to different verticals and a single ranked list of blended results is created. While federated search is regarded as collection fusion, data fusion techniques aim at improving search coverage and precision by combining multiple search runs on a single document collection. In this paper, we study in depth and extend a neural network-based approach, LambdaMerge, for merging results of ranked lists drawn from one (i.e., data fusion) or more (i.e., collection fusion) verticals. The proposed model considers the impact of the quality of documents, ranked lists and verticals for producing the final merged result in an optimization framework. We further investigate the potential of incorporating deep structures into the model with an aim of determining better combinations of different evidence. In the experiments on collection fusion and data fusion, the proposed approach significantly outperforms several standard baselines and state-of-the-art learning-based approaches.

Innovative Visualizations Shed Light on Avian Nocturnal Migration.

Globally, billions of flying animals undergo seasonal migrations, many of which occur at night. The temporal and spatial scales at which migrations occur and our inability to directly observe these nocturnal movements makes monitoring and characterizing this critical period in migratory animals’ life cycles difficult. Remote sensing, therefore, has played an important role in our understanding of large-scale nocturnal bird migrations. Weather surveillance radar networks in Europe and North America have great potential for long-term low-cost monitoring of bird migration at scales that have previously been impossible to achieve. Such long-term monitoring, however, poses a number of challenges for the ornithological and ecological communities: how does one take advantage of this vast data resource, integrate information across multiple sensors and large spatial and temporal scales, and visually represent the data for interpretation and dissemination, considering the dynamic nature of migration? We assembled an interdisciplinary team of ecologists, meteorologists, computer scientists, and graphic designers to develop two different flow visualizations, which are interactive and open source, in order to create novel representations of broad-front nocturnal bird migration to address a primary impediment to long-term, large-scale nocturnal migration monitoring. We have applied these visualization techniques to mass bird migration events recorded by two different weather surveillance radar networks covering regions in Europe and North America. These applications show the flexibility and portability of such an approach. The visualizations provide an intuitive representation of the scale and dynamics of these complex systems, are easily accessible for a broad interest group, and are biologically insightful. Additionally, they facilitate fundamental ecological research, conservation, mitigation of human–wildlife conflicts, improvement of meteorological products, and public outreach, education, and engagement.

Hamming approximation of NP witnesses

Given an instance of an NP-hard problem, how hard is it to compute a (possibly infeasible) solution x, such that x is guaranteed to agree with some feasible solution x in at least half its bits? Such questions about “structural” approximability are motivated by applications such as Computer Tomography, in which one wants to reconstruct as much of the full structure of the solution as possible. In this spirit, Feige et al. [1] (following Kumar and Sivakumar [4]) show that, for some ǫ > 0, given an instance Ψ of 3-SAT, it is NP-hard to compute an assignment x that agrees with any satisfying assignment x of Ψ in at least n/2 + n of the n variables. They show similar negative results for other natural NP-complete problems. Guruswami and Rudra [2] strengthen their bounds to n/2 + n (for all fixed ǫ > 0). The main result in this paper is as follows. For the “universal” NP-complete language U , for any positive ǫ, it is NP-hard to compute an x that agrees with a witness x in at least n/2 − ǫ√n logn bits. In contrast to previous results, this is less than half the bits. This result extends to randomized algorithms, for which it is essentially tight. We also give improved negative results for several natural NP-complete problems, as well as the first positive (algorithmic) results for Vertex Cover, Independent Set, Clique, and U .