Nathan W. Fuller

A protocol and calibration method for accurate multi-camera field videography

Stereo videography is a powerful technique for quantifying the kinematics and behavior of animals, but it can be challenging to use in an outdoor field setting. We here present a workflow and associated software for performing calibration of cameras placed in a field setting and estimating the accuracy of the resulting stereoscopic reconstructions. We demonstrate the workflow through example stereoscopic reconstructions of bat and bird flight. We provide software tools for planning experiments and processing the resulting calibrations that other researchers may use to calibrate their own cameras. Our field protocol can be deployed in a single afternoon, requiring only short video clips of light, portable calibration objects.

A Thermal Infrared Video Benchmark for Visual Analysis

We hereby publish a new thermal infrared video benchmark, called TIV, for various visual analysis tasks, which include single object tracking in clutter, multi-object tracking in single or multiple views, analyzing motion patterns of large groups, and censusing wild animals in flight. Our data describe real world scenarios, such as bats emerging from their caves in large numbers, a crowded street view during a marathon competition, and students walking through an atrium during class break. We also introduce baseline methods and evaluation protocols for these tasks. Our TIV benchmark enriches and diversifies video data sets available to the research community with thermal infrared footage, which poses new and challenging video analysis problems. We hope the TIV benchmark will help the community to better understand these interesting problems, generate new ideas, and value it as a testbed to compare solutions.

Sebaceous Lipid Profiling of Bat Integumentary Tissues: Quantitative Analysis of Free Fatty Acids, Monoacylglycerides, Squalene, and Sterols

White‐nose syndrome (WNS) is a fungal disease caused by Pseudogymnoascus destructans and is devastating North American bat populations. Sebaceous lipids secreted from host integumentary tissues are implicated in the initial attachment and recognition of host tissues by pathogenic fungi. We are interested in determining if ratios of lipid classes in sebum can be used as biomarkers to diagnose severity of fungal infection in bats. To first establish lipid compositions in bats, we isolated secreted and integral lipid fractions from the hair and wing tissues of three species: big brown bats (Eptesicus fuscus), Eastern red bats (Lasiurus borealis), and evening bats (Nycticeius humeralis). Sterols, FFAs, MAGs, and squalene were derivatized as trimethylsilyl esters, separated by gas chromatography, and identified by mass spectrometry. Ratios of sterol to squalene in different tissues were determined, and cholesterol as a disease biomarker was assessed. Free sterol was the dominant lipid class of bat integument. Squalene/sterol ratio is highest in wing sebum. Secreted wing lipid contained higher proportions of saturated FFAs and MAGs than integral wing or secreted hair lipid. These compounds are targets for investigating responses of P. destructans to specific host lipid compounds and as biomarkers to diagnose WNS.

Study of bat flight behavior by combining thermal image analysis with a LiDAR forest reconstruction

The nature of forest structure plays an important role in the study of foraging behaviors of bats. In this study, we demonstrate a new combined methodology that uses both thermal imaging technology and a ground-based LiDAR system to record and reconstruct Eptesicus fuscus (big brown bats) flight trajectories in three-dimensional (3-D) space. The combination of the two 3-D datasets provided a fine-scale reconstruction of the flight characteristics adjacent to and within the forests. A 3-D forest reconstruction, assembled from nine Echidna Validation Instrument LiDAR scans over the 1 ha site area, provided the essential environmental variables for the study of bat foraging behaviors, such as the canopy height, terrain, location of the obstacles, and canopy openness at a bat roosting and maternity site in Petersham, Massachusetts. Flight trajectories of 24 bats were recorded over the 25 m × 37.5 m region within the LiDAR forest reconstruction area. The trajectories were reconstructed using imaging data from multiple FLIR ThermoVision SC8000 cameras and were co-registered to the 3-D forest reconstruction. Twenty-four of these flight trajectories were categorized into four different behavior groups according to velocity and altitude analysis of the flight trajectories. Initial results showed that although all bats were guided by echolocation and avoided hitting a tree that was in all of their flight paths, different bats chose different flight routes. This study is an initial demonstration of the power of coupling thermal image analysis and LiDAR forest reconstructions. Our goal was to break ground for future ecological studies, where more extensive flight trajectories of bats can be coupled with the canopy reconstructions to better establish responses of bats to different habitat characteristics and clutter, which includes both static (trees) and dynamic (other bats) obstacles.

Collision avoidance in biological systems using collision cones

The focus of the work in this paper is the comparison of a mathematical deconfliction algorithm to biological data in a range of species that demonstrate agile flight beyond the current capabilities of engineered systems. The algorithm was tailored to two coordinate systems, global and body relative, and two velocity changing criteria, constant and variable speed. Three species of animals were considered: fish, birds and bats. Overall, strong correlations were found between the data and the algorithm in two of the species with data indicating a bias toward a body-fixed coordinate system with variable speed maneuvering. Results also suggested future development of a fully three dimensional algorithm rather than the planar version considered here.

Stable isotope tracing of trout hatchery carbon to sediments and foodwebs of limestone spring creeks

Limestone springs support productive ecosystems and fisheries, yet aquaculture may modify or impair these ecosystems. We determined trout hatchery organic contribution to spring creek sediments and foodwebs with natural abundance stable isotope methods. Hatchery feed, waste, and trout were significantly enriched in delta(13)C relative to autotrophs and wild fish. Spring creek sediments were enriched in delta(13)C toward the hatchery endmember relative to reference streams without hatcheries and relative to a larger larger-order, spring-influenced stream. Contribution of hatchery C to spring creek sediments was greatest during March and associated with greatest sediment %C. Contribution of hatchery C to pollution-tolerant isopod diet was 39-51% in a stream receiving limestone spring water via hatchery effluent. Isopods of one spring creek also relied on hatchery-derived C within one month of hatchery closure. Four years later, less pollution pollution-tolerant amphipods dominated and consumed non-vascular over vascular autotrophs (86%). Isopods of a second spring creek with an active hatchery did not appear to be using hatchery matter directly, but were enriched in delta(34)S relative to a spring creek tributary with no hatchery influence. Isopods in both of these streams were relatively enriched in delta(15)N, indicating general nutrient enrichment from surrounding agricultural land use. The contribution of hatchery vs. wild fish in diet of herons and egrets was traced with delta(13)C of guano. These birds were strongly dependent on stocked trout in a spring creek with a recently closed state trout hatchery, and also near another large, state-run hatchery. Heron dependence on hatchery fish in the spring creek decreased with time since hatchery closure. Use of stable isotope natural abundance techniques in karst spring creeks can reveal stream impairment due to aquaculture, specific C sources to bio-indicating consumers, losses of farmed fish to predation, and potential exposure of higher order consumers to contaminants associated with aquaculture.

Perceptual modalities guiding bat flight in a native habitat

Flying animals accomplish high-speed navigation through fields of obstacles using a suite of sensory modalities that blend spatial memory with input from vision, tactile sensing, and, in the case of most bats and some other animals, echolocation. Although a good deal of previous research has been focused on the role of individual modes of sensing in animal locomotion, our understanding of sensory integration and the interplay among modalities is still meager. To understand how bats integrate sensory input from echolocation, vision, and spatial memory, we conducted an experiment in which bats flying in their natural habitat were challenged over the course of several evening emergences with a novel obstacle placed in their flight path. Our analysis of reconstructed flight data suggests that vision, echolocation, and spatial memory together with the possible exercise of an ability in using predictive navigation are mutually reinforcing aspects of a composite perceptual system that guides flight. Together with the recent development in robotics, our paper points to the possible interpretation that while each stream of sensory information plays an important role in bat navigation, it is the emergent effects of combining modalities that enable bats to fly through complex spaces.

Interannual Survival of Myotis lucifugus (Chiroptera: Vespertilionidae) near the Epicenter of White-Nose Syndrome

Abstract Reduced populations of Myotis lucifugus (Little Brown Myotis) devastated by white-nose syndrome (WNS) persist in eastern North America. Between 2009 and 2013, we recaptured 113 marked individuals that survived between 1 and 6 winters in New England since the arrival of WNS. We also observed signs of reproductive success in 57 recaptured bats.

3D pose estimation of bats in the wild

Vision-based methods have gained popularity as a tool for helping to analyze the behavior of bats. Though, for bats in the wild, there are still no tools capable of estimating and subsequently analyzing articulated 3D bat pose. We propose a model-based multi-view articulated 3D bat pose estimation framework for this novel problem. Key challenges include the large search space associated with articulated 3D pose, the ambiguities that arise from 2D projections of 3D bodies, and the low resolution image data we have available. Our method uses multi-view camera geometry and temporal constraints to reduce the state space of possible articulated 3D bat poses and finds an optimal set using a Markov Random Field based model. Our experiments use real video data of flying bats and gold-standard annotations by a bat biologist. Our results show, for the first time in the literature, articulated 3D pose estimates being generated automatically for video sequences of bats flying in the wild. The average differences in body orientation and wing joint angles, between estimates produced by our method and those based on gold-standard annotations, ranged from 16° - 21° (i.e., ≈ 17% - 23%) for orientation and 14° - 26° (i.e., ≈ 7%- 14%) for wing joint angles.

Using collision cones to assess biological deconfliction methods

Biological systems consistently outperform autonomous systems governed by engineered algorithms in their ability to reactively avoid collisions. To better understand this discrepancy, a collision avoidance algorithm was applied to frames of digitized video trajectory data from bats, swallows and fish (Myotis velifer, Petrochelidon pyrrhonota and Danio aequipinnatus). Information available from visual cues, specifically relative position and velocity, was provided to the algorithm which used this information to define collision cones that allowed the algorithm to find a safe velocity requiring minimal deviation from the original velocity. The subset of obstacles provided to the algorithm was determined by the animals sensing range in terms of metric and topological distance. The algorithmic calculated velocities showed good agreement with observed biological velocities, indicating that the algorithm was an informative basis for comparison with the three species and could potentially be improved for engineered applications with further study.