Malika Meghjani

Multi-domain monitoring of marine environments using a heterogeneous robot team

In this paper we describe a heterogeneous multi-robot system for assisting scientists in environmental monitoring tasks, such as the inspection of marine ecosystems. This team of robots is comprised of a fixed-wing aerial vehicle, an autonomous airboat, and an agile legged underwater robot. These robots interact with off-site scientists and operate in a hierarchical structure to autonomously collect visual footage of interesting underwater regions, from multiple scales and mediums. We discuss organizational and scheduling complexities associated with multi-robot experiments in a field robotics setting. We also present results from our field trials, where we demonstrated the use of this heterogeneous robot team to achieve multi-domain monitoring of coral reefs, based on real-time interaction with a remotely-located marine biologist.

Bimodal information analysis for emotion recognition

We present a bimodal information analysis system for automatic emotion recognition. Our approach is based on the analysis of video sequences which combines facial expressions observed visually with acoustic features to automatically recognize five universal emotion classes: Anger, Disgust, Happiness, Sadness and Surprise. We address the challenges posed during the temporal analysis of the bimodal data and introduce a novel technique for combining the best features of instantaneous and temporal based visual recognition systems. We obtain robust appearance-based visual features which we classify instantaneously and aggregate it temporally to improve the recognition rates when compared to single-frame based instantaneous classification. The performance of the system is further boosted by using the complementary audio information for the bimodal emotion recognition. We combine the two modalities at both feature and score level to compare the respective joint emotion recognition rates. The emotions are instantaneously classified using a Support Vector Machine and sequentially aggregated based on their classification probabilities. This approach is validated on a posed audio-visual database and a natural interactive database. The experiments performed on these databases provide encouraging results with the best combined recognition rate being 82%.

MARE: Marine Autonomous Robotic Explorer

We present MARE, an autonomous airboat robot that is suitable for exploration-oriented tasks, such as inspection of coral reefs and shallow seabeds. The combination of this platforms particular mechanical properties and its powerful software framework enables it to function in a multitude of potential capacities, including autonomous surveillance, mapping, and search operations. In this paper we describe two different exploration strategies and their implementation using the MARE platform. First, we discuss the application of an efficient coverage algorithm, for the purpose of achieving systematic exploration of a known and bounded environment. Second, we present an exploration strategy driven by surprise, which steers the robot on a path that might lead to potentially surprising observations.

Combining Multi-robot Exploration and Rendezvous

We consider the problem of exploring an unknown environment with a pair of mobile robots. The goal is to make the robots meet (or rendezvous) in minimum time such that there is a maximum speed gain of the exploration task. The key challenge in achieving this goal is to rendezvous with the least possible dependency on communication. This single constraint involves several sub-problems: finding unique potential rendezvous locations in the environment, ranking these locations based on their uniqueness and synchronizing with the other robot to meet at one of the locations at a scheduled time. In addition, these tasks are to be performed simultaneously while exploring and mapping the environment. We propose an approach for efficiently combining the exploration and rendezvous tasks by considering the cost of reaching a rendezvous location and the reward of its uniqueness. This cost and reward model is combined with a set of deterministic and probabilistic rendezvous strategies for the robots to meet during exploration. Experimental results suggest that the joint tasks of exploration and rendezvous are substantially improved by ranking the potential rendezvous locations based on the combined cost-reward criterion when compared to the ranking solely based on the uniqueness of the location.

Asymmetric Rendezvous Search at Sea

In this paper we address the rendezvous problem between an autonomous underwater vehicle (AUV) and a passively floating drifter on the sea surface. The AUVs mission is to keep an estimate of the floating drifters position while exploring the underwater environment and periodically attempting to rendezvous with it. We are interested in the case where the AUV loses track of the drifter, predicts its location and searches for it in the vicinity of the predicted location. We parameterize this search problem with respect to both the uncertainty in the drifters position estimate and the ratio between the drifter and the AUV speeds. We examine two search strategies for the AUV, an inward spiral and an outward spiral. We derive conditions under which these patterns are guaranteed to find a drifter, and we empirically analyze them with respect to different parameters in simulation. In addition, we present results from field trials in which an AUV successfully found a drifter after periods of communication loss during which the robot was exploring.

Multi-robot exploration and rendezvous on graphs

We address the problem of arranging a meeting (or rendezvous) between two or more robots in an unknown bounded topological environment, starting at unknown locations, without any communication. The goal is to rendezvous in minimum time such that the robots can share resources for performing any global task. We specifically consider a global exploration task executed by two or more robots. Each robot explores the environment simultaneously, for a specified time, then selects potential rendezvous locations, where it expects to find other robots, and visits them. We propose a ranking criterion for selecting the order in which potential rendezvous locations will be visited. This ranking criterion associates a cost for visiting a rendezvous location and gives an expected reward of finding other agents. We evaluate the time taken to rendezvous by varying a set of conditions including: world size, number of robots, starting location of each robot and the presence of sensor noise. We present simulation results to quantify the effect of the aforementioned factors on the rendezvous time.

Efficient Terrain Driven Coral Coverage Using Gaussian Processes for Mosaic Synthesis

In this paper we present an efficient method for visual mapping of open water environments using exploration and reward identification followed by selective visual coverage. In particular, we consider the problem of visual mapping a shallow water coral reef to provide an environmental assay. Our approach has two stages based on two classes of sensors: bathymetric mapping and visual mapping. We use a robotic boat to collect bathymetric data using a sonar sensor for the first stage and video data using a visual sensor for the second stage. Since underwater environments have varying visibility, we use the sonar map to select regions of potential value, and efficiently construct the bathymetric map from sparse data using a Gaussian Process model. In the second stage, we collect visual data only where there is good potential pay-off, and we use a reward-driven finite-horizon model akin to a Markov Decision Process to extract the maximum amount of valuable data in the least amount of time. We show that a very small number of sonar readings suffice on a typical fringing reef. We validate and demonstrate our surveying technique using real robot in the presence of real world conditions such as wind and current. We also show that our proposed approach is suitable for visual surveying by presenting a visual collage of the reef.

Multi-target rendezvous search

In this paper, we examine multi-target search, where one or more targets must be found by a moving robot. Given the targets initial probability distribution or the expected search region, we present an analysis of three search strategies - Global maxima search, Local maxima search, and Spiral search. We aim at minimizing the mean-time-to-find and maximizing the total probability of finding the target. This leads to two types of illustrative performance metrics: minimum time capture and guaranteed capture. We validate the search strategies with respect to these two performance metrics. In addition, we study the effect of different target distributions on the performance of the search strategies. We also consider the practical realization of the proposed algorithms for multi-target search. The search strategies are analytically evaluated, through simulations and illustrative deployments, in open-water with an Autonomous Surface Vehicle (ASV) and drifting sensor targets.

Multi-agent rendezvous on street networks

In this paper we present an algorithm for finding a distance optimal rendezvous location with respect to both initial and target locations of the mobile agents. These agents can be humans or robots, who need to meet and split while performing a collaborative task. Our aim is to embed the meeting process within a background activity such that the agents travel through the rendezvous location while taking the shortest paths to their respective target locations. We analyze this problem in a street network scenario with two agents who are given their individual scheduled routes to complete with an underlying common goal. The agents are allowed to select any combination of the waypoints along their routes as long as they travel the shortest path and pass through the same potential rendezvous location. The total number of path combinations that the agents need to evaluate for the shortest path increases rapidly with the number of waypoints along their routes. We address this computational cost by proposing a combination of Euclidean and street network distances for a trade-off between the number of queries and a distance optimal solution.

Multi-target search strategies

This paper addresses the problem of searching multiple non-adversarial targets using a mobile searcher in an obstacle-free environment. In practice, we are particularly interested in marine applications where the targets drift on the ocean surface. These targets can be surface sensors used for marine environmental monitoring, drifting debris, or lost divers in open water. Searching for a floating target requires prior knowledge about the search region and an estimate of the targets motion. This task becomes challenging when searching for multiple targets where persistent searching for one of the targets can result in the loss of other targets. Hence, the searcher needs to trade-off between guaranteed and fast searches. We propose three classes of search strategies for addressing the multi-target search problem. These include, data-independent, probabilistic and hybrid search. The data-independent search strategy follow a pre-defined search pattern and schedule. The probabilistic search strategy is guided by the estimated probability distribution of the search target. The hybrid strategy combines data-independent search patterns with a probabilistic search schedule. We evaluate these search strategies in simulation and compare their performance characteristics in the context of searching multiple drifting targets using an Autonomous Surface Vehicle (ASV).