Vana Kalogeraki

Intelligent Urban Data Monitoring for Smart Cities

Urban data management is already an essential element of modern cities. The authorities can build on the variety of automatically generated information and develop intelligent services that improve citizens daily life, save environmental resources or aid in coping with emergencies. From a data mining perspective, urban data introduce a lot of challenges. Data volume, velocity and veracity are some obvious obstacles. However, there are even more issues of equal importance like data quality, resilience, privacy and security. In this paper we describe the development of a set of techniques and frameworks that aim at effective and efficient urban data management in real settings. To do this, we collaborated with the city of Dublin and worked on real problems and data. Our solutions were integrated in a system that was evaluated and is currently utilized by the city.

Location privacy for crowdsourcing applications

This paper contributes to mobile crowdsourcing applications by developing a privacy preserving framework that enables users to contribute content to the community while controlling their privacy exposure. One fundamental challenge in such applications is how to preserve user privacy, as participants may end up revealing a great deal of user-identified, geo-located data, which can easily unfold user trajectories or sensitive locations (e.g., users home or work location). In this paper we develop PROMPT, a highly efficient privacy preserving framework that runs locally on mobile devices. PROMPT relies on a novel geometric approximation approach to preserve user privacy, by evaluating the privacy exposure of users before sharing their geo-located data. Our detailed experimental evaluation using real-world datasets illustrates that our approach is effective, practical and has low overhead on smartphones.

Maximizing Determinism in Stream Processing Under Latency Constraints

The problem of coping with the demands of determinism and meeting latency constraints is challenging in distributed data stream processing systems that have to process high volume data streams that arrive from different unsynchronized input sources. In order to deterministically process the streaming data, they need mechanisms that synchronize the order in which tuples are processed by the operators. On the other hand, achieving real-time response in such a system requires careful tradeoff between determinism and low latency performance. We build on a recently proposed approach to handle data exchange and synchronization in stream processing, namely ScaleGate, which comes with guarantees for determinism and an efficient lock-free implementation, enabling high scalability. Considering the challenge and trade-offs implied by real-time constraints, we propose a system which comprises (a) a novel data structure called Slack-ScaleGate (SSG), along with its algorithmic implementation; SSG enables us to guarantee the deterministic processing of tuples as long as they are able to meet their latency constraints, and (b) a method to dynamically tune the maximum amount of time that a tuple can wait in the SSG data-structure, relaxing the determinism guarantees when needed, in order to satisfy the latency constraints. Our detailed experimental evaluation using a traffic monitoring application deployed in the city of Dublin, illustrates the working and benefits of our approach.

A Fast and Efficient Entity Resolution Approach for Preserving Privacy in Mobile Data

With the advent of mobile networking and the widespread adoption of smartphone devices, a number of location-based services have emerged, where users actively participate by sharing and receiving mobility data. However, the collection and analysis of user mobility data, such as user location information and trajectory data, especially when exploited together with external sources, such as social networks that often provide rich and publicly available information, can reveal sensitive user information. This paper proposes an approach based on entity resolution which enables users to disclose their mobility information without compromising their privacy, even if these data are linked with external publicly available information. We present detailed experimental results using four real datasets to illustrate that our approach is practical, efficient and effectively preserves privacy by eliminating potential links among the data.

DIsCO: DynamIc Data COmpression in Distributed Stream Processing Systems

Supporting high throughput in Distributed Stream Processing Systems (DSPSs) has been an important goal in recent years. Current works either focus on automatically increasing the system resources whenever the current setup is inadequate or apply load shedding techniques discarding some of the incoming data. However, both approaches have significant shortcomings as they require on the fly application reconfiguration where the application needs to be stopped and re-uploaded in the cluster with the new configurations, and can lead to significant information loss. One approach that has not yet been considered for improving the throughput of DSPSs is exploiting compression algorithms to minimize the communication overhead between components especially in cases where we have large-sized data like live CCTV camera reports. This work is the first that provides a novel framework, built on top of Apache Storm, which enables dynamic compression of incoming streaming data. Our approach uses a profiling algorithm to automatically determine the compression algorithm that should be applied and supports both lossless and lossy compression techniques. Furthermore, we propose a novel algorithm for determining when profiling should be applied. Finally, our detailed experimental evaluation with commonly used stream processing applications, indicates a clear improvement on the applications’ throughput when our proposed techniques are applied.

A Pareto-based scheduler for exploring cost-performance trade-offs for MapReduce workloads

In recent years, we are observing an increased demand for processing large amounts of data. The MapReduce programming model has been utilized by major computing companies and has been integrated by novel cyber physical systems (CPS) in order to perform large-scale data processing. However, the problem of efficiently scheduling MapReduce workloads in cluster environments, like Amazon’s EC2, can be challenging due to the observed trade-off between the need for performance and the corresponding monetary cost. The problem is exacerbated by the fact that cloud providers tend to charge users based on their I/O operations, increasing dramatically the spending budget. In this paper, we describe our approach for scheduling MapReduce workloads in cluster environments taking into consideration the performance/budget trade-off. Our approach makes the following contributions: (i) we propose a novel Pareto-based scheduler for identifying near-optimal resource allocations for user workloads with respect to performance and monetary cost, and (ii) we develop an automatic configuration of basic tasks’ parameters that allows us to further minimize the user’s spending budget and the jobs’ execution times. Our detailed experimental evaluation using both real and synthetic datasets illustrate that our approach improves the performance of the workloads as much as 50%, compared to its competitors.

INSIGHT: Dynamic Traffic Management Using Heterogeneous Urban Data

In this demo we present INSIGHT, a system that provides traffic event detection in Dublin by exploiting Big Data and Crowdsourcing techniques. Our system is able to process and analyze input from multiple heterogeneous urban data sources.

Using Human Social Sensors for Robust Event Location Detection

Recently, the massive prevalence of mobile devices has led to the development of mobile social sensing systems where humans are enlisted to act as social sensors to perform geo-located tasks that require human intelligence or intervention. Social sensing from ubiquitous users can provide significant benefits particularly during crisis management and emergency scenarios. However, an important problem during such emergencies is how to exploit social sensors to accurately determine the location, extent and severity of the event. In this paper we develop a social sensing system that uses humans as social sensors where we apply particle filtering to iteratively determine the spatial areas to be investigated to accurately detect the location and state of the target event. Our experiments illustrate that our approach can accurately identify critical real-world events using feedback from the social sensors.

Dynamic reduce task adjustment for hadoop workloads

In recent years, we observe an increasing demand for systems that are capable of efficiently managing and processing huge amounts of data. Apaches Hadoop, an open-source implementation of Googles MapReduce programming model, has emerged as one of the most popular systems for Big Data processing and is supported by major companies like Facebook, Yahoo! and Amazon. One of the most challenging aspects of executing a Hadoop job, is to configure appropriately the number of reduce tasks. The problem is exacerbated when multiple jobs are executing concurrently competing for the available system resources. Our approach consists of the following components: (i) an algorithm for computing the appropriate number of reduce tasks per job, (ii) the usage of profiler-jobs for gathering information necessary for the reduce task computation and (iii) two different policies for fragmenting the reduce tasks to the available system resources when multiple jobs execute concurrently in the cluster. Our detailed experimental evaluation using traffic monitoring Hadoop jobs on our local cluster, illustrates that our approach is practical and exhibits solid performance.