Vidarshana W. Bandara

Space-Time Signal Processing for Distributed Pattern Detection in Sensor Networks

A theory and algorithm for detecting and classifying weak, distributed patterns in network data is presented. The patterns we consider are anomalous temporal correlations between signals recorded at sensor nodes in a network. We use robust matrix completion and second order analysis to detect distributed patterns that are not discernible at the level of individual sensors. When viewed independently, the data at each node cannot provide a definitive determination of the underlying pattern, but when fused with data from across the network the relevant patterns emerge. We are specifically interested in detecting weak patterns in computer networks where the nodes (terminals, routers, servers, etc.) are sensors that provide measurements (of packet rates, user activity, central processing unit usage, etc.). The approach is applicable to many other types of sensor networks including wireless networks, mobile sensor networks, and social networks where correlated phenomena are of interest.

Extracting baseline patterns in Internet traffic using Robust Principal Components

Robust BaseLine (RBL) is a formal technique for extracting the baseline of network traffic to capture the underlying traffic trend. A range of applications such as anomaly detection and load balancing rely on baseline estimation. Once the fundamental period of the pattern for analysis is recognized, e.g., based on user interest or a period detector such as Autocorrelation Function (ACF), the basic extraction is carried out in two steps. First, the common component across the dataset is separated using Robust Principal Component Analysis (RPCA). The fundamental pattern in the common component is extracted using Principal Component Analysis (PCA) in the second step. Scaling factors required to fit the base-pattern back into the data are returned automatically by PCA. Two types of traffic baselines may be extracted: RBL-L captures the common behavior across time on a single link, and RBL-N captures the common behavior across a network of links, i.e., in space. RBL-N is particularly useful for specifying traffic matrices more efficiently over time, which normally requires multiple updates to follow baseline trends. The derived base-patterns for a single link or a single time period is then extended over the entire network or thru the entire observation period with a compressive analysis. The compressed base-pattern provides a smoother baseline and also a filter to separate baseline traffic and the deviations on the fly from traffic measurements. When compared against BLGBA (Baseline for Automatic Backbone Management) the proposed scheme provides a less noisy, more precisely fitting baseline. It is also more effective in revealing anomalies.

An adaptive compressive sensing scheme for network tomography based fault localization

A scalable network fault localization scheme based on compressive sensing is proposed. Aimed at large networks, the proposed scheme monitors a network with a few paths covering the network, and upon detection of anomalies in one or more paths, adaptively carries out additional end-to-end measurements to localize the faulty links. Each adaptive measurement covers a set of links identified based on the previous resolution. The scheme is highly scalable as the total number of measurements required grows logarithmically with the number of links in the network - a level of scalability not practically achieved for network data inference with compressive sensing so far. The scheme is tested on realistic Internet topologies with Gilbert-Elliott loss model calibrated with measurements made on Planet-Lab infrastructure. Results indicate that the converged solution of the proposed scheme achieves over 99% detection rates and less than 1% false positive rates. The proposed scalable scheme is accurate in terms of detection, cost effective in terms of implementation, and casts a minimal monitoring traffic load.

Phenomena discovery in WSNs: A compressive sensing based approach

A Compressive Sensing (CS) based solution is proposed for centralized and distributed discovery of physical phenomena in large scale Wireless Sensor Networks (WSNs). WSNs monitoring environmental phenomena over large geographic areas collect measurements from a large number of distributed sensors. Compressive Sensing provides an effective means of discovery and reconstruction of functions with only a subset of samples. Traditional CS relies on uniformly distributed samples which limits practicality of CS based recovery. To enhance the flexibility of sampling and implementation, the proposed approach uses random walk based samples. Unlike uniform sampling, random walk based sampling enables individual nodes achieve phenomenon awareness, i.e., the physical distribution of the phenomenon. We also derive a theoretical upper bound for the reconstruction failure probability. Simulation results on the number of samples required and error show that random walk based sampling is comparable to uniform sampling but with superior energy efficiency. More importantly, the proposed scheme provides a practical solution for a range of applications where uniform sampling is less economical or even infeasible.

Detecting radicalization trajectories using graph pattern matching algorithms

This paper outlines our on-going efforts to address the radicalization detection problem, the automated or semi-automated task of dynamically detecting and tracking behavioral changes in individuals who undergo the process of increasingly espousing jihadist beliefs and transition to the use of violent action in support of those beliefs. Leveraging the notion that personal trajectories towards violent radicalization exist, we take a graph pattern matching approach to track individual-level indicators using data fused from available public and government/law enforcement databases. We show that our approach provides analysts with the ability to find full or partial matches against a query pattern of radicalization, and a means to quantify the pace of the appearance of the indicators that may help prioritize investigative efforts and resources to prevent planned attacks.

Pattern Matching Trajectories for Investigative Graph Searches

Investigative graph search is the process of searching for and prioritizing entities of interest that may exhibit part or all of a pattern of attributes or connections for a latent behavior. In this work we formulate a related sub-problem of determining the pattern matching trajectories of such entities. The goal is to not only provide analysts with the ability to find full or partial matches against a query pattern, but also a means to quantify the pace of the appearance of the indicators. This technology has a variety of potential applications such as aiding in the detection of homegrown violent extremists before they carry out acts of domestic terrorism, detecting signs for post-traumatic stress in veterans, or tracking potential customer activities and experiences along a consumer journey. We propose a vectorized graph pattern matching approach that calculates the multi-hop class similarities between nodes in query and data graphs over time. By tracking partial match trajectories, we provide another dimension of analysis in investigative graph searches to highlight entities on a pathway towards a pattern of a latent behavior. We demonstrate the performance of our approach on a real-world BlogCatalog dataset of over 470K nodes and 4 million edges, where 98.56% of nodes and 99.65% of edges were filtered out with preprocessing steps, and successfully detected the trajectory of the top 1,327 nodes towards a query pattern.

INSiGHT: A system to detect violent extremist radicalization trajectories in dynamic graphs

Abstract The number and lethality of violent extremist plots motivated by the Salafi-jihadist ideology have been growing for nearly the last decade in many parts of the world including both the U.S and Western Europe. While detecting the radicalization of violent extremists is a key component in preventing future terrorist attacks, it remains a significant challenge to law enforcement due to the issues of both scale and dynamics. We propose the development of a radicalization trend detection system as a risk assessment assistance technology that relies on data mined from public data and government databases for individuals who exhibit risk indicators for extremist violence, and enables law enforcement to monitor those individuals at the scope and scale that is lawful, and accounts for the dynamic indicative behaviors of the individuals and their associates rigorously and automatically. We frame our approach to monitoring the radicalization pattern of behaviors as a unique dynamic graph pattern matching problem, and develop a technology called INSiGHT ( In vestigative S earch for G rap h - T rajectories) to help identify individuals or small groups with conforming subgraphs to a radicalization query pattern, and follow the match trajectories over time. This paper presents the overall INSiGHT architecture and is aimed at assisting law enforcement and intelligence agencies in monitoring and screening for those individuals whose behaviors indicate a significant risk for violence, and allow for the better prioritization of limited investigative resources. We demonstrated the performance of INSiGHT on a variety of datasets, to include small synthetic radicalization-specific datasets and a real behavioral dataset of time-stamped radicalization indicators of recent U.S. violent extremists.

INSiGHT: A system for detecting radicalization trajectories in large heterogeneous graphs

Detecting the radicalization of violent extremists is a key component in preventing future terrorist attacks, but it remains a significant challenge to law enforcement. Framing our approach as a unique dynamic graph pattern matching problem, we address this challenge by introducing an analyst-in-the-loop framework and related technology called INSiGHT (Investigative Search for Graph-Trajectories) to help identify individuals or small groups with conforming subgraphs to a radicalization query pattern, and follow the match trajectories over time. INSiGHT is aimed at developing tools for assisting law enforcement and intelligence agencies in monitoring and screening for those individuals whose behaviors indicate a significant risk for violence, and allow for the better prioritization of limited investigative resources. We demonstrate the validity of INSiGHT on two small synthetic datasets, and confirm that we can scale to and provide consistent results for a large, real world proxy dataset of over 470K nodes and 4 million edges.

Spatiotemporal model for Internet traffic anomalies

Models for Internet traffic anomalies greatly benefit a range of applications including robust network design, network provisioning and performance studies. A novel approach to analyse and model network traffic anomalies is presented. The proposed approach individually characterises different aspects of anomalies, such as origin, termination, propagation and changes in duration and volume, with common random processes. These characteristics are then integrated into a single model that successfully captures the overall anomaly behaviours. Characterisation of each anomaly property requires only a few parameters, leading to a concise set of parameters for the entire model. Although the model is calibrated with local measurements made at nodes, it successfully represents the global behaviours of anomalies over the network. The proposed model is applicable both at nodal level and at subnet level. This enables hierarchically analysing large and sophisticated networks. Anomalies are analysed using a multi-scale analysis framework based on which, a real-time monitoring system that efficiently communicate ongoing anomaly information across the network is developed. The system is also used for learning regional model parameters distributively. Internet2 traffic data is analysed using the framework, and the corresponding model parameters are derived. These results provide insight on the nature of anomalies in networks.