Teodora Sandra Buda

CogNet: A network management architecture featuring cognitive capabilities

It is expected that the fifth generation mobile networks (5G) will support both human-to-human and machine-to-machine communications, connecting up to trillions of devices and reaching formidable levels of complexity and traffic volume. This brings a new set of challenges for managing the network due to the diversity and the sheer size of the network. It will be necessary for the network to largely manage itself and deal with organisation, configuration, security, and optimisation issues. This paper proposes an architecture of an autonomic self-managing network based on Network Function Virtualization, which is capable of achieving or balancing objectives such as high QoS, low energy usage and operational efficiency. The main novelty of the architecture is the Cognitive Smart Engine introduced to enable Machine Learning, particularly (near) real-time learning, in order to dynamically adapt resources to the immediate requirements of the virtual network functions, while minimizing performance degradations to fulfill SLA requirements. This architecture is built within the CogNet European Horizon 2020 project, which refers to Cognitive Networks.

Can machine learning aid in delivering new use cases and scenarios in 5G

5G represents the next generation of communication networks and services, and will bring a new set of use cases and scenarios. These in turn will address a new set of challenges from the network and service management perspective, such as network traffic and resource management, big data management and energy efficiency. Consequently, novel techniques and strategies are required to address these challenges in a smarter way. In this paper, we present the limitations of the current network and service management and describe in detail the challenges that 5G is expected to face from a management perspective. The main contribution of this paper is presenting a set of use cases and scenarios of 5G in which machine learning can aid in addressing their management challenges. It is expected that machine learning can provide a higher and more intelligent level of monitoring and management of networks and applications, improve operational efficiencies and facilitate the requirements of the future 5G network.

ADE: An ensemble approach for early Anomaly Detection

Proactive anomaly detection refers to anticipating anomalies or abnormal patterns within a dataset in a timely manner. Discovering anomalies such as failures or degradations before their occurrence can lead to great benefits such as the ability to avoid the anomaly happening by applying some corrective measures in advance (e.g., allocating more resources for a nearly saturated system in a data centre). In this paper we address the proactive anomaly detection problem through machine learning and in particular ensemble learning. We propose an early Anomaly Detection Ensemble approach, ADE, which combines results of state-of-the-art anomaly detection techniques in order to provide more accurate results than each single technique. Moreover, we utilise a a weighted anomaly window as ground truth for training the model, which prioritises early detection in order to discover anomalies in a timely manner. Various strategies are explored for generating ground truth windows. Results show that ADE shows improvements of at least 10% in earliest detection score compared to each individual technique across all datasets considered. The technique proposed detected anomalies in advance up to ∼16h before they actually occurred.

RCMC: Recognizing Crowd-Mobility Patterns in Cities Based on Location Based Social Networks Data

During the past few years, the analysis of data generated from Location-Based Social Networks (LBSNs) have aided in the identification of urban patterns, understanding activity behaviours in urban areas, as well as producing novel recommender systems that facilitate users’ choices. Recognizing crowd-mobility patterns in cities is very important for public safety, traffic managment, disaster management, and urban planning. In this article, we propose a framework for Recognizing the Crowd Mobility Patterns in Cities using LBSN data. Our proposed framework comprises four main components: data gathering, recurrent crowd-mobility patterns extraction, temporal functional regions detection, and visualization component. More specifically, we employ a novel approach based on Non-negative Matrix Factorization and Gaussian Kernel Density Estimation for extracting the recurrent crowd-mobility patterns in cities illustrating how crowd shifts from one area to another during each day across various time slots. Moreover, the framework employs a hierarchical clustering-based algorithm for identifying what we refer to as temporal functional regions by modeling functional areas taking into account temporal variation by means of check-ins’ categories. We build the framework using a spatial-temporal dataset crawled from Twitter for two entire years (2013 and 2014) for the area of Manhattan in New York City. We perform a detailed analysis of the extracted crowd patterns with an exploratory visualization showing that our proposed approach can identify clearly obvious mobility patterns that recur over time and location in the urban scenario. Using same time interval, we show that correlating the temporal functional regions with the recognized recurrent crowd-mobility patterns can yield to a deeper understanding of city dynamics and the motivation behind the crowd mobility. We are confident that our proposed framework not only can help in managing complex city environments and better allocation of resources based on the expected crowd mobility and temporal functional regions but also can have a direct implication on a variety of applications such as personalized recommender systems, anomalous event detection, disaster resilience management systems, and others.

DeepAD: A Generic Framework Based on Deep Learning for Time Series Anomaly Detection

This paper presents a generic anomaly detection approach for time-series data. Existing anomaly detection approaches have several drawbacks such as a large number of false positives, parameters tuning difficulties, the need for a labeled dataset for training, use-case restrictions, or difficulty of use. We propose DeepAD, an anomaly detection framework that leverages a plethora of time-series forecasting models in order to detect anomalies more accurately, irrespective of the underlying complex patterns to be learnt. Our solution does not rely on the labels of the anomalous class for training the model, nor for optimizing the threshold based on highest detection given the labels in the training data. We compare our framework against EGADS framework on real and synthetic data with varying time-series characteristics. Results show significant improvements on average of 25% and up to \(40-50\)% in \(F_1{\text{- }}score\), precision, and recall on the Yahoo Webscope Benchmark.

Investigation of Replication Factor for Performance Enhancement in the Hadoop Distributed File System

The massive growth in the volume of data and the demand for big data utilisation has led to an increasing prevalence of Hadoop Distributed File System (HDFS) solutions. However, the performance of Hadoop and indeed HDFS has some limitations and remains an open problem in the research community. The ultimate goal of our research is to develop an adaptive replication system; this paper presents the first phase of the work - an investigation into the replication factor used in HDFS to determine whether increasing the replication factor for in-demand data can improve the performance of the system. We constructed a physical Hadoop cluster for our experimental environment, using TestDFSIO and both the real world and the synthetic data sets, NOAA and TPC-H, with Hive to validate our proposal. Results show that increasing the replication factor of the »hot» data increases the availability and locality of the data, and thus, decreases the job execution time.

ReX: Representative extrapolating relational databases

Abstract Generating synthetic data is useful in multiple application areas (e.g., database testing, software testing). Nevertheless, existing synthetic data generators are either limited to generating data that only respect the database schema constraints, or they are not accurate in terms of representativeness, unless a complex set of inputs are given from the user (such as the data characteristics of the desired generated data). In this paper, we present an extension of a prior representative extrapolation technique, namely ReX [20], limited to natural scaling rates. The objective is to produce in an automated and efficient way a representative extrapolated database, given an original database O and a rational scaling rate, s ∈ Q . In the extended version, the ReX system can handle rational scaling rates by combining existing efficient sampling and extrapolation techniques. Furthermore, we propose a novel sampling technique, RVFDS for handling positive rational values for the desired size of the generated database. We evaluate ReX in comparison with a realistic scaling method, namely UpSizeR [43], on both real and synthetic databases. We show that our solution statistically and significantly outperforms the compared method for rational scaling rates in terms of representativeness.

Cognitive Applications and Their Supporting Architecture for Smart Cities

A smart city enables a new and comprehensive approach to manage infrastructural, social, and institutional aspects of an urban ecosystem. The state information on technological, economic, and social factors forms the basis for such management, which is normally tracked by sensors. It is expected that billions of sensors and controllers will be connected and share knowledge. The proliferation of sensor networks and their different characteristics pose new challenges in relation to resource management in the context of smart cities but also lead to the rise of a new type of data on social networks—location-based social network (LBSN). To tackle these challenges, this chapter proposes a cognitive architecture to enable big data applications to largely manage themselves and to deal with organization, configuration, security, and optimization. Specially, this architecture work supports anomaly detection, which is essential to ensure quality of service and reduce cost of smart city services. Meanwhile, to harness the power of LBSN, this chapter presents solutions to identify functional regions of modern cities and discovery urban patterns, which enable us to better understand complex cities and activity behaviors.