Yosef Moatti

Guaranteeing High Availability Goals for Virtual Machine Placement

The placement of virtual machines (VMs) on a cluster of hosts under multiple constraints, including administrative (security, regulations) resource-oriented (capacity, energy), and QoS-oriented (performance) is a highly complex task. We define a new high-availability property for a VM, when a VM is marked as k-resilient, as long as there are up to k host failures, it should be guaranteed that it can be relocated to a non-failed host without relocating other VMs. Together with Hardware Predictive Failure Analysis and live migration, which enable VMs to be evacuated from a host before it fails, this property allows the continuous running of VMs on the cluster despite host failures. The complexity of the constraints associated with k-resiliency, which are naturally expressed by Second Order logic statements, prevented their integration into the placement computation until now. We present a novel algorithm which enables this integration by transforming the k-resiliency constraints to rules consumable by a generic Constraint Programming engine, prove that it guarantees the required resiliency and describe the implementation. We provide some preliminary results and compare our high availability support with naive solutions.

VM Placement Strategies for Cloud Scenarios

The problem of Virtual Machine (VM) placement in a compute cloud infrastructure is well-studied in the literature. However, the majority of the existing works ignore the dynamic nature of the incoming stream of VM deployment requests that continuously arrive to the cloud provider infrastructure. In this paper we provide a practical model of cloud placement management under a stream of requests and present a novel technique called Backward Speculative Placement (BSP) that projects the past demand behavior of a VM to a candidate target host. We exploit the BSP technique in two algorithms, first for handling the stream of deployment requests, second in a periodic optimization, to handle the dynamic aspects of the demands. We show the benefits of our BSP technique by comparing the results on a simulation period with a strategy of choosing an optimal placement at each time instant, produced by a generic MIP solver.

IOStack: Software-Defined Object Storage

As the complexity and scale of cloud storage systems grow, software-defined storage (SDS) has become a prime candidate to simplify cloud storage management. Here, the authors present IOStack, the first SDS architecture for object stores (such as OpenStack Swift). At the control plane, the provisioning of SDS services to tenants is made according to a set of policies managed via a high-level domain-specific language (DSL). Policies can target storage automation or specific service-level agreement (SLA) objectives. At the data plane, policies define the enforcement of SDS services, namely filters, on a tenants requests. Moreover, IOStack is a framework to build a variety of filters, ranging from general-purpose computations close to the data to specialized data management mechanisms. Experiments illustrate that IOStack enables easy and effective policy-based provisioning, which can significantly improve the operation of a multitenant object store.

Security intelligence for cloud management infrastructures

cloud management infrastructures S. Berger S. Garion Y. Moatti D. Naor D. Pendarakis A. Shulman-Peleg J. R. Rao E. Valdez Y. Weinsberg In this paper, we address the problem of protecting cloud infrastructures and customer workloads via smart auditing and logging, satisfying regulatory and compliance requirements. We observe that traditional approaches of logging and auditing events in cloud-scale infrastructures will not be effective without taking into account other controls. We introduce the concept of Cloud Security Intelligence (CSI), a new systematic approach for collecting, aggregating, correlating, and analyzing data from management, control, and data planes of cloud infrastructures, using a closed-loop architecture. Our approach cross-correlates control and data plane events, automatically deriving rules for monitoring and audits. Specifically, it sets dynamic rules concerning what and how to audit, adapting the logging accordingly, while comparing the data access patterns and configurations with the desired privileges and specifications. We have implemented CSI on two OpenStack-based systems: a closed loop network protection scheme and a cloud storage audit and risk analysis scheme for monitoring data access. In order to make cloud security approaches effective and scalable, we suggest that it is essential to use an intelligent approach such as correlating cloud logic from multiple cloud layers and components-e.g., IaaS (Infrastructure as a Service) or PaaS (Platform as a Service)-providing workload context that is maintained by cloud management systems, and using analytics on historical logs.

WatchIT: Who Watches Your IT Guy?

System administrators have unlimited access to system resources. As the Snowden case highlighted, these permissions can be exploited to steal valuable personal, classified, or commercial data. This problem is exacerbated when a third party administers the system. For example, a bank outsourcing its IT would not want to allow administrators access to the actual data. We propose WatchIT: a strategy that constrains IT personnels view of the system and monitors their actions. To this end, we introduce the abstraction of perforated containers -- while regular Linux containers are too restrictive to be used by system administrators, by punching holes in them, we strike a balance between information security and required administrative needs. Following the principle of least privilege, our system predicts which system resources should be accessible for handling each IT issue, creates a perforated container with the corresponding isolation, and deploys it as needed for fixing the problem. Under this approach, the system administrator retains superuser privileges, however only within the perforated container limits. We further provide means for the administrator to bypass the isolation, but such operations are monitored and logged for later analysis and anomaly detection. We provide a proof-of-concept implementation of our strategy, which includes software for deploying perforated containers, monitoring mechanisms, and changes to the Linux kernel. Finally, we present a case study conducted on the IT database of IBM Research in Israel, showing that our approach is feasible.

Too Big to Eat: Boosting Analytics Data Ingestion from Object Stores with Scoop

Extracting value from data stored in object stores,such as OpenStack Swift and Amazon S3, can be problematicin common scenarios where analytics frameworks and objectstores run in physically disaggregated clusters. One of the mainproblems is that analytics frameworks must ingest large amountsof data from the object store prior to the actual computation;this incurs a significant resources and performance overhead. Toovercome this problem, we present Scoop. Scoop enables analyticsframeworks to benefit from the computational resources of objectstores to optimize the execution of analytics jobs. Scoop achievesthis by enabling the addition of ETL-type actions to the dataupload path and by offloading querying functions to the objectstore through a rich and extensible active object storage layer. Asa proof-of-concept, Scoop enables Apache Spark SQL selectionsand projections to be executed close to the data in OpenStackSwift for accelerating analytics workloads of a smart energy gridcompany (GridPocket). Our experiments in a 63-machine clusterwith real IoT data and SQL queries from GridPocket show thatScoop exhibits query execution times up to 30x faster than thetraditional “ingest-then-compute” approach.

Brief Announcement: A Consent Management Solution for Enterprises

Technologies such as cloud, mobile and the Internet of Things (IoT) are resulting in the collection of more and more personal data. While this sensitive data can be a gold mine for enterprises, it can also constitute a major risk for them. Legislation and privacy norms are becoming stricter when it comes to collecting and processing personal data, requiring the informed consent of individuals to process their data for specific purposes. However, IT solutions that can address these privacy issues are still lacking. We briefly outline our solution and its main component called “Consent Manager”, for the management, automatic enforcement and auditing of user consent. We then describe how the Consent Manager was adopted as part of the European FP7 project COSMOS.