VMShield: Memory Introspection-Based Malware Detection to Secure Cloud-Based Services Against Stealthy Attacks

Abstract

With the rapid evolution of the industrial Internet, cloud service has emerged as a next-generation industrial standard that has the potential to revolutionize and transform the enterprise industry. In recent years, numerous enterprises have acknowledged the benefits of cloud-based service models. However, the security issues are a major concern, such as stealthy malware attacks against virtual domains. In this article, we propose an introspection based security approach, called VMShield for securing virtual domains in a cloud based service platform, which is designed to detect malware in cloud infrastructure. VMShield performs virtual memory introspection from the hypervisor (trusted-domain) to collect the run-time behavior of processes, making it impossible for the malware to evade the security tool. The use of introspection makes the proposed approach a better choice over traditional static and dynamic state-of-the-art techniques which fail to detect stealthy attacks. The VMShield extracts the system call features using Bag of n-gram approach and selects important features using the meta-heuristic algorithm, binary particle swarm optimization. Random Forest (RF) classifier is used to classify the monitored programs into benign and malign processes, making it capable of detecting the variants of malware thus, an advantage over the typical signature-matching approach. The University of New Mexico (UNM) Dataset and Bare cloud Dataset (University of California) has been used for the demonstration and validation of VMShield. The results prove that VMShield achieves a higher attack detection rate and reduced storage compared to previously proposed techniques.