Jeongseob Ahn

Architectural support for secure virtualization under a vulnerable hypervisor

Although cloud computing has emerged as a promising future computing model, security concerns due to malicious tenants have been deterring its fast adoption. In cloud computing, multiple tenants may share physical systems by using virtualization techniques. In such a virtualized system, a software hypervisor creates virtual machines (VMs) from the physical system, and provides each user with an isolated VM. However, the hypervisor, with a full control over hardware resources, can access the memory pages of guest VMs without any restriction. By compromising the hypervisor, a malicious user can access the memory contents of the VMs used by other users. In this paper, we propose a hardware-based mechanism to protect the memory of guest VMs from unauthorized accesses, even with an untrusted hypervisor. With this mechanism, memory isolation is provided by the secure hardware, which is much less vulnerable than the software hypervisor. The proposed mechanism extends the current hardware support for memory virtualization with a small extra hardware cost. The hypervisor can still flexibly allocate physical memory pages to virtual machines for efficient resource management. However, the hypervisor can update nested page tables only through the secure hardware mechanism, which verifies each mapping change. Using the hardware-oriented mechanism in each system securing guest VMs under a vulnerable hypervisor, this paper also proposes a cloud system architecture, which supports the authenticated launch and migration of guest VMs.

Subspace snooping: filtering snoops with operating system support

Although snoop-based coherence protocols provide fast cache-to-cache transfers with a simple and robust coherence mechanism, scaling the protocols has been difficult due to the overheads of broadcast snooping. In this paper, we propose a coherence filtering technique called subspace snooping, which stores the potential sharers of each memory page in the page table entry. By using the sharer information in the page table entry, coherence transactions for a page generate snoop requests only to the subset of nodes in the system (subspace). However, the coherence subspace of a page may evolve, as the phases of applications may change or the operating system may migrate threads to different nodes. To adjust subspaces dynamically, subspace snooping supports a shrinking mechanism, which removes obsolete nodes from subspaces. Subspace snooping can be integrated to any type of coherence protocols and network topologies. As subspace snooping guarantees that a subspace always contains the precise sharers of a page, it does not restrict the designs of coherence protocols and networks. We evaluate subspace snooping with Token Coherence on un-ordered mesh networks. For scientific and server applications on a 16-core system, subspace snooping reduces 44% of snoops on average.

Revisiting hardware-assisted page walks for virtualized systems

Recent improvements in architectural supports for virtualization have extended traditional hardware page walkers to traverse nested page tables. However, current two-dimensional (2D) page walkers have been designed under the assumption that the usage patterns of guest and nested page tables are similar. In this paper, we revisit the architectural supports for nested page table walks to incorporate the unique characteristics of memory management by hypervisors. Unlike page tables in native systems, nested page table sizes do not impose significant overheads on the overall memory usage. Based on this observation, we propose to use flat nested page tables to reduce unnecessary memory references for nested walks. A competing mechanism to HW 2D page walkers is shadow paging, which duplicates guest page tables but provides direct translations from guest virtual to system physical addresses. However, shadow paging has been suffering from the overheads of synchronization between guest and shadow page tables. The second mechanism we propose is a speculative shadow paging mechanism, called speculative inverted shadow paging, which is backed by non-speculative flat nested page tables. The speculative mechanism provides a direct translation with a single memory reference for common cases, and eliminates the page table synchronization overheads. We evaluate the proposed schemes with the real Xen hypervisor running on a full system simulator. The flat page tables improve a state-of-the-art 2D page walker with a page walk cache and nested TLB by 7%. The speculative shadow paging improves the same 2D page walker by 14%.

Micro-Sliced Virtual Processors to Hide the Effect of Discontinuous CPU Availability for Consolidated Systems

Although time-sharing CPUs has been an essential technique to virtualize CPUs for threads and virtual machines, most of the commercial operating systems and hyper visors maintain relatively coarse-grained time slices to mitigate the costs of context switching. However, the proliferation of system virtualization poses a new challenge for the coarse-grained time sharing techniques, since operating systems are running on virtual CPUs. The current system stack was designed under the assumption that operating systems can seize CPU resources at any moment. However, for the guest operating system on a virtual machine (VM), such assumption cannot be guaranteed, since virtual CPUs of VMs share limited physical cores. Due to the time-sharing of physical cores, the execution of a virtual CPU is not contiguous, with a gap between the virtual and real time spaces. Such a virtual time discontinuity problem leads to significant inefficiency for lock and interrupt handling, which rely on the immediate availability of CPUs whenever the operating system requires computation. This paper investigates the impact of virtual time discontinuity problem for lock and interrupt handling in guest operating systems. To reduce the gap between virtual and physical time spaces, the paper proposes to shorten time slices for CPU virtualization to reduce scheduling latencies of virtual CPUs. However, shortening time slices may lead to the increased overhead of context switching costs across virtual machines. We explore the design space of architectural solutions to reduce context switching overheads with low-cost context-aware cache insertion policies combined with a state-of-the-art context prefetcher.

The effect of multi-core on HPC applications in virtualized systems

In this paper, we evaluate the overheads of virtualization in commercial multicore architectures with shared memory and MPI-based applications. We find that the non-uniformity of memory latencies affects the performance of virtualized systems significantly. Due to the lack of support for non-uniform memory access (NUMA) in the Xen hypervisor, shared memory applications suffer from a significant performance degradation by virtualization. MPI-based applications show more resilience on sub-optimal NUMA memory allocation and virtual machine (VM) scheduling. However, using multiple VMs on a physical system for the same instance of MPI applications may adversely affect the overall performance, by increasing I/O operations through the domain 0 VM. As the number of cores increases on a chip, the cache hierarchy and external memory will become more asymmetric. As such non-uniformity in memory systems increases, NUMA and cache awareness in VM scheduling will be critical for shared memory applications.

H-SVM: Hardware-Assisted Secure Virtual Machines under a Vulnerable Hypervisor

With increasing demands on cloud computing, protecting guest virtual machines (VMs) from malicious attackers has become critical to provide secure services. The current cloud security model with software-based virtualization relies on the invulnerability of the software hypervisor and its trustworthy administrator with the root permission. However, compromising the hypervisor with remote attacks or root permission grants the attackers with a full access capability to the memory and context of a guest VM. This paper proposes a HW-based approach to protect guest VMs even under an untrusted hypervisor. With the proposed mechanism, memory isolation is provided by the secure hardware, which is much less vulnerable than the software hypervisor. The proposed mechanism extends the current hardware support for memory virtualization based on nested paging with a small extra hardware cost. The hypervisor can still flexibly allocate physical memory pages to virtual machines for efficient resource management. In addition to the system design for secure virtualization, this paper presents a prototype implementation using system management mode. Although the current system management mode is not intended for security functions and thus limits the performance and complete protection, the prototype implementation proves the feasibility of the proposed design.

Fast Two-Level Address Translation for Virtualized Systems

Recently, there have been several improvements in architectural supports for two-level address translation for virtualized systems. However, those improvements including HW-based two-dimensional (2D) page walkers have extended the traditional multi-level page tables, without considering the memory management characteristics of virtual machines. This paper exploits the unique behaviors of the hypervisor, and proposes three new nested address translation schemes for virtualized systems. The first scheme called nested segmentation is designed for static memory allocation, and uses HW segmentation to map the VM memory directly to large contiguous memory regions. The second scheme proposes to use a flat nested page table for each VM, reducing memory accesses by the current 2D page walkers. The third scheme uses speculative inverted shadow paging, backed by non-speculative flat nested page tables. The speculative mechanism provides direct translation with a single memory reference for common cases without page table synchronization overheads. We evaluate the proposed schemes with the Xen hypervisor running on a full system simulator. Nested segmentation can reduce the overheads of two-level translation significantly for a certain cloud computing model. The nested segmentation, flat page tables, and speculative shadowing improve a state-of-the-art 2D page walker by 10, 7, and 14 percent respectively.

Accelerating critical OS services in virtualized systems with flexible micro-sliced cores

Consolidating multiple virtual machines into a single server has been widely adopted in cloud computing to improve system utilization. However, the sharing of physical CPUs among virtual machines in consolidated systems poses a new challenge in providing an illusion of continuous CPU execution to the guest operating systems (OS). Due to the time-sharing of physical CPUs, the execution of a guest OS is periodically interrupted, while the guest OS may not be aware of the discontinuity of virtual time against the real time. The virtual time discontinuity problem causes the delayed processing of critical OS operations, such as interrupt handling and lock processing. Although there have been several prior studies to mitigate the problem, they address only a subset of symptoms, require the modification of guest OSes, or change the processor architecture. This paper proposes a novel way to comprehensively reduce the inefficiency of guest OS execution in consolidated systems. It migrates the short-lived critical OS tasks to dedicated micro-sliced cores, minimizing the delays caused by time sharing. The hypervisor identifies the critical OS tasks without any OS intervention, and schedules the critical code sections onto the dynamically partitioned cores at runtime. The dedicated micro-sliced cores employ a short sub-millisecond quantum to minimize the response latencies for consolidated virtual machines. By readily servicing the critical tasks, the proposed scheme can minimize the adverse artifact of virtual machine consolidation without any modification of guest OSes.

Deduplicating TLB entries for shared pages

Although the page sharing among different address spaces can effectively reduce the memory footprint, the corresponding address mappings still require their own TLB entries. Consequently, redundant address mappings for the shared pages reside in TLBs. Our goal in this study is to increase the TLB density by deduplicating redundant copies of address mappings into a single mapping. In virtualized servers, the case of hosting the same guest operating systems, the same third-party libraries, and the same application is commonly found. Such environments generate the multiple identical address mappings, virtual to physical address, for shared pages among different virtual machines. We exploit this unique characteristic of the virtual memory layout being analogous in the same execution environment, and explore TLB deduplication, called DeTLB, which merges redundant virtual to physical address mappings of different address spaces into a single TLB entry. To demonstrate the performance impact, we develop a QEMU based trace simulator and evaluate the number of TLB misses by running an Apache Spark ALS and a microbenchmark with the Linux KSM module. The results indicate that DeTLB can effectively reduce the number of TLB misses for both data and instructions.

Subspace Snooping: Exploiting Temporal Sharing Stability for Snoop Reduction

Although snoop-based coherence protocols provide fast cache-to-cache transfers with a simple and robust coherence mechanism, scaling the protocols has been difficult due to the overheads of broadcast snooping. In this paper, we propose a coherence filtering technique called subspace snooping, which stores the potential sharers of each memory page in the page table entry. By using the sharer information in the page table entry, coherence transactions for a page generate snoop requests only to the subset of nodes in the system. However, the coherence subspace of a page may evolve, as the phases of applications may change or the operating system may migrate threads to different nodes. To adjust subspaces dynamically, subspace snooping supports two different shrinking mechanisms, which remove obsolete nodes from subspaces. Among the two shrinking mechanisms, subspace snooping with safe shrinking can be integrated to any type of coherence protocols and network topologies, as it guarantees that a subspace always contains the precise sharers of a page. Speculative shrinking breaks the subspace superset property, but achieves better snoop reductions than safe shrinking. We evaluate subspace snooping with Token Coherence on unordered mesh networks. Subspace snooping reduces 58 percent of snoops on average for a set of parallel scientific and server workloads, and 87 percent for our multiprogrammed workloads.