Wes Lloyd

A software engineering perspective on environmental modeling framework design: The Object Modeling System

The environmental modeling community has historically been concerned with the proliferation of models and the effort associated with collective model development tasks (e.g., code generation, data transformation, etc.). Environmental modeling frameworks (EMFs) have been developed to address this problem, but much work remains before EMFs are adopted as mainstream modeling tools. Environmental model development requires both scientific understanding of environmental phenomena and software developer proficiency. EMFs support the modeling process through streamlining model code development, allowing seamless access to data, and supporting data analysis and visualization. EMFs also support aggregation of model components into functional units, component interaction and communication, temporal-spatial stepping, scaling of spatial data, multi-threading/multi-processor support, and cross-language interoperability. Some EMFs additionally focus on high-performance computing and are tailored for particular modeling domains such as ecosystem, socio-economic, or climate change research. The Object Modeling System Version 3 (OMS3) EMF employs new advances in software framework design to better support the environmental model development process. This paper discusses key EMF design goals/constraints and addresses software engineering aspects that have made OMS3 framework development efficacious and its application practical, as demonstrated by leveraging software engineering efforts outside of the modeling community and lessons learned from over a decade of EMF development. Software engineering approaches employed in OMS3 are highlighted including a non-invasive lightweight framework design supporting component-based model development, use of implicit parallelism in system design, use of domain specific language design patterns, and cloud-based support for computational scalability. The key advancements in EMF design presented herein may be applicable and beneficial for other EMF developers seeking to better support environmental model development through improved framework design.

Performance implications of multi-tier application deployments on Infrastructure-as-a-Service clouds: Towards performance modeling

Hosting a multi-tier application using an Infrastructure-as-a-Service (IaaS) cloud requires deploying components of the application stack across virtual machines (VMs) to provide the applications infrastructure while considering factors such as scalability, fault tolerance, performance and deployment costs (# of VMs). This paper presents results from an empirical study which investigates implications for application performance and resource requirements (CPU, disk and network) resulting from how multi-tier applications are deployed to IaaS clouds. We investigate the implications of: (1) component placement across VMs, (2) VM memory size, (3) VM hypervisor type (KVM vs. Xen), and (4) VM placement across physical hosts (provisioning variation). All possible deployment configurations for two multi-tier application variants are tested. One application variant was computationally bound by the application middleware, the other bound by geospatial queries. The best performing deployments required as few as 2 VMs, half the number required for VM-level service isolation, demonstrating potential cost savings when components can be consolidated. Resource utilization (CPU time, disk I/O, and network I/O) varied with component deployment location, VM memory allocation, and the hypervisor used (Xen or KVM) demonstrating how application deployment decisions impact required resources. Isolating application components using separate VMs produced performance overhead of ~1%-2%. Provisioning variation of VMs across physical hosts produced overhead up to 3%. Relationships between resource utilization and performance were assessed using multiple linear regression to develop a model to predict application deployment performance. Our model explained over 84% of the variance and predicted application performance with mean absolute error of only ~0.3 s with CPU time, disk sector reads, and disk sector writes serving as the most powerful predictors of application performance.

Environmental modeling framework invasiveness: Analysis and implications

Environmental modeling frameworks support scientific model development by providing model developers with domain specific software libraries which are used to aid model implementation. This paper presents an investigation on the framework invasiveness of environmental modeling frameworks. Invasiveness, similar to object-oriented coupling, is defined as the quantity of dependencies between model code and a modeling framework. We investigated relationships between invasiveness and the quality of modeling code, and also the utility of using a lightweight framework design approach in an environmental modeling framework. Five metrics to measure framework invasiveness were proposed and applied to measure dependencies between model and framework code of several implementations of Thornthwaite and the Precipitation-Runoff Modeling System (PRMS), two well-known hydrological models. Framework invasiveness measures were compared with existing common software metrics including size (lines of code), cyclomatic complexity, and object-oriented coupling. Models with lower framework invasiveness tended to be smaller, less complex, and have less coupling. In addition, the lightweight framework implementations of the Thornthwaite and PRMS models were less invasive than the traditional framework model implementations. Our results show that model implementations with higher degrees of framework invasiveness also had structural characteristics which previously have been shown to predict poor maintainability, a non-functional code quality attribute of concern. We conclude that using a framework with a lightweight framework design shows promise in helping to improve the quality of model code and that the lightweight framework design approach merits further attention by environmental modeling framework developers.

Migration of Multi-tier Applications to Infrastructure-as-a-Service Clouds: An Investigation Using Kernel-Based Virtual Machines

To investigate challenges of multi-tier application migration to Infrastructure-as-a-Service (IaaS) clouds we performed an experimental investigation by deploying a processor bound and input-output bound variant of the RUSLE2 erosion model to an IaaS based private cloud. Scaling the applications to achieve optimal system throughput is complex and involves much more than simply increasing the number of allotted virtual machines (VMs). While scaling the application variants a series of bottlenecks were encountered unique to an applications processing, I/O, and memory requirements, herein referred to as an applications profile. To investigate the impact of provisioning variation for hosting multi-tier applications we tested four schemes of VM deployments across the physical nodes of our cloud. Performance degradation was more pronounced when multiple I/O or CPU resource intensive application components were co-located on the same physical hardware. We investigated the virtualization overhead incurred using Kernel-based virtual machines (KVM) by deploying our application variants to both physical and virtual machines. Overhead varied based on the unique characteristics of each applications profile. We observed ~112% overhead for the input/output bound application and just ~ 10% overhead for the processor bound application. Understanding an applications profile was found to be important for optimal IaaS-based cloud migration and scaling.

Performance Modeling to Support Multi-tier Application Deployment to Infrastructure-as-a-Service Clouds

Infrastructure-as-a-service (IaaS) clouds support migration of multi-tier applications through virtualization of diverse application stack(s) of components which may require various operating systems and environments. To maximize performance of applications deployed to IaaS clouds while minimizing deployment costs, it is necessary to create virtual machine images to host application components with consideration for component dependencies that may affect load balancing of physical resources of VM hosts including CPU time, disk and network bandwidth. This paper presents results of an investigation utilizing physical machine (PM) and virtual machine (VM) resource utilization statistics to build performance models to predict application performance and rank performance of application component deployment configurations deployed across VMs. Our objective was to predict which component compositions provide best performance while requiring the fewest number of VMs. Eighteen individual resource utilization statistics were investigated for use as independent variables to predict service execution time using four different modeling approaches. Overall CPU time was the strongest predictor of execution time. The strength of individual predictors varied with respect to the resource utilization profiles of the applications. CPU statistics including idle time and number of context switches were good predictors when the test application was more disk I/O bound, while disk I/O statistics were better predictors when the application was more CPU bound. All performance models built were effective at determining the best performing service composition deployments validating the utility of our approach.

Demystifying the Clouds: Harnessing Resource Utilization Models for Cost Effective Infrastructure Alternatives

Deployment of service oriented applications (SOAs) to public infrastructure-as-a-service (IaaS) clouds presents challenges to system analysts. Public clouds offer an increasing array of virtual machine types with qualitatively defined CPU, disk, and network I/O capabilities. Determining cost effective application deployments requires selecting both the quantity and type of virtual machine (VM) resources for hosting SOA workloads of interest. Hosting decisions must utilize sufficient infrastructure to meet service level objectives and cope with service demand. To support these decisions, analysts must: (1) understand how their SOA behaves in the cloud; (2) quantify representative workload(s) for execution; and (3) support service level objectives regardless of the performance limits of the hosting infrastructure. In this paper we introduce a workload cost prediction methodology which harnesses operating system time accounting principles to support equivalent SOA workload performance using alternate virtual machine types. We demonstrate how the use of resource utilization checkpointing supports capturing the total resource utilization profile for SOA workloads executed across a pool of VMs. Given these workload profiles, we develop and evaluate our cost prediction methodology using six SOAs. We demonstrate how our methodology can support finding alternate infrastructures that afford lower hosting costs while offering equal or better performance using any VM type on Amazons public elastic compute cloud.

Mitigating Resource Contention and Heterogeneity in Public Clouds for Scientific Modeling Services

Abstraction of physical hardware using infrastructure-as-a-service (IaaS) clouds leads to the simplistic view that resources are homogeneous and that infinite scaling is possible with linear increases in performance. Hosting scientific modeling services using IaaS clouds requires awareness of application resource requirements and careful management of cloud-based infrastructure. In this paper, we present multiple methods to improve public cloud infrastructure management to support hosting scientific model services. We investigate public cloud VM-host heterogeneity and noisy neighbor detection to inform VM trial-and-better selection to favor worker VMs with better placements in public clouds. We present a cpuSteal noisy neighbor detection method (NN-Detect) which harnesses the cpuSteal CPU metric to identify worker VMs with resource contention from noisy neighbors. We evaluate potential performance improvements provided from leveraging these techniques in support of providing modeling-as-a-service for two environmental science models.

A minimally invasive model data passing interface for integrating legacy environmental system models

This paper presents the Model Data Passing Interface (MODPI). The approach provides fine-grained, multidirectional feedbacks between legacy environmental system models through read and write access to relevant model data during simulation using a bidirectional, event-based, publish-subscribe system with a message broker. MODPI only requires commented directives in the original code and an XML linkage file with an optional custom data conversion module. Automated code generation, compilation, and execution reduce the programming burden on the modeler. Case study results indicated that MODPI required less code modifications within each model code base both before and after automated code generation, outperforming a baseline subroutine approach. Performance overhead for MODPI was minimal for the use case, offering speedup in some cases through parallel execution. MODPI is much less invasive than other techniques, potentially encouraging adoption by the modeling community in addition to maintainability and reusability of integrated model code. Display Omitted We developed a minimally invasive model data passing interface.MODPI requires very minimal modifications to original model code bases.MODPI uses code generation to minimize the programming work.MODPI provides fine-grained, multidirectional feedback between models.We provide analysis of computational overhead concerns.