Oleg Sukhoroslov

A Web-Based Platform for Publication and Distributed Execution of Computing Applications

Researchers increasingly rely on using web-based systems for accessing and running scientific applications across distributed computing resources. However existing systems lack a number of important features, such as publication and sharing of scientific applications as online services, decoupling of applications from computing resources and providing remote programmatic access. This paper presents Everest, a Web-based platform for researchers supporting publication, execution and composition of applications running across distributed computing resources. Everest addresses the described challenges by relying on modern Web technologies and cloud computing models. It follows the Platform as a Service (PaaS) cloud delivery model by providing all its functionality via remote Web and programming interfaces. Any application added to Everest is automatically published both as a user-facing Web form and a Web service. Another distinct feature of Everest is the ability to attach external computing resources by any user and flexibly use these resources for running applications. The paper provides an overview of the platforms architecture and its main components, describes recent developments, presents results of experimental evaluation of the platform and discusses remaining challenges.

MathCloud: Publication and Reuse of Scientific Applications as RESTful Web Services

The paper presents MathCloud platform which enables wide-scale sharing, publication and reuse of scientific applications as RESTful web services. A unified interface of computational web service based on REST architectural style is proposed. Main components of MathCloud platform including service container, service catalogue, workflow management system, and security mechanism are described. In contrast to other similar efforts based on WS-* specifications, the described platform provides a more lightweight solution with native support for modern Web applications. The platform has been successfully used in several applications from various fields of computational science that confirm the viability of proposed approach and software platform.

Everest: A Cloud Platform for Computational Web Services

The ability to effortlessly reuse and combine existing computational tools is an important factor influencing research productivity in many scientific domains. While the service-oriented approach proved to be essential in order to enable wide-scale sharing of applications, we argue that its full potential in scientific computing is still not realized. In this paper, we present Everest, a cloud platform that supports publication, sharing and reuse of scientific applications as web services. The underlying approach is based on a uniform representation of computational web services and its implementation using REST architectural style. In comparison with existing work, Everest has a number of novel features such as the use of PaaS model, flexible binding of services with externally provisioned computing resources and remotely accessible API.

An Experimental Study of Workflow Scheduling Algorithms for Heterogeneous Systems

The paper studies the efficiency of nine state-of-the-art algorithms for scheduling of workflow applications in heterogeneous computing systems (HCS). The comparison of algorithms is performed on the base of discrete-event simulation for a wide range of workflow and system configurations. The developed open source simulation framework based on SimGrid toolkit allowed us to perform a large number of experiments in a reasonable amount of time and to ensure reproducible results. The accuracy of the used network model helped to reveal drawbacks of simpler models commonly used for studying scheduling algorithms.

Implementation and Use of Coarse-grained Parallel Branch-and-bound in Everest Distributed Environment

Abstract This paper examines the coarse-grained approach to parallelization of the branch-and-bound (B&B) algorithm in a distributed computing environment. This approach is based on preliminary decomposition of a feasible domain of mixed-integer programming problem into a set of subproblems. The produced subproblems are solved in parallel by a distributed pool of standalone B&B solvers. The incumbent values found by individual solvers are intercepted and propagated to other solvers to speed up the traversal of B&B search tree. Presented implementation of the approach is based on SCIP, a non-commercial MINLP solver, and Everest, a web-based distributed computing platform. The implementation was tested on several mixed-integer programming problems and a noticeable speedup has been achieved. In the paper, results of a number of experiments with the Traveling Salesman Problem are presented.

Using Everest Platform for Teaching Parallel and Distributed Computing

The paper presents a practical approach for building high-level services for teaching parallel and distributed computing based on Everest platform. Originally designed for publication of computing applications, the platform is suitable for rapid development of services for running different types of parallel programs on high-performance resources, as well as services for evaluation of practical assignments. As was demonstrated by using Everest for teaching two introductory PDC courses, the proposed approach helps to enhance students’ practical experience while avoiding low-level interfaces and providing a level of automation necessary for scaling the course to a large number of students. In contrast to other solutions, the exploited Platform as a Service model provides the ability to quickly reuse this approach by other PDC educators without installation of the platform.

An experimental study of scheduling algorithms for many-task applications

The paper studies the performance of algorithms for scheduling of many-task applications in distributed computing systems. Two important classes of such applications are considered: bags-of-tasks and workflows. The comparison of algorithms is performed on the basis of discrete-event simulation for various application cases and system configurations. The developed simulation framework based on SimGrid toolkit provides the necessary tools for implementation of scheduling algorithms, generation of synthetic systems and applications, execution of simulation experiments and analysis of results. This allowed to perform a large number of experiments in a reasonable amount of time and to ensure reproducible results. The conducted experiments demonstrate the dependence of the performance of studied algorithms on various application and system characteristics. While confirming the performance advantage of advanced static algorithms, the presented results reveal some interesting insights. In particular, the accuracy of the used network model helped to demonstrate the limitations of simple analytical models for scheduling of data-intensive parallel applications with static algorithms.

Building web-based services for practical exercises in parallel and distributed computing

Abstract The paper presents an approach to the design and implementation of web-based environments for practical exercises in parallel and distributed computing (PDC). The presented approach introduces minimal development and operational costs by relying on Everest, a general-purpose platform for building computational web services. The flexibility of proposed service-oriented architecture enables the development of different types of services targeting various use cases and PDC topics. The generic execution services support the execution of different types of parallel and distributed programs on corresponding computing systems, while the assignment evaluation services implement the execution and evaluation of solutions to programming assignments. As was demonstrated by teaching two introductory PDC courses, the presented approach helps to enhance students’ practical experience while avoiding low-level interfaces, reducing the grading time and providing a level of automation necessary for scaling the course to a large number of students. In contrast to other efforts, the exploited Platform as a Service model provides the ability to quickly reuse this approach by other PDC educators without installation of the Everest platform.

Using Simulation to Improve Workflow Scheduling in Heterogeneous Computing Systems

Workflows is an important class of parallel applications that consist of many tasks with logical or data dependencies. A multitude of scheduling algorithms have been proposed to optimize the workflow execution in heterogeneous computing systems. However, in order to be efficiently applied in practice, these algorithms require accurate estimates of task execution and communication times. In this paper two modifications of the well-known HEFT algorithm are investigated that use simulation instead of simple analytical models in order to better estimate data transfer times. The results of experimental study show that the proposed approach can improve makespan for data-intensive workflows with high parallelism and communication-to-computation ratio.

A high-level toolkit for development of distributed scientific applications

The paper presents IARnet toolkit, a set of high-level tools and services simplifying integration of software resources into a distributed computing environment and development of distributed applications involving dynamic discovery and composition of resources. A case study of using IARnet for solving large scale discrete optimization problems is discussed.