Benhur de Oliveira Stein

Pajé, an interactive visualization tool for tuning multi-threaded parallel applications

Abstract This paper describes Paje, an interactive visualization tool for displaying the execution of parallel applications where a potentially large number of communicating threads of various life-times execute on each node of a distributed memory parallel system. Paje is capable of representing a wide variety of interactions between threads. The main characteristics of Paje, interactivity and scalability, are exemplified by the performance tuning of a molecular dynamics application. In order to be easily extensible, the architecture of the system was based on components which are connected in a data flow graph to produce a given visualization tool. Innovative components were designed, in addition to “classical” components existing in similar visualization systems, to support scalability and interactivity.

Visualisation of Distributed Applications for Performance Debugging

This paper presents a method to perform visualisations of the behaviour of distributed applications, for performance analysis and debugging. This method is applied to a Java distributed application. Application level traces are recorded without any modification of the monitored applications nor of the JVMs. Trace recording includes records from the JVM, through the JVMPI, and records from the OS, through the data structure associated to each process. Recorded traces are visualised post mortem, using the interactive Paje visualisation tool, which can be conveniently specialised to visualise the dynamic behaviour of distributed Java applications. Applying this method to the execution of a book server, we were able to observe a situation where both the computation or the communications could be at the origin of a lack of performances. The observation helped finding the origin of the problem coming in this case from the computation.

Pajé: An Extensible Environment for Visualizing Multi-threaded Programs Executions

PajE is an interactive visualization tool for displaying the execution of parallel applications where a (potentially) large number of communicating threads of various life-times execute on each node of a distributed memory parallel system. The main novelty of PajE is an original combination of three of the most desirable properties of visualization tools for parallel programs: extensibility, interactivity and scalability. This article mainly focuses on the extensibility property of PajE, ability to easily add new functionalities to the tool. PajE was designed as a data-flow graph of modular components to ease the replacement of existing modules or the implementation of new ones. In addition the genericity of PajE allows application programmers to tailor the visualization to their needs, by simply adding tracing orders to the programs being traced.

MAPREDUCE CHALLENGES ON PERVASIVE GRIDS

This study presents the advances on designing and implementing scalable techniques to support the development and execution of MapReduce application in pervasive distributed computing infrastructures, in the context of the PER-MARE project. A pervasive framework for MapReduce applications is very useful in practice, especially in those scientific, enterprises and educational centers which have many unused or underused computing resources, which can be fully exploited to solve relevant problems that demand large computing power, such as scientific computing applications, big data processing, etc. In this study, we pro-pose the study of multiple techniques to support volatility and heterogeneity on MapReduce, by applying two complementary approaches: Improving the Apache Hadoop middleware by including context-awareness and fault-tolerance features; and providing an alternative pervasive grid implementation, fully adapted to dynamic environments. The main design and implementation decisions for both alternatives are described and validated through experiments, demonstrating that our approaches provide high reliability when executing on pervasive environments. The analysis of the experiments also leads to several insights on the requirements and constraints from dynamic and volatile systems, reinforcing the importance of context-aware information and advanced fault-tolerance features to provide efficient and reliable MapReduce services on pervasive grids.

PER-MARE: Adaptive Deployment of MapReduce over Pervasive Grids

Map Reduce is a parallel programming paradigm successfully used to perform computations on massive amounts of data, being widely deployed on clusters, grid, and cloud infrastructures. Interestingly, while the emergence of cloud infrastructures has opened new perspectives, several enterprises hesitate to put sensible data on the cloud and prefer to rely on internal resources. In this paper we introduce the PER-MARE initiative, which aims at proposing scalable techniques to support existent Map Reduce data-intensive applications in the context of loosely coupled networks such as pervasive and desktop grids. By relying on the Map Reduce programming model, PER-MARE proposes to explore the potential advantages of using free unused resources available at enterprises as pervasive grids, alone or in a hybrid environment. This paper presents the main lines that orient the PER-MARE approach and some preliminary results.

Test case generation from BPMN models for automated testing of Web-based BPM applications

This article proposes an approach to generate test cases from BPMN models, for automated testing of Web applications implemented with the support of BPM suites. The work is primarily focused on functional testing and has the following objectives: (i) identify execution paths from the flow analysis in the BPMN model and (ii) generate the initial code of test scripts to be run on a given Web application testing tool. Throughout the article, we describe the design and implementation of a solution to achieve these goals, targeting automated tests using Selenium and Cucumber as tools. The approach was applied to processes from a public repository and was able to generate test scenarios from different BPMN models.

Automated refactorings for high performance Fortran programmes

Refactoring is a software engineering technique aimed at improving the design of software applications, without changing their external behaviour. Several refactorings have been proposed for object-oriented languages, but there are few related works focusing on procedural programming. Fortran is a procedural language heavily used in high performance computing, which is not fully explored considering refactoring support. In this paper, we describe a set of automated refactorings for Fortran based on the Photran plug-in, which is integrated with the Eclipse integrated development environment (IDE). We present a set of experiments to evaluate the impact of the proposed refactorings in third-party Fortran applications. The results show that the proposed refactorings improve the design of existing applications without compromising their performance.

SocialCount - Detecting Social Interactions on Mobile Devices.

With mobile devices increasingly powerful and accessible to the majority of the population, applications have begun to become increasingly intelligent, customizable and adaptable to users’ needs. To do this, contextaware applications are developed. In this work, we create an approach to infer social interactions through the identification of the user’s voice and to recognize their social context. Data from the social context of the user has been useful in many real-life situations, such as identifying and controlling infectious disease epidemics.

SQL Query Performance on Hadoop: An Analysis Focused on Large Databases of Brazilian Electronic Invoices.

Hadoop clusters have established themselves as a foundation for various applications and experiments in the field of high-performance processing of large datasets. In this context, SQL-on-Hadoop emerged as trend that combines the popularity of SQL with the performance of Hadoop. In this work, we analyze the performance of SQL queries on Hadoop, using the Impala engine, comparing it with a RDBMS-based approach. The analysis focuses on a large set of electronic invoice data, representing an important application to support fiscal audit operations. The experiments performed included frequent queries in this context, which were implemented with and without data partitioning in both RDBMS and Impala/Hadoop. The results show speedups from 2.7 to 14x with Impala/Hadoop for the queries considered, on a lower cost hardware/software platform.

Performance Evaluation of Cloud-based RDBMS through a Cloud Scripting Language.

Cloud computing has brought new opportunities, but also new concerns, for developing enterprise information systems. In this work, we investigated the performance of two cloud-based relational database services, accessing them via scripts which also execute on a cloud platform, using Google Apps Script technology. Preliminary results show little differences between the services in their trial versions, considering limitations imposed by the Google platform.