Mel Krokos

VisIVO-integrated tools and services for large-scale astrophysical visualization

VisIVO is an integrated suite of tools and services specifically designed for the Virtual Observatory. This suite constitutes a software framework for effective visual discovery in currently available (and next-generation) very large-scale astrophysical data sets. VisIVO consists of VisiVO Desktop, a stand alone application for interactive visualization on standard PCs; VisIVO Server, a grid-enabled platform for high performance visualization; and VisIVO Web, a custom designed web portal supporting services based on the VisIVO Server functionality. The main characteristic of VisIVO is support for high-performance, multidimensional visualization of very large-scale astrophysical data sets. Users can obtain meaningful visualizations rapidly while preserving full and intuitive control of the relevant visualization parameters. This paper focuses on newly developed integrated tools in VisIVO Server allowing intuitive visual discovery with three-dimensional (3D) views being created from data tables. VisIVO Server can be installed easily on any web server with a database repository. We discuss briefly aspects of our implementation of VisiVO Server on a computational grid and also outline the functionality of the services offered by VisIVO Web. Finally we conclude with a summary of our work and pointers to future developments.

Science gateway technologies for the astrophysics community

The availability of large‐scale digital surveys offers tremendous opportunities for advancing scientific knowledge in the astrophysics community. Nevertheless, the analysis of these data often requires very powerful computational resources. Science gateway technologies offer Web‐based environments to run applications with little concern for learning and managing the underlying infrastructures that execute them. This paper focuses on the issues related to the development of a science gateway customized for the needs of the astrophysics community. The VisIVO Science Gateway is wrapped around a WS‐PGRADE/grid User Support Environment portal integrating services for processing and visualizing large‐scale multidimensional astrophysical data sets on distributed computing infrastructures. We discuss the core tools and services supported including an application for mobile access to the gateway. We report our experiences in supporting specialized astrophysical communities requiring development of complex workflows for visualization and numerical simulations. Further, available platforms are discussed for sharing workflows in collaborative environments. Finally, we outline our vision for creating a federation of science gateways to benefit astrophysical communities by sharing a set of services for authentication, computing infrastructure access and data/workflow repositories. Copyright

High-performance astrophysical visualization using Splotch

The scientific community is presently witnessing an unprecedented growth in the quality and quantity of data sets coming from simulations and real-world experiments. To access effectively and extract the scientific content of such large-scale data sets (often sizes are measured in hundreds or even millions of Gigabytes) appropriate tools are needed. Visual data exploration and discovery is a robust approach for rapidly and intuitively inspecting large-scale data sets, e.g. for identifying new features and patterns or isolating small regions of interest within which to apply time-consuming algorithms. This paper presents a high performance parallelized implementation of Splotch, our previously developed visual data exploration and discovery algorithm for large-scale astrophysical data sets coming from particle-based simulations. Splotch has been improved in order to exploit modern massively parallel architectures, e.g. multicore CPUs and CUDA-enabled GPUs. We present performance and scalability benchmarks on a number of test cases, demonstrating the ability of our high performance parallelized Splotch to handle efficiently large-scale data sets, such as the outputs of the Millennium II simulation, the largest cosmological simulation ever performed.

An Innovative Science Gateway for the Cherenkov Telescope Array

The Cherenkov Telescope Array (CTA) is currently building the next generation, ground-based, very high-energy gamma-ray instrumentation. CTA is expected to collect very large datasets (in the order of petabytes) which will have to be stored, managed and processed. This paper presents a graphical user interface built inside a science gateway aiming at providing CTA-users with a common working framework. The gateway is WS-PGRADE/gUSE workflow-oriented and is equipped with a flexible SSO (based on SAML) to control user access for authentication and authorization. An interactive desktop environment is provided, called Astronomical & Physics Cloud Interactive Desktop (ACID). Users are able to exploit the graphical interface as provided natively by the tools included in ACID. A cloud data service shares and synchronizes data files and output results between the user desktop and the science gateway. Our solution is a first attempt towards an ecosystem of new technologies with a high level of flexibility to suit present and future requirements of the CTA community.

VisIVOWeb: A WWW Environment for Large-Scale Astrophysical Visualization

ABSTRACT. This article presents a newly developed Web portal called VisIVOWeb that aims to provide the astrophysical community with powerful visualization tools for large-scale data sets in the context of Web 2.0. VisIVOWeb can effectively handle modern numerical simulations and real-world observations. Our open-source software is based on established visualization toolkits offering high-quality rendering algorithms. The underlying data management is discussed with the supported visualization interfaces and movie-making functionality. We introduce VisIVOWeb Network, a robust network of customized Web portals for visual discovery, and VisIVOWeb Connect, a lightweight and efficient solution for seamlessly connecting to existing astrophysical archives. A significant effort has been devoted for ensuring interoperability with existing tools by adhering to IVOA standards. We conclude with a summary of our work and a discussion on future developments.

VisIVO Workflow-Oriented Science Gateway for Astrophysical Visualization

Nowadays visualization-based knowledge discovery can play an important role in astrophysics. Collaborative visualization can enable multiple users to share visualization experiences, e.g. by interacting simultaneously with astrophysical datasets giving feedback on what other participants are doing/seeing. Further, workflow-driven applications allow reproduction of specific visualization results, a challenging task as selecting suitable visualization parameters may not be a straightforward process. This paper presents VisIVO Science Gateway, a web-based workflow-enabled framework integrating large-scale, multidimensional datasets and applications for visualization and data filtering on Distributed Computing Infrastructures (DCIs). Advanced users are able to create, change, invoke, and monitor workflows while standard users are provided with easy-to-use customised web interfaces hiding all technical aspects of the visualization algorithms and DCI configurations.

Creating Gateway Alliances Using WS-PGRADE/gUSE

The STARnet Gateway Federation is a unique example of a federated network of science gateways based on WS-PGRADE/gUSE technologies, and explicitly designed and tuned to the needs of the astronomical and astrophysical (A&A) community in Europe. The use of a federated gateway infrastructure allows scientists to explore new collaboration opportunities and advancing the scientific research activity within A&A. STARnet Gateways share a common authentication system, a distributed computing infrastructure, data archives, portlets, and workflow repositories. Building upon these technologies, a number of challenging applications from different A&A domains have been successfully prototyped and tested.

GPU accelerated particle visualization with Splotch

Splotch is a rendering algorithm for exploration and visual discovery in particle-based datasets coming from astronomical observations or numerical simulations. The strengths of the approach are production of high quality imagery and support for very large-scale datasets through an effective mix of the OpenMP and MPI parallel programming paradigms. This article reports our experiences in re-designing Splotch for exploiting emerging HPC architectures nowadays increasingly populated with GPUs. A performance model is introduced to guide our re-factoring of Splotch. A number of parallelization issues are discussed, in particular relating to race conditions and workload balancing, towards achieving optimal performances. Our implementation was accomplished by using the CUDA programming paradigm. Our strategy is founded on novel schemes achieving optimized data organization and classification of particles. We deploy a reference cosmological simulation to present performance results on acceleration gains and scalability. We finally outline our vision for future work developments including possibilities for further optimizations and exploitation of hybrid systems and emerging accelerators.

Towards a big data exploration framework for astronomical archives

Exploiting big data astronomical archives is a mandatory and challenging activity due to dramatically increasing sizes and high complexity of datasets coming from radio telescopes or space missions. Visual exploration and discovery can be invaluable tools providing prompt and intuitive insights into the intrinsic data characteristics, enabling scientists to rapidly identify interesting areas within which to apply computationally expensive algorithms or to discover correlations in data patterns. The paper outlines a new approach for creating a user-friendly, integrated and cross-platform framework to facilitate big data access, visualization and exploration, thus empowering astrophysicists to focus on pitching new ideas for scientific advances. We present a flexible distributed architecture striking a balance between local interactive exploration tools and remote services responsible for hiding data complexity. Remote services communicate with advanced distributed computing infrastructures presenting a meaningful lightweight version of the archive dataset obtained by mining or noise filtering methods. They are interfaced with science gateway technologies in order to allow collaborative activity between users and to provide customization and scalability of data analysis/processing workflows hiding underlying technicalities. Local tools enable interactive visualization optimized for ubiquitous computing environments, intuitively controlling the resulting visualisation. The motivations behind such a framework are envisaged to meet the requirements of the exploitation of the Gaia mission outcomes and are shown in the paper by a number of case studies. The presented framework can potentially have a profound impact on astronomical and astrophysical communities in the big data era, allowing to quickly understand datasets, thus aiding in adopting novel ways for scientific discovery.

Visual Discovery in Large-Scale Astrophysical Datasets; Experiences Using the Sloan Digital Sky Survey

Nowadays astronomers are experiencing an unprecedented growth in the quality and quantity of datasets coming from numerical simulations and real-world observations. For example, the increasing availability of high performance computing facilities has given the possibility to perform large-scale simulations of several dimensions. Also, forthcoming astronomical surveys are expected to collect datasets of several petabytes. The emerging need is thus for efficient visual discovery tools for rapid inspection to identify regions of interest in large-scale datasets prior to applying computationally expensive data analysis algorithms. This paper reports our experiences in developing visual discovery tools for the Sloan Digital Sky Survey (SDSS), the most ambitious astronomical survey ever undertaken. We present existing tools and visualization requirements collected from SDSS users for new functionality. We then discuss a range of newly developed visual discovery tools and their applicability to SDSS and finally we conclude with pointers to future developments.