Luca Morandini

The Australia urban research gateway

The

A data-driven urban research environment for Australia

20m Australian Urban Research Infrastructure Network (AURIN) project (www.aurin.org.au) began in July 2010. AURIN has been tasked with developing a secure, Web‐based virtual environment (e‐Infrastructure) offering seamless, secure access to diverse, distributed and extremely heterogeneous data sets from numerous agencies with an extensive portfolio of targeted analytical and visualization tools. This is being provisioned for Australia‐wide urban and built environment researchers – itself a highly heterogeneous collection of research communities with diverse demands, through a unified urban research gateway. This paper describes these demands and how the e‐Infrastructure and gateway is being designed and implemented to accommodate this diversity of requirements, both from the user/researcher perspective and from the data provider perspective. The scaling of the infrastructure is presented and the way in which it copes with the spectrum of big data challenges (volume, veracity, variability and velocity) and associated big data analytics. The utility of the e‐Infrastructure is also demonstrated through a range of scenarios illustrating and reflecting the interdisciplinary urban research now possible. Copyright

The design and benchmarking of a Cloud-based platform for processing and visualization of traffic data

The Australian Urban Research Infrastructure Network (AURIN) project (www.aurin.org.au) is tasked with developing an e-Infrastructure to support urban and built environment research across Australia. As identified in [1], this e-Infrastructure must provide seamless access to highly distributed and heterogeneous data sets from multiple organisations with accompanying analytical and visualization capabilities. The project is tasked with delivering a secure, web-based unifying environment offering a one-stop-shop for Australia-wide urban and built environment research. This paper describes the architectural design and implementation of the AURIN data-driven e-Infrastructure, where data is not just a passive entity that is accessed and used as a consequence of research demand, but is instead, directly shaping the computational access, processing and intelligent utilization possibilities. This is demonstrated in a situational context.

SMASH: A Cloud-Based Architecture for Big Data Processing and Visualization of Traffic Data

Nowadays, more and more data are being generated and collected. This is especially so in our cities. Big data as a popular research topic has brought about a range of new challenges that must to be tackled to support both commercial and research demands. The transport area is one example that has much to benefit from big data capabilities in allowing to process voluminous amounts of data that is created in real time and in vast quantities. Tackling these big data issues requires capabilities not typically found in common Cloud platforms. The high speed of production of data (velocity) of high volume data demands a distributed file system for storing and duplicating data; high performance computing engines capable of processing such large quantities of data in real time; a reliable database system able to optimize the indexing and querying of the data, and essentially a flexible implementation for scaling the usage of computation resources based on the bursty nature of data production. In this paper, we present and benchmark SMASH, a generic and highly scalable Cloud-based solution. We focus here on the implementation and especially on the utilization of the SMASH software stack and how it can be used to process large scale traffic data - although we note that the solution can be applied to other high velocity big data processing areas.

Privacy Preserving Geo-Linkage in the Big Urban Data Era

In recent times, big data has become a popular research topic and brought about a range of new challenges that must be tackled to support many commercial and research demands. The transport arena is one example that has much to benefit from big data capabilities in allowing to process voluminous amounts of data that is created in real time and in vast quantities. Tackling these big data issues requires capabilities not typically found in common Cloud platforms. This includes a distributed file system for capturing and storing data, a high performance computing engine able to process such large quantities of data, a reliable database system able to optimize the indexing and querying of the data, and geospatial capabilities to visualize the resultant analyzed data. In this paper we present SMASH, a generic and highly scalable Cloud-based architecture and its implementation that meets these many demands. We focus here specifically on the utilization of the SMASH software stack to process large scale traffic data for Adelaide and Victoria although we note that the solution can be applied to other big data processing areas. We provide performance results on SMASH and compare it with other big data solutions that have been developed.

The design of a flexible web-based analytical platform for urban research

Big data technologies and a range of Government open data initiatives provide the basis for discovering new insights into cities; how they are planned, how they managed and the day-to-day challenges they face in health, transport and changing population profiles. The Australian Urban Research Infrastructure Network (AURIN – www.aurin.org.au) project is one example of such a big data initiative that is currently running across Australia. AURIN provides a single gateway providing online (live) programmatic access to over 2000 data sets from over 70 major and typically definitive data-driven organizations across federal and State government, across industry and across academia. However whilst open (public) data is useful to bring data-driven intelligence to cities, more often than not, it is the data that is not-publicly accessible that is essential to understand city challenges and needs. Such sensitive (unit-level) data has unique requirements on access and usage to meet the privacy and confidentiality demands of the associated organizations. In this paper we highlight a novel geo-privacy supporting solution implemented as part of the AURIN project that provides seamless and secure access to individual (unit-level) data from the Department of Health in Victoria. We illustrate this solution across a range of typical city challenges in localized contexts around Melbourne. We show how unit level data can be combined with other data in a privacy-protecting manner. Unlike other secure data access and usage solutions that have been developed/deployed, the AURIN solution allows any researcher to access and use the data in a manner that meets all of the associated privacy and confidentiality concerns, without obliging them to obtain ethical approval or any other hurdles that are normally put in place on access to and use of sensitive data. This provides a paradigm shift in secure access to sensitive data with geospatial content.