D. M. Lindholm

TSDS: high-performance merge, subset, and filter software for time series-like data

Time Series Data Server (TSDS) is a software package for implementing a server that provides fast super-setting, sub-setting, filtering, and uniform gridding of time series-like data. TSDS was developed to respond quickly to requests for long time spans of data. Data may be served from a fast database, typically created by aggregating granules (e.g., data files) from a remote data source and storing them in a local cache that is optimized for serving time series. The system was designed specifically for time series data, and is optimized for requests where the longest dimension of the requested data structure is time. Scalar, vector, and spectrogram time series types are supported. The user can interact with the server by requesting a time series, a date range, and an optional filter to apply to the data. Available filters include strides, block average/minimum/maximum, exclude, and inequality. Constraint expressions are supported, which allow such operations as a request for data from one time series when a different time series satisfied a specified relationship. TSDS builds upon DAP (Data Access Protocol), NcML (netCDF Mark-up language) and related software libraries. In this work, we describe the current design of this server, as well as planned features and potential implementation strategies.

A semantically enabled metadata repository for scientific data

The LASP Extended Metadata Repository (LEMR) is a semantically enabled repository of information (metadata) about the scientific datasets that LASP offers to the public. The repository enables the provision of consistent, current, verified metadata to our users. It serves as a Single Source of Truth for this information, enabling more rigorous metadata management and addressing problems related to duplication of information. The linked open data aspect of the repository allows interlinking of concepts both within and across organizations and web sites. Associated interfaces allow users to browse and search the metadata. This information can be dynamically incorporated into web pages, so web page content is always up-to-date and consistent across the lab. With this information we can generate metadata records in a variety of schemas, such as ISO or SPASE, allowing federation with other organizations interested in our data. We leveraged open source technologies to build the repository and the dynamic web pages that read from it. VIVO, an open source semantic web application, provided key capabilities such as ontology and triple store management interfaces. AngularJS, an open source JavaScript framework for building web dynamic applications, was also invaluable in developing web pages that provide semantically enabled public interfaces to the metadata. In this paper we discuss our use of these tools and what we had to craft in order to meet our lab-specific needs.