IVOA Data Access Layer: roadmap as of year 2020
aa r X i v : . [ a s t r o - ph . I M ] D ec IVOA Data Access Layer: roadmap as of year 2020
Marco Molinaro and James Dempsey INAF - Osservatorio Astronomico di Trieste, Via G.B.Tiepolo 11, 34143Trieste, Italy; [email protected] CSIRO Information Management and Technology, GPO Box 1700 Canberra,ACT 2601, Australia
Abstract.
The International Virtual Observatory Alliance (IVOA) produces stan-dards to enable the sharing of astronomical data and services to the global astrophysicalcommunity. Within the IVOA, the Data Access Layer (DAL) working group aims toprovide standards for querying and accessing data holdings. The standards are primar-ily implemented by observatories and other data providers, so that the community canuse standard tools to interact with the data holdings. Recently, the DAL communityhas addressed the discovery and exchange of multi-dimensional and multi-messengerdata. It has also tackled new topics such as retrieval of observation location and objectvisibility information. They are now examining further support for the time domain andradio astronomy communities. We present the current DAL status and progress, in orderto keep implementors up to date with the DAL landscape. We also discuss upcomingchanges to DAL standards. Community contribution and feedback on these standardsare needed. We particularly encourage feedback from the data providers and projectsthat are using VO technologies to address their scientific community’s requirements.
1. Introduction
The International Virtual Observatory Alliance (IVOA ) has a mission to facilitate theinternational coordination and collaboration necessary for the development and de-ployment of the tools, systems and organisational structures necessary to enable theinternational utilisation of astronomical archives as an integrated and interoperatingvirtual observatory . To fulfil its mission the IVOA promotes technological standards,called Recommendations , among which the Data Access Layer Working Group (DALWG ) manages those related to the discovery and access of catalogues and datasets forastrophysical data holdings.The full set of IVOA Recommendations (and other documents ) provides a consis-tent ecosystem that evolves in time, following astrophysical community requirementsand technical landscape updates. This paper gives a view on the status and near futuredevelopments of the subset of this ecosystem that is related to the DAL WG activities. https://wiki.ivoa.net/twiki/bin/view/IVOA/IvoaDAL All IVOA published documents are made available at ff orts needed in implementingthose Recommendations, as well as some new challenges the standards themselves willbe facing, are then reported in the summary Sec. 5.
2. Architecture
Figure 1. IVOA Architecture diagram, modified to report the DAL landscape ofstandards. Blue rectangles identify standards directly managed by DAL, while greenones identify those co-managed with other WGs or on behalf of IGs. A border isapplied to standards that are already IVOA Recommendations, with an orange borderidentifying standards that are currently under revision.
Each IVOA Recommendation includes a role diagram , derived from the IVOAArchitecture (Arviset et al. 2010) Level 2 diagram, to visually describe the role it playsin the VO landscape and how it connects to other standards. Fig. 1 provides a similardiagram, modified from the common IVOA one, to provide a quick-look on the currentstatus of the DAL related standards.What’s visible from Fig. 1 is an even split between the standards under develop-ment or revision and the ones in standby, i.e. awaiting for community feedback. Amongthose under revision, most are so in response to community implementation feedback(DataLink, ADQL, DALI, VOEvent). However the ConeSearch revision is driven bothby an attempt to bring it up to compliance with the DALI interface specifications (seealso Fig. 3 and Sec. 4) and new requirements from the Time Domain community toallow filtering of catalogues based on a time window of interest. There are also newstandards under development (ObsLocTap & ObjVisSAP, ProvTAP & ProvSAP): someare meant to provide a discovery interface leveraging an existing Data Model standardVOADAL:roadmap asof 2020 3(like the Provenance one); others aim to standardise access to the observation sched-ules and the visibility of sources in the sky for di ff erent ground-based or space-bornefacilities.What’s not visible with the quick-look diagram is the connection to the Time Do-main Interest Group (IG) and the (just formed) Radio Astronomy IG. Both are provid-ing or will provide new requirements for the DAL standards. An example of this, asalready said, is the ConeSearch specification revision that will include a TIME parame-ter to accommodate filtering on time alongside the original positional one. The RadioAstronomy IG, on the other side, has just started work and will potentially impact theTAP / ObsCore based discovery scenario.
3. Roadmap
Figure 2. DAL short term roadmap. Recommendations awaiting community feed-back are in the
STANDBY box on the right. Current status, and the expected one bymid 2021, are depicted on the left. Dashed boxes indicate a less certain evolution.There’s no solid background on the + lane because this is not a fixedrelease schedule, only the expected outcome given the available e ff orts. The development expected for the DAL related standards on the short time scaleis depicted in Fig. 2. A few standards are expected to reach or approach the Recom-mendation status, moving the load from the ongoing development (pre-Working Draftor Working Draft - WD) to final comments and issue of the final standard (ProposedRecommendation - PR, Request for Comments phase - RFC, or Recommendation).On the other side there are many standards that are awaiting community feedbackfrom implementation. Here is where data providers and implementors have a majorrole, identifying missing features, issues and suggesting new use cases to be taken intoaccount.
4. Dependencies
One more concept to keep in mind, while reading the status and the roadmap forthe DAL related standards, is how the various standards are linked together. This issketched (without pretending to be exhaustive) in Fig. 3. From the diagram it is clearthe role played by the DALI Recommendation, that factorises the interface componentsof DAL access protocols and it can also be seen that some standards haven’t yet beenupdated to comply with it. The role of TAP also emerges as a support to model drivendiscovery, a solution initially introduced by ObsCore. Molinaro and Dempsey
Figure 3. DAL related standards inter-dependency graph.
DAL standards of course don’t rely only on themselves but have dependencies onmany other standards managed by other WGs: VOTable, UCD1 + , VOSI, only to citethe more obvious ones.
5. Summary
From the above description a few things can be taken away. Implementing an IVOAdiscovery or access protocol is not just a matter of reading one specification. Moreover,standards are revised over time and a new version of a standard can potentially lead torevisions in those that depend on it. The overall landscape, however, looks pretty stableand new features, and even standards, can fit in without major disruptions. Some morefactorisation can be applied (longer vision) on the non-TAP branch of the protocols,while TAP itself might have to cater for NoSQL solutions.In the widened landscape of the IVOA, DAL will also have to understand require-ments coming from computational use cases (where SODA and DataLink might needfurther updates) and will need to work (following Grid & Web Service WG activities)on attaching Authentication and Authorization solutions to its protocol interfaces.Further knowledge on the DAL history and activities can be found in Bonnarelet al. (2017) and Molinaro & Bonnarel (2019).
Acknowledgments.
MM acknowledges support from the project ESCAPE, fundedby the EU Horizon 2020 programme under the Grant Agreement n. 824064. DaveMorris provided support in placing the VOEvent specification in the diagrams. The ar-chitecture diagram has been built copying from the ivoaTeX ( https://github.com/ivoa-std/ivoatex ) role diagram SVG solution. References
Arviset, C., Gaudet, S., & IVOA Technical Coordination Group 2010, IVOA Architecture Ver-sion 1.0, IVOA Note 23 November 2010Bonnarel, F., et al. 2017, in ADASS XXV, edited by N. P. F. Lorente, K. Shortridge, & R. Wayth,vol. 512 of ASP Conf. Series, 549.1612.00299