AvesTerra: a framework for global-scale knowledge representation and analytic interoperability

Abstract

AvesTerra is a distributed knowledge representation framework for integrating many large and disparate data systems and analytic components at global scale. This framework allows data created or curated by many different institutions to be linked into a single unified, dynamic knowledge representation structure. The resulting fabric provides participants with a means to engage in multidisciplinary research and collaboration spanning many information systems without requiring a sophisticated computer science understanding of the mechanics of Big Data manipulation. Furthermore, AvesTerra enables this integration without the need for centralized data aggregation or local high-performance computational infrastructure, leveraging instead the distributed resources of a diverse and highly distributed analytic community. At a core technical level, AvesTerra consists of a system of peer-to-peer servers that collectively form a readily scalable knowledge space. The mathematical structure of this space is that of a generalized, recursive hypergraph, enabling the representation of complex dependency structures often encountered when working towards global scale. The framework incorporates numerous computational constructs including event publication and subscription, parallel threading and timer support, a unique distributed rendezvous mechanism for agent-based organization, privacy isolation, and semantic structure execution. This presentation provides an overview of the full framework and a sampling of the applications currently under development.