Nick Spadaccini

Specification of the Crystallographic Information File format, version 2.0

Version 2.0 of the CIF format incorporates novel features implemented in STAR 2.0. Among these are an expanded character repertoire, new and more flexible forms for quoted data values, and new compound data types. The CIF 2.0 format is compared with both CIF 1.1 and STAR 2.0, and a formal syntax specification is provided.

Extensions to the STAR File Syntax

The STAR File [Hall J. Chem. Inf. Comput. Sci. 1991, 31, 326-333; Hall and Spadaccini J. Chem. Inf. Comput. Sci. 1994, 34, 505-508] format represents a universal language adopted for electronic data and metadata exchange in the molecular-structure sciences [Hall et al. Acta Crystallogr. Sect. A 1991, 47, 655-685; International Tables for International Tables for Crystallography. Vol. G: Definition and exchange of crystallographic data; Springer: Dordrecht, The Netherlands, 2005; Allen et al. J. Chem. Inf. Comput. Sci. 1995, 35, 412-427] and used by the International Union for Crystallography for publication submissions and database depositions. This paper describes an extended STAR syntax that facilitates richer and more specific data definition and typing and a commensurate improvement in precise data description.

DDLm: A New Dictionary Definition Language

A previous paper [Spadaccini and Hall J. Chem. Inf. Model. doi:10.1021/ci300074v] details extensions to the STAR File [Hall J. Chem. Inf. Comput. Sci. 1991, 31, 326-333] syntax that will improve the exchange and archiving of electronic data. This paper describes a dictionary definition language (DDLm) for defining STAR File data items in a domain dictionary. A dictionary that defines the ontology and vocabulary of a discipline is built with DDLm, which is itself implemented in STAR, and is extensible and machine parsable. The DDLm is semantically rich and highly specific; provides strong data typing, data enumerations, and ranges; enables relationship keys between data items; and uses imbedded methods written in dREL [Spadaccini et al. J. Chem. Inf. Model. doi:10.1021/ci300076w] for data validation and evaluation and for refining data definitions. It promotes the modular definition of the discipline ontology and reuse through the ability to import definitions from other local and remote dictionaries, thus encouraging the sharing of data dictionaries within and across domains.

Relational expressions in STAR file dictionaries.

The STAR File (J. Chem. Inf Comput. Sci. 1994, 34, 505-508) is used widely in structural chemistry for exchanging numerical and text information with scientific journals and databases. These exchanges are increasingly dependent on data dictionaries to facilitate automatic data validation and checking. Definitions in data dictionaries are constructed using attribute descriptors, and this paper describes a method attribute for specifying the relationships between data items as an executable script written in a new relational expression language called dREL. The addition of this attribute improves the precision and the semantic content of dictionaries by providing relational representations of data, as well as facilitating the direct evaluation of derivable data items. The capacity to evaluate derivative data directly from the combination of primitive data and dictionary expressions is expected to change future archival approaches. The design concepts of the relational expression language dREL parser, which are applicable to any discipline, are described.

dREL: a relational expression language for dictionary methods.

The provision of precise metadata is an important but a largely underrated challenge for modern science [Nature 2009, 461, 145]. We describe here a dictionary methods language dREL that has been designed to enable complex data relationships to be expressed as formulaic scripts in data dictionaries written in DDLm [Spadaccini and Hall J. Chem. Inf. Model.2012 doi:10.1021/ci300075z]. dREL describes data relationships in a simple but powerful canonical form that is easy to read and understand and can be executed computationally to evaluate or validate data. The execution of dREL expressions is not a substitute for traditional scientific computation; it is to provide precise data dependency information to domain-specific definitions and a means for cross-validating data. Some scientific fields apply conventional programming languages to methods scripts but these tend to inhibit both dictionary development and accessibility. dREL removes the programming barrier and encourages the production of the metadata needed for seamless data archiving and exchange in science.

Automated Structure Detection and Analysis in Televiewer Images

Borehole televiewer data is an important source of data on structural and stratigraphic discontinuities in both the mining and petroleum industries. Manually picking features in downhole image logs is a labour-intensive and hence expensive task and as such is a significant bottleneck in data processing. It is also a subjective process. We present a new algorithm and workflow for automatically detecting and analysing planar structures in downhole acoustic and optical televiewer images. First, an image complexity measure highlights areas most suitable for automated structure detection. Changes in the image complexity can be used to locate geological boundaries. Second, structures are automatically detected, with each structure having an associated confidence level; users can apply a threshold to the confidence values to adjust the quality and quantity of the detected structures based on the image quality and geological complexity. Third, structures that have been detected but that do not meet the structure confidence threshold can be interactively assessed and if necessary selected. We also provide tools for rapidly picking sets of equivalent structures and reducing structures to a set of representative picks.

Specification of the STAR File

The general concepts and requirements of a universal exchange file are introduced in this chapter. The Self-defining Text Archive and Retrieval (STAR) File was designed to meet the requirements for a universal data-exchange mechanism, and it is the format widely used in structural science to exchange and archive numerical and text data. The STAR File structure is simple. Discrete data values, always represented by ASCII character strings, are identified by distinguishable tags. A file is partitioned into one or more blocks of data. Within each such block, one or more subsidiary blocks (known as save frames) may encapsulate a set of data items. In each block, a given tag may appear once only, but may index a single data value or a set of looped values. Loops of related items may be combined in tabular structures. Global blocks may occur that establish data values inherited by succeeding data blocks. This chapter describes the STAR File syntax and scoping rules in detail, and provides a formal grammar in extended Backus–Naur form. Keywords: STAR File specification; data models; universal data languages; STAR File syntax; STAR File grammar; text strings; data names; data values; data items; looped lists; save frames; data blocks; global blocks; data sets; privileged constructs; Backus–Naur form; lexical tokens

STAR File Utilities

The STAR File is a general-purpose data format that explicitly tags data values with identifiers, allowing them to be extracted and manipulated without needing to conform to a particular data model. Star_Base is a powerful stand-alone program that parses STAR Files and extracts specific items according to requests formulated in a simple, flexible but powerful query language. Several examples of its use are described, illustrating how the context of a data item is returned along with its value. Star.vim and StarMarkUp are applications for editing and browsing STAR Files. Some libraries and object classes for STAR and generalized CIF applications are also reviewed.

Specification of the Crystallographic Information File (CIF)

The Crystallographic Information File (CIF) is an electronic file containing specific data items relevant to crystallographic structure determinations and descriptions in the form of numbers and text, which are represented as ASCII character strings. The structure and format of a CIF are based on a restricted subset of the syntax of a Self-defining Text Archive and Retrieval (STAR) File. The CIF is intended as a general, flexible and extensible free-format file suitable for the transmission, storage and publication of crystallographic data. Since 1990 it has been recommended by the International Union of Crystallography as the preferred data structure for the exchange of data. This chapter provides a detailed description of the syntax features of a CIF and its conventions for associating meaning or semantic content to included data. Particular care is taken to describe how the file format handles issues of portability and archival integrity. The chapter includes the complete official specification, and a formal grammar in extended Backus–Naur form.

Star ontologies: knowledge retention through functional relationships

A key objective for databases is to delineate and capture computerinterpretable “knowledge” (i.e. meta-data) from non-derivative information (e.g. postulates and measurements) so that data can be modelled as part of a database validation and mining process. Current data handling approaches based on customized software are inflexible because they cannot respond quickly to changing methodologies. They are also difficult to extend so as to prevent the seamless addition of new data types and structures. Anther fundamental weakness is that software-encoded knowledge, which is usually encrypted in an imperative language, is neither easily human-accessible nor reuseable – and this inhibits knowledge retention, knowledge evolution and direct pedagogical applications. This talk will describe a generic approach to coalescing meta-data into domainspecific ontologies in which methods expressions, written in a symbolic text language dREL, describe complex relationships between defined data items [1]. A parser-compiler converts ontologies into a series of Java objects, forming an executable knowledge kernel for data manipulation, validation and interpretation tools. A prototype crystallographic ontology, based on the Core CIF dictionary, will be used to demonstrate the evaluation of derivative data and the redefinition of enumeration states. References [1] Spadaccini, N., Hall, S.R., and Castleden, I.R. “Relational Expressions in STAR File Dictionaries” 2000 J Chemical Information and Computer Science 40, 1289-1301.