Dave Braines

A Visual Approach to Semantic Query Design Using a Web-Based Graphical Query Designer

Query formulation is a key aspect of information retrieval, contributing to both the efficiency and usability of many semantic applications. A number of query languages, such as SPARQL, have been developed for the Semantic Web; however, there are, as yet, few tools to support end users with respect to the creation and editing of semantic queries. In this paper we introduce NITELIGHT, a Web-based graphical tool for semantic query construction that is based on the W3C SPARQL specification. NITELIGHT combines a number of features to support end-users with respect to the creation of SPARQL queries. These include a columnar ontology browser, an interactive graphical design surface, a SPARQL-compliant visual query language, a SPARQL syntax viewer and an integrated semantic query results browser. The functionality of each of these components is described in the current paper. In addition, we discuss the potential contribution of the NITELIGHT tool to rule creation/editing and semantic integration capabilities.

Controlled English to facilitate human/machine analytical processing

Controlled English is a human-readable information representation format that is implemented using a restricted subset of the English language, but which is unambiguous and directly accessible by simple machine processes. We have been researching the capabilities of CE in a number of contexts, and exploring the degree to which a flexible and more human-friendly information representation format could aid the intelligence analyst in a multi-agent collaborative operational environment; especially in cases where the agents are a mixture of other human users and machine processes aimed at assisting the human users. CE itself is built upon a formal logic basis, but allows users to easily specify models for a domain of interest in a human-friendly language. In our research we have been developing an experimental component known as the “CE Store” in which CE information can be quickly and flexibly processed and shared between human and machine agents. The CE Store environment contains a number of specialized machine agents for common processing tasks and also supports execution of logical inference rules that can be defined in the same CE language. This paper outlines the basic architecture of this approach, discusses some of the example machine agents that have been developed, and provides some typical examples of the CE language and the way in which it has been used to support complex analytical tasks on synthetic data sources. We highlight the fusion of human and machine processing supported through the use of the CE language and CE Store environment, and show this environment with examples of highly dynamic extensions to the model(s) and integration between different user-defined models in a collaborative setting.

Development of a controlled natural language interface for semantic MediaWiki

Semantic wikis support the collaborative creation, editing and utilization of semantically-enriched content, and they may therefore be well-suited to addressing problems associated with the limited availability of high-quality online semantic content. Unfortunately, however, many popular semantic wikis, such as Semantic MediaWiki (SMW), are not sufficiently expressive to support full-scale ontology authoring. Furthermore, the grounding of the Semantic Web in formal logic makes both the comprehension and production of ontological content difficult for many end-users. In order to address these issues, the expressivity of SMW was extended using a combination of semantic templates and a Web Ontology Language (OWL) meta-model. Semantic templates were also used to provide an ontology verbalization capability for SMW using the Rabbit Controlled Natural Language (CNL). The resulting system demonstrates how CNL interfaces can be implemented on top of SMW. The proposed solution introduces no changes to the underlying functionality of the SMW system, and the use of semantic templates as an ontology verbalization solution means that end-users can exploit all the usual features of conventional wiki systems to collaboratively create new CNL verbalization capabilities.

A Semantic Wiki Based Light-Weight Web Application Model

Wiki is a well-known Web 2.0 content management platform. The recent advance of semantic wikis enriches the conventional wikis by allowing users to edit and query structured semantic annotations (e.g., categories and typed links) beyond plain wiki text. This new feature provided by semantic wikis, as shown in this paper, enables a novel, transparent, and light-weight social Web application model. This model let developers collectively build Web applications using semantic wikis, including for data modeling, data management, data processing and data presentation. The source scripts and data of such applications are transparent to Web users. Beyond a generic description for the Web application model, we show two proof-of-concept prototypes, namely RPI Map and CNL (Controlled Natural Language) Wiki, both of which are based on Semantic MediaWiki (SMW).

Human-machine conversations to support mission-oriented information provision

Mission-oriented sensor networks present challenging problems in terms of human-machine collaboration. Human users need to task the network to help them achieve mission objectives, while humans (sometimes the same individuals) are also sources of mission-critical information. We propose a natural language-based conversational approach to supporting human-machine working in mission-oriented sensor networks. We present a model for human-machine and machine-machine interactions in a realistic mission context, and evaluate the model using an existing surveillance mission scenario. The model supports the flow of conversations from full natural language to a form of Controlled Natural Language (CNL) amenable to machine processing and automated reasoning, including high-level information fusion tasks. We introduce a mechanism for presenting the gist of verbose CNL expressions in a more convenient form for human users. We show how the conversational interactions supported by the model include requests for expansions and explanations of machine-processed information.

Sherlock: Experimental Evaluation of a Conversational Agent for Mobile Information Tasks

Controlled natural language (CNL) has great potential to support human–machine interaction (HMI) because it provides an information representation that is both human readable and machine processable. We investigated the effectiveness of a CNL-based conversational interface for HMI in a behavioral experiment called simple human experiment regarding locally observed collective knowledge (Sherlock). In Sherlock, individuals acted in groups to discover and report information to the machine using natural language (NL), which the machine then processed into CNL. The machine fused responses from different users to form a common operating picture, a dashboard showing the level of agreement for distinct information. To obtain information to add to this dashboard, users explored the real world in a simulated crowdsourced sensing scenario. This scenario represented a simplified controlled analog for tactical intelligence (i.e., direct intelligence of the environment), which is key for rapidly planning military, law enforcement, and emergency operations. Overall, despite close to zero training, 74% of the users inputted NL that was machine interpretable and addressed the assigned tasks. An experimental manipulation aimed to increase user–machine interaction, however, did not improve performance as hypothesized. Nevertheless, results indicate that the conversational interface may be effective in assisting humans with collection and fusion of information in a crowdsourcing context.

From open source communications to knowledge

Rapid processing and exploitation of open source information, including social media sources, in order to shorten decision-making cycles, has emerged as an important issue in intelligence analysis in recent years. Through a series of case studies and natural experiments, focussed primarily upon policing and counter-terrorism scenarios, we have developed an approach to information foraging and framing to inform decision making, drawing upon open source intelligence, in particular Twitter, due to its real-time focus and frequent use as a carrier for links to other media. Our work uses a combination of natural language (NL) and controlled natural language (CNL) processing to support information collection from human sensors, linking and schematising of collected information, and the framing of situational pictures. We illustrate the approach through a series of vignettes, highlighting (1) how relatively lightweight and reusable knowledge models (schemas) can rapidly be developed to add context to collected social media data, (2) how information from open sources can be combined with reports from trusted observers, for corroboration or to identify con icting information; and (3) how the approach supports users operating at or near the tactical edge, to rapidly task information collection and inform decision-making. The approach is supported by bespoke software tools for social media analytics and knowledge management.

Subjective Bayesian Networks and Human-in-the-Loop Situational Understanding

In this paper we present a methodology to exploit human-machine coalitions for situational understanding. Situational understanding refers to the ability to relate relevant information and form logical conclusions, as well as identify gaps in information. This process for comprehension of the meaning information requires the ability to reason inductively, for which we will exploit the machines’ ability to ‘learn’ from data. However, important phenomena are often rare in occurrence with high degrees of uncertainty, thus severely limiting the availability of instance data for training, and hence the applicability of many machine learning approaches. Therefore, we present the benefits of Subjective Bayesian Networks—i.e., Bayesian Networks with imprecise probabilities—for situational understanding, and the role of conversational interfaces for supporting decision makers in the evolution of situational understanding.

Agile development of ontologies through conversation

Ontologies and semantic systems are necessarily complex but offer great potential in terms of their ability to fuse information from multiple sources in support of situation awareness. Current approaches do not place the ontologies directly into the hands of the end user in the field but instead hide them away behind traditional applications. We have been experimenting with human-friendly ontologies and conversational interactions to enable non-technical business users to interact with and extend these dynamically. In this paper we outline our approach via a worked example, covering: OWL ontologies, ITA Controlled English, Sensor/mission matching and conversational interactions between human and machine agents.

Context-rich semantic framework for effective data-to-decisions in coalition networks

In a coalition context, data fusion involves combining of soft (e.g., field reports, intelligence reports) and hard (e.g., acoustic, imagery) sensory data such that the resulting output is better than what it would have been if the data are taken individually. However, due to the lack of explicit semantics attached with such data, it is difficult to automatically disseminate and put the right contextual data in the hands of the decision makers. In order to understand the data, explicit meaning needs to be added by means of categorizing and/or classifying the data in relationship to each other from base reference sources. In this paper, we present a semantic framework that provides automated mechanisms to expose real-time raw data effectively by presenting appropriate information needed for a given situation so that an informed decision could be made effectively. The system utilizes controlled natural language capabilities provided by the ITA (International Technology Alliance) Controlled English (CE) toolkit to provide a human-friendly semantic representation of messages so that the messages can be directly processed in human/machine hybrid environments. The Real-time Semantic Enrichment (RTSE) service adds relevant contextual information to raw data streams from domain knowledge bases using declarative rules. The rules define how the added semantics and context information are derived and stored in a semantic knowledge base. The software framework exposes contextual information from a variety of hard and soft data sources in a fast, reliable manner so that an informed decision can be made using semantic queries in intelligent software systems.