Alistair Willis

Analysing anaphoric ambiguity in natural language requirements

Many requirements documents are written in natural language (NL). However, with the flexibility of NL comes the risk of introducing unwanted ambiguities in the requirements and misunderstandings between stakeholders. In this paper, we describe an automated approach to identify potentially nocuous ambiguity, which occurs when text is interpreted differently by different readers. We concentrate on anaphoric ambiguity, which occurs when readers may disagree on how pronouns should be interpreted. We describe a number of heuristics, each of which captures information that may lead a reader to favor a particular interpretation of the text. We use these heuristics to build a classifier, which in turn predicts the degree to which particular interpretations are preferred. We collected multiple human judgements on the interpretation of requirements exhibiting anaphoric ambiguity and showed how the distribution of these judgements can be used to assess whether a particular instance of ambiguity is nocuous. Given a requirements document written in natural language, our approach can identify sentences that contain anaphoric ambiguity, and use the classifier to alert the requirements writer of text that runs the risk of misinterpretation. We report on a series of experiments that we conducted to evaluate the performance of the automated system we developed to support our approach. The results show that the system achieves high recall with a consistent improvement on baseline precision subject to some ambiguity tolerance levels, allowing us to explore and highlight realistic and potentially problematic ambiguities in actual requirements documents.

Extending Nocuous Ambiguity Analysis for Anaphora in Natural Language Requirements

This paper presents an approach to automatically identify potentially nocuous ambiguities, which occur when text is interpreted differently by different readers of requirements written in natural language. We extract a set of anaphora ambiguities from a range of requirements documents, and collect multiple human judgments on their interpretations. The judgment distribution is used to determine if an ambiguity is nocuous or innocuous. We investigate a number of antecedent preference heuristics that we use to explore aspects of anaphora which may lead a reader to favour a particular interpretation. Using machine learning techniques, we build an automated tool to predict the antecedent preference of noun phrase candidates, which in turn is used to identify nocuous ambiguity. We report on a series of experiments that we conducted to evaluate the performance of our automated system. The results show that the system achieves high recall with a consistent improvement on baseline precision subject to some ambiguity tolerance levels, allowing us to explore and highlight realistic and potentially problematic ambiguities in actual requirements documents.

Automatic detection of nocuous coordination ambiguities in natural language requirements

Natural language is prevalent in requirements documents. However, ambiguity is an intrinsic phenomenon of natural language, and is therefore present in all such documents. Ambiguity occurs when a sentence can be interpreted differently by different readers. In this paper, we describe an automated approach for characterizing and detecting so-called nocuous ambiguities, which carry a high risk of misunderstanding among different readers. Given a natural language requirements document, sentences that contain specific types of ambiguity are first extracted automatically from the text. A machine learning algorithm is then used to determine whether an ambiguous sentence is nocuous or innocuous, based on a set of heuristics that draw on human judgments, which we collected as training data. We implemented a prototype tool for Nocuous Ambiguity Identification (NAI), in order to illustrate and evaluate our approach. The tool focuses on coordination ambiguity. We report on the results of a set of experiments to assess the performance and usefulness of the approach.

Making Tacit Requirements Explicit

The importance of tacit knowledge in Requirements Engineering (RE) is widely acknowledged. While valuable work has developed techniques to expose sources of tacit knowledge during requirements elicitation, such techniques are not universally applied and tacit knowledge, continues to negatively affect the quality of the requirements. In this position paper we present a brief review and interpretation of the literature on tacit knowledge that, we believe, is useful for RE. We describe a number of techniques that offer analysts the means to reason about the effect of tacit knowledge and improve the quality of requirements and their management.

A Hybrid Model for Automatic Emotion Recognition in Suicide Notes

We describe the Open University teams submission to the 2011 i2b2/VA/Cincinnati Medical Natural Language Processing Challenge, Track 2 Shared Task for sentiment analysis in suicide notes. This Shared Task focused on the development of automatic systems that identify, at the sentence level, affective text of 15 specific emotions from suicide notes. We propose a hybrid model that incorporates a number of natural language processing techniques, including lexicon-based keyword spotting, CRF-based emotion cue identification, and machine learning-based emotion classification. The results generated by different techniques are integrated using different vote-based merging strategies. The automated system performed well against the manually-annotated gold standard, and achieved encouraging results with a micro-averaged F-measure score of 61.39% in textual emotion recognition, which was ranked 1st place out of 24 participant teams in this challenge. The results demonstrate that effective emotion recognition by an automated system is possible when a large annotated corpus is available.

Unpacking Tacit Knowledge for Requirements Engineering

The use of tacit knowledge is a common feature in everyday communication. It allows people to communicate effectively without forcing them to make everything tediously and painstakingly explicit, provided they all share a common understanding of whatever is not made explicit. If this latter criterion does not hold, confusion and misunderstanding will ensue. Tacit knowledge is also commonplace in requirements where it also affords economy of expression. However, the use of tacit knowledge also suffers from the same risk of misunder-standing, with the associated problems of anticipating where it has the potential for confusion, and of unraveling where it has played an actual role in misunder-standing. Thus the effective communication of requirements knowledge (whether verbally, through a document or some other medium) requires an understanding of what knowledge is and isn’t (necessarily) held in common. This is very hard to get right as people from different professional and cultural backgrounds are typically involved. At its worst, tacit requirements knowledge may lead to software that fails to satisfy the customer’s requirements. In this chapter we review the diverse views of tacit knowledge discussed in the literature from a wide range of disci-plines, reflect on their commonalities and differences, and propose a conceptual framework for requirements engineering that characterizes the different facets of tacit knowledge that distinguish the different views. We then identify methodolog-ical and technical challenges for future research on the role of tacit knowledge in requirements engineering.

Speculative requirements: Automatic detection of uncertainty in natural language requirements

Stakeholders frequently use speculative language when they need to convey their requirements with some degree of uncertainty. Due to the intrinsic vagueness of speculative language, speculative requirements risk being misunderstood, and related uncertainty overlooked, and may benefit from careful treatment in the requirements engineering process. In this paper, we present a linguistically-oriented approach to automatic detection of uncertainty in natural language (NL) requirements. Our approach comprises two stages. First we identify speculative sentences by applying a machine learning algorithm called Conditional Random Fields (CRFs) to identify uncertainty cues. The algorithm exploits a rich set of lexical and syntactic features extracted from requirements sentences. Second, we try to determine the scope of uncertainty. We use a rule-based approach that draws on a set of hand-crafted linguistic heuristics to determine the uncertainty scope with the help of dependency structures present in the sentence parse tree. We report on a series of experiments we conducted to evaluate the performance and usefulness of our system.

Improving search personalisation with dynamic group formation

Recent research has shown that the performance of search engines can be improved by enriching a users personal profile with information about other users with shared interests. In the existing approaches, groups of similar users are often statically determined, e.g., based on the common documents that users clicked. However, these static grouping methods are query-independent and neglect the fact that users in a group may have different interests with respect to different topics. In this paper, we argue that common interest groups should be dynamically constructed in response to the users input query. We propose a personalisation framework in which a user profile is enriched using information from other users dynamically grouped with respect to an input query. The experimental results on query logs from a major commercial web search engine demonstrate that our framework improves the performance of the web search engine and also achieves better performance than the static grouping method.

Temporal latent topic user profiles for search personalisation

The performance of search personalisation largely depends on how to build user profiles effectively. Many approaches have been developed to build user profiles using topics discussed in relevant documents, where the topics are usually obtained from human-generated online ontology such as Open Directory Project. The limitation of these approaches is that many documents may not contain the topics covered in the ontology. Moreover, the human-generated topics require expensive manual effort to determine the correct categories for each document. This paper addresses these problems by using Latent Dirichlet Allocation for unsupervised extraction of the topics from documents. With the learned topics, we observe that the search intent and user interests are dynamic, i.e., they change from time to time. In order to evaluate the effectiveness of temporal aspects in personalisation, we apply three typical time scales for building a long-term profile, a daily profile and a session profile. In the experiments, we utilise the profiles to re-rank search results returned by a commercial web search engine. Our experimental results demonstrate that our temporal profiles can significantly improve the ranking quality. The results further show a promising effect of temporal features in correlation with click entropy and query position in a search session.

On Presuppositions in Requirements

Tacit knowledge in requirements documents can lead to miscommunication between software engineers and