Genevieve Gorrell

Analysis of named entity recognition and linking for tweets

Applying natural language processing for mining and intelligent information access to tweets (a form of microblog) is a challenging, emerging research area. Unlike carefully authored news text and other longer content, tweets pose a number of new challenges, due to their short, noisy, context-dependent, and dynamic nature. Information extraction from tweets is typically performed in a pipeline, comprising consecutive stages of language identification, tokenisation, part-of-speech tagging, named entity recognition and entity disambiguation (e.g. with respect to DBpedia). In this work, we describe a new Twitter entity disambiguation dataset, and conduct an empirical analysis of named entity recognition and disambiguation, investigating how robust a number of state-of-the-art systems are on such noisy texts, what the main sources of error are, and which problems should be further investigated to improve the state of the art.

Countering method bias in questionnaire‐based user studies

Purpose – This paper seeks to discuss reliability problems associated with questionnaires, commonly employed in library and information science. It aims to focus on the effects of “common method variance” (CMV), which is a form of bias, and ways of countering these effects.Design/methodology/approach – The paper critically reviews the use of existing tools for demonstrating reliability in questionnaire‐based studies. In particular, it focuses on Cronbachs alpha, “Harmans single factor test” and Lindell and Whitneys “marker variable” approach. The paper introduces an illustrative case study based on the work on metacognition and web searching. It goes on to make recommendations for researchers considering using a questionnaire‐based approach.Findings – CMV is a problem affecting questionnaire‐based studies in different disciplines across social and information science. Where questionnaire items are more abstract, CMV has been found to be more of a problem. The widely used Cronbach alpha measure, of the ...

GATE Teamware: a web-based, collaborative text annotation framework

This paper presents GATE Teamware—an open-source, web-based, collaborative text annotation framework. It enables users to carry out complex corpus annotation projects, involving distributed annotator teams. Different user roles are provided (annotator, manager, administrator) with customisable user interface functionalities, in order to support the complex workflows and user interactions that occur in corpus annotation projects. Documents may be pre-processed automatically, so that human annotators can begin with text that has already been pre-annotated and thus making them more efficient. The user interface is simple to learn, aimed at non-experts, and runs in an ordinary web browser, without need of additional software installation. GATE Teamware has been evaluated through the creation of several gold standard corpora and internal projects, as well as through external evaluation in commercial and EU text annotation projects. It is available as on-demand service on GateCloud.net, as well as open-source for self-installation.

Negative symptoms in schizophrenia: a study in a large clinical sample of patients using a novel automated method

Objectives To identify negative symptoms in the clinical records of a large sample of patients with schizophrenia using natural language processing and assess their relationship with clinical outcomes. Design Observational study using an anonymised electronic health record case register. Setting South London and Maudsley NHS Trust (SLaM), a large provider of inpatient and community mental healthcare in the UK. Participants 7678 patients with schizophrenia receiving care during 2011. Main outcome measures Hospital admission, readmission and duration of admission. Results 10 different negative symptoms were ascertained with precision statistics above 0.80. 41% of patients had 2 or more negative symptoms. Negative symptoms were associated with younger age, male gender and single marital status, and with increased likelihood of hospital admission (OR 1.24, 95% CI 1.10 to 1.39), longer duration of admission (β-coefficient 20.5 days, 7.6–33.5), and increased likelihood of readmission following discharge (OR 1.58, 1.28 to 1.95). Conclusions Negative symptoms were common and associated with adverse clinical outcomes, consistent with evidence that these symptoms account for much of the disability associated with schizophrenia. Natural language processing provides a means of conducting research in large representative samples of patients, using data recorded during routine clinical practice.

Towards “metacognitively aware” IR systems: an initial user study

Purpose – The purpose of this paper is to describe: a new taxonomy of metacognitive skills designed to support the study of metacognition in the context of web searching; a data collection instrument based on the taxonomy; and the results of testing the instrument on a sample of university students and staff.Design/methodology/approach – The taxonomy is based on a review of the literature, and is extended to cover web searching. This forms the basis for the design of the data collection instrument, which is tested with 405 students and staff of Sheffield University.Findings – Subjects regard the range of metacognitive skills focused on as broadly similar. However, a number of significant differences in reported metacognition usage relating to age, gender and discipline.Practical implications – These findings contribute to the long‐term aims of the research which are to: develop a model of the actual and potential role of metacognition in web searching, and identify strategic “metacognitive interventions” ...

Talking through procedures: an intelligent space station procedure assistant

We present a prototype system aimed at providing spoken dialogue support for complex procedures aboard the International Space Station. The system allows navigation one line at a time or in larger steps. Other user functions include issuing spoken corrections, requesting images and diagrams, recording voice notes and spoken alarms, and controlling audio volume.

Natural language processing to extract symptoms of severe mental illness from clinical text: the Clinical Record Interactive Search Comprehensive Data Extraction (CRIS-CODE) project

Objectives We sought to use natural language processing to develop a suite of language models to capture key symptoms of severe mental illness (SMI) from clinical text, to facilitate the secondary use of mental healthcare data in research. Design Development and validation of information extraction applications for ascertaining symptoms of SMI in routine mental health records using the Clinical Record Interactive Search (CRIS) data resource; description of their distribution in a corpus of discharge summaries. Setting Electronic records from a large mental healthcare provider serving a geographic catchment of 1.2 million residents in four boroughs of south London, UK. Participants The distribution of derived symptoms was described in 23 128 discharge summaries from 7962 patients who had received an SMI diagnosis, and 13 496 discharge summaries from 7575 patients who had received a non-SMI diagnosis. Outcome measures Fifty SMI symptoms were identified by a team of psychiatrists for extraction based on salience and linguistic consistency in records, broadly categorised under positive, negative, disorganisation, manic and catatonic subgroups. Text models for each symptom were generated using the TextHunter tool and the CRIS database. Results We extracted data for 46 symptoms with a median F1 score of 0.88. Four symptom models performed poorly and were excluded. From the corpus of discharge summaries, it was possible to extract symptomatology in 87% of patients with SMI and 60% of patients with non-SMI diagnosis. Conclusions This work demonstrates the possibility of automatically extracting a broad range of SMI symptoms from English text discharge summaries for patients with an SMI diagnosis. Descriptive data also indicated that most symptoms cut across diagnoses, rather than being restricted to particular groups.

Using @Twitter Conventions to Improve #LOD-Based Named Entity Disambiguation

State-of-the-art named entity disambiguation approaches tend to perform poorly on social media content, and microblogs in particular. Tweets are processed individually and the richer, microblog-specific context is largely ignored. This paper focuses specifically on quantifying the impact on entity disambiguation performance when readily available contextual information is included from URL content, hash tag definitions, and Twitter user profiles. In particular, including URL content significantly improves performance. Similarly, user profile information for @mentions improves recall by over 10i¾?% with no adverse impact on precision. We also share a new corpus of tweets, which have been hand-annotated with DBpedia URIs, with high inter-annotator agreement.

Metacognition and web credibility

Purpose – The research reported here generated a list of criteria adopted by postgraduate students when evaluating websites. The analysis presented aims to determine whether metacognition played any part in the evaluation of websites by volunteers.Design/methodology/approach – Forty‐eight students participated in the study. They carried out a series of searches designed to bring them into contact with a range of websites, from forums to electronic books. The students were encouraged to “think aloud” as they searched, and to explain their actions and strategies. Search sessions were recorded, transcribed, and subjected to ethnographic content analysis.Findings – A range of evaluation criteria is presented. The criteria were applied at different stages of the search process and demonstrate varying degrees of metacognition. Observations on evaluation processes are also presented. Factors affecting evaluation included the purpose of the search, advice received from lecturers, and the perceived nature of the w...

ResToRinG CaPitaLiZaTion in #TweeTs

The rapid proliferation of microblogs such as Twitter has resulted in a vast quantity of written text becoming available that contains interesting information for NLP tasks. However, the noise level in tweets is so high that standard NLP tools perform poorly. In this pa- per, we present a statistical truecaser for tweets using a 3-gram language model built with truecased newswire texts and tweets. Our truecasing method shows an improvement in named entity recognition and part-of-speech tagging tasks.