Andrew Caines

Incremental Dependency Parsing and Disfluency Detection in Spoken Learner English

This paper investigates the suitability of state-of-the-art natural language processing NLP tools for parsing the spoken language of second language learners of English. The task of parsing spoken learner-language is important to the domains of automated language assessment ALA and computer-assisted language learning CALL. Due to the non-canonical nature of spoken language containing filled pauses, non-standard grammatical variations, hesitations and other disfluencies and compounded by a lack of available training data, spoken language parsing has been a challenge for standard NLP tools. Recently the Redshift parser Honnibal et al. In: Proceedings of CoNLL 2013 has been shown to be successful in identifying grammatical relations and certain disfluencies in native speaker spoken language, returning unlabelled dependency accuracy of 90.5% and a disfluency F-measure of 84.1% Honnibal & Johnson: TACL 2, 131-142 2014. We investigate how this parser handles spoken data from learners of English at various proficiency levels. Firstly, we find that Redshifts parsing accuracy on non-native speech data is comparable to Honnibal & Johnsons results, with 91.1% of dependency relations correctly identified. However, disfluency detection is markedly down, with an F-measure of just 47.8%. We attempt to explain why this should be, and investigate the effect of proficiency level on parsing accuracy. We relate our findings to the use of NLP technology for CALL and ALA applications.

Characterizing Eve: Analysing Cybercrime Actors in a Large Underground Forum.

Underground forums contain many thousands of active users, but the vast majority will be involved, at most, in minor levels of deviance. The number who engage in serious criminal activity is small. That being said, underground forums have played a significant role in several recent high-profile cybercrime activities. In this work we apply data science approaches to understand criminal pathways and characterize key actors related to illegal activity in one of the largest and longest-running underground forums. We combine the results of a logistic regression model with k-means clustering and social network analysis, verifying the findings using topic analysis. We identify variables relating to forum activity that predict the likelihood a user will become an actor of interest to law enforcement, and would therefore benefit the most from intervention. This work provides the first step towards identifying ways to deter the involvement of young people away from a career in cybercrime.