Gregor Wiedemann

Applying LDA Topic Modeling in Communication Research: Toward a Valid and Reliable Methodology

ABSTRACT Latent Dirichlet allocation (LDA) topic models are increasingly being used in communication research. Yet, questions regarding reliability and validity of the approach have received little attention thus far. In applying LDA to textual data, researchers need to tackle at least four major challenges that affect these criteria: (a) appropriate pre-processing of the text collection; (b) adequate selection of model parameters, including the number of topics to be generated; (c) evaluation of the model’s reliability; and (d) the process of validly interpreting the resulting topics. We review the research literature dealing with these questions and propose a methodology that approaches these challenges. Our overall goal is to make LDA topic modeling more accessible to communication researchers and to ensure compliance with disciplinary standards. Consequently, we develop a brief hands-on user guide for applying LDA topic modeling. We demonstrate the value of our approach with empirical data from an ongoing research project.

Computer-Assisted Text Analysis in the Social Sciences

Despite there is a long tradition of Computer Assisted Text Analysis (CATA) in social sciences, it followed a rather parallel development to QDA. Only a few years ago, realization of TM potentials for QDA started to emerge slowly. In this chapter, I reflect on the debate of the use of software in qualitative social science research together with approaches of text analysis from the NLP perspective.

Content Analysis between Quality and Quantity

Social science research using Text Mining tools requires—due to the lack of a canonical heuristics in the digital humanities—a blended reading approach. Integrating quantitative and qualitative analyses of complex textual data progressively, blended reading brings up various requirements for the implementation of Text Mining infrastructures. The article presents the Leipzig Corpus Miner (LCM), developed in the joint research project ePol—Post-Democracy and Neoliberalism and responding to social science research requirements. The functionalities offered by the LCM may serve as best practice of processing data in accordance with blended reading.

Analyse qualitativer Daten mit dem „Leipzig Corpus Miner“

Der Leipzig Corpus Miner (LCM) ist eine Webanwendung, die verschiedene Text Mining-Verfahren fur die Analyse groser Mengen qualitativer Daten bundelt. Durch eine einfach zu bedienende Benutzeroberflache ermoglicht der LCM Volltextzugriff auf 3,5 Millionen Zeitungstexte, die nach Suchbegriffen und Metadaten zu Subkollektionen gefiltert werden konnen. Auf dem Gesamtdatenbestand sowie auf den Subkollektionen konnen verschiedene computergestutzte Auswertungsverfahren angewendet und zu Analyseworkflows kombiniert werden. Damit ermoglicht der LCM die empirische Analyse sozialwissenschaftlicher Fragestellungen auf Basis groser Dokumentkollektionen, wobei qualitative und quantitative Analyseschritte miteinander verschrankt werden konnen. Dieser Artikel gibt einen Uberblick uber die Analysekapazitaten und mogliche Workflows zur Anwendung des LCM.

Zwischen »Neonazismus« und »Ideologien der Ungleichwertigkeit«

Haufig wird Forderungen nach Aufgabe des Extremismusbegriffs mit dem Verweis auf einen Mangel an Alternativen begegnet. Dieser Beitrag zeigt dagegen exemplarisch anhand eines fur Leipzig erstellten Handlungskonzeptes zur Starkung der demokratischen Kultur, wie eine Problematisierung bestimmter Ereignisse, Strukturen und Ideologien ohne Ruckgriff auf die analytischen Fallstricke des »(Rechts-)Extremismus« gelingen kann. Er argumentiert fur eine Verwendung des Terminus »Neonazismus« sowie die konkrete Benennung einzelner Ungleichwertigkeitsideologien wie Rassismus, Antisemitismus, Homophobie etc. und begrundet diese jeweils auf Basis der spezifischen lokalen Situationsanalyse. Der Anspruch eines »Rechtsextremismus« ersetzenden, neuen Sammelbegriffes wird dabei tendenziell fallen gelassen.

Integrating Text Mining Applications for Complex Analysis

The last chapter already has demonstrated that Text Mining (TM) applications can be a valid approach to social science research questions and that existing studies employ single TM procedures to investigate larger text collections. However, to benefit most effectively from the use of TM and to be able to develop complex research designs meeting requirements of established QDA methodologies, one needs specific adaptions of several procedures as well as a systematic integration of them. Therefore, this chapter introduces an integrated application of various TM methods to answer a specific political science research question.

Exemplary Study: Democratic Demarcation in Germany

The Text Mining (TM) workflows presented in the previous chapter provided a variety of results which will be combined in the following to a comprehensive study on democratic demarcation in Germany. The purpose of this chapter is to present an example of how findings from the introduced set of TM applications on large text collections contribute to investigations of abstract political and social science questions. Consequently, the character of this chapter differs from the previous ones with respect to the disciplinary perspective I take to describe the applied methods and results.

Einleitung Text Mining in den Sozialwissenschaften

Qualitative Methoden, die durch die Analyse von Texten Aussagen uber die soziale Wirklichkeit ermoglichen sollen, gehoren zweifelsohne zum zeitgenossischen Kanon sozialwissenschaftlicher Forschung (vgl. dazu Stulpe / Lemke in diesem Band). Wissenssoziologie und Hermeneutik sind ebenso einschlagige Konzepte, wie Grounded Theory, Diskursanalyse oder Qualitative Inhaltsanalyse, die als konkrete Auswertungsmethoden weite Verbreitung gefunden haben. Seit den 1980er Jahren unterstutzen Computerprogramme wie MAXQDA oder ATLAS.ti SozialwissenschaftlerInnen beim Datenmanagement in der Anwendung dieser Methoden – zunachst noch mit groserer Skepsis ob ihres Einflusses auf den Forschungsprozess verwendet, sind sie heutzutage jedoch fest etabliert (vgl. Kelle 2008).

Summary: Integrating Qualitative and Computational Text Analysis

In the light of recent research debates on computational social science and Digital Humanities (DH) as meanwhile adolescent disciplines dealing with big data (Reichert, 2014), I strove for answering in which ways Text Mining (TM) applications are able to support Qualitative Data Analysis (QDA) in the social sciences in a manner that fruitfully integrates a qualitative with a quantitative perspective. The guiding assumption was, the more modern Natural Language Processing (NLP) and Machine Learning (ML) algorithms enable us to identify patterns of `meaning’ from global contexts of mass data collections, while at the same time preserving opportunities to retrieve identified patterns again in local contexts of single documents, the more they allow for a fruitful integration of qualitative and quantitative text analysis. By combining extraction of qualitative knowledge from text to buttress understanding of social reality with quantification of extracted knowledge structures to infer on their relevancy, utilizing TM for QDA is inherently a mixed method research design.

Introduction: Qualitative Data Analysis in a Digital World

Digitalization and informatization of science during the last decades have widely transformed the ways in which empirical research is conducted in various disciplines. Computer-assisted data collection and analysis procedures even led to the emergence of new subdisciplines such as bioinformatics or medical informatics. The humanities (including social sciences) so far seem to lag somewhat behind this development—at least when it comes to analysis of textual data.