Brendan R. Watson

Issues and Best Practices in Content Analysis

This article discusses three issues concerning content analysis method and ends with a list of best practices in conducting and reporting content analysis projects. Issues addressed include the use of search and databases for sampling, the differences between content analysis and algorithmic text analysis, and which reliability coefficients should be calculated and reported. The “Best Practices” section provides steps to produce reliable and valid content analysis data and the appropriate reporting of those steps so the project can be properly evaluated and replicated.

Assessing the Reporting of Reliability in Published Content Analyses: 1985–2010

Content analysis is a common research method employed in communication studies. An important part of content analysis is establishing the reliability of the coding protocol, and reporting must be detailed enough to allow for replication of methodological procedures. This study employed a content analysis of published content analysis articles (N=581) in three communication journals over a 26-year period to examine changes in reliability sampling procedures and reporting of reliability coefficients across time. Findings indicate that general improvements have been made in the detail of reporting reliability, in the practice of reporting reliability coefficients that take chance into consideration, and in the reporting of reliability coefficients for more than one variable. However, explaining the reliability sampling process and use of a probability or census reliability sample did not change over time. In recent years, the preponderance of articles did not explain the reliability sampling method or report a reliability coefficient for all key study variables, and few utilized a census or probability sampling frame. Implications are discussed and recommendations made for reporting of reliability in content analysis.

When Critical Voices Should Speak Up

Routine journalistic coverage relies heavily on official sources, which are least likely to raise critical questions about the causes and consequences of disasters such as the 2010 BP oil spill. It is possible, though, that disasters—precisely because they are unexpected—at least temporarily dislodge journalistic routines that sideline independent, more critical voices. This study uses Gulf Coast newspaper coverage of the BP oil spill to examine whether there are two discernible stages of crisis reporting, the first marked by the more frequent use of unofficial sources (and thus, a more critical tone to the coverage), followed by a narrowing of those perspectives seen as being legitimate as the official interpretation of the crisis emerges, and journalists index their coverage to match that interpretation.

Structural Determinants of Local Public Affairs Place Blogging: Structural Pluralism and Community Stress

This study examines the relationship between community-level characteristics and the presence of public affairs place blogs in 232 U.S. cities. Two models to predict the presence of these sites are tested: a structural pluralism model, which suggests that the presence of one of these sites reflects more pluralistic voices, and a community stress model, which suggests that the presence of these sites reflects citizens’ efforts to cope with community problems. Analysis of demographic and crime data using logistic regression suggests that the community stress model is the stronger predictor. Public affairs place blogs are more likely in cities with higher murder rates, poverty rates, more physical decay, and more residents with professional occupations. It is these residents—with more education and income, living on the periphery of the most affected urban neighborhoods—who are most likely to go online to write about obtrusive community problems.

Is Twitter an Alternative Medium? Comparing Gulf Coast Twitter and Newspaper Coverage of the 2010 BP Oil Spill

This study compares Gulf Coast journalists and Twitter users’ coverage of the BP oil spill. In addition to examining authors’ attitudes toward and coverage of the BP oil spill, the study examines community-level variables that shaped attitudes and coverage. The community structure literature has suggested that news media in smaller, more homogeneous communities, which are economically dependent on a polluting industry (as are many communities along the Gulf Coast), are more reticent to be critical in their coverage of pollution. Scholars have suggested, though, that the Internet transcends local geography and that the Internet is more open to alternative perspectives. This study suggests, though, that while the distribution of online content may make local geography less relevant, its production is still rooted in local communities. As a result, Tweets about the oil spill were shaped by many of the same social and economic forces that shaped journalists’ coverage.

Ideologies drive journalists' attitudes toward oil industry

A survey of Gulf Coast journalists revealed that their environmental and political ideologies are the strongest predictors of their attitudes toward the oil industry following the BP oil spill. Conservative journalists are most likely to believe that the industry behaved responsibly.

Three Decades of Reliability in Communication Content Analyses: Reporting of Reliability Statistics and Coefficient Levels in Three Top Journals

This study examines reliability reporting in content analysis articles (N = 672) in three flagship communication journals. Data from 1985 to 2014 suggest improvements in reporting across time and also identify areas for additional improvement. Data show increased reporting of chance-corrected reliability coefficients and reporting reliability for all study variables, although increases were inconsistent among journals and the most recent time period showed slight declines. In general, the most often used coefficient was Scott’s Pi; however, Krippendorff’s Alpha was most used in the latest study period. Reporting of low reliability coefficients increased but then decreased most recently. Implications and areas for improvement are discussed.

Assessing Ideological, Professional, and Structural Biases in Journalists’ Coverage of the 2010 BP Oil Spill

Previous studies based on aggregate data have not found consistent evidence that journalists’ personal beliefs and attitudes bias their coverage. This study, however, uses individual-level survey data on Gulf Coast journalists’ beliefs and attitudes toward the BP oil spill, matched with a content analysis of respondents’ stories about the disaster, and community structure data. The study examines the effect that journalists’ perceptions of professional norms and the social and economic contexts of the communities in which they work had on their attitudes toward and coverage of the crisis.

Community Information Needs: A Theory and Methodological Framework

In the mass media age, mass communication researchers interested in civic engagement and participation focused primarily on media and media use. However, given the upending of traditional media institutions and the rise of social media and other digital communication technologies that are giving rise to new sources of information, the contemporary media environment calls for a focus on information received from media (and non-media sources), and whether that information meets communities’ information needs. This article unifies different theoretical traditions into a definition of community information needs and a model of community information needs, information seeking and processing, and civic engagement, which can begin to guide theoretically grounded need assessments. The community information seeking and processing model proposed also requires a fresh look at methodological traditions; community-based participatory (CBPR) research is suggested as a possible framework to follow. The CBPR framework capitalizes on the unique scholarly and professional strengths of the mass communication discipline not only to assess information needs but also to help fulfill those needs and study the community-level effects of a robust information environment.

Are Demographics Adequate Controls for Cell-Phone-Only Coverage Bias in Mass Communication Research?

Cell-phone-only (CPO) households differ along key variables from non-CPO households, creating potential coverage biases in landline-only random-digit-dialing (RDD) surveys. Researchers have attempted to correct for this by weighting their data based on demographic differences. Previous research, however, has not examined CPO coverage biases in media-use surveys—an important oversight as cell phone use is itself a media choice. This article presents a secondary analysis of Pew’s 2012 media consumption survey and concludes that demographics alone are not adequate controls for the CPO bias in media-use surveys.