R. Burke Johnson

Mixed Methods Research: A Research Paradigm Whose Time Has Come

The purposes of this article are to position mixed methods research (mixed research is a synonym) as the natural complement to traditional qualitative and quantitative research, to present pragmatism as offering an attractive philosophical partner for mixed methods research, and to provide a framework for designing and conducting mixed methods research. In doing this, we briefly review the paradigm “wars” and incompatibility thesis, we show some commonalities between quantitative and qualitative research, we explain the tenets of pragmatism, we explain the fundamental principle of mixed research and how to apply it, we provide specific sets of designs for the two major types of mixed methods research (mixed-model designs and mixed-method designs), and, finally, we explain mixed methods research as following (recursively) an eight-step process. A key feature of mixed methods research is its methodological pluralism or eclecticism, which frequently results in superior research (compared to monomethod research). Mixed methods research will be successful as more investigators study and help advance its concepts and as they regularly practice it.

Call for Mixed Analysis: A Philosophical Framework for Combining Qualitative and Quantitative Approaches

Abstract We provide a philosophical justification for analyzing qualitative and quantitative data within the same study. First, we present several recent typologies of analyses in social science research that incorporate both monomethod (i.e. purely quantitative research or purely qualitative research) and mixed research studies. Second, we discuss what has been referred to as the fundamental principle of empirical data analysis, wherein both qualitative and quantitative data analysis techniques are shaped by an attempt to analyze data in a way that yields at least one of five types of generalizations. Third, building on the frameworks of Denzin and Lincoln (2005), Heron and Reason (1997) and Johnson and Onwuegbuzie (2004), we compare and contrast three qualitative-based paradigms (i.e. constructivism, critical theory, participatory), one quantitative-based paradigm (i.e. postpositivism) and one mixed research-based paradigm (i.e. pragmatism) with respect to three axiomatic components (i.e. ontological, epistemological and methodological foundations) and seven issues (i.e. nature of knowledge, knowledge accumulation, goodness or quality criteria, values, ethics, inquirer posture and training). Also, we link each paradigm to data analysis strategies. Fourth, we illustrate similarities in goals between some qualitative and quantitative analyses; in so doing, we deconstruct the strong claim that analysis must be either qualitative or quantitative and illustrate that regardless of perspective (e.g. postpositivist or constructivist), both qualitative and quantitative data can be jointly analyzed. Finally, we compare and contrast 11 mixed research paradigms/worldviews, linking them to mixed analysis strategies, thereby situating mixed analyses in the philosophy of social science and promoting mixed research as a distinctive methodology.

Toward a theoretical model of evaluation utilization

Abstract Implicit evaluation utilization process-models were constructed from evaluation theorists, ideas, and explicit evaluation utilization process-models (i.e. already developed models) were located in the literature. The meta-model (i.e. a model developed from other models) was developed from the implicit and explicit process-models and from important ideas reported in recent research on evaluation use (e.g. participation, organizational development and complexity). The model depicts evaluation use as occurring in an internal environment situated in an external environment. The three sets of variables in the theoretical model are the background variables, the interactional or social psychological variables and the evaluation use variables. It is contended that evaluation-for-use will result in longer term effects when ideas from complexity theory, organizational learning and organizational design are employed. The meta-model reported here should be viewed as a theoretical model, offered in an attempt to promote theory development in the evaluation utilization literature.

Coming at Things Differently Future Directions of Possible Engagement With Mixed Methods Research

The field of mixed methods (MM) research facilitates ‘‘coming at things differently.’’ Traditional forms of data gathering using one method for data collection may not be adequate for answering complex questions that sometimes require a variety of qualitative and quantitative methods in one study. Although the number of MM studies continues to grow, it is clear from the literature that newcomers and seasoned MM researchers encounter praxis/practice barriers that concern the ‘‘how-tos’’ of integrating MM findings. More philosophical concerns center on issues regarding whether or not paradigmatic stances can be mixed or can coexist. Still other concerns center on research conundrums that reside within and between our disciplinary contexts, such as the impact of and interfacing with newly emergent technologies for data analysis and collection with MM research. Some of these barriers are conscious whereas others are unconscious.

Securing a Place at the Table A Review and Extension of Legitimation Criteria for the Conduct of Mixed Research

The purpose of this article is to recommend legitimation criteria pertinent for conducting mixed research. First, a discussion of mixed research as a distinctive methodology is presented, and challenges unique to mixed research are outlined. Second, an overview of quality criteria published in the mixed research literature is summarized. Third, the authors extend the discussion of quality criteria by introducing what we call the Holistic and Synergistic Legitimation Research Process, whereby emphasis on philosophical clarity and dialogue with values/perspectives of multiple communities of practice play continuous, iterative, holistic, and synergistic roles in the legitimation process. In so doing, the authors hope to clarify criteria that are important in terms of legitimating inferences drawn throughout a study, and, at the inquiry’s conclusion, to provide a perspective for guiding researchers’ evaluations of the quality of their research findings, inferences, and applications.

Comments on Howe: Toward a More Inclusive “Scientific Research in Education”:

In response to Howe (2009), the author argues that educational research needs multiple thoughtful perspectives. The author’s standpoint is that of a mixed methods research methodologist. Mixed methods research provides an antidualistic and syncretic philosophy and set of approaches or possibilities for merging insights from diverse perspectives; its working goal is to provide pragmatic, ethical solutions to local and societal problems. To achieve this goal, researchers should cease writing articles that construct straw figures (based on old literature), knock them down, and claim Truth. The author of the present article provides a new set of guidelines for an education science that includes a respectful and important place for all. The author also provides a working value theory that resolves some objectivist and subjectivist differences.

Common Survey Sampling Techniques

In this chapter we provide an introduction to probability and non-probability sampling methods commonly used in quantitative research. Although we pay special attention to sampling issues in survey research, we also briefly explain additional sampling methods used in other types of research.

Dialectical Pluralism A Metaparadigm Whose Time Has Come

There has been much debate about the role of paradigms in mixed methods research. In the face of past calls for each researcher to operate within a single paradigm, it turns out that some researchers/practitioners find many positive features in more than one paradigm. This “multiparadigmatic perspective” used in mixed methods research needs a systematic framework for the practice of engaging in difference. Also, individuals committed to a single paradigm need a philosophical/theoretical framework for working in multiparadigmatic teams. This article provides such a framework. It is a metaparadigm, and it is labeled dialectical pluralism 2.0 or more simply dialectical pluralism. The word “pluralism” refers to the acceptance and expectancy of difference in virtually every realm of inquiry, including reality, and the age-old word “dialectical” refers to the operative process which is both dialectical and dialogical. Dialectical pluralism provides a way for researchers, practitioners, clients, policy makers, and other stakeholders to work together and produce new workable “wholes” while, concurrently, thriving on differences and intellectual tensions.

Adding Qualitative and Mixed Methods Research to Health Intervention Studies Interacting With Differences

The purpose of this article is to explain how to improve intervention designs, such as randomized controlled trials (RCTs), in health science research using a process philosophy and theory known as dialectical pluralism (DP). DP views reality as plural and uses dialectical, dialogical, and hermeneutical approaches to knowledge construction. Using DP and its “both/and” logic, and its attempt to produce new creative syntheses, researchers on heterogeneous teams can better dialogue with qualitative and mixed methods approaches, concepts, paradigms, methodologies, and methods to improve their intervention research studies. The concept of reflexivity is utilized but is expanded when it is a component of DP. Examples of strategies for identifying, inviting, and creating divergence and integrative strategies for producing strong mixed methods intervention studies are provided and illustrated using real-life examples.

Practice note: Using debriefing interviews to promote authenticity and transparency in mixed research

Abstract Our goal in this editorial is to promote transparency in mixed research by detailing how to conduct d ebriefing interviews throughout the research process to document the degree that quality criteria are designed and implemented. First, we examine the role of transparency in establishing credibility of findings and conclusions. We provide a rationale for using debriefing interviews to elevate transparency in mixed research. Second, we discuss the process of peer debriefing in the context of qualitative studies because it is this context that formed the catalyst for developing a debriefing interview protocol. Third, we apply in the context of mixing approaches, five authenticity criteria (Guba & Lincoln, 1989) developed in accordance with a constructivist tradition. We present a debriefing interview technique accompanied by example interview questions to document the degree that researchers have met, in practice, these five quality criteria and other quality criteria such as philosophical clarity and incorporation of multiple standpoints. We conclude by discussing the process and benefits of debriefing interviews in mixed research studies.