William M. K. Trochim

An introduction to concept mapping for planning and evaluation.

Concept mapping is a type of structured conceptualization which can be used by groups to develop a conceptual framework which can guide evaluation or planning. In the typical case, six steps are involved: (1) Preparation (including selection of participants and development of focus for the conceptualization); (2) the Generation of statements; (3) the Structuring of statements; (4) the Representation of Statements in the form of a concept map (using multidimensional scaling and cluster analysis); (5) the Interpretation of maps; and, (6) the Utilization of maps. Concept mapping: encourages the group to stay on task; results relatively quickly in an interpretable conceptual framework; expresses this framework entirely in the language of the participants; yields a graphic or pictorial product which simultaneously shows all major ideas and their interrelationships; and often improves group or organizational cohesiveness and morale. This paper describes each step in the process, considers major methodological issues and problems, and discusses computer programs which can be used to accomplish the process.

Concept Mapping as an Alternative Approach for the Analysis of Open-Ended Survey Responses:

This article presents concept mapping as an alternative method to existing code-based and word-based text analysis techniques for one type of qualitative text data—open-ended survey questions. It is argued that the concept mapping method offers a unique blending of the strengths of these approaches while minimizing some of their weaknesses. This method appears to be especially well suited for the type of text generated by open-ended questions as well for organizational research questions that are exploratory in nature, aimed at scale or interview question development, and/or developing conceptual coding schemes. A detailed example of concept mapping on open-ended survey data is presented. Reliability and validity issues associated with concept mapping are also discussed.

Outcome pattern matching and program theory

Abstract Pattern matching is presented as a general framework which can guide the use of theory within program evaluation. Pattern matching minimally involves the specification of a theoretical pattern, the acquisition of an observed pattern, and an attempt to match these two. Pattern matching logic assumes that more complex theoretical patterns, if corroborated, provide a stronger basis for valid inference. Pattern matches in program evaluation can be divided into two types: process pattern matches which assess the construct validity of the program, participants, or measures, and outcome pattern matches which assess the causal hypothesis and address the traditional concerns of internal and external validity. Each of the three types of process pattern matches can be further divided into characteristic pattern matches (which examine the interrelationships between key characteristics across programs, participants, or measures) or object pattern matches (which view interrelationships between programs, participants, or measures based on their overall degree of similarity). Outcome pattern matching can be accomplished for any process pattern match by examining outcomes across programs, participants, or measures viewed either in terms of their characteristics or as molar objects. Hypothetical examples of pattern matching in program outcome evaluation contexts are presented along with consideration of the value of pattern matching for theory-based research .

Evaluating transdisciplinary science.

The past two decades have seen a growing interest and investment in transdisciplinary research teams and centers. The Transdisciplinary Tobacco Use Research Centers (TTURCs) exemplify large-scale scientific collaborations undertaken for the explicit purpose of promoting novel conceptual and methodological integrations bridging two or more fields. Until recently, few efforts have been made to evaluate the collaborative processes, and the scientific and public policy outcomes, of such centers. This manuscript offers a conceptual framework for understanding and evaluating transdisciplinary science and describes two ongoing evaluation studies covering the initial phase of the TTURC initiative. The methods and measures used by these studies are described, and early evaluative findings from the first 4 years of the initiative are presented. These data reveal progress toward intellectual integration within and between several of the TTURCs, and cumulative changes in the collaborative behaviors and values of participants over the course of the initiative. The data also suggest that different centers may follow alternative pathways toward transdisciplinary integration and highlight certain environmental, organizational, and institutional factors that influence each centers readiness for collaboration. Methodological challenges posed by the complexities of evaluating large-scale scientific collaborations (including those that specifically aspire toward transdisciplinary integrations spanning multiple fields) are discussed. Finally, new directions for future evaluative studies of transdisciplinary scientific collaboration, both within and beyond the field of tobacco science, are described.

Practical Challenges of Systems Thinking and Modeling in Public Health

OBJECTIVES Awareness of and support for systems thinking and modeling in the public health field are growing, yet there are many practical challenges to implementation. We sought to identify and describe these challenges from the perspectives of practicing public health professionals. METHODS A systems-based methodology, concept mapping, was used in a study of 133 participants from 2 systems-based public health initiatives (the Initiative for the Study and Implementation of Systems and the Syndemics Prevention Network). This method identified 100 key challenges to implementation of systems thinking and modeling in public health work. RESULTS The project resulted in a map identifying 8 categories of challenges and the dynamic interactions among them. CONCLUSIONS Implementation by public health professionals of the 8 simple rules we derived from the clusters in the map identified here will help to address challenges and improve the organization of systems that protect the publics health.

Research Design for Program Evaluation: The Regression-Discontinuity Approach.

An electric power source system for a gyroscopic instrument in which the kinetic energy of the gyro rotor revolving at high speed is used as an induction generator during times when the normal electrical power source for the gyroscopic instrument fails and to maintain the gyroscopic instrument in its normal operative condition during the time the electrical power source is disconnected.

Systems Thinking to Improve the Public's Health

Improving population health requires understanding and changing societal structures and functions, but countervailing forces sometimes undermine those changes, thus reflecting the adaptive complexity inherent in public health systems. The purpose of this paper is to propose systems thinking as a conceptual rubric for the practice of team science in public health, and transdisciplinary, translational research as a catalyst for promoting the functional efficiency of science. The paper lays a foundation for the conceptual understanding of systems thinking and transdisciplinary research, and will provide illustrative examples within and beyond public health. A set of recommendations for a systems-centric approach to translational science will be presented.

Concept mapping: Soft science or hard art?

Abstract Is concept mapping “science” or “art”? Can we legitimately claim that concept maps represent reality, or are they primarily suggestive devices which might stimulate new ways to look at our experiences? Here, the scientific side of concept mapping is viewed as “soft science” and the artistic one as “hard art” to imply that the process has some qualities of both, but probably does not fall exclusively within eithers domain. In the spirit of hard art, a “gallery” of final concept maps from twenty projects is presented, partly to illustrate more examples of the process when used in a variety of subject areas and for different purposes, and partly for their aesthetic value alone. In the spirit of soft science, two major issues are considered. First, the evidence for the validity and reliability of concept mapping is introduced, along with some suggestions for further research which might be undertaken to examine those characteristics. Second, the role of concept mapping is discussed, with special emphasis on its use in a pattern matching framework.

An Introduction to Concept Mapping as a Participatory Public Health Research Method

In this article, the authors introduce concept mapping as a useful participatory research method for public health researchers interested in generating hypotheses and developing theory. The authors first provide an overview of concept mapping, which combines qualitative approaches with quantitative analytical tools to produce visual displays of the relationship between ideas. Then, they present an illustrative research application of the method to the exploration of women’s perceptions of the relationship between residential neighborhood factors and intimate partner violence experiences. They give attention to the data collection and analysis procedures and to demonstrating the intricacies of using concept mapping for public health research purposes. Finally, the article concludes with a discussion of the unique contributions and challenges associated with concept mapping.

PATTERN MATCHING, VALIDITY, AND CONCEPTUALIZATION IN PROGRAM EVALUATION

All social research is based on pattern matching ideas. A pattern match involves a correspondence between a theoretical or conceptual expectation pattern and an observed or measured pattern. Two quasi-experimental designs-the nonequivalent dependent variable design and the reversed treatment design—illustrate pattern matching logic well. In program evaluation three pattern matches are important: the program pattern match that assesses program implementation; the measurement pattern match that assesses the validity of the measures; and the effect pattern match that assesses the causal hypothesis Conceptualization methods are needed to facilitate the articulation of rich theoretical patterns. An example of a conceptualization study is presented and the utility of conceptualization methods for pattern-matching research is discussed.