Karen A. Monsen

Descriptive Analysis and Interpretation

This chapter presents a step-by-step data analysis process beginning with data cleaning and preprocessing, which leads to and may include descriptive analyses of all variables. Once the data are clean and final descriptive statistics are completed, inferential statistics (Chap. 6) may be leveraged to better understand the practical and statistical significance of the data. Use of exploratory Data Analysis (Chap. 7) at any stage may reveal patterns in the data for subsequent formal hypothesis testing. Interpretation of the analysis is discussed for each of the methods. Your project may employ a number of complementary analysis methods based on study design, the variables representing the PIO MM concepts, and the measures used to operationalize them.

Exploratory Data Analysis

This chapter presents Exploratory Data Analysis (EDA) as an approach for gaining understanding and insight about a particular dataset, in order to support and validate statistical findings and also to potentially generate, identify, and create new hypotheses based on patterns in data. Examples of EDA are provided and interpretations are discussed. EDA may be used at any stage in the data analysis process from cleaning through transformation and descriptive analysis, as well as using results from every stage. Discovery of patterns may inspire a new direction in intervention effectiveness research, as well as further supporting or validating existing projects. Data visualization skills are essential for individuals working with large datasets.

Key Concepts, Definitions, and Frameworks

This chapter explores the worldviews that underlie intervention effectiveness research, quality improvement, and program evaluation, and explains the similarities and differences between these perspectives. At the heart of all perspectives is the need to know if an efficacious or research-based intervention is effective in a real-world setting, producing the intended and desired outcomes. Understanding how these worldviews arose and converge provides a basis for conducting intervention effectiveness research, quality improvement activities, or program evaluation using rigorous methods, correctly interpreting findings, and disseminating results. Definitions of these terms and other related terms such as translational research and comparative effectiveness research are provided in order to clearly establish the scope of the guidance provided within this book.

Inferential Analysis and Interpretation

This chapter builds on the concepts related to project design (Chap. 4) and descriptive (Chap. 5) to present a number of possible inferential statistics approaches suitable for the PIO MM (this chapter) to better understand the practical and statistical significance of the data. Interpretation of the analysis is discussed for each of the inferential methods. Selecting appropriate inferential statistics depends on your project study design, the variables representing the PIO MM concepts, and the measures used to operationalize them.

Use the Worksheets and PIO MM Figure

This chapter briefly reviews Part I of the book and offers an overview of Part II. The heart of Part II is a series of worksheets that are designed to guide you through each major phase of your project. Chapters 10– 16 give detailed instructions for completing each worksheet along with examples of completed worksheets.

Interpret the Results (Worksheet F)

The goal of this chapter is to clearly explicate the project results statements based on PIO MM. To complete the results statements for each analysis there are three heuristics: State what can be known from the findings, do not go beyond the data, and give alternative explanations for the findings. Completing and interpreting the analysis provides the information needed to complete statements for the Characteristics of the Sample, Description of the Outcomes, Statistical Significance of the Outcomes, and Practical Significance of the Outcomes. If measured, Description of the Intervention with fidelity measures (alignment with guideline or expected interventions) and dose (frequency counts) may also be described.

Define the Problem (Worksheet B)

The goal of this chapter is to clearly define and substantiate the problem that the intervention is expected to prevent or improve, based on the literature. Worksheet B is presented, described relative to making a difference in practice, and shown in completed from using data abstracted from a published article. Upon completion of Worksheet B, the reader will have defined the problem, substantiated the importance of the problem with relevant literature, and identified the method for measuring the problem over time. This information, together with the information from Worksheet C (Chap. 12) and Worksheet D (Chap. 13), will set the stage for launching the study.

Define the Outcome (Worksheet D)

The goal of this chapter is to clearly define and substantiate the outcome and related measures that will be used to operationalize the concepts described in PIO MM for the study. Worksheet D should be completed for each measure before moving on to the next chapter. This chapter provides an example of completed worksheets based on a fictitious study of home visiting program outcomes of individuals receiving brief family home visiting services compared to those receiving long term visits. Upon completion of Worksheet D, the reader will have defined the measures for identified PIO MM concepts including population characteristics, problem, intervention, and context; substantiated the psychometrics of the measures, and described the data collection process for each measure. This information, together with the information from Worksheets B and C enables the description of the study purpose and sets the stage for launching the study.

Describe the Intervention (Worksheet C)

The goal of this chapter is to clearly define and substantiate the intervention that is used to address the problem defined in Worksheet B, based on the literature. Worksheet C is presented and is completed based on an example from a fictitious study of intervention patterns associated with outcomes of adult and adolescent mothers with and without the Mental health problem. Worksheet C should be completed before moving on to the next chapter. Upon completion of Worksheet C, the reader will have defined the intervention, substantiated the efficacy of the intervention with relevant literature, and identified the method for measuring the intervention dose and fidelity over time. This information, together with the information from Worksheets B (Chapter 11) and D (Chapter 13), enables the description of the study purpose and sets the stage for launching the study.

Know the Literature (Worksheet A)

The first step in a research project is a thorough review of the literature related to your project. Here the purpose is to understand the state of the science and continue scientific inquiry based on that understanding. Worksheet A offers a structured format for recording relevant information (based on the PIO MM) from published articles. The result of Worksheet A is a “literature matrix” around the intervention effectiveness, quality improvement, or program evaluation problem of interest. After completing Worksheet A, the reader will have a ready resource for completing Worksheets B–F—information that will underlie and support all aspects of the study.