Gary Uhl

Increasing evaluation capacity within community-based HIV prevention programs

Abstract Funding agencies use technical assistance to strengthen the evaluation capacity of community-based organizations (CBOs). We used qualitative methods to describe beliefs and attitudes related to evaluation and to identify factors influencing evaluation capacity, based on interviews with 61 CBOs, nine health departments, and 28 technical assistance providers. Four factors influencing evaluation behavior among CBOs were identified: funding agency expectations, resources, leadership and staff, and evaluation tools and technology. Using these factors, we developed a model that describes three stages of evaluation capacity: compliance, investment, and advancement. We propose strategies by which funding agencies and technical assistance providers can help strengthen evaluation capacity within CBOs.

Taxonomy for strengthening the identification of core elements for evidence-based behavioral interventions for HIV/AIDS prevention

The concept of core elements was developed to denote characteristics of an intervention, such as activities or delivery methods, presumed to be responsible for the efficacy of evidence-based behavioral interventions (EBIs) for HIV/AIDS prevention. This paper describes the development of a taxonomy of core elements based on a literature review of theoretical approaches and characteristics of EBIs. Sixty-one categories of core elements were identified from the literature and grouped into three distinct domains: implementation, content and pedagogy. The taxonomy was tested by categorizing core elements from 20 HIV prevention EBIs disseminated by Centers for Disease Control and Prevention. Results indicated that core elements represented all three domains but several were difficult to operationalize due to vague language or the inclusion of numerous activities or constructs. A process is proposed to describe core elements in a method that overcomes some of these challenges. The taxonomy of core elements can be used to identify core elements of EBIs, strengthen the translation of EBIs from research to practice and guide future research seeking to identify essential core elements in prevention interventions.

Evaluation of an HIV prevention intervention for African Americans and Hispanics: findings from the VOICES/VOCES Community-based Organization Behavioral Outcomes Project.

There is limited knowledge about whether the delivery of evidence-based, HIV prevention interventions in ‘real world’ settings will produce outcomes similar to efficacy trial outcomes. In this study, we describe longitudinal changes in sexual risk outcomes among African American and Hispanic participants in the Video Opportunities for Innovative Condom Education and Safer Sex (VOICES/VOCES) program at four CDC-funded agencies. VOICES/VOCES was delivered to 922 high-risk individuals in a variety of community settings such as substance abuse treatment centers, housing complex centers, private residences, shelters, clinics, and colleges. Significant risk reductions were consistently observed at 30- and 120-days post-intervention for all outcome measures (e.g., unprotected sex, self-reported STD infection). Risk reductions were strongest for African American participants, although Hispanic participants also reported reducing their risky behaviors. These results suggest that, over a decade after the first diffusion of VOICES/VOCES across the U.S. by CDC, this intervention remains an effective tool for reducing HIV risk behaviors among high-risk African American and Hispanic individuals.

Building evaluation capacity for HIV prevention programs

HIV prevention programs, even those using science-based interventions, need to conduct evaluation to support the implementation and transfer of effective interventions, account for services, demonstrate effectiveness, and improve programs. The Program Evaluation Research Branch of the Division of HIV/AIDS Prevention, National Center for HIV, STD, and TB Prevention, Centers for Disease Control and Prevention, assists health department grantees and other CDC grantees by providing evaluation guidance, technical assistance (TA), and training in order to build their HIV prevention program evaluation capacity. Together, these evaluation resources assist grantees with overall implementation of evaluation and identify specific types of evaluation appropriate to each stage of intervention development. This paper describes the evaluation developmental process for different types of evaluation activities, provides a framework for building evaluation capacity, discusses the evaluation resources provided by CDC and gives examples of how evaluation TA and training support the overall technology transfer goals.

Opening up their doors: perspectives on the involvement of the African American faith community in HIV prevention in four communities.

In 1998, the U.S. government launched the Minority AIDS Initiative (MAI) to address growing ethnic and racial disparities in HIV/AIDS cases. The CDC performed an evaluation of its MAI-funded programs, including an assessment of community stakeholders’ perspective on the involvement of the faith community in HIV prevention. Individual interviews (N = 113) were conducted annually over 3 years in four communities. The majority of participants described a change in faith community’s attitudes toward HIV and a rise in HIV-related activities conducted by faith-based organizations. Participants attributed changes to faith-based funding, acknowledgment by African American community leadership that HIV is a serious health issue, and faith leaders’ desire to become more educated on HIV/AIDS. Participants reported conservative faith doctrine and stigma as barriers to faith community involvement. The findings suggest that although barriers remain, there is an increased willingness to address HIV/AIDS, and the faith community serves as a vital resource in HIV prevention.

Preventing Substance Use in High Risk Youth: Evaluation Challenges and Solutions

Programs to prevent substance use among high risk youth can pose numerous challenges for program evaluators including FEASIBILITY ISSUES such as Participant Recruitment and Retention, Identifying High Risk Youth, and Obtaining a Control/Comparison Group; MEASUREMENT ISSUES such as Social Desirability Bias and Instrument Reliability; METHODOLOGIC ISSUES such as Attrition (both selective and differential), Inadequate Implementation and Variable Dose, Low Statistical Power, Contamination of Comparison Groups, and Low Literacy Skills of Participants. For each of these challenges specific solutions are offered for researchers and practitioners. In general the solutions relate to three themes; 1) incorporate evaluation early into program design; 2) develop a strong partnership with program staff; and 3) be flexible. Also addressed are the need to acknowledge possible bias toward documenting positive outcomes as well as the need to customize evaluation designs for different settings and populations.

Provision of HIV Counseling and Testing Services at Five Community-Based Organizations Among Young Men of Color Who Have Sex with Men

In the context of monitoring and improving CDC-funded HIV prevention programs, we describe HIV tests and infections, provision of results, previous HIV tests, and risk behaviors for young (aged 13–29) men of color who have sex with men who received HIV tests at five community-based organizations. Of 1,723 tests provided, 2.1% were positive and 75.7% of positives were previously unaware of their infection. The highest positivity rate was among men aged 25–29 (4.7%). Thirty-four percent of tests were provided to men who were tested for the first time. Over half the tests (53.2%) were provided to men who reported sex with a person of unknown HIV status, and 34% to men who reported sex with an anonymous partner. Continued and more focused prevention efforts are needed to reach and test young men of color who have sex with men and to identify previously undiagnosed HIV infections among this target population.

Community stakeholders’ perspectives on the impact of the minority AIDS initiative in strengthening HIV prevention capacity in four communities1 The findings and conclusions in this paper are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention.

Objective. The Minority AIDS Initiative (MAI) was launched in 1998 to address the disproportionate rates of human immunodeficiency virus/acquired immunodeficiency syndrome (HIV/AIDS) among racial and ethnic minorities in the United States. The Centers for Disease Control and Prevention (CDC) conducted an evaluation to assess the influence of MAI in four communities, and the extent to which these communities increased their capacity to meet the HIV prevention needs of racial and ethnic minorities. Design. Retrospective data were collected annually through individual interviews over three years. Individual interviews were conducted with community stakeholders across the three waves of data collection. Data were analyzed using standardized qualitative methods including codebook development, coding, inter-coder agreement assessments, and data interpretation. This paper will highlight one area of inquiry—community stakeholders’ perceptions of the impact of MAI in their communities. Results. Community stakeholders reported that MAI increased capacity to respond to the HIV epidemic and provide services to racial and ethnic minorities. Specifically, MAI was perceived to have increased community empowerment, involvement, and awareness of HIV/AIDS; expanded HIV-related services and organizational self-sufficiency; and improved collaboration and the coordination of services in the community. Although recognizing MAI gave national focus to the impact of the epidemic on minority communities, respondents raised concerns about the implementation process and the lack of sustainability planning. Conclusion. MAI represented an initial national attempt to address the disproportionate rates of HIV/AIDS among racial and ethnic minorities. However, other strategies are also needed to address these significant health disparities. At CDC, steps are currently underway to develop a comprehensive strategy to prevent and reduce the burden of HIV/AIDS among racial and ethnic minorities. As community stakeholders are critical partners in the effort to prevent the spread of HIV, strengthening their capacity and promoting their involvement can help combat the epidemic.

Involving the community in HIV prevention program evaluation.

This article describes the challenges and benefits of involving the community in evaluating an HIV prevention intervention for African American women. The intervention, Women’s Initiative for Sexual Health, was evaluated using a randomized controlled trial. The intervention and the evaluation involved the community in which the intervention was delivered. To solicit criticism and suggestions for the evaluation, the research team conducted a focus group within each of three collaborating community-based organizations. Our goal was to increase the relevance and appropriateness of the evaluation by showing respect for program participants and consequently for cultures other than our own. We recommend that other researchers involve the community in program evaluation.

An Evaluation of Mpowerment on Individual-Level HIV Risk Behavior, Testing, and Psychosocial Factors Among Young MSM of Color: The Monitoring and Evaluation of MP (MEM) Project

Young men who have sex with men (MSM) of color are at increased risk for HIV infection. Mpowerment (MP) is an intervention designed to reduce risky sexual behavior and increase HIV testing among young MSM ages 18-29. From 2009 to 2012, three community-based organizations with support from the U.S. Centers for Disease Control and Prevention evaluated MP among N = 298 participants. Following a repeated measures design, data from 3- and 6-month follow-ups were compared to baseline. HIV testing and self-efficacy for safer sex increased at both follow-up time points; self-acceptance as an MSM was higher at follow-up 2. Condomless anal/vaginal sex was lower at follow-up 1 only. Frequency of exchange of safer sex messages among gay/bisexual/transgender friends was lower at follow-up 1, but similar to baseline at follow-up 2. Exposure to MP was associated with improved perceived positive social norms about safer sex and safer sex messages among gay/bisexual/transgender friends.