Sunny S. Kim

Learning How Programs Achieve their Impact: Embedding Theory-Driven Process Evaluation and Other Program Learning Mechanisms in Alive & Thrive:

Background Traditionally, impact evaluations have focused primarily on answering what impact programs or interventions have, with less attention to how or why impacts are achieved, or not achieved. The Alive & Thrive initiative, a 6-year program that aims to improve infant and young child feeding (IYCF) practices and reduce stunting in Bangladesh, Ethiopia, and Vietnam, has a specific objective to generate learning on how to achieve and replicate Alive & Thrives impact. Objective In Alive & Thrive, theory-driven process evaluation methods are the primary mechanism through which data are generated to address this objective. This paper focuses on the different methodological approaches that are being utilized, to answer the critical “how”; questions, and to generate information on the many processes and pathways to program impact. Methods We identify four key principles in our methodological approach that guides all process evaluation activities: 1) developing detailed program impact pathway (PIP) models, 2) linking data collection to PIPs utilizing mixed methods and multiple data sources, 3) linking evaluation activities with program implementation timelines, and 4) engaging with the program implementation and management teams. Results Beginning with the launch of the program, we outline the steps that have been taken in the design and implementation of the process evaluations of Alive & Thrive, and provide examples of how these steps have been operationalized in different country contexts. Conclusions This theory-driven and country- and component-specific approach, centered on careful analysis of PIPs, is intended to generate information on implementation and utilization pathways of Alive & Thrives interventions, thereby answering the questions of how impacts are achieved, or why not. This evaluation approach is not without challenges, and we highlight some of these key challenges.

Assessing implementation fidelity of a community-based infant and young child feeding intervention in Ethiopia identifies delivery challenges that limit reach to communities: a mixed-method process evaluation study

BackgroundProgram effectiveness is influenced by the degree and quality of implementation, thus requiring careful examination of delivery processes and how the program is or is not being implemented as intended. Implementation fidelity is defined by adherence to intervention design, exposure or dose, quality of delivery, and participant responsiveness. As part of the process evaluation (PE) of Alive & Thrive in Ethiopia, a large-scale initiative to improve infant and young child feeding (IYCF), we assessed these four fidelity elements along three components of its community-based intervention: training of frontline workers (FLWs), delivery of program tools and messages, and supportive supervision.MethodsData from a qualitative study among three levels of FLWs (n = 54), i.e. supervisors, health extension workers (HEWs), and community volunteers, and among mothers with children under two years of age (n = 60); and cross-sectional PE surveys with FLWs (n = 504) and mothers (n = 750) in two regions (Tigray and SNNPR) were analyzed to examine program fidelity.ResultsThere was strong adherence to the intended cascading design (i.e. transfer of knowledge and information from higher to lower FLW levels) and high exposure to training (95% HEWs and 94% volunteers in Tigray, 68% and 81% respectively in SNNPR). Training quality, assessed by IYCF knowledge and self-reported capacity, was high and increased since baseline. Job aids were used regularly by most supervisors and HEWs, but only 54% of volunteers in Tigray and 39% in SNNPR received them. Quality of program message delivery was lower among volunteers, and aided recall of key messages among mothers was also low. Although FLW supervision exposure was high, content and frequency were irregular.ConclusionsThere is evidence of strong fidelity in training and delivery of program tools and messages at higher FLW levels, but gaps in the reach of these to community volunteers and mothers and variability between regions could limit the potential for impact. Strengthening the linkages between HEWs and volunteers further can help to reach the target households and deliver IYCF results at scale.

Program Impact Pathway Analysis of a Social Franchise Model Shows Potential to Improve Infant and Young Child Feeding Practices in Vietnam

By mapping the mechanisms through which interventions are expected to achieve impact, program impact pathway (PIP) analysis lays out the theoretical causal links between program activities, outcomes, and impacts. This study examines the pathways through which the Alive & Thrive (A&T) social franchise model is intended to improve infant and young child feeding (IYCF) practices in Vietnam. Mixed methods were used, including qualitative interviews with franchise management board members (n = 12), surveys with health providers (n = 120), counseling observations (n = 160), and household surveys (n = 2045). Six PIP components were assessed: 1) franchise management, 2) training and IYCF knowledge of health providers, 3) service delivery, 4) program exposure and utilization, 5) maternal behavioral determinants (knowledge, beliefs, and intentions) toward optimal IYCF practices, and 6) IYCF practices. Data were collected from A&T-intensive areas (A&T-I; mass media + social franchise) and A&T-nonintensive areas (A&T-NI; mass media only) by using a cluster-randomized controlled trial design. Data from 2013 were compared with baseline where similar measures were available. Results indicate that mechanisms are in place for effective management of the franchise system, despite challenges to routine monitoring. A&T training was associated with increased capacity of providers, resulting in higher-quality IYCF counseling (greater technical knowledge and communication skills during counseling) in A&T-I areas. Franchise utilization increased from 10% in 2012 to 45% in 2013 but fell below the expected frequency of 9-15 contacts per mother-child dyad. Improvements in breastfeeding knowledge, beliefs, intentions, and practices were greater among mothers in A&T-I areas than among those in A&T-NI areas. In conclusion, there are many positive changes along the impact pathway of the franchise services, but challenges in utilization and demand creation should be addressed to achieve the full intended impact.

Exposure to Large-Scale Social and Behavior Change Communication Interventions Is Associated with Improvements in Infant and Young Child Feeding Practices in Ethiopia.

Optimal breastfeeding (BF) practices in Ethiopia are far below the government’s targets, and complementary feeding practices are poor. The Alive & Thrive initiative aimed to improve infant and young child feeding (IYCF) practices through large-scale implementation of social and behavior change communication interventions in four regions of Ethiopia. The study assessed the effects of the interventions on IYCF practices and anthropometry over time in two regions–Southern Nations, Nationalities and Peoples Region and Tigray. A pre- and post-intervention adequacy evaluation design was used; repeated cross-sectional surveys of households with children aged 0–23.9 mo (n = 1481 and n = 1494) and with children aged 24–59.9 mo (n = 1481 and n = 1475) were conducted at baseline (2010) and endline (2014), respectively. Differences in outcomes over time were estimated using regression models, accounting for clustering and covariates. Plausibility analyses included tracing recall of key messages and promoted foods and dose-response analyses. We observed improvements in most WHO-recommended IYCF indicators. Early BF initiation and exclusive BF increased by 13.7 and 9.4 percentage points (pp), respectively. Differences for timely introduction of complementary foods, minimum dietary diversity (MDD), minimum meal frequency (MMF), minimum acceptable diet (MAD), and consumption of iron-rich foods were 22.2, 3.3, 26.2, 3.5, and 2.7 pp, respectively. Timely introduction and intake of foods promoted by the interventions improved significantly, but anthropometric outcomes did not. We also observed a dose-response association between health post visits and early initiation of BF (OR: 1.8); higher numbers of home visits by community volunteers and key messages recalled were associated with 1.8–4.4 times greater odds of achieving MDD, MMF, and MAD, and higher numbers of radio spots heard were associated with 3 times greater odds of achieving MDD and MAD. The interventions were associated with plausible improvements in IYCF practices, but large gaps in improving children’s diets in Ethiopia remain, particularly during complementary feeding.

Exposure to mass media and interpersonal counseling has additive effects on exclusive breastfeeding and its psychosocial determinants among Vietnamese mothers.

Abstract The pathways through which behavior change interventions impact breastfeeding practices have not been well studied. This study aimed to examine: (1) the effects of exposure to mass media and interpersonal counseling on exclusive breastfeeding (EBF) and hypothesized psychosocial determinants (i.e. knowledge, intention, beliefs, social norms, and self‐efficacy); and (2) the pathways through which exposure to mass media and interpersonal counseling are associated with EBF. We used survey data from mothers with children < 2 year (n = 2045) from the 2013 process evaluation of Alive & Thrives program in Viet Nam. Multiple linear regression analyses and structural equation modeling were used to estimate effects. Exposure to mass media only, interpersonal counseling only, both or neither was 51%, 5%, 19% and 25%, respectively. Exposure to both mass media and interpersonal counseling had additive effects on EBF as well as on related psychosocial factors, compared with no exposure. For example, EBF prevalence was 26.1 percentage points (pp) higher in the group that received interpersonal counseling only, 3.9 pp higher in the mass media group and 31.8 pp higher in the group that received both interventions. As hypothesized, more than 90% of the total effect of the two interventions on EBF was explained by the psychosocial factors measured. Our findings suggest that combining different behavior change interventions leads to greater changes in psychosocial factors, which in turn positively affects breastfeeding behaviors.

Is Exposure to Animal Feces Harmful to Child Nutrition and Health Outcomes? A Multicountry Observational Analysis

It has recently been hypothesized that exposure to livestock constitutes a significant risk factor for diarrhea and environmental enteric disorder in young children, which may significantly contribute to undernutrition. To date, though, very little research has documented the extent of exposure to animal feces and whether this exposure is associated with child anthropometry in large samples and diverse settings. This study investigates these issues using data from the Alive and Thrive study conducted in rural areas of Ethiopia, Bangladesh, and Vietnam. These surveys used spot-checks to collect data on proxies of hygiene behaviors such as the cleanliness of mothers, young children, and the homestead environment, including the presence of animal feces. Animal feces were visible in 38–42% of household compounds across the three countries and were positively associated with household livestock ownership and negatively associated with maternal and child cleanliness. One-sided tests from multivariate least squares models for children 6–24 months of age indicate that the presence of animal feces is significantly and negatively associated with child height-for-age z scores in Ethiopia (β = −0.22), Bangladesh (β = −0.13), and in a pooled sample (β = −0.11), but not in Vietnam. There is also suggestive evidence that animal feces may be positively associated with diarrhea symptoms in Bangladesh. The results in this article, therefore, contribute to a growing body of evidence suggesting that animal ownership may pose a significant risk to child nutrition and health outcomes in developing countries.

Factors influencing maternal nutrition practices in a large scale maternal, newborn and child health program in Bangladesh

Improving maternal nutrition practices during pregnancy is essential to save lives and improve health outcomes for both mothers and babies. This paper examines the maternal, household, and health service factors influencing maternal nutrition practices in the context of a large scale maternal, newborn, and child health (MNCH) program in Bangladesh. Data were from a household survey of pregnant (n = 600) and recently delivered women (n = 2,000). Multivariate linear and logistic regression analyses were used to examine the determinants of three outcomes: consumption of iron and folic acid (IFA) tablets, calcium tablets, and diverse diets. Women consumed 94 ± 68 IFA and 82 ± 66 calcium tablets (out of 180 as recommended) during pregnancy, and only half of them consumed an adequately diverse diet. Good nutrition knowledge was the key maternal factor associated with higher consumption of IFA (β = 32.5, 95% CI: 19.5, 45.6) and calcium tablets (β ~31.9, 95% CI: 20.9, 43.0) and diverse diet (OR = 1.8, 95% CI: 1.0–3.1), compared to poor knowledge. Women’s self-efficacy in achieving the recommended practices and perception of enabling social norms were significantly associated with dietary diversity. At the household level, women who reported a high level of husband’s support were more likely to consume IFA (β = 25.0, 95% CI: 18.0, 32.1) and calcium (β = 26.6, 95% CI: 19.4, 33.7) tablets and diverse diet (OR = 1.9, 95% CI: 1.2, 3.3), compared to those who received low support. Health service factors associated with higher intakes of IFA and calcium tablets were early and more prenatal care visits and receipt of free supplements. Combined exposure to several of these factors was attributed to the consumption of an additional 46 IFA and 53 calcium tablets and 17% higher proportions of women consuming diverse diets. Our study shows that improving knowledge, self-efficacy and perceptions of social norms among pregnant women, and increasing husbands’ support, early registration in prenatal care, and provision of free supplements will largely improve maternal nutrition practices.

Integrating Nutrition Interventions into an Existing Maternal, Neonatal, and Child Health Program Increased Maternal Dietary Diversity, Micronutrient Intake, and Exclusive Breastfeeding Practices in Bangladesh: Results of a Cluster-Randomized Program Evaluation

Background: Maternal undernutrition is a major concern globally, contributing to poor birth outcomes. Limited evidence exists on delivering multiple interventions for maternal nutrition simultaneously. Alive & Thrive addressed this gap by integrating nutrition-focused interpersonal counseling, community mobilization, distribution of free micronutrient supplements, and weight-gain monitoring through an existing Maternal, Neonatal, and Child Health (MNCH) program in Bangladesh. Objectives: We evaluated the effect of providing nutrition-focused MNCH compared with standard MNCH (antenatal care with standard nutrition counseling) on coverage of nutrition interventions, maternal dietary diversity, micronutrient supplement intake, and early breastfeeding practices. Methods: We used a cluster-randomized design with cross-sectional surveys at baseline (2015) and endline (2016) (n ∼ 300 and 1000 pregnant or recently delivered women, respectively, per survey round). We derived difference-in-difference effect estimates, adjusted for geographic clustering and infant age and sex. Results: Coverage of interpersonal counseling was high; >90% of women in the nutrition-focused MNCH group were visited at home by health workers for maternal nutrition and breastfeeding counseling. The coverage of community mobilization activities was ∼50%. Improvements were significantly greater in the nutrition-focused MNCH group than in the standard MNCH group for consumption of iron and folic acid [effect: 9.8 percentage points (pp); 46 tablets] and calcium supplements (effect: 12.8 pp; 50 tablets). Significant impacts were observed for the number of food groups consumed (effect: 1.6 food groups), percentage of women who consumed ≥5 food groups/d (effect: 30.0 pp), and daily intakes of several micronutrients. A significant impact was also observed for exclusive breastfeeding (EBF; effect: 31 pp) but not for early initiation of breastfeeding. Conclusions: Addressing nutrition during pregnancy by delivering interpersonal counseling and community mobilization, providing free supplements, and ensuring weight-gain monitoring through an existing MNCH program improved maternal dietary diversity, micronutrient supplement consumption, and EBF practices. This trial was registered at clinicaltrials.gov as NCT02745249.

Changes in the policy environment for infant and young child feeding in Vietnam, Bangladesh, and Ethiopia, and the role of targeted advocacy

BackgroundThere is limited literature examining shifts in policy environments for nutrition and infant and young child feeding (IYCF) over time, and on the potential contribution of targeted advocacy to improved policy environments in low- and middle-income countries. This study tracked changes in the policy environment over a four-year period in three countries, and examined the role of targeted nutrition and IYCF advocacy strategies by a global initiative.MethodsQualitative methods, including key informant interviews, social network mapping, document and literature review, and event tracking, were used to gather data on nutrition and IYCF policies and programs, actor networks, and perceptions and salience of nutrition as an issue in 2010 and 2014 in Bangladesh, Ethiopia, and Vietnam. Theoretical frameworks from the policy sciences were used to analyze policy change over time, and drivers of change, across countries.ResultsThe written policy environment improved to differing extents in each country. By 2014, the discourse in all three countries mirrored international priorities of stunting reduction and exclusive breastfeeding. Yet competing nutrition priorities such as acute malnutrition, food insecurity, and nutrition transitions remained in each context. Key actor groups in each country were government, civil society, development partners and the private sector. Infant formula companies, in particular, emerged as key players against enforcement of IYCF legislation. The role of a targeted IYCF advocacy and policy support initiative was well-recognized in supporting multiple facets of the policy environment in each country, ranging from alliances to legislation and implementation support. Despite progress, however, government commitment to funding, implementation, and enforcement is still emerging in each country, thus challenging the potential impact of new and improved policies.ConclusionTargeted policy advocacy can catalyze change in national nutrition and IYCF policy environments, especially actor commitment, policy guidance, and legislation. Implementation constraints – financing, capacity and commitment of systems, and competing priorities and actors – are essential to address to sustain further progress. The lack of pressing political urgency for nutrition and IYCF, and the uncertain role of international networks in national policy spaces, has implications for the potential for change.

Supply- and Demand-Side Factors Influencing Utilization of Infant and Young Child Feeding Counselling Services in Viet Nam

Adequate utilization of services is critical to maximize the impact of counselling on infant and young child feeding (IYCF), but little is known about factors affecting utilization. Our study examined supply- and demand-side factors associated with the utilization of IYCF counselling services in Viet Nam. We used survey data from mothers with children <2y (n = 1,008) and health staff (n = 60) from the evaluation of a program that embedded IYCF counseling into the existing government health system. The frequency of never users, one-time users, repeat users, and achievers of the recommended minimum number of visits at health facilities were 45.1%, 13.0%, 28.4% and 13.5%, respectively. Poisson regression showed that demand-generation strategies, especially invitation cards, were the key factors determining one-time use (Prevalence ratio, PR 3.0, 95% CI: 2.2–4.2), repeated use (PR 3.2, 95% CI: 2.4–4.2), and achievement of minimum visits (PR 5.5, 95% CI: 3.6–8.4). Higher maternal education was associated with higher utilization both for one-time and repeated use. Being a farmer, belonging to an ethnic minority, and having a wasted child were associated with greater likelihood of achieving the minimum recommended number of visits, whereas child stunting or illness were not. Distance to health center was a barrier to repeated visits. Among supply-side factors, good counselling skills (PR: 1.3–1.8) was the most important factor associated with any service use, whereas longer employment duration and greater work pressure of health center staff were associated with lower utilization. Population attributable risk estimations showed that an additional 25% of the population would have achieved the minimum number of visits if exposed to three demand-generation strategies, and further increased to 49% if the health staff had good counseling skills and low work pressure. Our study provides evidence that demand-generation strategies are essential to increase utilization of facility-based IYCF counselling services in Viet Nam, and may be relevant for increasing and sustaining use of nutrition services in similar contexts.