Lourdes R. Guerrero

Assessing the Connection Between Health and Education: Identifying Potential Leverage Points for Public Health to Improve School Attendance

OBJECTIVES We examined multiple variables influencing school truancy to identify potential leverage points to improve school attendance. METHODS A cross-sectional observational design was used to analyze inner-city data collected in Los Angeles County, California, during 2010 to 2011. We constructed an ordinal logistic regression model with cluster robust standard errors to examine the association between truancy and various covariates. RESULTS The sample was predominantly Hispanic (84.3%). Multivariable analysis revealed greater truancy among students (1) with mild (adjusted odds ratio [AOR] = 1.57; 95% confidence interval [CI] = 1.22, 2.01) and severe (AOR = 1.80; 95% CI = 1.04, 3.13) depression (referent: no depression), (2) whose parents were neglectful (AOR = 2.21; 95% CI = 1.21, 4.03) or indulgent (AOR = 1.71; 95% CI = 1.04, 2.82; referent: authoritative parents), (3) who perceived less support from classes, teachers, and other students regarding college preparation (AOR = 0.87; 95% CI = 0.81, 0.95), (4) who had low grade point averages (AOR = 2.34; 95% CI = 1.49, 4.38), and (5) who reported using alcohol (AOR = 3.47; 95% CI = 2.34, 5.14) or marijuana (AOR = 1.59; 95% CI = 1.06, 2.38) during the past month. CONCLUSIONS Study findings suggest depression, substance use, and parental engagement as potential leverage points for public health to intervene to improve school attendance.

Using collaborative approaches with a multi-method, multi-site, multi-target intervention: Evaluating the National Research Mentoring Network

Background and purposeThe NIH-funded National Research Mentoring Network (NRMN) aims to increase the representation and success of underrepresented groups (URGs) in biomedical research by enhancing the training and career development of individuals from diverse backgrounds, communities, and cultures. The national scope of NRMN, its wide array of innovative programs in mentor and mentee matching and training across the career spectrum, requires a collaborative evaluation strategy that addresses both internal and external evaluation needs. Due to the variety of programs implemented for each target group, the NRMN program is responsible for its own process and short-term outcome evaluations and the national Coordination and Evaluation Center (CEC) is responsible for assessing the medium and long-term effectiveness of the implemented strategies and program sustainability. Using a collaborative, utilization-focused evaluation framework, both internal NRMN evaluators and the CEC are working to translate findings into information that can be used to make both short term and long-term decisions about the efficacy and reach of the NRMN model. This important information can then inform efforts to institutionalize the current programs and potentially replicate them elsewhere.Program and key highlightsThe overall evaluation of NRMN is guided by both outcome and process questions that are tailored for each target group. The different target groups include faculty and others who serve as mentors, mentees across academic training and career stages, and researchers without a history of independently funded research. NRMN is also building the capacity for training those pursuing biomedical careers by developing “master trainers” for both mentoring and grantsmanship programs in organizations and institutions that can support expanded training efforts aimed at diversifying the biomedical workforce.ImplicationsResults of this evaluation will be used to inform the design and implementation of sustainable, effective, and comprehensive mentoring and career development initiatives that promote diversity in the biomedical research workforce. Our collaborative evaluation design, theoretically-derived measurement instruments, efficient data systems, and timely reporting serve as an example of how to put evaluation principles described into practice for large, multi-site, and multi-dimensional research training programs like NRMN.

Developing the Translational Research Workforce: A Pilot Study of Common Metrics for Evaluating the Clinical and Translational Award KL2 Program

This pilot study describes the career development programs (i.e., NIH KL2 awards) across five Clinical and Translational Science Award (CTSA) institutions within the University of California (UC) system, and examines the feasibility of a set of common metrics for evaluating early outcomes.

A participatory approach to evaluating a national training and institutional change initiative: the BUILD longitudinal evaluation

Background and purposeThe National Institutes of Health (NIH) funds training programs to increase the numbers and skills of scientists who obtain NIH research grants, but few programs have been rigorously evaluated. The sizeable recent NIH investment in developing programs to increase the diversity of the NIH-funded workforce, implemented through the Diversity Program Consortium (DPC), is unusual in that it also funds a Consortium-wide evaluation plan, which spans the activities of the 10 BUilding Infrastructure Leading to Diversity (BUILD) awardees and the National Research Mentoring Network (NRMN). The purpose of this article is to describe the evaluation design and innovations of the BUILD Program on students, faculty, and institutions of the 10 primarily undergraduate BUILD sites.Key highlights of the projectOur approach to this multi-methods quasi-experimental longitudinal evaluation emphasizes stakeholder participation and collaboration. The evaluation plan specifies the major evaluation questions and key short- to long-term outcome measures (or Hallmarks of Success). The Coordination and Evaluation Center (CEC) embarked on a comprehensive evaluation strategy by developing a set of logic models that incorporate the Hallmarks of Success and other outcomes that were collaboratively identified by the DPC. Data were collected from each BUILD site through national surveys from the Higher Education Research Institute at UCLA (HERI), annual followup surveys that align with the HERI instruments, site visits and case studies, program encounter data (“tracker” data), and institutional data. The analytic approach involves comparing changes in Hallmarks (key outcomes) within institutions for biomedical students who participated versus those who did not participate in the BUILD program at each institution, as well as between institution patterns of biomedical students at the BUILD sites, and matched institutions that were not BUILD grantees. Case studies provide insights into the institutionalization of these new programs and help to explain the processes that lead to the observed outcomes.ImplicationsUltimately, the results of the consortium-wide evaluation will be used to inform national policy in higher education and will provide relevant examples of institutional and educational programmatic changes required to diversify the biomedical workforce in the USA.

NIH Career Development Awards in Clinical and Translational Science Award Institutions: Distinguishing Characteristics of Top Performing Sites

To identify and disseminate the organizational characteristics of “top performing” National Institute of Health (NIH) Clinical and Translational Science Awards (CTSA) institutions in regards to career development, using the number of new K awards received per year to rank institutions and comparing these with non‐CTSA institutions.

Geriatric Workforce Development for the Underserved: Using RCQI Methodology to Evaluate the Training of IHSS Caregivers

Caregivers play an important role in the in-home care of community dwelling older adults living with Alzheimer’s disease or related dementias (ADRD); however, many of these caregivers lack training in caring for this vulnerable population. In 2015, we developed and implemented an interactive, community-based, knowledge and skills-based training program for In-Home Supportive Services (IHSS) caregivers. This report shares the results of a process evaluation of this training program as it evolved over the course of three training sessions in Riverside County, California. Our iterative evaluation process reveals the unique needs of training and assessing a population of demographically diverse adult learners and provides guidance for those planning to implement similar training in underserved communities. Factors such as reliance on self-reported abilities, language readability level, and test anxiety may have confounded attempts to capture learner feedback and actual knowledge gains from our caregiver training program.

Instruction and Assessment of Competencies: Two Sides of the Same Coin

Six core competencies, as defined by the Accreditation Council for Graduate Medical Education (ACGME), offer a conceptual framework to address the knowledge and skills needed by students in training and doctors to perform competently. The question of interest for educators is “how” residents perceive they acquire proficiency in the core competencies. An annual survey was sent to all residents at UCLA from 2007 to 2010. Survey questions asked trainees across various programs about the helpfulness of specific predefined learning activities in acquiring the competencies. Responses from 1378 PGY1-3 residents in 12 ACGME-accredited residency programs were analyzed. Patient care activities and observation of attendants and peers were listed as the two most helpful learning activities for acquiring all six core competencies. The findings reinforce the importance of learning from role models during patient care activities and the heterogeneity of learning activities needed for acquiring all the competencies. The fact that competencies are multidimensional and interconnected makes it highly unlikely that a single approach to teaching or assessment will be sufficient for their acquisition. Hence, multiple methods for teaching and learning are necessary for the acquisition of the competencies.