Pavel V. Ovseiko

Assessing research impact in academic clinical medicine: a study using Research Excellence Framework pilot impact indicators

BackgroundFunders of medical research the world over are increasingly seeking, in research assessment, to complement traditional output measures of scientific publications with more outcome-based indicators of societal and economic impact. In the United Kingdom, the Higher Education Funding Council for England (HEFCE) developed proposals for the Research Excellence Framework (REF) to allocate public research funding to higher education institutions, inter alia, on the basis of the social and economic impact of their research. In 2010, it conducted a pilot exercise to test these proposals and refine impact indicators and criteria.MethodsThe impact indicators proposed in the 2010 REF impact pilot exercise are critically reviewed and appraised using insights from the relevant literature and empirical data collected for the University of Oxford’s REF pilot submission in clinical medicine. The empirical data were gathered from existing administrative sources and an online administrative survey carried out by the university’s Medical Sciences Division among 289 clinical medicine faculty members (48.1% response rate).ResultsThe feasibility and scope of measuring research impact in clinical medicine in a given university are assessed. Twenty impact indicators from seven categories proposed by HEFCE are presented; their strengths and limitations are discussed using insights from the relevant biomedical and research policy literature.ConclusionsWhile the 2010 pilot exercise has confirmed that the majority of the proposed indicators have some validity, there are significant challenges in operationalising and measuring these indicators reliably, as well as in comparing evidence of research impact across different cases in a standardised manner. It is suggested that the public funding agencies, medical research charities, universities, and the wider medical research community work together to develop more robust methodologies for capturing and describing impact, including more valid and reliable impact indicators.

Organizational culture in an academic health center: an exploratory study using a competing values framework.

Purpose Implementing cultural change and aligning organizational cultures could enhance innovation, quality, safety, and job satisfaction. The authors conducted this mixed-methods study to assess academic physician–scientists’ perceptions of the current and preferred future organizational culture at a university medical school and its partner health system. Method In October 2010, the authors surveyed academic physicians and scientists jointly employed by the University of Oxford and its local, major partner health system. The survey included the U.S. Veterans Affairs Administration’s 14-item Competing Values Framework instrument and two extra items prompting respondents to identify their substantive employer and to provide any additional open-ended comments. Results Of 436 academic physicians and scientists, 170 (39%) responded. Of these, 69 (41%) provided open-ended comments. Dominant hierarchical culture, moderate rational and team cultures, and underdeveloped entrepreneurial culture characterized the health system culture profile. The university profile was more balanced, with strong rational and entrepreneurial cultures, and moderate-to-strong hierarchical and team cultures. The preferred future culture (within five years) would emphasize team and entrepreneurial cultures and—to a lesser degree—rational culture, and would deemphasize hierarchical culture. Conclusions Whereas the university and the health system currently have distinct organizational cultures, academic physicians and scientists would prefer the same type of culture across the two organizations so that both could more successfully pursue the shared mission of academic medicine. Further research should explore strengthening the validity and reliability of the organizational culture instrument for academic medicine and building an evidence base of effective culture change strategies and interventions.

A global call for action to include gender in research impact assessment

Global investment in biomedical research has grown significantly over the last decades, reaching approximately a quarter of a trillion US dollars in 2010. However, not all of this investment is distributed evenly by gender. It follows, arguably, that scarce research resources may not be optimally invested (by either not supporting the best science or by failing to investigate topics that benefit women and men equitably). Women across the world tend to be significantly underrepresented in research both as researchers and research participants, receive less research funding, and appear less frequently than men as authors on research publications. There is also some evidence that women are relatively disadvantaged as the beneficiaries of research, in terms of its health, societal and economic impacts. Historical gender biases may have created a path dependency that means that the research system and the impacts of research are biased towards male researchers and male beneficiaries, making it inherently difficult (though not impossible) to eliminate gender bias. In this commentary, we – a group of scholars and practitioners from Africa, America, Asia and Europe – argue that gender-sensitive research impact assessment could become a force for good in moving science policy and practice towards gender equity. Research impact assessment is the multidisciplinary field of scientific inquiry that examines the research process to maximise scientific, societal and economic returns on investment in research. It encompasses many theoretical and methodological approaches that can be used to investigate gender bias and recommend actions for change to maximise research impact. We offer a set of recommendations to research funders, research institutions and research evaluators who conduct impact assessment on how to include and strengthen analysis of gender equity in research impact assessment and issue a global call for action.

Organizational models of emerging academic health science centers in England.

Recent government policy initiatives to foster medical innovation and high-quality care in England have prompted academic and clinical leaders to develop new organizational models to support the tripartite Flexnerian mission of academic medicine. Medical schools and health care providers have responded by aligning their missions and creating integrated governance structures that strengthen their partnerships. In March 2009, the government officially designated five academic-clinical partnerships as Englands first academic health science centers (AHSCs). As academic-clinical integration is likely to continue, future AHSC leaders could benefit from an analysis of models for organizing medical school-clinical enterprise relationships in Englands emerging AHSCs. In addition, as the United States ponders health systems reform and universal coverage, U.S. medical leaders may benefit from insight into the workings of academic medicine in Englands universal health system. In this article, the authors briefly characterize the organization and financing of the National Health Service and how it supports academic medicine. They review the policy behind the designation of AHSCs. Then, the authors describe contrasting organizational models adopted in two of the newly designated AHSCs and analyze these models using a framework derived from U.S. literature. The authors conclude by outlining the major challenges facing academic medicine in England and offer suggestions for future research collaborations between leaders of AHSCs in the United States and England.

Improving accountability through alignment: the role of academic health science centres and networks in England

BackgroundAs in many countries around the world, there are high expectations on academic health science centres and networks in England to provide high-quality care, innovative research, and world-class education, while also supporting wealth creation and economic growth. Meeting these expectations increasingly depends on partnership working between university medical schools and teaching hospitals, as well as other healthcare providers. However, academic-clinical relationships in England are still characterised by the “unlinked partners” model, whereby universities and their partner teaching hospitals are neither fiscally nor structurally linked, creating bifurcating accountabilities to various government and public agencies.DiscussionThis article focuses on accountability relationships in universities and teaching hospitals, as well as other healthcare providers that form core constituent parts of academic health science centres and networks. The authors analyse accountability for the tripartite mission of patient care, research, and education, using a four-fold typology of accountability relationships, which distinguishes between hierarchical (bureaucratic) accountability, legal accountability, professional accountability, and political accountability. Examples from North West London suggest that a number of mechanisms can be used to improve accountability for the tripartite mission through alignment, but that the simple creation of academic health science centres and networks is probably not sufficient.SummaryAt the heart of the challenge for academic health science centres and networks is the separation of accountabilities for patient care, research, and education in different government departments. Given that a fundamental top-down system redesign is now extremely unlikely, local academic and clinical leaders face the challenge of aligning their institutions as a matter of priority in order to improve accountability for the tripartite mission from the bottom up. It remains to be seen which alignment mechanisms are most effective, and whether they are strong enough to counter the separation of accountabilities for the tripartite mission at the national level, the on-going structural fragmentation of the health system in England, and the unprecedented financial challenges that it faces. Future research should focus on determining the comparative effectiveness of different alignment mechanisms, developing standardised metrics and key performance indicators, evaluating and assessing academic health science centres and networks, and empirically addressing leadership issues.

Implementation of collaborative governance in cross-sector innovation and education networks: evidence from the National Health Service in England

BackgroundIncreasingly, health policy-makers and managers all over the world look for alternative forms of organisation and governance in order to add more value and quality to their health systems. In recent years, the central government in England mandated several cross-sector health initiatives based on collaborative governance arrangements. However, there is little empirical evidence that examines local implementation responses to such centrally-mandated collaborations.MethodsData from the national study of Health Innovation and Education Clusters (HIECs) are used to provide comprehensive empirical evidence about the implementation of collaborative governance arrangements in cross-sector health networks in England. The study employed a mixed-methods approach, integrating both quantitative and qualitative data from a national survey of the entire population of HIEC directors (N = 17; response rate = 100%), a group discussion with 7 HIEC directors, and 15 in-depth interviews with HIEC directors and chairs.ResultsThe study provides a description and analysis of local implementation responses to the central government mandate to establish HIECs. The latter represent cross-sector health networks characterised by a vague mandate with the provision of a small amount of new resources. Our findings indicate that in the case of HIECs such a mandate resulted in the creation of rather fluid and informal partnerships, which over the period of three years made partial-to-full progress on governance activities and, in most cases, did not become self-sustaining without government funding.ConclusionThis study has produced valuable insights into the implementation responses in HIECs and possibly other cross-sector collaborations characterised by a vague mandate with the provision of a small amount of new resources. There is little evidence that local dominant coalitions appropriated the central HIEC mandate to their own ends. On the other hand, there is evidence of interpretation and implementation of the central mandate by HIEC leaders to serve their local needs. These findings augur well for Academic Health Science Networks, which pick up the mantle of large-scale, cross-sector collaborations for health and innovation. This study also highlights that a supportive policy environment and sufficient time would be crucial to the successful implementation of new cross-sector health collaborations.

Closing the gender leadership gap: a multi-centre cross-country comparison of women in management and leadership in academic health centres in the European Union.

BackgroundWomen’s participation in medicine and the need for gender equality in healthcare are increasingly recognised, yet little attention is paid to leadership and management positions in large publicly funded academic health centres. This study illustrates such a need, taking the case of four large European centres: Charité – Universitätsmedizin Berlin (Germany), Karolinska Institutet (Sweden), Medizinische Universität Wien (Austria), and Oxford Academic Health Science Centre (United Kingdom).CaseThe percentage of female medical students and doctors in all four countries is now well within the 40–60% gender balance zone. Women are less well represented among specialists and remain significantly under-represented among senior doctors and full professors. All four centres have made progress in closing the gender leadership gap on boards and other top-level decision-making bodies, but a gender leadership gap remains relevant. The level of achieved gender balance varies significantly between the centres and largely mirrors country-specific welfare state models, with more equal gender relations in Sweden than in the other countries. Notably, there are also similar trends across countries and centres: gender inequality is stronger within academic enterprises than within hospital enterprises and stronger in middle management than at the top level. These novel findings reveal fissures in the ‘glass ceiling’ effects at top-level management, while the barriers for women shift to middle-level management and remain strong in academic positions. The uneven shifts in the leadership gap are highly relevant and have policy implications.ConclusionSetting gender balance objectives exclusively for top-level decision-making bodies may not effectively promote a wider goal of gender equality. Academic health centres should pay greater attention to gender equality as an issue of organisational performance and good leadership at all levels of management, with particular attention to academic enterprises and newly created management structures. Developing comprehensive gender-sensitive health workforce monitoring systems and comparing progress across academic health centres in Europe could help to identify the gender leadership gap and utilise health human resources more effectively.

Personalized medical education: Reappraising clinician-scientist training.

Revitalizing the Oslerian ideal of the clinician-scientist-teacher may help in the training of the next generation of translational researchers. Revitalizing the Oslerian ideal of the clinician-scientist-teacher may help in the training of the next generation of translational researchers.

Markers of achievement for assessing and monitoring gender equity in translational research organisations: a rationale and study protocol

Introduction Translational research organisations (TROs) are a core component of the UKs expanding research base. Equity of career opportunity is key to ensuring a diverse and internationally competitive workforce. The UK now requires TROs to demonstrate how they are supporting gender equity. Yet, the evidence base for documenting such efforts is sparse. This study is designed to inform the acceleration of womens advancement and leadership in two of the UKs leading TROs—the National Institute for Health Research (NIHR) Biomedical Research Centres (BRCs) in Oxford and London—through the development, application and dissemination of a conceptual framework and measurement tool. Methods and analysis A cross-sectional retrospective evaluation. A conceptual framework with markers of achievement and corresponding candidate metrics has been specifically designed for this study based on an adapted balanced scorecard approach. It will be refined with an online stakeholder consultation and semistructured interviews to test the face validity and explore practices and mechanisms that influence gender equity in the given settings. Data will be collected via the relevant administrative databases. A comparison of two funding periods (2007–2012 and 2012–2017) will be carried out. Ethics and dissemination The University of Oxford Clinical Trials and Research Governance Team and the Research and Development Governance Team of Guys and St Thomas’ National Health Service (NHS) Foundation Trust reviewed the study and deemed it exempt from full ethics review. The results of the study will be used to inform prospective planning and monitoring within the participating NIHR BRCs with a view to accelerating womens advancement and leadership. Both the results of the study and its methodology will be further disseminated to academics and practitioners through the networks of collaborating TROs, relevant conferences and articles in peer-reviewed journals.

Organisational culture and post-merger integration in an academic health centre: a mixed-methods study

BackgroundAround the world, the last two decades have been characterised by an increase in the numbers of mergers between healthcare providers, including some of the most prestigious university hospitals and academic health centres. However, many mergers fail to bring the anticipated benefits, and successful post-merger integration in university hospitals and academic health centres is even harder to achieve. An increasing body of literature suggests that organisational culture affects the success of post-merger integration and academic-clinical collaboration.MethodsThis paper reports findings from a mixed-methods single-site study to examine 1) the perceptions of organisational culture in academic and clinical enterprises at one National Health Service (NHS) trust, and 2) the major cultural issues for its post-merger integration with another NHS trust and strategic partnership with a university. From the entire population of 72 clinician-scientists at one of the legacy NHS trusts, 38 (53%) completed a quantitative Competing Values Framework survey and 24 (33%) also provided qualitative responses. The survey was followed up by semi-structured interviews with six clinician-scientists and a group discussion including five senior managers.ResultsThe cultures of two legacy NHS trusts differed and were primarily distinct from the culture of the academic enterprise. Major cultural issues were related to the relative size, influence, and history of the legacy NHS trusts, and the implications of these for respective identities, clinical services, and finances. Strategic partnership with a university served as an important ameliorating consideration in reaching trust merger. However, some aspects of university entrepreneurial culture are difficult to reconcile with the NHS service delivery model and may create tension.ConclusionsThere are challenges in preserving a more desirable culture at one of the legacy NHS trusts, enhancing cultures in both legacy NHS trusts during their post-merger integration, and in aligning academic and clinical cultures following strategic partnership with a university. The seeds of success may be found in current best practice, good will, and a near identical ideal of the future preferred culture. Strong, fair leadership will be required both nationally and locally for the success of mergers and post-merger integration in university hospitals and academic health centres.