Romi Haas

Breakeven, Cost Benefit, Cost Effectiveness, and Willingness to Pay for Web-Based Versus Face-to-Face Education Delivery for Health Professionals

Background The introduction of Web-based education and open universities has seen an increase in access to professional development within the health professional education marketplace. Economic efficiencies of Web-based education and traditional face-to-face educational approaches have not been compared under randomized controlled trial conditions. Objective To compare costs and effects of Web-based and face-to-face short courses in falls prevention education for health professionals. Methods We designed two short courses to improve the clinical performance of health professionals in exercise prescription for falls prevention. One was developed for delivery in face-to-face mode and the other for online learning. Data were collected on learning outcomes including participation, satisfaction, knowledge acquisition, and change in practice, and combined with costs, savings, and benefits, to enable a break-even analysis from the perspective of the provider, cost-effectiveness analysis from the perspective of the health service, and cost-benefit analysis from the perspective of the participant. Results Face-to-face and Web-based delivery modalities produced comparable outcomes for participation, satisfaction, knowledge acquisition, and change in practice. Break-even analysis identified the Web-based educational approach to be robustly superior to face-to-face education, requiring a lower number of enrollments for the program to reach its break-even point. Cost-effectiveness analyses from the perspective of the health service and cost-benefit analysis from the perspective of the participant favored face-to-face education, although the outcomes were contingent on the sensitivity analysis applied (eg, the fee structure used). Conclusions The Web-based educational approach was clearly more efficient from the perspective of the education provider. In the presence of relatively equivocal results for comparisons from other stakeholder perspectives, it is likely that providers would prefer to deliver education via a Web-based medium. Trial Registration Australian New Zealand Clinical Trials Registry (ACTRN): 12610000135011; http://www.anzctr.org.au/trial_view.aspx?id=335135 (Archived by WebCite at http://www.webcitation.org/668POww4L)

Effectiveness of Web-Based Versus Face-To-Face Delivery of Education in Prescription of Falls-Prevention Exercise to Health Professionals: Randomized Trial

Background Exercise is an effective intervention for the prevention of falls; however, some forms of exercises have been shown to be more effective than others. There is a need to identify effective and efficient methods for training health professionals in exercise prescription for falls prevention. Objective The objective of our study was to compare two approaches for training clinicians in prescribing exercise to prevent falls. Methods This study was a head-to-head randomized trial design. Participants were physiotherapists, occupational therapists, nurses, and exercise physiologists working in Victoria, Australia. Participants randomly assigned to one group received face-to-face traditional education using a 1-day seminar format with additional video and written support material. The other participants received Web-based delivery of the equivalent educational material over a 4-week period with remote tutor facilitation. Outcomes were measured across levels 1 to 3 of Kirkpatrick’s hierarchy of educational outcomes, including attendance, adherence, satisfaction, knowledge, and self-reported change in practice. Results Of the 166 participants initially recruited, there was gradual attrition from randomization to participation in the trial (n = 67 Web-based, n = 68 face-to-face), to completion of the educational content (n = 44 Web-based, n = 50 face-to-face), to completion of the posteducation examinations (n = 43 Web-based, n = 49 face-to-face). Participant satisfaction was not significantly different between the intervention groups: mean (SD) satisfaction with content and relevance of course material was 25.73 (5.14) in the Web-based and 26.11 (5.41) in the face-to-face group; linear regression P = .75; and mean (SD) satisfaction with course facilitation and support was 11.61 (2.00) in the Web-based and 12.08 (1.54) in the face-to-face group; linear regression P = .25. Knowledge test results were comparable between the Web-based and face-to-face groups: median (interquartile range [IQR]) for the Web-based group was 90.00 (70.89–90.67) and for the face-to-face group was 80.56 (70.67–90.00); rank sum P = .07. The median (IQR) scores for the exercise assignment were also comparable: Web-based, 78.6 (68.5–85.1), and face-to-face, 78.6 (70.8–86.9); rank sum P = .61. No significant difference was identified in Kirkpatrick’s hierarchy domain change in practice: mean (SD) Web-based, 21.75 (4.40), and face-to-face, 21.88 (3.24); linear regression P = .89. Conclusion Web-based and face-to-face approaches to the delivery of education to clinicians on the subject of exercise prescription for falls prevention produced equivalent results in all of the outcome domains. Practical considerations should arguably drive choice of delivery method, which may favor Web-based provision for its ability to overcome access issues for health professionals in regional and remote settings. Trial Registration Australian New Zealand Clinical Trials Registry number: ACTRN12610000135011; http://www.anzctr.org.au/ACTRN12610000135011.aspx (Archived by WebCite at http://www.webcitation.org/63MicDjPV)

Clinical Decision Making in Exercise Prescription for Fall Prevention

Background Physical therapists often prescribe exercises for fall prevention. Understanding the factors influencing the clinical decision-making processes used by expert physical therapists working in specialist fall and balance clinics may assist other therapists in prescribing exercises for fall prevention with greater efficacy. Objectives The objective of this study was to describe the factors influencing the clinical decision-making processes used by expert physical therapists to prescribe exercises for fall prevention. Design This investigation was a qualitative study from a phenomenological perspective. Methods Semistructured telephone interviews were conducted with 24 expert physical therapists recruited primarily from the Victorian Falls Clinic Coalition. Interviews focused on 3 exercise prescription contexts: face-to-face individual therapy, group exercise programs, and home exercise programs. Interviews elicited information about therapist practices and the therapist, patient, and environmental factors influencing the clinical decision-making processes for the selection of exercise setting, type, dosage (intensity, quantity, rest periods, duration, and frequency), and progression. Strategies for promoting adherence and safety were also discussed. Data were analyzed with a framework approach by 3 investigators. Results Participants described highly individualized exercise prescription approaches tailored to address key findings from physical assessments. Dissonance between prescribing a program that was theoretically correct on the basis of physiological considerations and prescribing one that a client would adhere to was evident. Safety considerations also were highly influential on the exercise type and setting prescribed. Terminology for describing the intensity of balance exercises was vague relative to terminology for describing the intensity of strength exercises. Conclusions Physical therapists with expertise in fall prevention adopted an individualized approach to exercise prescription that was based on physical assessment findings rather than “off-the-shelf” exercise programs commonly used in fall prevention research. Training programs for people who prescribe exercises for older adults at risk of falling should encompass these findings.

Study protocol for two randomized controlled trials examining the effectiveness and safety of current weekend allied health services and a new stakeholder-driven model for acute medical/surgical patients versus no weekend allied health services

BackgroundDisinvestment from inefficient or ineffective health services is a growing priority for health care systems. Provision of allied health services over the weekend is now commonplace despite a relative paucity of evidence supporting their provision. The relatively high cost of providing this service combined with the paucity of evidence supporting its provision makes this a potential candidate for disinvestment so that resources consumed can be used in other areas.This study aims to determine the effectiveness, cost-effectiveness and safety of the current model of weekend allied health service and a new stakeholder-driven model of weekend allied health service delivery on acute medical and surgical wards compared to having no weekend allied health service.Methods/DesignTwo stepped wedge, cluster randomised trials of weekend allied health services will be conducted in six acute medical/surgical wards across two public metropolitan hospitals in Melbourne (Australia). Wards have been chosen to participate by management teams at each hospital. The allied health services to be investigated will include physiotherapy, occupational therapy, speech therapy, dietetics, social work and allied health assistants. At baseline, all wards will be receiving weekend allied health services. Study 1 intervention will be the sequential disinvestment (roll-in) of the current weekend allied health service model from each participating ward in monthly intervals and study 2 will be the roll-out of a new stakeholder-driven model of weekend allied health service delivery. The order in which weekend allied health services will be rolled in and out amongst participating wards will be determined randomly. This trial will be conducted in each of the two participating hospitals at a different time interval. Primary outcomes will be length of stay, rate of unplanned hospital readmission within 28 days and rate of adverse events. Secondary outcomes will be number of complaints and compliments, staff absenteeism, and patient discharge destination, satisfaction, and functional independence at discharge.DiscussionThis is the world’s first application of the recently described non-inferiority (roll-in) stepped wedge trial design, and the largest investigation of the effectiveness of weekend allied health services on acute medical surgical wards to date.Trial registrationAustralian New Zealand Clinical Trials Registry.Registration number: ACTRN12613001231730 (first study) and ACTRN12613001361796 (second study).Was this trial prospectively registered?: Yes.Date registered: 8 November 2013 (first study), 12 December 2013 (second study).Anticipated completion: June 2015.Protocol version: 1.Role of trial sponsor: KP and DL are directly employed by one of the trial sponsors, their roles were: KP assisted with overall development of research design and assisted with overall project management; DL contributed to project management, administration and communications strategy.

Data collection methods in health services research: hospital length of stay and discharge destination.

BACKGROUND Hospital length of stay and discharge destination are important outcome measures in evaluating effectiveness and efficiency of health services. Although hospital administrative data are readily used as a data collection source in health services research, no research has assessed this data collection method against other commonly used methods. OBJECTIVE Determine if administrative data from electronic patient management programs are an effective data collection method for key hospital outcome measures when compared with alternative hospital data collection methods. METHOD Prospective observational study comparing the completeness of data capture and level of agreement between three data collection methods; manual data collection from ward-based sources, administrative data from an electronic patient management program (i.PM), and inpatient medical record review (gold standard) for hospital length of stay and discharge destination. RESULTS Manual data collection from ward-based sources captured only 376 (69%) of the 542 inpatient episodes captured from the hospital administrative electronic patient management program. Administrative data from the electronic patient management program had the highest levels of agreement with inpatient medical record review for both length of stay (93.4%) and discharge destination (91%) data. CONCLUSION This is the first paper to demonstrate differences between data collection methods for hospital length of stay and discharge destination. Administrative data from an electronic patient management program showed the highest level of completeness of capture and level of agreement with the gold standard of inpatient medical record review for both length of stay and discharge destination, and therefore may be an acceptable data collection method for these measures.

Impact of disinvestment from weekend allied health services across acute medical and surgical wards: 2 stepped-wedge cluster randomised controlled trials

Background Disinvestment (removal, reduction, or reallocation) of routinely provided health services can be difficult when there is little published evidence examining whether the services are effective or not. Evidence is required to understand if removing these services produces outcomes that are inferior to keeping such services in place. However, organisational imperatives, such as budget cuts, may force healthcare providers to disinvest from these services before the required evidence becomes available. There are presently no experimental studies examining the effectiveness of allied health services (e.g., physical therapy, occupational therapy, and social work) provided on weekends across acute medical and surgical hospital wards, despite these services being routinely provided internationally. The aim of this study was to understand the impact of removing weekend allied health services from acute medical and surgical wards using a disinvestment-specific non-inferiority research design. Methods and findings We conducted 2 stepped-wedge cluster randomised controlled trials between 1 February 2014 and 30 April 2015 among patients on 12 acute medical or surgical hospital wards spread across 2 hospitals. The hospitals involved were 2 metropolitan teaching hospitals in Melbourne, Australia. Data from n = 14,834 patients were collected for inclusion in Trial 1, and n = 12,674 in Trial 2. Trial 1 was a disinvestment-specific non-inferiority stepped-wedge trial where the ‘current’ weekend allied health service was incrementally removed from participating wards each calendar month, in a random order, while Trial 2 used a conventional non-inferiority stepped-wedge design, where a ‘newly developed’ service was incrementally reinstated on the same wards as in Trial 1. Primary outcome measures were patient length of stay (proportion staying longer than expected and mean length of stay), the proportion of patients experiencing any adverse event, and the proportion with an unplanned readmission within 28 days of discharge. The ‘no weekend allied health service’ condition was considered to be not inferior if the 95% CIs of the differences between this condition and the condition with weekend allied health service delivery were below a 2% increase in the proportion of patients who stayed in hospital longer than expected, a 2% increase in the proportion who had an unplanned readmission within 28 days, a 2% increase in the proportion who had any adverse event, and a 1-day increase in the mean length of stay. The current weekend allied health service included physical therapy, occupational therapy, speech therapy, dietetics, social work, and allied health assistant services in line with usual care at the participating sites. The newly developed weekend allied health service allowed managers at each site to reprioritise tasks being performed and the balance of hours provided by each professional group and on which days they were provided. Analyses conducted on an intention-to-treat basis demonstrated that there was no estimated effect size difference between groups in the proportion of patients staying longer than expected (weekend versus no weekend; estimated effect size difference [95% CI], p-value) in Trial 1 (0.40 versus 0.38; estimated effect size difference 0.01 [−0.01 to 0.04], p = 0.31, CI was both above and below non-inferiority margin), but the proportion staying longer than expected was greater with the newly developed service compared to its no weekend service control condition (0.39 versus 0.40; estimated effect size difference 0.02 [0.01 to 0.04], p = 0.04, CI was completely below non-inferiority margin) in Trial 2. Trial 1 and 2 findings were discordant for the mean length of stay outcome (Trial 1: 5.5 versus 6.3 days; estimated effect size difference 1.3 days [0.9 to 1.8], p < 0.001, CI was both above and below non-inferiority margin; Trial 2: 5.9 versus 5.0 days; estimated effect size difference −1.6 days [−2.0 to −1.1], p < 0.001, CI was completely below non-inferiority margin). There was no difference between conditions for the proportion who had an unplanned readmission within 28 days in either trial (Trial 1: 0.01 [−0.01 to 0.03], p = 0.18, CI was both above and below non-inferiority margin; Trial 2: −0.01 [−0.02 to 0.01], p = 0.62, CI completely below non-inferiority margin). There was no difference between conditions in the proportion of patients who experienced any adverse event in Trial 1 (0.01 [−0.01 to 0.03], p = 0.33, CI was both above and below non-inferiority margin), but a lower proportion of patients had an adverse event in Trial 2 when exposed to the no weekend allied health condition (−0.03 [−0.05 to −0.004], p = 0.02, CI completely below non-inferiority margin). Limitations of this research were that 1 of the trial wards was closed by the healthcare provider after Trial 1 and could not be included in Trial 2, and that both withdrawing the current weekend allied health service model and installing a new one may have led to an accommodation period for staff to adapt to the new service settings. Stepped-wedge trials are potentially susceptible to bias from naturally occurring change over time at the service level; however, this was adjusted for in our analyses. Conclusions In Trial 1, criteria to say that the no weekend allied health condition was non-inferior to current weekend allied health condition were not met, while neither the no weekend nor current weekend allied health condition demonstrated superiority. In Trial 2, the no weekend allied health condition was non-inferior to the newly developed weekend allied health condition across all primary outcomes, and superior for the outcomes proportion of patients staying longer than expected, proportion experiencing any adverse event, and mean length of stay. Trial registration Australian New Zealand Clinical Trials Registry ACTRN12613001231730 and ACTRN12613001361796

Twelve month follow up of a falls prevention program in older adults from diverse populations in Australia: A qualitative study

Several randomised trials demonstrate that multi dimensional falls prevention programs are effective in reducing falls in older adults. There is a need to examine the impact of these programs in real life settings where diverse populations exist. The aim of this study was to examine the acceptability and impact on sustained participation in falls prevention activities of a combined exercise and education falls prevention program. A semi structured telephone interview was conducted with 23 participants 12 months following the completion of a 15 week falls prevention program tailored to diverse communities in Victoria, Australia and provided in both a group and home based format. Reported benefits of the falls prevention program included physical improvements in joint flexibility, mobility and balance and enjoyment derived from both the exercises and socialisation. Recall of the educational component was minimal as were ongoing behavioral changes to reduce the risk of falling other than exercise. Participation in sustained exercise for falls prevention following the completion of the program was also inconsistent. Future improvements of such programs could focus upon ensuring the exercises prescribed are sufficiently challenging for each individual in order to be of physical benefit, altering the educational style to be goal directed and more enjoyable, and integrating further strategies to support sustained participation in falls prevention behavioral changes. Linking participants with alternate ongoing exercise opportunities or potential sources of ongoing support may be advantageous in enhancing long term participation in exercise for falls prevention following cessation of the program.

The effectiveness of research implementation strategies for promoting evidence-informed policy and management decisions in healthcare: A systematic review

BackgroundIt is widely acknowledged that health policy and management decisions rarely reflect research evidence. Therefore, it is important to determine how to improve evidence-informed decision-making. The primary aim of this systematic review was to evaluate the effectiveness of research implementation strategies for promoting evidence-informed policy and management decisions in healthcare. The secondary aim of the review was to describe factors perceived to be associated with effective strategies and the inter-relationship between these factors.MethodsAn electronic search was developed to identify studies published between January 01, 2000, and February 02, 2016. This was supplemented by checking the reference list of included articles, systematic reviews, and hand-searching publication lists from prominent authors. Two reviewers independently screened studies for inclusion, assessed methodological quality, and extracted data.ResultsAfter duplicate removal, the search strategy identified 3830 titles. Following title and abstract screening, 96 full-text articles were reviewed, of which 19 studies (21 articles) met all inclusion criteria. Three studies were included in the narrative synthesis, finding policy briefs including expert opinion might affect intended actions, and intentions persisting to actions for public health policy in developing nations. Workshops, ongoing technical assistance, and distribution of instructional digital materials may improve knowledge and skills around evidence-informed decision-making in US public health departments. Tailored, targeted messages were more effective in increasing public health policies and programs in Canadian public health departments compared to messages and a knowledge broker. Sixteen studies (18 articles) were included in the thematic synthesis, leading to a conceptualisation of inter-relating factors perceived to be associated with effective research implementation strategies. A unidirectional, hierarchal flow was described from (1) establishing an imperative for practice change, (2) building trust between implementation stakeholders and (3) developing a shared vision, to (4) actioning change mechanisms. This was underpinned by the (5) employment of effective communication strategies and (6) provision of resources to support change.ConclusionsEvidence is developing to support the use of research implementation strategies for promoting evidence-informed policy and management decisions in healthcare. The design of future implementation strategies should be based on the inter-relating factors perceived to be associated with effective strategies.Trial registrationThis systematic review was registered with Prospero (record number: 42016032947).

Application of a novel disinvestment research design to the use of weekend allied health services on acute medical and surgical wards - randomised trial and economic evaluation protocol

Background Some currently provided health services have an absence of evidence for effectiveness, cost-effectiveness and/or safety. These are candidates for disinvestment. It is possible that such services would prove valuable if trials were to be conducted however, making disinvestment a clear risk. Provision of these services in the context of usual care is a considerable barrier to conducting a conventional trial of these interventions. Our team has recently developed a novel research approach to conduct a trial in this context [1]. In this paper, we describe the first application of this design. Allied health services include those provided by a range of health professional groups. Weekend allied health services on acute medical or surgical wards are widely provided internationally but are inconsistent in their composition and focus. There is no direct evidence of efficacy for these weekend services, and higher rates of pay on the weekend make their likely cost-effectiveness questionable. This research examines the efficacy, cost-effectiveness and safety of disinvesting from weekend allied health services on acute medical or surgical wards.

What makes weekend allied health services effective and cost-effective (or not) in acute medical and surgical wards? Perceptions of medical, nursing, and allied health workers

BackgroundThere is strong public support for acute hospital services to move to genuine 7-day models, including access to multidisciplinary team assessment. This study aimed to identify factors that might enable an effective and cost-effective weekend allied health services on acute hospital wards.MethodsThis qualitative study included 22 focus groups within acute wards with a weekend allied health service and 11 telephone interviews with weekend service providers. Data were collected from 210 hospital team members, including 17 medical, 97 nursing, and 96 allied health professionals from two Australian tertiary public hospitals. All were recorded and imported into nVivo 10 for analysis. Thematic analysis methods were used to develop a coding framework from the data and to identify emerging themes.ResultsKey themes identified were separated into issues perceived as being enablers or barriers to the effective or cost-effective delivery of weekend allied health services. Perceived enablers of effectiveness and cost-effectiveness included prioritizing interventions that prevent decline, the right person delivering the right service, improved access to the patient’s family, and ability to impact patient flow. Perceived barriers were employment of inexperienced weekend staff, insufficient investment to see tangible benefit, inefficiencies related to double-handling, unnecessary interventions and/or inappropriate referrals, and difficulty recruiting and retaining skilled staff.ConclusionsSuggestions for ensuring effective and cost effective weekend allied health care models include minimization of task duplication and targeting interventions so that the right patients receive the right interventions at the right time. Further research into the effectiveness and cost effectiveness of these services should factor in hidden costs, including those associated with managing the service.