Kenneth A. Getz

Assessing the impact of protocol design changes on clinical trial performance.

Although it is widely acknowledged that protocol design plays a crucial role in the success of clinical research studies, how protocols have changed over time and the impact of these changes on clinical trial performance have never been quantified. To measure protocol design trends, the Tufts Center for the Study of Drug Development analyzed data on 10,038 unique phase 1-4 protocols conducted between 1999 and 2005. Tufts Center for the Study of Drug Development analyzed study conduct performance data on 57 individual phase 2 and 3 protocols administered at US-based investigative sites. The results of this study indicate that the number of unique procedures and the frequency of procedures per protocol have increased at the annual rate of 6.5% and 8.7%, respectively, during the time period measured. Investigative site work burden to administer each protocol increased at an even faster rate of 10.5% between 1999 and 2005. Additionally, during this time period, study conduct performance-that is, cycle time and patient recruitment and retention rates-worsened; and the number of protocol amendments, observed serious adverse events, and length of case report forms increased substantially. Implications of these results for simplifying protocol designs and minimizing negative effects on study conduct performance are discussed.

Racial disparities among clinical research investigators.

Objective:The objective of the current study was to analyze the influence of a short-term and long-term custom foot orthotic (CFO) intervention on the lower extremity dynamics in a group of female runners with a history of overuse running knee injury. Design:Descriptive laboratory study. Setting:University of Massachusetts Biomechanics Laboratory, Amherst, MA. Participants:This study included a group of female recreational runners (15 to 40 km per week) who had a history of overuse running knee injury in the 6 months leading up to the study. Intervention:Semi-rigid, custom foot orthoses manufactured from a neutral suspension cast and designed to meet the specific needs of each subject. Subjects wore the custom foot orthoses during all running activities for a period of 6 weeks. Main Outcome Measures:Three-dimensional ankle and knee dynamics were collected while subjects performed over-ground running trials with and without a CFO intervention. Data were collected before and after a 6-week CFO intervention during all running activities. Results:For ankle parameters, short-term intervention led to significant decreases in maximum values for rearfoot eversion angle and velocity, impact peak, and loading rate. Ankle inversion impulse was also significantly decreased during the loading phase. At the knee, maximum knee external rotation moment was significantly increased when subjects wore the custom foot orthoses. Conclusions:The 6-week intervention led to subjective changes, including a significant decrease in pain. An improvement in symptoms did occur with the 6-week intervention. In addition, dynamic results revealed that custom foot orthoses have an immediate effect on dynamics and that this influence occurs only when orthoses are worn in the footwear. The short-term CFO intervention led to significant decreases in rearfoot kinematics (maximum eversion angle and velocity) but no changes observed in knee kinematics. The kinetic analysis revealed that these subjects exhibited significant decreases in maxima for ankle inversion moment and angular impulse during the loading phase, impact peak, and vertical loading rate with short-term, CFO intervention. At the knee, the CFO condition led to increases in knee external rotation moment maxima and angular impulse.Evidence shows that minority patients are underrepresented in clinical trials. The development of new drugs and treatments, however, requires that clinical research studies include representative participants, particularly in light of evidence indicating that minority populations sometimes respond differently to prescription medications. Racial disparities among clinical investigators are often cited as a major reason why minority patients are underrepresented in clinical trials. However, there is little to no empirical data to support or refute the prevalence of disparities among clinical investigators. The Tufts Center conducted two online surveys of 1376 physicians. The first survey (N = 859 respondents; 31% response rate) assessed the overall incidence of minority physician involvement in clinical research. The second survey (N = 768 respondents; 20% response rate) assessed the demographics, experience, and infrastructure of minority physicians who have participated in clinical research as a principal investigator or subinvestigator. The results of this study indicate that significant racial disparities exist among clinical investigators. The results also support assertions that physician race influences race of the clinical trial volunteer. The incidence of participation in clinical research among minority physicians is well below that observed among white physicians, more so with regard to U.S. Food & Drug Administration-regulated clinical trials funded by industry. Minority investigators tend to conduct and initiate fewer clinical trials annually. Yet minority and white physician interest in participating in clinical research is similarly high. Minority investigators tend be younger, with more limited clinical research infrastructure and support than their white counterparts. New strategies, policies, incentives, and reforms are needed to address racial disparities among clinical investigators. In addition, disparities among both volunteers and investigators need to be tracked more closely and methodically to monitor and assess the impact of newly implemented programs and reforms.

Measuring the Incidence, Causes, and Repercussions of Protocol Amendments

Drug development companies frequently amend finalized clinical trial protocols. Yet the incidence, causes, and impact of protocol amendments have never been quantified. Tufts Center for the Study of Drug Development (Tufts CSDD) conducted a study, in collaboration with 17 large and midsized pharmaceutical and biotechnology companies, examining more than 3,400 clinical trial protocols across development phases and therapeutic areas. Data on protocol characteristics, the number of amendments, the nature and incidence of changes per amendment, the causes of amendments, and the time and cost to implement amendments were among those analyzed. Tufts CSDD found that more than 40% of protocols were amended prior to the first subject/first visit, and one third of amendments were avoidable. Each amended protocol had an average of 2.3 amendments resulting in 4 months of incremental time to implement. Protocol amendments translate into significant unplanned expense and delays for research sponsors and unexpected burden for investigative sites. These findings underscore the substantial impact of protocol amendments on drug development efficiency and present an opportunity to realize substantial cycle time and cost savings.

Improving Protocol Design Feasibility to Drive Drug Development Economics and Performance

Protocol design complexity has increased substantially during the past decade and this in turn has adversely impacted drug development economics and performance. This article reviews the results of two major Tufts Center for the Study of Drug Development studies quantifying the direct cost of conducting less essential and unnecessary protocol procedures and of implementing amendments to protocol designs. Indirect costs including personnel time, work load and cycle time delays associated with complex protocol designs are also discussed. The author concludes with an overview of steps that research sponsors are taking to improve protocol design feasibility.

Variability in the costs of institutional review board oversight.

Background Previous studies have shown wide differences between institutions in economies of scale with regard to the costs of institutional review board (IRB) oversight of research. In this study, the authors explored variability among IRB costs, taking into account organizational size, components of the costs of oversight, and protocol type. Method The authors conducted a survey of academic medical centers to collect information on resource utilization associated with IRB oversight in 2002. They used national cost weights to assign a cost to each type of resource used, and summed weighted resource utilization for IRB costs. Descriptive statistics were generated for costs over all, tertile of protocol volume, cost component, and type of review. They also determined where the greatest cost variability is found. Results IRB costs per protocol reviewed are highly variable both overall and within tertiles of volume. Higher-volume institutions have lower costs, which is indicative of economies of scale. However, not all components of IRB costs (e.g., board time) are subject to economies of scale. Expedited reviews of protocols are not less expensive at low-volume institutions. Conclusions IRB costs for oversight are highly variable, and only some of the variation may be attributable to economies of scale. Given such wide variation in costs, the authors conclude that some institutions are conducting reviews in a manner that is inefficient or of low quality. Future work is needed to determine specific practices in reviews, and what leads to the best quality and most efficient oversight and review system.

Evaluating the completeness and accuracy of MedWatch data.

The Food and Drug Administration’s MedWatch system—a voluntary surveillance program—received 600,000 adverse event reports on marketed drugs and devices in 2011. The Food and Drug Administration credits the MedWatch system with improving awareness, and expediting early detection, of drug and device risks and in illuminating the adoption of medical treatments. Reporting bias has been acknowledged as a limitation of the MedWatch system. No systematic assessment of the accuracy and completeness of adverse event reporting has been conducted, yet inaccurate adverse event reporting may lead drug safety professionals to draw incorrect conclusions, manufacturers may be wrongly forced to suspend and withdraw medications and interventions, health professionals may mistakenly alter their clinical practices, and patients may be denied safe and effective treatments. In 2011, the Tufts Center for the Study of Drug Development gathered and analyzed 10.2 million adverse event reports filed with the MedWatch system. Patient information was generally complete and accurate. Suspect product information, on the other hand, showed high levels of incomplete and inaccurate data. Start and end dates of suspect product use had 37% and 23% completion rates, respectively. Dosage level was completed only 31% of the time, and product lot numbers had only a 9% completion rate. More than 25% of the names of reported suspect products were inaccurate, and 31% of suspect product start dates were inaccurate. Higher levels of completion and accuracy were associated with reports filed closer to the date when the adverse event was observed. Implications of the results and suggested improvements are discussed.

Quantifying the magnitude and cost of collecting extraneous protocol data.

Although most research professionals believe that protocol designs contain a growing number of unnecessary and redundant procedures generating unused data, incurring high cost, and jeopardizing study success, there are no published studies systematically examining this issue. Between November 2011 and May 2012, Tufts Center for the Study of Drug Development conducted a study among a working group of 15 pharmaceutical companies in which a total of 25,103 individual protocol procedures were evaluated and classified using clinical study reports and analysis plans. The results show that the typical later-stage protocol had an average of 7 objectives and 13 end points of which 53.8% are supplementary. One (24.7%) of every 4 procedures performed per phase-III protocol and 17.7% of all phase-II procedures per protocol were classified as “Noncore” in that they supported supplemental secondary, tertiary, and exploratory end points. For phase-III protocols, 23.6% of all procedures supported regulatory compliance requirements and 15.9% supported those for phase-II protocols. The study also found that on average,

Oncology Drug Development and Approval of Systemic Anticancer Therapy by the U.S. Food and Drug Administration

1.7 million (18.5% of the total) is spent in direct costs to administer Noncore procedures per phase-III protocol and

Subjects’ views of obligations to ensure post-trial access to drugs, care and information: qualitative results from the Experiences of Participants in Clinical Trials (EPIC) study

0.3 million (13.1% of the total) in direct costs are spent on Noncore procedures for each phase-II protocol. Based on the results of this study, the total direct cost to perform Noncore procedures for all active annual phase-II and phase-III protocols is conservatively estimated at

Open innovation: the new face of pharmaceutical research and development

3.7 billion annually, not including the indirect costs associated with collecting and managing Noncore procedure data and the ethical costs of exposing study volunteers to unnecessary risks associated with conducting extraneous procedures.