SSample-Targeted Clinical Trial Adaptation
Ognjen Arandjelovi´c
Centre for Pattern Recognition and Data Analytics, Deakin University, Australia
Abstract
Clinical trial adaptation refers to any adjustment of thetrial protocol after the onset of the trial. The main goalis to make the process of introducing new medical inter-ventions to patients more efficient by reducing the costand the time associated with evaluating their safety andefficacy. The principal question is how should adapta-tion be performed so as to minimize the chance of dis-torting the outcome of the trial. We propose a novelmethod for achieving this. Unlike previous work ourapproach focuses on trial adaptation by sample sizeadjustment. We adopt a recently proposed stratifica-tion framework based on collected auxiliary data andshow that this information together with the primarymeasured variables can be used to make a probabilis-tically informed choice of the particular sub-group asample should be removed from. Experiments on simu-lated data are used to illustrate the effectiveness of ourmethod and its application in practice.
Introduction
Robust evaluation is a crucial component in the processof introducing new medical interventions. Amongst others,these include newly developed medications, novel meansof administering known treatments, new screening proce-dures, diagnostic methodologies, physio-therapeutical ma-nipulations, and many others. Such evaluations usually takeon the form of a controlled clinical trial (or a series thereof),the framework widely accepted as best suited for a rigourousstatistical analysis of the effects of interest (Meinert, 1986;Piantadosi, 1997; Friedman, Furberg, and DeMets, 1998)(for a related discussion and critique also see (Penston,2005)). Driven both by legislating bodies, as well as thescientific community and the public, the standards that theassessment of novel interventions are expected to meet con-tinue to rise. Generally, this necessitates trials which em-ploy larger sample sizes and which perform assessmentover longer periods of time. A series of practical challengesemerge as a consequence. Increasing the number of individ-uals in a trial can be difficult because some trials necessitatethat participants meet specific criteria; volunteers are also
Copyright c (cid:13) less likely to commit to participation over extended peri-ods of time. The financial impact is another major issue –both the increase in the duration of a trial and the number ofparticipants result in additional cost to an already expensiveprocess. In response to these challenges, the use of adaptivetrials has emerged as a potential solution (Fisher, 1998; U.S.Department of Health and Human Services, 2010; Hung,Wang, and ONeill, 2006). The key idea underlying the con-cept of an adaptive trial design is that instead of fixing theparameters of a trial before its onset, greater efficiency canbe achieved by adjusting them as the trial progresses (Chowand Chan, 2011). For example, the trial sample size (e.g.the number of participants in a trial), treatment dose or fre-quency, or the duration of the trial may be increased or de-creased depending on the accumulated evidence (Cui, Hung,and Wang, 1999; Nissen, 2006; Lang, 2011).
Method overview
The method for trial adaptation we de-scribe in this paper has been influenced by recent work onthe analysis of imperfectly blinded clinical trials (Arand-jelovi´c, 2012a,b). Its key contribution was to introduce theidea of trial outcome analysis by patient sub-groups whichcomprise trial participants matched by the administered in-tervention (treatment or control) and their responses to anauxiliary questionnaire in which the participants are askedto express their belief regarding their assignment interven-tion in the closed-form. This framework was shown to besuitable for robust inference in the presence of “unblinding”(Arandjelovi´c, 2012a; Haahr and Hr´objartsson, 2006). Themethod proposed in the present paper emerges from the re-alization that the same framework can be used for trial adap-tation by providing information which can be used to make astatistically informed selection of the trial participants whichcan be dropped from the trial before its completion, with-out significantly affecting the trial outcome. Thus, the pro-posed approach falls under the category of trial adaptationsby “amending sample size”, in contrast to “dose finding”or “response adapting” methods which dominate previouswork (Lang, 2011).In (Arandjelovi´c, 2012a) it was shown that the analy-sis of a trial’s outcome should be performed by aggregat-ing evidence provided by matched participant sub-groups,where two sub-groups are matched if they contain partic-ipants who were administered different interventions butnonetheless had the same responses in the auxiliary ques- a r X i v : . [ c s . L G ] N ov ionnaire. Therefore, our idea advanced here is that an in-formed trial sample size reduction can be made by comput-ing which matched sub-group pair’s contribution of usefulinformation is affected the least with the removal of partici-pants from one of its groups. Contrast with previous work
Before introducing the pro-posed method in detail, it is worthwhile emphasizing twofundamental aspects in which it differs from the methodspreviously described in the literature. The first differenceconcerns the nature of the statistical framework which un-derlies our approach. Most of the existing work on trialadaptation by sample size adjustment adopts the frequen-tist paradigm. These methods follow a common pattern: aparticular null hypothesis is formulated which is then re-jected or accepted using a suitable statistic and the desiredconfidence requirement (Jennison and Turnbull, 2003). Incontrast, the method described in this paper is thoroughlyBayesian in nature. The second major conceptual novelty ofthe proposed method lies in the question it seeks to answer.All previous work on trial adaptation by sample size adjust-ment addresses the question of whether the sample size canbe reduced while maintaining a certain level of statisticalsignificance of the trial’s outcome. In contrast, the presentwork is the first to ask a complementary question of which particular individuals in the sample should be removed fromthe trial once the decision of sample size reduction has beenmade. Thus, the proposed method should not be seen as analternative to the any of the previously proposed methods butrather as a complementary element of the same framework.
Auxiliary data collection
The type of auxiliary data collection we utilize in this workwas originally proposed for the assessment of blinding inclinical trials (James et al., 1996). Since then it has beenadopted for the same purpose in a number of subsequentworks (Bang, Ni, and Davis, 2004; Hr´objartsson et al., 2007;Kolahi, Bang, and Park, 2009; Arandjelovi´c, 2012a) (alsosee (Sackett, 2007) for related commentary). The question-naire allows the trial participants to express their belief onthe nature of the intervention they have been administered(control or treatment) using a fixed number of choices. Themost commonly used, coarse-grained questionnaire admitsthe following three choices:1: belief that control intervention was administered,2: belief that treatment intervention was administered, and3: undecidedness about the nature of the intervention.
Matching sub-groups outcome model
In the general case, the effectiveness of a particular inter-vention in a trial participant depends on the inherent effectsof the intervention, as well as the participant’s expectations(conscious or not). Thus, as in (Arandjelovi´c, 2012a), in theinterpretation of trial results, we separately consider eachpopulation of participants which share the same combina-tion of the type of intervention and the expressed belief re-garding this group assignment. For example, when a 3-tierquestionnaire is used in a trial comparing the administration of the treatment of interest and control, we recognize controlsub-groups: G C − : participants of the control group who believe theywere assigned to the control group, G C : participants of the control group who are unsure oftheir group assignment, G C + : participants of the control group who believe theywere assigned to the treatment group,and the three corresponding treatment sub-groups. The keyidea underlying the method proposed in (Arandjelovi´c,2012a) is that because the outcome of an intervention de-pends on both the inherent effects of the intervention andthe participants’ expectations, the effectiveness should be in-ferred in a like-for-like fashion. In other words, the responseobserved in, say, the sub-group of participants assigned tothe control group whose feedback professes belief in thecontrol group assignment should be compared with the re-sponse of only the sub-group of the treatment group whoequally professed belief in the control group assignment. Sub-group selection
The primary aim of the statistical framework describedin (Arandjelovi´c, 2012a) is to facilitate an analysis of trialdata robust to the presence of partial or full unblinding of pa-tients, or indeed patient preconceptions which too may affectthe measured outcomes. Herein we propose to exploit andextend this framework to guide the choice of which patientsare removed from the trial after its onset, in a manner whichminimizes the loss of statistical significance of the ultimateoutcomes.At the onset of the trial, the trial should be randomizedaccording to the current best clinical practice; this problemis comprehensively covered in the influential work by Berger(2005). If a reduction in the number of trial participants wasattempted at this stage, by the very definition of a properlyrandomized trial, statistically speaking there is no reason toprefer the removal of any particular subject (or indeed a setof subjects) over another. Instead, any trial size adaptationmust be performed at a later stage after some meaningfuldifferentiation between subjects takes place (Nelson, 2010).The most obvious observable differentiation that takesplace between patients as the trial progresses is that of theoutcomes of primary interest in the trial (the “response”).This differentiation may allow for a statistically informedchoice to be made about which trial participants can bedropped from the trial in a manner which minimizes the ex-pected distortion of the ultimate findings. For example, thiscan be done by seeking to preserve the distribution of mea-sured outcomes within a group (treatment or control) butwith the constraint of a smaller number of participants; in-deed, our approach partially exploits this idea. However, ourkey contribution lies in a more innovative approach, whichexploits additional, yet readily collected discriminative in-formation. The proposed approach not only minimizes theeffect of smaller participant groups but also ensures that nounintentional bias is injected due to imperfect blinding. Re-call that the problem of inference robust to imperfect blind-ing should always be considered, as blinding can only bettempted with respect to those variables of the trial whichhave been identified as revealing of the administered treat-ment (and even for these it is fundamentally impossible to ensure perfect blinding).Our idea is to administer an auxiliary questionnaire of theform described in (James et al., 1996; Bang, Ni, and Davis,2004) every time an adaptation of the trial group size issought. As in (Arandjelovi´c, 2012a), this leads to the differ-entiation of each group of participants (control or treatment)into sub-groups, based on their belief regarding their groupassignment. In general, this means that even if no partici-pants are removed from the trial, a participant may changehis/her sub-group membership status. This is illustrated witha hypothetical example in Fig. 1. The first time an auxiliaryquestionnaire is administered (top plot), most of the treat-ment group participants are still unsure of their assignment(solid blue line); a smaller number of participants have cor-rectly guessed (or inferred) their assignment (bold blue line);lastly, an even smaller number holds the incorrect belief thatthey are in fact members of the control group (dotted blueline). All of the sub-groups show a spread of responses tothe treatment, such as may be expected due to various per-sonal variations of their members. At the time of the secondsnapshot (middle plot), at the next instance when auxiliarydata is collected, the proportions of participants in each sub-group has changed, as do the associated treatment responsestatistics. A similar observation can be made with respect tothe third and the last snapshot pictured in the figure (bottomplot). This sort of a development would not be unexpected– if the treatment is effective, as the trial progresses therewill be an increase in the number of treatment group par-ticipants who observe and correctly interpret these changes(note that this also means that there will be an associatedincrease in the number of participants who may exhibit anadditional positive effect from the fortunate realization thatthey are receiving the studied treatment intervention, ratherthan the control intervention). That being said, it should beemphasized that no assumption on the statistics of sub-groupmemberships or their relative sizes is made in the proposedmethod. The example in Fig. 1 is merely used for illustra-tion.The question is: how does this differentiation of patientsby auxiliary data sub-groups help us make a statisticallyrobust choice of which participants in the trial should bepreferentially dropped if a reduction in the trial size issought? To answer this question, recall that the main premiseof (Arandjelovi´c, 2012a) is“that it is meaningful to com-pare only the corresponding treatment and control partici-pant sub-groups, that is, sub-groups matched by their auxil-iary responses.” Each sub-group comparison contributes in-formation used to infer the probability density of the differ-ential effects of the treatment. We can then reformulate theoriginal question as: from which matching sub-group pairshould participants be preferentially dismissed from furtherconsideration so as to best conserve the sub-group pair’s in-formation contribution? Consider how the information onthe differential effects between a single pair of matchingsub-groups is inferred. In its general form, we can esti-mate some distance between the distributions of the two sub- P r obab ili t y den s i t y P r obab ili t y den s i t y Responses of control group sub−groups at adaptation step P r obab ili t y den s i t y Figure 1:
A conceptual illustration on a hypothetical example ofthe phenomenon whereby trial participants change their sub-groupmembership (recall that each sub-group is defined by its members’intervention assignment and auxiliary questionnaire responses).This is quite likely to occur when the effects of the treatment arevery readily apparent but various other mechanisms can act so asto cause a non-zero and changing sub-group flux. groups using a Bayesian approach: ρ ∗ ∝ (cid:90) Θ c (cid:90) Θ t ρ ( p c ( x ; Θ c ) , p t ( x ; Θ t )) (cid:124) (cid:123)(cid:122) (cid:125) Distance between distributionsfor specific parameter values × p ( D c | Θ c ) p ( D t | Θ t ) (cid:124) (cid:123)(cid:122) (cid:125) Model likelihoods p (Θ c ) p (Θ t ) (cid:124) (cid:123)(cid:122) (cid:125) Parameter priors × d Θ t d Θ c (1) where Θ c and Θ t are the sets of variables parameterizingthe two corresponding distributions p c ( x ; Θ c ) and p t ( x ; Θ t ) , p (Θ c ) and p (Θ t ) the parameter priors, ρ ( p c ( x ; Θ c ) , p t ( x ; Θ t )) a particular distance function (e.g. the Kullback-Leibler di-vergence (Kullback, 1951), Bhattacharyya (Bhattacharyya,1943) or Hellinger distances (Hellinger, 1909), or indeed theposterior of the difference of means used in (Arandjelovi´c,2012a)), and D c and D t the measured trial outcomes (e.g.the reduction in blood plasma LDL in a statin trial etc).Note that by changing (reducing) the number of partici-pants in one of the groups, the only affected term on the righthand side of (1) is one of the likelihood terms, p ( D c | Θ c ) or p ( D t | Θ t ) . Seen another way, a change in the number ofparticipants in the trial changes the weighting of the prod-uct of the distance term ρ ( p c ( x ; Θ c ) , p t ( x ; Θ t )) and the priors p (Θ c ) p (Θ t ) . Our idea is then to choose to remove a trialparticipant from that sub-group which produces the small-est change in the estimate ρ ∗ . However, it is not clear howthis may be achieved, since it is the size of the set D c that ischanging (so, for example, treating D c and D t as vectors and f as a function of vectors would not achieve the desired aim).Examining the sensitivity of ρ ∗ with the removal of each da-tum (i.e. trial participant) from D c and D t is also unsatisfac-tory since the problem does not lend itself to a greedy strat-egy: the optimal choice of which n rem trial participants todrop from the trial cannot be made by making n rem optimalchoices of which one participant to drop. An approach fol-lowing this direction but attempting to examine all possiblesets of size n rem would encounter computational tractabilityobstacles since this problem is NP-complete. The alternativewhich we propose is to consider and compare the magni-tudes of partial derivatives of ρ ∗ with respect to the sizes ofata sets D c and D t , but with an important constraint – thederivatives are taken of the expected functional form of ρ ∗ over different members of D c and D t . Formalizing this, wecompute: E (cid:20) ∂ρ ∗ ∂n c (cid:21) D c and E (cid:20) ∂ρ ∗ ∂n t (cid:21) D t , (2) where E [ ρ ∗ ] D c and E [ ρ ∗ ] D t are respectively the expectedvalues of ρ ∗ across the space of possible observations in D c and D t . Thus E [ ρ ∗ ] D c and E [ ρ ∗ ] D t are functions of twoscalars, the sizes n c and n t of sets D c and D t i.e. the numbersof members of the corresponding sub-groups.The proposed solution is not only theoretically justifiedbut it also lends itself to simple and efficient implementation.Since the expected values E [ ρ ∗ ] D c and E [ ρ ∗ ] D t are evalu-ated over sets D c and D t , in (1) the only term affected is p ( D c | Θ c ) p ( D t | Θ t ) , so the solution is readily obtained as aclosed form expression. Equally, the integration is readilyperformed using one of the standard Markov chain MonteCarlo integration methods (Gilks, 1995). Application example
In order to illustrate how the described method could beapplied in practice, let us consider a hypothetical example.Let the trial observation data in two matching sub-groupsbe drawn from the random variables X c and X t , which areappropriately modelled using normal distributions (Aitchi-son and Brown, 1957): X t ∼ /σ t exp − ( x − m t ) / (2 σ t ) and X c ∼ /σ c exp − ( x − m c ) / (2 σ c ) . The next step is to choosean appropriate distance function ρ in (1). In practice, thischoice would be governed by the goals of the study. Herein,for illustrative purposes we choose ρ to be the probabilitythat a patient will do better when the treatment interventionis administered: ρ ( p t ( x ; Θ t ) ,p c ( y ; Θ c )) = (cid:90) ∞ (cid:90) x p t ( x ; Θ t ) p c ( y ; Θ c ) dy dx where Θ c = ( m c , σ c ) and Θ t = ( m t , σ t ) are the mean andstandard deviation parameters specifying the correspondingnormal distributions. ρ ∗ ∝ (cid:90) ∞ (cid:90) ∞−∞ (cid:90) ∞ (cid:90) ∞−∞ (cid:90) ∞ (cid:90) x p t ( x | m t , σ t ) p ( m t , σ t ) dx dm t dσ t × p c ( y | m c , σ c ) p ( m c , σ c ) dy dm c dσ c = (cid:90) ∞ (cid:90) x I t ( x ) I c ( y ) dy dx (3) where – assuming uninformed priors on m c , m t , σ c , and σ t – each of the integrals I t ( x ) and I c ( y ) has the form: I = (cid:90) ∞ (cid:90) ∞−∞ σ exp (cid:26) − ( x − m ) σ (cid:27) × σ n exp (cid:26) − (cid:80) ni =1 ( x i − m ) σ (cid:27) dm dσ, (4) and { x i } and n stand for either { x ( c ) i } and n c or { x ( t ) i } and n t , and { x ( c ) i } and { x ( t ) i } ( i = 1 . . . n t ) are exponen-tially transformed measured trial variables. This integral canbe evaluated by combining the two exponential terms andcompleting the square of the numerator of the exponent as in (Arandjelovi´c, 2012a) which leads to the following sim-plification of (3): I ∝ (cid:90) ∞ σ n +1 exp (cid:110) − c σ (cid:111) dσ, (5) where the value of the only non-constant term is c = x + (cid:80) ni =1 x i − ( x + (cid:80) ni =1 x i ) / ( n + 1) . The form of theintegrand in (5) matches that of the inverse gamma distribu-tion Gamma ( z ; α, β ) = β α Γ( α ) z − α − exp {− β/z } . The variable z and the two parameters of the distribution, α and β , can bematched with the terms in (5) and the density integrated out,leaving: I ∝ c − n − . (6) Remembering that the functional form of c is different forthe control and the trial groups (since it is dependent on x i which stands for either x ( c ) i or x ( t ) i ), and substituting the re-sult from (6) back into (3) gives the following expression forthe distance function: ρ ∗ = (cid:90) ∞ (cid:90) x p t ( x ) p c ( y ) dy dx ∝ (cid:90) ∞ c − nt − t (cid:90) x c − nc − c dy dx Our goal now is to evaluate S c ( ρ ∗ ) and S t ( ρ ∗ ) , the sensitiv-ities of the distance function to the change in the size of thecontrol and the treatment groups. Without loss of generality,let us consider S t ( ρ ∗ ) : S t ( ρ ∗ ) ∝ (cid:90) ∞−∞ (cid:90) x −∞ S [ I t ( x )] I c ( y ) dy dx (7) To evaluate S [ I t ( x )] we will employ the standard chain ruleand perform differentiation with respect to n t when the cor-responding term is a function of the number of treatmentparticipants but not any x ( t ) i . On the other hand, as describedpreviously, to handle those terms which do depend on x ( t ) i (through c t ), we will use the expected value of the changein the term, averaged over all possible x ( t ) i that a unitarydecrease in n t can be achieved. Applying this idea on theexpression in (4): S [ I t ( x )] = − (cid:90) ∞ (cid:90) ∞−∞ σ exp (cid:26) − ( x − m ) σ (cid:27) × (cid:34) ln σσ n exp (cid:26) − (cid:80) ni =1 ( x i − m ) σ (cid:27) +1 σ n exp (cid:26) − (cid:80) ni =1 ( x i − m ) σ (cid:27) (cid:80) ni =1 ( x i − m ) σ n (cid:35) dm dσ (8) noting that we used the standard result ddn σ n = − ln σσ n with-out including its derivation with intermediary steps shownexplicitly. Full double integration in (8) is difficult to per-form analytically. However, one level of integration – withrespect to m – is readily achieved. Note that the first term,as a function of m , has the same form as the integral in(4) which we already evaluated. The same procedure whichuses the completion of the square in the exponential termcan be applied here as well (note that unlike in (4) here itis important to keep track of the multiplicative constants asthese will be different for the second term in (8)). The in-tegrand in the second term can be expressed in the form ( z − λ ) exp − z dz . This integration is also readily per-formed using the standard results (cid:82) ∞−∞ √ π z exp − z dz =1 and (cid:82) ∞−∞ √ π exp − z dz = 1 and by noting that the in-tegrand is an odd function: (cid:82) ∞−∞ √ π z exp − z dz = 0 .A straightforward application to (8) leads to the followingexpression for the sensitivity S [ I t ( x )] of the integral I t tochanges in the size of the corresponding sub-group: S [ I t ( x )] = − (cid:90) ∞ ln σσ n +1 σa √ π exp (cid:110) − c σ (cid:111) dσ (9) − (cid:90) ∞ exp (cid:8) − c σ (cid:9) σ n +3 n (cid:34) n (cid:16) σa (cid:17) + √ π σa n (cid:88) i =1 x i (cid:35) dσ This result, together with the expression in (6), can be sub-stituted into (7) and the remaining integration performed nu-merically.
From target sub-groups to specific participants
Adopting the framework proposed in (Arandjelovi´c, 2012a)whereby the analysis of a trial takes into account sub-groupsof trial participants, which emerge from grouping partici-pants according to their assigned intervention and auxiliarydata, thus far we focused on the problem of choosing thesub-group from which participants should be preferentiallyremoved if a reduction in trial size is sought. The other ques-tion which needs to be considered is how specific sub-groupmembers are to be chosen, once the target sub-group is iden-tified. Fortunately, the proposed framework makes this asimple task. Recall that the observed trial data within eachsub-group is assumed to comprise an identically and inde-pendently distributed sample from the underlying distribu-tion, i.e. x ( c ) i ∼ X c and x ( t ) i ∼ X t . This means that it is suf-ficient to randomly sample the set of target sub-group mem-bers to select those which can be removed.The simplicity of the selection process that our approachallows has an additional welcome consequence. Recall thatin the proposed method the choice of the target sub-group ismade by comparing differentials in (2). It is important to ob-serve that their values are computed for the initial values of n c and n t . Thus, as the number of participants in either of thesub-group is changed, so do the values of the differentials,and thus possibly the optimal sub-group choice. This is whythe removal of participants should proceed sequentially. Evaluation
The primary novelty introduced in this paper is of a method-ological nature. In the previous section we explained in de-tail the mathematical process involved in applying the pro-posed methodology in practice. Pertinent results were de-rived for a specific distance function used to quantify thedifference in the outcomes between the control and treat-ment groups in a trial. The choice of the distance function –which would in practice be made by the clinicians to suit theaims of a specific trial – governs the relative loss of infor-mation when participants are removed from a specific sub-group, and consequently dictates the choice of the optimalsub-group from which the removal should be performed ifthe overall trial sample size needs to be reduced. In this section we apply the derived results on experimen-tal data, and evaluate and discuss the performance of the pro-posed methodology. We adopt the evaluation protocol stan-dard in the domain of adaptive trials research, and obtaindata using a simulated experiment.
Experimental setup
We simulated a trial involving 180 individuals, half of whichwere assigned to the control and the other half to the treat-ment group. For each individual we maintain a variablewhich describes that person’s belief regarding his/her groupassignment. Thus, for the control group we have n c beliefs b ( c ) i ( i = 1 . . . n c ) and similarly for the treatment group n t be-liefs b ( t ) i ( i = 1 . . . n t ) . Belief is expressed by a real number, ∀ i. b ( c ) i , b ( t ) i ∈ ( −∞ , + ∞ ) , with 0 indicating true undecided-ness. Negative beliefs express a preference towards the be-lief in control group assignment, and positive towards the be-lief in treatment group assignment. The greater the absolutevalue of a belief variable is, the greater is the person’s con-viction. We employ a three-tier questionnaire. To simulate aparticipant’s response, we map the corresponding belief toone of the three possible questionnaire responses accordingto the following thresholding rule: b < − → Belief in control group assignment (10) − ≤ b ≤ → Uncertain (“don’t know”) (11) < b → Belief in treatment group assignment (12)
The starting beliefs of participants, i.e. their beliefs beforethe onset of the trial, are initialized to: b ( c ) i = b ( t ) i = − for i = 1 . . . for i = 10 . . . for i = 82 . . . (13) This initialization models the conservative belief of most in-dividuals, and the tendency of a smaller number of individu-als to exhibit “pessimistic” or “optimistic” expectations. Thesame distribution was used both for the control and the treat-ment groups, reflecting a well performed randomization.
Effect accumulation
As the trial progresses the effectsof the treatment accumulate. These are modelled as posi-tive i.e. the treatment is modelled as successful in the sensethat on average it produces a superior outcome in compar-ison with the control intervention. We model this using astochastic process which captures the variability in partici-pants’ responses to the same treatment. Specifically, at thediscrete time step k + 1 (the onset of the trial correspondingto k = 0 ), the effects on the i -th treatment and control groupparticipants at the preceding time step k are updated as: e ( t ) i ( k + 1) = e ( t ) i ( k ) + w ( t ) i ( k + 1) × exp (cid:26) − k + 110 (cid:27) (14) e ( c ) i ( k + 1) = e ( c ) i ( k ) + w ( c ) i ( k + 1) × exp (cid:26) − k + 110 (cid:27) (15) where w ( t ) i ( k + 1) and w ( c ) i ( k + 1) are drawn from W t ∼N (0 . , . and W c ∼ N (0 . , . respectively. At theonset there is no effect of the treatment; thus: ∀ .i = 1 . . . n t . e ( t ) i (0) = 0 and ∀ .i = 1 . . . n c . e ( c ) i (0) = 0 − − P o s t e r i o r p r obab ili t y den s i t y Proposed methodRandom participant removal (a) D i ff e r en t i a l e ff e c t o f t r ea t m en t Proposed methodRandom participant removalGround truth (b) N u m be r , o f, pa r t i c i pan t s , pe r , s ub − g r oup Control,,negative,beliefControl,,neutral,beliefControl,,positive,beliefTreatment,,negative,beliefTreatment,,neutral,beliefTreatment,,positive,belief (c) N u m be r , o f, pa r t i c i pan t s , pe r , s ub − g r oup Control,,negative,beliefControl,,neutral,beliefControl,,positive,beliefTreatment,,negative,beliefTreatment,,neutral,beliefTreatment,,positive,belief (d)
Figure 2: (a) Posteriors of the differential effect of treatment afterthe removal of 120 participants; (b) the maximum a posteriori es-timates of the differential effect of treatment during the course ofthe trial; the changes in the sample sizes within each of the six par-ticipant sub-groups observed in our experiment using (c) randomselection based participant removal, and (d) the proposed method.
Belief refinement
As the effects of the respective inter-ventions are exhibited, the trial participants have increasingamounts of evidence available guiding them towards form-ing the correct belief regarding their group assignment. Inour experiment this process is also modelled using a stochas-tic process which is dependent on the magnitude of the effectthat an intervention has in a particular participant, as well asuncertainty and differences in people’s inference from ob-servations. At the time step k + 1 , the beliefs of the i -th treat-ment and control group participants at the preceding timestep k are updated as follows: b ( t ) i ( k + 1) = b ( t ) i ( k ) + 0 . e ( t ) i ( k + 1) + ω ( t ) i ( k + 1) (16) b ( c ) i ( k + 1) = b ( c ) i ( k ) + 0 . e ( c ) i ( k + 1) + ω ( c ) i ( k + 1) (17) where ω ( t ) i ( k + 1) and ω ( c ) i ( k + 1) are drawn from Ω t ∼N (0 . , . and Ω c ∼ N (0 . , . respectively. Results and discussion
Using the same data obtained by simulating the experimentoutlined in the previous section, we compared the proposedmethod with the current practice of randomly selecting par-ticipants which are to be removed from the trial. In bothcases, data was analyzed using the Bayesian method pro-posed in (Arandjelovi´c, 2012a). A typical result is illustratedin Fig. 2(a); the plot shows the posterior distributions of thedifferential effect of the treatment inferred after the removalof 120 individuals, obtained using the proposed method (redline) and random selection (blue line). The most notable dif-ference between the two posteriors is in the associated un-certainties – the proposed method results in a much morepeaked posterior i.e. a much more definite estimate. In com-parison, the posterior obtained using random selection ismuch broader, admitting a lower degree of certainty asso-ciated with the corresponding estimate.The accuracy of two methods is better assessed by observ-ing their behaviour over time. The plot in Fig. 2(b) shows the maximum a posteriori estimates of the differential ef-fect of treatment obtained using the two methods during thecourse of the trial. Also shown is the ‘ground truth’, thatis, the actual differential effect which we can compute ex-actly from the setup of the experiment. In the early stagesof the trial, while the magnitude of the accumulated effectis small and the number of participants large, the two es-timates are virtually indistinguishable, and they follow theground truth plot closely. As expected, as the number of par-ticipants removed increases both estimates start to exhibitgreater stochastic perturbations. However, both the accuracy(that is, the closeness to the ground truth) and the reliability(that is, the magnitude of stochastic variability) of the pro-posed method can be seen to show superior performance –its maximum a posteriori estimate follows the ground truthmore closely and fluctuates less than the estimate obtainedwhen random selection is employed instead. It is also impor-tant to observe the rapid degradation of performance of therandom selection method as the number of remaining par-ticipants becomes small, which is not seen in the proposedmethod. This too can be expected from the theoretical argu-ment put forward earlier – the statistically optimal choice ofthe sub-group from which participants are removed ensuresthat the posterior is not highly dependent on a small num-ber of samples which would make it highly sensitive to thechange in sample size.Lastly, it is interesting to observe the differences betweenthe changes in the sample sizes within each sub-group us-ing the two approaches. This is illustrated using the plotsin Fig. 2(c) and 2(d). As expected, when random partici-pant removal is employed, the sizes of all sub-groups de-crease roughly linearly (save for stochastic variability), asshown in Fig. 2(c). In contrast, the sub-group size changeseffected by the proposed method show more complex struc-ture, governed by the specific values of the belief and effectvariables in our experiment. It is particularly interesting tonote that the size changes are not only non-linear, but alsonon-monotonic. For example, the size of the control sub-group which includes individuals which correctly identifiedtheir group assignment (i.e. the sub-group G C − ) begins toincrease notably after the removal of 30 participants andstarts to decrease only after the removal of further 78 par-ticipants. Summary and conclusions
We introduced a novel method for clinical trial adaptationby amending sample size. In contrast to all previous work inthis area, the problem we considered was not when samplesize should be adjusted but rather which particular samplesshould be removed. Our approach is based on the adoptedstratification recently proposed for the analysis of trial out-comes in the presence of imperfect blinding. This strati-fication is based on the trial participants’ responses to ageneric auxiliary questionnaire that allows each participantto express belief concerning his/her intervention assignment(treatment or control). Experiments on a simulated trial wereused to illustrate the effectiveness of our method and its su-periority over the currently practiced random selection. eferences
Aitchison, J., and Brown, J. A. C. 1957.
The LognormalDistribution.
Cambridge University Press.Arandjelovi´c, O. 2012a. Assessing blinding in clinical tri-als.
Advances in Neural Information Processing Systems(NIPS)
PLOS ONE
Contemp Clin Trials
Selection Bias and Covariate Imbalances inRandomized Clinical Trials.
Hoboken, New Jersey: Wiley.Bhattacharyya, A. 1943. On a measure of divergence betweentwo statistical populations defined by their probability dis-tributions.
Bulletin of the Calcutta Mathematical Society
Adaptive Design Methodsin Clinical Trials.
Chapman & Hall.Cui, L.; Hung, H. M. J.; and Wang, S. J. 1999. Modificationof sample size in group sequential clinical trials.
Biomet-rics
Stat Med
Fundamen-tals of Clinical Trials.
New York, New York: Springer-Verlag, 3rd edition.Gilks, W. R. 1995.
Markov Chain Monte Carlo in Practice.
Chapman and Hall/CRC, 1 edition.Haahr, M. T., and Hr´objartsson, A. 2006. Who is blindedin randomized clinical trials? A study of 200 trials and asurvey of authors.
Clin Trials
Journalf¨ur die reine und angewandte Mathematik
Int J Epidemiol
Pharmaceut Statist
Stat Med
Stat Med
Community Dent Oral Epidemiol
Annals of Mathematical Statistics
Am J Trop Med Hyg
Clinical Trials: Design, Conduct, andAnalysis.
New York, New York: Oxford University Press,3rd edition.Nelson, N. J. 2010. Adaptive clinical trial design: Has itstime come?
JNCI J Natl Cancer Inst
PLoS Clin Trials
Med Hypotheses
Clinical Trials: A Methodologic Per-spective.
Hoboken, New Jersey: Wiley, 3rd edition.Sackett, D. L. 2007. Commentary: Measuring the successof blinding in RCTs: don’t, must, can’t or needn’t?
IntJ Epidemiol