aa r X i v : . [ s t a t . O T ] A p r Statistical Science (cid:13)
Institute of Mathematical Statistics, 2015
A Conversation with Nancy Flournoy
William F. Rosenberger
Abstract.
Nancy Flournoy was born in Long Beach, California, onMay 4, 1947. After graduating from Polytechnic School in Pasadena in1965, she earned a B.S. (1969) and M.S. (1971) in biostatistics fromUCLA. Between her bachelors and masters degrees, she worked as aStatistician I for Regional Medical Programs at UCLA. After receivingher master’s degree, she spend three years at the Southwest Laboratoryfor Education Research and Development in Seal Beach, California.Flournoy joined the Seattle team pioneering bone marrow transplan-tation in 1973. She moved with the transplant team into the newlyformed Fred Hutchinson Cancer Research Center in 1975 as Directorof Clinical Statistics, where she supervised a group responsible for thedesign and analysis of about 80 simultaneous clinical trials. To supportthe Clinical Division, she supervised the development of an interdisci-plinary shared data software system. She recruited Leonard B. Hearneto create this database management system in 1975 (and married himin 1978). While at the Cancer Center, she was also at the Universityof Washington, where she received her doctorate in biomathematics in1982. She became the first female director of the program in statistics atthe National Science Foundation (NSF) in 1986. She received serviceawards from the NSF in 1988 and the National Institute of Statisti-cal Science in 2006 for facilitating interdisciplinary research. Flournoyjoined the Department of Mathematics and Statistics at American Uni-versity in 1988. She moved as department chair to the University ofMissouri in 2002, where she became Curators’ Distinguished Professorin 2012.While at the Cancer Center, Flournoy documented the graft-versus-leukemia effect in humans and discovered a source of frequent lethalviral infections in the bone marrow transplant patients. Later she wasinfluential in developing adaptive experimental designs. Her numeroushonors include fellow of the Institute of Mathematical Statistics (1990),the American Statistical Association (1992), the World Academy ofArts and Sciences (1992) and the American Academy for the Advance-ment of Science (1993). She has received the COPSS Scott (2000) andDavid (2007) awards, and the Norwood (2012) award from the Univer-sity of Alabama.
Key words and phrases:
Adaptive designs, clinical trials, data coordi-nating center, random walk rules, up-and-down procedures.
William F. Rosenberger is University Professor andChairman, Department of Statistics, George MasonUniversity, 4400 University Drive MS 4A7, Fairfax,Virginia 22030-4444, USA e-mail: [email protected].
This is an electronic reprint of the original articlepublished by the Institute of Mathematical Statistics in
Statistical Science , 2015, Vol. 30, No. 1, 133–146. Thisreprint differs from the original in pagination andtypographic detail. W. F. ROSENBERGER
Fig. 1.
Nancy at home in a part of Los Angeles County thatwas then called Potero Heights, 1949.
1. EARLY LIFE
Rosenberger:
Tell us a little about your early life.Where did you grow up and what did your parentsdo?
Flournoy:
I was born in Long Beach, CA, and grewup in Los Angeles County in a lemon orchard sur-rounded by oil wells and a flood plain. There was adairy farm nearby and we had a donkey. My fatherwas a plumbing contractor who plumbed Los An-geles: restaurants, dormitories, cemeteries. He had11 trucks go out every day. My Mom was alwaysunhappy about not finishing college, so she enrolledin college when I went to college and then directeda preschool for many years. I have three brothersand one sister. I was sent to Polytechnic School inPasadena as a sophomore in high school. On the en-trance exam I had the second highest score in mathin history, but I flunked the English exam because Ididn’t know the words in the instructions. (So eventhen I had a one-sided brain!)
Rosenberger:
As a young person, were you inter-ested in mathematics, statistics, data? What madeyou excited about statistics?
Flournoy:
High school algebra really made mehappy; I would lay on the floor and work problemsfor hours. I had a new female instructor whose hus-band had gotten a professorship across the street at
Fig. 2.
Nancy at her graduation from Polytechnic School,Pasadena, 1965.
Cal Tech while she just landed a high school job;her anger came through and I got the message thatmathematics is worth being passionate about.My love of statistics came as a junior at UCLA,when I took a course taught by Don Ylvisaker. I justassumed that Don was a great teacher for all time,but he later told me that he never had another classlike it. Four or five students from that class went onto get doctorates in statistics.
Rosenberger:
You were fortunate to be at UCLAat a time when there were some of the great names inbiostatistics: Abdelmonen Afifi, Frank Massey, WilDixon, Olive Dunn, Virginia Clark. What professorsexcited you at UCLA?
Flournoy:
Afifi was the young dynamic professorand taught out of Scheff´e; all the students lovedAfifi. Dixon had a bimodal distribution among thestudents; you either loved him or hated him. He putout a thousand ideas a minute; if you paid close at-tention, you would find they were pearls. It was achallenge to get what he was saying as he didn’tchange the tone of his voice when he switched fromone topic to another. He taught the power of data
CONVERSATION WITH NANCY FLOURNOY analysis as a tool for learning and a thousand lit-tle ways to make the data sing. I had a class withFrank Massey; I learned a lot, but he was quiet andnot dynamic. Rosenberger:
Did you have any connection to theDepartment of Statistics? You mentioned Ylvisaker.What about Paul Hoel?
Flournoy:
A separate statistics department didnot exist at that time; it was a math departmentwith a few statisticians. I used the Hoel, Port andStone probability book when it was just a set ofnotes. I don’t think Hoel was the instructor though.
Rosenberger:
What interested you in biostatis-tics?
Flournoy:
Most of the statistics courses that wereoffered at UCLA were in the Department of Bio-statistics. Prior to taking statistics, I had loved bio-chemistry and was a nutrition major, leading to mymajor in the School of Public Health (SPA). WhenI recognized that with a degree in nutrition, I wouldprobably only be able to run a cafeteria in a hos-pital, I decided to get my degree in mathematicsinstead. I applied repeatedly to change from SPA tothe College of Arts and Sciences (CAS), but my ap-plication would get turned down. In tears, I didn’tknow what to do. Then the SPA Dean asked whyI was flunking out, which didn’t make sense sinceI always had gotten As and Bs. They had lost all my records because I had changed names when Iwas previously married, and nothing followed me.So that’s why they didn’t accept me at CAS. Bythe time this got settled, I had enough credits toget a degree in biostatistics.
2. GRADUATE SCHOOL
Rosenberger:
What did you do after you gradu-ated? How did you get to graduate school?
Flournoy:
When I got a job at Regional Medi-cal Programs as a Statistician I, one old man wouldcome around and ask me to add numbers; I told himhe could hire a statistical clerk for half my salary.I was told that, as a young woman, my presenta-tions were not credible. So they hired a male DrPhto present my reports in his name. As a mild wayof protesting, I put my hair in a bun, dyed it white,and got fired. They said I was an “uppity woman.”At that time, Virginia Clark was department chair.She said, “We have a fellowship, why don’t you cometo grad school?” I have some happy memories of mymaster’s program at UCLA: Olive Dunn supervisedmy master’s thesis; Mary Ann Hill was a great teach-ing assistant for Dixon’s class; Carol Newton taughta mean FORTRAN programming course; and Rayand Jean Mickey were influential in my career deci-sions.
Fig. 3.
Nancy with her parents, Elizabeth Blincoe and Carr Irvine Flournoy, at her graduation from the University ofWashington in 1982.
W. F. ROSENBERGER
Rosenberger:
When you won the David Award,you talked about meeting F. N. David. Tell us aboutthat.
Flournoy:
I was in the Los Angeles chapter of theASA; around 1972, a group of us carpooled out toUC Riverside where David was giving a talk. Shehad a strong presence, standing with one leg up ona stairstep and smoking a cigar while she talked.It was a roomful of people, and she exuded suchconfidence. So I immediately started smoking cigars.I had been used to seeing female statisticians such asClark and Dunn behind a desk and not commandingan audience.
Rosenberger:
How did you get to University ofWashington (UW)?
Flournoy:
After the M.S., I thought I knew every-thing about statistics. I got a job at Southwest Ed-ucation and Laboratory for Research, where therewere a lot of education psychologists who were intoexperimental design. On my second day, they pre-sented me with computer output that had more thanone error term; I had the good sense to keep mymouth shut. I immediately called Wil Dixon andasked what they were talking about. He replied, “Ohwell, we can’t teach you everything.” He suggestedI get a book by Walt Federer. The book was out ofprint, but Walt got a preprint from India and sent itto me; so I spent my nights reading Federer’s book.Later, I was trying to read the
Journal of theAmerican Statistical Association to implement someof the stuff I wanted to do, and I found I couldn’tread the literature. I also wanted to escape the smogof Los Angeles. So I applied to the UW, my onlyapplication. Dick Kronmal said there was a researchassistant position with the bone marrow transplantteam, which was then located in the Old Public Hos-pital (recently Amazon) in Seattle.At that time, there was no statistics departmentat UW. The mathematical statistics courses weretaught in the Department of Mathematics. I tookthe mathematical statistics sequence from GalenShorack. I had courses from Ron Pyke and FritzSchultz in nonparametrics. Shortly after Fritz leftfor Boeing, the remaining statistics faculty formedthe Department of Statistics. In the Department ofBiostatistics, there were some female faculty: PaulaDiehr and Pat Wahl. I took the first categorical dataanalysis class taught at UW from Norman Breslow.He gave quizzes at the end of class, so I never paidso much attention in a course before. I took survival from Ross Prentice early in the days of the Cox pro-portional hazards model.
Rosenberger:
What was it like working with yourdissertation advisor, Lloyd Fisher?
Flournoy:
It worked out well because we have sim-ilar work styles. Both of us had busy consulting lives;we would schedule meetings and get our businessdone.
3. THE SEATTLE BONE MARROWTRANSPLANTATION TEAM
Rosenberger:
Today every street corner seems tohave a contract research organization for data co-ordinating centers on large clinical trials. But whenyou went to the Fred Hutchinson Cancer ResearchCenter, information technology was primitive, suchplaces did not exist. You had to create that envi-ronment on your own. What was it like? What werethe challenges?
Flournoy:
That’s an interesting story. Dick Kron-mal had invested a lot of effort in creating a databasemanagement system without requiring a rectangulardata structure. Updates required physically sortingthe cards (remember all data records had to fit intothe 80 digits of a Hollerith punch card—so I tend touse the words “card” and “record” interchangeably).There was a transplant data set in place with sevendifferent kinds of cards. Kronmal had E. Donnal“Don” Thomas (Director of the Clinical ResearchDivision at the Cancer Center) buy a computer. Thecomputer weighed 50 pounds (I could toss it in myvan and take it home; the cost was about $50,000),and data storage was on Phillips cassette tapes.Records could be transmitted across the phone wiresand then integrated into the database at UW. Ini-tially, there was not much data (only 10 patients)and the first update took my whole computing bud-get for the year! What I did then for some period oftime was, when it was time to do an update, punchcards of the whole database and the new dataset; Iwould physically sort and merge the cards by handand load them into SPSS. That was my “dirty laun-dry” story because the laundromat had big long ta-bles and I sorted cards while doing laundry. Kronmaltold me that, if I had any trouble with my new com-puter, I should call Leonard Hearne. Index sequen-tial files were brand new at that time, and Leonardused them to create an early database managementsystem before the word was in the literature (seeFlournoy and Hearne, 1981, 1990a, 1990b). We used
CONVERSATION WITH NANCY FLOURNOY it for several years until a commercial system cameon the market. At site visits, someone would ask aquestion and I would pass a note down to a program-mer, who would extract the answer in 15 minutes orso. We set the bar for oncology programs.Ross Prentice came from the University of Wa-terloo with a box of cards on the Cox proportionalhazards model; we were really early using that. Thedoctors were smart enough to understand the limi-tations in using discriminant analysis and they werethrilled to be able to incorporate censored survivaldata in their regression models. My work document-ing graft vs. leukemia in humans was very impor-tant (see Weiden et al., 1979, 1981a, 1981b, 1981c).One hypothesis motivating the development of bonemarrow transplantation was that the marrow graftwould attack residual leukemia also. Immunologicalactivity of the graft was apparent when the graftinstigated an immunological attack on the patient.I modeled the impact of this attack on the relapserate. The protection of the graft attack against re-lapse greatly complicated post-transplant treatmentstrategies. But our findings have withstood the testof time. It was, perhaps, the first major applica-tion of the proportional hazards model with time-dependent covariates. Rosenberger:
When you think of the success of thebone marrow transplant program (Don Thomas wonthe Nobel prize in 1990 for developing bone marrowtransplantation as a treatment for leukemia), howmuch did statistics and data management play a role in that? Do you think statistics and data man-agement will ever get its due?
Flournoy:
We had this rudimentary set of recordsthat could be added onto infinitely. It started outthat bacteriology wanted to add a card, then vi-rology, then specific studies would add a card withtheir data. Before you knew it, we had an inter-disciplinary shared database with assigned patientnumbers so all the integrated data was available.I was able to say “do you know what they’re do-ing in virology that’s related” because I knew ev-erybody’s data. It wasn’t until many years laterthat people started talking about having integratedshared databases. Most were established for billingpurposes, not for research purposes. They are differ-ent constructs. Hospitals would archive data afterthe bill was paid but we wanted to keep it aroundforever.When the program started, there was one of ev-erybody (one statistician, one virologist, etc.), andwe would sit around the table and share results. Itwas important to be influential and to catch prob-lems in data collection and quality control beforethey got big. When working with new doctors, therewere humps you had to get over because they wouldclaim that there were no quality control issues: theirlab people never made a mistake. A lot of negotia-tion had to go on before we could agree. Yes, we hada huge influence. Even randomization and blindingwas controversial. If it was in the middle of the nightthe cards might get shuffled; there was too muchroom for bias. We introduced them to a very careful
Fig. 4.
Yash Mittal, first female director of the probability program, and Nancy, first female director of the statistics program,at NSF.
W. F. ROSENBERGER randomization regimen for treatment assignments,with a 24 hour on-call person.It will be hard for statistics and data manage-ment to ever get its full due because the doctorsare so enamored of themselves (laughs). It’s really astrange system where the people with the least sci-ence background usually run the science. Also, thedata management budget was always the first to becut; yet it is very expensive to do a quality job.
Rosenberger:
What has your role been in fosteringinterdisciplinary research?
Flournoy:
Having conducted interdisciplinary re-search for more than a decade at the Cancer Center,I knew the power that teams of interdisciplinaryresearchers could bring to bear on important sci-entific questions. Coincidentally, when I went tothe National Science Foundation (NSF) in 1986,the Division of Mathematical Sciences (DMS) hadfunded the Institute of Mathematical Statistics(IMS) to write a report on cross-disciplinary re-search. I watched the growth in their thinking asthey interacted with each other. At the time, thediscipline did not appreciate the role of applicationsin academic settings. I think I was able to influ-ence the IMS cross disciplinary committee on thevaluable nature of interdisciplinary work. The re-port of the committee had a dramatic effect on thediscipline. The report proposed establishing the Na-tional Institute of Statistical Science. Since I was atNSF, I was able to promote the idea of establishinga broad institute that would work on problems ofnational importance.At the same time, I would receive proposals fromstatisticians motivated by applications. Because ourbudget was small, I took such proposals around tothe relevant disciplines that were involved, and wasable to get some joint funding. This resulted in mygetting an award in 1988 for facilitating the fund-ing of these interdisciplinary projects. This also ledto specific DMS requests for proposals for interdis-ciplinary research projects, which are now commonthroughout NSF.
4. ADAPTIVE DESIGNS
Rosenberger:
How did you get interested in adap-tive designs?
Flournoy:
While at the Cancer Center, the majorprogram project grant had five-year reviews. Whenwe prepared for the third one of these, we spent ayear reviewing what we had done and how we would go forward. In the course of that retrospective, Ideveloped some feelings about the two arm clinicaltrial. The standard ideas about the two arm clini-cal trials came from the Peto paper in the mid 70s(Peto et al., 1976, 1977). But, in my experience, atreatment is a point in a high dimensional space: in-volving drugs, radiation, including how much, howoften; and one learned little about this high dimen-sional space using the traditional two arm clinicaltrial. For instance, we spent five years comparingA to B; but then we go back to the high dimen-sional space and pick out point C, and have anotherfive years of experimentation and compare A to C.Then we compare C to D, and after 15 years wehave knowledge of four points in a high dimensionalspace. I believed it would be more efficient and in-formative to know which direction we should headin the high dimensional space. So that led me tothink about adaptive designs. I recommended sev-eral to the group and the physicians liked the ideas,but thought they may be too radical to get funded.Another thing that promoted my interest waslooking at pilot studies to decide what to take for-ward to larger studies. Bob Tsutukawa was visitingthe Cancer Center from the University of Missouriat the time. I thought his Bayesian ideas were ap-pealing and I used expert opinion for prior elicita-tion (see Flournoy, 1993). The prior was way off, sowe wound up with a lot of toxicities. You just can’ttrust the best expert opinion of the best experts,and so there needed to be some way to use interimdata faster to adapt and put much less weight onthe prior. My later work showed how random walkrules could be constructed to do this (see Durhamand Flournoy, 1994; Durham, Flournoy and Rosen-berger, 1997; Flournoy and Oron, 2015).
Rosenberger:
The first time I heard the nameNancy Flournoy was in the context of the 1989 ses-sion on adaptive designs at Joint Statistical Meet-ings (JSM) in Washington. It turned out to be oneof the most controversial sessions in the history ofJSM. Talk about that.
Flournoy:
My experience at NSF was that youdon’t make progress without community. One per-son alone doesn’t get much done. So I had the ideathat a JSM session on adaptive design would bringtogether people who are interested in adaptive de-signs. I didn’t know anyone personally. I invitedbased on my impressions of their interests. I in-vited Don Berry, Richard Simon and Janis Hard-wick. I gave a straightforward technical talk on the
CONVERSATION WITH NANCY FLOURNOY Fig. 5. topic. The remainder of the session focused primar-ily on criticism of the extracorporeal membrane oxy-genation (ECMO) trials (e.g., Barlett et al., 1985;O’Rourke et al., 1989; Ware, 1989). (The ECMOtrial was an implementation of the randomized play-the-winner rule of Wei and Durham, 1978, in which11 babies were assigned to an experimental arm, andall survived, while one baby assigned to the conven-tional arm, died. The historical death rate on theconventional arm was 80 percent.) To my dismay,all the negative focus of the session was directed to-ward the adaptive design aspect of the clinical trial,rather than on the sample size and what kind ofsample size would be needed for the trial to be con-vincing. The press that was generated by this sessionset adaptive designs back a long time.
Rosenberger:
How much do you think the failedECMO trial inhibited the development of adaptivedesigns?
Flournoy:
What would have been a reasonable ap-proach? The original trial was unconvincing due tohaving few patients, in spite of the fact that a prob-abilistically reasonable stopping rule was applied.The controversy over the subsequent two arm trial inclinical research set back adaptive designs wrongly.The adaptive trial was so successful that only onebaby died; is that bad?
Rosenberger:
In your 1992 AMS/IMS/SIAM con-ference on adaptive designs (Flournoy and Rosen-berger, 1995), you brought together some of thegroundbreakers of adaptive designs along with anumber of younger faculty who are now at the fore-front of the discipline. At the opening session, youstarted by talking about the need to streamline theprocess of clinical trials, the end to phases and theincorporation of dynamic interim decisions. You saidthat will revolutionize the way we do clinical tri-als, and that this conference would be an ambi-tious beginning to that revolution. Now, over two
W. F. ROSENBERGER decades later, there are 70-some sessions on adap-tive designs at the Joint Statistical Meetings, “big-pharma” working groups, Food and Drug Adminis-tration white papers and guidelines, companies likeADDPLAN, and CYTEL devoted to adaptive de-sign software and everyone wants to do adaptive de-signs. What took so long?
Flournoy:
ECMO made a steep hole to climb. Wealso had to develop theory. It was one thing to say“this is a good idea,” and another to adequatelysupport it. Some ideas were NOT good. This in-cludes a class of procedures that derive from stochas-tic approximation, that Val Fedorov coined “bestintention” designs. In these designs, a target doseis estimated (such as the dose having a particularpercent toxicity or one that maximizes some utilityfunction); then that estimate is the dose given tothe next subject. Some, including Lai and Robbins(1982), understood early on that using this proce-dure without safeguards may result in treatment se-quences that converge to the wrong dose. But others,including myself (Li, Durham and Flournoy, 1995),were enamored of this idea and ignorant of ear-lier warnings. This approach remains popular todayeven as recent publications are exposing just howmisleading it can be (e.g., Azriel, 2012, Oron andHoff, 2013).In the 1980s, John Whitehead spent a year visitingthe Cancer Center from the University of Reading,and promoted the idea of using sequential stoppingrules taking censoring into account. Its value was soobvious that I expected that by 1990 every clinicaltrial would be using these techniques. So I focusedinstead on adaptive allocation. At American Uni-versity (AU), I worked on theoretical problems inthese areas. When I pulled my head out and lookedaround I was shocked to see that stopping rules in-corporating censoring were not being used, except abit in cancer. So things that seem obvious to somecan take a long time to enter the medical arena.Take the “3 + 3” dose escalation design as an exam-ple. It has been soundly discredited (Reiner, Pao-letti and O’Quigley, 1999; Lin and Shih, 2001), andyet remains a standard practice in oncology phase Itrials.Adaptive allocation is still in its infancy comparedto sequential monitoring and stopping. Now therehas developed a new belief that simulation is ad-equate for assessing an adaptive design. But rely-ing solely on simulation muddies the water becausethere is no global view of what is driving the design. In addition, there are many papers in the literaturethat report only averages over simulations withoutmeasures of variability. When you consider measuresof variability, a completely different picture emerges(Oron and Hoff, 2013). I fervently believe in devel-oping the theory underlying classes of designs. For-tunately, many people are interested in working onthe theoretical challenges, and there are a lot of in-teresting open questions.
Rosenberger:
Many times when I hear talks onadaptive designs I want to scream out “NancyFlournoy thought of that in the 1980s.” How do youfeel about some of your early ideas being ignored?
Flournoy:
Well, I’m hardly alone in this. For in-stance, Chris Jennison invented many clever tech-niques for sequential and adaptive clinical trialsvery early that are sometimes “rediscovered” with-out reference (e.g., Jennison, Johnstone and Turn-bull, 1982; Kulkarni and Jennison, 1986; Jennison,1987). In my case, it amazes me that there are alarge number of people who will reference a paperfrom the 1980s and ignore 30 years of my research.For example, the early up and down paper of Storer(1989) is often cited without reference to my laterpapers that have much more sophisticated controlof the adaptive process. This early paper is used asa whipping post to declare up and down proceduresinferior. An up and down design is a random walkthat can end anywhere. The last state (dose) vis-ited should not be used as an estimator. But thisis done when the up and down design is comparedto other procedures that derive from stochastic ap-proximation (e.g., Zacks, 2009). That bothers me alot.
Rosenberger:
How did you meet Steve Durham?This began one of the great collaborations in statis-tics. Tell us about that.
Flournoy:
One of the few positive consequences ofthe 1989 JSM session was meeting Steve Durhamfrom the University of South Carolina. When Iwalked out the door after the session, Steve intro-duced himself and was very excited because we werebasically working on the same mathematical prob-lems, his from an engineering motivation, and minefrom a medical motivation. We began working to-gether right away. He would come to Washington,DC, to meet me, and I went to South Carolina. Aftera stint as Chair at AU, I was on a sabbatical at theUniversity of North Carolina Chapel Hill; Leonardand I bought a house close to campus so that wecould host visitors. In particular, Steve Durham and
CONVERSATION WITH NANCY FLOURNOY I worked together quite a lot in that house and atthe Department of Statistics. Several other collab-orators came down for extended periods, includingyou (W.F.R.) and two of my doctoral students: EloiKpamagen (now at Novavax) and Misrak Gezmu(now at National Institutes of Health).
Rosenberger:
The introduction of the randomwalk rules coincided with the introduction of thecontinual reassessment method (CRM; O’Quigley,Pepe, and Fisher, 1990) in the Bayesian context. Inparticular, you and Steve worked out the entire ex-act distribution theory of a class of designs, whileothers were relying on simulation. How does thisrank in terms of your contributions to statistics?
Flournoy:
The random walk rules are extremelypractical and mathematically elegant, so it was alot of fun to develop the theory. They are the stan-dard in many areas of science (e.g., American So-ciety for Testing and Materials, 2010; Treutwein,1995; National Institute of Environmental HealthSciences, 2001). The key property that we discoveredwas how to control the allocation coverage by intro-ducing an appropriate bias (Durham and Flournoy,1994). Steve was always thinking in terms of engi-neering applications; I was always thinking in thedose-response context. We did “reverse engineer-ing,” in that we had a target allocation in mind, andwe found design parameters to facilitate this. Thedesigns are nonparametric in that allocation doesnot depend on estimates of model parameters. Theyare extraordinarily simple to illustrate and have ex-act distribution theory that is unavailable for other,more complicated designs.
Rosenberger:
Some have lumped random walkrules in the context of generic dose escalation de-signs, such as the 3 + 3 design, that has no op-timal properties. At the same time, Bayesian ap-proaches, such as the CRM were becoming increas-ingly well-known. Talk about the historic interplayamong these approaches.
Flournoy:
Lloyd Fisher and John O’Quigley (fromthe University of Leeds) were hired at the Can-cer Center to replace me when I left for the NSF.Lloyd and I laughed that it is not often one’s dis-sertation advisor replaces his student! John was ini-tially responsible for implementing a random walkrule that I had designed in a pilot study for a bonemarrow clinical trial. He let them get away with asimple dose escalation procedure, but he and Lloydgot introduced to the subject at that time. They immediately thought of doing a Bayesian alterna-tive, and it was published in 1990 in
Biometrics (O’Quigley, Pepe and Fisher, 1990); the major ran-dom walk paper appeared in
Biometrics in 1997(Durham, Flournoy and Rosenberger, 1997). Mostof the Bayesian literature was, by necessity, simula-tion based, whereas Steve and I were busy obtaininga complete workable probabilistic theory of the ran-dom walk procedures.There are a number of philosophical differencesamong the approaches. Fedorov would call the CRMa “best intention” approach, because it involves pre-dicting a target dose and treating the next patient atthat dose, sequentially. Our approach is estimation-motivated. The idea is to get allocations into a re-gion of interest that allows us to efficiently estimatethe dose-response curve in that region.There is also a short-memory and long-memorydistinction: allocation probabilities for the ran-dom walk rule converge exponentially fast to theirasymptotic limits. Alternatively with best intentiondesigns (which to date are long-memory designs),nonrepresentative early allocations can cause the de-sign to converge to the wrong dose (see, e.g., Azriel,Mandel and Rinott, 2011; Oron, Azriel and Hoff,2011; Azriel, 2012). Such phenomena were observedearly on in the context of stochastic approxima-tion designs (e.g., Lai and Robbins, 1982; Bozin andZarrop, 1991).Adaptive optimal designs are promising longmemory designs, but they depend on parameter esti-mates to get started. Random walk procedures thattarget optimal design points provide good start-up information with small sample sizes. Alterna-tively, one can regularize the information matrix, a“fix” that is often called “Bayesian designs” eventhough no posterior distribution is obtained. TrueBayesian estimator updates coupled with dose al-locations made in some stable optimal way, ratherthan in a “best intention” way are also promising.
Rosenberger:
What is the future of adaptive de-signs? Do you think all clinical trials will eventuallybe adaptive?
Flournoy:
I think there is a great future for adap-tive designs. I think experimentation will always in-volve a series of trials; the question is how well oneutilizes information from one to the next. There isa lot of value in relatively small but sequential tri-als (see Flournoy, 2014), because these trials involvemany design features, including the grid size andrange on which you are operating. The best use of W. F. ROSENBERGER one experiment may be to tell you how you couldhave better selected design characteristics; then youcan refine the estimate of the target of interest.Some of my work has been on inference and es-timation following adaptive designs (e.g., Rosen-berger, Flournoy and Durham, 1997; Ivanova andFlournoy, 2001; May and Flournoy, 2009; Lane, Yaoand Flournoy, 2014). One has to be careful doingeverything sequentially because some of the interimchanges may cause final estimates to lack normal-ity. For example, in best-intention designs, the esti-mate of a slope parameter can march off to infinityfor some common models. Also, even if an adaptivedose-finding procedure has a fixed total sample size,the sample sizes at each dose are random variables.In up-and-down procedures, the proportion of sub-jects allocated to each dose tends to a constant andstandard asymptotic normality results. But in manyother adaptive designs, proportions of subjects allo-cated to each dose tend to a random variable. Thiscauses the conditional information matrix to be ran-dom, even in the limit, in which case standard condi-tions for asymptotic normality fail. These are manyinteresting questions to be explored about adaptivedesigns.
5. WOMEN IN STATISTICS
Rosenberger:
Talk about the creation of Pathwaysto the Future, its successes, and its legacy.
Flournoy:
I went to the 1984 Annual IMS Meet-ing in Lake Tahoe. At that meeting, there were five women out of about 200 attendees. It became quiteclear to me that this was an important place for aca-demic statisticians to meet and focus on academicinterests. In anticipation of the 1988 Fort CollinsIMS meeting, which was separate from the JointStatistical Meetings, I decided it would be great tosee more women there. So I bounced ideas off MaryEllen Bock (Purdue University) and Lynne Billard(University of Georgia). Lynne agreed to take thelead in organizing a workshop for women at the up-coming IMS meeting. Lynne had the brilliant ideaof having Elizabeth Scott (University of California,Berkeley) give the keynote lecture. At that time,we were debating whether there was any gender in-equity in academia, and we weren’t sure. I had neverexperienced problems at UCLA or UW. However,when I went to the NSF, Yash Mittal (the first fe-male director of the probability program) and I sawthat there were almost no female grantees, and veryfew were even applying for grants.The evening presentation by Scott really hit usvery hard: she had tons of data and randomizedstudies on gender inequity. Any questions about in-equities in how women were recruited, judged andvalued were thrown out the window. Scott’s wayof handling this lecture was wonderful because shewent through all this horribly depressing data, butshe then turned around and finished the lecture bytelling us what we could do to protect ourselves. Sheended with two positive notes: that outcomes arenot predetermined, and one can take one’s career in
Fig. 6.
Nancy, husband Leonard, and Lynne Billard at their home in Chapel Hill, NC, 1994.
CONVERSATION WITH NANCY FLOURNOY Fig. 7.
Nancy, Ingram Olkin and Elizabeth Margosches (formerly with the Environmental Protection Agency) at the Cam-panile at University of California, Berkeley, 2003. one’s own hands. Lynne ran the workshop for thenext two decades, and she presented Scott’s lecturewith updated data every year. That lecture was thelast lecture Scott gave before she passed away. I re-member well that there was a palpable sigh of relieffrom Scott—that she could turn over her cause tothe next generation.
Rosenberger:
How did you become NSF programdirector? What was your experience with gender is-sues there?
Flournoy:
Ingram Olkin has long been a greatfriend and mentor. He is the one who recommendedme to the NSF for the program director position. Iwas the first female director in the statistics programthe same year that Yash Mittal was the first femaleprobability director. Some people had indicated tothe division director their fear I was going to giveall the grant money to biostatistics. I convinced himthat I could represent the entire statistics field.One day I remember answering the phone and aprofessor on the line yelled “I said I wanted to speakto the director,” thinking a woman on the phonemust be a secretary.We had a good travel budget and I went to asmany young women’s lectures as I could. I would goup at the end of their talk and ask if they would be interested in applying for a grant. By the time I leftNSF, the proportion of grant proposals from womenwas proportional to their presence in the field. Asuggestion is such a small thing, and yet clearly im-portant messages weren’t being transmitted to fe-male faculty.
Rosenberger:
Was discrimination subtle or not sosubtle when your career was developing?
Flournoy:
Well, there was always sexist behaviorand many things that were said and done are consid-ered inappropriate or even sexual misconduct today.When I went on the job market for a fully academicposition I found that many men were incredulous.Some would make outrageous comments directly tome as if I were invisible (or a man). Men in my ownage category were often dismissive or oblivious tomy presence. Some of the older generation was veryhelpful and supportive (I think of Shanti Gupta,Purdue University; Norman Johnson, University ofNorth Carolina at Chapel Hill; Lucien LeCam, Uni-versity of California, Berkeley; Ingram Olkin, Stan-ford University; and Manny Parzen, Texas A&MUniversity). The younger generation just thought ofme as another senior person, so they were fine.
Rosenberger:
What is your feeling about the roleof women in statistics today? I can say, from my W. F. ROSENBERGER perspective on 20 years of search committees, thatfrom a hiring perspective, we are thrilled to havequalified women candidates and compete hard toget them. And certainly policies on tenure to allowmaternity leave have vastly improved over the years,as have the composition of committees and senioradministrators. Is there any work left to be done?
Flournoy:
You can see improvement, but there arestill troubling facts: just try to find a woman in the2013 JSM awards brochure, for instance. Women aregetting hired at proportional rates now, but awards,tenure and advancement are areas where there muchis left to be done. See Lynne Billard’s new update ofScott’s old data on the subject (Billard and Kafadar,2015). That will depress you.
6. CONCLUSION
Rosenberger:
You talked a little about your tran-sition into a fully academic position. The latter partof your career was spent at AU and University ofMissouri (MU), and considerable time as depart-ment chair, and a mentor to many diverse students.Talk about this.
Flournoy:
AU was a great place for me when Iwent there in 1988. I had left the Cancer Centerwith a staff of 23, a budget of $700,000 and re-sponsibilities that had become a burden when I be-came convinced of the need for more nimble learningstrategies in dose-finding clinical trials. I had eightdoctoral students at AU, and all but two of themdeveloped mechanisms to control random walks andurn models, and to provide mathematical descrip-tions of their controlled behavior. One worked onissues of inference following an adaptive design andone worked on a problem in economics. I am proudthat four of these students are black and two arewomen.Unfortunately, a very destructive president cameto AU, and by 2000 it was clear that STEM gradu-ate programs were going to be dismantled. AU hadone of the oldest statistics doctoral programs in thecountry and it was sad to see it threatened by ig-norance and arrogance. To remain in a departmentwith a doctoral program, I needed to move and thisled me to accept the chair at Missouri in 2002. WhenI stepped down as chair in 2011, I had doubled thenumber of tenure-track faculty and added five teach-ing faculty positions. I increased the presence of thedepartment across campus through joint appoint-ments and a targeted increase in service courses, and
Fig. 8.
Nancy near Aasgard Pass in the Enchantment LakesWilderness Area, Washington, where she was hiking with herhusband Leonard and her colleague Lori L. Thombs (Univer-sity of Missouri) following the 2006 Joint Statistical Meetingsin Seattle.
I increased the prestige of the department nation-ally, personally promoting our faculty and enablingtheir participation in national and international ac-tivities. More details can be found in a Chapter Irecently wrote on the history of statistics at MU(Flournoy and Galen, 2012).I have graduated seven doctoral students fromMU. We worked on adaptive and optimal designs;we developed new models for specific, challengingdose-response problems and we have illuminated theeffect of having dose allocations depend on the his-tory of prior allocations and responses. My studentscontinue to bring me a great deal of pleasure.
Rosenberger:
What are your hobbies and inter-ests?
Flournoy:
I love hiking. I am not happy with atrip that takes less than four days. A four-day triphas two days out and two days back—so one is neververy far from a road. After hiking for more than twodays, one must rely on one’s self much more com-pletely. It is so peaceful. I gave up trying to hikein the East and the Midwest United States. Onejust can’t get far enough away from roads; and themountains aren’t high enough. I like trekking around
CONVERSATION WITH NANCY FLOURNOY timberline for a week or more where the views arespectacular. I keep going back to Yosemite, KingsCanyon and Sequoia National Forests. Nepal wasgreat, too. I try to get in one long hike each year.In the meantime, I dance. I resumed ballet classeswhile at AU; it is great mind-to-body exercise andwonderful for strength and balance. Leonard and Ienjoy English country dance together. Throw in Pi-lates and yoga and I am happy.To survive a severe health challenge that had thedoctors stumped, I gained considerable knowledgeof alternative methods and became accomplished insome. But that is another story. Rosenberger:
What’s next for Nancy Flournoy?
Flournoy:
Well I have a lot of ideas. I’m really in-terested in questions of inference following adaptivedesigns. We have some examples in two stage designsthat maximum likelihood estimators are mixtures ofnormals; some designs lead to estimators that arenormal with random variances. I think our prelim-inary results are generalizable, but this remains tobe shown. I’m optimistic that tractable solutions toseemingly intractable problems are at hand.REFERENCES
American Society for Testing and Materials (2010). Stan-dard test method for estimating acute oral toxicity in rats.American Society for Testing and Materials, ASTM E1163-10. ASTM International, West Conshohocken, PA.
Azriel, D. (2012). A note on the robustness of the contin-ual reassessment method.
Statist. Probab. Lett. Azriel, D. , Mandel, M. and
Rinott, Y. (2011). The treat-ment versus experimentation dilemma in dose finding stud-ies.
J. Statist. Plann. Inference
Bartlett, R. H. , Roloff, D. W. , Cornell, R. G. , Andrews, A. F. , Dillon, P. W. and
Zwischen-berger, J. B. (1985). Extracorporeal circulation in neona-tal respiratory failure: A prospective randomized study.
Pediatrics Billard, L. and
Kafadar, K. (2015). Women in statis-tics: Scientific contributions versus rewards. In
AdvancingWomen in Science ( W. Pearson Jr. , L. M. Frehill and
C. L. McNeely , eds.) Chapter 7. Springer, New York.
Bozin, A. and
Zarrop, M. (1991). Self-tuning extremum op-timizer convergence and robustness. In
Proc. 1st EuropeanControl Conf. 91
Durham, S. D. and
Flournoy, N. (1994). Random walksfor quantile estimation. In
Statistical Decision Theory andRelated Topics, V (West Lafayette, IN, 1992)
Durham, S. D. , Flournoy, N. and
Rosenberger, W. F. (1997). A random walk rule for phase I clinical trials.
Bio-metrics Flournoy, N. (1993). A clinical experiment in bone marrowtransplantation: Estimating a percentile point of a quan-tal response curve. In
Case Studies in Bayesian Statis-tics ( C. Gatsonis , J. S. Hodges , R. E. Kass and
N. D. Singpurwalla , eds.) 324–335. Springer, New York.
Flournoy, N. (2014). A vignette of discovery. In
Past,Present and Future of Statistical Science, in Celebration ofthe COPSS 50th Anniversary ( X. Lin , D. Banks , C. Gen-est , G. Molenberghs , D. Scott and
J.-L. Wang , eds.)349–358. Chapman & Hall/CRC, Boca Raton, FL.
Flournoy, N. and
Galen, M. (2012). History of the statis-tics department at the University of Missouri. In
Historyof Statistics Departments ( A. Agresti and
X.-L. Meng ,eds.). Springer, New York.
Flournoy, N. and
Hearne, L. B. (1981). Effects ofdatabase management on the organization and adminis-tration of clinical trials. In
First Lawrence Berkeley Labo-ratory Workshop on Statistical Database Management ( H.K. T. Wong , ed.) 383–387. Lawrence Berkeley NationalLaboratory, Menlo Park, CA.
Flournoy, N. and
Hearne, L. B. (1990a). Quality con-trol for a shared multidisciplinary database. In
Data Qual-ity Control: Theory and Pragmatics ( G. E. Liepens and
V. R. R. Uppuluri , eds.) 43–56. Dekker, New York.
Flournoy, N. and
Hearne, L. B. (1990b). Sharing scien-tific data III: Planning and the research proposal.
IRB: AReview of Human Subject Research Flournoy, N. and
Oron, A. P. (2015). Up-and-down de-signs for dose-finding. In
Handbook of Design and Analysisof Experiments ( D. Bingham , A. M. Dean , M. Morris and
J. Stufken , eds.) 862–898. Chapman & Hall/CRC,Boca Raton, FL.
Flournoy, N. and
Rosenberger, W. F. , eds. (1995).
Adaptive Designs . Institute of Mathematical Statistics Lec-ture Notes—Monograph Series . IMS, Hayward, CA.MR1477667 Ivanova, A. and
Flournoy, N. (2001). A birth and deathurn for ternary outcomes: Stochastic processes applied tourn models. In
Advances in Statistical Theory—A Volumein Honor of Theophilos Cacoulous ( C. Charalambides , M. V. Koutras and
N. Balakrishnan , eds.) 583–600.CRC Press/Chapman & Hall, Boca Raton, FL.
Jennison, C. (1987). Efficient group sequential testswith unpredictable group sizes.
Biometrika Jennison, C. , Johnstone, I. M. and
Turnbull, B. W. (1982). Asymptotically optimal procedures for sequentialadaptive selection of the best of several normal means. In
Statistical Decision Theory and Related Topics, III, Vol. 2(West Lafayette, Ind., 1981)
Kulkarni, R. V. and
Jennison, C. (1986). Optimal prop-erties of the Bechhofer-Kulkarni Bernoulli selection proce-dure.
Ann. Statist. Lai, T. L. and
Robbins, H. (1982). Iterated least squaresin multiperiod control.
Adv. in Appl. Math. Lane, A. , Yao, P. and
Flournoy, N. (2014). Informationin a two-stage adaptive optimal design.
J. Statist. Plann.Inference W. F. ROSENBERGER
Li, Z. , Durham, S. D. and
Flournoy, N. (1995).An adaptive design for maximization of a contingentbinary response. In
Adaptive Designs (South Hadley,MA, 1992) . Institute of Mathematical Statistics LectureNotes—Monograph Series Lin, Y. and
Shih, W. J. (2001). Statistical properties of thetraditional algorithm-based designs for phase I cancer clin-ical trials.
Biostatistics May, C. and
Flournoy, N. (2009). Asymptotics in response-adaptive designs generated by a two-color, randomly rein-forced urn.
Ann. Statist. O’Quigley, J. , Pepe, M. and
Fisher, L. (1990). Continualreassessment method: A practical design for phase 1 clinicaltrials in cancer.
Biometrics O’Rourke, P. P. , Crone, R. K. , Vacanti, J. P. , Ware, J. H. , Lillehei, C. W. , Parad, R. B. and
Ep-stein, M. F. (1989). Extracorporeal membrane oxygena-tion and conventional medical therapy in neonates withpersistent pulmonary hypertension of the newborn: Aprospective randomized study.
Pediatrics Oron, A. P. , Azriel, D. and
Hoff, P. D. (2011). Dose-finding designs: The role of convergence properties.
Int. J.Biostat. Art. 39, 19. MR2873999
Oron, A. P. and
Hoff, P. D. (2013). Small sample behaviorof novel phase I cancer trial designs.
Clinical Trials Peto, R. , Pike, M. C. , Armitage, P. , Breslow, N. E. , Cox, D. R. , Howard, S. V. , Mantel, N. , McPher-son, K. , Peto, J. and
Smith, P. G. (1976). Design andanalysis of randomized clinical trials requiring prolongedobservation of each patient. I. Introduction and design.
Br.J. Cancer Peto, R. , Pike, M. C. , Armitage, P. , Breslow, N. E. , Cox, D. R. , Howard, S. V. , Mantel, N. , McPher-son, K. , Peto, J. and
Smith, P. G. (1977). Design andanalysis of randomized clinical trials requiring prolongedobservation of each patient. II. Analysis and examples.
Br.J. Cancer Reiner, E. , Paoletti, X. and
O’Quigley, J. (1999). Op-erating characteristics of the standard phase I clinical trialdesign.
Comput. Statist. Data Anal. Rosenberger, W. F. , Flournoy, N. and
Durham, S. D. (1997). Asymptotic normality of maximum likelihood es-timators from multiparameter response-driven designs.
J.Statist. Plann. Inference Storer, B. E. (1989). Design and analysis of phase I clinicaltrials.
Biometrics Thall, P. F. and
Cook, J. D. (2004). Dose-finding basedon efficacy-toxicity trade-offs.
Biometrics Treutwein, B. (1995). Minireview: Adaptive psychophysicalprocedures.
Vision Res. Ware, J. H. (1989). Investigating therapies of potentiallygreat benefit: ECMO.
Statist. Sci. Wei, L. J. and
Durham, S. (1978). The randomized play-the-winner rule in medical trials.
J. Amer. Statist. Assoc. Weiden, P. L. , Flournoy, N. , Thomas, E. D. , Pren-tice, R. , Fefer, A. , Buckner, C. D. and
Storb, R. (1979). Antileukemic effect of graft-versus-host disease inhuman recipients of allogenic-marrow grafts.
New Engl. J.Med.
Weiden, P. L. , Flournoy, N. , Sanders, J. E. , Sulli-van, K. M. and
Thomas, E. D. (1981a). Antileukemiceffect of graft-versus-host disease contributes to improvedsurvival after allogeneic marrow transplantation.
Trans-plant. Proc. Weiden, P. L. , Sullivan, K. M. , Flournoy, N. , Storb, R. , Thomas, E. D. and the Seattle BoneMarrow Transplant Team (1981b). Antileukemic ef-fect of chronic graft-versus-host disease. Contribution toimproved survival after allogeneic marrow transplantation.
New Engl. J. Med.
Weiden, P. L. , Flournoy, N. , Thomas, E. D. , Fefer, A. and
Storb, R. (1981c). Antitumor effect of marrow trans-plantation in human recipients of syngeneic or allogeneicgrafts. In
Graft-Versus-Leukemia in Man and Animal Mod-els ( J. Okunewick and
R. Meredith , eds.) 11–23. CRCPress, Boca Raton, FL.
Zacks, S. (2009).