Research by John Birge points the way to safer and cheaper drug trials.
It’s a formula that vexes researchers and patients alike: spend years conducting a clinical trial on a new drug, only to discover at the end of the trial that the drug is useless—or, worse, harms patients. Also, testing new drugs is expensive, and getting more so. More than 90 percent of the cost of a drug’s development comes from Phase III clinical trials, according to a report last year from the Manhattan Institute, a center for policy research. The average length of a clinical trial increased by 70 percent from 1995 to 2005, and the cost of a single trial has ballooned to as much as $100 million.
For years the medical community has maintained this status quo—a traditional clinical trial that can be long, expensive, and inconclusive—but John R. Birge, Jerry W. and Carol Lee Levin Professor of Operations Management, and PhD student Vishal Ahuja are trying to change that. The operations management experts are developing a model for clinical trials that could speed up research, save money, get better treatments to patients, and save lives. The details for these humble aspirations are in the math.
Birge and Ahuja’s research into clinical trials grew out of the frustration felt by Anirban Basu, now an associate professor at the University of Washington. Three years ago, when he was an assistant professor in the Department of Medicine at the University of Chicago, Basu was analyzing data from a large national trial that compared secondgeneration antipsychotic drugs used to treat schizophrenia with older drugs. The trial found the new drugs weren’t more effective, but Basu wondered if that conclusion was due to design flaws that are inherent in a conventional trial.
In such a trial, researchers divide patients into roughly equal groups. Then in various rounds of study, they randomly assign patients either a treatment or a placebo. At the end of the trial, researchers use statistical analysis to understand how well each treatment worked. Birge has some firsthand knowledge of this: in the mid-1990s, he participated as a patient in one such trial, a study for the blockbuster drug Lipitor, which treats high cholesterol. Every two weeks, he was randomly assigned
either Lipitor or a placebo, with equal chances of receiving either. That meant that he was often off the medication.
But could trials work more like doctors, who switch patients from one treatment to another based on how each patient reacts to a drug or device? As researchers began to understand that Lipitor was highly effective and safe, could they have randomly assigned a greater percentage of the patients to be taking the drug, potentially improving patients’—including Birge’s—overall health, without sacrificing any learning?
| Traditional design | Adaptive design | |
|
In a traditional for a Phase III design clinical trial, patients are divided into roughly equal groups and randomly assigned a particular treatment. The goal of the trial is to determine which treatment is more effective. For example, suppose researchers want to compare the effectiveness of a new pain reliever to ibuprofen. If they have 10 patients enroll in the first round of the trial, five will receive the new drug and five will receive ibuprofen. Patients will be randomly assigned to each group. In each subsequent round of the trial, the same numbers of patients will be randomly assigned to each treatment. At the end of the trial, researchers analyze the results to determine whether the new pain reliever or ibuprofen was more effective. |
In the adaptive design for clinical trials that John R. Birge and Vishal Ahuja propose, researchers may adjust the number of patients randomly assigned to each group as they learn about a particular treatment. Again, imagine researchers are testing the effectiveness of a new pain reliever compared with ibuprofen. In the first study group, as in the traditional model, they randomly assign equal numbers of patients to each treatment.
This time, after the first round of study, two patients taking ibuprofen say their pain was relieved, while three say they were not helped. In the group taking the new pain reliever, four say the drug helped their symptoms, while one says it did not. In the next round of the trial, researchers can adjust the sizes of the study groups, randomly assigning a greater proportion of patients to the new drug, which appeared more effective than ibuprofen. The proportions can continue to change in subsequent rounds as researchers learn more. |
Trials can work this way when they are designed to be “adaptive.” Adaptively designed drug trials were pioneered in the 1970s by Don Berry, a faculty member at the University of Texas MD Anderson Cancer Center. In an adaptively designed trial, patients continue to be randomly assigned treatments or placebos, as they would be in a traditional study. However, the proportions of patients assigned a given treatment can change in each round of assignments as researchers learn more about each treatment's safety and efficacy.
Suppose researchers are comparing two diabetes drugs, and drug A isn’t working well for a number of patients in the first round of a trial. In a traditional trial, the same proportion of patients would be randomly assigned drug A in the next round. By contrast, in an adaptive trial, researchers might see that more patients taking drug B were improving relative to the number of patients taking drug A. In the next round of assignments—which could include either a new group of patients or the ones already enrolled in the trial—a higher percentage of patients would randomly be assigned drug B.
The problem with adaptive trials, and the main reason they’re not more widely used, is that the trials have been slow and small by necessity. Researchers learn how a treatment affects a single patient, and that knowledge is incorporated into how the next patient is treated. But the trials don’t learn from multiple patients participating in a study simultaneously. That makes it hard to run large adaptive trials, which are frequently needed to study new drugs and treatments.
To solve that constraint, Birge and Ahuja combine two mathematical frameworks. The first is a Markov Decision Process (MDP), which can be applied when event outcomes are partly decided and partly random. The second is a Bayesian learning framework, which involves using new data to update the probability that an event will occur. The result, to borrow a term from a 2003 paper by Michael O’Gordon Duff, is called a Bayesadaptive Markov decision process. In this, the probabilities of an event happening are unknown and may vary over time as more information is observed. While Duff’s work was largely theoretical and focused on computer programming, Birge and Ahuja apply the Bayes-adaptive Markov decision process to drug trials. The probabilities at the beginning of a trial are derived from what clinicians know and believe at the time. As the trial progresses and clinicians obtain more information, they update their beliefs dynamically.
To test their model, Birge and Ahuja apply it to data from a 2008 trial that was conducted at 50 medical centers nationwide and involved 451 patients. The trial tested a stent, a device designed to improve blood flow to an artery in the brains of stroke patients, but the trial was halted when researchers discovered that patients receiving the stents were more than twice as likely to have a second stroke or die than those treated with conventional medical therapies. By the time the study was terminated, five people who had received stents had died, and a total of 46 participants in the trial had experienced a stroke or died within 30 days of receiving treatment.
The researchers in the trial ultimately learned that the stent was riskier than the alternative treatment. But their new model, the Booth researchers believe, would have allowed them to gain the same knowledge in less time, at less cost, and with less harm to patients. A trial “failure” is defined as a patient who suffers a stroke or dies within 30 days of treatment, and their research says the model would have prevented 17 failures, more than a third of the total.
Moreover, the design would have provided an additional layer of protection to patients who participated in the trial. Generally speaking, regulators scrutinize testing on medical devices less than they scrutinize it for pharmaceutical drugs. The stent being tested had been approved for general use by the Food and Drug Administration (FDA) in 2005 under a fasttrack process called the Humanitarian Device Exemption, meant to help devices reach the market that would benefit a few patients (fewer than 4,000 annually). Devices approved under this exemption tend to undergo less rigorous testing than those that go through the regular approval process. So patients participating in a study involving such a device take on additional risk, as they did in this one.
The new model has received some attention: Birge and Ahuja’s working paper, which they are revising for publication, last year won the Pierskalla Award from the Institute for Operations Research and the Management Sciences for the best research paper in healthcare management science.
But publishing a better model would be just the first step towards actually implementing it, which requires FDA approval. Beyond that, the model has to be refined. It works well with diseases and treatments when effects reveal themselves quickly, says Elbert Huang, director of the University of Chicago’s Center for Translational and Policy Research of Chronic Diseases. But the model, and adaptively designed studies in general, work less well when it comes to diseases and treatments whose effects manifest more slowly.
At his primary care practice, Huang has many older patients with Type 2 diabetes, and he tries to find the best treatment for each patient. That can be tough because there are many drugs to choose from, and they’re often used in combinations that haven’t been studied. In March the FDA added to the complexity doctors face when it approved the first of a new class of medicines to treat diabetes. On top of that, diabetes can take different forms that aren’t fully understood, and many patients have other health problems that complicate treatment.
One challenge driving up drug trial costs is the difficulty of attracting and retaining patients. Researchers find it hard to recruit volunteers since many people are wary of having a new drug or medical device tested on their bodies. Once they're in a trial, patients frequently drop out, often because the treatment they're receiving doesn't work as well as the previous approach their doctors have tried, or because they experience unpleasant or risky side effects.
Who could blame them? Some elements of traditional drug trials have their roots in farming, according to Anirban Basu of the University of Washington, which can make them feel rudimentary.
The method of randomly assigning patients to different groups for testing medical treatments evolved from a plan used by researchers in the 1920s to analyze crop yields. They divided a plot of land into segments and randomly sowed different seeds in each segment to see which produced the biggest harvest.
While those elements may have been cutting-edge at the time, technology has progressed.
"The problem in translating that type of design—the design used in all randomized clinical trials today—is that now you're 'sowing' medicine in a human body," Basu says. "The human body is not homogenous like a piece of land. You have to understand not the average result, but how different people respond in different ways."
Basu believes the new Patient-Centered Outcomes Research Institute, created by the Patient Protection and Affordable Care Act, will push clinical trials toward designs that maximize patient health—such as the trial model developed by Booth researchers.
To address the challenge of applying the model to diabetes, Birge and Ahuja are doing some more research, studying doctors who treat diabetes to understand how they determine the best sequences of treatments to offer patients. The researchers are using data from a large, government-funded study Huang is overseeing, which is investigating what drugs doctors have prescribed for roughly 500,000 diabetes patients treated through the US Department of Veterans Affairs.
The research, as it is further developed, could find applications beyond health care. Companies such as Amazon and Netflix are using mathematical models to determine what products to recommend to individual customers and how much to charge. This Booth-led model could potentially improve those recommendations by considering and learning from individual preferences.
But the researchers are in no rush to seek new applications. After all, the health-care industry is burdened by skyrocketing expenses, and doctors’ offices are full of people in need of treatments. Creating a better drug trial is a fine place to start.
How the calculus behind exit vs. voice has changed
Does Social Media Make Companies Behave Better?
Testing assumptions in the field can avoid costly errors in decision-making.
You Should Run More Experiments
Why now is the right time for small companies to take a hard look at their business models.
Pivoting in a Time of CrisisYour Privacy
We want to demonstrate our commitment to your privacy. Please review Chicago Booth's privacy notice, which provides information explaining how and why we collect particular information when you visit our website.
