EVERY
spring, some 30,000 oncologists, medical researchers and marketers gather in an
American city to showcase the latest advances in cancer treatment.
But
at the annual meeting of the American Society of Clinical Oncology last month,
much of the buzz surrounded a study that was anything but a breakthrough. To a
packed and whisper-quiet room at the McCormick Place convention center in
Chicago, Mark R. Gilbert, a professor of neuro-oncology at the University of
Texas M. D. Anderson Cancer Center in Houston, presented the results of a
clinical trial testing the drug Avastin in patients newly diagnosed with
glioblastoma multiforme, an aggressive brain cancer. In two earlier, smaller
studies of patients with recurrent brain cancers, tumors shrank and the disease
seemed to stall for several months when patients were given the drug, an
antibody that targets the blood supply of these fast-growing masses of cancer
cells.
But
to the surprise of many, Dr. Gilbert’s study found no difference in survival
between those who were given Avastin and those who were given a placebo.
Disappointing
though its outcome was, the study represented a victory for science over guesswork,
of hard data over hunches. As far as clinical trials went, Dr. Gilbert’s study
was the gold standard. The earlier studies had each been “single-arm,” in the
lingo of clinical trials, meaning there had been no comparison group. In Dr.
Gilbert’s study, more than 600 brain cancer patients were randomly assigned to
two evenly balanced groups: an intervention arm (those who got Avastin along
with a standard treatment) and a control arm (those who got the latter and a
placebo). What’s more, the study was “double-blind” — neither the patients nor
the doctors knew who was in which group until after the results had been
assessed.
The
centerpiece of the country’s drug-testing system — the randomized, controlled
trial — had worked.
Except
in one respect: doctors had no more clarity after the trial about how to treat
brain cancer patients than they had before. Some patients did do better on the
drug, and indeed, doctors and patients insist that some who take Avastin
significantly beat the average. But the trial was unable to discover these
“responders” along the way, much less examine what might have accounted for the
difference. (Dr. Gilbert is working to figure that out now.)
Indeed,
even after some 400 completed clinical trials in various cancers, it’s not
clear why Avastin works (or doesn’t work) in any single patient. “Despite
looking at hundreds of potential predictive biomarkers, we do not currently
have a way to predict who is most likely to respond to Avastin and who is not,”
says a spokesperson for Genentech, a division of the Swiss pharmaceutical giant
Roche, which makes the drug.
That
we could be this uncertain about any medicine with $6 billion in annual global
sales — and after 16 years of human trials involving tens of thousands of
patients — is remarkable in itself. And yet this is the norm, not the
exception. We are just as confused about a host of other long-tested therapies:
neuroprotective
drugs for stroke, erythropoiesis-stimulating agents for anemia, the antiviral
drug Tamiflu
— and, as recent headlines
have shown, rosiglitazone (Avandia) for diabetes, a controversy that has now
embroiled a related class
of molecules. Which brings us to perhaps a more fundamental question, one
that few people really want to ask: do clinical trials even work? Or are the
diseases of individuals so particular that testing experimental medicines in
broad groups is doomed to create more frustration than knowledge?
Researchers
are coming to understand just how individualized human physiology and human
pathology really are. On a genetic level, the tumors in one person with
pancreatic cancer almost surely won’t be identical to those of any other. Even
in a more widespread condition like high cholesterol, the variability between
individuals can be great, meaning that any two patients may have starkly
different reactions to a drug.
That’s
one reason that, despite the rigorous monitoring of clinical trials, 16 novel
medicines were withdrawn from the market from 2000 through 2010, a figure equal
to 6 percent of the total approved during the period. The pharmacogenomics of
each of us — the way our genes influence our response to drugs — is unique.
HUMAN
drug trials are typically divided into three phases. In the first, researchers
evaluate the safety of a new experimental compound in a small number of people,
determining the best way to deliver it and the optimal dosage. In Phase 2,
investigators give the drug to a larger number of patients, continuing to
monitor its safety as they assess whether the agent works.
“Works”
in this stage is broadly defined. Seeing that the drug has any positive effect
at all — say, that it decreases the level of a blood marker associated with a
disease — is often enough to move a drug to Phase 3. Even so, most experimental
drugs fail before they get to Phase 3.
The
few that make it to Phase 3 are then tested for safety and efficacy in hundreds
or thousands of patients. This time, the outcomes for those taking the new drug
are typically compared head-to-head with outcomes for those getting a placebo
or the standard-of-care therapy. Generally, the Food and Drug Administration
requires that two “adequate and well-controlled” trials confirm that a drug is
safe and effective before it approves it for sale, though the bar can be lower
in the case of medicines aimed at life-threatening conditions.
Rigorous
statistical tests are done to make sure that the drug’s demonstrated benefit is
genuine, not the result of chance. But chance turns out to be a hard thing to
rule out. When the measured effects are small — as they are in the vast majority
of clinical trials — mere chance is often the difference between whether a drug
is deemed to work or not, says John P. A. Ioannidis,
a professor of medicine at Stanford.
In a
famous 2005 paper
published in The Journal of the American Medical Association, Dr. Ioannidis, an
authority on statistical analysis, examined nearly four dozen high-profile
trials that found a specific medical intervention to be effective. Of the 26
randomized, controlled studies that were followed up by larger trials
(examining the same therapy in a bigger pool of patients), the initial finding
was wholly contradicted in three cases (12 percent). And in another 6 cases (23
percent), the later trials found the benefit to be less than half of what was
first reported.
It
wasn’t the therapy that changed in each case, but rather the sample size. And
Dr. Ioannidis believes that if more rigorous, follow-up studies were actually
done, the refutation rate would be far higher.
Donald
A. Berry, a professor of biostatistics at M. D. Anderson, agrees. He, too, can
rattle off dozens of examples of this evaporation effect and has made a sport,
he says, of predicting it. The failures of the last 20 or so Phase 3 trials
testing drugs for Alzheimer’s disease, he says, could have been predicted based
on the lackluster results from Phase 2. Still, the payoff for a successful
Phase 3 trial can be so enormous that drug makers will often roll the dice —
not on the prospect that the therapy will suddenly work, but on the chance that
a trial will suggest that it does.
At a
round-table discussion a few years ago, focused on the high failure rate for
Alzheimer’s drugs, Dr. Berry was amazed to hear one drug company researcher
admit to such thinking out loud. The researcher said that when he and his team
designed the Phase 3 trial, he thought the drug would probably fail. But if
they could get an approval for a drug for Alzheimer’s disease, it would be “a
huge success.”
“What
he was saying,” marvels Dr. Berry, “was, ‘We’re playing the lottery.’ ”
The
fact that the pharmaceutical companies sponsor and run the bulk of
investigative drug trials brings what Dr. Ioannidis calls a “constellation of
biases” to the process. Too often, he says, trials are against “a straw-man
comparator” like a placebo rather than a competing drug. So the studies don’t
really help us understand which treatments for a disease work best.
But
a more fundamental challenge has to do with the nature of clinical trials
themselves. “When you do any kind of trial, you’re really trying to answer a
question about truth in the universe,” says Hal Barron, the chief medical
officer and head of global development at Roche and Genentech. “And, of course, we can’t know that.
So we try to design an experiment on a subpopulation of the world that we think
is generalizable to the overall universe” — that is, to the patients who will
use the drug.
That’s
a very hard thing to pull off. The rules that govern study enrollment end up
creating trial populations that invariably are much younger, have fewer health
complications and have been exposed to far less medical treatment than those
who are likely to use the drug.
Roughly
53 percent of new cancer diagnoses, for example, are in people 65 or older, but
this age group accounts for just 33 percent of participants in cancer drug
trials.
Even
if clinical researchers could match the demographics of study populations to
those of the likely users of these medicines, no group of trial volunteers
could ever match the extraordinary biological diversity of the drugs’ eventual
consumers.
Drug
makers are well aware of the challenge. “Listen, it’s not lost on anybody that
about 95 percent of drugs that enter clinical testing fail to ever get
approved,” says Dr. Barron. “It’s not hard to imagine that at least some of
those might have failed because they work very, very well in a small group. We
can’t continue to have failures due to a lack of appreciation of this
heterogeneity in diseases.”
So
what’s the solution? For subtypes of disease that are already known, it may be
feasible to design small clinical trials and enroll only those who have the
appropriate genetic or molecular signature. That’s what Genentech did in
developing the breast cancer drug Herceptin, which homes in on tumor cells that
have an abundance of a protein called HER2.
And
that’s the strategy the company says it’s pursuing now. Sixty percent of the
new drugs in the works at Genentech/Roche are being developed with a companion
diagnostic test to identify the patients who are most likely to benefit.
But
given the dismal success rate for drug development, this piecemeal approach is
bound to be slow and arduous. Rather than try to fit patients, a handful at a
time, into the decades-old clinical-trials framework, we’d be far better off
changing the trials themselves.
In
fact, a breast cancer trial called I-SPY 2, already under way, may be a good
model to follow. The aim of the trial, sponsored by the Biomarkers Consortium,
a partnership that includes the Foundation for the National Institutes of
Health, the F.D.A., and others, is to figure out whether neoadjuvant therapy
for breast cancer — administering drugs before a tumor is surgically removed —
reduces recurrence of the disease, and if so, which drugs work best.
As
with the Herceptin model, patients are being matched with experimental
medicines that are designed to target a particular molecular subtype of breast
cancer. But unlike in other trials, I-SPY 2 investigators, including Dr. Berry,
are testing up to a dozen drugs from multiple companies, phasing out those that
don’t appear to be working and subbing in others, without stopping the study.
Part
of the novelty lies in a statistical technique called Bayesian analysis that
lets doctors quickly glean information about which therapies are working best.
There’s no certainty in the assessment, but doctors get to learn during the process
and then incorporate that knowledge into the ongoing trial.
Mark
Gilbert, for his part, would even settle for something simpler in his next
glioblastoma study. His definition of a successful clinical trial? “At the end
of the day,” he says, “regardless of the result, you’ve learned something.”
Clifton
Leaf is the author
of “The Truth in Small Doses: Why We’re Losing the War on Cancer — and How to
Win It.”
