RAND Review: Spring 2003: Preponderance of Evidence: Judging What to Do About Ephedra
Preponderance of Evidence
Judging What to Do About Ephedra
By Paul G. Shekelle, Margaret Maglione, and Sally C. Morton
Paul Shekelle, a RAND and Veterans Affairs physician, is director of the Southern California Evidence-Based Practice Center. Margaret Maglione is a RAND policy analyst. Sally Morton holds the RAND Endowed Chair in Statistics.
Since the 1980s, the herb ephedra has been purported to increase weight loss and to enhance athletic performance. In recent years, though, a string of athletes and other individuals who apparently had ingested the herb have collapsed and, in some cases, died. In the wake of the death of Baltimore Orioles pitcher Steve Bechler this spring, the voices of government officials and the public at large have grown increasingly louder in demanding proof of the safety and efficacy of the herb.
Unfortunately, scientific proof has become exceedingly difficult to attain. As a result of the 1994 Dietary Supplement Health and Education Act (DSHEA), substances that are classified as "dietary supplements" (including herbs such as ephedra) are not considered to be "drugs" and thus are not subject to the same rigorous regulatory standards as drugs are. According to the legislation, manufacturers of dietary supplements need not show evidence of the efficacy or safety of the products prior to marketing them. Therefore, the usual regulatory pressure to produce clinical studies assessing efficacy and risk does not exist. Consequently, we could find only limited scientific data about ephedra.
However, we did find sufficient evidence of danger associated with ephedra, along with insufficient evidence of its efficacy for athletic performance, to compel the U.S. Department of Health and Human Services and the U.S. Food and Drug Administration (FDA) to take preliminary regulatory measures against the herb. On Feb. 28, 2003, the FDA proposed a strong new warning label on ephedra products, warned manufacturers against making unsubstantiated claims that the products could enhance athletic performance, and invited public comment on the risks associated with ephedra to support new restrictions on the products.
The FDA is now seeking public comment to determine whether dietary supplements containing ephedra present a "significant or unreasonable risk of illness or injury." This is the standard that must be met under DSHEA before the FDA can take further regulatory action. This standard also reveals an indirect effect of the DSHEA law. By presuming that dietary supplements are safe and absolving manufacturers from proving that the supplements are safe, the law places the burden of proof on the FDA instead. The agency must somehow prove that a supplement is risky, even in the absence of clinical studies conducted by manufacturers.
Our study could not prove with scientific certainty that ephedra is unsafe. However, we compiled enough evidence to reach fairly confident conclusions. Our efforts could serve as an example of how policymakers and researchers can help to keep the public safe despite the absence of incontrovertible scientific proof of danger.
AP/WIDE WORLD PHOTOS/TERU IWASAKI
AP/WIDE WORLD PHOTOS/GREG WAHL-STEPHENS
AP WIDE WORLD PHOTOS/RANDY SQUIRES
|Food and Drug Administration Commissioner Mark McClellan, foreground, with Health and Human Services Secretary Tommy Thompson, tells reporters on Feb. 28 that bottles of the popular dietary supplement ephedra should bear warning labels that the pills can cause heart attacks, strokes, or even death.||Pat Bechler, mother of Baltimore Orioles pitcher Steve Bechler, addresses a packed auditorium during a March 8 memorial service in Medford, Ore., for the 23-year-old pitching prospect, who died of heatstroke on Feb. 17. A Florida medical examiner linked the death to ephedra. Steve's older brother Mike is at left.||Kevin Riggins—the father of Sean Riggins, a 16-year-old high school football player who died last fall after taking ephedra—speaks at the state capitol building in Springfield, Ill., on March 12. On March 20, the Illinois state senate voted unanimously to ban the sale of ephedra products.|
The active stimulant in ephedra is called ephedrine, which is found in over-the-counter drugs used to treat stuffy nose and asthma. The difference between the herb ephedra and the drug ephedrine is analogous to the difference between coffee beans and caffeine. Ephedra, known as "ma huang" among Chinese herbalists, is a shrub. Ephedrine is the active stimulant found in the shrub.
To determine if either substance could improve weight loss or enhance athletic performance, we searched the medical literature as well as other sources for published and unpublished clinical trials of the substances. We based our conclusions about efficacy on a detailed review of 52 trials of ephedra or ephedrine for weight loss or athletic performance.
Many of the 52 trials involved only small numbers of people, covered only short periods of time, or suffered from other limitations. For example, all of the trials for athletic performance involved just a couple of dozen young fit males, who were not representative of the general population. Even in aggregate, the 52 trials offered only weak evidence for assessing the relationship between rare adverse events and the use of ephedra or ephedrine.
To determine with greater confidence if it is safe to take ephedra or ephedrine, we analyzed nearly 18,000 case reports of adverse events. Consumers had contacted the FDA to provide the agency with 1,820 "adverse event reports," the vast majority of which dealt with ephedra rather than ephedrine. The FDA shared these reports with us. We discovered 71 additional reports in the medical literature. The largest repository of reports came from a computer file of 15,951 cases reported to Metabolife, a San Diego-based maker of ephedra-containing dietary supplements.
Except in extraordinary circumstances, case reports cannot be considered to be conclusive evidence of a cause-and-effect relationship. However, case reports can be useful to establish the potential for a causal relationship.
We found some evidence of the benefits of ephedra and ephedrine for weight loss. Dietary supplements containing ephedra alone, ephedrine alone, ephedra with herbs containing caffeine, or ephedrine plus caffeine promoted modest short-term weight loss of about two pounds per month more than among people taking a placebo. But none of the available studies followed participants for longer than six months. A study of 12 months is generally accepted as necessary to establish a drug's value as a weight-loss aid.
Of the 52 clinical trials, 44 assessed ephedra, ephedrine, or ephedrine plus other compounds used for weight loss. Of these 44 trials, we excluded 18 because the duration of treatment was less than eight weeks. We excluded six other trials for a variety of reasons.
In the remaining 20 trials, we found comparisons made in six categories. We highlight the results below and in Figure 1.
- Ephedrine versus placebo. In five trials, the average weight loss for a person treated with ephedrine was 1.3 pounds per month more than the average weight loss for a person treated with a placebo.
- Ephedrine and caffeine versus placebo. In 12 trials, the average weight loss for a person treated with ephedrine and caffeine was 2.2 pounds per month more than the average weight loss for a person treated with a placebo.
- Ephedrine and caffeine versus ephedrine alone. In three trials, the average weight loss for a person treated with ephedrine and caffeine was 0.8 pounds per month more than the average weight loss for a person treated with ephedrine alone.
- Ephedrine and caffeine versus another active pharmaceutical for weight loss. In two trials, we found no statistically significant difference in weight loss. One trial compared the combination of ephedrine and caffeine to dexfenfluramine. The other trial compared the same combination to diethylpropion.
- Ephedra versus placebo. In a single trial, the average weight loss for a person treated with ephedra was 1.8 pounds per month more than the average weight loss for a person treated with a placebo.
- Ephedra with herbs containing caffeine versus placebo. In four trials, the average weight loss for a person treated with this ephedra mixture was 2.1 pounds per month more than the average weight loss for a person treated with a placebo.
In Figure 1, we show the average increase in weight loss in each case as a black square and give its value. We also include a vertical line that illustrates the 95 percent confidence interval around the average value. For example, the average additional weight loss for a person treated with ephedrine alone was 1.3 pounds per month. We are 95 percent confident that the true additional average weight loss would fall somewhere between 0.4 and 2.2 pounds per month.
We found no trials of ephedra on athletic performance—and thus no evidence that ephedra could enhance athletic performance. We found only minimal evidence that ephedrine could enhance athletic performance. Even here, there appeared to be no athletic benefit from ephedrine beyond an immediate boost.
There were eight trials of ephedrine on athletic performance. All but one included caffeine. Each trial involved different types of exercise and different outcome measures, so we analyzed each trial individually.
Six of the eight trials assessed the exercise capacity of small groups of healthy males. Each trial included 24 or fewer subjects. The trials concluded that neither caffeine nor ephedrine alone had significant effects on various parameters of exercise performance, such as oxygen consumption, time to exhaustion, or carbon dioxide production. However, the combination of ephedrine and caffeine consistently demonstrated a 20-30 percent increase in short-term performance.
One trial of strength training showed an improvement in muscle endurance—but only on the first of three repetitions. The remaining trial reported no statistically significant improvement in a battery of tests of physical function, including oxygen uptake, measures of endurance and power, reaction time, hand-eye coordination, speed, and self-perceived exertion.
The clinical trials of ephedra and ephedrine reported numerous adverse side effects. We grouped the symptoms into clinically similar categories, as follows:
- psychiatric symptoms: those described in the trials as euphoria, neurotic behavior, agitation, neuropsychiatric symptoms, depressed mood, giddiness, irritability, or anxiety
- autonomic hyperactivity: those symptoms described as tremor, twitching, jitteriness, insomnia, difficulty sleeping, increased perspiration, or sweating
- palpitations: those symptoms described as palpitations, irregular heartbeat, loud heartbeat, heart pounding, or increased or stronger heartbeat
- hypertension: those symptoms described as hypertension, increased systolic blood pressure, or increased diastolic blood pressure
- upper gastrointestinal symptoms: those described as nausea, vomiting, abdominal pain, upset stomach, heartburn, or gastroesophageal reflux
- tachycardia: those symptoms described as tachycardia, or slightly elevated heart rate.
Figure 2 shows our estimates of the increased odds of suffering these adverse events when taking ephedra or ephedrine. Once again, we show each estimate as a black square and give its value. We also include a vertical line that illustrates the 95-percent confidence interval around the estimated value. For example, we estimate that the odds of a person suffering psychiatric symptoms were 3.6 times higher if the person took ephedra or ephedrine. We are 95 percent confident that the true value of the increased odds of suffering psychiatric symptoms would fall somewhere between 1.9 and 7.3.
Overall, people who received ephedra or ephedrine had between 2.2 and 3.6 times higher odds of suffering harmful side effects—including psychiatric symptoms, jitteriness, palpitations, nausea, and vomiting—than did people taking a placebo. There appeared to be a similar increase in the incidence of hypertension, but the increase was not statistically significant. There was also a trend toward a higher risk of adverse events with higher doses of ephedrine, but the data were sparse, and, once again, the differences were not statistically significant.
We could not estimate the increased odds of experiencing tachycardia because there were zero cases of tachycardia reported among people taking a placebo. In comparison, there were six cases of tachycardia among people taking ephedra or ephedrine.
It was impossible to estimate the degree to which caffeine contributed to the results, because there were too few trials of ephedrine alone or ephedra alone to isolate the role of caffeine. However, the one trial of ephedra without herbs containing caffeine reported a statistically significant twofold increase in gastrointestinal symptoms.
The additional evidence we gathered from the case reports raised even greater concerns about consumer safety. We screened the nearly 18,000 case reports and then reviewed in detail 284 reports of either death, heart attack, other cardiac symptoms, strokes, neurologic symptoms, seizures, or serious psychiatric symptoms.
We searched each of the 284 reports to assess whether ephedra or ephedrine was, indeed, a likely cause of the adverse event. We judged each case report by the following three criteria:
1. Documentation that an adverse event had occurred.
2. Either documentation that the subject had consumed ephedra or ephedrine within 24 hours prior to the adverse event or a toxicological examination revealing the presence of ephedrine or an associated product in the blood or urine. (For example, we did not require psychiatric symptoms to become manifest within 24 hours of using ephedra or ephedrine.)
3. Documentation that an adequate investigation had excluded other potential causes.
Cases meeting all three criteria were labeled "sentinel events." Cases meeting the first two criteria but having other possible causes were labeled "possible sentinel events." We solicited the judgment of expert clinicians to assess whether causes other than ephedra or ephedrine had been adequately evaluated and excluded.
From the 284 reports of serious adverse events, we identified two deaths, three heart attacks, nine strokes, three seizures, and five psychiatric cases as sentinel events with prior ephedra consumption. We identified three deaths, two heart attacks, two strokes, one seizure, and three psychiatric cases as sentinel events with prior ephedrine consumption. About half of the sentinel events occurred in people aged 30 years or younger. We identified 43 additional cases as possible sentinel events with prior ephedra consumption and 8 additional cases as possible sentinel events with prior ephedrine consumption.
In aggregate, the case reports suggest a link between products containing either ephedra or ephedrine and catastrophic events, such as sudden death, heart attack, stroke, seizures, and serious psychiatric symptoms.
Regarding weight loss, we found enough evidence to conclude that the short-term use of either ephedrine alone, ephedrine and caffeine combined, ephedra alone, or ephedra with herbs containing caffeine all promote weight loss in selected patient populations. However, all but three of the trials lasted for less than six months. Ideally, the trials should assess not only the results of a full year of treatment but also what happens after the treatment is discontinued.
Caffeine clearly adds additional efficacy to ephedrine in promoting weight loss. The effects of ephedrine and caffeine together are roughly equal to the effects of ephedra with or without herbs containing caffeine. Each results in about two pounds per month of weight loss over four months.
To put these pounds in context, though, competing FDA-approved weight loss drugs have been shown to be about equally as effective. The drugs sibutramine (Meridia) and orlistat (Xenical) have both resulted in average weight loss of 6-10 pounds over 6-12 months, and the drug phentermine (often used in combination with fenfluramine as "phen-fen") has resulted in average weight loss of 16 pounds over 9 months.
AP/WIDE WORLD PHOTOS/ED BAILEY
|Bottles of Ripped Fuel Metabolic Enhancer, which contains ephedra, are shown in New York on June 18, 2002. A bottle of Ripped Fuel was found in the locker of Minnesota Vikings tackle Korey Stringer after he collapsed and died during a training camp practice on Aug. 1, 2001.|
Regarding athletic performance, the few trials of ephedrine that we identified did not study the drug as used by the general population—that is, repeated use. Therefore, the effect of ephedra or ephedrine to enhance athletic performance over the long term is completely unknown.
Regarding safety, we conclude from the clinical trials that ephedrine and ephedra are associated with two to three times the odds of experiencing psychiatric symptoms, autonomic symptoms, upper gastrointestinal symptoms, and palpitations. It is not possible to separate out the effect that caffeine may contribute to these events.
We conclude from the case reports of ephedra and ephedrine that serious adverse events have occurred in young adults without other apparent causes. There may be a causal relationship between taking the substances and suffering rare serious adverse events. Catastrophic effects of ephedra, including death, cannot be ruled out at a rate of less than one person per thousand.
Our study has several limitations. As we note above, many of the clinical trials themselves had design limitations, and all of the weight loss trials were of short duration. In addition, the results of the weight loss trials may understate the dangers for the general population. These trials frequently involved medical screening to exclude people with preexisting conditions, such as heart disease, that could have predisposed the people to increased risks. It is unknown whether administering ephedra or ephedrine without such screening would increase the risks.
Despite these limitations, we found sufficient evidence to conclude that dietary supplements containing ephedra or ephedrine are associated with a modest increase in weight loss in the short term—but also an increase in a variety of serious health risks. We hope that our efforts in compiling medical evidence, even when it cannot be gleaned from clinical trials alone, can help others in the field find ways to reach similarly useful conclusions.
"Efficacy and Safety of Ephedra and Ephedrine for Weight Loss and Athletic Performance: A Meta-Analysis," Journal of the American Medical Association, Vol. 289, No. 12, March 26, 2003, pp. 1537-1545, Paul G. Shekelle, Mary L. Hardy, Sally C. Morton, Margaret Maglione, Walter A. Mojica, Marika J. Suttorp, Shannon L. Rhodes, Lara Jungvig, James Gagné.
Ephedra and Ephedrine for Weight Loss and Athletic Performance Enhancement: Clinical Efficacy and Side Effects, Paul Shekelle, Sally Morton, Margaret Maglione, et al., Evidence Report/Technology Assessment No. 76, Rockville, MD: Agency for Healthcare Research and Quality, February 2003, http://www.fda.gov/OHRMS/DOCKETS/98fr/95n-0304-bkg0003-ref-07-01-index.htm.