- Residency and Fellowship Programs
- Resident and Fellow Information
- Program Staff Information
SIU SOM Related
User's Guide to the Medical Literature MCXXVIII:How to Read Pharmaceutical Advertisements.
The Lack of Evidence Based Medicine Working Group (in no way associated with the Evidence Based Medicine Working Group)
"The truth, the half-truth, and nothing like the truth." -- Andrew Herxheimer
It is a busy day in your practice and you are sitting at your desk, legs up, leafing through a recent issue of "Physicians at Leisure." You come across and ad for PlavixTM, which indicates that this medication reduces the risk of cardiovascular events by 8% compared to aspirin. You wonder if you should be switching all your patients to PlavixTM.
Relative Risk Reduction ad absurdum: Are the risk reductions shown relative risk reductions ("RRR") or absolute risk reductions ("ARR")?
Pharmaceutical advertisements almost without exception use relative risk reductions rather than absolute risk reductions, as the RRR is almost always the much larger of the two.1 Both are of course valid measures of effect. However, the ARR is more relevant to individual patients, and it is from this that the "number needed to treat (NNT)" is calculated.2 For example, a decrease in risk from 3% to 2% is a 33% relative risk reduction, but only a 1% absolute risk reduction. You are not likely to see an ad that states "Prilozac decreased your risk of MI by 1%!" An ad referring to the SSSS trial (renamed the Zocor Survival Study in the ad!) points out the 42% RRR in coronary mortality in this study. The absolute risk reduction was 3.5%. Studies have demonstrated that physicians are more likely to prescribe when results are presented as RRRs than ARRs.3
Is the result statistically significant?
Look for p-values, and you may need to get out your bifocals. Look for exact values, and when these are missing assume the value is just less than the value you are given (e.g., p<.05 is probably p=.049). Confidence intervals have yet to make their appearance in the pharmaceutical literature.
Is the result clinically significant?
A result can be statistically significant but be clinically insignificant either because it is very small or because it is not a clinically important outcome. A recent ad for alendronate is instructive: On the left hand side of the page are very small (5-10%) relative increases in bone mineral density among subjects on alendronate. P-values of <0.001 are displayed for each of the three endpoints.
However, p-values are missing on the right hand side of the page, where large risk reductions for vertebral fractures (35-63%) are given for patients on alendronate. These results were in fact not statistically significant. The clinically significant endpoints are those on the right hand side of the page, which are alas lacking in statistical significance. The clinically insignificant endpoints (small and at least less clinically important) are the highly statistically significant endpoints on the left.
Are the graphs telling the truth?
Does the size of the effect shown equal the size of the effect in the data? Tufte's "Lie Factor" = (Size of effect shown in graphic) / (Size of effect in data). Lie Factors >1.05 or <0.95 represent substantial distortion. A recent quinapril ad inexplicably used different sized cones to represent exercise time in patients on varying doses of medication as well as on placebo. A cone twice as high as another (representing a doubling of exercise time), has a volume 4.4X that of the other. The lie factor in this case is 2.2 (4.4/2). Interestingly, in the same ad, when adverse events were compared, triangles were used rather than cones!
Is only a small percentage of the possible event rate displayed?
Often times the y-axes are only slightly higher than the event rate of one of the groups, which will tend to magnify things, while making it appear that one of the groups had the highest rate "possible." For example, in an add for pravastatin, referring to the West of Scotland Study, the y-axis ends at 8%. The rate of MI in the placebo group was 7.9%. The 2.4% difference between the treatment and placebo group is thus magnified. When portraying side-effects, however, advertisers usually demonstrate that they know y-axes can reach 100%, thus making microscopic a few percent incidence of adverse events.
Does the y-axis start at zero?
The y-axis should always begin at zero. If this is not so, someone is trying to make you believe that one of the groups has reached the lowest rate or number possible when this is not the case.
Does the graph extend beyond the study period?
Though survival curves in orignal reports usually indicate the number of eligible patients in each study year from time of randomization, ads rarely do. In another simvistatin ad, a survival curve going out to six years is portrayed. The numbers of eligible patients for each year since randomization is not displayed (as it is in the orginal article), and in fact, only a very small number of stragglers were followed (228 out of 4444) for six years. The difference at six years in the ad clearly appears larger than that at five.
And watch out for:
- Inconsistent time intervals on the x-axis, which can make increases or decreases look more precipitous;
- Graphs that are taller than they are wide, which exagerate increases or decreases;
- Graphs that use areas (or volumes!) to portray one-dimensional data; the number of dimensions depicted should not exceed the number of dimensions in the data.
Are references "real?"
Chances are, if the reference is to "Data on file" you are never going to see them.4
Is Cal Ripken in the ad?
It is now widely known that Cal is neither hypertensive nor taking lisinopril.5 Nor is he on the job everyday.
Resolution of the Scenario
The 8.7% risk reduction given is of course a relative risk. The absolute risk reduction was a whopping 0.9%, the NNT therefore = 1/.009 = 111, 95%CI 57 to 2,500. The result was marginally statistically significant, p=.045 (95% confidence interval on the RRR 0.3% to 16.5%, not given), though the clinical significance of this difference is questionable. The 8.7% reduction for Plavix vs aspirin is portrayed in an area approximately the same as the 25% reduction for aspirin vs placebo, in violation of Tufte's rule. This figure also suggests that Plavix and placebo have been compared head to head, when in fact, as the small print indicates, they have not.
The reference is to a real study, the CAPRIE trial published in the Lancet in 1996, and though you are also reassured by the abscence of Cal Ripken anywhere in the ad, you nevertheless decide to save your patients the $112.00/month cost of Plavix and keep them on aspirin.
- Lexchin, J. How patient outcomes are reported in drug advertisements: Review of Canadian medical journals. Can Fam Phys. 1999; 45; 1213-16.
- Chatellier, G, et al. The number needed to treat: A clinically useful nomogram in its proper context. BMJ. 1196; 312; 426-29.
- Bobbio M, Demichelis B, Giustetto G. Completeness of reporting trial results: Effect on physicians' willingness to prescribe. Lancet. 1994; 343:1209-11.
- Herxheimer, A. Data on file cited in pharmaceutical advertisements: What are they? Presented at International Congress on Biomedical Peer Review and Global Communications, Sept. 1997 (abstract).
- Jefferson BK. Cal Ripken, Jr., and Prinivil [letter]. New England Journal of Medicine. 339(27):2024-5.
Copyright ©1999-2001. No Free Lunch