WHEN GOOD GUIDELINES GO BAD Henry Green, MD, FACC, FACP May 29, 2009 A hair perhaps divides the false from true… -- Omar Khayyam Introduction Clinical practice guidelines are designed to assist physicians with patient management decisions. While failure to follow the guidelines adequately is of obvious concern, they are not meant to be embraced blindly and rigidly. Treatment should be individualized. Ideally guidelines should be based on documentation rather than prevailing convictions. In the real world hard evidence often does not exist. We are then forced to rely on contemporaneous expert opinion.1 Both the strength of recommendations and the level of evidence are included in many major guidelines. They are typically graded as follows: Strength of recommendations: Class I: there is evidence and/or general agreement that a given procedure or treatment is useful and effective Class II: there is conflicting evidence and/or a divergence of opinion about the usefulness/efficacy of a procedure or treatment Class IIa: weight of evidence or opinion is in favor of usefulness and efficacy Class IIb: usefulness and efficacy is less well established by evidence or opinion Class III: there is evidence and/or general agreement that the procedure is not useful or effective and in some cases may be harmful. Levels of evidence: Level of evidence A: based on evidence from multiple randomized trials or meta-analyses Level of evidence B: based on evidence from a single randomized trial or nonrandomized studies Level of evidence C: based on expert opinion, case studies, or standards of care. Most guidelines are based on level C evidence. About 35% are level B. Only 10% are level A. Randomized controlled trials Development2 Phase I trials involve a small number of healthy volunteers. Their purpose is to study the pharmacology, metabolism and side effects of a drug or treatment. Phase II trials are controlled clinical trials, typically including several hundred patients. This is done to determine the efficacy of the treatment and to disclose common side effects.

1

Phase III trials involve several hundred patients over a period of 6-18 months, to assess the longer term safety of the treatment. Phase IV trials are performed after the treatment has already been licensed. This is done in a health care setting with a variety of representative patient types. The purpose is to determine the long-term risks and benefits. Patient selection Controlled studies are usually conducted using relatively “pristine” patients. That is, the patients must have the condition under investigation, but certain specific comorbidities (such as renal disease) are often excluded. Typically selection is limited to a specific age range. Often women and some races are underrepresented. Control groups An attempt is made to provide a control group with characteristics similar to those of the treated patients. This is not always entirely successful, even though it may initially seem to be. In addition, to use a placebo as a comparator is often unethical. For example, it would be wrong to deprive control patients with coronary artery disease of a statin or a betablocker. Outcomes Hard endpoints Ideally the effectiveness of a treatment should be measured in terms of clinically significant outcomes. Examples might be survival, regression of disease or improvement of symptoms. Surrogate endpoints3 A surrogate marker is a substitute for a clinically relevant endpoint. It is assumed that the surrogate will reflect changes in the outcome. For example, raising HDL ought to be associated with an improvement atherosclerosis. Similarly, the rate of change in carotid intimal medial thickness should correlate with the severity of coronary atherosclerosis. Waiting for hard endpoints, such as death, myocardial infarction or need for coronary revascularization takes a long time. Using a surrogate brings the trial to a conclusion more quickly and at less expense. Unfortunately this does not always work. The surrogate may operate through a different pathway than the disease process. Even if the treatment improves the surrogate, it may not actually benefit the disease. Conversely, the treatment may work but the surrogate may fail to show this. Subgroup analysis Authors frequently analyze selected categories of patients from within their study population. For example, they may find that an intervention yields a more impressive result in subjects within a certain age range than it does in the overall group. These findings are often less statistically robust than the main trial results. They should be applied with caution.

2

Meta-analyses A meta-analysis combines data from multiple studies in an attempt to eliminate chance variability. In doing so, borderline outcomes may be shown to be significant. It may be possible to draw new conclusions. While these documents can be valuable, there are limitations. The search for relevant literature may be unwittingly biased, as may be the choice of studies considered worthy of inclusion. Furthermore, not all references are equally reliable. Some may be non-randomized trials. Older studies often did not utilize what is now state of the art treatment. Integration of data is also handicapped because individual papers may have used different patient inclusion criteria. They also define outcomes differently. For example, in investigating the relative safety of various antithrombotic regimens “significant blood loss” is variously defined by different workers (need for transfusion, hypotension, change in hemoglobin concentration, etc). Moreover, unpublished data are rarely if ever included. Sponsorship and funding Most trials are paid for by manufacturers. Their zeal to bring a product to market has occasionally resulted in cynical indifference to scientific authenticity. There have been cases in which the sponsor controlled what questions may be asked, the methodology used, how the data is analyzed, and even whether an unfavorable result may be published.4 Publication bias Because of space limitations, editors must reject many manuscripts. This is a difficult and sometimes arbitrary process; some valuable research never comes to light. Negative trials are seldom published. They are relevant, but are less interesting than those with positive outcomes. Life cycle of a guideline If guidelines were irrefutable they would never require updating. Medical and surgical treatments are continually changing, even as the guidelines are being written. Newer evidence comes to light and conclusions must be altered. It has been shown that the mean time to a signal of error is 5.5 years, and in 7% of guidelines, a signal of error has already appeared by the time of publication. The “half-life” of a medical guideline appears to be about 5 years. In the field of cardiology it is two years.5 Some examples CAST trial Post-myocardial infarction patients who have ventricular arrhythmias suffer a fourfold greater mortality than those who do not. It was therefore believed that suppressing these arrhythmias would lower the death rate. In 1989, the Cardiac Arrhythmia Suppression Trial was conducted to test this hypothesis. To everyone’s disappointment, antiarrhythmic drugs actually increased the mortality. The CAST II trial, using a different drug, reached the same conclusion. 6,7 Hormone replacement therapy In 1997, Pines et al found that postmenopausal women who were taking estrogen had a lower incidence of adverse cardiac events.8 By 2001, a paper was presented at the annual 3

meeting of the American Heart Association corroborating this. The speaker concluded that such treatment was the standard of care and that “to practice otherwise may be considered malpractice.” In 2002, a new trial from the Women’s Health Initiative reversed this recommendation, stating that hormone supplementation could increase the risk of myocardial infarction.9 The problem with the earlier work appears to have been the “healthy cohort effect.” Women who took hormones were also caring for their health in other ways. They were less likely to smoke, exercised more, and received better medical care. Preoperative beta blocker therapy The 2007 ACC/AHA guidelines for perioperative management of cardiac patients recommended the administration of carefully adjusted doses of beta-blockers to certain higher risk patients. This had been shown to reduce the incidence of myocardial infarctions.10 The following year, the POISE trial11 found that, while this was true, there was a higher incidence of stroke and death when beta-blocker therapy was employed. A flurry of editorial comment, sometimes scathing, followed. One major criticism of the POISE trial was the methodology employed. In contrast to the recommendations in the guidelines, a large dose of metoprolol was given close to the time of surgery. A more balanced view is to use beta-blockers selectively. Patients with comorbidities that predispose to hypotension should be excluded. One should also titrate the dose of betablocker slowly over a period of weeks prior to the surgery. 12 Any retrospective study of the medical literature will reveal innumerable similar examples in virtually all fields of medicine. How guidelines are used In spite of their limitations, guidelines summarize the best of our current knowledge. They are used for clinical decision-making, approval of drugs and devices, generation of health care policy, reimbursement, performance measured, hospital accreditation and, unfortunately, litigation. Your patients are not like the trial patients Ideally, one should only apply a therapy to those patients that match the subjects of the trials on which guidelines are based. We should also apply the treatment in exactly the same way. However, in real-world practice this is not possible. The age range of our patients is not restricted, and they usually have comorbidities. Polypharmacy is the rule. “Real patients” are less compliant than those in trials. Furthermore, we follow patients for decades, not just a couple of years. In that span of time, many unexpected things can happen. 13 Where do we go from here? While the guidelines are usually very helpful in patient management, there are many times when thoughtful dissent is called for. It may be wrong to administer a suggested treatment to a particular patient. One must be aware of drug interactions, side-effects and contraindications. Drug dosage usually must be individualized. Not all patients can handle the recommended 4

amount of a drug, and some may not tolerate it at all. Moreover not every patient has a given disease to the same degree. Comorbidities must be taken into account as well. Individualize patient management. Use the guidelines to enhance your judgment, not to replace it. Be sure to document your reasoning in the chart. Above all, keep up with the literature.14 References 1. Tricoci P et al. Scientific evidence underlying the ACC/AHA clinical practice guidelines. JAMA 2009;301:831-841 2. Code of Federal Regulations, Title 21, Volume 5, Revised April1, 2007 3. Fleming TR and DeMets DL. Surrogate end points in clinical trials: are we being misled? Ann Intern Med 1996; 125:605-613 4. Bodenheimer T. Uneasy alliance -- Clinical investigators and the pharmaceutical industry N Engl J Med 2000;342:1539-1544 5. Shojania KG et al. How quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med 2007;147:224-233. 6. The Cardiac Arrhythmia Suppression Trial (CAST) Investigators. Preliminary report: Effect of encainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. N Engl J Med 1989 Aug 10 321 406-412. 7. Ibid. N Engl J Med, 1992; 327:227-33 8. Pines A et al. Hormone replacement therapy and cardioprotection: basic concepts and clinical considerations. Eur J Obst Gyn and Reproductive Biol. 1997;71:193-197 9. Writing Group for the Women's Health Initiative Investigators. Risks and Benefits of Estrogen Plus Progestin in Healthy Postmenopausal Women: Principal Results From the Women's Health Initiative Randomized Controlled Trial JAMA 2002; 288: 321 - 333. 10. ACC/AHA 2007 Guidelines on Perioperative Cardiovascular Evaluation and Care for Noncardiac Surgery: Executive Summary. J Am Coll Cardiol 2007; 50:1707-1732; 11. POISE Study Group. Effects of extended-release metoprolol succinate in patients undergoing non-cardiac surgery (POISE trial): a randomised controlled trial. The Lancet 2008. 371:1839 -1847 12. Harte B and Jaffir AK. Perioperative beta-blockers in noncardiac surgery: evolution of the evidence. Cleveland Clinic Jour Med 2008;75:513-519 13. Tinetti ME et al. Potential pitfalls of disease-specific guidelines for patients with multiple conditions. N Engl J Med 2004; 351;2870-2874 14. Demaria AN. Clinical trials and clinical judgment. J Am Coll Cardiol 2008; 51:1120-1122

5