STATISTICS AND RESEARCH DESIGN

The evidence pyramid and introduction to randomized controlled trials Nikolaos Pandis, Associate Editor of Statistics and Research Design Corfu, Greece

B

iomedical scientists seek to develop interventions such as drugs to fight diseases, to invent devices that improve patients’ lives, and to develop noninvasive or invasive procedures to cure or improve medical or dental problems. However, it is important that the proposed interventions are tested in a systematic and transparent fashion based on scientific and ethical principles that ensure validity of results and patient safety. The development of rules and regulations for evaluation of interventions was motivated by past mistakes such as the thalidomide crisis that caused great harm to several thousands of people.1 The pyramid of evidence (Fig) includes several types of studies used to evaluate treatment effects, starting from in-vitro and animal studies at the lower level and going up to opinions, case reports, observational studies, and randomized controlled trials (RCTs).2 At the tip of the pyramid are systematic reviews, which constitute the highest level of evidence because they attempt to collect, combine, and report the best available evidence using systematic, transparent, and reproducible methodology. The pyramid of evidence has helped in assessing the quality of evidence and has been pivotal in translating the available evidence into clinical practice. Lower-level evidence is not necessarily false; in fact, lower-level studies have resulted in important breakthroughs, such as the discovery of penicillin. However, lower-level studies carry a greater risk of “false-positive” results and thus have a higher chance of leading to recommendations that are not in patients’ best interests. In the context of evidence-based dentistry, the position of a study design on the pyramid of evidence does not necessarily indicate the validity of the results but, rather, the priority it is given in decision making for treatment recommendations. The first theme of this series deals with clinical trials: specifically, RCTs. A clinical trial is a preplanned experiment that aims to assess the effects or benefits of at least

Am J Orthod Dentofacial Orthop 2011;140:446-7 0889-5406/$36.00 Copyright Ó 2011 by the American Association of Orthodontists. doi:10.1016/j.ajodo.2011.04.016

446

1 treatment in humans. An RCT uses a control group and randomization to assign participants to treatment arms, and aims to create similar treatment groups with regard to known and unknown baseline characteristics: in other words, to create groups that differ only by the intervention they receive.3 The use of a control group is important because (1) trial participants could naturally get better with time, (2) a patient who might respond better could be preferentially selected (selection bias), and (3) trial participants could respond better just because they are included in the study through some sort of behavioral modification. An RCT aims at producing valid and precise estimates of treatment effects while keeping participants safe and not necessarily producing statistically significant results. RCT results can be either true or false because of random error, bias, or confounding; therefore, every available and feasible methodologic effort should be taken to minimize the chance of obtaining invalid results. Randomization in clinical trial methodology has a specific meaning and is implemented by using explicit procedures. Randomization, as described in the CONSORT guidelines,3 consists of 3 steps, which will be described in more detail in subsequent articles: generation of random allocation, allocation concealment, and implementation of randomization. Randomization aims at producing participant groups that are in all respects similar except for the intervention and in which any observed differences are expected to be due to chance. Additionally, randomization ensures that treatment allocation cannot be predicted in advance, because prediction of allocation has been associated with biased treatment effects.3 It should be clearly understood that the frequently used terms in the orthodontic literature to describe randomization such as consequent allocation and allocation based on some deterministic measure (eg, date of birth or day of the week) are not considered true randomization approaches in RCT methodology. RCTs are the highest level of evidence for health care intervention efficacy; however, it is not always possible or ethical to conduct such a trial. For example, let us assume that we want to evaluate the effect of premolar

Pandis

Fig. The pyramid of evidence. SRs, systematic reviews; RCTs, randomized controlled trials.

extractions on profile changes in patients with crowding and protrusion for whom current orthodontic treatment standards would unanimously dictate premolar extractions. It would be unethical to conduct an RCT for this research question because it would require us to treat 1 patient group without premolar extractions even though we accept that the ideal therapy should include extractions. In medicine, a classic example when it is unethical to conduct an RCT is the exploration of the association between smoking and lung cancer, since it would be unacceptable to randomize subjects to smoking and nonsmoking groups, and to record prospectively the development of lung cancer. When RCTs are not feasible or ethical, we must resort to observational or epidemiologic studies.4 Observational studies are divided into 3 main categories: cross-sectional, case-control and cohort studies. In general, observational studies are more prone to bias and confounding compared with RCTs and make it more difficult to establish causality. We intend to cover observational studies in more detail in future articles. Observational studies are used extensively to describe the distribution of disease and exposure in populations and to generate hypotheses; hypotheses can be further assessed, when feasible, with RCTs. On the other hand, RCTs, although highly controlled and likely to be less biased, because they are often conducted in highly

447

selected settings, might yield less pragmatic results that are of lower external validity. The term external validity or generalizabilty is used to describe the extent to which RCT results apply to other populations and settings. An orthodontic example of this issue is the assessment of Class II correction. Class II malocclusions seem to be multifactorial problems expressed with a plethora of variations.5 Applying an intervention for Class II correction in a highly selective group of participants might provide evidence that will not be easily translated into clinical practice in the wider range of the Class II population. To develop predictive (and diagnostic) models, large observational studies are more appropriate because the interventions can be applied to many participants in different populations and settings, thus yielding better insights of the effectiveness of the interventions of interest in real life. KEY POINTS

 RCTs, when applicable, are considered the gold standard for establishing causality.  Observational studies, although more prone to bias and confounding, are suitable for hypothesis generation and also for developing predictive and diagnostic models. The next article in this series will discuss bias in clinical trials. REFERENCES 1. Santoro MA, Gorrie TM, editors. Ethics and the pharmaceutical industry. Cambridge, United Kingdom: Cambridge University Press; 2005. 2. Guide to research methods, the evidence pyramid. Available at: http://library.downstate.edu/EBM2/2100.htm. Accessed on May 11, 2011. 3. Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al. CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 2010;340:c869. 4. Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, et al. STROBE initiative. Strengthening the reporting of observational studies in epidemiology (STROBE): explanation and elaboration. Epidemiology 2007; 18:805-35. 5. Wiltshire W. Class II growth modification in perspective. Semin Orthod 2006;12:2-3.

American Journal of Orthodontics and Dentofacial Orthopedics

September 2011  Vol 140  Issue 3