STATISTICS AND RESEARCH DESIGN

Use of controls in clinical trials Nikolaos Pandis, Associate Editor of Statistics and Research Design Bern, Switzerland, and Corfu, Greece

I

n 3 previous articles in this series on randomized clinical trials (RCTs), we discussed the process and steps of randomization. In this article, we will discuss the purpose and use of controls in RCTs. In RCTs, it is important to create treatment groups that are similar in all possible known and unknown factors except for the treatment that the trial groups will receive. If the treatment groups are similar, then we are more certain that any differences in the outcomes are related to the intervention rather than to other factors. The control group in an RCT is required for the following reasons.1-3  Participants with a certain condition might get well with time regardless of any therapy received. Therefore, if a study is conducted without a control group, we cannot be certain whether any improvements are related to the intervention or to the effect of time. For example, suspected canine impaction might resolve automatically without intervention as patients get older.  Selection bias can cloud the results. An investigator might select participants who have the best prognosis, thus permitting overestimation of the effects of the treatment of interest. For example, investigators designing a trial to assess resolution of canine impaction after deciduous canine extraction (the intervention) without using a control group might select subjects who are more likely to experience the outcome (resolution of canine impaction) and conclude that the intervention is effective when it is not.  The placebo effect can be powerful. Study participants might respond better to treatment just because they are included in a study and not because they are receiving active therapy. The use of a placebo in the control neutralizes the potential positive response to a therapy just because of inclusion in a study. In other words, patients receiving the active therapy might experience a response, which would be the sum of the placebo effect (invoked by inclusion in the trial) and the effect of the intervention. Adding

a control allows for the comparison to be fair, since the control group will also experience the placebo effect; therefore, the difference between the 2 groups would be only the effect of the treatment.  The Hawthorne effect can affect subjects’ behaviors. Inclusion in a study might elicit behavioral changes that can predispose participants to respond in an exaggerated fashion to the therapy. For example, when subjective outcomes, such as pain levels, are used, participants might modify their response because they know that they are being tested.4 It is essential to realize the importance of using controls and that the use of historical controls compromises the evidence and tends to overestimate the effect of the new therapy by introducing bias at different levels; therefore every effort should be made to include only concurrent controls, where feasible.3 Use of historical controls means to compare outcomes of patients with a particular condition currently receiving the intervention of interest with past outcomes of those with the same or a similar condition who have not been treated with the same intervention or have received standard therapy. Historical controls are problematic because they do not ensure that the comparisons between the control or standard treatment and the new treatment are fair; the 2 groups might differ in more ways than just the intervention, thus failing to fulfill the important purposes of the use of control and randomization in a trial. Let us summarize in detail the possible limitations associated with historical controls3 (Table). 1.

2.

3. Am J Orthod Dentofacial Orthop 2012;141:250-1 0889-5406/$36.00 Copyright Ó 2012 by the American Association of Orthodontists. doi:10.1016/j.ajodo.2011.10.018

250

4.

Randomization, the best method for ensuring equal and random allocation to the 2 intervention groups of known and unknown factors, would not be possible. Baseline differences will be difficult to determine, since the historical controls might not have the same baseline characteristics and prognostic factors with the intervention group; therefore, it will be impossible to separate the effect of the intervention from possible confounding. Blinding is not feasible when historical controls are used. Time trends can cause outcomes to change with time for reasons that cannot be predicted.

Statistics and research design

251

who might not have been included in a study at the time of their observation. These systematic differences can bias the outcome by showing preferential care to the new treatment group; thus, it is likely to exaggerate the effect of the new intervention. Observer bias is related to the methodologic differences in outcome assessment with time but also might imply that recording of the outcome could be biased if the outcome in the control group is known because it is historical. Therefore, if the outcome assessors, who might favor the new treatment, know the outcome in the control group, they might be inclined to manipulate the recorded values in the new treatment group. Additionally, the quality of outcome information extracted from the historical controls’ records is likely to be inferior or even inadequate compared with the new treatment group, since no study was underway requiring rigorous methodology at that time. Outcome recording criteria and interpretations might differ between historical controls and new treatment participants. Additionally, patient exclusion, for example, due to adverse events might be different between the 2 treatment arms, since there is a difference in time, settings, and possibly other criteria (postrandomization selection bias).

Table. Problems and types of bias related to historical

controls Reasons to avoid the use of historical controls No randomization Baseline differences Lack of blinding Time-trend effects Selection bias Bias due to difference in diagnostic methodology Bias due to outcome assessment methodology Bias in study management or concomitant care Observer bias Postrandomization bias Biased statistical analysis

5.

6.

7.

U U U U U U U U U U U

Selection bias is possible because unbiased patient selection and baseline similarity between the historical control and the new treatment groups cannot be certain. Patient selection in RCTs follows clearly defined inclusion and exclusion criteria, which are determined at the design stage. It is unlikely that exactly the same criteria were used for the historical controls, and it would also be difficult to determine this by looking at the participants’ records retrospectively. Additionally, investigators of the new treatment are likely to have received more training in the selection procedure and could be more meticulous and possibly more restrictive during participant selection, since they know that they are working on a clinical study. Differences in diagnostic and outcome assessment methodologies are possible. Historical controls might have limitations because of changes in accuracy of recording and assessing the outcomes. It is reasonable to assume that with time better diagnostic and outcome assessment methods have been developed that can result in differences in patient selection (selection bias) and outcome assessment (observer bias) between current patients and historical controls. A more recent and improved method is likely to be more accurate in identifying the required eligibility criteria for entry in a study. Since participant selection might become more accurate with time, there could be baseline discrepancies between the characteristics of historical controls compared with the sample selected to receive the new treatment. Imagine, for example, making assessments by comparing images obtained before and after the cone-beam era. Biases in study management and concomitant treatment are possible. Participants in a predefined study are likely to be followed and monitored better compared with the historical control participants,

8.

Finally, statistical adjustments for known factors are possible, but this is not possible for unknown factors, and this can only be remedied by using the methods of randomization with concurrent controls in an RCT (biased analysis). KEY POINTS

 Use of concurrent controls is important for valid results because they allow for fair comparisons between treatment groups.  Use of historical controls should be avoided if at all possible. The next article in this series will discuss blinding. REFERENCES 1. Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al. CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 2010;340:c869. 2. Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P. Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med 2008;148: 295-309. 3. Pocock SJ. Clinical trials: a practical approach. Chichester, United Kingdom: Wiley; 1993. p. 50-65. 4. Piantadosi S. Clinical trials: a methodologic perspective. 2nd ed. Hoboken, NJ: Wiley; 2005. p. 192.

American Journal of Orthodontics and Dentofacial Orthopedics

February 2012  Vol 141  Issue 2