Decision Letter (MDM-07-054.R1) From: [email protected] To: [email protected] Cc: [email protected] Subject: Manuscript ID MDM-07-054.R1 Body: 30-Oct-2007 Dear Dr. Sadatsafavi: It is a pleasure to accept your manuscript entitled "Acceptability curves could be misleading when correlated strategies are compared" as a "Brief Report" in Medical Decision Making. If you have not done so already, you should complete a copyright form, which you can download from http://mc.manuscriptcentral.com/mdm (click on the "Instructions and Forms" link to get the form. On behalf of the Editors of the Medical Decision Making, we look forward to your continued contributions to the Journal. Sincerely, Mark Helfand Editor, Medical Decision Making [email protected] Date Sent: 30-Oct-2007

Acceptability curves could be misleading when correlated strategies are compared Sadatsafavi M, Najafzadeh M, Marra CA The recent discussions1,2,3 on the limitation of the cost-effectiveness acceptability curves (CEACs) was helpful in reevaluating the value of CEACs in cost-effectiveness studies. One of the limitations of the CEACs, as pointed out by Groot Koerkamp et al1and other authors4,5 , is that they may mislead policy makers regarding the preferred alternative. This refers to the fact that the ranking of strategies based on CEAC might be different from their ranking based on expected net benefit. This known phenomenon has so far been attributed to the higher positive skew of the distribution of the incremental net benefit (INB) of the optimal strategy1,4,5 . We would like to point out that CEACs can also be misleading when correlated strategies are compared, in the absence of any skewness in the distribution of INBs.

To be explicit, let us imagine that the decision maker is interested in a model-based cost-utility analysis of two new strategies versus a current strategy for treating a malignancy. Strategy 0 is the current (baseline) practice, strategy 1 is based on the new drug A, and strategy 2 is based on drug A plus the palliative drug B, with slightly additional costs and small increase in quality of life. Assume that the output of the model can be approximated using the following distributions:

∆ c1,0 ~ Normal (− 1000,200), ∆ e1,0 ~ Normal (0.1,0.4) ∆ c 2,1 ~ Normal (50,50), ∆ e2,1 ~ Normal (0.001,0.001)

Where Δci,j and Δei,j are the incremental cost and quality-adjusted life years (QALY), respectively, of strategy i versus j. Since the outcome of strategy 2 is defined relative to strategy 1, there would be a high degree of correlation between the INBs of strategies 1 and 2 versus the baseline strategy. Such correlation could be the result of the fact that strategies 1 and 2 share several parameters such as the probability of remission, cure, mortality and several costs and effectiveness components.

Obviously, both treatments result in lower costs and higher effectiveness compared with the baseline strategy. Therefore, it is obvious that given the observed data, the best strategy is either strategy 1 or 2 at any level of willingness to pay (λ). In addition, the Incremental cost-effectiveness ratio (ICER) of strategy 2 versus 1 is 50,000$/QALY. So a decision maker with λ of 50,000$/QALY will be indifferent between strategies 1 and 2.

However, the CEAC is non-intuitive and is in conflict with the results based on the expected net benefit. As shown in Figure 1, the CEAC of the baseline strategy is above that of the two alternative strategies at a wide range of λ, despite the fact that this strategy is dominated and should not be selected at any positive value of λ.

The reason why this situation happens can be explained using Figure 2, which illustrates the joint distribution of INB1,0 and INB2,0 for a given λ (not to be confused with a cost-effectiveness plane). When all strategies are compared simultaneously, the probability of strategy 1 and 2 being the best strategy is the proportion of the joint distribution falling in regions Ia+ IV and Ib+II, respectively. The proportion of the joint distribution falling in region III is the probability of strategy 0 being the best

option, which is greater that those of the other two strategies. This is despite the fact that both alternative strategies individually have higher probability of costeffectiveness than strategy 0 when compared on a one-to-one basis (that is, 1 vs. 0 and 2 vs. 0). In other words, strategies 1 and 2, which have a large overlap in their region of optimality, carve up their share of the joint distribution, rendering their probability of cost-effectiveness below that of the baseline strategy.

From a theoretical viewpoint, there is nothing wrong with such CEAC as it serves its purpose of illustrating the probability of each option being the optimal choice at different values of λ. Such CEAC is, however, counter-intuitive for the decision maker given the dominancy of strategies 1 and 2 over 0. Being indifferent in choosing strategy 1 or 2, the decision maker might proceed to select the strategy that has the highest value on CEAC at λ=50,000$, and will find the results totally conflicting with the ranking of the strategies based on their expected INBs.

The high degree of correlation between the alternative strategies was the essential part of this phenomenon in our example. In the absence of correlation, the proportion of the joint distribution falling in quadrant III while its center is in quadrant I is no more than 25%, so it cannot make the baseline strategy ranking first in CEAC. Such correlations are fairly common in economic comparisons, especially when alternative strategies are based on similar interventions (the example presented here) or the same intervention applied to different populations.

As this example demonstrated, there are other conditions besides the skewness of the distribution of INB that could alter the rank of strategies in CEAC. As such, and given

other points brought about by Groot Koerkamp1 and Schwartz3, we agree with them that unless the consumer of a cost-effectiveness analysis is sophisticated enough not to over-interpret acceptability curves, CEACs might defeat the purpose of facilitating decision making under uncertainty.

Probability of being the optimal strategy

Figure 1: Cost-effectiveness acceptability curve (CEAC) for the three hypothetical strategies described in the text

1

0.8

Strategy 1 0.6

Strategy 0

0.4

0.2 0

Strategy 2

0

20

40 60 80 Willingness to pay (x1000$)

100

Figure 2: Joint distribution of the incremental net benefits (INB) of strategy 1 vs. 0 and 2 vs. 0 at a fixed willingness to pay (λ) y=x

INB (2 vs. 0) Ib II Ia

INB (1 vs. 0)

III

IV

References:

1

Groot Koerkamp B, Hunink MG, Stijnen T, Hammitt JK, Kuntz KM, Weinstein MC, .Limitations of acceptability curves for presenting uncertainty in cost-effectiveness analysis..Med Decis Making. 2007 Mar-Apr;27(2):101-11. 2

Fenwick E, Briggs A, .Cost-effectiveness acceptability curves in the dock: case not proven? Med Decis Making. 2007 Mar-Apr;27(2):93-5. 3

Schwartz A. The acceptability of acceptability curves: comments on Groot Koerkamp and others. Med Decis Making. 2007 Mar-Apr;27(2):96-7. 4

Fenwick E, Claxton K, Sculpher M..Representing uncertainty: the role of cost-effectiveness acceptability curves..Health Econ. 2001 Dec;10(8):779-87. 5

O'Hagan A, Stevens JW, .The probability of cost-effectiveness..BMC Med Res Methodol. 2002;2:5. Epub 2002 Mar 11.