Jacob Barhak Ph.D. http://sites.google.com/site/jacobbarhak/ ISPOR ...

Viewer
Transcript

Jacob Barhak Ph.D. http://sites.google.com/site/jacobbarhak/ th ISPOR 18 Annual international Meeting 18-22 May 2013

THE REFERENCE MODEL SCORES FITNESS OF MODELS AND POPULATIONS In a Nutshell Competitive Disease Forecast A League of Disease Models/Consumers Report Rotate Populations

Population 1

Population Characteristics: Age, Male, Race, SBP, DBP, A1c, Smoke, BMI, HDL, LDL, Trig, TC, ACR, Lipid Ratio, Age At Diagnosis Of Diabetes, AF, Survive Stroke, Survive MI, MicroAlbuminuria, MacroAlbuminuria, Diabetes Type 2, Treated for Hypertension, Family History CHD, Rheumatoid Arthritis, Townsend Index, A1c Change, BMI Change, DBP Change, SBP Change, HDL Change, LDL Change, Trig Change, TC Change, Smoke Change, Year

Population 2

Process CHD

No CHD

MI

Stroke

Process Competing Mortality

Inputs are based on secondary data: • Published Risk Equations • Published Clinical Trials: i.e. no real individual data • Other publications

Rotate equation/ hypothesis variations

End of Trial

A1c

7.48

7.24

BMI

28

28

DBP

80.5

74.3

SBP

145

137.9

HDL

1.25

1.25

LDL

3.11

2.65

Trig

1.64

1.59

Without Biomarker change, Use the baseline distribution in risk equation

1

2

3

*

2

*

*

3

4

5

*

*

*

*

6

7

*

*

5 6

*

7

*

8

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

*

*

*

*

4

*

*

*

*

*

*

*

*

*

*

* *

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

* *

*

Outputs: 1. Clinical outcomes / Deaths 2. Costs / Quality of Life 3. Population / Equation / Hypothesis fitness

*

9

*

*

With biomarker change, from 1st year use end of trial distribution in risk equation

Best Model

MODELS Model Characteristics

A1c changes BMI changes BP changes Lipid change Smoke changes MI Equation # Stroke Equation # CHD Death Equation # Stroke Death Equation # Treatment Improvement Correction

0 0 0 0 0 1 1 1 1 0

0 0 0 0 0 1 1 1 1 1

0 0 0 0 0 1 2 1 1 0

0 0 0 0 0 1 2 1 1 1

0 0 0 0 0 1 3 1 1 0

0 0 0 0 0 1 3 1 1 1

0 0 0 0 0 1 4 1 1 0

0 0 0 0 0 1 4 1 1 1

CONCLUSIONS: 1) Modelers should talk in terms of best fitting model rather than best model since in many cases, the best model changes according to specifics of the query. 2) Disease modelers who develop risk equations should add a temporal correction term within their risk equations to prevent model outdating. 3) The fitness score function allows comparing behavior of different populations using a similar metric and provides a visual explanation of current understanding of disease progression.

Potentially access more populations and cover more phenomena

*

*

*

*

RESULTS: The fitness score matrix uses color coding and ranking to visually demonstrate the fitness between 6/34 populations/cohorts and 64 combinations of published risk equations and hypotheses. The results show that different combinations of risk equations behave differently on different population cohorts. For each query, the system ranks the models. Models that implement the following two corrections generally behaved better: 1) Temporal correction for treatment improvement 2) Biomarker change introduced in the first year.

Other Death

Alive

Risk Equation #

ADVANCE Collaborative Group, Intensive Blood Glucose Control and Vascular Outcomes in Patients with Type 2 Diabetes, NEJM. 2008 Jun 12;358(24):2560-72. doi:10.1056/NEJMoa0802987. Epub 2008 Jun 6.

METHODS: 1) Extraction of multiple published disease models/risk equations for cardiovascular disease and multiple diabetic population distributions and outcomes. The populations used were UKPDS, ASPEN, ADVANCE, ACCORD, NDR, KP. 2) Implementation of these components and systematic cross validation of models against populations using micro-simulation and relying on computing power. 3) Defining a fitness score to convert multiple outcome differences into a single number. 4) Defining different queries with weights to rank model / population fitness. The methods avoid using individual data and rely on more accessible summary data.

Death

Stroke Death

Survive Stroke

Population n

1

Example From Table 1 in:

OBJECTIVES: To figure out what disease models equations/hypothesis work best on what populations.

CHD Death

Survive MI

Parameter: BioMarker/State/Treatment Hypothesis 1: Biomarker Change:

ABSTRACT

Process Stroke

No Stroke

Baseline

High Performance Computing

0 0 0 0 0 2 1 1 1 0

0 0 0 0 0 2 1 1 1 1

0 0 0 0 0 2 2 1 1 0

0 0 0 0 0 2 2 1 1 1

0 0 0 0 0 2 3 1 1 0

0 0 0 0 0 2 3 1 1 1

0 0 0 0 0 2 4 1 1 0

0 0 0 0 0 2 4 1 1 1

0 0 0 0 0 3 1 1 1 0

0 0 0 0 0 3 1 1 1 1

0 0 0 0 0 3 2 1 1 0

0 0 0 0 0 3 2 1 1 1

0 0 0 0 0 3 3 1 1 0

0 0 0 0 0 3 3 1 1 1

0 0 0 0 0 3 4 1 1 0

0 0 0 0 0 3 4 1 1 1

0 0 0 0 0 4 1 1 1 0

0 0 0 0 0 4 1 1 1 1

0 0 0 0 0 4 2 1 1 0

0 0 0 0 0 4 2 1 1 1

0 0 0 0 0 4 3 1 1 0

0 0 0 0 0 4 3 1 1 1

0 0 0 0 0 4 4 1 1 0

0 0 0 0 0 4 4 1 1 1

1 1 1 1 1 1 1 1 1 0

1 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 2 1 1 0

1 1 1 1 1 1 2 1 1 1

1 1 1 1 1 1 3 1 1 0

1 1 1 1 1 1 3 1 1 1

1 1 1 1 1 1 4 1 1 0

1 1 1 1 1 1 4 1 1 1

1 1 1 1 1 2 1 1 1 0

1 1 1 1 1 2 1 1 1 1

1 1 1 1 1 2 2 1 1 0

1 1 1 1 1 2 2 1 1 1

1 1 1 1 1 2 3 1 1 0

1 1 1 1 1 2 3 1 1 1

1 1 1 1 1 2 4 1 1 0

1 1 1 1 1 2 4 1 1 1

1 1 1 1 1 3 1 1 1 0

1 1 1 1 1 3 1 1 1 1

1 1 1 1 1 3 2 1 1 0

1 1 1 1 1 3 2 1 1 1

1 1 1 1 1 3 3 1 1 0

1 1 1 1 1 3 3 1 1 1

1 1 1 1 1 3 4 1 1 0

1 1 1 1 1 3 4 1 1 1

1 1 1 1 1 4 1 1 1 0

1 1 1 1 1 4 1 1 1 1

1 1 1 1 1 4 2 1 1 0

1 1 1 1 1 4 2 1 1 1

1 1 1 1 1 4 3 1 1 0

1 1 1 1 1 4 3 1 1 1

1 1 1 1 1 4 4 1 1 0

1 1 1 1 1 4 4 1 1 1

40.0 36.7 36.2 14.6 18.4 24.8 25.4 20.5 15.9 10.7 35.1 46.6 28.7 36.0 75.8 80.6 65.3 68.0 55.8 45.2 41.5 44.4 9.2 18.3 42.7 81.5 151.5 60.3 11.3 13.3 40.5 54.5 59.3 44.4

27.6 33.1 31.2 22.9 28.2 28.8 29.4 6.8 16.0 16.6 19.9 24.4 22.6 22.2 45.5 53.8 42.3 43.9 40.0 20.3 15.6 17.3 8.2 14.8 35.8 64.7 123.1 50.2 11.5 17.9 39.3 58.6 55.0 48.2

37.8 33.1 31.4 15.3 20.8 23.7 23.3 18.2 12.4 12.3 28.8 30.8 22.2 35.4 65.6 73.3 64.1 59.6 51.8 45.0 41.9 44.9 11.4 21.0 50.9 84.7 128.0 62.3 15.0 17.8 42.1 51.8 53.3 43.1

32.2 36.0 33.8 22.6 26.5 30.6 32.9 11.6 20.4 16.6 24.3 23.2 23.1 26.1 53.2 59.5 44.0 48.2 50.4 25.6 23.8 23.6 8.3 18.1 41.5 73.4 123.7 54.4 14.0 17.7 42.5 56.2 53.0 42.3

39.2 33.3 34.9 16.7 20.1 24.3 24.6 20.8 13.6 10.5 29.9 36.9 30.1 33.5 72.9 74.2 54.5 63.5 61.2 49.7 49.1 46.9 9.9 24.8 53.7 83.8 138.0 63.5 12.1 16.3 40.3 49.8 51.0 38.4

35.4 36.3 38.6 24.9 31.8 32.4 30.4 15.8 29.1 16.5 28.7 32.2 22.8 26.9 59.8 60.2 45.1 47.2 51.2 31.3 32.4 28.6 10.5 20.5 42.5 72.9 125.8 56.1 13.5 16.7 38.5 52.2 54.5 41.7

40.4 35.4 33.9 16.5 20.2 25.8 24.7 20.6 12.7 10.7 31.0 38.6 26.8 34.7 78.2 77.6 60.0 60.3 53.1 50.5 45.4 48.9 10.5 25.1 53.8 88.9 164.2 67.7 13.2 14.9 39.3 52.9 52.0 43.8

32.6 34.7 37.1 23.5 28.1 28.4 33.0 12.7 20.9 16.4 25.4 29.6 22.2 25.6 51.6 58.2 48.2 48.6 46.4 28.8 27.4 29.0 9.2 19.2 41.5 80.0 145.8 58.8 15.0 15.0 36.1 52.7 50.8 44.3

38.2 34.3 34.6 11.5 18.4 17.6 17.3 18.7 25.7 8.8 54.4 55.3 54.1 63.2 77.1 81.9 60.6 68.8 55.1 58.9 53.7 60.2 10.4 18.4 42.3 92.1 162.2 64.2 10.9 9.6 28.6 46.5 76.6 35.0

33.9 42.7 40.5 17.9 28.3 22.3 27.6 16.9 41.5 17.2 44.6 45.1 43.8 46.0 49.1 57.0 43.8 49.3 50.9 32.1 30.3 32.0 10.3 17.7 34.0 72.2 141.9 55.8 12.5 12.6 33.6 55.7 76.7 41.4

41.1 33.8 35.4 12.0 16.9 14.6 19.0 13.2 22.8 6.9 46.3 48.7 48.5 56.0 75.6 76.7 60.5 64.1 54.9 60.2 56.6 56.9 11.4 21.6 54.3 96.8 140.4 71.8 12.9 12.1 30.6 46.5 61.4 34.0

39.0 45.8 43.3 20.0 29.5 23.6 26.2 17.0 43.0 15.4 41.6 45.3 43.4 49.6 60.0 60.4 50.8 55.2 52.4 39.7 36.3 36.7 10.8 21.8 46.2 85.4 149.5 60.8 14.1 16.1 33.9 60.3 77.4 40.2

38.5 31.8 33.4 12.9 14.6 17.7 15.6 15.6 28.3 7.5 50.5 52.4 50.6 57.6 70.2 72.9 53.9 60.0 50.3 64.8 62.0 64.6 12.3 22.9 51.5 91.6 151.8 69.3 10.9 12.4 26.9 47.3 58.6 34.7

39.0 46.2 44.9 21.2 29.0 21.7 25.6 27.4 49.1 16.4 52.3 51.2 46.9 58.9 59.5 66.2 55.0 54.3 54.8 48.8 45.5 50.9 12.2 20.1 47.0 89.4 152.3 64.3 12.7 14.6 30.1 59.4 70.6 46.8

40.7 30.6 31.9 11.0 17.3 14.7 16.0 14.1 22.6 7.9 49.2 60.0 48.6 58.3 73.5 70.8 57.7 60.5 59.2 65.4 59.1 64.6 12.7 22.9 50.8 102.0 180.9 72.0 11.2 12.3 27.7 49.8 60.7 32.4

33.1 43.2 44.0 19.9 27.8 23.3 24.2 19.4 42.9 15.6 45.7 51.0 47.1 52.6 57.7 59.6 48.3 52.1 56.4 45.5 39.2 44.1 11.4 23.0 45.0 94.0 169.1 63.2 11.8 11.3 32.2 55.2 73.7 44.7

37.9 43.8 43.1 23.4 27.0 34.0 32.5 20.2 18.7 17.4 67.2 71.3 70.9 75.4 94.4 92.9 78.2 75.9 65.2 49.8 48.7 50.3 10.6 37.4 64.3 82.6 110.9 70.3 21.8 30.1 57.7 60.7 46.5 48.1

21.6 21.8 22.9 10.7 9.6 8.2 10.0 45.3 25.6 15.4 14.0 9.3 23.3 15.3 15.9 15.3 23.8 18.2 15.0 35.6 33.8 33.7 4.7 5.9 13.4 31.9 88.6 28.7 7.3 1.6 11.1 14.8 37.8 12.7

33.4 43.1 40.3 25.2 29.6 31.4 30.4 14.8 16.0 18.0 58.3 60.4 65.9 71.4 85.7 89.1 68.0 73.9 64.7 53.0 46.2 49.9 14.6 41.9 75.5 83.1 93.5 68.9 20.6 30.0 54.5 56.3 26.2 48.9

30.0 27.9 28.4 11.0 11.6 11.5 12.2 47.2 28.8 16.6 10.6 8.4 20.6 13.9 18.4 19.5 21.6 20.8 12.5 34.7 35.0 35.1 6.3 10.2 24.0 37.2 87.7 34.0 9.2 9.2 10.4 5.4 38.0 9.1

34.3 47.7 39.0 23.0 29.1 32.6 29.8 16.1 18.6 16.6 62.5 65.6 64.1 72.0 81.2 88.3 69.2 78.3 61.6 54.2 53.0 54.3 13.6 41.4 73.8 89.3 97.5 69.8 20.1 30.5 56.4 59.2 20.8 49.6

35.3 28.5 30.0 12.7 11.4 7.3 9.3 51.1 32.3 16.5 12.2 12.3 12.6 7.4 22.0 23.8 20.8 18.3 13.9 37.2 36.0 35.9 7.0 10.9 21.4 42.8 95.4 34.2 8.7 7.5 6.9 7.8 36.1 8.0

32.7 41.3 43.4 22.4 30.2 31.6 34.8 16.7 19.0 18.5 67.4 67.0 65.4 76.9 84.1 90.5 76.3 76.3 64.2 53.8 54.4 53.2 12.8 44.6 75.1 95.2 130.4 75.7 20.4 32.0 58.6 57.4 31.3 48.2

30.5 28.9 28.8 12.2 11.5 7.6 9.3 48.0 28.6 16.5 12.1 10.1 14.1 5.2 20.8 17.9 23.1 20.4 11.5 35.6 35.4 34.6 6.6 10.2 20.6 40.3 108.3 36.1 5.9 5.2 7.3 6.3 39.7 7.2

26.7 23.2 24.1 12.4 14.6 21.8 17.1 22.7 12.2 6.8 45.7 45.6 50.8 40.9 64.0 60.6 51.2 48.1 35.8 44.1 35.2 40.7 8.9 21.1 40.4 73.7 136.9 60.0 12.3 12.1 31.8 54.9 78.1 39.8

24.0 22.5 21.1 10.4 10.2 9.8 11.2 37.1 23.8 12.2 12.3 7.9 20.0 19.7 12.8 13.3 18.4 17.0 15.6 30.4 29.7 29.4 6.0 6.7 12.1 32.8 83.9 28.0 6.1 0.7 9.3 17.6 9.6 12.3

25.6 22.2 20.9 13.5 9.0 21.9 17.8 18.2 7.3 10.3 39.8 34.5 49.1 41.6 58.4 50.2 51.4 41.3 37.4 46.6 37.7 41.4 9.8 22.4 43.7 71.7 115.1 59.6 14.0 15.1 34.4 53.4 67.5 40.8

31.0 25.8 26.7 9.8 11.1 8.3 11.1 37.1 25.4 12.6 7.6 7.1 17.3 14.2 18.9 16.2 17.3 15.1 8.5 30.0 31.2 29.7 7.5 9.2 22.4 45.3 94.0 35.0 7.9 9.3 11.6 13.5 7.6 10.4

23.3 23.8 22.5 12.2 13.6 23.5 16.7 18.8 10.4 7.2 42.3 36.3 49.2 43.9 58.8 51.8 51.7 38.3 38.4 51.1 42.9 44.1 13.6 24.1 45.6 73.1 124.4 57.6 14.0 11.6 35.0 56.2 62.3 43.6

34.1 29.8 31.4 10.2 9.8 8.7 8.8 40.8 28.3 11.9 8.9 6.8 11.3 8.1 18.8 17.6 17.0 15.5 9.7 33.2 34.4 33.8 8.2 10.5 23.3 47.6 99.0 36.3 7.5 7.6 7.7 13.9 8.3 7.4

27.5 22.4 23.5 10.9 10.3 21.6 17.9 22.9 10.8 8.8 47.9 36.4 44.2 44.9 62.4 60.8 47.1 40.7 37.0 53.7 44.6 49.3 11.6 21.9 43.8 91.1 151.7 70.9 12.1 13.7 30.0 56.1 72.3 41.1

32.6 27.9 26.5 8.6 11.4 9.0 9.9 39.0 25.8 10.5 10.2 8.5 10.3 9.5 19.5 17.5 17.0 15.4 8.7 30.8 30.9 32.7 7.1 8.2 21.5 50.7 116.2 37.8 7.2 5.9 8.2 12.6 17.7 12.2

39.8 34.7 37.0 16.2 18.0 22.9 19.8 22.0 12.3 11.0 31.9 29.3 21.3 27.4 64.1 64.9 56.7 53.0 47.4 43.3 36.0 40.6 10.3 20.7 44.5 77.1 149.3 62.1 11.7 15.9 38.2 50.9 63.1 45.0

26.2 32.6 31.9 22.7 22.0 29.5 25.5 9.8 11.2 15.4 16.8 20.3 22.9 21.2 41.2 43.6 34.7 35.4 36.2 19.1 15.2 17.9 7.7 16.7 33.7 65.7 120.2 52.1 13.2 13.5 38.3 58.5 51.4 45.0

43.2 33.4 32.1 15.1 17.0 24.9 21.4 19.7 6.9 9.8 21.9 24.6 23.3 26.6 58.7 61.2 49.2 46.5 44.4 44.3 37.2 39.4 11.2 23.1 54.4 85.8 132.0 65.9 14.4 16.7 39.6 54.9 44.9 39.6

31.8 35.4 33.5 21.7 23.3 27.3 25.9 9.9 14.1 17.0 20.0 22.7 21.1 19.6 47.2 47.5 41.9 40.4 39.9 23.4 20.2 23.4 9.1 22.4 42.5 71.2 129.1 53.4 13.2 17.7 40.0 57.2 53.8 50.0

40.6 33.9 36.3 17.5 16.2 24.0 22.0 18.0 13.5 9.6 21.8 27.2 22.2 25.6 61.3 66.0 46.8 46.7 43.4 50.4 41.7 43.3 11.1 23.2 50.7 83.0 137.6 65.1 13.8 16.9 36.6 52.4 48.9 41.7

33.5 41.1 37.2 21.1 25.7 29.7 27.5 20.0 21.6 17.0 21.7 27.4 19.6 23.4 51.6 51.0 39.1 42.9 45.3 32.6 28.5 29.2 10.5 21.5 42.5 78.6 133.2 54.3 12.8 15.1 39.5 56.8 53.0 44.5

41.4 36.2 37.4 14.5 17.8 24.4 21.3 20.2 12.1 7.8 30.1 30.1 23.1 32.3 64.8 63.1 47.4 49.2 46.0 49.5 40.1 45.3 11.6 22.8 51.8 94.6 159.4 64.8 13.8 14.7 34.0 55.0 51.3 45.7

30.7 34.6 34.5 22.7 24.4 31.8 30.8 12.3 16.2 17.6 22.2 27.5 18.4 23.2 43.8 47.0 36.8 38.9 38.3 29.6 26.0 26.0 8.1 19.2 43.6 78.6 150.1 60.9 12.6 16.8 36.2 52.0 55.4 45.6

39.4 31.2 35.6 11.6 14.7 18.4 15.5 18.1 21.4 8.8 45.4 43.0 45.8 43.2 66.0 68.3 54.8 47.3 40.3 58.5 44.2 51.7 10.6 18.4 45.7 92.9 167.3 63.1 11.5 9.8 28.3 47.1 73.6 35.3

32.5 40.1 36.9 21.3 25.3 25.3 23.9 17.5 31.2 18.5 36.4 32.0 36.2 35.2 42.8 43.8 41.3 32.2 35.9 30.9 21.5 28.3 9.2 18.7 36.8 79.3 143.5 54.2 11.8 12.6 28.3 58.3 72.8 41.6

36.6 34.1 33.2 11.6 12.4 14.1 13.9 12.8 18.3 6.5 32.5 35.5 43.7 35.5 57.5 56.9 52.1 40.3 48.4 56.3 43.7 51.2 10.8 26.3 49.9 91.2 147.7 63.3 11.6 13.6 32.0 47.3 63.4 36.9

35.9 44.1 42.7 22.1 23.2 28.0 22.5 18.9 36.2 17.4 33.0 33.7 36.7 34.7 50.4 51.7 42.7 37.0 45.5 37.2 29.7 35.9 11.8 20.7 43.2 88.4 145.8 58.7 15.0 14.1 32.4 62.3 67.7 43.7

37.8 32.1 31.6 15.5 14.5 16.9 15.0 16.1 21.9 8.4 39.2 32.4 41.3 43.4 58.2 54.3 45.1 42.7 45.6 61.2 50.4 57.0 12.0 23.0 48.6 91.5 155.1 63.8 12.4 11.0 29.7 46.4 56.4 36.0

41.8 43.8 43.3 20.7 23.9 23.7 21.1 25.8 39.6 16.2 37.9 38.8 40.6 38.7 49.9 49.9 43.9 36.0 44.8 43.3 37.9 42.7 13.1 20.8 46.0 88.0 151.6 61.8 13.2 12.7 30.7 59.6 71.6 40.5

40.4 30.8 33.6 12.1 14.0 14.7 16.3 14.3 18.9 6.7 43.6 41.6 44.0 44.0 62.0 59.3 45.7 45.8 48.6 62.5 54.0 59.0 12.8 21.5 48.6 91.4 182.2 72.5 12.8 13.0 28.3 47.4 63.3 36.2

38.3 43.4 46.0 21.3 22.2 22.5 22.2 19.0 37.7 15.3 37.3 37.1 36.1 39.7 51.5 48.9 44.1 36.9 43.9 42.4 38.3 37.1 12.0 19.8 45.9 87.8 168.1 67.8 11.6 14.3 35.2 56.2 71.9 46.3

36.0 46.4 43.6 22.7 19.5 33.1 22.9 18.9 11.0 20.1 47.5 51.5 45.1 50.2 73.1 66.9 59.8 57.6 48.8 44.5 35.1 38.3 11.3 36.9 71.0 86.4 109.6 71.9 21.0 29.1 58.7 58.0 47.7 49.3

21.5 23.0 22.4 12.2 12.1 9.1 11.5 49.9 32.0 16.7 18.9 16.6 30.0 18.5 20.9 17.6 27.8 24.6 18.6 37.7 40.0 37.7 4.3 6.1 15.4 31.3 85.4 28.4 6.1 2.8 9.0 16.9 38.5 12.9

35.5 43.3 41.6 20.9 20.1 31.5 22.1 16.1 5.9 18.9 43.7 46.4 49.2 46.4 66.9 62.0 51.2 49.4 52.1 45.6 36.2 39.0 11.9 39.3 73.1 85.1 89.6 68.3 21.9 32.1 55.4 56.8 25.4 52.2

30.4 28.1 29.2 15.8 13.7 8.6 11.3 47.4 31.4 17.4 17.0 11.6 28.5 17.3 20.3 19.1 25.6 22.3 12.6 36.1 38.2 36.9 6.5 11.3 22.8 40.8 89.0 32.4 9.2 9.2 11.5 5.4 41.4 7.7

35.0 44.3 43.5 21.3 19.5 35.7 23.0 17.1 10.7 16.9 46.0 49.0 50.8 51.7 59.5 62.9 57.1 56.2 51.4 48.5 42.0 44.4 13.9 42.0 70.8 86.7 96.5 72.2 18.6 31.2 54.7 59.0 21.9 47.8

34.1 29.8 29.7 15.5 12.4 9.5 10.3 51.4 35.2 17.3 14.6 11.9 16.6 13.5 21.9 19.2 24.2 21.3 14.5 39.8 39.9 39.4 6.4 11.0 23.9 44.3 91.9 35.0 9.5 7.6 9.9 5.8 40.3 7.2

36.0 43.2 44.2 23.0 18.3 32.1 23.3 16.9 10.1 17.9 50.3 48.3 50.0 54.0 70.3 62.7 56.1 55.9 46.7 50.6 41.3 45.2 10.9 41.8 73.6 96.8 131.8 72.9 19.8 29.5 52.2 55.7 31.3 49.9

29.3 28.2 27.4 12.0 13.4 7.5 10.5 49.5 32.3 17.2 14.7 12.0 19.0 10.1 22.5 19.7 24.5 22.8 13.1 38.9 38.7 37.5 5.9 9.0 23.8 43.7 106.2 34.1 7.8 3.7 6.6 7.4 40.8 8.6

Hypothesis 2: Temporal Correction for Treatment Improvement: Each model variation was executed: With Temporal Treatment Correction

Without Temporal Treatment Correction

UKPDS ASPEN

ADVANCE

ACCORD KP

NDR

28.4 21.6 24.1 14.0 19.5 19.9 23.7 22.3 17.4 9.1 62.9 71.7 68.8 78.1 87.4 89.4 78.5 73.0 55.0 46.4 45.4 49.3 8.5 19.4 41.0 73.3 140.3 61.2 11.0 14.0 31.7 51.4 75.7 41.7

23.7 22.2 22.2 12.1 7.6 10.6 11.9 38.2 18.0 12.0 11.2 8.1 19.5 19.2 14.5 19.7 15.8 11.7 11.9 28.9 27.1 28.6 6.6 6.0 14.7 35.4 91.7 28.1 5.3 1.8 11.8 17.2 3.3 14.3

25.2 23.2 23.7 12.1 21.7 20.7 22.7 20.3 9.8 8.9 59.7 62.1 61.7 67.2 78.9 81.0 70.6 73.3 57.5 48.9 42.7 45.1 10.9 19.9 46.6 81.6 119.8 59.1 13.9 13.7 35.0 54.4 70.6 42.6

28.8 26.8 24.5 12.3 10.7 12.5 14.0 39.9 20.5 12.4 8.8 12.8 15.8 18.3 22.1 27.8 19.3 17.2 16.5 28.5 27.0 28.6 7.3 11.4 21.8 42.0 94.6 36.7 9.0 8.3 15.0 10.8 6.0 9.7

27.0 22.6 22.9 12.8 17.3 21.9 20.8 21.9 16.2 7.7 62.8 63.7 61.6 68.0 76.9 84.6 67.7 66.7 55.8 48.5 50.2 49.9 11.4 26.7 47.5 71.9 117.2 62.7 12.7 13.4 34.7 52.7 68.5 42.5

34.9 29.0 31.6 10.2 9.0 7.4 9.9 39.4 25.3 12.5 15.4 19.1 13.1 18.0 27.1 28.0 20.4 20.5 17.3 35.0 31.0 33.2 7.7 10.2 24.2 49.9 99.8 37.1 9.2 6.4 7.9 14.5 6.9 10.6

25.3 21.5 23.1 11.9 17.0 23.6 22.2 23.5 10.8 10.6 59.8 62.8 62.8 72.6 78.3 85.5 70.2 71.4 53.8 52.3 55.0 51.6 12.4 23.2 49.7 89.6 155.8 66.0 11.9 12.3 37.7 54.6 68.7 45.4

31.2 29.1 26.7 9.1 9.9 7.2 11.6 40.3 19.1 11.5 12.2 14.4 13.7 19.5 22.2 28.0 18.0 16.4 11.9 32.8 32.3 29.4 6.8 7.3 21.4 49.8 116.3 37.8 6.9 6.0 9.4 12.5 17.2 7.1

OVERALL MODEL RANKING RESULTS Model Characteristics Sorted

Method_A1c Method_BMI Method_BP Method_Lipids Method_Smoke Method_MI Method_Stroke Method_DeathCHD Method_DeathStroke Method_TimeImprove Weighted Mean

1 1 1 1 1 1 1 1 1 1

0 0 0 0 0 1 1 1 1 1

0 0 0 0 0 4 1 1 1 1

1 1 1 1 1 1 2 1 1 1

1 1 1 1 1 4 1 1 1 1

0 0 0 0 0 1 2 1 1 1

1 1 1 1 1 1 3 1 1 1

0 0 0 0 0 4 2 1 1 1

0 0 0 0 0 4 3 1 1 1

19.384 20.191 21.622 22.664 22.814 22.919 23.265 23.301 23.841

0 0 0 0 0 4 4 1 1 1

1 1 1 1 1 1 4 1 1 1

0 0 0 0 0 1 3 1 1 1

1 1 1 1 1 4 2 1 1 1

1 1 1 1 1 4 3 1 1 1

1 1 1 1 1 4 4 1 1 1

0 0 0 0 0 1 4 1 1 1

1 1 1 1 1 2 1 1 1 1

0 0 0 0 0 2 1 1 1 1

23.92 24.293 24.561 24.584 24.979 24.997 25.048 41.618 43.138

1 1 1 1 1 2 2 1 1 1

0 0 0 0 0 2 2 1 1 1

1 1 1 1 1 1 2 1 1 0

0 0 0 0 0 2 3 1 1 1

1 1 1 1 1 2 4 1 1 1

0 0 0 0 0 2 4 1 1 1

1 1 1 1 1 3 1 1 1 1

1 1 1 1 1 2 3 1 1 1

1 1 1 1 1 1 3 1 1 0

1 1 1 1 1 1 1 1 1 0

0 0 0 0 0 3 1 1 1 1

1 1 1 1 1 2 3 1 1 0

1 1 1 1 1 2 1 1 1 0

1 1 1 1 1 2 2 1 1 0

0 0 0 0 0 2 2 1 1 0

1 1 1 1 1 3 3 1 1 0

1 1 1 1 1 3 2 1 1 0

45.39 45.947 45.984 46.484 46.517 46.637 46.954 47.201 47.606 48.054 48.305 48.784 48.995 49.249 49.797 49.893 50.228

1 1 1 1 1 1 4 1 1 0

1 1 1 1 1 3 2 1 1 1

0 0 0 0 0 1 3 1 1 0

0 0 0 0 0 2 3 1 1 0

0 0 0 0 0 1 2 1 1 0

0 0 0 0 0 1 1 1 1 0

0 0 0 0 0 2 1 1 1 0

1 1 1 1 1 3 1 1 1 0

1 1 1 1 1 2 4 1 1 0

1 1 1 1 1 3 3 1 1 1

0 0 0 0 0 3 3 1 1 0

1 1 1 1 1 3 4 1 1 0

1 1 1 1 1 3 4 1 1 1

0 0 0 0 0 3 2 1 1 0

0 0 0 0 0 3 1 1 1 0

0 0 0 0 0 3 2 1 1 1

0 0 0 0 0 2 4 1 1 0

0 0 0 0 0 3 4 1 1 1

0 0 0 0 0 3 3 1 1 1

50.47 50.575 50.652 51.027 51.147 51.213 51.319 51.614 51.682 51.809 52.312 52.432 52.627 52.801 52.956 53.445 53.539 54.148 54.212

1 1 1 1 1 4 2 1 1 0

0 0 0 0 0 1 4 1 1 0

1 1 1 1 1 4 3 1 1 0

1 1 1 1 1 4 1 1 1 0

0 0 0 0 0 4 2 1 1 0

1 1 1 1 1 4 4 1 1 0

0 0 0 0 0 4 3 1 1 0

54.63 55.122 55.353 55.509 58.025 58.034 58.667 59.531

Best 20 models have temporal treatment correction !

Best Model

0 0 0 0 0 3 4 1 1 0

Temporal treatment correction is a good idea! It accounts for model outdating

0 0 0 0 0 4 1 1 1 0

0 0 0 0 0 4 4 1 1 0

60.4 62.932

Time

Adjustment Time

FITNESS: LOW SCORE = GOOD FITNESS UKPDS33 Conventional UKPDS33 Intensive UKPDS33 Full ASPEN All Placebo ASPEN All Atorvastatin ASPEN Primary Placebo ASPEN Primary Atorvastatin ASPEN Secondary Placebo ASPEN Secondary Atorvastatin ASPEN Full ADVANCE Standard ADVANCE Intensive ADVANCE Asia Standard ADVANCE Asia Intensive ADVANCE EME Standard ADVANCE EME Intensive ADVANCE Eastern Europe Standard ADVANCE Eastern Europe Intensive ADVANCE Full ACCORD BP Standard Therapy ACCORD BP Intensive Therapy ACCORD BP Full KP 20-34 KP 35-50 KP 50-65 KP 65-75 KP 75+ KP Total NDR 20-34 NDR 35-50 NDR 50-65 NDR 65-75 NDR 75+ NDR Total

Simulated Time Stamp

Model Data Time Interval

Fitness Score: Simulated Outcomes at time T

Observed Outcomes at time T

X1

Y1

X2

Y2

Xn

Yn

Better fitness

Z=f(X,Y) Fitness of Model M and Population P

Z

Additional Query information leads to: • Ranking Models per populations • Ranking Populations per model • Overall Model Ranking

Worst fitness

References: • Barhak J., Isaman D.J.M., Ye W., Lee D.: Chronic disease modeling and simulation software. Journal of Biomedical Informatics, Volume 43, Issue 5, October 2010, Pages 791–799, http://dx.doi.org/10.1016/j.jbi.2010.06.003 • Barhak J., The Reference Model in the Mount Hood #6-2012 validation challenge and the uncertainty challenge. The Mt hood challenge 6, June 7-8, 2012. Johns Hopkins Mt. Washington Conference Center. • You Tube Video - The Reference Model for Disease Progression: Online: http://www.youtube.com/watch?v=7qxPSgINaD8 • Barhak J., The Reference Model for Disease Progression. Poster presentation, SciPy 2012, Austin Tx, 18-19 July 2012. SciPy 2012 proceedings can be accessed publically online: https://github.com/Jacob-Barhak/scipy_proceedings/blob/2012/papers/Jacob_Barhak/TheReferenceModelSciPy2012.rst • J. Barhak, The Reference Model for Chronic Disease Progression. 2012 Multiscale Modeling (MSM) Consortium Meeting, October 22-23, 2012, Abstract: http://www.imagwiki.nibib.nih.gov/mediawiki/images/7/77/Reference_Model_for_Chronic_Disease_Progression_Barhak.pdf . Poster: http://www.imagwiki.nibib.nih.gov/mediawiki/images/c/c4/PosterTheReferenceModel_IMAGE_MSM_Submit_2012_10_17.pdf

Acknowledgments: The GPL modeling framework was supported by the Biostatistics and Economic Modeling Core of the MDRTC (P60DK020572) and by the Methods and Measurement Core of the MCDTR (P30DK092926), both funded by the National Institute of Diabetes and Digestive and Kidney Diseases. The modeling framework was initially defined as GPL and was funded by Chronic Disease Modeling for Clinical Research Innovations grant (R21DK075077) from the same institute. The Reference Model was developed independently without financial support.