Milliman Depression Predictive Modeling Report – January 2017

Depression Predictive Modeling Report January 2017

Prepared by:

Milliman, Inc. 1400 Wewatta Street Suite 300 Denver, CO 80202-5549 USA

Stephen P. Melek FSA, MAAA Travis Gray FSA, MAAA

Tel +1 303 299 9400 Fax +1 303 299 9018

Stoddard Davenport

milliman.com

January 30, 2017 Offices in Principal Cities Worldwide

Milliman Depression Predictive Modeling Report – January 2017

Table of Contents INTRODUCTION................................................................................................................................................................... 1  METHODOLOGY.................................................................................................................................................................. 2  RESULTS ............................................................................................................................................................................. 5  Predictive Value ............................................................................................................................................................. 5  C-Statistic....................................................................................................................................................................... 7  CONCLUSION ...................................................................................................................................................................... 8  CAVEATS ............................................................................................................................................................................. 8  APPENDIX ............................................................................................................................................................................ 9  Table A1......................................................................................................................................................................... 9  Table A2......................................................................................................................................................................... 9  Table A3......................................................................................................................................................................... 9  Table A4....................................................................................................................................................................... 10  Table A5....................................................................................................................................................................... 10  Table A6....................................................................................................................................................................... 11  Table A7....................................................................................................................................................................... 12 

January 30, 2017 Offices in Principal Cities Worldwide

Milliman Depression Predictive Modeling Report – January 2017

INTRODUCTION In April 2016, we presented the benefits of predictive modeling for SIM primary care practices, particularly with regard to identifying and treating behavioral health conditions. We demonstrated an approach for building a depression predictive model using regional commercially insured data, noting that building and calibrating models specific to Colorado’s population that account for the differences between commercial, Medicaid, and Medicare enrollees would be a useful next step. For this analysis, we have developed initial versions of three depression predictive models, creating distinct models for the commercial, Medicaid, and Medicare populations, using Colorado’s All Payer Claims Database (APCD). The results of this modeling show that the members with the highest likelihood of being diagnosed with depression in the year following a base year of claim data can be effectively identified, and we are hopeful that this identification model will improve the timely diagnosis and treatment of depression in SIM primary care practices.

January 30, 2017

1 Offices in Principal Cities Worldwide

Milliman Depression Predictive Modeling Report – January 2017

METHODOLOGY We identified a population of insured members with at least two years of continuous eligibility in Colorado, separately for the commercial, Medicaid, and Medicare populations, using calendar years 2012 through 2015 data in the APCD. We then compiled medical and pharmacy claims data for these members, using diagnosis codes and pharmaceutical therapeutic classes to identify depression diagnoses and treatments. The criteria used to identify depression is listed in Table A1 of the Appendix. We restricted the population to only include members who did not appear to have been receiving treatment for depression in their first year of eligibility, with our intent being to predict the presence of untreated or undiagnosed depression. We then separated the remaining population into two cohorts; a cohort of members with a depression indicator after the initial period and a cohort of members without a depression indicator. For each enrollee, we examined a twenty four month period of continuous eligibility, where a member’s traits and utilization patterns in the first twelve months were used to predict the likelihood of that member beginning treatment for depression in the final twelve months of eligibility assessed. For members who never appeared to receive treatment for depression, we selected their most recent two years of continuous eligibility for modeling. For members that received treatment for depression (not in their first year of eligibility), we assigned the calendar year of their earliest depression claim as the final twelve months and the prior calendar year as the period of time to identify traits predictive of future depression, retaining only members who maintained continuous eligibility during that timeframe. The total number of members that satisfied this criteria are presented by line of business in Table 1 below. This table also shows the percentage of these members that were diagnosed with depression or treated for depression in the second year. Table 1: Members selected for modeling by line of business Total Members in Year 1 (no depression diagnoses or treatment)

Percent of Members Diagnosed with Depression or Treated for Depression in Year 2

1,183,269

4.5%

Medicaid

822,733

7.2%

Medicare

860,222

12.6%

Line of Business Commercial

We then built predictive models for each line of business to identify the healthcare utilization and demographic traits that were associated with an increased probability of being diagnosed with depression or treated for depression in the second year. Each model was built using 90% of the total members listed above, with 10% set aside as a validation set to test the model’s effectiveness in predicting depression. We started with an extensive list of traits that could potentially be associated with a higher risk of depression, using many of the same traits identified in the April 2016 predictive modeling report. The family risk factors identified in the previous report were eliminated from these predictive models, as these traits are not identifiable in the APCD. We also added the use of several psychotropic drug classes (other than antidepressants) to the list of traits that might be potential predictors of depression. Traits that we considered for inclusion in each of the commercial, Medicaid, and Medicare models include the following broad categories: 

Demographics – including age and gender.

January 30, 2017

2 Offices in Principal Cities Worldwide

Milliman Depression Predictive Modeling Report – January 2017



Behavioral conditions – presence of diagnoses of dementia, delirium, alcohol abuse, drug abuse, anxiety, schizophrenia, bipolar disorders, pervasive developmental disorders, anorexia, bulimia, posttraumatic stress disorder, attention deficit disorder, intellectual disabilities, and other psychotic or psychiatric conditions.



Chronic medical conditions – presence of diagnoses of arthritis, cancer, chronic pain, pulmonary hypertension, chronic obstructive pulmonary disease, diabetes, irritable bowel syndrome, menopause, obesity, osteoporosis, asthma, chronic kidney disease, congestive heart failure, back pain, ischemic heart disease, and infertility.



Medical events or acute medical conditions – presence of diagnoses indicating transplants, cardiac events, chest pain, migraines, hyperventilation, insomnia, major injuries, myocardial infarction, strokes, and concussions.



Other symptoms or stresses – child neglect, physical abuse, relational problems, occupational problems, academic problems, acculturation problems, bereavement, malingering, antisocial behavior, suicidal ideation, non-compliance with treatment, and stress.



Maternity – presence of inpatient or professional revenue and CPT codes associated with maternity visits.



Medical utilization measures – inpatient admits, surgeries, emergency room visits, dialysis, chemotherapy, chiropractor visits, and other services.



Prescription drug claims – use of sleeping pills, blood pressure medications, pain medications, weight loss drugs, anti-anxiety medications, Central Nervous System (CNS) agents, anti-psychotic medications, and erectile dysfunction medications.



Cost measures – total annual medical costs and total annual out of pocket costs for the member.



Risk score – based on Chronic Illness and Disability Payment System (CDPS) which is frequently used for risk adjustment by Medicaid.

This list of traits was then passed through a stepwise regression algorithm, first identifying the traits with the strongest relationship with a future diagnosis or treatment of depression, and adding traits one at a time that met statistical significance thresholds until the model fit no longer improved. The development of a solid predictive model requires testing different variables, populations, and model formats. Some of the variations we tested included: 

Using continuous variables versus variables segmented into different ranges,



Grouping conditions into categories versus modeling each condition independently,



Developing one model that can apply to Medicare, Medicaid, and commercial populations versus individual models for each cohort, and



Modeling members with claims along with members without claims, versus modeling all members together.

The results of several different model iterations suggested that the most effective modeling approach was to create two models for each line of business: one set of models for members with medical or prescription drug claims and another set for members that did not have any claims, which uses demographic traits only. Using this approach, we developed six unique models, and each model utilizes the traits most predictive of future depression for the cohort of interest. Details of the variables selected for each model can be found in Appendix Tables A2 through A7. Tables A2A4 in the Appendix provide the lists of demographic variables identified as predictive of future depression for

January 30, 2017

3 Offices in Principal Cities Worldwide

Milliman Depression Predictive Modeling Report – January 2017

each of the models that focuses on members with no medical or pharmaceutical utilization in the first year. Tables A5 through A7 in the Appendix list the variables selected for each model that focuses on members with medical and/or pharmaceutical utilization in the first year. The predictive models built using these variables provide an estimate of the impact that each trait has on a member’s risk of future diagnosis or treatment for depression. Each member is assigned a score between 0% and 100% that represents the estimated probability of having depression in year two. These scores can then be compared to actual outcomes in year two to see how effectively the model stratifies the population based on the risks determined by member traits in year one.

January 30, 2017

4 Offices in Principal Cities Worldwide

Milliman Depression Predictive Modeling Report – January 2017

RESULTS As mentioned above, each model was constructed using 90% of the members presented in Table 1, with 10% set aside in order to validate the model’s performance on data that was not included in model development. The results presented in this section use validation sets to compare model predicted outcomes to actual outcomes. The goal of these models is to identify members with a high risk of having undiagnosed or untreated depression so that depression screening or treatment can be made earlier in the course of their illness. The results of the depression predictive models for members who had either a pharmacy or medical claim show that a high proportion of the undiagnosed depressed population can be identified by looking at the highest risk members: 

The top 20% riskiest members in year one produced by the commercial model represented about 53% of the total actual depressed population in year two. It is about 2.6 times more likely to identify members with depression in this cohort than if the population were randomly sampled. Alternatively, the 20% least risky members in year one, as identified by the model, account for less than 3% of the total actual depressed population in year two.



The top 20% riskiest members in year one produced by the Medicaid model represented about 60% of the total actual depressed population in year two. It is about 3 times more likely to identify members with depression in this cohort than if the population were randomly sampled. Alternatively, the 20% least risky members in year one, as identified by the model, account for less than 0.5% of the total actual depressed population in year two.



The top 20% riskiest members in year one produced by the Medicare model represented about 39% of the total actual depressed population in year two. It is about 1.9 times more likely to identify members with depression in this cohort than if the population were randomly sampled. Alternatively, the 20% least risky members in year one, as identified by the model, account for less than 9% of the total actual depressed population in year two.

These results show that by focusing on just the highest risk members in the population identified by the depression predictive models, a large number of depressed individuals could be positively identified. This information can be particularly useful for practices with limited resources for depression screening and outreach. In addition, there are a variety of statistical measures that can be used to assess the effectiveness of these predictive models. Predictive values and c-statistics are presented in the following sections.

Predictive Value Positive and negative predictive value are the proportions of positive and negative results (scores above and below a given threshold) that are true positives or true negatives. In this context, the positive predictive value is the percentage of members with a depression risk score above a certain threshold that are actually diagnosed or treated for depression in year two, and the negative predictive value is the percentage of members with a depression risk score below a certain threshold that are not diagnosed or treated with depression in year two. The threshold can be selected based on the resources available for depression outreach or screening. Tables 2 through 4 below show the positive and negative predictive values at different depression risk score thresholds for the commercial, Medicaid, and Medicare predictive models when tested against the validation set.

January 30, 2017

5 Offices in Principal Cities Worldwide

Milliman Depression Predictive Modeling Report – January 2017

Table 2: Positive and negative predictive values for commercial model Depression Risk Score Threshold

Members with Members with Scores Depression Risk Scores Above Threshold with Above Threshold Depression in Year Two

Positive Predictive Value

Negative Predictive Value

65%

23

12

52%

95%

60%

49

17

35%

95%

55%

87

28

32%

95%

50%

155

50

32%

95%

45%

244

73

30%

95%

40%

378

106

28%

95%

35%

625

176

28%

95%

30%

1,016

273

27%

95%

25%

1,746

432

25%

95%

20%

3,068

702

23%

96%

15%

6,141

1,165

19%

96%

10%

15,666

2,245

14%

97%

5%

68,734

4,652

7%

99%

0%

102,940

5,036

5%

Table 3: Positive and negative predictive values for Medicaid model Depression Risk Score Threshold

Members with Members with Scores Depression Risk Scores Above Threshold with Depression in Year Two Above Threshold

Positive Predictive Value

Negative Predictive Value

65%

161

92

57%

92%

60%

259

136

53%

92%

55%

383

193

50%

93%

50%

576

273

47%

93%

45%

837

367

44%

93%

40%

1,277

534

42%

93%

35%

1,935

761

39%

93%

30%

3,014

1,115

37%

94%

25%

4,812

1,603

33%

94%

20%

8,189

2,304

28%

95%

15%

14,492

3,257

22%

96%

10%

26,562

4,482

17%

98%

5%

40,970

5,201

13%

99%

0%

70,214

5,385

8%

January 30, 2017

6 Offices in Principal Cities Worldwide

Milliman Depression Predictive Modeling Report – January 2017

Table 4: Positive and negative predictive values for Medicare model Depression Risk Score Threshold

Members with Members with Scores Depression Risk Scores Above Threshold with Above Threshold Depression in Year Two

Positive Predictive Value

Negative Predictive Value

65%

94

58

62%

87%

60%

169

90

53%

87%

55%

295

139

47%

87%

50%

518

232

45%

87%

45%

840

356

42%

88%

40%

1,388

550

40%

88%

35%

2,266

811

36%

88%

30%

3,938

1,306

33%

88%

25%

6,769

2,041

30%

89%

20%

13,802

3,514

25%

90%

15%

29,678

6,041

20%

92%

10%

64,867

9,331

14%

95%

5%

79,067

10,091

13%

100%

0%

79,068

10,091

13%

As seen in the above tables, a threshold of 30% would allow for the screening of about 1,000 commercial members, 3,000 Medicaid members, and 4,000 Medicare members. About 27% of commercial members, 37% of Medicaid members, and 33% of Medicare members with scores at or above 30% were diagnosed with or treated for depression in year two. Among members with scores below 30%, 95% of commercial members, 94% of Medicaid members, and 88% of Medicare members were not diagnosed with or treated with depression in year two.

C-Statistic The c-statistic is the probability that the model predicts outcomes better than chance, with values ranging from 0.5 (no better than chance) to 1.0 (predicts outcomes perfectly). A model that scores 0.7 or higher is typically considered reasonable, and a model that scores 0.8 or higher is considered strong. Most of the model configurations that we tested scored similarly within each population type, although the Medicaid versions tended to have the highest scores, followed by commercial and Medicare. The c-statistics for each of the models for members who had medical or prescription drug claims in year one are presented in Table 5 below. Table 5: C-statistics for predictive models by population type Line of Business

c-statistic

Commercial

0.760

Medicaid

0.822

Medicare

0.673

As evidenced above, the commercial and Medicaid models appear strong, while the Medicare model is slightly below our preferred reasonability threshold.

January 30, 2017

7 Offices in Principal Cities Worldwide

Milliman Depression Predictive Modeling Report – January 2017

CONCLUSION The primary goal of the depression predictive models developed is to identify the members most likely to be impacted by depression so that SIM practices can focus on these individuals for screening and treatment in order to have the greatest impact with limited resources. The results of these initial model versions for each of the commercial, Medicaid, and Medicare populations show that a significant amount of lift in identifying members with undiagnosed depression can be gained by screening members in high-risk cohorts as opposed to randomly screening the entire population. We can run these models through the most recent APCD data for all attributed SIM members for each SIM practice and develop scored lists of members based on their probability of having depression. Even for members identified as having a high risk of being diagnosed with or treated for depression in year two that did not actually have depression in year two, interventions may still be impactful to address their unmet healthcare needs. Further, some of these members may be diagnosed with or treated for depression in later years not captured by our current analysis. Thus, it is likely that the positive predictive values presented in Tables 2 through 4 above understate the outcomes that could be achieved under a depression screening and outreach program. Following this analysis, there are a few next steps that could be taken to improve the usefulness of predictive modeling for SIM practices: 

Incorporating data from behavioral health organizations (BHOs) in Colorado to supplement restricted behavioral health claims in the APCD due to 42 CFR Part 2.



Getting feedback from SIM practices regarding other behavioral conditions that could be useful to target in predictive modeling, as well as how to best deliver the results to practices.

CAVEATS This report was prepared for the Colorado SIM office for use in identifying individuals with potentially undiagnosed depression, and should not be shared externally without prior written consent from Milliman. Other uses are inappropriate. These results were developed using the All Payer Claims Database (APCD), received from CIVHC on September 27, 2016. We have not audited the data, but have reviewed it for reasonability. Federal regulation restricts behavioral healthcare claims in the APCD, which may impact the presented modeling approaches and results. To the extent that the historical experience used to develop these models differs from future claim patterns, the modeled results will be inaccurate. Milliman does not intend to benefit or create a legal duty towards any third party recipient of this work. Guidelines issued by the American Academy of Actuaries require actuaries to include their professional qualifications in all actuarial communications. Steve Melek is a member of the American Academy of Actuaries, and meets the qualification standards for performing this analysis.

January 30, 2017

8 Offices in Principal Cities Worldwide

Milliman Depression Predictive Modeling Report – January 2017

APPENDIX Table A1 Table A1 lists the criteria used to flag whether or not a member has depression in the predictive model. Table A1: Depression identification criteria Claim Type

Criteria

Medical

Diagnosis codes 311.x, 296.2x, 296.3x, and 300.4x

Pharmacy

Therapeutic class 69 (Anti-Depressants)

Table A2 Table A2 lists the age/gender variables chosen in the commercial model for members that did not have a medical or prescription drug claim. Table A2: Variables in the commercial non-claimants model Number 1

Variable Age 0-5

Number 5

Variable Male Age 11-13

2

Age 6-10

6

Male Age 14-70

3

Female Age 41-50

7

Male Age 71+

4

Female Age 71+

Table A3 Table A3 lists the age/gender variables chosen in the Medicaid model for members that did not have a medical or prescription drug claim. Table A3: Variables in the Medicaid non-claimants model Number 1

Variable Age 0-5

Number 6

Variable Female Age 61+

2

Age 6-10

7

Male Age 11-13

3

Female Age 22-30

8

Male Age 14-21

4

Female Age 31-45 Female Age 46-60

9

Male Age 46-56

5

January 30, 2017

9 Offices in Principal Cities Worldwide

Milliman Depression Predictive Modeling Report – January 2017

Table A4 Table A4 lists the age/gender variables chosen in the Medicare model for members that did not have a medical or prescription drug claim. Table A4: Variables in the Medicare non-claimants model Number 1

Variable Female Age 20-28

Number 3

Variable Female Age 65-80

2

Female Age 29-64

4

Male Age 20-64

Table A5 Table A5 lists the variables chosen in the commercial model for members that had medical or prescription drug claims. Table A5: Variables in the commercial claimants-only model Number 1

Variable Age 0-5

Number 22

2

Age 6-10

23

Variable 1 Inpatient Visit 2+ Inpatient Visits

3

Annual Medical Cost Category

24

Insomnia

4

Annual Out of Pocket Cost Category

25

Kidney Condition

5

Anxiety Disorder or Treatment

26

Male Age 11-13

6

Asthma

27

Male Age 14-70

7

Concussion

28

Male Age 71+

8

Chronic Obstructive Pulmonary Disorder

29

Maternity Visit

9

Developmental Disorder

30

Migraines

10

Drug or Alcohol Abuse

31

Pain Condition

11

Eating Disorder

32

Psychiatric Disorder

12

Erectile Dysfunction Medication

33

Psychotic Disorder

13

Emergency Visit Category

34

Risk Score 0.5-1

14

Female Age 11-13

35

Risk Score 1-2

15

Female Age 24-40

36

Risk Score 2-10

16

Female Age 51-70

37

Risk Score 10+

17

Female Age 71+

38

Sleeping Pills

18

Heart or Blood Pressure Condition

39

Surgeries 1-2

19

Irritable Bowel Syndrome or Menopause

40

Surgeries 3+

41

Weight Loss Drugs or Attention Deficit Disorder

20 21

Infertility Injury, Arthritis, or Osteoporosis

January 30, 2017

10 Offices in Principal Cities Worldwide

Milliman Depression Predictive Modeling Report – January 2017

Table A6 Table A6 lists the variables chosen in the Medicaid model for members that had medical or prescription drug claims. Table A6: Variables in the Medicaid claimants-only model Number 1

Variable Abuse

Number 22

Variable 2+ Inpatient Visits

2

Age 6-10

23

Insomnia

3

Annual Medical Cost Category

24

Intellectual Disability

4

Annual Out of Pocket Cost Category

25

Kidney Condition

5

Antisocial Disorder

26

Male Age 11-13

6

Anxiety Disorder or Treatment

27

Male Age 14-21

7

Asthma

28

Male Age 22-45

8

Chronic Obstructive Pulmonary Disorder

29

Male Age 46-56

9

Drug or Alcohol Abuse

30

Male Age 57-plus

10

Eating Disorder

31

Migraines

11

32

Pain Condition

12

Emergency Visit Category Female Age 11-13

33

Psychiatric Disorder

13

Female Age 14-21

34

Psychotic Disorder

14

Female Age 22-30

35

Risk Score 0.5-1

15

Female Age 31-45

36

Risk Score 1-2

16

Female Age 46-60

37

Risk Score 2-10

17

Female Age 61+

38

Risk Score 10+

18

Heart or Blood Pressure Condition

39

Sleeping Pills

19

Irritable Bowel Syndrome or Menopause

40

Stroke

20

Injury, Arthritis, or Osteoporosis

41

Weight Loss Drugs or Attention Deficit Disorder

21

1 Inpatient Visit

January 30, 2017

11 Offices in Principal Cities Worldwide

Milliman Depression Predictive Modeling Report – January 2017

Table A7 Table A7 lists the variables chosen in the Medicare model for members that had medical or prescription drug claims. Table A7: Variables in the Medicare claimants-only model Number 1

Variable Annual Medical Cost Category

Number 21

Variable 2+ Inpatient Visits

2

Annual Out of Pocket Cost Category

22

Insomnia

3

Anxiety Disorder or Treatment

23

Intellectual Disability

4

Asthma

24

Kidney Condition

5

Cancer

25

Male Age 20-64

6

1-14 Chiropractor Visits

26

Male Age 65-80

7

15+ Chiropractor Visits

27

Male Age 81+

8

Chronic Obstructive Pulmonary Disorder

28

Migraines

9

Developmental Disorder

29

10

Drug or Alcohol Abuse

30

Obesity Pain Condition

11

Eating Disorder

31

Psychiatric Disorder

12

Erectile Dysfunction Medication

32

Psychotic Disorder

13

33

Risk Score 0.5-1

14

Emergency Visit Category Female Age 20-28

34

Risk Score 2-10

15

Female Age 29-64

35

Risk Score 10+

16

Female Age 65-80

36

Sleeping Pills

17

Heart or Blood Pressure Condition

37

Stroke

18

38

3+ Surgeries

19

Irritable Bowel Syndrome or Menopause Injury, Arthritis, or Osteoporosis

39

Transplant

20

1 Inpatient Visit

40

Weight Loss Drugs or Attention Deficit Disorder

January 30, 2017

12 Offices in Principal Cities Worldwide