Using an Immunization Information System for Assessment, Feedback, Incentives, and Exchange (AFIX) Program Site Visits National Immunization Conference September 29, 2014 Bianca Perri, MPH New York City Department of Health and Mental Hygiene Bureau of Immunization

Outline • Background – – – –

Vaccines for Children (VFC) Program in New York City (NYC) NYC’s Immunization Information System (IIS) AFIX site visits in NYC Preparing for IIS-based AFIX site visits

• Steps for conducting IIS-based AFIX site visits • NYC immunization coverage at AFIX site visit and follow-up • Challenges, conclusions, and next steps

BACKGROUND

NYC and the VFC Program • NYC population estimate is 8.2 million – 1.9 million persons are 0-18 years old

• 74% of NYC 0-18 year-olds eligible for publicly funded vaccines – 66% VFC – 1% 317 (underinsured at non-Federally Qualified Health Centers) – 7% New York State Child Health Plus

• 1,436 (80%) of ~1,800 pediatric provider sites enrolled and active in VFC • Approximately 3.3 million VFC vaccine doses costing $138.6 million were distributed in 2013

NYC’s IIS: Citywide Immunization Registry (CIR) • State and City reporting mandate for immunizations administered to patients 0-18 years old • IIS reporting linked to VFC distribution in 2006 – Developed IIS-generated Doses Administered Report (DAR): –

# publicly funded doses administered and reported to IIS in year # publicly funded doses received in year

– Providers with DAR <90% subject to vaccine order reductions – 70% increase in IIS reporting, resulting in more complete data • 5.3 million patient records and 72 million immunizations (Sep 2014) • Immunization coverage within confidence intervals of National Immunization Survey

AFIX Site Visits in NYC • AFIX visits conducted among VFC providers to improve immunization coverage – Prioritize sites with low childhood series coverage

• Before 2006, AFIX assessments based on chart review – Piloted using IIS for AFIX visits at the end of 2006 – Transitioned to 100% AFIX visits via IIS by July 2010 – Expanded age groups of populations in 2012-2013

• In 2008, AFIX and VFC field operations teams merged – Conduct combined AFIX and VFC compliance site visits – 2/3 of VFC site visits include AFIX

Preparing to Implement IIS-based AFIX Site Visits • Develop a user-friendly tool to assess immunization coverage at provider sites – In NYC: Web Up-To-Date (UTD) application

• Train field staff to use Web UTD • Improve existing patient de-duplication process in IIS • Add fields to IIS user interface (Online Registry) – Moved or Gone Elsewhere (MOGE) – Disease history

Additional Fields in NYC’s IIS Online Registry MOGE Field

Disease History Field

NYC’s Web UTD Application • Generates IIS-based immunization coverage and reminder/recall reports by facility according to parameters: – Facility – Age cohort • All 19-35 month-olds and 13-17 year-olds

– Population inclusion criteria • Child received last series immunization at that facility • After a certain age for each patient – 1 year (361 days) for 19-35 month-olds – 9 years (3,287 days) for 13-17 year-olds

– Date coverage is run • Can be run as of a date in the past

– Type and number of antigens for UTD

Web UTD Application Screenshots

CONDUCTING IIS-BASED AFIX SITE VISITS

AFIX- Assessment • Identify facility’s patient list and merge duplicate immunization records in the IIS prior to assessing immunization coverage • Run immunization coverage reports using IIS-based Web UTD application • Review Web UTD reports – Summary statistics • Number of children assessed • Number and percentage of children UTD

– List of children for reminder/recall • Names of children missing ≥1 shot from specified series • Type and dose number of missing shot • Patients receiving an invalid dose

• Identify patterns of noncompliance with immunization schedule based on IIS clinical decision support

2013 Example: Facility Immunization Coverage Report, 19-35 month-olds Facility: 1111X11 Minimum DOB: 08/01/2011 Maximum DOB: 12/31/2012 Review Date (Immunization): 08/25/2014 Date Produced: 08/25/2014 14:19:41

FACILITY CODE 1111X11

SELECTION METHOD LAST SERIES SHOT

COUNT OF PATIENTS WITH AT TOTAL TOTAL TOTAL LEAST ONE PATIENTS PATIENTS NUMBER INVALID VACCINE (EXCLUDES (INCLUDES OF SHOT-ANY GROUP MOGE) MOGE) MOGEs FACILITY ALL

Percent UTD (last series shot) 4314314 Series % UTD: 431431 Series % UTD: 4 DTaPs: 3 Polios: 1 MMR: 4 Hibs (Full Hib Series): 3 Hep Bs: 1 Varicella: 4 PCV (Full PCV Series): 2 Hep A:

86.00% 88.00% 93.00% 98.80% 99.80% 96.10% 95.20% 99.00% 94.90% 71.80%

415

419

4

19

COUNT # OF PTs OF # OF PTs # OF PTs WITH COUNT OF INVALID WITH WITH MMR AND INVALID SHOTS- MMR < 1 VARICELLA VARICELLA SHOTS-THIS ANY YEAR OF < 1 YEAR < 28 DAYS FACILITY FACILITY AGE OF AGE APART 18

22

1

0

0

# OF PTs WITH 3rd HEPB < 6 MONTHS OF AGE

# OF PTs WITH 4th DTaP < 4 MONTHS AFTER 3rd DOSE

5

0

2013 Example: Facility Immunization Coverage Report, 13-17 year-olds Facility: 2222X02 Minimum DOB: 07/01/1996 Maximum DOB: 06/30/2001 Review Date (Immunization): 07/14/2014 Imm Age Days (Last): 3287 (Patient will be included if last shot was given in >= n days of age.) Imm Age Days (Any): 0 (Patient will be included if any shot was given in >= n days of age.) Last Shot Within Days: 10000 (Patient will be included if last shot was given within n days from Review Date.) Patient Age Days: 10000 (UTD calculated as of the age specified. Shots given after that age, will be excluded.) FACILITY CODE 2222X02 2222X02 2222X02

SELECTION METHOD LAST SERIES SHOT LAST SERIES SHOT LAST SERIES SHOT

VACCINE GROUP - ALL 1:1 (1 Tdap, 1 Meningococcal) 1 Tdap 1 Meningococcal

TOTAL PATIENTS 391 391 391

TOTAL UTD 350 362 355

PERCENT UTD 89.50% 92.60% 90.80%

FACILITY CODE 2222X02 2222X02 2222X02 2222X02

SELECTION METHOD LAST SERIES SHOT LAST SERIES SHOT LAST SERIES SHOT LAST SERIES SHOT

VACCINE GROUP - ALL 1:1:3 (1 Tdap, 1 Meningococcal, 3 HPV) 1:1:1 (1 Tdap, 1 Meningococcal, 1 HPV) 3 HPV 1 HPV

TOTAL PATIENTS 391 391 391 391

TOTAL UTD 205 318 206 325

PERCENT UTD 52.43% 81.33% 52.69% 83.12%

FACILITY CODE 2222X02 2222X02 2222X02 2222X02

SELECTION METHOD LAST SERIES SHOT LAST SERIES SHOT LAST SERIES SHOT LAST SERIES SHOT

VACCINE GROUP - GIRLS 1:1:3 (1 Tdap, 1 Meningococcal, 3 HPV) 1:1:1 (1 Tdap, 1 Meningococcal, 1 HPV) 3 HPV 1 HPV

TOTAL PATIENTS 183 183 183 183

TOTAL UTD 123 161 123 165

PERCENT UTD 67.20% 88.00% 67.20% 90.20%

FACILITY CODE 2222X02 2222X02 2222X02 2222X02

SELECTION METHOD LAST SERIES SHOT LAST SERIES SHOT LAST SERIES SHOT LAST SERIES SHOT

VACCINE GROUP - BOYS 1:1:3 (1 Tdap, 1 Meningococcal, 3 HPV) 1:1:1 (1 Tdap, 1 Meningococcal, 1 HPV) 3 HPV 1 HPV

TOTAL PATIENTS 208 208 208 208

TOTAL UTD 82 157 83 160

PERCENT UTD 39.40% 75.50% 39.90% 76.90%

2013 Example: Facility Recall List, 19-35 month-olds

Detailed List of Patients Not UTD (last series shot)

LAST FIRST GEN COUNT CIR NUMBER NAME NAME DOB DER MEDREC ***The following patients are not UTD on the series: 1 1111 A a 6/3/2012 F 100 2 1112 B b 12/18/2012 F 101 3 1113 C c 8/13/2011 M 102 4 1114 D d 12/1/2012 M 103 5 1115 E e 11/4/2011 M 104 6 1116 F f 4/18/2012 F 105 7 1117 G g 6/1/2012 F 106 8 1118 H h 8/21/2012 F 107 9 1119 I i 10/30/2012 M 108 10 1120 J j 6/9/2012 M 109 11 1121 K k 9/12/2012 F 110 12 1122 L l 10/18/2011 F 111 13 1123 M m 10/11/2012 M 112 14 1124 N n 4/11/2012 F 113 15 1125 O o 11/13/2012 M 114 16 1126 P p 4/3/2012 F 115 17 1127 Q q 5/8/2012 M 116 18 1128 R r 3/22/2012 M 117 19 1129 S s 5/30/2012 M 118

SERIES DUE

NOT UTD BUT NOT YET DUE SERIES HepA-2

HepA-2 HepA-2 DTP-4, HepA-1 HepA-2 HepA-2 HepA-2 HepA-2 Pneumo Conj-4 HepA-2 HepA-2 HepA-2 HepA-2 HepA-2

VARICEL MMR LA BEFORE BEFORE 1 YEAR 1 YEAR OF AGE OF AGE N N N N N N N N N N N N N N N N N N N

N N N N N N N N N N N N N N N N N N N

MMR AND VARICEL LA LESS THAN 28 DAYS APART N N N N N N N N N N N N N N N N N N N

THIRD THIRD AND HEPB FOURTH BEFORE DTP LESS SIX THAN MONTH FOUR S OF MONTHS AGE APART N N N N N N N N N N N N N N N N N N N

N N N N N N N N N N N N N N N N N N N

HepA-2 HepA-2 HepB-3, Pneumo Conj-3 HepA-2 HepA-2 HepB-3, Hib-4*, Varicella-1, Pneumo 20 1130 T t 12/23/2011 F 119 Conj-4, HepA-2 N N N N N *Hib requirements vary, based on age at first dose, total number of doses already received, and current age (four doses of Hib are necessary only for children aged 12 mos5 years who received 3 doses before age 12 months). UTD rates have been adjusted accordingly.

2013 Example: Facility Recall List, 13-17 year-olds CIR LAST COUNT NUMBER NAME FIRST NAME DOB ***The following patients are not UTD on the series: 1 10001 A A 09/23/00 2 10002 B B 08/11/97 3 10003 C c 07/05/97 4 10004 D d 07/05/97 5 10005 E e 04/16/00 6 10006 F f 07/19/97 7 10007 G g 12/25/98 8 10008 H h 08/25/97 9 10009 I i 03/21/01 10 10010 J j 02/11/99 11 10011 K k 05/15/01 12 10012 L l 09/14/99 13 10013 M m 06/17/00 14 10014 N n 02/09/99 15 10015 O o 03/24/00 16 10016 P p 09/13/97 17 10017 Q q 08/20/98 18 10018 R r 06/25/99 19 10019 S s 09/29/99 20 10020 T t 05/06/01

GENDER

MEDREC

SERIES DUE

M M M M M M M M F M F M M M M F M M M F

501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520

HPV-1 HPV-1 HPV-2 HPV-2

NOT UTD BUT NOT YET DUE SERIES

HPV-2 HPV-1 HPV-3 Mening.-1, HPV-1 HPV-3 HPV-3 HPV-3 HPV-3 Mening.-1, HPV-1 HPV-3 HPV-2 HPV-3 Mening.-1, HPV-1 HPV-2 Tdap, Mening.-1, HPV-1 HPV-2

AFIX- Feedback • Provide feedback during site visit – – – –

IIS-generated immunization coverage levels Areas of noncompliance with schedule Immunization practices needing quality improvement (QI) Additional recommendations for improving coverage

• Demonstrate Online Registry tools to help providers improve immunization coverage – Obtain individual patient immunization history and recommendations of immunizations due now and in the future – Generate recall lists of children not UTD – Update MOGE status

• Refer providers to internal Bureau of Immunization staff – Additional reminder/recall training – Immunization reporting problems – Vaccine ordering problems

Generating Recall List in Online Registry

AFIX- Incentives • Give providers recall lists at site visit • Recognize providers with high immunization coverage – 2013 criteria for Honor Roll • > 90% for 4 DTaP:3 polio:1 MMR:4 Hib:3 HepB :1 varicella: 4 PCV; (4:3:1:4:3:1:4) among 19-35 month-olds; • > 80% for 1 Tdap:1 MCV (1:1) among 13-17 year-olds; • > 90% VFC Doses Administered Report (DAR); and • > 10 patients in either age group

– Methods of recognition • Quarterly Childhood Coalition Meeting • American Academy of Pediatrics • Publish provider name on NYC Bureau of Immunization Web site

NYC Provider Honor Roll

AFIX- eXchange • Exchange information with providers to follow-up on their progress towards improving immunization services and coverage levels • Contact providers to discuss progress on QI activities

• Reassess coverage 3-4 months after original assessment – Coverage reassessments conducted via IIS – Sites with <90% coverage and ≥ 10 patients in either age group

• Send follow-up letter and report to show reassessment results – – – –

Immunizations since original assessment Immunization coverage at assessment vs. at follow-up Recall lists as of follow-up Progress on QI activities

• Exchange additional information when follow-up needed

2013 Example: Facility Immunization Coverage Report at Reassessment, 13-17 year-olds Facility: 1000X01 Minimum DOB: 04/01/1996 Maximum DOB: 03/31/2001 Initial AFIX Date: 04/28/2014 Follow-up AFIX Date : 07/30/2014 Imm Age Days (Last): 3287 (Patient will be included if last shot was given in >= n days of age.) Imm Age Days (Any): 0 (Patient will be included if any shot was given in >= n days of age.) Last Shot Within Days: 10000 (Patient will be included if last shot was given within n days from Review Date.) Patient Age Days: 10000 (UTD calculated as of the age specified. Shots given after that age, will be excluded.)

#Patients with shots administered since original AFIX visit (04/28/2014) by this facility #Shots administered since original AFIX visit (04/28/2014) at this facility #Shots reported since original AFIX visit (04/28/2014) at this facility #Patients MOGEd since original AFIX visit (04/28/2014)

88 107 146 68

FACILITY CODE SELECTION METHOD 1000X01 LAST SERIES SHOT 1000X01 LAST SERIES SHOT 1000X01 LAST SERIES SHOT

VACCINE GROUP - ALL 1:1 (1 Tdap, 1 Meningococcal) 1 Tdap 1 Meningococcal

TOTAL PATIENTS 435 435 435

TOTAL UTD 369 389 381

PERCENT UTD 84.80% 89.40% 87.60%

FACILITY CODE SELECTION METHOD 1000X01 LAST SERIES SHOT 1000X01 LAST SERIES SHOT 1000X01 LAST SERIES SHOT 1000X01 LAST SERIES SHOT

VACCINE GROUP - ALL 1:1:3 (1 Tdap, 1 Meningococcal, 3 HPV) 1:1:1 (1 Tdap, 1 Meningococcal, 1 HPV) 3 HPV 1 HPV

TOTAL PATIENTS 435 435 435 435

TOTAL UTD 252 321 262 343

PERCENT UTD 57.93% 73.79% 60.23% 78.85%

FACILITY CODE SELECTION METHOD 1000X01 LAST SERIES SHOT 1000X01 LAST SERIES SHOT 1000X01 LAST SERIES SHOT 1000X01 LAST SERIES SHOT

VACCINE GROUP - GIRLS 1:1:3 (1 Tdap, 1 Meningococcal, 3 HPV) 1:1:1 (1 Tdap, 1 Meningococcal, 1 HPV) 3 HPV 1 HPV

TOTAL PATIENTS 202 202 202 202

TOTAL UTD 135 156 142 168

PERCENT UTD 66.80% 77.20% 70.30% 83.20%

FACILITY CODE SELECTION METHOD 1000X01 LAST SERIES SHOT 1000X01 LAST SERIES SHOT 1000X01 LAST SERIES SHOT 1000X01 LAST SERIES SHOT

VACCINE GROUP - BOYS 1:1:3 (1 Tdap, 1 Meningococcal, 3 HPV) 1:1:1 (1 Tdap, 1 Meningococcal, 1 HPV) 3 HPV 1 HPV

TOTAL PATIENTS 233 233 233 233

TOTAL UTD 117 165 120 175

PERCENT UTD 50.20% 70.80% 51.50% 75.10%

IMMUNIZATION COVERAGE AT AFIX SITE VISIT AND FOLLOW-UP

2013 AFIX Site Visits • 483 provider sites received AFIX visit • 426 sites (88%) had AFIX assessment for both age groups

• 292 provider sites met the criteria for AFIX reassessment of immunization coverage and received a reassessment report in 2013-2014 – Mean follow-up time: 5.1 months

AFIX Assessment Site Visit vs. Post-AFIX Follow-up Immunization Coverage, NYC 2013 19-35 month-olds 100

% Complete

80

70 62

60

60 46 40

AFIX Site Visit (N=40,272 patients) Post-AFIX Follow-Up (N=40,249 patients)

20 0 4:3:1:4:3:1:4 Hepatitis A Series/Antigen

-Among 292 AFIX site visits conducted in 2013 with a subsequent follow-up in 2013-2014. -Mean follow-up time: 5.1 months

AFIX Assessment Site Visit vs. Post-AFIX Follow-up Immunization Coverage, NYC 2013 13-17 year-olds 100

% Complete

80

74 76

60 48

51

40

31

37

AFIX Site Visit (N = 121,885 patients)

36 30 17

20

11

Post-AFIX Follow-Up (N = 121,243 patients)

0

1:1 (girls 1:1:1 and boys) (girls)

1:1:1 (boys) Series

1:1:3 (girls)

1:1:3 (boys)

-Among 292 AFIX site visits conducted in 2013 with a subsequent follow-up in 2013-2014. -Mean follow-up time: 5.1 months

AFIX: 2006 vs. 2013 Number assessed Sites

Children

2006 (Chart Review)

2013 (IIS)

197 (15% of VFC sites)

483 (34% of VFC sites*)

8,001**

240,664***

* 483 of the 1,436 provider sites that were enrolled in VFC and active in 2013 ** 24-35 month olds *** 63,060 19-35 month-olds; 177,604 13-17 year-olds

Based on 2010 Census, the number of children included in 2013 AFIX assessments represent more than 1/3 of the total population in each age group

Challenges • Combined VFC/AFIX site visits cover many topics • Not seeing substantial improvements in immunization coverage among 13-17 yearolds – Recall lists for 13-17 year-olds may overwhelm providers

Conclusions • Development of Web UTD application greatly facilitated the transition from chart-based AFIX assessments to IIS-based AFIX assessments • Compared to chart review, advantages of IIS-based AFIX assessments include: – Less resources (e.g., less time for site visit, no data entry required to assess immunization coverage) – Helps improve completeness of IIS data by giving providers further incentive to report to the IIS

• IIS-based AFIX has helped improved our efficiency – Increased number of provider sites assessed – Increased number and age groups of children assessed • Age groups can be added quickly • Additional single antigens and series can be added quickly

Next Steps • Potential enhancements in 2014-2015 – Update current AFIX reminder/recall lists to include contact information – Roll-out text messaging and email-based reminder/recall via the Online Registry – Make Web UTD application more efficient for running IIS-based immunization coverage – Add missed opportunities to Web UTD – Add HPV to adolescent series for Provider Honor Roll

• Reconsider the structure of VFC/AFIX site visits – Initiate site visit with AFIX component, followed by VFC compliance? – Re-assess which providers receive an AFIX site visit?

• Conduct outcome evaluation to assess impact of AFIX site visits on immunization coverage

Acknowledgements • • • • • •

Karen Fernandez Amy Metroka Melissa Mickle-Hope Vikki Papadouka Jane R. Zucker Citywide Immunization Registry Unit and Provider Quality Assurance Unit Staff