Misspecification Effects in Zero-Inflated Negative Binomial Regression Models: Common Cases Andrew J. Civettini, University of Iowa Eric Hines, University of Iowa

This paper has been prepared for the 2005 Annual Meeting of the Southern Political Science Association, January 5-8, 2005 in New Orleans, Louisiana.

ABSTRACT We report the results of a Monte Carlo simulation testing the effects of two common types of misspecification in zero-inflated negative binomial regression models. The first type of misspecification occurs when a variable that contributes to the inflation portion of the model is omitted from the estimation. The second type occurs when that same variable is incorrectly included in the negative binomial portion of the model rather than the inflation portion. We find in both types that the majority of the bias created by the misspecification is captured by the constant values of the two portions of the model, and not by the coefficients on the independent variables of interest. In cases where bias is exhibited in coefficients on independent variables, that bias is against finding significance. Our findings suggest that with respect to these two common types of misspecification, coefficients on independent variables of interest are unbiased and can be reported with confidence.

1

While count models in general have become more commonly employed in political science, zero-inflated count models are rare in published studies of political science. Count models are employed in lieu of ordinary least squares (OLS) when the data are described by the number of times some occurrence has taken place. Count data by definition are bounded at zero and discrete (non-negative integer values), and thus violate the standard OLS assumption of continuity in the dependent variable. Except in cases where the counts are all large and non-zero, estimating count data with OLS leads to bias and inefficiency of the estimators. King (1988) shows that an exponential Poisson distribution generates count data, and Cameron and Trivedi (1986) demonstrate the superiority of exponential Poisson estimators compared with OLS estimators. In cases where the conditional variance is greater than the conditional mean, known as overdispersion, the underlying distribution is negative binomial rather than Poisson (Cameron and Trivedi 1986; Long 1997). The zero-inflated family of models was developed by Mullahy (1986) and extended by Lambert (1992) and Greene (1994). Zero-inflated count models are appropriate when some observations have no chance of experiencing the event. In other words, there is one process that determines whether a unit is likely to experience the event at all, and a second process determining the number of times that unit experiences the event, assuming it is the type that does in fact experience the event. The numbers of zeros is said to be “inflated” by the process that determines whether or not a unit is likely to experience the event, since in non-inflated count data some units may experience a zero count and thus not all zeros may describe those who are not likely to experience the event. Zero-inflated negative binomial (ZINB) regression can be useful for explaining

2

political phenomena of the count variety where one set of factors accounts for whether the unit of analysis is likely to experience the event at all, and a separate set of factors accounts for how many times. For example, Hines and Civettini (2004) utilize ZINB regression to explain which Members of Congress get appointed to conference committees and also how often they are appointed in a session. In this paper, we report the results of a Monte Carlo simulation testing the effects of two common types of misspecification in ZINB regression models. The first type of misspecification occurs when a variable that contributes to the inflation portion of the model is omitted from the estimation. The second type occurs when that same variable is incorrectly included in the negative binomial portion of the model rather than the inflation portion. We find in both types that the majority of the bias created by the misspecification is captured by the constant values of the two portions of the model, and not by the coefficients on the independent variables of interest. In cases where bias is exhibited in coefficients on independent variables, that bias is against finding significance. Our findings suggest that with respect to these two common types of misspecification, coefficients on independent variables of interest are unbiased and can be reported with confidence.

METHODS Monte Carlo experiments are useful tools for examining the properties of estimators, particularly when analytic solutions are complex or intractable. King (1988) utilized Monte Carlo experiments to demonstrate that count data were best characterized as resulting from an exponential Poisson process. Signorino (1999) employed Monte

3

Carlo simulation to demonstrate the shortcomings of logit for the statistical analysis of international conflict. DeBoef and Granato (1997) demonstrated that near-integrated data exhibited many of the properties of non-stationarity as integrated data, showing that the common assumption that near-integrated data were stationary is not founded. Monte Carlo simulation is best described by Mooney (1997). A pseudopopulation is constructed and set sampling procedures are defined. Then, estimates are drawn by sampling from the pseudo-population. This is repeated a number of times, and the resulting estimates are examined for the properties under question. In our case, our pseudo-population is generated according to a process characterized by zero-inflated negative binomial. The inflation portion of the model specifies whether an observation is likely to experience any event, that is, whether it is or is not likely to be subject to the process that determines the count. To generate a pseudo-population, 1000 observations are generated according to a ZINB process. First, a count value is generated for each of the observations, and that count is a function of two independent variables, hereafter X1 and X2, a constant and random error. X1 is distributed normally and X2 is distributed uniformly. Next, a selection value is generated for each of the observations, and that selection value is a function of two independent variables, hereafter X3 and X4, a constant and random error. X3 is distributed uniformly and X4 is distributed normally. When the selection value exceeds zero, the variable takes on the value of the count generated in the first process. When the selection variable is equal or less than zero, then the variable takes on a value of zero. Thus, an observation can take on a zero value either by being in the subset of observations that do not experience a count (less than or equal to zero in the inflation) or by having experienced a zero count.

4

After creating the pseudo-population, we estimate three models on the resulting data. The first model is correctly specified, in that X1 and X2 are included in the negative binomial portion of the model estimation, and X3 and X4 are included in the inflation portion of the model. This model will hereafter be referred to as the True Model. We then estimate two common types of misspecification in ZINB regression models. The first type of misspecification occurs when a variable that contributes to the inflation portion of the model is omitted from the estimation. In this case, X1 and X2 are included in the negative binomial portion of the model estimation, and X3 is included in the inflation portion. This model will hereafter be referred to as the Omitted Model. The second type of misspecification occurs when that same variable that was previously omitted is instead incorrectly included in the negative binomial portion of the model rather than the inflation portion. In this case, X1, X2, and X4 are included in the negative binomial portion of the model estimation, and X3 is included in the inflation portion. This model will hereafter be referred to as the Misspecified Model. It is possible for zero-inflated models to perform poorly when the number of allzeros, or observations that are described as not being in the group that has the possibility of experiencing the event, is either very low (or very high). This stems from the relatively small number of zeros (or ones) in the logit estimation of the inflation portion of the model. Logistic regression has been shown to have estimation inefficiencies at the tail ends of the distribution. As such, we vary the size of the constant in the process that generates the selection value. This corresponds rather nicely with the overall size of the all-zeros population, and as such is a convenient way to alter the size of that group without altering the coefficients on the independent variables. We varied the constant in

5

the selection process for a range of twenty values, generating all-zeros populations ranging from roughly 50 to roughly 950 out of the 1000 observations. The variable X4 is either omitted or otherwise incorrectly specified in out second and third estimations of the pseudo-population. We are interested in the bias that results in interpretation of the remaining coefficients. It is likely that the bias may vary with the size of the coefficient on X4, or how important X4 is in explaining the selection process that generates the pseudo-population. As such, we vary the true coefficient on X4 for a range of sixteen values, eight positive and eight negative. This results in 320 combinations of the coefficient on X4 and the constant of the selection process. For each of these 320 combinations, we generated 1000 Monte Carlo trials. In each trial, a pseudo-population was generated, the three models were estimated, and the resulting values were stored in a vector. The means of the values for 1000 trials are the resulting Monte Carlo estimates of the parameters of interest. In the next section, we detail our results.

RESULTS Five parameters are comparable across the three model estimations: both the alpha coefficient of the inflation as well as the alpha coefficient of the negative binomial portion of the model, the coefficients on X1 and X2 from the negative binomial portion, and the coefficient on X3 from the inflation portion of the three models. For each parameter, we report the bias in magnitude, standard error, and interpretation between the two models with specification error and the True Model. For ease of presentation, only bias in interpretation will be presented graphically. The most important coefficients in any model of statistical estimation, from a social science perspective, are typically the

6

coefficients on the independent variables. Thus we will begin with examining the coefficients on X1 and X2 from the negative binomial portion of the model. The results for the coefficient estimates on X1 and X2 are consistent when comparing the Omitted Model to the True Model. The magnitudes of the coefficient estimates are biased upwards in the Omitted Model by a slight degree, and those magnitudes increase as the size of the true coefficient on the omitted variable increases. The biases decrease slightly as the size of the all-zeros population increases, though this effect is less apparent than the effect of the true value of the coefficient on X4 from the generation of the pseudo-population. The same pattern holds for the standard error of the coefficient estimates on X1 and X2 for the Omitted Model. There are statistically significant effects of the size of the true coefficient on the omitted variable on the magnitudes of the biases. The magnitudes of the biases in the estimates of the standard errors of the coefficients on X1 and X2 increase with an increase in the value of the true coefficient on the omitted variable at a greater rate than do the biases on the coefficient estimates. This results in a bias against finding significance increasing in the true coefficient on the omitted variable, as shown in Figures 1 and 2 for the Omitted Model on X1 and X2, respectively. As the magnitude of the coefficient on X4 from the generation of the pseudo-population increases, the biases in interpretation of the coefficient estimates on X1 and X2 for the Omitted Model increase and the resulting biases are against finding significance. The biases against significance also increase slightly as the size of the all-zeros population increases, which suggests that there may be increases biased against significance precisely when it is most appropriate to employ ZINB

7

regression. This is attenuated by an increased bias towards finding significance at the tail ends of the distribution of values of all-zeros. [INSERT FIGURE 1 HERE] [INSERT FIGURE 2 HERE] The results for the coefficient estimates on X1 and X2 are also consistent when comparing the Misspecified Model to the True Model, and those results are strikingly comparable to the results for the Omitted Model. The magnitudes of the coefficient estimates are biased upwards in the Misspecified Model by a slight degree, and those magnitudes increase as the size of the true coefficient on the misspecified variable increases. The biases decrease slightly as the size of the all-zeros population increases, though this effect is less apparent than the effect of the true value of the coefficient on X4 from the generation of the pseudo-population. The same pattern holds for the standard errors of the coefficient estimates on X1 and X2 for the Misspecified Model. There are statistically significant effects of the size of the true coefficient on the misspecified variable on the magnitudes of the biases. The magnitudes of the biases in the estimates of the standard errors of the coefficients on X1 and X2 increase with an increase in the value of the true coefficient on the misspecified variable at a greater rate than does the biases on the coefficient estimates. This results in biases against finding significance increasing in the true coefficient on the misspecified variable, as shown in Figures 3 and 4 for the Misspecified Model on X1 and X2, respectively. As the magnitude of the coefficient on X4 from the generation of the pseudo-population increases, the biases in interpretation of the coefficient estimates on X1 and X2 for the Misspecified Model increase and the resulting biases are against finding significance. The biases against

8

significance also increase slightly as the size of the all-zeros population increases, which suggests that there may be increases biased against significance precisely when it is most appropriate to employ ZINB regression. Much like in the Omitted Model, this is attenuated by the increased bias towards finding significance at the tail ends of the distribution of values of all-zeros. However, the finding that the biases increase as the magnitude of the true coefficient on the misspecified variable, or the omitted variable in the first case of specification error, increases is one that deserves additional consideration. [INSERT FIGURE 3 HERE] [INSERT FIGURE 4 HERE] The results for the coefficient estimates on X3 for the two models are not as immediately clear. For both the Omitted Model and the Misspecified Model, the magnitudes of the biases in the coefficient estimates on X3 increase as the magnitude of the true coefficient on X4, the variable either omitted or misspecified, increases. The magnitudes of the biases in the estimates of the standard errors increase as well in increasing true values on X4. Finally, the magnitudes of the biases increase as the size of the all-zeros population increases. This could result from problems of estimation rather than problems of specification. The biases of the standard errors from both the Omitted and Misspecified Model on X3 increase at a greater rate than do the biases on the coefficient estimates. Thus, the bias in interpretation decreases as the value of the true coefficient on X4 from the pseudo-population increases. This result, a bias against significance, is much more striking than in the negative binomial portion of the model, and is consistent for both the Omitted and Misspecified Model. The bias in interpretation slightly increases as the size of the all-zeros population increases. This could result from

9

a problem of estimation when there are few zeros in the logistic regression that characterizes the estimation of the inflation portion of the model. The biases in interpretation of X3 for the Omitted Model and the Misspecified Model, respectively, are shown in Figures 5 and 6. [INSERT FIGURE 5 HERE] [INSERT FIGURE 6 HERE] The estimate of the alpha coefficient of the negative binomial portion of the Omitted Model is biased in magnitude. The magnitude of the bias of the estimate of the alpha coefficient increases as the value of the true coefficient on X4 from the generation of the pseudo-population increases. The magnitude of the bias also increases at relatively high and low ends of the distribution of all-zeros. For purposes of interpretation, the bias of the estimate of alpha on the negative binomial portion of the Omitted Model is usually biased against finding significance, except in those cases where the proportion of allzeros is particularly high or low. The bias in interpretation of the alpha coefficient of the negative binomial portion of the Omitted Model is shown in Figure 7. This suggests that with regard to the error of omission, the alpha coefficient of the negative e binomial portion of the model estimation is biased against significance, though the degree depends both on the importance of the omitted variable in determining the inflation portion of the model and the extremity of the proportion of all-zeros. [INSERT FIGURE 7 HERE] The estimate of the alpha coefficient of the negative binomial portion of the Misspecified Model is biased in magnitude. The magnitude of the bias of the estimate of the alpha coefficient increases as the result of the interaction between the size of the all-

10

zeros population and the value of the true coefficient on X4 from the generation of the pseudo-population. As that interaction increases, so does the magnitude of the bias. Both the size of the all-zeros population as well as the true value on X4 have primary effects, but the largest effect is exhibited by the interaction. The estimate of the standard error of the alpha coefficient exhibits a similar pattern based on the interaction of the size of the all-zeros population and the true value on X4, but the magnitude of the effect is smaller than for the coefficient. This results in an increasing amount of bias of interpretation as the either the size of the all-zeros population increases or the true coefficient on X4 increases. The bias in interpretation of the alpha coefficient of the negative binomial portion of the Misspecified Model is shown in Figure 8. This suggests that with regard to the error of misspecification, the alpha coefficient of the negative binomial portion of the model estimation is biased towards finding significance, though the degree depends both on the importance of the omitted variable in determining the inflation portion of the model and the proportion of all-zeros, as well as the interaction of the two. [INSERT FIGURE 8 HERE] There are no solid, interpretable patterns resulting from the alpha of the inflation estimates in either the Omitted or Specified Model. The bias of interpretation from the Omitted Model is shown in Figure 9, and for the Misspecified Model in Figure 10. The estimates are problematic when the true alpha in the data generation of the pseudopopulation is close to zero. This is due to a series of calculations dividing by small numbers, and as a result of random error these values are often at or near zero. In general, the biases of interpretation decrease as the magnitude of the true coefficient on X4

11

increases, and the biases decrease as the size of the all-zeros population increases. These results are tentative at best and may need independent evaluation at much greater depth through additional Monte Carlo simulations. [INSERT FIGURE 9 HERE] [INSERT FIGURE 10 HERE]

DISCUSSION AND CONCLUSIONS The most important finding of this Monte Carlo analysis, from the perspective of social science research, is that the coefficients on the independent variables in the negative binomial portion of the model are biased against finding significance with both types of misspecification. This implies that previous studies employing ZINB regression can be confident in the significance of their estimates even in the face of criticism that a variable has been left out of the inflation o incorrectly specified as part of the count process when it rightfully belongs in the inflation. However, the results are less direct or clear with respect to the independent variables in the inflation portion of the model. More analysis is warranted on this question and whether the same degree of confidence can be assigned to coefficient estimates in the inflation portion of the model. The bias seems to be captured in large part by some combination of the alpha coefficients. This makes intuitive sense and is a demonstration that the constant portion of the model is behaving as anticipating by capturing the effects not otherwise specified in the model. However, this analysis suggests more research is necessary regarding the nature of not merely the alpha coefficients but all coefficients at the tail ends of the distribution of all-zeros.

12

Future Monte Carlo analyses of ZINB regression should focus on several important considerations. First, this analysis does not ask what happens to the coefficients when a variable that belongs in he negative binomial portion of the model has been omitted from that portion of the model, nor what happens to the coefficient estimates when that variable is incorrectly specified as contributing to the inflation portion of the model. Further, this analysis ignores the possibility that an independent variable may contribute simultaneously to the inflation portion of the model as well as the negative binomial portion of the model. Finally, a separate Monte Carlo analysis is warranted to examine the tail ends of the all-zero distribution to develop a clear analytical picture of what level of all-zeros leads to biased estimation.

13

14

Bias in Interpretation of Mean NB Beta Estimates on X1 .9 1 1.1 1.2 1.3

0

800 True Beta on X4 =1.1 True Beta on X4 = .3

400 600 Mean Size of All-Zeros Population

True Beta on X4 = 1.5 True Beta on X4 = .7

200

Figure 1. Bias in Interpretation of Mean NB Beta Estimates on X1: Omitted Model

1000

15

Bias in Interpretation of Mean NB Beta Estimates on X2 .95 1 1.05 1.1 1.15

.9

0

800 True Beta on X4 =1.1 True Beta on X4 = .3

400 600 Mean Size of All-Zeros Population

True Beta on X4 = 1.5 True Beta on X4 = .7

200

Figure 2. Bias in Interpretation of Mean NB Beta Estimates on X2: Omitted Model

1000

16

Bias in Interpretation of Mean NB Beta Estimates on X1 .9 1 1.1 1.2 1.3

0

800 True Beta on X4 =1.1 True Beta on X4 = .3

400 600 Mean Size of All-Zeros Population

True Beta on X4 = 1.5 True Beta on X4 = .7

200

Figure 3. Bias in Interpretation of Mean NB Beta Estimates on X1: Misspecified Model

1000

17

Bias in Interpretation of Mean NB Beta Estimates on X2 .9 .95 1 1.05 1.1 1.15

0

800 True Beta on X4 =1.1 True Beta on X4 = .3

400 600 Mean Size of All-Zeros Population

True Beta on X4 = 1.5 True Beta on X4 = .7

200

Figure 4. Bias in Interpretation of Mean NB Beta Estimates on X2: Misspecified Model

1000

18

Bias in Interpretation of Mean Inflation Beta Estimates on X3 0 .5 1 1.5 2

0

800 True Beta on X4 =1.1 True Beta on X4 = .3

400 600 Mean Size of All-Zeros Population

True Beta on X4 = 1.5 True Beta on X4 = .7

200

Figure 5. Bias in Interpretation of Mean Inflation Beta Estimates on X3: Omitted Model

1000

19

Bias in Interpretation of Mean Inflation Beta Estimates on X3 0 1 2 3 4 5

0

800 True Beta on X4 =1.1 True Beta on X4 = .3

400 600 Mean Size of All-Zeros Population

True Beta on X4 = 1.5 True Beta on X4 = .7

200

Figure 6. Bias in Interpretation of Mean Inflation Beta Estimates on X3: Misspecified Model

1000

20

Bias in Interpretation of Mean NB Alpha Estimates .8 .9 1 1.1

.7

0

800 True Beta on X4 =1.1 True Beta on X4 = .3

400 600 Mean Size of All-Zeros Population

True Beta on X4 = 1.5 True Beta on X4 = .7

200

Figure 7. Bias in Interpretation of Mean NB Alpha Estimates: Omitted Model

1000

21

2.5

Bias in Interpretation of Mean NB Alpha Estimates 1.5 2

1

0

800 True Beta on X4 =1.1 True Beta on X4 = .3

400 600 Mean Size of All-Zeros Population

True Beta on X4 = 1.5 True Beta on X4 = .7

200

Figure 8. Bias in Interpretation of Mean NB Alpha Estimates: Misspecified Model

1000

22

Bias in Interpretation of Mean Inflation Alpha Estimates 0 1 2 3 4 5

0

800 True Beta on X4 =1.1 True Beta on X4 = .3

400 600 Mean Size of All-Zeros Population

True Beta on X4 = 1.5 True Beta on X4 = .7

200

Figure 9. Bias in Interpretation of Mean Inflation Alpha Estimates: Omitted Model

1000

23

Bias in Interpretation of Mean Inflation Alpha Estimates -1 0 1 2 3 4

0

800 True Beta on X4 =1.1 True Beta on X4 = .3

400 600 Mean Size of All-Zeros Population

True Beta on X4 = 1.5 True Beta on X4 = .7

200

1000

Figure 10. Bias in Interpretation of Mean Inflation Alpha Estimates: Misspecified Model

WORKS CITED DeBoef, Suzanna and Jim Granato (1997). “Near-Integrated Data and the Analysis of Political Relationships.” American Journal of Political Science 41(2): 619640. Cameron, A. Colin and Pravin K. Trivedi (1986). “Econometric Models Based on Count Data: Comparisons and Applications of Some Estimators and Tests.” Journal of Applied Econometrics 1(1): 29-53. Greene, William H. “Accounting for Excess Zeros and Sample Selection in Poisson and Negative Binomial Regression Models.” Working Paper No. 94-10. New York: New York University Department of Economics. Hines, Eric and Andrew J. Civettini (2004). “Strategic Selection of Conferees in the United States Senate" Paper presented at the annual meeting of the The American Political Science Association, Chicago, IL. King, Gary (1988). “Statistical Models for Political Science Event Counts: Bias in Conventional Procedures and Evidence for the Exponential Poisson Regression Model.” American Journal of Political Science 32: 838-863. Lambert, Diane (1992). “Zero-inflated Poisson Regression, With an Application to Defects in Manufacturing.” Technometrics 34(1): 1-14. Long, J. Scott (1997). Regression Models for Categorical and Limited Dependent Variables. Thousand Oaks, CA: Sage Publications. Mooney, Christopher Z. (1997). Monte Carlo Simulation. Thousand Oaks, CA: Sage Publications. Mullahy, John (1986). “Specification and Testing of Some Modified Count Data Models.” Journal of Econometrics 33: 341-365. Signorino, Curtis S. (1999). “Strategic Interaction and the Statistical Analysis of International Conflict.” American Political Science Review 93(2): 279-297.

24