Poker Superstars: Skill or Luck?*

Peter Fishman Department of Economics University of California, Berkeley

Devin G. Pope Department of Economics University of California, Berkeley

This Draft: December 4, 2006 (Please do not cite without authors’ permission)

Abstract Poker, a zero-sum game of incomplete information, has recently experienced a surge of popularity among both amateur and professional players. Several players who have done well in multiple tournaments have achieved celebrity status as poker “superstars.” However, because of poker’s outward simplicity, it has been questioned whether the success achieved by these superstars is due to skill or luck. This analysis uses data from high-stakes poker tournaments to test for skill discrepancies among elite players. We find significant evidence suggesting that certain players are able to consistently outperform their experienced counterparts.

Contact Fishman at [email protected] and Pope at [email protected]. *We thank Botond Koszegi, Matthew Rabin, and Justin Sydnor for helpful comments and suggestions. All errors are our own.

“Why do you think the same five guys make it to the final table of the World Series of Poker every year? What are they, the luckiest guys in Las Vegas?” -- Mike McDermott (Matt Damon in the 1998 film Rounders)

The popularity of poker has exploded in recent years.1 The premier event, the World Series of Poker Main Event, which costs $10,000 to enter (buy-in), has increased from a field of 6 in 1971, to 839 in 2003, and to 5619 in 2005. Broadcasts of poker tournaments can frequently be found on television stations such as ESPN, Fox Sports, the Travel Channel, Bravo, and the Game Show Network. These tournaments consistently receive high television ratings.2 Poker has also garnered the attention of many influential economists and served as a key inspiration in the development of game theory (von Neumann & Morgenstern (1944), Kuhn (1950), Nash & Shapley (1950)). To economists, poker is a zero-sum game of incomplete information requiring the use of not just probabilities, but beliefs in order to play well. Dreef, Borm, & van der Genugten (2003, 2004a, 2004b) compute optimal mixed-strategy solutions for simple versions of poker and show, theoretically, that skilled players have higher expected payouts than “beginners.”3 Because poker resembles many important economic situations (price competition, litigation, negotiation, etc.), it is of interest to know whether all experienced agents playing for large stakes are able to converge to optimal strategies. If evidence of skill

1

PartyGaming, a leading online poker website, released an IPO in May 2005 which valued the company at $8.46 billion. 2 “America’s Hottest Sport: Poker.” CNN/Money, July 9, 2004. 3 The authors note that the role of skill in poker has policy implications in that a Dutch gaming act requires a license to exploit games of chance, but nor games of skill.

2

discrepancies is found among elite poker players, it suggests that not all players are adopting optimal mixed strategies. Several “star” poker players have performed well in multiple high-stakes tournaments. While this is suggestive of skill differentials, it is far from conclusive. In how many poker tournaments have these “stars” participated in which they did not do well? Furthermore, even if poker competition among top players were random, we would expect a few players to get lucky and do well in multiple tournaments. Using data from high-stakes poker tournaments, we employ an identification strategy that allows us to properly test for whether skill differentials exist among top-place finishers of highstakes tournaments. We find evidence of substantial skill differences among these top players. To better understand the implications and magnitude of our results, we replicate our empirical analysis using data on outcomes from professional golf tournaments.

Data and Results Identifying skill discrepancies among top poker players is complicated by the lack of precise tournament data. The lists of entrants for large poker tournaments are not available and typically outcomes are only recorded for players that finish in the final two or three tables. Thus it is not possible to know the total number of tournaments for which a given player has participated. In order to circumvent the paucity of data on overall tournament participation, we focus our attention on the individuals who finished in the top 18 in high-stakes tournaments.4 Thus, while we are unable to identify the number of tournaments that an

4

A full poker table consists of 9 players. Thus, the top 18 represents the final two tables. Almost all recent high-stakes tournaments record the outcomes of the final two tables.

3

individual has played in, we are able to identify the number of times that a player has played in a “tournament of 18 players”. We can analyze whether or not the outcomes of these tournaments of 18 players appear to be random or whether certain players consistently do better conditional on being among the final 18 contestants. We use data on limit or no-limit Texas Hold’em tournaments that are a part of the World Series of Poker, World Poker Tour, or World Poker Open.5 Using information gleaned from pokerpages.com, we record outcomes for the top 18 finishers of tournaments since 2001 that had at least a $3,000 buy-in.6 A total of 81 separate poker tournaments fit these criteria. Table 1 presents summary statistics for the players in these tournaments. We similarly collect data on all 48 Professional Golfers' Association (PGA) tournaments in 2005. We record the name and final rank of each player that finished in the top 18 in each tournament.7 Table 1 provides summary statistics for the golf players in these tournaments. Empirically, we are interested in whether or not it is possible to use past information to predict the outcome of individuals conditional on being among the final 18 contestants of a given tournament.8 Being able to predict tournament success implies that outcomes are not random conditional on being in the top 18. We explore two sources of 5

Texas Hold’em is a variant of poker in which all players are given 2 cards; the goal is to make the best 5 card hand from the 2 cards and 5 community cards which apply to all players’ hands. Betting occurs after each player receives his or her cards, again after three of the five community cards are revealed, again after the 4th community card, and finally after the 5th community card. In limit Texas Hold’em, the bet amounts each round are fixed, whereas in no-limit Texas Hold’em a player can wager as many chips as he or she wants above a set minimum wager. 6 For a small number of tournaments after 2001 (and for all tournaments prior to 2001), the top 18 finishers were not recorded or not available and thus were not included in the analysis. 7 In golf, there are often ties. We record an average rank for these situations (e.g. if two players tie for 3rd place, each player is given a rank of 3.5). 8 We do not predict success using future results since the decision of whether to play or not in a tournament is endogenous to the outcome of previous tournaments.

4

past information that may predict current success: having played in previous tournaments and performance in the previous tournament played. The basic specifications that we employ are (1)

Rank it     Experience it   it

(2)

Rank it     One Prev it   Two Prev it   Three or More Prev it   it

(3)

Rank it     Top7 - 9it   Top4 - 6it   Top1 - 3it   it

where Rank it represents the ending rank (1st through 18th) of player i in tournament t. Experience it is a dummy variable indicating whether or not individual i had finished in the top 18 of another tournament prior to tournament t.

One Prev it , Two Prev it , and

Three or More Prev it are dummy variables set equal to one if individual i placed in the top 18 in exactly 1, 2, or at least 3 tournaments prior to tournament t, respectively (no previous tournament placements serves as the base group). Top7 - 9it , Top4 - 6it , and Top1 - 3it indicate if individual i placed in the top 7-9, 4-6, or 1-3 in the most recent tournament for which individual i placed in the top 18 (missing if t is the first top 18 finish for individual i). If there are no skill differentials among poker players who made one of the final two tables in these high-stakes poker tournaments, we would expect all of the coefficients in specifications (1) - (3) to be statistically indistinguishable from zero, which would indicate that, conditional on making it to the final 18, winning is random. However, if some players are more skilled than others among the top 18 finishers, we would expect to find negative coefficients in the above specifications (past experience and success should be associated with a reduction in rank – e.g. from 7th place to 6th place).

5

Table 2 presents the results from specifications (1) – (3) for both the poker and golf data. Robust standard errors are presented in brackets below the coefficient values. Beginning with the poker results, Column (1) indicates that having placed in the top 18 in a prior tournament is associated with a significant decrease in rank of .78 (this represents an improvement in rank – e.g. moving from 3rd place to 2nd place is a decrease in rank). Column (2) indicates that the more tournaments for which an individual has placed in the top 18, the better the predicted rank for the current tournament. For example, individuals who placed in the top 18 in three or more previous tournaments are predicted to finish one rank better than individuals who never placed in the top 18 in a previous tournament. Column (3) attempts to predict current rank based on the rank that a player placed in a previous tournament.9 The coefficients suggest that ranking in the top 1-3, top 4-6, or top 7-9 in the previous tournament predicts a better rank in this tournament relative to individuals who placed 10th-18th in the previous tournament. While the coefficients are negative and increasing with last tournament rank, only top 1-3 is statistically significant; individuals who placed in the top 3 in the last tournament on average end 2.5 spots higher in the current tournament (relative to individuals who placed 10th -18th in the last tournament). The results in Column (3) are of particular interest because the sample is reduced to individuals who have placed in the top 18 in at least two high-stakes tournaments. These individuals should all be experienced players and yet still can be shown to have significant skill differences. Columns (4) – (6) present the analogous results using data on professional golfers. The results are qualitatively similar to the poker results. Indicators of past experience in

9

By “previous tournament,” we are recording the last tournament in which the player participated and finished in the top 18.

6

columns (1) and (2) are approximately twice as predictive of current golf success relative to the same indicators of past experience for current poker success. Last tournament ranking is less predictive of current golf success relative to last tournament ranking of current poker success.

Discussion and Conclusion We present evidence of skill differentials among poker players finishing in one of the final two tables in high-stakes poker tournaments. The results are similar in size to the skill discrepancies identified among professional golf players. While we provide evidence of skill differentials, we cannot provide any insight regarding the cause of these discrepancies. Our main hypothesis is that even though poker is simple to learn and play, optimal strategies are very complex and require substantial cognitive ability to compute.

An alternative explanation for these skill

discrepancies is that some players are better at exploiting unintended signals made by the opponent. In classic poker lore, this is referred to as “reading” another player’s “tells.” A third potential explanation for outcome differentials is heterogeneous preferences.10 The skills required by poker players such as the understanding of probabilities and beliefs are similar to those needed by agents in real markets such as price setters and litigation negotiators. Our results suggest that executing optimal game theoretic solutions in situations like these may be very difficult for people, even those who are experienced and select into playing these types of games for high stakes.

10

We consider this explanation to be unlikely due to the fact that all players face the exact same incentives structure, which provides very large incentives to achieve the highest rank possible rather than make a “respectable showing.” Both winnings (and fame) grow exponentially as a player places higher up in the rankings.

7

References Dreef, M., P. Borm, and B. van der Genugten. “A new relative skill measure for games with chance elements.” Managerial and Decision Economics, 2004a. Dreef, M., P. Borm, and B. van der Genugten. “On Strategy and Relative Skill in Poker.” International Game Theory Review, 2003. Dreef, M., P. Borm, and B. van der Genugten. “Measuring skill in games: several approaches discussed.” Mathematical Methods of Operations Research, 2004b. Kuhn, H.W. "A Simplified Two-Person Poker." Contributions to the Theory of Games, I. H.W. Kuhn and A.W. Tucker, eds. Annals of mathematics Studies, Number 24. Princeton, New Jersey: Princeton University Press. 1950, pp. 97-103. Nash, J.F. and L.S. Shapley. "A Simple Three-Person Poker Game." Contributions to the Theory of Games, I. H.W. Kuhn and A.W. Tucker, eds. Annals of mathematics Studies, Number 24. Princeton, New Jersey: Princeton University Press, 1950, pp. 105116. von Neumann, J. and O. Morgenstern. Princeton University Press, 1944.

Theory of Games and Economic Behavior.

8

Table 1. Summary Statistics % with 1 top 18 finish % with 2 top 18 finishes % with 3 top 18 finishes % with 4 or more top 18 finishes Number of tournaments

Poker 70.1 14.7 6.9 8.3

Golf 24.3 14.7 18.9 42.2

81

48

Number of individuals 899 218 Notes: Poker summary statistics represent data from all high-stakes ($3,000 or greater buy-in) limit and no-limit Texas Hold’em tournaments between 2001 and 2005 from the World Series of Poker, the World Poker Tour, or World Poker Open. The 899 players represent those that finished in the top 18 of at least one of these 81 tournaments. The golf summary statistics represent data from all Professional Golfers’ Association Tournaments in 2005. The 218 players represent those that finished in the top 18 of at least one of these 48 tournaments.

9

Table 2. Predicting Poker and Golf Success Dependent Variable: Rank (1st - 18th)

Experience

(1) -0.781 [.274]***

Poker (2)

(3)

(4) -1.420 [.384]***

Golf (5)

1 Prev

-0.558 [.361]

-1.323 [.500]***

2 Prev

-0.920 [.474]*

-0.565 [.548]

3 or More Prev

-1.000 [.415]**

-1.865 [.438]***

(6)

Top7-9

-0.137 [.598]

0.107 [.582]

Top4-6

-0.552 [.595]

1.170 [.697]*

Top1-3

-2.510 [.554]***

-0.440 [.595]

Constant R-Squared

9.810 [.172]***

9.810 [.173]***

9.660 [.310]***

10.270 [.323]***

10.268 [.323]***

8.770 [.271]***

0.005

0.006

0.035

0.016

0.024

0.007

Observations 1494 1494 595 811 811 593 st th Notes: Columns (1) - (6) present coefficients from the regressions with finishing rank (1 – 18 ) as the dependent variable. Experience is an indicator that equals one if the player had previously finished in the top 18 of a tournament in our sample. 1 Prev, 2 Prev, and 3 or More Prev indicate the exact number of previous tournament finishes in our sample. Top 7-9, Top 4-6, and Top 1-3 are indicators that equal one if the player finished in one of those spots in the previous tournament (missing if no previous tournament in our sample). * significant at 10%; ** significant at 5%; *** significant at 1%

10