Home > Research > Small Grants > Hodgins (S23)

 Print this page

CALLING THEIR BLUFF: INVESTIGATING THE ACCURACY OF SKILL ASSESSMENT AMONG POKER PLAYERS
Project Approved 2009-10

Ms. Terri-Lynn MacKay (Principal Investigator)
Department of Psychology, University of Calgary

Co-investigators:

Dr. Michael Bowling
Department of Computing Science, University of Alberta

Dr. David Hodgins (Grant Holder)
Department of Psychology, University of Calgary

The purpose of the present study is to investigate the relationship between cognitive distortions and actual play using a poker interface with a low-variance analysis estimator of player skill. The proposed study will focus on each participant’s assessment of his or her performance compared with that measured by empirical skill. The Computer Poker Research Group at the University of Alberta has developed a program that can provide an estimate of a player’s true skill by factoring out much of the luck component. The program allows for a computational analysis of how much a player should expect to win/lose relative to an automated computer opponent. This analysis will provide a far more accurate estimate of a player’s skill than how much money was actually won or lost over those hands. The assessment of skill will then be correlated with level of cognitive distortions to determine a player’s accuracy of skill estimation. For example, does a good player who loses money correctly estimate they are winning (and were unlucky) or does a bad player who wins money think they are losing (and were lucky)? The key to this study is that after the hands are completed, a post-match analysis can be run to make unbiased estimates of the players' actual performance (i.e., remove much of the luck from the match outcome).

The secondary goal of the study is to seek to determine which variables influence the accuracy of a poker player's self-assessment. Additional questions to be answered are: a.) Is self-assessment tied to actual performance or only observed performance? b.) Are players with more experience better at assessing their performance? c) Are better poker players superior in assessing their performance? In addition, level of cognitive distortion (actual vs. perceived skill) will be assessed by problem gambling level. It is hypothesized that, in general, players will overestimate their skill level and problem gamblers will have a greater propensity for distortion. It is unclear what the relationship might be between actual skill and problem gambling level.

Status (Complete)

This research aims to assess to cognitive distortions among poker plays using a simulated online poker game. Ethics approval for this study was received on November 25, 2009.

The University of Alberta's Computer Poker Research Group built a complete platform for subjects to play Texas Hold'em against computer-controlled opponents. In order to simplify the participant's experience, the interface was built with Java Web Start in order to be accessible through any modern web browser. The interface connects to a backend server that handles the computer opponents and logs the data for later analysis. This software development took approximately two months of time to complete. It was developed by a programmer analyst and supported by a graduate student in the Poker Research Group.

A questionnaire for the study was developed collaboratively by the University of Calgary and the University of Alberta. Participants were recruited from the University of Calgary and the questionnaire was administered through Survey Monkey. Data collection was completed in April, 2010 with 540 participants. The analysis has been completed and written up as part of a doctoral dissertation. It is also being prepared for publication and has been disseminated through conference presentations. 

The Computer Poker Research Group is refining the platform (without the computer controlled opponents) in preparation to have publicly available under an open-source license.  Researchers interested in studying poker players’ behaviour directly will be able to use the software as opposed to using aggregate survey data. This release is likely to occur in 2010.

Scholarly Conference Papers:

MacKay, T. L., & Hodgins, D. C. (2011, April). Calling their bluff: Investigating the accuracy of skill assessment among poker players. Poster session presented at the Engaging the Big Questions in Gambling Studies, 10th Alberta Gaming Research Institute Conference, Banff, AB.

MacKay, T. L., & Hodgins, D. C. (2011, March). Cognitive distortions among Internet gamblers. Paper presented at the European Psychiatry Conference, Vienna, Austria.

MacKay, T. L., Hodgins, D. C., Bard, N., & Bowling, M. (2010, November). Calling their bluff: Investigating the accuracy of skill assessment among poker players. Poster session presented at the National Center for Responsible Gaming Conference on Gambling and Addiction, Las Vegas, NV.

Public Lectures:

MacKay, T. L. (2011, February). Calling their bluff: Investigating the accuracy of skill assessment among online poker players. Lecture presented at the Sebastian Littmann Research Day, Department of Psychiatry, University of Calgary, Calgary, AB.


Grant period: January 1, 2010 to December 31, 2010

The Alberta Gaming Research Institute provides grant funding to support peer-endorsed academic investigations into many aspects of gambling research. The contents, recommendations, and findings of the associated research reports, posted on this website, represent the views of the researcher(s).

Last Updated: 07/14/11

Links

• Volume 9, Issue 6 (.PDF 730K)