forked from dvatvani/dvatvani.github.io
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathBGG-Analysis-Part-2.html
309 lines (294 loc) · 17.4 KB
/
BGG-Analysis-Part-2.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
<!DOCTYPE html>
<html lang="en">
<head>
<link href='//fonts.googleapis.com/css?family=Source+Sans+Pro:300,400,700,400italic' rel='stylesheet' type='text/css'>
<link rel="stylesheet" type="text/css" href="/theme/stylesheet/style.min.css">
<link rel="stylesheet" type="text/css" href="/theme/stylesheet/pygments.min.css">
<link rel="stylesheet" type="text/css" href="/theme/stylesheet/font-awesome.min.css">
<link href="https://dvatvani.github.io/feeds/all.atom.xml" type="application/atom+xml" rel="alternate" title="Dinesh Vatvani Atom">
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="robots" content="" />
<meta name="author" content="Dinesh Vatvani" />
<meta name="description" content="An analysis of the BGG top 100 and complexity bias in BGG" />
<meta name="keywords" content="Board games">
<meta property="og:site_name" content="Dinesh Vatvani"/>
<meta property="og:title" content="An analysis of board games: Part II - Complexity bias in BGG"/>
<meta property="og:description" content="An analysis of the BGG top 100 and complexity bias in BGG"/>
<meta property="og:locale" content="en_US"/>
<meta property="og:url" content="https://dvatvani.github.io/BGG-Analysis-Part-2.html"/>
<meta property="og:type" content="article"/>
<meta property="article:published_time" content="2018-12-08 03:30:00+00:00"/>
<meta property="article:modified_time" content="2018-12-09 02:30:00+00:00"/>
<meta property="article:author" content="https://dvatvani.github.io/author/dinesh-vatvani.html">
<meta property="article:section" content="Analysis"/>
<meta property="article:tag" content="Board games"/>
<meta property="og:image" content="/static/Logo/radial_barh.png">
<title>Dinesh Vatvani – An analysis of board games: Part II - Complexity bias in BGG</title>
</head>
<body>
<aside>
<div>
<a href="https://dvatvani.github.io">
<img src="/static/Logo/radial_barh.png" alt="Dinesh Vatvani" title="Dinesh Vatvani">
</a>
<h1><a href="/">Dinesh Vatvani</a></h1>
<p>A Python and data analysis blog</p>
<nav>
<ul class="list">
<li><a href="/pages/about.html#about">About</a></li>
</ul>
</nav>
<ul class="social">
<li><a class="sc-github-square" href="https://github.com/dvatvani" target="_blank"><i class="fa fa-github-square"></i></a></li>
<li><a class="sc-twitter-square" href="https://twitter.com/d_vatvani" target="_blank"><i class="fa fa-twitter-square"></i></a></li>
<li><a class="sc-rss-square" href="http://dvatvani.github.io/feeds/all.atom.xml" target="_blank"><i class="fa fa-rss-square"></i></a></li>
</ul>
</div>
</aside>
<main>
<nav>
<a href="/">Home</a>
<a href="/archives.html">Archives</a>
<a href="/categories.html">Categories</a>
<a href="/tags.html">Tags</a>
<a href="https://dvatvani.github.io/feeds/all.atom.xml">Atom</a>
</nav>
<article>
<header>
<h1 id="BGG-Analysis-Part-2">An analysis of board games: Part <span class="caps">II</span> - Complexity bias in <span class="caps">BGG</span></h1>
<p>Posted on Sat 08 December 2018 in <a href="/category/analysis.html">Analysis</a></p>
</header>
<div>
<p>This is part <span class="caps">II</span> in my series on analysing BoardGameGeek data. Other parts can be found here:</p>
<ul>
<li><a href="./BGG-Analysis-Part-1.html">Part I: Introduction and general trends</a></li>
<li>Part <span class="caps">II</span>: Complexity bias in BoardGameGeek</li>
<li><a href="./BGG-Analysis-Part-3.html">Part <span class="caps">III</span>: Mapping the board game landscape</a></li>
</ul>
<hr />
<h1>Introduction</h1>
<p>In <a href="./BGG-Analysis-Part-1.html">Part I</a>, I describe how I generated a dataset from BoardGameGeek and explored general trends in the rate of release, ratings and complexity. It also looked at the prevalence of different mechanics and themes throughout the hobby and how this has changed in the past 30 years. In this post, we’ll investigate complexity bias in <span class="caps">BGG</span> ratings.</p>
<h1>Complexity bias in ratings</h1>
<p>BoardGameGeek’s top 100 list is a very visible “beacon” for the hobby and many players will use this list to make decisions about which games to try or buy. It is comparable to the IMDb top 250 in the role it plays in shaping what the community perceives as the apex of Board Game experiences. However, one of the problems with the <span class="caps">BGG</span> top 100 is that it is disproportionately dominated by big and complex games. This makes it less useful for a sizeable majority of board game players looking for good new games to play, since many of the games on that list will look inaccessible and daunting. The relationship between a game’s complexity and how highly rated it is on <span class="caps">BGG</span> is not just limited to the top 100. In fact, there is a pretty clear correlation between how complex a game is and how highly rated it is on BoardGameGeek, as shown below.</p>
<p><center><img alt="Rating vs weight" src="https://dvatvani.github.io/static/BGG-analysis/complexity-bias.png" /></center>
<center><em>Note: The above graph only includes games with > 100 votes for game weight</em></center></p>
<p>The existence of this correlation in the <span class="caps">BGG</span> dataset makes it easier to understand why the top 100 is disproportionately populated with big, complex games.</p>
<p>It is worth making a couple of comments based on the graph above:</p>
<ol>
<li>This graph does not necessarily mean that more complex board games are inherently better. While the graph above does show a clear (and statistically significant) relationship between perceived complexity and overall rating, we need to appreciate that there is a strong sampling bias present in our dataset that leads to this result i.e. Complex board games disproportionately appeal to the <span class="caps">BGG</span> user base.</li>
<li>A curious feature of the graph above is the tail of games of low complexity and low ratings at the bottom left of the plot. This “tail of spite” consists of relatively old mass-appeal games. Every single game in the tail of spite was released pre-1980, with many being considerably older than that. The games that form the tail of spite are shown in the table below:</li>
</ol>
<p><center></p>
<table>
<thead>
<tr>
<th align="left">Name</th>
<th align="center">Avg. rating</th>
<th align="center">Avg. weight</th>
<th align="center">Year published</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">Tic-Tac-Toe</td>
<td align="center">2.6</td>
<td align="center">1.11</td>
<td align="center">-1300</td>
</tr>
<tr>
<td align="left">Monopoly</td>
<td align="center">4.4</td>
<td align="center">1.68</td>
<td align="center">1933</td>
</tr>
<tr>
<td align="left">Trouble</td>
<td align="center">3.7</td>
<td align="center">1.05</td>
<td align="center">1965</td>
</tr>
<tr>
<td align="left">Pay Day</td>
<td align="center">4.7</td>
<td align="center">1.23</td>
<td align="center">1975</td>
</tr>
<tr>
<td align="left">Checkers</td>
<td align="center">4.9</td>
<td align="center">1.79</td>
<td align="center">1150</td>
</tr>
<tr>
<td align="left">Pachisi</td>
<td align="center">4.5</td>
<td align="center">1.21</td>
<td align="center">400</td>
</tr>
<tr>
<td align="left">Sorry!</td>
<td align="center">4.5</td>
<td align="center">1.17</td>
<td align="center">1929</td>
</tr>
<tr>
<td align="left">Battleship</td>
<td align="center">4.5</td>
<td align="center">1.23</td>
<td align="center">1931</td>
</tr>
<tr>
<td align="left">Mouse Trap</td>
<td align="center">4.1</td>
<td align="center">1.12</td>
<td align="center">1963</td>
</tr>
<tr>
<td align="left">Connect Four</td>
<td align="center">4.8</td>
<td align="center">1.20</td>
<td align="center">1974</td>
</tr>
<tr>
<td align="left">The Game of Life</td>
<td align="center">4.1</td>
<td align="center">1.19</td>
<td align="center">1960</td>
</tr>
<tr>
<td align="left">Operation</td>
<td align="center">4.0</td>
<td align="center">1.08</td>
<td align="center">1965</td>
</tr>
<tr>
<td align="left">Guess Who?</td>
<td align="center">4.8</td>
<td align="center">1.12</td>
<td align="center">1979</td>
</tr>
<tr>
<td align="left">Candy Land</td>
<td align="center">3.2</td>
<td align="center">1.05</td>
<td align="center">1949</td>
</tr>
<tr>
<td align="left">Snakes and Ladders</td>
<td align="center">2.8</td>
<td align="center">1.00</td>
<td align="center">-200</td>
</tr>
<tr>
<td align="left">Twister</td>
<td align="center">4.6</td>
<td align="center">1.09</td>
<td align="center">1966</td>
</tr>
<tr>
<td align="left">Pick Up Sticks</td>
<td align="center">4.2</td>
<td align="center">1.05</td>
<td align="center">1850</td>
</tr>
<tr>
<td align="left">Bingo</td>
<td align="center">2.7</td>
<td align="center">1.02</td>
<td align="center">1530</td>
</tr>
<tr>
<td align="left">Memory</td>
<td align="center">4.7</td>
<td align="center">1.16</td>
<td align="center">1959</td>
</tr>
</tbody>
</table>
<p></center></p>
<h1>Correcting for the complexity bias</h1>
<p>Since the regression in the graph above reveals how games’ ratings are related to complexity within the <span class="caps">BGG</span> dataset, we can artificially correct for the correlation by adjusting the game ratings to penalize complex games and reward simpler games. For the more mathematically inclined among you, I’m referring to the residuals of the regression between rating and complexity.</p>
<p>A short illustration goes a long way to intuitively explain what the process does.</p>
<p><center>
<img alt="Alt Text" src="https://dvatvani.github.io/static/BGG-analysis/manual_animation.gif" />
</center></p>
<p>Applying that artificial correction gives us a “complexity-agnostic” rating for all games. Below is an interactive plot showing the rating vs complexity after the rating correction. Hover over any point to see the name of the game and the game’s new <span class="caps">BGG</span> rank and rating.</p>
<p><center>
<iframe width="820" height="470" src="https://dvatvani.github.io/static/BGG-analysis/toolbar.html" frameborder="0" allowfullscreen>
</iframe><br>
<strong><em>Hover your mouse over (or tap if you’re on mobile) any point for more information about the game</em></strong>
</center></p>
<p>We can use these corrected ratings to re-rank all of the games and obtain a complexity-agnostic top 100 list. Note that <span class="caps">BGG</span> use something called a Bayesian mean to rank their games instead of taking the raw average ratings. What this does is effectively give each game a certain number of additional “average” rating votes. This is designed to push games with a very low number of ratings towards the average to prevent the top games list being dominated by games with only a couple of perfect score ratings. I’ve used a similar approach, using the same Bayesian prior as <span class="caps">BGG</span> (Bayesian prior of about <code>5.5</code> with a weight of around <code>1,000</code> ratings). As a result, there may be some cases where a game with a higher average rating end up having a lower rank than a game with a slightly lower average rating if the second has significantly more rating votes. The re-ranked <span class="caps">BGG</span> list using these corrected ratings has the complex games evenly spread throughout the ranked list of games rather than disproportionately skewed towards the top, thereby allowing some of the great, but less complex, games to shine through to the top 100.</p>
<p>I have applied the complexity-bias correction to all games with over 30 rating votes. Below is an interactive table that allows you to navigate the full list. It also includes a search function to find the impact of the complexity-bias correction on specific games.</p>
<p><object width="100%" height="700" type="text/html" data="https://dvatvani.github.io/static/BGG-analysis/data.html"></object>
<center><em>Note: This table only includes games with >= 30 rating votes</em></center></p>
<p>Some of the games experienced a fairly substantial push up/down the rankings ladder as a result of the complexity bias correction. Some of the games that benefitted the most from this rating correction and have risen to the top 100 are <code>Skull</code>, <code>BANG! The Dice Game</code>, <code>Love Letter: Batman</code>, <code>No Thanks!</code>, <code>Time's Up!</code>, <code>Spyfall</code> and <code>Sushi Go!</code>. Conversely, some of the games that have been penalized the most are <code>Twilight Imperium (Third Edition)</code>, <code>Alchemists</code>, <code>War of the Ring (first edition)</code>, <code>A Game of Thrones: The Board Game (Second Edition)</code>, <code>Through the Ages: A Story of Civilization</code> and <code>Caverna: The Cave Farmers</code>.</p>
<p>Looking at the revised top 100 from the list above, I still have some reservations about it, but it looks much more reasonable to me than the original <span class="caps">BGG</span> top 100 list. I suspect that for most board game players looking to try out new good games, this list would look far more approachable, while still being filled with excellent games.</p>
<p>I hope that you’ve enjoyed learning about the complexity bias inherent in the BoardGameGeek dataset and how we can correct for it. The discussion on whether or not complex games really are better is far from over, but hopefully people looking for some of the lighter great games to play will find this more welcoming take on the <span class="caps">BGG</span> top 100 useful.</p>
<p><em>The code I wrote for this analysis can be found <a href="http://nbviewer.jupyter.org/github/dvatvani/dvatvani.github.io/blob/master/static/BGG-analysis/BGG_analysis_-_complexity_bias_correction.ipynb">here</a> in the form of a Jupyter Notebook (Python).</em></p>
<hr />
<p><em>Thanks to</em>:</p>
<ul>
<li><em><a href="https://twitter.com/ColmSeeley">Colm Seeley</a> for co-authoring this work with me</em></li>
<li><em>Catherine Maddox for great feedback on the writing and presentation of the post</em></li>
<li><em>Quintin Smith (Quinns) from Shut Up <span class="amp">&</span> Sit Down for allowing me to use material from one of his talks in a presentation of this analysis</em></li>
<li><em>GitHub user <code>TheWeatherman</code> for creating the <a href="https://github.com/ThaWeatherman"><span class="caps">BGG</span> scraper</a> that I modified to collect the data used for this analysis.</em></li>
<li>_GitHub user <code>vividvilla</code> for building the useful <a href="https://github.com/vividvilla/csvtotable">CSVtoTable</a> tool</li>
</ul>
<p><em>If you enjoyed reading this, you may also enjoy:</em></p>
<ul>
<li><a href="https://www.shutupandsitdown.com/bgg100-100-81/"><em>Shut Up <span class="amp">&</span> Sit Down’s take on the <span class="caps">BGG</span> top 100</em></a></li>
<li><em><a href="https://opinionatedgamers.com/2015/08/17/are-boardgames-getting-better-an-empirical-analysis/"><em>Are Boardgames Getting Better? An Empirical Analysis</em></a> by <a href="https://opinionatedgamers.com/"><em>Opinionated Gamers</em></a></em></li>
<li><em><a href="https://boardgamegeek.com/blogpost/11991/numbers-bgg-rank-data-analysis"><em>By the Numbers - <span class="caps">BGG</span> Rank Data + Analysis</em></a> by Oliver Kiley</em></li>
</ul>
</div>
<div class="tag-cloud">
<p>
<a href="/tag/board-games.html">Board games</a>
</p>
</div>
<div id="disqus_thread"></div>
<script type="text/javascript">
var disqus_shortname = 'dvatvani';
(function() {
var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js';
(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
})();
</script>
<noscript>Please enable JavaScript to view the <a href="https://disqus.com/?ref_noscript" rel="nofollow">comments powered by Disqus.</a></noscript>
</article>
<footer>
<p>© Dinesh Vatvani </p>
<p>Built using <a href="http://getpelican.com" target="_blank">Pelican</a> - <a href="https://github.com/alexandrevicenzi/flex" target="_blank">Flex</a> theme by <a href="http://alexandrevicenzi.com" target="_blank">Alexandre Vicenzi</a></p> </footer>
</main>
<!-- Google Analytics -->
<script type="text/javascript">
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-75651339-1', 'auto');
ga('send', 'pageview');
</script>
<!-- End Google Analytics -->
<script type="application/ld+json">
{
"@context": "http://schema.org",
"@type": "BlogPosting",
"name": "An analysis of board games: Part II - Complexity bias in BGG",
"headline": "An analysis of board games: Part II - Complexity bias in BGG",
"datePublished": "2018-12-08 03:30:00+00:00",
"dateModified": "2018-12-09 02:30:00+00:00",
"author": {
"@type": "Person",
"name": "Dinesh Vatvani",
"url": "https://dvatvani.github.io/author/dinesh-vatvani.html"
},
"image": "/static/Logo/radial_barh.png",
"url": "https://dvatvani.github.io/BGG-Analysis-Part-2.html",
"description": "An analysis of the BGG top 100 and complexity bias in BGG"
}
</script></body>
</html>