Skip to content

Commit

Permalink
Merge pull request #248 from michalc/docs/paper-more-tweaks
Browse files Browse the repository at this point in the history
docs: more changes to paper
  • Loading branch information
michalc authored Dec 14, 2024
2 parents 4804767 + 79834ef commit 1f008fa
Showing 1 changed file with 6 additions and 6 deletions.
12 changes: 6 additions & 6 deletions paper/paper.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,24 +17,24 @@ bibliography: paper.bib

# Summary

OpenTTD [@openttdteam2004openttd] is a business simulation game created for recreational play, where one or more human players build companies by constructing a transportation network of buses, trains, planes and ships for use in moving passengers and cargo. OpenTTD can be extended by allowing autonomous so-called AI players, and OpenTTDLab leverages this capability to allow researchers to run experiments with their own AIs. Doing so converts OpenTTD from a game into a system for researching algorithms and their effects on companies and supply chains, and helps to ensure the results of such research are reproducible.
OpenTTD [@openttdteam2004openttd] is a business simulation game created for recreational play, where one or more human players build companies by constructing a transportation network of buses, trains, planes and ships for use in moving passengers and cargo. OpenTTD can be extended by allowing autonomous so-called AI players written using the Squirrel programming language, and OpenTTDLab leverages this capability to allow researchers to run experiments using their own AIs. Doing so converts OpenTTD from a game into a system for researching algorithms and their effects on companies and supply chains, and helps to ensure the results of such research are reproducible.

![The OpenTTDLab logo. Adapted from the OpenTTD Logo, © OpenTTD Team, licenced under the GNU General Public License Version 2.](../docs/assets/openttdlab-logo.svg){height="100pt"}

# Statement of need

OpenTTD is remarkably flexible: by leveraging the OpenTTD AI system it has been successfully used as a tool to research algorithms including artificial intelligence (AI), machine learning (ML), and anomaly detection algorithms [@rios2009trains; @wisniewski2011artificial; @bijlsma2014evolving; @konijnendijk2015mcts; @volna2017fuzzy; @beuneker2019autonomous; @lakomy2020railroad]. However, OpenTTD does not include the capability to easily repeat experiments over ranges of configuration--unsurprising given it was created for recreational play--and this results in much of the existing research not being reproducible [@charemza2024reusable, chap. 1]. OpenTTDLab addresses this by providing a framework through which experiments similar to those in the literature can be easily run via writing and running Python code, and encourages the code to be shared so future researchers can easily reproduce results.
OpenTTD is remarkably flexible: by leveraging the OpenTTD AI system it has been successfully used to research algorithms including artificial intelligence (AI), machine learning (ML), and anomaly detection algorithms [@rios2009trains; @wisniewski2011artificial; @bijlsma2014evolving; @konijnendijk2015mcts; @volna2017fuzzy; @beuneker2019autonomous; @lakomy2020railroad]. However, OpenTTD does not include the capability to easily repeat experiments over ranges of configuration--unsurprising given it was created for recreational play--and this results in much of the existing research not being reproducible [@charemza2024reusable, chap. 1]. OpenTTDLab addresses this by providing a framework through which experiments similar to those in the literature can be easily run via writing and running Python code, and encourages such code to be shared so future researchers can easily reproduce results.

There are no other similar frameworks that take a game created for recreational play and convert it into a reusable simulator for reproducible research, and in this respect OpenTTDLab is novel. The closest equivalent is Axelrod [@knight2016open], a Python framework that researches the Iterated Prisoner's Dilemma, allowing investigations of existing and new gameplay strategies. Similar to OpenTTDLab, Axelrod was created in response to difficulties in reproducing existing research.
There are no other open source frameworks that take a game created for recreational play and convert it into a reusable simulator for reproducible research, and in this respect OpenTTDLab is novel. The closest equivalent is Axelrod [@knight2016open], a Python framework for researching strategies for playing the Iterated Prisoner's Dilemma. Similar to OpenTTDLab, Axelrod was created in response to difficulties in reproducing existing research.

# Usage and results

Through a single Python function call, `run_experiments`, OpenTTDLab runs OpenTTD over a range of configurations, most often over a range of random seeds and a range of parameters of an AI. It then returns data from every month of in-game time, which can be processed and visualised. \autoref{fig:example-results} shows example results: comparing how the distributions of money in the bank change over time for two configurations of a simple AI. The differences in the distributions show that OpenTTDLab can be used to investigate risk/benefit trades offs in supply chains.
Through a single Python function call, OpenTTDLab runs OpenTTD over a range of configurations, most often over a range of random seeds and a range of parameters of an AI. It then returns data from every month of in-game time, which can be processed and visualised. \autoref{fig:example-results} shows example results: comparing how the distributions of money in the bank change over time for two configurations of a simple AI. The differences in the distributions show that OpenTTDLab can be used to investigate risk-benefit trades-offs in supply chains.

![How the distribution of money in the bank changes over time for a simple AI [@charemza2024parameterised]. The results of 100 runs of OpenTTD are shown, 50 when the AI is configured to build 1 bus, and 50 for when it is configured to build 16 buses. The results are generated by a single call to OpenTTDLab's `run_experiments` function. Image adapted from [@charemza2024reusable, chap. 5].\label{fig:example-results}](example-results-charemza2004reproducible.pdf){height="150pt"}
![How the distribution of money in the bank changes over time for an AI programmed to construct a single bus route with a configurable number of buses [@charemza2024parameterised]. The results of 100 runs of OpenTTD are shown, 50 when the AI is configured to build 1 bus, and 50 for when it is configured to build 16 buses. Image adapted from [@charemza2024reusable, chap. 5].\label{fig:example-results}](example-results-charemza2004reproducible.pdf){height="150pt"}

# Acknowledgments

OpenTTDLab could not have been created without incredible work of all the [contributors to OpenTTD, its dependencies, and precursors](https://github.com/OpenTTD/OpenTTD/blob/master/CREDITS.md). A further thank you to Patric Stout (A.K.A. TrueBrain), who originally wrote the OpenTTD savegame parser that OpenTTDLab forked from. Finally, thank you to Michael Herrmann, whose invaluable supervision of the original dissertation resulted in the first versions of OpenTTDLab [@charemza2024reusable].
OpenTTDLab could not have been created without incredible work of all the [contributors to OpenTTD, its dependencies, and forebears](https://github.com/OpenTTD/OpenTTD/blob/master/CREDITS.md). A further thank you to Patric Stout, a.k.a. TrueBrain, who originally wrote the OpenTTD savegame parser that OpenTTDLab forked from. Finally, thank you to Michael Herrmann, whose invaluable supervision of the the current author's dissertation [@charemza2024reusable] resulted in the first versions of OpenTTDLab.

# References

0 comments on commit 1f008fa

Please sign in to comment.