diff --git a/paper/paper.md b/paper/paper.md index 4f966d8..7d6291a 100644 --- a/paper/paper.md +++ b/paper/paper.md @@ -17,21 +17,21 @@ bibliography: paper.bib # Summary -OpenTTD [@openttdteam2004openttd] is a business simulation game created for recreational play, where one or more human players build companies by constructing a transportation network of buses, trains, planes and ships for use in moving passengers and cargo. OpenTTD can be extended by allowing autonomous so-called AI players written using the Squirrel programming language, and OpenTTDLab leverages this capability to allow researchers to run experiments using their own AIs. Doing so converts OpenTTD from a game into a system for researching algorithms and their effects on companies and supply chains, and helps to ensure the results of such research are reproducible. +OpenTTD [@openttdteam2004openttd] is a business simulation game created for recreational play, where one or more human players build commercial companies by constructing a transportation network of buses, trains, planes and ships for use in moving passengers and cargo. OpenTTD can be extended by allowing autonomous so-called AI players written using the Squirrel programming language, and OpenTTDLab leverages this capability to allow researchers to run experiments using their own AIs. Doing so converts OpenTTD from a game into a system for researching algorithms and their effects on companies and supply chains, and helps to ensure the results of such research are reproducible. ![The OpenTTDLab logo. Adapted from the OpenTTD Logo, © OpenTTD Team, licenced under the GNU General Public License Version 2.](../docs/assets/openttdlab-logo.svg){height="100pt"} # Statement of need -OpenTTD is remarkably flexible: by leveraging the OpenTTD AI system it has been successfully used to research algorithms including artificial intelligence (AI), machine learning (ML), and anomaly detection algorithms [@rios2009trains; @wisniewski2011artificial; @bijlsma2014evolving; @konijnendijk2015mcts; @volna2017fuzzy; @beuneker2019autonomous; @lakomy2020railroad]. However, OpenTTD does not include the capability to easily repeat experiments over ranges of configuration--unsurprising given it was created for recreational play--and this results in much of the existing research not being reproducible [@charemza2024reusable, chap. 1]. OpenTTDLab addresses this by providing a framework through which experiments similar to those in the literature can be easily run via writing and running Python code, and encourages such code to be shared so future researchers can easily reproduce results. +OpenTTD is remarkably flexible: its AI system has been successfully used to research artificial intelligence (AI), machine learning (ML), genetic algorithms and anomaly detection [@rios2009trains; @wisniewski2011artificial; @bijlsma2014evolving; @konijnendijk2015mcts; @volna2017fuzzy; @beuneker2019autonomous; @lakomy2020railroad]. However, OpenTTD does not include the capability to easily repeat experiments over ranges of configuration--unsurprising given it was created for recreational play--and much of the existing research is correspondingly not reproducible [@charemza2024reusable]. OpenTTDLab addresses this by providing a framework through which experiments similar to those in the literature can be easily run via writing and running Python code, and encourages such code to be shared so future researchers can easily reproduce results. -There are no other open source frameworks that take a game created for recreational play and convert it into a reusable simulator for reproducible research, and in this respect OpenTTDLab is novel. The closest equivalent is Axelrod [@knight2016open], a Python framework for researching strategies for playing the Iterated Prisoner's Dilemma. Similar to OpenTTDLab, Axelrod was created in response to difficulties in reproducing existing research. +There are no other open source frameworks that take a simulation game created for recreational play and convert it into a reusable simulator for reproducible research; in this respect OpenTTDLab is novel. The closest equivalent is Axelrod [@knight2016open], a Python framework for researching strategies for playing the Iterated Prisoner's Dilemma. Similar to OpenTTDLab, Axelrod was created in response to difficulties in reproducing existing research. # Usage and results Through a single Python function call, OpenTTDLab runs OpenTTD over a range of configurations, most often over a range of random seeds and a range of parameters of an AI. It then returns data from every month of in-game time, which can be processed and visualised. \autoref{fig:example-results} shows example results: comparing how the distributions of money in the bank change over time for two configurations of a simple AI. The differences in the distributions show that OpenTTDLab can be used to investigate risk-benefit trades-offs in supply chains. -![How the distribution of money in the bank changes over time for an AI programmed to construct a single bus route with a configurable number of buses [@charemza2024parameterised]. The results of 100 runs of OpenTTD are shown, 50 when the AI is configured to build 1 bus, and 50 for when it is configured to build 16 buses. Image adapted from [@charemza2024reusable, chap. 5].\label{fig:example-results}](example-results-charemza2004reproducible.pdf){height="150pt"} +![How the distribution of money in the bank changes over time for an AI programmed to construct a single bus route with a configurable number of buses [@charemza2024parameterised]. The results of 100 runs of OpenTTD are shown, 50 when the AI is configured to build 1 bus, and 50 for when it is configured to build 16 buses. Adapted from [@charemza2024reusable, chap. 5].\label{fig:example-results}](example-results-charemza2004reproducible.pdf){height="150pt"} # Acknowledgments