From f21d2423dfb4d997b7c6ff2c8c026d5912603758 Mon Sep 17 00:00:00 2001 From: Ewout ter Hoeven Date: Mon, 15 Jan 2024 12:16:52 +0100 Subject: [PATCH] intro tutorial: Analysing model reporters (#1955) * intro tutorial: Analysing model reporters --- docs/tutorials/intro_tutorial.ipynb | 517 +++++++++++++++------------- 1 file changed, 274 insertions(+), 243 deletions(-) diff --git a/docs/tutorials/intro_tutorial.ipynb b/docs/tutorials/intro_tutorial.ipynb index f130dbccaa5..69e9b8fb7e5 100644 --- a/docs/tutorials/intro_tutorial.ipynb +++ b/docs/tutorials/intro_tutorial.ipynb @@ -74,10 +74,10 @@ "pip install jupyter\n", "```\n", "\n", - "Install Visualization Tools:\n", + "Install [Seaborn](https://seaborn.pydata.org/) (which is used for data visualization):\n", "\n", "```bash\n", - "pip install matplotlib\n", + "pip install seaborn\n", "```\n", "\n", "\n", @@ -144,14 +144,7 @@ "cell_type": "code", "execution_count": null, "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:41.694690Z", - "start_time": "2023-04-25T18:23:41.573145Z" - }, - "collapsed": false, - "jupyter": { - "outputs_hidden": false - } + "collapsed": false }, "outputs": [], "source": [ @@ -190,14 +183,7 @@ "cell_type": "code", "execution_count": null, "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:41.696198Z", - "start_time": "2023-04-25T18:23:41.693472Z" - }, - "collapsed": false, - "jupyter": { - "outputs_hidden": false - } + "collapsed": false }, "outputs": [], "source": [ @@ -232,12 +218,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:41.700374Z", - "start_time": "2023-04-25T18:23:41.698198Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "class MoneyModel(mesa.Model):\n", @@ -269,12 +250,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:41.705824Z", - "start_time": "2023-04-25T18:23:41.703377Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "class MoneyAgent(mesa.Agent):\n", @@ -337,12 +313,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:41.711368Z", - "start_time": "2023-04-25T18:23:41.706825Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "starter_model = MoneyModel(10)\n", @@ -353,14 +324,7 @@ "cell_type": "code", "execution_count": null, "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:41.713554Z", - "start_time": "2023-04-25T18:23:41.711368Z" - }, - "collapsed": false, - "jupyter": { - "outputs_hidden": false - } + "collapsed": false }, "outputs": [], "source": [ @@ -380,14 +344,7 @@ "cell_type": "code", "execution_count": null, "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:41.716975Z", - "start_time": "2023-04-25T18:23:41.714555Z" - }, - "collapsed": false, - "jupyter": { - "outputs_hidden": false - } + "collapsed": false }, "outputs": [], "source": [ @@ -418,14 +375,7 @@ "cell_type": "code", "execution_count": null, "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:41.719722Z", - "start_time": "2023-04-25T18:23:41.716975Z" - }, - "collapsed": false, - "jupyter": { - "outputs_hidden": false - } + "collapsed": false }, "outputs": [], "source": [ @@ -459,12 +409,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:41.723390Z", - "start_time": "2023-04-25T18:23:41.721719Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "import copy\n", @@ -509,12 +454,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:41.790021Z", - "start_time": "2023-04-25T18:23:41.724975Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "model = MoneyModel(10)\n", @@ -526,38 +466,15 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Next, we need to get some data out of the model. Specifically, we want to see the distribution of the agent's wealth. We can get the wealth values with list comprehension, and then use matplotlib (or another graphics library) to visualize the data in a histogram." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you are running from a text editor or IDE, you'll also need to add this line, to make the graph appear.\n", - "\n", - "```python\n", - "import matplotlib.pyplot as plt\n", - "plt.show()\n", - "```" + "Next, we need to get some data out of the model. Specifically, we want to see the distribution of the agent's wealth. We can get the wealth values with list comprehension, and then use seaborn (or another graphics library) to visualize the data in a histogram." ] }, { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:41.882917Z", - "start_time": "2023-04-25T18:23:41.727492Z" - } - }, + "metadata": {}, "outputs": [], "source": [ - "# For a jupyter notebook add the following line:\n", - "%matplotlib inline\n", - "\n", - "# The below is needed for both notebooks and scripts\n", - "import matplotlib.pyplot as plt\n", - "\n", "agent_wealth = [a.wealth for a in model.schedule.agents]\n", "# Create a histogram with seaborn\n", "g = sns.histplot(agent_wealth, discrete=True)\n", @@ -583,12 +500,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:42.027373Z", - "start_time": "2023-04-25T18:23:41.886918Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "all_wealth = []\n", @@ -642,12 +554,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:42.030776Z", - "start_time": "2023-04-25T18:23:42.028374Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "class MoneyModel(mesa.Model):\n", @@ -733,12 +640,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:42.036559Z", - "start_time": "2023-04-25T18:23:42.033158Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "class MoneyAgent(mesa.Agent):\n", @@ -798,12 +700,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:42.042251Z", - "start_time": "2023-04-25T18:23:42.040741Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "model = MoneyModel(100, 10, 10)\n", @@ -815,18 +712,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now let's use matplotlib and numpy to visualize the number of agents residing in each cell. To do that, we create a numpy array of the same size as the grid, filled with zeros. Then we use the grid object's `coord_iter()` feature, which lets us loop over every cell in the grid, giving us each cell's positions and contents in turn." + "Now let's use seaborn and numpy to visualize the number of agents residing in each cell. To do that, we create a numpy array of the same size as the grid, filled with zeros. Then we use the grid object's `coord_iter()` feature, which lets us loop over every cell in the grid, giving us each cell's positions and contents in turn." ] }, { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:42.207690Z", - "start_time": "2023-04-25T18:23:42.043252Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "agent_counts = np.zeros((model.grid.width, model.grid.height))\n", @@ -857,12 +749,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:42.215443Z", - "start_time": "2023-04-25T18:23:42.214438Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "def compute_gini(model):\n", @@ -945,12 +832,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:42.232493Z", - "start_time": "2023-04-25T18:23:42.216442Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "model = MoneyModel(100, 10, 10)\n", @@ -968,12 +850,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:42.372795Z", - "start_time": "2023-04-25T18:23:42.233495Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "gini = model.datacollector.get_model_vars_dataframe()\n", @@ -992,12 +869,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:42.385021Z", - "start_time": "2023-04-25T18:23:42.372795Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "agent_wealth = model.datacollector.get_agent_vars_dataframe()\n", @@ -1014,12 +886,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:42.558338Z", - "start_time": "2023-04-25T18:23:42.383031Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "last_step = agent_wealth.index.get_level_values(\"Step\").max()\n", @@ -1043,12 +910,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:42.702477Z", - "start_time": "2023-04-25T18:23:42.520333Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "# Get the wealth of agent 14 over time\n", @@ -1125,12 +987,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:42.711736Z", - "start_time": "2023-04-25T18:23:42.702477Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "# save the model data (stored in the pandas gini object) to CSV\n", @@ -1146,25 +1003,25 @@ "source": [ "### Batch Run\n", "\n", - "Like we mentioned above, you usually won't run a model only once, but multiple times, with fixed parameters to find the overall distributions the model generates, and with varying parameters to analyze how they drive the model's outputs and behaviors. Instead of needing to write nested for-loops for each model, Mesa provides a [`batch_run`](https://github.com/projectmesa/mesa/blob/main/mesa/batchrunner.py) function which automates it for you." + "Like we mentioned above, you usually won't run a model only once, but multiple times, with fixed parameters to find the overall distributions the model generates, and with varying parameters to analyze how they drive the model's outputs and behaviors. Instead of needing to write nested for-loops for each model, Mesa provides a [`batch_run`](https://github.com/projectmesa/mesa/blob/main/mesa/batchrunner.py) function which automates it for you.\n", + "\n", + "The batch runner also requires an additional variable `self.running` for the MoneyModel class. This variable enables conditional shut off of the model once a condition is met. In this example it will be set as True indefinitely." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "The batch runner also requires an additional variable `self.running` for the MoneyModel class. This variable enables conditional shut off of the model once a condition is met. In this example it will be set as True indefinitely." + "#### Additional agent reporter\n", + "To make the results a little bit more interesting, we will also calculate the number of consecutive time steps an agent hasn't given any wealth as an agent reporter.\n", + "\n", + "This way we can see how data is handled when multiple reporters are used." ] }, { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:23:42.716831Z", - "start_time": "2023-04-25T18:23:42.714736Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "def compute_gini(model):\n", @@ -1179,6 +1036,7 @@ " \"\"\"A model with some number of agents.\"\"\"\n", "\n", " def __init__(self, N, width, height):\n", + " super().__init__()\n", " self.num_agents = N\n", " self.grid = mesa.space.MultiGrid(width, height, True)\n", " self.schedule = mesa.time.RandomActivation(self)\n", @@ -1194,53 +1052,71 @@ " self.grid.place_agent(a, (x, y))\n", "\n", " self.datacollector = mesa.DataCollector(\n", - " model_reporters={\"Gini\": compute_gini}, agent_reporters={\"Wealth\": \"wealth\"}\n", + " model_reporters={\"Gini\": compute_gini},\n", + " agent_reporters={\"Wealth\": \"wealth\", \"Steps_not_given\": \"steps_not_given\"},\n", " )\n", "\n", " def step(self):\n", " self.datacollector.collect(self)\n", - " self.schedule.step()" + " self.schedule.step()\n", + "\n", + "\n", + "class MoneyAgent(mesa.Agent):\n", + " \"\"\"An agent with fixed initial wealth.\"\"\"\n", + "\n", + " def __init__(self, unique_id, model):\n", + " super().__init__(unique_id, model)\n", + " self.wealth = 1\n", + " self.steps_not_given = 0\n", + "\n", + " def move(self):\n", + " possible_steps = self.model.grid.get_neighborhood(\n", + " self.pos, moore=True, include_center=False\n", + " )\n", + " new_position = self.random.choice(possible_steps)\n", + " self.model.grid.move_agent(self, new_position)\n", + "\n", + " def give_money(self):\n", + " cellmates = self.model.grid.get_cell_list_contents([self.pos])\n", + " if len(cellmates) > 1:\n", + " other = self.random.choice(cellmates)\n", + " other.wealth += 1\n", + " self.wealth -= 1\n", + " self.steps_not_given = 0\n", + " else:\n", + " self.steps_not_given += 1\n", + "\n", + " def step(self):\n", + " self.move()\n", + " if self.wealth > 0:\n", + " self.give_money()\n", + " else:\n", + " self.steps_not_given += 1" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ + "#### Batch run\n", + "\n", "We call `batch_run` with the following arguments:\n", "\n", "* `model_cls`\n", - "

\n", - "The model class that is used for the batch run.\n", - "

\n", - "\n", + " The model class that is used for the batch run.\n", "* `parameters`\n", - "

\n", - "A dictionary containing all the parameters of the model class and desired values to use for the batch run as key-value pairs. Each value can either be fixed ( e.g. `{\"height\": 10, \"width\": 10}`) or an iterable (e.g. `{\"N\": range(10, 500, 10)}`). `batch_run` will then generate all possible parameter combinations based on this dictionary and run the model `iterations` times for each combination.\n", - "

\n", - "\n", + " A dictionary containing all the parameters of the model class and desired values to use for the batch run as key-value pairs. Each value can either be fixed ( e.g. `{\"height\": 10, \"width\": 10}`) or an iterable (e.g. `{\"N\": range(10, 500, 10)}`). `batch_run` will then generate all possible parameter combinations based on this dictionary and run the model `iterations` times for each combination.\n", "* `number_processes`\n", - "

\n", - "If not specified, defaults to 1. Set it to `None` to use all the available processors.\n", - "

\n", - "Note: Multiprocessing does make debugging challenging. If your parameter sweeps are resulting in unexpected errors set `number_processes = 1`.\n", - "

\n", - "\n", + " If not specified, defaults to 1. Set it to `None` to use all the available processors.\n", + " Note: Multiprocessing does make debugging challenging. If your parameter sweeps are resulting in unexpected errors set `number_processes=1`.\n", "* `iterations`\n", - "

\n", - "The number of iterations to run each parameter combination for. Optional. If not specified, defaults to 1.

\n", + " The number of iterations to run each parameter combination for. Optional. If not specified, defaults to 1.\n", "* `data_collection_period`\n", - "

\n", - "The length of the period (number of steps) after which the model and agent reporters collect data. Optional. If not specified, defaults to -1, i.e. only at the end of each episode.\n", - "

\n", - "\n", + " The length of the period (number of steps) after which the model and agent reporters collect data. Optional. If not specified, defaults to -1, i.e. only at the end of each episode.\n", "* `max_steps`\n", - "

\n", - "The maximum number of time steps after which the model halts. An episode does either end when `self.running` of the model class is set to `False` or when `model.schedule.steps == max_steps` is reached. Optional. If not specified, defaults to 1000.\n", - "

\n", - "\n", + " The maximum number of time steps after which the model halts. An episode does either end when `self.running` of the model class is set to `False` or when `model.schedule.steps == max_steps` is reached. Optional. If not specified, defaults to 1000.\n", "* `display_progress`\n", - "

\n", - "Display the batch run progress. Optional. If not specified, defaults to `True`." + " Display the batch run progress. Optional. If not specified, defaults to `True`." ] }, { @@ -1251,10 +1127,10 @@ "\n", "We want to keep track of\n", "\n", - "1. the Gini coefficient value and\n", - "2. the individual agent's wealth development.\n", + "1. the Gini coefficient value at each time step, and\n", + "2. the individual agent's wealth development and steps without giving money.\n", "\n", - "Since for the latter changes at each time step might be interesting, we set `data_collection_period = 1`.\n", + "**Important:** Since for the latter changes at each time step might be interesting, we set `data_collection_period=1`. By default, it only collects data at the end of each episode.\n", "\n", "Note: The total number of runs is 245 (= 49 different populations * 5 iterations per population). However, the resulting list of dictionaries will be of length 6186250 (= 250 average agents per population * 49 different populations * 5 iterations per population * 101 steps per iteration). " ] @@ -1269,20 +1145,15 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:24:02.422337Z", - "start_time": "2023-04-25T18:23:42.717833Z" - } - }, + "metadata": {}, "outputs": [], "source": [ - "params = {\"width\": 10, \"height\": 10, \"N\": range(10, 500, 10)}\n", + "params = {\"width\": 10, \"height\": 10, \"N\": range(5, 100, 5)}\n", "\n", "results = mesa.batch_run(\n", " MoneyModel,\n", " parameters=params,\n", - " iterations=5,\n", + " iterations=7,\n", " max_steps=100,\n", " number_processes=1,\n", " data_collection_period=1,\n", @@ -1300,12 +1171,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:24:10.090556Z", - "start_time": "2023-04-25T18:24:02.423340Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "results_df = pd.DataFrame(results)\n", @@ -1322,12 +1188,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:24:10.237362Z", - "start_time": "2023-04-25T18:24:10.090556Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "# Filter the results to only contain the data of one agent (the Gini coefficient will be the same for the entire population at any time) at the 100th step of each episode\n", @@ -1343,6 +1204,33 @@ ");" ] }, + { + "cell_type": "markdown", + "source": [ + "We can create different kinds of plot from this filtered DataFrame. For example, a point plot with error bars." + ], + "metadata": { + "collapsed": false + } + }, + { + "cell_type": "code", + "outputs": [], + "source": [ + "# Create a point plot with error bars\n", + "g = sns.pointplot(data=results_filtered, x=\"N\", y=\"Gini\", linestyle='none')\n", + "g.figure.set_size_inches(8, 4)\n", + "g.set(\n", + " xlabel=\"Number of agents\",\n", + " ylabel=\"Gini coefficient\",\n", + " title=\"Gini coefficient vs. number of agents\",\n", + ");" + ], + "metadata": { + "collapsed": false + }, + "execution_count": null + }, { "cell_type": "markdown", "metadata": {}, @@ -1356,12 +1244,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:24:10.257241Z", - "start_time": "2023-04-25T18:24:10.239363Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "# First, we filter the results\n", @@ -1369,7 +1252,7 @@ "# Then, print the columns of interest of the filtered data frame\n", "print(\n", " one_episode_wealth.to_string(\n", - " index=False, columns=[\"Step\", \"AgentID\", \"Wealth\"], max_rows=25\n", + " index=False, columns=[\"Step\", \"AgentID\", \"Wealth\"], max_rows=10\n", " )\n", ")\n", "# For a prettier display we can also convert the data frame to html, uncomment to test in a Jupyter Notebook\n", @@ -1387,20 +1270,168 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "ExecuteTime": { - "end_time": "2023-04-25T18:24:10.314122Z", - "start_time": "2023-04-25T18:24:10.258232Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "results_one_episode = results_df[\n", " (results_df.N == 10) & (results_df.iteration == 1) & (results_df.AgentID == 0)\n", "]\n", - "print(results_one_episode.to_string(index=False, columns=[\"Step\", \"Gini\"], max_rows=25))" + "print(results_one_episode.to_string(index=False, columns=[\"Step\", \"Gini\"], max_rows=10))" ] }, + { + "cell_type": "markdown", + "source": [ + "### Analyzing model reporters: Comparing 5 scenarios\n", + "Other insight might be gathered when we compare the Gini coefficient of different scenarios. For example, we can compare the Gini coefficient of a population with 25 agents to the Gini coefficient of a population with 400 agents. While doing this, we increase the number of iterations to 25 to get a better estimate of the Gini coefficient for each population size and get usable error estimations." + ], + "metadata": { + "collapsed": false + } + }, + { + "cell_type": "code", + "outputs": [], + "source": [ + "params = {\"width\": 10, \"height\": 10, \"N\": [5, 10, 20, 40, 80]}\n", + "\n", + "results_5s = mesa.batch_run(\n", + " MoneyModel,\n", + " parameters=params,\n", + " iterations=100,\n", + " max_steps=120,\n", + " number_processes=1,\n", + " data_collection_period=1, # Important, otherwise the datacollector will only collect data of the last time step\n", + " display_progress=True,\n", + ")\n", + "\n", + "results_5s_df = pd.DataFrame(results_5s)" + ], + "metadata": { + "collapsed": false + }, + "execution_count": null + }, + { + "cell_type": "code", + "outputs": [], + "source": [ + "# Again filter the results to only contain the data of one agent (the Gini coefficient will be the same for the entire population at any time)\n", + "results_5s_df_filtered = results_5s_df[(results_5s_df.AgentID == 0)]\n", + "results_5s_df_filtered.head(3)" + ], + "metadata": { + "collapsed": false + }, + "execution_count": null + }, + { + "cell_type": "code", + "outputs": [], + "source": [ + "# Create a lineplot with error bars\n", + "g = sns.lineplot(\n", + " data=results_5s_df,\n", + " x=\"Step\",\n", + " y=\"Gini\",\n", + " hue=\"N\",\n", + " errorbar=(\"ci\", 95),\n", + " palette=\"tab10\",\n", + ")\n", + "g.figure.set_size_inches(8, 4)\n", + "plot_title = \"Gini coefficient for different population sizes\\n(mean over 100 runs, with 95% confidence interval)\"\n", + "g.set(title=plot_title, ylabel=\"Gini coefficient\");" + ], + "metadata": { + "collapsed": false + }, + "execution_count": null + }, + { + "cell_type": "markdown", + "source": [ + "In this case it looks like the Gini coefficient increases slower for smaller populations. This can be because of different things, either because the Gini coefficient is a measure of inequality and the smaller the population, the more likely it is that the agents are all in the same wealth class, or because there are less interactions between agents in smaller populations, which means that the wealth of an agent is less likely to change." + ], + "metadata": { + "collapsed": false + } + }, + { + "cell_type": "markdown", + "source": [ + "### Analyzing agent reporters\n", + "From the agents we collected the wealth and the number of consecutive rounds without a transaction. We can compare the 5 different population sizes by plotting the average number of consecutive rounds without a transaction for each population size.\n", + "\n", + "Note that we're aggregating multiple times here: First we take the average of all agents for each single replication. Then we plot the averages for all replications, with the error band showing the 95% confidence interval of that first average (over all agents). So this error band is representing the uncertainty of the mean value of the number of consecutive rounds without a transaction for each population size." + ], + "metadata": { + "collapsed": false + } + }, + { + "cell_type": "code", + "outputs": [], + "source": [ + "# Calculate the mean of the wealth and the number of consecutive rounds for all agents in each episode\n", + "agg_results_df = (\n", + " results_5s_df.groupby([\"iteration\", \"N\", \"Step\"])\n", + " .agg({\"Wealth\": \"mean\", \"Steps_not_given\": \"mean\"})\n", + " .reset_index()\n", + ")\n", + "agg_results_df.head(3)" + ], + "metadata": { + "collapsed": false + }, + "execution_count": null + }, + { + "cell_type": "code", + "outputs": [], + "source": [ + "# Create a line plot with error bars\n", + "g = sns.lineplot(\n", + " data=agg_results_df, x=\"Step\", y=\"Steps_not_given\", hue=\"N\", palette=\"tab10\"\n", + ")\n", + "g.figure.set_size_inches(8, 4)\n", + "g.set(\n", + " title=\"Average number of consecutive rounds without a transaction for different population sizes\\n(mean with 95% confidence interval)\",\n", + " ylabel=\"Consecutive rounds without a transaction\",\n", + ");" + ], + "metadata": { + "collapsed": false + }, + "execution_count": null + }, + { + "cell_type": "markdown", + "source": [ + "It can be clearly seen that the lower the number of agents, the higher the number of consecutive rounds without a transaction. This is because the agents have fewer interactions with each other and therefore the wealth of an agent is less likely to change." + ], + "metadata": { + "collapsed": false + } + }, + { + "cell_type": "markdown", + "source": [ + "#### General steps for analyzing results\n", + "\n", + "Many other analysis are possible based on the policies, scenarios and uncertainties that you might be interested in. In general, you can follow these steps to do your own analysis:\n", + "\n", + "1. Determine which metrics you want to analyse. Add these as model and agent reporters to the datacollector of your model.\n", + "2. Determine the input parameters you want to vary. Add these as parameters to the batch_run function, using ranges or lists to test different values.\n", + "3. Determine the hyperparameters of the batch_run function. Define the number of iterations, the number of processes, the number of steps, the data collection period, etc.\n", + "4. Run the batch_run function and save the results.\n", + "5. Transform, filter and aggregate the results to get the data you want to analyze. Make sure it's in long format, so that each row represents a single value.\n", + "6. Choose a plot type, what to plot on the x and y axis, which columns to use for the hue. Seaborn also has an amazing [Example Gallery](https://seaborn.pydata.org/examples/index.html).\n", + "7. Plot the data and analyze the results." + ], + "metadata": { + "collapsed": false + } + }, { "cell_type": "markdown", "metadata": {},