From 4df52fbead0dccc72be7766cc55d509d426ef489 Mon Sep 17 00:00:00 2001 From: "Wouter J. de Bruin" Date: Thu, 27 Aug 2020 14:15:49 +0200 Subject: [PATCH 01/10] Add changelog template and CHANGELOG.md --- .github/pull_request_template.md | 13 +++++++++++ CHANGELOG.md | 40 ++++++++++++++++++++++++++++++++ 2 files changed, 53 insertions(+) create mode 100644 .github/pull_request_template.md create mode 100644 CHANGELOG.md diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 000000000..cd5c6ca7f --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,13 @@ +*Insert a description of your pull request (PR) here, and check off the boxes below when they are done.* + +--- + +### Contributor checklist + +- [ ] :tada: This PR closes #ISSUE_NUMBER. +- [ ] :scroll: I have broken down my PR into the following tasks: + - [ ] Task 1 + - [ ] Task 2 +- [ ] :robot: I have added tests, or extended existing tests, to cover any new features or bugs fixed in this PR. +- [ ] :book: I have considered adding a new entry in `CHANGELOG.md`. +- [ ] :books: I have considered updating the documentation. \ No newline at end of file diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 000000000..ce78086f0 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,40 @@ +# Change Log for FlowNet +All notable changes to `FlowNet` will be documented in this file. +This project adheres to [Semantic Versioning](https://semver.org/). + +## [Unreleased] +### Added +- [#146](https://github.com/equinor/flownet/pull/146) Added about page to documentation with logo of industry and research institute partners. +- [#138](https://github.com/equinor/flownet/pull/138) Print message to terminal when the schedule is being generated instead of utter silence. +- [#134](https://github.com/equinor/flownet/pull/134) Egg [model](https://github.com/equinor/flownet-testdata/blob/master/egg/ci_config/assisted_history_matching.yml) added in CI. +- [#128](https://github.com/equinor/flownet/pull/128) Add possibility to use the original simulation model volume as volumetric constraints for adding new nodes as well as the option to set a maximum connection length. +- [#115](https://github.com/equinor/flownet/pull/115) Documentation of the preliminary Egg model results have been added. +- [#104](https://github.com/equinor/flownet/pull/104) Adds a new FlowNet logo. +- [#98](https://github.com/equinor/flownet/pull/98) Rock and Aquifer keywords are now allowed to be optional parameters. +- [#95](https://github.com/equinor/flownet/pull/95), [#97](https://github.com/equinor/flownet/pull/97) Relative permeability and `training_set_end_date` are now allowed to be optional. The training set definition will be checked upon parsing the configfile. +- [#90](https://github.com/equinor/flownet/pull/90) Running the analysis workflow is now optional. +- [#77](https://github.com/equinor/flownet/pull/77) Adds a warning message when the `WSTAT` keyword is not present in the original simulation data deck. The default status is `OPEN`. + +### Fixed +- [#141](https://github.com/equinor/flownet/pull/141) Fixed misplacement of `WCONINJH` target (`RATE` / `BHP`). The default `RATE` is however still always used. +- [#137](https://github.com/equinor/flownet/pull/137) After having moved all example data to [flownet-testdata](https://github.com/equinor/flownet-testdata) the examples folder had no use longer. Removed. +- [#136](https://github.com/equinor/flownet/pull/136) The ERT queue name was wrongly required in the config file. It is now allowed to have no value. +- [#119](https://github.com/equinor/flownet/pull/119) Workaround for [equinor/libres#984](https://github.com/equinor/libres/issues/984) +- [#92](https://github.com/equinor/flownet/pull/92) Fixed problem where the last production well would not correctly be shut. +- [#88](https://github.com/equinor/flownet/pull/88), [#106](https://github.com/equinor/flownet/pull/92), [#108](https://github.com/equinor/flownet/pull/108), [#110](https://github.com/equinor/flownet/pull/110) Fixed bugs in parsing the config that were a result of a changed behavior in configsuite where an item that has subitems with subitems, no longer can be tested for all `None` as the subitem with subitems in fact is a `NamedDict` with all `None`. +- [#89](https://github.com/equinor/flownet/pull/89), [#91](https://github.com/equinor/flownet/pull/91), [#99](https://github.com/equinor/flownet/pull/99) Updates and fixes in git workflow. +- [#81](https://github.com/equinor/flownet/pull/81) Various bugfixes in the analysis workflow are now solved. +- [#75](https://github.com/equinor/flownet/pull/75) Resampling via the config path is no longer supported. This PR removes also the resampling option from the config file parser. + +### Changed +- [#132](https://github.com/equinor/flownet/pull/132) The observation uncertainty, used for conditioning models, is now exposed to the user via the configuration file instead of hidden in the `observations.yamlobs.jinja2` template. This PR also introduced a wider usage of passing around the `ConfigSuit.snapshot`. +- [#128](https://github.com/equinor/flownet/pull/117) Simulation models and configuration files used in CI are now coming from [flownet-testdata](https://github.com/equinor/flownet-testdata) and are thus no longer an integral part of the flownet repository. +- [#125](https://github.com/equinor/flownet/pull/125) AHM and prediction `subprocess` call are now being performed through a common `run_ert_subprocess` function call. +- [#123](https://github.com/equinor/flownet/pull/123) Significant speed-up of the fault raytracing by limiting the ray tracing to the bounding box of individual connections instead of ray tracing over all triangles of all faults for each connection. +- [#111](https://github.com/equinor/flownet/pull/111) Phases present in the FlowNet model are to be set manually in configuration by the user. +- [#63](https://github.com/equinor/flownet/pull/63) The default simulator name has now been changed to _Flow_. +- [#80](https://github.com/equinor/flownet/pull/80) FlowNet now supports relative paths in the config file. + +## [0.2.0] - 2020-06-25 + +- No changelog was maintained until this release From 4978eb7e862b12caea13501012447b85242b5f50 Mon Sep 17 00:00:00 2001 From: "Wouter J. de Bruin" Date: Thu, 27 Aug 2020 14:20:08 +0200 Subject: [PATCH 02/10] Add changelog item --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index ce78086f0..f79f229a3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ This project adheres to [Semantic Versioning](https://semver.org/). ## [Unreleased] ### Added +- [#115](https://github.com/equinor/flownet/pull/150) Adds this changelog. - [#146](https://github.com/equinor/flownet/pull/146) Added about page to documentation with logo of industry and research institute partners. - [#138](https://github.com/equinor/flownet/pull/138) Print message to terminal when the schedule is being generated instead of utter silence. - [#134](https://github.com/equinor/flownet/pull/134) Egg [model](https://github.com/equinor/flownet-testdata/blob/master/egg/ci_config/assisted_history_matching.yml) added in CI. From 57d1d171014f891f2161f22c2fde4a2b1dc12559 Mon Sep 17 00:00:00 2001 From: "Wouter J. de Bruin" Date: Thu, 27 Aug 2020 14:25:26 +0200 Subject: [PATCH 03/10] Updated contributing md --- CONTRIBUTING.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 84a6d9be2..30260e532 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -13,23 +13,24 @@ The following is a set of guidelines for contributing to FlowNet. ## Pull Request Process 1. Work on your own fork of the main repo -1. Push your commits and make a draft pull request if you are still working further on it; otherwise make a - normal pull request -1. Check that your pull request passes all tests -1. When all tested have passed and your are happy with your changes, ask for a code review +1. Push your commits and make a draft pull request using the pull request template; follow the + instructions in the template. +1. Check that your pull request passes all tests. +1. When all tests have passed and your are happy with your changes, change your pull request to "ready for review" + and ask for a code review. 1. When your code has been approved - merge your changes. -### Testdata +### Test data -Testdata is available at https://github.com/equinor/flownet-testdata. +Test data is available at https://github.com/equinor/flownet-testdata. Note that if you do changes to the source code that needs corresponding changes in the configuration file, you should make a pull request to the -testdata repository as well, and temporarily change the environment variables +`flownet-testdata` repository as well, and temporarily change the environment variables in the [CI GitHub workflow](./.github/workflows/flownet.yml) to make sure CI downloads your branch during initial automatic testing. -Good practice is to merge the testdata PR first, and then change the workflow +Good practice is to merge the PR on `flownet-testdata` first, and then change the workflow back to official `equinor/master` in the main FlowNet PR. ### Build documentation From 6f8030d683fd424d171fa47506c7be7560770883 Mon Sep 17 00:00:00 2001 From: "Wouter J. de Bruin" Date: Mon, 31 Aug 2020 21:08:01 +0200 Subject: [PATCH 04/10] Added time_avg_open_location perforation handling strategy --- src/flownet/data/from_flow.py | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/src/flownet/data/from_flow.py b/src/flownet/data/from_flow.py index 1ade88e7c..3826087be 100644 --- a/src/flownet/data/from_flow.py +++ b/src/flownet/data/from_flow.py @@ -1,6 +1,7 @@ import warnings from pathlib import Path from typing import Union, List +import datetime import numpy as np import pandas as pd @@ -69,6 +70,37 @@ def multi_xyz_append(append_obj_list): elif self._perforation_handling_strategy == "multiple": xyz = [global_conns] coord_append = multi_xyz_append + elif self._perforation_handling_strategy == "time_avg_open_location": + connection_open_time = {} + + for conn_status in self._wells[well_name]: + time = datetime.datetime.strptime( + str(conn_status.simulationTime()), "%Y-%m-%d %H:%M:%S" + ) + for i, connection in enumerate(conn_status.globalConnections()): + if connection.ijk() not in connection_open_time: + connection_open_time[connection.ijk()] = 0 + elif connection.isOpen(): + connection_open_time[connection.ijk()] += ( + time - prev_time + ).total_seconds() + else: + connection_open_time[connection.ijk()] += 0 + + prev_time = time + + xyz = np.array([0.0, 0.0, 0.0]) + total_time = sum(connection_open_time.values()) + for connection, time in connection_open_time.items(): + if total_time > 0: + xyz += np.multiply( + np.array(self._grid.get_xyz(ijk=connection)), time + ) + else: + xyz = np.array(self._grid.get_xyz(ijk=connection)) + if total_time > 0: + xyz = tuple(np.divide(xyz, total_time)) + else: raise Exception( f"perforation strategy {self._perforation_handling_strategy} unknown" From 0bc44891035537b4bd46dd933a45796b4bce81b4 Mon Sep 17 00:00:00 2001 From: "Wouter J. de Bruin" Date: Tue, 1 Sep 2020 09:37:48 +0200 Subject: [PATCH 05/10] Code optimizations --- src/flownet/data/from_flow.py | 36 +++++++++++++++++++++-------------- 1 file changed, 22 insertions(+), 14 deletions(-) diff --git a/src/flownet/data/from_flow.py b/src/flownet/data/from_flow.py index 3826087be..39d6c88fa 100644 --- a/src/flownet/data/from_flow.py +++ b/src/flownet/data/from_flow.py @@ -43,6 +43,7 @@ def __init__( self._perforation_handling_strategy: str = perforation_handling_strategy + # pylint: disable=too-many-branches def _coordinates(self) -> pd.DataFrame: """ Function to extract well coordinates from an Flow simulation. @@ -73,40 +74,47 @@ def multi_xyz_append(append_obj_list): elif self._perforation_handling_strategy == "time_avg_open_location": connection_open_time = {} - for conn_status in self._wells[well_name]: + for i, conn_status in enumerate(self._wells[well_name]): time = datetime.datetime.strptime( str(conn_status.simulationTime()), "%Y-%m-%d %H:%M:%S" ) - for i, connection in enumerate(conn_status.globalConnections()): + if i == 0: + prev_time = time + + for connection in conn_status.globalConnections(): if connection.ijk() not in connection_open_time: - connection_open_time[connection.ijk()] = 0 + connection_open_time[connection.ijk()] = 0.0 elif connection.isOpen(): connection_open_time[connection.ijk()] += ( time - prev_time ).total_seconds() else: - connection_open_time[connection.ijk()] += 0 + connection_open_time[connection.ijk()] += 0.0 prev_time = time - xyz = np.array([0.0, 0.0, 0.0]) - total_time = sum(connection_open_time.values()) - for connection, time in connection_open_time.items(): - if total_time > 0: + xyz = np.zeros((1, 3), dtype=np.float64) + total_open_time = sum(connection_open_time.values()) + + if total_open_time > 0: + for connection, open_time in connection_open_time.items(): xyz += np.multiply( - np.array(self._grid.get_xyz(ijk=connection)), time + np.array(self._grid.get_xyz(ijk=connection)), + open_time / total_open_time, + ) + else: + for connection, open_time in connection_open_time.items(): + xyz += np.divide( + np.array(self._grid.get_xyz(ijk=connection)), + len(connection_open_time.items()), ) - else: - xyz = np.array(self._grid.get_xyz(ijk=connection)) - if total_time > 0: - xyz = tuple(np.divide(xyz, total_time)) else: raise Exception( f"perforation strategy {self._perforation_handling_strategy} unknown" ) - coord_append([well_name, *xyz]) + coord_append([well_name, *tuple(*xyz)]) return pd.DataFrame(coords, columns=["WELL_NAME", "X", "Y", "Z"]) From 0a407a7fa834201f5a8f4e69cfc2726467c5679f Mon Sep 17 00:00:00 2001 From: "Wouter J. de Bruin" Date: Tue, 1 Sep 2020 09:43:06 +0200 Subject: [PATCH 06/10] Reverted line 77 --- src/flownet/data/from_flow.py | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/src/flownet/data/from_flow.py b/src/flownet/data/from_flow.py index 39d6c88fa..38312d587 100644 --- a/src/flownet/data/from_flow.py +++ b/src/flownet/data/from_flow.py @@ -93,28 +93,30 @@ def multi_xyz_append(append_obj_list): prev_time = time - xyz = np.zeros((1, 3), dtype=np.float64) + xyz_values = np.zeros((1, 3), dtype=np.float64) total_open_time = sum(connection_open_time.values()) if total_open_time > 0: for connection, open_time in connection_open_time.items(): - xyz += np.multiply( + xyz_values += np.multiply( np.array(self._grid.get_xyz(ijk=connection)), open_time / total_open_time, ) else: for connection, open_time in connection_open_time.items(): - xyz += np.divide( + xyz_values += np.divide( np.array(self._grid.get_xyz(ijk=connection)), len(connection_open_time.items()), ) + xyz = tuple(*xyz_values) + else: raise Exception( f"perforation strategy {self._perforation_handling_strategy} unknown" ) - coord_append([well_name, *tuple(*xyz)]) + coord_append([well_name, *xyz]) return pd.DataFrame(coords, columns=["WELL_NAME", "X", "Y", "Z"]) From b92e74332384dfde9a1140ab4c8151050cc3e928 Mon Sep 17 00:00:00 2001 From: "Wouter J. de Bruin" Date: Tue, 1 Sep 2020 10:43:11 +0200 Subject: [PATCH 07/10] Test with own test data --- .github/workflows/flownet.yml | 4 ++-- setup.py | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/flownet.yml b/.github/workflows/flownet.yml index 31dfea749..77faca658 100644 --- a/.github/workflows/flownet.yml +++ b/.github/workflows/flownet.yml @@ -53,10 +53,10 @@ jobs: INPUT_MODEL_FOLDER: ./flownet-testdata/${{ matrix.flownet-model }}/input_model # If you want the CI to (temporarily) run against your fork of the testdada, # change the value her from "equinor" to your username. - TESTDATA_REPO_OWNER: equinor + TESTDATA_REPO_OWNER: wouterjdb # If you want the CI to (temporarily) run against another branch than master, # change the value her from "master" to the relevant branch name. - TESTDATA_REPO_BRANCH: master + TESTDATA_REPO_BRANCH: i157-new-strat run: | git clone --depth 1 --branch $TESTDATA_REPO_BRANCH https://github.com/$TESTDATA_REPO_OWNER/flownet-testdata.git if [ -f "$INPUT_MODEL_FOLDER/${{ matrix.flownet-model }}.tar.gz" ]; then diff --git a/setup.py b/setup.py index 19ce603f9..d2beff345 100644 --- a/setup.py +++ b/setup.py @@ -65,6 +65,6 @@ "Operating System :: OS Independent", "Natural Language :: English", "Topic :: Scientific/Engineering", - "License :: OSI Approved :: GNU General Public License v3 (GPLv3)", + "License :: OSI Approved :: GNU General Public License v3 (GPLv3) ", ], ) From 62bc697293f048561214111c9d95a978acea3328 Mon Sep 17 00:00:00 2001 From: "Wouter J. de Bruin" Date: Tue, 1 Sep 2020 10:51:03 +0200 Subject: [PATCH 08/10] Add changelog item. --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index f79f229a3..3dec56bf5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ This project adheres to [Semantic Versioning](https://semver.org/). ## [Unreleased] ### Added +- [#157](https://github.com/equinor/flownet/pull/157) Adds a new 'time-weighted average open perforation location' perforation strategy called `time_avg_open_location`. - [#115](https://github.com/equinor/flownet/pull/150) Adds this changelog. - [#146](https://github.com/equinor/flownet/pull/146) Added about page to documentation with logo of industry and research institute partners. - [#138](https://github.com/equinor/flownet/pull/138) Print message to terminal when the schedule is being generated instead of utter silence. From ab8d94bb6309a3c8e84085f470e004f947966f6e Mon Sep 17 00:00:00 2001 From: "Wouter J. de Bruin" Date: Tue, 1 Sep 2020 10:59:13 +0200 Subject: [PATCH 09/10] Black --- tests/test_one_dimensional.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tests/test_one_dimensional.py b/tests/test_one_dimensional.py index 4fdb9b70c..0df8fc23b 100644 --- a/tests/test_one_dimensional.py +++ b/tests/test_one_dimensional.py @@ -4,8 +4,7 @@ def test_one_dimensional(tmp_path: pathlib.Path) -> None: - """Single one dimensional flow Model example - """ + """Single one dimensional flow Model example""" n_grid_cells = 10 model_cross_section_area = 40 # m^2 From 194377a7d477835831ae33b6607bd96ed19f4964 Mon Sep 17 00:00:00 2001 From: "Wouter J. de Bruin" Date: Tue, 1 Sep 2020 12:14:06 +0200 Subject: [PATCH 10/10] Revert setup and workflow --- .github/workflows/flownet.yml | 4 ++-- setup.py | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/flownet.yml b/.github/workflows/flownet.yml index 77faca658..31dfea749 100644 --- a/.github/workflows/flownet.yml +++ b/.github/workflows/flownet.yml @@ -53,10 +53,10 @@ jobs: INPUT_MODEL_FOLDER: ./flownet-testdata/${{ matrix.flownet-model }}/input_model # If you want the CI to (temporarily) run against your fork of the testdada, # change the value her from "equinor" to your username. - TESTDATA_REPO_OWNER: wouterjdb + TESTDATA_REPO_OWNER: equinor # If you want the CI to (temporarily) run against another branch than master, # change the value her from "master" to the relevant branch name. - TESTDATA_REPO_BRANCH: i157-new-strat + TESTDATA_REPO_BRANCH: master run: | git clone --depth 1 --branch $TESTDATA_REPO_BRANCH https://github.com/$TESTDATA_REPO_OWNER/flownet-testdata.git if [ -f "$INPUT_MODEL_FOLDER/${{ matrix.flownet-model }}.tar.gz" ]; then diff --git a/setup.py b/setup.py index d2beff345..19ce603f9 100644 --- a/setup.py +++ b/setup.py @@ -65,6 +65,6 @@ "Operating System :: OS Independent", "Natural Language :: English", "Topic :: Scientific/Engineering", - "License :: OSI Approved :: GNU General Public License v3 (GPLv3) ", + "License :: OSI Approved :: GNU General Public License v3 (GPLv3)", ], )