Create ScenarioInfo class #94

BainanXia · 2020-02-21T02:48:05Z

This PR consists of three parts:

An empty __init__.py is added to the tests folder to fix the issue when using mock_scenario object in PostREISE, it complains not finding mock_grid in PowerSimData
All functions and attributes of scenario_info object is in a single python script, scenario_info.py under scenario folder. Discussed with @rouille , both of us think the user should be able to access scenario_info in a general sense not just use it to calculate target capacities in scenario design.
A full test of all the functions of the scenario_info object is implemented in test_scenario_info.py. All 6 functions passed the tests via mock scenario.

Note that I noticed the commit message should be a 'feat' instead of 'fix', a force push is conducted to fix this.

rouille · 2020-02-21T18:00:40Z

We should create a tests folder in PowerSimData/powersimdata/scemario and move the test_scenario_info.py there. The tests folder in PowerSimData/powersimdata/ houses the mock objects that are used in the tests.

rouille · 2020-02-21T18:06:03Z

Some lines are too long. All lines should be < 80 characters (PEP8).

rouille · 2020-02-21T18:10:19Z

powersimdata/scenario/scenario_info.py

+            'wind': wind,
+            'hydro': hydro
+        }
+


I would recommend to write a _check_state function that makes sure that the scenario instance that you pass to the constructor is in the analyze state (scenario.state.name == analyze). Otherwise raise an error.

The reason I did not add this after our discussion is it fails the tests. It seems the MockAnalyze object has no 'name' attribute.

AttributeError: 'MockAnalyze' object has no attribute 'name'

Good catch. I had been faking the class so you can import Analyze and do an isinstance on the mock, but I didn't enable the lower-tech way of just checking the name attribute. We should probably add this.

self.name = 'analyze' in MockAnalyze should do it

rouille · 2020-02-21T18:12:28Z

powersimdata/scenario/scenario_info.py

+            print("%s is incorrect." % area)
+            raise Exception('Invalid area')
+        return loadzone_set
+


I guess we can be more specific here and raise a ValueError inplace of a standard Exception.

Currently, area supports loadzone, state, state abbreviation, interconnect and 'all'. Potentially, it will be more in the future, such as combinations of interconnect, state or loadzones. Not sure whether we should put the specific guidance here or in function docs.

rouille · 2020-02-21T18:14:26Z

powersimdata/scenario/scenario_info.py

+            print('start_time falls behind end_time!')
+            raise Exception('Invalid time range')
+        return start_i, end_i
+


Same here. A ValueError would be more explicit.

Should I simply change Exception to ValueError here? I was thinking about several conditions:

start_time and end_time are in wrong format or some random strings.

start_time and end_time are in right format but not in the valid range of the scenario

start_time and end_time are in right format but in wrong order, i.e. should be switched

I would say that ValueError points to wrong argument(s). This would conceren #2 and #3. If you want to check the type you can add another condition that uses isinstance and raise a TypeError if not str.

Just saw the changes you made. Looks good.

danielolsen · 2020-02-21T18:14:48Z

powersimdata/tests/test_scenario_info.py

+    'plant_id': [101, 102, 103, 104, 105, 106],
+    'bus_id': [1001, 1002, 1003, 1004, 1005, 1006],
+    'type': ['solar', 'wind', 'ng', 'coal', 'dfo', 'hydro'],
+    'zone_name': ['zone1', 'zone1', 'zone1', 'zone1', 'zone1', 'zone1'],


Let's make sure this test data covers more edge cases: e.g. 2+ zones, some resources are in both zones, some are in just one or the other.

That's exactly what I did originally and I figured out that in MockGrid we only have one zone in the zone2id attribute. This test uses MockScenario which calls MockAnalyze then takes MockGrid. We could add attributes to the grid but seems not be able to overwrite the existing attributes in the grid.

Let's expand MockGrid to have at least two zones/ids natively.

Can we add other values in the zoneid dict of the MockGrid object?

danielolsen · 2020-02-21T18:16:46Z

powersimdata/tests/test_scenario_info.py

+mock_wind = mock_pg[wind_plant_id] * 4
+mock_hydro = mock_pg[hydro_plant_id] * 1.5
+
+start_time = '2016-01-01 00:00:00'


Similarly, let's have our dates span more than one day, so that we can ensure that check_time_range is comparing full datetimes and not just times. E.g. 6am one day to 3am the next day.

Agree. The function is comparing the index order of the start time and the end time, thus it should work. But you are right we could span the mock profiles to longer range. The down side is it makes the tests less eyeball tractable.

rouille · 2020-02-21T18:17:47Z

powersimdata/scenario/scenario_info.py

+        total_capacity = self.grid.plant[(self.grid.plant['type'] == gentype) &
+                                         (self.grid.plant['zone_name'].isin(loadzone_set))]['Pmax'].sum()
+        if total_capacity == 0:
+            print('Warning: no such type of generator in the area specified!')


There is the warnings package (https://docs.python.org/3/library/warnings.html#module-warnings) that could be useful here.

danielolsen · 2020-02-21T18:22:06Z

powersimdata/scenario/scenario_info.py

+        elif area in {'Texas', 'Western', 'Eastern'}:
+            loadzone_set = interconnect2loadzone[area]
+        elif area == 'all':
+            loadzone_set = list(self.grid.zone2id.keys())


This option does not return a set: we should wrap the grid keys in set() instead of list().

Good catch.

danielolsen · 2020-02-21T18:26:40Z

powersimdata/scenario/scenario_info.py

+        """
+        total_generation = self.get_generation(gentype, area, start_time, end_time)
+        total_profile_resource = self.get_profile_resource(gentype, area, start_time, end_time)
+        if total_profile_resource == 0 and total_generation == 0:


Good thinking ahead.

danielolsen · 2020-02-21T18:28:53Z

powersimdata/scenario/scenario_info.py

+        total_hours = end_i - start_i + 1
+        total_capacity = self.get_capacity(gentype, area)
+        if total_capacity == 0:
+            raise ZeroDivisionError('Division by zero')


Should this create an exception, or return 0? Either one seems okay, but it seems strange to me that it doesn't match get_curtailment

I thought about this before. The idea is to separate between a reasonable zero and an irregular zero:
--A reasonable zero: 1) zero profile, zero generation, thus zero curtailment 2) zero generation, nonzero capacity, zero capacity factor
--An irregular zero: zero generation, zero capacity, also zero capacity factor.
Or we have a better way to handle this difference?

As long as you've thought about it in the context of how we will use this object/method, I'm happy. If we can run the capacity scaling successfully in Western, including zones like WA where there's no solar, then I don't have strong feelings on the underlying implementation. Maybe a more descriptive message saying that the zero comes from zero capacity for the cases where we do run into this error.

BainanXia · 2020-02-21T18:49:59Z

Some lines are too long. All lines should be < 80 characters (PEP8).

Done with this.

BainanXia · 2020-02-21T19:53:28Z

Should I modify MockGrid and MockAnalyze (in PostREISE) as suggested and create two separate PRs at this point @rouille @danielolsen ?

danielolsen · 2020-02-21T21:08:01Z

@BainanXia yes, go ahead.

BainanXia · 2020-02-24T20:36:53Z

The current version of this PR:
-Done with all the suggested changes and pep8 cleanings so far
-Modified MockGrid(in PowerSimData) and MockAnalyze (in PostREISE) to cooperate with scenario_info tests
-Rebased this PR after changes in MockGrid were merged and force-pushed after the rebase
-Add get_demand function and corresponding tests. Now scenario_info object is capable to calculate total demand of a specific area during valid time range of a given scenario.

rouille

Great work
Check the following link: https://github.com/intvenlab/REISE/wiki/Pull-Request-Etiquette. It has been written by Merielle and it standardized the PR process.

The test using MockScenario is extended into 2 zones and 25 hours.

BainanXia requested review from dmuldrew, danielolsen and rouille February 21, 2020 02:48

BainanXia self-assigned this Feb 21, 2020

BainanXia force-pushed the add_scenario_info branch from 3d5e491 to da54b90 Compare February 21, 2020 02:55

rouille reviewed Feb 21, 2020

View reviewed changes

danielolsen reviewed Feb 21, 2020

View reviewed changes

rouille reviewed Feb 21, 2020

View reviewed changes

danielolsen reviewed Feb 21, 2020

View reviewed changes

BainanXia force-pushed the add_scenario_info branch from 2cd4b2f to 1798743 Compare February 21, 2020 23:02

BainanXia requested review from danielolsen, rouille and kasparm February 24, 2020 20:37

rouille approved these changes Feb 25, 2020

View reviewed changes

BainanXia added 4 commits February 27, 2020 17:31

feat: add scenario_info object and corresponding tests

dc17f58

test: add scenario state check and span tests

1613ce2

The test using MockScenario is extended into 2 zones and 25 hours.

feat: add get_demand function and corresponding tests

5f0cc1f

feat: add get_available_resource and corresponding tests

28648be

BainanXia force-pushed the add_scenario_info branch from d15c220 to 28648be Compare February 28, 2020 01:32

BainanXia merged commit 74768f8 into develop Feb 28, 2020

BainanXia deleted the add_scenario_info branch February 28, 2020 01:35

Lab-ITTeam unassigned BainanXia Jul 15, 2020

ahurli mentioned this pull request Mar 11, 2021

Develop into Master #410

Merged

rouille changed the title ~~fix: add scenario_info object and corresponding tests~~ Create Scenarioinfo class Mar 16, 2021

rouille changed the title ~~Create Scenarioinfo class~~ Create ScenarioInfo class Mar 16, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create ScenarioInfo class #94

Create ScenarioInfo class #94

BainanXia commented Feb 21, 2020 •

edited

Loading

rouille commented Feb 21, 2020

rouille commented Feb 21, 2020 •

edited

Loading

rouille Feb 21, 2020 •

edited

Loading

BainanXia Feb 21, 2020

danielolsen Feb 21, 2020

rouille Feb 21, 2020

rouille Feb 21, 2020

BainanXia Feb 21, 2020

rouille Feb 21, 2020

BainanXia Feb 21, 2020

rouille Feb 21, 2020

rouille Feb 21, 2020

danielolsen Feb 21, 2020

BainanXia Feb 21, 2020

danielolsen Feb 21, 2020

rouille Feb 21, 2020

danielolsen Feb 21, 2020

BainanXia Feb 21, 2020

rouille Feb 21, 2020

danielolsen Feb 21, 2020

BainanXia Feb 21, 2020

danielolsen Feb 21, 2020

danielolsen Feb 21, 2020

BainanXia Feb 21, 2020

danielolsen Feb 21, 2020

BainanXia commented Feb 21, 2020

BainanXia commented Feb 21, 2020

danielolsen commented Feb 21, 2020

BainanXia commented Feb 24, 2020

rouille left a comment

Create ScenarioInfo class #94

Create ScenarioInfo class #94

Conversation

BainanXia commented Feb 21, 2020 • edited Loading

rouille commented Feb 21, 2020

rouille commented Feb 21, 2020 • edited Loading

rouille Feb 21, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BainanXia commented Feb 21, 2020

BainanXia commented Feb 21, 2020

danielolsen commented Feb 21, 2020

BainanXia commented Feb 24, 2020

rouille left a comment

Choose a reason for hiding this comment

BainanXia commented Feb 21, 2020 •

edited

Loading

rouille commented Feb 21, 2020 •

edited

Loading

rouille Feb 21, 2020 •

edited

Loading