feat: add module to 'design' scenarios, starting with renewable stub scaling #80

danielolsen · 2020-01-08T00:55:52Z

This feature adds the automatic scaling of renewable stub branches that was used in the Western, Eastern, and USA Scenario runs. It analyzes the topology of the network, starting at each renewable generator and finding 'stub' branches--branches which are the only path between a given node and anywhere else in the network. In graph theory, these branches would be considered 'bridges', although this is currently a cruder implementation that will not detect when a stub is 'tree-shaped' rather than a series of singular links (this could be improved via the use of networkx or a similar graph theory package).

Once a builder is assigned, this feature is called via:
scenario.state.builder.scale_renewable_stubs(), with optional kwargs passed through to the implementation in powersimdata/input/design.py.

To support testing, I've created mock_builder.py and mock_change_table.py to contain new mock objects used in the tests for the functions in powersimdata/input/design.py.

Unit tests:

python -m unittest
....................
----------------------------------------------------------------------
Ran 20 tests in 0.090s

Implementation test. First we start with a vanilla Western scenario. The change_table is initially empty, and remains functionally empty after stub scaling (since renewables haven't been overbuilt). Once we scale up renewables, calling the stub scaling again adds branch scaling to the change_table.

C:\Users\dolsen\repos\iv\PowerSimData> python
Python 3.7.4 (tags/v3.7.4:e09359112e, Jul  8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from powersimdata.scenario.scenario import Scenario
>>> scenario = Scenario('')
>>> scenario.state.set_builder(['Western'])
--> Loading Western interconnect
Loading zone
Loading sub
Loading bus2sub
Loading bus
Loading plant
Loading plant cost
Loading branch
Loading DC line
--> Summary
# Existing study
base | ca2045 | test | pos_base
# Available profiles
demand: ca2020 | ca2030 | v3 | v4
hydro: v1 | v2
solar: v2
wind: v1 | v2
>>> scenario.state.builder.change_table.ct
{}
>>> scenario.state.builder.scale_renewable_stubs(verbose=False)
--> Loading Western interconnect
Loading zone
Loading sub
Loading bus2sub
Loading bus
Loading plant
Loading plant cost
Loading branch
Loading DC line
>>> scenario.state.builder.change_table.ct
{'branch': {'branch_id': {}}}
>>> scenario.state.builder.change_table.scale_plant_capacity(
...     'solar', zone_name={'Washington': 5, 'Arizona': 2.5})
>>> scenario.state.builder.scale_renewable_stubs(verbose=False)
--> Loading Western interconnect
Loading zone
Loading sub
Loading bus2sub
Loading bus
Loading plant
Loading plant cost
Loading branch
Loading DC line
>>> scenario.state.builder.change_table.ct
{'branch': {'branch_id': {89285: 2.8359614273946807, 89290: 2.602842944083592, 89291: 3.1023323953799795, 89460: 3.3102947835576577, 89461: 3.9487181015384616, 89462: 3.9487181015384616, 97296: 1.974359051282051, 97331: 1.974359051282051, 97363: 1.444444368888889, 97364: 1.9379843947525344, 97365: 1.7424846129289704, 97400: 1.9635627287449395, 97428: 2.008547085042735, 97482: 1.9615384903846154, 97533: 2.000000028846154, 97609: 1.974359051282051, 97617: 1.9635627287449395}}, 'solar': {'zone_id': {201: 5, 209: 2.5}}}

fix: ensure renewable stub scaling works starting with empty change table fix: ensure renewable stub scaling works starting with empty change table (for real) fix: ensure renewable stub scaling works starting with empty change table (for real for real)

rouille · 2020-01-08T01:38:19Z

I will look in more details tomorrow or, if I don't find the time, Thursday. Quick question, did you consider having scale_renewable_stubs in the ChangeTable object located in powersimdata.input.change_table. This is where the scaling function are and the rference grid is loaded there.

danielolsen · 2020-01-08T17:43:53Z

@rouille do you mean where the method is invoked from, or where the code lives? I am agnostic on where it's invoked from, and it could be from change_table if you think that fits better. I see that the ChangeTable has a Grid object which we could use to do topology analysis.

For where the code lives, I like having a new design.py module, which will eventually include other design features as well such as upgrading mesh branch branches.

rouille · 2020-01-08T17:58:24Z

I am talking about where the method is invoked. Right now it is in the Create object. I believe it should be in the ChangeTable object and instead of calling: scenario.state.builder.scale_renewable_stubs(verbose=False), you would do scenario.state.builder.change_table.scale_renewable_stubs(verbose=False).

danielolsen · 2020-01-08T18:10:53Z

Okay, I will change it.

danielolsen · 2020-01-08T19:24:30Z

Invocation has been moved. Bonus, it cuts down on the printing because we don't instantiate a new grid!

Unit testing:

PS C:\Users\dolsen\repos\iv\PowerSimData> python -m unittest
....................
----------------------------------------------------------------------
Ran 20 tests in 0.074s

OK

Integration testing:

>>> from powersimdata.scenario.scenario import Scenario
>>> scenario = Scenario('')
>>> scenario.state.set_builder(['Western'])
--> Loading Western interconnect
Loading zone
Loading sub
Loading bus2sub
Loading bus
Loading plant
Loading plant cost
Loading branch
Loading DC line
--> Summary
# Existing study
base | ca2045 | test | pos_base
# Available profiles
demand: ca2020 | ca2030 | v3 | v4
hydro: v1 | v2
solar: v2
wind: v1 | v2
>>> scenario.state.builder.change_table.ct
{}
>>> scenario.state.builder.change_table.scale_renewable_stubs(verbose=False)
>>> scenario.state.builder.change_table.ct
{'branch': {'branch_id': {}}}
>>> scenario.state.builder.change_table.scale_plant_capacity(
...     'solar', zone_name={'Washington': 5, 'Arizona': 2.5})
>>> scenario.state.builder.change_table.scale_renewable_stubs(verbose=False)
>>> scenario.state.builder.change_table.ct
{'branch': {'branch_id': {89285: 2.8359614273946807, 89290: 2.602842944083592, 89291: 3.1023323953799795, 89460: 3.3102947835576577, 89461: 3.9487181015384616, 89462: 3.9487181015384616, 97296: 1.974359051282051, 97331: 1.974359051282051, 97363: 1.444444368888889, 97364: 1.9379843947525344, 97365: 1.7424846129289704, 97400: 1.9635627287449395, 97428: 2.008547085042735, 97482: 1.9615384903846154, 97533: 2.000000028846154, 97609: 1.974359051282051, 97617: 1.9635627287449395}}, 'solar': {'zone_id': {201: 5, 209: 2.5}}}

rouille · 2020-01-08T20:16:38Z

Yes, I was expecting the printing to be reduced. So, if I read the code well, it seems that scale_renewable_stubs won't interfere with scale_branch_capacity, in other words, we can do both scaling if we want to.

danielolsen · 2020-01-08T20:27:26Z

If you call scale_branch_capacity and then scale_renewable_stubs, you're fine. If you call scale_renewable_stubs and then scale_branch_capacity, then you're in trouble because scale_branch_capacity will destroy any previous branch changes.

rouille · 2020-01-08T22:31:39Z

I remember doing that. The idea behind was that if you make a mistake in the scaling, you can easily overwrite without having to manually pop entries in the dictionary.

danielolsen · 2020-01-08T22:33:27Z

Maybe it would be worth adding an option to the scale_branch_capacity call, whether the new scaling should totally replace any existing scaling or if they should merge together?

rouille · 2020-01-08T22:41:04Z

Not a bad idea

danielolsen · 2020-01-08T23:56:15Z

scale_branch_capacity() no longer throws out all previous branch scale factors. Same for scale_plant_capacity, scale_dcline_capacity, scale_demand, and add_storage_capacity. To easily clear each of these, we have the ChangeTable.clear() method, which accepts a set of keywords (or a single keyword str) and clears the appropriate parts of the change table dict. The demo shows the use of this new method, and that now scale_renewable_stubs and scale_branch_capacity can be called in either order and the resulting change table dicts are identical.

Demo:

>>> from powersimdata.scenario.scenario import Scenario
>>> scenario = Scenario('')
>>> scenario.state.set_builder(['Western'])
--> Loading Western interconnect
Loading zone
Loading sub
Loading bus2sub
Loading bus
Loading plant
Loading plant cost
Loading branch
Loading DC line
--> Summary
# Existing study
base | ca2045 | test | pos_base
# Available profiles
demand: ca2020 | ca2030 | v3 | v4
hydro: v1 | v2
solar: v2
wind: v1 | v2
>>> scenario.state.builder.change_table.ct
{}
>>> scenario.state.builder.change_table.scale_plant_capacity(
...     'solar', zone_name={'Washington': 5, 'Arizona': 2.5})
>>> scenario.state.builder.change_table.ct
{'solar': {'zone_id': {201: 5, 209: 2.5}}}
>>> scenario.state.builder.change_table.scale_renewable_stubs(verbose=False)
>>> scenario.state.builder.change_table.scale_branch_capacity(
...     branch_id={90000: 10})
>>> scenario.state.builder.change_table.ct
{'solar': {'zone_id': {201: 5, 209: 2.5}}, 'branch': {'branch_id': {89285: 2.8359614273946807, 89290: 2.602842944083592, 89291: 3.1023323953799795, 89460: 3.3102947835576577, 89461: 3.9487181015384616, 89462: 3.9487181015384616, 97296: 1.974359051282051, 97331: 1.974359051282051, 97363: 1.444444368888889, 97364: 1.9379843947525344, 97365: 1.7424846129289704, 97400: 1.9635627287449395, 97428: 2.008547085042735, 97482: 1.9615384903846154, 97533: 2.000000028846154, 97609: 1.974359051282051, 97617: 1.9635627287449395, 90000: 10}}}
>>> ct1 = scenario.state.builder.change_table.ct.copy()
>>> scenario.state.builder.change_table.clear({'plant'})
>>> scenario.state.builder.change_table.ct
{'branch': {'branch_id': {89285: 2.8359614273946807, 89290: 2.602842944083592, 89291: 3.1023323953799795, 89460: 3.3102947835576577, 89461: 3.9487181015384616, 89462: 3.9487181015384616, 97296: 1.974359051282051, 97331: 1.974359051282051, 97363: 1.444444368888889, 97364: 1.9379843947525344, 97365: 1.7424846129289704, 97400: 1.9635627287449395, 97428: 2.008547085042735, 97482: 1.9615384903846154, 97533: 2.000000028846154, 97609: 1.974359051282051, 97617: 1.9635627287449395, 90000: 10}}}
>>> scenario.state.builder.change_table.clear()
>>> scenario.state.builder.change_table.ct
{}
>>> scenario.state.builder.change_table.scale_plant_capacity(
...     'solar', zone_name={'Washington': 5, 'Arizona': 2.5})
>>> scenario.state.builder.change_table.scale_branch_capacity(
...     branch_id={90000: 10})
>>> scenario.state.builder.change_table.scale_renewable_stubs(verbose=False)
>>> scenario.state.builder.change_table.ct
{'solar': {'zone_id': {201: 5, 209: 2.5}}, 'branch': {'branch_id': {90000: 10, 89285: 2.8359614273946807, 89290: 2.602842944083592, 89291: 3.1023323953799795, 89460: 3.3102947835576577, 89461: 3.9487181015384616, 89462: 3.9487181015384616, 97296: 1.974359051282051, 97331: 1.974359051282051, 97363: 1.444444368888889, 97364: 1.9379843947525344, 97365: 1.7424846129289704, 97400: 1.9635627287449395, 97428: 2.008547085042735, 97482: 1.9615384903846154, 97533: 2.000000028846154, 97609: 1.974359051282051, 97617: 1.9635627287449395}}}
>>> ct2 = scenario.state.builder.change_table.ct.copy()
>>> ct1 == ct2
True

rouille · 2020-01-09T00:01:37Z

Looks good. Thanks.

rouille

Super duper nice

…ious scalings, add clear() method

BainanXia

Using this for a while and finally I got a chance to give a pass of the code. Nice work.

danielolsen added 2 commits January 7, 2020 16:45

tests: adding mock objects and tests for testing design.py

db8b854

danielolsen requested review from rouille and BainanXia January 8, 2020 00:55

danielolsen assigned danielolsen, rouille and BainanXia Jan 8, 2020

refactor: move scale_renewable_stubs to change_table

d8a7168

rouille approved these changes Jan 9, 2020

View reviewed changes

feat: prevent change_table scaling from automatically discarding prev…

3620e52

…ious scalings, add clear() method

danielolsen force-pushed the design_transmission branch from 8bd0439 to 3620e52 Compare January 9, 2020 00:05

danielolsen merged commit bc72c96 into develop Jan 9, 2020

BainanXia approved these changes Jan 9, 2020

View reviewed changes

danielolsen deleted the design_transmission branch February 26, 2020 00:48

Lab-ITTeam unassigned BainanXia and danielolsen Jul 15, 2020

danielolsen mentioned this pull request Dec 2, 2020

Improve clearing of Change Table entries #348

Merged

ahurli mentioned this pull request Mar 11, 2021

Develop into Master #410

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add module to 'design' scenarios, starting with renewable stub scaling #80

feat: add module to 'design' scenarios, starting with renewable stub scaling #80

danielolsen commented Jan 8, 2020

rouille commented Jan 8, 2020

danielolsen commented Jan 8, 2020 •

edited

Loading

rouille commented Jan 8, 2020

danielolsen commented Jan 8, 2020

danielolsen commented Jan 8, 2020

rouille commented Jan 8, 2020

danielolsen commented Jan 8, 2020

rouille commented Jan 8, 2020

danielolsen commented Jan 8, 2020

rouille commented Jan 8, 2020

danielolsen commented Jan 8, 2020

rouille commented Jan 9, 2020

rouille left a comment

BainanXia left a comment

feat: add module to 'design' scenarios, starting with renewable stub scaling #80

feat: add module to 'design' scenarios, starting with renewable stub scaling #80

Conversation

danielolsen commented Jan 8, 2020

rouille commented Jan 8, 2020

danielolsen commented Jan 8, 2020 • edited Loading

rouille commented Jan 8, 2020

danielolsen commented Jan 8, 2020

danielolsen commented Jan 8, 2020

rouille commented Jan 8, 2020

danielolsen commented Jan 8, 2020

rouille commented Jan 8, 2020

danielolsen commented Jan 8, 2020

rouille commented Jan 8, 2020

danielolsen commented Jan 8, 2020

rouille commented Jan 9, 2020

rouille left a comment

Choose a reason for hiding this comment

BainanXia left a comment

Choose a reason for hiding this comment

danielolsen commented Jan 8, 2020 •

edited

Loading