feat: more wind power curves #53

danielolsen · 2019-12-13T22:25:53Z

This branch contains changes to the way wind profiles are generated that have been implemented in the latest profiles for Eastern. EIA Form 860 data are used to create state average turbine power curves (based on turbine models installed at real wind farms, the heights they're installed at, and the power curve for each turbine). When power_curves.py is imported, state power curves are build and saved if they are not already present in the expected directory.

Where turbine power curves are not available, or there are no wind farms listed in EIA Form 860, the IEC Class 2 curve is used by default. Tests are contained in test_power_curves.py to ensure that these functions are working properly.

Wind power profiles for farms are generated from state average turbine power curves based on a distribution of wind speeds throughout an hour (temporally) and throughout a farm (spatially). More information is given in the README.

python -m unittest -v
test_build_state_curves (prereise.gather.winddata.rap.tests.test_power_curves.TestBuildStateCurves) ... building state_power_curves
ok
test_shift_turbine_curve_higher_hub (prereise.gather.winddata.rap.tests.test_power_curves.TestShiftTurbineCurve) ... ok
test_shift_turbine_curve_lower_hub (prereise.gather.winddata.rap.tests.test_power_curves.TestShiftTurbineCurve) ... ok
test_shift_turbine_curve_sameheight_hub (prereise.gather.winddata.rap.tests.test_power_curves.TestShiftTurbineCurve) ... ok

----------------------------------------------------------------------
Ran 4 tests in 0.015s

OK

merrielle

I left a bunch of comments, most of them are small. Nice tests!

merrielle · 2019-12-17T22:35:46Z

README.md

+Once the U and V components of the wind are converted to a non-directional
+wind speed magnitude, this speed is converted to power using wind turbine


Don't most wind turbines only work with wind in a specific direction?

I believe all modern turbines pivot to follow the wind.

merrielle · 2019-12-17T22:37:47Z

README.md

+wind speed. This distribution tends to boost the power produced at lower
+wind speeds (since the power curve in this region is convex) and lower the
+power produced at higher wind speeds (since the power curve in this region is
+concave as the turbine tops out, and shuts down at higher wind speeds). This
+tracks with the wind-farm level data shown in NREL's validation report.


merrielle · 2019-12-17T23:24:26Z

prereise/gather/winddata/rap/power_curves.py

+
+data_dir = path.abspath(path.join(path.dirname(__file__), '..', 'data'))
+Form860 = get_form_860(data_dir)
+powercurves_path = path.join(data_dir, 'PowerCurves.csv')


Nice. I believe this makes the paths work on both windows and mac?

path.join makes path-gluing work on all systems. path.abspath takes any relative path (including ..s) and transforms it to an absolute path.

merrielle · 2019-12-17T23:26:36Z

prereise/gather/winddata/rap/power_curves.py

+    if not path.isdir(data_dir):
+        raise ValueError('data_dir is not a valid directory')


merrielle · 2019-12-17T23:29:29Z

prereise/gather/winddata/rap/power_curves.py

+    if not isinstance(year, int):
+        raise TypeError('year is not an int')
+
+    form_860_filename = ''.join(['3_2_Wind_Y', str(year), '.csv'])


hot tip: you can use a format string and it's way simpler: f'3_2_Wind_Y{year}.csv'

I hadn't seen this before! Very handy.

merrielle · 2019-12-17T23:36:48Z

prereise/gather/winddata/rap/power_curves.py

+    except FileNotFoundError:
+        regex_str = r'3_2_Wind_Y(\d{4}).csv'
+        matching_years = [
+            re.match(regex_str, f).group(1)
+            for f in os.listdir(data_dir) if re.match(regex_str, f)]
+        err_msg = ' '.join(['form data for year', str(year), 'not found. '])
+        err_msg += 'Years with data: ' + ', '.join(matching_years)
+        raise ValueError(err_msg)


Here's some stuff on glob if you want to check it out https://docs.python.org/3/library/glob.html

Maybe I'm not grepping glob yet, but I think os + regex is a bit clearer for this application. I want to look through the directory, for filenames that match a pattern, and then extract part of that filename.

merrielle · 2019-12-17T23:43:27Z

prereise/gather/winddata/rap/power_curves.py

+data_dir = path.abspath(path.join(path.dirname(__file__), '..', 'data'))
+Form860 = get_form_860(data_dir)
+powercurves_path = path.join(data_dir, 'PowerCurves.csv')
+PowerCurves = pd.read_csv(powercurves_path, index_col=0, header=None).T
+PowerCurves.set_index('Speed bin (m/s)', inplace=True)
+statepowercurves_path = path.join(data_dir, 'StatePowerCurves.csv')
+try:
+    StatePowerCurves = pd.read_csv(statepowercurves_path, index_col=0)
+except FileNotFoundError:
+    StatePowerCurves = build_state_curves(Form860, PowerCurves, rsd=0.4)
+    StatePowerCurves.to_csv(statepowercurves_path)


Discussed in person and we're planning on making this into two functions instead

merrielle · 2019-12-17T23:52:00Z

prereise/gather/winddata/rap/power_curves.py

+        cumulative_curve = np.zeros_like(curve_x)
+        cumulative_capacity = 0
+        state_wind_farms = Form860[Form860['State'] == s]
+        for i, f in enumerate(state_wind_farms.index):


Didn't know enumerate() was a thing! That's super useful. Also I think you might be able to operate over the entire dataframe at once instead of looping over it. Let me get back to you on that.

enumerate is great. It beats the hell out of for i in range(len(foo)):

merrielle · 2019-12-17T23:58:18Z

prereise/gather/winddata/rap/power_curves.py

+        for s in states:
+            x = state_curves.index
+            y = np.zeros_like(x)
+            for i, w in enumerate(x):


starting to get hard to follow the variable names

Good point.

merrielle · 2019-12-17T23:59:32Z

prereise/gather/winddata/rap/power_curves.py

+                sample_points = np.logical_and((x > min_point),(x < max_point))
+                cdf_points = norm.cdf(x[sample_points], loc=w, scale=sd)
+                pdf_points = np.concatenate((np.zeros(1), np.diff(cdf_points)))
+                #pdf_points *= 1 / np.sum(pdf_points)


forgot to delete this

danielolsen · 2019-12-19T01:02:11Z

Changes made based on the review by @merrielle. @rouille, I think you'll want to take a look at some of the changes I made here. The biggest one is that get_power() now takes the PowerCurves and StatePowerCurves dataframes as inputs, and therefore avoids running any code upon import of power_curves.py. When we want to call get_power(), first we have to import and call get_turbine_power_curves() and get_state_power_curves(), and then we pass these on each call of get_power(). I made what I think are the appropriate changes in rap.py and impute.py, but you may have other notebook code that I can't see.

More tests added as well.

> python -m unittest -v
test_build_state_curves (prereise.gather.winddata.rap.tests.test_power_curves.TestBuildStateCurves) ... building state_power_curves
ok
test_bad_dir (prereise.gather.winddata.rap.tests.test_power_curves.TestGetForm860) ... ok
test_bad_year (prereise.gather.winddata.rap.tests.test_power_curves.TestGetForm860) ... ok
test_default_year (prereise.gather.winddata.rap.tests.test_power_curves.TestGetForm860) ... ok
test_good_year (prereise.gather.winddata.rap.tests.test_power_curves.TestGetForm860) ... ok
test_year_str (prereise.gather.winddata.rap.tests.test_power_curves.TestGetForm860) ... ok
test_get_power_badstate_ten_different_default (prereise.gather.winddata.rap.tests.test_power_curves.TestGetPower) ... AR not found, defaulting to GE 1.5 SLE
ok
test_get_power_default_five (prereise.gather.winddata.rap.tests.test_power_curves.TestGetPower) ... foo not found, defaulting to IEC class 2
ok
test_get_power_default_ten (prereise.gather.winddata.rap.tests.test_power_curves.TestGetPower) ... foo not found, defaulting to IEC class 2
ok
test_get_power_default_thirty (prereise.gather.winddata.rap.tests.test_power_curves.TestGetPower) ... foo not found, defaulting to IEC class 2
ok
test_get_power_default_twenty (prereise.gather.winddata.rap.tests.test_power_curves.TestGetPower) ... foo not found, defaulting to IEC class 2
ok
test_get_power_default_zero (prereise.gather.winddata.rap.tests.test_power_curves.TestGetPower) ... foo not found, defaulting to IEC class 2
ok
test_get_power_named_ten (prereise.gather.winddata.rap.tests.test_power_curves.TestGetPower) ... ok
test_get_power_named_ten2 (prereise.gather.winddata.rap.tests.test_power_curves.TestGetPower) ... ok
test_get_state_power_curves (prereise.gather.winddata.rap.tests.test_power_curves.TestGetStatePowerCurves) ... ok
test_get_turbine_power_curves (prereise.gather.winddata.rap.tests.test_power_curves.TestGetTurbinePowerCurves) ... ok
test_shift_turbine_curve_higher_hub (prereise.gather.winddata.rap.tests.test_power_curves.TestShiftTurbineCurve) ... ok
test_shift_turbine_curve_lower_hub (prereise.gather.winddata.rap.tests.test_power_curves.TestShiftTurbineCurve) ... ok
test_shift_turbine_curve_sameheight_hub (prereise.gather.winddata.rap.tests.test_power_curves.TestShiftTurbineCurve) ... ok

----------------------------------------------------------------------
Ran 19 tests in 0.485s

OK

rouille · 2019-12-19T17:56:21Z

It is working.

merrielle

👍 👍 This looks awesome. Great job with the changes and excellent tests!

merrielle · 2019-12-19T21:28:59Z

prereise/gather/winddata/rap/power_curves.py

-                if w == 0:
+            xs = state_curves.index
+            ys = np.zeros_like(xs)
+            for i, x in enumerate(xs):


thanks, this actually helps a lot

w for wind speed totally made sense in my head at the time...

danielolsen requested review from dmuldrew, rouille and merrielle December 13, 2019 22:25

danielolsen assigned dmuldrew, danielolsen, rouille and merrielle Dec 13, 2019

danielolsen and others added 11 commits December 16, 2019 10:27

data: more wind power curves

c4be10f

feat: create state averaged wind power curves

48e9087

feat: query power from turbine or state, with default

83c42e9

refactor: more modular wind power curve analyses

470f6be

fix: bugfix for state wind power curves, adding tests

170a4a4

refactor: Move power curves in specific module

b1f5c4d

chore: Rerun notebook using new power estimation

6676438

feat: wind power curve spatio-temporal smoothing

2dd7bb8

chore: cleanup to final chosen values

cb08b53

doc: updates to wind power curve readme

d1a6563

docs: Improve README

ecb5402

rouille force-pushed the more_wind_power_curves branch from ad7a9df to ecb5402 Compare December 16, 2019 18:34

merrielle reviewed Dec 18, 2019

View reviewed changes

refactor: based on PR review, pass dataframes to get_power()

0651ccc

merrielle approved these changes Dec 19, 2019

View reviewed changes

danielolsen merged commit f954c4a into develop Dec 20, 2019

danielolsen deleted the more_wind_power_curves branch March 14, 2020 00:09

Lab-ITTeam unassigned merrielle, dmuldrew and danielolsen Jul 15, 2020

ahurli mentioned this pull request Mar 16, 2021

Develop into Master #155

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: more wind power curves #53

feat: more wind power curves #53

danielolsen commented Dec 13, 2019

merrielle left a comment •

edited

Loading

merrielle Dec 17, 2019

danielolsen Dec 18, 2019

merrielle Dec 17, 2019

merrielle Dec 17, 2019

danielolsen Dec 18, 2019

merrielle Dec 17, 2019

merrielle Dec 17, 2019

danielolsen Dec 18, 2019

merrielle Dec 17, 2019

danielolsen Dec 18, 2019

merrielle Dec 17, 2019

merrielle Dec 17, 2019

danielolsen Dec 18, 2019

merrielle Dec 17, 2019

danielolsen Dec 19, 2019

merrielle Dec 17, 2019

danielolsen commented Dec 19, 2019

rouille commented Dec 19, 2019

merrielle left a comment

merrielle Dec 19, 2019

danielolsen Dec 20, 2019

		Once the U and V components of the wind are converted to a non-directional
		wind speed magnitude, this speed is converted to power using wind turbine

		if not path.isdir(data_dir):
		raise ValueError('data_dir is not a valid directory')

feat: more wind power curves #53

feat: more wind power curves #53

Conversation

danielolsen commented Dec 13, 2019

merrielle left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

danielolsen commented Dec 19, 2019

rouille commented Dec 19, 2019

merrielle left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

merrielle left a comment •

edited

Loading