Use base profile directly #217
Conversation
rouille
added
new feature
|
I am testing this with Scenarios 923 and 924 on Heracles and Scenarios 925 and 926 on Zeus. |
| @@ -7,8 +7,6 @@ | |||
| from powersimdata.input.transform_grid import TransformGrid | |||
| from powersimdata.input.transform_profile import TransformProfile | |||
|
|
|||
| scenario_id = "564" | |||
There was a problem hiding this comment.
I still like this line, and def scenario(): return Scenario(scenario_id), just so that all hardcoded values are explicitly declared up top, not within functions. Is there a reason to remove this?
powersimdata/scenario/execute.py
Outdated
| program = "'{for(i=1; i<=NF; i++){if($1==%s) $2=\"%s\"}};1'" % ( | ||
| self._scenario_info["id"], | ||
| status, | ||
| ) | ||
| program = "'{if($1==%s) $2=\"%s\"};1'" % (self._scenario_info["id"], status) |
There was a problem hiding this comment.
I'm still not very familiar with awk, what is the impact of the change here? Can we add a comment that makes this section a bit more clear?
There was a problem hiding this comment.
I will add a comment. What I was doing before i.e looping over the columns (NF is the number of fields) was useless. I guess at that time I was a awk novice. You just want to locate the id through the the if and do something.
| self.grid = grid | ||
| self.ct = ct | ||
|
|
||
| self._tmp_dir = "%s/scenario_%s" % (const.EXECUTE_DIR, scenario_id) | ||
| self.TMP_DIR = "%s/scenario_%s" % (const.EXECUTE_DIR, scenario_info["id"]) |
There was a problem hiding this comment.
Is there any significance to this name change? Is it to match the variable naming convention for the other directories in const?
There was a problem hiding this comment.
Yes. I need to access other info of the scenario, the interconnect and the base profiles version. So I can no longer just pass the id.
There was a problem hiding this comment.
I mean the change from self._tmp_dir to self.TMP_DIR.
There was a problem hiding this comment.
No sorry for the misunderstanding. I decided to follow the synthax for folder location that we have in the powersimdata.utility.const module
|
The changes are indeed scattered throughout the code, but mostly are changes to use the scenario info from the ScenarioList rather than just the ID, which makes sense. I don't see anything that concerns me, and the Eastern scenarios I'm running on Heracles should finish by tomorrow afternoon so we will get some independent verification that this works in our most complicated scenario-running process so far. |
|
I think changing from scenario id to scenario info is reasonable given we will need other information of the scenario we are dealing with as well from now on. I've gone through the code and it looks reasonable to me. I think it is good to go if the ongoing integration tests pass. |
Purpose
Manipulate base profiles and not symbolic links to these profiles. This refactoring does the following:
/home/EGM/v2/data/input/tmpfolder of the scenario.The benefits are:
Note that the existing profiles in your local folder named id_demand.csv, id_hydro.csv, id_solar.csv and id_wind.csv will no longer be used and can be deleted.
What is the code doing?
There a lot of files that has been refactored. The major change happens in the
InputDataclass where we don't download the link but the raw profiles.Where to look
Start with the powersimdata/input/profiles.py file. We remove the symlingk generation In powersimdata/scenario/create.py file. The other modified files accomodate the changes in powersimdata/input/profiles.py.
Time estimate
30 min
Test
I have succressfully created, prepared, ran and extracted scenario 876. You can see that there are no more links in
/home/EGM/v2/data/input/for this scenario.