Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port linearize_gencost and pmin modifications from REISE.jl #701

Merged
merged 10 commits into from
Nov 10, 2022

Conversation

jenhagg
Copy link
Collaborator

@jenhagg jenhagg commented Nov 3, 2022

Purpose

To facilitate using a grid.pkl and remove the need for mapping between a case.mat/grid.mat/etc, we move grid modifications previously done in REISE.jl to the scenario preparation process.

This PR should be merged at the same time as Breakthrough-Energy/REISE.jl#187, after which we should update the existing grid.mat files. I believe we also need to update the ramp30 values in plant.csv as discussed in this issue #680 and release a new dataset on zenodo.

What the code is doing

Scale plant Pmin values and linearize cost curves the same way we were doing in REISE.jl - after we apply the change table, and just before we upload the grid.pkl to the server (or, container, or wherever). I also ported the linearize_gencost tests from julia since those will be unused/deleted.

Testing

Ran a scenario in a container and reloaded it after it finished for some cursory spot checking.

Time estimate

20 min

@rouille
Copy link
Collaborator

rouille commented Nov 3, 2022

Is powersimdata/design/generation/ a good location for the cost_curves module? Should we move it in powersimdata/scenario/ or powersimdata/input/? What do you think @BainanXia?

@BainanXia
Copy link
Collaborator

Is powersimdata/design/generation/ a good location for the cost_curves module? Should we move it in powersimdata/scenario/ or powersimdata/input/? What do you think @BainanXia?

I agree. The original idea of design module is a place for functions that are optionally used in the scenario workflow. However, linearization is essential given the engine can't deal with quadratic cost curves (at least for now) and similarly for the ramp module in the future. Thus we can create a new module in powersimdata/scenario/. Not sure what is a good name for it, maybe configure?

Comment on lines 210 to 224
def _adjust_pmin(self):
mi = self.grid.model_immutables
plant = self.grid.plant
pmin_factor = mi.plants["pmin_as_share_of_pmax"]

def _scale(x):
factor = pmin_factor.get(x.type)
return x.Pmin if factor is None else factor * x.Pmax

plant.Pmin = plant.apply(_scale, axis=1)

# set Pmin to 0 for generators that are off or profile based
profile_resource = list(mi.plants["profile_resources"])
plant.loc[plant.type.isin(profile_resource), "Pmin"] = 0
plant.loc[plant.status == 0, "Pmin"] = 0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe out of the scope of this PR, two high level questions:

  1. hydro is one of profile_resources, the Pmin of which will be set 0 at the end, however the pmin_factor of hydro is grid dependent .
  2. We will need to build user interface to pass user specified pmin_as_share_of_pmax (either plant based or plant type based or plant type * zone based) in the future, right?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. resources with profile have all Pmin set to 0 in the plant data frame but it is not used in the optimization. We did that for backward compatibility since this is what was hard coded in REISE.jl.
  2. If we want a finer resolution on pmin_as_share_of_pmax, we can have either an extra column in the plant data frame or a dedicated data structure than can be passed to the engine. Either way we will need a user interface to specify the values

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. True. Just pointing out such entries are modified twice in the current process.
  2. Agreed.

Comment on lines 217 to 222
def test_linearize_gencost_pmin_equal_pmax():
plant = mock_grid_gc.plant.copy()
plant.Pmin = plant.Pmax
grid = MockGrid({"plant": plant.reset_index().to_dict(), "gencost_before": mock_gc})
actual = linearize_gencost(grid, num_segments=3)
assert_frame_equal(expected_all_equal, actual, check_dtype=False)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! We thought about adding such tests a while ago as in the very first version of implementation, dividing by zero wasn't avoided.

@lanesmith
Copy link
Collaborator

I agree. The original idea of design module is a place for functions that are optionally used in the scenario workflow. However, linearization is essential given the engine can't deal with quadratic cost curves (at least for now) and similarly for the ramp module in the future. Thus we can create a new module in powersimdata/scenario/. Not sure what is a good name for it, maybe configure?

Is the ramp module essential? Especially once (if?) plant.csv gets updated, I'm not sure we would need the ramp-rate correction anymore.

@BainanXia
Copy link
Collaborator

BainanXia commented Nov 3, 2022

I agree. The original idea of design module is a place for functions that are optionally used in the scenario workflow. However, linearization is essential given the engine can't deal with quadratic cost curves (at least for now) and similarly for the ramp module in the future. Thus we can create a new module in powersimdata/scenario/. Not sure what is a good name for it, maybe configure?

Is the ramp module essential? Especially once (if?) plant.csv gets updated, I'm not sure we would need the ramp-rate correction anymore.

We will need that when scaling existing plants as well as adding new plants without specifying ramp rates until we have a better solution.

@rouille
Copy link
Collaborator

rouille commented Nov 3, 2022

Is powersimdata/design/generation/ a good location for the cost_curves module? Should we move it in powersimdata/scenario/ or powersimdata/input/? What do you think @BainanXia?

I agree. The original idea of design module is a place for functions that are optionally used in the scenario workflow. However, linearization is essential given the engine can't deal with quadratic cost curves (at least for now) and similarly for the ramp module in the future. Thus we can create a new module in powersimdata/scenario/. Not sure what is a good name for it, maybe configure?

It can be done in a separate PR. I am fine with having the linearize_gencost function in a new powersimdata.scenario.configure. We could have the ramp rate setting there too. Or we could have a linearize_cost_curve module and a assign_ramp_rate modules in powersimdata.input. Hat is your opinion @jenhagg?

@jenhagg
Copy link
Collaborator Author

jenhagg commented Nov 4, 2022

Is powersimdata/design/generation/ a good location for the cost_curves module? Should we move it in powersimdata/scenario/ or powersimdata/input/? What do you think @BainanXia?

I agree. The original idea of design module is a place for functions that are optionally used in the scenario workflow. However, linearization is essential given the engine can't deal with quadratic cost curves (at least for now) and similarly for the ramp module in the future. Thus we can create a new module in powersimdata/scenario/. Not sure what is a good name for it, maybe configure?

It can be done in a separate PR. I am fine with having the linearize_gencost function in a new powersimdata.scenario.configure. We could have the ramp rate setting there too. Or we could have a linearize_cost_curve module and a assign_ramp_rate modules in powersimdata.input. Hat is your opinion @jenhagg?

Based on the current package structure, I think having those in powersimdata.input makes sense, although the distinction between input and design isn't super clear to me. I'm fine with either option; there is always the possibility of reorganizing in the future.

Copy link
Collaborator

@lanesmith lanesmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

Copy link
Collaborator

@rouille rouille left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for tackling this!

Copy link
Collaborator

@BainanXia BainanXia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice 🚀

@jenhagg jenhagg merged commit 4a46483 into develop Nov 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants