[Question] Different Action Spaces?

[kad99kev]

Question ❓

Are there any environments that do not have an action space of shape 2? In the documentation for previous versions here, it shows that there are multiple environments with a Box(4), Box(5) etc., but when I go into the configuration JSON files on GitHub, they have only two actions defined. Could you please assist with this?

Thank you!

[kad99kev]

Related to the same question, when I use the Eplus-datacenter-hot-continuous-stochastic-v1 enviornment, I get 100% comfort violations all the time. On further inspection, it seems like the actions are only being applied to the west zone and the east zone is ignored. When I save the zone indoor temperature from the observations, only the west zone is within the defined comfort range. The east zone constantly get values above the comfort zone (i.e. approx 28 degree Celsius). Is there a way to redefine the action space to ensure that both zones are controlled?

[AlejandroCN7]

Hello @kad99kev!

We reduced the action spaces of the default Sinergym environments to make them simpler. In the environment configuration files, you can define those building components you want to control and the new action space (documentation here). Or you can directly use the constructor of the environment class.

In the case of the datacenter, as the building is designed, the temperature setpoints are applied to both zones simultaneously. However, as both zones are not the same, they do not have the same results. If you want to control each zone separately, you need to have explicit knowledge about the building model and change it yourself. Another option would be to modify the parameters of the reward function so that it only takes into account the west zone, but it would optimize only based on that zone of course.

I hope this is helpful!
Regards

[kad99kev]

Hi @AlejandroCN7.

Thank you so much for your reply. I completely understand your answer.

However, there might be a bug in the latest version of the data center. Irrespective of the weather, configuration, or heating/cooling setpoint values, the east zone indoor temperature does not change at all. When I track the indoor temperatures, only the west zone seems to be influenced by the setpoint temperatures. Could you please confirm this on your end? I have noticed this behaviour on versions 2.5.2, 3.1.0 and 3.20.

[AlejandroCN7]

Then, I will check the data you tell me and see if it is related to any of the Sinergym versions. I leave this issue open and as soon as I have more information I will comment here.

Thank you very much for the information and help @kad99kev!

[kad99kev]

Thank you @AlejandroCN7! Do let me know if you observe the same thing. I will be happy to share my observations once you get back to me!

[AlejandroCN7]

Hi @kad99kev!

I've researched this topic and here's what I found. I tried downloading the sample Datacenter file provided by the EnergyPlus v23.1.0 installation. The only significant change between this file and the one we use in Sinergym is as follows:

However, this change actually worsens the internal temperature of the east zone:

So, if that part was well-controlled before, I assume it's related to one of the EnergyPlus updates. The only changes made to the building in the updates were to upgrade its version and convert it to epJSON using EnergyPlus's automatic tools.

For testing, I think it's best to focus on optimizing the west zone until we have more information. For now, I will update the default reward functions for this building to focus only on this zone.

If you find anything that could help or improve this, please feel free to make a PR or comment on an issue.

Thank you very much for everything!

[AlejandroCN7]

I've attempted to control each zone independently, but unfortunately, it seems to have no impact on the east zone. Should I make any progress, I will reopen this issue.

Thank you very much for everything and I apologize for any inconvenience.

[kad99kev]

Hi @AlejandroCN7,

Thank you for the updates! I apologise for the naive questions, but the east zone issue is occurring irrespective of the datacenter file, meaning it is a EnergyPlus v23.1.0 issue? Would updating EnergyPlus to v23.2.0 fix it? Is there a way to flag this problem to EnergyPlus?

Thank you so much for working on this!

[AlejandroCN7]

Hi @kad99kev,

Maybe the latest EnergyPlus update will fix the problem, I can't tell you at this time. As the simulation engine is pending update anyway, I will retest after the update to see if there has been any improvement.

Thank you very much and sorry for the inconvenience.
Regards.

[kad99kev]

Hello @AlejandroCN7! I was just wondering if I were to update the EnergyPlus version from 23.1.0 to 23.2.0 to see if there is still a Datacenter issue, would that affect the library's workings? Would there be any side effects that would not be very obvious?

I haven't tried it yet, so I thought I would ask you in case you have already. Thank you!