Entsoe support (#21)

README.md

@@ -1,8 +1,9 @@
 # nordpool_diff custom component for Home Assistant
 
-Requires https://github.com/custom-components/nordpool
+Requires https://github.com/JaccoR/hass-entso-e and/or https://github.com/custom-components/nordpool
 
-[Nord Pool](https://www.nordpoolgroup.com/) gives you spot prices[^1], but making good use of those prices is not easy.
+[ENTSO-E](https://transparency.entsoe.eu/) and [Nord Pool](https://www.nordpoolgroup.com/) provide spot prices,
+but making good use of those prices is not easy.
 This component provides various algorithms whose output can be used for deciding when to turn water heater or
 car charger on/off, or for adjusting target temperature of a heater so that it will heat more just before prices
 will go up (to allow heating less when prices are high), and heat less just before prices will go down.
@@ -16,9 +17,18 @@ suitable for controlling things that require N contiguous hours to work, such as
 no guarantees about how many hours per day the output will stay above some threshold, even if typical price variations
 may make the output typically behave this or that way most of the time.
 
+## ENTSO-E vs. Nord Pool
+
+This component was initially (in 2021) created to support https://github.com/custom-components/nordpool, hence the name.
+But after that (in 2022) https://github.com/JaccoR/hass-entso-e became available. Besides being 100 % legal to use[^1],
+ENTSO-E also covers wider range of markets than Nord Pool.
+
+Since v0.2.0 / https://github.com/jpulakka/nordpool_diff/issues/21 hass-entso-e is preferred and default, but nordpool
+still works, and can also be used together with hass-entso-e as an automatic fallback mechanism when ENTSO-E API is down.
+
 ## Installation
 
-Install and configure https://github.com/custom-components/nordpool first.
+Install and configure https://github.com/JaccoR/hass-entso-e/ and/or https://github.com/custom-components/nordpool first.
 
 ### Option 1: HACS
 1. Go to HACS -> Integrations
@@ -39,17 +49,28 @@ Install and configure https://github.com/custom-components/nordpool first.
     ```yaml
     sensor:
       - platform: nordpool_diff
+    ```
+
+    The default setup assumes that hass-entso-e provides `sensor.current_price` entity,
+    which it does, if you left optional "Name" empty when configuring hass-entso-e.
+
+    Explicit `entsoe_entity` and/or `nordpool_entity` IDs can also be specified:
+
+    ```yaml
+    sensor:
+      - platform: nordpool_diff
+        entsoe_entity: sensor.current_price
         nordpool_entity: sensor.nordpool_kwh_fi_eur_3_095_024
     ```
 
-   Modify the `nordpool_entity` value according to your exact nordpool entity ID.
+   Modify the `entsoe_entity` and/or `nordpool_entity` values according to your exact entity IDs.
 
 2. Restart HA again to load the configuration. Now you should see `nordpool_diff_triangle_10` sensor, where
    the `triangle_10` part corresponds to default values of optional parameters, explained below.
 
 ## Optional parameters
 
-Optional parameters to configure include `filter_length`, `filter_type`, `unit` and `normalize`, defaults are `10`, `triangle`,
+Other parameters to configure include `filter_length`, `filter_type`, `unit` and `normalize`, defaults are `10`, `triangle`,
 `EUR/kWh/h` and `no`, respectively:
 
  ```yaml
@@ -83,17 +104,17 @@ current hour and the next multipliers correspond to upcoming hours.
 Smallest possible `filter_length: 2` creates FIR `[-1, 1]`. That is, price for the current hour is subtracted from the
 price of the next hour. In this case `filter_type: rectangle` and `filter_type: triangle` are identical.
 
-`filter_length: 3`, `filter_type: rectangle` creates FIR `[-1, 1/2, 1/2]`
-
-`filter_length: 3`, `filter_type: triangle` creates FIR `[-1, 2/3, 1/3]`
-
-`filter_length: 4`, `filter_type: rectangle` creates FIR `[-1, 1/3, 1/3, 1/3]`
-
-`filter_length: 4`, `filter_type: triangle` creates FIR `[-1, 3/6, 2/6, 1/6]`
+`filter_length: 3`,
+* `filter_type: rectangle` creates FIR `[-1, 1/2, 1/2]`
+* `filter_type: triangle` creates FIR `[-1, 2/3, 1/3]`
 
-`filter_length: 5`, `filter_type: rectangle` creates FIR `[-1, 1/4, 1/4, 1/4, 1/4]`
+`filter_length: 4`,
+* `filter_type: rectangle` creates FIR `[-1, 1/3, 1/3, 1/3]`
+* `filter_type: triangle` creates FIR `[-1, 3/6, 2/6, 1/6]`
 
-`filter_length: 5`, `filter_type: triangle` creates FIR `[-1, 4/10, 3/10, 2/10, 1/10]`
+`filter_length: 5`,
+* `filter_type: rectangle` creates FIR `[-1, 1/4, 1/4, 1/4, 1/4]`
+* `filter_type: triangle` creates FIR `[-1, 4/10, 3/10, 2/10, 1/10]`
 
 And so on. With rectangle, the right side of the filter is "flat". With triangle, the right side is weighting soon
 upcoming hours more than the farther away "tail" hours. First entry is always -1 and the filter is normalized so that
@@ -137,12 +158,21 @@ threshold but the next hour value is below the threshold, and we would like to a
 shouldn't turn the thing on at xx:59 if we would turn it off only after 1 minute. This can be avoided by taking the next
 hour value into account.
 
+## Debug logging
+
+Add the following to `configuration.yaml`:
+
+ ```yaml
+logger:
+  default: info
+  logs:
+    custom_components.nordpool_diff.sensor: debug
+```
+
 [^1]: [Nord Pool API documentation](https://www.nordpoolgroup.com/en/trading/api/) states
 _If you are a Nord Pool customer, using our trading APIs is for free. All others must become a customer to use our APIs._
-Regardless, the API is technically public and appears to work without any tokens.
-[ENTSO-E](https://transparency.entsoe.eu/) would be the correct place to fetch the prices from, and now (10/2022)
-there's also a HASS integration for that: https://github.com/JaccoR/hass-entso-e . That will be integrated in 
-https://github.com/jpulakka/nordpool_diff/issues/21
+Which apparently means that almost nobody should be using it, even though the API is technically public and appears to work without any tokens.
+It's more correct to use [ENTSO-E](https://transparency.entsoe.eu/) which is intended to be used by anyone.
 
 [^2]: Fancy way of saying that the price for the current hour is subtracted from the average price for the next few
 hours.

custom_components/nordpool_diff/manifest.json

@@ -4,8 +4,8 @@
   "documentation": "https://github.com/jpulakka/nordpool_diff",
   "issue_tracker": "https://github.com/jpulakka/nordpool_diff/issues",
   "requirements": [],
-  "dependencies": ["nordpool"],
+  "dependencies": [],
   "codeowners": ["@jpulakka"],
   "iot_class": "calculated",
-  "version": "0.1.5"
+  "version": "0.2.0"
 }

custom_components/nordpool_diff/sensor.py

@@ -1,5 +1,6 @@
 from __future__ import annotations
 
+import logging
 import voluptuous as vol
 import homeassistant.helpers.config_validation as cv
 from homeassistant.components.sensor import PLATFORM_SCHEMA, SensorEntity
@@ -8,8 +9,12 @@
 from homeassistant.helpers.entity_platform import AddEntitiesCallback
 from homeassistant.helpers.typing import ConfigType, DiscoveryInfoType
 from homeassistant.util import dt
+from datetime import datetime, timedelta
+
+_LOGGER = logging.getLogger(__name__)
 
 NORDPOOL_ENTITY = "nordpool_entity"
+ENTSOE_ENTITY = "entsoe_entity"
 FILTER_LENGTH = "filter_length"
 FILTER_TYPE = "filter_type"
 RECTANGLE = "rectangle"
@@ -25,7 +30,8 @@
 # https://developers.home-assistant.io/docs/development_validation/
 # https://github.com/home-assistant/core/blob/dev/homeassistant/helpers/config_validation.py
 PLATFORM_SCHEMA = PLATFORM_SCHEMA.extend({
-    vol.Required(NORDPOOL_ENTITY): cv.entity_id,
+    vol.Optional(NORDPOOL_ENTITY, default=""): cv.string,  # Is there a way to require EITHER nordpool OR entsoe being valid cv.entity_id?
+    vol.Optional(ENTSOE_ENTITY, default="sensor.current_price"): cv.string,  # hass-entso-e's default entity id
     vol.Optional(FILTER_LENGTH, default=10): vol.All(vol.Coerce(int), vol.Range(min=2, max=20)),
     vol.Optional(FILTER_TYPE, default=TRIANGLE): vol.In([RECTANGLE, TRIANGLE, INTERVAL, RANK]),
     vol.Optional(NORMALIZE, default=NO): vol.In([NO, MAX, MAX_MIN]),
@@ -40,12 +46,13 @@ def setup_platform(
         discovery_info: DiscoveryInfoType | None = None
 ) -> None:
     nordpool_entity_id = config[NORDPOOL_ENTITY]
+    entsoe_entity_id = config[ENTSOE_ENTITY]
     filter_length = config[FILTER_LENGTH]
     filter_type = config[FILTER_TYPE]
     normalize = config[NORMALIZE]
     unit = config[UNIT]
 
-    add_entities([NordpoolDiffSensor(nordpool_entity_id, filter_length, filter_type, normalize, unit)])
+    add_entities([NordpoolDiffSensor(nordpool_entity_id, entsoe_entity_id, filter_length, filter_type, normalize, unit)])
 
 def _with_interval(prices):
     p_min = min(prices)
@@ -60,11 +67,34 @@ def _with_rank(prices):
 def _with_filter(filter, normalize):
     return lambda prices : sum([a * b for a, b in zip(prices, filter)]) * normalize(prices)
 
+def _get_next_n_hours_from_nordpool(n, np):
+    prices = np.attributes["today"]
+    hour = dt.now().hour
+    # Get tomorrow if needed:
+    if len(prices) < hour + n and np.attributes["tomorrow_valid"]:
+        prices = prices + np.attributes["tomorrow"]
+    # Nordpool sometimes returns null prices, https://github.com/custom-components/nordpool/issues/125
+    # The nulls are typically at (tail of) "tomorrow", so simply removing them is reasonable:
+    prices = [x for x in prices if x is not None]
+    return prices[hour: hour + n]
+
+def _get_next_n_hours_from_entsoe(n, e):
+    prices = []
+    if p := e.attributes["prices"]:
+        hour_before_now = dt.utcnow() - timedelta(hours=1)
+        for item in p:
+            if prices or hour_before_now <= datetime.fromisoformat(item["time"]):
+                prices.append(item["price"])
+                if len(prices) == n:
+                    break
+    return prices
+
 class NordpoolDiffSensor(SensorEntity):
     _attr_icon = "mdi:flash"
 
-    def __init__(self, nordpool_entity_id, filter_length, filter_type, normalize, unit):
+    def __init__(self, nordpool_entity_id, entsoe_entity_id, filter_length, filter_type, normalize, unit):
         self._nordpool_entity_id = nordpool_entity_id
+        self._entsoe_entity_id = entsoe_entity_id
         self._filter_length = filter_length
         if normalize == MAX:
             normalize = lambda prices : 1 / (max(prices) if max(prices) > 0 else 1)
@@ -109,17 +139,24 @@ def update(self):
         prices = self._get_next_n_hours(self._filter_length + 1)  # +1 to calculate next hour
         self._state = round(self._compute(prices[:-1]), 3)
         self._next_hour = round(self._compute(prices[1:]), 3)
+        # TODO here could add caching, this really needs to be recalculated only each xx:00 if successful.
 
     def _get_next_n_hours(self, n):
-        np = self.hass.states.get(self._nordpool_entity_id)
-        prices = np.attributes["today"]
-        hour = dt.now().hour
-        # Get tomorrow if needed:
-        if len(prices) < hour + n and np.attributes["tomorrow_valid"]:
-            prices = prices + np.attributes["tomorrow"]
-        # Nordpool sometimes returns null prices, https://github.com/custom-components/nordpool/issues/125
-        # The nulls are typically at (tail of) "tomorrow", so simply removing them is reasonable:
-        prices = [x for x in prices if x is not None]
-        # Pad if needed, using last element:
-        prices = prices + (hour + n - len(prices)) * [prices[-1]]
-        return prices[hour: hour + n]
+        prices = []
+        # Prefer entsoe:
+        if e := self.hass.states.get(self._entsoe_entity_id):
+            prices = _get_next_n_hours_from_entsoe(n, e)
+            _LOGGER.debug(f"{n} prices from entsoe {prices}")
+        # Fall back to nordpool:
+        if (len(prices) < n) and (np := self.hass.states.get(self._nordpool_entity_id)):
+            np_prices = _get_next_n_hours_from_nordpool(n, np)
+            _LOGGER.debug(f"{n} prices from nordpool {np_prices}")
+            if len(np_prices) > len(prices):
+                prices = np_prices
+        # Fail gracefully if nothing works:
+        if not prices:
+            return n * [0]
+        # Pad if needed, using last element.
+        prices = prices + (n - len(prices)) * [prices[-1]]
+        _LOGGER.debug(f"{n} prices after padding {prices}")
+        return prices