Developement discussion

[MadOne]

Hi,

i have a few questions/suggestions for the futher developement.

• I suggest to "unfork" this repo. It has nearly nothing in common with the old one. Also remove the Info in readme https://github.com/orgs/community/discussions/52663
• We should try to catch the users of my old integration. I am not sure about the best approach. Is there a possibility of forwarding to this repo? Or just do a release with the information (I think most users wont read). Or maybe intentionally break the integration to wake up the users to switch.
• I activated the necessary checks for HACS: Error: R] [MANIFEST] Domain does not match dir name . So we have to either rename the domain or change the repo name. Both will break instrallations :(. This is also relevant for the icons, as they are tied to the domain.

[OStrama]

Yes I also thought about this. We should have a look at the "install base": Move to the naming that the majority of users use. I have no idea how many and how active they are.

The most annoying would be a change of the entity IDs, especially when using them for long time statistics, i.e. energy dashboards. In the newest HA version there was an anouncement that statistics can be attached to entities easier than before. Maybe, I do an upgrade to check.

Yes, since I use the remote environment including reformatting, I also remove warnings etc. when editing a file.

[MadOne]

There are a lot possibilities but none of them are flawless.

1. We could change back the DOMAIN to weishaupt_modbus: This would fix the icon and stop the HACS checks from complaining. But will break the entitys as the unique id is dependent on the domain for now.
   self._attr_unique_id = CONST.DOMAIN + self._attr_name ->self._attr_unique_id = CONST.DOMAIN_OLD + self._attr_name is a hacky fix.
   Users would have to setup a new configuration for the heatpump. I think statistics values would be found if we take the workaround above.
   This might also be a good time to introduce the prefix as the prefix would break the unique_id and also the statistics anyway.

2. Change the repository name and the folder name: This will disconnect HACS users from the integration developement. They would have to add a new repo to HACS.

I like none of the above. It would be sad if we loose some users, On the other side we might just ignore the consequences and do it right. But then we sould introduce all breaking changes at once.
For now my prefered solution is variant one with breaking the statistics and introduce the prefix.

[OStrama]

I‘d also prefer 1.
When we implement the prefix before, there‘s a simple migration path:

• standard prefix will be Weishaupt Modbus
• Users who used weishaupt wbb before have the choice:
  a) change prefix to it and keep statistics
  b) use new default and re-assign statistics via database hack (see Simon42 lost entity video)
  c) use new default and start with new statistics

we only have to give clear instructions in the readme and perhaps in the config dialog..

[MadOne]

I changed the domain to weishaupt_modbus.
The Prefix comes from const.py for now and is set to weishaupt_wbb.
You have to remove the heatpump and setup a new one, because of the changed domain.
This fixes the logos and HACS / hassfest checks.

There is a possibility in the config flow to react to updates. So we could devide in users update from 0.0.xx to > 0.0.8 and users who start with a fresh install > 0.0.8

[OStrama]

Good Idea! I had a look at other custom integrations (Fronius). I think I understood how to add custom parameters. When ready we can put the prefix in as a parameter.

[MadOne]

I changed modbusclient.disconnect() to sync. This is acording to the pymodbus example. I guess the disconnect is only client sided so no async is needed. This threw an error the whole time, but we didnt see it because if the bare try except. Now nearly all values are up.

There is still a lot of debugging to do. We have to get rid of those bare try except statements.

[OStrama]

Thanks, I took it over and enabled the heat power again. I'll try to do a pre-release that I can test it on my production system for a day or two.

My problem with the try-except sectios is, that I did not find a documentation of possible exceptions. (I know it from C++ docu that a function description usually contains infos like "throws the following exceptions:")

[MadOne]

I think you can select a branch in hacs. So no need for a release.

What exceptions are you missing? The modbus exception is displayed when i hover over the functions.

[OStrama]

Can you have a look at the readme? I tried to put in some migration info...

[MadOne]

Im in work right now. I hope i have time to think about the Migration when im at home. Too difficult on the phone

[OStrama]

Some insights from my production test. the number of faults decreased compared with my tests in the devenv. Maybe, because now we do not have two instances talking with the heatpump.

Nevertheless,

• I have stable entity values with no gaps anymore
• I can control the parameters reliably

I see the following in the log:

the 1st one happens constantly:

Connection failures from time to time:

But these only at startup:

[OStrama]

Digging a bit deeper it looks more clear:

At the beginning every request fails. You see it also in slow updating of entities after 1st start-up:

Zeile  13: 2024-10-25 11:25:44.540 WARNING (MainThread) [py.warnings] /home/homeassistant/.homeassistant/custom_components/weishaupt_modbus/modbusobject.py:52: UserWarning: Connection to heatpump succeeded
Zeile  16: 2024-10-25 11:25:44.541 WARNING (MainThread) [py.warnings] /home/homeassistant/.homeassistant/custom_components/weishaupt_modbus/entities.py:201: UserWarning: Item:Aussentemperatur failed

.

.
.

Zeile 268: 2024-10-25 11:25:44.545 WARNING (MainThread) [py.warnings] /home/homeassistant/.homeassistant/custom_components/weishaupt_modbus/entities.py:201: UserWarning: Item:Elektr. Energie Monat failed
Zeile 271: 2024-10-25 11:25:44.545 WARNING (MainThread) [py.warnings] /home/homeassistant/.homeassistant/custom_components/weishaupt_modbus/entities.py:201: UserWarning: Item:Elektr. Energie Jahr failed

In normal operation, the two others only happen mostly at exactly the same time. Maybe because of asynch HA tries 2 requests in parallel?

Zeile 274: 2024-10-25 11:26:14.690 WARNING (MainThread) [py.warnings] /home/homeassistant/.homeassistant/custom_components/weishaupt_modbus/modbusobject.py:52: UserWarning: Connection to heatpump succeeded
Zeile 278: 2024-10-25 11:26:14.712 WARNING (MainThread) [py.warnings] /home/homeassistant/.homeassistant/custom_components/weishaupt_modbus/entities.py:227: UserWarning: connection to the heatpump failed
Zeile 356: 2024-10-25 11:48:14.691 WARNING (MainThread) [py.warnings] /home/homeassistant/.homeassistant/custom_components/weishaupt_modbus/modbusobject.py:52: UserWarning: Connection to heatpump succeeded
Zeile 359: 2024-10-25 11:48:14.691 WARNING (MainThread) [py.warnings] /home/homeassistant/.homeassistant/custom_components/weishaupt_modbus/entities.py:227: UserWarning: connection to the heatpump failed
Zeile 601: 2024-10-25 13:05:14.690 WARNING (MainThread) [py.warnings] /home/homeassistant/.homeassistant/custom_components/weishaupt_modbus/modbusobject.py:52: UserWarning: Connection to heatpump succeeded
Zeile 604: 2024-10-25 13:05:14.690 WARNING (MainThread) [py.warnings] /home/homeassistant/.homeassistant/custom_components/weishaupt_modbus/entities.py:227: UserWarning: connection to the heatpump failed
Zeile 746: 2024-10-25 13:47:44.690 WARNING (MainThread) [py.warnings] /home/homeassistant/.homeassistant/custom_components/weishaupt_modbus/modbusobject.py:52: UserWarning: Connection to heatpump succeeded
Zeile 749: 2024-10-25 13:47:44.690 WARNING (MainThread) [py.warnings] /home/homeassistant/.homeassistant/custom_components/weishaupt_modbus/entities.py:227: UserWarning: connection to the heatpump failed

[MadOne]

I managed to reduce the errors and also get a "IllegalAddress" response. This could be used to filter the modbus addresses not supported by the heatpump.

[OStrama]

I installed it some minutes ago. I only see this

2024-10-25 21:09:14.395 WARNING (MainThread) [py.warnings] /home/homeassistant/.homeassistant/custom_components/weishaupt_modbus/modbusobject.py:52: UserWarning: Connection to heatpump succeeded
warnings.warn("Connection to heatpump succeeded")
2024-10-25 21:09:14.939 ERROR (MainThread) [pymodbus.logging] Cancel send, because not connected!

I have some not working entities, too. But I do not see messages from them..

[MadOne]

Run it in dev Container and you will see.
I also read about an integration that can display debug info.

[OStrama]

Thanks! Will do tomorrow..

[OStrama]

Removed another await on close which crashed entity changes ;-) Made an update

[MadOne]

I pushed out a new release because i made some heavy progress.

[OStrama]

Great! I used the new flag for checking is an item should be added to the list. It works for my WEZ2.
Have to leave now. When I have some time tomorrow, I'll add all entities to the list that are in the Excel Sheet..

[OStrama]

When looking at the Excel sheet with the addresses: A temperature value of -32k means not available. Should we treat them as _is_invalid, too?

[MadOne]

I think so. It unlikely hat the appear mid session.

[OStrama]

Another point: It seems that the connections sometimes left open and that there‘s an issue with too many open files after a while. Had to restart my Pi after HA was not responding anymore. That never happens before 0.0.8..

[MadOne]

You seem to have fixed this perfectly fine. I really like this.

[MadOne]

HA complains about file io (kennfeld.py) is not async

[OStrama]

For me it looks like this:

[OStrama]

It takes the names of the config entries but not the strings from the files..

[OStrama]

It Works!

• according to https://community.home-assistant.io/t/hacs-custom-integration-config-flow-strings-are-not-loading/584597/3 the translations files are not auto-generated for custom integrations. So we have to generate all language files by ourselves. I simply copied the string.json to translation and it worked..

[OStrama]

I'll check in this now in en and de. Please feel free to adapt ;-)

[MadOne]

Good job. This is a big step.

[MadOne]

What about adding our integration to HACS.
Acording to:
https://www.hacs.xyz/docs/publish/include/
we have all requirements.
Whis would enable people to find our integration much easier.

[OStrama]

Yes, that would be a good idea! (I‘m out this weekend, sorry for late replies..)

[MadOne]

When playing around with the translation i did: python -m script.hassfest

It reported:
Integration weishaupt_modbus:

• [ERROR] [MANIFEST] Invalid manifest: extra keys not allowed @ data['version']. Got '0.0.8'
• [ERROR] [REQUIREMENTS] Requirement pymodbus need to be pinned "==".
• [ERROR] [REQUIREMENTS] Requirement aiofiles need to be pinned "==".

Not sure what the complain about version is, maybe because its 0.x.x?
Should we pin the requirements?

[OStrama]

Yes, good idea!

[OStrama]

Is it possible to state diffewrent version numbers? Or is a higher number also allowed?
I recently upgraded to HA 2024.10 and now I have pymodbus 3.6.9 installed.
(I'd like top avoid that we have to update the manifest with every new HA version..)

[MadOne]

Maybe we should just go back to unpinned versions as we are not a native ha integration.

[OStrama]

Let's try with your settings in Manifest first and look what happens. Maybe HA is intelligent enough to handle it..

[OStrama]

Made a try: When pinning with "==" I see an error in my HA version 2024.10. Whern I use ">=" it works. Will it pass the test?

[OStrama]

Yes, makes sense. I will be available Mon Tue to adaptations regardin hacs integration.
But doing 1.0.0 before absolutely makes sense.

[OStrama]

First time below zero. I think they use a special format for that (Einerkomplement?)

[MadOne]

Can't confirm at the moment. We had +2°C

[OStrama]

OK, here it was -0.5.

If value > 32768 we have to substract 65536 then it fits with WEM portal.. And 32768 seems to be zero..
Will check it in later

[OStrama]

I check in the following changes:

• Temperature calculation as described above
• Changing Addr. 3311 to "Vorlauftemperatur präzise"
• Updating config schema to version 4 and updating async_migrate_entry (When adding/changing config flow parameters we always should take care that default values are set during update..)

Would it make sense to build a 1.0.0pre for testing?

[OStrama]

I'm not sure if 33111 is available on every model. The info about 33111 is from here: #20

[OStrama]

Point 1 and 3 sounds good. What is Vorlauftemperatur präzise?

Jeah we should to some pre while still developing. Rc should be end of new features and just bugfixing.

OK, should we freeze features and shift the last 3 enhancements to 1.1.0?

[MadOne]

Yes i think so. We should make a new developement branch for new features and and the main one for bugfixes only.

[OStrama]

OK, makes sense..

[OStrama]

Cretated new branch Version-1.0 and made 1.0.0rc1

[OStrama]

I made some bugfixing in branch Version-1.0.
Maybe, using the HA const for minutes in WW Push affects the translation you made?
I did not add translation up to now nor any changes in registers. Only missing spaces and crash if "Luftansaugtemperatur" not available. You were right, that it chrashed because of it.

[MadOne]

The translation system is quite a heavy change. When its done we should merge all branches.

[OStrama]

Ahh, yes, sorry, was too fast with the debugging changes to main. Should I reverse?

[MadOne]

Nope its ok.

[OStrama]

@MadOne Made a pull request in your old repository by mistake. Please ignore and/or decline it..

[MadOne]

@OStrama : Somehow my dev container broke and i rebuild it. Since then no branch of our integration works. Do you have similar problems?

[OStrama]

No, I currently check translate branch. It works in general..

[MadOne]

  File "/workspaces/HA_core/config/custom_components/weishaupt_modbus/modbusobject.py", line 156, in value
    if len(mbr.registers) > 0:
           ^^^^^^^^^^^^^
AttributeError: 'ExceptionResponse' object has no attribute 'registers'

Even in the main branch. Maybe this is due to an homassistent update or the increase of pymobus?

[OStrama]

I switched from German to English and vice versa. Works well and entities remain. But you have to restart HA to change language. But this is OK I guess..

[MadOne]

Are sensor values and select values translated?

[OStrama]

Yes

[MadOne]

Weird it did not work for me. Well now nothing works...
Any chance you could join discord?
https://discord.gg/ZYUqBF8n
This would make conversation much easier....

[MadOne]

What pymobus version do you have? I have 3.7.4 now

[OStrama]

pymodbus in /home/vscode/.local/ha-venv/lib/python3.12/site-packages (3.6.8)

WHat kind of update did you do?

[OStrama]

and on my production system: pymodbus in ./lib/python3.12/site-packages (3.6.9)

[OStrama]

In the devcontainer:
Successfully installed pymodbus-3.7.4

and a lot of exceptions:

File "/workspaces/core/config/custom_components/weishaupt_modbus/modbusobject.py", line 156, in value
if len(mbr.registers) > 0:
^^^^^^^^^^^^^
AttributeError: 'ExceptionResponse' object has no attribute 'registers'