Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Camera.record service not working with entity_id.name in template, says entity_id is undefined #40241

Closed
thrust15 opened this issue Sep 18, 2020 · 61 comments · Fixed by #128592
Closed

Comments

@thrust15
Copy link
Contributor

When I call the camera.record service with a filename template, it fails with the following error: jinja2.exceptions.UndefinedError: 'entity_id' is undefined.

This was not the case in the previous version of home assistant.

Environment

  • Home Assistant Core release with the issue: 0.115
  • Last working Home Assistant Core release (if known): 0.114
  • Operating environment (OS/Container/Supervised/Core): Supervised
  • Integration causing this issue: Generic Camera
  • Link to integration documentation on our website: https://www.home-assistant.io/integrations/camera/

Problem-relevant configuration.yaml

- alias: 'persoon voor de deur'
  trigger:
    - platform: numeric_state
      entity_id: image_processing.voortuin
      above: 0
  action:
    - service: camera.record
      entity_id: camera.frontyard
      data:
        duration: 60
        lookback: 30
        filename: '/share/motioneye/tensorflow/{{ entity_id.name }}_{{ now().strftime("%Y%m%d-%H%M%S") }}.mp4'
    - service: notify.security_telegram
      data_template:
        title: "Tensorflow"
        message: "Tensorflow"
        data:
          photo:
            file: "/config/www/tensorflow.jpg"
            caption: "Persoon in voortuin gedetecteerd {{states.image_processing.voortuin.attributes.matches.person[0].score}}%"

Traceback/Error logs

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 285, in async_render
    return compiled.render(kwargs).strip()
  File "/usr/local/lib/python3.8/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/usr/local/lib/python3.8/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/usr/local/lib/python3.8/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 1, in top-level template code
  File "/usr/local/lib/python3.8/site-packages/jinja2/sandbox.py", line 407, in getattr
    value = getattr(obj, attribute)
jinja2.exceptions.UndefinedError: 'entity_id' is undefined

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 138, in async_prepare_call_from_config
    service_data.update(template.render_complex(config[conf], variables))
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 81, in render_complex
    return {
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 82, in <dictcomp>
    render_complex(key, variables): render_complex(item, variables)
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 86, in render_complex
    return value.async_render(variables)
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 287, in async_render
    raise TemplateError(err) from err
homeassistant.exceptions.TemplateError: UndefinedError: 'entity_id' is undefined

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 206, in _async_step
    await getattr(
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 385, in _async_call_service_step
    domain, service, service_data = async_prepare_call_from_config(
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 140, in async_prepare_call_from_config
    raise HomeAssistantError(f"Error rendering data template: {ex}") from ex
homeassistant.exceptions.HomeAssistantError: Error rendering data template: UndefinedError: 'entity_id' is undefined

Additional information

@probot-home-assistant
Copy link

camera documentation
camera source
(message by IssueLinks)

@ruimarinho
Copy link

Workaround for now is to also set entity_id on the parent definition. Example:

  - trigger:
      platform: state
      entity_id: binary_sensor.main_door
      to: 'on'
    action:
      - service: camera.record
        entity_id: camera.hall
        data:
          entity_id: camera.hall
          filename: '/tmp/{{ entity_id.name }}_{{ now().strftime("%Y%m%d-%H%M%S") }}.mp4'
          duration: 8

@parautenbach
Copy link

parautenbach commented Oct 18, 2020

I can confirm this issue from another angle. I've posted it on the forum for feedback and then discovered this bug report.

This fails for me running core 0.116.2 on Python 3.9.0 (the last known version this worked on for me was also 0.114 – I skipped 0.115).

test_snapshot:
  sequence:
    - service: camera.snapshot
      data:
        entity_id: camera.security_camera
        filename: '/tmp/snapshot_{{ entity_id.entity_id }}.jpg'

Also, the example in the docs won't work.

(I'll try your suggestion to see if it works for me in this case.)

@justin-arens
Copy link

@parautenbach

(I'll try your suggestion to see if it works for me in this case.)

I am running into this same issue currently and my test scenario is nearly identical to yours. Have you been able to utilize the suggested workaround to find success? Unfortunately I haven't been able to get around it :(

@uchagani
Copy link
Contributor

uchagani commented Dec 9, 2020

Also having this issue. The workaround posted above doesn't seem to work for me with the snapshot service:

image

 1: Error rendering data template: UndefinedError: 'entity_id' is undefined
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 353, in async_render
    render_result = compiled.render(kwargs)
  File "/usr/local/lib/python3.8/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/usr/local/lib/python3.8/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/usr/local/lib/python3.8/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 1, in top-level template code
  File "/usr/local/lib/python3.8/site-packages/jinja2/sandbox.py", line 407, in getattr
    value = getattr(obj, attribute)
jinja2.exceptions.UndefinedError: 'entity_id' is undefined

@parautenbach were you able to get this working?

@parautenbach
Copy link

Also having this issue. The workaround posted above doesn't seem to work for me with the snapshot service:

image
 1: Error rendering data template: UndefinedError: 'entity_id' is undefined
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 353, in async_render
    render_result = compiled.render(kwargs)
  File "/usr/local/lib/python3.8/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/usr/local/lib/python3.8/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/usr/local/lib/python3.8/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 1, in top-level template code
  File "/usr/local/lib/python3.8/site-packages/jinja2/sandbox.py", line 407, in getattr
    value = getattr(obj, attribute)
jinja2.exceptions.UndefinedError: 'entity_id' is undefined

@parautenbach were you able to get this working?

I decided to wait for the fix so didn't try the workaround. It would be better to see more of your config but from what I understand something like this should work (based on my setup):

    - service: camera.snapshot
      entity_id: camera.security_camera  # note entity id here
      data:
        entity_id: camera.security_camera
        filename: '/tmp/{{ entity_id.entity_id }}_{{ now().strftime("%Y%m%d-%H%M%S") }}.jpg'

@parautenbach
Copy link

parautenbach commented Dec 9, 2020

Hmm, sorry, I lied (inadvertently): I just discovered that I had a test script. It's the same format as above but nonetheless:

test_snapshot:
  sequence:
    - service: camera.snapshot
      entity_id: camera.security_camera
      data:
        entity_id: camera.security_camera
        filename: '/tmp/snapshot_{{ entity_id.name }}.jpg'

@memoryz
Copy link

memoryz commented Dec 28, 2020

I am running into the same issue, and the suggested workaround does not work for me.

@memoryz
Copy link

memoryz commented Dec 28, 2020

I think I just figured out the proper workaround.

The issue is not that 'entity_id' needs to be defined on the parent definition, but rather that the 'filename' template compilation runs into undefined error when looking for the reference 'entity_id'.

So to workaround this issue, you just need to remove the reference to 'entity_id' in the 'filename' template, and hardcode it.

  - trigger:
      platform: state
      entity_id: binary_sensor.main_door
      to: 'on'
    action:
      - service: camera.record
        entity_id: camera.hall
        data:
          entity_id: camera.hall
          filename: '/tmp/hall_{{ now().strftime("%Y%m%d-%H%M%S") }}.mp4'
          duration: 8

@parautenbach
Copy link

I am running into the same issue, and the suggested workaround does not work for me.

You need to give more details. What exactly did you do? Which version of HA? I can categorically say that my test script illustrates the issue – and still does, at least with HA 0.118.5. Removing the parent entity_id will cause the filename to fail – for a given valid camera entity.

The issue is not that 'entity_id' needs to be defined on the parent definition, but rather that the 'filename' template compilation runs into undefined error when looking for the reference 'entity_id'.

What you say above is what's stated in the original bug report, and confirmed by this error:

jinja2.exceptions.UndefinedError: 'entity_id' is undefined

So to workaround this issue, you just need to remove the reference to 'entity_id' in the 'filename' template, and hardcode it.

This completely defeats the purpose. The point is not having to hardcode it. I don't understand how your workaround is one. It's just the non-generic way to do it.

@memoryz
Copy link

memoryz commented Dec 28, 2020

I am running into the same issue, and the suggested workaround does not work for me.

You need to give more details. What exactly did you do? Which version of HA? I can categorically say that my test script illustrates the issue – and still does, at least with HA 0.118.5. Removing the parent entity_id will cause the filename to fail – for a given valid camera entity.

The configuration I used was almost identical to @ruimarinho's config, but it doesn't work. The error was the same: jinja2.exceptions.UndefinedError: 'entity_id' is undefined. HA version is 2020.12.1. Whether or not having the parent entity_id does not make any difference. I get same error as long as I reference entity_id in the filename template.

The issue is not that 'entity_id' needs to be defined on the parent definition, but rather that the 'filename' template compilation runs into undefined error when looking for the reference 'entity_id'.

What you say above is what's stated in the original bug report, and confirmed by this error:

jinja2.exceptions.UndefinedError: 'entity_id' is undefined

So to workaround this issue, you just need to remove the reference to 'entity_id' in the 'filename' template, and hardcode it.

This completely defeats the purpose. The point is not having to hardcode it. I don't understand how your workaround is one. It's just the non-generic way to do it.

I didn't realize that's the purpose. For me, I started with having entity_id in the filename template and ran into this issue. Hardcoding the entity id is acceptable in my scenario.

@github-actions
Copy link

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale label Mar 28, 2021
@parautenbach
Copy link

This is still an issue on version 2021.3.3.

@github-actions github-actions bot removed the stale label Mar 29, 2021
@bennydente
Copy link

I think I just figured out the proper workaround.

The issue is not that 'entity_id' needs to be defined on the parent definition, but rather that the 'filename' template compilation runs into undefined error when looking for the reference 'entity_id'.

So to workaround this issue, you just need to remove the reference to 'entity_id' in the 'filename' template, and hardcode it.

  - trigger:
      platform: state
      entity_id: binary_sensor.main_door
      to: 'on'
    action:
      - service: camera.record
        entity_id: camera.hall
        data:
          entity_id: camera.hall
          filename: '/tmp/hall_{{ now().strftime("%Y%m%d-%H%M%S") }}.mp4'
          duration: 8

Hello,
This issue is still on version 2021.4.4. Hard code entity name in filename is not useful if you have multiple camera as entity_id i think

service: camera.record
target:
  entity_id:
    - camera.one
    - camera.two
data:
  duration: 20
  filename: >-
    /somepath/whatnamehere_{{ now().strftime("%d%m%Y-%H%M%S") }}.mp4
  lookback: 10 

@blalor
Copy link
Contributor

blalor commented Jun 22, 2021

Still an issue on core-2021.6.6.

@Extreme79
Copy link

Still an issue :(

@extera-nl
Copy link

Unbelievable... still not working

Someone has a good workaround?

@github-actions
Copy link

github-actions bot commented Jan 3, 2022

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale label Jan 3, 2022
@blalor
Copy link
Contributor

blalor commented Jan 3, 2022

Still exists on 2021.12.7.

@github-actions github-actions bot removed the stale label Jan 3, 2022
@mcoms
Copy link

mcoms commented Jan 3, 2022

The two template rendering lines are here and here for reference. They do not appear to be setting up entity_id in the Jinja context.

@loca5790
Copy link

Can confirm still an issue.... unless I'm missing something. How else are people enabling recording? enable_motion_recording or an addon?

@polcape
Copy link

polcape commented Jan 15, 2022

Issue still present. Anyone have a workaround for it?
I want take snapshot of multiple camera and store with name..

Thanks

@parautenbach
Copy link

The workaround is to hardcode whatever information you want to use in the template.

As for taking multiple camera snapshots, I can't speak for that, except to say this issue has no relation to that.

@loca5790
Copy link

issue is you can't remote record with the camera and access it at the same time through the IP. I've resorted to storing local on SD and having a back up pull from it.

@parautenbach
Copy link

That has nothing to do with this issue, which is about templating. What you mention has to do with your camera and the number of streams it can handle or is configured to handle.

@aneisch
Copy link
Contributor

aneisch commented Sep 25, 2023

2023.9.3: Still experiencing this when attempting to create a snapshot as documented at https://www.home-assistant.io/integrations/camera/#service-snapshot.

#40241 (comment) workaround worked for me.

@parautenbach
Copy link

Since it seems like this will never be fixed, better to remove the stated functionality:

home-assistant/home-assistant.io#29834

@illuzn
Copy link

illuzn commented Jan 14, 2024

Just adding to the above because I struggled with escaping... if you want to remove the "camera." prefix from the filename you can use the following template (I would not recommend using entity_id.name because you can have names which contain invalid characters for filenames).

service: camera.record
target:
  entity_id: 
    - camera.front_yard
    - camera.front_door
data:
  filename: /media/{{ '{{ entity_id.entity_id|replace(\'camera.\', \'\') }}' }}_{{ now().strftime('%Y%m%d-%H%M%S') }}.mp4

This will create recordings in the form /media/front_yard_20240114_133059.mp4.

@pilot51
Copy link

pilot51 commented Mar 24, 2024

I just started setting up my first camera automations and ran into this issue in 2024.3.3.

Crazy that this bug has been open for over 3 years and both the visual editor and online documentation still suggest doing something you can't do.
If it's not going to be fixed soon, a little notice in the docs mentioning the bug and workaround would go a long way.

@issue-triage-workflows
Copy link

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

@parautenbach
Copy link

This is still an issue that needs to be fixed, regardless of the less than ideal workarounds.

@issue-triage-workflows
Copy link

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

@DubyaDude
Copy link

This is still an issue that needs to be fixed as of version 2024.9.2

@emontnemery
Copy link
Contributor

This is not really a bug, the issue is the implicit templating of action service data which requires the nested templating suggested above.

The nested templating is difficult to understand and document though, and I think the best option is to remove the pre-defined template variable and let users create their own variables.

After looking through the source code, camera.record and camera.snapshot are the only services which have these kind of pre-defined variables.

@parautenbach
Copy link

Thanks a mil @emontnemery for the feedback. Glad to get first-hand feedback from the core team. :-)

The reason I logged it as a bug, is that the behaviour changed at some point (and doesn't work as documented).

Your explanation of nested templating helps and confirms a previous comment.

The templates in the service call are now evaluated before the service calls are sent.

Perhaps this change was made to be consistent with how other actions/operations internal work, but it still means it changed the original behaviour.

The nested templating is difficult to understand and document though

Indeed, and I don't have a suggestion, to be honest, because one will need to explain the internal operation of how templates get evaluated, which leads to a bleeding abstraction, which isn't ideal.

I think the best option is to remove the pre-defined template variable and let users create their own variables.

This sounds good. Are you thinking in a similar fashion to trigger and action variables? In such fields, one would still need some kind of reference to a trigger or service entity (from the context). Maybe a this built-in?

(Side note: I want to add that entity_id.entity_id also reads oddly to me, since I'd normally assume that the first entity_id is a string and not an object.)

After looking through the source code, camera.record and camera.snapshot are the only services which have these kind of pre-defined variables.

There might be other actions that might benefit from your suggestion too. I'll need to think a bit though, but if there are, it might help it towards an architectural proposal (if needed).

@emontnemery
Copy link
Contributor

@parautenbach What I mean is to change a script with a template like this, the latter example is what I suggest for the updated documentation in home-assistant/home-assistant.io#35267

actions:
  - action: camera.record
    target:
      entity_id: camera.yourcamera
    data:
      filename: '/tmp/{{ "{{ entity_id.entity_id }}" }}_{{ now().strftime("%Y%m%d-%H%M%S") }}.mp4'

To:

actions:
  - variables:
      entity_id: camera.yourcamera  # Store the camera entity_id in a variable for reuse
  - action: camera.record
    target:
      entity_id: '{{ entity_id }}'
    data:
      filename: '/tmp/{{ entity_id }}_{{ now().strftime("%Y%m%d-%H%M%S") }}.mp4'

Perhaps this change was made to be consistent with how other actions/operations internal work, but it still means it changed the original behaviour.

Yeah, the functionality changed when implicit templating was added to service data four years ago.

I want to add that entity_id.entity_id also reads oddly to me, since I'd normally assume that the first entity_id is a string and not an object

Yes, that was a mistake in the PR which added the entity_id variable to the camera services.

@parautenbach
Copy link

Thanks for the clarification @emontnemery! I'm on the same page now and would support this.

@NODeeJay
Copy link

NODeeJay commented Nov 28, 2024

There is unfortunately a very unpractical side effect with this solution and the better fix would be to properly procces the template and variables. With the change multiple entities are not supported anymore and existing recording scripts and automations need to be adapted, see:
#40241 (comment)
#40241 (comment)
#40241 (comment)

Since the current functionality is still working, is there a chance to fix the issue with the filename template instead of removing the functionality?

@parautenbach
Copy link

Could you elaborate? Are you referring to the issue of keeping this dynamic if you're re-using a script or trigger an automation for multiple cameras?

Remember that you can use templates in variables, so in an automation's actions you can get the trigger's entity ID and use that in the variable to do what you need (like mapping from a motion sensor entity to a camera entity). Same for scripts: You don't need to set that variable within the script. You can pass it in as a script field, and that field can be dynamic.

@NODeeJay
Copy link

I see the point of reusing the once build camera automation/script, but it's not the same as starting the recording for independent cameras the same time with the same values.

How should this look like in future?

metadata: {}
target:
  entity_id:
    - camera.cam_entrance
    - camera.cam_garageentrance
    - camera.cam_garden_low_latency
    - camera.cam_garden2_substream
data:
  duration: 15
  lookback: 5
  filename: >-
    /media/recordings/{{ '{{ entity_id.name|replace(\' MainStream\',\'\') }}'
    }}_{{ now().strftime("%Y%m%d-%H%M%S") }}.mp4
action: camera.record

without creating a complex automation calling over and over the same script until all entities are recording.
On dozens of ESP-cams, Octoprint, outside cameras etc. I have those groups of 2 - 5 cameras that record the same time. I tried the new suggestion with a reusable script but even though I use parallel in the script to execute the record action there is a noticeable time difference that was not there before.

Or we are missing the arrays and a way to step through variables in the templates. In current functionality I can provide n-entities with the same settings and start recording the same time, this does not work anymore the new way. See also the discussion in the HA forum starting 18 days ago: https://community.home-assistant.io/t/camera-snapshot-entity-id-in-filename/31697/19?u=nodeejay

IMHO, the old solution was not optimal, but the new one is even worse. Then I really prefer the hack and the functionality and I strongly assume at least @bennydente, @mcoms & @illuzn might agree?

@illuzn
Copy link

illuzn commented Jan 12, 2025

@NODeeJay and everyone else... dropping this here because the new documentation doesn't explicitly mention how to do this for multiple camera records.

I tried using a for_each repeat loop but that records each camera sequentially i.e. it waits for one camera to finish before recording the next.

The kind of inelegant way I got this to work is as follows:

  1. Create a helper script that actually performs the record action and accepts a camera entity:
alias: Camera Record Helper
description: "Helper Script to Record Multiple Cameras in Parallel"
fields:
  camera_entity:
    selector:
      text: null
    name: Camera Entity
    required: true
    description: Camera to Record
sequence:
  - action: camera.record
    data:
      duration: 300
      lookback: 300
      # Turns camera.Front into Front_20250131-130102.mp4 change to suit your usecase.
      # Putting year month day helps when sorting your recordings.
      filename: >-
        /share/recordings/{{ camera_entity|replace('camera.', '') }}_{{
        now().strftime('%Y%m%d-%H%M%S') }}.mp4
    target:
      entity_id: "{{ camera_entity }}"
mode: parallel
# Change this if you need to record more than 10 cameras at once.
max: 10
  1. Now my automation is effectively as follows (I record my cameras when an alarm triggers but customise for your own use):
alias: Record Cameras on Alarm Trigger
description: Records 5 minutes of camera footage when the alarm is triggered
triggers:
  - entity_id:
      - alarm_control_panel.alarm_whole_house
    to: triggered
    trigger: state
actions:
  - repeat:
      for_each:
        - camera.backyard
        - camera.front_door
        - camera.frontyard
      sequence:
        - action: script.turn_on
          data:
            variables:
              camera_entity: "{{ repeat.item }}"
          target:
            entity_id: script.camera_record_helper
mode: single

There is probably a way to do this without having the seperate helper but as they say, perfect is the enemy of good enough and this works good enough for me.

@parautenbach
Copy link

parautenbach commented Jan 12, 2025

There is probably a way to do this without having the seperate helper but as they say, perfect is the enemy of good enough and this works good enough for me.

script.turn_on will indeed execute in a non-blocking way. To avoid the helper script, you can have the script contents in the automation and just use the parallel key.

actions:
  - repeat:
      for_each:
        - camera.backyard
        - camera.front_door
        - camera.frontyard
      sequence:
        - parallel:
            ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment