HA Intents

[jaswalters]

If I create an intent and intent script with custom sentences for devices in HA, will Willow leverage these to alter a device?

[nikito]

Yes, I have numerous custom intents and intent scripts in HA that work with willow. Just note you need to be using WIS (not local commands) for it to work. Also in current code if your speech responses exceed 64 characters they will get cut off in tts, but this is being looked into in issue #159 😊

[nikito]

It's possible, but requires some custom intent sentence/script coding. They details this at https://developers.home-assistant.io/docs/voice/intent-recognition/template-sentence-syntax https://www.home-assistant.io/integrations/conversation/ and https://www.home-assistant.io/integrations/intent_script/

To do the above example, you would likely need to have some collection of locations stored in a list; I don't know that home assistant allows "wild card" words/slots that take in any random value.

[rob-odwyer]

OK cool - that lines up with what I'd seen so far.

I can define sentences and intent_scripts and get Willow to match them, but the "wild card" part is specifically what I was hoping to do.

The weather example isn't a great one since defining a list of locations isn't that bad - specifically what I want to do is add items to a grocery list.

Sounds like I'll either need to wait for support in the hassil language or build some kind of proxy in front of my HA that handles these commands instead.

[kristiankielhofner]

The "proxy in front of HA" is one of the use cases for the Willow Application Server (in development). The first release will only support management of Willow devices but it will move far beyond that.

There are a lot of voice assistant tasks that don't make any sense for Home Assistant to be involved in - such as your use case of adding items to a grocery list. Or things like calendars, reminders, timers, alarms, and many other (arguably most) tasks.

We hope with WAS and the functionality enabled by Willow and WIS we'll be able to kick-off a very robust open voice application and integration ecosystem.

The general concept is to use NLP and routing to applications and integrations - basically Alexa skills without the awkward and confusing "Ask XYZ" grammar where the user can define general grammar and inputs/outputs to various providers.

Full functionality for this is likely months away but it is very much on our radar.

[rob-odwyer]

That makes a lot of sense as a long term direction, thanks for the explanation!

Are you thinking of designing WAS to use a more specific grammar, e.g. certain sentence templates map to certain integrations?

Or something LLM-powered, like the way Langchain and similar tools construct agents? It would be interesting to use an agent that you could pass possible integrations to as actions along with a query. Might not be appropriate for everyone's use case though, since its inherently a bit unpredictable.

[kristiankielhofner]

WAS will use NLP approaches - LLMs don't really make sense for this use case and are impractical to run on most GPUs and CPUs.

We will likely use Rasa (or similar) for flexibility in grammar and defining of rules for intent matching to actions, apps, integrations, etc.