Commit Graph

51 Commits

Author SHA1 Message Date
Allen Porter
1fc624c7a7 Update LLM selector serializer to support ObjectSelector fields and arrays (#148094) 2025-07-04 13:05:16 +02:00
Paulus Schoutsen
7cccdf2205 Add accept keyword to Media selector (#145527)
* Add accept keyword to Media selector

* Adjust test
2025-06-24 07:36:48 -05:00
Paulus Schoutsen
059c12798d Drop user prompt from LLMContext (#146787) 2025-06-13 22:01:39 -04:00
Allen Porter
b16151ac6d Add an LLM tool for fetching todo list items (#143777)
* Add a tool for fetching todo list items

* Simplify the todo list interface by adding an "all" status

* Update prompt to improve performance on smaller models
2025-04-30 08:49:33 -04:00
Allen Porter
db043b26da Fix quality loss for LLM conversation agent question answering (#142873)
* Fix a bug parsing a streaming response with no json

* Remove debug lines

* Fix  quality loss for LLM conversation agent question answering

* Update tests
2025-04-14 10:05:34 +02:00
Allen Porter
d91528648f Update ollama to allow selecting mutiple LLM APIs (#142445)
* Update ollama to allow selecting mutiple LLM APIs

* Update homeassistant/helpers/llm.py

* Avoid gather since these don't do I/O

---------

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2025-04-13 15:37:46 -07:00
John Karabudak
85d2e3d006 Fix LLM to speed up prefill (#141156)
* fix: two minor LLM changes to speed up prefill

- moved the current date/time to the end of the prompt
- started sorting all entities by last_changed

* addressed PR comments

* fixed tests

* reduced scope of try/catch in LLM prompt

* addressed more PR comments

* fixed Anthropic test

* addressed another PR comment

* fixed remainder of tests
2025-03-30 13:30:40 -07:00
Allen Porter
4e2dfba45f Omit state from the Assist LLM prompts (#141034)
* Omit state from the Assist LLM prompts

* Add back the stateful prompt
2025-03-22 15:41:51 -04:00
Allen Porter
a83bf4f514 Add a GetHomeState tool to return the current state of the home (#140971)
* Add a GetHomeState tool to return the current state of the home

* Fix check for exposing entities

* Add "all" to get home state description
2025-03-20 19:37:54 -07:00
Paulus Schoutsen
ac42c9386c Simplify llm calendar tool (#137402)
* Simplify calendar tool

* Clean up exposed entities
2025-02-05 11:42:41 +01:00
Denis Shulyaka
85540cea3f Add LLM ActionTool (#136591)
Add ActionTool
2025-01-27 14:21:27 -05:00
Paulus Schoutsen
005ae3ace6 Allow LLMs to get calendar events from exposed calendars (#136304) 2025-01-23 17:54:04 -05:00
Allen Porter
6e111d18ec Allow unregistering LLM APIs (#135162) 2025-01-09 08:18:25 -08:00
Paulus Schoutsen
ae34a6b375 Do not double expose scripts in LLM tools (#131726) 2024-11-27 18:04:08 +01:00
Paulus Schoutsen
7e03100af2 Allow an LLM to see script response values (#131683) 2024-11-26 23:51:21 -06:00
Jan-Philipp Benecke
1a71fbe427 Add intent to cancel all timers (#130873)
* Add intent to cancel all timers

* Add intent to llm test
2024-11-26 09:59:41 -06:00
Denis Shulyaka
e0e61b5262 Expose scripts with no fields as entities (#123061) 2024-10-22 23:14:07 -07:00
Denis Shulyaka
ed6d6575d7 Add aliases to script llm tool description (#122380)
* Add aliases to script llm tool description

* Also add name
2024-08-02 08:05:06 +02:00
Denis Shulyaka
be24475cee Update selector converters for llm script tools (#122830) 2024-07-30 08:24:03 -07:00
Allen Porter
e5bb1b2cc6 Update LLM prompt to improve quality for local LLMs (#122746) 2024-07-29 12:04:23 +02:00
Denis Shulyaka
064d7261b4 Ensure script llm tool name does not start with a digit (#122349)
* Ensure script tool name does not start with a digit

* Fix test name
2024-07-22 11:11:09 +02:00
Alan
8ff4991f07 LLM to handle int attributes (#121037) 2024-07-06 11:41:18 +02:00
Michael Hansen
d3ceaef098 Allow timer management from any device (#120440) 2024-06-26 09:06:56 +02:00
Denis Shulyaka
2386ed3830 Add script llm tool (#118936)
* Add script llm tool

* Add tests

* More tests

* more test

* more test

* Add area and floor resolving

* coverage

* coverage

* fix ColorTempSelector

* fix mypy

* fix mypy

* add script reload test

* Cache script tool parameters

* Make custom_serializer a part of api

---------

Co-authored-by: Michael Hansen <mike@rhasspy.org>
2024-06-25 10:43:26 -05:00
Paulus Schoutsen
19f97a3e53 LLM to handle decimal attributes (#120257) 2024-06-23 17:09:57 -04:00
tronikos
6b8bddf6e3 Make remaining time of timers available to LLMs (#118696)
* Include speech_slots in IntentResponse.as_dict

* Populate speech_slots only if available

* fix typo

* Add test

* test all fields

* Fix another test

---------

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2024-06-14 20:47:41 +02:00
epenet
c70cfbb535 Fix arguments-renamed pylint warning in tests (#119473) 2024-06-12 12:25:29 +02:00
Paulus Schoutsen
c49ca5ed56 Ensure intent tools have safe names (#119144) 2024-06-08 17:53:47 +02:00
Paulus Schoutsen
52ad90a68d Include script description in LLM exposed entities (#118749)
* Include script description in LLM exposed entities

* Fix race in test

* Fix type

* Expose script

* Remove fields
2024-06-04 11:18:07 -04:00
Paulus Schoutsen
bdcfd93129 Automatically fill in slots based on LLM context (#118619)
* Automatically fill in slots from LLM context

* Add tests

* Apply suggestions from code review

Co-authored-by: Allen Porter <allen@thebends.org>

---------

Co-authored-by: Allen Porter <allen@thebends.org>
2024-06-03 10:36:41 -04:00
tronikos
dfb407728f Stop instructing LLM to not pass the domain as a list (#118590) 2024-05-31 20:21:18 -04:00
Paulus Schoutsen
6dd01dbff7 Rename llm.ToolContext to llm.LLMContext (#118566) 2024-05-31 11:11:24 -04:00
tronikos
2bd142d3a6 Improve LLM prompt (#118520) 2024-05-30 22:03:57 -04:00
tronikos
0d6c7d0973 Fix LLMs asking which area when there is only one device (#118518)
* Ignore deprecated open and close cover intents for LLMs

* Fix LLMs asking which area when there is only one device

* remove unrelated changed

* remove unrelated changes
2024-05-30 17:14:11 -04:00
tronikos
4893faa671 Instruct LLM to not pass a list to the domain (#118451) 2024-05-30 08:37:45 +02:00
tronikos
639f6c640c Improve LLM prompt (#118443)
* Improve LLM prompt

* test

* improvements

* improvements
2024-05-29 21:44:33 -04:00
tronikos
4056c4c2cc Ask LLM to pass area name and domain (#118357) 2024-05-29 08:03:43 -04:00
Paulus Schoutsen
d22871f1fd Reduce the intent response data sent to LLMs (#118346)
* Reduce the intent response data sent to LLMs

* No longer delete speech
2024-05-28 23:07:00 -04:00
Paulus Schoutsen
c097a05ed4 Tweak Assist LLM API prompt (#118343) 2024-05-28 22:43:22 -04:00
Paulus Schoutsen
615a1eda51 LLM Assist API to ignore intents if not needed for exposed entities or calling device (#118283)
* LLM Assist API to ignore timer intents if device doesn't support it

* Refactor to use API instances

* Extract ToolContext class

* Limit exposed intents based on exposed entities
2024-05-28 21:29:18 -04:00
Paulus Schoutsen
f0d7f48930 Handle generic commands as area commands in the LLM Assist API (#118276)
* Handle generic commands as area commands in the LLM Assist API

* Add word area
2024-05-28 11:21:17 -04:00
Paulus Schoutsen
ecb05989ca Add exposed entities to the Assist LLM API prompt (#118203)
* Add exposed entities to the Assist LLM API prompt

* Check expose entities in Google test

* Copy Google default prompt test cases to LLM tests
2024-05-27 00:27:08 -04:00
Paulus Schoutsen
87fc27eeae Teach Context about deepcopy (#118206)
Teach context about deepcopy
2024-05-26 23:14:02 -04:00
Denis Shulyaka
cee3be5f7a Break long strings in LLM tools (#118114)
* Break long code strings

* Address comments

---------

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2024-05-25 14:24:51 -04:00
Denis Shulyaka
da74ac06d7 Add user name and location to the LLM assist prompt (#118071)
Add user name and location to the llm assist prompt
2024-05-24 22:23:05 -04:00
Paulus Schoutsen
7554ca9460 Allow llm API to render dynamic template prompt (#118055)
* Allow llm API to render dynamic template prompt

* Make rendering api prompt async so it can become a RAG

* Fix test
2024-05-24 16:04:48 -04:00
Denis Shulyaka
009c9e79ae LLM Tools: Add device_id (#117884) 2024-05-21 21:24:46 -04:00
Michael Hansen
8079cc0464 Add description to intent handlers and use in LLM helper (#117864) 2024-05-21 12:54:34 -04:00
Denis Shulyaka
c3196a5667 LLM Tools support for Google Generative AI integration (#117644)
* initial commit

* Undo prompt chenges

* Move format_tool out of the class

* Only catch HomeAssistantError and vol.Invalid

* Add config flow option

* Fix type

* Add translation

* Allow changing API access from options flow

* Allow model picking

* Remove allowing HASS Access in main flow

* Move model to the top in options flow

* Make prompt conditional based on API access

* convert only once to dict

* Reduce debug logging

* Update title

* re-order models

* Address comments

* Move things

* Update labels

* Add tool call tests

* coverage

* Use LLM APIs

* Fixes

* Address comments

* Reinstate the title to not break entity name

---------

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2024-05-19 22:11:25 -04:00
Paulus Schoutsen
d001e7daea Add API class to LLM helper (#117707)
* Add API class to LLM helper

* Add more tests

* Rename intent to assist to broaden scope
2024-05-18 21:14:05 -04:00