jeans/home-assistant.io
jeans/home-assistant.io
1
0
Fork 0
mirror of https://github.com/home-assistant/home-assistant.io.git synced 2025-07-24 09:46:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

Google Generative AI: Document service for prompts consisting of text and images using Gemini Pro Vision (#30354)

Browse Source 
Co-authored-by: c0ffeeca7 <38767475+c0ffeeca7@users.noreply.github.com>
This commit is contained in:
tronikos 2024-01-08 01:22:19 -08:00 committed by GitHub
parent adb23685de
commit 76eec1bde2
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 45 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

45
source/_integrations/google_generative_ai_conversation.markdown
View File

@ -44,3 +44,48 @@ Top K:
description: Number of top-scored tokens to consider during generation.
{% endconfiguration_basic %}
## Services
### Service `google_generative_ai_conversation.generate_content`
Allows you to ask Gemini Pro or Gemini Pro Vision to generate content from a prompt consisting of text and optionally images.
This service populates [response data](/docs/scripts/service-calls#use-templates-to-handle-response-data) with the generated content.
| Service data attribute | Optional | Description | Example |
| ---------------------- | -------- | ---------------------------------------------- | ------------------- |
| `prompt` | no | The prompt for generating the content. | Describe this image |
| `image_filename` | yes | File names for images to include in the prompt. | /tmp/image.jpg |
{% raw %}
```yaml
service: google_generative_ai_conversation.generate_content
data:
prompt: >-
Very briefly describe what you see in this image from my doorbell camera.
Your message needs to be short to fit in a phone notification. Don't
describe stationary objects or buildings.
image_filename: /tmp/doorbell_snapshot.jpg
response_variable: generated_content
```
{% endraw %}
The response data field `text` will contain the generated content.
Another example with multiple images:
{% raw %}
```yaml
service: google_generative_ai_conversation.generate_content
data:
prompt: >-
Briefly describe what happened in the following sequence of images
from my driveway camera.
image_filename:
- /tmp/driveway_snapshot1.jpg
- /tmp/driveway_snapshot2.jpg
- /tmp/driveway_snapshot3.jpg
- /tmp/driveway_snapshot4.jpg
response_variable: generated_content
```
{% endraw %}