Voice tutorial: Add procedure on setting up esp32 for voice control (#27138)
* Assist: add procedure on setting up ESP32 for voice control - Draft * Assist ESP32: Add ref to troubleshooting section * Apply suggestions from code review Co-authored-by: Paulus Schoutsen <balloob@gmail.com> * ESP32 voice remote: update procedure * Fix typos * Add link to procedure on how to create custom sentences * Add step on installing driver * Implement review feedback by ballob * Fix typo * Add some more procedures - Add procedure on creating a voice remote if no Cloud subscription is available - add procedure on creating a local assistant using Whisper and Piper - add procedure on exposing entities to Assist * Minor tweaks * Apply suggestions from code review Co-authored-by: Paulus Schoutsen <balloob@gmail.com> * Implement review feedback * Implement review feedback * Fix typo * Remove instructions on creating fully local esp32 setup * Delete step to check server language * Remove dead link in aside * Fix link * Improve wording * Improve wording * Fix broken link * Implement review feedback * Fix navigation text --------- Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
@ -46,6 +46,9 @@
|
||||
<li>{% active_link /docs/assist/apple/ Assist for Apple devices %}</li>
|
||||
<li>{% active_link /docs/assist/builtin_sentences/ Built-in sentences %}</li>
|
||||
<li>{% active_link /docs/assist/custom_sentences/ Custom sentences %}</li>
|
||||
<li>{% active_link /docs/assist/voice_remote_local_assistant/ Configuring a local assistant %}</li>
|
||||
<li>{% active_link /projects/thirteen-usd-voice-remote/ Tutorial: $13 voice remote %}
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
|
@ -15,6 +15,8 @@
|
||||
{% include asides/getting_started_navigation.html %}
|
||||
{% elsif root == 'docs' %}
|
||||
{% include asides/docs_navigation.html %}
|
||||
{% elsif root == 'projects' %}
|
||||
{% include asides/docs_navigation.html %}
|
||||
{% elsif root == 'faq' %}
|
||||
{% include asides/faq_navigation.html %}
|
||||
{% elsif root == 'hassio' or root == 'addons' %}
|
||||
|
39
source/docs/assist/voice_remote_local_assistant.markdown
Normal file
@ -0,0 +1,39 @@
|
||||
---
|
||||
title: "Configuring a local Assist pipeline"
|
||||
---
|
||||
|
||||
In Home Assistant, the Assist pipelines are made up of various components that together form a voice assistant.
|
||||
|
||||
For each component you can choose from different options. We have prepared a speech-to-text and text-to-speech option that runs fully local.
|
||||
|
||||
The speech-to-text option is [Whisper](https://github.com/openai/whisper). It's an open source AI model that supports [various languages](https://github.com/openai/whisper#available-models-and-languages). We use a forked version called [faster-whisper](https://github.com/guillaumekln/faster-whisper). On a Raspberry Pi 4, it takes around 8 seconds to process incoming voice commands. On an Intel NUC it is done in under a second.
|
||||
|
||||
For text-to-speech we have developed [Piper](https://github.com/rhasspy/piper). Piper is a fast, local neural text to speech system that sounds great and is optimized for the Raspberry Pi 4. It supports [many languages](https://rhasspy.github.io/piper-samples/). On a Raspberry Pi, using medium quality models, it can generate 1.6s of voice in a second.
|
||||
|
||||
## Setting up a local Assist pipeline
|
||||
|
||||
1. Install the add-ons to convert text into speech and vice versa.
|
||||
* Install the {% my supervisor_addon addon="whisper" title="**Whisper**" %} and the {% my supervisor_addon addon="piper" title="**Piper**" %} add-ons.
|
||||

|
||||
* Start both add-ons. This may take a while.
|
||||
* Once the add-ons are started, head over to the integrations under {% my integrations title="**Settings** > **Devices & Services**" %}.
|
||||
* You should now see both being discovered by the [Wyoming integration](/integations/wyoming).
|
||||

|
||||
* For both integrations, select **Configure**.
|
||||
* Once the setup is complete, you should see both Piper and Whisper in one integration.
|
||||

|
||||
* **Whisper** converts speech into text.
|
||||
* **Piper** converts text into speech.
|
||||
* **Wyoming** is the protocol they are both using to communicate.
|
||||
1. Setup your assistant.
|
||||
* Go to **Settings** > **Voice assistants** and select **Add assistant**.
|
||||

|
||||
* Enter a name. You can pick any name that is meaningful to you.
|
||||
* Select the language that you want to speak.
|
||||
* Under **Conversation agent**, select **Home Assistant**.
|
||||
* Under **Speech-to-text**, select **faster-whisper**.
|
||||
* Under **Text-to-speech**, select **piper**.
|
||||
* Depending on your language, you may be able to select different language variants.
|
||||
1. That's it. You ensured your voice commands are processed locally on your device.
|
||||
1. If you haven't done so yet, [expose your devices to Assist](/projects/private-voice-assistant/voice_remote_expose_devices/#exposing-your-devices).
|
||||
* Otherwise you won't be able to control them by voice.
|
BIN
source/images/assist/assistant-expose-01.png
Normal file
After Width: | Height: | Size: 20 KiB |
BIN
source/images/assist/assistant-expose-02.png
Normal file
After Width: | Height: | Size: 29 KiB |
BIN
source/images/assist/esp32-atom-flash-06.png
Normal file
After Width: | Height: | Size: 34 KiB |
BIN
source/images/assist/esp32-atom-flash-07.png
Normal file
After Width: | Height: | Size: 11 KiB |
BIN
source/images/assist/esp32-atom-flash-no-port.png
Normal file
After Width: | Height: | Size: 234 KiB |
BIN
source/images/assist/esp32-atom-flash-select-port.png
Normal file
After Width: | Height: | Size: 36 KiB |
BIN
source/images/assist/m5stack-atom-echo-discovered-03.png
Normal file
After Width: | Height: | Size: 18 KiB |
BIN
source/images/assist/piper-whisper-install-01.png
Normal file
After Width: | Height: | Size: 57 KiB |
BIN
source/images/assist/piper-whisper-install-02.png
Normal file
After Width: | Height: | Size: 18 KiB |
BIN
source/images/assist/piper-whisper-install-03.png
Normal file
After Width: | Height: | Size: 9.0 KiB |
BIN
source/images/assist/piper-whisper-install-05.png
Normal file
After Width: | Height: | Size: 139 KiB |
BIN
source/images/assist/server-language-01.png
Normal file
After Width: | Height: | Size: 11 KiB |
80
source/projects/thirteen-usd-voice-remote.markdown
Normal file
@ -0,0 +1,80 @@
|
||||
---
|
||||
title: "$13 voice remote for Home Assistant"
|
||||
---
|
||||
|
||||
This tutorial will guide you to turn an ATOM Echo into the
|
||||
world's most private voice assistant. Pick up the tiny device to talk to
|
||||
your smart home. Issue commands and get responses!
|
||||
|
||||
<lite-youtube videoid="w6QxGdxVMJs" videotitle="$13 voice remote for Home Assistant
|
||||
"></lite-youtube>
|
||||
|
||||
## Required material
|
||||
|
||||
* Home Assistant 2023.5 or later
|
||||
* [Home Assistant Cloud](https://www.nabucasa.com) or a manually configured [Assist Pipeline](/docs/assist/voice_remote_local_assistant)
|
||||
* The password to your 2.4 GHz Wi-Fi network
|
||||
* Chrome (or a Chromium-based browser like Edge) on desktop (not Android/iOS)
|
||||
* [M5Stack ATOM Echo Development Kit](https://shop.m5stack.com/products/atom-echo-smart-speaker-dev-kit?ref=NabuCasa)
|
||||
* USB-C cable to connect the ATOM Echo
|
||||
|
||||
|
||||
|
||||
## Installing the software onto the ATOM Echo
|
||||
|
||||
Before you can use this device with Home Assistant, you need to install a bit of software on it.
|
||||
|
||||
1. Make sure this page is opened in a Chromium-based browser on a desktop. It does not work on a tablet or phone.
|
||||
* Select the **Connect** button below. If your browser does not support web serial, there is no button but a text.
|
||||
|
||||
<script type="module" src="https://unpkg.com/esp-web-tools@9/dist/web/install-button.js?module"></script>
|
||||
<esp-web-install-button manifest="https://firmware.esphome.io/voice-assistant/m5stack-atom-echo/manifest.json"></esp-web-install-button>
|
||||
|
||||
2. Connect the ATOM Echo to your computer.
|
||||
* In the popup window, view the available ports.
|
||||
* Plug the USB-C cable into the ATOM Echo and connect it to your computer.
|
||||
* In the pop-up window, there should now appear a new entry. Select this USB serial port and select **Connect**.
|
||||
* Depending on your computer, the entry might look different.
|
||||

|
||||
* If no new port shows, your system may be missing a driver. Close the pop-up window.
|
||||
* In the dialog, select the CH342 driver, install it, then **Try again**.
|
||||

|
||||
3. Select **Install Voice Assistant**, then **Install**.
|
||||
* Follow the instructions provided by the installation wizard.
|
||||
* Add the ATOM Echo to your Wi-Fi:
|
||||
* When prompted, select your network from the list and enter the credentials to your 2.4 GHz Wi-Fi network.
|
||||
* Select **Connect**.
|
||||
* The ATOM Echo now joined your network. Select **Add to Home Assistant**.
|
||||
4. This opens the **My** link to Home Assistant.
|
||||
* If you have not used My Home Assistant before, you will need to configure it. If your Home Assistant URL is not accessible on `http://homeassistant.local:8123`, replace it with the URL to your Home Assistant instance.
|
||||
* Open the link.
|
||||

|
||||
5. Select **OK**.
|
||||
|
||||

|
||||
6. To add the newly discovered device, select the ATOM Echo from the list.
|
||||
* Add your ATOM Echo to a room and select **Finish**.
|
||||
7. You should now see a new **M5Stack Atom Echo** integration.
|
||||

|
||||
* Your ATOM Echo is connected to Home Assistant over Wi-Fi. You can now move it to any place in your home with a USB power supply.
|
||||
8. Congratulations! You can now voice control Home Assistant using a button with build-in microphone. Now give some commands.
|
||||
|
||||
## Controlling Home Assistant over the ATOM Echo
|
||||
|
||||
1. Press and hold the button on your ATOM Echo.
|
||||
* The LED should light up in blue.
|
||||
1. Say a [supported voice command](/docs/assist/builtin_sentences/). For example, *Turn off the light in the kitchen*.
|
||||
* Make sure you’re using the area name exactly as you defined it in Home Assistant.
|
||||
* You can also ask a question, such as
|
||||
* *Is the front door locked?*
|
||||
* *Which lights are on in the living room?*
|
||||
1. Let go of the button.
|
||||
* The LED should light up in green.
|
||||
* Home Assistant will confirm the action.
|
||||
1. Your command is not supported? [Add your own commands](/integrations/conversation/).
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
Are things not working as expected?
|
||||
|
||||
* Checkout the [general troubleshooting section for Assist](/projects/private-voice-assistant/troubleshooting/).
|