Voice tutorial: Add procedure on setting up esp32 for voice control (#27138)

* Assist: add procedure on setting up ESP32 for voice control

- Draft

* Assist ESP32: Add ref to troubleshooting section

* Apply suggestions from code review

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>

* ESP32 voice remote: update procedure

* Fix typos

* Add link to procedure on how to create custom sentences

* Add step on installing driver

* Implement review feedback by ballob

* Fix typo

* Add some more procedures

- Add procedure on creating a voice remote if no Cloud subscription is available
- add procedure on creating a local assistant using Whisper and Piper
- add procedure on exposing entities to Assist

* Minor tweaks

* Apply suggestions from code review

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>

* Implement review feedback

* Implement review feedback

* Fix typo

* Remove instructions on creating fully local esp32 setup

* Delete step to check server language

* Remove dead link in aside

* Fix link

* Improve wording

* Improve wording

* Fix broken link

* Implement review feedback

* Fix navigation text

---------

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
This commit is contained in:
c0ffeeca7 2023-04-27 17:06:46 +02:00 committed by GitHub
parent 6b48b21f1d
commit e4b00c8fd3
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
16 changed files with 124 additions and 0 deletions

View File

@ -46,6 +46,9 @@
<li>{% active_link /docs/assist/apple/ Assist for Apple devices %}</li>
<li>{% active_link /docs/assist/builtin_sentences/ Built-in sentences %}</li>
<li>{% active_link /docs/assist/custom_sentences/ Custom sentences %}</li>
<li>{% active_link /docs/assist/voice_remote_local_assistant/ Configuring a local assistant %}</li>
<li>{% active_link /projects/thirteen-usd-voice-remote/ Tutorial: $13 voice remote %}
</li>
</ul>
</li>
<li>

View File

@ -15,6 +15,8 @@
{% include asides/getting_started_navigation.html %}
{% elsif root == 'docs' %}
{% include asides/docs_navigation.html %}
{% elsif root == 'projects' %}
{% include asides/docs_navigation.html %}
{% elsif root == 'faq' %}
{% include asides/faq_navigation.html %}
{% elsif root == 'hassio' or root == 'addons' %}

View File

@ -0,0 +1,39 @@
---
title: "Configuring a local Assist pipeline"
---
In Home Assistant, the Assist pipelines are made up of various components that together form a voice assistant.
For each component you can choose from different options. We have prepared a speech-to-text and text-to-speech option that runs fully local.
The speech-to-text option is [Whisper](https://github.com/openai/whisper). It's an open source AI model that supports [various languages](https://github.com/openai/whisper#available-models-and-languages). We use a forked version called [faster-whisper](https://github.com/guillaumekln/faster-whisper). On a Raspberry Pi 4, it takes around 8 seconds to process incoming voice commands. On an Intel NUC it is done in under a second.
For text-to-speech we have developed [Piper](https://github.com/rhasspy/piper). Piper is a fast, local neural text to speech system that sounds great and is optimized for the Raspberry Pi 4. It supports [many languages](https://rhasspy.github.io/piper-samples/). On a Raspberry Pi, using medium quality models, it can generate 1.6s of voice in a second.
## Setting up a local Assist pipeline
1. Install the add-ons to convert text into speech and vice versa.
* Install the {% my supervisor_addon addon="whisper" title="**Whisper**" %} and the {% my supervisor_addon addon="piper" title="**Piper**" %} add-ons.
![Install the Whisper and Piper add-ons](/images/assist/piper-whisper-install-01.png)
* Start both add-ons. This may take a while.
* Once the add-ons are started, head over to the integrations under {% my integrations title="**Settings** > **Devices & Services**" %}.
* You should now see both being discovered by the [Wyoming integration](/integations/wyoming).
![Whisper and Piper integrations](/images/assist/piper-whisper-install-02.png)
* For both integrations, select **Configure**.
* Once the setup is complete, you should see both Piper and Whisper in one integration.
![Whisper and Piper integration](/images/assist/piper-whisper-install-03.png)
* **Whisper** converts speech into text.
* **Piper** converts text into speech.
* **Wyoming** is the protocol they are both using to communicate.
1. Setup your assistant.
* Go to **Settings** > **Voice assistants** and select **Add assistant**.
![Enter a name for your voice assistant](/images/assist/piper-whisper-install-05.png)
* Enter a name. You can pick any name that is meaningful to you.
* Select the language that you want to speak.
* Under **Conversation agent**, select **Home Assistant**.
* Under **Speech-to-text**, select **faster-whisper**.
* Under **Text-to-speech**, select **piper**.
* Depending on your language, you may be able to select different language variants.
1. That's it. You ensured your voice commands are processed locally on your device.
1. If you haven't done so yet, [expose your devices to Assist](/projects/private-voice-assistant/voice_remote_expose_devices/#exposing-your-devices).
* Otherwise you won't be able to control them by voice.

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 234 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 36 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 57 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.0 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 139 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

View File

@ -0,0 +1,80 @@
---
title: "$13 voice remote for Home Assistant"
---
This tutorial will guide you to turn an ATOM Echo into the
world's most private voice assistant. Pick up the tiny device to talk to
your smart home. Issue commands and get responses!
<lite-youtube videoid="w6QxGdxVMJs" videotitle="$13 voice remote for Home Assistant
"></lite-youtube>
## Required material
* Home Assistant 2023.5 or later
* [Home Assistant Cloud](https://www.nabucasa.com) or a manually configured [Assist Pipeline](/docs/assist/voice_remote_local_assistant)
* The password to your 2.4&nbsp;GHz Wi-Fi network
* Chrome (or a Chromium-based browser like Edge) on desktop (not Android/iOS)
* [M5Stack ATOM Echo Development Kit](https://shop.m5stack.com/products/atom-echo-smart-speaker-dev-kit?ref=NabuCasa)
* USB-C cable to connect the ATOM Echo
## Installing the software onto the ATOM Echo
Before you can use this device with Home Assistant, you need to install a bit of software on it.
1. Make sure this page is opened in a Chromium-based browser on a desktop. It does not work on a tablet or phone.
* Select the **Connect** button below. If your browser does not support web serial, there is no button but a text.
<script type="module" src="https://unpkg.com/esp-web-tools@9/dist/web/install-button.js?module"></script>
<esp-web-install-button manifest="https://firmware.esphome.io/voice-assistant/m5stack-atom-echo/manifest.json"></esp-web-install-button>
2. Connect the ATOM Echo to your computer.
* In the popup window, view the available ports.
* Plug the USB-C cable into the ATOM Echo and connect it to your computer.
* In the pop-up window, there should now appear a new entry. Select this USB serial port and select **Connect**.
* Depending on your computer, the entry might look different.
![Select USB port](/images/assist/esp32-atom-flash-select-port.png)
* If no new port shows, your system may be missing a driver. Close the pop-up window.
* In the dialog, select the CH342 driver, install it, then **Try again**.
![Open My link](/images/assist/esp32-atom-flash-no-port.png)
3. Select **Install Voice Assistant**, then **Install**.
* Follow the instructions provided by the installation wizard.
* Add the ATOM Echo to your Wi-Fi:
* When prompted, select your network from the list and enter the credentials to your 2.4&nbsp;GHz Wi-Fi network.
* Select **Connect**.
* The ATOM Echo now joined your network. Select **Add to Home Assistant**.
4. This opens the **My** link to Home Assistant.
* If you have not used My Home Assistant before, you will need to configure it. If your Home Assistant URL is not accessible on `http://homeassistant.local:8123`, replace it with the URL to your Home Assistant instance.
* Open the link.
![Open My link](/images/assist/esp32-atom-flash-06.png)
5. Select **OK**.
![Set up ESPHome](/images/assist/esp32-atom-flash-07.png)
6. To add the newly discovered device, select the ATOM Echo from the list.
* Add your ATOM Echo to a room and select **Finish**.
7. You should now see a new **M5Stack Atom Echo** integration.
![ATOM Echo discovered](/images/assist/m5stack-atom-echo-discovered-03.png)
* Your ATOM Echo is connected to Home Assistant over Wi-Fi. You can now move it to any place in your home with a USB power supply.
8. Congratulations! You can now voice control Home Assistant using a button with build-in microphone. Now give some commands.
## Controlling Home Assistant over the ATOM Echo
1. Press and hold the button on your ATOM Echo.
* The LED should light up in blue.
1. Say a [supported voice command](/docs/assist/builtin_sentences/). For example, *Turn off the light in the kitchen*.
* Make sure youre using the area name exactly as you defined it in Home Assistant.
* You can also ask a question, such as
* *Is the front door locked?*
* *Which lights are on in the living room?*
1. Let go of the button.
* The LED should light up in green.
* Home Assistant will confirm the action.
1. Your command is not supported? [Add your own commands](/integrations/conversation/).
## Troubleshooting
Are things not working as expected?
* Checkout the [general troubleshooting section for Assist](/projects/private-voice-assistant/troubleshooting/).