jeans/home-assistant.io
jeans/home-assistant.io
1
0
Fork 0
mirror of https://github.com/home-assistant/home-assistant.io.git synced 2025-07-19 15:26:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update TensorFlow docs (#14114)

Browse Source 
Co-authored-by: Franck Nijhof <git@frenck.dev>
This commit is contained in:
Jason Hunter 2020-08-11 06:35:50 -04:00 committed by Franck Nijhof
parent 4c291fd029
commit 73a35d7622
No known key found for this signature in database
GPG Key ID: D62583BA8AB11CA3
1 changed files with 49 additions and 23 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

72
source/_integrations/tensorflow.markdown
View File

@ -10,43 +10,64 @@ ha_domain: tensorflow
The `tensorflow` image processing platform allows you to detect and recognize objects in a camera image using [TensorFlow](https://www.tensorflow.org/). The state of the entity is the number of objects detected, and recognized objects are listed in the `summary` attribute along with quantity. The `matches` attribute provides the confidence `score` for recognition and the bounding `box` of the object for each detection category. The `tensorflow` image processing platform allows you to detect and recognize objects in a camera image using [TensorFlow](https://www.tensorflow.org/). The state of the entity is the number of objects detected, and recognized objects are listed in the `summary` attribute along with quantity. The `matches` attribute provides the confidence `score` for recognition and the bounding `box` of the object for each detection category.
<div class='note warning'> ## Home Assistant Core
The following packages must be installed on Raspbian before following the setup for the integration to work: If you are using the Home Assistant Core installation type, some additional requirements and steps apply.
`sudo apt-get install libatlas-base-dev libopenjp2-7 libtiff5` For all other installation types, this section can be skipped.
</div> The following packages must be installed on Debian before following the setup for the integration to work:
`sudo apt-get install libatlas-base-dev libopenjp2-7 libtiff5`
## Setup It is possible that Home Assistant is unable to install the Python TensorFlow bindings. If that is the case,
you'll need to install those manually using: `pip install tensorflow==2.2.0`, as the Python wheel is
not available for all platforms.
You need to install the `tensorflow` Python packages with: `$ pip3 install tensorflow==1.13.2`. The wheel is not available for all platforms. See [the official install guide](https://www.tensorflow.org/install/) for other options. The required packages are included in Home Assistant Supervised installations but only supported on amd64 architecture. See [the official install guide](https://www.tensorflow.org/install/) for other options.
This integration requires files to be downloaded, compiled on your computer, and added to the Home Assistant configuration directory. These steps can be performed using the sample script at [this gist](https://gist.github.com/hunterjm/6f9332f92b60c3d5e448ad936d7353c3). Alternatively, if you wish to perform the process manually, the process is as follows: Furthermore, the official Python TensorFlow wheels by Google, require your CPU to support the `avx` extension.
If your CPU lacks those capabilities, Home Assistant will crash when using TensorFlow, without any message.
- Clone [tensorflow/models](https://github.com/tensorflow/models/tree/master/research/object_detection) ## Preparation
- Compile protobuf models located in `research/object_detection/protos` with `protoc`
- Create the following directory structure inside your configuration directory: This integration requires files to be downloaded, compiled on your computer, and added to the Home Assistant configuration directory. These steps can be performed by cloning [this repository](https://github.com/hunterjm/hass-tensorflow) into your configuration directory. Alternatively, if you wish to perform the process manually, the process is as follows:
Create the following folder structure in your configuration directory.
```bash ```bash
|- {config_dir} |- {config_dir}
| - tensorflow/ |- tensorflow/
|- object_detection/ |- models/
|- __init__.py
``` ```
- Copy required object_detection dependencies to the `object_detection` folder inside of the `tensorflow` folder: Follow these steps (Linux) to compile the object detection library.
- `research/object_detection/data` ```bash
- `research/object_detection/utils` # Clone tensorflow/models
- `research/object_detection/protos` git clone https://github.com/tensorflow/models.git
# Compile Protobuf (apt-get install protobuf-compiler)
cd models/research
protoc object_detection/protos/*.proto --python_out=.
# Copy object_detection to {config_dir}
cp -r object_detection {config_dir}/tensorflow
```
Your final folder structure should look as follows
```bash
|- {config_dir}
|- tensorflow/
|- models/
|- object_detection/
|- ...
```
## Model Selection ## Model Selection
Lastly, it is time to pick a model. It is recommended to start with one of the COCO models available in the [Model Detection Zoo](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md). Lastly, it is time to pick a model. It is recommended to start with one of the COCO models available in the [Model Detection Zoo](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md).
The trade-off between the different models is accuracy vs speed. Users with a decent CPU should start with the `faster_rcnn_inception_v2_coco` model. If you are running on an ARM device like a Raspberry Pi, start with the `ssd_mobilenet_v2_coco` model. The trade-off between the different models is accuracy vs speed. Users with a decent CPU should start with one of the `EfficientDet` models. If you are running on an ARM device like a Raspberry Pi, start with the `SSD MobileNet v2 320x320` model.
Whichever model you choose, download it and place the `frozen_inference_graph.pb` file in the `tensorflow` folder in your configuration directory. Whichever model you choose, download it and extract in to the `tensorflow/models` folder in your configuration directory.
## Configuration ## Configuration
@ -59,7 +80,7 @@ image_processing:
source: source:
- entity_id: camera.local_file - entity_id: camera.local_file
model: model:
graph: /home/homeassistant/.homeassistant/tensorflow/frozen_inference_graph.pb graph: /config/tensorflow/models/efficientdet_d0_coco17_tpu-32/
``` ```
{% configuration %} {% configuration %}
@ -86,7 +107,7 @@ model:
type: map type: map
keys: keys:
graph: graph:
description: Full path to `frozen_inference_graph.pb`. description: Full path to the base model directory.
required: true required: true
type: string type: string
labels: labels:
@ -94,6 +115,11 @@ model:
required: false required: false
type: string type: string
default: tensorflow/object_detection/data/mscoco_label_map.pbtxt default: tensorflow/object_detection/data/mscoco_label_map.pbtxt
label_offset:
description: Offset for mapping label ID to a name (only use for custom models)
required: false
type: integer
default: 1
model_dir: model_dir:
description: Full path to TensorFlow models directory. description: Full path to TensorFlow models directory.
required: false required: false
@ -143,7 +169,7 @@ image_processing:
- "/tmp/{% raw %}{{ camera_entity.split('.')[1] }}{% endraw %}_latest.jpg" - "/tmp/{% raw %}{{ camera_entity.split('.')[1] }}{% endraw %}_latest.jpg"
- "/tmp/{% raw %}{{ camera_entity.split('.')[1] }}_{{ now().strftime('%Y%m%d_%H%M%S') }}{% endraw %}.jpg" - "/tmp/{% raw %}{{ camera_entity.split('.')[1] }}_{{ now().strftime('%Y%m%d_%H%M%S') }}{% endraw %}.jpg"
model: model:
graph: /home/homeassistant/.homeassistant/tensorflow/frozen_inference_graph.pb graph: /config/tensorflow/models/efficientdet_d0_coco17_tpu-32/
categories: categories:
- category: person - category: person
area: area: