diff --git a/tools/decode-config.md b/tools/decode-config.md
index 4097638fa..10019a23f 100644
--- a/tools/decode-config.md
+++ b/tools/decode-config.md
@@ -1,407 +1,3 @@
-# decode-config.py
-_decode-config.py_ is able to backup and restore Tasmota configuration.
+A tool to backup and restore the configuration of [Tasmota](http://tasmota.com/)-devices.
-In comparison with the Tasmota build-in "Backup/Restore Configuration" function _decode-config.py_
-* uses human readable and editable [JSON](http://www.json.org/)-format for backup/restore,
-* can restore previously backup and changed [JSON](http://www.json.org/)-format files,
-* is able to create Tasmota compatible command list with related config parameter
-
-Comparing backup files created by *decode-config.py* and *.dmp files created by Tasmota "Backup/Restore Configuration":
-
-| | decode-config.py
*.json file | Tasmota
*.dmp file |
-|-------------------------|:-------------------------------:|:-----------------------------------:|
-| Encrypted | No | Yes |
-| Readable | Yes | No |
-| Simply editable | Yes | No |
-| Simply batch processing | Yes | No |
-
-_decode-config.py_ is compatible with Tasmota version from v5.10.0 up to now.
-
-# Content
-* [Prerequisite](decode-config.md#prerequisite)
-* [File Types](decode-config.md#file-types)
- * [.dmp File Format](decode-config.md#-dmp-format)
- * [.json File Format](decode-config.md#-json-format)
- * [.bin File Format](decode-config.md#-bin-format)
- * [File extensions](decode-config.md#file-extensions)
-* [Usage](decode-config.md#usage)
- * [Basics](decode-config.md#basics)
- * [Save backup file](decode-config.md#save-backup-file)
- * [Restore backup file](decode-config.md#restore-backup-file)
- * [Output to screen](decode-config.md#output-to-screen)
- * [JSON output](decode-config.md#json-output)
- * [Tasmota command output](decode-config.md#tasmota-command-output)
- * [Filter data](decode-config.md#filter-data)
- * [Configuration file](decode-config.md#configuration-file)
- * [More program arguments](decode-config.md#more-program-arguments)
- * [Examples](decode-config.md#examples)
- * [Config file](decode-config.md#config-file)
- * [Using Tasmota binary configuration files](decode-config.md#using-tasmota-binary-configuration-files)
- * [Use batch processing](decode-config.md#use-batch-processing)
- * [Notes](decode-config.md#notes)
-
-## Prerequisite
-* This program is written in [Python](https://en.wikipedia.org/wiki/Python_(programming_language)) so you need to install a working python environment for your operating system.
-
-### Linux
-```
-sudo apt-get install python python-pip libcurl4-openssl-dev libssl-dev
-```
-```
-pip install pycurl configargparse
-```
-
-### Windows 10
-
-Install [Python 2.7](https://www.python.org/download/releases/2.7/) then install dependencies. For PyCurl you need to [download pycurl‑7.43.0.3‑cp27‑cp27m‑win_amd64.whl](https://www.lfd.uci.edu/~gohlke/pythonlibs/#pycurl) for Windows 10 64bit.
-```
-pip install pycurl-7.43.0.3-cp27-cp27m-win_amd64.whl
-// run the command from the folder where you downloaded the file
-
-pip install configargparse
-```
-
-* [Tasmota](https://github.com/arendst/Tasmota) [Firmware](https://github.com/arendst/Tasmota/releases) with Web-Server enabled:
- * To backup or restore configurations from or to a Tasmota device you need a firmare with enabled web-server in admin mode (command [WebServer 2](https://tasmota.github.io/docs/#/Commands#wifi)). This is the Tasmota default.
- * If using your own compiled firmware be aware to enable the web-server (`#define USE_WEBSERVER` and `#define WEB_SERVER 2`).
-
-## File Types
-_decode-config.py_ can handle the following backup file types:
-### .dmp Format
-Configuration data as used by Tasmota "Backup/Restore Configuration" web interface.
-This format is binary and encrypted.
-### .json Format
-Configuration data in [JSON](http://www.json.org/)-format.
-This format is decrypted, human readable and editable and can also be used for the `--restore-file` parameter.
-This file will be created by _decode-config.py_ using the `--backup-file` with `--backup-type json` parameter, this is the default.
-### .bin Format
-Configuration data in binary format.
-This format is binary decryptet, editable (e.g. using a hex editor) and can also be used for `--restore-file` command.
-It will be created by _decode-config.py_ using `--backup-file` with `--backup-type bin`.
-Note:
-The .bin file contains the same information as the original .dmp file from Tasmota "Backup/Restore Configuration" but it is decrpted and 4 byte longer than an original (it is a prefix header at the beginning). .bin file data starting at address 4 contains the same as the **struct SYSCFG** from Tasmota [settings.h](https://github.com/arendst/Tasmota/blob/master/tasmota/settings.h) in decrypted format.
-
-#### File extensions
-You don't need to append exensions for your file name as _decode-config.py_ uses auto extension as default. The extension will be choose based on file contents and `--backup-type` parameter.
-If you do not want using auto extensions use the `--no-extension` parameter.
-
-## Usage
-After download don't forget to set the executable flag under linux with `chmod +x decode-config.py` or call the program using `python decode-config.py...`.
-
-### Basics
-At least pass a source where you want to read the configuration data from using `-f ` or `-d `:
-
-The source can be either
-* a Tasmota device hostname or IP using the `-d ` parameter
-* a Tasmota `*.dmp` configuration file using `-f ` parameter
-
-Example:
-
- decode-config.py -d tasmota-4281
-
-will output a human readable configuration in [JSON](http://www.json.org/)-format:
-
- {
- "altitude": 112,
- "baudrate": 115200,
- "blinkcount": 10,
- "blinktime": 10,
- ...
- "ws_width": [
- 1,
- 3,
- 5
- ]
- }
-
-
-### Save backup file
-To save the output as backup file use `--backup-file `, you can use placeholder for Version, Friendlyname and Hostname:
-
- decode-config.py -d tasmota-4281 --backup-file Config_@f_@v
-
-If you have setup a WebPassword within Tasmota, use
-
- decode-config.py -d tasmota-4281 -p --backup-file Config_@f_@v
-
-will create a file like `Config_Tasmota_6.4.0.json` (the part `Tasmota` and `6.4.0` will choosen related to your device configuration). Because the default backup file format is JSON, you can read and change it with any raw text editor.
-
-### Restore backup file
-Reading back a saved (and possible changed) backup file use the `--restore-file ` parameter. This will read the (changed) configuration data from this file and send it back to the source device or filename.
-
-To restore the previously save backup file `Config_Tasmota_6.2.1.json` to device `tasmota-4281` use:
-
- decode-config.py -d tasmota-4281 --restore-file Config_Tasmota_6.2.1.json
-
-with password set by WebPassword:
-
- decode-config.py -d tasmota-4281 -p --restore-file Config_Tasmota_6.2.1.json
-
-### Output to screen
-To force screen output use the `--output` parameter.
-
-Output to screen is default enabled when calling the program with a source parameter (-f or -d) but without any backup or restore parameter.
-
-#### JSON output
-The default output format is [JSON](decode-config.md#-json-format). You can force JSON output using the `--output-format json` parameter.
-
-Example:
-
- decode-config.py -d tasmota-4281 -c my.conf -x Wifi --output-format json
-
- {
- ...
- "hostname": "%s-%04d",
- "ip_address": [
- "0.0.0.0",
- "192.168.12.1",
- "255.255.255.0",
- "192.168.12.1"
- ],
- "ntp_server": [
- "ntp.localnet.home",
- "ntp2.localnet.home",
- "192.168.12.1"
- ],
- "sta_active": 0,
- "sta_config": 5,
- "sta_pwd": [
- "myWlAnPaszxwo!z",
- "myWlAnPaszxwo!z2"
- ],
- "sta_ssid": [
- "wlan.1",
- "my-wlan"
- ],
- "web_password": "myPaszxwo!z",
- "webserver": 2
- ...
- }
-
-Note: JSON output always contains all configuration data like the backup file except you are using `--group` arg.
-
-
-#### Tasmota command output
-_decode-config.py_ is able to translate the configuration data to (most all) Tasmota commands. To output your configuration as Tasmota commands use `--output-format cmnd` or `--output-format command`.
-
-Example:
-
- decode-config.py -d tasmota-4281 -c my.conf -g Wifi --output-format cmnd
-
- # Wifi:
- AP 0
- Hostname %s-%04d
- IPAddress1 0.0.0.0
- IPAddress2 192.168.12.1
- IPAddress3 255.255.255.0
- IPAddress4 192.168.12.1
- NtpServer1 ntp.localnet.home
- NtpServer2 ntp2.localnet.home
- NtpServer3 192.168.12.1
- Password1 myWlAnPaszxwo!z
- Password2 myWlAnPaszxwo!z2
- SSId1 wlan.1
- SSId2 wlan.1
- WebPassword myPaszxwo!z
- WebServer 2
- WifiConfig 5
-
-Note: A few very specific module commands like MPC230xx, KNX and some Display commands are not supported. These are still available by JSON output.
-
-### Filter data
-The huge number of Tasmota configuration data can be overstrained and confusing, so the most of the configuration data are grouped into categories.
-
-With _decode-config.py_ the following categories are available: `Display`, `Domoticz`, `Internal`, `KNX`, `Led`, `Logging`, `MCP230xx`, `MQTT`, `Main`, `Management`, `Pow`, `Sensor`, `Serial`, `SetOption`, `RF`, `System`, `Timers`, `Wifi`
-
-These are similary to the categories on [https://tasmota.github.io/docs/#/Commands](Tasmota Command Wiki).
-
-To filter outputs to a subset of groups use the `-g` or `--group` arg concatenating the grooup you want, e. g.
-
- decode-config.py -d tasmota-4281 -c my.conf --output-format cmnd --group Main MQTT Management Wifi
-
-
-### Configuration file
-Each argument that start with `--` (eg. `--file`) can also be set in a config file (specified via -c). Config file syntax allows: key=value, flag=true, stuff=[a,b,c] (for details, see syntax at [https://pypi.org/project/ConfigArgParse](https://pypi.org/project/ConfigArgParse/)).
-
-If an argument is specified in more than one place, then commandline values override config file values which override defaults. This is usefull if you always use the same argument or a basic set of arguments.
-
-The http authentication credentials `--username` and `--password` is predestinated to store it in a file instead using it on your command line as argument:
-
-e.g. my.conf:
-
- [source]
- username = admin
- password = myPaszxwo!z
-
-To make a backup file from example above you can now pass the config file instead using the password on command line:
-
- decode-config.py -d tasmota-4281 -c my.conf --backup-file Config_@f_@v
-
-
-
-### More program arguments
-For better reading each short written arg (minus sign `-`) has a corresponding long version (two minus signs `--`), eg. `--device` for `-d` or `--file` for `-f` (note: not even all `--` arg has a corresponding `-` one).
-
-A short list of possible program args is displayed using `-h` or `--help`.
-
-For advanced help use `-H` or `--full-help`:
-
- usage: decode-config.py [-f ] [-d ] [-P ]
- [-u ] [-p ] [-i ]
- [-o ] [-t json|bin|dmp] [-E] [-e] [-F]
- [--json-indent ] [--json-compact]
- [--json-hide-pw] [--json-show-pw]
- [--cmnd-indent ] [--cmnd-groups]
- [--cmnd-nogroups] [--cmnd-sort] [--cmnd-unsort]
- [-c ] [-S] [-T json|cmnd|command]
- [-g {Control,Devices,Display,Domoticz,Internal,Knx,Light,Management,Mqtt,Power,Rf,Rules,Sensor,Serial,Setoption,Shutter,System,Timer,Wifi} [{Control,Devices,Display,Domoticz,Internal,Knx,Light,Management,Mqtt,Power,Rf,Rules,Sensor,Serial,Setoption,Shutter,System,Timer,Wifi} ...]]
- [--ignore-warnings] [-h] [-H] [-v] [-V]
-
- Backup/Restore Tasmota configuration data. Args that start with '--'
- (eg. -f) can also be set in a config file (specified via -c). Config file
- syntax allows: key=value, flag=true, stuff=[a,b,c] (for details, see syntax at
- https://goo.gl/R74nmi). If an arg is specified in more than one place, then
- commandline values override config file values which override defaults.
-
- Source:
- Read/Write Tasmota configuration from/to
-
- -f, --file, --tasmota-file
- file to retrieve/write Tasmota configuration from/to
- (default: None)'
- -d, --device, --host
- hostname or IP address to retrieve/send Tasmota
- configuration from/to (default: None)
- -P, --port TCP/IP port number to use for the host connection
- (default: 80)
- -u, --username
- host HTTP access username (default: admin)
- -p, --password
- host HTTP access password (default: None)
-
- Backup/Restore:
- Backup & restore specification
-
- -i, --restore-file
- file to restore configuration from (default: None).
- Replacements: @v=firmware version from config,
- @f=device friendly name from config, @h=device
- hostname from config, @H=device hostname from device
- (-d arg only)
- -o, --backup-file
- file to backup configuration to (default: None).
- Replacements: @v=firmware version from config,
- @f=device friendly name from config, @h=device
- hostname from config, @H=device hostname from device
- (-d arg only)
- -t, --backup-type json|bin|dmp
- backup filetype (default: 'json')
- -E, --extension append filetype extension for -i and -o filename
- (default)
- -e, --no-extension do not append filetype extension, use -i and -o
- filename as passed
- -F, --force-restore force restore even configuration is identical
-
- JSON output:
- JSON format specification
-
- --json-indent
- pretty-printed JSON output using indent level
- (default: 'None'). -1 disables indent.
- --json-compact compact JSON output by eliminate whitespace
- --json-hide-pw hide passwords
- --json-show-pw, --json-unhide-pw
- unhide passwords (default)
-
- Tasmota command output:
- Tasmota command output format specification
-
- --cmnd-indent
- Tasmota command grouping indent level (default: '2').
- 0 disables indent
- --cmnd-groups group Tasmota commands (default)
- --cmnd-nogroups leave Tasmota commands ungrouped
- --cmnd-sort sort Tasmota commands (default)
- --cmnd-unsort leave Tasmota commands unsorted
-
- Common:
- Optional arguments
-
- -c, --config
- program config file - can be used to set default
- command args (default: None)
- -S, --output display output regardsless of backup/restore usage
- (default do not output on backup or restore usage)
- -T, --output-format json|cmnd|command
- display output format (default: 'json')
- -g, --group {Control,Devices,Display,Domoticz,Internal,Knx,Light,Management,Mqtt,Power,Rf,Rules,Sensor,Serial,Setoption,Shutter,System,Timer,Wifi}
- limit data processing to command groups (default no
- filter)
- --ignore-warnings do not exit on warnings. Not recommended, used by your
- own responsibility!
-
- Info:
- Extra information
-
- -h, --help show usage help message and exit
- -H, --full-help show full help message and exit
- -v, --verbose produce more output about what the program does
- -V, --version show program's version number and exit
-
- Either argument -d or -f must be given.
-
-### Program parameter notes
-
-_decode-config.py_
-
-
-### Examples
-The most of the examples are for linux command line. Under Windows call the program using `python decode-config.py ...`.
-
-#### Config file
-Note: The example contains .ini style sections `[...]`. Sections are always treated as comment and serves as clarity only.
-For further details of config file syntax see [https://pypi.org/project/ConfigArgParse](https://pypi.org/project/ConfigArgParse/).
-
-*my.conf*
-
- [Source]
- username = admin
- password = myPaszxwo!z
-
- [JSON]
- json-indent 2
-
-#### Using Tasmota binary configuration files
-
-1. Restore a Tasmota configuration file
-
- `decode-config.py -c my.conf -d tasmota --restore-file Config_Tasmota_6.2.1.dmp`
-
-2. Backup device using Tasmota configuration compatible format
-
- a) use file extension to choice the file format
-
- `decode-config.py -c my.conf -d tasmota --backup-file Config_@f_@v.dmp`
-
- b) use args to choice the file format
-
- `decode-config.py -c my.conf -d tasmota --backup-type dmp --backup-file Config_@f_@v`
-
-#### Use batch processing
-
- for device in tasmota1 tasmota2 tasmota3; do ./decode-config.py -c my.conf -d $device -o Config_@f_@v
-
-or under windows
-
- for device in (tasmota1 tasmota2 tasmota3) do python decode-config.py -c my.conf -d %device -o Config_@f_@v
-
-will produce JSON configuration files for host tasmota1, tasmota2 and tasmota3 using friendly name and Tasmota firmware version for backup filenames.
-
-## Notes
-Some general notes:
-* Filename replacement macros **@h** and **@H**:
- * **@h**
-The **@h** replacement macro uses the hostname configured with the Tasomta Wifi `Hostname ` command (defaults to `%s-%04d`). It will not use the network hostname of your device because this is not available when working with files only (e.g. `--file ` as source).
-To prevent having a useless % in your filename, **@h** will not replaced by configuration data hostname if this contains '%' characters.
- * **@H**
-If you want to use the network hostname within your filename, use the **@H** replacement macro instead - but be aware this will only replaced if you are using a network device as source (`-d`, `--device`, `--host`); it will not work when using a file as source (`-f`, `--file`)
+## decode-config has moved to [https://github.com/tasmota/decode-config](https://github.com/tasmota/decode-config)
diff --git a/tools/decode-config.py b/tools/decode-config.py
deleted file mode 100755
index f15c696ae..000000000
--- a/tools/decode-config.py
+++ /dev/null
@@ -1,3322 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-from __future__ import print_function
-from past.builtins import long
-VER = '2.4.0039'
-
-"""
- decode-config.py - Backup/Restore Tasmota configuration data
-
- Copyright (C) 2019 Norbert Richter
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see .
-
-
-Requirements:
- - Python 2.x:
- pip install json requests urllib2 configargparse
-
-
-Instructions:
- Execute command with option -d to retrieve config data from a host
- or use -f to read a configuration file saved using Tasmota Web-UI
-
- For further information read 'decode-config.md'
-
- For help execute command with argument -h (or -H for advanced help)
-
-
-Usage: decode-config.py [-f ] [-d ] [-P ]
- [-u ] [-p ] [-i ]
- [-o ] [-t json|bin|dmp] [-E] [-e] [-F]
- [--json-indent ] [--json-compact]
- [--json-hide-pw] [--json-show-pw]
- [--cmnd-indent ] [--cmnd-groups]
- [--cmnd-nogroups] [--cmnd-sort] [--cmnd-unsort]
- [-c ] [-S] [-T json|cmnd|command]
- [-g {Control,Devices,Display,Domoticz,Internal,Knx,Light,Management,Mqtt,Power,Rules,Sensor,Serial,Setoption,Shutter,Rf,System,Timer,Wifi} [{Control,Devices,Display,Domoticz,Internal,Knx,Light,Management,Mqtt,Power,Rules,Sensor,Serial,Setoption,Shutter,Rf,System,Timer,Wifi} ...]]
- [--ignore-warnings] [-h] [-H] [-v] [-V]
-
- Backup/Restore Tasmota configuration data. Args that start with '--'
- (eg. -f) can also be set in a config file (specified via -c). Config file
- syntax allows: key=value, flag=true, stuff=[a,b,c] (for details, see syntax at
- https://goo.gl/R74nmi). If an arg is specified in more than one place, then
- commandline values override config file values which override defaults.
-
- Source:
- Read/Write Tasmota configuration from/to
-
- -f, --file, --tasmota-file
- file to retrieve/write Tasmota configuration from/to
- (default: None)'
- -d, --device, --host
- hostname or IP address to retrieve/send Tasmota
- configuration from/to (default: None)
- -P, --port TCP/IP port number to use for the host connection
- (default: 80)
- -u, --username
- host HTTP access username (default: admin)
- -p, --password
- host HTTP access password (default: None)
-
- Backup/Restore:
- Backup & restore specification
-
- -i, --restore-file
- file to restore configuration from (default: None).
- Replacements: @v=firmware version from config,
- @f=device friendly name from config, @h=device
- hostname from config, @H=device hostname from device
- (-d arg only)
- -o, --backup-file
- file to backup configuration to (default: None).
- Replacements: @v=firmware version from config,
- @f=device friendly name from config, @h=device
- hostname from config, @H=device hostname from device
- (-d arg only)
- -t, --backup-type json|bin|dmp
- backup filetype (default: 'json')
- -E, --extension append filetype extension for -i and -o filename
- (default)
- -e, --no-extension do not append filetype extension, use -i and -o
- filename as passed
- -F, --force-restore force restore even configuration is identical
-
- JSON output:
- JSON format specification
-
- --json-indent
- pretty-printed JSON output using indent level
- (default: 'None'). -1 disables indent.
- --json-compact compact JSON output by eliminate whitespace
- --json-hide-pw hide passwords
- --json-show-pw, --json-unhide-pw
- unhide passwords (default)
-
- Tasmota command output:
- Tasmota command output format specification
-
- --cmnd-indent
- Tasmota command grouping indent level (default: '2').
- 0 disables indent
- --cmnd-groups group Tasmota commands (default)
- --cmnd-nogroups leave Tasmota commands ungrouped
- --cmnd-sort sort Tasmota commands (default)
- --cmnd-unsort leave Tasmota commands unsorted
-
- Common:
- Optional arguments
-
- -c, --config
- program config file - can be used to set default
- command args (default: None)
- -S, --output display output regardsless of backup/restore usage
- (default do not output on backup or restore usage)
- -T, --output-format json|cmnd|command
- display output format (default: 'json')
- -g, --group {Control,Devices,Display,Domoticz,Internal,Knx,Light,Management,Mqtt,Power,Rules,Sensor,Serial,Setoption,Shutter,Rf,System,Timer,Wifi}
- limit data processing to command groups (default no
- filter)
- --ignore-warnings do not exit on warnings. Not recommended, used by your
- own responsibility!
-
- Info:
- Extra information
-
- -h, --help show usage help message and exit
- -H, --full-help show full help message and exit
- -v, --verbose produce more output about what the program does
- -V, --version show program's version number and exit
-
- Either argument -d or -f must be given.
-
-
-Returns:
- 0: successful
- 1: restore skipped
- 2: program argument error
- 3: file not found
- 4: data size mismatch
- 5: data CRC error
- 6: unsupported configuration version
- 7: configuration file read error
- 8: JSON file decoding error
- 9: Restore file data error
- 10: Device data download error
- 11: Device data upload error
- 20: python module missing
- 21: Internal error
- >21: python library exit code
- 4xx, 5xx: HTTP errors
-
-"""
-
-class ExitCode:
- OK = 0
- RESTORE_SKIPPED = 1
- ARGUMENT_ERROR = 2
- FILE_NOT_FOUND = 3
- DATA_SIZE_MISMATCH = 4
- DATA_CRC_ERROR = 5
- UNSUPPORTED_VERSION = 6
- FILE_READ_ERROR = 7
- JSON_READ_ERROR = 8
- RESTORE_DATA_ERROR = 9
- DOWNLOAD_CONFIG_ERROR = 10
- UPLOAD_CONFIG_ERROR = 11
- MODULE_NOT_FOUND = 20
- INTERNAL_ERROR = 21
-
-# ======================================================================
-# imports
-# ======================================================================
-import os.path
-import io
-import sys, platform
-def ModuleImportError(module):
- er = str(module)
- print('{}, try "pip install {}"'.format(er,er.split(' ')[len(er.split(' '))-1]), file=sys.stderr)
- sys.exit(ExitCode.MODULE_NOT_FOUND)
-try:
- from datetime import datetime
- import time
- import copy
- import struct
- import socket
- import re
- import math
- import inspect
- import json
- import configargparse
- import requests
- if sys.version_info.major==2:
- import urllib2
- else:
- import urllib
-except ImportError as e:
- ModuleImportError(e)
-
-# ======================================================================
-# globals
-# ======================================================================
-PROG='{} v{} by Norbert Richter '.format(os.path.basename(sys.argv[0]),VER)
-
-CONFIG_FILE_XOR = 0x5A
-BINARYFILE_MAGIC = 0x63576223
-STR_ENCODING = 'utf8'
-HIDDEN_PASSWORD = '********'
-INTERNAL = 'Internal'
-
-DEFAULTS = {
- 'source':
- {
- 'device': None,
- 'port': 80,
- 'username': 'admin',
- 'password': None,
- 'tasmotafile': None,
- },
- 'backup':
- {
- 'restorefile': None,
- 'backupfile': None,
- 'backupfileformat': 'json',
- 'extension': True,
- 'forcerestore': False,
- },
- 'jsonformat':
- {
- 'jsonindent': None,
- 'jsoncompact': False,
- 'jsonsort': True,
- 'jsonhidepw': False,
- },
- 'cmndformat':
- {
- 'cmndindent': 2,
- 'cmndgroup': True,
- 'cmndsort': True,
- },
- 'common':
- {
- 'output': False,
- 'outputformat': 'json',
- 'configfile': None,
- 'ignorewarning':False,
- 'filter': None,
- },
-}
-args = {}
-exitcode = 0
-
-
-# ======================================================================
-# Settings mapping
-# ======================================================================
-"""
-Settings dictionary describes the config file fields definition:
-
- = { : }
-
- : "string"
- a python valid dictionary key (string)
-
- : ( , , [,] )
- a tuple containing the following items:
-
- : |
- data type & format definition
- :
- defines the use of data at
- format is defined in 'struct module format string'
- see
- https://docs.python.org/3.8/library/struct.html#format-strings
- :
- A dictionary describes a (sub)setting dictonary
- and can recursively define another
-
- : | (, , )
- address definition
- :
- The address (starting from 0) within binary config data.
- :
- number of bits used (positive integer)
- :
- bit shift :
- >= 0: shift the result right
- < 0: shift the result left
-
- : | (, [,cmd])
- data definition
- : None | | [] | [ ,...]
- None:
- Single value, not an array
- :
- []
- Defines a one-dimensional array of size
- [ ,...]
- Defines a one- or multi-dimensional array
- :
- value validation function
- : (, )
- Tasmota command definition
- :
- command group string
- : | (,...)
- convert data into Tasmota command function
-
- : | (, )
- read/write converter
- : None |
- Will be used in Bin2Mapping to convert values read
- from the binary data object into mapping dictionary
- None
- None indicates not read conversion
-
- to convert value from binary object to JSON.
- : None | False |
- Will be used in Mapping2Bin to convert values read
- from mapping dictionary before write to binary
- data object
- None
- None indicates not write conversion
- False
- False indicates the value is readonly and will
- not be written into the binary object.
-
- to convert value from JSON back to binary object
-
- Common definitions
-
- : | | None
- function to be called or string to evaluate:
- :
- A function name will be called with one or two parameter:
- The value to be processed
- (optional) the current array index (1,n)
-
- A string will be evaluate as is. The following
- placeholder can be used to replace it by runtime values:
- '$':
- will be replaced by the mapping name value
- '#':
- will be replace by array index (if any)
- '@':
- can be used as reference to other mapping values
- see definition below for examples
-
- : 'string' | "string"
- characters enclosed in ' or "
-
- : integer
- numbers in the range -2147483648 through 2147483647
- : unsigned integer
- numbers in the range 0 through 4294967295
-
-"""
-# ----------------------------------------------------------------------
-# Settings helper
-# ----------------------------------------------------------------------
-def passwordread(value):
- return HIDDEN_PASSWORD if args.jsonhidepw else value
-def passwordwrite(value):
- return None if value == HIDDEN_PASSWORD else value
-def bitsRead(x, n=0, c=1):
- """
- Reads bit(s) of a number
-
- @param x:
- the number from which to read
-
- @param n:
- which bit position to read
-
- @param c:
- how many bits to read (1 if omitted)
-
- @return:
- the bit value(s)
- """
- if isinstance(x,str):
- x = int(x, 0)
- if isinstance(x,str):
- n = int(n, 0)
-
- if n >= 0:
- x >>= n
- else:
- x <<= abs(n)
- if c>0:
- x &= (1<, , [,]
- 'cfg_holder': ('0 and bitsRead($,0,11)>(12*60) else "",time=time.strftime("%H:%M",time.gmtime((bitsRead($,0,11) if bitsRead($,29,2)==0 else bitsRead($,0,11) if bitsRead($,0,11)<=(12*60) else bitsRead($,0,11)-(12*60))*60)),window=bitsRead($,11,4),repeat=bitsRead($,15),days="{:07b}".format(bitsRead($,16,7))[::-1],device=bitsRead($,23,4)+1,power=bitsRead($,27,2) )')), ('"0x{:08x}".format($)', False) ),
- 'time': (' 0 and type_ is not None else '',
- sstatus=status if status is not None and status > 0 else '',
- scolon=': ' if type_ is not None or line is not None else '',
- smgs=msg,
- slineno=' (@{:04d})'.format(line) if line is not None else '')
- , file=sys.stderr)
-
-
-def exit(status=0, msg="end", type_=LogType.ERROR, src=None, doexit=True, line=None):
- """
- Called when the program should be exit
-
- @param status:
- the exit status program returns to callert
- @param msg:
- the msg logged before exit
- @param type_:
- msg type: 'INFO', 'WARNING' or 'ERROR'
- @param doexit:
- True to exit program, otherwise return
- """
-
- if src is not None:
- msg = '{} ({})'.format(src, msg)
- message(msg, type_=type_ if status!=ExitCode.OK else LogType.INFO, status=status, line=line)
- exitcode = status
- if doexit:
- sys.exit(exitcode)
-
-
-def debug(args):
- """
- Get debug level
-
- @param args:
- configargparse.parse_args() result
-
- @return:
- debug level
- """
- return 0 if args.debug is None else args.debug
-
-
-def instance(type_):
- """
- Creates Python2/3 compatible isinstance test type(s)
-
- @param args:
- Python3 instance type
-
- @return:
- Python2/3 compatible isinstance type(s)
- """
- newtype = type_
- if sys.version_info.major==2:
- if type_==str:
- newtype = (str,unicode)
- elif isinstance(type_, tuple) and str in type_:
- newtype = newtype + (unicode,)
- return newtype
-
-
-def ShortHelp(doexit=True):
- """
- Show short help (usage) only - ued by own -h handling
-
- @param doexit:
- sys.exit with OK if True
- """
- print(parser.description)
- print
- parser.print_usage()
- print
- print("For advanced help use '{prog} -H' or '{prog} --full-help'".format(prog=os.path.basename(sys.argv[0])))
- if doexit:
- sys.exit(ExitCode.OK)
-
-
-class CustomHelpFormatter(configargparse.HelpFormatter):
- """
- Class for customizing the help output
- """
-
- def _format_action_invocation(self, action):
- """
- Reformat multiple metavar output
- -d , --device , --host
- to single output
- -d, --device, --host
- """
-
- orgstr = configargparse.HelpFormatter._format_action_invocation(self, action)
- if orgstr and orgstr[0] != '-': # only optional arguments
- return orgstr
- res = getattr(action, '_formatted_action_invocation', None)
- if res:
- return res
-
- options = orgstr.split(', ')
- if len(options) <= 1:
- action._formatted_action_invocation = orgstr
- return orgstr
-
- return_list = []
- for option in options:
- meta = ""
- arg = option.split(' ')
- if len(arg) > 1:
- meta = arg[1]
- return_list.append(arg[0])
- if len(meta) > 0 and len(return_list) > 0:
- return_list[len(return_list)-1] += " "+meta
- action._formatted_action_invocation = ', '.join(return_list)
- return action._formatted_action_invocation
-
-
-# ======================================================================
-# Tasmota config data handling
-# ======================================================================
-def GetTemplateSizes():
- """
- Get all possible template sizes as list
-
- @return:
- template sizes as list []
- """
- sizes = []
- for cfg in Settings:
- sizes.append(cfg[1])
- # return unique sizes only (remove duplicates)
- return list(set(sizes))
-
-
-def GetTemplateSetting(decode_cfg):
- """
- Search for version, size and settings to be used depending on given binary config data
-
- @param decode_cfg:
- binary config data (decrypted)
-
- @return:
- version, size, settings to use; None if version is invalid
- """
- version = 0x0
- size = setting = None
- version = GetField(decode_cfg, 'version', Setting_6_2_1['version'], raw=True)
- # search setting definition top-down
- for cfg in sorted(Settings, key=lambda s: s[0], reverse=True):
- if version >= cfg[0]:
- size = cfg[1]
- setting = cfg[2]
- break
-
- return version, size, setting
-
-
-def GetGroupList(setting):
- """
- Get all avilable group definition from setting
-
- @return:
- configargparse.parse_args() result
- """
- groups = set()
-
- for name in setting:
- dev = setting[name]
- format_, group = GetFieldDef(dev, fields="format_, group")
- if group is not None and len(group) > 0:
- groups.add(group.title())
- if isinstance(format_, dict):
- subgroups = GetGroupList(format_)
- if subgroups is not None and len(subgroups) > 0:
- for group in subgroups:
- groups.add(group.title())
-
- groups=list(groups)
- groups.sort()
- return groups
-
-
-class FileType:
- FILE_NOT_FOUND = None
- DMP = 'dmp'
- JSON = 'json'
- BIN = 'bin'
- UNKNOWN = 'unknown'
- INCOMPLETE_JSON = 'incomplete json'
- INVALID_JSON = 'invalid json'
- INVALID_BIN = 'invalid bin'
-
-def GetFileType(filename):
- """
- Get the FileType class member of a given filename
-
- @param filename:
- filename of the file to analyse
-
- @return:
- FileType class member
- """
- filetype = FileType.UNKNOWN
-
- # try filename
- try:
- isfile = os.path.isfile(filename)
- try:
- with open(filename, "r") as f:
- try:
- # try reading as json
- inputjson = json.load(f)
- if 'header' in inputjson:
- filetype = FileType.JSON
- else:
- filetype = FileType.INCOMPLETE_JSON
- except ValueError:
- filetype = FileType.INVALID_JSON
- # not a valid json, get filesize and compare it with all possible sizes
- try:
- size = os.path.getsize(filename)
- except:
- filetype = FileType.UNKNOWN
- sizes = GetTemplateSizes()
-
- # size is one of a dmp file size
- if size in sizes:
- filetype = FileType.DMP
- elif (size - ((len(hex(BINARYFILE_MAGIC))-2)/2)) in sizes:
- # check if the binary file has the magic header
- with open(filename, "rb") as inputfile:
- inputbin = inputfile.read()
- if struct.unpack_from('>24) & 0xff)
- minor = ((version>>16) & 0xff)
- release = ((version>> 8) & 0xff)
- subrelease = (version & 0xff)
- if major >= 6:
- if subrelease > 0:
- subreleasestr = str(subrelease)
- else:
- subreleasestr = ''
- else:
- if subrelease > 0:
- subreleasestr = str(chr(subrelease+ord('a')-1))
- else:
- subreleasestr = ''
- return "{:d}.{:d}.{:d}{}{}".format( major, minor, release, '.' if (major >= 6 and subreleasestr != '') else '', subreleasestr)
-
-
-def MakeFilename(filename, filetype, configmapping):
- """
- Replace variables within a filename
-
- @param filename:
- original filename possible containing replacements:
- @v:
- Tasmota version from config data
- @f:
- friendlyname from config data
- @h:
- hostname from config data
- @H:
- hostname from device (-d arg only)
- @param filetype:
- FileType.x object - creates extension if not None
- @param configmapping:
- binary config data (decrypted)
-
- @return:
- New filename with replacements
- """
- config_version = config_friendlyname = config_hostname = device_hostname = ''
-
- if 'version' in configmapping:
- config_version = GetVersionStr( int(str(configmapping['version']), 0) )
- if 'friendlyname' in configmapping:
- config_friendlyname = re.sub('[^0-9a-zA-Z]','_', configmapping['friendlyname'][0])
- if 'hostname' in configmapping:
- if configmapping['hostname'].find('%') < 0:
- config_hostname = re.sub('[^0-9a-zA-Z]','_', configmapping['hostname'])
- if filename.find('@H') >= 0 and args.device is not None:
- device_hostname = GetTasmotaHostname(args.device, args.port, username=args.username, password=args.password)
- if device_hostname is None:
- device_hostname = ''
-
- dirname = basename = ext = ''
-
- # split file parts
- dirname = os.path.normpath(os.path.dirname(filename))
- basename = os.path.basename(filename)
- name, ext = os.path.splitext(basename)
-
- # make a valid filename
- try:
- name = name.decode('unicode-escape').translate(dict((ord(char), None) for char in '\/*?:"<>|'))
- except:
- pass
- name = str(name.replace(' ','_'))
-
- # append extension based on filetype if not given
- if len(ext) and ext[0]=='.':
- ext = ext[1:]
- if filetype is not None and args.extension and (len(ext)<2 or all(c.isdigit() for c in ext)):
- ext = filetype.lower()
-
- # join filename + extension
- if len(ext):
- name_ext = name+'.'+ext
- else:
- name_ext = name
-
- # join path and filename
- try:
- filename = os.path.join(dirname, name_ext)
- except:
- pass
-
- filename = filename.replace('@v', config_version)
- filename = filename.replace('@f', config_friendlyname )
- filename = filename.replace('@h', config_hostname )
- filename = filename.replace('@H', device_hostname )
-
- return filename
-
-
-def MakeUrl(host, port=80, location=''):
- """
- Create a Tasmota host url
-
- @param host:
- hostname or IP of Tasmota host
- @param port:
- port number to use for http connection
- @param location:
- http url location
-
- @return:
- Tasmota http url
- """
- return "http://{shost}{sdelimiter}{sport}/{slocation}".format(\
- shost=host,
- sdelimiter=':' if port != 80 else '',
- sport=port if port != 80 else '',
- slocation=location )
-
-
-def LoadTasmotaConfig(filename):
- """
- Load config from Tasmota file
-
- @param filename:
- filename to load
-
- @return:
- binary config data (encrypted) or None on error
- """
-
- encode_cfg = None
-
- # read config from a file
- if not os.path.isfile(filename): # check file exists
- exit(ExitCode.FILE_NOT_FOUND, "File '{}' not found".format(filename),line=inspect.getlineno(inspect.currentframe()))
- try:
- with open(filename, "rb") as tasmotafile:
- encode_cfg = tasmotafile.read()
- except Exception as e:
- exit(e.args[0], "'{}' {}".format(filename, e[1]),line=inspect.getlineno(inspect.currentframe()))
-
- return encode_cfg
-
-
-def TasmotaGet(cmnd, host, port, username=DEFAULTS['source']['username'], password=None, contenttype = None):
- """
- Tasmota http request
-
- @param host:
- hostname or IP of Tasmota device
- @param port:
- http port of Tasmota device
- @param username:
- optional username for Tasmota web login
- @param password
- optional password for Tasmota web login
-
- @return:
- binary config data (encrypted) or None on error
- """
-
- # read config direct from device via http
- url = MakeUrl(host, port, cmnd)
- auth = None
- if username is not None and password is not None:
- auth = (username, password)
- res = requests.get(url, auth=auth)
-
- if not res.ok:
- exit(res.status_code, "Error on http GET request for {} - {}".format(url,res.reason), line=inspect.getlineno(inspect.currentframe()))
-
- if contenttype is not None and res.headers['Content-Type']!=contenttype:
- exit(ExitCode.DOWNLOAD_CONFIG_ERROR, "Device did not response properly, may be Tasmota webserver admin mode is disabled (WebServer 2)",line=inspect.getlineno(inspect.currentframe()))
-
- return res.status_code, res.content
-
-
-def GetTasmotaHostname(host, port, username=DEFAULTS['source']['username'], password=None):
- """
- Get Tasmota hostname from device
-
- @param host:
- hostname or IP of Tasmota device
- @param port:
- http port of Tasmota device
- @param username:
- optional username for Tasmota web login
- @param password
- optional password for Tasmota web login
-
- @return:
- Tasmota real hostname or None on error
- """
- hostname = None
-
- loginstr = ""
- if password is not None:
- loginstr = "user={}&password={}&".format(urllib2.quote(username), urllib2.quote(password))
- # get hostname
- responsecode, body = TasmotaGet("cm?{}cmnd=status%205".format(loginstr), host, port, username=username, password=password)
- if body is not None:
- jsonbody = json.loads(body)
- if "StatusNET" in jsonbody and "Hostname" in jsonbody["StatusNET"]:
- hostname = jsonbody["StatusNET"]["Hostname"]
- if args.verbose:
- message("Hostname for '{}' retrieved: '{}'".format(host, hostname), type_=LogType.INFO)
-
- return hostname
-
-
-def PullTasmotaConfig(host, port, username=DEFAULTS['source']['username'], password=None):
- """
- Pull config from Tasmota device
-
- @param host:
- hostname or IP of Tasmota device
- @param port:
- http port of Tasmota device
- @param username:
- optional username for Tasmota web login
- @param password
- optional password for Tasmota web login
-
- @return:
- binary config data (encrypted) or None on error
- """
- responsecode, body = TasmotaGet('dl', host, port, username, password, contenttype='application/octet-stream')
-
- return body
-
-
-def PushTasmotaConfig(encode_cfg, host, port, username=DEFAULTS['source']['username'], password=None):
- """
- Upload binary data to a Tasmota host using http
-
- @param encode_cfg:
- encrypted binary data or filename containing Tasmota encrypted binary config
- @param host:
- hostname or IP of Tasmota device
- @param port:
- http port of Tasmota device
- @param username:
- optional username for Tasmota web login
- @param password
- optional password for Tasmota web login
-
- @return
- errorcode, errorstring
- errorcode=0 if success, otherwise http response or exception code
- """
- if isinstance(encode_cfg, (bytes,bytearray)):
- encode_cfg = str(encode_cfg)
-
- # get restore config page first to set internal Tasmota vars
- responsecode, body = TasmotaGet('rs?', host, port, username, password, contenttype='text/html')
- if body is None:
- return responsecode, "ERROR"
-
- # ~ # post data
- url = MakeUrl(host, port, "u2")
- auth = None
- if username is not None and password is not None:
- auth = (username, password)
- files = {'u2':('{sprog}_v{sver}.dmp'.format(sprog=os.path.basename(sys.argv[0]), sver=VER), encode_cfg)}
- res = requests.post(url, auth=auth, files=files)
-
- if not res.ok:
- exit(res.status_code, "Error on http POST request for {} - {}".format(url,res.reason), line=inspect.getlineno(inspect.currentframe()))
-
- if res.headers['Content-Type']!='text/html':
- exit(ExitCode.DOWNLOAD_CONFIG_ERROR, "Device did not response properly, may be Tasmota webserver admin mode is disabled (WebServer 2)",line=inspect.getlineno(inspect.currentframe()))
-
- body = res.content
-
- findUpload = body.find("Upload")
- if findUpload < 0:
- return ExitCode.UPLOAD_CONFIG_ERROR, "Device did not response properly with upload result page"
-
- body = body[findUpload:]
- findSuccessful = body.find("Successful")
- if findSuccessful < 0:
- errmatch = re.search("(\S*)
(.*)
", body)
- reason = "Unknown error"
- if errmatch and len(errmatch.groups()) > 1:
- reason = errmatch.group(2)
- return ExitCode.UPLOAD_CONFIG_ERROR, reason
-
- return 0, 'OK'
-
-
-def DecryptEncrypt(obj):
- """
- Decrpt/Encrypt binary config data
-
- @param obj:
- binary config data
-
- @return:
- decrypted configuration (if obj contains encrypted data)
- """
- if isinstance(obj, (bytes,bytearray)):
- obj = str(obj)
- dobj = obj[0:2]
- for i in range(2, len(obj)):
- dobj += chr( (ord(obj[i]) ^ (CONFIG_FILE_XOR +i)) & 0xff )
- return dobj
-
-
-def GetSettingsCrc(dobj):
- """
- Return binary config data calclulated crc
-
- @param dobj:
- decrypted binary config data
-
- @return:
- 2 byte unsigned integer crc value
-
- """
- if isinstance(dobj, (bytes,bytearray)):
- dobj = str(dobj)
- version, size, setting = GetTemplateSetting(dobj)
- if version < 0x06060007 or version > 0x0606000A:
- size = 3584
- crc = 0
- for i in range(0, size):
- if not i in [14,15]: # Skip crc
- byte = ord(dobj[i])
- crc += byte * (i+1)
-
- return crc & 0xffff
-
-
-def GetSettingsCrc32(dobj):
- """
- Return binary config data calclulated crc32
-
- @param dobj:
- decrypted binary config data
-
- @return:
- 4 byte unsigned integer crc value
-
- """
- if isinstance(dobj, (bytes,bytearray)):
- dobj = str(dobj)
- crc = 0
- for i in range(0, len(dobj)-4):
- crc ^= ord(dobj[i])
- for j in range(0, 8):
- crc = (crc >> 1) ^ (-int(crc & 1) & 0xEDB88320);
-
- return ~crc & 0xffffffff
-
-
-def GetFieldDef(fielddef, fields="format_, addrdef, baseaddr, bits, bitshift, datadef, arraydef, validate, cmd, group, tasmotacmnd, converter, readconverter, writeconverter"):
-
- """
- Get field definition items
-
- @param fielddef:
- field format - see "Settings dictionary" above
- @param fields:
- comma separated string list of values to be returned
- possible values see fields default
-
- @return:
- set of values defined in
- """
- format_ = addrdef = baseaddr = datadef = arraydef = validate = cmd = group = tasmotacmnd = converter = readconverter = writeconverter = None
- bits = bitshift = 0
-
- # calling with nothing is wrong
- if fielddef is None:
- print(' is None', file=sys.stderr)
- raise SyntaxError(' error')
-
- # get top level items
- if len(fielddef) == 3:
- # converter not present
- format_, addrdef, datadef = fielddef
- elif len(fielddef) == 4:
- # converter present
- format_, addrdef, datadef, converter = fielddef
- else:
- print('wrong {} length ({}) in setting'.format(fielddef, len(fielddef)), file=sys.stderr)
- raise SyntaxError(' error')
-
- # ignore calls with 'root' setting
- if isinstance(format_, instance(dict)) and baseaddr is None and datadef is None:
- return eval(fields)
-
- if not isinstance(format_, instance((str,dict))):
- print('wrong {} type {} in {}'.format(format_, type(format_), fielddef), file=sys.stderr)
- raise SyntaxError(' error')
-
- # extract addrdef items
- baseaddr = addrdef
- if isinstance(baseaddr, instance((list,tuple))):
- if len(baseaddr) == 3:
- # baseaddr bit definition
- baseaddr, bits, bitshift = baseaddr
- if not isinstance(bits, instance(int)):
- print(' must be defined as integer in {}'.format(bits, fielddef), file=sys.stderr)
- raise SyntaxError(' error')
- if not isinstance(bitshift, instance(int)):
- print(' must be defined as integer in {}'.format(bitshift, fielddef), file=sys.stderr)
- raise SyntaxError(' error')
- else:
- print('wrong {} length ({}) in {}'.format(addrdef, len(addrdef), fielddef), file=sys.stderr)
- raise SyntaxError(' error')
- if not isinstance(baseaddr, instance(int)):
- print(' must be defined as integer in {}'.format(baseaddr, fielddef), file=sys.stderr)
- raise SyntaxError(' error')
-
- # extract datadef items
- arraydef = datadef
- if isinstance(datadef, instance((tuple))):
- if len(datadef) == 2:
- # datadef has a validator
- arraydef, validate = datadef
- elif len(datadef) == 3:
- # datadef has a validator and cmd set
- arraydef, validate, cmd = datadef
- # cmd must be a tuple with 2 objects
- if isinstance(cmd, instance((tuple))) and len(cmd) == 2:
- group, tasmotacmnd = cmd
- if group is not None and not isinstance(group, instance(str)):
- print('wrong {} in {}'.format(group, fielddef), file=sys.stderr)
- raise SyntaxError(' error')
- if tasmotacmnd is isinstance(tasmotacmnd, instance(tuple)):
- tasmotacmnds = tasmotacmnd
- for tasmotacmnd in tasmotacmnds:
- if tasmotacmnd is not None and not callable(tasmotacmnd) and not isinstance(tasmotacmnd, instance(str)):
- print('wrong {} in {}'.format(tasmotacmnd, fielddef), file=sys.stderr)
- raise SyntaxError(' error')
- else:
- if tasmotacmnd is not None and not callable(tasmotacmnd) and not isinstance(tasmotacmnd, instance(str)):
- print('wrong {} in {}'.format(tasmotacmnd, fielddef), file=sys.stderr)
- raise SyntaxError(' error')
- else:
- print('wrong {} length ({}) in {}'.format(cmd, len(cmd), fielddef), file=sys.stderr)
- raise SyntaxError(' error')
- else:
- print('wrong {} length ({}) in {}'.format(datadef, len(datadef), fielddef), file=sys.stderr)
- raise SyntaxError(' error')
-
- if validate is not None and (not isinstance(validate, instance(str)) and not callable(validate)):
- print('wrong {} type {} in {}'.format(validate, type(validate), fielddef), file=sys.stderr)
- raise SyntaxError(' error')
-
- # convert single int into one-dimensional list
- if isinstance(arraydef, instance(int)):
- arraydef = [arraydef]
-
- if arraydef is not None and not isinstance(arraydef, instance((list))):
- print('wrong {} type {} in {}'.format(arraydef, type(arraydef), fielddef), file=sys.stderr)
- raise SyntaxError(' error')
-
- # get read/write converter items
- readconverter = converter
- if isinstance(converter, instance((tuple))):
- if len(converter) == 2:
- # converter has read/write converter
- readconverter, writeconverter = converter
- if readconverter is not None and not isinstance(readconverter, instance(str)) and not callable(readconverter):
- print('wrong {} type {} in {}'.format(readconverter, type(readconverter), fielddef), file=sys.stderr)
- raise SyntaxError(' error')
- if writeconverter is not None and (not isinstance(writeconverter, instance((bool,str))) and not callable(writeconverter)):
- print('wrong {} type {} in {}'.format(writeconverter, type(writeconverter), fielddef), file=sys.stderr)
- raise SyntaxError(' error')
- else:
- print('wrong {} length ({}) in {}'.format(converter, len(converter), fielddef), file=sys.stderr)
- raise SyntaxError(' error')
-
-
- return eval(fields)
-
-
-def ReadWriteConverter(value, fielddef, read=True, raw=False):
- """
- Convert field value based on field desc
-
- @param value:
- original value
- @param fielddef
- field definition - see "Settings dictionary" above
- @param read
- use read conversion if True, otherwise use write conversion
- @param raw
- return raw values (True) or converted values (False)
-
- @return:
- (un)converted value
- """
- converter, readconverter, writeconverter = GetFieldDef(fielddef, fields='converter, readconverter, writeconverter')
-
- # call password functions even if raw value should be processed
- if read and callable(readconverter) and readconverter == passwordread:
- raw = False
- if not read and callable(writeconverter) and writeconverter == passwordwrite:
- raw = False
-
- if not raw and converter is not None:
- conv = readconverter if read else writeconverter
- try:
- if isinstance(conv, instance(str)): # evaluate strings
- return eval(conv.replace('$','value'))
- elif callable(conv): # use as format function
- return conv(value)
- except Exception as e:
- exit(e.args[0], e.args[1], type_=LogType.WARNING, line=inspect.getlineno(inspect.currentframe()))
-
- return value
-
-
-def CmndConverter(valuemapping, value, idx, fielddef):
- """
- Convert field value into Tasmota command if available
-
- @param valuemapping:
- data mapping
- @param value:
- original value
- @param fielddef
- field definition - see "Settings dictionary" above
-
- @return:
- converted value, list of values or None if unable to convert
- """
- converter, readconverter, writeconverter, group, tasmotacmnd = GetFieldDef(fielddef, fields='converter, readconverter, writeconverter, group, tasmotacmnd')
-
- result = None
-
- if (callable(readconverter) and readconverter == passwordread) or (callable(writeconverter) and writeconverter == passwordwrite):
- if value == HIDDEN_PASSWORD:
- return None
- else:
- result = value
-
- if tasmotacmnd is not None and (callable(tasmotacmnd) or len(tasmotacmnd) > 0):
- if idx is not None:
- idx += 1
- if isinstance(tasmotacmnd, instance(str)): # evaluate strings
- if idx is not None:
- evalstr = tasmotacmnd.replace('$','value').replace('#','idx').replace('@','valuemapping')
- else:
- evalstr = tasmotacmnd.replace('$','value').replace('@','valuemapping')
- result = eval(evalstr)
-
- elif callable(tasmotacmnd): # use as format function
- if idx is not None:
- result = tasmotacmnd(value, idx)
- else:
- result = tasmotacmnd(value)
-
- return result
-
-
-def ValidateValue(value, fielddef):
- """
- Validate a value if validator is defined in fielddef
-
- @param value:
- original value
- @param fielddef
- field definition - see "Settings dictionary" above
-
- @return:
- True if value is valid, False if invalid
- """
- validate = GetFieldDef(fielddef, fields='validate')
-
- if value == 0:
- # can not complete all validate condition
- # some Tasmota values are not allowed to be 0 on input
- # even though these values are set to 0 on Tasmota initial.
- # so we can't validate 0 values
- return True;
-
- valid = True
- try:
- if isinstance(validate, instance(str)): # evaluate strings
- valid = eval(validate.replace('$','value'))
- elif callable(validate): # use as format function
- valid = validate(value)
- except:
- valid = False
-
- return valid
-
-
-def GetFormatCount(format_):
- """
- Get format prefix count
-
- @param format_:
- format specifier
-
- @return:
- prefix count or 1 if not specified
- """
-
- if isinstance(format_, instance(str)):
- match = re.search("\s*(\d+)", format_)
- if match:
- return int(match.group(0))
-
- return 1
-
-
-def GetFormatType(format_):
- """
- Get format type and bitsize without prefix
-
- @param format_:
- format specifier
-
- @return:
- (format_, 0) or (format without prefix, bitsize)
- """
-
- formattype = format_
- bitsize = 0
- if isinstance(format_, instance(str)):
- match = re.search("\s*(\D+)", format_)
- if match:
- formattype = match.group(0)
- bitsize = struct.calcsize(formattype) * 8
- return formattype, bitsize
-
-
-def GetFieldMinMax(fielddef):
- """
- Get minimum, maximum of field based on field format definition
-
- @param fielddef:
- field format - see "Settings dictionary" above
-
- @return:
- min, max
- """
- minmax = {'c': (0, 0xff),
- '?': (0, 1),
- 'b': (~0x7f, 0x7f),
- 'B': (0, 0xff),
- 'h': (~0x7fff, 0x7fff),
- 'H': (0, 0xffff),
- 'i': (~0x7fffffff, 0x7fffffff),
- 'I': (0, 0xffffffff),
- 'l': (~0x7fffffff, 0x7fffffff),
- 'L': (0, 0xffffffff),
- 'q': (~0x7fffffffffffffff, 0x7fffffffffffffff),
- 'Q': (0, 0x7fffffffffffffff),
- 'f': (sys.float_info.min, sys.float_info.max),
- 'd': (sys.float_info.min, sys.float_info.max),
- }
- format_ = GetFieldDef(fielddef, fields='format_')
- min_ = 0
- max_ = 0
-
- if format_[-1:] in minmax:
- min_, max_ = minmax[format_[-1:]]
- max_ *= GetFormatCount(format_)
- elif format_[-1:] in ['s','p']:
- # s and p may have a prefix as length
- max_ = GetFormatCount(format_)
-
- return min_,max_
-
-
-def GetFieldLength(fielddef):
- """
- Get length of a field in bytes based on field format definition
-
- @param fielddef:
- field format - see "Settings dictionary" above
-
- @return:
- length of field in bytes
- """
-
- length=0
- format_, addrdef, arraydef = GetFieldDef(fielddef, fields='format_, addrdef, arraydef')
-
- # contains a integer list
- if isinstance(arraydef, instance(list)) and len(arraydef) > 0:
- # arraydef contains a list
- # calc size recursive by sum of all elements
- for i in range(0, arraydef[0]):
- subfielddef = GetSubfieldDef(fielddef)
- if len(arraydef) > 1:
- length += GetFieldLength( (format_, addrdef, subfielddef) )
- # single array
- else:
- length += GetFieldLength( (format_, addrdef, None) )
-
- elif isinstance(format_, instance(dict)):
- # -> iterate through format
- addr = None
- setting = format_
- for name in setting:
- baseaddr, bits, bitshift = GetFieldDef(setting[name], fields='baseaddr, bits, bitshift')
- _len = GetFieldLength(setting[name])
- if addr != baseaddr:
- addr = baseaddr
- length += _len
-
- # a simple value
- elif isinstance(format_, instance(str)):
- length = struct.calcsize(format_)
-
- return length
-
-
-def GetSubfieldDef(fielddef):
- """
- Get subfield definition from a given field definition
-
- @param fielddef:
- see Settings desc above
-
- @return:
- subfield definition
- """
-
- format_, addrdef, datadef, arraydef, validate, cmd, converter = GetFieldDef(fielddef, fields='format_, addrdef, datadef, arraydef, validate, cmd, converter')
-
- # create new arraydef
- if len(arraydef) > 1:
- arraydef = arraydef[1:]
- else:
- arraydef = None
-
- # create new datadef
- if isinstance(datadef, instance(tuple)):
- if cmd is not None:
- datadef = (arraydef, validate, cmd)
- else:
- datadef = (arraydef, validate)
- else:
- datadef = arraydef
-
- # set new field def
- subfielddef = None
- if converter is not None:
- subfielddef = (format_, addrdef, datadef, converter)
- else:
- subfielddef = (format_, addrdef, datadef)
-
- return subfielddef
-
-
-def IsFilterGroup(group):
- """
- Check if group is valid on filter
-
- @param grooup:
- group name to check
-
- @return:
- True if group is in filter, otherwise False
- """
-
- if args.filter is not None:
- if group is None:
- return False
- if group == '*':
- return True
- if group.title() != INTERNAL.title() and group.title() not in (groupname.title() for groupname in args.filter):
- return False
- return True
-
-
-def GetFieldValue(fielddef, dobj, addr):
- """
- Get single field value from definition
-
- @param fielddef:
- see Settings desc
- @param dobj:
- decrypted binary config data
- @param addr
- addr within dobj
-
- @return:
- value read from dobj
- """
-
- format_, bits, bitshift = GetFieldDef(fielddef, fields='format_, bits, bitshift')
-
- value_ = 0
- unpackedvalue = struct.unpack_from(format_, dobj, addr)
- singletype, bitsize = GetFormatType(format_)
-
- if not format_[-1:].lower() in ['s','p']:
- for val in unpackedvalue:
- value_ <<= bitsize
- value_ = value_ + val
- value_ = bitsRead(value_, bitshift, bits)
- else:
- value_ = unpackedvalue[0]
- s = str(value_).split('\0')[0] # use left string until \0
- value_ = unicode(s, errors='ignore') # remove character > 127
-
- return value_
-
-
-def SetFieldValue(fielddef, dobj, addr, value):
- """
- Set single field value from definition
-
- @param fielddef:
- see Settings desc
- @param dobj:
- decrypted binary config data
- @param addr
- addr within dobj
- @param value
- new value
-
- @return:
- new decrypted binary config data
- """
-
- format_, bits, bitshift = GetFieldDef(fielddef, fields='format_, bits, bitshift')
- formatcnt = GetFormatCount(format_)
- singletype, bitsize = GetFormatType(format_)
- if debug(args) >= 2:
- print("SetFieldValue(): fielddef {}, addr 0x{:04x} value {} formatcnt {} singletype {} bitsize {} ".format(fielddef,addr,value,formatcnt,singletype,bitsize), file=sys.stderr)
- if not format_[-1:].lower() in ['s','p']:
- addr += (bitsize / 8) * formatcnt
- for _ in range(0, formatcnt):
- addr -= (bitsize / 8)
- maxunsigned = ((2**bitsize) - 1)
- maxsigned = ((2**bitsize)>>1)-1
- val = value & maxunsigned
- if isinstance(value,instance(int)) and value < 0 and val > maxsigned:
- val = ((maxunsigned+1)-val) * (-1)
- if debug(args) >= 3:
- print("SetFieldValue(): Single type - fielddef {}, addr 0x{:04x} value {} singletype {} bitsize {}".format(fielddef,addr,val,singletype,bitsize), file=sys.stderr)
- try:
- struct.pack_into(singletype, dobj, addr, val)
- except struct.error as e:
- exit(ExitCode.RESTORE_DATA_ERROR,
- "Single type {} [fielddef={}, addr=0x{:04x}, value={}] - skipped!".format(e,fielddef,addr,val),
- type_=LogType.WARNING,
- doexit=not args.ignorewarning,
- line=inspect.getlineno(inspect.currentframe()))
- value >>= bitsize
- else:
- if debug(args) >= 3:
- print("SetFieldValue(): String type - fielddef {}, addr 0x{:04x} value {} format_ {}".format(fielddef,addr,value,format_), file=sys.stderr)
- try:
- struct.pack_into(format_, dobj, addr, value)
- except struct.error as e:
- exit(ExitCode.RESTORE_DATA_ERROR,
- "String type {} [fielddef={}, addr=0x{:04x}, value={}} - skipped!".format(e,fielddef,addr,value),
- type_=LogType.WARNING,
- doexit=not args.ignorewarning,
- line=inspect.getlineno(inspect.currentframe()))
-
- return dobj
-
-
-def GetField(dobj, fieldname, fielddef, raw=False, addroffset=0):
- """
- Get field value from definition
-
- @param dobj:
- decrypted binary config data
- @param fieldname:
- name of the field
- @param fielddef:
- see Settings desc above
- @param raw
- return raw values (True) or converted values (False)
- @param addroffset
- use offset for baseaddr (used for recursive calls)
-
- @return:
- field mapping
- """
-
- if isinstance(dobj, instance((bytes,bytearray))):
- dobj = str(dobj)
-
- valuemapping = None
-
- # get field definition
- format_, baseaddr, bits, bitshift, arraydef, group, tasmotacmnd = GetFieldDef(fielddef, fields='format_, baseaddr, bits, bitshift, arraydef, group, tasmotacmnd')
-
- # filter groups
- if not IsFilterGroup(group):
- return valuemapping
-
- # contains a integer list
- if isinstance(arraydef, instance(list)) and len(arraydef) > 0:
- valuemapping = []
- offset = 0
- for i in range(0, arraydef[0]):
- subfielddef = GetSubfieldDef(fielddef)
- length = GetFieldLength(subfielddef)
- if length != 0:
- value = GetField(dobj, fieldname, subfielddef, raw=raw, addroffset=addroffset+offset)
- valuemapping.append(value)
- offset += length
-
- # contains a dict
- elif isinstance(format_, instance(dict)):
- mapping_value = {}
- # -> iterate through format
- for name in format_:
- value = None
- value = GetField(dobj, name, format_[name], raw=raw, addroffset=addroffset)
- if value is not None:
- mapping_value[name] = value
- # copy complete returned mapping
- valuemapping = copy.deepcopy(mapping_value)
-
- # a simple value
- elif isinstance(format_, instance((str, bool, int, float, long))):
- if GetFieldLength(fielddef) != 0:
- valuemapping = ReadWriteConverter(GetFieldValue(fielddef, dobj, baseaddr+addroffset), fielddef, read=True, raw=raw)
-
- else:
- exit(ExitCode.INTERNAL_ERROR, "Wrong mapping format definition: '{}'".format(format_), type_=LogType.WARNING, doexit=not args.ignorewarning, line=inspect.getlineno(inspect.currentframe()))
-
- return valuemapping
-
-
-def SetField(dobj, fieldname, fielddef, restore, addroffset=0, filename=""):
- """
- Get field value from definition
-
- @param dobj:
- decrypted binary config data
- @param fieldname:
- name of the field
- @param fielddef:
- see Settings desc above
- @param restore
- restore mapping with the new value(s)
- @param addroffset
- use offset for baseaddr (used for recursive calls)
- @param filename
- related filename (for messages only)
-
- @return:
- new decrypted binary config data
- """
- format_, baseaddr, bits, bitshift, arraydef, group, writeconverter = GetFieldDef(fielddef, fields='format_, baseaddr, bits, bitshift, arraydef, group, writeconverter')
- # cast unicode
- fieldname = str(fieldname)
-
- # filter groups
- if not IsFilterGroup(group):
- return dobj
-
- # do not write readonly values
- if writeconverter is False:
- if debug(args) >= 2:
- print("SetField(): Readonly '{}' using '{}'/{}{} @{} skipped".format(fieldname, format_, arraydef, bits, hex(baseaddr+addroffset)), file=sys.stderr)
- return dobj
-
- # contains a list
- if isinstance(arraydef, instance(list)) and len(arraydef) > 0:
- offset = 0
- if len(restore) > arraydef[0]:
- exit(ExitCode.RESTORE_DATA_ERROR, "file '{sfile}', array '{sname}[{selem}]' exceeds max number of elements [{smax}]".format(sfile=filename, sname=fieldname, selem=len(restore), smax=arraydef[0]), type_=LogType.WARNING, doexit=not args.ignorewarning, line=inspect.getlineno(inspect.currentframe()))
- for i in range(0, arraydef[0]):
- subfielddef = GetSubfieldDef(fielddef)
- length = GetFieldLength(subfielddef)
- if length != 0:
- if i >= len(restore): # restore data list may be shorter than definition
- break
- subrestore = restore[i]
- dobj = SetField(dobj, fieldname, subfielddef, subrestore, addroffset=addroffset+offset, filename=filename)
- offset += length
-
- # contains a dict
- elif isinstance(format_, instance(dict)):
- for name in format_: # -> iterate through format
- if name in restore:
- dobj = SetField(dobj, name, format_[name], restore[name], addroffset=addroffset, filename=filename)
-
- # a simple value
- elif isinstance(format_, instance((str, bool, int, float, long))):
- valid = True
- err = ""
- errformat = ""
-
- min_, max_ = GetFieldMinMax(fielddef)
- value = _value = None
- skip = False
-
- # simple char value
- if format_[-1:] in ['c']:
- try:
- value = ReadWriteConverter(restore.encode(STR_ENCODING)[0], fielddef, read=False)
- except Exception as e:
- exit(e.args[0], e.args[1], type_=LogType.WARNING, line=inspect.getlineno(inspect.currentframe()))
- valid = False
-
- # bool
- elif format_[-1:] in ['?']:
- try:
- value = ReadWriteConverter(bool(restore), fielddef, read=False)
- except Exception as e:
- exit(e.args[0], e.args[1], type_=LogType.WARNING, line=inspect.getlineno(inspect.currentframe()))
- valid = False
-
- # integer
- elif format_[-1:] in ['b','B','h','H','i','I','l','L','q','Q','P']:
- value = ReadWriteConverter(restore, fielddef, read=False)
- if isinstance(value, instance(str)):
- value = int(value, 0)
- else:
- value = int(value)
- # bits
- if bits != 0:
- bitvalue = value
- value = struct.unpack_from(format_, dobj, baseaddr+addroffset)[0]
- # validate restore value
- valid = ValidateValue(bitvalue, fielddef)
- if not valid:
- err = "valid bit range exceeding"
- value = bitvalue
- else:
- mask = (1< mask:
- min_ = 0
- max_ = mask
- _value = bitvalue
- valid = False
- else:
- if bitshift >= 0:
- bitvalue <<= bitshift
- mask <<= bitshift
- else:
- bitvalue >>= abs(bitshift)
- mask >>= abs(bitshift)
- v=value
- value &= (0xffffffff ^ mask)
- value |= bitvalue
-
- # full size values
- else:
- # validate restore function
- valid = ValidateValue(value, fielddef)
- if not valid:
- err = "valid range exceeding"
- _value = value
-
- # float
- elif format_[-1:] in ['f','d']:
- try:
- value = ReadWriteConverter(float(restore), fielddef, read=False)
- except:
- valid = False
-
- # string
- elif format_[-1:] in ['s','p']:
- value = ReadWriteConverter(restore.encode(STR_ENCODING), fielddef, read=False)
- err = "string length exceeding"
- if value is not None:
- max_ -= 1
- valid = min_ <= len(value) <= max_
- else:
- skip = True
- valid = True
-
- if value is None and not skip:
- # None is an invalid value
- valid = False
-
- if valid is None and not skip:
- # validate against object type size
- valid = min_ <= value <= max_
- if not valid:
- err = "type range exceeding"
- errformat = " [{smin},{smax}]"
-
- if _value is None:
- # copy value before possible change below
- _value = value
-
- if isinstance(_value, instance(str)):
- _value = "'{}'".format(_value)
-
- if valid:
- if not skip:
- if debug(args) >= 2:
- sbits = " {} bits shift {}".format(bits, bitshift) if bits else ""
- strvalue = "{} [{}]".format(_value, hex(value)) if isinstance(_value, instance(int)) else _value
- print("SetField(): Set '{}' using '{}'/{}{} @{} to {}".format(fieldname, format_, arraydef, sbits, hex(baseaddr+addroffset), strvalue), file=sys.stderr)
- if fieldname != 'cfg_crc' and fieldname != '_':
- prevvalue = GetFieldValue(fielddef, dobj, baseaddr+addroffset)
- dobj = SetFieldValue(fielddef, dobj, baseaddr+addroffset, value)
- curvalue = GetFieldValue(fielddef, dobj, baseaddr+addroffset)
- if prevvalue != curvalue and args.verbose:
- message("Value for '{}' changed from {} to {}".format(fieldname, prevvalue, curvalue), type_=LogType.INFO)
- else:
- if debug(args) >= 2:
- print("SetField(): Special field '{}' using '{}'/{}{} @{} skipped".format(fieldname, format_, arraydef, bits, hex(baseaddr+addroffset)), file=sys.stderr)
- else:
- sformat = "file '{sfile}' - {{'{sname}': {svalue}}} ({serror})"+errformat
- exit(ExitCode.RESTORE_DATA_ERROR, sformat.format(sfile=filename, sname=fieldname, serror=err, svalue=_value, smin=min_, smax=max_), type_=LogType.WARNING, doexit=not args.ignorewarning)
-
- return dobj
-
-
-def SetCmnd(cmnds, fieldname, fielddef, valuemapping, mappedvalue, addroffset=0, idx=None):
- """
- Get field value from definition
-
- @param cmnds:
- Tasmota command mapping: { 'group': ['cmnd' <,'cmnd'...>] ... }
- @param fieldname:
- name of the field
- @param fielddef:
- see Settings desc above
- @param valuemapping:
- data mapping
- @param mappedvalue
- mappedvalue mapping with the new value(s)
- @param addroffset
- use offset for baseaddr (used for recursive calls)
- @param idx
- optional array index
-
- @return:
- new Tasmota command mapping
- """
- format_, baseaddr, bits, bitshift, arraydef, group, tasmotacmnd, writeconverter = GetFieldDef(fielddef, fields='format_, baseaddr, bits, bitshift, arraydef, group, tasmotacmnd, writeconverter')
-
- # cast unicode
- fieldname = str(fieldname)
-
- # filter groups
- if not IsFilterGroup(group):
- return cmnds
-
- # contains a list
- if isinstance(arraydef, instance(list)) and len(arraydef) > 0:
- offset = 0
- if len(mappedvalue) > arraydef[0]:
- exit(ExitCode.RESTORE_DATA_ERROR, "array '{sname}[{selem}]' exceeds max number of elements [{smax}]".format(sname=fieldname, selem=len(mappedvalue), smax=arraydef[0]), type_=LogType.WARNING, doexit=not args.ignorewarning, line=inspect.getlineno(inspect.currentframe()))
- for i in range(0, arraydef[0]):
- subfielddef = GetSubfieldDef(fielddef)
- length = GetFieldLength(subfielddef)
- if length != 0:
- if i >= len(mappedvalue): # mappedvalue data list may be shorter than definition
- break
- subrestore = mappedvalue[i]
- cmnds = SetCmnd(cmnds, fieldname, subfielddef, valuemapping, subrestore, addroffset=addroffset+offset, idx=i)
- offset += length
-
- # contains a dict
- elif isinstance(format_, instance(dict)):
- for name in format_: # -> iterate through format
- if name in mappedvalue:
- cmnds = SetCmnd(cmnds, name, format_[name], valuemapping, mappedvalue[name], addroffset=addroffset, idx=idx)
-
- # a simple value
- elif isinstance(format_, instance((str, bool, int, float, long))):
- if group is not None:
- group = group.title();
- if isinstance(tasmotacmnd, instance(tuple)):
- tasmotacmnds = tasmotacmnd
- for tasmotacmnd in tasmotacmnds:
- cmnd = CmndConverter(valuemapping, mappedvalue, idx, fielddef)
- if group is not None and cmnd is not None:
- if group not in cmnds:
- cmnds[group] = []
- if isinstance(cmnd, instance(list)):
- for c in cmnd:
- cmnds[group].append(c)
- else:
- cmnds[group].append(cmnd)
- else:
- cmnd = CmndConverter(valuemapping, mappedvalue, idx, fielddef)
- if group is not None and cmnd is not None:
- if group not in cmnds:
- cmnds[group] = []
- if isinstance(cmnd, instance(list)):
- for c in cmnd:
- cmnds[group].append(c)
- else:
- cmnds[group].append(cmnd)
-
- return cmnds
-
-
-def Bin2Mapping(decode_cfg):
- """
- Decodes binary data stream into pyhton mappings dict
-
- @param decode_cfg:
- binary config data (decrypted)
-
- @return:
- valuemapping data as mapping dictionary
- """
- if isinstance(decode_cfg, instance((bytes,bytearray))):
- decode_cfg = str(decode_cfg)
-
- # get binary header and template to use
- version, size, setting = GetTemplateSetting(decode_cfg)
-
- # if we did not found a mathching setting
- if setting is None:
- exit(ExitCode.UNSUPPORTED_VERSION, "Tasmota configuration version {} not supported".format(version),line=inspect.getlineno(inspect.currentframe()))
-
- if 'version' in setting:
- cfg_version = GetField(decode_cfg, 'version', setting['version'], raw=True)
-
- # check size if exists
- if 'cfg_size' in setting:
- cfg_size = GetField(decode_cfg, 'cfg_size', setting['cfg_size'], raw=True)
- # read size should be same as definied in setting
- if cfg_size > size:
- # may be processed
- exit(ExitCode.DATA_SIZE_MISMATCH, "Number of bytes read does ot match - read {}, expected {} byte".format(cfg_size, size), type_=LogType.ERROR,line=inspect.getlineno(inspect.currentframe()))
- elif cfg_size < size:
- # less number of bytes can not be processed
- exit(ExitCode.DATA_SIZE_MISMATCH, "Number of bytes read to small to process - read {}, expected {} byte".format(cfg_size, size), type_=LogType.ERROR,line=inspect.getlineno(inspect.currentframe()))
-
- # check crc if exists
- if 'cfg_crc' in setting:
- cfg_crc = GetField(decode_cfg, 'cfg_crc', setting['cfg_crc'], raw=True)
- else:
- cfg_crc = GetSettingsCrc(decode_cfg)
- if 'cfg_crc32' in setting:
- cfg_crc32 = GetField(decode_cfg, 'cfg_crc32', setting['cfg_crc32'], raw=True)
- else:
- cfg_crc32 = GetSettingsCrc32(decode_cfg)
- if version < 0x0606000B:
- if cfg_crc != GetSettingsCrc(decode_cfg):
- exit(ExitCode.DATA_CRC_ERROR, 'Data CRC error, read 0x{:4x} should be 0x{:4x}'.format(cfg_crc, GetSettingsCrc(decode_cfg)), type_=LogType.WARNING, doexit=not args.ignorewarning,line=inspect.getlineno(inspect.currentframe()))
- else:
- if cfg_crc32 != GetSettingsCrc32(decode_cfg):
- exit(ExitCode.DATA_CRC_ERROR, 'Data CRC32 error, read 0x{:8x} should be 0x{:8x}'.format(cfg_crc32, GetSettingsCrc32(decode_cfg)), type_=LogType.WARNING, doexit=not args.ignorewarning,line=inspect.getlineno(inspect.currentframe()))
-
- # get valuemapping
- valuemapping = GetField(decode_cfg, None, (setting,0,(None, None, (INTERNAL, None))))
-
- # add header info
- timestamp = datetime.now()
- valuemapping['header'] = { 'timestamp':timestamp.strftime("%Y-%m-%d %H:%M:%S"),
- 'format': {
- 'jsonindent': args.jsonindent,
- 'jsoncompact': args.jsoncompact,
- 'jsonsort': args.jsonsort,
- 'jsonhidepw': args.jsonhidepw,
- },
- 'template': {
- 'version': hex(version),
- 'crc': hex(cfg_crc),
- },
- 'data': {
- 'crc': hex(GetSettingsCrc(decode_cfg)),
- 'size': len(decode_cfg),
- },
- 'script': {
- 'name': os.path.basename(__file__),
- 'version': VER,
- },
- 'os': (platform.machine(), platform.system(), platform.release(), platform.version(), platform.platform()),
- 'python': platform.python_version(),
- }
- if 'cfg_crc' in setting:
- valuemapping['header']['template'].update({'size': cfg_size})
- if 'cfg_crc32' in setting:
- valuemapping['header']['template'].update({'crc32': hex(cfg_crc32)})
- valuemapping['header']['data'].update({'crc32': hex(GetSettingsCrc32(decode_cfg))})
- if 'version' in setting:
- valuemapping['header']['data'].update({'version': hex(cfg_version)})
-
- return valuemapping
-
-
-def Mapping2Bin(decode_cfg, jsonconfig, filename=""):
- """
- Encodes into binary data stream
-
- @param decode_cfg:
- binary config data (decrypted)
- @param jsonconfig:
- restore data mapping
- @param filename:
- name of the restore file (for error output only)
-
- @return:
- changed binary config data (decrypted) or None on error
- """
- if isinstance(decode_cfg, instance(str)):
- decode_cfg = bytearray(decode_cfg)
-
-
- # get binary header data to use the correct version template from device
- version, size, setting = GetTemplateSetting(decode_cfg)
-
- # make empty binarray array
- _buffer = bytearray()
- # add data
- _buffer.extend(decode_cfg)
-
- if setting is not None:
- # iterate through restore data mapping
- for name in jsonconfig:
- # key must exist in both dict
- if name in setting:
- SetField(_buffer, name, setting[name], jsonconfig[name], addroffset=0, filename=filename)
- else:
- if name != 'header':
- exit(ExitCode.RESTORE_DATA_ERROR, "Restore file '{}' contains obsolete name '{}', skipped".format(filename, name), type_=LogType.WARNING, doexit=not args.ignorewarning)
-
- if 'cfg_crc' in setting:
- crc = GetSettingsCrc(_buffer)
- struct.pack_into(setting['cfg_crc'][0], _buffer, setting['cfg_crc'][1], crc)
- if 'cfg_crc32' in setting:
- crc32 = GetSettingsCrc32(_buffer)
- struct.pack_into(setting['cfg_crc32'][0], _buffer, setting['cfg_crc32'][1], crc32)
- return _buffer
-
- else:
- exit(ExitCode.UNSUPPORTED_VERSION,"File '{}', Tasmota configuration version 0x{:x} not supported".format(filename, version), type_=LogType.WARNING, doexit=not args.ignorewarning)
-
- return None
-
-
-def Mapping2Cmnd(decode_cfg, valuemapping, filename=""):
- """
- Encodes mapping data into Tasmota command mapping
-
- @param decode_cfg:
- binary config data (decrypted)
- @param valuemapping:
- data mapping
- @param filename:
- name of the restore file (for error output only)
-
- @return:
- Tasmota command mapping {group: [cmnd <,cmnd <,...>>]}
- """
- if isinstance(decode_cfg, instance(str)):
- decode_cfg = bytearray(decode_cfg)
-
- # get binary header data to use the correct version template from device
- version, size, setting = GetTemplateSetting(decode_cfg)
-
- cmnds = {}
-
- if setting is not None:
- # iterate through restore data mapping
- for name in valuemapping:
- # key must exist in both dict
- if name in setting:
- cmnds = SetCmnd(cmnds, name, setting[name], valuemapping, valuemapping[name], addroffset=0)
- else:
- if name != 'header':
- exit(ExitCode.RESTORE_DATA_ERROR, "Restore file '{}' contains obsolete name '{}', skipped".format(filename, name), type_=LogType.WARNING, doexit=not args.ignorewarning)
-
- return cmnds
-
- else:
- exit(ExitCode.UNSUPPORTED_VERSION,"File '{}', Tasmota configuration version 0x{:x} not supported".format(filename, version), type_=LogType.WARNING, doexit=not args.ignorewarning)
-
- return None
-
-
-def Backup(backupfile, backupfileformat, encode_cfg, decode_cfg, configmapping):
- """
- Create backup file
-
- @param backupfile:
- Raw backup filename from program args
- @param backupfileformat:
- Backup file format
- @param encode_cfg:
- binary config data (encrypted)
- @param decode_cfg:
- binary config data (decrypted)
- @param configmapping:
- config data mapppings
- """
-
- name, ext = os.path.splitext(backupfile)
- if ext.lower() == '.'+FileType.BIN.lower():
- backupfileformat = FileType.BIN
- elif ext.lower() == '.'+FileType.DMP.lower():
- backupfileformat = FileType.DMP
- elif ext.lower() == '.'+FileType.JSON.lower():
- backupfileformat = FileType.JSON
-
- fileformat = ""
- # Tasmota format
- if backupfileformat.lower() == FileType.DMP.lower():
- fileformat = "Tasmota"
- backup_filename = MakeFilename(backupfile, FileType.DMP, configmapping)
- if args.verbose:
- message("Writing backup file '{}' ({} format)".format(backup_filename, fileformat), type_=LogType.INFO)
- try:
- with open(backup_filename, "wb") as backupfp:
- backupfp.write(encode_cfg)
- except Exception as e:
- exit(e.args[0], "'{}' {}".format(backup_filename, e[1]),line=inspect.getlineno(inspect.currentframe()))
-
- # binary format
- elif backupfileformat.lower() == FileType.BIN.lower():
- fileformat = "binary"
- backup_filename = MakeFilename(backupfile, FileType.BIN, configmapping)
- if args.verbose:
- message("Writing backup file '{}' ({} format)".format(backup_filename, fileformat), type_=LogType.INFO)
- try:
- with open(backup_filename, "wb") as backupfp:
- backupfp.write(struct.pack('>]}
- """
- def OutputTasmotaSubCmnds(cmnds):
- if args.cmndsort:
- for cmnd in sorted(cmnds, key = lambda cmnd:[int(c) if c.isdigit() else c for c in re.split('(\d+)', cmnd)]):
- print("{}{}".format(" "*args.cmndindent, cmnd))
- else:
- for cmnd in cmnds:
- print("{}{}".format(" "*args.cmndindent, cmnd))
-
- groups = GetGroupList(Settings[0][2])
-
- if args.cmndgroup:
- for group in groups:
- if group.title() in (groupname.title() for groupname in tasmotacmnds):
- cmnds = tasmotacmnds[group]
- print
- print("# {}:".format(group))
- OutputTasmotaSubCmnds(cmnds)
-
- else:
- cmnds = []
- for group in groups:
- if group.title() in (groupname.title() for groupname in tasmotacmnds):
- cmnds.extend(tasmotacmnds[group])
- OutputTasmotaSubCmnds(cmnds)
-
-def ParseArgs():
- """
- Program argument parser
-
- @return:
- configargparse.parse_args() result
- """
- global parser
- parser = configargparse.ArgumentParser(description='Backup/Restore Tasmota configuration data.',
- epilog='Either argument -d or -f must be given.',
- add_help=False,
- formatter_class=lambda prog: CustomHelpFormatter(prog))
-
- source = parser.add_argument_group('Source', 'Read/Write Tasmota configuration from/to')
- source.add_argument('-f', '--file', '--tasmota-file',
- metavar='',
- dest='tasmotafile',
- default=DEFAULTS['source']['tasmotafile'],
- help="file to retrieve/write Tasmota configuration from/to (default: {})'".format(DEFAULTS['source']['tasmotafile']))
- source.add_argument('-d', '--device', '--host',
- metavar='',
- dest='device',
- default=DEFAULTS['source']['device'],
- help="hostname or IP address to retrieve/send Tasmota configuration from/to (default: {})".format(DEFAULTS['source']['device']) )
- source.add_argument('-P', '--port',
- metavar='',
- dest='port',
- default=DEFAULTS['source']['port'],
- help="TCP/IP port number to use for the host connection (default: {})".format(DEFAULTS['source']['port']) )
- source.add_argument('-u', '--username',
- metavar='',
- dest='username',
- default=DEFAULTS['source']['username'],
- help="host HTTP access username (default: {})".format(DEFAULTS['source']['username']))
- source.add_argument('-p', '--password',
- metavar='',
- dest='password',
- default=DEFAULTS['source']['password'],
- help="host HTTP access password (default: {})".format(DEFAULTS['source']['password']))
-
- backup = parser.add_argument_group('Backup/Restore', 'Backup & restore specification')
- backup.add_argument('-i', '--restore-file',
- metavar='',
- dest='restorefile',
- default=DEFAULTS['backup']['backupfile'],
- help="file to restore configuration from (default: {}). Replacements: @v=firmware version from config, @f=device friendly name from config, @h=device hostname from config, @H=device hostname from device (-d arg only)".format(DEFAULTS['backup']['restorefile']))
- backup.add_argument('-o', '--backup-file',
- metavar='',
- dest='backupfile',
- default=DEFAULTS['backup']['backupfile'],
- help="file to backup configuration to (default: {}). Replacements: @v=firmware version from config, @f=device friendly name from config, @h=device hostname from config, @H=device hostname from device (-d arg only)".format(DEFAULTS['backup']['backupfile']))
- backup_file_formats = ['json', 'bin', 'dmp']
- backup.add_argument('-t', '--backup-type',
- metavar='|'.join(backup_file_formats),
- dest='backupfileformat',
- choices=backup_file_formats,
- default=DEFAULTS['backup']['backupfileformat'],
- help="backup filetype (default: '{}')".format(DEFAULTS['backup']['backupfileformat']) )
- backup.add_argument('-E', '--extension',
- dest='extension',
- action='store_true',
- default=DEFAULTS['backup']['extension'],
- help="append filetype extension for -i and -o filename{}".format(' (default)' if DEFAULTS['backup']['extension'] else '') )
- backup.add_argument('-e', '--no-extension',
- dest='extension',
- action='store_false',
- default=DEFAULTS['backup']['extension'],
- help="do not append filetype extension, use -i and -o filename as passed{}".format(' (default)' if not DEFAULTS['backup']['extension'] else '') )
- backup.add_argument('-F', '--force-restore',
- dest='forcerestore',
- action='store_true',
- default=DEFAULTS['backup']['forcerestore'],
- help="force restore even configuration is identical{}".format(' (default)' if DEFAULTS['backup']['forcerestore'] else '') )
-
- jsonformat = parser.add_argument_group('JSON output', 'JSON format specification')
- jsonformat.add_argument('--json-indent',
- metavar='',
- dest='jsonindent',
- type=int,
- default=DEFAULTS['jsonformat']['jsonindent'],
- help="pretty-printed JSON output using indent level (default: '{}'). -1 disables indent.".format(DEFAULTS['jsonformat']['jsonindent']) )
- jsonformat.add_argument('--json-compact',
- dest='jsoncompact',
- action='store_true',
- default=DEFAULTS['jsonformat']['jsoncompact'],
- help="compact JSON output by eliminate whitespace{}".format(' (default)' if DEFAULTS['jsonformat']['jsoncompact'] else '') )
-
- jsonformat.add_argument('--json-sort',
- dest='jsonsort',
- action='store_true',
- default=DEFAULTS['jsonformat']['jsonsort'],
- help=configargparse.SUPPRESS) #"sort json keywords{}".format(' (default)' if DEFAULTS['jsonformat']['jsonsort'] else '') )
- jsonformat.add_argument('--json-unsort',
- dest='jsonsort',
- action='store_false',
- default=DEFAULTS['jsonformat']['jsonsort'],
- help=configargparse.SUPPRESS) #"do not sort json keywords{}".format(' (default)' if not DEFAULTS['jsonformat']['jsonsort'] else '') )
-
- jsonformat.add_argument('--json-hide-pw',
- dest='jsonhidepw',
- action='store_true',
- default=DEFAULTS['jsonformat']['jsonhidepw'],
- help="hide passwords{}".format(' (default)' if DEFAULTS['jsonformat']['jsonhidepw'] else '') )
- jsonformat.add_argument('--json-show-pw', '--json-unhide-pw',
- dest='jsonhidepw',
- action='store_false',
- default=DEFAULTS['jsonformat']['jsonhidepw'],
- help="unhide passwords{}".format(' (default)' if not DEFAULTS['jsonformat']['jsonhidepw'] else '') )
-
- cmndformat = parser.add_argument_group('Tasmota command output', 'Tasmota command output format specification')
- cmndformat.add_argument('--cmnd-indent',
- metavar='',
- dest='cmndindent',
- type=int,
- default=DEFAULTS['cmndformat']['cmndindent'],
- help="Tasmota command grouping indent level (default: '{}'). 0 disables indent".format(DEFAULTS['cmndformat']['cmndindent']) )
- cmndformat.add_argument('--cmnd-groups',
- dest='cmndgroup',
- action='store_true',
- default=DEFAULTS['cmndformat']['cmndgroup'],
- help="group Tasmota commands{}".format(' (default)' if DEFAULTS['cmndformat']['cmndgroup'] else '') )
- cmndformat.add_argument('--cmnd-nogroups',
- dest='cmndgroup',
- action='store_false',
- default=DEFAULTS['cmndformat']['cmndgroup'],
- help="leave Tasmota commands ungrouped{}".format(' (default)' if not DEFAULTS['cmndformat']['cmndgroup'] else '') )
- cmndformat.add_argument('--cmnd-sort',
- dest='cmndsort',
- action='store_true',
- default=DEFAULTS['cmndformat']['cmndsort'],
- help="sort Tasmota commands{}".format(' (default)' if DEFAULTS['cmndformat']['cmndsort'] else '') )
- cmndformat.add_argument('--cmnd-unsort',
- dest='cmndsort',
- action='store_false',
- default=DEFAULTS['cmndformat']['cmndsort'],
- help="leave Tasmota commands unsorted{}".format(' (default)' if not DEFAULTS['cmndformat']['cmndsort'] else '') )
-
- common = parser.add_argument_group('Common', 'Optional arguments')
- common.add_argument('-c', '--config',
- metavar='',
- dest='configfile',
- default=DEFAULTS['common']['configfile'],
- is_config_file=True,
- help="program config file - can be used to set default command args (default: {})".format(DEFAULTS['common']['configfile']) )
-
- common.add_argument('-S', '--output',
- dest='output',
- action='store_true',
- default=DEFAULTS['common']['output'],
- help="display output regardsless of backup/restore usage{}".format(" (default)" if DEFAULTS['common']['output'] else " (default do not output on backup or restore usage)") )
- output_formats = ['json', 'cmnd','command']
- common.add_argument('-T', '--output-format',
- metavar='|'.join(output_formats),
- dest='outputformat',
- choices=output_formats,
- default=DEFAULTS['common']['outputformat'],
- help="display output format (default: '{}')".format(DEFAULTS['common']['outputformat']) )
- groups = GetGroupList(Settings[0][2])
- if '*' in groups:
- groups.remove('*')
- common.add_argument('-g', '--group',
- dest='filter',
- choices=groups,
- nargs='+',
- type=lambda s : s.title(),
- default=DEFAULTS['common']['filter'],
- help="limit data processing to command groups (default {})".format("no filter" if DEFAULTS['common']['filter'] == None else DEFAULTS['common']['filter']) )
- common.add_argument('--ignore-warnings',
- dest='ignorewarning',
- action='store_true',
- default=DEFAULTS['common']['ignorewarning'],
- help="do not exit on warnings{}. Not recommended, used by your own responsibility!".format(' (default)' if DEFAULTS['common']['ignorewarning'] else '') )
-
-
- info = parser.add_argument_group('Info','Extra information')
- info.add_argument('-D', '--debug',
- dest='debug',
- action='count',
- help=configargparse.SUPPRESS)
- info.add_argument('-h', '--help',
- dest='shorthelp',
- action='store_true',
- help='show usage help message and exit')
- info.add_argument("-H", "--full-help",
- action="help",
- help="show full help message and exit")
- info.add_argument('-v', '--verbose',
- dest='verbose',
- action='store_true',
- help='produce more output about what the program does')
- info.add_argument('-V', '--version',
- action='version',
- version=PROG)
-
- args = parser.parse_args()
-
- if debug(args) >= 1:
- print(parser.format_values(), file=sys.stderr)
- print("Settings:", file=sys.stderr)
- for k in args.__dict__:
- print(" "+str(k), "= ",eval('args.{}'.format(k)), file=sys.stderr)
- return args
-
-
-if __name__ == "__main__":
- args = ParseArgs()
- if args.shorthelp:
- ShortHelp()
-
- # check source args
- if args.device is not None and args.tasmotafile is not None:
- exit(ExitCode.ARGUMENT_ERROR, "Unable to select source, do not use -d and -f together",line=inspect.getlineno(inspect.currentframe()))
-
- # default no configuration available
- encode_cfg = None
-
- # pull config from Tasmota device
- if args.tasmotafile is not None:
- if args.verbose:
- message("Load data from file '{}'".format(args.tasmotafile), type_=LogType.INFO)
- encode_cfg = LoadTasmotaConfig(args.tasmotafile)
-
- # load config from Tasmota file
- if args.device is not None:
- if args.verbose:
- message("Load data from device '{}'".format(args.device), type_=LogType.INFO)
- encode_cfg = PullTasmotaConfig(args.device, args.port, username=args.username, password=args.password)
-
- if encode_cfg is None:
- # no config source given
- ShortHelp(False)
- print
- print(parser.epilog)
- sys.exit(ExitCode.OK)
-
- if len(encode_cfg) == 0:
- exit(ExitCode.FILE_READ_ERROR, "Unable to read configuration data from {} '{}'".format('device' if args.device is not None else 'file', \
- args.device if args.device is not None else args.tasmotafile) \
- ,line=inspect.getlineno(inspect.currentframe()) )
- # decrypt Tasmota config
- decode_cfg = DecryptEncrypt(encode_cfg)
-
- # decode into mappings dictionary
- configmapping = Bin2Mapping(decode_cfg)
- if args.verbose and 'version' in configmapping:
- message("{} '{}' is using Tasmota {}".format('File' if args.tasmotafile is not None else 'Device',
- args.tasmotafile if args.tasmotafile is not None else args.device,
- GetVersionStr(configmapping['version'])),
- type_=LogType.INFO)
-
- # backup to file
- if args.backupfile is not None:
- Backup(args.backupfile, args.backupfileformat, encode_cfg, decode_cfg, configmapping)
-
- # restore from file
- if args.restorefile is not None:
- Restore(args.restorefile, args.backupfileformat, encode_cfg, decode_cfg, configmapping)
-
- # json screen output
- if (args.backupfile is None and args.restorefile is None) or args.output:
- if args.outputformat == 'json':
- print(json.dumps(configmapping, sort_keys=args.jsonsort, indent=None if args.jsonindent<0 else args.jsonindent, separators=(',', ':') if args.jsoncompact else (', ', ': ') ))
-
- if args.outputformat == 'cmnd' or args.outputformat == 'command':
- tasmotacmnds = Mapping2Cmnd(decode_cfg, configmapping)
- OutputTasmotaCmnds(tasmotacmnds)
-
- sys.exit(exitcode)