J. Nick Koston
1bd9767d8c
Handle InnoDB deadlocks during migration ( #89073 )
...
* Handle slow InnoDB rollback when encountering duplicates during migration
fixes #89069
* adjust
* fix mock
* tests
* return on success
2023-03-03 22:00:13 -05:00
epenet
f2b736fad0
Adjust entity registry access in core platforms ( #88944 )
...
* Adjust entity registry access in platforms
* Adjust more core components
2023-03-01 08:02:16 +01:00
J. Nick Koston
9c82cfecca
Avoid asking recorder platforms for list_statistic_ids when already complete ( #88495 )
...
* Avoid asking recorder platforms for list_statistic_ids when already complete
If we already had all the data needed for list_statistic_ids, we would
still query recorder platforms and throw away the results
* Update homeassistant/components/recorder/statistics.py
2023-02-20 12:39:42 -05:00
J. Nick Koston
c2b770bcb9
Load pending state attributes and event data ids at startup ( #88444 )
...
* Load pending state attributes and event data ids at startup
Since we queue all events to be processed after startup
we can have a thundering herd of queries to prime the
LRUs of event data and state attributes ids. Since we
know we are about to process a chunk of events we can
fetch all the ids in two queries
* lru
* fix hang
* Fix recorder LRU being destroyed if event session is reopened
We would clear the LRU in _close_event_session but
it would never get replaced with an LRU again so
it would leak memory if the event session is reopened
* Fix recorder LRU being destroyed if event session is reopened
We would clear the LRU in _close_event_session but
it would never get replaced with an LRU again so
it would leak memory if the event session is reopened
* cleanup
2023-02-19 21:26:38 -05:00
J. Nick Koston
0b311c8c7f
Fix statistics_at_time query not using index ( #88460 )
...
* Fix statistics_at_time query not using index
fixes #82411
* fix refactoring error
* fix query so sqlalc does not get confused
* split it
* write as subquery
* reduce
* cleanup
* reduce
* Revert "reduce"
This reverts commit 43b4b55778207d87e3f24543a876ae73e4a7107a.
2023-02-19 21:07:31 -05:00
J. Nick Koston
d4376b2041
Use a single session to compile statistics ( #88405 )
...
We had two session_scope contexts in compile_statistics which causes
a commit to happen twice during the compile.
2023-02-18 20:23:47 -05:00
J. Nick Koston
5fe8829cf6
Speed up compiling hourly statistics ( #88225 )
2023-02-16 12:55:25 -06:00
J. Nick Koston
6c542bd314
Speed up formatting statistics data ( #88228 )
...
* Speed up fetching statistics by using attrgetter
Uses the faster native code vs a lambda
4aeae28671/Modules/_operator.c (L1406)
2023-02-16 12:34:20 -05:00
J. Nick Koston
32a6280139
Optimize _sorted_statistics_to_dict to avoid checking which types each loop ( #87901 )
...
* Optimize _sorted_statistics_to_dict to avoid checking which types each loop
* tweak
* avoid some more dict lookups
2023-02-11 23:21:40 -05:00
J. Nick Koston
86a93e9fce
Optimize fetching statistics by avoiding recalculating time boundaries ( #87859 )
...
* predictive
* fix conversion error
* fix conversion error
* fix conversion error
* convert day to use new algo
* reduce
2023-02-11 11:41:57 -05:00
J. Nick Koston
c35661947a
Optimize fetching statistics by avoiding timestamp conversions ( #87845 )
...
* Optimize fetching statistics by avoiding timestamp conversions
* remove accidential unrelated change
* fix test
* recreate so we handle timezone changes
2023-02-10 11:26:46 -05:00
J. Nick Koston
ea356ad260
Optimize fetching weekly/monthly/yearly statistics ( #87747 )
...
* Optimize fetching statistics
* speed up
* avoid double groupby
* avoid another loop
* tweak flow
* fixes
* tweak
* avoid a new dt object in the cache for week/month
* avoid a new dt object in the cache for week/month
2023-02-09 21:22:32 -05:00
J. Nick Koston
abf0c87e40
Migrate statistics to use timestamp columns ( #87321 )
2023-02-09 12:24:19 -06:00
Erik Montnemery
94519de8dd
Upgrade SQLAlchemy to 2.0.2 ( #86436 )
...
Co-authored-by: J. Nick Koston <nick@koston.org>
2023-02-08 08:17:32 -06:00
epenet
6806e85eab
Add missing converters to recorder statistics ( #87137 )
2023-02-02 09:44:26 +01:00
Franck Nijhof
a79885ceaf
Enable Ruff SIM117 ( #86783 )
2023-01-27 11:52:49 +01:00
Franck Nijhof
c5dedb7a79
Code styling tweaks to the recorder integration ( #86030 )
2023-01-16 19:51:11 +01:00
Erik Montnemery
b0d4b73874
Add unitless unit converter ( #85694 )
...
* Add unitless unit converter
* Adjust type hints
* Adjust tests
* Rename to UnitlessRatioConverter
2023-01-12 09:20:00 +01:00
Franck Nijhof
f39f3b612a
String formatting and max line length - Part 5 ( #84501 )
...
Co-authored-by: jjlawren <jjlawren@users.noreply.github.com>
2022-12-23 16:43:17 +01:00
Franck Nijhof
94755a5773
String formatting and max line length - Part 4 ( #84445 )
...
Co-authored-by: jjlawren <jjlawren@users.noreply.github.com>
2022-12-23 13:27:27 +01:00
Erik Montnemery
587381440f
Fix repairing datetime precision for PostgreSQL ( #83351 )
2022-12-05 22:33:06 -05:00
Erik Montnemery
f869ce9d06
Validate common statistics DB schema errors on start ( #79707 )
...
* Validate common statistics db schema errors on start
* Fix test
* Add tests
* Adjust tests
* Disable statistics schema validation in tests
* Update after rebase
2022-11-29 10:16:08 +01:00
Erik Montnemery
58b3a00b16
Remove unnecessary DB access from statistic_during_period ( #82871 )
2022-11-28 13:48:06 -05:00
Erik Montnemery
a038314d8b
Add display unit to WS recorder/get_statistics_metadata ( #82870 )
2022-11-28 13:46:57 -05:00
Erik Montnemery
b94e1e9ef8
Fire events when long term statistics is updated ( #82492 )
...
* Fire events when long term statistics is updated
* Allow the new events to be subscribed to by anyone
* Address review comments
* Finish renaming events
* Finish renaming events
* Fix do_adhoc_statistics
* Adjust tests
* Adjust tests
2022-11-24 22:01:36 +01:00
Erik Montnemery
607a0e7697
Reduce size of get_statistics_during_period WS API response ( #82131 )
2022-11-16 17:36:30 +01:00
Erik Montnemery
9b8f94363c
Fix statistic_during_period for data with holes ( #81847 )
2022-11-16 12:46:29 +01:00
Erik Montnemery
68346599d2
Add WS API recorder/statistic_during_period ( #80663 )
2022-10-27 15:51:09 -04:00
Michael
8175dab7ab
Add week period to recorder statistics api ( #80784 )
...
* add week period to get statistics api
* add test
2022-10-25 20:07:28 +02:00
Erik Montnemery
b42e26fbdd
Use SupportedDialect enum in recorder ( #80319 )
2022-10-14 08:45:57 -04:00
Erik Montnemery
2e261d5dc2
Allow specifying the target table when importing statistics ( #80230 )
...
Allow specifying the table when importing statistics
2022-10-14 08:32:19 +02:00
Erik Montnemery
577f7904b5
Minor improvements of recorder typing ( #80165 )
...
* Minor improvements of recorder typing
* Only allow specifying statistic_ids as lists
2022-10-12 14:59:10 +02:00
Erik Montnemery
92ca95ca81
Fix preserving long term statistics when entity_id is changed ( #79556 )
2022-10-03 21:13:48 -04:00
Erik Montnemery
a2e3978d53
Don't normalize units of long term statistics ( #79320 )
...
* Don't normalize units of long term statistics
* Update statistics.py
2022-10-03 21:42:44 +02:00
Erik Montnemery
2de273500e
Remove state_unit_of_measurement from metadata DB table ( #79370 )
...
* Remove state_unit_of_measurement from metadata DB table
* Adjust test
2022-10-01 18:55:00 +02:00
Erik Montnemery
bc2dffabc4
Improve naming of units used in statistics ( #79276 )
2022-09-30 08:38:44 +02:00
epenet
5438552d4a
Cleanup unit conversion ( #79197 )
...
* Move UNIT_RATIO to BaseUnitConverter
* Make UNIT_CONVERSION private
* Remove STATISTIC_UNIT_TO_UNIT_CLASS constant
* Cleanup websocket_api
* Imrpove valid_units check
2022-09-28 13:49:46 +02:00
epenet
5389ff3253
Add new weight device class ( #79185 )
2022-09-28 12:13:49 +02:00
epenet
7c448416e1
Add speed to SensorDeviceClass ( #77953 )
...
* Add speed to SensorDeviceClass
* Adjust recorder
* Adjust tests
* Adjust sensor UNIT_CONVERTERS
* Add tests
* Add websocket tests
* Update strings.json
2022-09-27 17:19:34 +01:00
epenet
bfcc18e5b8
Add distance to SensorDeviceClass ( #77951 )
...
* Add distance to SensorDeviceClass
* Adjust recorder
* Adjust tests
* Adjust recorder
* Update __init__.py
* Update test_websocket_api.py
* Update test_websocket_api.py
* Update test_websocket_api.py
* Update strings.json
* Fix tests
* Adjust docstring
2022-09-27 15:34:00 +01:00
Erik Montnemery
2167cf540d
Drop some unused constants from recorder ( #79138 )
2022-09-27 10:52:16 +02:00
Erik Montnemery
c52d0f7495
Support converting statistics to another unit ( #79117 )
2022-09-27 08:44:58 +02:00
epenet
83b426deb0
Adjust normalization routines in recorder statistics ( #78966 )
2022-09-23 08:27:27 -04:00
epenet
ddf56baf7a
Move temperature utility to unit_conversion ( #78960 )
2022-09-22 18:31:50 +02:00
epenet
c8491c4404
Move volume utility to unit_conversion ( #78955 )
...
* Move volume utility to unit_conversion
* Split tests
2022-09-22 17:49:45 +02:00
epenet
523d8d246b
Move pressure utility to unit_conversion ( #78953 )
2022-09-22 16:44:09 +02:00
epenet
0767cdd935
Move energy and power utilites to unit_conversion ( #78950 )
...
* Move energy and power utilites to unit_conversion
* Move tests
2022-09-22 15:39:49 +02:00
epenet
6002377d4f
Convert UnitConverter protocol to a class ( #78934 )
...
* Convert UnitConverter protocl to a class
* Remove logic change
* Use TypeVar
* Remove NORMALIZED_UNIT from pressure
* Reduce size of PR
* Reduce some more
* Once more
* Once more
* Remove DEVICE_CLASS
2022-09-22 14:15:22 +02:00
epenet
713fb874a8
Add NORMALISED_UNIT to UnitConverter ( #78920 )
...
* Add NORMALISED_UNIT to UnitConverter
* Adjust statistics
* Rename
2022-09-22 08:50:08 +02:00
epenet
39315b7fe3
Introduce UnitConverter protocol ( #78888 )
...
* Introduce ConversionUtility
* Use ConversionUtility in number
* Use ConversionUtility in sensor
* Use ConversionUtility in sensor recorder
* Add normalise to ConversionUtility
* Revert changes to recorder.py
* Reduce size of PR
* Adjust recorder statistics
* Rename variable
* Rename
* Apply suggestion
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* Apply suggestion
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* Apply suggestion
Co-authored-by: Erik Montnemery <erik@montnemery.com>
Co-authored-by: Erik Montnemery <erik@montnemery.com>
2022-09-22 07:18:00 +02:00
