629 Commits

Author SHA1 Message Date
Robert Resch
e3f2f30395
Add circular mean statistics and sensor state class MEASUREMENT_ANGLE (#138453)
* Add circular mean statistics

* fixes

* Add has_circular_mean and fix tests

* Fix mypy

* Rename to MEASUREMENT_ANGLE

* Fix kitchen_sink tests

* Fix sensor tests

* for testing only

* Revert ws command change

* Apply suggestions

* test only

* add custom handling for postgres

* fix recursion limit

* Check if column is already available

* Set default false and not nullable for has_circular_mean

* Proper fix to be backwards compatible

* Fix value is None

* Align with schema

* Remove has_circular_mean from test schemas as it's not required anymore

* fix wrong column type

* Use correct variable to reduce stats

* Add guard that the uom is matching a valid one from the state class

* Add some tests

* Fix tests again

* Use mean_type in StatisticsMetato difference between different mean type algorithms

* Fix leftovers

* Fix kitchen_sink tests

* Fix postgres

* Add circular mean test

* Add mean_type_changed stats issue

* Align the attributes with unit_changed

* Fix mean_type_change stats issue

* Add missing sensor recorder tests

* Add test_statistic_during_period_circular_mean

* Add mean_weight

* Add test_statistic_during_period_hole_circular_mean

* Use seperate migration step to null has_mean

* Typo ARITHMETIC

* Implement requested changes

* Implement requested changes

* Split into #141444

* Add StatisticMeanType.NONE and forbid that mean_type can be None

* Fix mean_type

* Implement requested changes

* Small leftover of latest StatisticMeanType changes
2025-03-26 13:15:58 +01:00
Erik Montnemery
cc5c8bf5e3
Make helpers.frame.report_usage work when called from any thread (#139836)
* Make helpers.frame.report_usage work when called from any thread

* Address review comments, update tests

* Add test

* Update test

* Update recorder test

* Update tests
2025-03-05 19:37:34 +01:00
J. Nick Koston
c9abe76023
Use multiple indexed group-by queries to get start time states for MySQL (#138786)
* tweaks

* mysql

* mysql

* Update homeassistant/components/recorder/history/modern.py

* Update homeassistant/components/recorder/history/modern.py

* Update homeassistant/components/recorder/const.py

* Update homeassistant/components/recorder/statistics.py

* Apply suggestions from code review

* mysql

* mysql

* cover

* make sure db is fully init on old schema

* fixes

* fixes

* coverage

* coverage

* coverage

* s/slow_dependant_subquery/slow_dependent_subquery/g

* reword

* comment that callers are responsible for staying under the limit

* comment that callers are responsible for staying under the limit

* switch to kwargs

* reduce branching complexity

* split stats query

* preen

* split tests

* split tests
2025-03-02 15:13:06 +01:00
Erik Montnemery
5cf56ec113
Adjust recorder backup platform tests (#139492) 2025-02-28 11:44:58 +01:00
Erik Montnemery
1be9836663
Fail recorder.backup.async_pre_backup if Home Assistant is not running (#139491)
Fail recorder.backup.async_pre_backup if hass is not running
2025-02-28 11:44:16 +01:00
Erik Montnemery
0c092f80c7
Add default_db_url flag to WS command recorder/info (#139333) 2025-02-26 14:09:38 +01:00
Erik Montnemery
6507955a14
Fix race in WS command recorder/info (#139177)
* Fix race in WS command recorder/info

* Add comment

* Remove unnecessary local import
2025-02-24 18:55:13 +01:00
J. Nick Koston
93b01a3bc3
Fix minimum schema version to run event_id_post_migration (#139014)
* Fix minimum version to run event_id_post_migration

The table rebuild to fix the foreign key constraint was added
in https://github.com/home-assistant/core/pull/120779 but the
schema version was not bumped so we need to make sure
any database that was created with schema 43 or older
still has the migration run as otherwise they will not
be able to purge the database with SQLite since each
delete in the events table will due a full table scan
of the states table to look for a foreign key that is
not there

fixes #138818

* Apply suggestions from code review

* Update homeassistant/components/recorder/migration.py

* Update homeassistant/components/recorder/migration.py

* Update homeassistant/components/recorder/const.py

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

* update tests, add more cover

* update tests, add more cover

* Update tests/components/recorder/test_migration_run_time_migrations_remember.py
2025-02-22 15:39:12 -05:00
Erik Montnemery
ff4f4111d0
Minor adjustment of recorder helper (#138941) 2025-02-20 16:28:39 +00:00
Erik Montnemery
fb57284561
Remove helper.recorder.async_wait_recorder (#138935) 2025-02-20 15:02:22 +00:00
Jan-Philipp Benecke
4ceced6405
Fix sqlalchemy deprecation warning that declarative_base has moved (#137360) 2025-02-04 15:31:05 -06:00
epenet
0e263aa427
Standardize homeassistant imports in full-CI tests (#136735) 2025-01-28 13:10:20 -05:00
Marc Mueller
f822fd82bb
Fix recorder fixture typing (#136174) 2025-01-21 18:18:05 -10:00
Joost Lekkerkerker
ef8b8fbbaa
Enable RUF023 (#135830) 2025-01-17 12:28:27 +01:00
epenet
02956f9a83
Improve formatting in component test files (#135267)
Improve formatting in test files
2025-01-10 10:53:45 +01:00
G Johansson
64752af4c2
Change minimum SQLite version to 3.40.1 (#135042)
Co-authored-by: J. Nick Koston <nick@koston.org>
2025-01-08 16:34:36 -10:00
Erik Montnemery
0bd22eabc7
Improve recorder schema migration error test (#134518) 2025-01-03 10:05:07 +01:00
Erik Montnemery
cf238cd8f7
Don't start recorder if a database from the future is used (#134467) 2025-01-02 18:56:23 +01:00
J. Nick Koston
942fbdedcf
Ensure all states have been migrated to use timestamps (#134007) 2024-12-26 07:48:55 -10:00
J. Nick Koston
43fab48d4e
Improve purge performance for PostgreSQL with large databases (#133699) 2024-12-21 10:53:15 +01:00
Erik Montnemery
a23b37114e
Improve recorder data migrator tests (#133628) 2024-12-20 07:41:14 -10:00
Erik Montnemery
dd215b3d5d
Revert "Revert "Simplify recorder RecorderRunsManager (#131785)"" (#133564)
Revert "Revert "Simplify recorder RecorderRunsManager" (#133201)"

This reverts commit 980b8a91e62c449fab558318573fa756818875a6.
2024-12-19 12:32:15 +01:00
Erik Montnemery
bb7abd037c
Revert "Revert "Improve recorder history queries (#131702)"" (#133561)
Revert "Revert "Improve recorder history queries (#131702)" (#133203)"

This reverts commit 74e4654c26177909e653921f27f838fd1366adc0.
2024-12-19 11:50:12 +01:00
J. Nick Koston
69a8d3f3c1
Revert "Optimize start time state queries for PostgreSQL" (#133555) 2024-12-18 23:01:58 -10:00
J. Nick Koston
99698ef95d
Optimize start time state queries for PostgreSQL (#133228) 2024-12-18 19:41:53 -10:00
Erik Montnemery
ff8bc763c3
Ensure indices needed by data migrators exist (#133367)
* Ensure indices needed by data migrators exist

* Update test

* Improve test

* Ignore index error on char(0) columns

* Adjust tests

* Address review comments

* Add comment motivating magic number
2024-12-18 21:29:52 +01:00
Erik Montnemery
d22668a166
Don't run recorder data migration on new databases (#133412)
* Don't run recorder data migration on new databases

* Add tests
2024-12-17 20:02:12 +01:00
Erik Montnemery
74e4654c26
Revert "Improve recorder history queries (#131702)" (#133203) 2024-12-15 12:28:32 +01:00
Erik Montnemery
980b8a91e6
Revert "Simplify recorder RecorderRunsManager" (#133201)
Revert "Simplify recorder RecorderRunsManager (#131785)"

This reverts commit cf0ee635077114961f6e508be56ce7620c718c18.
2024-12-14 14:21:19 +01:00
J. Nick Koston
f177336025
Add missing last_reported_timestamp to LazyState (#132761)
followup to #132752
2024-12-09 23:08:01 +01:00
J. Nick Koston
e4ba94f939
Fix LazyState compatibility with State under_cached_property change (#132752) 2024-12-09 21:41:08 +01:00
Erik Montnemery
552613d949
Remove support for live recorder data migration of event type IDs (#131826) 2024-12-06 21:08:08 +01:00
Erik Montnemery
d26d483a2f
Improve recorder util resolve_period (#132264) 2024-12-06 21:06:56 +01:00
Erik Montnemery
deab285db8
Improve tests of recorder util resolve_period (#132259) 2024-12-04 14:01:49 +01:00
Pete
a417d3dcf8
Fix recorder "year" period in leap year (#132167)
* FIX: make "year" period work in leap year

* Add test

* Set second and microsecond to non-zero in test start times

* FIX: better fix for leap year problem

* Revert "FIX: better fix for leap year problem"

This reverts commit 06aba46ec6a0a1e944c88fe99d9bc6181a73cc1c.

---------

Co-authored-by: Erik <erik@montnemery.com>
2024-12-04 13:21:10 +01:00
Erik Montnemery
1a714276cc
Remove support for live recorder data migration of entity IDs (#131952) 2024-12-03 14:43:33 -06:00
Erik Montnemery
cf0ee63507
Simplify recorder RecorderRunsManager (#131785) 2024-12-01 11:26:29 -06:00
J. Nick Koston
8878d0f0e1
Reduce time syscalls needed to insert new statistics (#131984) 2024-12-01 08:55:07 -06:00
Erik Montnemery
bbce183faf
Deprecate dt_util.utc_to_timestamp (#131787) 2024-11-28 17:00:20 +01:00
Erik Montnemery
d9832f8c3a
Rename constant in tests/components/recorder/test_migration_from_schema_32.py (#131819) 2024-11-28 13:26:58 +01:00
Erik Montnemery
381d5453b1
Improve recorder history queries (#131702)
* Improve recorder history queries

* Remove some comments

* Update StatesManager._oldest_ts when adding pending state

* Update after review

* Improve tests

* Improve post-purge logic

* Avoid calling dt_util.utc_to_timestamp in new code

---------

Co-authored-by: J. Nick Koston <nick@koston.org>
2024-11-27 21:12:42 +01:00
Erik Montnemery
44f90dca0c
Fix logic for purge of recorder runs (#130378)
* Fix logic for purge of recorder runs

* Make test more explicit

* Explicitly don't remove unclosed recorder runs in purge
2024-11-26 07:47:47 +01:00
Davin Kevin
f42386de44
Prevent endless loop in recorder when using a filter and there are no more states to purge (#126149)
Co-authored-by: J. Nick Koston <nick@koston.org>
2024-11-21 20:50:49 +01:00
Michael Arthur
d8549409f7
Area units and conversion between metric and US (#123563)
* area conversions

* start work on tests

* add number device class

* update unit conversions to utilise distance constants

* add area unit

* update test unit system

* update device condition and trigger

* update statistic unit converters

* further tests work WIP

* update test unit system

* add missing string translations

* fix websocket tests

* add deprecated notice

* add more missing strings and missing initialisation of unit system

* adjust icon and remove strings from scrape and random

* Fix acre to meters conversion

Co-authored-by: epenet <6771947+epenet@users.noreply.github.com>

* Tidy up valid units

Co-authored-by: epenet <6771947+epenet@users.noreply.github.com>

* fix ordering of area

* update order alphabetically

* fix broken test

* update test_init

* Update homeassistant/const.py

Co-authored-by: epenet <6771947+epenet@users.noreply.github.com>

* remove deprecated unit and fix alphabetical order

* change deprecation and add tests, change to millimeter conversion for inches

* fix order

* re-order defs alphabetically

* add measurement as well

* update icons

* fix up Deprecation of area square meters

* Update core integrations to UnitOfArea

* update test recorder tests

* unit system tests in alphabetical

* update snapshot

* rebuild

* revert alphabetization of functions

* other revert of alphabetical order

---------

Co-authored-by: epenet <6771947+epenet@users.noreply.github.com>
2024-11-21 16:10:44 +01:00
Erik Montnemery
db5cb6233c
Correct condition signalling non-live DB migration is in progress (#129464) 2024-10-29 12:26:52 -10:00
Marc Mueller
2de161ce0e
Fix mariadb recorder tests for Python 3.13 (#129303) 2024-10-29 09:17:47 +01:00
Erik Montnemery
d8b618f7c3
Remove support for live recorder data migration of context ids (#125309) 2024-10-26 07:19:03 +02:00
J. Nick Koston
db494de809
Restore __slots__ to core objects (#127441) 2024-10-03 22:23:47 +02:00
Erik Montnemery
3f1acdc9ec
Make recorder WS command recorder/clear_statistics wait (#127120) 2024-10-02 10:43:40 +02:00
Erik Montnemery
2a2af01d12
Make recorder WS command recorder/update_statistics_metadata wait (#127179) 2024-10-02 09:56:36 +02:00