1
0
mirror of https://github.com/home-assistant/core.git synced 2025-08-18 01:40:11 +00:00
Commit Graph

74 Commits

Author SHA1 Message Date
Joost Lekkerkerker
6bb4e7d62c Bump ruff to 0.3.4 ()
Co-authored-by: Sid <27780930+autinerd@users.noreply.github.com>
Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
2024-03-26 00:02:16 +01:00
Sid
d5fd005db8 Add ruff rule PIE808 () 2024-03-17 09:56:26 +01:00
Marc Mueller
2ef2172b01 Convert test helpers to get hass instance to contextmanagers ()
* Convert get_test_home_assistant helper to contextmanager

* Convert async_test_home_assistant helper to contextmanager

* Move timezone reset to async_test_home_assistant helper
2024-02-11 21:23:51 +01:00
J. Nick Koston
d8c6534aff Refactor recorder for common event case ()
Almost 99% of items that are put into the recorder queue are
Events. Avoid wrapping them in tasks since we have to unwrap
them right away and its must faster to check for both RecorderTask
and Events since events are the common case.
2024-01-07 22:31:40 -05:00
J. Nick Koston
2ec1359063 Dynamically size recorder max backlog based on available memory ()
Co-authored-by: Franck Nijhof <git@frenck.dev>
2023-04-18 12:35:49 +02:00
J. Nick Koston
49079691d4 Reduce overhead of legacy database columns on new installs ()
* Reduce overhead of legacy database columns on new installs

* Reduce overhead of legacy database columns on new installs

* Reduce overhead of legacy database columns on new installs

* Reduce overhead of legacy database columns on new installs

* not working as expected

* override the type compiler

* override the type compiler

* override the type compiler

* override the type compiler

* Apply suggestions from code review

* pgsql char1

* make entity filter test setup with old schema

* fix some more tests that were mutating state

* fix some more tests that were mutating state

* fix some more tests that were mutating state

* fix more dbstate mutations

* add shim for older tests

* split migration tests

* add coverage for purging legacy data

* tweak

* more fixes

* drop some legacy

* fix another test

* fix a few more

* add casts for postgresql in case someone deletes the schema changes table

* dry

* dry

* dry
2023-04-10 10:08:46 -04:00
J. Nick Koston
a2efe2445a Fix migration when encountering a NULL entity_id/event_type ()
* Fix migration when encountering a NULL entity_id/event_type

reported in #beta on discord

* simplify
2023-03-30 20:54:13 -04:00
J. Nick Koston
6b0c98045e Handle garbage in the context_id column during migration ()
* Handle garbage in the context_id column during migration

* Update homeassistant/components/recorder/migration.py

* lint
2023-03-30 20:53:47 -04:00
J. Nick Koston
31c988c4f0 Fix index not being dropped on postgresql databases with a schema prefix ()
* Fix index not being dropped on postgresql databases with a schema prefix

Added logging in case index drops fail so we can tell
why in the future

* coverage
2023-03-22 21:03:26 -04:00
J. Nick Koston
96225bb287 Rename recorder run_history to table_managers.recorder_runs_manager () 2023-03-21 18:38:33 -10:00
J. Nick Koston
a244749712 Make StatesMetaManager thread-safe when an entity_id is fully deleted from the database and than re-added ()
* refactor to make StatesMetaManager threadsafe

* refactor to make StatesMetaManager threadsafe

* refactor to make StatesMetaManager threadsafe

* refactor to make StatesMetaManager threadsafe

* reduce

* comments
2023-03-15 08:54:02 -04:00
J. Nick Koston
b9ac6b4a7c Improve reliability of context id migration ()
* Split context id migration into states and events tasks

Since events can finish much earlier than states we
would keep looking at the table because states as not
done. Make them seperate tasks

* add retry dec

* fix migration happening twice

* another case
2023-03-12 21:41:48 -04:00
J. Nick Koston
c41f91be89 Deduplicate entity_id in the states table () 2023-03-12 10:01:58 -10:00
J. Nick Koston
8bd43760b6 Deduplicate event_types in the events table ()
* Deduplicate event_types in the events table

* Deduplicate event_types in the events table

* more fixes

* adjust

* adjust

* fix product

* fix tests

* adjust

* migrate

* migrate

* migrate

* more test fixes

* more test fixes

* fix

* migration test

* adjust

* speed up

* fix index

* fix more tests

* handle db failure

* preload

* tweak

* adjust

* fix stale docs strings, remove dead code

* refactor

* fix slow tests

* coverage

* self join to resolve query performance

* fix typo

* no need for quiet

* no need to drop index already dropped

* remove index that will never be used

* drop index sooner as we no longer use it

* Revert "remove index that will never be used"

This reverts commit 461aad2c52.

* typo
2023-03-11 14:54:55 -05:00
J. Nick Koston
170a13302c Reduce overhead to store context ids in the database () 2023-03-08 14:51:45 -10:00
J. Nick Koston
1bd9767d8c Handle InnoDB deadlocks during migration ()
* Handle slow InnoDB rollback when encountering duplicates during migration

fixes 

* adjust

* fix mock

* tests

* return on success
2023-03-03 22:00:13 -05:00
J. Nick Koston
c2b770bcb9 Load pending state attributes and event data ids at startup ()
* Load pending state attributes and event data ids at startup

Since we queue all events to be processed after startup
we can have a thundering herd of queries to prime the
LRUs of event data and state attributes ids. Since we
know we are about to process a chunk of events we can
fetch all the ids in two queries

* lru

* fix hang

* Fix recorder LRU being destroyed if event session is reopened

We would clear the LRU in _close_event_session but
it would never get replaced with an LRU again so
it would leak memory if the event session is reopened

* Fix recorder LRU being destroyed if event session is reopened

We would clear the LRU in _close_event_session but
it would never get replaced with an LRU again so
it would leak memory if the event session is reopened

* cleanup
2023-02-19 21:26:38 -05:00
epenet
185cd61cbd Add type hints to integration tests (recorder) () 2023-02-17 18:50:09 +01:00
Franck Nijhof
ed79265843 Enable Ruff PT006 ()
* Enable Ruff PT006

* Adjust existing cases

* Fix tests

* Remove unneeded parentheses
2023-02-15 14:09:50 +01:00
J. Nick Koston
cbaf4764e7 Ensure MySQL tests cleanup connections and raise an exception if they do not ()
Co-authored-by: epenet <6771947+epenet@users.noreply.github.com>
Co-authored-by: Erik <erik@montnemery.com>
2023-02-09 16:27:53 -06:00
J. Nick Koston
abf0c87e40 Migrate statistics to use timestamp columns () 2023-02-09 12:24:19 -06:00
Erik Montnemery
2fae82a9ef Really ensure recorder test fixture is setup before hass fixture () 2023-02-09 11:43:14 -06:00
J. Nick Koston
f4a81a8b5b Disable test_migration_in_progress on MySQL ()
This test was recently enabled on MySQL in
https://github.com/home-assistant/core/pull/87753

Since it the migration is still in progress in the background
when the test ends, it causes deadlock with InnoDB when its dropped
out from under it
2023-02-09 15:27:27 +01:00
Erik Montnemery
b0f5bdd504 Improve migration tests to use MariaDB / PostgreSQL database () 2023-02-09 08:42:32 +01:00
epenet
ba85fdcd61 Add type hints to integration tests (q-s) () 2023-02-08 08:12:54 -08:00
epenet
59ca7780fa Add typing to tests with single hass argument () 2023-02-07 15:01:16 +01:00
epenet
4142f0d15d Add return type to tests without arguments ()
* Add return type to tests without arguments

* Black

* Cancel fixture amends
2023-02-07 14:20:06 +01:00
Franck Nijhof
5e81d28116 Update black to 23.1.0 () 2023-02-02 18:35:24 +01:00
epenet
30bf0634fe Add per-file-ignore to pylint () 2023-01-22 17:26:24 +01:00
Erik Montnemery
466c4656ca Refactor recorder migration ()
* Refactor recorder migration

* Improve test coverage
2022-10-13 08:11:54 +02:00
Erik
4a1c40f09b Revert "Refactor recorder migration"
This reverts commit 69e10e5982.
2022-10-12 15:12:12 +02:00
Erik
69e10e5982 Refactor recorder migration 2022-10-12 15:00:28 +02:00
Erik Montnemery
2de273500e Remove state_unit_of_measurement from metadata DB table ()
* Remove state_unit_of_measurement from metadata DB table

* Adjust test
2022-10-01 18:55:00 +02:00
Erik Montnemery
dd20a7ea62 Display statistics in the source's unit () 2022-09-15 12:01:24 -04:00
Erik Montnemery
0a1fd36e03 Catch up with statistics after DB migration is done ()
* Catch up with statistics after DB migration is done

* Don't access the database from the event loop

* Fix deadlocking test

* Fix test
2022-09-15 07:40:56 +02:00
Erik Montnemery
fd6ffef52f Support non-live database migration ()
* Support non-live database migration

* Tweak startup order, add test

* Address review comments

* Fix typo

* Clarify comment about promoting dependencies

* Tweak

* Fix merge mistake

* Fix some tests

* Fix additional test

* Fix additional test

* Adjust tests

* Improve test coverage
2022-07-22 15:11:34 +02:00
Erik Montnemery
606d544157 Use recorder get_instance function to improve typing () 2022-07-22 11:58:26 +02:00
Erik Montnemery
5f2b4001f3 Separate recorder database schema from other classes ()
* Separate recorder database schema from other classes

* fix logbook imports

* migrate new tests

* few more

* last one

* fix merge

Co-authored-by: J. Nick Koston <nick@koston.org>
2022-06-07 14:41:43 +02:00
J. Nick Koston
eb77f8db85 Complete strict typing for recorder ()
* Complete strict typing for recorder

* update tests

* Update tests/components/recorder/test_migrate.py

Co-authored-by: Martin Hjelmare <marhje52@gmail.com>

* Update tests/components/recorder/test_migrate.py

Co-authored-by: Martin Hjelmare <marhje52@gmail.com>

* Remove the asserts

* remove ignore comments

Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2022-05-04 12:22:50 -05:00
J. Nick Koston
13ce0a7d6a Fix history using pre v25 queries during v26 migration () 2022-05-04 08:56:50 -07:00
J. Nick Koston
29bda196b5 Break apart recorder into tasks and core modules () 2022-05-02 21:53:56 -07:00
J. Nick Koston
1c4a785fb3 Prevent autoflush from happening during attrs lookup () 2022-04-26 13:04:58 -07:00
J. Nick Koston
f073f17040 Refactor tracking of the recorder run history ()
Co-authored-by: Erik Montnemery <erik@montnemery.com>
2022-04-26 09:59:43 -10:00
J. Nick Koston
89e3a48751 Use database executor in migration test () 2022-04-26 07:42:28 -07:00
Erik Montnemery
28ebab9c5a Simplify waiting for recorder in tests () 2022-04-25 12:04:47 +02:00
J. Nick Koston
66f0a3816a Reduce memory pressure during database migration () 2022-04-07 18:29:31 -10:00
Erik Montnemery
949b0e1b65 Don't allow in-memory SQLite database () 2022-04-08 00:43:09 +02:00
J. Nick Koston
e911936a0d Remove direct usage of concurrent.futures from recorder () 2022-03-24 09:48:49 -10:00
Erik Montnemery
41ab12cb88 Don't use shared session during recorder migration () 2022-02-04 09:55:11 -08:00
Erik Montnemery
e5ee38d3b0 Adjust recorder tests according to () 2021-11-05 10:49:58 +01:00