412 Commits

Author SHA1 Message Date
J. Nick Koston
61f6df527e
Import cryptography early since importing openssl is not thread-safe (#111768) 2024-02-28 19:00:17 -05:00
J. Nick Koston
459f430575
Use eager tasks in a few more places in bootstrap (#111697)
I missed a few of the ones that run before we call
_async_set_up_integrations as I was too focused
on the timings of that function
2024-02-28 12:41:03 +01:00
Joakim Sørensen
f622ddef47
Move backup/* WS commands to the backup integration (#111636)
Revert "Revert "Move backup/* WS commands to the backup integration" (#111136)"

This reverts commit 8f834268955966d93bbc85b2ae7e86e6da46e37f.
2024-02-27 11:56:07 -05:00
J. Nick Koston
f08122084b
Reduce bootstrap code (#111450)
* Reduce bootstrap code

Adds a SETUP_ORDER const dict which will be used
for future refactoring

* tweak

* fix bad merge
2024-02-26 23:58:16 -05:00
J. Nick Koston
de48ad5931
Create bootstrap tasks eagerly (#111497)
We can avoid one event loop iteration to start the tasks here
2024-02-26 23:56:16 -05:00
J. Nick Koston
145ce183d0
Create component setup tasks eagerly in bootstrap (#111511)
This saves waiting one iteration of the event loop
2024-02-26 23:45:55 -05:00
Joakim Sørensen
8f83426895
Revert "Move backup/* WS commands to the backup integration" (#111136)
Revert "Move backup/* WS commands to the backup integration (#110651)"

This reverts commit ec4e6c3a7435400cf1a9afdf2c84165be957f294.
2024-02-22 16:08:30 +01:00
Joakim Sørensen
ec4e6c3a74
Move backup/* WS commands to the backup integration (#110651)
* Move backup/* WS commands to the backup integration

* Call correct command

* Use debug for logging

* Remove assertion of hass.data for setup test

* parametrize token fixture
2024-02-22 10:25:38 +01:00
Franck Nijhof
e0a9dcd996
Add label registry (#110821) 2024-02-19 11:59:08 +01:00
J. Nick Koston
3633700d97
Fix sending multiple empty integration timing in bootstrap (#110872) 2024-02-18 19:53:05 +01:00
J. Nick Koston
16653ff5d0
Group loading of translations for integrations to reduce executor jobs at startup (#110674) 2024-02-17 21:08:55 -06:00
J. Nick Koston
def6c5c21c
Refactor integration startup time tracking to reduce overhead (#110136)
* Refactor integration startup time tracking to reduce overhead

- Use monotonic time for watching integration startup time as it avoids incorrect values if time moves backwards because of ntp during startup and reduces many time conversions since we want durations in seconds and not local time

- Use loop scheduling instead of a task

- Moves all the dispatcher logic into the new _WatchPendingSetups

* websocket as well

* tweaks

* simplify logic

* preserve logic

* preserve logic

* lint

* adjust
2024-02-17 21:47:55 -05:00
Franck Nijhof
4570eed6f6
Add floor registry (#110741)
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2024-02-17 21:21:15 +01:00
J. Nick Koston
1260c5a909
Speed up bootstrap by preloading manifests for base platforms (#110130) 2024-02-16 09:35:46 -06:00
Piotr Machowski
a2f4e99994
Add state_translated function to jinja templates (#96906)
* Add state_translated jinja function

* Add tests for load_state_translations_to_cache and get_cached_translations

* Cleanup state_translated template

* Add tests for state_translated jinja function

* Apply black formatting

* Improve code quality

* Apply suggestions from code review

Co-authored-by: Erik Montnemery <erik@montnemery.com>

* Apply suggestions from code review

* Prevent invalid components from loading translations

* Refactor loading translations to cache

* Adjust code issues

* Update homeassistant/helpers/translation.py

Co-authored-by: Erik Montnemery <erik@montnemery.com>

* Refactor listeners that trigger translation loading

* Apply suggestions from code review

Co-authored-by: Erik Montnemery <erik@montnemery.com>

* Apply suggestions from code review

* Adjust invalid function calls, fix code styling

* Adjust code quality

* Extract async_translate_state function

* Apply suggestions from code review

Co-authored-by: Erik Montnemery <erik@montnemery.com>

* Apply suggestions from code review

* Fix tests

* Fix tests

---------

Co-authored-by: Piotr Machowski <PiotrMachowski@users.noreply.github.com>
Co-authored-by: Erik Montnemery <erik@montnemery.com>
2024-02-10 10:47:56 +01:00
Marc Mueller
cd0ee98dba
Use builtin TimeoutError [core + helpers] (#109684) 2024-02-05 12:09:54 +01:00
Franck Nijhof
80207835d7
Move core fundamental components into bootstrap (#105560)
Co-authored-by: Erik <erik@montnemery.com>
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2024-01-22 20:09:48 +01:00
J. Nick Koston
69307374f4
Signficantly reduce executor contention during bootstrap (#107312)
* Signficantly reduce executor contention during bootstrap

At startup we have a thundering herd wanting to use the executor
to load manifiest.json. Since we know which integrations we are
about to load in each resolver step, group the manifest loads
into single executor jobs by calling async_get_integrations on
the deps of the integrations after they are resolved.

In practice this reduced the number of executor jobs
by 80% during bootstrap

* merge

* naming

* tweak

* tweak

* not enough contention to be worth it there

* refactor to avoid waiting

* refactor to avoid waiting

* tweaks

* tweaks

* tweak

* background is fine

* comment
2024-01-07 22:55:40 -05:00
Erik Montnemery
3bcc6194ef
Add domain key config validation (#104242)
* Drop use of regex in helpers.extract_domain_configs

* Update test

* Revert test update

* Add domain_from_config_key helper

* Add validator

* Address review comment

* Update snapshots

* Inline domain_from_config_key in validator
2023-12-05 15:07:32 +01:00
Jan Bouwhuis
9962301b42
Do not notify config errors during logging (#104466) 2023-11-24 21:34:09 +01:00
Erik Montnemery
654c4b6e35
Use core domain constant in bootstrap (#104061) 2023-11-16 11:26:57 +01:00
Erik Montnemery
5b37096b5f
Refactor config.async_log_exception (#104034)
* Refactor config.async_log_exception

* Improve test coverage

* Make functions public
2023-11-15 19:09:49 +01:00
Marc Mueller
a70ec64408
Update mypy to 1.7.0 (#103800) 2023-11-11 23:31:04 +01:00
Erik Montnemery
97cc05d0b4
Make it possible to restart core in safe mode (#102606) 2023-10-24 14:47:58 +02:00
Erik Montnemery
b953f2998c
Rename the safe_mode integration to recovery_mode (#102581)
* Rename safe mode integration to recovery mode

* Update code
2023-10-24 09:11:14 +02:00
Erik Montnemery
c481fdb7d0
Rename safe mode to recovery mode (#102580) 2023-10-23 20:33:08 +02:00
Erik Montnemery
91faa53843
Don't allow hass.config.config_dir to be None (#98442) 2023-08-16 13:00:14 +02:00
Erik Montnemery
3b9d6f2dde
Add setup function to the component loader (#98148)
* Add setup function to the component loader

* Update test

* Setup the loader in safe mode and in check_config script
2023-08-15 10:59:42 +02:00
Jan-Philipp Benecke
99ee4e8a42
Set httpx log level to warning (#94217)
Set log level of httpx to warning
2023-06-08 09:39:06 +02:00
J. Nick Koston
fba826ae9e
Migrate restore_state helper to use registry loading pattern (#93773)
* Migrate restore_state helper to use registry loading pattern

As more entities have started using restore_state over time, it
has become a startup bottleneck as each entity being added is
creating a task to load restore state data that is already loaded
since it is a singleton

We now use the same pattern as the registry helpers

* fix refactoring error -- guess I am tired

* fixes

* fix tests

* fix more

* fix more

* fix zha tests

* fix zha tests

* comments

* fix error

* add missing coverage

* s/DATA_RESTORE_STATE_TASK/DATA_RESTORE_STATE/g
2023-05-30 20:48:17 -05:00
Raman Gupta
24290e5d08
Add datetime platform (#81943)
Co-authored-by: Franck Nijhof <frenck@frenck.nl>
Co-authored-by: Franck Nijhof <git@frenck.dev>
2023-05-29 23:24:15 +02:00
Ville Skyttä
cb22154b36
Capture warnings.warn and friends messages in logs (#85875)
The default behavior of these warnings is to go to stderr, which in
some setups goes easily unnoticed. For example in Docker based ones,
they end up only in the container logs, and not e.g. in the HA log.

Capture these to make them available in logs where other such messages
are, and to make them subject to filtering as usual.
https://docs.python.org/3/library/logging.html#logging.captureWarnings
2023-05-18 21:41:13 +03:00
Jan Bouwhuis
da26b0a930
Ensure dependencies are awaited correctly when setting up integrations (#91454)
* Do not wait

* Correct tests

* Manage after dependencies stage 1

* test bootstrap dependencies

* Assert log the dependenciy is waited for

* Improve docstrings

* Assert outside callback

* Patch async_get_integrations

* Revert changes made to snips integration

* Undo changes to mqtt_statestream
2023-04-21 08:33:50 +02:00
J. Nick Koston
17719663f0
Fix memory churn in state templates (#90685)
* Fix memory churn in state templates

The LRU for state templates was limited to 512 states. As soon
as it was exaused, system performance would tank as each template
that iterated all states would have to create and GC any state
> 512

* does it scale?

* avoid copy on all

* comment

* preen

* cover

* cover

* comments

* comments

* comments

* preen

* preen
2023-04-02 20:51:25 -04:00
Erik Montnemery
5bc9545b81
Rename custom_jinja to custom_templates (#90473)
Co-authored-by: Franck Nijhof <git@frenck.dev>
2023-03-29 21:58:25 +02:00
J. Nick Koston
f60e9c71a2
Make bootstrap cancelation safe (#90420) 2023-03-28 11:22:41 -10:00
David Poll
7284af6a3e
Add an in-memory-preloading loader for Jinja imports (#88850)
* Adds a loader to enable jinja imports.

* Switch to in-memory

* Move loading custom_jinja off of the event loop

* Raise TemplateNotFound if template doesn't exist

* Fix docstring

* Adds a service to reload custom jinja

* Remove IO from test setup

* Improve coverage and small refactor

* Incorporate feedback and use .jinja extension

* Check the loaded sources in test.

* Incorporate PR feedback.

* Update homeassistant/helpers/template.py

Co-authored-by: Erik Montnemery <erik@montnemery.com>

---------

Co-authored-by: Erik Montnemery <erik@montnemery.com>
2023-03-13 11:00:05 +01:00
J. Nick Koston
11681f3f31
Pass a helpful name when creating common asyncio tasks in core (#89171) 2023-03-05 12:46:02 +01:00
Erik Montnemery
83e5bf7ae8
Use entity_sources to determine integration in recorder platforms (#88382) 2023-02-18 07:21:41 -06:00
Franck Nijhof
5e81d28116
Update black to 23.1.0 (#87188) 2023-02-02 18:35:24 +01:00
Franck Nijhof
79b52a2b41
Stricter pylint message control (#86154) 2023-01-20 13:47:55 +01:00
Franck Nijhof
64c2340fab
Core code styling improvements (#85963) 2023-01-15 23:00:51 +01:00
Franck Nijhof
b0cee0bc46
String formatting and max line length - Part 1 (#84390)
Co-authored-by: Erik Montnemery <erik@montnemery.com>
2022-12-22 10:12:50 +01:00
puddly
8c8994352d
Allow only specific packages to be skipped during startup dependency installation (#82758) 2022-11-30 08:38:52 +01:00
Erik Montnemery
e1338adf1a
Allow configuring country and language in core config (#81734)
* Allow configuring country and language in core config

* Add script for updating list of countries

* Use black for formatting

* Fix quoting

* Move country codes to a separate file

* Address review comments

* Add generated/countries.py

* Get default language from owner account

* Remove unused variable

* Add script to generate list of supported languages

* Add tests

* Fix stale docsring

* Use format_python_namespace

* Correct async_user_store

* Improve typing

* Fix with_store decorator

* Initialize language in core store migration

* Fix startup

* Tweak

* Apply suggestions from code review

Co-authored-by: Franck Nijhof <git@frenck.dev>

* Update storage.py

Co-authored-by: Franck Nijhof <git@frenck.dev>
2022-11-24 23:25:50 +01:00
Franck Nijhof
6a1bb8c421
Deprecate Python 3.9 (#82193) 2022-11-16 15:38:10 +01:00
Aarni Koskela
8038485ca4
Use partition instead of split where possible in core (#81806) 2022-11-15 21:45:48 +01:00
Charles Garwood
58d531841b
Fix typo SIGNAL_BOOTSTRAP_INTEGRATONS -> SIGNAL_BOOTSTRAP_INTEGRATIONS (#79970) 2022-10-09 17:06:28 -10:00
Erik Montnemery
dfed3ba75e
Move issue_registry to homeassistant.helpers (#77299)
* Move issue_registry to homeassistant.helpers

* Add backwards compatibility
2022-08-25 11:32:06 +02:00
Erik Montnemery
fd6ffef52f
Support non-live database migration (#72433)
* Support non-live database migration

* Tweak startup order, add test

* Address review comments

* Fix typo

* Clarify comment about promoting dependencies

* Tweak

* Fix merge mistake

* Fix some tests

* Fix additional test

* Fix additional test

* Adjust tests

* Improve test coverage
2022-07-22 15:11:34 +02:00