Merge remote-tracking branch 'upstream/dev' into memory

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>

.ai/instructions.md

@@ -0,0 +1,222 @@
+# ESPHome AI Collaboration Guide
+
+This document provides essential context for AI models interacting with this project. Adhering to these guidelines will ensure consistency and maintain code quality.
+
+## 1. Project Overview & Purpose
+
+*   **Primary Goal:** ESPHome is a system to configure microcontrollers (like ESP32, ESP8266, RP2040, and LibreTiny-based chips) using simple yet powerful YAML configuration files. It generates C++ firmware that can be compiled and flashed to these devices, allowing users to control them remotely through home automation systems.
+*   **Business Domain:** Internet of Things (IoT), Home Automation.
+
+## 2. Core Technologies & Stack
+
+*   **Languages:** Python (>=3.10), C++ (gnu++20)
+*   **Frameworks & Runtimes:** PlatformIO, Arduino, ESP-IDF.
+*   **Build Systems:** PlatformIO is the primary build system. CMake is used as an alternative.
+*   **Configuration:** YAML.
+*   **Key Libraries/Dependencies:**
+    *   **Python:** `voluptuous` (for configuration validation), `PyYAML` (for parsing configuration files), `paho-mqtt` (for MQTT communication), `tornado` (for the web server), `aioesphomeapi` (for the native API).
+    *   **C++:** `ArduinoJson` (for JSON serialization/deserialization), `AsyncMqttClient-esphome` (for MQTT), `ESPAsyncWebServer` (for the web server).
+*   **Package Manager(s):** `pip` (for Python dependencies), `platformio` (for C++/PlatformIO dependencies).
+*   **Communication Protocols:** Protobuf (for native API), MQTT, HTTP.
+
+## 3. Architectural Patterns
+
+*   **Overall Architecture:** The project follows a code-generation architecture. The Python code parses user-defined YAML configuration files and generates C++ source code. This C++ code is then compiled and flashed to the target microcontroller using PlatformIO.
+
+*   **Directory Structure Philosophy:**
+    *   `/esphome`: Contains the core Python source code for the ESPHome application.
+    *   `/esphome/components`: Contains the individual components that can be used in ESPHome configurations. Each component is a self-contained unit with its own C++ and Python code.
+    *   `/tests`: Contains all unit and integration tests for the Python code.
+    *   `/docker`: Contains Docker-related files for building and running ESPHome in a container.
+    *   `/script`: Contains helper scripts for development and maintenance.
+
+*   **Core Architectural Components:**
+    1.  **Configuration System** (`esphome/config*.py`): Handles YAML parsing and validation using Voluptuous, schema definitions, and multi-platform configurations.
+    2.  **Code Generation** (`esphome/codegen.py`, `esphome/cpp_generator.py`): Manages Python to C++ code generation, template processing, and build flag management.
+    3.  **Component System** (`esphome/components/`): Contains modular hardware and software components with platform-specific implementations and dependency management.
+    4.  **Core Framework** (`esphome/core/`): Manages the application lifecycle, hardware abstraction, and component registration.
+    5.  **Dashboard** (`esphome/dashboard/`): A web-based interface for device configuration, management, and OTA updates.
+
+*   **Platform Support:**
+    1.  **ESP32** (`components/esp32/`): Espressif ESP32 family. Supports multiple variants (S2, S3, C3, etc.) and both IDF and Arduino frameworks.
+    2.  **ESP8266** (`components/esp8266/`): Espressif ESP8266. Arduino framework only, with memory constraints.
+    3.  **RP2040** (`components/rp2040/`): Raspberry Pi Pico/RP2040. Arduino framework with PIO (Programmable I/O) support.
+    4.  **LibreTiny** (`components/libretiny/`): Realtek and Beken chips. Supports multiple chip families and auto-generated components.
+
+## 4. Coding Conventions & Style Guide
+
+*   **Formatting:**
+    *   **Python:** Uses `ruff` and `flake8` for linting and formatting. Configuration is in `pyproject.toml`.
+    *   **C++:** Uses `clang-format` for formatting. Configuration is in `.clang-format`.
+
+*   **Naming Conventions:**
+    *   **Python:** Follows PEP 8. Use clear, descriptive names following snake_case.
+    *   **C++:** Follows the Google C++ Style Guide.
+
+*   **Component Structure:**
+    *   **Standard Files:**
+        ```
+        components/[component_name]/
+        ├── __init__.py          # Component configuration schema and code generation
+        ├── [component].h        # C++ header file (if needed)
+        ├── [component].cpp      # C++ implementation (if needed)
+        └── [platform]/         # Platform-specific implementations
+            ├── __init__.py      # Platform-specific configuration
+            ├── [platform].h     # Platform C++ header
+            └── [platform].cpp   # Platform C++ implementation
+        ```
+
+    *   **Component Metadata:**
+        - `DEPENDENCIES`: List of required components
+        - `AUTO_LOAD`: Components to automatically load
+        - `CONFLICTS_WITH`: Incompatible components
+        - `CODEOWNERS`: GitHub usernames responsible for maintenance
+        - `MULTI_CONF`: Whether multiple instances are allowed
+
+*   **Code Generation & Common Patterns:**
+    *   **Configuration Schema Pattern:**
+        ```python
+        import esphome.codegen as cg
+        import esphome.config_validation as cv
+        from esphome.const import CONF_KEY, CONF_ID
+
+        CONF_PARAM = "param"  # A constant that does not yet exist in esphome/const.py
+
+        my_component_ns = cg.esphome_ns.namespace("my_component")
+        MyComponent = my_component_ns.class_("MyComponent", cg.Component)
+
+        CONFIG_SCHEMA = cv.Schema({
+            cv.GenerateID(): cv.declare_id(MyComponent),
+            cv.Required(CONF_KEY): cv.string,
+            cv.Optional(CONF_PARAM, default=42): cv.int_,
+        }).extend(cv.COMPONENT_SCHEMA)
+
+        async def to_code(config):
+            var = cg.new_Pvariable(config[CONF_ID])
+            await cg.register_component(var, config)
+            cg.add(var.set_key(config[CONF_KEY]))
+            cg.add(var.set_param(config[CONF_PARAM]))
+        ```
+
+    *   **C++ Class Pattern:**
+        ```cpp
+        namespace esphome {
+        namespace my_component {
+
+        class MyComponent : public Component {
+         public:
+          void setup() override;
+          void loop() override;
+          void dump_config() override;
+
+          void set_key(const std::string &key) { this->key_ = key; }
+          void set_param(int param) { this->param_ = param; }
+
+         protected:
+          std::string key_;
+          int param_{0};
+        };
+
+        }  // namespace my_component
+        }  // namespace esphome
+        ```
+
+    *   **Common Component Examples:**
+        - **Sensor:**
+          ```python
+          from esphome.components import sensor
+          CONFIG_SCHEMA = sensor.sensor_schema(MySensor).extend(cv.polling_component_schema("60s"))
+          async def to_code(config):
+              var = await sensor.new_sensor(config)
+              await cg.register_component(var, config)
+          ```
+
+        - **Binary Sensor:**
+          ```python
+          from esphome.components import binary_sensor
+          CONFIG_SCHEMA = binary_sensor.binary_sensor_schema().extend({ ... })
+          async def to_code(config):
+              var = await binary_sensor.new_binary_sensor(config)
+          ```
+
+        - **Switch:**
+          ```python
+          from esphome.components import switch
+          CONFIG_SCHEMA = switch.switch_schema().extend({ ... })
+          async def to_code(config):
+              var = await switch.new_switch(config)
+          ```
+
+*   **Configuration Validation:**
+    *   **Common Validators:** `cv.int_`, `cv.float_`, `cv.string`, `cv.boolean`, `cv.int_range(min=0, max=100)`, `cv.positive_int`, `cv.percentage`.
+    *   **Complex Validation:** `cv.All(cv.string, cv.Length(min=1, max=50))`, `cv.Any(cv.int_, cv.string)`.
+    *   **Platform-Specific:** `cv.only_on(["esp32", "esp8266"])`, `cv.only_with_arduino`.
+    *   **Schema Extensions:**
+        ```python
+        CONFIG_SCHEMA = cv.Schema({ ... })
+         .extend(cv.COMPONENT_SCHEMA)
+         .extend(uart.UART_DEVICE_SCHEMA)
+         .extend(i2c.i2c_device_schema(0x48))
+         .extend(spi.spi_device_schema(cs_pin_required=True))
+        ```
+
+## 5. Key Files & Entrypoints
+
+*   **Main Entrypoint(s):** `esphome/__main__.py` is the main entrypoint for the ESPHome command-line interface.
+*   **Configuration:**
+    *   `pyproject.toml`: Defines the Python project metadata and dependencies.
+    *   `platformio.ini`: Configures the PlatformIO build environments for different microcontrollers.
+    *   `.pre-commit-config.yaml`: Configures the pre-commit hooks for linting and formatting.
+*   **CI/CD Pipeline:** Defined in `.github/workflows`.
+
+## 6. Development & Testing Workflow
+
+*   **Local Development Environment:** Use the provided Docker container or create a Python virtual environment and install dependencies from `requirements_dev.txt`.
+*   **Running Commands:** Use the `script/run-in-env.py` script to execute commands within the project's virtual environment. For example, to run the linter: `python3 script/run-in-env.py pre-commit run`.
+*   **Testing:**
+    *   **Python:** Run unit tests with `pytest`.
+    *   **C++:** Use `clang-tidy` for static analysis.
+    *   **Component Tests:** YAML-based compilation tests are located in `tests/`. The structure is as follows:
+        ```
+        tests/
+        ├── test_build_components/ # Base test configurations
+        └── components/[component]/ # Component-specific tests
+        ```
+        Run them using `script/test_build_components`. Use `-c <component>` to test specific components and `-t <target>` for specific platforms.
+*   **Debugging and Troubleshooting:**
+    *   **Debug Tools:**
+        - `esphome config <file>.yaml` to validate configuration.
+        - `esphome compile <file>.yaml` to compile without uploading.
+        - Check the Dashboard for real-time logs.
+        - Use component-specific debug logging.
+    *   **Common Issues:**
+        - **Import Errors**: Check component dependencies and `PYTHONPATH`.
+        - **Validation Errors**: Review configuration schema definitions.
+        - **Build Errors**: Check platform compatibility and library versions.
+        - **Runtime Errors**: Review generated C++ code and component logic.
+
+## 7. Specific Instructions for AI Collaboration
+
+*   **Contribution Workflow (Pull Request Process):**
+    1.  **Fork & Branch:** Create a new branch in your fork.
+    2.  **Make Changes:** Adhere to all coding conventions and patterns.
+    3.  **Test:** Create component tests for all supported platforms and run the full test suite locally.
+    4.  **Lint:** Run `pre-commit` to ensure code is compliant.
+    5.  **Commit:** Commit your changes. There is no strict format for commit messages.
+    6.  **Pull Request:** Submit a PR against the `dev` branch. The Pull Request title should have a prefix of the component being worked on (e.g., `[display] Fix bug`, `[abc123] Add new component`). Update documentation, examples, and add `CODEOWNERS` entries as needed. Pull requests should always be made with the PULL_REQUEST_TEMPLATE.md template filled out correctly.
+
+*   **Documentation Contributions:**
+    *   Documentation is hosted in the separate `esphome/esphome-docs` repository.
+    *   The contribution workflow is the same as for the codebase.
+
+*   **Best Practices:**
+    *   **Component Development:** Keep dependencies minimal, provide clear error messages, and write comprehensive docstrings and tests.
+    *   **Code Generation:** Generate minimal and efficient C++ code. Validate all user inputs thoroughly. Support multiple platform variations.
+    *   **Configuration Design:** Aim for simplicity with sensible defaults, while allowing for advanced customization.
+
+*   **Security:** Be mindful of security when making changes to the API, web server, or any other network-related code. Do not hardcode secrets or keys.
+
+*   **Dependencies & Build System Integration:**
+    *   **Python:** When adding a new Python dependency, add it to the appropriate `requirements*.txt` file and `pyproject.toml`.
+    *   **C++ / PlatformIO:** When adding a new C++ dependency, add it to `platformio.ini` and use `cg.add_library`.
+    *   **Build Flags:** Use `cg.add_build_flag(...)` to add compiler flags.

.clang-tidy.hash

@@ -0,0 +1 @@
+32b0db73b3ae01ba18c9cbb1dabbd8156bc14dded500471919bd0a3dc33916e0

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -0,0 +1,92 @@
+name: Report an issue with ESPHome
+description: Report an issue with ESPHome.
+body:
+  - type: markdown
+    attributes:
+      value: |
+        This issue form is for reporting bugs only!
+
+        If you have a feature request or enhancement, please [request them here instead][fr].
+
+        [fr]: https://github.com/orgs/esphome/discussions
+  - type: textarea
+    validations:
+      required: true
+    id: problem
+    attributes:
+      label: The problem
+      description: >-
+        Describe the issue you are experiencing here to communicate to the
+        maintainers. Tell us what you were trying to do and what happened.
+
+        Provide a clear and concise description of what the problem is.
+
+  - type: markdown
+    attributes:
+      value: |
+        ## Environment
+  - type: input
+    id: version
+    validations:
+      required: true
+    attributes:
+      label: Which version of ESPHome has the issue?
+      description: >
+        ESPHome version like 1.19, 2025.6.0 or 2025.XX.X-dev.
+  - type: dropdown
+    validations:
+      required: true
+    id: installation
+    attributes:
+      label: What type of installation are you using?
+      options:
+        - Home Assistant Add-on
+        - Docker
+        - pip
+  - type: dropdown
+    validations:
+      required: true
+    id: platform
+    attributes:
+      label: What platform are you using?
+      options:
+        - ESP8266
+        - ESP32
+        - RP2040
+        - BK72XX
+        - RTL87XX
+        - LN882X
+        - Host
+        - Other
+  - type: input
+    id: component_name
+    attributes:
+      label: Component causing the issue
+      description: >
+        The name of the component or platform. For example, api/i2c or ultrasonic.
+
+  - type: markdown
+    attributes:
+      value: |
+        # Details
+  - type: textarea
+    id: config
+    attributes:
+      label: YAML Config
+      description: |
+        Include a complete YAML configuration file demonstrating the problem here. Preferably post the *entire* file - don't make assumptions about what is unimportant. However, if it's a large or complicated config then you will need to reduce it to the smallest possible file *that still demonstrates the problem*. If you don't provide enough information to *easily* reproduce the problem, it's unlikely your bug report will get any attention. Logs do not belong here, attach them below.
+      render: yaml
+  - type: textarea
+    id: logs
+    attributes:
+      label: Anything in the logs that might be useful for us?
+      description: For example, error message, or stack traces. Serial or USB logs are much more useful than WiFi logs.
+      render: txt
+  - type: textarea
+    id: additional
+    attributes:
+      label: Additional information
+      description: >
+        If you have any additional information for us, use the field below.
+        Please note, you can attach screenshots or screen recordings here, by
+        dragging and dropping files in the field below.

.github/ISSUE_TEMPLATE/config.yml

@@ -1,15 +1,21 @@
 ---
 blank_issues_enabled: false
 contact_links:
-  - name: Issue Tracker
-    url: https://github.com/esphome/issues
-    about: Please create bug reports in the dedicated issue tracker.
-  - name: Feature Request Tracker
-    url: https://github.com/esphome/feature-requests
-    about: |
-      Please create feature requests in the dedicated feature request tracker.
+  - name: Report an issue with the ESPHome documentation
+    url: https://github.com/esphome/esphome-docs/issues/new/choose
+    about: Report an issue with the ESPHome documentation.
+  - name: Report an issue with the ESPHome web server
+    url: https://github.com/esphome/esphome-webserver/issues/new/choose
+    about: Report an issue with the ESPHome web server.
+  - name: Report an issue with the ESPHome Builder / Dashboard
+    url: https://github.com/esphome/dashboard/issues/new/choose
+    about: Report an issue with the ESPHome Builder / Dashboard.
+  - name: Report an issue with the ESPHome API client
+    url: https://github.com/esphome/aioesphomeapi/issues/new/choose
+    about: Report an issue with the ESPHome API client.
+  - name: Make a Feature Request
+    url: https://github.com/orgs/esphome/discussions
+    about: Please create feature requests in the dedicated feature request tracker.
   - name: Frequently Asked Question
     url: https://esphome.io/guides/faq.html
-    about: |
-      Please view the FAQ for common questions and what
-      to include in a bug report.
+    about: Please view the FAQ for common questions and what to include in a bug report.

.github/PULL_REQUEST_TEMPLATE.md

@@ -26,6 +26,7 @@
 - [ ] RP2040
 - [ ] BK72xx
 - [ ] RTL87xx
+- [ ] nRF52840
 
 ## Example entry for `config.yaml`:
 

.github/actions/restore-python/action.yml

@@ -41,7 +41,7 @@ runs:
       shell: bash
       run: |
         python -m venv venv
-        ./venv/Scripts/activate
+        source ./venv/Scripts/activate
         python --version
         pip install -r requirements.txt -r requirements_test.txt
         pip install -e .

.github/copilot-instructions.md

@@ -0,0 +1 @@
+../.ai/instructions.md

.github/dependabot.yml

@@ -9,6 +9,9 @@ updates:
       # Hypotehsis is only used for testing and is updated quite often
       - dependency-name: hypothesis
   - package-ecosystem: github-actions
+    labels:
+      - "dependencies"
+      - "github-actions"
     directory: "/"
     schedule:
       interval: daily
@@ -20,11 +23,17 @@ updates:
           - "docker/login-action"
           - "docker/setup-buildx-action"
   - package-ecosystem: github-actions
+    labels:
+      - "dependencies"
+      - "github-actions"
     directory: "/.github/actions/build-image"
     schedule:
       interval: daily
     open-pull-requests-limit: 10
   - package-ecosystem: github-actions
+    labels:
+      - "dependencies"
+      - "github-actions"
     directory: "/.github/actions/restore-python"
     schedule:
       interval: daily

.github/workflows/auto-label-pr.yml

@@ -0,0 +1,620 @@
+name: Auto Label PR
+
+on:
+  # Runs only on pull_request_target due to having access to a App token.
+  # This means PRs from forks will not be able to alter this workflow to get the tokens
+  pull_request_target:
+    types: [labeled, opened, reopened, synchronize, edited]
+
+permissions:
+  pull-requests: write
+  contents: read
+
+env:
+  SMALL_PR_THRESHOLD: 30
+  MAX_LABELS: 15
+  TOO_BIG_THRESHOLD: 1000
+  COMPONENT_LABEL_THRESHOLD: 10
+
+jobs:
+  label:
+    runs-on: ubuntu-latest
+    if: github.event.action != 'labeled' || github.event.sender.type != 'Bot'
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4.2.2
+
+      - name: Generate a token
+        id: generate-token
+        uses: actions/create-github-app-token@v2
+        with:
+          app-id: ${{ secrets.ESPHOME_GITHUB_APP_ID }}
+          private-key: ${{ secrets.ESPHOME_GITHUB_APP_PRIVATE_KEY }}
+
+      - name: Auto Label PR
+        uses: actions/github-script@v7.0.1
+        with:
+          github-token: ${{ steps.generate-token.outputs.token }}
+          script: |
+            const fs = require('fs');
+
+            // Constants
+            const SMALL_PR_THRESHOLD = parseInt('${{ env.SMALL_PR_THRESHOLD }}');
+            const MAX_LABELS = parseInt('${{ env.MAX_LABELS }}');
+            const TOO_BIG_THRESHOLD = parseInt('${{ env.TOO_BIG_THRESHOLD }}');
+            const COMPONENT_LABEL_THRESHOLD = parseInt('${{ env.COMPONENT_LABEL_THRESHOLD }}');
+            const BOT_COMMENT_MARKER = '<!-- auto-label-pr-bot -->';
+            const CODEOWNERS_MARKER = '<!-- codeowners-request -->';
+            const TOO_BIG_MARKER = '<!-- too-big-request -->';
+
+            const MANAGED_LABELS = [
+              'new-component',
+              'new-platform',
+              'new-target-platform',
+              'merging-to-release',
+              'merging-to-beta',
+              'core',
+              'small-pr',
+              'dashboard',
+              'github-actions',
+              'by-code-owner',
+              'has-tests',
+              'needs-tests',
+              'needs-docs',
+              'needs-codeowners',
+              'too-big',
+              'labeller-recheck'
+            ];
+
+            const DOCS_PR_PATTERNS = [
+              /https:\/\/github\.com\/esphome\/esphome-docs\/pull\/\d+/,
+              /esphome\/esphome-docs#\d+/
+            ];
+
+            // Global state
+            const { owner, repo } = context.repo;
+            const pr_number = context.issue.number;
+
+            // Get current labels and PR data
+            const { data: currentLabelsData } = await github.rest.issues.listLabelsOnIssue({
+              owner,
+              repo,
+              issue_number: pr_number
+            });
+            const currentLabels = currentLabelsData.map(label => label.name);
+            const managedLabels = currentLabels.filter(label =>
+              label.startsWith('component: ') || MANAGED_LABELS.includes(label)
+            );
+
+            // Check for mega-PR early - if present, skip most automatic labeling
+            const isMegaPR = currentLabels.includes('mega-pr');
+
+            // Get all PR files with automatic pagination
+            const prFiles = await github.paginate(
+              github.rest.pulls.listFiles,
+              {
+                owner,
+                repo,
+                pull_number: pr_number
+              }
+            );
+
+            // Calculate data from PR files
+            const changedFiles = prFiles.map(file => file.filename);
+            const totalChanges = prFiles.reduce((sum, file) => sum + (file.additions || 0) + (file.deletions || 0), 0);
+
+            console.log('Current labels:', currentLabels.join(', '));
+            console.log('Changed files:', changedFiles.length);
+            console.log('Total changes:', totalChanges);
+            if (isMegaPR) {
+              console.log('Mega-PR detected - applying limited labeling logic');
+            }
+
+            // Fetch API data
+            async function fetchApiData() {
+              try {
+                const response = await fetch('https://data.esphome.io/components.json');
+                const componentsData = await response.json();
+                return {
+                  targetPlatforms: componentsData.target_platforms || [],
+                  platformComponents: componentsData.platform_components || []
+                };
+              } catch (error) {
+                console.log('Failed to fetch components data from API:', error.message);
+                return { targetPlatforms: [], platformComponents: [] };
+              }
+            }
+
+            // Strategy: Merge branch detection
+            async function detectMergeBranch() {
+              const labels = new Set();
+              const baseRef = context.payload.pull_request.base.ref;
+
+              if (baseRef === 'release') {
+                labels.add('merging-to-release');
+              } else if (baseRef === 'beta') {
+                labels.add('merging-to-beta');
+              }
+
+              return labels;
+            }
+
+            // Strategy: Component and platform labeling
+            async function detectComponentPlatforms(apiData) {
+              const labels = new Set();
+              const componentRegex = /^esphome\/components\/([^\/]+)\//;
+              const targetPlatformRegex = new RegExp(`^esphome\/components\/(${apiData.targetPlatforms.join('|')})/`);
+
+              for (const file of changedFiles) {
+                const componentMatch = file.match(componentRegex);
+                if (componentMatch) {
+                  labels.add(`component: ${componentMatch[1]}`);
+                }
+
+                const platformMatch = file.match(targetPlatformRegex);
+                if (platformMatch) {
+                  labels.add(`platform: ${platformMatch[1]}`);
+                }
+              }
+
+              return labels;
+            }
+
+            // Strategy: New component detection
+            async function detectNewComponents() {
+              const labels = new Set();
+              const addedFiles = prFiles.filter(file => file.status === 'added').map(file => file.filename);
+
+              for (const file of addedFiles) {
+                const componentMatch = file.match(/^esphome\/components\/([^\/]+)\/__init__\.py$/);
+                if (componentMatch) {
+                  try {
+                    const content = fs.readFileSync(file, 'utf8');
+                    if (content.includes('IS_TARGET_PLATFORM = True')) {
+                      labels.add('new-target-platform');
+                    }
+                  } catch (error) {
+                    console.log(`Failed to read content of ${file}:`, error.message);
+                  }
+                  labels.add('new-component');
+                }
+              }
+
+              return labels;
+            }
+
+            // Strategy: New platform detection
+            async function detectNewPlatforms(apiData) {
+              const labels = new Set();
+              const addedFiles = prFiles.filter(file => file.status === 'added').map(file => file.filename);
+
+              for (const file of addedFiles) {
+                const platformFileMatch = file.match(/^esphome\/components\/([^\/]+)\/([^\/]+)\.py$/);
+                if (platformFileMatch) {
+                  const [, component, platform] = platformFileMatch;
+                  if (apiData.platformComponents.includes(platform)) {
+                    labels.add('new-platform');
+                  }
+                }
+
+                const platformDirMatch = file.match(/^esphome\/components\/([^\/]+)\/([^\/]+)\/__init__\.py$/);
+                if (platformDirMatch) {
+                  const [, component, platform] = platformDirMatch;
+                  if (apiData.platformComponents.includes(platform)) {
+                    labels.add('new-platform');
+                  }
+                }
+              }
+
+              return labels;
+            }
+
+            // Strategy: Core files detection
+            async function detectCoreChanges() {
+              const labels = new Set();
+              const coreFiles = changedFiles.filter(file =>
+                file.startsWith('esphome/core/') ||
+                (file.startsWith('esphome/') && file.split('/').length === 2)
+              );
+
+              if (coreFiles.length > 0) {
+                labels.add('core');
+              }
+
+              return labels;
+            }
+
+            // Strategy: PR size detection
+            async function detectPRSize() {
+              const labels = new Set();
+              const testChanges = prFiles
+                .filter(file => file.filename.startsWith('tests/'))
+                .reduce((sum, file) => sum + (file.additions || 0) + (file.deletions || 0), 0);
+
+              const nonTestChanges = totalChanges - testChanges;
+
+              if (totalChanges <= SMALL_PR_THRESHOLD) {
+                labels.add('small-pr');
+              }
+
+              // Don't add too-big if mega-pr label is already present
+              if (nonTestChanges > TOO_BIG_THRESHOLD && !isMegaPR) {
+                labels.add('too-big');
+              }
+
+              return labels;
+            }
+
+            // Strategy: Dashboard changes
+            async function detectDashboardChanges() {
+              const labels = new Set();
+              const dashboardFiles = changedFiles.filter(file =>
+                file.startsWith('esphome/dashboard/') ||
+                file.startsWith('esphome/components/dashboard_import/')
+              );
+
+              if (dashboardFiles.length > 0) {
+                labels.add('dashboard');
+              }
+
+              return labels;
+            }
+
+            // Strategy: GitHub Actions changes
+            async function detectGitHubActionsChanges() {
+              const labels = new Set();
+              const githubActionsFiles = changedFiles.filter(file =>
+                file.startsWith('.github/workflows/')
+              );
+
+              if (githubActionsFiles.length > 0) {
+                labels.add('github-actions');
+              }
+
+              return labels;
+            }
+
+            // Strategy: Code owner detection
+            async function detectCodeOwner() {
+              const labels = new Set();
+
+              try {
+                const { data: codeownersFile } = await github.rest.repos.getContent({
+                  owner,
+                  repo,
+                  path: 'CODEOWNERS',
+                });
+
+                const codeownersContent = Buffer.from(codeownersFile.content, 'base64').toString('utf8');
+                const prAuthor = context.payload.pull_request.user.login;
+
+                const codeownersLines = codeownersContent.split('\n')
+                  .map(line => line.trim())
+                  .filter(line => line && !line.startsWith('#'));
+
+                const codeownersRegexes = codeownersLines.map(line => {
+                  const parts = line.split(/\s+/);
+                  const pattern = parts[0];
+                  const owners = parts.slice(1);
+
+                  let regex;
+                  if (pattern.endsWith('*')) {
+                    const dir = pattern.slice(0, -1);
+                    regex = new RegExp(`^${dir.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}`);
+                  } else if (pattern.includes('*')) {
+                    // First escape all regex special chars except *, then replace * with .*
+                    const regexPattern = pattern
+                      .replace(/[.+?^${}()|[\]\\]/g, '\\$&')
+                      .replace(/\*/g, '.*');
+                    regex = new RegExp(`^${regexPattern}$`);
+                  } else {
+                    regex = new RegExp(`^${pattern.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}$`);
+                  }
+
+                  return { regex, owners };
+                });
+
+                for (const file of changedFiles) {
+                  for (const { regex, owners } of codeownersRegexes) {
+                    if (regex.test(file) && owners.some(owner => owner === `@${prAuthor}`)) {
+                      labels.add('by-code-owner');
+                      return labels;
+                    }
+                  }
+                }
+              } catch (error) {
+                console.log('Failed to read or parse CODEOWNERS file:', error.message);
+              }
+
+              return labels;
+            }
+
+            // Strategy: Test detection
+            async function detectTests() {
+              const labels = new Set();
+              const testFiles = changedFiles.filter(file => file.startsWith('tests/'));
+
+              if (testFiles.length > 0) {
+                labels.add('has-tests');
+              }
+
+              return labels;
+            }
+
+            // Strategy: Requirements detection
+            async function detectRequirements(allLabels) {
+              const labels = new Set();
+
+              // Check for missing tests
+              if ((allLabels.has('new-component') || allLabels.has('new-platform')) && !allLabels.has('has-tests')) {
+                labels.add('needs-tests');
+              }
+
+              // Check for missing docs
+              if (allLabels.has('new-component') || allLabels.has('new-platform')) {
+                const prBody = context.payload.pull_request.body || '';
+                const hasDocsLink = DOCS_PR_PATTERNS.some(pattern => pattern.test(prBody));
+
+                if (!hasDocsLink) {
+                  labels.add('needs-docs');
+                }
+              }
+
+              // Check for missing CODEOWNERS
+              if (allLabels.has('new-component')) {
+                const codeownersModified = prFiles.some(file =>
+                  file.filename === 'CODEOWNERS' &&
+                  (file.status === 'modified' || file.status === 'added') &&
+                  (file.additions || 0) > 0
+                );
+
+                if (!codeownersModified) {
+                  labels.add('needs-codeowners');
+                }
+              }
+
+              return labels;
+            }
+
+            // Generate review messages
+            function generateReviewMessages(finalLabels) {
+              const messages = [];
+              const prAuthor = context.payload.pull_request.user.login;
+
+              // Too big message
+              if (finalLabels.includes('too-big')) {
+                const testChanges = prFiles
+                  .filter(file => file.filename.startsWith('tests/'))
+                  .reduce((sum, file) => sum + (file.additions || 0) + (file.deletions || 0), 0);
+                const nonTestChanges = totalChanges - testChanges;
+
+                const tooManyLabels = finalLabels.length > MAX_LABELS;
+                const tooManyChanges = nonTestChanges > TOO_BIG_THRESHOLD;
+
+                let message = `${TOO_BIG_MARKER}\n### 📦 Pull Request Size\n\n`;
+
+                if (tooManyLabels && tooManyChanges) {
+                  message += `This PR is too large with ${nonTestChanges} line changes (excluding tests) and affects ${finalLabels.length} different components/areas.`;
+                } else if (tooManyLabels) {
+                  message += `This PR affects ${finalLabels.length} different components/areas.`;
+                } else {
+                  message += `This PR is too large with ${nonTestChanges} line changes (excluding tests).`;
+                }
+
+                message += ` Please consider breaking it down into smaller, focused PRs to make review easier and reduce the risk of conflicts.\n\n`;
+                message += `For guidance on breaking down large PRs, see: https://developers.esphome.io/contributing/submitting-your-work/#how-to-approach-large-submissions`;
+
+                messages.push(message);
+              }
+
+              // CODEOWNERS message
+              if (finalLabels.includes('needs-codeowners')) {
+                const message = `${CODEOWNERS_MARKER}\n### 👥 Code Ownership\n\n` +
+                  `Hey there @${prAuthor},\n` +
+                  `Thanks for submitting this pull request! Can you add yourself as a codeowner for this integration? ` +
+                  `This way we can notify you if a bug report for this integration is reported.\n\n` +
+                  `In \`__init__.py\` of the integration, please add:\n\n` +
+                  `\`\`\`python\nCODEOWNERS = ["@${prAuthor}"]\n\`\`\`\n\n` +
+                  `And run \`script/build_codeowners.py\``;
+
+                messages.push(message);
+              }
+
+              return messages;
+            }
+
+            // Handle reviews
+            async function handleReviews(finalLabels) {
+              const reviewMessages = generateReviewMessages(finalLabels);
+              const hasReviewableLabels = finalLabels.some(label =>
+                ['too-big', 'needs-codeowners'].includes(label)
+              );
+
+              const { data: reviews } = await github.rest.pulls.listReviews({
+                owner,
+                repo,
+                pull_number: pr_number
+              });
+
+              const botReviews = reviews.filter(review =>
+                review.user.type === 'Bot' &&
+                review.state === 'CHANGES_REQUESTED' &&
+                review.body && review.body.includes(BOT_COMMENT_MARKER)
+              );
+
+              if (hasReviewableLabels) {
+                const reviewBody = `${BOT_COMMENT_MARKER}\n\n${reviewMessages.join('\n\n---\n\n')}`;
+
+                if (botReviews.length > 0) {
+                  // Update existing review
+                  await github.rest.pulls.updateReview({
+                    owner,
+                    repo,
+                    pull_number: pr_number,
+                    review_id: botReviews[0].id,
+                    body: reviewBody
+                  });
+                  console.log('Updated existing bot review');
+                } else {
+                  // Create new review
+                  await github.rest.pulls.createReview({
+                    owner,
+                    repo,
+                    pull_number: pr_number,
+                    body: reviewBody,
+                    event: 'REQUEST_CHANGES'
+                  });
+                  console.log('Created new bot review');
+                }
+              } else if (botReviews.length > 0) {
+                // Dismiss existing reviews
+                for (const review of botReviews) {
+                  try {
+                    await github.rest.pulls.dismissReview({
+                      owner,
+                      repo,
+                      pull_number: pr_number,
+                      review_id: review.id,
+                      message: 'Review dismissed: All requirements have been met'
+                    });
+                    console.log(`Dismissed bot review ${review.id}`);
+                  } catch (error) {
+                    console.log(`Failed to dismiss review ${review.id}:`, error.message);
+                  }
+                }
+              }
+            }
+
+            // Main execution
+            const apiData = await fetchApiData();
+            const baseRef = context.payload.pull_request.base.ref;
+
+            // Early exit for non-dev branches
+            if (baseRef !== 'dev') {
+              const branchLabels = await detectMergeBranch();
+              const finalLabels = Array.from(branchLabels);
+
+              console.log('Computed labels (merge branch only):', finalLabels.join(', '));
+
+              // Apply labels
+              if (finalLabels.length > 0) {
+                await github.rest.issues.addLabels({
+                  owner,
+                  repo,
+                  issue_number: pr_number,
+                  labels: finalLabels
+                });
+              }
+
+              // Remove old managed labels
+              const labelsToRemove = managedLabels.filter(label => !finalLabels.includes(label));
+              for (const label of labelsToRemove) {
+                try {
+                  await github.rest.issues.removeLabel({
+                    owner,
+                    repo,
+                    issue_number: pr_number,
+                    name: label
+                  });
+                } catch (error) {
+                  console.log(`Failed to remove label ${label}:`, error.message);
+                }
+              }
+
+              return;
+            }
+
+            // Run all strategies
+            const [
+              branchLabels,
+              componentLabels,
+              newComponentLabels,
+              newPlatformLabels,
+              coreLabels,
+              sizeLabels,
+              dashboardLabels,
+              actionsLabels,
+              codeOwnerLabels,
+              testLabels
+            ] = await Promise.all([
+              detectMergeBranch(),
+              detectComponentPlatforms(apiData),
+              detectNewComponents(),
+              detectNewPlatforms(apiData),
+              detectCoreChanges(),
+              detectPRSize(),
+              detectDashboardChanges(),
+              detectGitHubActionsChanges(),
+              detectCodeOwner(),
+              detectTests()
+            ]);
+
+            // Combine all labels
+            const allLabels = new Set([
+              ...branchLabels,
+              ...componentLabels,
+              ...newComponentLabels,
+              ...newPlatformLabels,
+              ...coreLabels,
+              ...sizeLabels,
+              ...dashboardLabels,
+              ...actionsLabels,
+              ...codeOwnerLabels,
+              ...testLabels
+            ]);
+
+            // Detect requirements based on all other labels
+            const requirementLabels = await detectRequirements(allLabels);
+            for (const label of requirementLabels) {
+              allLabels.add(label);
+            }
+
+            let finalLabels = Array.from(allLabels);
+
+            // For mega-PRs, exclude component labels if there are too many
+            if (isMegaPR) {
+              const componentLabels = finalLabels.filter(label => label.startsWith('component: '));
+              if (componentLabels.length > COMPONENT_LABEL_THRESHOLD) {
+                finalLabels = finalLabels.filter(label => !label.startsWith('component: '));
+                console.log(`Mega-PR detected - excluding ${componentLabels.length} component labels (threshold: ${COMPONENT_LABEL_THRESHOLD})`);
+              }
+            }
+
+            // Handle too many labels (only for non-mega PRs)
+            const tooManyLabels = finalLabels.length > MAX_LABELS;
+
+            if (tooManyLabels && !isMegaPR && !finalLabels.includes('too-big')) {
+              finalLabels = ['too-big'];
+            }
+
+            console.log('Computed labels:', finalLabels.join(', '));
+
+            // Handle reviews
+            await handleReviews(finalLabels);
+
+            // Apply labels
+            if (finalLabels.length > 0) {
+              console.log(`Adding labels: ${finalLabels.join(', ')}`);
+              await github.rest.issues.addLabels({
+                owner,
+                repo,
+                issue_number: pr_number,
+                labels: finalLabels
+              });
+            }
+
+            // Remove old managed labels
+            const labelsToRemove = managedLabels.filter(label => !finalLabels.includes(label));
+            for (const label of labelsToRemove) {
+              console.log(`Removing label: ${label}`);
+              try {
+                await github.rest.issues.removeLabel({
+                  owner,
+                  repo,
+                  issue_number: pr_number,
+                  name: label
+                });
+              } catch (error) {
+                console.log(`Failed to remove label ${label}:`, error.message);
+              }
+            }

.github/workflows/ci-clang-tidy-hash.yml

@@ -0,0 +1,75 @@
+name: Clang-tidy Hash CI
+
+on:
+  pull_request:
+    paths:
+      - ".clang-tidy"
+      - "platformio.ini"
+      - "requirements_dev.txt"
+      - ".clang-tidy.hash"
+      - "script/clang_tidy_hash.py"
+      - ".github/workflows/ci-clang-tidy-hash.yml"
+
+permissions:
+  contents: read
+  pull-requests: write
+
+jobs:
+  verify-hash:
+    name: Verify clang-tidy hash
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4.2.2
+
+      - name: Set up Python
+        uses: actions/setup-python@v5.6.0
+        with:
+          python-version: "3.11"
+
+      - name: Verify hash
+        run: |
+          python script/clang_tidy_hash.py --verify
+
+      - if: failure()
+        name: Show hash details
+        run: |
+          python script/clang_tidy_hash.py
+          echo "## Job Failed" | tee -a $GITHUB_STEP_SUMMARY
+          echo "You have modified clang-tidy configuration but have not updated the hash." | tee -a $GITHUB_STEP_SUMMARY
+          echo "Please run 'script/clang_tidy_hash.py --update' and commit the changes." | tee -a $GITHUB_STEP_SUMMARY
+
+      - if: failure()
+        name: Request changes
+        uses: actions/github-script@v7.0.1
+        with:
+          script: |
+            await github.rest.pulls.createReview({
+              pull_number: context.issue.number,
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              event: 'REQUEST_CHANGES',
+              body: 'You have modified clang-tidy configuration but have not updated the hash.\nPlease run `script/clang_tidy_hash.py --update` and commit the changes.'
+            })
+
+      - if: success()
+        name: Dismiss review
+        uses: actions/github-script@v7.0.1
+        with:
+          script: |
+            let reviews = await github.rest.pulls.listReviews({
+              pull_number: context.issue.number,
+              owner: context.repo.owner,
+              repo: context.repo.repo
+            });
+            for (let review of reviews.data) {
+              if (review.user.login === 'github-actions[bot]' && review.state === 'CHANGES_REQUESTED') {
+                await github.rest.pulls.dismissReview({
+                  pull_number: context.issue.number,
+                  owner: context.repo.owner,
+                  repo: context.repo.repo,
+                  review_id: review.id,
+                  message: 'Clang-tidy hash now matches configuration.'
+                });
+              }
+            }

.github/workflows/ci-docker.yml

@@ -47,7 +47,7 @@ jobs:
       - name: Set up Python
         uses: actions/setup-python@v5.6.0
         with:
-          python-version: "3.10"
+          python-version: "3.11"
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v3.11.1
 

.github/workflows/ci.yml

@@ -20,8 +20,8 @@ permissions:
   contents: read
 
 env:
-  DEFAULT_PYTHON: "3.10"
-  PYUPGRADE_TARGET: "--py310-plus"
+  DEFAULT_PYTHON: "3.11"
+  PYUPGRADE_TARGET: "--py311-plus"
 
 concurrency:
   # yamllint disable-line rule:line-length
@@ -39,7 +39,7 @@ jobs:
         uses: actions/checkout@v4.2.2
       - name: Generate cache-key
         id: cache-key
-        run: echo key="${{ hashFiles('requirements.txt', 'requirements_test.txt') }}" >> $GITHUB_OUTPUT
+        run: echo key="${{ hashFiles('requirements.txt', 'requirements_test.txt', '.pre-commit-config.yaml') }}" >> $GITHUB_OUTPUT
       - name: Set up Python ${{ env.DEFAULT_PYTHON }}
         id: python
         uses: actions/setup-python@v5.6.0
@@ -58,56 +58,16 @@ jobs:
           python -m venv venv
           . venv/bin/activate
           python --version
-          pip install -r requirements.txt -r requirements_test.txt
+          pip install -r requirements.txt -r requirements_test.txt pre-commit
           pip install -e .
 
-  ruff:
-    name: Check ruff
-    runs-on: ubuntu-24.04
-    needs:
-      - common
-    steps:
-      - name: Check out code from GitHub
-        uses: actions/checkout@v4.2.2
-      - name: Restore Python
-        uses: ./.github/actions/restore-python
-        with:
-          python-version: ${{ env.DEFAULT_PYTHON }}
-          cache-key: ${{ needs.common.outputs.cache-key }}
-      - name: Run Ruff
-        run: |
-          . venv/bin/activate
-          ruff format esphome tests
-      - name: Suggested changes
-        run: script/ci-suggest-changes
-        if: always()
-
-  flake8:
-    name: Check flake8
-    runs-on: ubuntu-24.04
-    needs:
-      - common
-    steps:
-      - name: Check out code from GitHub
-        uses: actions/checkout@v4.2.2
-      - name: Restore Python
-        uses: ./.github/actions/restore-python
-        with:
-          python-version: ${{ env.DEFAULT_PYTHON }}
-          cache-key: ${{ needs.common.outputs.cache-key }}
-      - name: Run flake8
-        run: |
-          . venv/bin/activate
-          flake8 esphome
-      - name: Suggested changes
-        run: script/ci-suggest-changes
-        if: always()
-
   pylint:
     name: Check pylint
     runs-on: ubuntu-24.04
     needs:
       - common
+      - determine-jobs
+    if: needs.determine-jobs.outputs.python-linters == 'true'
     steps:
       - name: Check out code from GitHub
         uses: actions/checkout@v4.2.2
@@ -124,27 +84,6 @@ jobs:
         run: script/ci-suggest-changes
         if: always()
 
-  pyupgrade:
-    name: Check pyupgrade
-    runs-on: ubuntu-24.04
-    needs:
-      - common
-    steps:
-      - name: Check out code from GitHub
-        uses: actions/checkout@v4.2.2
-      - name: Restore Python
-        uses: ./.github/actions/restore-python
-        with:
-          python-version: ${{ env.DEFAULT_PYTHON }}
-          cache-key: ${{ needs.common.outputs.cache-key }}
-      - name: Run pyupgrade
-        run: |
-          . venv/bin/activate
-          pyupgrade ${{ env.PYUPGRADE_TARGET }} `find esphome -name "*.py" -type f`
-      - name: Suggested changes
-        run: script/ci-suggest-changes
-        if: always()
-
   ci-custom:
     name: Run script/ci-custom
     runs-on: ubuntu-24.04
@@ -173,7 +112,6 @@ jobs:
       fail-fast: false
       matrix:
         python-version:
-          - "3.10"
           - "3.11"
           - "3.12"
           - "3.13"
@@ -189,14 +127,10 @@ jobs:
             os: windows-latest
           - python-version: "3.12"
             os: windows-latest
-          - python-version: "3.10"
-            os: windows-latest
           - python-version: "3.13"
             os: macOS-latest
           - python-version: "3.12"
             os: macOS-latest
-          - python-version: "3.10"
-            os: macOS-latest
     runs-on: ${{ matrix.os }}
     needs:
       - common
@@ -204,6 +138,7 @@ jobs:
       - name: Check out code from GitHub
         uses: actions/checkout@v4.2.2
       - name: Restore Python
+        id: restore-python
         uses: ./.github/actions/restore-python
         with:
           python-version: ${{ matrix.python-version }}
@@ -213,7 +148,7 @@ jobs:
       - name: Run pytest
         if: matrix.os == 'windows-latest'
         run: |
-          ./venv/Scripts/activate
+          . ./venv/Scripts/activate.ps1
           pytest -vv --cov-report=xml --tb=native -n auto tests --ignore=tests/integration/
       - name: Run pytest
         if: matrix.os == 'ubuntu-latest' || matrix.os == 'macOS-latest'
@@ -224,12 +159,59 @@ jobs:
         uses: codecov/codecov-action@v5.4.3
         with:
           token: ${{ secrets.CODECOV_TOKEN }}
+      - name: Save Python virtual environment cache
+        if: github.ref == 'refs/heads/dev'
+        uses: actions/cache/save@v4.2.3
+        with:
+          path: venv
+          key: ${{ runner.os }}-${{ steps.restore-python.outputs.python-version }}-venv-${{ needs.common.outputs.cache-key }}
+
+  determine-jobs:
+    name: Determine which jobs to run
+    runs-on: ubuntu-24.04
+    needs:
+      - common
+    outputs:
+      integration-tests: ${{ steps.determine.outputs.integration-tests }}
+      clang-tidy: ${{ steps.determine.outputs.clang-tidy }}
+      python-linters: ${{ steps.determine.outputs.python-linters }}
+      changed-components: ${{ steps.determine.outputs.changed-components }}
+      component-test-count: ${{ steps.determine.outputs.component-test-count }}
+    steps:
+      - name: Check out code from GitHub
+        uses: actions/checkout@v4.2.2
+        with:
+          # Fetch enough history to find the merge base
+          fetch-depth: 2
+      - name: Restore Python
+        uses: ./.github/actions/restore-python
+        with:
+          python-version: ${{ env.DEFAULT_PYTHON }}
+          cache-key: ${{ needs.common.outputs.cache-key }}
+      - name: Determine which tests to run
+        id: determine
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          . venv/bin/activate
+          output=$(python script/determine-jobs.py)
+          echo "Test determination output:"
+          echo "$output" | jq
+
+          # Extract individual fields
+          echo "integration-tests=$(echo "$output" | jq -r '.integration_tests')" >> $GITHUB_OUTPUT
+          echo "clang-tidy=$(echo "$output" | jq -r '.clang_tidy')" >> $GITHUB_OUTPUT
+          echo "python-linters=$(echo "$output" | jq -r '.python_linters')" >> $GITHUB_OUTPUT
+          echo "changed-components=$(echo "$output" | jq -c '.changed_components')" >> $GITHUB_OUTPUT
+          echo "component-test-count=$(echo "$output" | jq -r '.component_test_count')" >> $GITHUB_OUTPUT
 
   integration-tests:
     name: Run integration tests
     runs-on: ubuntu-latest
     needs:
       - common
+      - determine-jobs
+    if: needs.determine-jobs.outputs.integration-tests == 'true'
     steps:
       - name: Check out code from GitHub
         uses: actions/checkout@v4.2.2
@@ -259,44 +241,15 @@ jobs:
           . venv/bin/activate
           pytest -vv --no-cov --tb=native -n auto tests/integration/
 
-  clang-format:
-    name: Check clang-format
-    runs-on: ubuntu-24.04
-    needs:
-      - common
-    steps:
-      - name: Check out code from GitHub
-        uses: actions/checkout@v4.2.2
-      - name: Restore Python
-        uses: ./.github/actions/restore-python
-        with:
-          python-version: ${{ env.DEFAULT_PYTHON }}
-          cache-key: ${{ needs.common.outputs.cache-key }}
-      - name: Install clang-format
-        run: |
-          . venv/bin/activate
-          pip install clang-format -c requirements_dev.txt
-      - name: Run clang-format
-        run: |
-          . venv/bin/activate
-          script/clang-format -i
-          git diff-index --quiet HEAD --
-      - name: Suggested changes
-        run: script/ci-suggest-changes
-        if: always()
-
   clang-tidy:
     name: ${{ matrix.name }}
     runs-on: ubuntu-24.04
     needs:
       - common
-      - ruff
-      - ci-custom
-      - clang-format
-      - flake8
-      - pylint
-      - pytest
-      - pyupgrade
+      - determine-jobs
+    if: needs.determine-jobs.outputs.clang-tidy == 'true'
+    env:
+      GH_TOKEN: ${{ github.token }}
     strategy:
       fail-fast: false
       max-parallel: 2
@@ -335,6 +288,10 @@ jobs:
     steps:
       - name: Check out code from GitHub
         uses: actions/checkout@v4.2.2
+        with:
+          # Need history for HEAD~1 to work for checking changed files
+          fetch-depth: 2
+
       - name: Restore Python
         uses: ./.github/actions/restore-python
         with:
@@ -346,14 +303,14 @@ jobs:
         uses: actions/cache@v4.2.3
         with:
           path: ~/.platformio
-          key: platformio-${{ matrix.pio_cache_key }}
+          key: platformio-${{ matrix.pio_cache_key }}-${{ hashFiles('platformio.ini') }}
 
       - name: Cache platformio
         if: github.ref != 'refs/heads/dev'
         uses: actions/cache/restore@v4.2.3
         with:
           path: ~/.platformio
-          key: platformio-${{ matrix.pio_cache_key }}
+          key: platformio-${{ matrix.pio_cache_key }}-${{ hashFiles('platformio.ini') }}
 
       - name: Register problem matchers
         run: |
@@ -367,10 +324,28 @@ jobs:
           mkdir -p .temp
           pio run --list-targets -e esp32-idf-tidy
 
+      - name: Check if full clang-tidy scan needed
+        id: check_full_scan
+        run: |
+          . venv/bin/activate
+          if python script/clang_tidy_hash.py --check; then
+            echo "full_scan=true" >> $GITHUB_OUTPUT
+            echo "reason=hash_changed" >> $GITHUB_OUTPUT
+          else
+            echo "full_scan=false" >> $GITHUB_OUTPUT
+            echo "reason=normal" >> $GITHUB_OUTPUT
+          fi
+
       - name: Run clang-tidy
         run: |
           . venv/bin/activate
+          if [ "${{ steps.check_full_scan.outputs.full_scan }}" = "true" ]; then
+            echo "Running FULL clang-tidy scan (hash changed)"
             script/clang-tidy --all-headers --fix ${{ matrix.options }} ${{ matrix.ignore_errors && '|| true' || '' }}
+          else
+            echo "Running clang-tidy on changed files only"
+            script/clang-tidy --all-headers --fix --changed ${{ matrix.options }} ${{ matrix.ignore_errors && '|| true' || '' }}
+          fi
         env:
           # Also cache libdeps, store them in a ~/.platformio subfolder
           PLATFORMIO_LIBDEPS_DIR: ~/.platformio/libdeps
@@ -380,59 +355,18 @@ jobs:
         # yamllint disable-line rule:line-length
         if: always()
 
-  list-components:
-    runs-on: ubuntu-24.04
-    needs:
-      - common
-    if: github.event_name == 'pull_request'
-    outputs:
-      components: ${{ steps.list-components.outputs.components }}
-      count: ${{ steps.list-components.outputs.count }}
-    steps:
-      - name: Check out code from GitHub
-        uses: actions/checkout@v4.2.2
-        with:
-          # Fetch enough history so `git merge-base refs/remotes/origin/dev HEAD` works.
-          fetch-depth: 500
-      - name: Get target branch
-        id: target-branch
-        run: |
-          echo "branch=${{ github.event.pull_request.base.ref }}" >> $GITHUB_OUTPUT
-      - name: Fetch ${{ steps.target-branch.outputs.branch }} branch
-        run: |
-          git -c protocol.version=2 fetch --no-tags --prune --no-recurse-submodules --depth=1 origin +refs/heads/${{ steps.target-branch.outputs.branch }}:refs/remotes/origin/${{ steps.target-branch.outputs.branch }}
-          git merge-base refs/remotes/origin/${{ steps.target-branch.outputs.branch }} HEAD
-      - name: Restore Python
-        uses: ./.github/actions/restore-python
-        with:
-          python-version: ${{ env.DEFAULT_PYTHON }}
-          cache-key: ${{ needs.common.outputs.cache-key }}
-      - name: Find changed components
-        id: list-components
-        run: |
-          . venv/bin/activate
-          components=$(script/list-components.py --changed --branch ${{ steps.target-branch.outputs.branch }})
-          output_components=$(echo "$components" | jq -R -s -c 'split("\n")[:-1] | map(select(length > 0))')
-          count=$(echo "$output_components" | jq length)
-
-          echo "components=$output_components" >> $GITHUB_OUTPUT
-          echo "count=$count" >> $GITHUB_OUTPUT
-
-          echo "$count Components:"
-          echo "$output_components" | jq
-
   test-build-components:
     name: Component test ${{ matrix.file }}
     runs-on: ubuntu-24.04
     needs:
       - common
-      - list-components
-    if: github.event_name == 'pull_request' && fromJSON(needs.list-components.outputs.count) > 0 && fromJSON(needs.list-components.outputs.count) < 100
+      - determine-jobs
+    if: github.event_name == 'pull_request' && fromJSON(needs.determine-jobs.outputs.component-test-count) > 0 && fromJSON(needs.determine-jobs.outputs.component-test-count) < 100
     strategy:
       fail-fast: false
       max-parallel: 2
       matrix:
-        file: ${{ fromJson(needs.list-components.outputs.components) }}
+        file: ${{ fromJson(needs.determine-jobs.outputs.changed-components) }}
     steps:
       - name: Install dependencies
         run: |
@@ -460,8 +394,8 @@ jobs:
     runs-on: ubuntu-24.04
     needs:
       - common
-      - list-components
-    if: github.event_name == 'pull_request' && fromJSON(needs.list-components.outputs.count) >= 100
+      - determine-jobs
+    if: github.event_name == 'pull_request' && fromJSON(needs.determine-jobs.outputs.component-test-count) >= 100
     outputs:
       matrix: ${{ steps.split.outputs.components }}
     steps:
@@ -470,7 +404,7 @@ jobs:
       - name: Split components into 20 groups
         id: split
         run: |
-          components=$(echo '${{ needs.list-components.outputs.components }}' | jq -c '.[]' | shuf | jq -s -c '[_nwise(20) | join(" ")]')
+          components=$(echo '${{ needs.determine-jobs.outputs.changed-components }}' | jq -c '.[]' | shuf | jq -s -c '[_nwise(20) | join(" ")]')
           echo "components=$components" >> $GITHUB_OUTPUT
 
   test-build-components-split:
@@ -478,9 +412,9 @@ jobs:
     runs-on: ubuntu-24.04
     needs:
       - common
-      - list-components
+      - determine-jobs
       - test-build-components-splitter
-    if: github.event_name == 'pull_request' && fromJSON(needs.list-components.outputs.count) >= 100
+    if: github.event_name == 'pull_request' && fromJSON(needs.determine-jobs.outputs.component-test-count) >= 100
     strategy:
       fail-fast: false
       max-parallel: 4
@@ -517,24 +451,41 @@ jobs:
             ./script/test_build_components -e compile -c $component
           done
 
+  pre-commit-ci-lite:
+    name: pre-commit.ci lite
+    runs-on: ubuntu-latest
+    needs:
+      - common
+    if: github.event_name == 'pull_request' && github.base_ref != 'beta' && github.base_ref != 'release'
+    steps:
+      - name: Check out code from GitHub
+        uses: actions/checkout@v4.2.2
+      - name: Restore Python
+        uses: ./.github/actions/restore-python
+        with:
+          python-version: ${{ env.DEFAULT_PYTHON }}
+          cache-key: ${{ needs.common.outputs.cache-key }}
+      - uses: pre-commit/action@v3.0.1
+        env:
+          SKIP: pylint,clang-tidy-hash
+      - uses: pre-commit-ci/lite-action@v1.1.0
+        if: always()
+
   ci-status:
     name: CI Status
     runs-on: ubuntu-24.04
     needs:
       - common
-      - ruff
       - ci-custom
-      - clang-format
-      - flake8
       - pylint
       - pytest
       - integration-tests
-      - pyupgrade
       - clang-tidy
-      - list-components
+      - determine-jobs
       - test-build-components
       - test-build-components-splitter
       - test-build-components-split
+      - pre-commit-ci-lite
     if: always()
     steps:
       - name: Success

.github/workflows/codeowner-review-request.yml

@@ -0,0 +1,324 @@
+# This workflow automatically requests reviews from codeowners when:
+# 1. A PR is opened, reopened, or synchronized (updated)
+# 2. A PR is marked as ready for review
+#
+# It reads the CODEOWNERS file and matches all changed files in the PR against
+# the codeowner patterns, then requests reviews from the appropriate owners
+# while avoiding duplicate requests for users who have already been requested
+# or have already reviewed the PR.
+
+name: Request Codeowner Reviews
+
+on:
+  # Needs to be pull_request_target to get write permissions
+  pull_request_target:
+    types: [opened, reopened, synchronize, ready_for_review]
+
+permissions:
+  pull-requests: write
+  contents: read
+
+jobs:
+  request-codeowner-reviews:
+    name: Run
+    if: ${{ !github.event.pull_request.draft }}
+    runs-on: ubuntu-latest
+    steps:
+      - name: Request reviews from component codeowners
+        uses: actions/github-script@v7.0.1
+        with:
+          script: |
+            const owner = context.repo.owner;
+            const repo = context.repo.repo;
+            const pr_number = context.payload.pull_request.number;
+
+            console.log(`Processing PR #${pr_number} for codeowner review requests`);
+
+            // Hidden marker to identify bot comments from this workflow
+            const BOT_COMMENT_MARKER = '<!-- codeowner-review-request-bot -->';
+
+            try {
+              // Get the list of changed files in this PR
+              const { data: files } = await github.rest.pulls.listFiles({
+                owner,
+                repo,
+                pull_number: pr_number
+              });
+
+              const changedFiles = files.map(file => file.filename);
+              console.log(`Found ${changedFiles.length} changed files`);
+
+              if (changedFiles.length === 0) {
+                console.log('No changed files found, skipping codeowner review requests');
+                return;
+              }
+
+              // Fetch CODEOWNERS file from root
+              const { data: codeownersFile } = await github.rest.repos.getContent({
+                owner,
+                repo,
+                path: 'CODEOWNERS',
+                ref: context.payload.pull_request.base.sha
+              });
+              const codeownersContent = Buffer.from(codeownersFile.content, 'base64').toString('utf8');
+
+              // Parse CODEOWNERS file to extract all patterns and their owners
+              const codeownersLines = codeownersContent.split('\n')
+                .map(line => line.trim())
+                .filter(line => line && !line.startsWith('#'));
+
+              const codeownersPatterns = [];
+
+              // Convert CODEOWNERS pattern to regex (robust glob handling)
+              function globToRegex(pattern) {
+                // Escape regex special characters except for glob wildcards
+                let regexStr = pattern
+                  .replace(/([.+^=!:${}()|[\]\\])/g, '\\$1') // escape regex chars
+                  .replace(/\*\*/g, '.*') // globstar
+                  .replace(/\*/g, '[^/]*') // single star
+                  .replace(/\?/g, '.'); // question mark
+                return new RegExp('^' + regexStr + '$');
+              }
+
+              // Helper function to create comment body
+              function createCommentBody(reviewersList, teamsList, matchedFileCount, isSuccessful = true) {
+                const reviewerMentions = reviewersList.map(r => `@${r}`);
+                const teamMentions = teamsList.map(t => `@${owner}/${t}`);
+                const allMentions = [...reviewerMentions, ...teamMentions].join(', ');
+
+                if (isSuccessful) {
+                  return `${BOT_COMMENT_MARKER}\n👋 Hi there! I've automatically requested reviews from codeowners based on the files changed in this PR.\n\n${allMentions} - You've been requested to review this PR as codeowner(s) of ${matchedFileCount} file(s) that were modified. Thanks for your time! 🙏`;
+                } else {
+                  return `${BOT_COMMENT_MARKER}\n👋 Hi there! This PR modifies ${matchedFileCount} file(s) with codeowners.\n\n${allMentions} - As codeowner(s) of the affected files, your review would be appreciated! 🙏\n\n_Note: Automatic review request may have failed, but you're still welcome to review._`;
+                }
+              }
+
+              for (const line of codeownersLines) {
+                const parts = line.split(/\s+/);
+                if (parts.length < 2) continue;
+
+                const pattern = parts[0];
+                const owners = parts.slice(1);
+
+                // Use robust glob-to-regex conversion
+                const regex = globToRegex(pattern);
+                codeownersPatterns.push({ pattern, regex, owners });
+              }
+
+              console.log(`Parsed ${codeownersPatterns.length} codeowner patterns`);
+
+              // Match changed files against CODEOWNERS patterns
+              const matchedOwners = new Set();
+              const matchedTeams = new Set();
+              const fileMatches = new Map(); // Track which files matched which patterns
+
+              for (const file of changedFiles) {
+                for (const { pattern, regex, owners } of codeownersPatterns) {
+                  if (regex.test(file)) {
+                    console.log(`File '${file}' matches pattern '${pattern}' with owners: ${owners.join(', ')}`);
+
+                    if (!fileMatches.has(file)) {
+                      fileMatches.set(file, []);
+                    }
+                    fileMatches.get(file).push({ pattern, owners });
+
+                    // Add owners to the appropriate set (remove @ prefix)
+                    for (const owner of owners) {
+                      const cleanOwner = owner.startsWith('@') ? owner.slice(1) : owner;
+                      if (cleanOwner.includes('/')) {
+                        // Team mention (org/team-name)
+                        const teamName = cleanOwner.split('/')[1];
+                        matchedTeams.add(teamName);
+                      } else {
+                        // Individual user
+                        matchedOwners.add(cleanOwner);
+                      }
+                    }
+                  }
+                }
+              }
+
+              if (matchedOwners.size === 0 && matchedTeams.size === 0) {
+                console.log('No codeowners found for any changed files');
+                return;
+              }
+
+              // Remove the PR author from reviewers
+              const prAuthor = context.payload.pull_request.user.login;
+              matchedOwners.delete(prAuthor);
+
+              // Get current reviewers to avoid duplicate requests (but still mention them)
+              const { data: prData } = await github.rest.pulls.get({
+                owner,
+                repo,
+                pull_number: pr_number
+              });
+
+              const currentReviewers = new Set();
+              const currentTeams = new Set();
+
+              if (prData.requested_reviewers) {
+                prData.requested_reviewers.forEach(reviewer => {
+                  currentReviewers.add(reviewer.login);
+                });
+              }
+
+              if (prData.requested_teams) {
+                prData.requested_teams.forEach(team => {
+                  currentTeams.add(team.slug);
+                });
+              }
+
+              // Check for completed reviews to avoid re-requesting users who have already reviewed
+              const { data: reviews } = await github.rest.pulls.listReviews({
+                owner,
+                repo,
+                pull_number: pr_number
+              });
+
+              const reviewedUsers = new Set();
+              reviews.forEach(review => {
+                reviewedUsers.add(review.user.login);
+              });
+
+              // Check for previous comments from this workflow to avoid duplicate pings
+              const comments = await github.paginate(
+                github.rest.issues.listComments,
+                {
+                  owner,
+                  repo,
+                  issue_number: pr_number
+                }
+              );
+
+              const previouslyPingedUsers = new Set();
+              const previouslyPingedTeams = new Set();
+
+              // Look for comments from github-actions bot that contain our bot marker
+              const workflowComments = comments.filter(comment =>
+                comment.user.type === 'Bot' &&
+                comment.body.includes(BOT_COMMENT_MARKER)
+              );
+
+              // Extract previously mentioned users and teams from workflow comments
+              for (const comment of workflowComments) {
+                // Match @username patterns (not team mentions)
+                const userMentions = comment.body.match(/@([a-zA-Z0-9_.-]+)(?![/])/g) || [];
+                userMentions.forEach(mention => {
+                  const username = mention.slice(1); // remove @
+                  previouslyPingedUsers.add(username);
+                });
+
+                // Match @org/team patterns
+                const teamMentions = comment.body.match(/@[a-zA-Z0-9_.-]+\/([a-zA-Z0-9_.-]+)/g) || [];
+                teamMentions.forEach(mention => {
+                  const teamName = mention.split('/')[1];
+                  previouslyPingedTeams.add(teamName);
+                });
+              }
+
+              console.log(`Found ${previouslyPingedUsers.size} previously pinged users and ${previouslyPingedTeams.size} previously pinged teams`);
+
+              // Remove users who have already been pinged in previous workflow comments
+              previouslyPingedUsers.forEach(user => {
+                matchedOwners.delete(user);
+              });
+
+              previouslyPingedTeams.forEach(team => {
+                matchedTeams.delete(team);
+              });
+
+              // Remove only users who have already submitted reviews (not just requested reviewers)
+              reviewedUsers.forEach(reviewer => {
+                matchedOwners.delete(reviewer);
+              });
+
+              // For teams, we'll still remove already requested teams to avoid API errors
+              currentTeams.forEach(team => {
+                matchedTeams.delete(team);
+              });
+
+              const reviewersList = Array.from(matchedOwners);
+              const teamsList = Array.from(matchedTeams);
+
+              if (reviewersList.length === 0 && teamsList.length === 0) {
+                console.log('No eligible reviewers found (all may already be requested, reviewed, or previously pinged)');
+                return;
+              }
+
+              const totalReviewers = reviewersList.length + teamsList.length;
+              console.log(`Requesting reviews from ${reviewersList.length} users and ${teamsList.length} teams for ${fileMatches.size} matched files`);
+
+              // Request reviews
+              try {
+                const requestParams = {
+                  owner,
+                  repo,
+                  pull_number: pr_number
+                };
+
+                // Filter out users who are already requested reviewers for the API call
+                const newReviewers = reviewersList.filter(reviewer => !currentReviewers.has(reviewer));
+                const newTeams = teamsList.filter(team => !currentTeams.has(team));
+
+                if (newReviewers.length > 0) {
+                  requestParams.reviewers = newReviewers;
+                }
+
+                if (newTeams.length > 0) {
+                  requestParams.team_reviewers = newTeams;
+                }
+
+                // Only make the API call if there are new reviewers to request
+                if (newReviewers.length > 0 || newTeams.length > 0) {
+                  await github.rest.pulls.requestReviewers(requestParams);
+                  console.log(`Successfully requested reviews from ${newReviewers.length} new users and ${newTeams.length} new teams`);
+                } else {
+                  console.log('All codeowners are already requested reviewers or have reviewed');
+                }
+
+                // Only add a comment if there are new codeowners to mention (not previously pinged)
+                if (reviewersList.length > 0 || teamsList.length > 0) {
+                  const commentBody = createCommentBody(reviewersList, teamsList, fileMatches.size, true);
+
+                  await github.rest.issues.createComment({
+                    owner,
+                    repo,
+                    issue_number: pr_number,
+                    body: commentBody
+                  });
+                  console.log(`Added comment mentioning ${reviewersList.length} users and ${teamsList.length} teams`);
+                } else {
+                  console.log('No new codeowners to mention in comment (all previously pinged)');
+                }
+              } catch (error) {
+                if (error.status === 422) {
+                  console.log('Some reviewers may already be requested or unavailable:', error.message);
+
+                  // Only try to add a comment if there are new codeowners to mention
+                  if (reviewersList.length > 0 || teamsList.length > 0) {
+                    const commentBody = createCommentBody(reviewersList, teamsList, fileMatches.size, false);
+
+                    try {
+                      await github.rest.issues.createComment({
+                        owner,
+                        repo,
+                        issue_number: pr_number,
+                        body: commentBody
+                      });
+                      console.log(`Added fallback comment mentioning ${reviewersList.length} users and ${teamsList.length} teams`);
+                    } catch (commentError) {
+                      console.log('Failed to add comment:', commentError.message);
+                    }
+                  } else {
+                    console.log('No new codeowners to mention in fallback comment');
+                  }
+                } else {
+                  throw error;
+                }
+              }
+
+            } catch (error) {
+              console.log('Failed to process codeowner review requests:', error.message);
+              console.error(error);
+            }

.github/workflows/external-component-bot.yml

@@ -0,0 +1,157 @@
+name: Add External Component Comment
+
+on:
+  pull_request_target:
+    types: [opened, synchronize]
+
+permissions:
+  contents: read       #  Needed to fetch PR details
+  issues: write        #  Needed to create and update comments (PR comments are managed via the issues REST API)
+  pull-requests: write  # also needed?
+
+jobs:
+  external-comment:
+    name: External component comment
+    runs-on: ubuntu-latest
+    steps:
+      - name: Add external component comment
+        uses: actions/github-script@v7.0.1
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          script: |
+            // Generate external component usage instructions
+            function generateExternalComponentInstructions(prNumber, componentNames, owner, repo) {
+                let source;
+                if (owner === 'esphome' && repo === 'esphome')
+                    source = `github://pr#${prNumber}`;
+                else
+                    source = `github://${owner}/${repo}@pull/${prNumber}/head`;
+                return `To use the changes from this PR as an external component, add the following to your ESPHome configuration YAML file:
+
+            \`\`\`yaml
+            external_components:
+              - source: ${source}
+                components: [${componentNames.join(', ')}]
+                refresh: 1h
+            \`\`\``;
+            }
+
+            // Generate repo clone instructions
+            function generateRepoInstructions(prNumber, owner, repo, branch) {
+                return `To use the changes in this PR:
+
+            \`\`\`bash
+            # Clone the repository:
+            git clone https://github.com/${owner}/${repo}
+            cd ${repo}
+
+            # Checkout the PR branch:
+            git fetch origin pull/${prNumber}/head:${branch}
+            git checkout ${branch}
+
+            # Install the development version:
+            script/setup
+
+            # Activate the development version:
+            source venv/bin/activate
+            \`\`\`
+
+            Now you can run \`esphome\` as usual to test the changes in this PR.
+            `;
+            }
+
+            async function createComment(octokit, owner, repo, prNumber, esphomeChanges, componentChanges) {
+                const commentMarker = "<!-- This comment was generated automatically by the external-component-bot workflow. -->";
+                const legacyCommentMarker = "<!-- This comment was generated automatically by a GitHub workflow. -->";
+                let commentBody;
+                if (esphomeChanges.length === 1) {
+                    commentBody = generateExternalComponentInstructions(prNumber, componentChanges, owner, repo);
+                } else {
+                    commentBody = generateRepoInstructions(prNumber, owner, repo, context.payload.pull_request.head.ref);
+                }
+                commentBody += `\n\n---\n(Added by the PR bot)\n\n${commentMarker}`;
+
+                // Check for existing bot comment
+                const comments = await github.paginate(
+                    github.rest.issues.listComments,
+                    {
+                        owner: owner,
+                        repo: repo,
+                        issue_number: prNumber,
+                        per_page: 100,
+                    }
+                );
+
+                const sorted = comments.sort((a, b) => new Date(b.updated_at) - new Date(a.updated_at));
+
+                const botComment = sorted.find(comment =>
+                    (
+                      comment.body.includes(commentMarker) ||
+                      comment.body.includes(legacyCommentMarker)
+                    ) && comment.user.type === "Bot"
+                );
+
+                if (botComment && botComment.body === commentBody) {
+                    // No changes in the comment, do nothing
+                    return;
+                }
+
+                if (botComment) {
+                    // Update existing comment
+                    await github.rest.issues.updateComment({
+                        owner: owner,
+                        repo: repo,
+                        comment_id: botComment.id,
+                        body: commentBody,
+                    });
+                } else {
+                    // Create new comment
+                    await github.rest.issues.createComment({
+                        owner: owner,
+                        repo: repo,
+                        issue_number: prNumber,
+                        body: commentBody,
+                    });
+                }
+            }
+
+            async function getEsphomeAndComponentChanges(github, owner, repo, prNumber) {
+                const changedFiles = await github.rest.pulls.listFiles({
+                    owner: owner,
+                    repo: repo,
+                    pull_number: prNumber,
+                });
+
+                const esphomeChanges = changedFiles.data
+                    .filter(file => file.filename !== "esphome/core/defines.h" && file.filename.startsWith('esphome/'))
+                    .map(file => {
+                        const match = file.filename.match(/esphome\/([^/]+)/);
+                        return match ? match[1] : null;
+                    })
+                    .filter(it => it !== null);
+
+                if (esphomeChanges.length === 0) {
+                    return {esphomeChanges: [], componentChanges: []};
+                }
+
+                const uniqueEsphomeChanges = [...new Set(esphomeChanges)];
+                const componentChanges = changedFiles.data
+                    .filter(file => file.filename.startsWith('esphome/components/'))
+                    .map(file => {
+                        const match = file.filename.match(/esphome\/components\/([^/]+)\//);
+                        return match ? match[1] : null;
+                    })
+                    .filter(it => it !== null);
+
+                return {esphomeChanges: uniqueEsphomeChanges, componentChanges: [...new Set(componentChanges)]};
+            }
+
+            // Start of main code.
+
+            const prNumber = context.payload.pull_request.number;
+            const {owner, repo} = context.repo;
+
+            const {esphomeChanges, componentChanges} = await getEsphomeAndComponentChanges(github, owner, repo, prNumber);
+            if (componentChanges.length !== 0) {
+                await createComment(github, owner, repo, prNumber, esphomeChanges, componentChanges);
+            }

.github/workflows/issue-codeowner-notify.yml

@@ -0,0 +1,163 @@
+# This workflow automatically notifies codeowners when an issue is labeled with component labels.
+# It reads the CODEOWNERS file to find the maintainers for the labeled components
+# and posts a comment mentioning them to ensure they're aware of the issue.
+
+name: Notify Issue Codeowners
+
+on:
+  issues:
+    types: [labeled]
+
+permissions:
+  issues: write
+  contents: read
+
+jobs:
+  notify-codeowners:
+    name: Run
+    if: ${{ startsWith(github.event.label.name, format('component{0} ', ':')) }}
+    runs-on: ubuntu-latest
+    steps:
+      - name: Notify codeowners for component issues
+        uses: actions/github-script@v7.0.1
+        with:
+          script: |
+            const owner = context.repo.owner;
+            const repo = context.repo.repo;
+            const issue_number = context.payload.issue.number;
+            const labelName = context.payload.label.name;
+
+            console.log(`Processing issue #${issue_number} with label: ${labelName}`);
+
+            // Hidden marker to identify bot comments from this workflow
+            const BOT_COMMENT_MARKER = '<!-- issue-codeowner-notify-bot -->';
+
+            // Extract component name from label
+            const componentName = labelName.replace('component: ', '');
+            console.log(`Component: ${componentName}`);
+
+            try {
+              // Fetch CODEOWNERS file from root
+              const { data: codeownersFile } = await github.rest.repos.getContent({
+                owner,
+                repo,
+                path: 'CODEOWNERS'
+              });
+              const codeownersContent = Buffer.from(codeownersFile.content, 'base64').toString('utf8');
+
+              // Parse CODEOWNERS file to extract component mappings
+              const codeownersLines = codeownersContent.split('\n')
+                .map(line => line.trim())
+                .filter(line => line && !line.startsWith('#'));
+
+              let componentOwners = null;
+
+              for (const line of codeownersLines) {
+                const parts = line.split(/\s+/);
+                if (parts.length < 2) continue;
+
+                const pattern = parts[0];
+                const owners = parts.slice(1);
+
+                // Look for component patterns: esphome/components/{component}/*
+                const componentMatch = pattern.match(/^esphome\/components\/([^\/]+)\/\*$/);
+                if (componentMatch && componentMatch[1] === componentName) {
+                  componentOwners = owners;
+                  break;
+                }
+              }
+
+              if (!componentOwners) {
+                console.log(`No codeowners found for component: ${componentName}`);
+                return;
+              }
+
+              console.log(`Found codeowners for '${componentName}': ${componentOwners.join(', ')}`);
+
+              // Separate users and teams
+              const userOwners = [];
+              const teamOwners = [];
+
+              for (const owner of componentOwners) {
+                const cleanOwner = owner.startsWith('@') ? owner.slice(1) : owner;
+                if (cleanOwner.includes('/')) {
+                  // Team mention (org/team-name)
+                  teamOwners.push(`@${cleanOwner}`);
+                } else {
+                  // Individual user
+                  userOwners.push(`@${cleanOwner}`);
+                }
+              }
+
+              // Remove issue author from mentions to avoid self-notification
+              const issueAuthor = context.payload.issue.user.login;
+              const filteredUserOwners = userOwners.filter(mention =>
+                mention !== `@${issueAuthor}`
+              );
+
+              // Check for previous comments from this workflow to avoid duplicate pings
+              const comments = await github.paginate(
+                github.rest.issues.listComments,
+                {
+                  owner,
+                  repo,
+                  issue_number: issue_number
+                }
+              );
+
+              const previouslyPingedUsers = new Set();
+              const previouslyPingedTeams = new Set();
+
+              // Look for comments from github-actions bot that contain codeowner pings for this component
+              const workflowComments = comments.filter(comment =>
+                comment.user.type === 'Bot' &&
+                comment.body.includes(BOT_COMMENT_MARKER) &&
+                comment.body.includes(`component: ${componentName}`)
+              );
+
+              // Extract previously mentioned users and teams from workflow comments
+              for (const comment of workflowComments) {
+                // Match @username patterns (not team mentions)
+                const userMentions = comment.body.match(/@([a-zA-Z0-9_.-]+)(?![/])/g) || [];
+                userMentions.forEach(mention => {
+                  previouslyPingedUsers.add(mention); // Keep @ prefix for easy comparison
+                });
+
+                // Match @org/team patterns
+                const teamMentions = comment.body.match(/@[a-zA-Z0-9_.-]+\/[a-zA-Z0-9_.-]+/g) || [];
+                teamMentions.forEach(mention => {
+                  previouslyPingedTeams.add(mention);
+                });
+              }
+
+              console.log(`Found ${previouslyPingedUsers.size} previously pinged users and ${previouslyPingedTeams.size} previously pinged teams for component ${componentName}`);
+
+              // Remove previously pinged users and teams
+              const newUserOwners = filteredUserOwners.filter(mention => !previouslyPingedUsers.has(mention));
+              const newTeamOwners = teamOwners.filter(mention => !previouslyPingedTeams.has(mention));
+
+              const allMentions = [...newUserOwners, ...newTeamOwners];
+
+              if (allMentions.length === 0) {
+                console.log('No new codeowners to notify (all previously pinged or issue author is the only codeowner)');
+                return;
+              }
+
+              // Create comment body
+              const mentionString = allMentions.join(', ');
+              const commentBody = `${BOT_COMMENT_MARKER}\n👋 Hey ${mentionString}!\n\nThis issue has been labeled with \`component: ${componentName}\` and you've been identified as a codeowner of this component. Please take a look when you have a chance!\n\nThanks for maintaining this component! 🙏`;
+
+              // Post comment
+              await github.rest.issues.createComment({
+                owner,
+                repo,
+                issue_number: issue_number,
+                body: commentBody
+              });
+
+              console.log(`Successfully notified new codeowners: ${mentionString}`);
+
+            } catch (error) {
+              console.log('Failed to process codeowner notifications:', error.message);
+              console.error(error);
+            }

.github/workflows/release.yml

@@ -96,7 +96,7 @@ jobs:
       - name: Set up Python
         uses: actions/setup-python@v5.6.0
         with:
-          python-version: "3.10"
+          python-version: "3.11"
 
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v3.11.1

.github/workflows/yaml-lint.yml

@@ -1,25 +0,0 @@
----
-name: YAML lint
-
-on:
-  push:
-    branches: [dev, beta, release]
-    paths:
-      - "**.yaml"
-      - "**.yml"
-  pull_request:
-    paths:
-      - "**.yaml"
-      - "**.yml"
-
-jobs:
-  yamllint:
-    name: yamllint
-    runs-on: ubuntu-latest
-    steps:
-      - name: Check out code from GitHub
-        uses: actions/checkout@v4.2.2
-      - name: Run yamllint
-        uses: frenck/action-yamllint@v1.5.0
-        with:
-          strict: true

.pre-commit-config.yaml

@@ -4,15 +4,14 @@
 
 ci:
   autoupdate_commit_msg: 'pre-commit: autoupdate'
-  autoupdate_schedule: weekly
-  autofix_prs: false
+  autoupdate_schedule: off  # Disabled until ruff versions are synced between deps and pre-commit
   # Skip hooks that have issues in pre-commit CI environment
-  skip: [pylint, yamllint]
+  skip: [pylint, clang-tidy-hash]
 
 repos:
   - repo: https://github.com/astral-sh/ruff-pre-commit
     # Ruff version.
-    rev: v0.12.2
+    rev: v0.12.5
     hooks:
       # Run the linter.
       - id: ruff
@@ -28,22 +27,25 @@ repos:
           - pydocstyle==5.1.1
         files: ^(esphome|tests)/.+\.py$
   - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v3.4.0
+    rev: v5.0.0
     hooks:
       - id: no-commit-to-branch
         args:
           - --branch=dev
           - --branch=release
           - --branch=beta
+      - id: end-of-file-fixer
+      - id: trailing-whitespace
   - repo: https://github.com/asottile/pyupgrade
     rev: v3.20.0
     hooks:
       - id: pyupgrade
-        args: [--py310-plus]
+        args: [--py311-plus]
   - repo: https://github.com/adrienverge/yamllint.git
     rev: v1.37.1
     hooks:
       - id: yamllint
+        exclude: ^(\.clang-format|\.clang-tidy)$
   - repo: https://github.com/pre-commit/mirrors-clang-format
     rev: v13.0.1
     hooks:
@@ -56,3 +58,10 @@ repos:
         entry: python3 script/run-in-env.py pylint
         language: system
         types: [python]
+      - id: clang-tidy-hash
+        name: Update clang-tidy hash
+        entry: python script/clang_tidy_hash.py --update-if-changed
+        language: python
+        files: ^(\.clang-tidy|platformio\.ini|requirements_dev\.txt)$
+        pass_filenames: false
+        additional_dependencies: []

CLAUDE.md

@@ -0,0 +1 @@
+.ai/instructions.md

CODEOWNERS

@@ -9,6 +9,7 @@
 pyproject.toml @esphome/core
 esphome/*.py @esphome/core
 esphome/core/* @esphome/core
+.github/** @esphome/core
 
 # Integrations
 esphome/components/a01nyub/* @MrSuicideParrot
@@ -28,7 +29,7 @@ esphome/components/aic3204/* @kbx81
 esphome/components/airthings_ble/* @jeromelaban
 esphome/components/airthings_wave_base/* @jeromelaban @kpfleming @ncareau
 esphome/components/airthings_wave_mini/* @ncareau
-esphome/components/airthings_wave_plus/* @jeromelaban
+esphome/components/airthings_wave_plus/* @jeromelaban @precurse
 esphome/components/alarm_control_panel/* @grahambrown11 @hwstar
 esphome/components/alpha3/* @jan-hofmeier
 esphome/components/am2315c/* @swoboda1337
@@ -245,6 +246,7 @@ esphome/components/lcd_menu/* @numo68
 esphome/components/ld2410/* @regevbr @sebcaps
 esphome/components/ld2420/* @descipher
 esphome/components/ld2450/* @hareeshmu
+esphome/components/ld24xx/* @kbx81
 esphome/components/ledc/* @OttoWinter
 esphome/components/libretiny/* @kuba2k2
 esphome/components/libretiny_pwm/* @kuba2k2
@@ -324,6 +326,7 @@ esphome/components/nextion/text_sensor/* @senexcrenshaw
 esphome/components/nfc/* @jesserockz @kbx81
 esphome/components/noblex/* @AGalfra
 esphome/components/npi19/* @bakerkj
+esphome/components/nrf52/* @tomaszduda23
 esphome/components/number/* @esphome/core
 esphome/components/one_wire/* @ssieb
 esphome/components/online_image/* @clydebarrow @guillempages
@@ -378,6 +381,7 @@ esphome/components/rp2040_pwm/* @jesserockz
 esphome/components/rpi_dpi_rgb/* @clydebarrow
 esphome/components/rtl87xx/* @kuba2k2
 esphome/components/rtttl/* @glmnet
+esphome/components/runtime_stats/* @bdraco
 esphome/components/safe_mode/* @jsuanet @kbx81 @paulmonigatti
 esphome/components/scd4x/* @martgras @sjtrny
 esphome/components/script/* @esphome/core
@@ -535,5 +539,6 @@ esphome/components/xiaomi_xmwsdj04mmc/* @medusalix
 esphome/components/xl9535/* @mreditor97
 esphome/components/xpt2046/touchscreen/* @nielsnl68 @numo68
 esphome/components/xxtea/* @clydebarrow
+esphome/components/zephyr/* @tomaszduda23
 esphome/components/zhlt01/* @cfeenstra1024
 esphome/components/zio_ultrasonic/* @kahrendt

CONTRIBUTING.md

@@ -7,7 +7,7 @@ project and be sure to join us on [Discord](https://discord.gg/KhAMKrd).
 
 **See also:**
 
-[Documentation](https://esphome.io) -- [Issues](https://github.com/esphome/issues/issues) -- [Feature requests](https://github.com/esphome/feature-requests/issues)
+[Documentation](https://esphome.io) -- [Issues](https://github.com/esphome/esphome/issues) -- [Feature requests](https://github.com/orgs/esphome/discussions)
 
 ---
 

Doxyfile

@@ -48,7 +48,7 @@ PROJECT_NAME           = ESPHome
 # could be handy for archiving the generated documentation or if some version
 # control system is used.
 
-PROJECT_NUMBER         = 2025.7.3
+PROJECT_NUMBER         = 2025.8.0-dev
 
 # Using the PROJECT_BRIEF tag one can provide an optional one line description
 # for a project that appears at the top of each page and should give viewer a

GEMINI.md

@@ -0,0 +1 @@
+.ai/instructions.md

README.md

@@ -9,7 +9,7 @@
 
 ---
 
-[Documentation](https://esphome.io) -- [Issues](https://github.com/esphome/issues/issues) -- [Feature requests](https://github.com/esphome/feature-requests/issues)
+[Documentation](https://esphome.io) -- [Issues](https://github.com/esphome/esphome/issues) -- [Feature requests](https://github.com/orgs/esphome/discussions)
 
 ---
 

esphome/__main__.py

@@ -34,6 +34,7 @@ from esphome.const import (
     CONF_PORT,
     CONF_SUBSTITUTIONS,
     CONF_TOPIC,
+    ENV_NOGITIGNORE,
     PLATFORM_ESP32,
     PLATFORM_ESP8266,
     PLATFORM_RP2040,
@@ -88,9 +89,9 @@ def choose_prompt(options, purpose: str = None):
 def choose_upload_log_host(
     default, check_default, show_ota, show_mqtt, show_api, purpose: str = None
 ):
-    options = []
-    for port in get_serial_ports():
-        options.append((f"{port.path} ({port.description})", port.path))
+    options = [
+        (f"{port.path} ({port.description})", port.path) for port in get_serial_ports()
+    ]
     if default == "SERIAL":
         return choose_prompt(options, purpose=purpose)
     if (show_ota and "ota" in CORE.config) or (show_api and "api" in CORE.config):
@@ -118,9 +119,7 @@ def mqtt_logging_enabled(mqtt_config):
         return False
     if CONF_TOPIC not in log_topic:
         return False
-    if log_topic.get(CONF_LEVEL, None) == "NONE":
-        return False
-    return True
+    return log_topic.get(CONF_LEVEL, None) != "NONE"
 
 
 def get_port_type(port):
@@ -209,6 +208,9 @@ def wrap_to_code(name, comp):
 
 
 def write_cpp(config):
+    if not get_bool_env(ENV_NOGITIGNORE):
+        writer.write_gitignore()
+
     generate_cpp_contents(config)
     return write_cpp_file()
 
@@ -225,10 +227,13 @@ def generate_cpp_contents(config):
 
 
 def write_cpp_file():
-    writer.write_platformio_project()
-
     code_s = indent(CORE.cpp_main_section)
     writer.write_cpp(code_s)
+
+    from esphome.build_gen import platformio
+
+    platformio.write_project()
+
     return 0
 
 
@@ -458,6 +463,13 @@ def command_vscode(args):
 
 
 def command_compile(args, config):
+    # Set memory analysis options in config
+    if args.analyze_memory:
+        config.setdefault(CONF_ESPHOME, {})["analyze_memory"] = True
+
+    if args.memory_report:
+        config.setdefault(CONF_ESPHOME, {})["memory_report_file"] = args.memory_report
+
     exit_code = write_cpp(config)
     if exit_code != 0:
         return exit_code
@@ -837,6 +849,17 @@ def parse_args(argv):
         help="Only generate source code, do not compile.",
         action="store_true",
     )
+    parser_compile.add_argument(
+        "--analyze-memory",
+        help="Analyze and display memory usage by component after compilation.",
+        action="store_true",
+    )
+    parser_compile.add_argument(
+        "--memory-report",
+        help="Save memory analysis report to a file (supports .json or .txt).",
+        type=str,
+        metavar="FILE",
+    )
 
     parser_upload = subparsers.add_parser(
         "upload",

esphome/build_gen/platformio.py

@@ -0,0 +1,102 @@
+import os
+
+from esphome.const import __version__
+from esphome.core import CORE
+from esphome.helpers import mkdir_p, read_file, write_file_if_changed
+from esphome.writer import find_begin_end, update_storage_json
+
+INI_AUTO_GENERATE_BEGIN = "; ========== AUTO GENERATED CODE BEGIN ==========="
+INI_AUTO_GENERATE_END = "; =========== AUTO GENERATED CODE END ============"
+
+INI_BASE_FORMAT = (
+    """; Auto generated code by esphome
+
+[common]
+lib_deps =
+build_flags =
+upload_flags =
+
+""",
+    """
+
+""",
+)
+
+
+def format_ini(data: dict[str, str | list[str]]) -> str:
+    content = ""
+    for key, value in sorted(data.items()):
+        if isinstance(value, list):
+            content += f"{key} =\n"
+            for x in value:
+                content += f"    {x}\n"
+        else:
+            content += f"{key} = {value}\n"
+    return content
+
+
+def get_ini_content():
+    CORE.add_platformio_option(
+        "lib_deps",
+        [x.as_lib_dep for x in CORE.platformio_libraries.values()]
+        + ["${common.lib_deps}"],
+    )
+    # Sort to avoid changing build flags order
+    CORE.add_platformio_option("build_flags", sorted(CORE.build_flags))
+
+    # Sort to avoid changing build unflags order
+    CORE.add_platformio_option("build_unflags", sorted(CORE.build_unflags))
+
+    # Add extra script for C++ flags
+    CORE.add_platformio_option("extra_scripts", [f"pre:{CXX_FLAGS_FILE_NAME}"])
+
+    content = "[platformio]\n"
+    content += f"description = ESPHome {__version__}\n"
+
+    content += f"[env:{CORE.name}]\n"
+    content += format_ini(CORE.platformio_options)
+
+    return content
+
+
+def write_ini(content):
+    update_storage_json()
+    path = CORE.relative_build_path("platformio.ini")
+
+    if os.path.isfile(path):
+        text = read_file(path)
+        content_format = find_begin_end(
+            text, INI_AUTO_GENERATE_BEGIN, INI_AUTO_GENERATE_END
+        )
+    else:
+        content_format = INI_BASE_FORMAT
+    full_file = f"{content_format[0] + INI_AUTO_GENERATE_BEGIN}\n{content}"
+    full_file += INI_AUTO_GENERATE_END + content_format[1]
+    write_file_if_changed(path, full_file)
+
+
+def write_project():
+    mkdir_p(CORE.build_path)
+
+    content = get_ini_content()
+    write_ini(content)
+
+    # Write extra script for C++ specific flags
+    write_cxx_flags_script()
+
+
+CXX_FLAGS_FILE_NAME = "cxx_flags.py"
+CXX_FLAGS_FILE_CONTENTS = """# Auto-generated ESPHome script for C++ specific compiler flags
+Import("env")
+
+# Add C++ specific flags
+"""
+
+
+def write_cxx_flags_script() -> None:
+    path = CORE.relative_build_path(CXX_FLAGS_FILE_NAME)
+    contents = CXX_FLAGS_FILE_CONTENTS
+    if not CORE.is_host:
+        contents += 'env.Append(CXXFLAGS=["-Wno-volatile"])'
+        contents += "\n"
+    write_file_if_changed(path, contents)

esphome/components/adc/__init__.py

@@ -5,6 +5,7 @@ from esphome.components.esp32.const import (
     VARIANT_ESP32,
     VARIANT_ESP32C2,
     VARIANT_ESP32C3,
+    VARIANT_ESP32C5,
     VARIANT_ESP32C6,
     VARIANT_ESP32H2,
     VARIANT_ESP32S2,
@@ -51,82 +52,93 @@ SAMPLING_MODES = {
     "max": sampling_mode.MAX,
 }
 
-adc1_channel_t = cg.global_ns.enum("adc1_channel_t")
-adc2_channel_t = cg.global_ns.enum("adc2_channel_t")
+adc_unit_t = cg.global_ns.enum("adc_unit_t", is_class=True)
+
+adc_channel_t = cg.global_ns.enum("adc_channel_t", is_class=True)
 
 # pin to adc1 channel mapping
 # https://github.com/espressif/esp-idf/blob/v4.4.8/components/driver/include/driver/adc.h
 ESP32_VARIANT_ADC1_PIN_TO_CHANNEL = {
     # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32/include/soc/adc_channel.h
     VARIANT_ESP32: {
-        36: adc1_channel_t.ADC1_CHANNEL_0,
-        37: adc1_channel_t.ADC1_CHANNEL_1,
-        38: adc1_channel_t.ADC1_CHANNEL_2,
-        39: adc1_channel_t.ADC1_CHANNEL_3,
-        32: adc1_channel_t.ADC1_CHANNEL_4,
-        33: adc1_channel_t.ADC1_CHANNEL_5,
-        34: adc1_channel_t.ADC1_CHANNEL_6,
-        35: adc1_channel_t.ADC1_CHANNEL_7,
+        36: adc_channel_t.ADC_CHANNEL_0,
+        37: adc_channel_t.ADC_CHANNEL_1,
+        38: adc_channel_t.ADC_CHANNEL_2,
+        39: adc_channel_t.ADC_CHANNEL_3,
+        32: adc_channel_t.ADC_CHANNEL_4,
+        33: adc_channel_t.ADC_CHANNEL_5,
+        34: adc_channel_t.ADC_CHANNEL_6,
+        35: adc_channel_t.ADC_CHANNEL_7,
     },
     # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32c2/include/soc/adc_channel.h
     VARIANT_ESP32C2: {
-        0: adc1_channel_t.ADC1_CHANNEL_0,
-        1: adc1_channel_t.ADC1_CHANNEL_1,
-        2: adc1_channel_t.ADC1_CHANNEL_2,
-        3: adc1_channel_t.ADC1_CHANNEL_3,
-        4: adc1_channel_t.ADC1_CHANNEL_4,
+        0: adc_channel_t.ADC_CHANNEL_0,
+        1: adc_channel_t.ADC_CHANNEL_1,
+        2: adc_channel_t.ADC_CHANNEL_2,
+        3: adc_channel_t.ADC_CHANNEL_3,
+        4: adc_channel_t.ADC_CHANNEL_4,
     },
     # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32c3/include/soc/adc_channel.h
     VARIANT_ESP32C3: {
-        0: adc1_channel_t.ADC1_CHANNEL_0,
-        1: adc1_channel_t.ADC1_CHANNEL_1,
-        2: adc1_channel_t.ADC1_CHANNEL_2,
-        3: adc1_channel_t.ADC1_CHANNEL_3,
-        4: adc1_channel_t.ADC1_CHANNEL_4,
+        0: adc_channel_t.ADC_CHANNEL_0,
+        1: adc_channel_t.ADC_CHANNEL_1,
+        2: adc_channel_t.ADC_CHANNEL_2,
+        3: adc_channel_t.ADC_CHANNEL_3,
+        4: adc_channel_t.ADC_CHANNEL_4,
+    },
+    # ESP32-C5 ADC1 pin mapping - based on official ESP-IDF documentation
+    # https://docs.espressif.com/projects/esp-idf/en/latest/esp32c5/api-reference/peripherals/gpio.html
+    VARIANT_ESP32C5: {
+        1: adc_channel_t.ADC_CHANNEL_0,
+        2: adc_channel_t.ADC_CHANNEL_1,
+        3: adc_channel_t.ADC_CHANNEL_2,
+        4: adc_channel_t.ADC_CHANNEL_3,
+        5: adc_channel_t.ADC_CHANNEL_4,
+        6: adc_channel_t.ADC_CHANNEL_5,
     },
     # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32c6/include/soc/adc_channel.h
     VARIANT_ESP32C6: {
-        0: adc1_channel_t.ADC1_CHANNEL_0,
-        1: adc1_channel_t.ADC1_CHANNEL_1,
-        2: adc1_channel_t.ADC1_CHANNEL_2,
-        3: adc1_channel_t.ADC1_CHANNEL_3,
-        4: adc1_channel_t.ADC1_CHANNEL_4,
-        5: adc1_channel_t.ADC1_CHANNEL_5,
-        6: adc1_channel_t.ADC1_CHANNEL_6,
+        0: adc_channel_t.ADC_CHANNEL_0,
+        1: adc_channel_t.ADC_CHANNEL_1,
+        2: adc_channel_t.ADC_CHANNEL_2,
+        3: adc_channel_t.ADC_CHANNEL_3,
+        4: adc_channel_t.ADC_CHANNEL_4,
+        5: adc_channel_t.ADC_CHANNEL_5,
+        6: adc_channel_t.ADC_CHANNEL_6,
     },
     # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32h2/include/soc/adc_channel.h
     VARIANT_ESP32H2: {
-        1: adc1_channel_t.ADC1_CHANNEL_0,
-        2: adc1_channel_t.ADC1_CHANNEL_1,
-        3: adc1_channel_t.ADC1_CHANNEL_2,
-        4: adc1_channel_t.ADC1_CHANNEL_3,
-        5: adc1_channel_t.ADC1_CHANNEL_4,
+        1: adc_channel_t.ADC_CHANNEL_0,
+        2: adc_channel_t.ADC_CHANNEL_1,
+        3: adc_channel_t.ADC_CHANNEL_2,
+        4: adc_channel_t.ADC_CHANNEL_3,
+        5: adc_channel_t.ADC_CHANNEL_4,
     },
     # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32s2/include/soc/adc_channel.h
     VARIANT_ESP32S2: {
-        1: adc1_channel_t.ADC1_CHANNEL_0,
-        2: adc1_channel_t.ADC1_CHANNEL_1,
-        3: adc1_channel_t.ADC1_CHANNEL_2,
-        4: adc1_channel_t.ADC1_CHANNEL_3,
-        5: adc1_channel_t.ADC1_CHANNEL_4,
-        6: adc1_channel_t.ADC1_CHANNEL_5,
-        7: adc1_channel_t.ADC1_CHANNEL_6,
-        8: adc1_channel_t.ADC1_CHANNEL_7,
-        9: adc1_channel_t.ADC1_CHANNEL_8,
-        10: adc1_channel_t.ADC1_CHANNEL_9,
+        1: adc_channel_t.ADC_CHANNEL_0,
+        2: adc_channel_t.ADC_CHANNEL_1,
+        3: adc_channel_t.ADC_CHANNEL_2,
+        4: adc_channel_t.ADC_CHANNEL_3,
+        5: adc_channel_t.ADC_CHANNEL_4,
+        6: adc_channel_t.ADC_CHANNEL_5,
+        7: adc_channel_t.ADC_CHANNEL_6,
+        8: adc_channel_t.ADC_CHANNEL_7,
+        9: adc_channel_t.ADC_CHANNEL_8,
+        10: adc_channel_t.ADC_CHANNEL_9,
     },
     # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32s3/include/soc/adc_channel.h
     VARIANT_ESP32S3: {
-        1: adc1_channel_t.ADC1_CHANNEL_0,
-        2: adc1_channel_t.ADC1_CHANNEL_1,
-        3: adc1_channel_t.ADC1_CHANNEL_2,
-        4: adc1_channel_t.ADC1_CHANNEL_3,
-        5: adc1_channel_t.ADC1_CHANNEL_4,
-        6: adc1_channel_t.ADC1_CHANNEL_5,
-        7: adc1_channel_t.ADC1_CHANNEL_6,
-        8: adc1_channel_t.ADC1_CHANNEL_7,
-        9: adc1_channel_t.ADC1_CHANNEL_8,
-        10: adc1_channel_t.ADC1_CHANNEL_9,
+        1: adc_channel_t.ADC_CHANNEL_0,
+        2: adc_channel_t.ADC_CHANNEL_1,
+        3: adc_channel_t.ADC_CHANNEL_2,
+        4: adc_channel_t.ADC_CHANNEL_3,
+        5: adc_channel_t.ADC_CHANNEL_4,
+        6: adc_channel_t.ADC_CHANNEL_5,
+        7: adc_channel_t.ADC_CHANNEL_6,
+        8: adc_channel_t.ADC_CHANNEL_7,
+        9: adc_channel_t.ADC_CHANNEL_8,
+        10: adc_channel_t.ADC_CHANNEL_9,
     },
 }
 
@@ -135,54 +147,56 @@ ESP32_VARIANT_ADC1_PIN_TO_CHANNEL = {
 ESP32_VARIANT_ADC2_PIN_TO_CHANNEL = {
     # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32/include/soc/adc_channel.h
     VARIANT_ESP32: {
-        4: adc2_channel_t.ADC2_CHANNEL_0,
-        0: adc2_channel_t.ADC2_CHANNEL_1,
-        2: adc2_channel_t.ADC2_CHANNEL_2,
-        15: adc2_channel_t.ADC2_CHANNEL_3,
-        13: adc2_channel_t.ADC2_CHANNEL_4,
-        12: adc2_channel_t.ADC2_CHANNEL_5,
-        14: adc2_channel_t.ADC2_CHANNEL_6,
-        27: adc2_channel_t.ADC2_CHANNEL_7,
-        25: adc2_channel_t.ADC2_CHANNEL_8,
-        26: adc2_channel_t.ADC2_CHANNEL_9,
+        4: adc_channel_t.ADC_CHANNEL_0,
+        0: adc_channel_t.ADC_CHANNEL_1,
+        2: adc_channel_t.ADC_CHANNEL_2,
+        15: adc_channel_t.ADC_CHANNEL_3,
+        13: adc_channel_t.ADC_CHANNEL_4,
+        12: adc_channel_t.ADC_CHANNEL_5,
+        14: adc_channel_t.ADC_CHANNEL_6,
+        27: adc_channel_t.ADC_CHANNEL_7,
+        25: adc_channel_t.ADC_CHANNEL_8,
+        26: adc_channel_t.ADC_CHANNEL_9,
     },
     # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32c2/include/soc/adc_channel.h
     VARIANT_ESP32C2: {
-        5: adc2_channel_t.ADC2_CHANNEL_0,
+        5: adc_channel_t.ADC_CHANNEL_0,
     },
     # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32c3/include/soc/adc_channel.h
     VARIANT_ESP32C3: {
-        5: adc2_channel_t.ADC2_CHANNEL_0,
+        5: adc_channel_t.ADC_CHANNEL_0,
     },
+    # ESP32-C5 has no ADC2 channels
+    VARIANT_ESP32C5: {},  # no ADC2
     # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32c6/include/soc/adc_channel.h
     VARIANT_ESP32C6: {},  # no ADC2
     # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32h2/include/soc/adc_channel.h
     VARIANT_ESP32H2: {},  # no ADC2
     # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32s2/include/soc/adc_channel.h
     VARIANT_ESP32S2: {
-        11: adc2_channel_t.ADC2_CHANNEL_0,
-        12: adc2_channel_t.ADC2_CHANNEL_1,
-        13: adc2_channel_t.ADC2_CHANNEL_2,
-        14: adc2_channel_t.ADC2_CHANNEL_3,
-        15: adc2_channel_t.ADC2_CHANNEL_4,
-        16: adc2_channel_t.ADC2_CHANNEL_5,
-        17: adc2_channel_t.ADC2_CHANNEL_6,
-        18: adc2_channel_t.ADC2_CHANNEL_7,
-        19: adc2_channel_t.ADC2_CHANNEL_8,
-        20: adc2_channel_t.ADC2_CHANNEL_9,
+        11: adc_channel_t.ADC_CHANNEL_0,
+        12: adc_channel_t.ADC_CHANNEL_1,
+        13: adc_channel_t.ADC_CHANNEL_2,
+        14: adc_channel_t.ADC_CHANNEL_3,
+        15: adc_channel_t.ADC_CHANNEL_4,
+        16: adc_channel_t.ADC_CHANNEL_5,
+        17: adc_channel_t.ADC_CHANNEL_6,
+        18: adc_channel_t.ADC_CHANNEL_7,
+        19: adc_channel_t.ADC_CHANNEL_8,
+        20: adc_channel_t.ADC_CHANNEL_9,
     },
     # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32s3/include/soc/adc_channel.h
     VARIANT_ESP32S3: {
-        11: adc2_channel_t.ADC2_CHANNEL_0,
-        12: adc2_channel_t.ADC2_CHANNEL_1,
-        13: adc2_channel_t.ADC2_CHANNEL_2,
-        14: adc2_channel_t.ADC2_CHANNEL_3,
-        15: adc2_channel_t.ADC2_CHANNEL_4,
-        16: adc2_channel_t.ADC2_CHANNEL_5,
-        17: adc2_channel_t.ADC2_CHANNEL_6,
-        18: adc2_channel_t.ADC2_CHANNEL_7,
-        19: adc2_channel_t.ADC2_CHANNEL_8,
-        20: adc2_channel_t.ADC2_CHANNEL_9,
+        11: adc_channel_t.ADC_CHANNEL_0,
+        12: adc_channel_t.ADC_CHANNEL_1,
+        13: adc_channel_t.ADC_CHANNEL_2,
+        14: adc_channel_t.ADC_CHANNEL_3,
+        15: adc_channel_t.ADC_CHANNEL_4,
+        16: adc_channel_t.ADC_CHANNEL_5,
+        17: adc_channel_t.ADC_CHANNEL_6,
+        18: adc_channel_t.ADC_CHANNEL_7,
+        19: adc_channel_t.ADC_CHANNEL_8,
+        20: adc_channel_t.ADC_CHANNEL_9,
     },
 }
 

esphome/components/adc/adc_sensor.h

@@ -3,11 +3,14 @@
 #include "esphome/components/sensor/sensor.h"
 #include "esphome/components/voltage_sampler/voltage_sampler.h"
 #include "esphome/core/component.h"
+#include "esphome/core/defines.h"
 #include "esphome/core/hal.h"
 
 #ifdef USE_ESP32
-#include <esp_adc_cal.h>
-#include "driver/adc.h"
+#include "esp_adc/adc_cali.h"
+#include "esp_adc/adc_cali_scheme.h"
+#include "esp_adc/adc_oneshot.h"
+#include "hal/adc_types.h"  // This defines ADC_CHANNEL_MAX
 #endif                      // USE_ESP32
 
 namespace esphome {
@@ -49,36 +52,72 @@ class Aggregator {
 
 class ADCSensor : public sensor::Sensor, public PollingComponent, public voltage_sampler::VoltageSampler {
  public:
-#ifdef USE_ESP32
-  /// Set the attenuation for this pin. Only available on the ESP32.
-  void set_attenuation(adc_atten_t attenuation) { this->attenuation_ = attenuation; }
-  void set_channel1(adc1_channel_t channel) {
-    this->channel1_ = channel;
-    this->channel2_ = ADC2_CHANNEL_MAX;
-  }
-  void set_channel2(adc2_channel_t channel) {
-    this->channel2_ = channel;
-    this->channel1_ = ADC1_CHANNEL_MAX;
-  }
-  void set_autorange(bool autorange) { this->autorange_ = autorange; }
-#endif  // USE_ESP32
-
-  /// Update ADC values
+  /// Update the sensor's state by reading the current ADC value.
+  /// This method is called periodically based on the update interval.
   void update() override;
-  /// Setup ADC
+
+  /// Set up the ADC sensor by initializing hardware and calibration parameters.
+  /// This method is called once during device initialization.
   void setup() override;
+
+  /// Output the configuration details of the ADC sensor for debugging purposes.
+  /// This method is called during the ESPHome setup process to log the configuration.
   void dump_config() override;
-  /// `HARDWARE_LATE` setup priority
+
+  /// Return the setup priority for this component.
+  /// Components with higher priority are initialized earlier during setup.
+  /// @return A float representing the setup priority.
   float get_setup_priority() const override;
+
+  /// Set the GPIO pin to be used by the ADC sensor.
+  /// @param pin Pointer to an InternalGPIOPin representing the ADC input pin.
   void set_pin(InternalGPIOPin *pin) { this->pin_ = pin; }
+
+  /// Enable or disable the output of raw ADC values (unprocessed data).
+  /// @param output_raw Boolean indicating whether to output raw ADC values (true) or processed values (false).
   void set_output_raw(bool output_raw) { this->output_raw_ = output_raw; }
+
+  /// Set the number of samples to be taken for ADC readings to improve accuracy.
+  /// A higher sample count reduces noise but increases the reading time.
+  /// @param sample_count The number of samples (e.g., 1, 4, 8).
   void set_sample_count(uint8_t sample_count);
+
+  /// Set the sampling mode for how multiple ADC samples are combined into a single measurement.
+  ///
+  /// When multiple samples are taken (controlled by set_sample_count), they can be combined
+  /// in one of three ways:
+  ///   - SamplingMode::AVG: Compute the average (default)
+  ///   - SamplingMode::MIN: Use the lowest sample value
+  ///   - SamplingMode::MAX: Use the highest sample value
+  /// @param sampling_mode The desired sampling mode to use for aggregating ADC samples.
   void set_sampling_mode(SamplingMode sampling_mode);
+
+  /// Perform a single ADC sampling operation and return the measured value.
+  /// This function handles raw readings, calibration, and averaging as needed.
+  /// @return The sampled value as a float.
   float sample() override;
 
-#ifdef USE_ESP8266
-  std::string unique_id() override;
-#endif  // USE_ESP8266
+#ifdef USE_ESP32
+  /// Set the ADC attenuation level to adjust the input voltage range.
+  /// This determines how the ADC interprets input voltages, allowing for greater precision
+  /// or the ability to measure higher voltages depending on the chosen attenuation level.
+  /// @param attenuation The desired ADC attenuation level (e.g., ADC_ATTEN_DB_0, ADC_ATTEN_DB_11).
+  void set_attenuation(adc_atten_t attenuation) { this->attenuation_ = attenuation; }
+
+  /// Configure the ADC to use a specific channel on a specific ADC unit.
+  /// This sets the channel for single-shot or continuous ADC measurements.
+  /// @param unit The ADC unit to use (ADC_UNIT_1 or ADC_UNIT_2).
+  /// @param channel The ADC channel to configure, such as ADC_CHANNEL_0, ADC_CHANNEL_3, etc.
+  void set_channel(adc_unit_t unit, adc_channel_t channel) {
+    this->adc_unit_ = unit;
+    this->channel_ = channel;
+  }
+
+  /// Set whether autoranging should be enabled for the ADC.
+  /// Autoranging automatically adjusts the attenuation level to handle a wide range of input voltages.
+  /// @param autorange Boolean indicating whether to enable autoranging.
+  void set_autorange(bool autorange) { this->autorange_ = autorange; }
+#endif  // USE_ESP32
 
 #ifdef USE_RP2040
   void set_is_temperature() { this->is_temperature_ = true; }
@@ -90,17 +129,28 @@ class ADCSensor : public sensor::Sensor, public PollingComponent, public voltage
   InternalGPIOPin *pin_;
   SamplingMode sampling_mode_{SamplingMode::AVG};
 
+#ifdef USE_ESP32
+  float sample_autorange_();
+  float sample_fixed_attenuation_();
+  bool autorange_{false};
+  adc_oneshot_unit_handle_t adc_handle_{nullptr};
+  adc_cali_handle_t calibration_handle_{nullptr};
+  adc_atten_t attenuation_{ADC_ATTEN_DB_0};
+  adc_channel_t channel_;
+  adc_unit_t adc_unit_;
+  struct SetupFlags {
+    uint8_t init_complete : 1;
+    uint8_t config_complete : 1;
+    uint8_t handle_init_complete : 1;
+    uint8_t calibration_complete : 1;
+    uint8_t reserved : 4;
+  } setup_flags_{};
+  static adc_oneshot_unit_handle_t shared_adc_handles[2];
+#endif  // USE_ESP32
+
 #ifdef USE_RP2040
   bool is_temperature_{false};
 #endif  // USE_RP2040
-
-#ifdef USE_ESP32
-  adc_atten_t attenuation_{ADC_ATTEN_DB_0};
-  adc1_channel_t channel1_{ADC1_CHANNEL_MAX};
-  adc2_channel_t channel2_{ADC2_CHANNEL_MAX};
-  bool autorange_{false};
-  esp_adc_cal_characteristics_t cal_characteristics_[SOC_ADC_ATTEN_NUM] = {};
-#endif  // USE_ESP32
 };
 
 }  // namespace adc

esphome/components/adc/adc_sensor_esp32.cpp

@@ -8,145 +8,315 @@ namespace adc {
 
 static const char *const TAG = "adc.esp32";
 
-static const adc_bits_width_t ADC_WIDTH_MAX_SOC_BITS = static_cast<adc_bits_width_t>(ADC_WIDTH_MAX - 1);
+adc_oneshot_unit_handle_t ADCSensor::shared_adc_handles[2] = {nullptr, nullptr};
 
-#ifndef SOC_ADC_RTC_MAX_BITWIDTH
-#if USE_ESP32_VARIANT_ESP32S2
-static const int32_t SOC_ADC_RTC_MAX_BITWIDTH = 13;
-#else
-static const int32_t SOC_ADC_RTC_MAX_BITWIDTH = 12;
-#endif  // USE_ESP32_VARIANT_ESP32S2
-#endif  // SOC_ADC_RTC_MAX_BITWIDTH
-
-static const int ADC_MAX = (1 << SOC_ADC_RTC_MAX_BITWIDTH) - 1;
-static const int ADC_HALF = (1 << SOC_ADC_RTC_MAX_BITWIDTH) >> 1;
-
-void ADCSensor::setup() {
-  ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->get_name().c_str());
-
-  if (this->channel1_ != ADC1_CHANNEL_MAX) {
-    adc1_config_width(ADC_WIDTH_MAX_SOC_BITS);
-    if (!this->autorange_) {
-      adc1_config_channel_atten(this->channel1_, this->attenuation_);
-    }
-  } else if (this->channel2_ != ADC2_CHANNEL_MAX) {
-    if (!this->autorange_) {
-      adc2_config_channel_atten(this->channel2_, this->attenuation_);
-    }
-  }
-
-  for (int32_t i = 0; i <= ADC_ATTEN_DB_12_COMPAT; i++) {
-    auto adc_unit = this->channel1_ != ADC1_CHANNEL_MAX ? ADC_UNIT_1 : ADC_UNIT_2;
-    auto cal_value = esp_adc_cal_characterize(adc_unit, (adc_atten_t) i, ADC_WIDTH_MAX_SOC_BITS,
-                                              1100,  // default vref
-                                              &this->cal_characteristics_[i]);
-    switch (cal_value) {
-      case ESP_ADC_CAL_VAL_EFUSE_VREF:
-        ESP_LOGV(TAG, "Using eFuse Vref for calibration");
-        break;
-      case ESP_ADC_CAL_VAL_EFUSE_TP:
-        ESP_LOGV(TAG, "Using two-point eFuse Vref for calibration");
-        break;
-      case ESP_ADC_CAL_VAL_DEFAULT_VREF:
+const LogString *attenuation_to_str(adc_atten_t attenuation) {
+  switch (attenuation) {
+    case ADC_ATTEN_DB_0:
+      return LOG_STR("0 dB");
+    case ADC_ATTEN_DB_2_5:
+      return LOG_STR("2.5 dB");
+    case ADC_ATTEN_DB_6:
+      return LOG_STR("6 dB");
+    case ADC_ATTEN_DB_12_COMPAT:
+      return LOG_STR("12 dB");
     default:
-        break;
-    }
+      return LOG_STR("Unknown Attenuation");
   }
 }
 
-void ADCSensor::dump_config() {
-  static const char *const ATTEN_AUTO_STR = "auto";
-  static const char *const ATTEN_0DB_STR = "0 db";
-  static const char *const ATTEN_2_5DB_STR = "2.5 db";
-  static const char *const ATTEN_6DB_STR = "6 db";
-  static const char *const ATTEN_12DB_STR = "12 db";
-  const char *atten_str = ATTEN_AUTO_STR;
+const LogString *adc_unit_to_str(adc_unit_t unit) {
+  switch (unit) {
+    case ADC_UNIT_1:
+      return LOG_STR("ADC1");
+    case ADC_UNIT_2:
+      return LOG_STR("ADC2");
+    default:
+      return LOG_STR("Unknown ADC Unit");
+  }
+}
 
+void ADCSensor::setup() {
+  ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->get_name().c_str());
+  // Check if another sensor already initialized this ADC unit
+  if (ADCSensor::shared_adc_handles[this->adc_unit_] == nullptr) {
+    adc_oneshot_unit_init_cfg_t init_config = {};  // Zero initialize
+    init_config.unit_id = this->adc_unit_;
+    init_config.ulp_mode = ADC_ULP_MODE_DISABLE;
+#if USE_ESP32_VARIANT_ESP32C3 || USE_ESP32_VARIANT_ESP32C5 || USE_ESP32_VARIANT_ESP32C6 || USE_ESP32_VARIANT_ESP32H2
+    init_config.clk_src = ADC_DIGI_CLK_SRC_DEFAULT;
+#endif  // USE_ESP32_VARIANT_ESP32C3 || USE_ESP32_VARIANT_ESP32C5 || USE_ESP32_VARIANT_ESP32C6 ||
+        // USE_ESP32_VARIANT_ESP32H2
+    esp_err_t err = adc_oneshot_new_unit(&init_config, &ADCSensor::shared_adc_handles[this->adc_unit_]);
+    if (err != ESP_OK) {
+      ESP_LOGE(TAG, "Error initializing %s: %d", LOG_STR_ARG(adc_unit_to_str(this->adc_unit_)), err);
+      this->mark_failed();
+      return;
+    }
+  }
+  this->adc_handle_ = ADCSensor::shared_adc_handles[this->adc_unit_];
+
+  this->setup_flags_.handle_init_complete = true;
+
+  adc_oneshot_chan_cfg_t config = {
+      .atten = this->attenuation_,
+      .bitwidth = ADC_BITWIDTH_DEFAULT,
+  };
+  esp_err_t err = adc_oneshot_config_channel(this->adc_handle_, this->channel_, &config);
+  if (err != ESP_OK) {
+    ESP_LOGE(TAG, "Error configuring channel: %d", err);
+    this->mark_failed();
+    return;
+  }
+  this->setup_flags_.config_complete = true;
+
+  // Initialize ADC calibration
+  if (this->calibration_handle_ == nullptr) {
+    adc_cali_handle_t handle = nullptr;
+    esp_err_t err;
+
+#if USE_ESP32_VARIANT_ESP32C3 || USE_ESP32_VARIANT_ESP32C5 || USE_ESP32_VARIANT_ESP32C6 || \
+    USE_ESP32_VARIANT_ESP32S3 || USE_ESP32_VARIANT_ESP32H2
+    // RISC-V variants and S3 use curve fitting calibration
+    adc_cali_curve_fitting_config_t cali_config = {};  // Zero initialize first
+#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 3, 0)
+    cali_config.chan = this->channel_;
+#endif  // ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 3, 0)
+    cali_config.unit_id = this->adc_unit_;
+    cali_config.atten = this->attenuation_;
+    cali_config.bitwidth = ADC_BITWIDTH_DEFAULT;
+
+    err = adc_cali_create_scheme_curve_fitting(&cali_config, &handle);
+    if (err == ESP_OK) {
+      this->calibration_handle_ = handle;
+      this->setup_flags_.calibration_complete = true;
+      ESP_LOGV(TAG, "Using curve fitting calibration");
+    } else {
+      ESP_LOGW(TAG, "Curve fitting calibration failed with error %d, will use uncalibrated readings", err);
+      this->setup_flags_.calibration_complete = false;
+    }
+#else  // Other ESP32 variants use line fitting calibration
+    adc_cali_line_fitting_config_t cali_config = {
+      .unit_id = this->adc_unit_,
+      .atten = this->attenuation_,
+      .bitwidth = ADC_BITWIDTH_DEFAULT,
+#if !defined(USE_ESP32_VARIANT_ESP32S2)
+      .default_vref = 1100,  // Default reference voltage in mV
+#endif  // !defined(USE_ESP32_VARIANT_ESP32S2)
+    };
+    err = adc_cali_create_scheme_line_fitting(&cali_config, &handle);
+    if (err == ESP_OK) {
+      this->calibration_handle_ = handle;
+      this->setup_flags_.calibration_complete = true;
+      ESP_LOGV(TAG, "Using line fitting calibration");
+    } else {
+      ESP_LOGW(TAG, "Line fitting calibration failed with error %d, will use uncalibrated readings", err);
+      this->setup_flags_.calibration_complete = false;
+    }
+#endif  // USE_ESP32_VARIANT_ESP32C3 || ESP32C5 || ESP32C6 || ESP32S3 || ESP32H2
+  }
+
+  this->setup_flags_.init_complete = true;
+}
+
+void ADCSensor::dump_config() {
   LOG_SENSOR("", "ADC Sensor", this);
   LOG_PIN("  Pin: ", this->pin_);
-
-  if (!this->autorange_) {
-    switch (this->attenuation_) {
-      case ADC_ATTEN_DB_0:
-        atten_str = ATTEN_0DB_STR;
-        break;
-      case ADC_ATTEN_DB_2_5:
-        atten_str = ATTEN_2_5DB_STR;
-        break;
-      case ADC_ATTEN_DB_6:
-        atten_str = ATTEN_6DB_STR;
-        break;
-      case ADC_ATTEN_DB_12_COMPAT:
-        atten_str = ATTEN_12DB_STR;
-        break;
-      default:  // This is to satisfy the unused ADC_ATTEN_MAX
-        break;
-    }
-  }
-
   ESP_LOGCONFIG(TAG,
+                "  Channel:       %d\n"
+                "  Unit:          %s\n"
                 "  Attenuation:   %s\n"
                 "  Samples:       %i\n"
                 "  Sampling mode: %s",
-                atten_str, this->sample_count_, LOG_STR_ARG(sampling_mode_to_str(this->sampling_mode_)));
+                this->channel_, LOG_STR_ARG(adc_unit_to_str(this->adc_unit_)),
+                this->autorange_ ? "Auto" : LOG_STR_ARG(attenuation_to_str(this->attenuation_)), this->sample_count_,
+                LOG_STR_ARG(sampling_mode_to_str(this->sampling_mode_)));
+
+  ESP_LOGCONFIG(
+      TAG,
+      "  Setup Status:\n"
+      "    Handle Init:  %s\n"
+      "    Config:       %s\n"
+      "    Calibration:  %s\n"
+      "    Overall Init: %s",
+      this->setup_flags_.handle_init_complete ? "OK" : "FAILED", this->setup_flags_.config_complete ? "OK" : "FAILED",
+      this->setup_flags_.calibration_complete ? "OK" : "FAILED", this->setup_flags_.init_complete ? "OK" : "FAILED");
+
   LOG_UPDATE_INTERVAL(this);
 }
 
 float ADCSensor::sample() {
-  if (!this->autorange_) {
+  if (this->autorange_) {
+    return this->sample_autorange_();
+  } else {
+    return this->sample_fixed_attenuation_();
+  }
+}
+
+float ADCSensor::sample_fixed_attenuation_() {
   auto aggr = Aggregator(this->sampling_mode_);
 
   for (uint8_t sample = 0; sample < this->sample_count_; sample++) {
-      int raw = -1;
-      if (this->channel1_ != ADC1_CHANNEL_MAX) {
-        raw = adc1_get_raw(this->channel1_);
-      } else if (this->channel2_ != ADC2_CHANNEL_MAX) {
-        adc2_get_raw(this->channel2_, ADC_WIDTH_MAX_SOC_BITS, &raw);
+    int raw;
+    esp_err_t err = adc_oneshot_read(this->adc_handle_, this->channel_, &raw);
+
+    if (err != ESP_OK) {
+      ESP_LOGW(TAG, "ADC read failed with error %d", err);
+      continue;
     }
+
     if (raw == -1) {
-        return NAN;
+      ESP_LOGW(TAG, "Invalid ADC reading");
+      continue;
     }
 
     aggr.add_sample(raw);
   }
+
+  uint32_t final_value = aggr.aggregate();
+
   if (this->output_raw_) {
-      return aggr.aggregate();
-    }
-    uint32_t mv =
-        esp_adc_cal_raw_to_voltage(aggr.aggregate(), &this->cal_characteristics_[(int32_t) this->attenuation_]);
-    return mv / 1000.0f;
+    return final_value;
   }
 
-  int raw12 = ADC_MAX, raw6 = ADC_MAX, raw2 = ADC_MAX, raw0 = ADC_MAX;
+  if (this->calibration_handle_ != nullptr) {
+    int voltage_mv;
+    esp_err_t err = adc_cali_raw_to_voltage(this->calibration_handle_, final_value, &voltage_mv);
+    if (err == ESP_OK) {
+      return voltage_mv / 1000.0f;
+    } else {
+      ESP_LOGW(TAG, "ADC calibration conversion failed with error %d, disabling calibration", err);
+      if (this->calibration_handle_ != nullptr) {
+#if USE_ESP32_VARIANT_ESP32C3 || USE_ESP32_VARIANT_ESP32C5 || USE_ESP32_VARIANT_ESP32C6 || \
+    USE_ESP32_VARIANT_ESP32S3 || USE_ESP32_VARIANT_ESP32H2
+        adc_cali_delete_scheme_curve_fitting(this->calibration_handle_);
+#else   // Other ESP32 variants use line fitting calibration
+        adc_cali_delete_scheme_line_fitting(this->calibration_handle_);
+#endif  // USE_ESP32_VARIANT_ESP32C3 || ESP32C5 || ESP32C6 || ESP32S3 || ESP32H2
+        this->calibration_handle_ = nullptr;
+      }
+    }
+  }
 
-  if (this->channel1_ != ADC1_CHANNEL_MAX) {
-    adc1_config_channel_atten(this->channel1_, ADC_ATTEN_DB_12_COMPAT);
-    raw12 = adc1_get_raw(this->channel1_);
-    if (raw12 < ADC_MAX) {
-      adc1_config_channel_atten(this->channel1_, ADC_ATTEN_DB_6);
-      raw6 = adc1_get_raw(this->channel1_);
-      if (raw6 < ADC_MAX) {
-        adc1_config_channel_atten(this->channel1_, ADC_ATTEN_DB_2_5);
-        raw2 = adc1_get_raw(this->channel1_);
-        if (raw2 < ADC_MAX) {
-          adc1_config_channel_atten(this->channel1_, ADC_ATTEN_DB_0);
-          raw0 = adc1_get_raw(this->channel1_);
+  return final_value * 3.3f / 4095.0f;
+}
+
+float ADCSensor::sample_autorange_() {
+  // Auto-range mode
+  auto read_atten = [this](adc_atten_t atten) -> std::pair<int, float> {
+    // First reconfigure the attenuation for this reading
+    adc_oneshot_chan_cfg_t config = {
+        .atten = atten,
+        .bitwidth = ADC_BITWIDTH_DEFAULT,
+    };
+
+    esp_err_t err = adc_oneshot_config_channel(this->adc_handle_, this->channel_, &config);
+
+    if (err != ESP_OK) {
+      ESP_LOGW(TAG, "Error configuring ADC channel for autorange: %d", err);
+      return {-1, 0.0f};
     }
+
+    // Need to recalibrate for the new attenuation
+    if (this->calibration_handle_ != nullptr) {
+      // Delete old calibration handle
+#if USE_ESP32_VARIANT_ESP32C3 || USE_ESP32_VARIANT_ESP32C5 || USE_ESP32_VARIANT_ESP32C6 || \
+    USE_ESP32_VARIANT_ESP32S3 || USE_ESP32_VARIANT_ESP32H2
+      adc_cali_delete_scheme_curve_fitting(this->calibration_handle_);
+#else
+      adc_cali_delete_scheme_line_fitting(this->calibration_handle_);
+#endif
+      this->calibration_handle_ = nullptr;
     }
+
+    // Create new calibration handle for this attenuation
+    adc_cali_handle_t handle = nullptr;
+
+#if USE_ESP32_VARIANT_ESP32C3 || USE_ESP32_VARIANT_ESP32C5 || USE_ESP32_VARIANT_ESP32C6 || \
+    USE_ESP32_VARIANT_ESP32S3 || USE_ESP32_VARIANT_ESP32H2
+    adc_cali_curve_fitting_config_t cali_config = {};
+#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 3, 0)
+    cali_config.chan = this->channel_;
+#endif
+    cali_config.unit_id = this->adc_unit_;
+    cali_config.atten = atten;
+    cali_config.bitwidth = ADC_BITWIDTH_DEFAULT;
+
+    err = adc_cali_create_scheme_curve_fitting(&cali_config, &handle);
+#else
+    adc_cali_line_fitting_config_t cali_config = {
+      .unit_id = this->adc_unit_,
+      .atten = atten,
+      .bitwidth = ADC_BITWIDTH_DEFAULT,
+#if !defined(USE_ESP32_VARIANT_ESP32S2)
+      .default_vref = 1100,
+#endif
+    };
+    err = adc_cali_create_scheme_line_fitting(&cali_config, &handle);
+#endif
+
+    int raw;
+    err = adc_oneshot_read(this->adc_handle_, this->channel_, &raw);
+
+    if (err != ESP_OK) {
+      ESP_LOGW(TAG, "ADC read failed in autorange with error %d", err);
+      if (handle != nullptr) {
+#if USE_ESP32_VARIANT_ESP32C3 || USE_ESP32_VARIANT_ESP32C5 || USE_ESP32_VARIANT_ESP32C6 || \
+    USE_ESP32_VARIANT_ESP32S3 || USE_ESP32_VARIANT_ESP32H2
+        adc_cali_delete_scheme_curve_fitting(handle);
+#else
+        adc_cali_delete_scheme_line_fitting(handle);
+#endif
       }
-  } else if (this->channel2_ != ADC2_CHANNEL_MAX) {
-    adc2_config_channel_atten(this->channel2_, ADC_ATTEN_DB_12_COMPAT);
-    adc2_get_raw(this->channel2_, ADC_WIDTH_MAX_SOC_BITS, &raw12);
-    if (raw12 < ADC_MAX) {
-      adc2_config_channel_atten(this->channel2_, ADC_ATTEN_DB_6);
-      adc2_get_raw(this->channel2_, ADC_WIDTH_MAX_SOC_BITS, &raw6);
-      if (raw6 < ADC_MAX) {
-        adc2_config_channel_atten(this->channel2_, ADC_ATTEN_DB_2_5);
-        adc2_get_raw(this->channel2_, ADC_WIDTH_MAX_SOC_BITS, &raw2);
-        if (raw2 < ADC_MAX) {
-          adc2_config_channel_atten(this->channel2_, ADC_ATTEN_DB_0);
-          adc2_get_raw(this->channel2_, ADC_WIDTH_MAX_SOC_BITS, &raw0);
+      return {-1, 0.0f};
     }
+
+    float voltage = 0.0f;
+    if (handle != nullptr) {
+      int voltage_mv;
+      err = adc_cali_raw_to_voltage(handle, raw, &voltage_mv);
+      if (err == ESP_OK) {
+        voltage = voltage_mv / 1000.0f;
+      } else {
+        voltage = raw * 3.3f / 4095.0f;
+      }
+      // Clean up calibration handle
+#if USE_ESP32_VARIANT_ESP32C3 || USE_ESP32_VARIANT_ESP32C5 || USE_ESP32_VARIANT_ESP32C6 || \
+    USE_ESP32_VARIANT_ESP32S3 || USE_ESP32_VARIANT_ESP32H2
+      adc_cali_delete_scheme_curve_fitting(handle);
+#else
+      adc_cali_delete_scheme_line_fitting(handle);
+#endif
+    } else {
+      voltage = raw * 3.3f / 4095.0f;
+    }
+
+    return {raw, voltage};
+  };
+
+  auto [raw12, mv12] = read_atten(ADC_ATTEN_DB_12);
+  if (raw12 == -1) {
+    ESP_LOGE(TAG, "Failed to read ADC in autorange mode");
+    return NAN;
+  }
+
+  int raw6 = 4095, raw2 = 4095, raw0 = 4095;
+  float mv6 = 0, mv2 = 0, mv0 = 0;
+
+  if (raw12 < 4095) {
+    auto [raw6_val, mv6_val] = read_atten(ADC_ATTEN_DB_6);
+    raw6 = raw6_val;
+    mv6 = mv6_val;
+
+    if (raw6 < 4095 && raw6 != -1) {
+      auto [raw2_val, mv2_val] = read_atten(ADC_ATTEN_DB_2_5);
+      raw2 = raw2_val;
+      mv2 = mv2_val;
+
+      if (raw2 < 4095 && raw2 != -1) {
+        auto [raw0_val, mv0_val] = read_atten(ADC_ATTEN_DB_0);
+        raw0 = raw0_val;
+        mv0 = mv0_val;
       }
     }
   }
@@ -155,19 +325,19 @@ float ADCSensor::sample() {
     return NAN;
   }
 
-  uint32_t mv12 = esp_adc_cal_raw_to_voltage(raw12, &this->cal_characteristics_[(int32_t) ADC_ATTEN_DB_12_COMPAT]);
-  uint32_t mv6 = esp_adc_cal_raw_to_voltage(raw6, &this->cal_characteristics_[(int32_t) ADC_ATTEN_DB_6]);
-  uint32_t mv2 = esp_adc_cal_raw_to_voltage(raw2, &this->cal_characteristics_[(int32_t) ADC_ATTEN_DB_2_5]);
-  uint32_t mv0 = esp_adc_cal_raw_to_voltage(raw0, &this->cal_characteristics_[(int32_t) ADC_ATTEN_DB_0]);
-
-  uint32_t c12 = std::min(raw12, ADC_HALF);
-  uint32_t c6 = ADC_HALF - std::abs(raw6 - ADC_HALF);
-  uint32_t c2 = ADC_HALF - std::abs(raw2 - ADC_HALF);
-  uint32_t c0 = std::min(ADC_MAX - raw0, ADC_HALF);
+  const int adc_half = 2048;
+  uint32_t c12 = std::min(raw12, adc_half);
+  uint32_t c6 = adc_half - std::abs(raw6 - adc_half);
+  uint32_t c2 = adc_half - std::abs(raw2 - adc_half);
+  uint32_t c0 = std::min(4095 - raw0, adc_half);
   uint32_t csum = c12 + c6 + c2 + c0;
 
-  uint32_t mv_scaled = (mv12 * c12) + (mv6 * c6) + (mv2 * c2) + (mv0 * c0);
-  return mv_scaled / (float) (csum * 1000U);
+  if (csum == 0) {
+    ESP_LOGE(TAG, "Invalid weight sum in autorange calculation");
+    return NAN;
+  }
+
+  return (mv12 * c12 + mv6 * c6 + mv2 * c2 + mv0 * c0) / csum;
 }
 
 }  // namespace adc

esphome/components/adc/adc_sensor_esp8266.cpp

@@ -56,8 +56,6 @@ float ADCSensor::sample() {
   return aggr.aggregate() / 1024.0f;
 }
 
-std::string ADCSensor::unique_id() { return get_mac_address() + "-adc"; }
-
 }  // namespace adc
 }  // namespace esphome
 

esphome/components/adc/sensor.py

@@ -10,13 +10,11 @@ from esphome.const import (
     CONF_NUMBER,
     CONF_PIN,
     CONF_RAW,
-    CONF_WIFI,
     DEVICE_CLASS_VOLTAGE,
     STATE_CLASS_MEASUREMENT,
     UNIT_VOLT,
 )
 from esphome.core import CORE
-import esphome.final_validate as fv
 
 from . import (
     ATTENUATION_MODES,
@@ -24,6 +22,7 @@ from . import (
     ESP32_VARIANT_ADC2_PIN_TO_CHANNEL,
     SAMPLING_MODES,
     adc_ns,
+    adc_unit_t,
     validate_adc_pin,
 )
 
@@ -57,21 +56,6 @@ def validate_config(config):
     return config
 
 
-def final_validate_config(config):
-    if CORE.is_esp32:
-        variant = get_esp32_variant()
-        if (
-            CONF_WIFI in fv.full_config.get()
-            and config[CONF_PIN][CONF_NUMBER]
-            in ESP32_VARIANT_ADC2_PIN_TO_CHANNEL[variant]
-        ):
-            raise cv.Invalid(
-                f"{variant} doesn't support ADC on this pin when Wi-Fi is configured"
-            )
-
-    return config
-
-
 ADCSensor = adc_ns.class_(
     "ADCSensor", sensor.Sensor, cg.PollingComponent, voltage_sampler.VoltageSampler
 )
@@ -99,8 +83,6 @@ CONFIG_SCHEMA = cv.All(
     validate_config,
 )
 
-FINAL_VALIDATE_SCHEMA = final_validate_config
-
 
 async def to_code(config):
     var = cg.new_Pvariable(config[CONF_ID])
@@ -119,13 +101,13 @@ async def to_code(config):
     cg.add(var.set_sample_count(config[CONF_SAMPLES]))
     cg.add(var.set_sampling_mode(config[CONF_SAMPLING_MODE]))
 
+    if CORE.is_esp32:
         if attenuation := config.get(CONF_ATTENUATION):
             if attenuation == "auto":
                 cg.add(var.set_autorange(cg.global_ns.true))
             else:
                 cg.add(var.set_attenuation(attenuation))
 
-    if CORE.is_esp32:
         variant = get_esp32_variant()
         pin_num = config[CONF_PIN][CONF_NUMBER]
         if (
@@ -133,10 +115,10 @@ async def to_code(config):
             and pin_num in ESP32_VARIANT_ADC1_PIN_TO_CHANNEL[variant]
         ):
             chan = ESP32_VARIANT_ADC1_PIN_TO_CHANNEL[variant][pin_num]
-            cg.add(var.set_channel1(chan))
+            cg.add(var.set_channel(adc_unit_t.ADC_UNIT_1, chan))
         elif (
             variant in ESP32_VARIANT_ADC2_PIN_TO_CHANNEL
             and pin_num in ESP32_VARIANT_ADC2_PIN_TO_CHANNEL[variant]
         ):
             chan = ESP32_VARIANT_ADC2_PIN_TO_CHANNEL[variant][pin_num]
-            cg.add(var.set_channel2(chan))
+            cg.add(var.set_channel(adc_unit_t.ADC_UNIT_2, chan))

esphome/components/airthings_wave_plus/__init__.py

@@ -1 +1 @@
-CODEOWNERS = ["@jeromelaban"]
+CODEOWNERS = ["@jeromelaban", "@precurse"]

esphome/components/airthings_wave_plus/airthings_wave_plus.cpp

@@ -73,11 +73,29 @@ void AirthingsWavePlus::dump_config() {
   LOG_SENSOR("  ", "Illuminance", this->illuminance_sensor_);
 }
 
-AirthingsWavePlus::AirthingsWavePlus() {
-  this->service_uuid_ = espbt::ESPBTUUID::from_raw(SERVICE_UUID);
-  this->sensors_data_characteristic_uuid_ = espbt::ESPBTUUID::from_raw(CHARACTERISTIC_UUID);
+void AirthingsWavePlus::setup() {
+  const char *service_uuid;
+  const char *characteristic_uuid;
+  const char *access_control_point_characteristic_uuid;
+
+  // Change UUIDs for Wave Radon Gen2
+  switch (this->wave_device_type_) {
+    case WaveDeviceType::WAVE_GEN2:
+      service_uuid = SERVICE_UUID_WAVE_RADON_GEN2;
+      characteristic_uuid = CHARACTERISTIC_UUID_WAVE_RADON_GEN2;
+      access_control_point_characteristic_uuid = ACCESS_CONTROL_POINT_CHARACTERISTIC_UUID_WAVE_RADON_GEN2;
+      break;
+    default:
+      // Wave Plus
+      service_uuid = SERVICE_UUID;
+      characteristic_uuid = CHARACTERISTIC_UUID;
+      access_control_point_characteristic_uuid = ACCESS_CONTROL_POINT_CHARACTERISTIC_UUID;
+  }
+
+  this->service_uuid_ = espbt::ESPBTUUID::from_raw(service_uuid);
+  this->sensors_data_characteristic_uuid_ = espbt::ESPBTUUID::from_raw(characteristic_uuid);
   this->access_control_point_characteristic_uuid_ =
-      espbt::ESPBTUUID::from_raw(ACCESS_CONTROL_POINT_CHARACTERISTIC_UUID);
+      espbt::ESPBTUUID::from_raw(access_control_point_characteristic_uuid);
 }
 
 }  // namespace airthings_wave_plus

esphome/components/airthings_wave_plus/airthings_wave_plus.h

@@ -9,13 +9,20 @@ namespace airthings_wave_plus {
 
 namespace espbt = esphome::esp32_ble_tracker;
 
+enum WaveDeviceType : uint8_t { WAVE_PLUS = 0, WAVE_GEN2 = 1 };
+
 static const char *const SERVICE_UUID = "b42e1c08-ade7-11e4-89d3-123b93f75cba";
 static const char *const CHARACTERISTIC_UUID = "b42e2a68-ade7-11e4-89d3-123b93f75cba";
 static const char *const ACCESS_CONTROL_POINT_CHARACTERISTIC_UUID = "b42e2d06-ade7-11e4-89d3-123b93f75cba";
 
+static const char *const SERVICE_UUID_WAVE_RADON_GEN2 = "b42e4a8e-ade7-11e4-89d3-123b93f75cba";
+static const char *const CHARACTERISTIC_UUID_WAVE_RADON_GEN2 = "b42e4dcc-ade7-11e4-89d3-123b93f75cba";
+static const char *const ACCESS_CONTROL_POINT_CHARACTERISTIC_UUID_WAVE_RADON_GEN2 =
+    "b42e50d8-ade7-11e4-89d3-123b93f75cba";
+
 class AirthingsWavePlus : public airthings_wave_base::AirthingsWaveBase {
  public:
-  AirthingsWavePlus();
+  void setup() override;
 
   void dump_config() override;
 
@@ -23,12 +30,14 @@ class AirthingsWavePlus : public airthings_wave_base::AirthingsWaveBase {
   void set_radon_long_term(sensor::Sensor *radon_long_term) { radon_long_term_sensor_ = radon_long_term; }
   void set_co2(sensor::Sensor *co2) { co2_sensor_ = co2; }
   void set_illuminance(sensor::Sensor *illuminance) { illuminance_sensor_ = illuminance; }
+  void set_device_type(WaveDeviceType wave_device_type) { wave_device_type_ = wave_device_type; }
 
  protected:
   bool is_valid_radon_value_(uint16_t radon);
   bool is_valid_co2_value_(uint16_t co2);
 
   void read_sensors(uint8_t *raw_value, uint16_t value_len) override;
+  WaveDeviceType wave_device_type_{WaveDeviceType::WAVE_PLUS};
 
   sensor::Sensor *radon_sensor_{nullptr};
   sensor::Sensor *radon_long_term_sensor_{nullptr};

esphome/components/airthings_wave_plus/sensor.py

@@ -7,6 +7,7 @@ from esphome.const import (
     CONF_ILLUMINANCE,
     CONF_RADON,
     CONF_RADON_LONG_TERM,
+    CONF_TVOC,
     DEVICE_CLASS_CARBON_DIOXIDE,
     DEVICE_CLASS_ILLUMINANCE,
     ICON_RADIOACTIVE,
@@ -15,6 +16,7 @@ from esphome.const import (
     UNIT_LUX,
     UNIT_PARTS_PER_MILLION,
 )
+from esphome.types import ConfigType
 
 DEPENDENCIES = airthings_wave_base.DEPENDENCIES
 
@@ -25,8 +27,27 @@ AirthingsWavePlus = airthings_wave_plus_ns.class_(
     "AirthingsWavePlus", airthings_wave_base.AirthingsWaveBase
 )
 
+CONF_DEVICE_TYPE = "device_type"
+WaveDeviceType = airthings_wave_plus_ns.enum("WaveDeviceType")
+DEVICE_TYPES = {
+    "WAVE_PLUS": WaveDeviceType.WAVE_PLUS,
+    "WAVE_GEN2": WaveDeviceType.WAVE_GEN2,
+}
 
-CONFIG_SCHEMA = airthings_wave_base.BASE_SCHEMA.extend(
+
+def validate_wave_gen2_config(config: ConfigType) -> ConfigType:
+    """Validate that Wave Gen2 devices don't have CO2 or TVOC sensors."""
+    if config[CONF_DEVICE_TYPE] == "WAVE_GEN2":
+        if CONF_CO2 in config:
+            raise cv.Invalid("Wave Gen2 devices do not support CO2 sensor")
+        # Check for TVOC in the base schema config
+        if CONF_TVOC in config:
+            raise cv.Invalid("Wave Gen2 devices do not support TVOC sensor")
+    return config
+
+
+CONFIG_SCHEMA = cv.All(
+    airthings_wave_base.BASE_SCHEMA.extend(
         {
             cv.GenerateID(): cv.declare_id(AirthingsWavePlus),
             cv.Optional(CONF_RADON): sensor.sensor_schema(
@@ -53,7 +74,12 @@ CONFIG_SCHEMA = airthings_wave_base.BASE_SCHEMA.extend(
                 device_class=DEVICE_CLASS_ILLUMINANCE,
                 state_class=STATE_CLASS_MEASUREMENT,
             ),
+            cv.Optional(CONF_DEVICE_TYPE, default="WAVE_PLUS"): cv.enum(
+                DEVICE_TYPES, upper=True
+            ),
         }
+    ),
+    validate_wave_gen2_config,
 )
 
 
@@ -73,3 +99,4 @@ async def to_code(config):
     if config_illuminance := config.get(CONF_ILLUMINANCE):
         sens = await sensor.new_sensor(config_illuminance)
         cg.add(var.set_illuminance(sens))
+    cg.add(var.set_device_type(config[CONF_DEVICE_TYPE]))

esphome/components/api/__init__.py

@@ -323,9 +323,10 @@ async def api_connected_to_code(config, condition_id, template_arg, args):
 
 
 def FILTER_SOURCE_FILES() -> list[str]:
-    """Filter out api_pb2_dump.cpp when proto message dumping is not enabled
-    and user_services.cpp when no services are defined."""
-    files_to_filter = []
+    """Filter out api_pb2_dump.cpp when proto message dumping is not enabled,
+    user_services.cpp when no services are defined, and protocol-specific
+    implementations based on encryption configuration."""
+    files_to_filter: list[str] = []
 
     # api_pb2_dump.cpp is only needed when HAS_PROTO_MESSAGE_DUMP is defined
     # This is a particularly large file that still needs to be opened and read
@@ -341,4 +342,16 @@ def FILTER_SOURCE_FILES() -> list[str]:
     if config and not config.get(CONF_ACTIONS) and not config[CONF_CUSTOM_SERVICES]:
         files_to_filter.append("user_services.cpp")
 
+    # Filter protocol-specific implementations based on encryption configuration
+    encryption_config = config.get(CONF_ENCRYPTION) if config else None
+
+    # If encryption is not configured at all, we only need plaintext
+    if encryption_config is None:
+        files_to_filter.append("api_frame_helper_noise.cpp")
+    # If encryption is configured with a key, we only need noise
+    elif encryption_config.get(CONF_KEY):
+        files_to_filter.append("api_frame_helper_plaintext.cpp")
+    # If encryption is configured but no key is provided, we need both
+    # (this allows a plaintext client to provide a noise key)
+
     return files_to_filter

esphome/components/api/api.proto

@@ -203,7 +203,7 @@ message DeviceInfoResponse {
   option (id) = 10;
   option (source) = SOURCE_SERVER;
 
-  bool uses_password = 1;
+  bool uses_password = 1 [(field_ifdef) = "USE_API_PASSWORD"];
 
   // The name of the node, given by "App.set_name()"
   string name = 2;
@@ -222,37 +222,39 @@ message DeviceInfoResponse {
   // The model of the board. For example NodeMCU
   string model = 6;
 
-  bool has_deep_sleep = 7;
+  bool has_deep_sleep = 7 [(field_ifdef) = "USE_DEEP_SLEEP"];
 
   // The esphome project details if set
-  string project_name = 8;
-  string project_version = 9;
+  string project_name = 8 [(field_ifdef) = "ESPHOME_PROJECT_NAME"];
+  string project_version = 9 [(field_ifdef) = "ESPHOME_PROJECT_NAME"];
 
-  uint32 webserver_port = 10;
+  uint32 webserver_port = 10 [(field_ifdef) = "USE_WEBSERVER"];
 
-  uint32 legacy_bluetooth_proxy_version = 11;
-  uint32 bluetooth_proxy_feature_flags = 15;
+  // Deprecated in API version 1.9
+  uint32 legacy_bluetooth_proxy_version = 11 [deprecated=true, (field_ifdef) = "USE_BLUETOOTH_PROXY"];
+  uint32 bluetooth_proxy_feature_flags = 15 [(field_ifdef) = "USE_BLUETOOTH_PROXY"];
 
   string manufacturer = 12;
 
   string friendly_name = 13;
 
-  uint32 legacy_voice_assistant_version = 14;
-  uint32 voice_assistant_feature_flags = 17;
+  // Deprecated in API version 1.10
+  uint32 legacy_voice_assistant_version = 14 [deprecated=true, (field_ifdef) = "USE_VOICE_ASSISTANT"];
+  uint32 voice_assistant_feature_flags = 17 [(field_ifdef) = "USE_VOICE_ASSISTANT"];
 
-  string suggested_area = 16;
+  string suggested_area = 16 [(field_ifdef) = "USE_AREAS"];
 
   // The Bluetooth mac address of the device. For example "AC:BC:32:89:0E:AA"
-  string bluetooth_mac_address = 18;
+  string bluetooth_mac_address = 18 [(field_ifdef) = "USE_BLUETOOTH_PROXY"];
 
   // Supports receiving and saving api encryption key
-  bool api_encryption_supported = 19;
+  bool api_encryption_supported = 19 [(field_ifdef) = "USE_API_NOISE"];
 
-  repeated DeviceInfo devices = 20;
-  repeated AreaInfo areas = 21;
+  repeated DeviceInfo devices = 20 [(field_ifdef) = "USE_DEVICES"];
+  repeated AreaInfo areas = 21 [(field_ifdef) = "USE_AREAS"];
 
   // Top-level area info to phase out suggested_area
-  AreaInfo area = 22;
+  AreaInfo area = 22 [(field_ifdef) = "USE_AREAS"];
 }
 
 message ListEntitiesRequest {
@@ -290,14 +292,14 @@ message ListEntitiesBinarySensorResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
 
   string device_class = 5;
   bool is_status_binary_sensor = 6;
   bool disabled_by_default = 7;
-  string icon = 8;
+  string icon = 8 [(field_ifdef) = "USE_ENTITY_ICON"];
   EntityCategory entity_category = 9;
-  uint32 device_id = 10;
+  uint32 device_id = 10 [(field_ifdef) = "USE_DEVICES"];
 }
 message BinarySensorStateResponse {
   option (id) = 21;
@@ -311,7 +313,7 @@ message BinarySensorStateResponse {
   // If the binary sensor does not have a valid state yet.
   // Equivalent to `!obj->has_state()` - inverse logic to make state packets smaller
   bool missing_state = 3;
-  uint32 device_id = 4;
+  uint32 device_id = 4 [(field_ifdef) = "USE_DEVICES"];
 }
 
 // ==================== COVER ====================
@@ -324,20 +326,22 @@ message ListEntitiesCoverResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
 
   bool assumed_state = 5;
   bool supports_position = 6;
   bool supports_tilt = 7;
   string device_class = 8;
   bool disabled_by_default = 9;
-  string icon = 10;
+  string icon = 10 [(field_ifdef) = "USE_ENTITY_ICON"];
   EntityCategory entity_category = 11;
   bool supports_stop = 12;
-  uint32 device_id = 13;
+  uint32 device_id = 13 [(field_ifdef) = "USE_DEVICES"];
 }
 
+// Deprecated in API version 1.1
 enum LegacyCoverState {
+  option deprecated = true;
   LEGACY_COVER_STATE_OPEN = 0;
   LEGACY_COVER_STATE_CLOSED = 1;
 }
@@ -356,15 +360,18 @@ message CoverStateResponse {
   fixed32 key = 1;
   // legacy: state has been removed in 1.13
   // clients/servers must still send/accept it until the next protocol change
-  LegacyCoverState legacy_state = 2;
+  // Deprecated in API version 1.1
+  LegacyCoverState legacy_state = 2 [deprecated=true];
 
   float position = 3;
   float tilt = 4;
   CoverOperation current_operation = 5;
-  uint32 device_id = 6;
+  uint32 device_id = 6 [(field_ifdef) = "USE_DEVICES"];
 }
 
+// Deprecated in API version 1.1
 enum LegacyCoverCommand {
+  option deprecated = true;
   LEGACY_COVER_COMMAND_OPEN = 0;
   LEGACY_COVER_COMMAND_CLOSE = 1;
   LEGACY_COVER_COMMAND_STOP = 2;
@@ -380,15 +387,17 @@ message CoverCommandRequest {
 
   // legacy: command has been removed in 1.13
   // clients/servers must still send/accept it until the next protocol change
-  bool has_legacy_command = 2;
-  LegacyCoverCommand legacy_command = 3;
+  // Deprecated in API version 1.1
+  bool has_legacy_command = 2 [deprecated=true];
+  // Deprecated in API version 1.1
+  LegacyCoverCommand legacy_command = 3 [deprecated=true];
 
   bool has_position = 4;
   float position = 5;
   bool has_tilt = 6;
   float tilt = 7;
   bool stop = 8;
-  uint32 device_id = 9;
+  uint32 device_id = 9 [(field_ifdef) = "USE_DEVICES"];
 }
 
 // ==================== FAN ====================
@@ -401,19 +410,21 @@ message ListEntitiesFanResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
 
   bool supports_oscillation = 5;
   bool supports_speed = 6;
   bool supports_direction = 7;
   int32 supported_speed_count = 8;
   bool disabled_by_default = 9;
-  string icon = 10;
+  string icon = 10 [(field_ifdef) = "USE_ENTITY_ICON"];
   EntityCategory entity_category = 11;
   repeated string supported_preset_modes = 12;
-  uint32 device_id = 13;
+  uint32 device_id = 13 [(field_ifdef) = "USE_DEVICES"];
 }
+// Deprecated in API version 1.6 - only used in deprecated fields
 enum FanSpeed {
+  option deprecated = true;
   FAN_SPEED_LOW = 0;
   FAN_SPEED_MEDIUM = 1;
   FAN_SPEED_HIGH = 2;
@@ -432,11 +443,12 @@ message FanStateResponse {
   fixed32 key = 1;
   bool state = 2;
   bool oscillating = 3;
-  FanSpeed speed = 4 [deprecated = true];
+  // Deprecated in API version 1.6
+  FanSpeed speed = 4 [deprecated=true];
   FanDirection direction = 5;
   int32 speed_level = 6;
   string preset_mode = 7;
-  uint32 device_id = 8;
+  uint32 device_id = 8 [(field_ifdef) = "USE_DEVICES"];
 }
 message FanCommandRequest {
   option (id) = 31;
@@ -448,8 +460,10 @@ message FanCommandRequest {
   fixed32 key = 1;
   bool has_state = 2;
   bool state = 3;
-  bool has_speed = 4 [deprecated = true];
-  FanSpeed speed = 5 [deprecated = true];
+  // Deprecated in API version 1.6
+  bool has_speed = 4 [deprecated=true];
+  // Deprecated in API version 1.6
+  FanSpeed speed = 5 [deprecated=true];
   bool has_oscillating = 6;
   bool oscillating = 7;
   bool has_direction = 8;
@@ -458,7 +472,7 @@ message FanCommandRequest {
   int32 speed_level = 11;
   bool has_preset_mode = 12;
   string preset_mode = 13;
-  uint32 device_id = 14;
+  uint32 device_id = 14 [(field_ifdef) = "USE_DEVICES"];
 }
 
 // ==================== LIGHT ====================
@@ -484,21 +498,25 @@ message ListEntitiesLightResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
 
   repeated ColorMode supported_color_modes = 12;
   // next four supports_* are for legacy clients, newer clients should use color modes
+  // Deprecated in API version 1.6
   bool legacy_supports_brightness = 5 [deprecated=true];
+  // Deprecated in API version 1.6
   bool legacy_supports_rgb = 6 [deprecated=true];
+  // Deprecated in API version 1.6
   bool legacy_supports_white_value = 7 [deprecated=true];
+  // Deprecated in API version 1.6
   bool legacy_supports_color_temperature = 8 [deprecated=true];
   float min_mireds = 9;
   float max_mireds = 10;
   repeated string effects = 11;
   bool disabled_by_default = 13;
-  string icon = 14;
+  string icon = 14 [(field_ifdef) = "USE_ENTITY_ICON"];
   EntityCategory entity_category = 15;
-  uint32 device_id = 16;
+  uint32 device_id = 16 [(field_ifdef) = "USE_DEVICES"];
 }
 message LightStateResponse {
   option (id) = 24;
@@ -520,7 +538,7 @@ message LightStateResponse {
   float cold_white = 12;
   float warm_white = 13;
   string effect = 9;
-  uint32 device_id = 14;
+  uint32 device_id = 14 [(field_ifdef) = "USE_DEVICES"];
 }
 message LightCommandRequest {
   option (id) = 32;
@@ -556,7 +574,7 @@ message LightCommandRequest {
   uint32 flash_length = 17;
   bool has_effect = 18;
   string effect = 19;
-  uint32 device_id = 28;
+  uint32 device_id = 28 [(field_ifdef) = "USE_DEVICES"];
 }
 
 // ==================== SENSOR ====================
@@ -567,7 +585,9 @@ enum SensorStateClass {
   STATE_CLASS_TOTAL = 3;
 }
 
+// Deprecated in API version 1.5
 enum SensorLastResetType {
+  option deprecated = true;
   LAST_RESET_NONE = 0;
   LAST_RESET_NEVER = 1;
   LAST_RESET_AUTO = 2;
@@ -582,19 +602,20 @@ message ListEntitiesSensorResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
 
-  string icon = 5;
+  string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"];
   string unit_of_measurement = 6;
   int32 accuracy_decimals = 7;
   bool force_update = 8;
   string device_class = 9;
   SensorStateClass state_class = 10;
   // Last reset type removed in 2021.9.0
-  SensorLastResetType legacy_last_reset_type = 11;
+  // Deprecated in API version 1.5
+  SensorLastResetType legacy_last_reset_type = 11 [deprecated=true];
   bool disabled_by_default = 12;
   EntityCategory entity_category = 13;
-  uint32 device_id = 14;
+  uint32 device_id = 14 [(field_ifdef) = "USE_DEVICES"];
 }
 message SensorStateResponse {
   option (id) = 25;
@@ -608,7 +629,7 @@ message SensorStateResponse {
   // If the sensor does not have a valid state yet.
   // Equivalent to `!obj->has_state()` - inverse logic to make state packets smaller
   bool missing_state = 3;
-  uint32 device_id = 4;
+  uint32 device_id = 4 [(field_ifdef) = "USE_DEVICES"];
 }
 
 // ==================== SWITCH ====================
@@ -621,14 +642,14 @@ message ListEntitiesSwitchResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
 
-  string icon = 5;
+  string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"];
   bool assumed_state = 6;
   bool disabled_by_default = 7;
   EntityCategory entity_category = 8;
   string device_class = 9;
-  uint32 device_id = 10;
+  uint32 device_id = 10 [(field_ifdef) = "USE_DEVICES"];
 }
 message SwitchStateResponse {
   option (id) = 26;
@@ -639,7 +660,7 @@ message SwitchStateResponse {
 
   fixed32 key = 1;
   bool state = 2;
-  uint32 device_id = 3;
+  uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"];
 }
 message SwitchCommandRequest {
   option (id) = 33;
@@ -650,7 +671,7 @@ message SwitchCommandRequest {
 
   fixed32 key = 1;
   bool state = 2;
-  uint32 device_id = 3;
+  uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"];
 }
 
 // ==================== TEXT SENSOR ====================
@@ -663,13 +684,13 @@ message ListEntitiesTextSensorResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
 
-  string icon = 5;
+  string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"];
   bool disabled_by_default = 6;
   EntityCategory entity_category = 7;
   string device_class = 8;
-  uint32 device_id = 9;
+  uint32 device_id = 9 [(field_ifdef) = "USE_DEVICES"];
 }
 message TextSensorStateResponse {
   option (id) = 27;
@@ -683,7 +704,7 @@ message TextSensorStateResponse {
   // If the text sensor does not have a valid state yet.
   // Equivalent to `!obj->has_state()` - inverse logic to make state packets smaller
   bool missing_state = 3;
-  uint32 device_id = 4;
+  uint32 device_id = 4 [(field_ifdef) = "USE_DEVICES"];
 }
 
 // ==================== SUBSCRIBE LOGS ====================
@@ -711,7 +732,6 @@ message SubscribeLogsResponse {
 
   LogLevel level = 1;
   bytes message = 3;
-  bool send_failed = 4;
 }
 
 // ==================== NOISE ENCRYPTION ====================
@@ -853,11 +873,11 @@ message ListEntitiesCameraResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
   bool disabled_by_default = 5;
-  string icon = 6;
+  string icon = 6 [(field_ifdef) = "USE_ENTITY_ICON"];
   EntityCategory entity_category = 7;
-  uint32 device_id = 8;
+  uint32 device_id = 8 [(field_ifdef) = "USE_DEVICES"];
 }
 
 message CameraImageResponse {
@@ -869,7 +889,7 @@ message CameraImageResponse {
   fixed32 key = 1;
   bytes data = 2;
   bool done = 3;
-  uint32 device_id = 4;
+  uint32 device_id = 4 [(field_ifdef) = "USE_DEVICES"];
 }
 message CameraImageRequest {
   option (id) = 45;
@@ -937,7 +957,7 @@ message ListEntitiesClimateResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
 
   bool supports_current_temperature = 5;
   bool supports_two_point_target_temperature = 6;
@@ -947,7 +967,8 @@ message ListEntitiesClimateResponse {
   float visual_target_temperature_step = 10;
   // for older peer versions - in new system this
   // is if CLIMATE_PRESET_AWAY exists is supported_presets
-  bool legacy_supports_away = 11;
+  // Deprecated in API version 1.5
+  bool legacy_supports_away = 11 [deprecated=true];
   bool supports_action = 12;
   repeated ClimateFanMode supported_fan_modes = 13;
   repeated ClimateSwingMode supported_swing_modes = 14;
@@ -955,14 +976,14 @@ message ListEntitiesClimateResponse {
   repeated ClimatePreset supported_presets = 16;
   repeated string supported_custom_presets = 17;
   bool disabled_by_default = 18;
-  string icon = 19;
+  string icon = 19 [(field_ifdef) = "USE_ENTITY_ICON"];
   EntityCategory entity_category = 20;
   float visual_current_temperature_step = 21;
   bool supports_current_humidity = 22;
   bool supports_target_humidity = 23;
   float visual_min_humidity = 24;
   float visual_max_humidity = 25;
-  uint32 device_id = 26;
+  uint32 device_id = 26 [(field_ifdef) = "USE_DEVICES"];
 }
 message ClimateStateResponse {
   option (id) = 47;
@@ -978,7 +999,8 @@ message ClimateStateResponse {
   float target_temperature_low = 5;
   float target_temperature_high = 6;
   // For older peers, equal to preset == CLIMATE_PRESET_AWAY
-  bool unused_legacy_away = 7;
+  // Deprecated in API version 1.5
+  bool unused_legacy_away = 7 [deprecated=true];
   ClimateAction action = 8;
   ClimateFanMode fan_mode = 9;
   ClimateSwingMode swing_mode = 10;
@@ -987,7 +1009,7 @@ message ClimateStateResponse {
   string custom_preset = 13;
   float current_humidity = 14;
   float target_humidity = 15;
-  uint32 device_id = 16;
+  uint32 device_id = 16 [(field_ifdef) = "USE_DEVICES"];
 }
 message ClimateCommandRequest {
   option (id) = 48;
@@ -1006,8 +1028,10 @@ message ClimateCommandRequest {
   bool has_target_temperature_high = 8;
   float target_temperature_high = 9;
   // legacy, for older peers, newer ones should use CLIMATE_PRESET_AWAY in preset
-  bool unused_has_legacy_away = 10;
-  bool unused_legacy_away = 11;
+  // Deprecated in API version 1.5
+  bool unused_has_legacy_away = 10 [deprecated=true];
+  // Deprecated in API version 1.5
+  bool unused_legacy_away = 11 [deprecated=true];
   bool has_fan_mode = 12;
   ClimateFanMode fan_mode = 13;
   bool has_swing_mode = 14;
@@ -1020,7 +1044,7 @@ message ClimateCommandRequest {
   string custom_preset = 21;
   bool has_target_humidity = 22;
   float target_humidity = 23;
-  uint32 device_id = 24;
+  uint32 device_id = 24 [(field_ifdef) = "USE_DEVICES"];
 }
 
 // ==================== NUMBER ====================
@@ -1038,9 +1062,9 @@ message ListEntitiesNumberResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
 
-  string icon = 5;
+  string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"];
   float min_value = 6;
   float max_value = 7;
   float step = 8;
@@ -1049,7 +1073,7 @@ message ListEntitiesNumberResponse {
   string unit_of_measurement = 11;
   NumberMode mode = 12;
   string device_class = 13;
-  uint32 device_id = 14;
+  uint32 device_id = 14 [(field_ifdef) = "USE_DEVICES"];
 }
 message NumberStateResponse {
   option (id) = 50;
@@ -1063,7 +1087,7 @@ message NumberStateResponse {
   // If the number does not have a valid state yet.
   // Equivalent to `!obj->has_state()` - inverse logic to make state packets smaller
   bool missing_state = 3;
-  uint32 device_id = 4;
+  uint32 device_id = 4 [(field_ifdef) = "USE_DEVICES"];
 }
 message NumberCommandRequest {
   option (id) = 51;
@@ -1074,7 +1098,7 @@ message NumberCommandRequest {
 
   fixed32 key = 1;
   float state = 2;
-  uint32 device_id = 3;
+  uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"];
 }
 
 // ==================== SELECT ====================
@@ -1087,13 +1111,13 @@ message ListEntitiesSelectResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
 
-  string icon = 5;
+  string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"];
   repeated string options = 6;
   bool disabled_by_default = 7;
   EntityCategory entity_category = 8;
-  uint32 device_id = 9;
+  uint32 device_id = 9 [(field_ifdef) = "USE_DEVICES"];
 }
 message SelectStateResponse {
   option (id) = 53;
@@ -1107,7 +1131,7 @@ message SelectStateResponse {
   // If the select does not have a valid state yet.
   // Equivalent to `!obj->has_state()` - inverse logic to make state packets smaller
   bool missing_state = 3;
-  uint32 device_id = 4;
+  uint32 device_id = 4 [(field_ifdef) = "USE_DEVICES"];
 }
 message SelectCommandRequest {
   option (id) = 54;
@@ -1118,7 +1142,7 @@ message SelectCommandRequest {
 
   fixed32 key = 1;
   string state = 2;
-  uint32 device_id = 3;
+  uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"];
 }
 
 // ==================== SIREN ====================
@@ -1131,15 +1155,15 @@ message ListEntitiesSirenResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
 
-  string icon = 5;
+  string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"];
   bool disabled_by_default = 6;
   repeated string tones = 7;
   bool supports_duration = 8;
   bool supports_volume = 9;
   EntityCategory entity_category = 10;
-  uint32 device_id = 11;
+  uint32 device_id = 11 [(field_ifdef) = "USE_DEVICES"];
 }
 message SirenStateResponse {
   option (id) = 56;
@@ -1150,7 +1174,7 @@ message SirenStateResponse {
 
   fixed32 key = 1;
   bool state = 2;
-  uint32 device_id = 3;
+  uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"];
 }
 message SirenCommandRequest {
   option (id) = 57;
@@ -1168,7 +1192,7 @@ message SirenCommandRequest {
   uint32 duration = 7;
   bool has_volume = 8;
   float volume = 9;
-  uint32 device_id = 10;
+  uint32 device_id = 10 [(field_ifdef) = "USE_DEVICES"];
 }
 
 // ==================== LOCK ====================
@@ -1194,9 +1218,9 @@ message ListEntitiesLockResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
 
-  string icon = 5;
+  string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"];
   bool disabled_by_default = 6;
   EntityCategory entity_category = 7;
   bool assumed_state = 8;
@@ -1206,7 +1230,7 @@ message ListEntitiesLockResponse {
 
   // Not yet implemented:
   string code_format = 11;
-  uint32 device_id = 12;
+  uint32 device_id = 12 [(field_ifdef) = "USE_DEVICES"];
 }
 message LockStateResponse {
   option (id) = 59;
@@ -1216,7 +1240,7 @@ message LockStateResponse {
   option (no_delay) = true;
   fixed32 key = 1;
   LockState state = 2;
-  uint32 device_id = 3;
+  uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"];
 }
 message LockCommandRequest {
   option (id) = 60;
@@ -1230,7 +1254,7 @@ message LockCommandRequest {
   // Not yet implemented:
   bool has_code = 3;
   string code = 4;
-  uint32 device_id = 5;
+  uint32 device_id = 5 [(field_ifdef) = "USE_DEVICES"];
 }
 
 // ==================== BUTTON ====================
@@ -1243,13 +1267,13 @@ message ListEntitiesButtonResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
 
-  string icon = 5;
+  string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"];
   bool disabled_by_default = 6;
   EntityCategory entity_category = 7;
   string device_class = 8;
-  uint32 device_id = 9;
+  uint32 device_id = 9 [(field_ifdef) = "USE_DEVICES"];
 }
 message ButtonCommandRequest {
   option (id) = 62;
@@ -1259,7 +1283,7 @@ message ButtonCommandRequest {
   option (base_class) = "CommandProtoMessage";
 
   fixed32 key = 1;
-  uint32 device_id = 2;
+  uint32 device_id = 2 [(field_ifdef) = "USE_DEVICES"];
 }
 
 // ==================== MEDIA PLAYER ====================
@@ -1298,9 +1322,9 @@ message ListEntitiesMediaPlayerResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
 
-  string icon = 5;
+  string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"];
   bool disabled_by_default = 6;
   EntityCategory entity_category = 7;
 
@@ -1308,7 +1332,7 @@ message ListEntitiesMediaPlayerResponse {
 
   repeated MediaPlayerSupportedFormat supported_formats = 9;
 
-  uint32 device_id = 10;
+  uint32 device_id = 10 [(field_ifdef) = "USE_DEVICES"];
 }
 message MediaPlayerStateResponse {
   option (id) = 64;
@@ -1320,7 +1344,7 @@ message MediaPlayerStateResponse {
   MediaPlayerState state = 2;
   float volume = 3;
   bool muted = 4;
-  uint32 device_id = 5;
+  uint32 device_id = 5 [(field_ifdef) = "USE_DEVICES"];
 }
 message MediaPlayerCommandRequest {
   option (id) = 65;
@@ -1342,7 +1366,7 @@ message MediaPlayerCommandRequest {
 
   bool has_announcement = 8;
   bool announcement = 9;
-  uint32 device_id = 10;
+  uint32 device_id = 10 [(field_ifdef) = "USE_DEVICES"];
 }
 
 // ==================== BLUETOOTH ====================
@@ -1354,12 +1378,17 @@ message SubscribeBluetoothLEAdvertisementsRequest {
   uint32 flags = 1;
 }
 
+// Deprecated - only used by deprecated BluetoothLEAdvertisementResponse
 message BluetoothServiceData {
+  option deprecated = true;
   string uuid = 1;
-  repeated uint32 legacy_data = 2 [deprecated = true];  // Removed in api version 1.7
+  // Deprecated in API version 1.7
+  repeated uint32 legacy_data = 2 [deprecated=true];  // Removed in api version 1.7
   bytes data = 3;  // Added in api version 1.7
 }
+// Removed in ESPHome 2025.8.0 - use BluetoothLERawAdvertisementsResponse instead
 message BluetoothLEAdvertisementResponse {
+  option deprecated = true;
   option (id) = 67;
   option (source) = SOURCE_SERVER;
   option (ifdef) = "USE_BLUETOOTH_PROXY";
@@ -1381,7 +1410,7 @@ message BluetoothLERawAdvertisement {
   sint32 rssi = 2;
   uint32 address_type = 3;
 
-  bytes data = 4;
+  bytes data = 4 [(fixed_array_size) = 62];
 }
 
 message BluetoothLERawAdvertisementsResponse {
@@ -1434,19 +1463,19 @@ message BluetoothGATTGetServicesRequest {
 }
 
 message BluetoothGATTDescriptor {
-  repeated uint64 uuid = 1;
+  repeated uint64 uuid = 1 [(fixed_array_size) = 2];
   uint32 handle = 2;
 }
 
 message BluetoothGATTCharacteristic {
-  repeated uint64 uuid = 1;
+  repeated uint64 uuid = 1 [(fixed_array_size) = 2];
   uint32 handle = 2;
   uint32 properties = 3;
   repeated BluetoothGATTDescriptor descriptors = 4;
 }
 
 message BluetoothGATTService {
-  repeated uint64 uuid = 1;
+  repeated uint64 uuid = 1 [(fixed_array_size) = 2];
   uint32 handle = 2;
   repeated BluetoothGATTCharacteristic characteristics = 3;
 }
@@ -1457,7 +1486,7 @@ message BluetoothGATTGetServicesResponse {
   option (ifdef) = "USE_BLUETOOTH_PROXY";
 
   uint64 address = 1;
-  repeated BluetoothGATTService services = 2;
+  repeated BluetoothGATTService services = 2 [(fixed_array_size) = 1];
 }
 
 message BluetoothGATTGetServicesDoneResponse {
@@ -1845,14 +1874,14 @@ message ListEntitiesAlarmControlPanelResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
-  string icon = 5;
+  reserved 4; // Deprecated: was string unique_id
+  string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"];
   bool disabled_by_default = 6;
   EntityCategory entity_category = 7;
   uint32 supported_features = 8;
   bool requires_code = 9;
   bool requires_code_to_arm = 10;
-  uint32 device_id = 11;
+  uint32 device_id = 11 [(field_ifdef) = "USE_DEVICES"];
 }
 
 message AlarmControlPanelStateResponse {
@@ -1863,7 +1892,7 @@ message AlarmControlPanelStateResponse {
   option (no_delay) = true;
   fixed32 key = 1;
   AlarmControlPanelState state = 2;
-  uint32 device_id = 3;
+  uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"];
 }
 
 message AlarmControlPanelCommandRequest {
@@ -1875,7 +1904,7 @@ message AlarmControlPanelCommandRequest {
   fixed32 key = 1;
   AlarmControlPanelStateCommand command = 2;
   string code = 3;
-  uint32 device_id = 4;
+  uint32 device_id = 4 [(field_ifdef) = "USE_DEVICES"];
 }
 
 // ===================== TEXT =====================
@@ -1892,8 +1921,8 @@ message ListEntitiesTextResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
-  string icon = 5;
+  reserved 4; // Deprecated: was string unique_id
+  string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"];
   bool disabled_by_default = 6;
   EntityCategory entity_category = 7;
 
@@ -1901,7 +1930,7 @@ message ListEntitiesTextResponse {
   uint32 max_length = 9;
   string pattern = 10;
   TextMode mode = 11;
-  uint32 device_id = 12;
+  uint32 device_id = 12 [(field_ifdef) = "USE_DEVICES"];
 }
 message TextStateResponse {
   option (id) = 98;
@@ -1915,7 +1944,7 @@ message TextStateResponse {
   // If the Text does not have a valid state yet.
   // Equivalent to `!obj->has_state()` - inverse logic to make state packets smaller
   bool missing_state = 3;
-  uint32 device_id = 4;
+  uint32 device_id = 4 [(field_ifdef) = "USE_DEVICES"];
 }
 message TextCommandRequest {
   option (id) = 99;
@@ -1926,7 +1955,7 @@ message TextCommandRequest {
 
   fixed32 key = 1;
   string state = 2;
-  uint32 device_id = 3;
+  uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"];
 }
 
 
@@ -1940,12 +1969,12 @@ message ListEntitiesDateResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
 
-  string icon = 5;
+  string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"];
   bool disabled_by_default = 6;
   EntityCategory entity_category = 7;
-  uint32 device_id = 8;
+  uint32 device_id = 8 [(field_ifdef) = "USE_DEVICES"];
 }
 message DateStateResponse {
   option (id) = 101;
@@ -1961,7 +1990,7 @@ message DateStateResponse {
   uint32 year = 3;
   uint32 month = 4;
   uint32 day = 5;
-  uint32 device_id = 6;
+  uint32 device_id = 6 [(field_ifdef) = "USE_DEVICES"];
 }
 message DateCommandRequest {
   option (id) = 102;
@@ -1974,7 +2003,7 @@ message DateCommandRequest {
   uint32 year = 2;
   uint32 month = 3;
   uint32 day = 4;
-  uint32 device_id = 5;
+  uint32 device_id = 5 [(field_ifdef) = "USE_DEVICES"];
 }
 
 // ==================== DATETIME TIME ====================
@@ -1987,12 +2016,12 @@ message ListEntitiesTimeResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
 
-  string icon = 5;
+  string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"];
   bool disabled_by_default = 6;
   EntityCategory entity_category = 7;
-  uint32 device_id = 8;
+  uint32 device_id = 8 [(field_ifdef) = "USE_DEVICES"];
 }
 message TimeStateResponse {
   option (id) = 104;
@@ -2008,7 +2037,7 @@ message TimeStateResponse {
   uint32 hour = 3;
   uint32 minute = 4;
   uint32 second = 5;
-  uint32 device_id = 6;
+  uint32 device_id = 6 [(field_ifdef) = "USE_DEVICES"];
 }
 message TimeCommandRequest {
   option (id) = 105;
@@ -2021,7 +2050,7 @@ message TimeCommandRequest {
   uint32 hour = 2;
   uint32 minute = 3;
   uint32 second = 4;
-  uint32 device_id = 5;
+  uint32 device_id = 5 [(field_ifdef) = "USE_DEVICES"];
 }
 
 // ==================== EVENT ====================
@@ -2034,15 +2063,15 @@ message ListEntitiesEventResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
 
-  string icon = 5;
+  string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"];
   bool disabled_by_default = 6;
   EntityCategory entity_category = 7;
   string device_class = 8;
 
   repeated string event_types = 9;
-  uint32 device_id = 10;
+  uint32 device_id = 10 [(field_ifdef) = "USE_DEVICES"];
 }
 message EventResponse {
   option (id) = 108;
@@ -2052,7 +2081,7 @@ message EventResponse {
 
   fixed32 key = 1;
   string event_type = 2;
-  uint32 device_id = 3;
+  uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"];
 }
 
 // ==================== VALVE ====================
@@ -2065,9 +2094,9 @@ message ListEntitiesValveResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
 
-  string icon = 5;
+  string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"];
   bool disabled_by_default = 6;
   EntityCategory entity_category = 7;
   string device_class = 8;
@@ -2075,7 +2104,7 @@ message ListEntitiesValveResponse {
   bool assumed_state = 9;
   bool supports_position = 10;
   bool supports_stop = 11;
-  uint32 device_id = 12;
+  uint32 device_id = 12 [(field_ifdef) = "USE_DEVICES"];
 }
 
 enum ValveOperation {
@@ -2093,7 +2122,7 @@ message ValveStateResponse {
   fixed32 key = 1;
   float position = 2;
   ValveOperation current_operation = 3;
-  uint32 device_id = 4;
+  uint32 device_id = 4 [(field_ifdef) = "USE_DEVICES"];
 }
 
 message ValveCommandRequest {
@@ -2107,7 +2136,7 @@ message ValveCommandRequest {
   bool has_position = 2;
   float position = 3;
   bool stop = 4;
-  uint32 device_id = 5;
+  uint32 device_id = 5 [(field_ifdef) = "USE_DEVICES"];
 }
 
 // ==================== DATETIME DATETIME ====================
@@ -2120,12 +2149,12 @@ message ListEntitiesDateTimeResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
 
-  string icon = 5;
+  string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"];
   bool disabled_by_default = 6;
   EntityCategory entity_category = 7;
-  uint32 device_id = 8;
+  uint32 device_id = 8 [(field_ifdef) = "USE_DEVICES"];
 }
 message DateTimeStateResponse {
   option (id) = 113;
@@ -2139,7 +2168,7 @@ message DateTimeStateResponse {
   // Equivalent to `!obj->has_state()` - inverse logic to make state packets smaller
   bool missing_state = 2;
   fixed32 epoch_seconds = 3;
-  uint32 device_id = 4;
+  uint32 device_id = 4 [(field_ifdef) = "USE_DEVICES"];
 }
 message DateTimeCommandRequest {
   option (id) = 114;
@@ -2150,7 +2179,7 @@ message DateTimeCommandRequest {
 
   fixed32 key = 1;
   fixed32 epoch_seconds = 2;
-  uint32 device_id = 3;
+  uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"];
 }
 
 // ==================== UPDATE ====================
@@ -2163,13 +2192,13 @@ message ListEntitiesUpdateResponse {
   string object_id = 1;
   fixed32 key = 2;
   string name = 3;
-  string unique_id = 4;
+  reserved 4; // Deprecated: was string unique_id
 
-  string icon = 5;
+  string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"];
   bool disabled_by_default = 6;
   EntityCategory entity_category = 7;
   string device_class = 8;
-  uint32 device_id = 9;
+  uint32 device_id = 9 [(field_ifdef) = "USE_DEVICES"];
 }
 message UpdateStateResponse {
   option (id) = 117;
@@ -2188,7 +2217,7 @@ message UpdateStateResponse {
   string title = 8;
   string release_summary = 9;
   string release_url = 10;
-  uint32 device_id = 11;
+  uint32 device_id = 11 [(field_ifdef) = "USE_DEVICES"];
 }
 enum UpdateCommand {
   UPDATE_COMMAND_NONE = 0;
@@ -2204,5 +2233,5 @@ message UpdateCommandRequest {
 
   fixed32 key = 1;
   UpdateCommand command = 2;
-  uint32 device_id = 3;
+  uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"];
 }

esphome/components/api/api_connection.h

@@ -13,13 +13,36 @@
 #include <vector>
 #include <functional>
 
-namespace esphome {
-namespace api {
+namespace esphome::api {
+
+// Client information structure
+struct ClientInfo {
+  std::string name;      // Client name from Hello message
+  std::string peername;  // IP:port from socket
+
+  std::string get_combined_info() const {
+    if (name == peername) {
+      // Before Hello message, both are the same
+      return name;
+    }
+    return name + " (" + peername + ")";
+  }
+};
 
 // Keepalive timeout in milliseconds
 static constexpr uint32_t KEEPALIVE_TIMEOUT_MS = 60000;
 // Maximum number of entities to process in a single batch during initial state/info sending
-static constexpr size_t MAX_INITIAL_PER_BATCH = 20;
+// This was increased from 20 to 24 after removing the unique_id field from entity info messages,
+// which reduced message sizes allowing more entities per batch without exceeding packet limits
+static constexpr size_t MAX_INITIAL_PER_BATCH = 24;
+// Maximum number of packets to process in a single batch (platform-dependent)
+// This limit exists to prevent stack overflow from the PacketInfo array in process_batch_
+// Each PacketInfo is 8 bytes, so 64 * 8 = 512 bytes, 32 * 8 = 256 bytes
+#if defined(USE_ESP32) || defined(USE_HOST)
+static constexpr size_t MAX_PACKETS_PER_BATCH = 64;  // ESP32 has 8KB+ stack, HOST has plenty
+#else
+static constexpr size_t MAX_PACKETS_PER_BATCH = 32;  // ESP8266/RP2040/etc have smaller stacks
+#endif
 
 class APIConnection : public APIServerConnection {
  public:
@@ -111,12 +134,11 @@ class APIConnection : public APIServerConnection {
   void send_homeassistant_service_call(const HomeassistantServiceResponse &call) {
     if (!this->flags_.service_call_subscription)
       return;
-    this->send_message(call);
+    this->send_message(call, HomeassistantServiceResponse::MESSAGE_TYPE);
   }
 #ifdef USE_BLUETOOTH_PROXY
   void subscribe_bluetooth_le_advertisements(const SubscribeBluetoothLEAdvertisementsRequest &msg) override;
   void unsubscribe_bluetooth_le_advertisements(const UnsubscribeBluetoothLEAdvertisementsRequest &msg) override;
-  bool send_bluetooth_le_advertisement(const BluetoothLEAdvertisementResponse &msg);
 
   void bluetooth_device_request(const BluetoothDeviceRequest &msg) override;
   void bluetooth_gatt_read(const BluetoothGATTReadRequest &msg) override;
@@ -125,15 +147,14 @@ class APIConnection : public APIServerConnection {
   void bluetooth_gatt_write_descriptor(const BluetoothGATTWriteDescriptorRequest &msg) override;
   void bluetooth_gatt_get_services(const BluetoothGATTGetServicesRequest &msg) override;
   void bluetooth_gatt_notify(const BluetoothGATTNotifyRequest &msg) override;
-  BluetoothConnectionsFreeResponse subscribe_bluetooth_connections_free(
-      const SubscribeBluetoothConnectionsFreeRequest &msg) override;
+  bool send_subscribe_bluetooth_connections_free_response(const SubscribeBluetoothConnectionsFreeRequest &msg) override;
   void bluetooth_scanner_set_mode(const BluetoothScannerSetModeRequest &msg) override;
 
 #endif
 #ifdef USE_HOMEASSISTANT_TIME
   void send_time_request() {
     GetTimeRequest req;
-    this->send_message(req);
+    this->send_message(req, GetTimeRequest::MESSAGE_TYPE);
   }
 #endif
 
@@ -144,8 +165,7 @@ class APIConnection : public APIServerConnection {
   void on_voice_assistant_audio(const VoiceAssistantAudio &msg) override;
   void on_voice_assistant_timer_event_response(const VoiceAssistantTimerEventResponse &msg) override;
   void on_voice_assistant_announce_request(const VoiceAssistantAnnounceRequest &msg) override;
-  VoiceAssistantConfigurationResponse voice_assistant_get_configuration(
-      const VoiceAssistantConfigurationRequest &msg) override;
+  bool send_voice_assistant_get_configuration_response(const VoiceAssistantConfigurationRequest &msg) override;
   void voice_assistant_set_configuration(const VoiceAssistantSetConfiguration &msg) override;
 #endif
 
@@ -172,11 +192,11 @@ class APIConnection : public APIServerConnection {
 #ifdef USE_HOMEASSISTANT_TIME
   void on_get_time_response(const GetTimeResponse &value) override;
 #endif
-  HelloResponse hello(const HelloRequest &msg) override;
-  ConnectResponse connect(const ConnectRequest &msg) override;
-  DisconnectResponse disconnect(const DisconnectRequest &msg) override;
-  PingResponse ping(const PingRequest &msg) override { return {}; }
-  DeviceInfoResponse device_info(const DeviceInfoRequest &msg) override;
+  bool send_hello_response(const HelloRequest &msg) override;
+  bool send_connect_response(const ConnectRequest &msg) override;
+  bool send_disconnect_response(const DisconnectRequest &msg) override;
+  bool send_ping_response(const PingRequest &msg) override;
+  bool send_device_info_response(const DeviceInfoRequest &msg) override;
   void list_entities(const ListEntitiesRequest &msg) override { this->list_entities_iterator_.begin(); }
   void subscribe_states(const SubscribeStatesRequest &msg) override {
     this->flags_.state_subscription = true;
@@ -191,15 +211,12 @@ class APIConnection : public APIServerConnection {
     this->flags_.service_call_subscription = true;
   }
   void subscribe_home_assistant_states(const SubscribeHomeAssistantStatesRequest &msg) override;
-  GetTimeResponse get_time(const GetTimeRequest &msg) override {
-    // TODO
-    return {};
-  }
+  bool send_get_time_response(const GetTimeRequest &msg) override;
 #ifdef USE_API_SERVICES
   void execute_service(const ExecuteServiceRequest &msg) override;
 #endif
 #ifdef USE_API_NOISE
-  NoiseEncryptionSetKeyResponse noise_encryption_set_key(const NoiseEncryptionSetKeyRequest &msg) override;
+  bool send_noise_encryption_set_key_response(const NoiseEncryptionSetKeyRequest &msg) override;
 #endif
 
   bool is_authenticated() override {
@@ -209,6 +226,7 @@ class APIConnection : public APIServerConnection {
     return static_cast<ConnectionState>(this->flags_.connection_state) == ConnectionState::CONNECTED ||
            this->is_authenticated();
   }
+  uint8_t get_log_subscription_level() const { return this->flags_.log_subscription; }
   void on_fatal_error() override;
   void on_unauthenticated_access() override;
   void on_no_setup_connection() override;
@@ -260,49 +278,55 @@ class APIConnection : public APIServerConnection {
   bool try_to_clear_buffer(bool log_out_of_space);
   bool send_buffer(ProtoWriteBuffer buffer, uint8_t message_type) override;
 
-  std::string get_client_combined_info() const {
-    if (this->client_info_ == this->client_peername_) {
-      // Before Hello message, both are the same (just IP:port)
-      return this->client_info_;
-    }
-    return this->client_info_ + " (" + this->client_peername_ + ")";
-  }
+  std::string get_client_combined_info() const { return this->client_info_.get_combined_info(); }
 
   // Buffer allocator methods for batch processing
   ProtoWriteBuffer allocate_single_message_buffer(uint16_t size);
   ProtoWriteBuffer allocate_batch_message_buffer(uint16_t size);
 
  protected:
-  // Helper function to fill common entity info fields
-  static void fill_entity_info_base(esphome::EntityBase *entity, InfoResponseProtoMessage &response) {
-    // Set common fields that are shared by all entity types
-    response.key = entity->get_object_id_hash();
-    response.object_id = entity->get_object_id();
-
-    if (entity->has_own_name())
-      response.name = entity->get_name();
-
-    // Set common EntityBase properties
-    response.icon = entity->get_icon();
-    response.disabled_by_default = entity->is_disabled_by_default();
-    response.entity_category = static_cast<enums::EntityCategory>(entity->get_entity_category());
-#ifdef USE_DEVICES
-    response.device_id = entity->get_device_id();
-#endif
-  }
-
-  // Helper function to fill common entity state fields
-  static void fill_entity_state_base(esphome::EntityBase *entity, StateResponseProtoMessage &response) {
-    response.key = entity->get_object_id_hash();
-#ifdef USE_DEVICES
-    response.device_id = entity->get_device_id();
-#endif
-  }
+  // Helper function to handle authentication completion
+  void complete_authentication_();
 
   // Non-template helper to encode any ProtoMessage
   static uint16_t encode_message_to_buffer(ProtoMessage &msg, uint8_t message_type, APIConnection *conn,
                                            uint32_t remaining_size, bool is_single);
 
+  // Helper to fill entity state base and encode message
+  static uint16_t fill_and_encode_entity_state(EntityBase *entity, StateResponseProtoMessage &msg, uint8_t message_type,
+                                               APIConnection *conn, uint32_t remaining_size, bool is_single) {
+    msg.key = entity->get_object_id_hash();
+#ifdef USE_DEVICES
+    msg.device_id = entity->get_device_id();
+#endif
+    return encode_message_to_buffer(msg, message_type, conn, remaining_size, is_single);
+  }
+
+  // Helper to fill entity info base and encode message
+  static uint16_t fill_and_encode_entity_info(EntityBase *entity, InfoResponseProtoMessage &msg, uint8_t message_type,
+                                              APIConnection *conn, uint32_t remaining_size, bool is_single) {
+    // Set common fields that are shared by all entity types
+    msg.key = entity->get_object_id_hash();
+    // IMPORTANT: get_object_id() may return a temporary std::string
+    std::string object_id = entity->get_object_id();
+    msg.set_object_id(StringRef(object_id));
+
+    if (entity->has_own_name()) {
+      msg.set_name(entity->get_name());
+    }
+
+    // Set common EntityBase properties
+#ifdef USE_ENTITY_ICON
+    msg.set_icon(entity->get_icon_ref());
+#endif
+    msg.disabled_by_default = entity->is_disabled_by_default();
+    msg.entity_category = static_cast<enums::EntityCategory>(entity->get_entity_category());
+#ifdef USE_DEVICES
+    msg.device_id = entity->get_device_id();
+#endif
+    return encode_message_to_buffer(msg, message_type, conn, remaining_size, is_single);
+  }
+
 #ifdef USE_VOICE_ASSISTANT
   // Helper to check voice assistant validity and connection ownership
   inline bool check_voice_assistant_api_connection_() const;
@@ -463,9 +487,8 @@ class APIConnection : public APIServerConnection {
   std::unique_ptr<camera::CameraImageReader> image_reader_;
 #endif
 
-  // Group 3: Strings (12 bytes each on 32-bit, 4-byte aligned)
-  std::string client_info_;
-  std::string client_peername_;
+  // Group 3: Client info struct (24 bytes on 32-bit: 2 strings × 12 bytes each)
+  ClientInfo client_info_;
 
   // Group 4: 4-byte types
   uint32_t last_traffic_;
@@ -699,6 +722,5 @@ class APIConnection : public APIServerConnection {
   }
 };
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api
 #endif

esphome/components/api/api_frame_helper.h

@@ -8,16 +8,17 @@
 
 #include "esphome/core/defines.h"
 #ifdef USE_API
-#ifdef USE_API_NOISE
-#include "noise/protocol.h"
-#endif
-
-#include "api_noise_context.h"
 #include "esphome/components/socket/socket.h"
 #include "esphome/core/application.h"
+#include "esphome/core/log.h"
 
-namespace esphome {
-namespace api {
+namespace esphome::api {
+
+// uncomment to log raw packets
+//#define HELPER_LOG_PACKETS
+
+// Forward declaration
+struct ClientInfo;
 
 class ProtoWriteBuffer;
 
@@ -40,7 +41,6 @@ struct PacketInfo {
 enum class APIError : uint16_t {
   OK = 0,
   WOULD_BLOCK = 1001,
-  BAD_HANDSHAKE_PACKET_LEN = 1002,
   BAD_INDICATOR = 1003,
   BAD_DATA_PACKET = 1004,
   TCP_NODELAY_FAILED = 1005,
@@ -51,16 +51,19 @@ enum class APIError : uint16_t {
   BAD_ARG = 1010,
   SOCKET_READ_FAILED = 1011,
   SOCKET_WRITE_FAILED = 1012,
+  OUT_OF_MEMORY = 1018,
+  CONNECTION_CLOSED = 1022,
+#ifdef USE_API_NOISE
+  BAD_HANDSHAKE_PACKET_LEN = 1002,
   HANDSHAKESTATE_READ_FAILED = 1013,
   HANDSHAKESTATE_WRITE_FAILED = 1014,
   HANDSHAKESTATE_BAD_STATE = 1015,
   CIPHERSTATE_DECRYPT_FAILED = 1016,
   CIPHERSTATE_ENCRYPT_FAILED = 1017,
-  OUT_OF_MEMORY = 1018,
   HANDSHAKESTATE_SETUP_FAILED = 1019,
   HANDSHAKESTATE_SPLIT_FAILED = 1020,
   BAD_HANDSHAKE_ERROR_BYTE = 1021,
-  CONNECTION_CLOSED = 1022,
+#endif
 };
 
 const char *api_error_to_str(APIError err);
@@ -68,7 +71,8 @@ const char *api_error_to_str(APIError err);
 class APIFrameHelper {
  public:
   APIFrameHelper() = default;
-  explicit APIFrameHelper(std::unique_ptr<socket::Socket> socket) : socket_owned_(std::move(socket)) {
+  explicit APIFrameHelper(std::unique_ptr<socket::Socket> socket, const ClientInfo *client_info)
+      : socket_owned_(std::move(socket)), client_info_(client_info) {
     socket_ = socket_owned_.get();
   }
   virtual ~APIFrameHelper() = default;
@@ -94,8 +98,6 @@ class APIFrameHelper {
     }
     return APIError::OK;
   }
-  // Give this helper a name for logging
-  void set_log_info(std::string info) { info_ = std::move(info); }
   virtual APIError write_protobuf_packet(uint8_t type, ProtoWriteBuffer buffer) = 0;
   // Write multiple protobuf packets in a single operation
   // packets contains (message_type, offset, length) for each message in the buffer
@@ -109,29 +111,28 @@ class APIFrameHelper {
   bool is_socket_ready() const { return socket_ != nullptr && socket_->ready(); }
 
  protected:
-  // Struct for holding parsed frame data
-  struct ParsedFrame {
-    std::vector<uint8_t> msg;
-  };
-
   // Buffer containing data to be sent
   struct SendBuffer {
-    std::vector<uint8_t> data;
-    uint16_t offset{0};  // Current offset within the buffer (uint16_t to reduce memory usage)
+    std::unique_ptr<uint8_t[]> data;
+    uint16_t size{0};    // Total size of the buffer
+    uint16_t offset{0};  // Current offset within the buffer
 
     // Using uint16_t reduces memory usage since ESPHome API messages are limited to UINT16_MAX (65535) bytes
-    uint16_t remaining() const { return static_cast<uint16_t>(data.size()) - offset; }
-    const uint8_t *current_data() const { return data.data() + offset; }
+    uint16_t remaining() const { return size - offset; }
+    const uint8_t *current_data() const { return data.get() + offset; }
   };
 
   // Common implementation for writing raw data to socket
-  APIError write_raw_(const struct iovec *iov, int iovcnt);
+  APIError write_raw_(const struct iovec *iov, int iovcnt, uint16_t total_write_len);
 
   // Try to send data from the tx buffer
   APIError try_send_tx_buf_();
 
   // Helper method to buffer data from IOVs
-  void buffer_data_from_iov_(const struct iovec *iov, int iovcnt, uint16_t total_write_len);
+  void buffer_data_from_iov_(const struct iovec *iov, int iovcnt, uint16_t total_write_len, uint16_t offset);
+
+  // Common socket write error handling
+  APIError handle_socket_write_error_();
   template<typename StateEnum>
   APIError write_raw_(const struct iovec *iov, int iovcnt, socket::Socket *socket, std::vector<uint8_t> &tx_buf,
                       const std::string &info, StateEnum &state, StateEnum failed_state);
@@ -161,10 +162,13 @@ class APIFrameHelper {
 
   // Containers (size varies, but typically 12+ bytes on 32-bit)
   std::deque<SendBuffer> tx_buf_;
-  std::string info_;
   std::vector<struct iovec> reusable_iovs_;
   std::vector<uint8_t> rx_buf_;
 
+  // Pointer to client info (4 bytes on 32-bit)
+  // Note: The pointed-to ClientInfo object must outlive this APIFrameHelper instance.
+  const ClientInfo *client_info_{nullptr};
+
   // Group smaller types together
   uint16_t rx_buf_len_ = 0;
   State state_{State::INITIALIZE};
@@ -179,105 +183,6 @@ class APIFrameHelper {
   APIError handle_socket_read_result_(ssize_t received);
 };
 
-#ifdef USE_API_NOISE
-class APINoiseFrameHelper : public APIFrameHelper {
- public:
-  APINoiseFrameHelper(std::unique_ptr<socket::Socket> socket, std::shared_ptr<APINoiseContext> ctx)
-      : APIFrameHelper(std::move(socket)), ctx_(std::move(ctx)) {
-    // Noise header structure:
-    // Pos 0: indicator (0x01)
-    // Pos 1-2: encrypted payload size (16-bit big-endian)
-    // Pos 3-6: encrypted type (16-bit) + data_len (16-bit)
-    // Pos 7+: actual payload data
-    frame_header_padding_ = 7;
-  }
-  ~APINoiseFrameHelper() override;
-  APIError init() override;
-  APIError loop() override;
-  APIError read_packet(ReadPacketBuffer *buffer) override;
-  APIError write_protobuf_packet(uint8_t type, ProtoWriteBuffer buffer) override;
-  APIError write_protobuf_packets(ProtoWriteBuffer buffer, std::span<const PacketInfo> packets) override;
-  // Get the frame header padding required by this protocol
-  uint8_t frame_header_padding() override { return frame_header_padding_; }
-  // Get the frame footer size required by this protocol
-  uint8_t frame_footer_size() override { return frame_footer_size_; }
+}  // namespace esphome::api
 
- protected:
-  APIError state_action_();
-  APIError try_read_frame_(ParsedFrame *frame);
-  APIError write_frame_(const uint8_t *data, uint16_t len);
-  APIError init_handshake_();
-  APIError check_handshake_finished_();
-  void send_explicit_handshake_reject_(const std::string &reason);
-
-  // Pointers first (4 bytes each)
-  NoiseHandshakeState *handshake_{nullptr};
-  NoiseCipherState *send_cipher_{nullptr};
-  NoiseCipherState *recv_cipher_{nullptr};
-
-  // Shared pointer (8 bytes on 32-bit = 4 bytes control block pointer + 4 bytes object pointer)
-  std::shared_ptr<APINoiseContext> ctx_;
-
-  // Vector (12 bytes on 32-bit)
-  std::vector<uint8_t> prologue_;
-
-  // NoiseProtocolId (size depends on implementation)
-  NoiseProtocolId nid_;
-
-  // Group small types together
-  // Fixed-size header buffer for noise protocol:
-  // 1 byte for indicator + 2 bytes for message size (16-bit value, not varint)
-  // Note: Maximum message size is UINT16_MAX (65535), with a limit of 128 bytes during handshake phase
-  uint8_t rx_header_buf_[3];
-  uint8_t rx_header_buf_len_ = 0;
-  // 4 bytes total, no padding
-};
-#endif  // USE_API_NOISE
-
-#ifdef USE_API_PLAINTEXT
-class APIPlaintextFrameHelper : public APIFrameHelper {
- public:
-  APIPlaintextFrameHelper(std::unique_ptr<socket::Socket> socket) : APIFrameHelper(std::move(socket)) {
-    // Plaintext header structure (worst case):
-    // Pos 0: indicator (0x00)
-    // Pos 1-3: payload size varint (up to 3 bytes)
-    // Pos 4-5: message type varint (up to 2 bytes)
-    // Pos 6+: actual payload data
-    frame_header_padding_ = 6;
-  }
-  ~APIPlaintextFrameHelper() override = default;
-  APIError init() override;
-  APIError loop() override;
-  APIError read_packet(ReadPacketBuffer *buffer) override;
-  APIError write_protobuf_packet(uint8_t type, ProtoWriteBuffer buffer) override;
-  APIError write_protobuf_packets(ProtoWriteBuffer buffer, std::span<const PacketInfo> packets) override;
-  uint8_t frame_header_padding() override { return frame_header_padding_; }
-  // Get the frame footer size required by this protocol
-  uint8_t frame_footer_size() override { return frame_footer_size_; }
-
- protected:
-  APIError try_read_frame_(ParsedFrame *frame);
-
-  // Group 2-byte aligned types
-  uint16_t rx_header_parsed_type_ = 0;
-  uint16_t rx_header_parsed_len_ = 0;
-
-  // Group 1-byte types together
-  // Fixed-size header buffer for plaintext protocol:
-  // We now store the indicator byte + the two varints.
-  // To match noise protocol's maximum message size (UINT16_MAX = 65535), we need:
-  // 1 byte for indicator + 3 bytes for message size varint (supports up to 2097151) + 2 bytes for message type varint
-  //
-  // While varints could theoretically be up to 10 bytes each for 64-bit values,
-  // attempting to process messages with headers that large would likely crash the
-  // ESP32 due to memory constraints.
-  uint8_t rx_header_buf_[6];  // 1 byte indicator + 5 bytes for varints (3 for size + 2 for type)
-  uint8_t rx_header_buf_pos_ = 0;
-  bool rx_header_parsed_ = false;
-  // 8 bytes total, no padding needed
-};
-#endif
-
-}  // namespace api
-}  // namespace esphome
-#endif
+#endif  // USE_API

esphome/components/api/api_frame_helper_noise.cpp

@@ -0,0 +1,583 @@
+#include "api_frame_helper_noise.h"
+#ifdef USE_API
+#ifdef USE_API_NOISE
+#include "api_connection.h"  // For ClientInfo struct
+#include "esphome/core/application.h"
+#include "esphome/core/hal.h"
+#include "esphome/core/helpers.h"
+#include "esphome/core/log.h"
+#include "proto.h"
+#include <cstring>
+#include <cinttypes>
+
+namespace esphome::api {
+
+static const char *const TAG = "api.noise";
+static const char *const PROLOGUE_INIT = "NoiseAPIInit";
+static constexpr size_t PROLOGUE_INIT_LEN = 12;  // strlen("NoiseAPIInit")
+
+#define HELPER_LOG(msg, ...) ESP_LOGVV(TAG, "%s: " msg, this->client_info_->get_combined_info().c_str(), ##__VA_ARGS__)
+
+#ifdef HELPER_LOG_PACKETS
+#define LOG_PACKET_RECEIVED(buffer) ESP_LOGVV(TAG, "Received frame: %s", format_hex_pretty(buffer).c_str())
+#define LOG_PACKET_SENDING(data, len) ESP_LOGVV(TAG, "Sending raw: %s", format_hex_pretty(data, len).c_str())
+#else
+#define LOG_PACKET_RECEIVED(buffer) ((void) 0)
+#define LOG_PACKET_SENDING(data, len) ((void) 0)
+#endif
+
+/// Convert a noise error code to a readable error
+std::string noise_err_to_str(int err) {
+  if (err == NOISE_ERROR_NO_MEMORY)
+    return "NO_MEMORY";
+  if (err == NOISE_ERROR_UNKNOWN_ID)
+    return "UNKNOWN_ID";
+  if (err == NOISE_ERROR_UNKNOWN_NAME)
+    return "UNKNOWN_NAME";
+  if (err == NOISE_ERROR_MAC_FAILURE)
+    return "MAC_FAILURE";
+  if (err == NOISE_ERROR_NOT_APPLICABLE)
+    return "NOT_APPLICABLE";
+  if (err == NOISE_ERROR_SYSTEM)
+    return "SYSTEM";
+  if (err == NOISE_ERROR_REMOTE_KEY_REQUIRED)
+    return "REMOTE_KEY_REQUIRED";
+  if (err == NOISE_ERROR_LOCAL_KEY_REQUIRED)
+    return "LOCAL_KEY_REQUIRED";
+  if (err == NOISE_ERROR_PSK_REQUIRED)
+    return "PSK_REQUIRED";
+  if (err == NOISE_ERROR_INVALID_LENGTH)
+    return "INVALID_LENGTH";
+  if (err == NOISE_ERROR_INVALID_PARAM)
+    return "INVALID_PARAM";
+  if (err == NOISE_ERROR_INVALID_STATE)
+    return "INVALID_STATE";
+  if (err == NOISE_ERROR_INVALID_NONCE)
+    return "INVALID_NONCE";
+  if (err == NOISE_ERROR_INVALID_PRIVATE_KEY)
+    return "INVALID_PRIVATE_KEY";
+  if (err == NOISE_ERROR_INVALID_PUBLIC_KEY)
+    return "INVALID_PUBLIC_KEY";
+  if (err == NOISE_ERROR_INVALID_FORMAT)
+    return "INVALID_FORMAT";
+  if (err == NOISE_ERROR_INVALID_SIGNATURE)
+    return "INVALID_SIGNATURE";
+  return to_string(err);
+}
+
+/// Initialize the frame helper, returns OK if successful.
+APIError APINoiseFrameHelper::init() {
+  APIError err = init_common_();
+  if (err != APIError::OK) {
+    return err;
+  }
+
+  // init prologue
+  size_t old_size = prologue_.size();
+  prologue_.resize(old_size + PROLOGUE_INIT_LEN);
+  std::memcpy(prologue_.data() + old_size, PROLOGUE_INIT, PROLOGUE_INIT_LEN);
+
+  state_ = State::CLIENT_HELLO;
+  return APIError::OK;
+}
+// Helper for handling handshake frame errors
+APIError APINoiseFrameHelper::handle_handshake_frame_error_(APIError aerr) {
+  if (aerr == APIError::BAD_INDICATOR) {
+    send_explicit_handshake_reject_("Bad indicator byte");
+  } else if (aerr == APIError::BAD_HANDSHAKE_PACKET_LEN) {
+    send_explicit_handshake_reject_("Bad handshake packet len");
+  }
+  return aerr;
+}
+
+// Helper for handling noise library errors
+APIError APINoiseFrameHelper::handle_noise_error_(int err, const char *func_name, APIError api_err) {
+  if (err != 0) {
+    state_ = State::FAILED;
+    HELPER_LOG("%s failed: %s", func_name, noise_err_to_str(err).c_str());
+    return api_err;
+  }
+  return APIError::OK;
+}
+
+/// Run through handshake messages (if in that phase)
+APIError APINoiseFrameHelper::loop() {
+  // During handshake phase, process as many actions as possible until we can't progress
+  // socket_->ready() stays true until next main loop, but state_action() will return
+  // WOULD_BLOCK when no more data is available to read
+  while (state_ != State::DATA && this->socket_->ready()) {
+    APIError err = state_action_();
+    if (err == APIError::WOULD_BLOCK) {
+      break;
+    }
+    if (err != APIError::OK) {
+      return err;
+    }
+  }
+
+  // Use base class implementation for buffer sending
+  return APIFrameHelper::loop();
+}
+
+/** Read a packet into the rx_buf_. If successful, stores frame data in the frame parameter
+ *
+ * @param frame: The struct to hold the frame information in.
+ *   msg_start: points to the start of the payload - this pointer is only valid until the next
+ *     try_receive_raw_ call
+ *
+ * @return 0 if a full packet is in rx_buf_
+ * @return -1 if error, check errno.
+ *
+ * errno EWOULDBLOCK: Packet could not be read without blocking. Try again later.
+ * errno ENOMEM: Not enough memory for reading packet.
+ * errno API_ERROR_BAD_INDICATOR: Bad indicator byte at start of frame.
+ * errno API_ERROR_HANDSHAKE_PACKET_LEN: Packet too big for this phase.
+ */
+APIError APINoiseFrameHelper::try_read_frame_(std::vector<uint8_t> *frame) {
+  if (frame == nullptr) {
+    HELPER_LOG("Bad argument for try_read_frame_");
+    return APIError::BAD_ARG;
+  }
+
+  // read header
+  if (rx_header_buf_len_ < 3) {
+    // no header information yet
+    uint8_t to_read = 3 - rx_header_buf_len_;
+    ssize_t received = this->socket_->read(&rx_header_buf_[rx_header_buf_len_], to_read);
+    APIError err = handle_socket_read_result_(received);
+    if (err != APIError::OK) {
+      return err;
+    }
+    rx_header_buf_len_ += static_cast<uint8_t>(received);
+    if (static_cast<uint8_t>(received) != to_read) {
+      // not a full read
+      return APIError::WOULD_BLOCK;
+    }
+
+    if (rx_header_buf_[0] != 0x01) {
+      state_ = State::FAILED;
+      HELPER_LOG("Bad indicator byte %u", rx_header_buf_[0]);
+      return APIError::BAD_INDICATOR;
+    }
+    // header reading done
+  }
+
+  // read body
+  uint16_t msg_size = (((uint16_t) rx_header_buf_[1]) << 8) | rx_header_buf_[2];
+
+  if (state_ != State::DATA && msg_size > 128) {
+    // for handshake message only permit up to 128 bytes
+    state_ = State::FAILED;
+    HELPER_LOG("Bad packet len for handshake: %d", msg_size);
+    return APIError::BAD_HANDSHAKE_PACKET_LEN;
+  }
+
+  // reserve space for body
+  if (rx_buf_.size() != msg_size) {
+    rx_buf_.resize(msg_size);
+  }
+
+  if (rx_buf_len_ < msg_size) {
+    // more data to read
+    uint16_t to_read = msg_size - rx_buf_len_;
+    ssize_t received = this->socket_->read(&rx_buf_[rx_buf_len_], to_read);
+    APIError err = handle_socket_read_result_(received);
+    if (err != APIError::OK) {
+      return err;
+    }
+    rx_buf_len_ += static_cast<uint16_t>(received);
+    if (static_cast<uint16_t>(received) != to_read) {
+      // not all read
+      return APIError::WOULD_BLOCK;
+    }
+  }
+
+  LOG_PACKET_RECEIVED(rx_buf_);
+  *frame = std::move(rx_buf_);
+  // consume msg
+  rx_buf_ = {};
+  rx_buf_len_ = 0;
+  rx_header_buf_len_ = 0;
+  return APIError::OK;
+}
+
+/** To be called from read/write methods.
+ *
+ * This method runs through the internal handshake methods, if in that state.
+ *
+ * If the handshake is still active when this method returns and a read/write can't take place at
+ * the moment, returns WOULD_BLOCK.
+ * If an error occurred, returns that error. Only returns OK if the transport is ready for data
+ * traffic.
+ */
+APIError APINoiseFrameHelper::state_action_() {
+  int err;
+  APIError aerr;
+  if (state_ == State::INITIALIZE) {
+    HELPER_LOG("Bad state for method: %d", (int) state_);
+    return APIError::BAD_STATE;
+  }
+  if (state_ == State::CLIENT_HELLO) {
+    // waiting for client hello
+    std::vector<uint8_t> frame;
+    aerr = try_read_frame_(&frame);
+    if (aerr != APIError::OK) {
+      return handle_handshake_frame_error_(aerr);
+    }
+    // ignore contents, may be used in future for flags
+    // Resize for: existing prologue + 2 size bytes + frame data
+    size_t old_size = prologue_.size();
+    prologue_.resize(old_size + 2 + frame.size());
+    prologue_[old_size] = (uint8_t) (frame.size() >> 8);
+    prologue_[old_size + 1] = (uint8_t) frame.size();
+    std::memcpy(prologue_.data() + old_size + 2, frame.data(), frame.size());
+
+    state_ = State::SERVER_HELLO;
+  }
+  if (state_ == State::SERVER_HELLO) {
+    // send server hello
+    const std::string &name = App.get_name();
+    const std::string &mac = get_mac_address();
+
+    std::vector<uint8_t> msg;
+    // Calculate positions and sizes
+    size_t name_len = name.size() + 1;  // including null terminator
+    size_t mac_len = mac.size() + 1;    // including null terminator
+    size_t name_offset = 1;
+    size_t mac_offset = name_offset + name_len;
+    size_t total_size = 1 + name_len + mac_len;
+
+    msg.resize(total_size);
+
+    // chosen proto
+    msg[0] = 0x01;
+
+    // node name, terminated by null byte
+    std::memcpy(msg.data() + name_offset, name.c_str(), name_len);
+    // node mac, terminated by null byte
+    std::memcpy(msg.data() + mac_offset, mac.c_str(), mac_len);
+
+    aerr = write_frame_(msg.data(), msg.size());
+    if (aerr != APIError::OK)
+      return aerr;
+
+    // start handshake
+    aerr = init_handshake_();
+    if (aerr != APIError::OK)
+      return aerr;
+
+    state_ = State::HANDSHAKE;
+  }
+  if (state_ == State::HANDSHAKE) {
+    int action = noise_handshakestate_get_action(handshake_);
+    if (action == NOISE_ACTION_READ_MESSAGE) {
+      // waiting for handshake msg
+      std::vector<uint8_t> frame;
+      aerr = try_read_frame_(&frame);
+      if (aerr != APIError::OK) {
+        return handle_handshake_frame_error_(aerr);
+      }
+
+      if (frame.empty()) {
+        send_explicit_handshake_reject_("Empty handshake message");
+        return APIError::BAD_HANDSHAKE_ERROR_BYTE;
+      } else if (frame[0] != 0x00) {
+        HELPER_LOG("Bad handshake error byte: %u", frame[0]);
+        send_explicit_handshake_reject_("Bad handshake error byte");
+        return APIError::BAD_HANDSHAKE_ERROR_BYTE;
+      }
+
+      NoiseBuffer mbuf;
+      noise_buffer_init(mbuf);
+      noise_buffer_set_input(mbuf, frame.data() + 1, frame.size() - 1);
+      err = noise_handshakestate_read_message(handshake_, &mbuf, nullptr);
+      if (err != 0) {
+        // Special handling for MAC failure
+        send_explicit_handshake_reject_(err == NOISE_ERROR_MAC_FAILURE ? "Handshake MAC failure" : "Handshake error");
+        return handle_noise_error_(err, "noise_handshakestate_read_message", APIError::HANDSHAKESTATE_READ_FAILED);
+      }
+
+      aerr = check_handshake_finished_();
+      if (aerr != APIError::OK)
+        return aerr;
+    } else if (action == NOISE_ACTION_WRITE_MESSAGE) {
+      uint8_t buffer[65];
+      NoiseBuffer mbuf;
+      noise_buffer_init(mbuf);
+      noise_buffer_set_output(mbuf, buffer + 1, sizeof(buffer) - 1);
+
+      err = noise_handshakestate_write_message(handshake_, &mbuf, nullptr);
+      APIError aerr_write =
+          handle_noise_error_(err, "noise_handshakestate_write_message", APIError::HANDSHAKESTATE_WRITE_FAILED);
+      if (aerr_write != APIError::OK)
+        return aerr_write;
+      buffer[0] = 0x00;  // success
+
+      aerr = write_frame_(buffer, mbuf.size + 1);
+      if (aerr != APIError::OK)
+        return aerr;
+      aerr = check_handshake_finished_();
+      if (aerr != APIError::OK)
+        return aerr;
+    } else {
+      // bad state for action
+      state_ = State::FAILED;
+      HELPER_LOG("Bad action for handshake: %d", action);
+      return APIError::HANDSHAKESTATE_BAD_STATE;
+    }
+  }
+  if (state_ == State::CLOSED || state_ == State::FAILED) {
+    return APIError::BAD_STATE;
+  }
+  return APIError::OK;
+}
+void APINoiseFrameHelper::send_explicit_handshake_reject_(const std::string &reason) {
+  std::vector<uint8_t> data;
+  data.resize(reason.length() + 1);
+  data[0] = 0x01;  // failure
+
+  // Copy error message in bulk
+  if (!reason.empty()) {
+    std::memcpy(data.data() + 1, reason.c_str(), reason.length());
+  }
+
+  // temporarily remove failed state
+  auto orig_state = state_;
+  state_ = State::EXPLICIT_REJECT;
+  write_frame_(data.data(), data.size());
+  state_ = orig_state;
+}
+APIError APINoiseFrameHelper::read_packet(ReadPacketBuffer *buffer) {
+  int err;
+  APIError aerr;
+  aerr = state_action_();
+  if (aerr != APIError::OK) {
+    return aerr;
+  }
+
+  if (state_ != State::DATA) {
+    return APIError::WOULD_BLOCK;
+  }
+
+  std::vector<uint8_t> frame;
+  aerr = try_read_frame_(&frame);
+  if (aerr != APIError::OK)
+    return aerr;
+
+  NoiseBuffer mbuf;
+  noise_buffer_init(mbuf);
+  noise_buffer_set_inout(mbuf, frame.data(), frame.size(), frame.size());
+  err = noise_cipherstate_decrypt(recv_cipher_, &mbuf);
+  APIError decrypt_err = handle_noise_error_(err, "noise_cipherstate_decrypt", APIError::CIPHERSTATE_DECRYPT_FAILED);
+  if (decrypt_err != APIError::OK)
+    return decrypt_err;
+
+  uint16_t msg_size = mbuf.size;
+  uint8_t *msg_data = frame.data();
+  if (msg_size < 4) {
+    state_ = State::FAILED;
+    HELPER_LOG("Bad data packet: size %d too short", msg_size);
+    return APIError::BAD_DATA_PACKET;
+  }
+
+  uint16_t type = (((uint16_t) msg_data[0]) << 8) | msg_data[1];
+  uint16_t data_len = (((uint16_t) msg_data[2]) << 8) | msg_data[3];
+  if (data_len > msg_size - 4) {
+    state_ = State::FAILED;
+    HELPER_LOG("Bad data packet: data_len %u greater than msg_size %u", data_len, msg_size);
+    return APIError::BAD_DATA_PACKET;
+  }
+
+  buffer->container = std::move(frame);
+  buffer->data_offset = 4;
+  buffer->data_len = data_len;
+  buffer->type = type;
+  return APIError::OK;
+}
+APIError APINoiseFrameHelper::write_protobuf_packet(uint8_t type, ProtoWriteBuffer buffer) {
+  // Resize to include MAC space (required for Noise encryption)
+  buffer.get_buffer()->resize(buffer.get_buffer()->size() + frame_footer_size_);
+  PacketInfo packet{type, 0,
+                    static_cast<uint16_t>(buffer.get_buffer()->size() - frame_header_padding_ - frame_footer_size_)};
+  return write_protobuf_packets(buffer, std::span<const PacketInfo>(&packet, 1));
+}
+
+APIError APINoiseFrameHelper::write_protobuf_packets(ProtoWriteBuffer buffer, std::span<const PacketInfo> packets) {
+  APIError aerr = state_action_();
+  if (aerr != APIError::OK) {
+    return aerr;
+  }
+
+  if (state_ != State::DATA) {
+    return APIError::WOULD_BLOCK;
+  }
+
+  if (packets.empty()) {
+    return APIError::OK;
+  }
+
+  std::vector<uint8_t> *raw_buffer = buffer.get_buffer();
+  uint8_t *buffer_data = raw_buffer->data();  // Cache buffer pointer
+
+  this->reusable_iovs_.clear();
+  this->reusable_iovs_.reserve(packets.size());
+  uint16_t total_write_len = 0;
+
+  // We need to encrypt each packet in place
+  for (const auto &packet : packets) {
+    // The buffer already has padding at offset
+    uint8_t *buf_start = buffer_data + packet.offset;
+
+    // Write noise header
+    buf_start[0] = 0x01;  // indicator
+    // buf_start[1], buf_start[2] to be set after encryption
+
+    // Write message header (to be encrypted)
+    const uint8_t msg_offset = 3;
+    buf_start[msg_offset] = static_cast<uint8_t>(packet.message_type >> 8);      // type high byte
+    buf_start[msg_offset + 1] = static_cast<uint8_t>(packet.message_type);       // type low byte
+    buf_start[msg_offset + 2] = static_cast<uint8_t>(packet.payload_size >> 8);  // data_len high byte
+    buf_start[msg_offset + 3] = static_cast<uint8_t>(packet.payload_size);       // data_len low byte
+    // payload data is already in the buffer starting at offset + 7
+
+    // Make sure we have space for MAC
+    // The buffer should already have been sized appropriately
+
+    // Encrypt the message in place
+    NoiseBuffer mbuf;
+    noise_buffer_init(mbuf);
+    noise_buffer_set_inout(mbuf, buf_start + msg_offset, 4 + packet.payload_size,
+                           4 + packet.payload_size + frame_footer_size_);
+
+    int err = noise_cipherstate_encrypt(send_cipher_, &mbuf);
+    APIError aerr = handle_noise_error_(err, "noise_cipherstate_encrypt", APIError::CIPHERSTATE_ENCRYPT_FAILED);
+    if (aerr != APIError::OK)
+      return aerr;
+
+    // Fill in the encrypted size
+    buf_start[1] = static_cast<uint8_t>(mbuf.size >> 8);
+    buf_start[2] = static_cast<uint8_t>(mbuf.size);
+
+    // Add iovec for this encrypted packet
+    size_t packet_len = static_cast<size_t>(3 + mbuf.size);  // indicator + size + encrypted data
+    this->reusable_iovs_.push_back({buf_start, packet_len});
+    total_write_len += packet_len;
+  }
+
+  // Send all encrypted packets in one writev call
+  return this->write_raw_(this->reusable_iovs_.data(), this->reusable_iovs_.size(), total_write_len);
+}
+
+APIError APINoiseFrameHelper::write_frame_(const uint8_t *data, uint16_t len) {
+  uint8_t header[3];
+  header[0] = 0x01;  // indicator
+  header[1] = (uint8_t) (len >> 8);
+  header[2] = (uint8_t) len;
+
+  struct iovec iov[2];
+  iov[0].iov_base = header;
+  iov[0].iov_len = 3;
+  if (len == 0) {
+    return this->write_raw_(iov, 1, 3);  // Just header
+  }
+  iov[1].iov_base = const_cast<uint8_t *>(data);
+  iov[1].iov_len = len;
+
+  return this->write_raw_(iov, 2, 3 + len);  // Header + data
+}
+
+/** Initiate the data structures for the handshake.
+ *
+ * @return 0 on success, -1 on error (check errno)
+ */
+APIError APINoiseFrameHelper::init_handshake_() {
+  int err;
+  memset(&nid_, 0, sizeof(nid_));
+  // const char *proto = "Noise_NNpsk0_25519_ChaChaPoly_SHA256";
+  // err = noise_protocol_name_to_id(&nid_, proto, strlen(proto));
+  nid_.pattern_id = NOISE_PATTERN_NN;
+  nid_.cipher_id = NOISE_CIPHER_CHACHAPOLY;
+  nid_.dh_id = NOISE_DH_CURVE25519;
+  nid_.prefix_id = NOISE_PREFIX_STANDARD;
+  nid_.hybrid_id = NOISE_DH_NONE;
+  nid_.hash_id = NOISE_HASH_SHA256;
+  nid_.modifier_ids[0] = NOISE_MODIFIER_PSK0;
+
+  err = noise_handshakestate_new_by_id(&handshake_, &nid_, NOISE_ROLE_RESPONDER);
+  APIError aerr = handle_noise_error_(err, "noise_handshakestate_new_by_id", APIError::HANDSHAKESTATE_SETUP_FAILED);
+  if (aerr != APIError::OK)
+    return aerr;
+
+  const auto &psk = ctx_->get_psk();
+  err = noise_handshakestate_set_pre_shared_key(handshake_, psk.data(), psk.size());
+  aerr = handle_noise_error_(err, "noise_handshakestate_set_pre_shared_key", APIError::HANDSHAKESTATE_SETUP_FAILED);
+  if (aerr != APIError::OK)
+    return aerr;
+
+  err = noise_handshakestate_set_prologue(handshake_, prologue_.data(), prologue_.size());
+  aerr = handle_noise_error_(err, "noise_handshakestate_set_prologue", APIError::HANDSHAKESTATE_SETUP_FAILED);
+  if (aerr != APIError::OK)
+    return aerr;
+  // set_prologue copies it into handshakestate, so we can get rid of it now
+  prologue_ = {};
+
+  err = noise_handshakestate_start(handshake_);
+  aerr = handle_noise_error_(err, "noise_handshakestate_start", APIError::HANDSHAKESTATE_SETUP_FAILED);
+  if (aerr != APIError::OK)
+    return aerr;
+  return APIError::OK;
+}
+
+APIError APINoiseFrameHelper::check_handshake_finished_() {
+  assert(state_ == State::HANDSHAKE);
+
+  int action = noise_handshakestate_get_action(handshake_);
+  if (action == NOISE_ACTION_READ_MESSAGE || action == NOISE_ACTION_WRITE_MESSAGE)
+    return APIError::OK;
+  if (action != NOISE_ACTION_SPLIT) {
+    state_ = State::FAILED;
+    HELPER_LOG("Bad action for handshake: %d", action);
+    return APIError::HANDSHAKESTATE_BAD_STATE;
+  }
+  int err = noise_handshakestate_split(handshake_, &send_cipher_, &recv_cipher_);
+  APIError aerr = handle_noise_error_(err, "noise_handshakestate_split", APIError::HANDSHAKESTATE_SPLIT_FAILED);
+  if (aerr != APIError::OK)
+    return aerr;
+
+  frame_footer_size_ = noise_cipherstate_get_mac_length(send_cipher_);
+
+  HELPER_LOG("Handshake complete!");
+  noise_handshakestate_free(handshake_);
+  handshake_ = nullptr;
+  state_ = State::DATA;
+  return APIError::OK;
+}
+
+APINoiseFrameHelper::~APINoiseFrameHelper() {
+  if (handshake_ != nullptr) {
+    noise_handshakestate_free(handshake_);
+    handshake_ = nullptr;
+  }
+  if (send_cipher_ != nullptr) {
+    noise_cipherstate_free(send_cipher_);
+    send_cipher_ = nullptr;
+  }
+  if (recv_cipher_ != nullptr) {
+    noise_cipherstate_free(recv_cipher_);
+    recv_cipher_ = nullptr;
+  }
+}
+
+extern "C" {
+// declare how noise generates random bytes (here with a good HWRNG based on the RF system)
+void noise_rand_bytes(void *output, size_t len) {
+  if (!esphome::random_bytes(reinterpret_cast<uint8_t *>(output), len)) {
+    ESP_LOGE(TAG, "Acquiring random bytes failed; rebooting");
+    arch_restart();
+  }
+}
+}
+
+}  // namespace esphome::api
+#endif  // USE_API_NOISE
+#endif  // USE_API

esphome/components/api/api_frame_helper_noise.h

@@ -0,0 +1,68 @@
+#pragma once
+#include "api_frame_helper.h"
+#ifdef USE_API
+#ifdef USE_API_NOISE
+#include "noise/protocol.h"
+#include "api_noise_context.h"
+
+namespace esphome::api {
+
+class APINoiseFrameHelper : public APIFrameHelper {
+ public:
+  APINoiseFrameHelper(std::unique_ptr<socket::Socket> socket, std::shared_ptr<APINoiseContext> ctx,
+                      const ClientInfo *client_info)
+      : APIFrameHelper(std::move(socket), client_info), ctx_(std::move(ctx)) {
+    // Noise header structure:
+    // Pos 0: indicator (0x01)
+    // Pos 1-2: encrypted payload size (16-bit big-endian)
+    // Pos 3-6: encrypted type (16-bit) + data_len (16-bit)
+    // Pos 7+: actual payload data
+    frame_header_padding_ = 7;
+  }
+  ~APINoiseFrameHelper() override;
+  APIError init() override;
+  APIError loop() override;
+  APIError read_packet(ReadPacketBuffer *buffer) override;
+  APIError write_protobuf_packet(uint8_t type, ProtoWriteBuffer buffer) override;
+  APIError write_protobuf_packets(ProtoWriteBuffer buffer, std::span<const PacketInfo> packets) override;
+  // Get the frame header padding required by this protocol
+  uint8_t frame_header_padding() override { return frame_header_padding_; }
+  // Get the frame footer size required by this protocol
+  uint8_t frame_footer_size() override { return frame_footer_size_; }
+
+ protected:
+  APIError state_action_();
+  APIError try_read_frame_(std::vector<uint8_t> *frame);
+  APIError write_frame_(const uint8_t *data, uint16_t len);
+  APIError init_handshake_();
+  APIError check_handshake_finished_();
+  void send_explicit_handshake_reject_(const std::string &reason);
+  APIError handle_handshake_frame_error_(APIError aerr);
+  APIError handle_noise_error_(int err, const char *func_name, APIError api_err);
+
+  // Pointers first (4 bytes each)
+  NoiseHandshakeState *handshake_{nullptr};
+  NoiseCipherState *send_cipher_{nullptr};
+  NoiseCipherState *recv_cipher_{nullptr};
+
+  // Shared pointer (8 bytes on 32-bit = 4 bytes control block pointer + 4 bytes object pointer)
+  std::shared_ptr<APINoiseContext> ctx_;
+
+  // Vector (12 bytes on 32-bit)
+  std::vector<uint8_t> prologue_;
+
+  // NoiseProtocolId (size depends on implementation)
+  NoiseProtocolId nid_;
+
+  // Group small types together
+  // Fixed-size header buffer for noise protocol:
+  // 1 byte for indicator + 2 bytes for message size (16-bit value, not varint)
+  // Note: Maximum message size is UINT16_MAX (65535), with a limit of 128 bytes during handshake phase
+  uint8_t rx_header_buf_[3];
+  uint8_t rx_header_buf_len_ = 0;
+  // 4 bytes total, no padding
+};
+
+}  // namespace esphome::api
+#endif  // USE_API_NOISE
+#endif  // USE_API

esphome/components/api/api_frame_helper_plaintext.cpp

@@ -0,0 +1,290 @@
+#include "api_frame_helper_plaintext.h"
+#ifdef USE_API
+#ifdef USE_API_PLAINTEXT
+#include "api_connection.h"  // For ClientInfo struct
+#include "esphome/core/application.h"
+#include "esphome/core/hal.h"
+#include "esphome/core/helpers.h"
+#include "esphome/core/log.h"
+#include "proto.h"
+#include <cstring>
+#include <cinttypes>
+
+namespace esphome::api {
+
+static const char *const TAG = "api.plaintext";
+
+#define HELPER_LOG(msg, ...) ESP_LOGVV(TAG, "%s: " msg, this->client_info_->get_combined_info().c_str(), ##__VA_ARGS__)
+
+#ifdef HELPER_LOG_PACKETS
+#define LOG_PACKET_RECEIVED(buffer) ESP_LOGVV(TAG, "Received frame: %s", format_hex_pretty(buffer).c_str())
+#define LOG_PACKET_SENDING(data, len) ESP_LOGVV(TAG, "Sending raw: %s", format_hex_pretty(data, len).c_str())
+#else
+#define LOG_PACKET_RECEIVED(buffer) ((void) 0)
+#define LOG_PACKET_SENDING(data, len) ((void) 0)
+#endif
+
+/// Initialize the frame helper, returns OK if successful.
+APIError APIPlaintextFrameHelper::init() {
+  APIError err = init_common_();
+  if (err != APIError::OK) {
+    return err;
+  }
+
+  state_ = State::DATA;
+  return APIError::OK;
+}
+APIError APIPlaintextFrameHelper::loop() {
+  if (state_ != State::DATA) {
+    return APIError::BAD_STATE;
+  }
+  // Use base class implementation for buffer sending
+  return APIFrameHelper::loop();
+}
+
+/** Read a packet into the rx_buf_. If successful, stores frame data in the frame parameter
+ *
+ * @param frame: The struct to hold the frame information in.
+ *   msg: store the parsed frame in that struct
+ *
+ * @return See APIError
+ *
+ * error API_ERROR_BAD_INDICATOR: Bad indicator byte at start of frame.
+ */
+APIError APIPlaintextFrameHelper::try_read_frame_(std::vector<uint8_t> *frame) {
+  if (frame == nullptr) {
+    HELPER_LOG("Bad argument for try_read_frame_");
+    return APIError::BAD_ARG;
+  }
+
+  // read header
+  while (!rx_header_parsed_) {
+    // Now that we know when the socket is ready, we can read up to 3 bytes
+    // into the rx_header_buf_ before we have to switch back to reading
+    // one byte at a time to ensure we don't read past the message and
+    // into the next one.
+
+    // Read directly into rx_header_buf_ at the current position
+    // Try to get to at least 3 bytes total (indicator + 2 varint bytes), then read one byte at a time
+    ssize_t received =
+        this->socket_->read(&rx_header_buf_[rx_header_buf_pos_], rx_header_buf_pos_ < 3 ? 3 - rx_header_buf_pos_ : 1);
+    APIError err = handle_socket_read_result_(received);
+    if (err != APIError::OK) {
+      return err;
+    }
+
+    // If this was the first read, validate the indicator byte
+    if (rx_header_buf_pos_ == 0 && received > 0) {
+      if (rx_header_buf_[0] != 0x00) {
+        state_ = State::FAILED;
+        HELPER_LOG("Bad indicator byte %u", rx_header_buf_[0]);
+        return APIError::BAD_INDICATOR;
+      }
+    }
+
+    rx_header_buf_pos_ += received;
+
+    // Check for buffer overflow
+    if (rx_header_buf_pos_ >= sizeof(rx_header_buf_)) {
+      state_ = State::FAILED;
+      HELPER_LOG("Header buffer overflow");
+      return APIError::BAD_DATA_PACKET;
+    }
+
+    // Need at least 3 bytes total (indicator + 2 varint bytes) before trying to parse
+    if (rx_header_buf_pos_ < 3) {
+      continue;
+    }
+
+    // At this point, we have at least 3 bytes total:
+    //   - Validated indicator byte (0x00) stored at position 0
+    //   - At least 2 bytes in the buffer for the varints
+    // Buffer layout:
+    //   [0]: indicator byte (0x00)
+    //   [1-3]: Message size varint (variable length)
+    //     - 2 bytes would only allow up to 16383, which is less than noise's UINT16_MAX (65535)
+    //     - 3 bytes allows up to 2097151, ensuring we support at least as much as noise
+    //   [2-5]: Message type varint (variable length)
+    // We now attempt to parse both varints. If either is incomplete,
+    // we'll continue reading more bytes.
+
+    // Skip indicator byte at position 0
+    uint8_t varint_pos = 1;
+    uint32_t consumed = 0;
+
+    auto msg_size_varint = ProtoVarInt::parse(&rx_header_buf_[varint_pos], rx_header_buf_pos_ - varint_pos, &consumed);
+    if (!msg_size_varint.has_value()) {
+      // not enough data there yet
+      continue;
+    }
+
+    if (msg_size_varint->as_uint32() > std::numeric_limits<uint16_t>::max()) {
+      state_ = State::FAILED;
+      HELPER_LOG("Bad packet: message size %" PRIu32 " exceeds maximum %u", msg_size_varint->as_uint32(),
+                 std::numeric_limits<uint16_t>::max());
+      return APIError::BAD_DATA_PACKET;
+    }
+    rx_header_parsed_len_ = msg_size_varint->as_uint16();
+
+    // Move to next varint position
+    varint_pos += consumed;
+
+    auto msg_type_varint = ProtoVarInt::parse(&rx_header_buf_[varint_pos], rx_header_buf_pos_ - varint_pos, &consumed);
+    if (!msg_type_varint.has_value()) {
+      // not enough data there yet
+      continue;
+    }
+    if (msg_type_varint->as_uint32() > std::numeric_limits<uint16_t>::max()) {
+      state_ = State::FAILED;
+      HELPER_LOG("Bad packet: message type %" PRIu32 " exceeds maximum %u", msg_type_varint->as_uint32(),
+                 std::numeric_limits<uint16_t>::max());
+      return APIError::BAD_DATA_PACKET;
+    }
+    rx_header_parsed_type_ = msg_type_varint->as_uint16();
+    rx_header_parsed_ = true;
+  }
+  // header reading done
+
+  // reserve space for body
+  if (rx_buf_.size() != rx_header_parsed_len_) {
+    rx_buf_.resize(rx_header_parsed_len_);
+  }
+
+  if (rx_buf_len_ < rx_header_parsed_len_) {
+    // more data to read
+    uint16_t to_read = rx_header_parsed_len_ - rx_buf_len_;
+    ssize_t received = this->socket_->read(&rx_buf_[rx_buf_len_], to_read);
+    APIError err = handle_socket_read_result_(received);
+    if (err != APIError::OK) {
+      return err;
+    }
+    rx_buf_len_ += static_cast<uint16_t>(received);
+    if (static_cast<uint16_t>(received) != to_read) {
+      // not all read
+      return APIError::WOULD_BLOCK;
+    }
+  }
+
+  LOG_PACKET_RECEIVED(rx_buf_);
+  *frame = std::move(rx_buf_);
+  // consume msg
+  rx_buf_ = {};
+  rx_buf_len_ = 0;
+  rx_header_buf_pos_ = 0;
+  rx_header_parsed_ = false;
+  return APIError::OK;
+}
+APIError APIPlaintextFrameHelper::read_packet(ReadPacketBuffer *buffer) {
+  APIError aerr;
+
+  if (state_ != State::DATA) {
+    return APIError::WOULD_BLOCK;
+  }
+
+  std::vector<uint8_t> frame;
+  aerr = try_read_frame_(&frame);
+  if (aerr != APIError::OK) {
+    if (aerr == APIError::BAD_INDICATOR) {
+      // Make sure to tell the remote that we don't
+      // understand the indicator byte so it knows
+      // we do not support it.
+      struct iovec iov[1];
+      // The \x00 first byte is the marker for plaintext.
+      //
+      // The remote will know how to handle the indicator byte,
+      // but it likely won't understand the rest of the message.
+      //
+      // We must send at least 3 bytes to be read, so we add
+      // a message after the indicator byte to ensures its long
+      // enough and can aid in debugging.
+      const char msg[] = "\x00"
+                         "Bad indicator byte";
+      iov[0].iov_base = (void *) msg;
+      iov[0].iov_len = 19;
+      this->write_raw_(iov, 1, 19);
+    }
+    return aerr;
+  }
+
+  buffer->container = std::move(frame);
+  buffer->data_offset = 0;
+  buffer->data_len = rx_header_parsed_len_;
+  buffer->type = rx_header_parsed_type_;
+  return APIError::OK;
+}
+APIError APIPlaintextFrameHelper::write_protobuf_packet(uint8_t type, ProtoWriteBuffer buffer) {
+  PacketInfo packet{type, 0, static_cast<uint16_t>(buffer.get_buffer()->size() - frame_header_padding_)};
+  return write_protobuf_packets(buffer, std::span<const PacketInfo>(&packet, 1));
+}
+
+APIError APIPlaintextFrameHelper::write_protobuf_packets(ProtoWriteBuffer buffer, std::span<const PacketInfo> packets) {
+  if (state_ != State::DATA) {
+    return APIError::BAD_STATE;
+  }
+
+  if (packets.empty()) {
+    return APIError::OK;
+  }
+
+  std::vector<uint8_t> *raw_buffer = buffer.get_buffer();
+  uint8_t *buffer_data = raw_buffer->data();  // Cache buffer pointer
+
+  this->reusable_iovs_.clear();
+  this->reusable_iovs_.reserve(packets.size());
+  uint16_t total_write_len = 0;
+
+  for (const auto &packet : packets) {
+    // Calculate varint sizes for header layout
+    uint8_t size_varint_len = api::ProtoSize::varint(static_cast<uint32_t>(packet.payload_size));
+    uint8_t type_varint_len = api::ProtoSize::varint(static_cast<uint32_t>(packet.message_type));
+    uint8_t total_header_len = 1 + size_varint_len + type_varint_len;
+
+    // Calculate where to start writing the header
+    // The header starts at the latest possible position to minimize unused padding
+    //
+    // Example 1 (small values): total_header_len = 3, header_offset = 6 - 3 = 3
+    // [0-2]  - Unused padding
+    // [3]    - 0x00 indicator byte
+    // [4]    - Payload size varint (1 byte, for sizes 0-127)
+    // [5]    - Message type varint (1 byte, for types 0-127)
+    // [6...] - Actual payload data
+    //
+    // Example 2 (medium values): total_header_len = 4, header_offset = 6 - 4 = 2
+    // [0-1]  - Unused padding
+    // [2]    - 0x00 indicator byte
+    // [3-4]  - Payload size varint (2 bytes, for sizes 128-16383)
+    // [5]    - Message type varint (1 byte, for types 0-127)
+    // [6...] - Actual payload data
+    //
+    // Example 3 (large values): total_header_len = 6, header_offset = 6 - 6 = 0
+    // [0]    - 0x00 indicator byte
+    // [1-3]  - Payload size varint (3 bytes, for sizes 16384-2097151)
+    // [4-5]  - Message type varint (2 bytes, for types 128-32767)
+    // [6...] - Actual payload data
+    //
+    // The message starts at offset + frame_header_padding_
+    // So we write the header starting at offset + frame_header_padding_ - total_header_len
+    uint8_t *buf_start = buffer_data + packet.offset;
+    uint32_t header_offset = frame_header_padding_ - total_header_len;
+
+    // Write the plaintext header
+    buf_start[header_offset] = 0x00;  // indicator
+
+    // Encode varints directly into buffer
+    ProtoVarInt(packet.payload_size).encode_to_buffer_unchecked(buf_start + header_offset + 1, size_varint_len);
+    ProtoVarInt(packet.message_type)
+        .encode_to_buffer_unchecked(buf_start + header_offset + 1 + size_varint_len, type_varint_len);
+
+    // Add iovec for this packet (header + payload)
+    size_t packet_len = static_cast<size_t>(total_header_len + packet.payload_size);
+    this->reusable_iovs_.push_back({buf_start + header_offset, packet_len});
+    total_write_len += packet_len;
+  }
+
+  // Send all packets in one writev call
+  return write_raw_(this->reusable_iovs_.data(), this->reusable_iovs_.size(), total_write_len);
+}
+
+}  // namespace esphome::api
+#endif  // USE_API_PLAINTEXT
+#endif  // USE_API

esphome/components/api/api_frame_helper_plaintext.h

@@ -0,0 +1,53 @@
+#pragma once
+#include "api_frame_helper.h"
+#ifdef USE_API
+#ifdef USE_API_PLAINTEXT
+
+namespace esphome::api {
+
+class APIPlaintextFrameHelper : public APIFrameHelper {
+ public:
+  APIPlaintextFrameHelper(std::unique_ptr<socket::Socket> socket, const ClientInfo *client_info)
+      : APIFrameHelper(std::move(socket), client_info) {
+    // Plaintext header structure (worst case):
+    // Pos 0: indicator (0x00)
+    // Pos 1-3: payload size varint (up to 3 bytes)
+    // Pos 4-5: message type varint (up to 2 bytes)
+    // Pos 6+: actual payload data
+    frame_header_padding_ = 6;
+  }
+  ~APIPlaintextFrameHelper() override = default;
+  APIError init() override;
+  APIError loop() override;
+  APIError read_packet(ReadPacketBuffer *buffer) override;
+  APIError write_protobuf_packet(uint8_t type, ProtoWriteBuffer buffer) override;
+  APIError write_protobuf_packets(ProtoWriteBuffer buffer, std::span<const PacketInfo> packets) override;
+  uint8_t frame_header_padding() override { return frame_header_padding_; }
+  // Get the frame footer size required by this protocol
+  uint8_t frame_footer_size() override { return frame_footer_size_; }
+
+ protected:
+  APIError try_read_frame_(std::vector<uint8_t> *frame);
+
+  // Group 2-byte aligned types
+  uint16_t rx_header_parsed_type_ = 0;
+  uint16_t rx_header_parsed_len_ = 0;
+
+  // Group 1-byte types together
+  // Fixed-size header buffer for plaintext protocol:
+  // We now store the indicator byte + the two varints.
+  // To match noise protocol's maximum message size (UINT16_MAX = 65535), we need:
+  // 1 byte for indicator + 3 bytes for message size varint (supports up to 2097151) + 2 bytes for message type varint
+  //
+  // While varints could theoretically be up to 10 bytes each for 64-bit values,
+  // attempting to process messages with headers that large would likely crash the
+  // ESP32 due to memory constraints.
+  uint8_t rx_header_buf_[6];  // 1 byte indicator + 5 bytes for varints (3 for size + 2 for type)
+  uint8_t rx_header_buf_pos_ = 0;
+  bool rx_header_parsed_ = false;
+  // 8 bytes total, no padding needed
+};
+
+}  // namespace esphome::api
+#endif  // USE_API_PLAINTEXT
+#endif  // USE_API

esphome/components/api/api_noise_context.h

@@ -3,8 +3,7 @@
 #include <cstdint>
 #include "esphome/core/defines.h"
 
-namespace esphome {
-namespace api {
+namespace esphome::api {
 
 #ifdef USE_API_NOISE
 using psk_t = std::array<uint8_t, 32>;
@@ -28,5 +27,4 @@ class APINoiseContext {
 };
 #endif  // USE_API_NOISE
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api

esphome/components/api/api_options.proto

@@ -23,3 +23,8 @@ extend google.protobuf.MessageOptions {
     optional bool no_delay = 1040 [default=false];
     optional string base_class = 1041;
 }
+
+extend google.protobuf.FieldOptions {
+    optional string field_ifdef = 1042;
+    optional uint32 fixed_array_size = 50007;
+}

esphome/components/api/api_pb2_service.cpp

@@ -3,8 +3,7 @@
 #include "api_pb2_service.h"
 #include "esphome/core/log.h"
 
-namespace esphome {
-namespace api {
+namespace esphome::api {
 
 static const char *const TAG = "api.service";
 
@@ -16,7 +15,7 @@ void APIServerConnectionBase::log_send_message_(const char *name, const std::str
 
 void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, uint8_t *msg_data) {
   switch (msg_type) {
-    case 1: {
+    case HelloRequest::MESSAGE_TYPE: {
       HelloRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -25,7 +24,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
       this->on_hello_request(msg);
       break;
     }
-    case 3: {
+    case ConnectRequest::MESSAGE_TYPE: {
       ConnectRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -34,7 +33,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
       this->on_connect_request(msg);
       break;
     }
-    case 5: {
+    case DisconnectRequest::MESSAGE_TYPE: {
       DisconnectRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -43,7 +42,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
       this->on_disconnect_request(msg);
       break;
     }
-    case 6: {
+    case DisconnectResponse::MESSAGE_TYPE: {
       DisconnectResponse msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -52,7 +51,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
       this->on_disconnect_response(msg);
       break;
     }
-    case 7: {
+    case PingRequest::MESSAGE_TYPE: {
       PingRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -61,7 +60,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
       this->on_ping_request(msg);
       break;
     }
-    case 8: {
+    case PingResponse::MESSAGE_TYPE: {
       PingResponse msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -70,7 +69,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
       this->on_ping_response(msg);
       break;
     }
-    case 9: {
+    case DeviceInfoRequest::MESSAGE_TYPE: {
       DeviceInfoRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -79,7 +78,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
       this->on_device_info_request(msg);
       break;
     }
-    case 11: {
+    case ListEntitiesRequest::MESSAGE_TYPE: {
       ListEntitiesRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -88,7 +87,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
       this->on_list_entities_request(msg);
       break;
     }
-    case 20: {
+    case SubscribeStatesRequest::MESSAGE_TYPE: {
       SubscribeStatesRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -97,7 +96,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
       this->on_subscribe_states_request(msg);
       break;
     }
-    case 28: {
+    case SubscribeLogsRequest::MESSAGE_TYPE: {
       SubscribeLogsRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -107,7 +106,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
       break;
     }
 #ifdef USE_COVER
-    case 30: {
+    case CoverCommandRequest::MESSAGE_TYPE: {
       CoverCommandRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -118,7 +117,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_FAN
-    case 31: {
+    case FanCommandRequest::MESSAGE_TYPE: {
       FanCommandRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -129,7 +128,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_LIGHT
-    case 32: {
+    case LightCommandRequest::MESSAGE_TYPE: {
       LightCommandRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -140,7 +139,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_SWITCH
-    case 33: {
+    case SwitchCommandRequest::MESSAGE_TYPE: {
       SwitchCommandRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -150,7 +149,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
       break;
     }
 #endif
-    case 34: {
+    case SubscribeHomeassistantServicesRequest::MESSAGE_TYPE: {
       SubscribeHomeassistantServicesRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -159,7 +158,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
       this->on_subscribe_homeassistant_services_request(msg);
       break;
     }
-    case 36: {
+    case GetTimeRequest::MESSAGE_TYPE: {
       GetTimeRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -168,7 +167,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
       this->on_get_time_request(msg);
       break;
     }
-    case 37: {
+    case GetTimeResponse::MESSAGE_TYPE: {
       GetTimeResponse msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -177,7 +176,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
       this->on_get_time_response(msg);
       break;
     }
-    case 38: {
+    case SubscribeHomeAssistantStatesRequest::MESSAGE_TYPE: {
       SubscribeHomeAssistantStatesRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -186,7 +185,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
       this->on_subscribe_home_assistant_states_request(msg);
       break;
     }
-    case 40: {
+    case HomeAssistantStateResponse::MESSAGE_TYPE: {
       HomeAssistantStateResponse msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -196,7 +195,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
       break;
     }
 #ifdef USE_API_SERVICES
-    case 42: {
+    case ExecuteServiceRequest::MESSAGE_TYPE: {
       ExecuteServiceRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -207,7 +206,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_CAMERA
-    case 45: {
+    case CameraImageRequest::MESSAGE_TYPE: {
       CameraImageRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -218,7 +217,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_CLIMATE
-    case 48: {
+    case ClimateCommandRequest::MESSAGE_TYPE: {
       ClimateCommandRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -229,7 +228,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_NUMBER
-    case 51: {
+    case NumberCommandRequest::MESSAGE_TYPE: {
       NumberCommandRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -240,7 +239,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_SELECT
-    case 54: {
+    case SelectCommandRequest::MESSAGE_TYPE: {
       SelectCommandRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -251,7 +250,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_SIREN
-    case 57: {
+    case SirenCommandRequest::MESSAGE_TYPE: {
       SirenCommandRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -262,7 +261,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_LOCK
-    case 60: {
+    case LockCommandRequest::MESSAGE_TYPE: {
       LockCommandRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -273,7 +272,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_BUTTON
-    case 62: {
+    case ButtonCommandRequest::MESSAGE_TYPE: {
       ButtonCommandRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -284,7 +283,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_MEDIA_PLAYER
-    case 65: {
+    case MediaPlayerCommandRequest::MESSAGE_TYPE: {
       MediaPlayerCommandRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -295,7 +294,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_BLUETOOTH_PROXY
-    case 66: {
+    case SubscribeBluetoothLEAdvertisementsRequest::MESSAGE_TYPE: {
       SubscribeBluetoothLEAdvertisementsRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -306,7 +305,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_BLUETOOTH_PROXY
-    case 68: {
+    case BluetoothDeviceRequest::MESSAGE_TYPE: {
       BluetoothDeviceRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -317,7 +316,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_BLUETOOTH_PROXY
-    case 70: {
+    case BluetoothGATTGetServicesRequest::MESSAGE_TYPE: {
       BluetoothGATTGetServicesRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -328,7 +327,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_BLUETOOTH_PROXY
-    case 73: {
+    case BluetoothGATTReadRequest::MESSAGE_TYPE: {
       BluetoothGATTReadRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -339,7 +338,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_BLUETOOTH_PROXY
-    case 75: {
+    case BluetoothGATTWriteRequest::MESSAGE_TYPE: {
       BluetoothGATTWriteRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -350,7 +349,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_BLUETOOTH_PROXY
-    case 76: {
+    case BluetoothGATTReadDescriptorRequest::MESSAGE_TYPE: {
       BluetoothGATTReadDescriptorRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -361,7 +360,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_BLUETOOTH_PROXY
-    case 77: {
+    case BluetoothGATTWriteDescriptorRequest::MESSAGE_TYPE: {
       BluetoothGATTWriteDescriptorRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -372,7 +371,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_BLUETOOTH_PROXY
-    case 78: {
+    case BluetoothGATTNotifyRequest::MESSAGE_TYPE: {
       BluetoothGATTNotifyRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -383,7 +382,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_BLUETOOTH_PROXY
-    case 80: {
+    case SubscribeBluetoothConnectionsFreeRequest::MESSAGE_TYPE: {
       SubscribeBluetoothConnectionsFreeRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -394,7 +393,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_BLUETOOTH_PROXY
-    case 87: {
+    case UnsubscribeBluetoothLEAdvertisementsRequest::MESSAGE_TYPE: {
       UnsubscribeBluetoothLEAdvertisementsRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -405,7 +404,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_VOICE_ASSISTANT
-    case 89: {
+    case SubscribeVoiceAssistantRequest::MESSAGE_TYPE: {
       SubscribeVoiceAssistantRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -416,7 +415,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_VOICE_ASSISTANT
-    case 91: {
+    case VoiceAssistantResponse::MESSAGE_TYPE: {
       VoiceAssistantResponse msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -427,7 +426,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_VOICE_ASSISTANT
-    case 92: {
+    case VoiceAssistantEventResponse::MESSAGE_TYPE: {
       VoiceAssistantEventResponse msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -438,7 +437,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_ALARM_CONTROL_PANEL
-    case 96: {
+    case AlarmControlPanelCommandRequest::MESSAGE_TYPE: {
       AlarmControlPanelCommandRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -449,7 +448,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_TEXT
-    case 99: {
+    case TextCommandRequest::MESSAGE_TYPE: {
       TextCommandRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -460,7 +459,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_DATETIME_DATE
-    case 102: {
+    case DateCommandRequest::MESSAGE_TYPE: {
       DateCommandRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -471,7 +470,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_DATETIME_TIME
-    case 105: {
+    case TimeCommandRequest::MESSAGE_TYPE: {
       TimeCommandRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -482,7 +481,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_VOICE_ASSISTANT
-    case 106: {
+    case VoiceAssistantAudio::MESSAGE_TYPE: {
       VoiceAssistantAudio msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -493,7 +492,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_VALVE
-    case 111: {
+    case ValveCommandRequest::MESSAGE_TYPE: {
       ValveCommandRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -504,7 +503,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_DATETIME_DATETIME
-    case 114: {
+    case DateTimeCommandRequest::MESSAGE_TYPE: {
       DateTimeCommandRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -515,7 +514,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_VOICE_ASSISTANT
-    case 115: {
+    case VoiceAssistantTimerEventResponse::MESSAGE_TYPE: {
       VoiceAssistantTimerEventResponse msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -526,7 +525,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_UPDATE
-    case 118: {
+    case UpdateCommandRequest::MESSAGE_TYPE: {
       UpdateCommandRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -537,7 +536,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_VOICE_ASSISTANT
-    case 119: {
+    case VoiceAssistantAnnounceRequest::MESSAGE_TYPE: {
       VoiceAssistantAnnounceRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -548,7 +547,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_VOICE_ASSISTANT
-    case 121: {
+    case VoiceAssistantConfigurationRequest::MESSAGE_TYPE: {
       VoiceAssistantConfigurationRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -559,7 +558,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_VOICE_ASSISTANT
-    case 123: {
+    case VoiceAssistantSetConfiguration::MESSAGE_TYPE: {
       VoiceAssistantSetConfiguration msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -570,7 +569,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_API_NOISE
-    case 124: {
+    case NoiseEncryptionSetKeyRequest::MESSAGE_TYPE: {
       NoiseEncryptionSetKeyRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -581,7 +580,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
     }
 #endif
 #ifdef USE_BLUETOOTH_PROXY
-    case 127: {
+    case BluetoothScannerSetModeRequest::MESSAGE_TYPE: {
       BluetoothScannerSetModeRequest msg;
       msg.decode(msg_data, msg_size);
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -597,36 +596,29 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type,
 }
 
 void APIServerConnection::on_hello_request(const HelloRequest &msg) {
-  HelloResponse ret = this->hello(msg);
-  if (!this->send_message(ret)) {
+  if (!this->send_hello_response(msg)) {
     this->on_fatal_error();
   }
 }
 void APIServerConnection::on_connect_request(const ConnectRequest &msg) {
-  ConnectResponse ret = this->connect(msg);
-  if (!this->send_message(ret)) {
+  if (!this->send_connect_response(msg)) {
     this->on_fatal_error();
   }
 }
 void APIServerConnection::on_disconnect_request(const DisconnectRequest &msg) {
-  DisconnectResponse ret = this->disconnect(msg);
-  if (!this->send_message(ret)) {
+  if (!this->send_disconnect_response(msg)) {
     this->on_fatal_error();
   }
 }
 void APIServerConnection::on_ping_request(const PingRequest &msg) {
-  PingResponse ret = this->ping(msg);
-  if (!this->send_message(ret)) {
+  if (!this->send_ping_response(msg)) {
     this->on_fatal_error();
   }
 }
 void APIServerConnection::on_device_info_request(const DeviceInfoRequest &msg) {
-  if (this->check_connection_setup_()) {
-    DeviceInfoResponse ret = this->device_info(msg);
-    if (!this->send_message(ret)) {
+  if (this->check_connection_setup_() && !this->send_device_info_response(msg)) {
     this->on_fatal_error();
   }
-  }
 }
 void APIServerConnection::on_list_entities_request(const ListEntitiesRequest &msg) {
   if (this->check_authenticated_()) {
@@ -655,12 +647,9 @@ void APIServerConnection::on_subscribe_home_assistant_states_request(const Subsc
   }
 }
 void APIServerConnection::on_get_time_request(const GetTimeRequest &msg) {
-  if (this->check_connection_setup_()) {
-    GetTimeResponse ret = this->get_time(msg);
-    if (!this->send_message(ret)) {
+  if (this->check_connection_setup_() && !this->send_get_time_response(msg)) {
     this->on_fatal_error();
   }
-  }
 }
 #ifdef USE_API_SERVICES
 void APIServerConnection::on_execute_service_request(const ExecuteServiceRequest &msg) {
@@ -671,12 +660,9 @@ void APIServerConnection::on_execute_service_request(const ExecuteServiceRequest
 #endif
 #ifdef USE_API_NOISE
 void APIServerConnection::on_noise_encryption_set_key_request(const NoiseEncryptionSetKeyRequest &msg) {
-  if (this->check_authenticated_()) {
-    NoiseEncryptionSetKeyResponse ret = this->noise_encryption_set_key(msg);
-    if (!this->send_message(ret)) {
+  if (this->check_authenticated_() && !this->send_noise_encryption_set_key_response(msg)) {
     this->on_fatal_error();
   }
-  }
 }
 #endif
 #ifdef USE_BUTTON
@@ -865,12 +851,9 @@ void APIServerConnection::on_bluetooth_gatt_notify_request(const BluetoothGATTNo
 #ifdef USE_BLUETOOTH_PROXY
 void APIServerConnection::on_subscribe_bluetooth_connections_free_request(
     const SubscribeBluetoothConnectionsFreeRequest &msg) {
-  if (this->check_authenticated_()) {
-    BluetoothConnectionsFreeResponse ret = this->subscribe_bluetooth_connections_free(msg);
-    if (!this->send_message(ret)) {
+  if (this->check_authenticated_() && !this->send_subscribe_bluetooth_connections_free_response(msg)) {
     this->on_fatal_error();
   }
-  }
 }
 #endif
 #ifdef USE_BLUETOOTH_PROXY
@@ -897,12 +880,9 @@ void APIServerConnection::on_subscribe_voice_assistant_request(const SubscribeVo
 #endif
 #ifdef USE_VOICE_ASSISTANT
 void APIServerConnection::on_voice_assistant_configuration_request(const VoiceAssistantConfigurationRequest &msg) {
-  if (this->check_authenticated_()) {
-    VoiceAssistantConfigurationResponse ret = this->voice_assistant_get_configuration(msg);
-    if (!this->send_message(ret)) {
+  if (this->check_authenticated_() && !this->send_voice_assistant_get_configuration_response(msg)) {
     this->on_fatal_error();
   }
-  }
 }
 #endif
 #ifdef USE_VOICE_ASSISTANT
@@ -920,5 +900,4 @@ void APIServerConnection::on_alarm_control_panel_command_request(const AlarmCont
 }
 #endif
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api

esphome/components/api/api_pb2_service.h

@@ -6,8 +6,7 @@
 
 #include "api_pb2.h"
 
-namespace esphome {
-namespace api {
+namespace esphome::api {
 
 class APIServerConnectionBase : public ProtoService {
  public:
@@ -18,11 +17,11 @@ class APIServerConnectionBase : public ProtoService {
  public:
 #endif
 
-  template<typename T> bool send_message(const T &msg) {
+  bool send_message(const ProtoMessage &msg, uint8_t message_type) {
 #ifdef HAS_PROTO_MESSAGE_DUMP
     this->log_send_message_(msg.message_name(), msg.dump());
 #endif
-    return this->send_message_(msg, T::MESSAGE_TYPE);
+    return this->send_message_(msg, message_type);
   }
 
   virtual void on_hello_request(const HelloRequest &value){};
@@ -207,22 +206,22 @@ class APIServerConnectionBase : public ProtoService {
 
 class APIServerConnection : public APIServerConnectionBase {
  public:
-  virtual HelloResponse hello(const HelloRequest &msg) = 0;
-  virtual ConnectResponse connect(const ConnectRequest &msg) = 0;
-  virtual DisconnectResponse disconnect(const DisconnectRequest &msg) = 0;
-  virtual PingResponse ping(const PingRequest &msg) = 0;
-  virtual DeviceInfoResponse device_info(const DeviceInfoRequest &msg) = 0;
+  virtual bool send_hello_response(const HelloRequest &msg) = 0;
+  virtual bool send_connect_response(const ConnectRequest &msg) = 0;
+  virtual bool send_disconnect_response(const DisconnectRequest &msg) = 0;
+  virtual bool send_ping_response(const PingRequest &msg) = 0;
+  virtual bool send_device_info_response(const DeviceInfoRequest &msg) = 0;
   virtual void list_entities(const ListEntitiesRequest &msg) = 0;
   virtual void subscribe_states(const SubscribeStatesRequest &msg) = 0;
   virtual void subscribe_logs(const SubscribeLogsRequest &msg) = 0;
   virtual void subscribe_homeassistant_services(const SubscribeHomeassistantServicesRequest &msg) = 0;
   virtual void subscribe_home_assistant_states(const SubscribeHomeAssistantStatesRequest &msg) = 0;
-  virtual GetTimeResponse get_time(const GetTimeRequest &msg) = 0;
+  virtual bool send_get_time_response(const GetTimeRequest &msg) = 0;
 #ifdef USE_API_SERVICES
   virtual void execute_service(const ExecuteServiceRequest &msg) = 0;
 #endif
 #ifdef USE_API_NOISE
-  virtual NoiseEncryptionSetKeyResponse noise_encryption_set_key(const NoiseEncryptionSetKeyRequest &msg) = 0;
+  virtual bool send_noise_encryption_set_key_response(const NoiseEncryptionSetKeyRequest &msg) = 0;
 #endif
 #ifdef USE_BUTTON
   virtual void button_command(const ButtonCommandRequest &msg) = 0;
@@ -303,7 +302,7 @@ class APIServerConnection : public APIServerConnectionBase {
   virtual void bluetooth_gatt_notify(const BluetoothGATTNotifyRequest &msg) = 0;
 #endif
 #ifdef USE_BLUETOOTH_PROXY
-  virtual BluetoothConnectionsFreeResponse subscribe_bluetooth_connections_free(
+  virtual bool send_subscribe_bluetooth_connections_free_response(
       const SubscribeBluetoothConnectionsFreeRequest &msg) = 0;
 #endif
 #ifdef USE_BLUETOOTH_PROXY
@@ -316,8 +315,7 @@ class APIServerConnection : public APIServerConnectionBase {
   virtual void subscribe_voice_assistant(const SubscribeVoiceAssistantRequest &msg) = 0;
 #endif
 #ifdef USE_VOICE_ASSISTANT
-  virtual VoiceAssistantConfigurationResponse voice_assistant_get_configuration(
-      const VoiceAssistantConfigurationRequest &msg) = 0;
+  virtual bool send_voice_assistant_get_configuration_response(const VoiceAssistantConfigurationRequest &msg) = 0;
 #endif
 #ifdef USE_VOICE_ASSISTANT
   virtual void voice_assistant_set_configuration(const VoiceAssistantSetConfiguration &msg) = 0;
@@ -445,5 +443,4 @@ class APIServerConnection : public APIServerConnectionBase {
 #endif
 };
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api

esphome/components/api/api_server.cpp

@@ -16,8 +16,7 @@
 
 #include <algorithm>
 
-namespace esphome {
-namespace api {
+namespace esphome::api {
 
 static const char *const TAG = "api";
 
@@ -31,7 +30,6 @@ APIServer::APIServer() {
 }
 
 void APIServer::setup() {
-  ESP_LOGCONFIG(TAG, "Running setup");
   this->setup_controller();
 
 #ifdef USE_API_NOISE
@@ -105,7 +103,7 @@ void APIServer::setup() {
             return;
           }
           for (auto &c : this->clients_) {
-            if (!c->flags_.remove)
+            if (!c->flags_.remove && c->get_log_subscription_level() >= level)
               c->try_send_log_message(level, tag, message, message_len);
           }
         });
@@ -185,9 +183,9 @@ void APIServer::loop() {
 
     // Rare case: handle disconnection
 #ifdef USE_API_CLIENT_DISCONNECTED_TRIGGER
-    this->client_disconnected_trigger_->trigger(client->client_info_, client->client_peername_);
+    this->client_disconnected_trigger_->trigger(client->client_info_.name, client->client_info_.peername);
 #endif
-    ESP_LOGV(TAG, "Remove connection %s", client->client_info_.c_str());
+    ESP_LOGV(TAG, "Remove connection %s", client->client_info_.name.c_str());
 
     // Swap with the last element and pop (avoids expensive vector shifts)
     if (client_index < this->clients_.size() - 1) {
@@ -205,22 +203,20 @@ void APIServer::loop() {
 
 void APIServer::dump_config() {
   ESP_LOGCONFIG(TAG,
-                "API Server:\n"
+                "Server:\n"
                 "  Address: %s:%u",
                 network::get_use_address().c_str(), this->port_);
 #ifdef USE_API_NOISE
-  ESP_LOGCONFIG(TAG, "  Using noise encryption: %s", YESNO(this->noise_ctx_->has_psk()));
+  ESP_LOGCONFIG(TAG, "  Noise encryption: %s", YESNO(this->noise_ctx_->has_psk()));
   if (!this->noise_ctx_->has_psk()) {
-    ESP_LOGCONFIG(TAG, "  Supports noise encryption: YES");
+    ESP_LOGCONFIG(TAG, "  Supports encryption: YES");
   }
 #else
-  ESP_LOGCONFIG(TAG, "  Using noise encryption: NO");
+  ESP_LOGCONFIG(TAG, "  Noise encryption: NO");
 #endif
 }
 
 #ifdef USE_API_PASSWORD
-bool APIServer::uses_password() const { return !this->password_.empty(); }
-
 bool APIServer::check_password(const std::string &password) const {
   // depend only on input password length
   const char *a = this->password_.c_str();
@@ -428,10 +424,11 @@ bool APIServer::save_noise_psk(psk_t psk, bool make_active) {
   ESP_LOGD(TAG, "Noise PSK saved");
   if (make_active) {
     this->set_timeout(100, [this, psk]() {
-      ESP_LOGW(TAG, "Disconnecting all clients to reset connections");
+      ESP_LOGW(TAG, "Disconnecting all clients to reset PSK");
       this->set_noise_psk(psk);
       for (auto &c : this->clients_) {
-        c->send_message(DisconnectRequest());
+        DisconnectRequest req;
+        c->send_message(req, DisconnectRequest::MESSAGE_TYPE);
       }
     });
   }
@@ -464,7 +461,8 @@ void APIServer::on_shutdown() {
 
   // Send disconnect requests to all connected clients
   for (auto &c : this->clients_) {
-    if (!c->send_message(DisconnectRequest())) {
+    DisconnectRequest req;
+    if (!c->send_message(req, DisconnectRequest::MESSAGE_TYPE)) {
       // If we can't send the disconnect request directly (tx_buffer full),
       // schedule it at the front of the batch so it will be sent with priority
       c->schedule_message_front_(nullptr, &APIConnection::try_send_disconnect_request, DisconnectRequest::MESSAGE_TYPE,
@@ -484,6 +482,5 @@ bool APIServer::teardown() {
   return this->clients_.empty();
 }
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api
 #endif

esphome/components/api/api_server.h

@@ -18,8 +18,7 @@
 
 #include <vector>
 
-namespace esphome {
-namespace api {
+namespace esphome::api {
 
 #ifdef USE_API_NOISE
 struct SavedNoisePsk {
@@ -39,7 +38,6 @@ class APIServer : public Component, public Controller {
   bool teardown() override;
 #ifdef USE_API_PASSWORD
   bool check_password(const std::string &password) const;
-  bool uses_password() const;
   void set_password(const std::string &password);
 #endif
   void set_port(uint16_t port);
@@ -197,6 +195,5 @@ template<typename... Ts> class APIConnectedCondition : public Condition<Ts...> {
   bool check(Ts... x) override { return global_api_server->is_connected(); }
 };
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api
 #endif

esphome/components/api/client.py

@@ -14,6 +14,8 @@ with warnings.catch_warnings():
     from aioesphomeapi import APIClient, parse_log_message
     from aioesphomeapi.log_runner import async_run
 
+import contextlib
+
 from esphome.const import CONF_KEY, CONF_PASSWORD, CONF_PORT, __version__
 from esphome.core import CORE
 
@@ -66,7 +68,5 @@ async def async_run_logs(config: dict[str, Any], address: str) -> None:
 
 def run_logs(config: dict[str, Any], address: str) -> None:
     """Run the logs command."""
-    try:
+    with contextlib.suppress(KeyboardInterrupt):
         asyncio.run(async_run_logs(config, address))
-    except KeyboardInterrupt:
-        pass

esphome/components/api/custom_api_device.h

@@ -6,8 +6,7 @@
 #ifdef USE_API_SERVICES
 #include "user_services.h"
 #endif
-namespace esphome {
-namespace api {
+namespace esphome::api {
 
 #ifdef USE_API_SERVICES
 template<typename T, typename... Ts> class CustomAPIDeviceService : public UserServiceBase<Ts...> {
@@ -148,7 +147,7 @@ class CustomAPIDevice {
    */
   void call_homeassistant_service(const std::string &service_name) {
     HomeassistantServiceResponse resp;
-    resp.service = service_name;
+    resp.set_service(StringRef(service_name));
     global_api_server->send_homeassistant_service_call(resp);
   }
 
@@ -168,12 +167,12 @@ class CustomAPIDevice {
    */
   void call_homeassistant_service(const std::string &service_name, const std::map<std::string, std::string> &data) {
     HomeassistantServiceResponse resp;
-    resp.service = service_name;
+    resp.set_service(StringRef(service_name));
     for (auto &it : data) {
-      HomeassistantServiceMap kv;
-      kv.key = it.first;
-      kv.value = it.second;
-      resp.data.push_back(kv);
+      resp.data.emplace_back();
+      auto &kv = resp.data.back();
+      kv.set_key(StringRef(it.first));
+      kv.set_value(StringRef(it.second));
     }
     global_api_server->send_homeassistant_service_call(resp);
   }
@@ -190,7 +189,7 @@ class CustomAPIDevice {
    */
   void fire_homeassistant_event(const std::string &event_name) {
     HomeassistantServiceResponse resp;
-    resp.service = event_name;
+    resp.set_service(StringRef(event_name));
     resp.is_event = true;
     global_api_server->send_homeassistant_service_call(resp);
   }
@@ -210,18 +209,17 @@ class CustomAPIDevice {
    */
   void fire_homeassistant_event(const std::string &service_name, const std::map<std::string, std::string> &data) {
     HomeassistantServiceResponse resp;
-    resp.service = service_name;
+    resp.set_service(StringRef(service_name));
     resp.is_event = true;
     for (auto &it : data) {
-      HomeassistantServiceMap kv;
-      kv.key = it.first;
-      kv.value = it.second;
-      resp.data.push_back(kv);
+      resp.data.emplace_back();
+      auto &kv = resp.data.back();
+      kv.set_key(StringRef(it.first));
+      kv.set_value(StringRef(it.second));
     }
     global_api_server->send_homeassistant_service_call(resp);
   }
 };
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api
 #endif

esphome/components/api/homeassistant_service.h

@@ -7,8 +7,7 @@
 #include "esphome/core/helpers.h"
 #include <vector>
 
-namespace esphome {
-namespace api {
+namespace esphome::api {
 
 template<typename... X> class TemplatableStringValue : public TemplatableValue<std::string, X...> {
  private:
@@ -36,6 +35,9 @@ template<typename... X> class TemplatableStringValue : public TemplatableValue<s
 
 template<typename... Ts> class TemplatableKeyValuePair {
  public:
+  // Keys are always string literals from YAML dictionary keys (e.g., "code", "event")
+  // and never templatable values or lambdas. Only the value parameter can be a lambda/template.
+  // Using pass-by-value with std::move allows optimal performance for both lvalues and rvalues.
   template<typename T> TemplatableKeyValuePair(std::string key, T value) : key(std::move(key)), value(value) {}
   std::string key;
   TemplatableStringValue<Ts...> value;
@@ -47,37 +49,42 @@ template<typename... Ts> class HomeAssistantServiceCallAction : public Action<Ts
 
   template<typename T> void set_service(T service) { this->service_ = service; }
 
-  template<typename T> void add_data(std::string key, T value) {
-    this->data_.push_back(TemplatableKeyValuePair<Ts...>(key, value));
-  }
+  // Keys are always string literals from the Python code generation (e.g., cg.add(var.add_data("tag_id", templ))).
+  // The value parameter can be a lambda/template, but keys are never templatable.
+  // Using pass-by-value allows the compiler to optimize for both lvalues and rvalues.
+  template<typename T> void add_data(std::string key, T value) { this->data_.emplace_back(std::move(key), value); }
   template<typename T> void add_data_template(std::string key, T value) {
-    this->data_template_.push_back(TemplatableKeyValuePair<Ts...>(key, value));
+    this->data_template_.emplace_back(std::move(key), value);
   }
   template<typename T> void add_variable(std::string key, T value) {
-    this->variables_.push_back(TemplatableKeyValuePair<Ts...>(key, value));
+    this->variables_.emplace_back(std::move(key), value);
   }
 
   void play(Ts... x) override {
     HomeassistantServiceResponse resp;
-    resp.service = this->service_.value(x...);
+    std::string service_value = this->service_.value(x...);
+    resp.set_service(StringRef(service_value));
     resp.is_event = this->is_event_;
     for (auto &it : this->data_) {
-      HomeassistantServiceMap kv;
-      kv.key = it.key;
-      kv.value = it.value.value(x...);
-      resp.data.push_back(kv);
+      resp.data.emplace_back();
+      auto &kv = resp.data.back();
+      kv.set_key(StringRef(it.key));
+      std::string value = it.value.value(x...);
+      kv.set_value(StringRef(value));
     }
     for (auto &it : this->data_template_) {
-      HomeassistantServiceMap kv;
-      kv.key = it.key;
-      kv.value = it.value.value(x...);
-      resp.data_template.push_back(kv);
+      resp.data_template.emplace_back();
+      auto &kv = resp.data_template.back();
+      kv.set_key(StringRef(it.key));
+      std::string value = it.value.value(x...);
+      kv.set_value(StringRef(value));
     }
     for (auto &it : this->variables_) {
-      HomeassistantServiceMap kv;
-      kv.key = it.key;
-      kv.value = it.value.value(x...);
-      resp.variables.push_back(kv);
+      resp.variables.emplace_back();
+      auto &kv = resp.variables.back();
+      kv.set_key(StringRef(it.key));
+      std::string value = it.value.value(x...);
+      kv.set_value(StringRef(value));
     }
     this->parent_->send_homeassistant_service_call(resp);
   }
@@ -91,6 +98,5 @@ template<typename... Ts> class HomeAssistantServiceCallAction : public Action<Ts
   std::vector<TemplatableKeyValuePair<Ts...>> variables_;
 };
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api
 #endif

esphome/components/api/list_entities.cpp

@@ -6,8 +6,7 @@
 #include "esphome/core/log.h"
 #include "esphome/core/util.h"
 
-namespace esphome {
-namespace api {
+namespace esphome::api {
 
 // Generate entity handler implementations using macros
 #ifdef USE_BINARY_SENSOR
@@ -86,10 +85,9 @@ ListEntitiesIterator::ListEntitiesIterator(APIConnection *client) : client_(clie
 #ifdef USE_API_SERVICES
 bool ListEntitiesIterator::on_service(UserServiceDescriptor *service) {
   auto resp = service->encode_list_service_response();
-  return this->client_->send_message(resp);
+  return this->client_->send_message(resp, ListEntitiesServicesResponse::MESSAGE_TYPE);
 }
 #endif
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api
 #endif

esphome/components/api/list_entities.h

@@ -4,8 +4,7 @@
 #ifdef USE_API
 #include "esphome/core/component.h"
 #include "esphome/core/component_iterator.h"
-namespace esphome {
-namespace api {
+namespace esphome::api {
 
 class APIConnection;
 
@@ -96,6 +95,5 @@ class ListEntitiesIterator : public ComponentIterator {
   APIConnection *client_;
 };
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api
 #endif

esphome/components/api/proto.cpp

@@ -3,12 +3,11 @@
 #include "esphome/core/helpers.h"
 #include "esphome/core/log.h"
 
-namespace esphome {
-namespace api {
+namespace esphome::api {
 
 static const char *const TAG = "api.proto";
 
-void ProtoMessage::decode(const uint8_t *buffer, size_t length) {
+void ProtoDecodableMessage::decode(const uint8_t *buffer, size_t length) {
   uint32_t i = 0;
   bool error = false;
   while (i < length) {
@@ -89,5 +88,4 @@ std::string ProtoMessage::dump() const {
 }
 #endif
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api

esphome/components/api/proto.h

@@ -3,16 +3,48 @@
 #include "esphome/core/component.h"
 #include "esphome/core/helpers.h"
 #include "esphome/core/log.h"
+#include "esphome/core/string_ref.h"
 
 #include <cassert>
+#include <cstring>
 #include <vector>
 
 #ifdef ESPHOME_LOG_HAS_VERY_VERBOSE
 #define HAS_PROTO_MESSAGE_DUMP
 #endif
 
-namespace esphome {
-namespace api {
+namespace esphome::api {
+
+/*
+ * StringRef Ownership Model for API Protocol Messages
+ * ===================================================
+ *
+ * StringRef is used for zero-copy string handling in outgoing (SOURCE_SERVER) messages.
+ * It holds a pointer and length to existing string data without copying.
+ *
+ * CRITICAL: The referenced string data MUST remain valid until message encoding completes.
+ *
+ * Safe StringRef Patterns:
+ * 1. String literals: StringRef("literal") - Always safe (static storage duration)
+ * 2. Member variables: StringRef(this->member_string_) - Safe if object outlives encoding
+ * 3. Global/static strings: StringRef(GLOBAL_CONSTANT) - Always safe
+ * 4. Local variables: Safe ONLY if encoding happens before function returns:
+ *    std::string temp = compute_value();
+ *    msg.set_field(StringRef(temp));
+ *    return this->send_message(msg);  // temp is valid during encoding
+ *
+ * Unsafe Patterns (WILL cause crashes/corruption):
+ * 1. Temporaries: msg.set_field(StringRef(obj.get_string())) // get_string() returns by value
+ * 2. Optional values: msg.set_field(StringRef(optional.value())) // value() returns a copy
+ * 3. Concatenation: msg.set_field(StringRef(str1 + str2)) // Result is temporary
+ *
+ * For unsafe patterns, store in a local variable first:
+ *    std::string temp = optional.value();  // or get_string() or str1 + str2
+ *    msg.set_field(StringRef(temp));
+ *
+ * The send_*_response pattern ensures proper lifetime management by encoding
+ * within the same function scope where temporaries are created.
+ */
 
 /// Representation of a VarInt - in ProtoBuf should be 64bit but we only use 32bit
 class ProtoVarInt {
@@ -135,6 +167,7 @@ class ProtoVarInt {
 
 // Forward declaration for decode_to_message and encode_to_writer
 class ProtoMessage;
+class ProtoDecodableMessage;
 
 class ProtoLengthDelimited {
  public:
@@ -142,15 +175,15 @@ class ProtoLengthDelimited {
   std::string as_string() const { return std::string(reinterpret_cast<const char *>(this->value_), this->length_); }
 
   /**
-   * Decode the length-delimited data into an existing ProtoMessage instance.
+   * Decode the length-delimited data into an existing ProtoDecodableMessage instance.
    *
    * This method allows decoding without templates, enabling use in contexts
-   * where the message type is not known at compile time. The ProtoMessage's
+   * where the message type is not known at compile time. The ProtoDecodableMessage's
    * decode() method will be called with the raw data and length.
    *
-   * @param msg The ProtoMessage instance to decode into
+   * @param msg The ProtoDecodableMessage instance to decode into
    */
-  void decode_to_message(ProtoMessage &msg) const;
+  void decode_to_message(ProtoDecodableMessage &msg) const;
 
  protected:
   const uint8_t *const value_;
@@ -175,23 +208,7 @@ class Proto32Bit {
   const uint32_t value_;
 };
 
-class Proto64Bit {
- public:
-  explicit Proto64Bit(uint64_t value) : value_(value) {}
-  uint64_t as_fixed64() const { return this->value_; }
-  int64_t as_sfixed64() const { return static_cast<int64_t>(this->value_); }
-  double as_double() const {
-    union {
-      uint64_t raw;
-      double value;
-    } s{};
-    s.raw = this->value_;
-    return s.value;
-  }
-
- protected:
-  const uint64_t value_;
-};
+// NOTE: Proto64Bit class removed - wire type 1 (64-bit fixed) not supported
 
 class ProtoWriteBuffer {
  public:
@@ -205,9 +222,9 @@ class ProtoWriteBuffer {
    * @param field_id Field number (tag) in the protobuf message
    * @param type Wire type value:
    *   - 0: Varint (int32, int64, uint32, uint64, sint32, sint64, bool, enum)
-   *   - 1: 64-bit (fixed64, sfixed64, double)
    *   - 2: Length-delimited (string, bytes, embedded messages, packed repeated fields)
    *   - 5: 32-bit (fixed32, sfixed32, float)
+   *   - Note: Wire type 1 (64-bit fixed) is not supported
    *
    * Following https://protobuf.dev/programming-guides/encoding/#structure
    */
@@ -221,12 +238,20 @@ class ProtoWriteBuffer {
 
     this->encode_field_raw(field_id, 2);  // type 2: Length-delimited string
     this->encode_varint_raw(len);
-    auto *data = reinterpret_cast<const uint8_t *>(string);
-    this->buffer_->insert(this->buffer_->end(), data, data + len);
+
+    // Using resize + memcpy instead of insert provides significant performance improvement:
+    // ~10-11x faster for 16-32 byte strings, ~3x faster for 64-byte strings
+    // as it avoids iterator checks and potential element moves that insert performs
+    size_t old_size = this->buffer_->size();
+    this->buffer_->resize(old_size + len);
+    std::memcpy(this->buffer_->data() + old_size, string, len);
   }
   void encode_string(uint32_t field_id, const std::string &value, bool force = false) {
     this->encode_string(field_id, value.data(), value.size(), force);
   }
+  void encode_string(uint32_t field_id, const StringRef &ref, bool force = false) {
+    this->encode_string(field_id, ref.c_str(), ref.size(), force);
+  }
   void encode_bytes(uint32_t field_id, const uint8_t *data, size_t len, bool force = false) {
     this->encode_string(field_id, reinterpret_cast<const char *>(data), len, force);
   }
@@ -258,20 +283,10 @@ class ProtoWriteBuffer {
     this->write((value >> 16) & 0xFF);
     this->write((value >> 24) & 0xFF);
   }
-  void encode_fixed64(uint32_t field_id, uint64_t value, bool force = false) {
-    if (value == 0 && !force)
-      return;
-
-    this->encode_field_raw(field_id, 1);  // type 1: 64-bit fixed64
-    this->write((value >> 0) & 0xFF);
-    this->write((value >> 8) & 0xFF);
-    this->write((value >> 16) & 0xFF);
-    this->write((value >> 24) & 0xFF);
-    this->write((value >> 32) & 0xFF);
-    this->write((value >> 40) & 0xFF);
-    this->write((value >> 48) & 0xFF);
-    this->write((value >> 56) & 0xFF);
-  }
+  // NOTE: Wire type 1 (64-bit fixed: double, fixed64, sfixed64) is intentionally
+  // not supported to reduce overhead on embedded systems. All ESPHome devices are
+  // 32-bit microcontrollers where 64-bit operations are expensive. If 64-bit support
+  // is needed in the future, the necessary encoding/decoding functions must be added.
   void encode_float(uint32_t field_id, float value, bool force = false) {
     if (value == 0.0f && !force)
       return;
@@ -324,7 +339,6 @@ class ProtoMessage {
   virtual ~ProtoMessage() = default;
   // Default implementation for messages with no fields
   virtual void encode(ProtoWriteBuffer buffer) const {}
-  void decode(const uint8_t *buffer, size_t length);
   // Default implementation for messages with no fields
   virtual void calculate_size(uint32_t &total_size) const {}
 #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -332,12 +346,18 @@ class ProtoMessage {
   virtual void dump_to(std::string &out) const = 0;
   virtual const char *message_name() const { return "unknown"; }
 #endif
+};
+
+// Base class for messages that support decoding
+class ProtoDecodableMessage : public ProtoMessage {
+ public:
+  void decode(const uint8_t *buffer, size_t length);
 
  protected:
   virtual bool decode_varint(uint32_t field_id, ProtoVarInt value) { return false; }
   virtual bool decode_length(uint32_t field_id, ProtoLengthDelimited value) { return false; }
   virtual bool decode_32bit(uint32_t field_id, Proto32Bit value) { return false; }
-  virtual bool decode_64bit(uint32_t field_id, Proto64Bit value) { return false; }
+  // NOTE: decode_64bit removed - wire type 1 not supported
 };
 
 class ProtoSize {
@@ -548,23 +568,40 @@ class ProtoSize {
   }
 
   /**
-   * @brief Calculates and adds the size of a fixed field to the total message size
-   *
-   * Fixed fields always take exactly N bytes (4 for fixed32/float, 8 for fixed64/double).
-   *
-   * @tparam NumBytes The number of bytes for this fixed field (4 or 8)
-   * @param is_nonzero Whether the value is non-zero
+   * @brief Calculates and adds the size of a float field to the total message size
    */
-  template<uint32_t NumBytes>
-  static inline void add_fixed_field(uint32_t &total_size, uint32_t field_id_size, bool is_nonzero) {
-    // Skip calculation if value is zero
-    if (!is_nonzero) {
-      return;  // No need to update total_size
+  static inline void add_float_field(uint32_t &total_size, uint32_t field_id_size, float value) {
+    if (value != 0.0f) {
+      total_size += field_id_size + 4;
+    }
   }
 
-    // Fixed fields always take exactly NumBytes
-    total_size += field_id_size + NumBytes;
+  // NOTE: add_double_field removed - wire type 1 (64-bit: double) not supported
+  // to reduce overhead on embedded systems
+
+  /**
+   * @brief Calculates and adds the size of a fixed32 field to the total message size
+   */
+  static inline void add_fixed32_field(uint32_t &total_size, uint32_t field_id_size, uint32_t value) {
+    if (value != 0) {
+      total_size += field_id_size + 4;
     }
+  }
+
+  // NOTE: add_fixed64_field removed - wire type 1 (64-bit: fixed64) not supported
+  // to reduce overhead on embedded systems
+
+  /**
+   * @brief Calculates and adds the size of a sfixed32 field to the total message size
+   */
+  static inline void add_sfixed32_field(uint32_t &total_size, uint32_t field_id_size, int32_t value) {
+    if (value != 0) {
+      total_size += field_id_size + 4;
+    }
+  }
+
+  // NOTE: add_sfixed64_field removed - wire type 1 (64-bit: sfixed64) not supported
+  // to reduce overhead on embedded systems
 
   /**
    * @brief Calculates and adds the size of an enum field to the total message size
@@ -662,46 +699,20 @@ class ProtoSize {
     total_size += field_id_size + varint(value);
   }
 
-  /**
-   * @brief Calculates and adds the size of a sint64 field to the total message size
-   *
-   * Sint64 fields use ZigZag encoding, which is more efficient for negative values.
-   */
-  static inline void add_sint64_field(uint32_t &total_size, uint32_t field_id_size, int64_t value) {
-    // Skip calculation if value is zero
-    if (value == 0) {
-      return;  // No need to update total_size
-    }
-
-    // ZigZag encoding for sint64: (n << 1) ^ (n >> 63)
-    uint64_t zigzag = (static_cast<uint64_t>(value) << 1) ^ (static_cast<uint64_t>(value >> 63));
-    total_size += field_id_size + varint(zigzag);
-  }
+  // NOTE: sint64 support functions (add_sint64_field, add_sint64_field_repeated) removed
+  // sint64 type is not supported by ESPHome API to reduce overhead on embedded systems
 
   /**
-   * @brief Calculates and adds the size of a sint64 field to the total message size (repeated field version)
-   *
-   * Sint64 fields use ZigZag encoding, which is more efficient for negative values.
+   * @brief Calculates and adds the size of a string field using length
    */
-  static inline void add_sint64_field_repeated(uint32_t &total_size, uint32_t field_id_size, int64_t value) {
-    // Always calculate size for repeated fields
-    // ZigZag encoding for sint64: (n << 1) ^ (n >> 63)
-    uint64_t zigzag = (static_cast<uint64_t>(value) << 1) ^ (static_cast<uint64_t>(value >> 63));
-    total_size += field_id_size + varint(zigzag);
-  }
-
-  /**
-   * @brief Calculates and adds the size of a string/bytes field to the total message size
-   */
-  static inline void add_string_field(uint32_t &total_size, uint32_t field_id_size, const std::string &str) {
+  static inline void add_string_field(uint32_t &total_size, uint32_t field_id_size, size_t len) {
     // Skip calculation if string is empty
-    if (str.empty()) {
+    if (len == 0) {
       return;  // No need to update total_size
     }
 
-    // Calculate and directly add to total_size
-    const uint32_t str_size = static_cast<uint32_t>(str.size());
-    total_size += field_id_size + varint(str_size) + str_size;
+    // Field ID + length varint + string bytes
+    total_size += field_id_size + varint(static_cast<uint32_t>(len)) + static_cast<uint32_t>(len);
   }
 
   /**
@@ -713,6 +724,19 @@ class ProtoSize {
     total_size += field_id_size + varint(str_size) + str_size;
   }
 
+  /**
+   * @brief Calculates and adds the size of a bytes field to the total message size
+   */
+  static inline void add_bytes_field(uint32_t &total_size, uint32_t field_id_size, size_t len) {
+    // Skip calculation if bytes is empty
+    if (len == 0) {
+      return;  // No need to update total_size
+    }
+
+    // Field ID + length varint + data bytes
+    total_size += field_id_size + varint(static_cast<uint32_t>(len)) + static_cast<uint32_t>(len);
+  }
+
   /**
    * @brief Calculates and adds the size of a nested message field to the total message size
    *
@@ -823,8 +847,8 @@ inline void ProtoWriteBuffer::encode_message(uint32_t field_id, const ProtoMessa
   assert(this->buffer_->size() == begin + varint_length_bytes + msg_length_bytes);
 }
 
-// Implementation of decode_to_message - must be after ProtoMessage is defined
-inline void ProtoLengthDelimited::decode_to_message(ProtoMessage &msg) const {
+// Implementation of decode_to_message - must be after ProtoDecodableMessage is defined
+inline void ProtoLengthDelimited::decode_to_message(ProtoDecodableMessage &msg) const {
   msg.decode(this->value_, this->length_);
 }
 
@@ -885,5 +909,4 @@ class ProtoService {
   }
 };
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api

esphome/components/api/subscribe_state.cpp

@@ -3,8 +3,7 @@
 #include "api_connection.h"
 #include "esphome/core/log.h"
 
-namespace esphome {
-namespace api {
+namespace esphome::api {
 
 // Generate entity handler implementations using macros
 #ifdef USE_BINARY_SENSOR
@@ -69,6 +68,5 @@ INITIAL_STATE_HANDLER(update, update::UpdateEntity)
 
 InitialStateIterator::InitialStateIterator(APIConnection *client) : client_(client) {}
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api
 #endif

esphome/components/api/subscribe_state.h

@@ -5,8 +5,7 @@
 #include "esphome/core/component.h"
 #include "esphome/core/component_iterator.h"
 #include "esphome/core/controller.h"
-namespace esphome {
-namespace api {
+namespace esphome::api {
 
 class APIConnection;
 
@@ -89,6 +88,5 @@ class InitialStateIterator : public ComponentIterator {
   APIConnection *client_;
 };
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api
 #endif

esphome/components/api/user_services.cpp

@@ -1,8 +1,7 @@
 #include "user_services.h"
 #include "esphome/core/log.h"
 
-namespace esphome {
-namespace api {
+namespace esphome::api {
 
 template<> bool get_execute_arg_value<bool>(const ExecuteServiceArgument &arg) { return arg.bool_; }
 template<> int32_t get_execute_arg_value<int32_t>(const ExecuteServiceArgument &arg) {
@@ -40,5 +39,4 @@ template<> enums::ServiceArgType to_service_arg_type<std::vector<std::string>>()
   return enums::SERVICE_ARG_TYPE_STRING_ARRAY;
 }
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api

esphome/components/api/user_services.h

@@ -8,14 +8,15 @@
 #include "api_pb2.h"
 
 #ifdef USE_API_SERVICES
-namespace esphome {
-namespace api {
+namespace esphome::api {
 
 class UserServiceDescriptor {
  public:
   virtual ListEntitiesServicesResponse encode_list_service_response() = 0;
 
   virtual bool execute_service(const ExecuteServiceRequest &req) = 0;
+
+  bool is_internal() { return false; }
 };
 
 template<typename T> T get_execute_arg_value(const ExecuteServiceArgument &arg);
@@ -31,14 +32,14 @@ template<typename... Ts> class UserServiceBase : public UserServiceDescriptor {
 
   ListEntitiesServicesResponse encode_list_service_response() override {
     ListEntitiesServicesResponse msg;
-    msg.name = this->name_;
+    msg.set_name(StringRef(this->name_));
     msg.key = this->key_;
     std::array<enums::ServiceArgType, sizeof...(Ts)> arg_types = {to_service_arg_type<Ts>()...};
     for (int i = 0; i < sizeof...(Ts); i++) {
-      ListEntitiesServicesArgument arg;
+      msg.args.emplace_back();
+      auto &arg = msg.args.back();
       arg.type = arg_types[i];
-      arg.name = this->arg_names_[i];
-      msg.args.push_back(arg);
+      arg.set_name(StringRef(this->arg_names_[i]));
     }
     return msg;
   }
@@ -72,6 +73,5 @@ template<typename... Ts> class UserServiceTrigger : public UserServiceBase<Ts...
   void execute(Ts... x) override { this->trigger(x...); }  // NOLINT
 };
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api
 #endif  // USE_API_SERVICES

esphome/components/audio/audio.h

@@ -15,7 +15,7 @@ class AudioStreamInfo {
    *  - An audio sample represents a unit of audio for one channel.
    *  - A frame represents a unit of audio with a sample for every channel.
    *
-   * In gneneral, converting between bytes, samples, and frames shouldn't result in rounding errors so long as frames
+   * In general, converting between bytes, samples, and frames shouldn't result in rounding errors so long as frames
    * are used as the main unit when transferring audio data. Durations may result in rounding for certain sample rates;
    * e.g., 44.1 KHz. The ``frames_to_milliseconds_with_remainder`` function should be used for accuracy, as it takes
    * into account the remainder rather than just ignoring any rounding.
@@ -76,7 +76,7 @@ class AudioStreamInfo {
 
   /// @brief Computes the duration, in microseconds, the given amount of frames represents.
   /// @param frames Number of audio frames
-  /// @return Duration in microseconds `frames` respresents. May be slightly inaccurate due to integer divison rounding
+  /// @return Duration in microseconds `frames` represents. May be slightly inaccurate due to integer division rounding
   ///         for certain sample rates.
   uint32_t frames_to_microseconds(uint32_t frames) const;
 

esphome/components/binary_sensor/__init__.py

@@ -266,8 +266,10 @@ async def delayed_off_filter_to_code(config, filter_id):
 async def autorepeat_filter_to_code(config, filter_id):
     timings = []
     if len(config) > 0:
-        for conf in config:
-            timings.append((conf[CONF_DELAY], conf[CONF_TIME_OFF], conf[CONF_TIME_ON]))
+        timings.extend(
+            (conf[CONF_DELAY], conf[CONF_TIME_OFF], conf[CONF_TIME_ON])
+            for conf in config
+        )
     else:
         timings.append(
             (
@@ -573,16 +575,15 @@ async def setup_binary_sensor_core_(var, config):
         await automation.build_automation(trigger, [], conf)
 
     for conf in config.get(CONF_ON_MULTI_CLICK, []):
-        timings = []
-        for tim in conf[CONF_TIMING]:
-            timings.append(
+        timings = [
             cg.StructInitializer(
                 MultiClickTriggerEvent,
                 ("state", tim[CONF_STATE]),
                 ("min_length", tim[CONF_MIN_LENGTH]),
                 ("max_length", tim.get(CONF_MAX_LENGTH, 4294967294)),
             )
-            )
+            for tim in conf[CONF_TIMING]
+        ]
         trigger = cg.new_Pvariable(conf[CONF_TRIGGER_ID], var, timings)
         if CONF_INVALID_COOLDOWN in conf:
             cg.add(trigger.set_invalid_cooldown(conf[CONF_INVALID_COOLDOWN]))

esphome/components/bluetooth_proxy/__init__.py

@@ -85,13 +85,13 @@ async def to_code(config):
     await cg.register_component(var, config)
 
     cg.add(var.set_active(config[CONF_ACTIVE]))
-    await esp32_ble_tracker.register_ble_device(var, config)
+    await esp32_ble_tracker.register_raw_ble_device(var, config)
 
     for connection_conf in config.get(CONF_CONNECTIONS, []):
         connection_var = cg.new_Pvariable(connection_conf[CONF_ID])
         await cg.register_component(connection_var, connection_conf)
         cg.add(var.register_connection(connection_var))
-        await esp32_ble_tracker.register_client(connection_var, connection_conf)
+        await esp32_ble_tracker.register_raw_client(connection_var, connection_conf)
 
     if config.get(CONF_CACHE_SERVICES):
         add_idf_sdkconfig_option("CONFIG_BT_GATTC_CACHE_NVS_FLASH", True)

esphome/components/bluetooth_proxy/bluetooth_connection.cpp

@@ -8,16 +8,182 @@
 
 #include "bluetooth_proxy.h"
 
-namespace esphome {
-namespace bluetooth_proxy {
+namespace esphome::bluetooth_proxy {
 
 static const char *const TAG = "bluetooth_proxy.connection";
 
+static void fill_128bit_uuid_array(std::array<uint64_t, 2> &out, esp_bt_uuid_t uuid_source) {
+  esp_bt_uuid_t uuid = espbt::ESPBTUUID::from_uuid(uuid_source).as_128bit().get_uuid();
+  out[0] = ((uint64_t) uuid.uuid.uuid128[15] << 56) | ((uint64_t) uuid.uuid.uuid128[14] << 48) |
+           ((uint64_t) uuid.uuid.uuid128[13] << 40) | ((uint64_t) uuid.uuid.uuid128[12] << 32) |
+           ((uint64_t) uuid.uuid.uuid128[11] << 24) | ((uint64_t) uuid.uuid.uuid128[10] << 16) |
+           ((uint64_t) uuid.uuid.uuid128[9] << 8) | ((uint64_t) uuid.uuid.uuid128[8]);
+  out[1] = ((uint64_t) uuid.uuid.uuid128[7] << 56) | ((uint64_t) uuid.uuid.uuid128[6] << 48) |
+           ((uint64_t) uuid.uuid.uuid128[5] << 40) | ((uint64_t) uuid.uuid.uuid128[4] << 32) |
+           ((uint64_t) uuid.uuid.uuid128[3] << 24) | ((uint64_t) uuid.uuid.uuid128[2] << 16) |
+           ((uint64_t) uuid.uuid.uuid128[1] << 8) | ((uint64_t) uuid.uuid.uuid128[0]);
+}
+
 void BluetoothConnection::dump_config() {
   ESP_LOGCONFIG(TAG, "BLE Connection:");
   BLEClientBase::dump_config();
 }
 
+void BluetoothConnection::loop() {
+  BLEClientBase::loop();
+
+  // Early return if no active connection or not in service discovery phase
+  if (this->address_ == 0 || this->send_service_ < 0 || this->send_service_ > this->service_count_) {
+    return;
+  }
+
+  // Handle service discovery
+  this->send_service_for_discovery_();
+}
+
+void BluetoothConnection::reset_connection_(esp_err_t reason) {
+  // Send disconnection notification
+  this->proxy_->send_device_connection(this->address_, false, 0, reason);
+
+  // Important: If we were in the middle of sending services, we do NOT send
+  // send_gatt_services_done() here. This ensures the client knows that
+  // the service discovery was interrupted and can retry. The client
+  // (aioesphomeapi) implements a 30-second timeout (DEFAULT_BLE_TIMEOUT)
+  // to detect incomplete service discovery rather than relying on us to
+  // tell them about a partial list.
+  this->set_address(0);
+  this->send_service_ = DONE_SENDING_SERVICES;
+  this->proxy_->send_connections_free();
+}
+
+void BluetoothConnection::send_service_for_discovery_() {
+  if (this->send_service_ == this->service_count_) {
+    this->send_service_ = DONE_SENDING_SERVICES;
+    this->proxy_->send_gatt_services_done(this->address_);
+    if (this->connection_type_ == espbt::ConnectionType::V3_WITH_CACHE ||
+        this->connection_type_ == espbt::ConnectionType::V3_WITHOUT_CACHE) {
+      this->release_services();
+    }
+    return;
+  }
+
+  // Early return if no API connection
+  auto *api_conn = this->proxy_->get_api_connection();
+  if (api_conn == nullptr) {
+    return;
+  }
+
+  // Send next service
+  esp_gattc_service_elem_t service_result;
+  uint16_t service_count = 1;
+  esp_gatt_status_t service_status = esp_ble_gattc_get_service(this->gattc_if_, this->conn_id_, nullptr,
+                                                               &service_result, &service_count, this->send_service_);
+  this->send_service_++;
+
+  if (service_status != ESP_GATT_OK) {
+    ESP_LOGE(TAG, "[%d] [%s] esp_ble_gattc_get_service error at offset=%d, status=%d", this->connection_index_,
+             this->address_str().c_str(), this->send_service_ - 1, service_status);
+    return;
+  }
+
+  if (service_count == 0) {
+    ESP_LOGE(TAG, "[%d] [%s] esp_ble_gattc_get_service missing, service_count=%d", this->connection_index_,
+             this->address_str().c_str(), service_count);
+    return;
+  }
+
+  api::BluetoothGATTGetServicesResponse resp;
+  resp.address = this->address_;
+  auto &service_resp = resp.services[0];
+  fill_128bit_uuid_array(service_resp.uuid, service_result.uuid);
+  service_resp.handle = service_result.start_handle;
+
+  // Get the number of characteristics directly with one call
+  uint16_t total_char_count = 0;
+  esp_gatt_status_t char_count_status =
+      esp_ble_gattc_get_attr_count(this->gattc_if_, this->conn_id_, ESP_GATT_DB_CHARACTERISTIC,
+                                   service_result.start_handle, service_result.end_handle, 0, &total_char_count);
+
+  if (char_count_status == ESP_GATT_OK && total_char_count > 0) {
+    // Only reserve if we successfully got a count
+    service_resp.characteristics.reserve(total_char_count);
+  } else if (char_count_status != ESP_GATT_OK) {
+    ESP_LOGW(TAG, "[%d] [%s] Error getting characteristic count, status=%d", this->connection_index_,
+             this->address_str().c_str(), char_count_status);
+  }
+
+  // Now process characteristics
+  uint16_t char_offset = 0;
+  esp_gattc_char_elem_t char_result;
+  while (true) {  // characteristics
+    uint16_t char_count = 1;
+    esp_gatt_status_t char_status =
+        esp_ble_gattc_get_all_char(this->gattc_if_, this->conn_id_, service_result.start_handle,
+                                   service_result.end_handle, &char_result, &char_count, char_offset);
+    if (char_status == ESP_GATT_INVALID_OFFSET || char_status == ESP_GATT_NOT_FOUND) {
+      break;
+    }
+    if (char_status != ESP_GATT_OK) {
+      ESP_LOGE(TAG, "[%d] [%s] esp_ble_gattc_get_all_char error, status=%d", this->connection_index_,
+               this->address_str().c_str(), char_status);
+      break;
+    }
+    if (char_count == 0) {
+      break;
+    }
+
+    service_resp.characteristics.emplace_back();
+    auto &characteristic_resp = service_resp.characteristics.back();
+    fill_128bit_uuid_array(characteristic_resp.uuid, char_result.uuid);
+    characteristic_resp.handle = char_result.char_handle;
+    characteristic_resp.properties = char_result.properties;
+    char_offset++;
+
+    // Get the number of descriptors directly with one call
+    uint16_t total_desc_count = 0;
+    esp_gatt_status_t desc_count_status =
+        esp_ble_gattc_get_attr_count(this->gattc_if_, this->conn_id_, ESP_GATT_DB_DESCRIPTOR, char_result.char_handle,
+                                     service_result.end_handle, 0, &total_desc_count);
+
+    if (desc_count_status == ESP_GATT_OK && total_desc_count > 0) {
+      // Only reserve if we successfully got a count
+      characteristic_resp.descriptors.reserve(total_desc_count);
+    } else if (desc_count_status != ESP_GATT_OK) {
+      ESP_LOGW(TAG, "[%d] [%s] Error getting descriptor count for char handle %d, status=%d", this->connection_index_,
+               this->address_str().c_str(), char_result.char_handle, desc_count_status);
+    }
+
+    // Now process descriptors
+    uint16_t desc_offset = 0;
+    esp_gattc_descr_elem_t desc_result;
+    while (true) {  // descriptors
+      uint16_t desc_count = 1;
+      esp_gatt_status_t desc_status = esp_ble_gattc_get_all_descr(
+          this->gattc_if_, this->conn_id_, char_result.char_handle, &desc_result, &desc_count, desc_offset);
+      if (desc_status == ESP_GATT_INVALID_OFFSET || desc_status == ESP_GATT_NOT_FOUND) {
+        break;
+      }
+      if (desc_status != ESP_GATT_OK) {
+        ESP_LOGE(TAG, "[%d] [%s] esp_ble_gattc_get_all_descr error, status=%d", this->connection_index_,
+                 this->address_str().c_str(), desc_status);
+        break;
+      }
+      if (desc_count == 0) {
+        break;
+      }
+
+      characteristic_resp.descriptors.emplace_back();
+      auto &descriptor_resp = characteristic_resp.descriptors.back();
+      fill_128bit_uuid_array(descriptor_resp.uuid, desc_result.uuid);
+      descriptor_resp.handle = desc_result.handle;
+      desc_offset++;
+    }
+  }
+
+  // Send the message (we already checked api_conn is not null at the beginning)
+  api_conn->send_message(resp, api::BluetoothGATTGetServicesResponse::MESSAGE_TYPE);
+}
+
 bool BluetoothConnection::gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_if,
                                               esp_ble_gattc_cb_param_t *param) {
   if (!BLEClientBase::gattc_event_handler(event, gattc_if, param))
@@ -25,22 +191,16 @@ bool BluetoothConnection::gattc_event_handler(esp_gattc_cb_event_t event, esp_ga
 
   switch (event) {
     case ESP_GATTC_DISCONNECT_EVT: {
-      this->proxy_->send_device_connection(this->address_, false, 0, param->disconnect.reason);
-      this->set_address(0);
-      this->proxy_->send_connections_free();
+      this->reset_connection_(param->disconnect.reason);
       break;
     }
     case ESP_GATTC_CLOSE_EVT: {
-      this->proxy_->send_device_connection(this->address_, false, 0, param->close.reason);
-      this->set_address(0);
-      this->proxy_->send_connections_free();
+      this->reset_connection_(param->close.reason);
       break;
     }
     case ESP_GATTC_OPEN_EVT: {
       if (param->open.status != ESP_GATT_OK && param->open.status != ESP_GATT_ALREADY_OPEN) {
-        this->proxy_->send_device_connection(this->address_, false, 0, param->open.status);
-        this->set_address(0);
-        this->proxy_->send_connections_free();
+        this->reset_connection_(param->open.status);
       } else if (this->connection_type_ == espbt::ConnectionType::V3_WITH_CACHE) {
         this->proxy_->send_device_connection(this->address_, true, this->mtu_);
         this->proxy_->send_connections_free();
@@ -72,10 +232,8 @@ bool BluetoothConnection::gattc_event_handler(esp_gattc_cb_event_t event, esp_ga
       api::BluetoothGATTReadResponse resp;
       resp.address = this->address_;
       resp.handle = param->read.handle;
-      resp.data.reserve(param->read.value_len);
-      // Use bulk insert instead of individual push_backs
-      resp.data.insert(resp.data.end(), param->read.value, param->read.value + param->read.value_len);
-      this->proxy_->get_api_connection()->send_message(resp);
+      resp.set_data(param->read.value, param->read.value_len);
+      this->proxy_->get_api_connection()->send_message(resp, api::BluetoothGATTReadResponse::MESSAGE_TYPE);
       break;
     }
     case ESP_GATTC_WRITE_CHAR_EVT:
@@ -89,7 +247,7 @@ bool BluetoothConnection::gattc_event_handler(esp_gattc_cb_event_t event, esp_ga
       api::BluetoothGATTWriteResponse resp;
       resp.address = this->address_;
       resp.handle = param->write.handle;
-      this->proxy_->get_api_connection()->send_message(resp);
+      this->proxy_->get_api_connection()->send_message(resp, api::BluetoothGATTWriteResponse::MESSAGE_TYPE);
       break;
     }
     case ESP_GATTC_UNREG_FOR_NOTIFY_EVT: {
@@ -103,7 +261,7 @@ bool BluetoothConnection::gattc_event_handler(esp_gattc_cb_event_t event, esp_ga
       api::BluetoothGATTNotifyResponse resp;
       resp.address = this->address_;
       resp.handle = param->unreg_for_notify.handle;
-      this->proxy_->get_api_connection()->send_message(resp);
+      this->proxy_->get_api_connection()->send_message(resp, api::BluetoothGATTNotifyResponse::MESSAGE_TYPE);
       break;
     }
     case ESP_GATTC_REG_FOR_NOTIFY_EVT: {
@@ -116,7 +274,7 @@ bool BluetoothConnection::gattc_event_handler(esp_gattc_cb_event_t event, esp_ga
       api::BluetoothGATTNotifyResponse resp;
       resp.address = this->address_;
       resp.handle = param->reg_for_notify.handle;
-      this->proxy_->get_api_connection()->send_message(resp);
+      this->proxy_->get_api_connection()->send_message(resp, api::BluetoothGATTNotifyResponse::MESSAGE_TYPE);
       break;
     }
     case ESP_GATTC_NOTIFY_EVT: {
@@ -125,10 +283,8 @@ bool BluetoothConnection::gattc_event_handler(esp_gattc_cb_event_t event, esp_ga
       api::BluetoothGATTNotifyDataResponse resp;
       resp.address = this->address_;
       resp.handle = param->notify.handle;
-      resp.data.reserve(param->notify.value_len);
-      // Use bulk insert instead of individual push_backs
-      resp.data.insert(resp.data.end(), param->notify.value, param->notify.value + param->notify.value_len);
-      this->proxy_->get_api_connection()->send_message(resp);
+      resp.set_data(param->notify.value, param->notify.value_len);
+      this->proxy_->get_api_connection()->send_message(resp, api::BluetoothGATTNotifyDataResponse::MESSAGE_TYPE);
       break;
     }
     default:
@@ -265,7 +421,6 @@ esp32_ble_tracker::AdvertisementParserType BluetoothConnection::get_advertisemen
   return this->proxy_->get_advertisement_parser_type();
 }
 
-}  // namespace bluetooth_proxy
-}  // namespace esphome
+}  // namespace esphome::bluetooth_proxy
 
 #endif  // USE_ESP32

esphome/components/bluetooth_proxy/bluetooth_connection.h

@@ -4,14 +4,14 @@
 
 #include "esphome/components/esp32_ble_client/ble_client_base.h"
 
-namespace esphome {
-namespace bluetooth_proxy {
+namespace esphome::bluetooth_proxy {
 
 class BluetoothProxy;
 
 class BluetoothConnection : public esp32_ble_client::BLEClientBase {
  public:
   void dump_config() override;
+  void loop() override;
   bool gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_if,
                            esp_ble_gattc_cb_param_t *param) override;
   void gap_event_handler(esp_gap_ble_cb_event_t event, esp_ble_gap_cb_param_t *param) override;
@@ -27,6 +27,9 @@ class BluetoothConnection : public esp32_ble_client::BLEClientBase {
  protected:
   friend class BluetoothProxy;
 
+  void send_service_for_discovery_();
+  void reset_connection_(esp_err_t reason);
+
   // Memory optimized layout for 32-bit systems
   // Group 1: Pointers (4 bytes each, naturally aligned)
   BluetoothProxy *proxy_;
@@ -39,7 +42,6 @@ class BluetoothConnection : public esp32_ble_client::BLEClientBase {
   // 1 byte used, 1 byte padding
 };
 
-}  // namespace bluetooth_proxy
-}  // namespace esphome
+}  // namespace esphome::bluetooth_proxy
 
 #endif  // USE_ESP32

esphome/components/bluetooth_proxy/bluetooth_proxy.cpp

@@ -3,30 +3,38 @@
 #include "esphome/core/log.h"
 #include "esphome/core/macros.h"
 #include "esphome/core/application.h"
+#include <cstring>
 
 #ifdef USE_ESP32
 
-namespace esphome {
-namespace bluetooth_proxy {
+namespace esphome::bluetooth_proxy {
 
 static const char *const TAG = "bluetooth_proxy";
-static const int DONE_SENDING_SERVICES = -2;
 
-std::vector<uint64_t> get_128bit_uuid_vec(esp_bt_uuid_t uuid_source) {
-  esp_bt_uuid_t uuid = espbt::ESPBTUUID::from_uuid(uuid_source).as_128bit().get_uuid();
-  return std::vector<uint64_t>{((uint64_t) uuid.uuid.uuid128[15] << 56) | ((uint64_t) uuid.uuid.uuid128[14] << 48) |
-                                   ((uint64_t) uuid.uuid.uuid128[13] << 40) | ((uint64_t) uuid.uuid.uuid128[12] << 32) |
-                                   ((uint64_t) uuid.uuid.uuid128[11] << 24) | ((uint64_t) uuid.uuid.uuid128[10] << 16) |
-                                   ((uint64_t) uuid.uuid.uuid128[9] << 8) | ((uint64_t) uuid.uuid.uuid128[8]),
-                               ((uint64_t) uuid.uuid.uuid128[7] << 56) | ((uint64_t) uuid.uuid.uuid128[6] << 48) |
-                                   ((uint64_t) uuid.uuid.uuid128[5] << 40) | ((uint64_t) uuid.uuid.uuid128[4] << 32) |
-                                   ((uint64_t) uuid.uuid.uuid128[3] << 24) | ((uint64_t) uuid.uuid.uuid128[2] << 16) |
-                                   ((uint64_t) uuid.uuid.uuid128[1] << 8) | ((uint64_t) uuid.uuid.uuid128[0])};
-}
+// Batch size for BLE advertisements to maximize WiFi efficiency
+// Each advertisement is up to 80 bytes when packaged (including protocol overhead)
+// Most advertisements are 20-30 bytes, allowing even more to fit per packet
+// 16 advertisements × 80 bytes (worst case) = 1280 bytes out of ~1320 bytes usable payload
+// This achieves ~97% WiFi MTU utilization while staying under the limit
+static constexpr size_t FLUSH_BATCH_SIZE = 16;
+
+// Verify BLE advertisement data array size matches the BLE specification (31 bytes adv + 31 bytes scan response)
+static_assert(sizeof(((api::BluetoothLERawAdvertisement *) nullptr)->data) == 62,
+              "BLE advertisement data array size mismatch");
 
 BluetoothProxy::BluetoothProxy() { global_bluetooth_proxy = this; }
 
 void BluetoothProxy::setup() {
+  // Pre-allocate response object
+  this->response_ = std::make_unique<api::BluetoothLERawAdvertisementsResponse>();
+
+  // Reserve capacity but start with size 0
+  // Reserve 50% since we'll grow naturally and flush at FLUSH_BATCH_SIZE
+  this->response_->advertisements.reserve(FLUSH_BATCH_SIZE / 2);
+
+  // Don't pre-allocate pool - let it grow only if needed in busy environments
+  // Many devices in quiet areas will never need the overflow pool
+
   this->parent_->add_scanner_state_callback([this](esp32_ble_tracker::ScannerState state) {
     if (this->api_connection_ != nullptr) {
       this->send_bluetooth_scanner_state_(state);
@@ -39,128 +47,91 @@ void BluetoothProxy::send_bluetooth_scanner_state_(esp32_ble_tracker::ScannerSta
   resp.state = static_cast<api::enums::BluetoothScannerState>(state);
   resp.mode = this->parent_->get_scan_active() ? api::enums::BluetoothScannerMode::BLUETOOTH_SCANNER_MODE_ACTIVE
                                                : api::enums::BluetoothScannerMode::BLUETOOTH_SCANNER_MODE_PASSIVE;
-  this->api_connection_->send_message(resp);
+  this->api_connection_->send_message(resp, api::BluetoothScannerStateResponse::MESSAGE_TYPE);
 }
 
+#ifdef USE_ESP32_BLE_DEVICE
 bool BluetoothProxy::parse_device(const esp32_ble_tracker::ESPBTDevice &device) {
-  if (!api::global_api_server->is_connected() || this->api_connection_ == nullptr || this->raw_advertisements_)
+  // This method should never be called since bluetooth_proxy always uses raw advertisements
+  // but we need to provide an implementation to satisfy the virtual method requirement
   return false;
-
-  ESP_LOGV(TAG, "Proxying packet from %s - %s. RSSI: %d dB", device.get_name().c_str(), device.address_str().c_str(),
-           device.get_rssi());
-  this->send_api_packet_(device);
-  return true;
 }
-
-// Batch size for BLE advertisements to maximize WiFi efficiency
-// Each advertisement is up to 80 bytes when packaged (including protocol overhead)
-// Most advertisements are 20-30 bytes, allowing even more to fit per packet
-// 16 advertisements × 80 bytes (worst case) = 1280 bytes out of ~1320 bytes usable payload
-// This achieves ~97% WiFi MTU utilization while staying under the limit
-static constexpr size_t FLUSH_BATCH_SIZE = 16;
-
-namespace {
-// Batch buffer in anonymous namespace to avoid guard variable (saves 8 bytes)
-// This is initialized at program startup before any threads
-// NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables)
-std::vector<api::BluetoothLERawAdvertisement> batch_buffer;
-}  // namespace
-
-static std::vector<api::BluetoothLERawAdvertisement> &get_batch_buffer() { return batch_buffer; }
+#endif
 
 bool BluetoothProxy::parse_devices(const esp32_ble::BLEScanResult *scan_results, size_t count) {
-  if (!api::global_api_server->is_connected() || this->api_connection_ == nullptr || !this->raw_advertisements_)
+  if (!api::global_api_server->is_connected() || this->api_connection_ == nullptr)
     return false;
 
-  // Get the batch buffer reference
-  auto &batch_buffer = get_batch_buffer();
+  auto &advertisements = this->response_->advertisements;
 
-  // Reserve additional capacity if needed
-  size_t new_size = batch_buffer.size() + count;
-  if (batch_buffer.capacity() < new_size) {
-    batch_buffer.reserve(new_size);
-  }
-
-  // Add new advertisements to the batch buffer
   for (size_t i = 0; i < count; i++) {
     auto &result = scan_results[i];
     uint8_t length = result.adv_data_len + result.scan_rsp_len;
 
-    batch_buffer.emplace_back();
-    auto &adv = batch_buffer.back();
+    // Check if we need to expand the vector
+    if (this->advertisement_count_ >= advertisements.size()) {
+      if (this->advertisement_pool_.empty()) {
+        // No room in pool, need to allocate
+        advertisements.emplace_back();
+      } else {
+        // Pull from pool
+        advertisements.push_back(std::move(this->advertisement_pool_.back()));
+        this->advertisement_pool_.pop_back();
+      }
+    }
+
+    // Fill in the data directly at current position
+    auto &adv = advertisements[this->advertisement_count_];
     adv.address = esp32_ble::ble_addr_to_uint64(result.bda);
     adv.rssi = result.rssi;
     adv.address_type = result.ble_addr_type;
-    adv.data.assign(&result.ble_adv[0], &result.ble_adv[length]);
+    adv.data_len = length;
+    std::memcpy(adv.data, result.ble_adv, length);
+
+    this->advertisement_count_++;
 
     ESP_LOGV(TAG, "Queuing raw packet from %02X:%02X:%02X:%02X:%02X:%02X, length %d. RSSI: %d dB", result.bda[0],
              result.bda[1], result.bda[2], result.bda[3], result.bda[4], result.bda[5], length, result.rssi);
-  }
 
-  // Only send if we've accumulated a good batch size to maximize batching efficiency
-  // https://github.com/esphome/backlog/issues/21
-  if (batch_buffer.size() >= FLUSH_BATCH_SIZE) {
+    // Flush if we have reached FLUSH_BATCH_SIZE
+    if (this->advertisement_count_ >= FLUSH_BATCH_SIZE) {
       this->flush_pending_advertisements();
     }
+  }
 
   return true;
 }
 
 void BluetoothProxy::flush_pending_advertisements() {
-  auto &batch_buffer = get_batch_buffer();
-  if (batch_buffer.empty() || !api::global_api_server->is_connected() || this->api_connection_ == nullptr)
+  if (this->advertisement_count_ == 0 || !api::global_api_server->is_connected() || this->api_connection_ == nullptr)
     return;
 
-  api::BluetoothLERawAdvertisementsResponse resp;
-  resp.advertisements.swap(batch_buffer);
-  this->api_connection_->send_message(resp);
-}
+  auto &advertisements = this->response_->advertisements;
 
-void BluetoothProxy::send_api_packet_(const esp32_ble_tracker::ESPBTDevice &device) {
-  api::BluetoothLEAdvertisementResponse resp;
-  resp.address = device.address_uint64();
-  resp.address_type = device.get_address_type();
-  if (!device.get_name().empty())
-    resp.name = device.get_name();
-  resp.rssi = device.get_rssi();
+  // Return any items beyond advertisement_count_ to the pool
+  if (advertisements.size() > this->advertisement_count_) {
+    // Move unused items back to pool
+    this->advertisement_pool_.insert(this->advertisement_pool_.end(),
+                                     std::make_move_iterator(advertisements.begin() + this->advertisement_count_),
+                                     std::make_move_iterator(advertisements.end()));
 
-  // Pre-allocate vectors based on known sizes
-  auto service_uuids = device.get_service_uuids();
-  resp.service_uuids.reserve(service_uuids.size());
-  for (auto &uuid : service_uuids) {
-    resp.service_uuids.emplace_back(uuid.to_string());
+    // Resize to actual count
+    advertisements.resize(this->advertisement_count_);
   }
 
-  // Pre-allocate service data vector
-  auto service_datas = device.get_service_datas();
-  resp.service_data.reserve(service_datas.size());
-  for (auto &data : service_datas) {
-    resp.service_data.emplace_back();
-    auto &service_data = resp.service_data.back();
-    service_data.uuid = data.uuid.to_string();
-    service_data.data.assign(data.data.begin(), data.data.end());
-  }
+  // Send the message
+  this->api_connection_->send_message(*this->response_, api::BluetoothLERawAdvertisementsResponse::MESSAGE_TYPE);
 
-  // Pre-allocate manufacturer data vector
-  auto manufacturer_datas = device.get_manufacturer_datas();
-  resp.manufacturer_data.reserve(manufacturer_datas.size());
-  for (auto &data : manufacturer_datas) {
-    resp.manufacturer_data.emplace_back();
-    auto &manufacturer_data = resp.manufacturer_data.back();
-    manufacturer_data.uuid = data.uuid.to_string();
-    manufacturer_data.data.assign(data.data.begin(), data.data.end());
-  }
-
-  this->api_connection_->send_message(resp);
+  // Reset count - existing items will be overwritten in next batch
+  this->advertisement_count_ = 0;
 }
 
 void BluetoothProxy::dump_config() {
   ESP_LOGCONFIG(TAG, "Bluetooth Proxy:");
   ESP_LOGCONFIG(TAG,
                 "  Active: %s\n"
-                "  Connections: %d\n"
-                "  Raw advertisements: %s",
-                YESNO(this->active_), this->connections_.size(), YESNO(this->raw_advertisements_));
+                "  Connections: %d",
+                YESNO(this->active_), this->connections_.size());
 }
 
 int BluetoothProxy::get_bluetooth_connections_free() {
@@ -188,139 +159,17 @@ void BluetoothProxy::loop() {
   }
 
   // Flush any pending BLE advertisements that have been accumulated but not yet sent
-  if (this->raw_advertisements_) {
-    static uint32_t last_flush_time = 0;
   uint32_t now = App.get_loop_component_start_time();
 
   // Flush accumulated advertisements every 100ms
-    if (now - last_flush_time >= 100) {
+  if (now - this->last_advertisement_flush_time_ >= 100) {
     this->flush_pending_advertisements();
-      last_flush_time = now;
-    }
-  }
-  for (auto *connection : this->connections_) {
-    if (connection->send_service_ == connection->service_count_) {
-      connection->send_service_ = DONE_SENDING_SERVICES;
-      this->send_gatt_services_done(connection->get_address());
-      if (connection->connection_type_ == espbt::ConnectionType::V3_WITH_CACHE ||
-          connection->connection_type_ == espbt::ConnectionType::V3_WITHOUT_CACHE) {
-        connection->release_services();
-      }
-    } else if (connection->send_service_ >= 0) {
-      esp_gattc_service_elem_t service_result;
-      uint16_t service_count = 1;
-      esp_gatt_status_t service_status =
-          esp_ble_gattc_get_service(connection->get_gattc_if(), connection->get_conn_id(), nullptr, &service_result,
-                                    &service_count, connection->send_service_);
-      connection->send_service_++;
-      if (service_status != ESP_GATT_OK) {
-        ESP_LOGE(TAG, "[%d] [%s] esp_ble_gattc_get_service error at offset=%d, status=%d",
-                 connection->get_connection_index(), connection->address_str().c_str(), connection->send_service_ - 1,
-                 service_status);
-        continue;
-      }
-      if (service_count == 0) {
-        ESP_LOGE(TAG, "[%d] [%s] esp_ble_gattc_get_service missing, service_count=%d",
-                 connection->get_connection_index(), connection->address_str().c_str(), service_count);
-        continue;
-      }
-      api::BluetoothGATTGetServicesResponse resp;
-      resp.address = connection->get_address();
-      resp.services.reserve(1);  // Always one service per response in this implementation
-      api::BluetoothGATTService service_resp;
-      service_resp.uuid = get_128bit_uuid_vec(service_result.uuid);
-      service_resp.handle = service_result.start_handle;
-      uint16_t char_offset = 0;
-      esp_gattc_char_elem_t char_result;
-      // Get the number of characteristics directly with one call
-      uint16_t total_char_count = 0;
-      esp_gatt_status_t char_count_status = esp_ble_gattc_get_attr_count(
-          connection->get_gattc_if(), connection->get_conn_id(), ESP_GATT_DB_CHARACTERISTIC,
-          service_result.start_handle, service_result.end_handle, 0, &total_char_count);
-
-      if (char_count_status == ESP_GATT_OK && total_char_count > 0) {
-        // Only reserve if we successfully got a count
-        service_resp.characteristics.reserve(total_char_count);
-      } else if (char_count_status != ESP_GATT_OK) {
-        ESP_LOGW(TAG, "[%d] [%s] Error getting characteristic count, status=%d", connection->get_connection_index(),
-                 connection->address_str().c_str(), char_count_status);
-      }
-
-      // Now process characteristics
-      while (true) {  // characteristics
-        uint16_t char_count = 1;
-        esp_gatt_status_t char_status = esp_ble_gattc_get_all_char(
-            connection->get_gattc_if(), connection->get_conn_id(), service_result.start_handle,
-            service_result.end_handle, &char_result, &char_count, char_offset);
-        if (char_status == ESP_GATT_INVALID_OFFSET || char_status == ESP_GATT_NOT_FOUND) {
-          break;
-        }
-        if (char_status != ESP_GATT_OK) {
-          ESP_LOGE(TAG, "[%d] [%s] esp_ble_gattc_get_all_char error, status=%d", connection->get_connection_index(),
-                   connection->address_str().c_str(), char_status);
-          break;
-        }
-        if (char_count == 0) {
-          break;
-        }
-        api::BluetoothGATTCharacteristic characteristic_resp;
-        characteristic_resp.uuid = get_128bit_uuid_vec(char_result.uuid);
-        characteristic_resp.handle = char_result.char_handle;
-        characteristic_resp.properties = char_result.properties;
-        char_offset++;
-
-        // Get the number of descriptors directly with one call
-        uint16_t total_desc_count = 0;
-        esp_gatt_status_t desc_count_status =
-            esp_ble_gattc_get_attr_count(connection->get_gattc_if(), connection->get_conn_id(), ESP_GATT_DB_DESCRIPTOR,
-                                         char_result.char_handle, service_result.end_handle, 0, &total_desc_count);
-
-        if (desc_count_status == ESP_GATT_OK && total_desc_count > 0) {
-          // Only reserve if we successfully got a count
-          characteristic_resp.descriptors.reserve(total_desc_count);
-        } else if (desc_count_status != ESP_GATT_OK) {
-          ESP_LOGW(TAG, "[%d] [%s] Error getting descriptor count for char handle %d, status=%d",
-                   connection->get_connection_index(), connection->address_str().c_str(), char_result.char_handle,
-                   desc_count_status);
-        }
-
-        // Now process descriptors
-        uint16_t desc_offset = 0;
-        esp_gattc_descr_elem_t desc_result;
-        while (true) {  // descriptors
-          uint16_t desc_count = 1;
-          esp_gatt_status_t desc_status =
-              esp_ble_gattc_get_all_descr(connection->get_gattc_if(), connection->get_conn_id(),
-                                          char_result.char_handle, &desc_result, &desc_count, desc_offset);
-          if (desc_status == ESP_GATT_INVALID_OFFSET || desc_status == ESP_GATT_NOT_FOUND) {
-            break;
-          }
-          if (desc_status != ESP_GATT_OK) {
-            ESP_LOGE(TAG, "[%d] [%s] esp_ble_gattc_get_all_descr error, status=%d", connection->get_connection_index(),
-                     connection->address_str().c_str(), desc_status);
-            break;
-          }
-          if (desc_count == 0) {
-            break;
-          }
-          api::BluetoothGATTDescriptor descriptor_resp;
-          descriptor_resp.uuid = get_128bit_uuid_vec(desc_result.uuid);
-          descriptor_resp.handle = desc_result.handle;
-          characteristic_resp.descriptors.push_back(std::move(descriptor_resp));
-          desc_offset++;
-        }
-        service_resp.characteristics.push_back(std::move(characteristic_resp));
-      }
-      resp.services.push_back(std::move(service_resp));
-      this->api_connection_->send_message(resp);
-    }
+    this->last_advertisement_flush_time_ = now;
   }
 }
 
 esp32_ble_tracker::AdvertisementParserType BluetoothProxy::get_advertisement_parser_type() {
-  if (this->raw_advertisements_)
   return esp32_ble_tracker::AdvertisementParserType::RAW_ADVERTISEMENTS;
-  return esp32_ble_tracker::AdvertisementParserType::PARSED_ADVERTISEMENTS;
 }
 
 BluetoothConnection *BluetoothProxy::get_connection_(uint64_t address, bool reserve) {
@@ -465,7 +314,7 @@ void BluetoothProxy::bluetooth_device_request(const api::BluetoothDeviceRequest
       call.success = ret == ESP_OK;
       call.error = ret;
 
-      this->api_connection_->send_message(call);
+      this->api_connection_->send_message(call, api::BluetoothDeviceClearCacheResponse::MESSAGE_TYPE);
 
       break;
     }
@@ -565,7 +414,6 @@ void BluetoothProxy::subscribe_api_connection(api::APIConnection *api_connection
     return;
   }
   this->api_connection_ = api_connection;
-  this->raw_advertisements_ = flags & BluetoothProxySubscriptionFlag::SUBSCRIPTION_RAW_ADVERTISEMENTS;
   this->parent_->recalculate_advertisement_parser_types();
 
   this->send_bluetooth_scanner_state_(this->parent_->get_scanner_state());
@@ -577,7 +425,6 @@ void BluetoothProxy::unsubscribe_api_connection(api::APIConnection *api_connecti
     return;
   }
   this->api_connection_ = nullptr;
-  this->raw_advertisements_ = false;
   this->parent_->recalculate_advertisement_parser_types();
 }
 
@@ -589,7 +436,7 @@ void BluetoothProxy::send_device_connection(uint64_t address, bool connected, ui
   call.connected = connected;
   call.mtu = mtu;
   call.error = error;
-  this->api_connection_->send_message(call);
+  this->api_connection_->send_message(call, api::BluetoothDeviceConnectionResponse::MESSAGE_TYPE);
 }
 void BluetoothProxy::send_connections_free() {
   if (this->api_connection_ == nullptr)
@@ -602,7 +449,7 @@ void BluetoothProxy::send_connections_free() {
       call.allocated.push_back(connection->address_);
     }
   }
-  this->api_connection_->send_message(call);
+  this->api_connection_->send_message(call, api::BluetoothConnectionsFreeResponse::MESSAGE_TYPE);
 }
 
 void BluetoothProxy::send_gatt_services_done(uint64_t address) {
@@ -610,7 +457,7 @@ void BluetoothProxy::send_gatt_services_done(uint64_t address) {
     return;
   api::BluetoothGATTGetServicesDoneResponse call;
   call.address = address;
-  this->api_connection_->send_message(call);
+  this->api_connection_->send_message(call, api::BluetoothGATTGetServicesDoneResponse::MESSAGE_TYPE);
 }
 
 void BluetoothProxy::send_gatt_error(uint64_t address, uint16_t handle, esp_err_t error) {
@@ -620,7 +467,7 @@ void BluetoothProxy::send_gatt_error(uint64_t address, uint16_t handle, esp_err_
   call.address = address;
   call.handle = handle;
   call.error = error;
-  this->api_connection_->send_message(call);
+  this->api_connection_->send_message(call, api::BluetoothGATTWriteResponse::MESSAGE_TYPE);
 }
 
 void BluetoothProxy::send_device_pairing(uint64_t address, bool paired, esp_err_t error) {
@@ -629,7 +476,7 @@ void BluetoothProxy::send_device_pairing(uint64_t address, bool paired, esp_err_
   call.paired = paired;
   call.error = error;
 
-  this->api_connection_->send_message(call);
+  this->api_connection_->send_message(call, api::BluetoothDevicePairingResponse::MESSAGE_TYPE);
 }
 
 void BluetoothProxy::send_device_unpairing(uint64_t address, bool success, esp_err_t error) {
@@ -638,7 +485,7 @@ void BluetoothProxy::send_device_unpairing(uint64_t address, bool success, esp_e
   call.success = success;
   call.error = error;
 
-  this->api_connection_->send_message(call);
+  this->api_connection_->send_message(call, api::BluetoothDeviceUnpairingResponse::MESSAGE_TYPE);
 }
 
 void BluetoothProxy::bluetooth_scanner_set_mode(bool active) {
@@ -654,7 +501,6 @@ void BluetoothProxy::bluetooth_scanner_set_mode(bool active) {
 
 BluetoothProxy *global_bluetooth_proxy = nullptr;  // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
 
-}  // namespace bluetooth_proxy
-}  // namespace esphome
+}  // namespace esphome::bluetooth_proxy
 
 #endif  // USE_ESP32

esphome/components/bluetooth_proxy/bluetooth_proxy.h

@@ -18,10 +18,10 @@
 #include <esp_bt.h>
 #include <esp_bt_device.h>
 
-namespace esphome {
-namespace bluetooth_proxy {
+namespace esphome::bluetooth_proxy {
 
 static const esp_err_t ESP_GATT_NOT_CONNECTED = -1;
+static const int DONE_SENDING_SERVICES = -2;
 
 using namespace esp32_ble_client;
 
@@ -51,7 +51,9 @@ enum BluetoothProxySubscriptionFlag : uint32_t {
 class BluetoothProxy : public esp32_ble_tracker::ESPBTDeviceListener, public Component {
  public:
   BluetoothProxy();
+#ifdef USE_ESP32_BLE_DEVICE
   bool parse_device(const esp32_ble_tracker::ESPBTDevice &device) override;
+#endif
   bool parse_devices(const esp32_ble::BLEScanResult *scan_results, size_t count) override;
   void dump_config() override;
   void setup() override;
@@ -129,7 +131,6 @@ class BluetoothProxy : public esp32_ble_tracker::ESPBTDeviceListener, public Com
   }
 
  protected:
-  void send_api_packet_(const esp32_ble_tracker::ESPBTDevice &device);
   void send_bluetooth_scanner_state_(esp32_ble_tracker::ScannerState state);
 
   BluetoothConnection *get_connection_(uint64_t address, bool reserve);
@@ -141,15 +142,21 @@ class BluetoothProxy : public esp32_ble_tracker::ESPBTDeviceListener, public Com
   // Group 2: Container types (typically 12 bytes on 32-bit)
   std::vector<BluetoothConnection *> connections_{};
 
-  // Group 3: 1-byte types grouped together
+  // BLE advertisement batching
+  std::vector<api::BluetoothLERawAdvertisement> advertisement_pool_;
+  std::unique_ptr<api::BluetoothLERawAdvertisementsResponse> response_;
+
+  // Group 3: 4-byte types
+  uint32_t last_advertisement_flush_time_{0};
+
+  // Group 4: 1-byte types grouped together
   bool active_;
-  bool raw_advertisements_{false};
+  uint8_t advertisement_count_{0};
   // 2 bytes used, 2 bytes padding
 };
 
 extern BluetoothProxy *global_bluetooth_proxy;  // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
 
-}  // namespace bluetooth_proxy
-}  // namespace esphome
+}  // namespace esphome::bluetooth_proxy
 
 #endif  // USE_ESP32

esphome/components/canbus/__init__.py

@@ -22,8 +22,7 @@ def validate_id(config):
     if CONF_CAN_ID in config:
         can_id = config[CONF_CAN_ID]
         id_ext = config[CONF_USE_EXTENDED_ID]
-        if not id_ext:
-            if can_id > 0x7FF:
+        if not id_ext and can_id > 0x7FF:
             raise cv.Invalid("Standard IDs must be 11 Bit (0x000-0x7ff / 0-2047)")
     return config
 

esphome/components/const/__init__.py

@@ -3,6 +3,12 @@
 CODEOWNERS = ["@esphome/core"]
 
 CONF_BYTE_ORDER = "byte_order"
+BYTE_ORDER_LITTLE = "little_endian"
+BYTE_ORDER_BIG = "big_endian"
+
+CONF_COLOR_DEPTH = "color_depth"
 CONF_DRAW_ROUNDING = "draw_rounding"
+CONF_ON_RECEIVE = "on_receive"
 CONF_ON_STATE_CHANGE = "on_state_change"
 CONF_REQUEST_HEADERS = "request_headers"
+CONF_USE_PSRAM = "use_psram"

esphome/components/dfrobot_sen0395/__init__.py

@@ -74,8 +74,7 @@ def range_segment_list(input):
     if isinstance(input, list):
         for list_item in input:
             if isinstance(list_item, list):
-                for item in list_item:
-                    flat_list.append(item)
+                flat_list.extend(list_item)
             else:
                 flat_list.append(list_item)
     else:

esphome/components/display_menu_base/__init__.py

@@ -17,6 +17,7 @@ from esphome.const import (
     CONF_MODE,
     CONF_NUMBER,
     CONF_ON_VALUE,
+    CONF_SWITCH,
     CONF_TEXT,
     CONF_TRIGGER_ID,
     CONF_TYPE,
@@ -33,7 +34,6 @@ CONF_LABEL = "label"
 CONF_MENU = "menu"
 CONF_BACK = "back"
 CONF_SELECT = "select"
-CONF_SWITCH = "switch"
 CONF_ON_TEXT = "on_text"
 CONF_OFF_TEXT = "off_text"
 CONF_VALUE_LAMBDA = "value_lambda"

esphome/components/esp32/__init__.py

@@ -31,6 +31,7 @@ from esphome.const import (
     KEY_TARGET_FRAMEWORK,
     KEY_TARGET_PLATFORM,
     PLATFORM_ESP32,
+    ThreadModel,
     __version__,
 )
 from esphome.core import CORE, HexInt, TimePeriod
@@ -39,7 +40,7 @@ import esphome.final_validate as fv
 from esphome.helpers import copy_file_if_changed, mkdir_p, write_file_if_changed
 from esphome.types import ConfigType
 
-from .boards import BOARDS
+from .boards import BOARDS, STANDARD_BOARDS
 from .const import (  # noqa
     KEY_BOARD,
     KEY_COMPONENTS,
@@ -309,19 +310,19 @@ def _format_framework_espidf_version(
 
 # The default/recommended arduino framework version
 #  - https://github.com/espressif/arduino-esp32/releases
-RECOMMENDED_ARDUINO_FRAMEWORK_VERSION = cv.Version(3, 1, 3)
+RECOMMENDED_ARDUINO_FRAMEWORK_VERSION = cv.Version(3, 2, 1)
 # The platform-espressif32 version to use for arduino frameworks
 #  - https://github.com/pioarduino/platform-espressif32/releases
-ARDUINO_PLATFORM_VERSION = cv.Version(53, 3, 13)
+ARDUINO_PLATFORM_VERSION = cv.Version(54, 3, 21)
 
 # The default/recommended esp-idf framework version
 #  - https://github.com/espressif/esp-idf/releases
 #  - https://api.registry.platformio.org/v3/packages/platformio/tool/framework-espidf
-RECOMMENDED_ESP_IDF_FRAMEWORK_VERSION = cv.Version(5, 3, 2)
+RECOMMENDED_ESP_IDF_FRAMEWORK_VERSION = cv.Version(5, 4, 2)
 # The platformio/espressif32 version to use for esp-idf frameworks
 #  - https://github.com/platformio/platform-espressif32/releases
 #  - https://api.registry.platformio.org/v3/packages/platformio/platform/espressif32
-ESP_IDF_PLATFORM_VERSION = cv.Version(53, 3, 13)
+ESP_IDF_PLATFORM_VERSION = cv.Version(54, 3, 21)
 
 # List based on https://registry.platformio.org/tools/platformio/framework-espidf/versions
 SUPPORTED_PLATFORMIO_ESP_IDF_5X = [
@@ -356,8 +357,8 @@ SUPPORTED_PIOARDUINO_ESP_IDF_5X = [
 def _arduino_check_versions(value):
     value = value.copy()
     lookups = {
-        "dev": (cv.Version(3, 1, 3), "https://github.com/espressif/arduino-esp32.git"),
-        "latest": (cv.Version(3, 1, 3), None),
+        "dev": (cv.Version(3, 2, 1), "https://github.com/espressif/arduino-esp32.git"),
+        "latest": (cv.Version(3, 2, 1), None),
         "recommended": (RECOMMENDED_ARDUINO_FRAMEWORK_VERSION, None),
     }
 
@@ -395,8 +396,8 @@ def _arduino_check_versions(value):
 def _esp_idf_check_versions(value):
     value = value.copy()
     lookups = {
-        "dev": (cv.Version(5, 3, 2), "https://github.com/espressif/esp-idf.git"),
-        "latest": (cv.Version(5, 3, 2), None),
+        "dev": (cv.Version(5, 4, 2), "https://github.com/espressif/esp-idf.git"),
+        "latest": (cv.Version(5, 2, 2), None),
         "recommended": (RECOMMENDED_ESP_IDF_FRAMEWORK_VERSION, None),
     }
 
@@ -487,25 +488,32 @@ def _platform_is_platformio(value):
 
 
 def _detect_variant(value):
-    board = value[CONF_BOARD]
-    if board in BOARDS:
-        variant = BOARDS[board][KEY_VARIANT]
-        if CONF_VARIANT in value and variant != value[CONF_VARIANT]:
+    board = value.get(CONF_BOARD)
+    variant = value.get(CONF_VARIANT)
+    if variant and board is None:
+        # If variant is set, we can derive the board from it
+        # variant has already been validated against the known set
+        value = value.copy()
+        value[CONF_BOARD] = STANDARD_BOARDS[variant]
+    elif board in BOARDS:
+        variant = variant or BOARDS[board][KEY_VARIANT]
+        if variant != BOARDS[board][KEY_VARIANT]:
             raise cv.Invalid(
                 f"Option '{CONF_VARIANT}' does not match selected board.",
                 path=[CONF_VARIANT],
             )
         value = value.copy()
         value[CONF_VARIANT] = variant
-    else:
-        if CONF_VARIANT not in value:
+    elif not variant:
         raise cv.Invalid(
             "This board is unknown, if you are sure you want to compile with this board selection, "
             f"override with option '{CONF_VARIANT}'",
             path=[CONF_BOARD],
         )
+    else:
         _LOGGER.warning(
-            "This board is unknown. Make sure the chosen chip component is correct.",
+            "This board is unknown; the specified variant '%s' will be used but this may not work as expected.",
+            variant,
         )
     return value
 
@@ -676,7 +684,7 @@ CONF_PARTITIONS = "partitions"
 CONFIG_SCHEMA = cv.All(
     cv.Schema(
         {
-            cv.Required(CONF_BOARD): cv.string_strict,
+            cv.Optional(CONF_BOARD): cv.string_strict,
             cv.Optional(CONF_CPU_FREQUENCY): cv.one_of(
                 *FULL_CPU_FREQUENCIES, upper=True
             ),
@@ -691,6 +699,7 @@ CONFIG_SCHEMA = cv.All(
     _detect_variant,
     _set_default_framework,
     set_core_data,
+    cv.has_at_least_one_key(CONF_BOARD, CONF_VARIANT),
 )
 
 
@@ -705,6 +714,7 @@ async def to_code(config):
     cg.add_define("ESPHOME_BOARD", config[CONF_BOARD])
     cg.add_build_flag(f"-DUSE_ESP32_VARIANT_{config[CONF_VARIANT]}")
     cg.add_define("ESPHOME_VARIANT", VARIANT_FRIENDLY[config[CONF_VARIANT]])
+    cg.add_define(ThreadModel.MULTI_ATOMICS)
 
     cg.add_platformio_option("lib_ldf_mode", "off")
     cg.add_platformio_option("lib_compat_mode", "strict")
@@ -943,8 +953,10 @@ def _write_idf_component_yml():
 
 # Called by writer.py
 def copy_files():
-    if CORE.using_arduino:
-        if "partitions.csv" not in CORE.data[KEY_ESP32][KEY_EXTRA_BUILD_FILES]:
+    if (
+        CORE.using_arduino
+        and "partitions.csv" not in CORE.data[KEY_ESP32][KEY_EXTRA_BUILD_FILES]
+    ):
         write_file_if_changed(
             CORE.relative_build_path("partitions.csv"),
             get_arduino_partition_csv(
@@ -970,7 +982,7 @@ def copy_files():
             __version__,
         )
 
-    for _, file in CORE.data[KEY_ESP32][KEY_EXTRA_BUILD_FILES].items():
+    for file in CORE.data[KEY_ESP32][KEY_EXTRA_BUILD_FILES].values():
         if file[KEY_PATH].startswith("http"):
             import requests
 

esphome/components/esp32/boards.py

@@ -2,13 +2,30 @@ from .const import (
     VARIANT_ESP32,
     VARIANT_ESP32C2,
     VARIANT_ESP32C3,
+    VARIANT_ESP32C5,
     VARIANT_ESP32C6,
     VARIANT_ESP32H2,
     VARIANT_ESP32P4,
     VARIANT_ESP32S2,
     VARIANT_ESP32S3,
+    VARIANTS,
 )
 
+STANDARD_BOARDS = {
+    VARIANT_ESP32: "esp32dev",
+    VARIANT_ESP32C2: "esp32-c2-devkitm-1",
+    VARIANT_ESP32C3: "esp32-c3-devkitm-1",
+    VARIANT_ESP32C5: "esp32-c5-devkitc-1",
+    VARIANT_ESP32C6: "esp32-c6-devkitm-1",
+    VARIANT_ESP32H2: "esp32-h2-devkitm-1",
+    VARIANT_ESP32P4: "esp32-p4-evboard",
+    VARIANT_ESP32S2: "esp32-s2-kaluga-1",
+    VARIANT_ESP32S3: "esp32-s3-devkitc-1",
+}
+
+# Make sure not missed here if a new variant added.
+assert all(v in STANDARD_BOARDS for v in VARIANTS)
+
 ESP32_BASE_PINS = {
     "TX": 1,
     "RX": 3,

esphome/components/esp32/gpio.cpp

@@ -114,7 +114,6 @@ void ESP32InternalGPIOPin::setup() {
   if (flags_ & gpio::FLAG_OUTPUT) {
     gpio_set_drive_capability(pin_, drive_strength_);
   }
-  ESP_LOGD(TAG, "rtc: %d", SOC_GPIO_SUPPORT_RTC_INDEPENDENT);
 }
 
 void ESP32InternalGPIOPin::pin_mode(gpio::Flags flags) {

esphome/components/esp32/post_build.py.script

@@ -1,77 +1,112 @@
-# Source https://github.com/letscontrolit/ESPEasy/pull/3845#issuecomment-1005864664
-
-# pylint: disable=E0602
-Import("env")  # noqa
+Import("env")
 
 import os
+import json
 import shutil
+import pathlib
+import itertools
 
-if os.environ.get("ESPHOME_USE_SUBPROCESS") is None:
+def merge_factory_bin(source, target, env):
+    """
+    Merges all flash sections into a single .factory.bin using esptool.
+    Attempts multiple methods to detect image layout: flasher_args.json, FLASH_EXTRA_IMAGES, fallback guesses.
+    """
+    firmware_name = os.path.basename(env.subst("$PROGNAME")) + ".bin"
+    build_dir = pathlib.Path(env.subst("$BUILD_DIR"))
+    firmware_path = build_dir / firmware_name
+    flash_size = env.BoardConfig().get("upload.flash_size", "4MB")
+    chip = env.BoardConfig().get("build.mcu", "esp32")
+
+    sections = []
+    flasher_args_path = build_dir / "flasher_args.json"
+
+    # 1. Try flasher_args.json
+    if flasher_args_path.exists():
         try:
-        import esptool
-    except ImportError:
-        env.Execute("$PYTHONEXE -m pip install esptool")
-else:
-    import subprocess
-from SCons.Script import ARGUMENTS
-
-# Copy over the default sdkconfig.
-from os import path
-
-if path.exists("./sdkconfig.defaults"):
-    os.makedirs(".temp", exist_ok=True)
-    shutil.copy("./sdkconfig.defaults", "./.temp/sdkconfig-esp32-idf")
-
-
-def esp32_create_combined_bin(source, target, env):
-    verbose = bool(int(ARGUMENTS.get("PIOVERBOSE", "0")))
-    if verbose:
-        print("Generating combined binary for serial flashing")
-    app_offset = 0x10000
-
-    new_file_name = env.subst("$BUILD_DIR/${PROGNAME}.factory.bin")
-    sections = env.subst(env.get("FLASH_EXTRA_IMAGES"))
-    firmware_name = env.subst("$BUILD_DIR/${PROGNAME}.bin")
-    chip = env.get("BOARD_MCU")
-    flash_size = env.BoardConfig().get("upload.flash_size")
-    cmd = [
-        "--chip",
-        chip,
-        "merge_bin",
-        "-o",
-        new_file_name,
-        "--flash_size",
-        flash_size,
-    ]
-    if verbose:
-        print("    Offset | File")
-    for section in sections:
-        sect_adr, sect_file = section.split(" ", 1)
-        if verbose:
-            print(f" -  {sect_adr} | {sect_file}")
-        cmd += [sect_adr, sect_file]
-
-    cmd += [hex(app_offset), firmware_name]
-
-    if verbose:
-        print(f" - {hex(app_offset)} | {firmware_name}")
-        print()
-        print(f"Using esptool.py arguments: {' '.join(cmd)}")
-        print()
-
-    if os.environ.get("ESPHOME_USE_SUBPROCESS") is None:
-        esptool.main(cmd)
+            with flasher_args_path.open() as f:
+                flash_data = json.load(f)
+            for addr, fname in sorted(flash_data["flash_files"].items(), key=lambda kv: int(kv[0], 16)):
+                file_path = pathlib.Path(fname)
+                if file_path.exists():
+                    sections.append((addr, str(file_path)))
                 else:
-        subprocess.run(["esptool.py", *cmd])
+                    print(f"Info: {file_path.name} not found - skipping")
+        except Exception as e:
+            print(f"Warning: Failed to parse flasher_args.json - {e}")
 
+    # 2. Try FLASH_EXTRA_IMAGES if flasher_args.json failed or was empty
+    if not sections:
+        flash_images = env.get("FLASH_EXTRA_IMAGES")
+        if flash_images:
+            print("Using FLASH_EXTRA_IMAGES from PlatformIO environment")
+            # flatten any nested lists
+            flat = list(itertools.chain.from_iterable(
+                x if isinstance(x, (list, tuple)) else [x] for x in flash_images
+            ))
+            entries = [env.subst(x) for x in flat]
+            for i in range(0, len(entries) - 1, 2):
+                addr, fname = entries[i], entries[i + 1]
+                if isinstance(fname, (list, tuple)):
+                    print(f"Warning: Skipping malformed FLASH_EXTRA_IMAGES entry: {fname}")
+                    continue
+                file_path = pathlib.Path(str(fname))
+                if file_path.exists():
+                    sections.append((addr, str(file_path)))
+                else:
+                    print(f"Info: {file_path.name} not found — skipping")
+
+    # 3. Final fallback: guess standard image locations
+    if not sections:
+        print("Fallback: guessing legacy image paths")
+        guesses = [
+            ("0x0", build_dir / "bootloader" / "bootloader.bin"),
+            ("0x8000", build_dir / "partition_table" / "partition-table.bin"),
+            ("0xe000", build_dir / "ota_data_initial.bin"),
+            ("0x10000", firmware_path)
+        ]
+        for addr, file_path in guesses:
+            if file_path.exists():
+                sections.append((addr, str(file_path)))
+            else:
+                print(f"Info: {file_path.name} not found — skipping")
+
+    # If no valid sections found, skip merge
+    if not sections:
+        print("No valid flash sections found — skipping .factory.bin creation.")
+        return
+
+    output_path = firmware_path.with_suffix(".factory.bin")
+    cmd = [
+        "--chip", chip,
+        "merge_bin",
+        "--flash_size", flash_size,
+        "--output", str(output_path)
+    ]
+    for addr, file_path in sections:
+        cmd += [addr, file_path]
+
+    print(f"Merging binaries into {output_path}")
+    result = env.Execute(
+        env.VerboseAction(
+            f"{env.subst('$PYTHONEXE')} -m esptool " + " ".join(cmd),
+            "Merging binaries with esptool"
+        )
+    )
+
+    if result == 0:
+        print(f"Successfully created {output_path}")
+    else:
+        print(f"Error: esptool merge_bin failed with code {result}")
 
 def esp32_copy_ota_bin(source, target, env):
+    """
+    Copy the main firmware to a .ota.bin file for compatibility with ESPHome OTA tools.
+    """
     firmware_name = env.subst("$BUILD_DIR/${PROGNAME}.bin")
     new_file_name = env.subst("$BUILD_DIR/${PROGNAME}.ota.bin")
-
     shutil.copyfile(firmware_name, new_file_name)
+    print(f"Copied firmware to {new_file_name}")
 
-
-# pylint: disable=E0602
-env.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", esp32_create_combined_bin)  # noqa
-env.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", esp32_copy_ota_bin)  # noqa
+# Run merge first, then ota copy second
+env.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", merge_factory_bin)
+env.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", esp32_copy_ota_bin)

esphome/components/esp32_ble/ble.cpp

@@ -19,8 +19,7 @@
 #include <esp32-hal-bt.h>
 #endif
 
-namespace esphome {
-namespace esp32_ble {
+namespace esphome::esp32_ble {
 
 static const char *const TAG = "esp32_ble";
 
@@ -538,7 +537,6 @@ uint64_t ble_addr_to_uint64(const esp_bd_addr_t address) {
 
 ESP32BLE *global_ble = nullptr;  // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
 
-}  // namespace esp32_ble
-}  // namespace esphome
+}  // namespace esphome::esp32_ble
 
 #endif

esphome/components/esp32_ble/ble.h

@@ -21,8 +21,7 @@
 #include <esp_gattc_api.h>
 #include <esp_gatts_api.h>
 
-namespace esphome {
-namespace esp32_ble {
+namespace esphome::esp32_ble {
 
 // Maximum number of BLE scan results to buffer
 // Sized to handle bursts of advertisements while allowing for processing delays
@@ -191,7 +190,6 @@ template<typename... Ts> class BLEDisableAction : public Action<Ts...> {
   void play(Ts... x) override { global_ble->disable(); }
 };
 
-}  // namespace esp32_ble
-}  // namespace esphome
+}  // namespace esphome::esp32_ble
 
 #endif

esphome/components/esp32_ble/ble_advertising.cpp

@@ -8,8 +8,7 @@
 #include "esphome/core/log.h"
 #include "esphome/core/application.h"
 
-namespace esphome {
-namespace esp32_ble {
+namespace esphome::esp32_ble {
 
 static const char *const TAG = "esp32_ble.advertising";
 
@@ -160,7 +159,6 @@ void BLEAdvertising::register_raw_advertisement_callback(std::function<void(bool
   this->raw_advertisements_callbacks_.push_back(std::move(callback));
 }
 
-}  // namespace esp32_ble
-}  // namespace esphome
+}  // namespace esphome::esp32_ble
 
 #endif

esphome/components/esp32_ble/ble_advertising.h

@@ -10,8 +10,7 @@
 #include <esp_gap_ble_api.h>
 #include <esp_gatts_api.h>
 
-namespace esphome {
-namespace esp32_ble {
+namespace esphome::esp32_ble {
 
 using raw_adv_data_t = struct {
   uint8_t *data;
@@ -55,7 +54,6 @@ class BLEAdvertising {
   int8_t current_adv_index_{-1};  // -1 means standard scan response
 };
 
-}  // namespace esp32_ble
-}  // namespace esphome
+}  // namespace esphome::esp32_ble
 
 #endif

esphome/components/esp32_ble/ble_event.h

@@ -11,8 +11,7 @@
 
 #include "ble_scan_result.h"
 
-namespace esphome {
-namespace esp32_ble {
+namespace esphome::esp32_ble {
 
 // Compile-time verification that ESP-IDF scan complete events only contain a status field
 // This ensures our reinterpret_cast in ble.cpp is safe
@@ -395,7 +394,6 @@ static_assert(sizeof(esp_ble_sec_t) <= 73, "esp_ble_sec_t is larger than BLEScan
 
 // BLEEvent total size: 84 bytes (80 byte union + 1 byte type + 3 bytes padding)
 
-}  // namespace esp32_ble
-}  // namespace esphome
+}  // namespace esphome::esp32_ble
 
 #endif

esphome/components/esp32_ble/ble_scan_result.h

@@ -4,8 +4,7 @@
 
 #include <esp_gap_ble_api.h>
 
-namespace esphome {
-namespace esp32_ble {
+namespace esphome::esp32_ble {
 
 // Structure for BLE scan results - only fields we actually use
 struct __attribute__((packed)) BLEScanResult {
@@ -18,7 +17,6 @@ struct __attribute__((packed)) BLEScanResult {
   uint8_t search_evt;
 };  // ~73 bytes vs ~400 bytes for full esp_ble_gap_cb_param_t
 
-}  // namespace esp32_ble
-}  // namespace esphome
+}  // namespace esphome::esp32_ble
 
 #endif

esphome/components/esp32_ble/ble_uuid.cpp

@@ -7,8 +7,7 @@
 #include <cinttypes>
 #include "esphome/core/log.h"
 
-namespace esphome {
-namespace esp32_ble {
+namespace esphome::esp32_ble {
 
 static const char *const TAG = "esp32_ble";
 
@@ -189,7 +188,6 @@ std::string ESPBTUUID::to_string() const {
   return "";
 }
 
-}  // namespace esp32_ble
-}  // namespace esphome
+}  // namespace esphome::esp32_ble
 
 #endif

esphome/components/esp32_ble/ble_uuid.h

@@ -8,8 +8,7 @@
 #include <string>
 #include <esp_bt_defs.h>
 
-namespace esphome {
-namespace esp32_ble {
+namespace esphome::esp32_ble {
 
 class ESPBTUUID {
  public:
@@ -41,7 +40,6 @@ class ESPBTUUID {
   esp_bt_uuid_t uuid_;
 };
 
-}  // namespace esp32_ble
-}  // namespace esphome
+}  // namespace esphome::esp32_ble
 
 #endif

esphome/components/esp32_ble_client/ble_client_base.cpp

@@ -105,6 +105,7 @@ void BLEClientBase::dump_config() {
   }
 }
 
+#ifdef USE_ESP32_BLE_DEVICE
 bool BLEClientBase::parse_device(const espbt::ESPBTDevice &device) {
   if (!this->auto_connect_)
     return false;
@@ -122,6 +123,7 @@ bool BLEClientBase::parse_device(const espbt::ESPBTDevice &device) {
   this->remote_addr_type_ = device.get_address_type();
   return true;
 }
+#endif
 
 void BLEClientBase::connect() {
   ESP_LOGI(TAG, "[%d] [%s] 0x%02x Attempting BLE connection", this->connection_index_, this->address_str_.c_str(),

esphome/components/esp32_ble_client/ble_client_base.h

@@ -31,7 +31,9 @@ class BLEClientBase : public espbt::ESPBTClient, public Component {
   void dump_config() override;
 
   void run_later(std::function<void()> &&f);  // NOLINT
+#ifdef USE_ESP32_BLE_DEVICE
   bool parse_device(const espbt::ESPBTDevice &device) override;
+#endif
   void on_scan_end() override {}
   bool gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_if,
                            esp_ble_gattc_cb_param_t *param) override;

esphome/components/esp32_ble_server/__init__.py

@@ -140,8 +140,11 @@ VALUE_TYPES = {
 
 
 def validate_char_on_write(char_config):
-    if CONF_ON_WRITE in char_config:
-        if not char_config[CONF_WRITE] and not char_config[CONF_WRITE_NO_RESPONSE]:
+    if (
+        CONF_ON_WRITE in char_config
+        and not char_config[CONF_WRITE]
+        and not char_config[CONF_WRITE_NO_RESPONSE]
+    ):
         raise cv.Invalid(
             f"{CONF_ON_WRITE} requires the {CONF_WRITE} or {CONF_WRITE_NO_RESPONSE} property to be set"
         )
@@ -149,8 +152,7 @@ def validate_char_on_write(char_config):
 
 
 def validate_descriptor(desc_config):
-    if CONF_ON_WRITE in desc_config:
-        if not desc_config[CONF_WRITE]:
+    if CONF_ON_WRITE in desc_config and not desc_config[CONF_WRITE]:
         raise cv.Invalid(
             f"{CONF_ON_WRITE} requires the {CONF_WRITE} property to be set"
         )

esphome/components/esp32_ble_tracker/__init__.py

@@ -30,7 +30,9 @@ from esphome.const import (
     CONF_SERVICE_UUID,
     CONF_TRIGGER_ID,
 )
-from esphome.core import CORE
+from esphome.core import CORE, coroutine_with_priority
+from esphome.enum import StrEnum
+from esphome.types import ConfigType
 
 AUTO_LOAD = ["esp32_ble"]
 DEPENDENCIES = ["esp32"]
@@ -50,6 +52,25 @@ IDF_MAX_CONNECTIONS = 9
 
 _LOGGER = logging.getLogger(__name__)
 
+
+# Enum for BLE features
+class BLEFeatures(StrEnum):
+    ESP_BT_DEVICE = "ESP_BT_DEVICE"
+
+
+# Set to track which features are needed by components
+_required_features: set[BLEFeatures] = set()
+
+
+def register_ble_features(features: set[BLEFeatures]) -> None:
+    """Register BLE features that a component needs.
+
+    Args:
+        features: Set of BLEFeatures enum members
+    """
+    _required_features.update(features)
+
+
 esp32_ble_tracker_ns = cg.esphome_ns.namespace("esp32_ble_tracker")
 ESP32BLETracker = esp32_ble_tracker_ns.class_(
     "ESP32BLETracker",
@@ -277,12 +298,19 @@ async def to_code(config):
     cg.add(var.set_scan_window(int(params[CONF_WINDOW].total_milliseconds / 0.625)))
     cg.add(var.set_scan_active(params[CONF_ACTIVE]))
     cg.add(var.set_scan_continuous(params[CONF_CONTINUOUS]))
+
+    # Register ESP_BT_DEVICE feature if any of the automation triggers are used
+    if (
+        config.get(CONF_ON_BLE_ADVERTISE)
+        or config.get(CONF_ON_BLE_SERVICE_DATA_ADVERTISE)
+        or config.get(CONF_ON_BLE_MANUFACTURER_DATA_ADVERTISE)
+    ):
+        register_ble_features({BLEFeatures.ESP_BT_DEVICE})
+
     for conf in config.get(CONF_ON_BLE_ADVERTISE, []):
         trigger = cg.new_Pvariable(conf[CONF_TRIGGER_ID], var)
         if CONF_MAC_ADDRESS in conf:
-            addr_list = []
-            for it in conf[CONF_MAC_ADDRESS]:
-                addr_list.append(it.as_hex)
+            addr_list = [it.as_hex for it in conf[CONF_MAC_ADDRESS]]
             cg.add(trigger.set_addresses(addr_list))
         await automation.build_automation(trigger, [(ESPBTDeviceConstRef, "x")], conf)
     for conf in config.get(CONF_ON_BLE_SERVICE_DATA_ADVERTISE, []):
@@ -334,10 +362,23 @@ async def to_code(config):
 
     cg.add_define("USE_OTA_STATE_CALLBACK")  # To be notified when an OTA update starts
     cg.add_define("USE_ESP32_BLE_CLIENT")
+
+    CORE.add_job(_add_ble_features)
+
     if config.get(CONF_SOFTWARE_COEXISTENCE):
         cg.add_define("USE_ESP32_BLE_SOFTWARE_COEXISTENCE")
 
 
+# This needs to be run as a job with very low priority so that all components have
+# chance to call register_ble_tracker and register_client before the list is checked
+# and added to the global defines list.
+@coroutine_with_priority(-1000)
+async def _add_ble_features():
+    # Add feature-specific defines based on what's needed
+    if BLEFeatures.ESP_BT_DEVICE in _required_features:
+        cg.add_define("USE_ESP32_BLE_DEVICE")
+
+
 ESP32_BLE_START_SCAN_ACTION_SCHEMA = cv.Schema(
     {
         cv.GenerateID(): cv.use_id(ESP32BLETracker),
@@ -382,13 +423,43 @@ async def esp32_ble_tracker_stop_scan_action_to_code(
     return var
 
 
-async def register_ble_device(var, config):
+async def register_ble_device(
+    var: cg.SafeExpType, config: ConfigType
+) -> cg.SafeExpType:
+    register_ble_features({BLEFeatures.ESP_BT_DEVICE})
     paren = await cg.get_variable(config[CONF_ESP32_BLE_ID])
     cg.add(paren.register_listener(var))
     return var
 
 
-async def register_client(var, config):
+async def register_client(var: cg.SafeExpType, config: ConfigType) -> cg.SafeExpType:
+    register_ble_features({BLEFeatures.ESP_BT_DEVICE})
+    paren = await cg.get_variable(config[CONF_ESP32_BLE_ID])
+    cg.add(paren.register_client(var))
+    return var
+
+
+async def register_raw_ble_device(
+    var: cg.SafeExpType, config: ConfigType
+) -> cg.SafeExpType:
+    """Register a BLE device listener that only needs raw advertisement data.
+
+    This does NOT register the ESP_BT_DEVICE feature, meaning ESPBTDevice
+    will not be compiled in if this is the only registration method used.
+    """
+    paren = await cg.get_variable(config[CONF_ESP32_BLE_ID])
+    cg.add(paren.register_listener(var))
+    return var
+
+
+async def register_raw_client(
+    var: cg.SafeExpType, config: ConfigType
+) -> cg.SafeExpType:
+    """Register a BLE client that only needs raw advertisement data.
+
+    This does NOT register the ESP_BT_DEVICE feature, meaning ESPBTDevice
+    will not be compiled in if this is the only registration method used.
+    """
     paren = await cg.get_variable(config[CONF_ESP32_BLE_ID])
     cg.add(paren.register_client(var))
     return var

esphome/components/esp32_ble_tracker/automation.h

@@ -5,8 +5,8 @@
 
 #ifdef USE_ESP32
 
-namespace esphome {
-namespace esp32_ble_tracker {
+namespace esphome::esp32_ble_tracker {
+#ifdef USE_ESP32_BLE_DEVICE
 class ESPBTAdvertiseTrigger : public Trigger<const ESPBTDevice &>, public ESPBTDeviceListener {
  public:
   explicit ESPBTAdvertiseTrigger(ESP32BLETracker *parent) { parent->register_listener(this); }
@@ -87,6 +87,7 @@ class BLEEndOfScanTrigger : public Trigger<>, public ESPBTDeviceListener {
   bool parse_device(const ESPBTDevice &device) override { return false; }
   void on_scan_end() override { this->trigger(); }
 };
+#endif  // USE_ESP32_BLE_DEVICE
 
 template<typename... Ts> class ESP32BLEStartScanAction : public Action<Ts...> {
  public:
@@ -106,7 +107,6 @@ template<typename... Ts> class ESP32BLEStopScanAction : public Action<Ts...>, pu
   void play(Ts... x) override { this->parent_->stop_scan(); }
 };
 
-}  // namespace esp32_ble_tracker
-}  // namespace esphome
+}  // namespace esphome::esp32_ble_tracker
 
 #endif

esphome/components/esp32_ble_tracker/esp32_ble_tracker.cpp

@@ -35,8 +35,7 @@
 // bt_trace.h
 #undef TAG
 
-namespace esphome {
-namespace esp32_ble_tracker {
+namespace esphome::esp32_ble_tracker {
 
 static const char *const TAG = "esp32_ble_tracker";
 
@@ -128,19 +127,26 @@ void ESP32BLETracker::loop() {
     uint8_t write_idx = this->ring_write_index_.load(std::memory_order_acquire);
 
     while (read_idx != write_idx) {
-      // Process one result at a time directly from ring buffer
-      BLEScanResult &scan_result = this->scan_ring_buffer_[read_idx];
+      // Calculate how many contiguous results we can process in one batch
+      // If write > read: process all results from read to write
+      // If write <= read (wraparound): process from read to end of buffer first
+      size_t batch_size = (write_idx > read_idx) ? (write_idx - read_idx) : (SCAN_RESULT_BUFFER_SIZE - read_idx);
 
+      // Process the batch for raw advertisements
       if (this->raw_advertisements_) {
         for (auto *listener : this->listeners_) {
-          listener->parse_devices(&scan_result, 1);
+          listener->parse_devices(&this->scan_ring_buffer_[read_idx], batch_size);
         }
         for (auto *client : this->clients_) {
-          client->parse_devices(&scan_result, 1);
+          client->parse_devices(&this->scan_ring_buffer_[read_idx], batch_size);
         }
       }
 
+      // Process individual results for parsed advertisements
       if (this->parse_advertisements_) {
+#ifdef USE_ESP32_BLE_DEVICE
+        for (size_t i = 0; i < batch_size; i++) {
+          BLEScanResult &scan_result = this->scan_ring_buffer_[read_idx + i];
           ESPBTDevice device;
           device.parse_scan_rst(scan_result);
 
@@ -163,9 +169,11 @@ void ESP32BLETracker::loop() {
             this->print_bt_device_info(device);
           }
         }
+#endif  // USE_ESP32_BLE_DEVICE
+      }
 
-      // Move to next entry in ring buffer
-      read_idx = (read_idx + 1) % SCAN_RESULT_BUFFER_SIZE;
+      // Update read index for entire batch
+      read_idx = (read_idx + batch_size) % SCAN_RESULT_BUFFER_SIZE;
 
       // Store with release to ensure reads complete before index update
       this->ring_read_index_.store(read_idx, std::memory_order_release);
@@ -511,6 +519,7 @@ void ESP32BLETracker::set_scanner_state_(ScannerState state) {
   this->scanner_state_callbacks_.call(state);
 }
 
+#ifdef USE_ESP32_BLE_DEVICE
 ESPBLEiBeacon::ESPBLEiBeacon(const uint8_t *data) { memcpy(&this->beacon_data_, data, sizeof(beacon_data_)); }
 optional<ESPBLEiBeacon> ESPBLEiBeacon::from_manufacturer_data(const ServiceData &data) {
   if (!data.uuid.contains(0x4C, 0x00))
@@ -751,13 +760,16 @@ void ESPBTDevice::parse_adv_(const uint8_t *payload, uint8_t len) {
     }
   }
 }
+
 std::string ESPBTDevice::address_str() const {
   char mac[24];
   snprintf(mac, sizeof(mac), "%02X:%02X:%02X:%02X:%02X:%02X", this->address_[0], this->address_[1], this->address_[2],
            this->address_[3], this->address_[4], this->address_[5]);
   return mac;
 }
+
 uint64_t ESPBTDevice::address_uint64() const { return esp32_ble::ble_addr_to_uint64(this->address_); }
+#endif  // USE_ESP32_BLE_DEVICE
 
 void ESP32BLETracker::dump_config() {
   ESP_LOGCONFIG(TAG, "BLE Tracker:");
@@ -796,6 +808,7 @@ void ESP32BLETracker::dump_config() {
   }
 }
 
+#ifdef USE_ESP32_BLE_DEVICE
 void ESP32BLETracker::print_bt_device_info(const ESPBTDevice &device) {
   const uint64_t address = device.address_uint64();
   for (auto &disc : this->already_discovered_) {
@@ -866,8 +879,8 @@ bool ESPBTDevice::resolve_irk(const uint8_t *irk) const {
   return ecb_ciphertext[15] == (addr64 & 0xff) && ecb_ciphertext[14] == ((addr64 >> 8) & 0xff) &&
          ecb_ciphertext[13] == ((addr64 >> 16) & 0xff);
 }
+#endif  // USE_ESP32_BLE_DEVICE
 
-}  // namespace esp32_ble_tracker
-}  // namespace esphome
+}  // namespace esphome::esp32_ble_tracker
 
-#endif
+#endif  // USE_ESP32

esphome/components/esp32_ble_tracker/esp32_ble_tracker.h

@@ -22,8 +22,7 @@
 #include "esphome/components/esp32_ble/ble.h"
 #include "esphome/components/esp32_ble/ble_uuid.h"
 
-namespace esphome {
-namespace esp32_ble_tracker {
+namespace esphome::esp32_ble_tracker {
 
 using namespace esp32_ble;
 
@@ -39,6 +38,7 @@ struct ServiceData {
   adv_data_t data;
 };
 
+#ifdef USE_ESP32_BLE_DEVICE
 class ESPBLEiBeacon {
  public:
   ESPBLEiBeacon() { memset(&this->beacon_data_, 0, sizeof(this->beacon_data_)); }
@@ -116,13 +116,16 @@ class ESPBTDevice {
   std::vector<ServiceData> service_datas_{};
   const BLEScanResult *scan_result_{nullptr};
 };
+#endif  // USE_ESP32_BLE_DEVICE
 
 class ESP32BLETracker;
 
 class ESPBTDeviceListener {
  public:
   virtual void on_scan_end() {}
+#ifdef USE_ESP32_BLE_DEVICE
   virtual bool parse_device(const ESPBTDevice &device) = 0;
+#endif
   virtual bool parse_devices(const BLEScanResult *scan_results, size_t count) { return false; };
   virtual AdvertisementParserType get_advertisement_parser_type() {
     return AdvertisementParserType::PARSED_ADVERTISEMENTS;
@@ -237,7 +240,9 @@ class ESP32BLETracker : public Component,
   void register_client(ESPBTClient *client);
   void recalculate_advertisement_parser_types();
 
+#ifdef USE_ESP32_BLE_DEVICE
   void print_bt_device_info(const ESPBTDevice &device);
+#endif
 
   void start_scan();
   void stop_scan();
@@ -315,7 +320,6 @@ class ESP32BLETracker : public Component,
 // NOLINTNEXTLINE
 extern ESP32BLETracker *global_esp32_ble_tracker;
 
-}  // namespace esp32_ble_tracker
-}  // namespace esphome
+}  // namespace esphome::esp32_ble_tracker
 
 #endif

esphome/components/esp32_camera/__init__.py

@@ -119,6 +119,12 @@ ENUM_SPECIAL_EFFECT = {
     "SEPIA": ESP32SpecialEffect.ESP32_SPECIAL_EFFECT_SEPIA,
 }
 
+camera_fb_location_t = cg.global_ns.enum("camera_fb_location_t")
+ENUM_FB_LOCATION = {
+    "PSRAM": cg.global_ns.CAMERA_FB_IN_PSRAM,
+    "DRAM": cg.global_ns.CAMERA_FB_IN_DRAM,
+}
+
 # pin assignment
 CONF_HREF_PIN = "href_pin"
 CONF_PIXEL_CLOCK_PIN = "pixel_clock_pin"
@@ -149,6 +155,7 @@ CONF_MAX_FRAMERATE = "max_framerate"
 CONF_IDLE_FRAMERATE = "idle_framerate"
 # frame buffer
 CONF_FRAME_BUFFER_COUNT = "frame_buffer_count"
+CONF_FRAME_BUFFER_LOCATION = "frame_buffer_location"
 
 # stream trigger
 CONF_ON_STREAM_START = "on_stream_start"
@@ -230,6 +237,9 @@ CONFIG_SCHEMA = cv.All(
                 cv.framerate, cv.Range(min=0, max=1)
             ),
             cv.Optional(CONF_FRAME_BUFFER_COUNT, default=1): cv.int_range(min=1, max=2),
+            cv.Optional(CONF_FRAME_BUFFER_LOCATION, default="PSRAM"): cv.enum(
+                ENUM_FB_LOCATION, upper=True
+            ),
             cv.Optional(CONF_ON_STREAM_START): automation.validate_automation(
                 {
                     cv.GenerateID(CONF_TRIGGER_ID): cv.declare_id(
@@ -301,6 +311,7 @@ SETTERS = {
     CONF_WB_MODE: "set_wb_mode",
     # test pattern
     CONF_TEST_PATTERN: "set_test_pattern",
+    CONF_FRAME_BUFFER_LOCATION: "set_frame_buffer_location",
 }
 
 
@@ -328,6 +339,7 @@ async def to_code(config):
     else:
         cg.add(var.set_idle_update_interval(1000 / config[CONF_IDLE_FRAMERATE]))
     cg.add(var.set_frame_buffer_count(config[CONF_FRAME_BUFFER_COUNT]))
+    cg.add(var.set_frame_buffer_location(config[CONF_FRAME_BUFFER_LOCATION]))
     cg.add(var.set_frame_size(config[CONF_RESOLUTION]))
 
     cg.add_define("USE_CAMERA")

esphome/components/esp32_camera/esp32_camera.cpp

@@ -133,6 +133,7 @@ void ESP32Camera::dump_config() {
   ESP_LOGCONFIG(TAG,
                 "  JPEG Quality: %u\n"
                 "  Framebuffer Count: %u\n"
+                "  Framebuffer Location: %s\n"
                 "  Contrast: %d\n"
                 "  Brightness: %d\n"
                 "  Saturation: %d\n"
@@ -140,8 +141,9 @@ void ESP32Camera::dump_config() {
                 "  Horizontal Mirror: %s\n"
                 "  Special Effect: %u\n"
                 "  White Balance Mode: %u",
-                st.quality, conf.fb_count, st.contrast, st.brightness, st.saturation, ONOFF(st.vflip),
-                ONOFF(st.hmirror), st.special_effect, st.wb_mode);
+                st.quality, conf.fb_count, this->config_.fb_location == CAMERA_FB_IN_PSRAM ? "PSRAM" : "DRAM",
+                st.contrast, st.brightness, st.saturation, ONOFF(st.vflip), ONOFF(st.hmirror), st.special_effect,
+                st.wb_mode);
   // ESP_LOGCONFIG(TAG, "  Auto White Balance: %u", st.awb);
   // ESP_LOGCONFIG(TAG, "  Auto White Balance Gain: %u", st.awb_gain);
   ESP_LOGCONFIG(TAG,
@@ -350,6 +352,9 @@ void ESP32Camera::set_frame_buffer_count(uint8_t fb_count) {
   this->config_.fb_count = fb_count;
   this->set_frame_buffer_mode(fb_count > 1 ? CAMERA_GRAB_LATEST : CAMERA_GRAB_WHEN_EMPTY);
 }
+void ESP32Camera::set_frame_buffer_location(camera_fb_location_t fb_location) {
+  this->config_.fb_location = fb_location;
+}
 
 /* ---------------- public API (specific) ---------------- */
 void ESP32Camera::add_image_callback(std::function<void(std::shared_ptr<camera::CameraImage>)> &&callback) {

esphome/components/esp32_camera/esp32_camera.h

@@ -152,6 +152,7 @@ class ESP32Camera : public camera::Camera {
   /* -- frame buffer */
   void set_frame_buffer_mode(camera_grab_mode_t mode);
   void set_frame_buffer_count(uint8_t fb_count);
+  void set_frame_buffer_location(camera_fb_location_t fb_location);
 
   /* public API (derivated) */
   void setup() override;