Compare commits

...

50 Commits

Author SHA1 Message Date
Aidan Timson b5da32f324 Address E2E review findings 2026-07-13 11:26:08 +01:00
Aidan Timson 1cbc60fe86 Limit gallery shell error filtering 2026-07-13 11:04:35 +01:00
Aidan Timson 41cba54094 Keep app shell error checks strict 2026-07-13 11:04:11 +01:00
Aidan Timson a9dc02952b Restore real demo sidebar clicks 2026-07-13 11:03:46 +01:00
Aidan Timson 60c181d5c8 Validate E2E worker overrides 2026-07-13 11:03:20 +01:00
Aidan Timson 86e0536de0 Avoid concurrent cold E2E builds 2026-07-13 09:48:30 +01:00
Aidan Timson c427acffe2 Stabilize demo sidebar navigation test 2026-07-13 09:48:10 +01:00
Aidan Timson 12d4b7207e Clarify local E2E server workflow 2026-07-13 09:32:29 +01:00
Aidan Timson 00b297b0fe Require helper reuse in test changes 2026-07-13 09:30:11 +01:00
Aidan Timson 307bb844fe Keep page error filtering internal 2026-07-13 09:27:49 +01:00
Aidan Timson 5291c37c88 Remove duplicate smoke test parallel mode 2026-07-13 09:27:12 +01:00
Aidan Timson c2e9c5e257 Remove unused route project filtering 2026-07-13 09:26:29 +01:00
Aidan Timson 1ce87aa5d4 Remove ineffective gallery test mode 2026-07-13 09:25:50 +01:00
Aidan Timson 086581d280 Restore demo page error tracking 2026-07-13 09:25:27 +01:00
Aidan Timson 0774f352fb Share parallel e2e smoke registration 2026-07-13 09:05:07 +01:00
Aidan Timson 5ee8f7b4a4 Restore concurrency 2026-07-13 09:05:07 +01:00
Aidan Timson 52788d7f71 Restore concurrency 2026-07-13 09:05:07 +01:00
Aidan Timson 7406cc8cea Avoid duplicate gallery readiness wait 2026-07-13 09:05:07 +01:00
Aidan Timson 2e5731ff82 Run app e2e readiness waits concurrently 2026-07-13 09:05:07 +01:00
Aidan Timson d78e9890bc Avoid slow gallery alert count wait 2026-07-13 09:05:07 +01:00
Aidan Timson 3eaf64bc97 Avoid duplicate gallery page waits 2026-07-13 09:05:07 +01:00
Aidan Timson 1595e3bffc Consolidate config link e2e checks 2026-07-13 09:05:07 +01:00
Aidan Timson df0969a4f5 Run e2e sidebar checks concurrently 2026-07-13 09:05:07 +01:00
Aidan Timson e3c917b65d Remove redundant e2e test cases 2026-07-13 09:05:07 +01:00
Aidan Timson d2ed98613b Use Playwright image for e2e tests 2026-07-13 09:05:07 +01:00
Aidan Timson 6185f5033f Remove local sharded e2e runner 2026-07-13 09:05:07 +01:00
Aidan Timson 78887965ed 4/4/2 sharding 2026-07-13 09:05:07 +01:00
Aidan Timson 2a5e4d5c88 Build demo e2e artifact modern only 2026-07-13 09:05:07 +01:00
Aidan Timson bbfe5b6be8 Build e2e app artifact modern only 2026-07-13 09:05:07 +01:00
Aidan Timson 5c0ef1c653 Fix e2e benchmark concurrency 2026-07-13 09:05:07 +01:00
Aidan Timson 7b64046a2f Use test mode for e2e builds 2026-07-13 09:05:07 +01:00
Aidan Timson e760df5a27 Queue e2e benchmark runs 2026-07-13 09:05:07 +01:00
Aidan Timson 8bafdbcec6 Add local sharded e2e runner 2026-07-13 09:05:07 +01:00
Aidan Timson a20745c274 6,4,6 2026-07-13 09:05:07 +01:00
Aidan Timson 598e0935f6 rm 2026-07-13 09:05:07 +01:00
Aidan Timson a9d560096a Shard demo (2) and gallery (3) 2026-07-13 09:05:07 +01:00
Aidan Timson f1e89688b9 Double sharding count 2026-07-13 09:05:07 +01:00
Aidan Timson 42c40fccdb Format 2026-07-13 09:05:07 +01:00
Aidan Timson b2d2eda50f Fix name 2026-07-13 09:05:07 +01:00
Aidan Timson d37a07b1b4 Fold e2e-local into report job 2026-07-13 09:05:07 +01:00
Aidan Timson 0a74b515ff Shard e2e tests 2026-07-13 09:05:07 +01:00
Aidan Timson e358d1478a Enable workers and full paralelism to tests. Use 60% for local (4 in my case) and 1 for CI 2026-07-13 09:05:07 +01:00
Aidan Timson ac4a4dcd70 Move app e2e helpers into src 2026-07-13 09:05:07 +01:00
Aidan Timson bd70df459b Consolidate app e2e smoke data 2026-07-13 09:05:07 +01:00
Aidan Timson da6f712d2c Split e2e helpers by suite 2026-07-13 09:05:07 +01:00
Aidan Timson 160eaaac4f Tighten app e2e smoke timeouts 2026-07-13 09:05:07 +01:00
Aidan Timson c70498c057 Move app e2e smoke data out of spec 2026-07-13 09:05:07 +01:00
Aidan Timson 1b9841185c Consolidate app route e2e helpers 2026-07-13 09:05:07 +01:00
Aidan Timson d1233489ab Make suites run as parallel jobs in CI. Merge reports after 2026-07-13 09:05:07 +01:00
Aidan Timson 5315c5710f Run e2e suites in parallel 2026-07-13 09:05:07 +01:00
25 changed files with 1438 additions and 981 deletions
+18 -10
View File
@@ -1,12 +1,18 @@
---
name: ha-frontend-testing
description: Home Assistant frontend validation workflow. Use when running lint, TypeScript checks, Vitest, Playwright e2e suites, dev servers, or chart-data benchmarks.
description: Home Assistant frontend testing and validation workflow. Use when adding or updating tests, running lint, TypeScript checks, Vitest, Playwright e2e suites, dev servers, or chart-data benchmarks.
---
# HA Frontend Testing
Use this skill when choosing or running validation for frontend changes.
## Test Helpers
- Before adding or changing tests, inspect the relevant suite's existing helpers and fixtures. Reuse them instead of duplicating setup, test data, navigation, interactions, waits, or assertions.
- When the same test flow appears more than once, move it into the closest suite-local helper with a focused interface.
- Keep one-off test behaviour in the test unless a helper makes the intent materially clearer. Do not hide the behaviour under test behind broad, configurable abstractions.
## Core Commands
```bash
@@ -35,23 +41,25 @@ For focused type feedback on one file, use editor diagnostics instead of a file-
`yarn dev:serve` also serves locally and supports `-c` for the core URL and `-p` for the port. Default local serving port is 8124.
Dev server commands support `--background`, `--status`, `--stop`, and `--logs [--follow]`.
Dev server commands support `--background`, `--status`, `--stop`, and `--logs [--follow]`. Prefer managed background mode while iterating so the watcher stays available across test runs without occupying the terminal.
## Playwright E2E
Each suite has its own dev server port. Playwright reuses an existing server locally when the port is already running; otherwise it performs a slow full build. The rspack watcher recompiles on save, so reruns should not need a restart.
Each suite has its own dev server port. Prefer running the relevant server in the background while iterating. Playwright reuses it when the port is already running; otherwise it performs a slow full build. The rspack watcher recompiles on save, so reruns should not need a restart.
Start the relevant suite server, then run that suite:
| Suite | Server | Test command |
| ------- | ------------------------------- | ----------------------- |
| App | `yarn test:e2e:app:dev` on 8095 | `yarn test:e2e:app` |
| Demo | `yarn dev:demo` on 8090 | `yarn test:e2e:demo` |
| Gallery | `yarn dev:gallery` on 8100 | `yarn test:e2e:gallery` |
| Suite | Background server | Test command |
| ------- | -------------------------------------------- | ----------------------- |
| App | `yarn test:e2e:app:dev --background` on 8095 | `yarn test:e2e:app` |
| Demo | `yarn dev:demo --background` on 8090 | `yarn test:e2e:demo` |
| Gallery | `yarn dev:gallery --background` on 8100 | `yarn test:e2e:gallery` |
Server reuse and `--stop` use the `/__ha_dev_status` health check, so starting or stopping twice is harmless.
Use the same server command with `--status`, `--logs [--follow]`, or `--stop` to manage it. Server reuse and `--stop` use the `/__ha_dev_status` health check, so starting or stopping twice is harmless.
Use `-g "<title>" --project=chromium` to narrow a run. `yarn test:e2e` runs all three suites. Run suites directly; piping through output truncation hides progress and failures.
Local runs against a watched development server do not always match CI's clean build artifacts, environment, sharding, or worker configuration. Use background servers for the fast iteration loop, but confirm the relevant CI jobs complete successfully before considering E2E changes verified.
Use `-g "<title>" --project=chromium` to narrow a run. `yarn test:e2e` runs all three suites in parallel when every managed server is available, otherwise it runs them sequentially to prevent cold builds racing over shared generated assets. Run suites directly; piping through output truncation hides progress and failures.
The app suite uses a stripped-down harness for e2e. Demo and gallery use their normal dev servers.
+4
View File
@@ -18,6 +18,10 @@ runs:
node-version-file: ".nvmrc"
cache: ${{ inputs.cache == 'true' && 'yarn' || '' }}
- name: Enable Corepack
shell: bash
run: corepack enable
- name: Install dependencies
shell: bash
run: yarn install ${{ inputs.immutable == 'true' && '--immutable' || '' }}
+161 -40
View File
@@ -38,8 +38,9 @@ jobs:
- name: Build demo
uses: ./.github/actions/build
with:
target: build-demo
target: build-demo-e2e
github-token: ${{ secrets.GITHUB_TOKEN }}
is-test: true
- name: Upload demo build
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
@@ -65,8 +66,9 @@ jobs:
- name: Build e2e test app
uses: ./.github/actions/build
with:
target: build-e2e-test-app
target: build-e2e-test-app-e2e
github-token: ${{ secrets.GITHUB_TOKEN }}
is-test: true
- name: Upload e2e test app build
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
@@ -103,15 +105,27 @@ jobs:
if-no-files-found: error
retention-days: 3
# ── Run Playwright tests locally against Chromium ──────────────────────────
e2e-local:
name: E2E (local Chromium)
needs: [build-demo, build-e2e-test-app, build-gallery]
# ── Run Playwright tests against Chromium ──────────────────────────────────
e2e-demo:
name: E2E demo (${{ matrix.shardIndex }}/${{ matrix.shardTotal }})
needs:
- build-demo
runs-on: ubuntu-latest
# Fail fast if anything hangs. The whole suite should take < 15 minutes on
# Chromium; anything longer is almost certainly an install or webServer
# hang.
timeout-minutes: 30
container:
image: mcr.microsoft.com/playwright:v1.61.1-noble
options: --user 1001 --ipc=host
defaults:
run:
shell: bash
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
shardIndex:
- 1
- 2
shardTotal:
- 2
steps:
- name: Check out files from GitHub
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
@@ -121,60 +135,133 @@ jobs:
- name: Setup Node and install
uses: ./.github/actions/setup
# Resolve the installed Playwright version so the browser cache tracks
# Playwright itself, not every unrelated dependency bump.
- name: Resolve Playwright version
id: playwright-version
run: echo "version=$(node -p 'require("@playwright/test/package.json").version')" >> "$GITHUB_OUTPUT"
# Cache the downloaded browser build keyed on the installed Playwright
# version, so re-runs skip the ~170 MB download unless Playwright changes.
- name: Cache Playwright browsers
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ~/.cache/ms-playwright
key: ${{ runner.os }}-playwright-${{ steps.playwright-version.outputs.version }}
- name: Install Playwright browsers
run: yarn playwright install --with-deps chromium
timeout-minutes: 10
- name: Download demo build
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: demo-dist
path: demo/dist/
- name: Run Playwright demo tests
run: yarn test:e2e:demo --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
timeout-minutes: 15
- name: Upload demo blob report
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: always()
with:
name: blob-report-demo-${{ matrix.shardIndex }}
path: test/e2e/reports/demo/
if-no-files-found: warn
retention-days: 3
e2e-app:
name: E2E app (${{ matrix.shardIndex }}/${{ matrix.shardTotal }})
needs:
- build-e2e-test-app
runs-on: ubuntu-latest
container:
image: mcr.microsoft.com/playwright:v1.61.1-noble
options: --user 1001 --ipc=host
defaults:
run:
shell: bash
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
shardIndex:
- 1
- 2
- 3
- 4
shardTotal:
- 4
steps:
- name: Check out files from GitHub
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- name: Setup Node and install
uses: ./.github/actions/setup
- name: Download e2e test app build
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: e2e-test-app-dist
path: test/e2e/app/dist/
- name: Run Playwright app tests
run: yarn test:e2e:app --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
timeout-minutes: 15
- name: Upload app blob report
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: always()
with:
name: blob-report-app-${{ matrix.shardIndex }}
path: test/e2e/reports/app/
if-no-files-found: warn
retention-days: 3
e2e-gallery:
name: E2E gallery (${{ matrix.shardIndex }}/${{ matrix.shardTotal }})
needs:
- build-gallery
runs-on: ubuntu-latest
container:
image: mcr.microsoft.com/playwright:v1.61.1-noble
options: --user 1001 --ipc=host
defaults:
run:
shell: bash
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
shardIndex:
- 1
- 2
- 3
- 4
shardTotal:
- 4
steps:
- name: Check out files from GitHub
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- name: Setup Node and install
uses: ./.github/actions/setup
- name: Download gallery build
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: gallery-dist
path: gallery/dist/
- name: Run Playwright tests (local)
run: yarn test:e2e
- name: Run Playwright gallery tests
run: yarn test:e2e:gallery --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
timeout-minutes: 15
- name: Upload blob report
- name: Upload gallery blob report
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: always()
with:
name: blob-report-local
path: test/e2e/reports/
name: blob-report-gallery-${{ matrix.shardIndex }}
path: test/e2e/reports/gallery/
if-no-files-found: warn
retention-days: 3
# ── Merge local blob reports and post PR comment ───────────────────────────
report:
name: Report
needs: [e2e-local]
needs:
- e2e-demo
- e2e-app
- e2e-gallery
runs-on: ubuntu-latest
if: ${{ !cancelled() }}
if: ${{ always() }}
permissions:
contents: read
pull-requests: write
@@ -187,12 +274,26 @@ jobs:
- name: Setup Node and install
uses: ./.github/actions/setup
- name: Download blob report (local)
- name: Download demo blob reports
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
continue-on-error: true
with:
name: blob-report-local
path: test/e2e/reports/
pattern: blob-report-demo-*
path: test/e2e/reports/demo/
- name: Download app blob reports
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
continue-on-error: true
with:
pattern: blob-report-app-*
path: test/e2e/reports/app/
- name: Download gallery blob reports
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
continue-on-error: true
with:
pattern: blob-report-gallery-*
path: test/e2e/reports/gallery/
- name: Stage blobs for merge
run: node test/e2e/collect-blob-reports.mjs
@@ -209,7 +310,11 @@ jobs:
retention-days: 14
- name: Post report to PR
if: github.event_name == 'pull_request' && needs.e2e-local.result == 'failure'
if: >-
github.event_name == 'pull_request' &&
(needs.e2e-demo.result == 'failure' ||
needs.e2e-app.result == 'failure' ||
needs.e2e-gallery.result == 'failure')
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
@@ -217,3 +322,19 @@ jobs:
`${process.env.GITHUB_WORKSPACE}/test/e2e/post-report-comment.mjs`
);
await postReportComment({ github, context, core });
- name: Check suite results
run: |
failed=0
for suite in \
"demo:${{ needs.e2e-demo.result }}" \
"app:${{ needs.e2e-app.result }}" \
"gallery:${{ needs.e2e-gallery.result }}"; do
name="${suite%%:*}"
result="${suite#*:}"
echo "E2E ${name}: ${result}"
if [ "$result" != "success" ]; then
failed=1
fi
done
exit "$failed"
+4 -2
View File
@@ -232,7 +232,7 @@ module.exports.config = {
};
},
demo({ isProdBuild, latestBuild, isStatsBuild }) {
demo({ isProdBuild, latestBuild, isStatsBuild, isTestBuild }) {
return {
name: "demo" + nameSuffix(latestBuild),
entry: {
@@ -247,6 +247,7 @@ module.exports.config = {
isProdBuild,
latestBuild,
isStatsBuild,
isTestBuild,
};
},
@@ -306,7 +307,7 @@ module.exports.config = {
};
},
e2eTestApp({ isProdBuild, latestBuild, isStatsBuild }) {
e2eTestApp({ isProdBuild, latestBuild, isStatsBuild, isTestBuild }) {
return {
name: "e2e-test-app" + nameSuffix(latestBuild),
entry: {
@@ -321,6 +322,7 @@ module.exports.config = {
isProdBuild,
latestBuild,
isStatsBuild,
isTestBuild,
};
},
};
+16
View File
@@ -42,6 +42,22 @@ gulp.task(
)
);
gulp.task(
"build-demo-e2e",
gulp.series(
async function setEnv() {
process.env.NODE_ENV = "production";
},
"clean-demo",
// Cast needs to be backwards compatible and older HA has no translations
"translations-enable-merge-backend",
gulp.parallel("gen-icons-json", "build-translations", "build-locale-data"),
"copy-static-demo",
"rspack-prod-demo-e2e",
"gen-pages-demo-prod-e2e"
)
);
gulp.task(
"analyze-demo",
gulp.series(
+15
View File
@@ -39,3 +39,18 @@ gulp.task(
"gen-pages-e2e-test-app-prod"
)
);
gulp.task(
"build-e2e-test-app-e2e",
gulp.series(
async function setEnv() {
process.env.NODE_ENV = "production";
},
"clean-e2e-test-app",
"translations-enable-merge-backend",
gulp.parallel("gen-icons-json", "build-translations", "build-locale-data"),
"copy-static-e2e-test-app",
"rspack-prod-e2e-test-app-e2e",
"gen-pages-e2e-test-app-prod"
)
);
+10
View File
@@ -225,6 +225,16 @@ gulp.task(
)
);
gulp.task(
"gen-pages-demo-prod-e2e",
genPagesProdTask(
DEMO_PAGE_ENTRIES,
paths.demo_dir,
paths.demo_output_root,
paths.demo_output_latest
)
);
const GALLERY_PAGE_ENTRIES = { "index.html": ["entrypoint"] };
gulp.task(
+24
View File
@@ -177,6 +177,18 @@ gulp.task("rspack-prod-demo", () =>
bothBuilds(createDemoConfig, {
isProdBuild: true,
isStatsBuild: env.isStatsBuild(),
isTestBuild: env.isTestBuild(),
})
)
);
gulp.task("rspack-prod-demo-e2e", () =>
prodBuild(
createDemoConfig({
isProdBuild: true,
latestBuild: true,
isStatsBuild: env.isStatsBuild(),
isTestBuild: env.isTestBuild(),
})
)
);
@@ -269,6 +281,18 @@ gulp.task("rspack-prod-e2e-test-app", () =>
bothBuilds(createE2eTestAppConfig, {
isProdBuild: true,
isStatsBuild: env.isStatsBuild(),
isTestBuild: env.isTestBuild(),
})
)
);
gulp.task("rspack-prod-e2e-test-app-e2e", () =>
prodBuild(
createE2eTestAppConfig({
isProdBuild: true,
latestBuild: true,
isStatsBuild: env.isStatsBuild(),
isTestBuild: env.isTestBuild(),
})
)
);
+19 -4
View File
@@ -387,9 +387,14 @@ const createAppConfig = ({
bundle.config.app({ isProdBuild, latestBuild, isStatsBuild, isTestBuild })
);
const createDemoConfig = ({ isProdBuild, latestBuild, isStatsBuild }) =>
const createDemoConfig = ({
isProdBuild,
latestBuild,
isStatsBuild,
isTestBuild,
}) =>
createRspackConfig(
bundle.config.demo({ isProdBuild, latestBuild, isStatsBuild })
bundle.config.demo({ isProdBuild, latestBuild, isStatsBuild, isTestBuild })
);
const createCastConfig = ({ isProdBuild, latestBuild }) =>
@@ -401,9 +406,19 @@ const createGalleryConfig = ({ isProdBuild, latestBuild }) =>
const createLandingPageConfig = ({ isProdBuild, latestBuild }) =>
createRspackConfig(bundle.config.landingPage({ isProdBuild, latestBuild }));
const createE2eTestAppConfig = ({ isProdBuild, latestBuild, isStatsBuild }) =>
const createE2eTestAppConfig = ({
isProdBuild,
latestBuild,
isStatsBuild,
isTestBuild,
}) =>
createRspackConfig(
bundle.config.e2eTestApp({ isProdBuild, latestBuild, isStatsBuild })
bundle.config.e2eTestApp({
isProdBuild,
latestBuild,
isStatsBuild,
isTestBuild,
})
);
module.exports = {
+16
View File
@@ -58,6 +58,17 @@
"depNameTemplate": "rhysd/actionlint",
"datasourceTemplate": "github-releases",
"extractVersionTemplate": "^v(?<version>.+)$"
},
{
"description": "Keep Playwright CI container image up to date",
"customType": "regex",
"managerFilePatterns": ["/^\\.github/workflows/e2e\\.yaml$/"],
"matchStrings": [
"mcr\\.microsoft\\.com/playwright:(?<currentValue>v\\d+\\.\\d+\\.\\d+-noble)"
],
"depNameTemplate": "mcr.microsoft.com/playwright",
"datasourceTemplate": "docker",
"versioningTemplate": "regex:^v(?<major>\\d+)\\.(?<minor>\\d+)\\.(?<patch>\\d+)-(?<compatibility>noble)$"
}
],
"packageRules": [
@@ -86,6 +97,11 @@
"description": "Group date-fns with dependent timezone package",
"groupName": "date-fns",
"matchPackageNames": ["date-fns", "date-fns-tz"]
},
{
"description": "Group Playwright package and CI container updates",
"groupName": "Playwright",
"matchPackageNames": ["@playwright/test", "mcr.microsoft.com/playwright"]
}
]
}
+67 -461
View File
@@ -4,164 +4,29 @@
* Run with:
* yarn test:e2e:app
*/
import { test, expect, type Page } from "@playwright/test";
import type { MoreInfoView } from "../../src/dialogs/more-info/const";
import { PANEL_TIMEOUT, QUICK_TIMEOUT, SHELL_TIMEOUT } from "./helpers";
import { e2ePanelRouteAssertions } from "./app/src/ha-test-panels";
/**
* Each More info view renders one root element inside the dialog, plus one or
* more characteristic descendants that prove the view actually populated rather
* than rendering an empty shell. `text`, when set, asserts the element's text
* instead of just its presence.
*/
const MORE_INFO_VIEW_ELEMENTS: {
view: MoreInfoView;
element: string;
content: { selector: string; text?: string }[];
}[] = [
{
view: "info",
element: "ha-more-info-info",
content: [
{ selector: "more-info-light" },
{ selector: "span.title", text: "Test Light" },
],
},
{
view: "history",
element: "ha-more-info-history-and-logbook",
// The demo loads the history component but not logbook.
content: [{ selector: "ha-more-info-history" }],
},
{
view: "settings",
element: "ha-more-info-settings",
// The scenario mocks config/entity_registry/get, so the real registry
// panel renders instead of the "no unique ID" warning.
content: [{ selector: "entity-registry-settings" }],
},
{
view: "related",
element: "ha-related-items",
// search/related is mocked to return no relations, so the empty list
// renders.
content: [{ selector: "ha-related-items >> ha-list" }],
},
{
view: "add_to",
element: "ha-more-info-add-to",
// Admin users get the default add-to action list.
content: [{ selector: "ha-add-to-action-list" }],
},
{
view: "details",
element: "ha-more-info-details",
// The details view renders the state and attributes cards.
content: [{ selector: "ha-card" }],
},
];
const URL_NORMALIZATION_ASSERTIONS: {
name: string;
path: string;
element: string;
url: RegExp;
action?: (page: Page) => Promise<void>;
}[] = [
{
name: "keeps the todo panel when adding the selected entity query",
path: "/todo",
element: "ha-panel-todo",
url: /\/\?entity_id=todo\.shopping_list#\/todo$/,
},
{
name: "keeps the history panel when removing the back query",
path: "/?back=1#/history",
element: "ha-panel-history, history-panel",
url: /\/#\/history$/,
},
{
name: "keeps the logbook panel when removing the back query",
path: "/?back=1#/logbook",
element: "ha-panel-logbook",
url: /\/#\/logbook$/,
},
{
name: "keeps the lovelace panel when adding the edit query",
path: "/lovelace",
element: "ha-panel-lovelace, hui-root",
url: /\/\?edit=1#\/lovelace\/home$/,
action: (page) => setLovelaceEditMode(page, true),
},
{
name: "keeps the lovelace panel when removing the edit query",
path: "/lovelace",
element: "ha-panel-lovelace, hui-root",
url: /\/#\/lovelace\/home$/,
action: async (page) => {
await setLovelaceEditMode(page, true);
await expect(page).toHaveURL(/\/\?edit=1#\/lovelace\/home$/, {
timeout: SHELL_TIMEOUT,
});
await setLovelaceEditMode(page, false);
},
},
];
interface E2ELovelaceRoot extends HTMLElement {
lovelace?: {
setEditMode: (editMode: boolean) => void;
};
}
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
// The test app is built with __DEMO__=true which enables hash-based routing.
// Panel paths must use hash URLs: /#/lovelace, /#/energy, etc.
// Scenario selection uses query params: /?scenario=foo (always at root).
/** Navigate to a panel (hash routing) and wait for app to initialize. */
async function goToPanel(page: Page, path: string) {
// Paths starting with /? are root-level (scenario selection); panel paths
// need to use hash routing (/#/panelname).
const url = path.startsWith("/?") ? path : `/#${path}`;
await page.goto(url);
await page.waitForSelector("ha-test", { state: "attached" });
// Wait for the app to finish initialising (hassConnected sets panels)
await page.waitForFunction(() => Boolean((window as any).__mockHass));
}
async function setLovelaceEditMode(page: Page, editMode: boolean) {
await page
.locator("hui-root")
.first()
.waitFor({ state: "attached", timeout: QUICK_TIMEOUT });
await page
.locator("hui-root")
.first()
.evaluate(async (el: Element, value) => {
const root = el as E2ELovelaceRoot;
const start = performance.now();
await new Promise<void>((resolve, reject) => {
const check = () => {
if (root.lovelace?.setEditMode) {
resolve();
return;
}
if (performance.now() - start > 2000) {
reject(new Error("Lovelace edit mode action was not available"));
return;
}
requestAnimationFrame(check);
};
check();
});
root.lovelace!.setEditMode(value);
}, editMode);
}
import { test, expect } from "@playwright/test";
import {
appSidebar,
appSidebarConfig,
appSidebarPanel,
assertElementContent,
defineLinkSmokeTests,
defineRouteSmokeTests,
ensureAppSidebarPanelVisible,
goToPanel,
} from "./app/src/helpers";
import {
expectNoPageErrors,
PANEL_TIMEOUT,
QUICK_TIMEOUT,
SHELL_TIMEOUT,
trackPageErrors,
} from "./helpers";
import {
appRouteSmokeGroups,
configLinks,
moreInfoViewElements,
} from "./app/src/smoke";
// ---------------------------------------------------------------------------
// App shell
@@ -169,60 +34,40 @@ async function setLovelaceEditMode(page: Page, editMode: boolean) {
test.describe("App shell", () => {
test("page loads and ha-test element mounts", async ({ page }) => {
const errors: string[] = [];
page.on("pageerror", (e) => errors.push(e.message));
const errors = trackPageErrors(page);
await goToPanel(page, "/");
await expect(page.locator("ha-test")).toBeAttached();
expect(errors).toHaveLength(0);
await expect(page.locator("ha-test")).toBeAttached({
timeout: QUICK_TIMEOUT,
});
expectNoPageErrors(errors, undefined, []);
});
test("sidebar renders with expected panels", async ({ page }) => {
await goToPanel(page, "/lovelace");
// Regular panels use #sidebar-panel-{urlPath} inside ha-sidebar's shadow root
for (const urlPath of ["lovelace", "map", "energy", "history"]) {
// eslint-disable-next-line no-await-in-loop
await expect(
page.locator(
`ha-test >> home-assistant-main >> ha-sidebar >> #sidebar-panel-${urlPath}`
)
).toBeAttached();
}
// Config has its own special element with id="sidebar-config"
await expect(
page.locator(
`ha-test >> home-assistant-main >> ha-sidebar >> #sidebar-config`
)
).toBeAttached();
await Promise.all([
// Regular panels use #sidebar-panel-{urlPath} inside ha-sidebar's shadow root.
...["lovelace", "map", "energy", "history"].map((urlPath) =>
expect(appSidebarPanel(page, urlPath)).toBeAttached({
timeout: QUICK_TIMEOUT,
})
),
// Config has its own special element with id="sidebar-config".
expect(appSidebarConfig(page)).toBeAttached({
timeout: QUICK_TIMEOUT,
}),
]);
});
test("sidebar navigation changes the active panel", async ({ page }) => {
await goToPanel(page, "/lovelace");
const sidebar = page.locator(
"ha-test >> home-assistant-main >> ha-sidebar"
);
await expect(sidebar).toBeAttached({ timeout: SHELL_TIMEOUT });
const historyLink = sidebar.locator("#sidebar-panel-history");
if (!(await historyLink.isVisible().catch(() => false))) {
await page.locator("ha-test >> home-assistant-main").evaluate((el) => {
el.dispatchEvent(
new CustomEvent("hass-toggle-menu", {
detail: { open: true },
bubbles: true,
composed: true,
})
);
});
}
await expect(historyLink).toBeVisible({ timeout: SHELL_TIMEOUT });
const historyLink = await ensureAppSidebarPanelVisible(page, "history");
await historyLink.click();
await expect(page).toHaveURL(/\/#\/history$/, { timeout: SHELL_TIMEOUT });
await expect(page).toHaveURL(/\/#\/history$/, { timeout: QUICK_TIMEOUT });
await expect(
page.locator("ha-panel-history, history-panel").first()
).toBeAttached({ timeout: PANEL_TIMEOUT });
@@ -231,34 +76,29 @@ test.describe("App shell", () => {
test("sidebar renders notification badge", async ({ page }) => {
await goToPanel(page, "/lovelace");
const sidebar = page.locator(
"ha-test >> home-assistant-main >> ha-sidebar"
);
await expect(sidebar).toBeAttached({ timeout: SHELL_TIMEOUT });
const sidebar = appSidebar(page);
await expect(sidebar).toBeAttached({ timeout: QUICK_TIMEOUT });
const notificationsLink = sidebar.locator("#sidebar-notifications");
await expect(notificationsLink).toBeAttached({ timeout: SHELL_TIMEOUT });
await expect(notificationsLink).toBeAttached({ timeout: QUICK_TIMEOUT });
await expect(notificationsLink.locator(".badge").first()).toHaveText("1", {
timeout: SHELL_TIMEOUT,
timeout: QUICK_TIMEOUT,
});
});
test("sidebar marks the active panel as selected", async ({ page }) => {
const sidebar = page.locator(
"ha-test >> home-assistant-main >> ha-sidebar"
);
const lovelaceLink = sidebar.locator("#sidebar-panel-lovelace");
const historyLink = sidebar.locator("#sidebar-panel-history");
const lovelaceLink = appSidebarPanel(page, "lovelace");
const historyLink = appSidebarPanel(page, "history");
await goToPanel(page, "/lovelace");
await expect(lovelaceLink).toHaveClass(/selected/, {
timeout: SHELL_TIMEOUT,
timeout: QUICK_TIMEOUT,
});
await expect(historyLink).not.toHaveClass(/selected/);
await goToPanel(page, "/history");
await expect(historyLink).toHaveClass(/selected/, {
timeout: SHELL_TIMEOUT,
timeout: QUICK_TIMEOUT,
});
await expect(lovelaceLink).not.toHaveClass(/selected/);
});
@@ -271,124 +111,16 @@ test.describe("App shell", () => {
await goToPanel(page, "/?scenario=non-admin#/lovelace");
// Wait for the sidebar to mount before asserting on its contents.
await expect(
page.locator("ha-test >> home-assistant-main >> ha-sidebar")
).toBeAttached({ timeout: SHELL_TIMEOUT });
await expect(appSidebar(page)).toBeAttached({ timeout: QUICK_TIMEOUT });
// Config panel is adminOnly — should not appear for non-admin.
const configLink = page.locator(
`ha-test >> home-assistant-main >> ha-sidebar >> #sidebar-config`
);
await expect(configLink).not.toBeAttached();
});
});
// ---------------------------------------------------------------------------
// Panel navigation
// ---------------------------------------------------------------------------
test.describe("Panel navigation", () => {
for (const [path, element] of e2ePanelRouteAssertions) {
test(`renders registered panel ${path}`, async ({ page }) => {
await goToPanel(page, path);
await expect(page.locator(element).first()).toBeAttached({
timeout: PANEL_TIMEOUT,
});
});
}
});
test.describe("Panel URL normalization", () => {
for (const {
name,
path,
element,
url,
action,
} of URL_NORMALIZATION_ASSERTIONS) {
test(name, async ({ page }) => {
await goToPanel(page, path);
await action?.(page);
await expect(page).toHaveURL(url, { timeout: SHELL_TIMEOUT });
await expect(page.locator(element).first()).toBeAttached({
timeout: PANEL_TIMEOUT,
});
});
}
});
// ---------------------------------------------------------------------------
// Tools panel (formerly Developer tools)
// ---------------------------------------------------------------------------
/**
* Every tool sub-page reachable under /config/tools, mapped to the custom
* element tools-router mounts for it (see tools-router.ts). Asserting on the
* specific element proves the route actually rendered its tool, not just the
* shared ha-panel-tools shell.
*/
const TOOLS_SUBPAGES: { route: string; element: string }[] = [
{ route: "yaml", element: "tools-yaml-config" },
{ route: "state", element: "tools-state" },
{ route: "action", element: "tools-action" },
{ route: "template", element: "tools-template" },
{ route: "event", element: "tools-event" },
{ route: "statistics", element: "tools-statistics" },
{ route: "assist", element: "tools-assist" },
{ route: "debug", element: "tools-debug" },
];
test.describe("Tools panel", () => {
test("base path renders the tools panel", async ({ page }) => {
await goToPanel(page, "/config/tools");
await expect(page.locator("ha-panel-tools")).toBeAttached({
timeout: PANEL_TIMEOUT,
});
});
for (const { route, element } of TOOLS_SUBPAGES) {
test(`renders the ${route} sub-page`, async ({ page }) => {
await goToPanel(page, `/config/tools/${route}`);
await expect(page.locator(element)).toBeAttached({
timeout: PANEL_TIMEOUT,
});
});
}
test("service is an alias for the action tool", async ({ page }) => {
await goToPanel(page, "/config/tools/service");
await expect(page.locator("tools-action")).toBeAttached({
timeout: PANEL_TIMEOUT,
await expect(appSidebarConfig(page)).not.toBeAttached({
timeout: QUICK_TIMEOUT,
});
});
});
// ---------------------------------------------------------------------------
// Tools redirects (old developer-tools URLs)
// ---------------------------------------------------------------------------
test.describe("Tools redirects", () => {
// The panel moved from top-level /developer-tools (pre-2026.2) to
// /config/developer-tools (2026.2), then was renamed to /config/tools
// (2026.8). Both old locations must redirect to the new one, and deep links
// must keep their sub-page. See the updateRoute() redirect in
// src/layouts/home-assistant.ts.
for (const oldBase of ["/developer-tools", "/config/developer-tools"]) {
test(`redirects ${oldBase} to the tools panel`, async ({ page }) => {
await goToPanel(page, oldBase);
await expect(page.locator("ha-panel-tools")).toBeAttached({
timeout: PANEL_TIMEOUT,
});
});
test(`redirects ${oldBase}/state to the state tool`, async ({ page }) => {
await goToPanel(page, `${oldBase}/state`);
await expect(page.locator("tools-state")).toBeAttached({
timeout: PANEL_TIMEOUT,
});
});
}
});
defineRouteSmokeTests(appRouteSmokeGroups);
// ---------------------------------------------------------------------------
// Lovelace
@@ -417,7 +149,7 @@ test.describe("Lovelace dashboard", () => {
// ---------------------------------------------------------------------------
test.describe("Light more-info dialog", () => {
for (const { view, element, content } of MORE_INFO_VIEW_ELEMENTS) {
for (const { view, element, content } of moreInfoViewElements) {
test(`opens more-info ${view} view for a light entity`, async ({
page,
}) => {
@@ -455,16 +187,7 @@ test.describe("Light more-info dialog", () => {
// Each view should render its own characteristic content, not just an
// empty shell.
for (const { selector, text } of content) {
const locator = dialog.locator(selector).first();
if (text) {
// eslint-disable-next-line no-await-in-loop
await expect(locator).toContainText(text, { timeout: QUICK_TIMEOUT });
} else {
// eslint-disable-next-line no-await-in-loop
await expect(locator).toBeAttached({ timeout: QUICK_TIMEOUT });
}
}
await assertElementContent(dialog, content);
});
}
});
@@ -519,144 +242,27 @@ test.describe("Theming", () => {
// ---------------------------------------------------------------------------
test.describe("Config panel", () => {
const DASHBOARD_LINKS = [
{ href: "/config/integrations", label: "Devices & services" },
{ href: "/config/automation", label: "Automations & scenes" },
{ href: "/config/areas", label: "Areas, labels & zones" },
{ href: "/config/apps", label: "Apps" },
{ href: "/config/lovelace/dashboards", label: "Dashboards" },
{ href: "/config/voice-assistants", label: "Voice assistants" },
{ href: "/config/matter", label: "Matter" },
{ href: "/config/zha", label: "Zigbee" },
{ href: "/config/zwave_js", label: "Z-Wave" },
{ href: "/knx", label: "KNX" },
{ href: "/config/thread", label: "Thread" },
{ href: "/config/bluetooth", label: "Bluetooth" },
{ href: "/config/infrared", label: "Infrared" },
{ href: "/config/radio-frequency", label: "Radio frequency" },
{ href: "/insteon", label: "Insteon" },
{ href: "/config/tags", label: "Tags" },
{ href: "/config/person", label: "People" },
{ href: "/config/system", label: "System" },
{ href: "/config/tools", label: "Tools" },
{ href: "/config/info", label: "About" },
];
const CONFIG_ROUTES: { path: string; element: string }[] = [
{ path: "/config/integrations", element: "ha-config-integrations" },
{ path: "/config/devices", element: "ha-config-devices" },
{ path: "/config/entities", element: "ha-config-entities" },
{ path: "/config/helpers", element: "ha-config-helpers" },
{ path: "/config/areas", element: "ha-config-areas" },
{ path: "/config/apps", element: "ha-config-apps" },
{ path: "/config/app", element: "ha-config-app-dashboard" },
{ path: "/config/automation", element: "ha-config-automation" },
{ path: "/config/backup", element: "ha-config-backup" },
{ path: "/config/scene", element: "ha-config-scene" },
{ path: "/config/script", element: "ha-config-script" },
{ path: "/config/blueprint", element: "ha-config-blueprint" },
{ path: "/config/cloud", element: "ha-config-cloud" },
{ path: "/config/energy", element: "ha-config-energy" },
{ path: "/config/hardware", element: "ha-config-hardware" },
{ path: "/config/labs", element: "ha-config-labs" },
{ path: "/config/lovelace", element: "ha-config-lovelace" },
{ path: "/config/person", element: "ha-config-person" },
{ path: "/config/storage", element: "ha-config-section-storage" },
{ path: "/config/tags", element: "ha-config-tags" },
{ path: "/config/users", element: "ha-config-users" },
{ path: "/config/voice-assistants", element: "ha-config-voice-assistants" },
{ path: "/config/system", element: "ha-config-system-navigation" },
{ path: "/config/info", element: "ha-config-info" },
{ path: "/config/logs", element: "ha-config-logs" },
{ path: "/config/general", element: "ha-config-section-general" },
{ path: "/config/updates", element: "ha-config-section-updates" },
{ path: "/config/repairs", element: "ha-config-repairs-dashboard" },
{ path: "/config/analytics", element: "ha-config-section-analytics" },
{ path: "/config/ai-tasks", element: "ha-config-section-ai-tasks" },
{ path: "/config/labels", element: "ha-config-labels" },
{ path: "/config/zone", element: "ha-config-zone" },
{ path: "/config/network", element: "ha-config-section-network" },
{
path: "/config/application_credentials",
element: "ha-config-application-credentials",
},
{ path: "/config/bluetooth", element: "bluetooth-config-dashboard-router" },
{ path: "/config/dhcp", element: "dhcp-config-panel" },
{ path: "/config/infrared", element: "infrared-config-dashboard-router" },
{ path: "/config/matter", element: "matter-config-panel" },
{ path: "/config/mqtt", element: "mqtt-config-panel" },
{
path: "/config/radio-frequency",
element: "radio-frequency-config-dashboard-router",
},
{ path: "/config/ssdp", element: "ssdp-config-panel" },
{ path: "/config/thread", element: "thread-config-panel" },
{ path: "/config/zeroconf", element: "zeroconf-config-panel" },
{ path: "/config/zha", element: "zha-config-dashboard-router" },
{ path: "/config/zwave_js", element: "zwave_js-config-router" },
];
const NESTED_CONFIG_ROUTES: { path: string; element: string }[] = [
{
path: "/config/integrations/dashboard",
element: "ha-config-integrations-dashboard",
},
{
path: "/config/devices/dashboard",
element: "ha-config-devices-dashboard",
},
{ path: "/config/areas/dashboard", element: "ha-config-areas-dashboard" },
{ path: "/config/backup/settings", element: "ha-config-backup-settings" },
];
test("config panel loads without JS errors", async ({ page }) => {
const errors: string[] = [];
page.on("pageerror", (e) => errors.push(e.message));
const errors = trackPageErrors(page);
await goToPanel(page, "/config");
await expect(
page.locator("ha-panel-config, ha-config-dashboard").first()
).toBeAttached({ timeout: PANEL_TIMEOUT + 5_000 });
// Filter known pre-existing errors from vendor code
const realErrors = errors.filter(
(e) => !e.includes("ResizeObserver") && !e.includes("Non-Error")
);
expect(realErrors).toHaveLength(0);
expectNoPageErrors(errors);
});
test("dashboard renders key settings links", async ({ page }) => {
const getDashboard = async (page) => {
await goToPanel(page, "/config");
const dashboard = page.locator("ha-config-dashboard");
await expect(dashboard).toBeAttached({ timeout: PANEL_TIMEOUT });
await expect(dashboard).toBeAttached({ timeout: QUICK_TIMEOUT });
return dashboard;
};
for (const { href, label } of DASHBOARD_LINKS) {
const link = dashboard.getByRole("link", {
name: new RegExp(`^${label.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}\\b`),
});
// eslint-disable-next-line no-await-in-loop
await expect(link).toHaveAttribute("href", href, {
timeout: QUICK_TIMEOUT,
});
}
});
for (const { path, element } of CONFIG_ROUTES) {
test(`renders ${path}`, async ({ page }) => {
await goToPanel(page, path);
await expect(page.locator(element)).toBeAttached({
timeout: PANEL_TIMEOUT,
});
});
}
for (const { path, element } of NESTED_CONFIG_ROUTES) {
test(`renders ${path}`, async ({ page }) => {
await goToPanel(page, path);
await expect(page.locator(element)).toBeAttached({
timeout: PANEL_TIMEOUT,
});
});
}
defineLinkSmokeTests(
"config links point to expected pages",
configLinks,
getDashboard
);
});
+172
View File
@@ -0,0 +1,172 @@
import { expect, test, type Locator, type Page } from "@playwright/test";
import {
defineParallelSmokeTests,
PANEL_TIMEOUT,
QUICK_TIMEOUT,
SHELL_TIMEOUT,
} from "../../helpers";
const APP_MAIN_SELECTOR = "ha-test >> home-assistant-main";
const APP_SIDEBAR_SELECTOR = `${APP_MAIN_SELECTOR} >> ha-sidebar`;
// The app e2e harness is built with __DEMO__=true, which enables hash routing.
// Scenario selection uses query params at root: /?scenario=foo#/lovelace.
export async function goToPanel(page: Page, path: string) {
const url = path.startsWith("/?") ? path : `/#${path}`;
await page.goto(url);
await Promise.all([
page.waitForSelector("ha-test", {
state: "attached",
timeout: SHELL_TIMEOUT,
}),
page.waitForFunction(
() => "__mockHass" in window && Boolean(window.__mockHass),
undefined,
{ timeout: SHELL_TIMEOUT }
),
]);
}
export const appMain = (page: Page) => page.locator(APP_MAIN_SELECTOR);
export const appSidebar = (page: Page) => page.locator(APP_SIDEBAR_SELECTOR);
export const appSidebarPanel = (page: Page, panel: string) =>
appSidebar(page).locator(`#sidebar-panel-${panel}`);
export const appSidebarConfig = (page: Page) =>
appSidebar(page).locator("#sidebar-config");
export async function openAppSidebar(page: Page) {
await appMain(page).evaluate((el) => {
el.dispatchEvent(
new CustomEvent("hass-toggle-menu", {
detail: { open: true },
bubbles: true,
composed: true,
})
);
});
}
export async function ensureAppSidebarPanelVisible(page: Page, panel: string) {
await expect(appSidebar(page)).toBeAttached({ timeout: QUICK_TIMEOUT });
const link = appSidebarPanel(page, panel);
if (!(await link.isVisible().catch(() => false))) {
await openAppSidebar(page);
}
await expect(link).toBeVisible({ timeout: QUICK_TIMEOUT });
return link;
}
const escapeRegExp = (value: string) =>
value.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
export interface LinkSmokeCase {
href: string;
label: string;
}
export async function assertLink(
root: Locator,
{ href, label }: LinkSmokeCase
) {
const link = root.getByRole("link", {
name: new RegExp(`^${escapeRegExp(label)}\\b`),
});
await expect(link).toHaveAttribute("href", href, {
timeout: QUICK_TIMEOUT,
});
}
export function defineLinkSmokeTests(
name: string,
links: LinkSmokeCase[],
getRoot: (page: Page) => Promise<Locator>
) {
test(name, async ({ page }) => {
const root = await getRoot(page);
await Promise.all(
links.map((link) =>
test.step(`${link.label} links to ${link.href}`, async () => {
await assertLink(root, link);
})
)
);
});
}
export interface ElementContentAssertion {
selector: string;
text?: string;
}
export interface ViewElementSmokeCase<TView extends string = string> {
view: TView;
element: string;
content: ElementContentAssertion[];
}
export async function assertElementContent(
root: Locator,
content: ElementContentAssertion[]
) {
await Promise.all(
content.map(({ selector, text }) => {
const locator = root.locator(selector).first();
return text
? expect(locator).toContainText(text, { timeout: QUICK_TIMEOUT })
: expect(locator).toBeAttached({ timeout: QUICK_TIMEOUT });
})
);
}
export interface RouteSmokeCase {
name?: string;
path: string;
element: string;
url?: RegExp;
action?: (page: Page) => Promise<void>;
}
export interface RouteSmokeGroup {
name: string;
routes: RouteSmokeCase[];
testName?: (route: RouteSmokeCase) => string;
}
export const routeCase = (path: string, element: string): RouteSmokeCase => ({
path,
element,
});
export const routeCases = (routes: [string, string][]): RouteSmokeCase[] =>
routes.map(([path, element]) => routeCase(path, element));
export const rendersRoute = (route: RouteSmokeCase) => `renders ${route.path}`;
async function assertRouteSmoke(page: Page, route: RouteSmokeCase) {
await goToPanel(page, route.path);
await route.action?.(page);
if (route.url) {
await expect(page).toHaveURL(route.url, { timeout: QUICK_TIMEOUT });
}
await expect(page.locator(route.element).first()).toBeAttached({
timeout: PANEL_TIMEOUT,
});
}
export function defineRouteSmokeTests(groups: RouteSmokeGroup[]) {
defineParallelSmokeTests({
groups,
groupName: (group) => group.name,
cases: (group) => group.routes,
testName: (route, group) =>
route.name ?? group.testName?.(route) ?? rendersRoute(route),
run: async ({ page, smokeCase }) => {
await assertRouteSmoke(page, smokeCase);
},
});
}
+273
View File
@@ -0,0 +1,273 @@
import { expect, type Page } from "@playwright/test";
import type { MoreInfoView } from "../../../../src/dialogs/more-info/const";
import { QUICK_TIMEOUT, SHELL_TIMEOUT } from "../../helpers";
import {
rendersRoute,
routeCase,
routeCases,
type LinkSmokeCase,
type RouteSmokeCase,
type RouteSmokeGroup,
type ViewElementSmokeCase,
} from "./helpers";
import { e2ePanelRouteAssertions } from "./ha-test-panels";
// ── Config dashboard links ───────────────────────────────────────────────────
export const configLinks: LinkSmokeCase[] = [
{ href: "/config/integrations", label: "Devices & services" },
{ href: "/config/automation", label: "Automations & scenes" },
{ href: "/config/areas", label: "Areas, labels & zones" },
{ href: "/config/apps", label: "Apps" },
{ href: "/config/lovelace/dashboards", label: "Dashboards" },
{ href: "/config/voice-assistants", label: "Voice assistants" },
{ href: "/config/matter", label: "Matter" },
{ href: "/config/zha", label: "Zigbee" },
{ href: "/config/zwave_js", label: "Z-Wave" },
{ href: "/knx", label: "KNX" },
{ href: "/config/thread", label: "Thread" },
{ href: "/config/bluetooth", label: "Bluetooth" },
{ href: "/config/infrared", label: "Infrared" },
{ href: "/config/radio-frequency", label: "Radio frequency" },
{ href: "/insteon", label: "Insteon" },
{ href: "/config/tags", label: "Tags" },
{ href: "/config/person", label: "People" },
{ href: "/config/system", label: "System" },
{ href: "/config/tools", label: "Tools" },
{ href: "/config/info", label: "About" },
];
// ── More-info dialog views ───────────────────────────────────────────────────
export const moreInfoViewElements: ViewElementSmokeCase<MoreInfoView>[] = [
{
view: "info",
element: "ha-more-info-info",
content: [
{ selector: "more-info-light" },
{ selector: "span.title", text: "Test Light" },
],
},
{
view: "history",
element: "ha-more-info-history-and-logbook",
// The demo loads the history component but not logbook.
content: [{ selector: "ha-more-info-history" }],
},
{
view: "settings",
element: "ha-more-info-settings",
// The scenario mocks config/entity_registry/get, so the real registry
// panel renders instead of the "no unique ID" warning.
content: [{ selector: "entity-registry-settings" }],
},
{
view: "related",
element: "ha-related-items",
// search/related is mocked to return no relations, so the empty list
// renders.
content: [{ selector: "ha-related-items >> ha-list" }],
},
{
view: "add_to",
element: "ha-more-info-add-to",
// Admin users get the default add-to action list.
content: [{ selector: "ha-add-to-action-list" }],
},
{
view: "details",
element: "ha-more-info-details",
// The details view renders the state and attributes cards.
content: [{ selector: "ha-card" }],
},
];
// ── Route smoke tests ────────────────────────────────────────────────────────
interface E2ELovelaceRoot extends HTMLElement {
lovelace?: {
setEditMode: (editMode: boolean) => void;
};
}
async function setLovelaceEditMode(page: Page, editMode: boolean) {
await page
.locator("hui-root")
.first()
.waitFor({ state: "attached", timeout: QUICK_TIMEOUT });
await page
.locator("hui-root")
.first()
.evaluate(async (el: Element, value) => {
const root = el as E2ELovelaceRoot;
const start = performance.now();
await new Promise<void>((resolve, reject) => {
const check = () => {
if (root.lovelace?.setEditMode) {
resolve();
return;
}
if (performance.now() - start > 2000) {
reject(new Error("Lovelace edit mode action was not available"));
return;
}
requestAnimationFrame(check);
};
check();
});
root.lovelace!.setEditMode(value);
}, editMode);
}
const PANEL_ROUTE_ASSERTIONS = Array.from(
e2ePanelRouteAssertions,
([path, element]) => routeCase(path, element)
);
const URL_NORMALIZATION_ASSERTIONS: RouteSmokeCase[] = [
{
name: "keeps the todo panel when adding the selected entity query",
path: "/todo",
element: "ha-panel-todo",
url: /\/\?entity_id=todo\.shopping_list#\/todo$/,
},
{
name: "keeps the history panel when removing the back query",
path: "/?back=1#/history",
element: "ha-panel-history, history-panel",
url: /\/#\/history$/,
},
{
name: "keeps the logbook panel when removing the back query",
path: "/?back=1#/logbook",
element: "ha-panel-logbook",
url: /\/#\/logbook$/,
},
{
name: "keeps the lovelace panel when removing the edit query",
path: "/lovelace",
element: "ha-panel-lovelace, hui-root",
url: /\/#\/lovelace\/home$/,
action: async (page) => {
await setLovelaceEditMode(page, true);
await expect(page).toHaveURL(/\/\?edit=1#\/lovelace\/home$/, {
timeout: SHELL_TIMEOUT,
});
await setLovelaceEditMode(page, false);
},
},
];
const TOOLS_SUBPAGES: { route: string; element: string }[] = [
{ route: "yaml", element: "tools-yaml-config" },
{ route: "state", element: "tools-state" },
{ route: "action", element: "tools-action" },
{ route: "template", element: "tools-template" },
{ route: "event", element: "tools-event" },
{ route: "statistics", element: "tools-statistics" },
{ route: "assist", element: "tools-assist" },
{ route: "debug", element: "tools-debug" },
];
const TOOLS_ROUTE_ASSERTIONS = [
routeCase("/config/tools", "ha-panel-tools"),
...TOOLS_SUBPAGES.map(({ route, element }) =>
routeCase(`/config/tools/${route}`, element)
),
routeCase("/config/tools/service", "tools-action"),
];
const TOOLS_REDIRECT_ASSERTIONS = [
...["/developer-tools", "/config/developer-tools"].flatMap((oldBase) => [
routeCase(oldBase, "ha-panel-tools"),
routeCase(`${oldBase}/state`, "tools-state"),
]),
];
const CONFIG_ROUTES = routeCases([
["/config/integrations", "ha-config-integrations"],
["/config/devices", "ha-config-devices"],
["/config/entities", "ha-config-entities"],
["/config/helpers", "ha-config-helpers"],
["/config/areas", "ha-config-areas"],
["/config/apps", "ha-config-apps"],
["/config/app", "ha-config-app-dashboard"],
["/config/automation", "ha-config-automation"],
["/config/backup", "ha-config-backup"],
["/config/scene", "ha-config-scene"],
["/config/script", "ha-config-script"],
["/config/blueprint", "ha-config-blueprint"],
["/config/cloud", "ha-config-cloud"],
["/config/energy", "ha-config-energy"],
["/config/hardware", "ha-config-hardware"],
["/config/labs", "ha-config-labs"],
["/config/lovelace", "ha-config-lovelace"],
["/config/person", "ha-config-person"],
["/config/storage", "ha-config-section-storage"],
["/config/tags", "ha-config-tags"],
["/config/users", "ha-config-users"],
["/config/voice-assistants", "ha-config-voice-assistants"],
["/config/system", "ha-config-system-navigation"],
["/config/info", "ha-config-info"],
["/config/logs", "ha-config-logs"],
["/config/general", "ha-config-section-general"],
["/config/updates", "ha-config-section-updates"],
["/config/repairs", "ha-config-repairs-dashboard"],
["/config/analytics", "ha-config-section-analytics"],
["/config/ai-tasks", "ha-config-section-ai-tasks"],
["/config/labels", "ha-config-labels"],
["/config/zone", "ha-config-zone"],
["/config/network", "ha-config-section-network"],
["/config/application_credentials", "ha-config-application-credentials"],
["/config/bluetooth", "bluetooth-config-dashboard-router"],
["/config/dhcp", "dhcp-config-panel"],
["/config/infrared", "infrared-config-dashboard-router"],
["/config/matter", "matter-config-panel"],
["/config/mqtt", "mqtt-config-panel"],
["/config/radio-frequency", "radio-frequency-config-dashboard-router"],
["/config/ssdp", "ssdp-config-panel"],
["/config/thread", "thread-config-panel"],
["/config/zeroconf", "zeroconf-config-panel"],
["/config/zha", "zha-config-dashboard-router"],
["/config/zwave_js", "zwave_js-config-router"],
]);
const NESTED_CONFIG_ROUTES = routeCases([
["/config/integrations/dashboard", "ha-config-integrations-dashboard"],
["/config/devices/dashboard", "ha-config-devices-dashboard"],
["/config/areas/dashboard", "ha-config-areas-dashboard"],
["/config/backup/settings", "ha-config-backup-settings"],
]);
export const appRouteSmokeGroups: RouteSmokeGroup[] = [
{
name: "Panel navigation",
routes: PANEL_ROUTE_ASSERTIONS,
testName: (route) => `renders registered panel ${route.path}`,
},
{
name: "Panel URL normalization",
routes: URL_NORMALIZATION_ASSERTIONS,
testName: (route) => route.name!,
},
{
name: "Tools panel",
routes: TOOLS_ROUTE_ASSERTIONS,
testName: rendersRoute,
},
{
name: "Tools redirects",
routes: TOOLS_REDIRECT_ASSERTIONS,
testName: (route) => `redirects ${route.path}`,
},
{
name: "Config routes",
routes: CONFIG_ROUTES,
testName: rendersRoute,
},
{
name: "Nested config routes",
routes: NESTED_CONFIG_ROUTES,
testName: rendersRoute,
},
];
+27 -5
View File
@@ -5,6 +5,29 @@
// Usage: node test/e2e/collect-blob-reports.mjs
import { cpSync, mkdirSync, readdirSync, rmSync } from "fs";
import { join, relative } from "path";
const findBlobReports = (dir) => {
const files = [];
const walk = (currentDir) => {
for (const entry of readdirSync(currentDir, { withFileTypes: true })) {
const entryPath = join(currentDir, entry.name);
if (entry.isDirectory()) {
walk(entryPath);
} else if (entry.name.endsWith(".zip")) {
files.push(entryPath);
}
}
};
try {
walk(dir);
} catch {
return undefined;
}
return files;
};
const dest = "test/e2e/reports/blob";
rmSync(dest, { recursive: true, force: true });
@@ -12,10 +35,8 @@ mkdirSync(dest, { recursive: true });
for (const suite of ["demo", "app", "gallery"]) {
const src = `test/e2e/reports/${suite}`;
let files;
try {
files = readdirSync(src).filter((f) => f.endsWith(".zip"));
} catch {
const files = findBlobReports(src);
if (!files?.length) {
// Suite report directory doesn't exist (e.g. job was skipped or failed
// before uploading). Skip gracefully.
process.stderr.write(
@@ -24,6 +45,7 @@ for (const suite of ["demo", "app", "gallery"]) {
continue;
}
for (const file of files) {
cpSync(`${src}/${file}`, `${dest}/${suite}-${file}`);
const name = relative(src, file).replace(/[\\/]/g, "-");
cpSync(file, join(dest, `${suite}-${name}`));
}
}
+29 -107
View File
@@ -1,146 +1,68 @@
import { expect, test } from "@playwright/test";
import {
expectNoPageErrors,
NAVIGATION_TIMEOUT,
PANEL_TIMEOUT,
QUICK_TIMEOUT,
SHELL_TIMEOUT,
appErrors as filterAppErrors,
trackPageErrors,
} from "./helpers";
import {
activateDemoSidebarPanel,
demoCardSelector,
moreInfoCardSelector,
openDemoSidebar,
waitForDemoReady,
} from "./demo/helpers";
test.describe("Home Assistant Demo", () => {
// Collect JS errors during each test so we can assert no unexpected crashes.
let pageErrors: Error[] = [];
let pageErrors: ReturnType<typeof trackPageErrors>;
test.beforeEach(async ({ page }) => {
pageErrors = [];
page.on("pageerror", (err) => pageErrors.push(err));
pageErrors = trackPageErrors(page);
await page.goto("/");
});
function appErrors() {
return filterAppErrors(pageErrors);
}
// ── 1. Page loads ──────────────────────────────────────────────────────────
test("page loads and ha-demo mounts without JS errors", async ({ page }) => {
// The custom element is present in the document
await expect(page.locator("ha-demo")).toBeAttached({
timeout: NAVIGATION_TIMEOUT,
});
await waitForDemoReady(page);
// The launch screen should disappear once the app is ready
await expect(page.locator("#ha-launch-screen")).toBeHidden({
timeout: NAVIGATION_TIMEOUT,
});
// No unhandled JS exceptions
expect(appErrors()).toHaveLength(0);
expectNoPageErrors(pageErrors);
});
// ── 2. Dashboard renders ───────────────────────────────────────────────────
test("dashboard renders Lovelace cards", async ({ page }) => {
await expect(page.locator("ha-demo")).toBeAttached({
timeout: NAVIGATION_TIMEOUT,
});
await expect(page.locator("#ha-launch-screen")).toBeHidden({
timeout: NAVIGATION_TIMEOUT,
});
await waitForDemoReady(page);
const cardSelector = [
"hui-tile-card",
"hui-entity-card",
"hui-glance-card",
"hui-button-card",
"hui-markdown-card",
].join(", ");
await expect(page.locator(cardSelector).first()).toBeVisible({
await expect(page.locator(demoCardSelector).first()).toBeVisible({
timeout: PANEL_TIMEOUT,
});
});
// ── 3. Sidebar navigation ─────────────────────────────────────────────────
test("sidebar navigation changes the active panel", async ({ page }) => {
await expect(page.locator("ha-demo")).toBeAttached({
timeout: NAVIGATION_TIMEOUT,
});
await expect(page.locator("#ha-launch-screen")).toBeHidden({
timeout: NAVIGATION_TIMEOUT,
});
await waitForDemoReady(page);
await openDemoSidebar(page);
await activateDemoSidebarPanel(page, "map");
// On narrow viewports (< 870 px — mobile / tablet) the sidebar lives
// inside a modal drawer that is closed by default. Open it first via
// the ha-menu-button in the top app-bar.
const menuButton = page.locator("ha-menu-button");
if (await menuButton.isVisible()) {
await menuButton.click();
await expect(page.locator("ha-sidebar")).toBeVisible({
timeout: SHELL_TIMEOUT,
});
} else {
await expect(page.locator("ha-sidebar")).toBeAttached({
timeout: NAVIGATION_TIMEOUT,
});
}
const candidatePanels = ["map", "logbook", "history", "config"];
let clicked = false;
for (const panel of candidatePanels) {
const navItem = page.locator(`#sidebar-panel-${panel}`);
// eslint-disable-next-line no-await-in-loop
const visible = await navItem.isVisible().catch(() => false);
if (visible) {
// eslint-disable-next-line no-await-in-loop
await navItem.click();
// eslint-disable-next-line no-await-in-loop
await expect(page).toHaveURL(new RegExp(`/${panel}`), {
timeout: SHELL_TIMEOUT,
});
clicked = true;
break;
}
}
expect(clicked, "No known sidebar panel was found to click").toBe(true);
expect(appErrors()).toHaveLength(0);
expectNoPageErrors(pageErrors);
});
// ── 4. More info dialog ───────────────────────────────────────────────────
test("clicking an entity card opens the more-info dialog", async ({
page,
}) => {
await expect(page.locator("ha-demo")).toBeAttached({
timeout: NAVIGATION_TIMEOUT,
});
await expect(page.locator("#ha-launch-screen")).toBeHidden({
await waitForDemoReady(page);
// Tile cards are the most common card type in the demo; fall back to other
// clickable card types in case this platform renders a different layout.
await expect(page.locator(moreInfoCardSelector).first()).toBeVisible({
timeout: NAVIGATION_TIMEOUT,
});
await page.locator(moreInfoCardSelector).first().click();
// Tile cards are the most common card type in the demo; they open the
// more-info dialog on click. Fall back to other clickable card types in
// case the demo layout on this platform doesn't include tile cards.
const cardSelector =
"hui-tile-card, hui-entity-card, hui-button-card, hui-glance-card";
await expect(page.locator(cardSelector).first()).toBeVisible({
timeout: NAVIGATION_TIMEOUT,
});
await page.locator(cardSelector).first().click();
// The more-info dialog is a top-level custom element appended to the body.
// We verify it is attached, then confirm it rendered by checking the title
// span which is slotted into the light DOM and has real layout dimensions.
const dialog = page.locator("ha-more-info-dialog");
await expect(dialog).toBeAttached({ timeout: SHELL_TIMEOUT });
await expect(dialog.locator("span.title")).toBeVisible({
timeout: QUICK_TIMEOUT,
});
const title = dialog.locator("span.title");
await expect(title).toBeVisible({ timeout: QUICK_TIMEOUT });
expect(appErrors()).toHaveLength(0);
expectNoPageErrors(pageErrors);
});
});
+54
View File
@@ -0,0 +1,54 @@
import { expect, type Page } from "@playwright/test";
import { NAVIGATION_TIMEOUT, SHELL_TIMEOUT } from "../helpers";
export const demoCardSelector = [
"hui-tile-card",
"hui-entity-card",
"hui-glance-card",
"hui-button-card",
"hui-markdown-card",
].join(", ");
export const moreInfoCardSelector =
"hui-tile-card, hui-entity-card, hui-button-card, hui-glance-card";
export async function waitForDemoReady(page: Page) {
await expect(page.locator("ha-demo")).toBeAttached({
timeout: NAVIGATION_TIMEOUT,
});
await expect(page.locator("#ha-launch-screen")).toBeHidden({
timeout: NAVIGATION_TIMEOUT,
});
}
export async function openDemoSidebar(page: Page) {
const menuButton = page.locator("ha-menu-button");
if (await menuButton.isVisible()) {
const modalDrawer = page.locator("ha-drawer").locator("wa-drawer");
await Promise.all([
modalDrawer.evaluate(
(element) =>
new Promise<void>((resolve) => {
element.addEventListener("wa-after-show", () => resolve(), {
once: true,
});
})
),
menuButton.click(),
]);
return;
}
await expect(page.locator("ha-sidebar")).toBeAttached({
timeout: NAVIGATION_TIMEOUT,
});
}
export async function activateDemoSidebarPanel(page: Page, panel: string) {
const navItem = page.locator(`#sidebar-panel-${panel}`);
await expect(navItem).toBeVisible({ timeout: SHELL_TIMEOUT });
await navItem.click();
await expect(page).toHaveURL(new RegExp(`/${panel}(?:/|$)`), {
timeout: SHELL_TIMEOUT,
});
}
+61 -306
View File
@@ -7,362 +7,117 @@
* Run with:
* yarn test:e2e:gallery
*/
import { test, expect, type Page } from "@playwright/test";
import { QUICK_TIMEOUT, SHELL_TIMEOUT } from "./helpers";
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
/** Navigate to a gallery page via hash and wait for it to render. */
async function goToGalleryPage(page: Page, hash: string) {
// First visit to let ha-gallery boot up
await page.goto(`/#${hash}`);
await page.waitForSelector("ha-gallery", { state: "attached" });
// Wait for the demo element to appear in ha-gallery's shadow root.
// The element name is derived from the hash: "components/ha-bar" → "demo-components-ha-bar".
// page-description is only rendered for pages that have a description field,
// so we cannot use it as a universal readiness signal.
const demoTag = `demo-${hash.replace("/", "-")}`;
await page.waitForFunction((tag) => {
const gallery = document.querySelector("ha-gallery") as any;
return gallery?.shadowRoot?.querySelector(tag) != null;
}, demoTag);
}
/** Assert a gallery page loads without console errors.
* Demo elements live inside ha-gallery's shadow root use >> to pierce it.
*/
async function assertPageLoads(page: Page, hash: string, selector: string) {
const errors: string[] = [];
page.on("pageerror", (e) => errors.push(e.message));
await goToGalleryPage(page, hash);
// Pierce ha-gallery's shadow root with >>
await expect(page.locator(`ha-gallery >> ${selector}`).first()).toBeAttached({
timeout: SHELL_TIMEOUT,
});
const realErrors = errors.filter(
(e) => !IGNORED_ERRORS.some((re) => re.test(e))
);
expect(
realErrors,
`JS errors on ${hash}: ${realErrors.join("; ")}`
).toHaveLength(0);
}
// Errors that are gallery-harness artifacts rather than bugs in the component
// under test. The Lit-context init-error family that used to live here is gone:
// ha-gallery now provides fallback contexts for every demo (mirroring the real
// app's root), so context-consuming components resolve `localize`, formatters,
// config, etc. synchronously instead of throwing during init.
const IGNORED_ERRORS: RegExp[] = [
/ResizeObserver/,
/Non-Error/,
/Extension context/,
// Plain objects thrown by mock WebSocket/data-fetch show up as "Object".
/^Object$/,
// hui-group-entity-row calls .some() on a possibly-undefined entity_id array
// from mock state data — pre-existing gallery data issue.
/Cannot read properties of undefined \(reading 'some'\)/,
];
// ---------------------------------------------------------------------------
// Gallery shell
// ---------------------------------------------------------------------------
import { test, expect } from "@playwright/test";
import {
expectNoPageErrors,
QUICK_TIMEOUT,
SHELL_TIMEOUT,
trackPageErrors,
} from "./helpers";
import {
defineGallerySmokeTests,
expectGalleryDemoElement,
galleryLocator,
getGalleryDemo,
goToGalleryHome,
GALLERY_SHELL_IGNORED_PAGE_ERRORS,
} from "./gallery/helpers";
import { componentPages, lovelacePages, moreInfoPages } from "./gallery/pages";
test.describe("Gallery shell", () => {
test("page loads and ha-gallery mounts", async ({ page }) => {
const errors: string[] = [];
page.on("pageerror", (e) => errors.push(e.message));
const errors = trackPageErrors(page);
await page.goto("/");
await expect(page.locator("ha-gallery")).toBeAttached({
timeout: SHELL_TIMEOUT,
});
await goToGalleryHome(page);
const realErrors = errors.filter(
(e) => !e.includes("ResizeObserver") && !e.includes("Non-Error")
);
expect(realErrors).toHaveLength(0);
expectNoPageErrors(errors, undefined, GALLERY_SHELL_IGNORED_PAGE_ERRORS);
});
test("sidebar renders navigation links", async ({ page }) => {
await page.goto("/");
await page.waitForSelector("ha-gallery", { state: "attached" });
// The gallery drawer sidebar is inside ha-gallery's shadow root
await expect(page.locator("ha-gallery >> ha-drawer")).toBeAttached({
await goToGalleryHome(page);
await expect(galleryLocator(page, "ha-drawer")).toBeAttached({
timeout: QUICK_TIMEOUT,
});
});
});
// ---------------------------------------------------------------------------
// Component pages
// ---------------------------------------------------------------------------
const componentPages: { name: string; selector: string }[] = [
{ name: "ha-alert", selector: "demo-components-ha-alert" },
{ name: "ha-badge", selector: "demo-components-ha-badge" },
{ name: "ha-bar", selector: "demo-components-ha-bar" },
{ name: "ha-button", selector: "demo-components-ha-button" },
{ name: "ha-chips", selector: "demo-components-ha-chips" },
{ name: "ha-control-button", selector: "demo-components-ha-control-button" },
{
name: "ha-control-circular-slider",
selector: "demo-components-ha-control-circular-slider",
},
{
name: "ha-control-number-buttons",
selector: "demo-components-ha-control-number-buttons",
},
{
name: "ha-control-select-menu",
selector: "demo-components-ha-control-select-menu",
},
{ name: "ha-control-select", selector: "demo-components-ha-control-select" },
{ name: "ha-control-slider", selector: "demo-components-ha-control-slider" },
{ name: "ha-control-switch", selector: "demo-components-ha-control-switch" },
{ name: "ha-dialog", selector: "demo-components-ha-dialog" },
{ name: "ha-dropdown", selector: "demo-components-ha-dropdown" },
{
name: "ha-expansion-panel",
selector: "demo-components-ha-expansion-panel",
},
{ name: "ha-faded", selector: "demo-components-ha-faded" },
{ name: "ha-form", selector: "demo-components-ha-form" },
{ name: "ha-gauge", selector: "demo-components-ha-gauge" },
{
name: "ha-hs-color-picker",
selector: "demo-components-ha-hs-color-picker",
},
{ name: "ha-input", selector: "demo-components-ha-input" },
{ name: "ha-label-badge", selector: "demo-components-ha-label-badge" },
{ name: "ha-list", selector: "demo-components-ha-list" },
{ name: "ha-marquee-text", selector: "demo-components-ha-marquee-text" },
{
name: "ha-progress-button",
selector: "demo-components-ha-progress-button",
},
{ name: "ha-select-box", selector: "demo-components-ha-select-box" },
{ name: "ha-selector", selector: "demo-components-ha-selector" },
{ name: "ha-slider", selector: "demo-components-ha-slider" },
{ name: "ha-spinner", selector: "demo-components-ha-spinner" },
{ name: "ha-switch", selector: "demo-components-ha-switch" },
{ name: "ha-textarea", selector: "demo-components-ha-textarea" },
{ name: "ha-tip", selector: "demo-components-ha-tip" },
{ name: "ha-tooltip", selector: "demo-components-ha-tooltip" },
{
name: "ha-adaptive-dialog",
selector: "demo-components-ha-adaptive-dialog",
},
{
name: "ha-adaptive-popover",
selector: "demo-components-ha-adaptive-popover",
},
];
test.describe("Components", () => {
for (const { name, selector } of componentPages) {
test(`${name} renders without errors`, async ({ page }) => {
await assertPageLoads(page, `components/${name}`, selector);
});
}
});
// ---------------------------------------------------------------------------
// More-info pages
// ---------------------------------------------------------------------------
const moreInfoPages: { name: string; selector: string }[] = [
{ name: "light", selector: "demo-more-info-light" },
{ name: "climate", selector: "demo-more-info-climate" },
{ name: "cover", selector: "demo-more-info-cover" },
{ name: "fan", selector: "demo-more-info-fan" },
{ name: "humidifier", selector: "demo-more-info-humidifier" },
{ name: "input-number", selector: "demo-more-info-input-number" },
{ name: "input-text", selector: "demo-more-info-input-text" },
{ name: "lawn-mower", selector: "demo-more-info-lawn-mower" },
{ name: "lock", selector: "demo-more-info-lock" },
{ name: "media-player", selector: "demo-more-info-media-player" },
{ name: "number", selector: "demo-more-info-number" },
{ name: "scene", selector: "demo-more-info-scene" },
{ name: "timer", selector: "demo-more-info-timer" },
{ name: "update", selector: "demo-more-info-update" },
{ name: "vacuum", selector: "demo-more-info-vacuum" },
{ name: "water-heater", selector: "demo-more-info-water-heater" },
];
test.describe("More-info dialogs", () => {
for (const { name, selector } of moreInfoPages) {
test(`more-info ${name} renders without errors`, async ({ page }) => {
await assertPageLoads(page, `more-info/${name}`, selector);
});
}
});
// ---------------------------------------------------------------------------
// Lovelace card pages
// ---------------------------------------------------------------------------
const lovelacePages: { name: string; selector: string }[] = [
{ name: "area-card", selector: "demo-lovelace-area-card" },
{ name: "conditional-card", selector: "demo-lovelace-conditional-card" },
{ name: "entities-card", selector: "demo-lovelace-entities-card" },
{ name: "entity-button-card", selector: "demo-lovelace-entity-button-card" },
{ name: "entity-filter-card", selector: "demo-lovelace-entity-filter-card" },
{ name: "gauge-card", selector: "demo-lovelace-gauge-card" },
{ name: "glance-card", selector: "demo-lovelace-glance-card" },
{
name: "grid-and-stack-card",
selector: "demo-lovelace-grid-and-stack-card",
},
{ name: "iframe-card", selector: "demo-lovelace-iframe-card" },
{ name: "light-card", selector: "demo-lovelace-light-card" },
{ name: "map-card", selector: "demo-lovelace-map-card" },
{ name: "markdown-card", selector: "demo-lovelace-markdown-card" },
{ name: "media-control-card", selector: "demo-lovelace-media-control-card" },
{ name: "media-player-row", selector: "demo-lovelace-media-player-row" },
{ name: "picture-card", selector: "demo-lovelace-picture-card" },
{
name: "picture-elements-card",
selector: "demo-lovelace-picture-elements-card",
},
{
name: "picture-entity-card",
selector: "demo-lovelace-picture-entity-card",
},
{
name: "picture-glance-card",
selector: "demo-lovelace-picture-glance-card",
},
{ name: "thermostat-card", selector: "demo-lovelace-thermostat-card" },
{ name: "tile-card", selector: "demo-lovelace-tile-card" },
{ name: "todo-list-card", selector: "demo-lovelace-todo-list-card" },
];
test.describe("Lovelace cards", () => {
for (const { name, selector } of lovelacePages) {
test(`${name} renders without errors`, async ({ page }) => {
await assertPageLoads(page, `lovelace/${name}`, selector);
});
}
});
// ---------------------------------------------------------------------------
// Specific interaction tests
// ---------------------------------------------------------------------------
defineGallerySmokeTests("Components", "components", componentPages);
defineGallerySmokeTests("More-info dialogs", "more-info", moreInfoPages);
defineGallerySmokeTests("Lovelace cards", "lovelace", lovelacePages);
test.describe("Component interactions", () => {
test("ha-alert renders all four types", async ({ page }) => {
await goToGalleryPage(page, "components/ha-alert");
const demo = page.locator("ha-gallery >> demo-components-ha-alert");
await expect(demo).toBeAttached({ timeout: SHELL_TIMEOUT });
const demo = await getGalleryDemo(page, "components/ha-alert");
// The demo uses property binding (.alertType) not attribute binding,
// so we verify that multiple ha-alert elements are present.
// The demo uses property binding (.alertType) not attribute binding, so we
// verify that multiple ha-alert elements are present.
const alerts = demo.locator("ha-alert");
await expect(alerts.first()).toBeAttached({ timeout: QUICK_TIMEOUT });
// There should be at least 4 alerts (one per type)
await expect(alerts)
.toHaveCount(4, { timeout: QUICK_TIMEOUT })
.catch(async () => {
// If not exactly 4, just verify there are some (demo may include more)
const count = await alerts.count();
expect(count).toBeGreaterThanOrEqual(4);
});
await expect(alerts.nth(3)).toBeAttached({ timeout: QUICK_TIMEOUT });
});
test("ha-button renders primary action button", async ({ page }) => {
await goToGalleryPage(page, "components/ha-button");
const demo = page.locator("ha-gallery >> demo-components-ha-button");
await expect(demo).toBeAttached({ timeout: SHELL_TIMEOUT });
await expect(demo.locator("ha-button, mwc-button").first()).toBeAttached({
timeout: QUICK_TIMEOUT,
});
const demo = await getGalleryDemo(page, "components/ha-button");
await expectGalleryDemoElement(demo, "ha-button, mwc-button");
});
test("ha-control-slider can be found in DOM", async ({ page }) => {
await goToGalleryPage(page, "components/ha-control-slider");
const demo = page.locator(
"ha-gallery >> demo-components-ha-control-slider"
);
await expect(demo).toBeAttached({ timeout: SHELL_TIMEOUT });
await expect(demo.locator("ha-control-slider").first()).toBeAttached({
timeout: QUICK_TIMEOUT,
});
const demo = await getGalleryDemo(page, "components/ha-control-slider");
await expectGalleryDemoElement(demo, "ha-control-slider");
});
test("ha-form renders schema-driven fields", async ({ page }) => {
await goToGalleryPage(page, "components/ha-form");
const demo = page.locator("ha-gallery >> demo-components-ha-form");
await expect(demo).toBeAttached({ timeout: SHELL_TIMEOUT });
await expect(demo.locator("ha-form").first()).toBeAttached({
timeout: QUICK_TIMEOUT,
});
});
const demo = await getGalleryDemo(page, "components/ha-form");
test("ha-dialog demo renders a dialog trigger", async ({ page }) => {
await goToGalleryPage(page, "components/ha-dialog");
const demo = page.locator("ha-gallery >> demo-components-ha-dialog");
await expect(demo).toBeAttached({ timeout: SHELL_TIMEOUT });
await expectGalleryDemoElement(demo, "ha-form");
});
test("tile-card renders entity state", async ({ page }) => {
await goToGalleryPage(page, "lovelace/tile-card");
const demo = page.locator("ha-gallery >> demo-lovelace-tile-card");
await expect(demo).toBeAttached({ timeout: SHELL_TIMEOUT });
await expect(demo.locator("hui-tile-card").first()).toBeAttached({
timeout: QUICK_TIMEOUT,
});
const demo = await getGalleryDemo(page, "lovelace/tile-card");
await expectGalleryDemoElement(demo, "hui-tile-card");
});
test("more-info light renders controls", async ({ page }) => {
await goToGalleryPage(page, "more-info/light");
const demo = page.locator("ha-gallery >> demo-more-info-light");
await expect(demo).toBeAttached({ timeout: SHELL_TIMEOUT });
// Light more-info should contain a brightness or color-temp control
await expect(
demo
.locator("ha-control-slider, ha-more-info-light, more-info-content")
.first()
).toBeAttached({ timeout: SHELL_TIMEOUT });
});
const demo = await getGalleryDemo(page, "more-info/light");
test("more-info cover renders position controls", async ({ page }) => {
await goToGalleryPage(page, "more-info/cover");
const demo = page.locator("ha-gallery >> demo-more-info-cover");
await expect(demo).toBeAttached({ timeout: SHELL_TIMEOUT });
await expectGalleryDemoElement(
demo,
"ha-control-slider, ha-more-info-light, more-info-content",
SHELL_TIMEOUT
);
});
test("ha-gauge renders a gauge element", async ({ page }) => {
await goToGalleryPage(page, "components/ha-gauge");
const demo = page.locator("ha-gallery >> demo-components-ha-gauge");
await expect(demo).toBeAttached({ timeout: SHELL_TIMEOUT });
// ha-gauge page is markdown-based; gauge elements render in the description area
await expect(page.locator("ha-gallery >> ha-gauge").first()).toBeAttached({
await getGalleryDemo(page, "components/ha-gauge");
// ha-gauge page is markdown-based; gauge elements render in the description area.
await expect(galleryLocator(page, "ha-gauge").first()).toBeAttached({
timeout: QUICK_TIMEOUT,
});
});
test("ha-switch toggles state on click", async ({ page }) => {
await goToGalleryPage(page, "components/ha-switch");
const demo = page.locator("ha-gallery >> demo-components-ha-switch");
await expect(demo).toBeAttached({ timeout: SHELL_TIMEOUT });
const demo = await getGalleryDemo(page, "components/ha-switch");
// Find the first interactive (non-disabled) switch. Pull its checked state
// from the property ha-switch toggles via property, not the attribute.
// from the property because ha-switch toggles via property, not attribute.
const switchEl = demo.locator("ha-switch:not([disabled])").first();
await expect(switchEl).toBeAttached({ timeout: QUICK_TIMEOUT });
const before = await switchEl.evaluate((el: any) => el.checked === true);
const before = await switchEl.evaluate(
(el: HTMLElement & { checked?: boolean }) => el.checked === true
);
await switchEl.click();
await expect
.poll(() => switchEl.evaluate((el: any) => el.checked === true), {
timeout: QUICK_TIMEOUT,
})
.poll(
() =>
switchEl.evaluate(
(el: HTMLElement & { checked?: boolean }) => el.checked === true
),
{ timeout: QUICK_TIMEOUT }
)
.toBe(!before);
});
});
+98
View File
@@ -0,0 +1,98 @@
import { expect, type Locator, type Page } from "@playwright/test";
import {
defineParallelSmokeTests,
expectNoPageErrors,
QUICK_TIMEOUT,
SHELL_TIMEOUT,
trackPageErrors,
} from "../helpers";
export const GALLERY_SHELL_IGNORED_PAGE_ERRORS: RegExp[] = [
/ResizeObserver/,
/Non-Error/,
];
export const GALLERY_IGNORED_PAGE_ERRORS: RegExp[] = [
...GALLERY_SHELL_IGNORED_PAGE_ERRORS,
/Extension context/,
// Plain objects thrown by mock WebSocket/data-fetch show up as "Object".
/^Object$/,
// hui-group-entity-row calls .some() on a possibly-undefined entity_id array
// from mock state data - pre-existing gallery data issue.
/Cannot read properties of undefined \(reading 'some'\)/,
];
export interface GalleryPageSmokeCase {
name: string;
selector: string;
}
export const galleryLocator = (page: Page, selector: string) =>
page.locator(`ha-gallery >> ${selector}`);
const galleryDemoTag = (hash: string) => `demo-${hash.replace(/\//g, "-")}`;
async function waitForGalleryReady(page: Page) {
await expect(page.locator("ha-gallery")).toBeAttached({
timeout: SHELL_TIMEOUT,
});
}
export async function goToGalleryHome(page: Page) {
await page.goto("/");
await waitForGalleryReady(page);
}
export async function goToGalleryPage(page: Page, hash: string) {
await page.goto(`/#${hash}`);
}
async function expectGalleryPageSelector(page: Page, selector: string) {
const locator = galleryLocator(page, selector).first();
await expect(locator).toBeAttached({ timeout: SHELL_TIMEOUT });
return locator;
}
export async function getGalleryDemo(page: Page, hash: string) {
await goToGalleryPage(page, hash);
return expectGalleryPageSelector(page, galleryDemoTag(hash));
}
export async function assertGalleryPageLoads(
page: Page,
hash: string,
selector: string
) {
const errors = trackPageErrors(page);
await goToGalleryPage(page, hash);
await expectGalleryPageSelector(page, selector);
expectNoPageErrors(errors, hash, GALLERY_IGNORED_PAGE_ERRORS);
}
export function defineGallerySmokeTests(
groupName: string,
routePrefix: string,
pages: GalleryPageSmokeCase[]
) {
defineParallelSmokeTests({
groups: [{ name: groupName, routePrefix, pages }],
groupName: (group) => group.name,
cases: (group) => group.pages,
testName: (smokeCase) => `${smokeCase.name} renders without errors`,
run: async ({ page, group, smokeCase }) => {
await assertGalleryPageLoads(
page,
`${group.routePrefix}/${smokeCase.name}`,
smokeCase.selector
);
},
});
}
export async function expectGalleryDemoElement(
demo: Locator,
selector: string,
timeout = QUICK_TIMEOUT
) {
await expect(demo.locator(selector).first()).toBeAttached({ timeout });
}
+117
View File
@@ -0,0 +1,117 @@
import type { GalleryPageSmokeCase } from "./helpers";
export const componentPages: GalleryPageSmokeCase[] = [
{ name: "ha-alert", selector: "demo-components-ha-alert" },
{ name: "ha-badge", selector: "demo-components-ha-badge" },
{ name: "ha-bar", selector: "demo-components-ha-bar" },
{ name: "ha-button", selector: "demo-components-ha-button" },
{ name: "ha-chips", selector: "demo-components-ha-chips" },
{ name: "ha-control-button", selector: "demo-components-ha-control-button" },
{
name: "ha-control-circular-slider",
selector: "demo-components-ha-control-circular-slider",
},
{
name: "ha-control-number-buttons",
selector: "demo-components-ha-control-number-buttons",
},
{
name: "ha-control-select-menu",
selector: "demo-components-ha-control-select-menu",
},
{ name: "ha-control-select", selector: "demo-components-ha-control-select" },
{ name: "ha-control-slider", selector: "demo-components-ha-control-slider" },
{ name: "ha-control-switch", selector: "demo-components-ha-control-switch" },
{ name: "ha-dialog", selector: "demo-components-ha-dialog" },
{ name: "ha-dropdown", selector: "demo-components-ha-dropdown" },
{
name: "ha-expansion-panel",
selector: "demo-components-ha-expansion-panel",
},
{ name: "ha-faded", selector: "demo-components-ha-faded" },
{ name: "ha-form", selector: "demo-components-ha-form" },
{ name: "ha-gauge", selector: "demo-components-ha-gauge" },
{
name: "ha-hs-color-picker",
selector: "demo-components-ha-hs-color-picker",
},
{ name: "ha-input", selector: "demo-components-ha-input" },
{ name: "ha-label-badge", selector: "demo-components-ha-label-badge" },
{ name: "ha-list", selector: "demo-components-ha-list" },
{ name: "ha-marquee-text", selector: "demo-components-ha-marquee-text" },
{
name: "ha-progress-button",
selector: "demo-components-ha-progress-button",
},
{ name: "ha-select-box", selector: "demo-components-ha-select-box" },
{ name: "ha-selector", selector: "demo-components-ha-selector" },
{ name: "ha-slider", selector: "demo-components-ha-slider" },
{ name: "ha-spinner", selector: "demo-components-ha-spinner" },
{ name: "ha-switch", selector: "demo-components-ha-switch" },
{ name: "ha-textarea", selector: "demo-components-ha-textarea" },
{ name: "ha-tip", selector: "demo-components-ha-tip" },
{ name: "ha-tooltip", selector: "demo-components-ha-tooltip" },
{
name: "ha-adaptive-dialog",
selector: "demo-components-ha-adaptive-dialog",
},
{
name: "ha-adaptive-popover",
selector: "demo-components-ha-adaptive-popover",
},
];
export const moreInfoPages: GalleryPageSmokeCase[] = [
{ name: "light", selector: "demo-more-info-light" },
{ name: "climate", selector: "demo-more-info-climate" },
{ name: "cover", selector: "demo-more-info-cover" },
{ name: "fan", selector: "demo-more-info-fan" },
{ name: "humidifier", selector: "demo-more-info-humidifier" },
{ name: "input-number", selector: "demo-more-info-input-number" },
{ name: "input-text", selector: "demo-more-info-input-text" },
{ name: "lawn-mower", selector: "demo-more-info-lawn-mower" },
{ name: "lock", selector: "demo-more-info-lock" },
{ name: "media-player", selector: "demo-more-info-media-player" },
{ name: "number", selector: "demo-more-info-number" },
{ name: "scene", selector: "demo-more-info-scene" },
{ name: "timer", selector: "demo-more-info-timer" },
{ name: "update", selector: "demo-more-info-update" },
{ name: "vacuum", selector: "demo-more-info-vacuum" },
{ name: "water-heater", selector: "demo-more-info-water-heater" },
];
export const lovelacePages: GalleryPageSmokeCase[] = [
{ name: "area-card", selector: "demo-lovelace-area-card" },
{ name: "conditional-card", selector: "demo-lovelace-conditional-card" },
{ name: "entities-card", selector: "demo-lovelace-entities-card" },
{ name: "entity-button-card", selector: "demo-lovelace-entity-button-card" },
{ name: "entity-filter-card", selector: "demo-lovelace-entity-filter-card" },
{ name: "gauge-card", selector: "demo-lovelace-gauge-card" },
{ name: "glance-card", selector: "demo-lovelace-glance-card" },
{
name: "grid-and-stack-card",
selector: "demo-lovelace-grid-and-stack-card",
},
{ name: "iframe-card", selector: "demo-lovelace-iframe-card" },
{ name: "light-card", selector: "demo-lovelace-light-card" },
{ name: "map-card", selector: "demo-lovelace-map-card" },
{ name: "markdown-card", selector: "demo-lovelace-markdown-card" },
{ name: "media-control-card", selector: "demo-lovelace-media-control-card" },
{ name: "media-player-row", selector: "demo-lovelace-media-player-row" },
{ name: "picture-card", selector: "demo-lovelace-picture-card" },
{
name: "picture-elements-card",
selector: "demo-lovelace-picture-elements-card",
},
{
name: "picture-entity-card",
selector: "demo-lovelace-picture-entity-card",
},
{
name: "picture-glance-card",
selector: "demo-lovelace-picture-glance-card",
},
{ name: "thermostat-card", selector: "demo-lovelace-thermostat-card" },
{ name: "tile-card", selector: "demo-lovelace-tile-card" },
{ name: "todo-list-card", selector: "demo-lovelace-todo-list-card" },
];
+64 -17
View File
@@ -1,6 +1,7 @@
/**
* Shared helpers and constants for Playwright e2e suites.
*/
import { expect, test, type Page } from "@playwright/test";
// ── Timeouts ────────────────────────────────────────────────────────────────
// Centralised so tweaks don't require search-and-replace across spec files.
@@ -17,21 +18,67 @@ export const NAVIGATION_TIMEOUT = 30_000;
// ── Error filtering ─────────────────────────────────────────────────────────
/**
* Filter out errors known to be unrelated to the app under test:
* - ResizeObserver loop notifications (browser quirk, harmless)
* - Non-Error rejections (mock data throws plain objects)
* - Browser extension noise
*/
export function appErrors(errors: { message: string }[] | string[]) {
const messages =
typeof errors[0] === "string"
? (errors as string[])
: (errors as { message: string }[]).map((e) => e.message);
return messages.filter(
(msg) =>
!msg.includes("ResizeObserver") &&
!msg.includes("Non-Error") &&
!msg.includes("Extension context")
);
type PageError = { message: string } | string;
export const IGNORED_PAGE_ERRORS: RegExp[] = [
/ResizeObserver/,
/Non-Error/,
/Extension context/,
];
export function trackPageErrors(page: Page) {
const errors: PageError[] = [];
page.on("pageerror", (error) => errors.push(error));
return errors;
}
function pageErrors(errors: PageError[], ignoredErrors = IGNORED_PAGE_ERRORS) {
return errors
.map((error) => (typeof error === "string" ? error : error.message))
.filter((message) =>
ignoredErrors.every((pattern) => !pattern.test(message))
);
}
export function expectNoPageErrors(
errors: PageError[],
context?: string,
ignoredErrors = IGNORED_PAGE_ERRORS
) {
const realErrors = pageErrors(errors, ignoredErrors);
const details = realErrors.length ? `: ${realErrors.join("; ")}` : "";
expect(
realErrors,
context ? `JS errors on ${context}${details}` : `JS errors${details}`
).toHaveLength(0);
}
export interface DefineParallelSmokeTestsOptions<TGroup, TCase> {
groups: readonly TGroup[];
groupName: (group: TGroup) => string;
cases: (group: TGroup) => readonly TCase[];
testName: (smokeCase: TCase, group: TGroup) => string;
run: (context: {
page: Page;
group: TGroup;
smokeCase: TCase;
}) => Promise<void>;
}
export function defineParallelSmokeTests<TGroup, TCase>({
groups,
groupName,
cases,
testName,
run,
}: DefineParallelSmokeTestsOptions<TGroup, TCase>) {
for (const group of groups) {
test.describe(groupName(group), () => {
for (const smokeCase of cases(group)) {
test(testName(smokeCase, group), async ({ page }) => {
await run({ page, group, smokeCase });
});
}
});
}
}
+21
View File
@@ -0,0 +1,21 @@
const DEFAULT_LOCAL_WORKERS = "60%";
const VALID_WORKERS = /^[1-9]\d*%?$/;
export const getE2EWorkers = (): number | string => {
if (process.env.CI) {
return 1;
}
const workers = process.env.E2E_WORKERS;
if (!workers) {
return DEFAULT_LOCAL_WORKERS;
}
if (!VALID_WORKERS.test(workers)) {
throw new Error(
`E2E_WORKERS must be a positive integer or percentage, received "${workers}".`
);
}
return workers.endsWith("%") ? workers : Number(workers);
};
+4 -1
View File
@@ -1,4 +1,5 @@
import { defineConfig, devices } from "@playwright/test";
import { getE2EWorkers } from "./playwright-workers";
const APP_PORT = 8095;
const APP_BASE_URL = `http://localhost:${APP_PORT}`;
@@ -11,8 +12,10 @@ export default defineConfig({
expect: { timeout: 15_000 },
retries: process.env.CI ? 1 : 0,
fullyParallel: true,
workers: getE2EWorkers(),
outputDir: "test-results",
outputDir: "test-results/app",
reporter: [["list"], ["blob", { outputDir: "reports/app" }]],
use: {
+4 -1
View File
@@ -1,4 +1,5 @@
import { defineConfig, devices } from "@playwright/test";
import { getE2EWorkers } from "./playwright-workers";
// Port 8090 matches the `develop_demo` dev server (rspack-dev-server-demo).
// This means running `demo/script/develop_demo` and then `yarn test:e2e:local`
@@ -16,8 +17,10 @@ export default defineConfig({
expect: { timeout: 15_000 },
retries: process.env.CI ? 1 : 0,
fullyParallel: true,
workers: getE2EWorkers(),
outputDir: "test-results",
outputDir: "test-results/demo",
reporter: [["list"], ["blob", { outputDir: "reports/demo" }]],
use: {
+4 -1
View File
@@ -1,4 +1,5 @@
import { defineConfig, devices } from "@playwright/test";
import { getE2EWorkers } from "./playwright-workers";
const GALLERY_PORT = 8100;
const GALLERY_BASE_URL = `http://localhost:${GALLERY_PORT}`;
@@ -11,8 +12,10 @@ export default defineConfig({
expect: { timeout: 15_000 },
retries: process.env.CI ? 1 : 0,
fullyParallel: true,
workers: getE2EWorkers(),
outputDir: "test-results",
outputDir: "test-results/gallery",
reporter: [["list"], ["blob", { outputDir: "reports/gallery" }]],
use: {
+156 -26
View File
@@ -1,16 +1,106 @@
#!/usr/bin/env node
// Runs each e2e suite (demo, app, gallery) regardless of individual failures,
// then collects and merges blob reports and exits with a non-zero code if any
// suite failed.
// then collects and merges blob reports locally and exits with a non-zero code
// if any suite failed.
//
// Usage: node test/e2e/run-suites.mjs <suite> [<suite> ...]
// Where <suite> matches a test:e2e:<suite> script in package.json,
// e.g. "demo", "app", "gallery".
//
// Using ; or running suites independently avoids the && short-circuit problem
// where a failing suite skips the remaining suites and their blob reports.
// Running suites independently avoids the && short-circuit problem where a
// failing suite skips the remaining suites and their blob reports.
// Set E2E_WORKERS to a number or percentage to override local workers.
// Cold local builds run sequentially because suites share generated assets.
import { execFileSync } from "child_process";
import { execFileSync, spawn } from "child_process";
const TRUE_VALUES = new Set(["1", "true", "yes"]);
const SUITE_SERVERS = {
demo: { port: 8090, suite: "demo" },
app: { port: 8095, suite: "e2e-app" },
gallery: { port: 8100, suite: "gallery" },
};
const isTruthy = (value) => TRUE_VALUES.has(value?.toLowerCase() ?? "");
const hasManagedServer = async (suite) => {
const server = SUITE_SERVERS[suite];
if (!server) return false;
try {
const response = await fetch(
`http://localhost:${server.port}/__ha_dev_status`,
{ signal: AbortSignal.timeout(1000) }
);
if (!response.ok) return false;
const status = await response.json();
return status.server === "ha-frontend-dev" && status.suite === server.suite;
} catch {
return false;
}
};
const formatDuration = (ms) => {
const totalSeconds = Math.round(ms / 1000);
const minutes = Math.floor(totalSeconds / 60);
const seconds = totalSeconds % 60;
return minutes ? `${minutes}m ${seconds}s` : `${seconds}s`;
};
const writePrefixed = (suite, stream, chunk, pending) => {
const lines = `${pending.value}${chunk}`.split(/\r?\n/);
pending.value = lines.pop() ?? "";
for (const line of lines) {
stream.write(`[${suite}] ${line}\n`);
}
};
const flushPrefixed = (suite, stream, pending) => {
if (!pending.value) return;
stream.write(`[${suite}] ${pending.value}\n`);
pending.value = "";
};
const runSuite = (suite, env) =>
new Promise((resolve) => {
const started = Date.now();
const child = spawn("yarn", [`test:e2e:${suite}`], {
stdio: ["ignore", "pipe", "pipe"],
env,
});
const pendingStdout = { value: "" };
const pendingStderr = { value: "" };
const workerLabel = env.E2E_WORKERS ? ` (workers: ${env.E2E_WORKERS})` : "";
process.stdout.write(
`\n--- Running suite: test:e2e:${suite}${workerLabel} ---\n`
);
child.stdout.on("data", (chunk) =>
writePrefixed(suite, process.stdout, chunk, pendingStdout)
);
child.stderr.on("data", (chunk) =>
writePrefixed(suite, process.stderr, chunk, pendingStderr)
);
child.on("error", (err) => {
flushPrefixed(suite, process.stdout, pendingStdout);
flushPrefixed(suite, process.stderr, pendingStderr);
process.stderr.write(`[${suite}] Failed to start: ${err.message}\n`);
resolve({ suite, code: 1, duration: Date.now() - started });
});
child.on("close", (code) => {
flushPrefixed(suite, process.stdout, pendingStdout);
flushPrefixed(suite, process.stderr, pendingStderr);
const duration = Date.now() - started;
process.stdout.write(
`--- Finished suite: test:e2e:${suite} (${formatDuration(duration)}) ---\n`
);
resolve({ suite, code: code ?? 1, duration });
});
});
const suites = process.argv.slice(2);
if (!suites.length) {
@@ -18,32 +108,72 @@ if (!suites.length) {
process.exit(1);
}
const failures = [];
const hasAllManagedServers = (
await Promise.all(suites.map(hasManagedServer))
).every(Boolean);
const sequential =
isTruthy(process.env.E2E_SEQUENTIAL) ||
(suites.length > 1 && !hasAllManagedServers);
const skipMerge = isTruthy(process.env.E2E_SKIP_MERGE);
const suiteWorkers =
!sequential &&
!process.env.CI &&
!process.env.E2E_WORKERS &&
suites.length > 1
? `${Math.max(1, Math.floor(60 / suites.length))}%`
: undefined;
const suiteEnv = suiteWorkers
? { ...process.env, E2E_WORKERS: suiteWorkers }
: process.env;
for (const suite of suites) {
process.stdout.write(`\n--- Running suite: test:e2e:${suite} ---\n`);
try {
execFileSync("yarn", [`test:e2e:${suite}`], { stdio: "inherit" });
} catch {
failures.push(suite);
const results = [];
if (sequential) {
if (!isTruthy(process.env.E2E_SEQUENTIAL)) {
process.stdout.write(
"Running suites sequentially because not all managed dev servers are available.\n"
);
}
for (const suite of suites) {
// eslint-disable-next-line no-await-in-loop
results.push(await runSuite(suite, suiteEnv));
}
} else {
results.push(
...(await Promise.all(suites.map((suite) => runSuite(suite, suiteEnv))))
);
}
const failures = results
.filter(({ code }) => code !== 0)
.map(({ suite }) => suite);
process.stdout.write("\nE2E suite timings:\n");
for (const { suite, duration } of results) {
process.stdout.write(`- test:e2e:${suite}: ${formatDuration(duration)}\n`);
}
// Collect and merge blob reports regardless of suite outcomes.
execFileSync("node", ["test/e2e/collect-blob-reports.mjs"], {
stdio: "inherit",
});
execFileSync(
"npx",
[
"playwright",
"merge-reports",
"-c",
"test/e2e/playwright.merge.config.ts",
"test/e2e/reports/blob",
],
{ stdio: "inherit" }
);
if (skipMerge) {
process.stdout.write(
"\nSkipping merged e2e report because E2E_SKIP_MERGE is set.\n"
);
} else {
execFileSync("node", ["test/e2e/collect-blob-reports.mjs"], {
stdio: "inherit",
});
execFileSync(
"npx",
[
"playwright",
"merge-reports",
"-c",
"test/e2e/playwright.merge.config.ts",
"test/e2e/reports/blob",
],
{ stdio: "inherit" }
);
}
if (failures.length) {
process.stderr.write(