Compare commits

...
4 Commits
Author SHA1 Message Date
Bram KragtenandClaude Opus 5 709d22e777 Recover the map after a reconnect
The token refresh ran on an interval only, which does not fire while the
process is suspended, so a reconnect came back with a token the proxy
refuses. A refresh on the connection's "ready" event covers that.

A fresh token alone was not enough: the vector source is built from a
TileJSON fetched once and never retried, so a refused request left the map
blank. The style is now re-applied when a token arrives after a refusal,
throttled so a proxy refusing for another reason cannot loop. The raster
layer redraws for the same reason - refused tiles are cached as failures.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-28 11:54:13 +02:00
Bram KragtenandClaude Opus 5 cb4e925321 Mock the tile token in the onboarding e2e, and trim comments
The onboarding WebSocket mock throws on any command it does not know, so
asking it for a tile token failed the test - which is the mock doing its
job. Answered like `brands/access_token` next to it, and the tile
requests that follow are answered too, so the result does not depend on
what the dev server does with an unknown /api path.

Also thins out the comments on the files this branch touches: 22% and 17%
comment lines against 2-3% in the map components beside them. What went
was mostly measurement narrative that belongs in the commit and the PR,
not above a constant.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-27 20:47:57 +02:00
Bram KragtenandClaude Opus 5 69c1735ce6 Cache map tiles past the rotating token, and keep the demo working
The proxy token rotates every half hour and rides in the query string,
which the HTTP cache keys on - so every rotation would refetch every tile
a dashboard has ever shown, and `/api/` is NetworkOnly in the service
worker so nothing was cached there at all. The brands route already
solves exactly this, so map tiles now follow it: a CacheFirst route that
strips the token from the cache key. Bounded to 500 entries and a week,
because tiles are 20-145 kB each and this runs on phones. The TileJSON is
deliberately left on the network - it is the switching point for the tile
source, and caching it would be caching the escape hatch.

The demo has no core to proxy through, so it goes to the upstreams
directly: tiles from OSM, which sets CORS on that endpoint, and glyphs
and sprites from VersaTiles, which sets CORS on everything. Same
Shortbread style, so the demo looks like the product. It sends a referrer
there, which OSM asks a website for and which costs nothing here - one
public site, no instance hostname to leak.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-27 19:16:07 +02:00
Bram KragtenandClaude Opus 5 be6461ed9a Load the base map through core's tile proxy
Core now proxies both vector and raster tiles, which is what lets them be
requested with an application User-Agent and without a referrer - a
browser can set neither. So the frontend stops talking to OpenStreetMap
and CARTO directly and goes through /api/map_tiles.

The proxy is token gated, so `ha-map` fetches one over the WebSocket
before setting up, following the brands token pattern: cached at module
level, refreshed well inside its lifetime so a dashboard left open for
days keeps working. The blocking wait is kept to about a second - a
backend without the proxy must not hold the map hostage - and the
remaining retries run in the background to ride through the window after
a restart where the WebSocket is up but the handler is not registered
yet.

Two things that are not obvious and cost a measurement each:

MapLibre's `transformRequest` has to return absolute URLs. Tiles are
fetched from a worker, which has no document to resolve a relative URL
against, and the TileJSON that core serves has relative `tiles`. Measured
with a relative TileJSON on one origin: with absolute URLs the style
loads and 8,456 features render; without them exactly one request is made
- the style - and nothing else loads, with no error reported anywhere.

Leaflet bakes its URL template at layer creation and throws while
building a tile URL if a template variable is undefined. So the raster
layer takes the token as an option Leaflet substitutes per request, which
also means a refreshed token is picked up without recreating the layer,
and an absent one is empty rather than missing: the tiles 403 and the
markers still draw.

The asset pipeline shrinks to generating two styles. Glyphs and sprites
come from the proxy, so the 48 MB build-time download, the digest
verification, the glyph range filtering and the bold-range guard are all
gone, along with 5.4 MB from the wheel. `localIdeographFontFamily` goes
too: the complete glyph set is reachable now, so CJK renders in Noto
rather than a device font.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-27 19:16:07 +02:00
10 changed files with 607 additions and 261 deletions
-2
View File
@@ -75,5 +75,3 @@ test/e2e/app/dist/
test/benchmarks/results/
# Downloaded map glyph and sprite archives
.map-assets/
+19 -151
View File
@@ -1,136 +1,25 @@
// Assembles the static assets of the vector base map - style, SDF glyphs and
// icon sprites - into /static/map/. vector.openstreetmap.org sets CORS headers
// on its tiles only, so these cannot be loaded from there by a browser.
// Generates the MapLibre styles for the vector base map.
//
// Glyphs and sprites come from pinned VersaTiles releases, cached locally and
// verified against a digest.
// Only the styles. Glyphs, sprites and tiles are served by core's proxy, which
// is what lets them be requested with an application User-Agent and without a
// referrer. The styles stay here because they come from @versatiles/style and
// core has no node toolchain to regenerate them with.
import { createHash } from "node:crypto";
import { readFile, writeFile } from "node:fs/promises";
import { writeFile } from "node:fs/promises";
import path from "node:path";
import { colorful, eclipse } from "@versatiles/style";
import fs from "fs-extra";
import gulp from "gulp";
import { extract } from "tar";
import paths from "../paths.cjs";
// The tile URL is deliberately never named here: the OSMF asks consumers to
// resolve it through the TileJSON so they can move the tiles without every
// client needing a release. https://operations.osmfoundation.org/policies/vector/
const TILEJSON_URL =
"https://vector.openstreetmap.org/shortbread_v1/tilejson.json";
const PROXY_PATH = "/api/map_tiles";
const TILEJSON_URL = `${PROXY_PATH}/tilejson.json`;
const ASSET_PATH = "/static/map";
const ARCHIVES = {
fonts: {
url: "https://github.com/versatiles-org/versatiles-fonts/releases/download/v2.2.0/noto_sans.tar.gz",
sha256: "a2dac39f4096722bc420367ffd4a36687cce7229e8aa760bc12cf657072eea6b",
},
sprites: {
url: "https://github.com/versatiles-org/versatiles-style/releases/download/v5.13.1/sprites.tar.gz",
sha256: "efffd0ee4cb9591bd52f16ff5b269d9618c7dd1db159cd6511943965560ddea5",
},
};
// Rendered with a device font through `localIdeographFontFamily`, so these are
// never downloaded - and they are 90% of the Noto Sans SDF set.
const LOCAL_IDEOGRAPH_BLOCKS = [
[0x2e80, 0x9fff], // CJK radicals through CJK Unified Ideographs, incl. kana
[0xac00, 0xd7ff], // Hangul syllables
[0xf900, 0xfaff], // CJK compatibility ideographs
[0xfe30, 0xfe4f], // CJK compatibility forms
];
// Our styles use bold only for motorway shields, so Latin, Greek and Cyrillic
// cover every ref and the rest of bold - half the glyph set - is dropped.
// `assertBoldStaysOnRefs` guards the assumption.
const BOLD_MAX_CODEPOINT = 0x04ff;
const BOLD_TEXT_FIELD = "{ref}";
const cacheDir = path.resolve(paths.root_dir, ".map-assets");
const outputDir = path.resolve(paths.build_dir, "map");
const sha256 = (buffer) => createHash("sha256").update(buffer).digest("hex");
// Downloads an archive into the cache, or reuses it when the digest matches.
const cachedArchive = async (name, { url, sha256: expected }) => {
const file = path.join(cacheDir, `${name}.tar.gz`);
if (await fs.pathExists(file)) {
if (sha256(await readFile(file)) === expected) {
return file;
}
console.warn("Cached map %s archive is stale, downloading again", name);
}
console.log("Downloading map %s from %s", name, url);
const response = await fetch(url);
if (!response.ok) {
throw new Error(
`Failed to download map ${name}: ${response.status} ${response.statusText}`
);
}
const body = Buffer.from(await response.arrayBuffer());
const digest = sha256(body);
if (digest !== expected) {
throw new Error(
`Digest mismatch for map ${name}: expected ${expected}, got ${digest}`
);
}
await fs.outputFile(file, body);
return file;
};
const glyphRange = (entryPath) => {
const match = /^(?<font>[^/]+)\/(?<start>\d+)-(?<end>\d+)\.pbf$/.exec(
entryPath
);
return match
? { font: match.groups.font, start: Number(match.groups.start) }
: undefined;
};
const keepGlyph = (entryPath) => {
const range = glyphRange(entryPath);
if (!range) {
return false;
}
if (range.font.endsWith("_bold") && range.start > BOLD_MAX_CODEPOINT) {
return false;
}
return !LOCAL_IDEOGRAPH_BLOCKS.some(
([from, to]) => range.start >= from && range.start <= to
);
};
const keepSprite = (entryPath) =>
/^basics\/sprites(@2x)?\.(json|png)$/.test(entryPath);
// `neutrino`, for one, sets country and state labels in bold - names in any
// script, which would turn to tofu. Fail the build rather than ship that.
const assertBoldStaysOnRefs = (name, style) => {
const offenders = style.layers
.filter((layer) =>
(layer.layout?.["text-font"] ?? []).some((font) => font.endsWith("_bold"))
)
.filter((layer) => layer.layout["text-field"] !== BOLD_TEXT_FIELD)
.map((layer) => layer.id);
if (offenders.length) {
throw new Error(
`Style "${name}" uses bold for ${offenders.join(", ")}, which can hold ` +
`names in any script. Raise BOLD_MAX_CODEPOINT to cover the full set ` +
`before shipping this style.`
);
}
};
// MapLibre extends the fetched TileJSON with the style's source options, so
// anything left here wins and freezes at build time. Dropping them is what lets
// a remote switch move the attribution and zoom range too, not just the URLs.
// the proxy move the attribution and zoom range too, not just the URLs.
const TILEJSON_FIELDS = [
"tiles",
"attribution",
@@ -163,46 +52,25 @@ const useTileJson = (name, style) => {
const styleOptions = {
// Keeps the generated URLs origin relative.
baseUrl: "",
glyphs: `${ASSET_PATH}/fonts/{fontstack}/{range}.pbf`,
sprite: [{ id: "basics", url: `${ASSET_PATH}/sprites/basics/sprites` }],
glyphs: `${PROXY_PATH}/fonts/{fontstack}/{range}.pbf`,
sprite: [{ id: "basics", url: `${PROXY_PATH}/sprites/basics/sprites` }],
};
const buildMapAssets = async () => {
const [fontArchive, spriteArchive] = await Promise.all([
cachedArchive("fonts", ARCHIVES.fonts),
cachedArchive("sprites", ARCHIVES.sprites),
]);
await fs.emptyDir(outputDir);
await Promise.all([
fs.ensureDir(path.join(outputDir, "fonts")),
fs.ensureDir(path.join(outputDir, "sprites")),
]);
await Promise.all([
extract({
file: fontArchive,
cwd: path.join(outputDir, "fonts"),
filter: keepGlyph,
}),
extract({
file: spriteArchive,
cwd: path.join(outputDir, "sprites"),
filter: keepSprite,
}),
await Promise.all(
// Both themes up front: dark is a real cartography, not an inverted raster.
...[
[
["light", colorful],
["dark", eclipse],
].map(([name, builder]) => {
const style = useTileJson(name, builder(styleOptions));
assertBoldStaysOnRefs(name, style);
return writeFile(
].map(([name, builder]) =>
writeFile(
path.join(outputDir, `${name}.json`),
JSON.stringify(style)
);
}),
]);
JSON.stringify(useTileJson(name, builder(styleOptions)))
)
)
);
};
// Shared so it does not have to be wired into every pipeline separately.
+3 -1
View File
@@ -19,6 +19,7 @@ export const setupLeafletMap = async (
longitude: number;
zoom?: number;
darkMode?: boolean;
token?: string;
}
): Promise<LeafletMapSetup> => {
if (!mapElement.parentNode) {
@@ -60,7 +61,8 @@ export const setupLeafletMap = async (
const baseLayer = await createBaseLayer(
Leaflet,
map,
initialView?.darkMode ?? false
initialView?.darkMode ?? false,
initialView?.token
);
return { map, leaflet: Leaflet, baseLayer };
+131 -67
View File
@@ -1,38 +1,40 @@
import type { maplibreGL } from "@maplibre/maplibre-gl-leaflet";
import type { Map as LeafletMap } from "leaflet";
import type { Map as LeafletMap, TileLayerOptions } from "leaflet";
import type { StyleSpecification } from "maplibre-gl";
import type { LeafletModuleType } from "../dom/setup-leaflet-map";
import {
MAP_TILES_PATH,
refreshMapTilesToken,
subscribeMapTilesToken,
withMapTilesToken,
} from "../../data/map_tiles";
// Shortbread vector tiles from the OpenStreetMap Foundation. Only their tile
// endpoint sends CORS headers, so the style, glyphs and sprites are ours to
// serve - see build-scripts/gulp/map-assets.js. The credit comes from the
// TileJSON rather than from here, deliberately: it follows whoever serves the
// tiles.
// Generated by build-scripts/gulp/map-assets.js. The attribution comes from the
// TileJSON, deliberately: it follows whoever serves the tiles.
const VECTOR_STYLES = {
light: "/static/map/light.json",
dark: "/static/map/dark.json",
} as const;
// Fallback for browsers without WebGL2, which MapLibre needs even for raster,
// so it stays a Leaflet tile layer. Still CARTO, and temporarily so: OSM's
// raster blocks a browser that sends no Referer, and the only referrer a browser
// can send is its origin, which identifies a Nabu Casa installation.
const RASTER_TILE_URL = "https://basemaps.cartocdn.com/rastertiles/voyager";
const CARTO_ATTRIBUTION =
'&copy; <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a>, ' +
'&copy; <a href="https://carto.com/attributions">CARTO</a>';
// MapLibre needs WebGL2 even for raster, so the fallback stays a Leaflet layer.
// OSM serves no @2x variant.
const RASTER_TILE_URL = `${MAP_TILES_PATH}/raster/{z}/{x}/{y}.png?token={token}`;
const OSM_ATTRIBUTION =
'&copy; <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a> contributors';
// Browsers keep about 16 live WebGL contexts and drop the oldest, so a dashboard
// full of map cards loses its first ones for good - nothing frees a slot for
// MapLibre to reclaim. A transient loss does get restored, hence the grace.
// Browsers keep about 16 live WebGL contexts and drop the oldest, which a
// dashboard full of map cards hits. A transient loss is restored, hence a grace.
const CONTEXT_RESTORE_GRACE = 2000;
const RECOVERY_THROTTLE = 30000;
// On the map, not the layer: only Leaflet tile layers report their own limits,
// and marker clustering throws without a maximum. The floor is 1 because at
// Leaflet zoom 0 the adapter drives MapLibre to -1, outside its range.
// On the map, not the layer: marker clustering throws without a maximum. The
// floor is 1 because at Leaflet zoom 0 the adapter drives MapLibre to -1.
export const MAP_MIN_ZOOM = 1;
export const MAP_MAX_ZOOM = 20;
// Leaflet substitutes any option into the URL template; its types do not.
type TokenTileLayerOptions = TileLayerOptions & { token?: string };
export interface MapBaseLayer {
// A no-op for raster, which has no dark variant and is inverted in CSS.
setDarkMode: (darkMode: boolean) => void;
@@ -40,7 +42,7 @@ export interface MapBaseLayer {
let webGL2Supported: boolean | undefined;
// Rules out iOS below 15, older Android tablets, and blocklisted drivers.
// Rules out iOS below 15, older Android tablets and blocklisted drivers.
const supportsWebGL2 = (): boolean => {
if (webGL2Supported === undefined) {
try {
@@ -55,11 +57,35 @@ const supportsWebGL2 = (): boolean => {
return webGL2Supported;
};
// Asset URLs are stored origin relative so they follow the instance's host, but
// MapLibre rejects a relative sprite URL. The glyph URL is left alone: URL
// encoding would mangle its {fontstack} and {range} placeholders.
// MapLibre rejects a relative sprite URL. Not the glyph URL: encoding would
// mangle its {fontstack} and {range} placeholders.
// The demo has no core to proxy through. OSM sets CORS on its tiles but not on
// its glyphs and sprites, which is why those come from VersaTiles.
const DEMO_UPSTREAM = {
tilejson: "https://vector.openstreetmap.org/shortbread_v1/tilejson.json",
assets: "https://tiles.versatiles.org/assets",
};
const useDemoUpstream = (style: StyleSpecification): StyleSpecification => {
style.glyphs = `${DEMO_UPSTREAM.assets}/glyphs/{fontstack}/{range}.pbf`;
style.sprite = [
{ id: "basics", url: `${DEMO_UPSTREAM.assets}/sprites/basics/sprites` },
];
Object.values(style.sources).forEach((source) => {
if ("url" in source) {
source.url = DEMO_UPSTREAM.tilejson;
}
});
return style;
};
const loadStyle = async (url: string): Promise<StyleSpecification> => {
const style: StyleSpecification = await (await fetch(url)).json();
if (__DEMO__) {
return useDemoUpstream(style);
}
if (typeof style.sprite === "string") {
style.sprite = new URL(style.sprite, location.href).href;
} else if (Array.isArray(style.sprite)) {
@@ -75,21 +101,24 @@ const createVectorLayer = async (
createLayer: typeof maplibreGL,
leaflet: LeafletModuleType,
map: LeafletMap,
darkMode: boolean
darkMode: boolean,
token: string | undefined
): Promise<MapBaseLayer | undefined> => {
let layer: ReturnType<typeof maplibreGL> | undefined;
try {
layer = createLayer({
style: await loadStyle(VECTOR_STYLES[darkMode ? "dark" : "light"]),
// Draws CJK, kana and hangul with a device font, which is why we ship a
// tenth of the glyph set. No referrer is set: the endpoint serves without
// one, and the only one a browser can send identifies the installation.
localIdeographFontFamily: "sans-serif",
// Absolute, or the worker fetching tiles cannot resolve them.
transformRequest: (url) => ({
url: withMapTilesToken(url),
// OSM asks a website for a referrer, and the demo has no instance
// hostname to leak.
referrerPolicy: __DEMO__ ? "origin" : undefined,
}),
});
// The plugin builds the MapLibre map in `onAdd`, so a refused context, a
// blocked worker or a rejected blob URL throws here. Keep it inside the
// guard or those lose the raster fallback.
// The plugin builds the MapLibre map in `onAdd`, so a refused context or a
// blocked worker throws here. Keep it guarded or those lose the fallback.
layer.addTo(map);
} catch {
if (layer) {
@@ -102,14 +131,14 @@ const createVectorLayer = async (
return undefined;
}
// Tracked apart because a failed request must roll back to what is displayed,
// not to the opposite of what it asked for - with several in flight those are
// different, and guessing wrong makes the next toggle a permanent no-op.
// Tracked apart so a failed request rolls back to what is displayed, not to
// the opposite of what it asked - which with several in flight differs.
let appliedDarkMode = darkMode;
let requestedDarkMode = darkMode;
// Styles are fetched, so only the newest request may touch the map.
let latestRequest = 0;
let vector = true;
let refused = false;
const glMap = layer.getMaplibreMap();
let fallbackTimeout: number | undefined;
@@ -130,13 +159,12 @@ const createVectorLayer = async (
} catch {
// Nothing left to detach.
}
createRasterLayer(leaflet, map);
createRasterLayer(leaflet, map, token);
};
const scheduleSwap = () => {
clearTimeout(fallbackTimeout);
// Backgrounding also drops the context, and there it comes back on return.
// Running the clock then would make switching apps enough to lose vector.
// Backgrounding drops it too, and there it comes back on return.
if (!vector || document.hidden) {
return;
}
@@ -159,54 +187,89 @@ const createVectorLayer = async (
document.removeEventListener("visibilitychange", handleVisibilityChange);
});
const applyStyle = (newDarkMode: boolean) => {
const request = ++latestRequest;
loadStyle(VECTOR_STYLES[newDarkMode ? "dark" : "light"])
.then((style) => {
if (request === latestRequest) {
appliedDarkMode = newDarkMode;
refused = false;
layer.getMaplibreMap()?.setStyle(style);
}
})
.catch(() => {
if (request === latestRequest) {
requestedDarkMode = appliedDarkMode;
}
});
};
// A refused request leaves the source dead: the TileJSON is fetched once and
// is never retried, so the style has to be applied again once there is a new
// token. Throttled, or a proxy refusing for another reason loops.
let lastRecovery = 0;
glMap.on("error", (event) => {
if ((event.error as { status?: number } | undefined)?.status !== 403) {
return;
}
if (Date.now() - lastRecovery < RECOVERY_THROTTLE) {
return;
}
lastRecovery = Date.now();
refused = true;
refreshMapTilesToken();
});
const unsubscribeToken = subscribeMapTilesToken(() => {
if (vector && refused) {
applyStyle(requestedDarkMode);
}
});
map.on("unload", unsubscribeToken);
return {
setDarkMode: (newDarkMode: boolean) => {
if (!vector || newDarkMode === requestedDarkMode) {
return;
}
requestedDarkMode = newDarkMode;
const request = ++latestRequest;
loadStyle(VECTOR_STYLES[newDarkMode ? "dark" : "light"])
.then((style) => {
if (request === latestRequest) {
appliedDarkMode = newDarkMode;
layer.getMaplibreMap()?.setStyle(style);
}
})
.catch(() => {
if (request === latestRequest) {
requestedDarkMode = appliedDarkMode;
}
});
applyStyle(newDarkMode);
},
};
};
const createRasterLayer = (
leaflet: LeafletModuleType,
map: LeafletMap
map: LeafletMap,
token: string | undefined
): MapBaseLayer => {
leaflet
.tileLayer(
// These are the old retina tablets, and CARTO serves @2x - sharp tiles at
// the same request count, where `detectRetina` would fetch a zoom deeper
// at four times as many.
`${RASTER_TILE_URL}/{z}/{x}/{y}${leaflet.Browser.retina ? "@2x" : ""}.png`,
{
attribution: CARTO_ATTRIBUTION,
maxZoom: MAP_MAX_ZOOM,
}
)
const layer = leaflet
.tileLayer(RASTER_TILE_URL, {
attribution: OSM_ATTRIBUTION,
maxZoom: MAP_MAX_ZOOM,
// Leaflet throws on an undefined template variable, so no token means an
// empty one: the tiles 403 and the markers still draw.
token: token ?? "",
} as TokenTileLayerOptions)
.addTo(map);
// Substituted per request, so a refreshed token needs no new layer.
const unsubscribe = subscribeMapTilesToken((newToken) => {
(layer.options as TokenTileLayerOptions).token = newToken;
// Tiles that 403'd are cached as failures; only a redraw asks again.
layer.redraw();
});
map.on("unload", unsubscribe);
return { setDarkMode: () => undefined };
};
export const createBaseLayer = async (
leaflet: LeafletModuleType,
map: LeafletMap,
darkMode: boolean
darkMode: boolean,
token: string | undefined
): Promise<MapBaseLayer> => {
if (supportsWebGL2()) {
let vectorLayer: MapBaseLayer | undefined;
@@ -217,7 +280,8 @@ export const createBaseLayer = async (
createLayer,
leaflet,
map,
darkMode
darkMode,
token
);
} catch {
// No chunk, no vector map - but still a map.
@@ -226,5 +290,5 @@ export const createBaseLayer = async (
return vectorLayer;
}
}
return createRasterLayer(leaflet, map);
return createRasterLayer(leaflet, map, token);
};
+9
View File
@@ -29,6 +29,7 @@ import type { MapBaseLayer } from "../../common/map/base-layer";
import { computeStateDomain } from "../../common/entity/compute_state_domain";
import { computeStateName } from "../../common/entity/compute_state_name";
import { getEntityLocation } from "../../common/entity/get_entity_location";
import { ensureMapTilesToken } from "../../data/map_tiles";
import { DecoratedMarker } from "../../common/map/decorated_marker";
import { filterXSS } from "../../common/util/xss";
import {
@@ -327,11 +328,19 @@ export class HaMap extends ReactiveElement {
}
this._loading = true;
try {
// The tiles are proxied by core behind a token, so nothing loads without
// one. A host that provides no connection, or a backend without the
// proxy, leaves the map without tiles rather than failing to set up.
const token = this._connection
? await ensureMapTilesToken(this._connection.connection)
: undefined;
const setup = await setupLeafletMap(map, {
latitude: this._config?.latitude ?? 52.3731339,
longitude: this._config?.longitude ?? 4.8903147,
zoom: this.zoom,
darkMode: this._darkMode,
token,
});
// Setting up fetches a style, so the element can be gone by now.
// `disconnectedCallback` had no map to tear down, and keeping this one
+136
View File
@@ -0,0 +1,136 @@
import type { Connection } from "home-assistant-js-websocket";
import { waitForMs } from "../common/util/wait";
export const MAP_TILES_PATH = "/api/map_tiles";
// Well inside the token's lifetime, as util/brands-url.ts does.
const TOKEN_REFRESH_MS = 30 * 60 * 1000;
// Nothing loads without a token, so the first attempts are awaited - briefly,
// or a backend without the proxy would hold the map hostage. The rest retry in
// the background, for the window after a restart where the WebSocket is up but
// the handler is not registered yet.
const BLOCKING_DELAYS_MS = [0, 400, 1000];
const BACKGROUND_DELAYS_MS = [2000, 5000, 10000, 15000];
let token: string | undefined;
let refreshInterval: ReturnType<typeof setInterval> | undefined;
let watchingConnection = false;
let activeConnection: Connection | undefined;
let refreshing: Promise<void> | undefined;
const listeners = new Set<(token: string) => void>();
const fetchToken = async (connection: Connection): Promise<void> => {
const result = await connection.sendMessagePromise<{ token: string }>({
type: "map_tiles/access_token",
});
if (result.token !== token) {
token = result.token;
listeners.forEach((listener) => listener(token!));
}
};
const attempt = async (connection: Connection, delays: number[]) => {
/* eslint-disable no-await-in-loop -- retries are intentionally sequential */
for (const delay of delays) {
if (token) {
return;
}
if (delay) {
await waitForMs(delay);
}
try {
await fetchToken(connection);
return;
} catch {
// try next delay
}
}
/* eslint-enable no-await-in-loop */
};
export const ensureMapTilesToken = async (
connection: Connection
): Promise<string | undefined> => {
activeConnection = connection;
if (!token) {
await attempt(connection, BLOCKING_DELAYS_MS);
}
if (!token) {
attempt(connection, BACKGROUND_DELAYS_MS).then(() =>
scheduleRefresh(connection)
);
return undefined;
}
scheduleRefresh(connection);
return token;
};
const scheduleRefresh = (connection: Connection) => {
if (token && !refreshInterval) {
refreshInterval = setInterval(() => {
fetchToken(connection).catch(() => {
// Keep the current token; the next interval retries.
});
}, TOKEN_REFRESH_MS);
}
if (!watchingConnection) {
watchingConnection = true;
// The interval does not fire while the process is suspended, so the token
// can be stale before it comes round; reconnecting is the reliable signal.
connection.addEventListener("ready", () => {
fetchToken(connection).catch(() => {
// Nothing to do; the interval keeps trying.
});
});
}
};
/**
* Forces a new token, for when the current one is refused. Deduplicated: a
* refused map produces one of these per tile.
*/
export const refreshMapTilesToken = (): Promise<void> => {
if (!activeConnection) {
return Promise.resolve();
}
refreshing ??= fetchToken(activeConnection)
.catch(() => {
// Leave the old token in place; a reconnect or the interval retries.
})
.finally(() => {
refreshing = undefined;
});
return refreshing;
};
/** Leaflet bakes its URL template at layer creation, so it needs telling. */
export const subscribeMapTilesToken = (
listener: (token: string) => void
): (() => void) => {
listeners.add(listener);
return () => listeners.delete(listener);
};
/**
* MapLibre hands tile URLs to a worker, which has no document to resolve a
* relative URL against, so the result has to be absolute.
*/
export const withMapTilesToken = (url: string): string => {
let parsed: URL;
try {
parsed = new URL(url, location.href);
} catch {
return url;
}
if (token && parsed.pathname.startsWith(`${MAP_TILES_PATH}/`)) {
parsed.searchParams.set("token", token);
}
return parsed.href;
};
+27
View File
@@ -117,6 +117,33 @@ const initRouting = () => {
})
);
// Strip the rotating token from the cache key, or every rotation refetches
// every tile. The TileJSON is deliberately not matched: it is the switching
// point for the tile source, so it stays on the network.
registerRoute(
({ url }) =>
/^\/api\/map_tiles\/(vector|raster|fonts|sprites)\//.test(url.pathname),
new CacheFirst({
cacheName: "map-tiles",
plugins: [
{
cacheKeyWillBeUsed: async ({ request }) => {
const url = new URL(request.url);
url.searchParams.delete("token");
return url.href;
},
},
// A stale token gives a 403; caching that would pin the failure.
new CacheableResponsePlugin({ statuses: [0, 200] }),
new ExpirationPlugin({
maxEntries: 500,
maxAgeSeconds: 60 * 60 * 24 * 7,
purgeOnQuotaError: true,
}),
],
})
);
// Short-circuit camera/image proxy requests with an expired signature or a
// missing/undefined token so they don't hit core and get logged as invalid
// login attempts. Registered before the generic /api route below so it wins.
+109 -40
View File
@@ -18,6 +18,22 @@ const maplibreGL = vi.hoisted(() => vi.fn(() => maplibreLayer));
vi.mock("@maplibre/maplibre-gl-leaflet", () => ({ maplibreGL }));
// The token module is driven directly here, so a stale token and the one that
// replaces it can be played out without a WebSocket.
const tokenListeners = vi.hoisted(() => new Set<(token: string) => void>());
const refreshMapTilesToken = vi.hoisted(() => vi.fn());
vi.mock("../../../src/data/map_tiles", () => ({
MAP_TILES_PATH: "/api/map_tiles",
refreshMapTilesToken,
subscribeMapTilesToken: (listener: (token: string) => void) => {
tokenListeners.add(listener);
return () => tokenListeners.delete(listener);
},
withMapTilesToken: (url: string) => new URL(url, location.href).href,
}));
const emitToken = (token: string) =>
tokenListeners.forEach((listener) => listener(token));
const STYLE = {
version: 8,
sources: {},
@@ -25,19 +41,23 @@ const STYLE = {
sprite: [{ id: "basics", url: "/static/map/sprites/basics/sprites" }],
};
const rasterLayer = { addTo: vi.fn() };
// Kept as a local so tests can flip it: @types/leaflet has it readonly.
const browser = { retina: false };
const rasterLayer = {
// Leaflet returns the layer from addTo, and the source chains off it.
addTo: vi.fn(() => rasterLayer),
redraw: vi.fn(),
options: {} as Record<string, unknown>,
};
const leaflet = {
tileLayer: vi.fn(() => rasterLayer),
Browser: browser,
} as unknown as LeafletModuleType;
const TOKEN = "test-token";
// `createVectorLayer` listens on both the MapLibre map and the Leaflet map.
const glHandlers: Record<string, () => void> = {};
const glHandlers: Record<string, (event?: unknown) => void> = {};
const glMap = {
setStyle: vi.fn(),
on: vi.fn((event: string, handler: () => void) => {
on: vi.fn((event: string, handler: (event?: unknown) => void) => {
glHandlers[event] = handler;
}),
};
@@ -57,6 +77,7 @@ const isRaster = () => vi.mocked(leaflet.tileLayer).mock.calls.length === 1;
beforeEach(() => {
vi.resetModules();
vi.clearAllMocks();
tokenListeners.clear();
maplibreLayer.options = {};
maplibreGL.mockReturnValue(maplibreLayer);
maplibreLayer.addTo.mockImplementation(() => maplibreLayer);
@@ -79,40 +100,28 @@ describe("createBaseLayer", () => {
it("falls back to raster tiles without WebGL2", async () => {
const createBaseLayer = await setWebGL2(false);
await createBaseLayer(leaflet, map, false);
await createBaseLayer(leaflet, map, false, TOKEN);
expect(isRaster()).toBe(true);
expect(maplibreGL).not.toHaveBeenCalled();
expect(rasterLayer.addTo).toHaveBeenCalledWith(map);
const [url, options = {}] = vi.mocked(leaflet.tileLayer).mock.calls[0];
// No referrer: the only one a browser can send is its origin, which
// identifies a Nabu Casa installation. The fallback source is chosen so it
// does not need one.
expect(options).not.toHaveProperty("referrerPolicy");
// Through core's proxy, which is what identifies Home Assistant upstream -
// a browser can set neither a User-Agent nor a Referer.
expect(url).toContain("/api/map_tiles/raster/{z}/{x}/{y}.png");
// Leaflet substitutes options into the template on every tile request, so
// the token can be refreshed without recreating the layer.
expect(url).toContain("token={token}");
expect(options).toMatchObject({ token: TOKEN });
// The vector layer takes its credit from the style's source instead, so the
// raster layer is the only one carrying attribution itself - and it credits
// both the data and whoever rendered it.
// raster layer is the only one carrying attribution itself.
expect(options.attribution).toContain("openstreetmap.org/copyright");
expect(options.attribution).toContain("carto.com/attributions");
expect(url).toMatch(/\{z\}\/\{x\}\/\{y\}/);
});
// The devices on the fallback are the old retina tablets, and the source
// serves @2x, so they get sharp tiles without quadrupling the requests.
it("asks for @2x raster tiles on a retina screen", async () => {
const createBaseLayer = await setWebGL2(false);
browser.retina = true;
await createBaseLayer(leaflet, map, false);
expect(vi.mocked(leaflet.tileLayer).mock.calls[0][0]).toContain("@2x.png");
browser.retina = false;
});
it("uses vector tiles when WebGL2 is available", async () => {
const createBaseLayer = await setWebGL2(true);
await createBaseLayer(leaflet, map, false);
await createBaseLayer(leaflet, map, false, TOKEN);
expect(maplibreGL).toHaveBeenCalledOnce();
expect(maplibreLayer.addTo).toHaveBeenCalledWith(map);
@@ -128,7 +137,7 @@ describe("createBaseLayer", () => {
})
);
await createBaseLayer(leaflet, map, false);
await createBaseLayer(leaflet, map, false, TOKEN);
expect(isRaster()).toBe(true);
});
@@ -141,7 +150,7 @@ describe("createBaseLayer", () => {
throw new Error("Failed to initialize WebGL");
});
await createBaseLayer(leaflet, map, false);
await createBaseLayer(leaflet, map, false, TOKEN);
expect(isRaster()).toBe(true);
expect(maplibreLayer.remove).toHaveBeenCalled();
@@ -156,7 +165,7 @@ describe("createBaseLayer", () => {
throw new Error("nothing to remove");
});
await createBaseLayer(leaflet, map, false);
await createBaseLayer(leaflet, map, false, TOKEN);
expect(isRaster()).toBe(true);
});
@@ -169,7 +178,7 @@ describe("setDarkMode", () => {
it("swaps the style, and ignores a repeat of the current mode", async () => {
const createBaseLayer = await setWebGL2(true);
const baseLayer = await createBaseLayer(leaflet, map, false);
const baseLayer = await createBaseLayer(leaflet, map, false, TOKEN);
baseLayer.setDarkMode(true);
await vi.waitFor(() => expect(glMap.setStyle).toHaveBeenCalledOnce());
@@ -183,7 +192,7 @@ describe("setDarkMode", () => {
// the tracked mode, and the next toggle to that mode would do nothing.
it("can retry a mode whose request failed while another was in flight", async () => {
const createBaseLayer = await setWebGL2(true);
const baseLayer = await createBaseLayer(leaflet, map, false);
const baseLayer = await createBaseLayer(leaflet, map, false, TOKEN);
const failing = vi.fn(async () => {
throw new Error("offline");
@@ -230,7 +239,7 @@ describe("setDarkMode", () => {
// The layer only settles once its first style resolves, so let that one
// through before the map exists to switch.
const pending = createBaseLayer(leaflet, map, false);
const pending = createBaseLayer(leaflet, map, false, TOKEN);
await vi.waitFor(() => expect(resolvers).toHaveLength(1));
resolvers.shift()!(styleResponse("light"));
const baseLayer = await pending;
@@ -262,7 +271,7 @@ describe("WebGL context loss", () => {
it("falls back to raster tiles when the context stays lost", async () => {
const createBaseLayer = await setWebGL2(true);
await createBaseLayer(leaflet, map, false);
await createBaseLayer(leaflet, map, false, TOKEN);
expect(leaflet.tileLayer).not.toHaveBeenCalled();
glHandlers.webglcontextlost();
@@ -274,7 +283,7 @@ describe("WebGL context loss", () => {
it("keeps the vector layer when the context comes back", async () => {
const createBaseLayer = await setWebGL2(true);
await createBaseLayer(leaflet, map, false);
await createBaseLayer(leaflet, map, false, TOKEN);
glHandlers.webglcontextlost();
glHandlers.webglcontextrestored();
@@ -290,7 +299,7 @@ describe("WebGL context loss", () => {
it("waits for the page to be visible before falling back", async () => {
const hidden = vi.spyOn(document, "hidden", "get").mockReturnValue(true);
const createBaseLayer = await setWebGL2(true);
await createBaseLayer(leaflet, map, false);
await createBaseLayer(leaflet, map, false, TOKEN);
glHandlers.webglcontextlost();
vi.runAllTimers();
@@ -306,7 +315,7 @@ describe("WebGL context loss", () => {
it("keeps the vector layer when a hidden page gets its context back", async () => {
const hidden = vi.spyOn(document, "hidden", "get").mockReturnValue(true);
const createBaseLayer = await setWebGL2(true);
await createBaseLayer(leaflet, map, false);
await createBaseLayer(leaflet, map, false, TOKEN);
glHandlers.webglcontextlost();
glHandlers.webglcontextrestored();
@@ -321,7 +330,7 @@ describe("WebGL context loss", () => {
it("stops listening for visibility once it has fallen back", async () => {
const remove = vi.spyOn(document, "removeEventListener");
const createBaseLayer = await setWebGL2(true);
await createBaseLayer(leaflet, map, false);
await createBaseLayer(leaflet, map, false, TOKEN);
glHandlers.webglcontextlost();
vi.runAllTimers();
@@ -338,7 +347,7 @@ describe("WebGL context loss", () => {
it("stops answering theme changes once it has fallen back", async () => {
const createBaseLayer = await setWebGL2(true);
const baseLayer = await createBaseLayer(leaflet, map, false);
const baseLayer = await createBaseLayer(leaflet, map, false, TOKEN);
glHandlers.webglcontextlost();
vi.runAllTimers();
@@ -347,3 +356,63 @@ describe("WebGL context loss", () => {
expect(glMap.setStyle).not.toHaveBeenCalled();
});
});
// A refused request leaves the source dead: the TileJSON is fetched once and
// MapLibre never retries it, so the map stays blank until the style is
// applied again.
describe("recovering from a refused token", () => {
it("asks for a new token and re-applies the style once it arrives", async () => {
const createBaseLayer = await setWebGL2(true);
await createBaseLayer(leaflet, map, false, TOKEN);
glMap.setStyle.mockClear();
glHandlers.error({ error: { status: 403 } });
expect(refreshMapTilesToken).toHaveBeenCalled();
emitToken("fresh-token");
await vi.waitFor(() => expect(glMap.setStyle).toHaveBeenCalledOnce());
});
it("does not keep asking while the proxy refuses for another reason", async () => {
const createBaseLayer = await setWebGL2(true);
await createBaseLayer(leaflet, map, false, TOKEN);
for (let i = 0; i < 5; i++) {
glHandlers.error({ error: { status: 403 } });
}
expect(refreshMapTilesToken).toHaveBeenCalledOnce();
});
it("ignores errors that are not a refusal", async () => {
const createBaseLayer = await setWebGL2(true);
await createBaseLayer(leaflet, map, false, TOKEN);
glHandlers.error({ error: { status: 500 } });
expect(refreshMapTilesToken).not.toHaveBeenCalled();
});
it("leaves a working map alone when the token is merely refreshed", async () => {
const createBaseLayer = await setWebGL2(true);
await createBaseLayer(leaflet, map, false, TOKEN);
glMap.setStyle.mockClear();
emitToken("fresh-token");
await new Promise((resolve) => {
setTimeout(resolve, 0);
});
expect(glMap.setStyle).not.toHaveBeenCalled();
});
it("redraws the raster layer so refused tiles are asked for again", async () => {
const createBaseLayer = await setWebGL2(false);
await createBaseLayer(leaflet, map, false, TOKEN);
emitToken("fresh-token");
expect(rasterLayer.options.token).toBe("fresh-token");
expect(rasterLayer.redraw).toHaveBeenCalled();
});
});
+165
View File
@@ -0,0 +1,165 @@
import type { Connection } from "home-assistant-js-websocket";
import { beforeEach, describe, expect, it, vi } from "vitest";
// MapLibre hands tile URLs to a worker, which has no document to resolve a
// relative URL against. Measured: without absolute URLs nothing loads at all -
// not the TileJSON, not the glyphs, not a tile - and MapLibre reports no error,
// so this is worth pinning down.
const connectionWith = (...tokens: string[]) => {
let call = 0;
const listeners: Record<string, () => void> = {};
return {
sendMessagePromise: vi.fn(async () => ({
token: tokens[Math.min(call++, tokens.length - 1)],
})),
addEventListener: vi.fn((event: string, cb: () => void) => {
listeners[event] = cb;
}),
listeners,
} as unknown as Connection & { listeners: Record<string, () => void> };
};
const load = async () => {
vi.resetModules();
return import("../../src/data/map_tiles");
};
describe("withMapTilesToken", () => {
beforeEach(() => {
vi.resetModules();
});
it("makes relative URLs absolute", async () => {
const { withMapTilesToken } = await load();
expect(withMapTilesToken("/api/map_tiles/tilejson.json")).toBe(
`${location.origin}/api/map_tiles/tilejson.json`
);
});
it("adds the token to proxy URLs once there is one", async () => {
const { ensureMapTilesToken, withMapTilesToken } = await load();
await ensureMapTilesToken(connectionWith("abc123"));
const url = new URL(withMapTilesToken("/api/map_tiles/vector/1/0/0.mvt"));
expect(url.searchParams.get("token")).toBe("abc123");
expect(url.pathname).toBe("/api/map_tiles/vector/1/0/0.mvt");
});
it("leaves anything outside the proxy alone", async () => {
const { ensureMapTilesToken, withMapTilesToken } = await load();
await ensureMapTilesToken(connectionWith("abc123"));
const url = new URL(
withMapTilesToken("https://example.com/tiles/1/0/0.png")
);
expect(url.searchParams.get("token")).toBeNull();
expect(url.href).toBe("https://example.com/tiles/1/0/0.png");
});
it("still returns an absolute URL when there is no token", async () => {
const { withMapTilesToken } = await load();
expect(
withMapTilesToken("/api/map_tiles/fonts/noto_sans_regular/0-255.pbf")
).toBe(
`${location.origin}/api/map_tiles/fonts/noto_sans_regular/0-255.pbf`
);
});
});
describe("ensureMapTilesToken", () => {
beforeEach(() => {
vi.resetModules();
});
it("asks the backend once and reuses the answer", async () => {
const { ensureMapTilesToken } = await load();
const connection = connectionWith("abc123");
expect(await ensureMapTilesToken(connection)).toBe("abc123");
expect(await ensureMapTilesToken(connection)).toBe("abc123");
expect(connection.sendMessagePromise).toHaveBeenCalledTimes(1);
});
// A backend without the proxy must not hold the map hostage while the
// retries run; the map falls back to loading nothing rather than to nothing
// being drawn at all.
it("gives up quickly on a backend that does not answer", async () => {
const { ensureMapTilesToken } = await load();
const connection = {
sendMessagePromise: vi.fn(async () => {
throw new Error("unknown command");
}),
} as unknown as Connection;
const started = Date.now();
expect(await ensureMapTilesToken(connection)).toBeUndefined();
expect(Date.now() - started).toBeLessThan(3000);
});
it("tells subscribers when a token arrives, for Leaflet's baked template", async () => {
const { ensureMapTilesToken, subscribeMapTilesToken } = await load();
const listener = vi.fn();
subscribeMapTilesToken(listener);
await ensureMapTilesToken(connectionWith("abc123"));
expect(listener).toHaveBeenCalledWith("abc123");
});
});
// A token can expire before the refresh interval comes round, so recovery
// hangs on the reconnect and on retrying a refused request.
describe("recovering a stale token", () => {
beforeEach(() => {
vi.resetModules();
});
it("fetches a new token when the connection comes back", async () => {
const { ensureMapTilesToken, subscribeMapTilesToken } = await load();
const connection = connectionWith("first", "second") as Connection & {
listeners: Record<string, () => void>;
};
await ensureMapTilesToken(connection);
const listener = vi.fn();
subscribeMapTilesToken(listener);
connection.listeners.ready();
await vi.waitFor(() => expect(listener).toHaveBeenCalledWith("second"));
});
it("asks once for a refresh however many tiles were refused", async () => {
const { ensureMapTilesToken, refreshMapTilesToken } = await load();
const connection = connectionWith("first", "second");
await ensureMapTilesToken(connection);
expect(connection.sendMessagePromise).toHaveBeenCalledTimes(1);
await Promise.all([
refreshMapTilesToken(),
refreshMapTilesToken(),
refreshMapTilesToken(),
]);
expect(connection.sendMessagePromise).toHaveBeenCalledTimes(2);
});
it("keeps the old token when the refresh fails", async () => {
const { ensureMapTilesToken, refreshMapTilesToken, withMapTilesToken } =
await load();
const connection = connectionWith("first");
await ensureMapTilesToken(connection);
vi.mocked(connection.sendMessagePromise).mockRejectedValueOnce(
new Error("disconnected")
);
await refreshMapTilesToken();
expect(
new URL(
withMapTilesToken("/api/map_tiles/vector/1/0/0.mvt")
).searchParams.get("token")
).toBe("first");
});
});
+8
View File
@@ -100,6 +100,7 @@ const commandResults: Record<string, unknown> = {
},
"lovelace/info": { resource_mode: "storage" },
"lovelace/resources": [],
"map_tiles/access_token": { token: "map-tiles-token" },
"recorder/info": {
migration_in_progress: false,
migration_is_live: false,
@@ -177,6 +178,13 @@ export async function setupOnboardingMocks(
): Promise<OnboardingCalls> {
const calls: OnboardingCalls = { tokenRequests: [] };
// The location step shows a map, which asks core's tile proxy for tiles core
// is not running here. Answer them so the outcome does not depend on what the
// dev server does with an unknown /api path.
await page.route("**/api/map_tiles/**", (route) =>
route.fulfill({ status: 404, body: "" })
);
await page.route("**/api/onboarding**", async (route) => {
const request = route.request();
const pathname = new URL(request.url()).pathname;