feat(api): configurable worker processes via API_WORKERS

Add a --workers / API_WORKERS option so the API can run multiple worker
processes in a single container for multi-core concurrency, without
needing to scale containers (which would conflict with the api service's
fixed container_name and complicate per-stack ops/monitoring).

The existing create_app() carries hardcoded defaults (sqlite:///./
meshcore.db, Redis off), so forked workers cannot use it — they would
open the wrong database and run without caching. Add an env-driven
factory, create_app_from_env(), that rebuilds the app from APISettings
plus the CLI-only env vars (CORS_ORIGINS, METRICS_ENABLED,
METRICS_CACHE_TTL), mirroring the single-process resolution. workers > 1
runs uvicorn against this factory via an import string; workers == 1
keeps the single-process object path so local CLI flags still apply.

Wire API_WORKERS into the api compose service (default 1, unchanged
behaviour) and document it in the README (new "Scaling the API"
section + env table) and the v0.12 upgrading notes, including the
SQLite single-host caveat and the env-vs-CLI-flag note for workers.

Tests: create_app_from_env reads DB/Redis/MQTT/metrics from env,
honours explicit overrides, and derives the collector DB path from
DATA_HOME rather than the bare create_app default.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Louis KingandClaude Opus 4.8 committed 2026-06-11 12:09:05 +01:00
1 parent e316b656fc
commit 03603a83e2
7 files changed
+197 -1

No files matched your search

+20
View File
@@ -4,6 +4,26 @@ This guide covers upgrading from a previous MeshCore Hub release to the current
## v0.12.0
### Multi-Worker API (`API_WORKERS`)
The API can now run multiple worker processes in a single container for multi-core concurrency, controlled by a new `API_WORKERS` environment variable (default `1`, unchanged behaviour). Each worker is an independent process sharing one listening socket.
**New environment variable:**
| Variable | Default | Description |
| ------------- | ------- | ------------------------------------------------------------------- |
| `API_WORKERS` | `1` | Number of API worker processes (increase for multi-core concurrency) |
**No action required to upgrade** — the default of `1` preserves the previous single-process behaviour. To use it, set `API_WORKERS` in your `.env` and recreate the `api` service.
**Important:** with more than one worker, configuration must come from **environment variables** — CLI flags passed to `meshcore-hub api` are not propagated to forked worker processes. Docker Compose deployments already configure everything via env, so they are unaffected. Enabling Redis (`REDIS_ENABLED=true`) is recommended so all workers share one response cache.
While on SQLite, all workers share the same database file on the same host (WAL mode allows concurrent reads alongside the collector's single writer). Writes do not scale and this does not extend across multiple hosts; switch `DATABASE_URL` to PostgreSQL to scale beyond a single host. See [Scaling the API](../README.md#scaling-the-api) for details.
### Read-Path Query Optimisations
Several read-heavy endpoints had their query patterns optimised (node `is_observer` filtering, dashboard node-count history, and message/dashboard sender-name resolution). These are internal performance improvements with no API or configuration changes — responses are unchanged. The `is_observer` change ships an Alembic migration that is applied automatically on startup (Docker) or via `meshcore-hub db upgrade`.
### Optional Redis API Cache
A new optional Redis-backed caching layer reduces database load for read-heavy API endpoints (nodes, advertisements, messages, channels, dashboard). Redis is entirely optional — the API works identically without it.