mc-webui/docs/architecture.md

mc-webui Architecture

Technical documentation for mc-webui, covering system architecture, project structure, and internal APIs.

Table of Contents

• Tech Stack
• Container Architecture
• DeviceManager Architecture
• Project Structure
• Database Architecture
• API Reference
• WebSocket API
• Versioning & Releases
• Offline Support

---

Tech Stack

• Backend: Python 3.11+, Flask, Flask-SocketIO (gevent), SQLite
• Frontend: HTML5, Bootstrap 5, vanilla JavaScript, Socket.IO client
• Deployment: Docker / Docker Compose (Single-container architecture)
• Communication: Direct hardware access (USB, BLE, or TCP) via meshcore library
• Data source: SQLite Database (./data/meshcore/<pubkey_prefix>.db)

---

Container Architecture

mc-webui uses a single-container architecture for simplified deployment and direct hardware communication:

┌─────────────────────────────────────────────────────────────┐
│                     Docker Network                           │
│                                                              │
│  ┌───────────────────────────────────────────────────────┐   │
│  │                       mc-webui                        │   │
│  │                                                       │   │
│  │  - Flask web app (Port 5000)                          │   │
│  │  - DeviceManager (Direct USB/BLE/TCP access)          │   │
│  │  - Database (SQLite)                                  │   │
│  │                                                       │   │
│  └─────────┬─────────────────────────────────────────────┘   │
│            │                                                 │
└────────────┼─────────────────────────────────────────────────┘
             │
             ▼
      ┌──────────────┐
      │ USB/BLE/TCP  │
      │    Device    │
      └──────────────┘

Three transport options are supported with the following priority: BLE > TCP > Serial (USB). Set the MC_BLE_ADDRESS or MC_TCP_HOST environment variable to activate BLE or TCP transport respectively; otherwise, USB serial is used by default.

This v2 architecture eliminates the need for a separate bridge container and relies on the native meshcore Python library for direct communication, ensuring lower latency and greater stability.

Docker Entrypoint (BLE cleanup)

scripts/docker-entrypoint.sh runs before the Flask app starts. When MC_BLE_ADDRESS is set, it uses D-Bus to check if BlueZ has an active session to the device and disconnects it. BlueZ auto-reconnects trusted devices, which leaves stale GATT notification handles that block bleak from establishing a new session. A clean disconnect at startup ensures the app starts with a fresh BLE state.

Multi-architecture Images

Official images are built via GitHub Actions for linux/amd64, linux/arm64, and linux/arm/v7 (Raspberry Pi 2/3/4/5 supported). Build dependencies (gcc, python3-dev, libjpeg-dev, zlib1g-dev) are installed and then purged to keep the final image size small while still compiling Pillow and pycryptodome from source when wheels are unavailable (notably on arm/v7). GHA layer cache (cache-from / cache-to) speeds up subsequent rebuilds. Images are pushed to both Docker Hub (mawoj/mc-webui) and GitHub Container Registry (ghcr.io/marekwo/mc-webui), with latest tag on main and dev tag on the dev branch.

---

DeviceManager Architecture

The DeviceManager handles the connection to the MeshCore device via a direct session:

• Single persistent session - One long-lived connection utilizing the meshcore library
• Event-driven - Subscribes to device events (e.g., incoming messages, advert receptions, ACKs) and triggers appropriate handlers
• Direct Database integration - Seamlessly syncs contacts, messages, and device settings to the SQLite database
• Real-time messages - Instant message processing via callback events without polling
• Thread-safe queue - Commands are serialized to prevent device lockups
• Auto-restart watchdog - Monitors connection health and restarts the session on crash
• BLE keepalive & reconnect - When using Bluetooth transport, a 60s keepalive loop detects "zombie" connections (reads still succeed but writes silently fail). On disconnect or keepalive failure, the manager marks the session as permanently failed and the /health endpoint returns 503, letting the Docker healthcheck trigger a fast container restart (~5s) to get a clean BLE state rather than attempting unreliable in-process reconnects
• Echo correlation - Sent channel messages pre-compute their expected pkt_payload using the channel secret and send timestamp (±3s for clock drift), so incoming echoes are matched exactly instead of only by 1-byte channel hash (prevents misattribution when two messages go out simultaneously on the same channel)
• Per-channel region scope - Before each channel send, the channel's mapped region scope key (16 bytes) is pushed to the firmware via CMD_SET_FLOOD_SCOPE_KEY (54). The scope-set + send pair is serialised under a _send_lock so concurrent sends on different channels can't swap each other's scope. Channels without a mapping get an all-zero key so a previously-set scope doesn't leak across channels
• Per-send channel-secret refresh - Channel indices on the device compact down after a deletion, so the boot-time _load_channel_secrets() cache can drift. send_channel_message calls _refresh_channel_secret(idx) first (one extra get_channel(idx) round-trip) to fetch the current secret straight from firmware, update the in-memory cache and DB if they had drifted, and use it for the pkt_payload echo correlation
• Liveness telemetry - Tracks _last_rx_at (bumped on every RX_LOG_DATA event) and _consecutive_stats_failures (incremented on get_stats_* / get_bat exceptions, cleared on success). Surfaced via /health/strict for the external watchdog
• TCP self-heal - A _liveness_watcher_loop task on the DM event loop calls force_reconnect() when no RX event has arrived for HEALTH_STRICT_MAX_RX_STALE_SEC (5 min). send_channel_message also detects empty-string concurrent.futures.TimeoutError from set_flood_scope_key (the symptom of a degraded long-lived TCP) and runs an in-place reconnect + one retry before failing. A 30 s cooldown and _reconnect_lock prevent churn; _intentional_disconnect keeps the DISCONNECTED handler from racing the reconnect. The watcher keeps re-checking staleness even after _connected has gone False, so a single failed reconnect (e.g. an empty self_info) no longer silently stops all further healing for non-BLE transports
• Raw packet resend - Own channel sends capture a full hex wire snapshot (header + transport_codes + path_len + encrypted payload) into channel_messages.raw_packet, rebuilt from the actual pkt_payload once echo correlation resolves it and honouring the device's cached path_hash_mode. resend_channel_message() re-broadcasts that snapshot verbatim via CMD_SEND_RAW_PACKET (0x41) so repeaters dedupe by packet hash (Mesh::hasSeen) and only previously-unreached nodes pick it up. Requires companion firmware ≥1.16 (fw_ver_code ≥ 13), gated via the cached supports_raw_resend flag captured from the connect-time DEVICE_INFO event. Self-echoes of a resend (the firmware seen-table can evict the hash within minutes on a busy mesh) are detected by recomputing the expected pkt_payload and matching an existing own row, so a resend never reappears as an inbound message from yourself

Observer (MQTT packet capture)

app/observer.py (ObserverManager) implements a meshcore-packet-capture-compatible observer on top of the existing device session — no second connection and no device-side enable command (the firmware pushes RX_LOG_DATA for every overheard frame):

• Hot path — _on_rx_log_data hands every packet (all payload types, before the GRP_TXT echo gate) to handle_raw_packet(): pure CPU work (header/path decode, firmware-exact packet hash, JSON build) plus a non-blocking paho publish(), wrapped in its own try/except so observer failures can never affect chat
• One paho-mqtt client per enabled broker, each with its own network thread (loop_start), auto-reconnect with backoff, QoS 0 (no backlog during outages), retained online/offline status plus an LWT on meshcore[/IATA/PUBKEY]/status; packets go to .../packets (flat meshcore/packets|status when IATA is unset)
• Wire-format fidelity — the decode/hash/format functions are verbatim ports of upstream meshcore-packet-capture (including legacy path-length fallbacks and the publish-with-defaults quirk for unknown payload versions), validated byte-identical against live packets from the reference observer network
• Config — observer_brokers table plus the observer_settings JSON key (enabled, iata, advert_interval_hours) in app_settings; every mutating API call triggers reload() on a daemon thread (the client list is swapped atomically and the hot path reads it lock-free), so changes apply without restart
• Advert scheduler — an observer-owned daemon thread checks every 10 minutes and sends a flood advert once advert_interval_hours has elapsed since the observer_last_advert_at timestamp persisted in app_settings (restart-safe)
• Live status — observer_status events on the /chat namespace (throttled to one per 2 s on the packet path) drive the Settings-tab badges and counters; GET /api/observer/status returns the full merged view

Repeater administration (My Repeaters)

The /repeaters (list) and /repeaters/manage (per-repeater tools) panels are standalone iframe pages built on the companion protocol's remote-request commands plus the repeater text CLI:

• Serialization — the companion firmware tracks a single pending remote request (each new send silently clears the previous one), so every repeater operation runs under DeviceManager._repeater_lock (180 s acquire timeout; a held lock returns {'busy': True} → HTTP 429). This also guards against two browser tabs operating at once
• Login sessions — repeater_login() waits for LOGIN_SUCCESS filtered by the contact's pubkey_prefix and stores {is_admin, permissions, logged_in_at} in the in-memory _repeater_sessions dict (cleared on logout/app restart; the UI auto-relogs with the saved password). All remote endpoints fail fast with 401 need_login when no session exists — a repeater only answers pubkeys in its ACL, so requesting without a login would just burn a multi-minute timeout. A wrong password is indistinguishable from an unreachable repeater (the firmware never replies to failed logins), so timeout messages always name both causes
• CLI reply correlation — repeater_cmd_wait() sends one text command and blocks for its reply. Replies arrive as CONTACT_MSG_RECV with txt_type=1 (CLI_DATA) and carry no protocol-level correlation, so correlation = the repeater lock (single command in flight) + a single-slot waiter matched against the sender's 12-hex pubkey prefix at the top of _on_dm_received. Matched replies are consumed there and never stored as chat DMs; unmatched CLI replies keep the legacy behavior (stored as a DM) so the Console's fire-and-forget cmd flow is unchanged. Wait time derives from the device-suggested timeout (clamped 10–45 s)
• Settings batches — reads run sequential get <field> commands per section and parse the firmware's > value replies, reporting per-field errors without failing the batch; writes send only dirty fields and classify each reply as ok (starts with ok/password now), reboot_required (reply mentions reboot — e.g. set radio), or failed (anything else, surfaced verbatim). Connection-level failures abort the rest of the batch
• Role gating — the firmware silently drops text CLI from non-admin logins (which would surface as a timeout), so the CLI/Settings/Actions endpoints reject guest sessions with 403 up front instead
• Actions whitelist — advert.zerohop, advert (flood), clock sync, reboot. reboot never replies (the firmware restarts immediately without building one), so a clean send followed by silence is reported as success. Text erase is firmware-gated to the USB serial console (sender_timestamp == 0), hence no erase in the UI

Path Analyzer

The /path-analyzer panel (standalone iframe page, path-analyzer.js) is a read-only analysis view over data the app already collects — no new tables, no background work:

• Data source — GET /api/path-analyzer/messages?days=N joins channel_messages with echoes (path hex + SNR + per-echo hash_size, keyed by pkt_payload). Echoes are fetched with db.get_echoes_for_payloads() — chunked IN queries (≤500 params, under SQLite's host-parameter limit) via idx_echoes_pkt — deliberately avoiding the per-message echo query the older /api/messages path still does. The pkt_payload reconstruction (raw_json text → channel-secret AES/HMAC compute) is shared with /api/messages via the _get_row_pkt_payload() helper
• All filtering/stats/map/routes logic is client-side over the bulk payload (hundreds of KB for 7 days — fine on a LAN): filters operate on per-hop tokens split with each echo's own hash_size (mixed 1/2/3-byte networks are real), so SQL-side token filtering was rejected. Every view always reflects the active filters for free
• Four views share the one payload: Messages (hop-by-hop echo detail), Repeaters (per-hash relay/SNR stats), Routes (consecutive hop-segment n-grams — user-selectable length 2–4, counted anywhere in a path), and Map (Leaflet path drawing). The repeater filter accepts a >-chained sequence (each element a hash prefix or contact name) matched as consecutive hops; Routes rows write such a sequence into that filter on click
• Deep link — GET /path-analyzer?hash=<packet_hash>&path=<echo path hex> opens straight on the Map view with that message selected and that exact echo drawn. Used by the chat path popup (app.js → openPathInAnalyzer stashes {hash, path} in window.paDeepLink; the modal's show.bs.modal handler builds the iframe URL). On load the analyzer resolves the message by packet_hash, widening the time range once to 7 days if it isn't in the current window, and matches the echo by raw path hex (fallback: shortest routed echo)
• Map layers — three L.layerGroups added in draw order (base repeater markers → alternative echoes → selected path), both extras gated by opt-in checkboxes in a topright L.Control (the shared filter bar is wrong for view-specific state). Alternative echoes are coloured by their index in echoView, so a hue survives changing which echo is primary; segments are deduplicated against a Set of endpoint-pair keys that the primary path fills first, so alternatives render only where they diverge instead of underneath the primary line. Toggling calls paRenderMapView(false) — a full re-render (the sidebar swatches must follow) with fitBounds suppressed, so overlay changes never discard the user's viewport
• Filter persistence — toolbar controls plus the Routes segment length are mirrored into localStorage under mc-webui-pa-filters (browser-local working set, deliberately not device state in SQLite). Only user-driven handlers write, via paApplyAndSaveFilters(), so programmatic changes — notably the deep link forcing 7 days — never overwrite the stored set; restore is skipped entirely when paDeepLink is present, and stored <select> values that no longer exist as options are ignored. paRestoreFilters() ends with paReadFilters() so paFilters, the Clear button, and the mobile badge match the restored DOM before the first render
• SNR attribution — echo SNR is measured at our receiver, so stats credit it to the final hop only; intermediate hops get relay counts but never SNR
• Hash→contact resolution — pubkey-prefix match against /api/contacts/cached?format=full (memoized per token). 1-byte hashes collide by design; the map renders unresolved hops as amber candidate markers with manual pick, and the repeater-name filter is intentionally inclusive over candidates
• Legacy rows whose pkt_payload cannot be recomputed (missing channel secret) are returned with packet_hash: null and no echoes rather than dropped

---

Project Structure

mc-webui/
├── Dockerfile                      # Main app Docker image
├── docker-compose.yml              # Single-container orchestration
├── app/
│   ├── __init__.py
│   ├── main.py                     # Flask entry point + Socket.IO handlers
│   ├── config.py                   # Configuration from env vars
│   ├── database.py                 # SQLite database models and CRUD operations
│   ├── device_manager.py           # Core logic for meshcore communication
│   ├── observer.py                 # Observer: MQTT packet-capture publishing
│   ├── contacts_cache.py           # Persistent contacts cache (DB-backed)
│   ├── read_status.py              # Server-side read status manager (DB-backed)
│   ├── version.py                  # Git-based version management
│   ├── migrate_v1.py               # Migration script from v1 flat files to v2 SQLite
│   ├── meshcore/
│   │   ├── __init__.py
│   │   ├── cli.py                  # Meshcore library wrapper interface
│   │   └── parser.py               # Data parsers
│   ├── archiver/
│   │   └── manager.py              # Archive scheduler and management
│   ├── routes/
│   │   ├── __init__.py
│   │   ├── api.py                  # REST API endpoints
│   │   └── views.py                # HTML views
│   ├── static/                     # Frontend assets (CSS, JS, images, vendors)
│   │   └── js/fab-utils.js         # Floating-button drag/collapse/sizing helpers
│   └── templates/                  # HTML templates
├── docs/                           # Documentation
├── scripts/
│   ├── update.sh                   # Automated update script
│   ├── docker-entrypoint.sh        # Container startup (BLE cleanup)
│   ├── updater/                    # Remote update webhook service
│   └── watchdog/                   # Container health monitor
└── README.md

---

Database Architecture

mc-webui v2 uses a robust SQLite Database with WAL (Write-Ahead Logging) enabled.

Location: ./data/meshcore/<pubkey_prefix>.db

Key tables:

• messages - All channel and direct messages (with FTS5 index for full-text search)
• contacts - Contact list with sync status, types, block/ignore flags, no_auto_flood flag
• channels - Channel configuration and keys
• echoes - Sent message tracking and repeater paths, hash_size for path_hash_mode
• direct_messages - DM messages with delivery tracking (delivery_status, delivery_attempt, delivery_max_attempts, delivery_path)
• acks - DM delivery status
• settings - Application settings (migrated from .webui_settings.json)
• regions - User-curated MeshCore flood scopes (name, key_hex, is_default)
• channel_scopes - Per-channel region mapping (channel_idx → region_id, CASCADE on region delete; absent row = no override → firmware default applies)
• read_status - Per-channel read counters and favorites (is_favorite column; used to pin channels in the sidebar/dropdown sort order)
• analyzers - User-configured MeshCore Analyzer services (name, url_template with {packetHash} placeholder, is_default, is_disabled; partial unique index enforces a single default)
• observer_brokers - MQTT brokers for the Observer packet-capture feature (name, host, port, username, password — stored plaintext, use_tls, tls_verify, is_disabled)
• repeaters - Repeaters saved in the My Repeaters panel (public_key PK, password — stored plaintext per the observer_brokers precedent, added_at, last_login_at, last_login_role). Everything else about a repeater (name, path, position) comes from the device contact at read time

direct_messages gained a delivery_path_hash_size column (auto-migrated, defaults to 1) so reloaded DM bubbles render multi-byte routes correctly. The path_len column on channel_messages, direct_messages, and paths now stores the raw firmware byte (masked hop count plus path_hash_mode in the upper bits), recombined at write time via pack_path_len(); the API endpoints decode it back into path_hash_size on read. channel_messages also gained a raw_packet column (the full hex wire snapshot captured at send time, indexed by idx_cm_pkt on pkt_payload for fast self-echo lookups) that powers raw resend; it is NULL for received and pre-migration rows, so the resend button stays disabled there.

The use of SQLite allows for fast queries, reliable data storage, full-text search, and complex filtering (such as contact ignoring/blocking) without the risk of file corruption inherent to flat JSON files.

Retention scheduler

Retention is enabled by default with 90 / 90 / 60 / 30 days for channel_messages / direct_messages / advertisements / diagnostics. The job runs daily at 03:30 local (TZ from .env) and cleanup_old_messages() also deletes from echoes, paths, and acks (the diagnostic tables — historically the bulk of DB size). When at least 1 000 rows are removed in a pass, the scheduler immediately runs VACUUM to reclaim file space (a SQLite DELETE only marks pages free).

The retention/cleanup scheduler runs APScheduler jobs in worker threads, so each job is decorated with @_with_app_context and the Flask app is passed in via set_flask_app(); the init_*_schedule() callers also wrap themselves in app.app_context() so the boot-time read of current_app.db doesn't blow up with "Working outside of application context".

The archiver builds the .msgs path from device_name, but the meshcore library strips non-ASCII when writing the file (so a device renamed to include an emoji breaks the strict path match). The archiver now falls back to globbing the data directory for a single non-archive .msgs file when the expected path is missing — mirroring migrate_v1.

The channels API reads from the channels DB table rather than iterating device slots. _load_channel_secrets() syncs the table on every device connect (and prunes stale rows), set_channel() / remove_channel() update it synchronously with the device, and _refresh_channel_secret() refreshes individual rows on per-send refresh. This makes /api/channels a single sub-millisecond SELECT and unaffected by device responsiveness — the original symptom (only "Public" showing up after a refresh when the device briefly stalls) is gone.

---

API Reference

Messages

Method	Endpoint	Description
GET	/api/messages	List messages (?archive_date, ?days, ?channel_idx)
POST	/api/messages	Send message ({text, channel_idx, reply_to?})
GET	/api/messages/updates	Check for new messages (smart refresh)
GET	/api/messages/<id>/meta	Get message metadata (echoes, paths)
POST	/api/messages/<id>/resend	Re-broadcast an own channel message verbatim via CMD_SEND_RAW_PACKET (same packet hash, so unreached repeaters pick it up). 400 for not-own / missing raw_packet snapshot / disconnected / firmware < 1.16, 404 for unknown id
GET	/api/messages/search	Full-text search (?q=, ?channel_idx=, ?limit=)
GET	/api/path-analyzer/messages	Bulk channel messages across all channels with batched echo data (?days=1..30, default 3); powers the Path Analyzer panel (GET /path-analyzer)

Contacts

Method	Endpoint	Description
GET	/api/contacts	List contacts
GET	/api/contacts/detailed	Full contact data (includes protection, ignore, block flags)
GET	/api/contacts/cached	Get cached contacts (superset of device contacts)
POST	/api/contacts/delete	Soft-delete contact ({selector})
POST	/api/contacts/cached/delete	Delete cached contact
GET	/api/contacts/protected	List protected public keys
POST	/api/contacts/<key>/protect	Toggle contact protection
POST	/api/contacts/<key>/ignore	Toggle contact ignore
POST	/api/contacts/<key>/block	Toggle contact block
GET	/api/contacts/blocked-names	Get blocked names count
POST	/api/contacts/block-name	Block a name pattern
GET	/api/contacts/blocked-names-list	List blocked name patterns
POST	/api/contacts/preview-cleanup	Preview cleanup criteria
POST	/api/contacts/cleanup	Remove contacts by filter
GET	/api/contacts/cleanup-settings	Get auto-cleanup settings
POST	/api/contacts/cleanup-settings	Update auto-cleanup settings
GET	/api/contacts/pending	Pending contacts (?types=1&types=2)
POST	/api/contacts/pending/approve	Approve pending contact
POST	/api/contacts/pending/reject	Reject pending contact
POST	/api/contacts/pending/clear	Clear all pending contacts
POST	/api/contacts/manual-add	Add contact from URI or params
POST	/api/contacts/<key>/push-to-device	Push cached contact to device
POST	/api/contacts/<key>/move-to-cache	Move device contact to cache
GET	/api/contacts/repeaters	List repeater contacts (for path picker)
GET	/api/contacts/<key>/paths	Get contact paths
POST	/api/contacts/<key>/paths	Add path to contact
PUT	/api/contacts/<key>/paths/<id>	Update path (star, label)
DELETE	/api/contacts/<key>/paths/<id>	Delete path
POST	/api/contacts/<key>/paths/reorder	Reorder paths
POST	/api/contacts/<key>/paths/<id>/apply	Push a configured path to the firmware as the active route (mirrors change_path); invalidates the contacts cache
POST	/api/contacts/<key>/paths/reset_flood	Reset to FLOOD routing
POST	/api/contacts/<key>/paths/clear	Clear all paths
GET	/api/contacts/<key>/no_auto_flood	Get "Keep path" flag
PUT	/api/contacts/<key>/no_auto_flood	Set "Keep path" flag

Channels

Method	Endpoint	Description
GET	/api/channels	List all channels
POST	/api/channels	Create new channel (idempotent — returns existing slot if name already used)
POST	/api/channels/join	Join existing channel (idempotent unless explicit index overrides)
DELETE	/api/channels/<index>	Remove channel
GET	/api/channels/<index>/qr	QR code (?format=json|png)
GET	/api/channels/muted	Get muted channels
POST	/api/channels/<index>/mute	Toggle channel mute
GET	/api/channels/scopes	Bulk per-channel region mapping for UI
PUT	/api/channels/<index>/scope	Assign/clear region scope ({region_id: int|null})
GET	/api/channels/favorites	List favorite channel indices
POST	/api/channels/<index>/favorite	Set favorite state ({favorite: bool})

Regions (MeshCore flood scopes)

Method	Endpoint	Description
GET	/api/regions	List the device's region registry
POST	/api/regions	Create region ({name}); key derived as SHA256('#'+name)[:16]
DELETE	/api/regions/<id>	Delete region; CASCADE clears channel mappings; if it was the firmware default, clears it on device
POST	/api/regions/<id>/default	Mark default in DB AND push to firmware (CMD_SET_DEFAULT_FLOOD_SCOPE = 63, requires firmware v1.15+)
DELETE	/api/regions/default	Clear default region in DB and on firmware

The PUT /api/channels/<index>/scope endpoint accepts any index in [0, device_manager._max_channels) (40 on current firmwares; falls back to 8 if the DM is unreachable).

Analyzers

Method	Endpoint	Description
GET	/api/analyzers	List configured analyzer services
POST	/api/analyzers	Create analyzer ({name, url_template}); template must contain {packetHash}
PUT	/api/analyzers/<id>	Update analyzer (name / url / is_disabled)
DELETE	/api/analyzers/<id>	Delete analyzer
POST	/api/analyzers/<id>/default	Mark as default (enforced single-default via partial unique index)
DELETE	/api/analyzers/default	Clear the default analyzer

The backend no longer ships a pre-built analyzer_url per message — channel-message payloads include packet_hash instead, and the frontend substitutes {packetHash} in the chosen URL template at click time. The Letsmesh Analyzer is not a hardcoded pseudo-row: seed_default_analyzers() inserts it as an ordinary analyzers row exactly once per install (guarded by the analyzer_letsmesh_seeded flag in app_settings), so it can be renamed, disabled, or deleted like any user entry. The row stores is_disabled (not is_enabled); the UI inverts it on read/write so the switch reads "Enabled", which is why no data migration was needed.

Observer

Method	Endpoint	Description
GET	/api/observer/settings	Get observer settings (enabled, iata, advert_interval_hours)
POST	/api/observer/settings	Partial update; IATA must be empty or exactly 3 letters, interval 0-8760 h
GET	/api/observer/brokers	List MQTT brokers (passwords never returned; a has_password flag instead)
POST	/api/observer/brokers	Create broker ({name, host, port?, username?, password?, use_tls?, tls_verify?})
PUT	/api/observer/brokers/<id>	Partial update; omitted password keeps the stored one, empty string clears it
DELETE	/api/observer/brokers/<id>	Delete broker
GET	/api/observer/status	Settings + broker rows merged with live connection state and packet counters

Every mutating endpoint hot-reloads the ObserverManager, so broker and setting changes take effect without an app restart.

My Repeaters (repeater administration)

Method	Endpoint	Description
GET	/api/repeaters	Saved repeaters merged with device contact truth (?refresh=true bypasses the contacts cache)
POST	/api/repeaters	Add a repeater ({public_key}; 409 when already saved)
GET	/api/repeaters/<pk>	Single merged entry + login session state
PUT	/api/repeaters/<pk>	Set or clear the saved password ({password}; empty string clears)
DELETE	/api/repeaters/<pk>	Remove from the list (device contact untouched)
GET	/api/repeaters/<pk>/password	Saved password (empty string when none); prefills the login-retry prompt for the trusted local UI
POST	/api/repeaters/<pk>/login	Log in with {password?, save?} — omitted password uses the saved one
GET	/api/repeaters/<pk>/session	In-memory session state (logged_in, is_admin, permissions)
POST	/api/repeaters/<pk>/logout	Log out and drop the session
GET	/api/repeaters/<pk>/status	Binary status request (battery, radio, packet stats)
GET	/api/repeaters/<pk>/clock	Repeater clock (LE epoch from req_basic_sync)
GET	/api/repeaters/<pk>/telemetry	All Cayenne LPP channels
GET	/api/repeaters/<pk>/neighbours	Zero-hop neighbours enriched with contact names/positions
POST	/api/repeaters/<pk>/cli	Text CLI command ({command} → {output, elapsed_ms}); admin only
GET	/api/repeaters/<pk>/settings	Read one settings section (?section=basic|radio|location|features|network|advert|operator|advanced) as a get batch; admin only
POST	/api/repeaters/<pk>/settings	Apply dirty fields ({values}) as a set batch with per-field ok|failed|reboot_required results; admin only
POST	/api/repeaters/<pk>/action	One-shot action ({action: zerohop_advert|flood_advert|clock_sync|reboot}); admin only

Error mapping is shared across the family: 401 need_login (no session), 403 (guest on an admin endpoint), 429 (another repeater operation in progress — single firmware request slot), 503 (device not connected), 504 (timeout: repeater unreachable or wrong password — indistinguishable by protocol). Passwords are never returned by any GET (password_set boolean only). The panels are served by GET /repeaters and GET /repeaters/manage?pubkey=<64-hex>.

Direct Messages

Method	Endpoint	Description
GET	/api/dm/conversations	List DM conversations
GET	/api/dm/messages	Get messages (?conversation_id=, ?limit=)
POST	/api/dm/messages	Send DM ({recipient, text})
GET	/api/dm/updates	Check for new DMs
GET	/api/dm/auto_retry	Get DM retry configuration
POST	/api/dm/auto_retry	Update DM retry configuration

Device & Settings

Method	Endpoint	Description
GET	/api/status	Connection status (device name, transport type, serial port / BLE address). When connected also surfaces fw_ver_code, supports_raw_resend, path_hash_mode, and path_hash_size so the frontend can show/hide the raw-resend button and verify the resend snapshot's hash size
GET	/api/device/info	Device information
GET	/api/device/stats	Device statistics
GET	/api/device/settings	Get device settings
POST	/api/device/settings	Update device settings
GET	/api/device/config	Get device configuration (name, coords, advert_loc_policy, path_hash_mode, radio params, tx_power)
POST	/api/device/config	Update device configuration from Settings > Device tab. Subset of fields incl. path_hash_mode (0=1B, 1=2B, 2=3B)
POST	/api/device/command	Execute command (advert, floodadv)
GET	/api/device/commands	List available special commands
GET	/api/chat/settings	Get chat settings (quote length, route popup timeout/no-autoclose)
POST	/api/chat/settings	Update chat settings
GET	/api/ui/settings	Get UI settings (toast timeout, no-autoclose, position)
POST	/api/ui/settings	Update UI settings
GET	/api/retention-settings	Get message retention settings
POST	/api/retention-settings	Update retention settings

Archives & Backup

Method	Endpoint	Description
GET	/api/archives	List archives
POST	/api/archive/trigger	Manual archive
GET	/api/backup/list	List database backups
POST	/api/backup/create	Create database backup
GET	/api/backup/download	Download backup file
GET	/api/db/size	Current DB file size (bytes)
POST	/api/db/vacuum	Kick off SQLite VACUUM in a worker thread. Returns 202 immediately; 409 if already running. The kickoff endpoint deliberately splits from polling so reverse proxies with ~30 s idle timeouts can't kill it mid-rewrite
GET	/api/db/vacuum/status	Poll vacuum progress: {running, elapsed_seconds, size_before, size_after}

Health endpoints

These are top-level routes (not under /api/), consumed by Docker's healthcheck and the host-level watchdog.

Method	Endpoint	Description
GET	/health	Lenient liveness check. Returns 503 only when BLE reconnection has permanently failed (so Docker triggers a container restart to clear BLE state). Returns 200 otherwise
GET	/health/strict	Strict device-health check for the external watchdog. JSON response. Returns 503 when (a) BLE permanently failed, (b) _consecutive_stats_failures ≥ 5, or (c) transport is serial/usb/tcp and no RX event for > HEALTH_STRICT_MAX_RX_STALE_SEC (5 min). Returns 200 with the same counters when healthy

Other

Method	Endpoint	Description
GET	/api/read_status	Get server-side read status
POST	/api/read_status/mark_read	Mark messages as read
POST	/api/read_status/mark_all_read	Mark all messages as read
GET	/api/version	Get app version
GET	/api/check-update	Check for available updates
GET	/api/updater/status	Get updater service status
POST	/api/updater/trigger	Trigger remote update
GET	/api/advertisements	Get recent advertisements
GET	/api/console/history	Get console command history
POST	/api/console/history	Save console command
DELETE	/api/console/history	Clear console history
GET	/api/console/output	Get persisted console output transcript (capped at 500 entries)
POST	/api/console/output	Append entry to transcript
DELETE	/api/console/output	Clear transcript
GET	/api/logs	Get application logs

---

WebSocket API

All Socket.IO clients (/chat, /console, /logs) are configured with transports: ['polling']. The Werkzeug dev server can't upgrade WebSockets, so every io() upgrade attempt previously returned HTTP 500 and clients fell into a polling/upgrade reconnect loop — visible as 10–15 s freezes on app load. Long-polling keeps real-time pushes working with ~1–2 s latency.

Console Namespace (/console)

Interactive console via Socket.IO WebSocket connection.

Client → Server:

• send_command - Execute command ({command: "infos"})

Server → Client:

• console_status - Connection status
• command_response - Command result ({success, command, output})

Chat Namespace (/chat)

Real-time message delivery via Socket.IO.

Server → Client:

• new_channel_message - New channel message received
• new_dm_message - New DM received
• message_echo - Echo/ACK update for sent message (includes hash_size)
• dm_ack - DM delivery confirmation
• dm_retry_status - Real-time retry progress (dm_id, attempt, max_attempts)
• dm_retry_failed - All retry attempts exhausted (dm_id)
• dm_delivered_info - Delivery details after ACK (dm_id, attempt, max_attempts, path, hash_size)
• path_changed - Contact path discovered/updated (public_key)
• observer_status - Observer live state (enabled, running, reason, packet counters, per-broker connected/last_error)

Logs Namespace (/logs)

Real-time log streaming via Socket.IO.

Server → Client:

• log_line - New log line

The MemoryLogHandler filters werkzeug access-log records for /socket.io/ and /api/logs/ paths before buffering/broadcasting. With async_mode='threading' Socket.IO falls back to long-polling; without this filter every poll is logged, the broadcast wakes the pending poll, the client re-polls immediately, and an open System Log tab spins at 10+ requests/sec.

---

Versioning & Releases

Two identities, deliberately separate — app/version.py exposes both:

	Source	Looks like	Used for
RELEASE_VERSION	VERSION file at the repo root	2.1.0	What users and testers quote; the git tag; the GitHub release
VERSION_STRING	git commit date + short hash	2026.07.26+95d96ec	Pinning down an exact deploy; what /api/check-update compares

Resolution order is unchanged: app/version_frozen.py (written by python -m app.version freeze, which scripts/update.sh runs before every rebuild) wins over live git, which wins over the built-in fallbacks. RELEASE_VERSION is read from the VERSION file and only overridden by the frozen file, so a frozen file written before releases existed still yields a correct release number. The container has no git and no .git, hence COPY VERSION ./ in the Dockerfile — without it a plain docker compose build would report 0.0.0.

Both values ship in GET /api/version (release and version) and in the template context ({{ release }}, {{ version }}). The menu shows the release number first with the build underneath; #versionText deliberately still holds the build string, because the remote-update poller compares it to detect that the server came back on a new build.

Cutting a release:

1. On dev: bump VERSION, and title the pending docs/whatsnew.md section ## <version> — <date> (open a fresh ## Unreleased above it)
2. Merge dev → main
3. On main: ./scripts/release.sh — validates the number, refuses a dirty tree, a non-main branch, or an existing tag, extracts the notes from that whatsnew section, then tags v<version>, pushes it, and publishes the GitHub release via gh. --dry-run prints the notes and changes nothing
4. Deploy as usual (mcupdate), which freezes the version into the image

Numbering is SemVer read through an operator's eyes: MAJOR when a deploy needs manual action (new env var, migration, breaking config), MINOR for new features, PATCH for fixes only. Releases start at 2.1.0 — the v2 line has been in production since 2026-03-28, and v1 is the archived pre-migration branch, so 1.x would have been ambiguous. Everything before 2.1.0 is dated but untagged.

---

Offline Support

The application works completely offline without internet connection. Vendor libraries (Bootstrap, Bootstrap Icons, Socket.IO, Emoji Picker) are bundled locally. A Service Worker provides hybrid caching to ensure functionality without connectivity.