feat(api): add public REST API for stats, nodes and messages

WHAT: Four read-only GET endpoints under /api/v1/ registered on the NiceGUI/FastAPI instance via register_routes(_shared) in __main__.py. Controlled by API_ENABLED in config.py (default: True). WHY: Enables the domca.nl PHP collector to pull live mesh data over HTTP without direct access to SQLite or SharedData internals. NOTES: Private channel messages are unconditionally excluded in public_api_service.py — filtering is server-side hard, no auth needed. Channel rule: idx==0 (Public) or name.startswith('#') (Hashtag) = expose.
2026-08-09 10:22:44 +02:00 · 2026-04-04 14:17:07 +02:00
parent a62b8d1733
commit 1275f78e36
8 changed files with 600 additions and 1 deletions
@@ -10,6 +10,47 @@ Format follows [Keep a Changelog](https://keepachangelog.com/) and [Semantic Ver

 ---

+## [1.17.0] - 2026-04-04
+
+### ADDED
+- **Public REST API** (`api/routes.py`, `api/__init__.py`): four read-only
+  GET endpoints registered on the NiceGUI/FastAPI application instance.
+  Enabled via `API_ENABLED = True` in `config.py` (default: on).
+  - `GET /api/v1/stats` — aggregate statistics for the last 72 hours:
+    total messages, unique senders, active nodes per type, average hops
+    and peak hour. Only public and hashtag channel messages are counted.
+  - `GET /api/v1/nodes` — full contact list with node type, GPS coordinates
+    and (when available) last-seen timestamp and battery voltage.
+  - `GET /api/v1/messages?limit=N&offset=N` — paginated message list
+    restricted to public (idx 0) and hashtag (`name.startswith('#')`)
+    channels. Private channel messages are unconditionally excluded.
+  - `GET /api/v1/channels` — channel list with `is_private` flag per entry.
+- **PublicApiService** (`services/public_api_service.py`): pure-Python
+  business logic for all four endpoints.  Contains the single source of
+  truth for channel-type classification (`is_public_channel`,
+  `is_private_channel`) used throughout the API layer.
+- **`API_ENABLED`** and **`API_CORS_ORIGINS`** constants (`config.py`):
+  toggle the API on/off and configure allowed CORS origins.
+
+### CHANGED
+- `config.py`: version bump `1.16.0 → 1.17.0`; `PUBLIC API` section added
+  with `API_ENABLED` and `API_CORS_ORIGINS`.
+- `__main__.py`: conditional `register_routes(_shared)` call after
+  `SharedData` construction; prints API URL or disabled notice at startup.
+
+### RATIONALE
+- Enables the domca.nl PHP collector to pull live mesh data over HTTP
+  without direct access to the SQLite archive or SharedData internals.
+- Filtering is enforced server-side: what is not public can never leak,
+  even without authentication.
+
+### IMPACT
+- No existing route, panel or BLE handler modified.
+- `SharedData` and `MessageArchive` accessed read-only.
+- Zero breaking changes to v1.16.0 behaviour when `API_ENABLED = False`.
+
+---
+
 ## [1.16.0] - 2026-04-04

 ### ADDED
@@ -56,6 +56,7 @@ A graphical user interface for MeshCore mesh network devices with native USB ser
  - [9.10. Keyword Bot](#910-keyword-bot)
  - [9.11. RX Log](#911-rx-log)
  - [9.12. Actions](#912-actions)
+  - [9.13. Public REST API](#913-public-rest-api)
 - [10. Architecture](#10-architecture)
 - [11. Cross-Frequency Bridge](#11-cross-frequency-bridge)
  - [11.1. Bridge Overview](#111-bridge-overview)
@@ -120,6 +121,7 @@ Under the hood it uses `meshcore` as the protocol layer, `meshcoredecoder` for r
 - **Dynamic Channel Discovery** — Channels are automatically discovered from the device at startup via probing, eliminating the need to manually configure `CHANNELS_CONFIG`
 <!-- ADDED: Dynamic channel discovery (v5.7.0) -->
 - **Add Channel** — Add hashtag or private channels directly from the GUI via the `＋ Add Channel` button in the Messages submenu. New private channels generate a shareable QR code and hex key for distribution to other users
+- **Public REST API** — Read-only JSON endpoints (`/api/v1/stats`, `/api/v1/nodes`, `/api/v1/messages`, `/api/v1/channels`) for external consumers such as statistics dashboards. Private channel messages are unconditionally excluded; no authentication required
 - **Keyword Bot** — Built-in auto-reply bot that responds to configurable keywords on selected channels, with cooldown and loop prevention
 - **Packet Decoding** — Raw LoRa packets from RX log are decoded and decrypted using channel keys, providing message hashes, path hashes and hop data
 - **Message Deduplication** — Dual-strategy dedup (hash-based and content-based) prevents duplicate messages from appearing
@@ -779,6 +781,66 @@ The built-in bot automatically replies to messages containing recognised keyword
 - Send advertisement
 - Set device name

+### 9.13. Public REST API
+
+MeshCore GUI exposes a lightweight read-only REST API under `/api/v1/`.
+It is designed for consumption by the [domca.nl](https://www.domca.nl)
+statistics pages but can be used by any HTTP client on the same network.
+
+Enable or disable the API in `meshcore_gui/config.py`:
+
+```python
+API_ENABLED: bool = True          # set False to disable all endpoints
+API_CORS_ORIGINS: list[str] = ["*"]   # restrict to your RPi IP in production
+```
+
+#### Endpoints
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/v1/stats` | Network statistics for the last 72 hours |
+| GET | `/api/v1/nodes` | All known mesh nodes with GPS and type |
+| GET | `/api/v1/messages?limit=100&offset=0` | Paginated public channel messages |
+| GET | `/api/v1/channels` | Channel list with `is_private` flag |
+
+**Privacy guarantee:** the `/api/v1/messages` endpoint returns messages from
+`Public` (index 0) and `#hashtag` channels **only**. Private channel messages
+are unconditionally excluded — no authentication is needed because there is
+nothing private in the response.
+
+#### Quick test
+
+```bash
+curl http://<meshcore-ip>:8081/api/v1/stats | python3 -m json.tool
+curl "http://<meshcore-ip>:8081/api/v1/messages?limit=5"
+```
+
+#### Example responses
+
+`GET /api/v1/stats`
+```json
+{
+  "generated_at": "2026-04-04T10:00:00+00:00",
+  "period_hours": 72,
+  "total_messages": 1240,
+  "unique_senders": 38,
+  "active_clients": 89,
+  "active_repeaters": 12,
+  "active_room_servers": 3,
+  "avg_hops": 1.8,
+  "peak_hour": 14
+}
+```
+
+`GET /api/v1/channels`
+```json
+[
+  {"idx": 0, "name": "Public",     "is_private": false},
+  {"idx": 1, "name": "#localmesh", "is_private": false},
+  {"idx": 2, "name": "TeamNL",     "is_private": true}
+]
+```
+
 ## 10. Architecture

 <!-- CHANGED: Architecture diagram updated — added RoomServerPanel and RoomPasswordStore (v5.7.0) -->
@@ -275,6 +275,14 @@ def main():
    from meshcore_gui.services.bbs_config_store import BbsConfigStore as _BCS
    _bbs_settings_page = BbsSettingsPage(_shared, _BCS())

+    # ── Register public REST API routes (optional) ──
+    if config.API_ENABLED:
+        from meshcore_gui.api.routes import register_routes
+        register_routes(_shared)
+        print(f"Public API enabled  — http://0.0.0.0:{port}/api/v1/stats")
+    else:
+        print("Public API disabled — set API_ENABLED=True in config.py to enable")
+
    # ── Start worker ──
    worker = create_worker(
        device_id,
@@ -0,0 +1,13 @@
+"""
+Public REST API package for MeshCore GUI.
+
+Exposes read-only JSON endpoints under /api/v1/ for consumption by
+external services (e.g. the domca.nl PHP statistics pages).
+
+Use :func:`register_routes` to wire the routes into the running
+NiceGUI/FastAPI application instance.
+"""
+
+from meshcore_gui.api.routes import register_routes
+
+__all__ = ["register_routes"]
@@ -0,0 +1,152 @@
+"""
+Public REST API route definitions for MeshCore GUI.
+
+Registers four read-only GET endpoints under ``/api/v1/`` on the
+NiceGUI/FastAPI application instance:
+
+    GET /api/v1/stats
+    GET /api/v1/nodes
+    GET /api/v1/messages
+    GET /api/v1/channels
+
+Call :func:`register_routes` once from ``__main__.py`` after
+:class:`~meshcore_gui.core.shared_data.SharedData` is constructed and
+before ``ui.run()`` is called.
+
+All routes are async and access shared data read-only.  CORS is
+configured from :data:`~meshcore_gui.config.API_CORS_ORIGINS`.
+"""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any, Dict, List
+
+from fastapi import Query
+from fastapi.middleware.cors import CORSMiddleware
+from nicegui import app as _nicegui_app
+
+import meshcore_gui.config as config
+from meshcore_gui.services.public_api_service import (
+    get_channels_payload,
+    get_messages_payload,
+    get_nodes_payload,
+    get_stats_payload,
+)
+
+if TYPE_CHECKING:
+    from meshcore_gui.core.shared_data import SharedData
+
+
+def register_routes(shared: "SharedData") -> None:
+    """Wire public API routes into the NiceGUI/FastAPI application.
+
+    Must be called after :class:`~meshcore_gui.core.shared_data.SharedData`
+    is constructed and **before** ``ui.run()`` so that FastAPI registers
+    the routes on startup.
+
+    CORS middleware is added once using the origins configured in
+    :data:`~meshcore_gui.config.API_CORS_ORIGINS`.  The middleware is
+    idempotent — calling this function more than once is safe (NiceGUI
+    guards against duplicate middleware).
+
+    Args:
+        shared: Application shared-data instance.  Passed to service
+                functions as a read-only data source.
+    """
+    # ── CORS ────────────────────────────────────────────────────────────
+    _nicegui_app.add_middleware(
+        CORSMiddleware,
+        allow_origins=config.API_CORS_ORIGINS,
+        allow_methods=["GET"],
+        allow_headers=["*"],
+    )
+
+    # ── Routes ──────────────────────────────────────────────────────────
+
+    @_nicegui_app.get(
+        "/api/v1/stats",
+        tags=["MeshCore Public API"],
+        summary="Network statistics for the last 72 hours",
+        response_model=None,
+    )
+    async def api_stats() -> Dict[str, Any]:
+        """Return aggregate statistics for the last 72 hours.
+
+        Only public (index 0) and hashtag channels are included in message
+        counts.  Node counts reflect the live contact list.
+
+        Returns:
+            JSON object with ``generated_at``, ``period_hours``,
+            ``total_messages``, ``unique_senders``, ``active_clients``,
+            ``active_repeaters``, ``active_room_servers``, ``avg_hops``
+            and ``peak_hour``.
+        """
+        return get_stats_payload(shared)
+
+    @_nicegui_app.get(
+        "/api/v1/nodes",
+        tags=["MeshCore Public API"],
+        summary="All known mesh nodes",
+        response_model=None,
+    )
+    async def api_nodes() -> List[Dict[str, Any]]:
+        """Return all contacts from the live contact list.
+
+        Fields not tracked by the current firmware interface (``last_seen``,
+        ``battery_mv``) are returned as ``null``.
+
+        Returns:
+            JSON array of node objects with ``name``, ``pubkey_prefix``,
+            ``type``, ``last_seen``, ``adv_lat``, ``adv_lon`` and
+            ``battery_mv``.
+        """
+        return get_nodes_payload(shared)
+
+    @_nicegui_app.get(
+        "/api/v1/messages",
+        tags=["MeshCore Public API"],
+        summary="Paginated public and hashtag channel messages",
+        response_model=None,
+    )
+    async def api_messages(
+        limit: int = Query(default=100, ge=1, le=500, description="Maximum items to return"),
+        offset: int = Query(default=0, ge=0, description="Items to skip"),
+    ) -> Dict[str, Any]:
+        """Return paginated messages from public and hashtag channels only.
+
+        Private channel messages are **never** returned, regardless of
+        authentication.  The filtering is enforced server-side.
+
+        Args:
+            limit:  Number of messages to return (1–500, default 100).
+            offset: Number of messages to skip for pagination (default 0).
+
+        Returns:
+            JSON object with ``total``, ``limit``, ``offset`` and ``items``
+            (list of message objects).
+        """
+        return get_messages_payload(shared, limit=limit, offset=offset)
+
+    @_nicegui_app.get(
+        "/api/v1/channels",
+        tags=["MeshCore Public API"],
+        summary="Channel list with privacy flag",
+        response_model=None,
+    )
+    async def api_channels() -> List[Dict[str, Any]]:
+        """Return all channels discovered from the device.
+
+        Each entry includes an ``is_private`` flag.  Private channels appear
+        in this list (so callers know they exist) but no keys or message
+        content is exposed.
+
+        Returns:
+            JSON array of channel objects with ``idx``, ``name`` and
+            ``is_private``.
+        """
+        return get_channels_payload(shared)
+
+    config.debug_print(
+        "Public API registered: /api/v1/stats, /api/v1/nodes, "
+        "/api/v1/messages, /api/v1/channels"
+    )
@@ -25,7 +25,7 @@ from typing import Any, Dict, List
 # ==============================================================================


-VERSION: str = "1.16.0"
+VERSION: str = "1.17.0"


 # ==============================================================================
@@ -291,6 +291,21 @@ MAX_CHANNELS: int = 100
 CHANNEL_CACHE_ENABLED: bool = False


+# ==============================================================================
+# PUBLIC API
+# ==============================================================================
+
+# Enable or disable the public REST API endpoints (/api/v1/*).
+# When False, no routes are registered and the API is unreachable.
+API_ENABLED: bool = True
+
+# CORS origins allowed to call the public API.
+# Set to the internal IP or hostname of the domca.nl Raspberry Pi.
+# Example: ["http://192.168.1.10", "https://www.domca.nl"]
+# Use ["*"] only on trusted internal networks.
+API_CORS_ORIGINS: list[str] = ["*"]
+
+
 # ==============================================================================
 # BOT DEVICE NAME
 # ==============================================================================
@@ -0,0 +1,308 @@
+"""
+Business logic for the MeshCore public REST API.
+
+This module contains pure data-transformation functions that are called
+by the API route handlers in :mod:`meshcore_gui.api.routes`.  It has
+**no** GUI, BLE or NiceGUI dependencies and may be imported from any layer.
+
+Channel-type rules (definitive — no exceptions):
+    idx == 0               → Public  — always expose
+    name.startswith('#')   → Hashtag — always expose
+    anything else          → Private — NEVER expose or store
+
+All functions access :class:`~meshcore_gui.core.shared_data.SharedData`
+and :class:`~meshcore_gui.services.message_archive.MessageArchive` in
+**read-only** mode.
+"""
+
+from __future__ import annotations
+
+from collections import Counter
+from datetime import datetime, timedelta, timezone
+from typing import TYPE_CHECKING, Any, Dict, List, Optional, Tuple
+
+if TYPE_CHECKING:
+    from meshcore_gui.core.shared_data import SharedData
+    from meshcore_gui.services.message_archive import MessageArchive
+
+
+# ---------------------------------------------------------------------------
+# Constants
+# ---------------------------------------------------------------------------
+
+#: Stats window in hours (72 h = 3 days).
+STATS_PERIOD_HOURS: int = 72
+
+#: Maximum messages fetched from the archive for stats computation.
+#: Adjust upwards if the archive grows very large and peak_hour is wrong.
+_STATS_FETCH_LIMIT: int = 50_000
+
+#: Node-type integer → API string mapping.
+#: Matches the MeshCore type field: 0/1 = Companion CLI, 2 = Repeater, 3 = Room Server.
+_NODE_TYPE_MAP: Dict[int, str] = {
+    0: "client",
+    1: "client",
+    2: "repeater",
+    3: "room_server",
+}
+
+
+# ---------------------------------------------------------------------------
+# Channel classification
+# ---------------------------------------------------------------------------
+
+def is_public_channel(idx: Optional[int], name: str) -> bool:
+    """Return True when the channel is public (idx 0) or a hashtag channel.
+
+    This is the single source of truth for channel-type classification used
+    throughout the public API.  Private channels are excluded from all
+    endpoints.
+
+    Args:
+        idx:  Channel index as stored on the device (``None`` for DMs).
+        name: Channel name string (e.g. ``"Public"``, ``"#localmesh"``).
+
+    Returns:
+        ``True`` for public or hashtag channels; ``False`` for everything else.
+    """
+    if idx == 0:
+        return True
+    if name and name.startswith("#"):
+        return True
+    return False
+
+
+def is_private_channel(idx: Optional[int], name: str) -> bool:
+    """Return True when the channel is private (inverse of :func:`is_public_channel`).
+
+    Args:
+        idx:  Channel index (``None`` for DMs).
+        name: Channel name string.
+
+    Returns:
+        ``True`` for private channels; ``False`` for public/hashtag.
+    """
+    return not is_public_channel(idx, name)
+
+
+# ---------------------------------------------------------------------------
+# Payload builders
+# ---------------------------------------------------------------------------
+
+def get_stats_payload(shared: "SharedData") -> Dict[str, Any]:
+    """Build the ``GET /api/v1/stats`` response payload.
+
+    Reads the last :data:`STATS_PERIOD_HOURS` hours of messages from the
+    archive, limited to public and hashtag channels.  All statistics are
+    derived from that filtered message set and from the live contact list.
+
+    Args:
+        shared: The application :class:`~meshcore_gui.core.shared_data.SharedData`
+                instance (read-only).
+
+    Returns:
+        Dict matching the ``/api/v1/stats`` JSON schema.
+    """
+    archive = shared.archive
+    now_utc = datetime.now(timezone.utc)
+    cutoff = now_utc - timedelta(hours=STATS_PERIOD_HOURS)
+
+    # ── Fetch messages from archive ──────────────────────────────────────
+    messages: List[Dict[str, Any]] = []
+    if archive is not None:
+        raw, _ = archive.query_messages(
+            after=cutoff,
+            limit=_STATS_FETCH_LIMIT,
+            offset=0,
+        )
+        messages = [
+            m for m in raw
+            if is_public_channel(m.get("channel"), m.get("channel_name", ""))
+        ]
+
+    # ── Aggregate stats ──────────────────────────────────────────────────
+    unique_senders: set = set()
+    hops_values: List[int] = []
+    hour_counter: Counter = Counter()
+
+    for msg in messages:
+        sender = msg.get("sender") or msg.get("sender_pubkey", "")
+        if sender:
+            unique_senders.add(sender)
+
+        path_len = msg.get("path_len", 0) or 0
+        if path_len > 0:
+            hops_values.append(path_len)
+
+        ts_str = msg.get("timestamp_utc", "")
+        if ts_str:
+            try:
+                ts = datetime.fromisoformat(ts_str)
+                hour_counter[ts.hour] += 1
+            except (ValueError, TypeError):
+                pass
+
+    avg_hops = round(sum(hops_values) / len(hops_values), 2) if hops_values else 0.0
+    peak_hour = hour_counter.most_common(1)[0][0] if hour_counter else None
+
+    # ── Node counts from live contact list ───────────────────────────────
+    active_clients = 0
+    active_repeaters = 0
+    active_room_servers = 0
+
+    with shared.lock:
+        for contact in shared.contacts.values():
+            node_type = int(contact.get("type", 0) or 0)
+            if node_type == 2:
+                active_repeaters += 1
+            elif node_type == 3:
+                active_room_servers += 1
+            else:
+                active_clients += 1
+
+    return {
+        "generated_at": now_utc.isoformat(),
+        "period_hours": STATS_PERIOD_HOURS,
+        "total_messages": len(messages),
+        "unique_senders": len(unique_senders),
+        "active_clients": active_clients,
+        "active_repeaters": active_repeaters,
+        "active_room_servers": active_room_servers,
+        "avg_hops": avg_hops,
+        "peak_hour": peak_hour,
+    }
+
+
+def get_nodes_payload(shared: "SharedData") -> List[Dict[str, Any]]:
+    """Build the ``GET /api/v1/nodes`` response payload.
+
+    Returns all known contacts from the live contact list.  Fields that are
+    not tracked by the current codebase (``last_seen``, ``battery_mv``) are
+    returned as ``null``.
+
+    Args:
+        shared: Application shared-data instance (read-only).
+
+    Returns:
+        List of node dicts matching the ``/api/v1/nodes`` JSON schema.
+    """
+    nodes: List[Dict[str, Any]] = []
+
+    with shared.lock:
+        for pubkey, contact in shared.contacts.items():
+            raw_type = int(contact.get("type", 0) or 0)
+            node_type = _NODE_TYPE_MAP.get(raw_type, "client")
+
+            adv_lat = contact.get("adv_lat") or None
+            adv_lon = contact.get("adv_lon") or None
+            # Zero-coordinates mean "unknown" — normalize to null
+            if adv_lat == 0.0 and adv_lon == 0.0:
+                adv_lat = None
+                adv_lon = None
+
+            nodes.append({
+                "name": contact.get("adv_name") or pubkey[:12],
+                "pubkey_prefix": pubkey[:12],
+                "type": node_type,
+                "last_seen": contact.get("last_seen"),        # null if absent
+                "adv_lat": adv_lat,
+                "adv_lon": adv_lon,
+                "battery_mv": contact.get("battery_mv"),     # null if absent
+            })
+
+    # Stable sort: repeaters first, then clients, then room servers
+    _order = {"repeater": 0, "client": 1, "room_server": 2}
+    nodes.sort(key=lambda n: (_order.get(n["type"], 9), n["name"]))
+    return nodes
+
+
+def get_messages_payload(
+    shared: "SharedData",
+    limit: int = 100,
+    offset: int = 0,
+) -> Dict[str, Any]:
+    """Build the ``GET /api/v1/messages`` response payload.
+
+    Returns paginated messages from public and hashtag channels **only**.
+    Private channel messages are excluded unconditionally — no authentication
+    is required precisely because the filtering is enforced server-side here.
+
+    Args:
+        shared: Application shared-data instance (read-only).
+        limit:  Maximum number of messages to return (capped at 500).
+        offset: Number of messages to skip (for pagination).
+
+    Returns:
+        Dict matching the ``/api/v1/messages`` JSON schema with ``total``,
+        ``limit``, ``offset`` and ``items`` keys.
+    """
+    # Hard cap: never return more than 500 messages per call
+    limit = min(max(1, limit), 500)
+    offset = max(0, offset)
+
+    archive = shared.archive
+    if archive is None:
+        return {"total": 0, "limit": limit, "offset": offset, "items": []}
+
+    # Fetch a large window so we can apply channel-type filtering before pagination.
+    # Because query_messages returns newest-first we fetch offset+limit+buffer rows.
+    fetch_limit = offset + limit + 1000  # generous buffer for filtered-out private msgs
+    raw, _ = archive.query_messages(limit=fetch_limit, offset=0)
+
+    # Filter: public + hashtag only
+    public_msgs = [
+        m for m in raw
+        if is_public_channel(m.get("channel"), m.get("channel_name", ""))
+    ]
+
+    total = len(public_msgs)
+    page = public_msgs[offset: offset + limit]
+
+    items: List[Dict[str, Any]] = []
+    for i, msg in enumerate(page):
+        items.append({
+            "id": offset + i + 1,                          # 1-based stable ID
+            "channel_idx": msg.get("channel"),
+            "channel_name": msg.get("channel_name", ""),
+            "sender": msg.get("sender", ""),
+            "text": msg.get("text", ""),
+            "timestamp": msg.get("timestamp_utc"),
+            "hops": msg.get("path_len", 0) or 0,
+            "path_hashes": msg.get("path_hashes") or [],
+        })
+
+    return {
+        "total": total,
+        "limit": limit,
+        "offset": offset,
+        "items": items,
+    }
+
+
+def get_channels_payload(shared: "SharedData") -> List[Dict[str, Any]]:
+    """Build the ``GET /api/v1/channels`` response payload.
+
+    Returns all channels discovered from the device, annotated with an
+    ``is_private`` flag.  Private channels are included in this list (so
+    callers know they exist) but their names are the only information
+    exposed — no keys or message content is ever returned.
+
+    Args:
+        shared: Application shared-data instance (read-only).
+
+    Returns:
+        List of channel dicts matching the ``/api/v1/channels`` JSON schema.
+    """
+    channels: List[Dict[str, Any]] = []
+
+    with shared.lock:
+        for ch in shared.channels:
+            idx = ch.get("idx")
+            name = ch.get("name", "")
+            channels.append({
+                "idx": idx,
+                "name": name,
+                "is_private": is_private_channel(idx, name),
+            })
+
+    return channels