diff --git a/CHANGELOG.md b/CHANGELOG.md index ba63a1d..6e1c1e8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,47 @@ Format follows [Keep a Changelog](https://keepachangelog.com/) and [Semantic Ver --- +## [1.17.0] - 2026-04-04 + +### ADDED +- **Public REST API** (`api/routes.py`, `api/__init__.py`): four read-only + GET endpoints registered on the NiceGUI/FastAPI application instance. + Enabled via `API_ENABLED = True` in `config.py` (default: on). + - `GET /api/v1/stats` — aggregate statistics for the last 72 hours: + total messages, unique senders, active nodes per type, average hops + and peak hour. Only public and hashtag channel messages are counted. + - `GET /api/v1/nodes` — full contact list with node type, GPS coordinates + and (when available) last-seen timestamp and battery voltage. + - `GET /api/v1/messages?limit=N&offset=N` — paginated message list + restricted to public (idx 0) and hashtag (`name.startswith('#')`) + channels. Private channel messages are unconditionally excluded. + - `GET /api/v1/channels` — channel list with `is_private` flag per entry. +- **PublicApiService** (`services/public_api_service.py`): pure-Python + business logic for all four endpoints. Contains the single source of + truth for channel-type classification (`is_public_channel`, + `is_private_channel`) used throughout the API layer. +- **`API_ENABLED`** and **`API_CORS_ORIGINS`** constants (`config.py`): + toggle the API on/off and configure allowed CORS origins. + +### CHANGED +- `config.py`: version bump `1.16.0 → 1.17.0`; `PUBLIC API` section added + with `API_ENABLED` and `API_CORS_ORIGINS`. +- `__main__.py`: conditional `register_routes(_shared)` call after + `SharedData` construction; prints API URL or disabled notice at startup. + +### RATIONALE +- Enables the domca.nl PHP collector to pull live mesh data over HTTP + without direct access to the SQLite archive or SharedData internals. +- Filtering is enforced server-side: what is not public can never leak, + even without authentication. + +### IMPACT +- No existing route, panel or BLE handler modified. +- `SharedData` and `MessageArchive` accessed read-only. +- Zero breaking changes to v1.16.0 behaviour when `API_ENABLED = False`. + +--- + ## [1.16.0] - 2026-04-04 ### ADDED diff --git a/README.md b/README.md index 3319d21..8cfa45e 100644 --- a/README.md +++ b/README.md @@ -56,6 +56,7 @@ A graphical user interface for MeshCore mesh network devices with native USB ser - [9.10. Keyword Bot](#910-keyword-bot) - [9.11. RX Log](#911-rx-log) - [9.12. Actions](#912-actions) + - [9.13. Public REST API](#913-public-rest-api) - [10. Architecture](#10-architecture) - [11. Cross-Frequency Bridge](#11-cross-frequency-bridge) - [11.1. Bridge Overview](#111-bridge-overview) @@ -120,6 +121,7 @@ Under the hood it uses `meshcore` as the protocol layer, `meshcoredecoder` for r - **Dynamic Channel Discovery** — Channels are automatically discovered from the device at startup via probing, eliminating the need to manually configure `CHANNELS_CONFIG` - **Add Channel** — Add hashtag or private channels directly from the GUI via the `＋ Add Channel` button in the Messages submenu. New private channels generate a shareable QR code and hex key for distribution to other users +- **Public REST API** — Read-only JSON endpoints (`/api/v1/stats`, `/api/v1/nodes`, `/api/v1/messages`, `/api/v1/channels`) for external consumers such as statistics dashboards. Private channel messages are unconditionally excluded; no authentication required - **Keyword Bot** — Built-in auto-reply bot that responds to configurable keywords on selected channels, with cooldown and loop prevention - **Packet Decoding** — Raw LoRa packets from RX log are decoded and decrypted using channel keys, providing message hashes, path hashes and hop data - **Message Deduplication** — Dual-strategy dedup (hash-based and content-based) prevents duplicate messages from appearing @@ -779,6 +781,66 @@ The built-in bot automatically replies to messages containing recognised keyword - Send advertisement - Set device name +### 9.13. Public REST API + +MeshCore GUI exposes a lightweight read-only REST API under `/api/v1/`. +It is designed for consumption by the [domca.nl](https://www.domca.nl) +statistics pages but can be used by any HTTP client on the same network. + +Enable or disable the API in `meshcore_gui/config.py`: + +```python +API_ENABLED: bool = True # set False to disable all endpoints +API_CORS_ORIGINS: list[str] = ["*"] # restrict to your RPi IP in production +``` + +#### Endpoints + +| Method | Path | Description | +|--------|------|-------------| +| GET | `/api/v1/stats` | Network statistics for the last 72 hours | +| GET | `/api/v1/nodes` | All known mesh nodes with GPS and type | +| GET | `/api/v1/messages?limit=100&offset=0` | Paginated public channel messages | +| GET | `/api/v1/channels` | Channel list with `is_private` flag | + +**Privacy guarantee:** the `/api/v1/messages` endpoint returns messages from +`Public` (index 0) and `#hashtag` channels **only**. Private channel messages +are unconditionally excluded — no authentication is needed because there is +nothing private in the response. + +#### Quick test + +```bash +curl http://:8081/api/v1/stats | python3 -m json.tool +curl "http://:8081/api/v1/messages?limit=5" +``` + +#### Example responses + +`GET /api/v1/stats` +```json +{ + "generated_at": "2026-04-04T10:00:00+00:00", + "period_hours": 72, + "total_messages": 1240, + "unique_senders": 38, + "active_clients": 89, + "active_repeaters": 12, + "active_room_servers": 3, + "avg_hops": 1.8, + "peak_hour": 14 +} +``` + +`GET /api/v1/channels` +```json +[ + {"idx": 0, "name": "Public", "is_private": false}, + {"idx": 1, "name": "#localmesh", "is_private": false}, + {"idx": 2, "name": "TeamNL", "is_private": true} +] +``` + ## 10. Architecture diff --git a/meshcore_gui/__main__.py b/meshcore_gui/__main__.py index 758e279..704e2b7 100644 --- a/meshcore_gui/__main__.py +++ b/meshcore_gui/__main__.py @@ -275,6 +275,14 @@ def main(): from meshcore_gui.services.bbs_config_store import BbsConfigStore as _BCS _bbs_settings_page = BbsSettingsPage(_shared, _BCS()) + # ── Register public REST API routes (optional) ── + if config.API_ENABLED: + from meshcore_gui.api.routes import register_routes + register_routes(_shared) + print(f"Public API enabled — http://0.0.0.0:{port}/api/v1/stats") + else: + print("Public API disabled — set API_ENABLED=True in config.py to enable") + # ── Start worker ── worker = create_worker( device_id, diff --git a/meshcore_gui/api/__init__.py b/meshcore_gui/api/__init__.py new file mode 100644 index 0000000..85e0159 --- /dev/null +++ b/meshcore_gui/api/__init__.py @@ -0,0 +1,13 @@ +""" +Public REST API package for MeshCore GUI. + +Exposes read-only JSON endpoints under /api/v1/ for consumption by +external services (e.g. the domca.nl PHP statistics pages). + +Use :func:`register_routes` to wire the routes into the running +NiceGUI/FastAPI application instance. +""" + +from meshcore_gui.api.routes import register_routes + +__all__ = ["register_routes"] diff --git a/meshcore_gui/api/routes.py b/meshcore_gui/api/routes.py new file mode 100644 index 0000000..b9a4034 --- /dev/null +++ b/meshcore_gui/api/routes.py @@ -0,0 +1,152 @@ +""" +Public REST API route definitions for MeshCore GUI. + +Registers four read-only GET endpoints under ``/api/v1/`` on the +NiceGUI/FastAPI application instance: + + GET /api/v1/stats + GET /api/v1/nodes + GET /api/v1/messages + GET /api/v1/channels + +Call :func:`register_routes` once from ``__main__.py`` after +:class:`~meshcore_gui.core.shared_data.SharedData` is constructed and +before ``ui.run()`` is called. + +All routes are async and access shared data read-only. CORS is +configured from :data:`~meshcore_gui.config.API_CORS_ORIGINS`. +""" + +from __future__ import annotations + +from typing import TYPE_CHECKING, Any, Dict, List + +from fastapi import Query +from fastapi.middleware.cors import CORSMiddleware +from nicegui import app as _nicegui_app + +import meshcore_gui.config as config +from meshcore_gui.services.public_api_service import ( + get_channels_payload, + get_messages_payload, + get_nodes_payload, + get_stats_payload, +) + +if TYPE_CHECKING: + from meshcore_gui.core.shared_data import SharedData + + +def register_routes(shared: "SharedData") -> None: + """Wire public API routes into the NiceGUI/FastAPI application. + + Must be called after :class:`~meshcore_gui.core.shared_data.SharedData` + is constructed and **before** ``ui.run()`` so that FastAPI registers + the routes on startup. + + CORS middleware is added once using the origins configured in + :data:`~meshcore_gui.config.API_CORS_ORIGINS`. The middleware is + idempotent — calling this function more than once is safe (NiceGUI + guards against duplicate middleware). + + Args: + shared: Application shared-data instance. Passed to service + functions as a read-only data source. + """ + # ── CORS ──────────────────────────────────────────────────────────── + _nicegui_app.add_middleware( + CORSMiddleware, + allow_origins=config.API_CORS_ORIGINS, + allow_methods=["GET"], + allow_headers=["*"], + ) + + # ── Routes ────────────────────────────────────────────────────────── + + @_nicegui_app.get( + "/api/v1/stats", + tags=["MeshCore Public API"], + summary="Network statistics for the last 72 hours", + response_model=None, + ) + async def api_stats() -> Dict[str, Any]: + """Return aggregate statistics for the last 72 hours. + + Only public (index 0) and hashtag channels are included in message + counts. Node counts reflect the live contact list. + + Returns: + JSON object with ``generated_at``, ``period_hours``, + ``total_messages``, ``unique_senders``, ``active_clients``, + ``active_repeaters``, ``active_room_servers``, ``avg_hops`` + and ``peak_hour``. + """ + return get_stats_payload(shared) + + @_nicegui_app.get( + "/api/v1/nodes", + tags=["MeshCore Public API"], + summary="All known mesh nodes", + response_model=None, + ) + async def api_nodes() -> List[Dict[str, Any]]: + """Return all contacts from the live contact list. + + Fields not tracked by the current firmware interface (``last_seen``, + ``battery_mv``) are returned as ``null``. + + Returns: + JSON array of node objects with ``name``, ``pubkey_prefix``, + ``type``, ``last_seen``, ``adv_lat``, ``adv_lon`` and + ``battery_mv``. + """ + return get_nodes_payload(shared) + + @_nicegui_app.get( + "/api/v1/messages", + tags=["MeshCore Public API"], + summary="Paginated public and hashtag channel messages", + response_model=None, + ) + async def api_messages( + limit: int = Query(default=100, ge=1, le=500, description="Maximum items to return"), + offset: int = Query(default=0, ge=0, description="Items to skip"), + ) -> Dict[str, Any]: + """Return paginated messages from public and hashtag channels only. + + Private channel messages are **never** returned, regardless of + authentication. The filtering is enforced server-side. + + Args: + limit: Number of messages to return (1–500, default 100). + offset: Number of messages to skip for pagination (default 0). + + Returns: + JSON object with ``total``, ``limit``, ``offset`` and ``items`` + (list of message objects). + """ + return get_messages_payload(shared, limit=limit, offset=offset) + + @_nicegui_app.get( + "/api/v1/channels", + tags=["MeshCore Public API"], + summary="Channel list with privacy flag", + response_model=None, + ) + async def api_channels() -> List[Dict[str, Any]]: + """Return all channels discovered from the device. + + Each entry includes an ``is_private`` flag. Private channels appear + in this list (so callers know they exist) but no keys or message + content is exposed. + + Returns: + JSON array of channel objects with ``idx``, ``name`` and + ``is_private``. + """ + return get_channels_payload(shared) + + config.debug_print( + "Public API registered: /api/v1/stats, /api/v1/nodes, " + "/api/v1/messages, /api/v1/channels" + ) diff --git a/meshcore_gui/config.py b/meshcore_gui/config.py index 74fac3f..6a126b0 100644 --- a/meshcore_gui/config.py +++ b/meshcore_gui/config.py @@ -25,7 +25,7 @@ from typing import Any, Dict, List # ============================================================================== -VERSION: str = "1.16.0" +VERSION: str = "1.17.0" # ============================================================================== @@ -291,6 +291,21 @@ MAX_CHANNELS: int = 100 CHANNEL_CACHE_ENABLED: bool = False +# ============================================================================== +# PUBLIC API +# ============================================================================== + +# Enable or disable the public REST API endpoints (/api/v1/*). +# When False, no routes are registered and the API is unreachable. +API_ENABLED: bool = True + +# CORS origins allowed to call the public API. +# Set to the internal IP or hostname of the domca.nl Raspberry Pi. +# Example: ["http://192.168.1.10", "https://www.domca.nl"] +# Use ["*"] only on trusted internal networks. +API_CORS_ORIGINS: list[str] = ["*"] + + # ============================================================================== # BOT DEVICE NAME # ============================================================================== diff --git a/meshcore_gui/services/public_api_service.py b/meshcore_gui/services/public_api_service.py new file mode 100644 index 0000000..b8770f6 --- /dev/null +++ b/meshcore_gui/services/public_api_service.py @@ -0,0 +1,308 @@ +""" +Business logic for the MeshCore public REST API. + +This module contains pure data-transformation functions that are called +by the API route handlers in :mod:`meshcore_gui.api.routes`. It has +**no** GUI, BLE or NiceGUI dependencies and may be imported from any layer. + +Channel-type rules (definitive — no exceptions): + idx == 0 → Public — always expose + name.startswith('#') → Hashtag — always expose + anything else → Private — NEVER expose or store + +All functions access :class:`~meshcore_gui.core.shared_data.SharedData` +and :class:`~meshcore_gui.services.message_archive.MessageArchive` in +**read-only** mode. +""" + +from __future__ import annotations + +from collections import Counter +from datetime import datetime, timedelta, timezone +from typing import TYPE_CHECKING, Any, Dict, List, Optional, Tuple + +if TYPE_CHECKING: + from meshcore_gui.core.shared_data import SharedData + from meshcore_gui.services.message_archive import MessageArchive + + +# --------------------------------------------------------------------------- +# Constants +# --------------------------------------------------------------------------- + +#: Stats window in hours (72 h = 3 days). +STATS_PERIOD_HOURS: int = 72 + +#: Maximum messages fetched from the archive for stats computation. +#: Adjust upwards if the archive grows very large and peak_hour is wrong. +_STATS_FETCH_LIMIT: int = 50_000 + +#: Node-type integer → API string mapping. +#: Matches the MeshCore type field: 0/1 = Companion CLI, 2 = Repeater, 3 = Room Server. +_NODE_TYPE_MAP: Dict[int, str] = { + 0: "client", + 1: "client", + 2: "repeater", + 3: "room_server", +} + + +# --------------------------------------------------------------------------- +# Channel classification +# --------------------------------------------------------------------------- + +def is_public_channel(idx: Optional[int], name: str) -> bool: + """Return True when the channel is public (idx 0) or a hashtag channel. + + This is the single source of truth for channel-type classification used + throughout the public API. Private channels are excluded from all + endpoints. + + Args: + idx: Channel index as stored on the device (``None`` for DMs). + name: Channel name string (e.g. ``"Public"``, ``"#localmesh"``). + + Returns: + ``True`` for public or hashtag channels; ``False`` for everything else. + """ + if idx == 0: + return True + if name and name.startswith("#"): + return True + return False + + +def is_private_channel(idx: Optional[int], name: str) -> bool: + """Return True when the channel is private (inverse of :func:`is_public_channel`). + + Args: + idx: Channel index (``None`` for DMs). + name: Channel name string. + + Returns: + ``True`` for private channels; ``False`` for public/hashtag. + """ + return not is_public_channel(idx, name) + + +# --------------------------------------------------------------------------- +# Payload builders +# --------------------------------------------------------------------------- + +def get_stats_payload(shared: "SharedData") -> Dict[str, Any]: + """Build the ``GET /api/v1/stats`` response payload. + + Reads the last :data:`STATS_PERIOD_HOURS` hours of messages from the + archive, limited to public and hashtag channels. All statistics are + derived from that filtered message set and from the live contact list. + + Args: + shared: The application :class:`~meshcore_gui.core.shared_data.SharedData` + instance (read-only). + + Returns: + Dict matching the ``/api/v1/stats`` JSON schema. + """ + archive = shared.archive + now_utc = datetime.now(timezone.utc) + cutoff = now_utc - timedelta(hours=STATS_PERIOD_HOURS) + + # ── Fetch messages from archive ────────────────────────────────────── + messages: List[Dict[str, Any]] = [] + if archive is not None: + raw, _ = archive.query_messages( + after=cutoff, + limit=_STATS_FETCH_LIMIT, + offset=0, + ) + messages = [ + m for m in raw + if is_public_channel(m.get("channel"), m.get("channel_name", "")) + ] + + # ── Aggregate stats ────────────────────────────────────────────────── + unique_senders: set = set() + hops_values: List[int] = [] + hour_counter: Counter = Counter() + + for msg in messages: + sender = msg.get("sender") or msg.get("sender_pubkey", "") + if sender: + unique_senders.add(sender) + + path_len = msg.get("path_len", 0) or 0 + if path_len > 0: + hops_values.append(path_len) + + ts_str = msg.get("timestamp_utc", "") + if ts_str: + try: + ts = datetime.fromisoformat(ts_str) + hour_counter[ts.hour] += 1 + except (ValueError, TypeError): + pass + + avg_hops = round(sum(hops_values) / len(hops_values), 2) if hops_values else 0.0 + peak_hour = hour_counter.most_common(1)[0][0] if hour_counter else None + + # ── Node counts from live contact list ─────────────────────────────── + active_clients = 0 + active_repeaters = 0 + active_room_servers = 0 + + with shared.lock: + for contact in shared.contacts.values(): + node_type = int(contact.get("type", 0) or 0) + if node_type == 2: + active_repeaters += 1 + elif node_type == 3: + active_room_servers += 1 + else: + active_clients += 1 + + return { + "generated_at": now_utc.isoformat(), + "period_hours": STATS_PERIOD_HOURS, + "total_messages": len(messages), + "unique_senders": len(unique_senders), + "active_clients": active_clients, + "active_repeaters": active_repeaters, + "active_room_servers": active_room_servers, + "avg_hops": avg_hops, + "peak_hour": peak_hour, + } + + +def get_nodes_payload(shared: "SharedData") -> List[Dict[str, Any]]: + """Build the ``GET /api/v1/nodes`` response payload. + + Returns all known contacts from the live contact list. Fields that are + not tracked by the current codebase (``last_seen``, ``battery_mv``) are + returned as ``null``. + + Args: + shared: Application shared-data instance (read-only). + + Returns: + List of node dicts matching the ``/api/v1/nodes`` JSON schema. + """ + nodes: List[Dict[str, Any]] = [] + + with shared.lock: + for pubkey, contact in shared.contacts.items(): + raw_type = int(contact.get("type", 0) or 0) + node_type = _NODE_TYPE_MAP.get(raw_type, "client") + + adv_lat = contact.get("adv_lat") or None + adv_lon = contact.get("adv_lon") or None + # Zero-coordinates mean "unknown" — normalize to null + if adv_lat == 0.0 and adv_lon == 0.0: + adv_lat = None + adv_lon = None + + nodes.append({ + "name": contact.get("adv_name") or pubkey[:12], + "pubkey_prefix": pubkey[:12], + "type": node_type, + "last_seen": contact.get("last_seen"), # null if absent + "adv_lat": adv_lat, + "adv_lon": adv_lon, + "battery_mv": contact.get("battery_mv"), # null if absent + }) + + # Stable sort: repeaters first, then clients, then room servers + _order = {"repeater": 0, "client": 1, "room_server": 2} + nodes.sort(key=lambda n: (_order.get(n["type"], 9), n["name"])) + return nodes + + +def get_messages_payload( + shared: "SharedData", + limit: int = 100, + offset: int = 0, +) -> Dict[str, Any]: + """Build the ``GET /api/v1/messages`` response payload. + + Returns paginated messages from public and hashtag channels **only**. + Private channel messages are excluded unconditionally — no authentication + is required precisely because the filtering is enforced server-side here. + + Args: + shared: Application shared-data instance (read-only). + limit: Maximum number of messages to return (capped at 500). + offset: Number of messages to skip (for pagination). + + Returns: + Dict matching the ``/api/v1/messages`` JSON schema with ``total``, + ``limit``, ``offset`` and ``items`` keys. + """ + # Hard cap: never return more than 500 messages per call + limit = min(max(1, limit), 500) + offset = max(0, offset) + + archive = shared.archive + if archive is None: + return {"total": 0, "limit": limit, "offset": offset, "items": []} + + # Fetch a large window so we can apply channel-type filtering before pagination. + # Because query_messages returns newest-first we fetch offset+limit+buffer rows. + fetch_limit = offset + limit + 1000 # generous buffer for filtered-out private msgs + raw, _ = archive.query_messages(limit=fetch_limit, offset=0) + + # Filter: public + hashtag only + public_msgs = [ + m for m in raw + if is_public_channel(m.get("channel"), m.get("channel_name", "")) + ] + + total = len(public_msgs) + page = public_msgs[offset: offset + limit] + + items: List[Dict[str, Any]] = [] + for i, msg in enumerate(page): + items.append({ + "id": offset + i + 1, # 1-based stable ID + "channel_idx": msg.get("channel"), + "channel_name": msg.get("channel_name", ""), + "sender": msg.get("sender", ""), + "text": msg.get("text", ""), + "timestamp": msg.get("timestamp_utc"), + "hops": msg.get("path_len", 0) or 0, + "path_hashes": msg.get("path_hashes") or [], + }) + + return { + "total": total, + "limit": limit, + "offset": offset, + "items": items, + } + + +def get_channels_payload(shared: "SharedData") -> List[Dict[str, Any]]: + """Build the ``GET /api/v1/channels`` response payload. + + Returns all channels discovered from the device, annotated with an + ``is_private`` flag. Private channels are included in this list (so + callers know they exist) but their names are the only information + exposed — no keys or message content is ever returned. + + Args: + shared: Application shared-data instance (read-only). + + Returns: + List of channel dicts matching the ``/api/v1/channels`` JSON schema. + """ + channels: List[Dict[str, Any]] = [] + + with shared.lock: + for ch in shared.channels: + idx = ch.get("idx") + name = ch.get("name", "") + channels.append({ + "idx": idx, + "name": name, + "is_private": is_private_channel(idx, name), + }) + + return channels diff --git a/meshcore_gui_Iteratie_A_result.zip b/meshcore_gui_Iteratie_A_result.zip new file mode 100644 index 0000000..2aecad9 Binary files /dev/null and b/meshcore_gui_Iteratie_A_result.zip differ