iarv/meshcore-gui
1
0
Fork 1
mirror of https://github.com/pe1hvh/meshcore-gui.git synced 2026-03-28 17:42:38 +01:00
Code Issues Packages Projects Releases Wiki Activity

Initial clean code

Browse Source
This commit is contained in:
pe1hvh
2026-03-09 17:53:29 +01:00
commit d8a7947c6b
111 changed files with 24373 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.gitattributes vendored Normal file
View File

@@ -0,0 +1,2 @@
# Auto detect text files and perform LF normalization
* text=auto

9
.gitignore vendored Normal file
View File

@@ -0,0 +1,9 @@
.venv/
venv/
__pycache__/
*.pyc
logs/*.log
logs/*.txt
.DS_Store
.idea/
.vscode/

89
AGENTS.md Normal file
View File

@@ -0,0 +1,89 @@
# MeshCore GUI — Agent Notes
## Purpose
MeshCore GUI is a NiceGUI-based desktop/headless web UI for MeshCore radios. It supports **two transport modes**:
- **USB Serial** via `MeshCore.create_serial()` — for wired connections
- **Bluetooth LE** via `MeshCore.create_ble()` — for wireless T1000e devices
The transport is **auto-detected** from the device argument.
## Entry Points
- `meshcore_gui.py` (primary)
- `python -m meshcore_gui` (`meshcore_gui/__main__.py`)
### Common Run
```bash
# Serial
./venv/bin/python meshcore_gui.py /dev/ttyACM0 --debug-on --baud=115200
# BLE
./venv/bin/python meshcore_gui.py literal:AA:BB:CC:DD:EE:FF --debug-on --ble-pin 123456
```
## Architecture (High-Level)
- **UI thread (NiceGUI)**: `meshcore_gui/gui/*`
- **Worker thread (transport + asyncio)**: `meshcore_gui/ble/worker.py`
- `_BaseWorker` — shared main loop, caching, data loading
- `SerialWorker` — USB serial transport
- `BLEWorker` — Bluetooth LE transport (PIN agent, bond management)
- `create_worker()` — factory function for auto-detection
- **Commands**: `meshcore_gui/ble/commands.py`
- **Events**: `meshcore_gui/ble/events.py`
- **Shared state**: `meshcore_gui/core/shared_data.py`
- **BLE PIN agent**: `meshcore_gui/ble/ble_agent.py` (D-Bus, Linux only)
- **BLE reconnect**: `meshcore_gui/ble/ble_reconnect.py` (bond cleanup)
## Transport Detection
`config.is_ble_address(device_id)` returns True when:
- Device ID starts with `literal:` prefix
- Device ID matches `XX:XX:XX:XX:XX:XX` MAC address pattern
Everything else is treated as a serial port path.
## Config (`meshcore_gui/config.py`)
- `TRANSPORT`: `"serial"` or `"ble"` (set at startup)
- Serial: `SERIAL_BAUDRATE`, `SERIAL_CX_DELAY`, `DEFAULT_TIMEOUT`
- BLE: `BLE_PIN`, `DEFAULT_TIMEOUT`
- Shared: `MESHCORE_LIB_DEBUG`, `RECONNECT_*`, `CONTACT_REFRESH_SECONDS`
## Dependencies
- **Serial mode**: `meshcore`, `nicegui`, `meshcoredecoder`
- **BLE mode**: additionally `bleak`, `dbus_fast` (Linux only)
- BLE dependencies are **lazily imported** — serial-only installs don't need them.
## Device Name Behavior
- BOT toggle changes device name via `set_device_name` command.
- Warning labels are shown next to BOT toggles.
- Explicit device name can be set via the **Actions** panel input.
## Map Centering
- Map is in `meshcore_gui/gui/panels/map_panel.py`.
- Centering happens on device updates or when the MAP panel is opened.
- There is a **Center on Device** button that uses the last known GPS.
- Leaflet size invalidation is called before centering to handle hidden panels.
- Map theme follows UI dark/light mode by default.
- Map theme can be overridden with the **Theme** toggle (Auto/Dark/Light).
## Panel URLs
- Drawer and sidebar actions navigate to `/?panel=<id>&channel=<optional>` so browser back restores the last panel.
- On load, the dashboard reads the query params and shows the requested panel.
## Route Viewer
- Clicking a message opens `/route/{msg_key}` in the **same tab**.
- The route page has **Back to Dashboard** and **Back to Archive** buttons.
## Refresh Behavior
- GUI refresh queues a full device reload.
- Contacts fetch is bounded by a timeout to prevent hangs.
## Persistent Data
Stored under `~/.meshcore-gui/`:
- `cache/`, `archive/`, `logs/`, `pins/`, `room_passwords/`
## Tests
- Tests live in `tests/`.
- `pytest` is not installed by default; use `pip install pytest` in the venv.
## Installer Scripts
- `install_serial.sh` — systemd service for serial connections
- `install_ble_stable.sh` — systemd service for BLE connections

333
BRIDGE.md Normal file
View File

@@ -0,0 +1,333 @@
# MeshCore Bridge — Cross-Frequency Message Bridge
### No MQTT, no broker, no cloud. Just LoRa ↔ LoRa.
![Status](https://img.shields.io/badge/Status-Production-green.svg)
![Python](https://img.shields.io/badge/Python-3.10+-blue.svg)
![License](https://img.shields.io/badge/License-MIT-green.svg)
![Platform](https://img.shields.io/badge/Platform-Linux-orange.svg)
![Transport](https://img.shields.io/badge/Transport-Dual%20USB%20Serial-blueviolet.svg)
![Bridge](https://img.shields.io/badge/Bridge-Cross--Frequency%20LoRa%20↔%20LoRa-ff6600.svg)
A standalone daemon that connects two MeshCore devices operating on different radio frequencies. It forwards messages on a configurable bridge channel from one device to the other, effectively extending your mesh network across frequency boundaries.
## Table of Contents
- [1. Overview](#1-overview)
- [2. Features](#2-features)
- [3. Requirements](#3-requirements)
- [3.1. Requirement Status](#31-requirement-status)
- [4. Installation](#4-installation)
- [4.1. Quick Start](#41-quick-start)
- [4.2. systemd Service](#42-systemd-service)
- [5. Configuration](#5-configuration)
- [5.1. Bridge Settings](#51-bridge-settings)
- [5.2. Command-Line Options](#52-command-line-options)
- [6. How It Works](#6-how-it-works)
- [6.1. Message Flow](#61-message-flow)
- [6.2. Loop Prevention](#62-loop-prevention)
- [6.3. Private (Encrypted) Channels](#63-private-encrypted-channels)
- [7. Dashboard](#7-dashboard)
- [8. File Structure](#8-file-structure)
- [9. Assumptions](#9-assumptions)
- [10. Troubleshooting](#10-troubleshooting)
- [10.1. Bridge Won't Start](#101-bridge-wont-start)
- [10.2. Messages Not Forwarding](#102-messages-not-forwarding)
- [10.3. Port Conflicts](#103-port-conflicts)
- [10.4. Service Issues](#104-service-issues)
- [11. License](#11-license)
- [12. Author](#12-author)
---
## 1. Overview
The bridge runs as an independent process alongside (or instead of) the regular meshcore_gui instances. It imports the existing meshcore_gui modules (SharedData, Worker, models, config) as a library and requires **zero modifications** to the meshcore_gui codebase.
```
┌───────────────────────────────────────────┐
│ meshcore_bridge daemon │
│ │
│ ┌──────────────┐ ┌────────────────┐ │
│ │ SharedData A │ │ BridgeEngine │ │
│ │ + Worker A │◄──►│ (forward & │ │
│ │ (ttyUSB1) │ │ dedup) │ │
│ └──────────────┘ └────────────────┘ │
│ ┌──────────────┐ │ │
│ │ SharedData B │◄────────┘ │
│ │ + Worker B │ │
│ │ (ttyUSB2) │ │
│ └──────────────┘ │
│ │
│ ┌───────────────────────────────────┐ │
│ │ Bridge Dashboard (NiceGUI :9092) │ │
│ │ - Device A & B status │ │
│ │ - Forwarded message log │ │
│ │ - Bridge config view │ │
│ └───────────────────────────────────┘ │
└───────────────────────────────────────────┘
```
Key properties:
- **Separate process** — the bridge runs independently from meshcore_gui; both can run simultaneously on the same host
- **Loop prevention** — three mechanisms prevent message loops: direction filter, message hash tracking, and echo suppression
- **Private channels** — encrypted channels work transparently because the bridge operates at the plaintext level between firmware decryption and encryption
- **DOMCA dashboard** — status page on its own port showing both device connections, bridge statistics and a forwarded message log
- **YAML configuration** — all settings in a single `bridge_config.yaml` file
## 2. Features
- **Bidirectional forwarding** — Messages on the bridge channel are forwarded A→B and B→A automatically
- **Loop prevention** — Direction filter, message hash tracking, and echo suppression prevent infinite loops
- **Private channel support** — Encrypted channels work transparently; bridge operates at the plaintext level
- **DOMCA dashboard** — Live status page with device connections, bridge statistics and forwarded message log
- **YAML configuration** — All settings in a single `bridge_config.yaml` file
- **systemd integration** — Install as a background daemon with automatic restart
- **Zero meshcore_gui changes** — Imports existing modules as a library, 0 changed files
## 3. Requirements
- Python 3.10+
- meshcore_gui (installed or on PYTHONPATH)
- meshcore Python library (`pip install meshcore`)
- pyyaml (`pip install pyyaml`)
- Two MeshCore devices connected via USB serial
### 3.1. Requirement Status
| ID | Requirement | Status |
|----|-------------|--------|
| M1 | Bridge as separate process, meshcore_gui unchanged | ✅ |
| M2 | Forward messages on #bridge channel A↔B within <2s | ✅ |
| M3 | YAML config for channel, ports, polling interval | ✅ |
| M4 | 0 changed files in meshcore_gui/ | ✅ |
| M5 | GUI identical to meshcore_gui (DOMCA theme) | ✅ |
| M6 | Configurable port (--port=9092) | ✅ |
| M7 | Loop prevention via forwarded-hash set | ✅ |
| M8 | Two devices on different frequencies | ✅ |
| M9 | Private (encrypted) channels fully supported | ✅ |
---
## 4. Installation
### 4.1. Quick Start
```bash
# 1. Install the dependency
pip install pyyaml
# 2. Copy the config template and edit it
cp bridge_config.yaml /etc/meshcore/bridge_config.yaml
nano /etc/meshcore/bridge_config.yaml
# 3. Run the bridge
python meshcore_bridge.py --config=/etc/meshcore/bridge_config.yaml
# 4. Open the dashboard at http://your-host:9092
```
**Prerequisites:** Two MeshCore devices connected via USB serial to the same host, with the bridge channel configured on both devices using the same channel secret/password.
### 4.2. systemd Service
Install the bridge as a systemd daemon for production use:
```bash
# Run the installer script
sudo bash install_bridge.sh
# Edit the configuration
sudo nano /etc/meshcore/bridge_config.yaml
# Start the service
sudo systemctl start meshcore-bridge
sudo systemctl enable meshcore-bridge
```
**Useful service commands:**
| Command | Description |
|---------|-------------|
| `sudo systemctl status meshcore-bridge` | Check if the service is running |
| `sudo journalctl -u meshcore-bridge -f` | Follow the live log output |
| `sudo systemctl restart meshcore-bridge` | Restart after a configuration change |
| `sudo systemctl stop meshcore-bridge` | Stop the service |
**Uninstall:**
```bash
sudo bash install_bridge.sh --uninstall
```
---
## 5. Configuration
### 5.1. Bridge Settings
All settings are defined in `bridge_config.yaml`:
```yaml
bridge:
channel_name: "bridge" # Channel name (for display)
channel_idx_a: 3 # Channel index on device A
channel_idx_b: 3 # Channel index on device B
poll_interval_ms: 200 # Polling interval (ms)
forward_prefix: true # Add [sender] prefix
max_forwarded_cache: 500 # Loop prevention cache size
device_a:
port: /dev/ttyUSB1 # Serial port device A
baud: 115200 # Baud rate
label: "869.525 MHz" # Dashboard label
device_b:
port: /dev/ttyUSB2 # Serial port device B
baud: 115200 # Baud rate
label: "868.000 MHz" # Dashboard label
gui:
port: 9092 # Dashboard port
title: "MeshCore Bridge" # Browser tab title
```
### 5.2. Command-Line Options
| Flag | Description | Default |
|------|-------------|---------|
| `--config=PATH` | Path to YAML config file | `./bridge_config.yaml` |
| `--port=PORT` | Override GUI port | From config (9092) |
| `--debug-on` | Enable debug logging | Off |
| `--help` | Show usage info | — |
---
## 6. How It Works
### 6.1. Message Flow
1. **Device A** receives a channel message on the bridge channel via LoRa
2. MeshCore firmware decrypts the message (if private channel) and passes plaintext to the Worker
3. The Worker's EventHandler stores the message in **SharedData A**
4. **BridgeEngine** polls SharedData A, detects the new message, checks dedup hash set
5. BridgeEngine injects a `send_message` command into **SharedData B**'s command queue
6. Worker B picks up the command and transmits the message on Device B's bridge channel
7. MeshCore firmware on Device B encrypts (if private channel) and transmits via LoRa
The reverse direction (B→A) works identically.
### 6.2. Loop Prevention
The bridge uses three mechanisms to prevent message loops:
1. **Direction filter** — Only incoming messages (`direction='in'`) are forwarded. Messages we transmitted (`direction='out'`) are never forwarded.
2. **Message hash tracking** — Each forwarded message's hash is stored in a bounded set (configurable via `max_forwarded_cache`). If the same hash appears again, it is blocked.
3. **Echo suppression** — When a message is forwarded, the hash of the forwarded text (including `[sender]` prefix) is also registered, preventing the forwarded message from being re-forwarded when it appears on the target device.
### 6.3. Private (Encrypted) Channels
The bridge works transparently with both public and private channels. No extra configuration is needed beyond ensuring that both devices have the same channel secret/password:
- **Inbound**: MeshCore firmware decrypts → Worker receives plaintext → BridgeEngine reads plaintext
- **Outbound**: BridgeEngine injects command → Worker sends via meshcore lib → Firmware encrypts → LoRa TX
> **Prerequisite:** The bridge channel MUST be configured on both devices with **identical channel secret/password**. Only the frequency and channel index may differ.
---
## 7. Dashboard
The bridge dashboard is accessible at `http://your-host:9092` (or your configured port) and shows:
- **Configuration summary** — active channel, indices, poll interval
- **Device A status** — connection state, device name, radio frequency
- **Device B status** — connection state, device name, radio frequency
- **Bridge statistics** — messages forwarded (total, A→B, B→A), duplicates blocked, uptime
- **Forwarded message log** — last 200 forwarded messages with timestamps and direction
The dashboard uses the same DOMCA theme as meshcore_gui with dark/light mode toggle.
---
## 8. File Structure
```
meshcore_bridge.py # Entry point (~25 lines)
meshcore_bridge/
├── __init__.py # Package init
├── __main__.py # CLI, dual-worker setup, NiceGUI server (~180 lines)
├── config.py # YAML config loading (~130 lines)
├── bridge_engine.py # Core bridge logic (~250 lines)
└── gui/
├── __init__.py # GUI package init
├── dashboard.py # Bridge dashboard page (~180 lines)
└── panels/
├── __init__.py # Panels package init
├── status_panel.py # Device connection status (~180 lines)
└── log_panel.py # Forwarded message log (~100 lines)
bridge_config.yaml # Configuration template
install_bridge.sh # systemd service installer
BRIDGE.md # This documentation
```
**Total new code:** ~1,050 lines
**Changed files in meshcore_gui/:** 0 (zero)
---
## 9. Assumptions
- Both MeshCore devices are connected via USB serial to the same host (Raspberry Pi / Linux server)
- The bridge channel exists on both devices with the same name (but possibly different index)
- The bridge channel has identical channel secret/password on both devices
- The meshcore_gui package is importable (installed via `pip install -e .` or on PYTHONPATH)
- Sufficient CPU/RAM for two simultaneous MeshCore connections (~100MB)
- Messages are forwarded with a sender prefix `[original_sender]` for identification
---
## 10. Troubleshooting
### 10.1. Bridge Won't Start
- Check that both serial ports exist: `ls -l /dev/ttyUSB*`
- Verify meshcore_gui is importable: `python -c "from meshcore_gui.core.shared_data import SharedData"`
- Check pyyaml is installed: `pip install pyyaml`
### 10.2. Messages Not Forwarding
- Verify the bridge channel index matches on both devices
- Check that the channel secret is identical on both devices
- Look at the dashboard: are both devices showing "Connected"?
- Enable debug mode: `python meshcore_bridge.py --debug-on`
### 10.3. Port Conflicts
| Daemon | Default Port |
|---|---|
| meshcore_gui | 8081 |
| **meshcore_bridge** | **9092** |
| meshcore_observer | 9093 |
Change via `--port=XXXX` or in `bridge_config.yaml`.
### 10.4. Service Issues
```bash
sudo systemctl status meshcore-bridge
journalctl -u meshcore-bridge -f
sudo systemctl restart meshcore-bridge
```
---
## 11. License
MIT License — Copyright (c) 2026 PE1HVH
## 12. Author
**PE1HVH** — [GitHub](https://github.com/pe1hvh) — DOMCA MeshCore Project

806
CHANGELOG.md Normal file
View File

@@ -0,0 +1,806 @@
## [1.13.1] - 2026-03-09
### Fixed
- Route map markers now use the same JS-rendered node icons as the main MAP instead of NiceGUI default blue markers.
- Route detail pages now bootstrap their Leaflet assets explicitly so the shared map icon runtime is available there too.
### Changed
- Route maps are now rendered browser-side through the shared Leaflet JS runtime for icon consistency with MAP, Messages, and Archive.
# CHANGELOG
<!-- CHANGED: Title changed from "CHANGELOG: Message & Metadata Persistence" to "CHANGELOG" —
a root-level CHANGELOG.md should be project-wide, not feature-specific. -->
All notable changes to MeshCore GUI are documented in this file.
Format follows [Keep a Changelog](https://keepachangelog.com/) and [Semantic Versioning](https://semver.org/).
---
## [1.13.1] - 2026-03-09 — Message Icon Consistency
### Changed
- 🔄 `meshcore_gui/gui/constants.py` — Added shared helper functions to resolve node-type icons and labels from the same contact type mapping used by the map and contacts panel
- 🔄 `meshcore_gui/core/models.py` — `Message.format_line()` now supports an optional sender prefix so message-related views can prepend the same node icon set without changing existing formatting logic
- 🔄 `meshcore_gui/gui/panels/messages_panel.py` — Message rows now prepend the sender with the same node icon mapping as the map/contact views
- 🔄 `meshcore_gui/gui/archive_page.py` — Archive rows now use the same sender icon mapping as the live messages panel and map/contact views
- 🔄 `meshcore_gui/gui/route_page.py` — Route header and route detail table now show node-type icons derived from the shared contact type mapping instead of generic hardcoded role icons
### Impact
- Message-driven views now use one consistent icon language across map, contacts, messages, archive and route detail
- Existing map runtime and panel behavior remain unchanged
- No breaking changes outside icon rendering
## [1.13.0] - 2026-03-09 — Leaflet Map Runtime Stabilization
### Added
- ✅ `meshcore_gui/static/leaflet_map_panel.js` — Dedicated browser-side Leaflet runtime responsible for map lifecycle, marker registry and theme handling independent from NiceGUI redraw cycles
- ✅ `meshcore_gui/static/leaflet_map_panel.css` — Styling for browser-side node markers and map container
- ✅ `meshcore_gui/services/map_snapshot_service.py` — Snapshot service that normalizes device/contact map data into a compact payload for the browser runtime
- ✅ Browser-side map state management for center, zoom and theme
- ✅ Theme persistence across reconnect events via browser storage fallback
### Changed
- 🔄 `meshcore_gui/gui/panels/map_panel.py` — Replaced NiceGUI Leaflet wrapper usage with a pure browser-managed Leaflet container while preserving the existing card layout, theme toggle and center-on-device control
- 🔄 Leaflet bootstrap moved out of inline Python into a dedicated browser runtime loaded from `/static`
- 🔄 Map initialization now occurs only once per container; NiceGUI refresh cycles no longer recreate the map
- 🔄 Dashboard update loop now sends compact map snapshots instead of triggering redraws
- 🔄 Snapshot processing in the browser is coalesced so only the newest payload is applied
- 🔄 Map markers are managed in separate device/contact layers and updated incrementally by stable node id
- 🔄 Theme switching moved to a dedicated theme channel instead of being embedded in snapshot data
### Fixed
- 🛠 **Map disappearing during dashboard refresh cycles** — prevented repeated map reinitialization caused by the 500 ms NiceGUI update loop
- 🛠 **Markers disappearing between refreshes** — marker updates are now incremental and keyed by node id
- 🛠 **Blank map container on load** — browser bootstrap now waits for DOM host, Leaflet runtime and panel runtime before initialization
- 🛠 **Race condition between queued snapshot and theme selection** — explicit theme changes can no longer be overwritten by stale snapshot payloads
- 🛠 **Viewport jumping back to default center/zoom** — stored viewport is no longer reapplied on each snapshot update
- 🛠 **Theme reverting to default during reconnect** — effective map theme is restored before snapshot processing resumes
### Impact
- Leaflet map is now managed entirely in the browser and is no longer recreated on each dashboard refresh
- Node markers remain stable and no longer flicker or disappear during the 500 ms update cycle
- Theme switching and viewport state persist reliably across reconnect events
- No breaking changes outside the map subsystem
---
## [1.12.1] - 2026-03-08 — Minor change bot
### Changed
- 🔄 `meshcore_gui/services/bot.py`: remove path id's
### Impact
- No breaking changes — all existing functionality preserved serial.
---
## [1.12.0] - 2026-02-26 — MeshCore Observer Fase 1
### Added
- ✅ **MeshCore Observer daemon** — New standalone read-only daemon (`meshcore_observer.py`) that reads archive JSON files produced by meshcore_gui and meshcore_bridge, aggregates them, and presents a unified NiceGUI monitoring dashboard on port 9093.
- ✅ **ArchiveWatcher** — Core component that polls `~/.meshcore-gui/archive/` for `*_messages.json` and `*_rxlog.json` files, tracks mtime changes, and returns only new entries since previous poll. Thread-safe, zero writes, graceful on corrupt JSON.
- ✅ **Observer dashboard panels** — Sources overview, aggregated messages feed (sorted by timestamp), aggregated RX log table, and statistics panel with uptime/counters/per-source breakdown. Full DOMCA theme (dark + light mode).
- ✅ **Source filter** — Dropdown to filter messages and RX log by archive source.
- ✅ **Channel filter** — Dropdown to filter messages by channel name.
- ✅ **ObserverConfig** — YAML-based configuration with `from_yaml()` classmethod, defaults work without config file.
- ✅ **observer_config.yaml** — Documented config template with all options.
- ✅ **install_observer.sh** — systemd installer (`/opt/meshcore-observer/`, `/etc/meshcore/observer_config.yaml`), with `--uninstall` option.
- ✅ **RxLogEntry raw packet fields** — 5 new fields on `RxLogEntry` dataclass: `raw_payload`, `packet_len`, `payload_len`, `route_type`, `packet_type_num` (all with defaults, backward compatible).
- ✅ **EventHandler.on_rx_log() metadata** — Raw payload hex and packet metadata now passed through to RxLogEntry and archived (preparation for Fase 2 LetsMesh uplink).
### Changed
- 🔄 `meshcore_gui/core/models.py`: RxLogEntry +5 fields with defaults (backward compatible).
- 🔄 `meshcore_gui/ble/events.py`: on_rx_log() fills raw_payload and metadata (~10 lines added).
- 🔄 `meshcore_gui/services/message_archive.py`: add_rx_log() serializes the 5 new RxLogEntry fields.
- 🔄 `meshcore_gui/config.py`: Version bumped to `1.12.0`.
### Impact
- **No breaking changes** — All new RxLogEntry fields have defaults; existing archives and code work identically.
- **New daemon** — meshcore_observer is fully standalone; no imports from meshcore_gui (reads only JSON files).
---
### Added
- ✅ **Serial CLI flags** — `--baud=BAUD` and `--serial-cx-dly=SECONDS` for serial configuration at startup.
### Changed
- 🔄 **Connection layer** — Switched from BLE to serial (`MeshCore.create_serial`) with serial reconnect handling.
- 🔄 `config.py`: Added `SERIAL_BAUDRATE`, `SERIAL_CX_DELAY`, `DEFAULT_TIMEOUT`, `MESHCORE_LIB_DEBUG`; removed BLE PIN settings; version bumped to `1.10.0`.
- 🔄 `meshcore_gui.py` / `meshcore_gui/__main__.py`: Updated usage, banners and defaults for serial ports.
- 🔄 Docs: Updated README and core docs for serial usage; BLE documents marked as legacy.
### Impact
- No breaking changes — all existing functionality preserved serial.
---
## [1.9.11] - 2026-02-19 — Message Dedup Hotfix
### Fixed
- 🛠 **Duplicate messages after (re)connect** — `load_recent_from_archive()` appended archived messages on every connect attempt without clearing existing entries; after N failed connects, each message appeared N times. Method is now idempotent: clears the in-memory list before loading.
- 🛠 **Persistent duplicate messages** — Live BLE events for messages already loaded from archive were not suppressed because the `DualDeduplicator` was never seeded with archived content. Added `_seed_dedup_from_messages()` in `BLEWorker` after cache/archive load and after reconnect.
- 🛠 **Last-line-of-defence dedup in SharedData** — `add_message()` now maintains a fingerprint set (`message_hash` or `channel:sender:text`) and silently skips messages whose fingerprint is already tracked. This guards against duplicates regardless of their source.
- 🛠 **Messages panel empty on first click** — `_show_panel()` made the container visible but relied on the next 500 ms timer tick to populate it. Added an immediate `_messages.update()` call so content is rendered the moment the panel becomes visible.
### Changed
- 🔄 `core/shared_data.py`: Added `_message_fingerprints` set and `_message_fingerprint()` static method; `add_message()` checks fingerprint before insert and evicts fingerprints when messages are rotated out; `load_recent_from_archive()` clears messages and fingerprints before loading (idempotent)
- 🔄 `ble/worker.py`: Added `_seed_dedup_from_messages()` helper; called after `_apply_cache()` and after reconnect `_load_data()` to seed `DualDeduplicator` with existing messages
- 🔄 `gui/dashboard.py`: `_show_panel()` now forces an immediate `_messages.update()` when the messages panel is shown, eliminating the stale-content flash
- 🔄 `config.py`: Version bumped to `1.9.11`
### Impact
- Eliminates all duplicate message display scenarios: initial connect, failed retries, reconnect, and BLE event replay
- No breaking changes — all existing functionality preserved
- Fingerprint set is bounded to the same 100-message cap as the message list
---
## [1.9.10] - 2026-02-19 — Map Tooltips & Separate Own-Position Marker
### Added
- ✅ **Map marker tooltips** — All markers on the Leaflet map now show a tooltip on hover with the node name and type icon (📱, 📡, 🏠) from `TYPE_ICONS`
- ✅ **Separate own-position marker** — The device's own position is now tracked as a dedicated `_own_marker`, independent from contact markers. This prevents the own marker from being removed/recreated on every contact update cycle
### Changed
- 🔄 `gui/panels/map_panel.py`: Renamed `_markers` to `_contacts_markers`; added `_own_marker` attribute; own position marker is only updated when `device_updated` flag is set (not every timer tick); contact markers are only rebuilt when `contacts_updated` is set; added `TYPE_ICONS` import for tooltip icons
- 🔄 `gui/dashboard.py`: Added `self._map.update(data)` call in the `device_updated` block so the own-position marker updates when device info changes (e.g. GPS position update)
- 🔄 `config.py`: Version bumped to `1.9.10`
### Impact
- Map centering on own device now works correctly and updates only when position actually changes
- Contact markers are no longer needlessly destroyed and recreated on every UI timer tick — only on actual contact data changes
- Tooltips make it easy to identify nodes on the map without clicking
- No breaking changes — all existing map functionality preserved
### Credits
- Based on [PR #16](https://github.com/pe1hvh/meshcore-gui/pull/16) by [@rich257](https://github.com/rich257)
---
## [1.9.9] - 2026-02-18 — Variable Landing Page & Operator Callsign
### Added
- ✅ **Configurable operator callsign** — New `OPERATOR_CALLSIGN` constant in `config.py` (default: `"PE1HVH"`). Used in the landing page SVG and the drawer footer copyright label. Change this single value to personalize the entire GUI for a different operator
- ✅ **External landing page SVG** — The DOMCA splash screen is now loaded from a standalone file (`static/landing_default.svg`) instead of being hardcoded in `dashboard.py`. New `LANDING_SVG_PATH` constant in `config.py` points to the SVG file. The placeholder `{callsign}` in the SVG is replaced at runtime with `OPERATOR_CALLSIGN`
- ✅ **Landing page customization** — To use a custom landing page: copy `landing_default.svg` (or create your own SVG), use `{callsign}` wherever the operator callsign should appear, and point `LANDING_SVG_PATH` to your file. The default SVG includes an instructive comment block explaining the placeholder mechanism
### Changed
- 🔄 `config.py`: Added `OPERATOR_CALLSIGN` and `LANDING_SVG_PATH` constants in new **OPERATOR / LANDING PAGE** section; version bumped to `1.9.9`
- 🔄 `gui/dashboard.py`: Removed hardcoded `_DOMCA_SVG` string (~70 lines); added `_load_landing_svg()` helper that reads SVG from disk and replaces `{callsign}` placeholder; CSS variable `--pe1hvh` renamed to `--callsign`; drawer footer copyright label now uses `config.OPERATOR_CALLSIGN`
### Added (files)
- ✅ `static/landing_default.svg` — The original DOMCA splash SVG extracted as a standalone file, with `{callsign}` placeholder and `--callsign` CSS variable. Serves as both the default landing page and a reference template for custom SVGs
### Impact
- Out-of-the-box behavior is identical to v1.9.8 (same DOMCA branding, same PE1HVH callsign)
- Operators personalize by changing 1–2 lines in `config.py` — no code modifications needed
- Fallback: if the SVG file is missing, a minimal placeholder text is shown instead of a crash
- No breaking changes — all existing dashboard functionality (panels, menus, timer, theming) unchanged
---
## [1.9.8] - 2026-02-17 — Bugfix: Route Page Sender ID, Type & Location Not Populated
### Fixed
- 🛠 **Sender ID, Type and Location empty in Route Page** — After the v4.1 refactoring to `RouteBuilder`/`RouteNode`, the sender contact lookup relied solely on `SharedData.get_contact_by_prefix()` (live lock-based) and `get_contact_by_name()`. When both failed (e.g. empty `sender_pubkey` from RX_LOG decode, or name mismatch), `route['sender']` remained `None` and the route table fell through to a hardcoded fallback with `type: '-'`, `location: '-'`. The contact data was available in the snapshot `data['contacts']` but was never searched
- 🛠 **Route table fallback row ignored available contact data** — When `route['sender']` was `None`, the `_render_route_table` method used a static fallback row without attempting to find the contact in the data snapshot. Even when the contact was present in `data['contacts']` with valid type and location, these fields showed as `'-'`
### Changed
- 🔄 `services/route_builder.py`: Added two additional fallback strategies in `build()` after the existing SharedData lookups: (3) bidirectional pubkey prefix match against `data['contacts']` snapshot, (4) case-insensitive `adv_name` match against `data['contacts']` snapshot. Added helper methods `_find_contact_by_pubkey()` and `_find_contact_by_adv_name()` for snapshot-based lookups
- 🔄 `gui/route_page.py`: Added defensive fallback in `_render_route_table()` sender section — when `route['sender']` is `None`, attempts to find the contact in the snapshot via `_find_sender_contact()` before falling back to the static `'-'` row. Added `_find_sender_contact()` helper method
### Impact
- Sender ID (hash), Type and Location are now populated correctly in the route table when the contact is known
- Four-layer lookup chain ensures maximum resolution: (1) SharedData pubkey lookup, (2) SharedData name lookup, (3) snapshot pubkey lookup, (4) snapshot name lookup
- Defensive fallback in route_page guarantees data is shown even if RouteBuilder misses it
- No breaking changes — all existing route page behavior, styling and data flows unchanged
---
## [1.9.7] - 2026-02-17 — Layout Fix: Archive Filter Toggle & Route Page Styling
### Changed
- 🔄 `gui/archive_page.py`: Archive filter card now hidden by default; toggle visibility via a `filter_list` icon button placed right-aligned on the same row as the "📚 Archive" title. Header restructured from single label to `ui.row()` with `justify-between` layout
- 🔄 `gui/route_page.py`: Route page now uses DOMCA theme (imported from `dashboard.py`) with dark mode as default, consistent with the main dashboard. Header restyled from `bg-blue-600` to Quasar-themed header with JetBrains Mono font. Content container changed from `w-full max-w-4xl mx-auto` to `domca-panel` class for consistent responsive sizing
- 🔄 `gui/dashboard.py`: Added `domca-header-text` CSS class with `@media (max-width: 599px)` rule to hide header text on narrow viewports; applied to version label and status label
- 🔄 `gui/route_page.py`: Header label also uses `domca-header-text` class for consistent responsive behaviour
### Added
- ✅ **Archive filter toggle** — `filter_list` icon button in archive header row toggles the filter card visibility on click
- ✅ **Route page close button** — `X` (close) icon button added right-aligned in the route page header; calls `window.close()` to close the browser tab
- ✅ **Responsive header** — On viewports < 600px, header text labels are hidden; only icon buttons (menu, dark mode toggle, close) remain visible
### Impact
- Archive page is cleaner by default — filters only shown when needed
- Route page visually consistent with the main dashboard (DOMCA theme, dark mode, responsive panel width)
- Headers degrade gracefully on mobile (< 600px): only icon buttons visible, no text overflow
- No functional changes — all event handlers, callbacks, data bindings, logic and imports are identical to the input
---
## [1.9.6] - 2026-02-17 — Bugfix: Channel Discovery Reliability
### Fixed
- 🛠 **Channels not appearing (especially on mobile)** — Channel discovery aborted too early on slow BLE connections. The `_discover_channels()` probe used a single attempt per channel slot and stopped after just 2 consecutive empty responses. On mobile BLE stacks (WebBluetooth via NiceGUI) where GATT responses are slower, this caused discovery to abort before finding any channels, falling back to only `[0] Public`
- 🛠 **Race condition: channel update flag lost between threads** — `get_snapshot()` and `clear_update_flags()` were two separate calls, each acquiring the lock independently. If the BLE worker set `channels_updated = True` between these two calls, the GUI consumed the flag via `get_snapshot()` but then `clear_update_flags()` reset it — causing the channel submenu and dropdown to never populate
- 🛠 **Channels disappear on browser reconnect** — When a browser tab is closed and reopened, `render()` creates new (empty) NiceGUI containers for the drawer submenus, but did not reset `_last_channel_fingerprint`. The `_update_submenus()` method compared the new fingerprint against the stale one, found them equal, and skipped the rebuild — leaving the new containers permanently empty. Fixed by resetting both `_last_channel_fingerprint` and `_last_rooms_fingerprint` in `render()`
### Changed
- 🔄 `core/shared_data.py`: New atomic method `get_snapshot_and_clear_flags()` that reads the snapshot and resets all update flags in a single lock acquisition. Internally refactored to `_build_snapshot_unlocked()` helper. Existing `get_snapshot()` and `clear_update_flags()` retained for backward compatibility
- 🔄 `ble/worker.py`: `_discover_channels()` — `max_attempts` increased from 1 to 2 per channel slot; inter-attempt `delay` increased from 0.5s to 1.0s; consecutive error threshold raised from 2 to 3; inter-channel pause increased from 0.15s to 0.3s for mobile BLE stack breathing room
- 🔄 `gui/dashboard.py`: `_update_ui()` now uses `get_snapshot_and_clear_flags()` instead of separate `get_snapshot()` + `clear_update_flags()`; `render()` now resets `_last_channel_fingerprint` and `_last_rooms_fingerprint` to `None` so that `_update_submenus()` rebuilds into the freshly created containers; channel-dependent updates (`update_filters`, `update_channel_options`, `_update_submenus`) now run unconditionally when channel data exists — safe because each method has internal idempotency checks
- 🔄 `gui/panels/messages_panel.py`: `update_channel_options()` now includes an equality check on options dict to skip redundant `.update()` calls to the NiceGUI client on every 500ms timer tick
### Impact
- Channel discovery now survives transient BLE timeouts that are common on mobile connections
- Atomic snapshot eliminates the threading race condition that caused channels to silently never appear
- Browser close+reopen no longer loses channels — the single-instance timer race on the shared `DashboardPage` is fully mitigated
- No breaking changes — all existing API methods retained, all other functionality unchanged
---
## [1.9.5] - 2026-02-16 — Layout Fix: RX Log Table Responsive Sizing
### Fixed
- 🛠 **RX Log table did not adapt to panel/card size** — The table used `max-h-48` (a maximum height cap) instead of a responsive fixed height, causing it to remain small regardless of available space. Changed to `h-40` which is overridden by the existing dashboard CSS to `calc(100vh - 20rem)` — the same responsive pattern used by the Messages panel
- 🛠 **RX Log table did not fill card width** — Added `w-full` class to the table element so it stretches to the full width of the parent card
- 🛠 **RX Log card did not fill panel height** — Added `flex-grow` class to the card container so it expands to fill the available panel space
### Changed
- 🔄 `gui/panels/rxlog_panel.py`: Card classes `'w-full'` → `'w-full flex-grow'` (line 45); table classes `'text-xs max-h-48 overflow-y-auto'` → `'w-full text-xs h-40 overflow-y-auto'` (line 65)
### Impact
- RX Log table now fills the panel consistently on both desktop and mobile viewports
- Layout is consistent with other panels (Messages, Contacts) that use the same `h-40` responsive height pattern
- No functional changes — all event handlers, callbacks, data bindings, logica and imports are identical to the input
---
## [1.9.4] - 2026-02-16 — BLE Address Log Prefix & Entry Point Cleanup
### Added
- ✅ **BLE address prefix in log filename** — Log file is now named `<BLE_ADDRESS>_meshcore_gui.log` (e.g. `AA_BB_CC_DD_EE_FF_meshcore_gui.log`) instead of the generic `meshcore_gui.log`. Makes it easy to identify which device produced which log file when running multiple instances
- New helper `_sanitize_ble_address()` strips `literal:` prefix and replaces colons with underscores
- New function `configure_log_file(ble_address)` updates `LOG_FILE` at runtime before the logger is initialised
- Rotated backups follow the same naming pattern automatically
### Removed
- ❌ **`meshcore_gui/meshcore_gui.py`** — Redundant copy of `main()` that was never imported. All three entry points (`meshcore_gui.py` root, `__main__.py`, and `meshcore_gui/meshcore_gui.py`) contained near-identical copies of the same logic, causing changes to be missed (as demonstrated by this fix). `__main__.py` is now the single source of truth; root `meshcore_gui.py` is a thin wrapper that imports from it
### Changed
- 🔄 `config.py`: Added `_sanitize_ble_address()` and `configure_log_file()`; version bumped to `1.9.4`
- 🔄 `__main__.py`: Added `config.configure_log_file(ble_address)` call before any debug output
- 🔄 `meshcore_gui.py` (root): Reduced to 4-line wrapper importing `main` from `__main__`
### Impact
- Log files are now identifiable per BLE device
- Single source of truth for `main()` eliminates future sync issues between entry points
- Both startup methods (`python meshcore_gui.py` and `python -m meshcore_gui`) remain functional
- No breaking changes — defaults and all existing behaviour unchanged
---
## [1.9.3] - 2026-02-16 — Bugfix: Map Default Location & Payload Type Decoding
### Fixed
- 🛠 **Map centred on hardcoded Zwolle instead of device location** — All Leaflet maps used magic-number coordinates `(52.5, 6.0)` as initial centre and fallback. These are now replaced by a single configurable constant `DEFAULT_MAP_CENTER` in `config.py`. Once the device reports a valid `adv_lat`/`adv_lon`, maps re-centre on the actual device position (existing behaviour, unchanged)
- 🛠 **Payload type shown as raw integer** — Payload type is now retrieved from the decoded payload and translated to human-readable text using MeshCoreDecoder functions, instead of displaying the raw numeric type value
### Changed
- 🔄 `config.py`: Added `DEFAULT_MAP_CENTER` (default: `(52.5168, 6.0830)`) and `DEFAULT_MAP_ZOOM` (default: `9`) constants in new **MAP DEFAULTS** section. Version bumped to `1.9.2`
- 🔄 `gui/panels/map_panel.py`: Imports `DEFAULT_MAP_CENTER` and `DEFAULT_MAP_ZOOM` from config; `ui.leaflet(center=...)` uses config constants instead of hardcoded values
- 🔄 `gui/route_page.py`: Imports `DEFAULT_MAP_CENTER` and `DEFAULT_MAP_ZOOM` from config; fallback coordinates (`or 52.5` / `or 6.0`) replaced by `DEFAULT_MAP_CENTER[0]` / `[1]`; zoom uses `DEFAULT_MAP_ZOOM`
### Impact
- Map default location is now a single-point-of-change in `config.py`
- Payload type is displayed as readable text instead of a raw number
- No breaking changes — all existing map behaviour (re-centre on device position, contact markers) unchanged
## [1.9.2] - 2026-02-15 — CLI Parameters & Cleanup
### Added
- ✅ **`--port=PORT` CLI parameter** — Web server port is now configurable at startup (default: `8081`). Allows running multiple instances simultaneously on different ports
- ✅ **`--ble-pin=PIN` CLI parameter** — BLE pairing PIN is now configurable at startup (default: `123456`). Eliminates the need to edit `config.py` for devices with a non-default PIN, and works in systemd service files
- ✅ **Per-device log file** — Debug log file now includes the BLE address in its filename (e.g. `F0_9E_9E_75_A3_01_meshcore_gui.log`), so multiple instances log to separate files
### Fixed
- 🛠 **BLE PIN not applied from CLI** — `ble/worker.py` imported `BLE_PIN` as a constant at module load time (`from config import BLE_PIN`), capturing the default value `"123456"` before CLI parsing could override `config.BLE_PIN`. Changed to runtime access via `config.BLE_PIN` so the `--ble-pin` parameter is correctly passed to the BLE agent
### Removed
- ❌ **Redundant `meshcore_gui/meshcore_gui.py`** — This file was a near-identical copy of both `meshcore_gui.py` (top-level) and `meshcore_gui/__main__.py`, but was never imported or referenced. Removed to eliminate maintenance risk. The two remaining entry points cover all startup methods: `python meshcore_gui.py` and `python -m meshcore_gui`
### Impact
- Multiple instances can run side-by-side with different ports, PINs and log files
- Service deployments no longer require editing `config.py` — all runtime settings via CLI
- No breaking changes — all defaults are unchanged
---
## [1.9.1] - 2026-02-14 — Bugfix: Dual Reconnect Conflict
### Fixed
- 🛠 **Library reconnect interfered with application reconnect** — The meshcore library's internal `auto_reconnect` (visible in logs as `"Attempting reconnection 1/3"`) ran a fast 3-attempt reconnect cycle without bond cleanup. This prevented the application's own `reconnect_loop` (which does `remove_bond()` + backoff) from succeeding, because BlueZ retained a stale bond → `"failed to discover service"`
### Changed
- 🔄 `ble/worker.py`: Set `auto_reconnect=False` in both `MeshCore.create_ble()` call sites (`_connect()` and `_create_fresh_connection()`), so only the application's bond-aware `reconnect_loop` handles reconnection
- 🔄 `ble/worker.py`: Added `"failed to discover"` and `"service discovery"` to disconnect detection keywords for defensive coverage
### Impact
- Eliminates the ~9 second wasted library reconnect cycle after every BLE disconnect
- Application's `reconnect_loop` (with bond cleanup) now runs immediately after disconnect detection
- No breaking changes — the application reconnect logic was already fully functional
---
## [1.9.0] - 2026-02-14 — BLE Connection Stability
### Added
- ✅ **Built-in BLE PIN agent** — New `ble/ble_agent.py` registers a D-Bus agent with BlueZ to handle PIN pairing requests automatically. Eliminates the need for external `bt-agent.service` and `bluez-tools` package
- Uses `dbus_fast` (already a dependency of `bleak`, no new packages)
- Supports `RequestPinCode`, `RequestPasskey`, `DisplayPasskey`, `RequestConfirmation`, `AuthorizeService` callbacks
- Configurable PIN via `BLE_PIN` in `config.py` (default: `123456`)
- ✅ **Automatic bond cleanup** — New `ble/ble_reconnect.py` provides `remove_bond()` function that removes stale BLE bonds via D-Bus, equivalent to `bluetoothctl remove <address>`. Called automatically on startup and before each reconnect attempt
- ✅ **Automatic reconnect after disconnect** — BLEWorker main loop now detects BLE disconnects (via connection error exceptions) and automatically triggers a reconnect sequence: bond removal → linear backoff wait → fresh connection → re-wire handlers → reload device data
- Configurable via `RECONNECT_MAX_RETRIES` (default: 5) and `RECONNECT_BASE_DELAY` (default: 5.0s)
- After all retries exhausted: waits 60s then starts a new retry cycle (infinite recovery)
- ✅ **Generic install script** — `install_ble_stable.sh` auto-detects user, project directory, venv path and entry point to generate systemd service and D-Bus policy. Supports `--uninstall` flag
### Changed
- 🔄 **`ble/worker.py`** — `_async_main()` rewritten with three phases: (1) start PIN agent, (2) remove stale bond, (3) connect + main loop with disconnect detection. Reconnect logic re-wires all event handlers and reloads device data after successful reconnection
- 🔄 **`config.py`** — Added `BLE_PIN`, `RECONNECT_MAX_RETRIES`, `RECONNECT_BASE_DELAY` constants
### Removed
- ❌ **`bt-agent.service` dependency** — No longer needed; PIN pairing is handled by the built-in agent
- ❌ **`bluez-tools` system package** — No longer needed
- ❌ **`~/.meshcore-ble-pin` file** — No longer needed
- ❌ **Manual `bluetoothctl remove` before startup** — Handled automatically
- ❌ **`ExecStartPre` in systemd service** — Bond cleanup is internal
### Impact
- Zero external dependencies for BLE pairing on Linux
- Automatic recovery from the T1000e ~2 hour BLE disconnect issue
- No manual intervention needed after BLE connection loss
- Single systemd service (`meshcore-gui.service`) manages everything
- No breaking changes to existing functionality
---
## [1.8.0] - 2026-02-14 — DRY Message Construction & Archive Layout Unification
### Fixed
- 🛠 **Case-sensitive prefix matching** — `get_contact_name_by_prefix()` and `get_contact_by_prefix()` in `shared_data.py` failed to match path hashes (uppercase, e.g. `'B8'`) against contact pubkeys (lowercase, e.g. `'b8a3f2...'`). Added `.lower()` to both sides of the comparison, consistent with `_resolve_path_names()` which already had it
- 🛠 **Route page 404 from archive** — Archive page linked to `/route/{hash}` but route was registered as `/route/{msg_index:int}`, causing a JSON parse error for hex hash strings. Route parameter changed to `str` with 3-strategy lookup (index → memory hash → archive fallback)
- 🛠 **Three entry points out of sync** — `meshcore_gui.py` (root), `meshcore_gui/meshcore_gui.py` (inner) and `meshcore_gui/__main__.py` had diverging route registrations. All three now use identical `/route/{msg_key}` with `str` parameter
### Changed
- 🔄 **`core/models.py` — DRY factory methods and formatting**
- `Message.now_timestamp()`: static method replacing 7× hardcoded `datetime.now().strftime('%H:%M:%S')` across `events.py` and `commands.py`
- `Message.incoming()`: classmethod factory for received messages (`direction='in'`, auto-timestamp)
- `Message.outgoing()`: classmethod factory for sent messages (`sender='Me'`, `direction='out'`, auto-timestamp)
- `Message.format_line(channel_names)`: single-line display formatting (`"12:34:56 ← [Public] [2h✓] PE1ABC: Hello mesh!"`), replacing duplicate inline formatting in `messages_panel.py` and `archive_page.py`
- 🔄 **`ble/events.py`** — 4× `Message(...)` constructors replaced by `Message.incoming()`; `datetime` import removed
- 🔄 **`ble/commands.py`** — 3× `Message(...)` constructors replaced by `Message.outgoing()`; `datetime` import removed
- 🔄 **`gui/panels/messages_panel.py`** — 15 lines inline formatting replaced by single `msg.format_line(channel_names)` call
- 🔄 **`gui/archive_page.py` — Layout unified with main page**
- Multi-row card layout replaced by single-line `msg.format_line()` in monospace container (same style as main page)
- DM added to channel filter dropdown (post-filter on `channel is None`)
- Message click opens `/route/{message_hash}` in new tab (was: no click handler on archive messages)
- Removed `_render_message_card()` (98 lines) and `_render_archive_route()` (75 lines)
- Removed `RouteBuilder` dependency and `TYPE_LABELS` import
- File reduced from 445 to 267 lines
- 🔄 **`gui/route_page.py`** — `render(msg_index: int)` → `render(msg_key: str)` with 3-strategy message lookup: (1) numeric index from in-memory list, (2) hash match in memory, (3) `archive.get_message_by_hash()` fallback
- 🔄 **`services/message_archive.py`** — New method `get_message_by_hash(hash)` for single-message lookup by packet hash
- 🔄 **`__main__.py` + `meshcore_gui.py` (both)** — Route changed from `/route/{msg_index}` (int) to `/route/{msg_key}` (str)
### Impact
- DRY: timestamp formatting 7→1 definition, message construction 7→2 factories, line formatting 2→1 method
- Archive page visually consistent with main messages panel (single-line, monospace)
- Archive messages now clickable to open route visualization (was: only in-memory messages)
- Case-insensitive prefix matching fixes path name resolution for contacts with uppercase path hashes
- No breaking changes to BLE protocol handling, dedup, bot, or data storage
### Known Limitations
- DM filter in archive uses post-filtering (query without channel filter + filter on `channel is None`); becomes exact when `query_messages()` gets native DM support
### Parked for later
- Multi-path tracking (enrich RxLogEntry with multiple path observations)
- Events correlation improvements (only if proven data loss after `.lower()` fix)
---
## [1.7.0] - 2026-02-13 — Archive Channel Name Persistence
### Added
- ✅ **Channel name stored in archive** — Messages now persist `channel_name` alongside the numeric `channel` index in `<ADDRESS>_messages.json`, so archived messages retain their human-readable channel name even when the device is not connected
- `Message` dataclass: new field `channel_name: str` (default `""`, backward compatible)
- `SharedData.add_message()`: automatically resolves `channel_name` from the live channels list when not already set (new helper `_resolve_channel_name()`)
- `MessageArchive.add_message()`: writes `channel_name` to the JSON dict
- ✅ **Archive channel selector built from archived data** — Channel filter dropdown on `/archive` now populated via `SELECT DISTINCT channel_name` on the archive instead of the live BLE channels list
- New method `MessageArchive.get_distinct_channel_names()` returns sorted unique channel names from stored messages
- Selector shows only channels that actually have archived messages
- ✅ **Archive filter on channel name** — `MessageArchive.query_messages()` parameter changed from `channel: Optional[int]` to `channel_name: Optional[str]` (exact match on name string)
### Changed
- 🔄 `core/models.py`: Added `channel_name` field to `Message` dataclass and `from_dict()`
- 🔄 `core/shared_data.py`: `add_message()` resolves channel name; added `_resolve_channel_name()` helper
- 🔄 `services/message_archive.py`: `channel_name` persisted in JSON; `query_messages()` filters by name; new `get_distinct_channel_names()` method
- 🔄 `gui/archive_page.py`: Channel selector built from `archive.get_distinct_channel_names()`; filter state changed from `_channel_filter` (int) to `_channel_name_filter` (str); message cards show `channel_name` directly from archive
### Fixed
- 🛠 **Main page empty after startup** — After a restart the messages panel showed no messages until new live BLE traffic arrived. `SharedData.load_recent_from_archive()` now loads up to 100 recent archived messages during the cache-first startup phase, so historical messages are immediately visible
- New method `SharedData.load_recent_from_archive(limit)` — reads from `MessageArchive.query_messages()` and populates the in-memory list without re-archiving
- `BLEWorker._apply_cache()` calls `load_recent_from_archive()` at the end of cache loading
### Impact
- Archived messages now self-contained — channel name visible without live BLE connection
- Main page immediately shows historical messages after startup (no waiting for live BLE traffic)
- Backward compatible — old archive entries without `channel_name` fall back to `"Ch <idx>"`
- No breaking changes to existing functionality
---
## [1.6.0] - 2026-02-13 — Dashboard Layout Consolidation
### Changed
- 🔄 **Messages panel consolidated** — Filter checkboxes (DM + channels) and message input (text field, channel selector, Send button) are now integrated into the Messages panel, replacing the separate Filter and Input panels
- DM + channel checkboxes displayed centered in the Messages header row, between the "💬 Messages" label and the "📚 Archive" button
- Message input row (text field, channel selector, Send button) placed below the message list within the same card
- `messages_panel.py`: Constructor now accepts `put_command` callable; added `update_filters(data)`, `update_channel_options(channels)` methods and `channel_filters`, `last_channels` properties (all logic 1:1 from FilterPanel/InputPanel); `update()` signature unchanged
- 🔄 **Actions panel expanded** — BOT toggle checkbox moved from Filter panel to Actions panel, below the Refresh/Advert buttons
- `actions_panel.py`: Constructor now accepts `set_bot_enabled` callable; added `update(data)` method for BOT state sync; `_on_bot_toggle()` logic 1:1 from FilterPanel
- 🔄 **Dashboard layout simplified** — Centre column reduced from 4 panels (Map → Input → Filter → Messages) to 2 panels (Map → Messages)
- `dashboard.py`: FilterPanel and InputPanel no longer rendered; all dependencies rerouted to MessagesPanel and ActionsPanel; `_update_ui()` call-sites updated accordingly
### Removed (from layout, files retained)
- ❌ **Filter panel** no longer rendered as separate panel — `filter_panel.py` retained in codebase but not instantiated in dashboard
- ❌ **Input panel** no longer rendered as separate panel — `input_panel.py` retained in codebase but not instantiated in dashboard
### Impact
- Cleaner, more compact dashboard: 2 fewer panels in the centre column
- All functionality preserved — message filtering, send, BOT toggle, archive all work identically
- No breaking changes to BLE, services, core or other panels
---
<!-- ADDED: v1.5.0 feature + bugfix entry -->
## [1.5.0] - 2026-02-11 — Room Server Support, Dynamic Channel Discovery & Contact Management
### Added
- ✅ **Room Server panel** — Dedicated per-room-server message panel in the centre column below Messages. Each Room Server (type=3 contact) gets its own `ui.card()` with login/logout controls and message display
- Click a Room Server contact to open an add/login dialog with password field
- After login: messages are displayed in the room card; send messages directly from the room panel
- Password row + login button automatically replaced by Logout button after successful login
- Room Server author attribution via `signature` field (txt_type=2) — real message author is resolved from the 4-byte pubkey prefix, not the room server pubkey
- New panel: `gui/panels/room_server_panel.py` — per-room card management with login state tracking
- ✅ **Room Server password store** — Passwords stored outside the repository in `~/.meshcore-gui/room_passwords/<ADDRESS>.json`
- New service: `services/room_password_store.py` — JSON-backed persistent password storage per BLE device, analogous to `PinStore`
- Room panels are restored from stored passwords on app restart
- ✅ **Dynamic channel discovery** — Channels are now auto-discovered from the device at startup via `get_channel()` BLE probing, replacing the hardcoded `CHANNELS_CONFIG`
- Single-attempt probe per channel slot with early stop after 2 consecutive empty slots
- Channel name and encryption key extracted in a single pass (combined discovery + key loading)
- Configurable channel caching via `CHANNEL_CACHE_ENABLED` (default: `False` — always fresh from device)
- `MAX_CHANNELS` setting (default: 8) controls how many slots are probed
- ✅ **Individual contact deletion** — 🗑️ delete button per unpinned contact in the contacts list, with confirmation dialog
- New command: `remove_single_contact` in BLE command handler
- Pinned contacts are protected (no delete button shown)
- ✅ **"Also delete from history" option** — Checkbox in the Clean up confirmation dialog to also remove locally cached contact data
<!-- ADDED: Research document reference -->
- ✅ **Room Server protocol research** — `RoomServer_Companion_App_Onderzoek.md` documents the full companion app message flow (login, push protocol, signature mechanism, auto_message_fetching)
### Changed
- 🔄 `config.py`: Removed `CHANNELS_CONFIG` constant; added `MAX_CHANNELS` (default: 8) and `CHANNEL_CACHE_ENABLED` (default: `False`)
- 🔄 `ble/worker.py`: Replaced hardcoded channel loading with `_discover_channels()` method; added `_try_get_channel_info()` helper; `_apply_cache()` respects `CHANNEL_CACHE_ENABLED` setting; removed `_load_channel_keys()` (integrated into discovery pass)
- 🔄 `ble/commands.py`: Added `login_room`, `send_room_msg` and `remove_single_contact` command handlers
- 🔄 `gui/panels/contacts_panel.py`: Contact click now dispatches by type — type=3 (Room Server) opens room dialog, others open DM dialog; added `on_add_room` callback parameter; added 🗑️ delete button per unpinned contact
- 🔄 `gui/panels/messages_panel.py`: Room Server messages filtered from general message view via `_is_room_message()` with prefix matching; `update()` accepts `room_pubkeys` parameter
- 🔄 `gui/dashboard.py`: Added `RoomServerPanel` in centre column; `_update_ui()` passes `room_pubkeys` to Messages panel; added `_on_add_room_server` callback
- 🔄 `gui/panels/filter_panel.py`: Channel filter checkboxes now built dynamically from discovered channels (no hardcoded references)
- 🔄 `services/bot.py`: Removed stale comment referencing hardcoded channels
### Fixed
- 🛠 **Room Server messages appeared as DM** — Messages from Room Servers (txt_type=2) were displayed in the general Messages panel as direct messages. They are now filtered out and shown exclusively in the Room Server panel
- 🛠 **Historical room messages not shown after login** — Post-login fetch loop was polling `get_msg()` before room server had time to push messages over LoRa RF (10–75s per message). Removed redundant fetch loop; the library's `auto_message_fetching` handles `MESSAGES_WAITING` events correctly and event-driven
- 🛠 **Author attribution incorrect for room messages** — Room server messages showed the room server name as sender instead of the actual message author. Now correctly resolved from the `signature` field (4-byte pubkey prefix) via contact lookup
### Impact
- Room Servers are now first-class citizens in the GUI with dedicated panels
- Channel configuration no longer requires manual editing of `config.py`
- Contact list management is more granular with per-contact deletion
- No breaking changes to existing functionality (messages, DM, map, archive, bot, etc.)
---
## [1.4.0] - 2026-02-09 — SDK Event Race Condition Fix
### Fixed
- 🛠 **BLE startup delay of ~2 minutes eliminated** — The meshcore Python SDK (`commands/base.py`) dispatched device response events before `wait_for_events()` registered its subscription. On busy networks with frequent `RX_LOG_DATA` events, this caused `send_device_query()` and `get_channel()` to fail repeatedly with `no_event_received`, wasting 110+ seconds in timeouts
### Changed
- 📄 `meshcore` SDK `commands/base.py`: Rewritten `send()` method to subscribe to expected events **before** transmitting the BLE command (subscribe-before-send pattern), matching the approach used by the companion apps (meshcore.js, iOS, Android). Submitted upstream as [meshcore_py PR #52](https://github.com/meshcore-dev/meshcore_py/pull/52)
### Impact
- Startup time reduced from ~2+ minutes to ~10 seconds on busy networks
- All BLE commands (`send_device_query`, `get_channel`, `get_bat`, `send_appstart`, etc.) now succeed on first attempt instead of requiring multiple retries
- No changes to meshcore_gui code required — the fix is entirely in the meshcore SDK
### Temporary Installation
Until the fix is merged upstream, install the patched meshcore SDK:
```bash
pip install --force-reinstall git+https://github.com/PE1HVH/meshcore_py.git@fix/event-race-condition
```
---
<!-- ADDED: v1.3.2 bugfix entry -->
## [1.3.2] - 2026-02-09 — Bugfix: Bot Device Name Restoration After Restart
### Fixed
- 🛠 **Bot device name not properly restored after restart/crash** — After a restart or crash with bot mode previously active, the original device name was incorrectly stored as the bot name (e.g. `NL-OV-ZWL-STDSHGN-WKC Bot`) instead of the real device name (e.g. `PE1HVH T1000e`). The original device name is now correctly preserved and restored when bot mode is disabled
### Changed
- 🔄 `commands.py`: `set_bot_name` handler now verifies that the stored original name is not already the bot name before saving
- 🔄 `shared_data.py`: `original_device_name` is only written when it differs from `BOT_DEVICE_NAME` to prevent overwriting with the bot name on restart
---
<!-- ADDED: v1.3.1 bugfix entry -->
## [1.3.1] - 2026-02-09 — Bugfix: Auto-add AttributeError
### Fixed
- 🛠 **Auto-add error on first toggle** — Setting auto-add for the first time raised `AttributeError: 'telemetry_mode_base'`. The `set_manual_add_contacts()` SDK call now handles missing `telemetry_mode_base` attribute gracefully
### Changed
- 🔄 `commands.py`: `set_auto_add` handler wraps `set_manual_add_contacts()` call with attribute check and error handling for missing `telemetry_mode_base`
---
<!-- ADDED: New v1.3.0 entry at top -->
## [1.3.0] - 2026-02-08 — Bot Device Name Management
### Added
- ✅ **Bot device name switching** — When the BOT checkbox is enabled, the device name is automatically changed to a configurable bot name; when disabled, the original name is restored
- Original device name is saved before renaming so it can be restored on BOT disable
- Device name written to device via BLE `set_name()` SDK call
- Graceful handling of BLE failures during name change
- ✅ **`BOT_DEVICE_NAME` constant** in `config.py` — Configurable fixed device name used when bot mode is active (default: `;NL-OV-ZWL-STDSHGN-WKC Bot`)
### Changed
- 🔄 `config.py`: Added `BOT_DEVICE_NAME` constant for bot mode device name
- 🔄 `bot.py`: Removed hardcoded `BOT_NAME` prefix ("Zwolle Bot") from bot reply messages — bot replies no longer include a name prefix
- 🔄 `filter_panel.py`: BOT checkbox toggle now triggers device name save/rename via command queue
- 🔄 `commands.py`: Added `set_bot_name` and `restore_name` command handlers for device name switching
- 🔄 `shared_data.py`: Added `original_device_name` field for storing the pre-bot device name
### Removed
- ❌ `BOT_NAME` constant from `bot.py` — bot reply prefix removed; replies no longer prepend a bot display name
---
## [1.2.0] - 2026-02-08 — Contact Maintenance Feature
### Added
- ✅ **Pin/Unpin contacts** (Iteration A) — Toggle to pin individual contacts, protecting them from bulk deletion
- Persistent pin state stored in `~/.meshcore-gui/cache/<ADDRESS>_pins.json`
- Pinned contacts visually marked with yellow background
- Pinned contacts sorted to top of contact list
- Pin state survives app restart
- New service: `services/pin_store.py` — JSON-backed persistent pin storage
- ✅ **Bulk delete unpinned contacts** (Iteration B) — Remove all unpinned contacts from device in one action
- "🧹 Clean up" button in contacts panel with confirmation dialog
- Shows count of contacts to be removed vs. pinned contacts kept
- Progress status updates during removal
- Automatic device resync after completion
- New service: `services/contact_cleaner.py` — ContactCleanerService with purge statistics
- ✅ **Auto-add contacts toggle** (Iteration C) — Control whether device automatically adds new contacts from mesh adverts
- "📥 Auto-add" checkbox in contacts panel (next to Clean up button)
- Syncs with device via `set_manual_add_contacts()` SDK call
- Inverted logic handled internally (UI "Auto-add ON" = `set_manual_add_contacts(false)`)
- Optimistic update with automatic rollback on BLE failure
- State synchronized from device on each GUI update cycle
### Changed
- 🔄 `contacts_panel.py`: Added pin checkbox per contact, purge button, auto-add toggle, DM dialog (all existing functionality preserved)
- 🔄 `commands.py`: Added `purge_unpinned` and `set_auto_add` command handlers
- 🔄 `shared_data.py`: Added `auto_add_enabled` field with thread-safe getter/setter
- 🔄 `protocols.py`: Added `set_auto_add_enabled` and `is_auto_add_enabled` to Writer and Reader protocols
- 🔄 `dashboard.py`: Passes `PinStore` and `set_auto_add_enabled` callback to ContactsPanel
- 🔄 **UI language**: All Dutch strings in `contacts_panel.py` and `commands.py` translated to English
---
### Fixed
- 🛠 **Route table names and IDs not displayed** — Route tables in both current messages (RoutePage) and archive messages (ArchivePage) now correctly show node names and public key IDs for sender, repeaters and receiver
### Changed
- 🔄 **CHANGELOG.md**: Corrected version numbering to semantic versioning, fixed inaccurate references (archive button location, filter state persistence)
- 🔄 **README.md**: Added Message Archive feature, updated project structure, configuration table and architecture diagram
- 🔄 **MeshCore_GUI_Design.docx**: Added ArchivePage, MessageArchive, Models components; updated project structure, protocols, configuration and version history
---
## [1.1.0] - 2026-02-07 — Archive Viewer Feature
### Added
- ✅ **Archive Viewer Page** (`/archive`) — Full-featured message archive browser
- Pagination (50 messages per page, configurable)
- Channel filter dropdown (All + configured channels)
- Time range filter (24h, 7d, 30d, 90d, All time)
- Text search (case-insensitive)
- Filter state stored in instance variables (reset on page reload)
- Message cards with same styling as main messages panel
- Clickable messages for route visualization (where available)
- **💬 Reply functionality** — Expandable reply panel per message
- **🗺️ Inline route table** — Expandable route display per archive message with sender, repeaters and receiver (names, IDs, node types)
- *(Note: Reply panels and inline route tables removed in v1.8.0, replaced by click-to-route navigation via message hash)*
<!-- CHANGED: "Filter state persistence (app.storage.user)" replaced with "Filter state stored in
instance variables" — the code (archive_page.py:36-40) uses self._current_page etc.,
not app.storage.user. The comment in the code is misleading. -->
<!-- ADDED: "Inline route table" entry — _render_archive_route() in archive_page.py:333-407
was not documented. -->
- ✅ **MessageArchive.query_messages()** method
- Filter by: time range, channel, text search, sender
- Pagination support (limit, offset)
- Returns tuple: (messages, total_count)
- Sorting: Newest first
- ✅ **UI Integration**
- "📚 Archive" button in Messages panel header (opens in new tab)
- Back to Dashboard button in archive page
<!-- CHANGED: "📚 View Archive button in Actions panel" corrected — the button is in
MessagesPanel (messages_panel.py:25), not in ActionsPanel (actions_panel.py).
ActionsPanel only contains Refresh and Advert buttons. -->
- ✅ **Reply Panel**
- Expandable reply per message (💬 Reply button)
- Pre-filled with @sender mention
- Channel selector
- Send button with success notification
- Auto-close expansion after send
### Changed
- 🔄 `SharedData.get_snapshot()`: Now includes `'archive'` field
- 🔄 `MessagesPanel`: Added archive button in header row
- 🔄 Both entry points (`__main__.py` and `meshcore_gui.py`): Register `/archive` route
<!-- CHANGED: "ActionsPanel: Added archive button" corrected to "MessagesPanel" -->
### Performance
- Query: ~10ms for 10k messages with filters
- Memory: ~10KB per page (50 messages)
- No impact on main UI (separate page)
### Known Limitations
- ~~Route visualization only works for messages in recent buffer (last 100)~~ — Fixed in v1.8.0: archive messages now support click-to-route via `get_message_by_hash()` fallback
- Text search is linear scan (no indexing yet)
- Sender filter exists in API but not in UI yet
---
## [1.0.3] - 2026-02-07 — Critical Bugfix: Archive Overwrite Prevention
### Fixed
- 🛠 **CRITICAL**: Fixed bug where archive was overwritten instead of appended on restart
- 🛠 Archive now preserves existing data when read errors occur
- 🛠 Buffer is retained for retry if existing archive cannot be read
### Changed
- 🔄 `_flush_messages()`: Early return on read error instead of overwriting
- 🔄 `_flush_rxlog()`: Early return on read error instead of overwriting
- 🔄 Better error messages for version mismatch and JSON decode errors
### Details
**Problem:** If the existing archive file had a JSON parse error or version mismatch,
the flush operation would proceed with `existing_messages = []`, effectively
overwriting all historical data with only the new buffered messages.
**Solution:** The flush methods now:
1. Try to read existing archive first
2. If read fails (JSON error, version mismatch, IO error), abort the flush
3. Keep buffer intact for next retry
4. Only clear buffer after successful write
**Impact:** No data loss on restart or when archive files have issues.
### Testing
- ✅ Added `test_append_on_restart_not_overwrite()` integration test
- ✅ Verifies data is appended across multiple sessions
- ✅ All existing tests still pass
---
## [1.0.2] - 2026-02-07 — RxLog message_hash Enhancement
### Added
- ✅ `message_hash` field added to `RxLogEntry` model
- ✅ RxLog entries now include message_hash for correlation with messages
- ✅ Archive JSON includes message_hash in rxlog entries
### Changed
- 🔄 `events.py`: Restructured `on_rx_log()` to extract message_hash before creating RxLogEntry
- 🔄 `message_archive.py`: Updated rxlog archiving to include message_hash field
- 🔄 Tests updated to verify message_hash persistence
### Benefits
- **Correlation**: Link RX log entries to their corresponding messages
- **Analysis**: Track which packets resulted in messages
- **Debugging**: Better troubleshooting of packet processing
---
## [1.0.1] - 2026-02-07 — Entry Point Fix
### Fixed
- ✅ `meshcore_gui.py` (root entry point) now passes ble_address to SharedData
- ✅ Archive works correctly regardless of how application is started
### Changed
- 🔄 Both entry points (`meshcore_gui.py` and `meshcore_gui/__main__.py`) updated
---
## [1.0.0] - 2026-02-07 — Message & Metadata Persistence
### Added
- ✅ MessageArchive class for persistent storage
- ✅ Configurable retention periods (MESSAGE_RETENTION_DAYS, RXLOG_RETENTION_DAYS, CONTACT_RETENTION_DAYS)
- ✅ Automatic daily cleanup of old data
- ✅ Batch writes for performance
- ✅ Thread-safe with separate locks
- ✅ Atomic file writes
- ✅ Contact retention in DeviceCache
- ✅ Archive statistics API
- ✅ Comprehensive tests (20+ unit, 8+ integration)
- ✅ Full documentation
### Storage Locations
- `~/.meshcore-gui/archive/<ADDRESS>_messages.json`
- `~/.meshcore-gui/archive/<ADDRESS>_rxlog.json`
### Requirements Completed
- R1: All incoming messages persistent ✅
- R2: All incoming RxLog entries persistent ✅
- R3: Configurable retention ✅
- R4: Automatic cleanup ✅
- R5: Backward compatibility ✅
- R6: Contact retention ✅
- R7: Archive stats API ✅
- Fix3: Leaflet asset injection is now per page render instead of process-global, and browser bootstrap now retries until the host element, Leaflet runtime, and MeshCore panel runtime are all available. This fixes blank map containers caused by missing or late-loaded JS/CSS assets.
- Fix5: Removed per-snapshot map invalidate calls, stopped forcing a default dark theme during map bootstrap, and added client-side interaction/resize guards so zooming stays responsive and the theme no longer jumps back during status-loop updates.
## 2026-03-09 map hotfix v2
- regular map snapshots no longer carry theme state
- explicit theme changes are now handled only via the dedicated theme channel
- initial map render now sends an ensure_map command plus an immediate theme sync
- added no-op ensure_map handling in the Leaflet runtime to avoid accidental fallback behaviour
## [1.13.0] - 2026-03-09
### Added
- Leaflet marker clustering using Leaflet.markercluster for contact nodes.
- Browser-side cluster rendering with the device marker kept outside the cluster layer.
- Cluster performance tuning with `chunkedLoading: true`.
- Spiderfy support at max zoom for overlapping markers.
### Fixed
- Wrong asset load order causing `L is not defined` in MarkerClusterGroup.
- Cluster initialization failure caused by missing `maxZoom` on map startup.
- Retry cascade causing `Map container is already initialized`.
### Changed
- Map lifecycle is browser-owned: NiceGUI hosts the container, Leaflet owns map state.
- Contact markers are updated incrementally in the existing cluster layer.

21
LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2025 PE1HVH
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

231
MAP_ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,231 @@
# Map Architecture — MeshCore GUI
## Overview
The MeshCore GUI map subsystem is implemented as a **browser-managed Leaflet runtime** embedded inside a NiceGUI container.
The key design decision is that the **map lifecycle is owned by the browser**, not by the Python UI update loop.
NiceGUI acts only as a container and data provider.
This architecture prevents map resets, marker flicker, and viewport jumps during the 500 ms dashboard refresh cycle.
---
# Architecture
```
NiceGUI Dashboard
│
│ snapshot (500 ms)
▼
MapPanel (Python)
│
│ JSON payload
▼
Leaflet Runtime (Browser)
│
├─ Map instance (persistent)
├─ Marker registry
├─ Contact cluster layer
├─ Theme state
└─ Viewport state
```
---
# Component Responsibilities
## MapPanel (Python)
Location:
```
meshcore_gui/gui/panels/map_panel.py
```
Responsibilities:
* provides the map container
* injects the Leaflet runtime assets
* sends compact map snapshots
* handles UI actions:
* theme toggle
* center on device
MapPanel **does NOT control the Leaflet map directly**.
It only sends data.
---
## MapSnapshotService
Location:
```
meshcore_gui/services/map_snapshot_service.py
```
Responsibilities:
* converts device/contact data into a compact JSON snapshot
* ensures stable node identifiers
* prepares payloads for the browser runtime
Example snapshot structure:
```json
{
"device": {...},
"contacts": [...],
"force_center": false
}
```
Snapshots are emitted every **500 ms** by the dashboard update loop.
---
## Leaflet Runtime
Location:
```
meshcore_gui/static/leaflet_map_panel.js
```
Responsibilities:
* initialize the Leaflet map once
* maintain persistent map instance
* manage marker registry
* maintain a persistent contact cluster layer
* keep the own-device marker outside clustering
* apply snapshots incrementally
* manage map theme and viewport state
Key design rules:
```
map is created once
markers updated incrementally
snapshots never recreate the map
clustering is attached only after maxZoom is known
```
---
# Update Flow
```
SharedData
│
▼
Dashboard update loop (500 ms)
│
▼
MapSnapshotService
│
▼
MapPanel
│
▼
Leaflet Runtime
```
Snapshots are **coalesced** so the browser applies only the newest payload.
---
# Theme Handling
Theme changes are handled via a **dedicated theme channel**.
Snapshots do **not** carry theme information.
Reason:
Embedding theme state in snapshots caused race conditions where queued snapshots overwrote explicit user selections.
Theme state is managed in the browser runtime and restored on reconnect.
---
# Marker Model
Markers are keyed by **stable node id**.
```
device marker (standalone)
contact markers (clustered)
```
Updates are applied incrementally:
```
add marker
update marker
remove marker
```
This prevents marker flicker during the refresh loop.
---
# Important Constraints
Developers must **not**:
* recreate the Leaflet map inside the dashboard refresh loop
* call `L.map(...)` from snapshot handlers, retry loops or timer callbacks
* embed theme state in snapshots
* call Leaflet APIs directly from Python
* force viewport resets during normal snapshot updates
* place the device marker inside the contact cluster layer
Violating these rules will reintroduce:
* disappearing maps
* marker flicker
* viewport resets
* theme resets
---
# Reconnect Behaviour
When the NiceGUI connection temporarily drops:
1. the Leaflet runtime persists in the browser
2. the map instance remains intact
3. theme and viewport state are restored
4. snapshot updates resume once the connection returns
---
# Future Extensions
Possible improvements without breaking the architecture:
* heatmap layers
* route overlays
* tile provider switching
* richer cluster icons or spiderfy tuning
All extensions must remain **browser-managed**.
---
# Summary
The MeshCore map subsystem follows a strict separation:
```
Python → data
Browser → map lifecycle
```
This prevents UI refresh cycles from interfering with map state and ensures smooth rendering even with frequent dashboard updates.

636
OBSERVER.md Normal file
View File

@@ -0,0 +1,636 @@
# MeshCore Observer — Read-Only Archive Monitor
### Multi-source aggregation dashboard with optional MQTT uplink to LetsMesh.
![Status](https://img.shields.io/badge/Status-Production-green.svg)
![Python](https://img.shields.io/badge/Python-3.10+-blue.svg)
![License](https://img.shields.io/badge/License-MIT-green.svg)
![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows-orange.svg)
![Device](https://img.shields.io/badge/Device-None%20Required-blueviolet.svg)
![MQTT](https://img.shields.io/badge/MQTT-LetsMesh%20Uplink-ff6600.svg)
A standalone daemon that reads JSON archive files produced by `meshcore_gui` and `meshcore_bridge`, aggregates them from all sources, and presents a unified live dashboard. It never connects to a device and never writes to the archive — it only watches and displays.
## Table of Contents
- [1. Why This Project Exists](#1-why-this-project-exists)
- [2. Features](#2-features)
- [3. Requirements](#3-requirements)
- [4. Installation](#4-installation)
- [4.1. Base Installation (dashboard only)](#41-base-installation-dashboard-only)
- [4.2. MQTT Uplink Dependencies](#42-mqtt-uplink-dependencies)
- [4.3. Verify Installation](#43-verify-installation)
- [5. Quick Start](#5-quick-start)
- [6. Command-Line Options](#6-command-line-options)
- [7. Configuration](#7-configuration)
- [7.1. Observer Settings](#71-observer-settings)
- [7.2. Port Allocation](#72-port-allocation)
- [8. MQTT Uplink to LetsMesh](#8-mqtt-uplink-to-letsmesh)
- [8.1. Prerequisites](#81-prerequisites)
- [8.2. Automatic Key Setup (same machine)](#82-automatic-key-setup-same-machine)
- [8.3. Manual Key Setup (remote GUI)](#83-manual-key-setup-remote-gui)
- [8.4. MQTT Configuration](#84-mqtt-configuration)
- [8.5. Test and Go Live](#85-test-and-go-live)
- [8.6. Privacy Controls](#86-privacy-controls)
- [8.7. Multiple Brokers](#87-multiple-brokers)
- [8.8. How Authentication Works](#88-how-authentication-works)
- [8.9. MQTT Topics](#89-mqtt-topics)
- [9. systemd Installation](#9-systemd-installation)
- [10. How It Works](#10-how-it-works)
- [11. Dashboard Panels](#11-dashboard-panels)
- [12. Running Alongside Other Daemons](#12-running-alongside-other-daemons)
- [13. Troubleshooting](#13-troubleshooting)
- [13.1. Dashboard](#131-dashboard)
- [13.2. MQTT](#132-mqtt)
- [14. Version History](#14-version-history)
- [15. License](#15-license)
- [16. Author](#16-author)
---
## 1. Why This Project Exists
When running multiple MeshCore devices — a GUI instance on 869 MHz, a bridge between 869 and 868 MHz, perhaps another GUI on a different frequency — each writes its own archive files. There is no single place to see all traffic at once.
The Observer solves this by watching all archive files from all sources, merging them into one live dashboard. It requires no device, no serial port and no meshcore library. Just point it at the archive directory and it works.
With MQTT uplink enabled, the Observer can also contribute your node's received packets to the global [LetsMesh analyzer](https://analyzer.letsmesh.net), helping map the mesh network's reach and signal quality.
```
[meshcore_gui] ──writes──► ~/.meshcore-gui/archive/*.json ◄──reads── [Observer]
[meshcore_bridge] ──writes──► │
┌────┴────┐
▼ ▼
NiceGUI MQTT Uplink
Dashboard (optional)
:9093 │
▼
analyzer.letsmesh.net
```
## 2. Features
- **Multi-source aggregation** — Automatically detects and merges archives from all GUI and Bridge instances
- **Live message feed** — Channel messages from all sources, sorted by timestamp, filterable by source and channel
- **Live RX log** — Packet log with SNR, RSSI, type, hops, and decoded path
- **Source overview** — Table of all detected archive files with entry counts
- **Statistics** — Uptime, totals, per-source breakdown
- **MQTT uplink to LetsMesh** — Publishes RX log packets to [analyzer.letsmesh.net](https://analyzer.letsmesh.net) via MQTT over WebSocket+TLS with Ed25519 JWT authentication. Privacy-configurable: choose which packet types to share
- **DOMCA theme** — Dark and light mode, consistent with meshcore_gui and meshcore_bridge
- **Zero device access** — No serial port, no BLE, no meshcore library required
## 3. Requirements
**Dashboard (always required):**
- Python 3.10+
- `nicegui`
- `pyyaml`
**MQTT uplink (optional):**
- `paho-mqtt` >= 2.0
- Node.js 18+ with `@michaelhart/meshcore-decoder` — **required** for signing JWT tokens with the orlp/ed25519 algorithm used by LetsMesh
**Note on PyNaCl:** PyNaCl can be used as a fallback for token signing, but **only** with legacy 64-char seed keys. The current `device_identity.json` format uses 128-char orlp/ed25519 expanded keys which are **not compatible** with PyNaCl. For new installations, use Node.js + meshcore-decoder.
Without the MQTT packages the Observer runs fine — only the LetsMesh uplink is disabled.
---
## 4. Installation
### 4.1. Base Installation (dashboard only)
```bash
cd ~/meshcore-gui
source venv/bin/activate
pip install nicegui pyyaml
```
Verify:
```bash
python meshcore_observer.py --help
```
### 4.2. MQTT Uplink Dependencies
**Step 1 — paho-mqtt:**
```bash
pip install paho-mqtt
```
**Step 2 — Node.js (if not already installed):**
```bash
# Check if Node.js is available
node --version
# If not installed (Debian/Ubuntu/Raspberry Pi OS):
sudo apt update && sudo apt install -y nodejs npm
```
**Step 3 — meshcore-decoder:**
```bash
sudo npm install -g @michaelhart/meshcore-decoder
```
**Step 4 — Verify Node.js can find meshcore-decoder:**
```bash
node -e "require('@michaelhart/meshcore-decoder').createAuthToken; console.log('OK')"
```
If this prints `OK`, you're good. If it says `Cannot find module`, Node.js can't find the global install. Fix with:
```bash
# Check where npm installs global packages:
npm root -g
# If it's /usr/local/lib/node_modules, set NODE_PATH:
export NODE_PATH=/usr/local/lib/node_modules
node -e "require('@michaelhart/meshcore-decoder').createAuthToken; console.log('OK')"
```
If you need `NODE_PATH`, add it to your shell profile or systemd service (see [section 9](#9-systemd-installation)).
> **Note:** The Observer auto-detects `NODE_PATH` from `/usr/lib/node_modules`, `/usr/local/lib/node_modules`, and `~/.npm-global/lib/node_modules`. You only need to set it manually if npm uses a non-standard location.
### 4.3. Verify Installation
```bash
cd ~/meshcore-gui
source venv/bin/activate
# Dashboard only:
python -c "import nicegui, yaml; print('Dashboard deps OK')"
# MQTT uplink:
python -c "import paho.mqtt; print('paho-mqtt OK')"
node -e "require('@michaelhart/meshcore-decoder'); console.log('meshcore-decoder OK')"
```
---
## 5. Quick Start
```bash
cd ~/meshcore-gui
source venv/bin/activate
# Run with defaults (dashboard on port 9093, reads ~/.meshcore-gui/archive/)
python meshcore_observer.py
```
The dashboard opens at **http://localhost:9093**. The Observer immediately starts scanning for archive JSON files. If meshcore_gui or meshcore_bridge is running and writing archives, they will appear within seconds.
---
## 6. Command-Line Options
| Flag | Description | Default |
|------|-------------|---------|
| `--config=PATH` | Path to YAML configuration file | `./observer_config.yaml` |
| `--port=PORT` | Override dashboard port | `9093` |
| `--debug-on` | Enable verbose debug logging | Off |
| `--mqtt-dry-run` | Log MQTT payloads without publishing (also enables MQTT) | Off |
| `--help` | Show usage information | — |
Examples:
```bash
python meshcore_observer.py
python meshcore_observer.py --config=/etc/meshcore/observer_config.yaml
python meshcore_observer.py --port=9094 --debug-on
python meshcore_observer.py --mqtt-dry-run --debug-on
```
---
## 7. Configuration
All settings are optional. The Observer works with sensible defaults and no config file.
### 7.1. Observer Settings
**observer_config.yaml:**
```yaml
observer:
archive_dir: "~/.meshcore-gui/archive"
poll_interval_s: 2.0
max_messages_display: 100
max_rxlog_display: 50
gui:
port: 9093
title: "MeshCore Observer"
```
### 7.2. Port Allocation
| Daemon | Default Port |
|---|---|
| meshcore_gui | 8081 / 9090 |
| meshcore_bridge | 9092 |
| **meshcore_observer** | **9093** |
---
## 8. MQTT Uplink to LetsMesh
The Observer can publish RX log packets to the LetsMesh network analyzer at [analyzer.letsmesh.net](https://analyzer.letsmesh.net). MQTT is **disabled by default**.
```
[Observer]
│
│ RX log entries from archive
│
├── filter by packet type (privacy)
├── transform to LetsMesh JSON format
├── sign JWT with Ed25519 private key
│
└──► mqtt-eu-v1.letsmesh.net:443 (WebSocket+TLS)
│
▼
analyzer.letsmesh.net
```
### 8.1. Prerequisites
Before enabling MQTT, ensure:
1. **MQTT dependencies are installed** (see [section 4.2](#42-mqtt-uplink-dependencies))
2. **meshcore_gui has the fixed `device_identity.py`** — version ≥ 1.2.0 that writes the full 128-char orlp/ed25519 private key. Without this fix, `device_identity.json` contains only a 64-char key that is **not the Ed25519 seed** but a clamped scalar, which causes "Not authorized" errors.
Verify your identity file:
```bash
cat ~/.meshcore-gui/device_identity.json | python -c "
import json, sys
d = json.load(sys.stdin)
pub = d.get('public_key', '')
priv = d.get('private_key', '')
print(f'Public key: {len(pub)} chars — {pub[:16]}...')
print(f'Private key: {len(priv)} chars')
if len(priv) == 128:
print('✅ Correct format (128-char orlp expanded key)')
elif len(priv) == 64:
print('❌ Legacy format (64 chars) — update device_identity.py and restart meshcore_gui')
else:
print(f'❌ Unexpected length: {len(priv)}')
"
```
3. **Your IATA airport code** — 3-letter code for your nearest airport (e.g. `AMS`, `JFK`, `LHR`)
### 8.2. Automatic Key Setup (same machine)
When meshcore_gui and the Observer run on the **same machine**, keys are shared automatically via `~/.meshcore-gui/device_identity.json`. No manual key configuration needed — just add the MQTT section to your config and the Observer reads the keys at startup.
### 8.3. Manual Key Setup (remote GUI)
When meshcore_gui runs on a **different machine**, you need to transfer the keys.
**Option A — Copy the identity file (recommended):**
Copy `~/.meshcore-gui/device_identity.json` from the GUI machine to the Observer machine, then:
```bash
cd ~/meshcore-gui
source venv/bin/activate
python -m meshcore_observer.setup_mqtt_keys --identity /path/to/device_identity.json
```
This saves the private key to `~/.meshcore-observer-key` (chmod 600) and writes the public key to `observer_config.yaml`.
**Option B — Interactive setup:**
```bash
cd ~/meshcore-gui
source venv/bin/activate
python -m meshcore_observer.setup_mqtt_keys
```
You will need:
- The **public key** (64 hex chars) — visible in meshcore_gui device info
- The **private key** (128 hex chars) — from `device_identity.json` on the GUI machine
**Option C — Environment variable:**
```bash
export MESHCORE_PRIVATE_KEY="<128-char hex private key>"
export MESHCORE_PUBLIC_KEY="<64-char hex public key>"
```
### 8.4. MQTT Configuration
Edit `observer_config.yaml`:
```yaml
mqtt:
enabled: true
iata: "AMS" # Your nearest airport code
device_name: "PE1HVH Observer" # Name shown on analyzer.letsmesh.net
# Keys — only needed if meshcore_gui runs on a different machine.
# On the same machine, keys are read from device_identity.json automatically.
# public_key: "D955E72C..."
# private_key_file: "~/.meshcore-observer-key"
brokers:
- name: "letsmesh-eu"
server: "mqtt-eu-v1.letsmesh.net"
port: 443
transport: "websockets"
tls: true
enabled: true
upload_packet_types: [] # [] = all types
status_interval_s: 300
reconnect_delay_s: 10
```
### 8.5. Test and Go Live
**Step 1 — Dry run** (log payloads without publishing):
```bash
python meshcore_observer.py --mqtt-dry-run --debug-on
```
Check the output for:
- `"Loaded device identity from ..."` — keys found
- `"Using Node.js meshcore-decoder for MQTT auth tokens"` — signing works
- `"[DRY RUN] Would connect to ..."` — broker config OK
**Step 2 — Live:**
```bash
python meshcore_observer.py
```
The dashboard MQTT panel shows connection status, packet counters, and any errors. Within minutes your packets should appear on [analyzer.letsmesh.net](https://analyzer.letsmesh.net).
### 8.6. Privacy Controls
Control which packet types are shared:
```yaml
# Only advertisements (network discovery, no message content)
upload_packet_types: [4]
# Advertisements and group text metadata
upload_packet_types: [4, 5]
# Everything (default)
upload_packet_types: []
```
Packet types: 0=REQ, 1=RESPONSE, 2=TXT_MSG, 3=ACK, 4=ADVERT, 5=GRP_TXT, 6=GRP_DATA, 7=ANON_REQ, 8=PATH, 9=TRACE.
The raw packet payload (hex bytes) is always included for shared types. If you do not want to share message content, use `[4]` (ADVERT only).
### 8.7. Multiple Brokers
Publish to EU and US brokers simultaneously for redundancy:
```yaml
brokers:
- name: "letsmesh-eu"
server: "mqtt-eu-v1.letsmesh.net"
port: 443
transport: "websockets"
tls: true
enabled: true
- name: "letsmesh-us"
server: "mqtt-us-v1.letsmesh.net"
port: 443
transport: "websockets"
tls: true
enabled: true
```
### 8.8. How Authentication Works
LetsMesh uses Ed25519 JWT tokens — no registration required. Your device key pair *is* your identity:
1. meshcore_gui exports the device's Ed25519 keypair to `device_identity.json`
2. The Observer generates a JWT signed with the 128-char orlp/ed25519 private key via Node.js meshcore-decoder
3. MQTT username: `v1_{PUBLIC_KEY}` (uppercase, 64 hex chars)
4. MQTT password: the signed JWT token
5. The broker verifies the signature against the public key from the username
6. Topics are scoped to `meshcore/{IATA}/{PUBLIC_KEY}/`
7. Tokens auto-refresh before expiry (default 1 hour)
**Key format:** The private key in `device_identity.json` is 128 hex chars (64 bytes) in orlp/ed25519 expanded format: `[clamped_scalar(32)][nonce_prefix(32)]`. This is **not** a seed+pubkey concatenation — it is the output of `SHA-512(seed)` with clamping. The public key comes from `send_appstart()` and is stored separately.
### 8.9. MQTT Topics
| Topic | Content | QoS | Retained |
|---|---|---|---|
| `meshcore/{IATA}/{KEY}/packets` | RX log entries (JSON) | 0 | No |
| `meshcore/{IATA}/{KEY}/status` | Online/offline status | 1 | Yes |
The status topic uses MQTT Last Will and Testament (LWT): if the Observer disconnects unexpectedly, the broker automatically publishes an offline status.
---
## 9. systemd Installation
For running the Observer as a background service on Linux.
**Install:**
```bash
cd ~/meshcore-gui
source venv/bin/activate
bash install_observer.sh
```
The installer detects the venv and current user automatically, creates a systemd service, and offers to start it immediately.
**If you need a custom NODE_PATH** (see [section 4.2](#42-mqtt-uplink-dependencies)), edit the service file after installation:
```bash
sudo systemctl edit meshcore-observer
```
Add:
```ini
[Service]
Environment="NODE_PATH=/usr/local/lib/node_modules"
```
Then reload:
```bash
sudo systemctl daemon-reload
sudo systemctl restart meshcore-observer
```
**Service commands:**
| Command | Description |
|---------|-------------|
| `sudo systemctl start meshcore-observer` | Start the service |
| `sudo systemctl stop meshcore-observer` | Stop the service |
| `sudo systemctl restart meshcore-observer` | Restart after config change |
| `sudo systemctl status meshcore-observer` | Check status |
| `sudo journalctl -u meshcore-observer -f` | Follow live logs |
**Uninstall:**
```bash
cd ~/meshcore-gui
bash install_observer.sh --uninstall
```
---
## 10. How It Works
The Observer uses a polling-based file watcher (`ArchiveWatcher`) that:
1. Scans the archive directory for `*_messages.json` and `*_rxlog.json` files
2. Checks each file's `mtime` (modification timestamp)
3. If unchanged since last poll → skip (no disk I/O)
4. If changed → read, parse, extract only new entries (delta detection)
5. Feeds new entries to the dashboard panels (and optionally to MQTT uplink)
This is efficient and safe:
- **No file locking conflicts** — meshcore_gui uses atomic writes (temp file + rename)
- **No race conditions** — Observer only reads completed files
- **No crash on corruption** — Malformed JSON is logged and skipped
- **No crash on missing files** — Vanished files are removed from tracking
---
## 11. Dashboard Panels
### Sources
Table of all detected archive files with source name, message count, and RX log count.
### Messages
Aggregated message feed from all sources. Newest messages on top. Filterable by source and channel.
### RX Log
Aggregated packet log from all sources. Columns: Time, Source, SNR, RSSI, Type, Hops, Path, Hash.
### Statistics
Observer uptime, total messages and RX log entries seen, number of active sources, per-source breakdown.
### MQTT Uplink
Connection status per broker (green/red dot), total packets published, filtered count, skipped count, last publish timestamp, and any errors.
---
## 12. Running Alongside Other Daemons
The Observer is designed to coexist with meshcore_gui and meshcore_bridge:
```
┌──────────────────┐ ┌──────────────────┐
│ meshcore_gui │ │ meshcore_bridge │
│ :8081 │ │ :9092 │
│ writes archive │ │ writes archive │
└────────┬─────────┘ └────────┬──────────┘
│ │
▼ ▼
~/.meshcore-gui/archive/
│
▼
┌──────────────────┐
│ meshcore_observer │
│ :9093 │──────► mqtt-eu-v1.letsmesh.net
│ reads archive │ (optional MQTT uplink)
└──────────────────┘
```
All three can run simultaneously. The Observer only reads atomically-written files and never interferes with the other daemons.
---
## 13. Troubleshooting
### 13.1. Dashboard
**"Waiting for archive files..."**
- Verify meshcore_gui or meshcore_bridge is running and has received at least one message
- Check: `ls ~/.meshcore-gui/archive/`
- If using a custom path, verify `archive_dir` in your config
**No messages despite archive files existing**
- Check file permissions
- Run with `--debug-on`
- Verify archive files have `"version": 1` in their JSON
**Port conflict**
- Change with `--port=9094` or in `observer_config.yaml`
### 13.2. MQTT
**"Not authorized" (rc=5)**
This is almost always a key format issue. Check in order:
1. **Is the private key 128 chars?**
```bash
python -c "
import json
d = json.load(open('$HOME/.meshcore-gui/device_identity.json'))
print(f\"private_key length: {len(d.get('private_key',''))}\")"
```
If 64: you need the fixed `device_identity.py` in meshcore_gui. Deploy it and restart meshcore_gui.
2. **Does the public key match the GUI?**
The `public_key` in `device_identity.json` must match what meshcore_gui shows under device info (uppercase hex). If it doesn't, the identity file was written by a buggy version — deploy the fix and restart.
3. **Is meshcore-decoder available?**
```bash
node -e "require('@michaelhart/meshcore-decoder').createAuthToken; console.log('OK')"
```
If this fails, see [section 4.2](#42-mqtt-uplink-dependencies).
4. **Run with debug:**
```bash
python meshcore_observer.py --mqtt-dry-run --debug-on
```
Look for `"Node.js token generation failed"` or `"PyNaCl"` fallback messages.
**"Connecting..." but never connects**
- Check firewall allows outbound connections to port 443
- Try the US broker: change `server` to `mqtt-us-v1.letsmesh.net`
- Check DNS resolution: `nslookup mqtt-eu-v1.letsmesh.net`
**Packets not appearing on analyzer.letsmesh.net**
- Use `--mqtt-dry-run` to verify payload format
- Check `upload_packet_types` is not filtering everything
- Verify archive files contain `raw_payload` data
- The analyzer may take a few minutes to index new nodes
**"PyNaCl fallback requires a 64-char Ed25519 seed"**
- Node.js meshcore-decoder is not available, and the private key is 128 chars
- Solution: install meshcore-decoder (see [section 4.2](#42-mqtt-uplink-dependencies))
---
## 14. Version History
| Version | Date | Description |
|---|---|---|
| 1.2.0 | 2026-02-26 | Fix: 128-char orlp/ed25519 private key support, NODE_PATH auto-detection, improved key validation |
| 1.1.0 | 2026-02-26 | Fase 2: MQTT uplink to LetsMesh (WebSocket+TLS, Ed25519 JWT, privacy filter) |
| 1.0.0 | 2026-02-26 | Fase 1: Read-only archive monitor dashboard |
---
## 15. License
MIT License — see LICENSE file
## 16. Author
**PE1HVH** — [GitHub](https://github.com/pe1hvh)

1212
README.md Normal file
View File

File diff suppressed because it is too large Load Diff

32
bridge_config.yaml Normal file
View File

@@ -0,0 +1,32 @@
# =============================================================================
# MeshCore Bridge — Configuration
# =============================================================================
#
# Cross-frequency message bridge daemon configuration.
# See BRIDGE.md for full documentation.
#
# IMPORTANT: The bridge channel must exist on BOTH devices with
# IDENTICAL channel secret/password. Only the frequency
# and channel index may differ.
bridge:
channel_name: "bridge" # Channel name (for display / logging)
channel_idx_a: 3 # Channel index on device A
channel_idx_b: 3 # Channel index on device B
poll_interval_ms: 200 # Polling interval (milliseconds)
forward_prefix: true # Add [sender] prefix to forwarded messages
max_forwarded_cache: 500 # Loop prevention cache size (number of hashes)
device_a:
port: /dev/ttyUSB1 # Serial port for device A
baud: 115200 # Baud rate
label: "869.525 MHz" # Display label for dashboard
device_b:
port: /dev/ttyUSB2 # Serial port for device B
baud: 115200 # Baud rate
label: "868.000 MHz" # Display label for dashboard
gui:
port: 9092 # Web dashboard port
title: "MeshCore Bridge" # Browser tab title

773
docs/CHANGELOG.md Normal file
View File

@@ -0,0 +1,773 @@
# CHANGELOG
<!-- CHANGED: Title changed from "CHANGELOG: Message & Metadata Persistence" to "CHANGELOG" —
a root-level CHANGELOG.md should be project-wide, not feature-specific. -->
All notable changes to MeshCore GUI are documented in this file.
Format follows [Keep a Changelog](https://keepachangelog.com/) and [Semantic Versioning](https://semver.org/).
---
## [1.13.0] - 2026-03-09 — Leaflet Map Runtime Stabilization
### Added
- ✅ `meshcore_gui/static/leaflet_map_panel.js` — Dedicated browser-side Leaflet runtime responsible for map lifecycle, marker registry, clustering and theme handling independent from NiceGUI redraw cycles
- ✅ `meshcore_gui/static/leaflet_map_panel.css` — Styling for browser-side node markers, cluster icons and map container
- ✅ `meshcore_gui/services/map_snapshot_service.py` — Snapshot service that normalizes device/contact map data into a compact payload for the browser runtime
- ✅ Browser-side map state management for center, zoom and theme
- ✅ Theme persistence across reconnect events via browser storage fallback
- ✅ Browser-side contact clustering via `Leaflet.markercluster`
- ✅ Separate non-clustered device marker layer so the own device remains individually visible
### Changed
- 🔄 `meshcore_gui/gui/panels/map_panel.py` — Replaced NiceGUI Leaflet wrapper usage with a pure browser-managed Leaflet container while preserving the existing card layout, theme toggle and center-on-device control
- 🔄 Leaflet bootstrap moved out of inline Python into a dedicated browser runtime loaded from `/static`
- 🔄 Asset loading order is now explicit: Leaflet first, then `Leaflet.markercluster`, then the MeshCore panel runtime
- 🔄 Map initialization now occurs only once per container; NiceGUI refresh cycles no longer recreate the map
- 🔄 Dashboard update loop now sends compact map snapshots instead of triggering redraws
- 🔄 Snapshot processing in the browser is coalesced so only the newest payload is applied
- 🔄 Map markers are managed in separate device/contact layers and updated incrementally by stable node id
- 🔄 Contact markers are rendered inside a persistent cluster layer while the device marker remains outside clustering
- 🔄 Theme switching moved to a dedicated theme channel instead of being embedded in snapshot data
### Fixed
- 🛠 **Map disappearing during dashboard refresh cycles** — prevented repeated map reinitialization caused by the 500 ms NiceGUI update loop
- 🛠 **Markers disappearing between refreshes** — marker updates are now incremental and keyed by node id
- 🛠 **Blank map container on load** — browser bootstrap now waits for DOM host, Leaflet runtime and panel runtime before initialization
- 🛠 **Leaflet clustering bootstrap failure (`L is not defined`)** — resolved by enforcing correct script dependency order before the panel runtime starts
- 🛠 **MarkerClusterGroup failure (`Map has no maxZoom specified`)** — the map now defines `maxZoom` during initial creation before the cluster layer is attached
- 🛠 **Half-initialized map retry cascade (`Map container is already initialized`)** — map state is now registered safely during initialization so a failed attempt cannot trigger a second `L.map(...)` on the same container
- 🛠 **Race condition between queued snapshot and theme selection** — explicit theme changes can no longer be overwritten by stale snapshot payloads
- 🛠 **Viewport jumping back to default center/zoom** — stored viewport is no longer reapplied on each snapshot update
- 🛠 **Theme reverting to default during reconnect** — effective map theme is restored before snapshot processing resumes
### Impact
- Leaflet map is now managed entirely in the browser and is no longer recreated on each dashboard refresh
- Node markers remain stable and no longer flicker or disappear during the 500 ms update cycle
- Dense contact sets can now be rendered with clustering without violating the browser-owned map lifecycle
- Theme switching and viewport state persist reliably across reconnect events
- No breaking changes outside the map subsystem
---
## [1.12.1] - 2026-03-08 — Minor change bot
### Changed
- 🔄 `meshcore_gui/services/bot.py`: remove path id's
### Impact
- No breaking changes — all existing functionality preserved serial.
---
## [1.12.0] - 2026-02-26 — MeshCore Observer Fase 1
### Added
- ✅ **MeshCore Observer daemon** — New standalone read-only daemon (`meshcore_observer.py`) that reads archive JSON files produced by meshcore_gui and meshcore_bridge, aggregates them, and presents a unified NiceGUI monitoring dashboard on port 9093.
- ✅ **ArchiveWatcher** — Core component that polls `~/.meshcore-gui/archive/` for `*_messages.json` and `*_rxlog.json` files, tracks mtime changes, and returns only new entries since previous poll. Thread-safe, zero writes, graceful on corrupt JSON.
- ✅ **Observer dashboard panels** — Sources overview, aggregated messages feed (sorted by timestamp), aggregated RX log table, and statistics panel with uptime/counters/per-source breakdown. Full DOMCA theme (dark + light mode).
- ✅ **Source filter** — Dropdown to filter messages and RX log by archive source.
- ✅ **Channel filter** — Dropdown to filter messages by channel name.
- ✅ **ObserverConfig** — YAML-based configuration with `from_yaml()` classmethod, defaults work without config file.
- ✅ **observer_config.yaml** — Documented config template with all options.
- ✅ **install_observer.sh** — systemd installer (`/opt/meshcore-observer/`, `/etc/meshcore/observer_config.yaml`), with `--uninstall` option.
- ✅ **RxLogEntry raw packet fields** — 5 new fields on `RxLogEntry` dataclass: `raw_payload`, `packet_len`, `payload_len`, `route_type`, `packet_type_num` (all with defaults, backward compatible).
- ✅ **EventHandler.on_rx_log() metadata** — Raw payload hex and packet metadata now passed through to RxLogEntry and archived (preparation for Fase 2 LetsMesh uplink).
### Changed
- 🔄 `meshcore_gui/core/models.py`: RxLogEntry +5 fields with defaults (backward compatible).
- 🔄 `meshcore_gui/ble/events.py`: on_rx_log() fills raw_payload and metadata (~10 lines added).
- 🔄 `meshcore_gui/services/message_archive.py`: add_rx_log() serializes the 5 new RxLogEntry fields.
- 🔄 `meshcore_gui/config.py`: Version bumped to `1.12.0`.
### Impact
- **No breaking changes** — All new RxLogEntry fields have defaults; existing archives and code work identically.
- **New daemon** — meshcore_observer is fully standalone; no imports from meshcore_gui (reads only JSON files).
---
### Added
- ✅ **Serial CLI flags** — `--baud=BAUD` and `--serial-cx-dly=SECONDS` for serial configuration at startup.
### Changed
- 🔄 **Connection layer** — Switched from BLE to serial (`MeshCore.create_serial`) with serial reconnect handling.
- 🔄 `config.py`: Added `SERIAL_BAUDRATE`, `SERIAL_CX_DELAY`, `DEFAULT_TIMEOUT`, `MESHCORE_LIB_DEBUG`; removed BLE PIN settings; version bumped to `1.10.0`.
- 🔄 `meshcore_gui.py` / `meshcore_gui/__main__.py`: Updated usage, banners and defaults for serial ports.
- 🔄 Docs: Updated README and core docs for serial usage; BLE documents marked as legacy.
### Impact
- No breaking changes — all existing functionality preserved serial.
---
## [1.9.11] - 2026-02-19 — Message Dedup Hotfix
### Fixed
- 🛠 **Duplicate messages after (re)connect** — `load_recent_from_archive()` appended archived messages on every connect attempt without clearing existing entries; after N failed connects, each message appeared N times. Method is now idempotent: clears the in-memory list before loading.
- 🛠 **Persistent duplicate messages** — Live BLE events for messages already loaded from archive were not suppressed because the `DualDeduplicator` was never seeded with archived content. Added `_seed_dedup_from_messages()` in `BLEWorker` after cache/archive load and after reconnect.
- 🛠 **Last-line-of-defence dedup in SharedData** — `add_message()` now maintains a fingerprint set (`message_hash` or `channel:sender:text`) and silently skips messages whose fingerprint is already tracked. This guards against duplicates regardless of their source.
- 🛠 **Messages panel empty on first click** — `_show_panel()` made the container visible but relied on the next 500 ms timer tick to populate it. Added an immediate `_messages.update()` call so content is rendered the moment the panel becomes visible.
### Changed
- 🔄 `core/shared_data.py`: Added `_message_fingerprints` set and `_message_fingerprint()` static method; `add_message()` checks fingerprint before insert and evicts fingerprints when messages are rotated out; `load_recent_from_archive()` clears messages and fingerprints before loading (idempotent)
- 🔄 `ble/worker.py`: Added `_seed_dedup_from_messages()` helper; called after `_apply_cache()` and after reconnect `_load_data()` to seed `DualDeduplicator` with existing messages
- 🔄 `gui/dashboard.py`: `_show_panel()` now forces an immediate `_messages.update()` when the messages panel is shown, eliminating the stale-content flash
- 🔄 `config.py`: Version bumped to `1.9.11`
### Impact
- Eliminates all duplicate message display scenarios: initial connect, failed retries, reconnect, and BLE event replay
- No breaking changes — all existing functionality preserved
- Fingerprint set is bounded to the same 100-message cap as the message list
---
## [1.9.10] - 2026-02-19 — Map Tooltips & Separate Own-Position Marker
### Added
- ✅ **Map marker tooltips** — All markers on the Leaflet map now show a tooltip on hover with the node name and type icon (📱, 📡, 🏠) from `TYPE_ICONS`
- ✅ **Separate own-position marker** — The device's own position is now tracked as a dedicated `_own_marker`, independent from contact markers. This prevents the own marker from being removed/recreated on every contact update cycle
### Changed
- 🔄 `gui/panels/map_panel.py`: Renamed `_markers` to `_contacts_markers`; added `_own_marker` attribute; own position marker is only updated when `device_updated` flag is set (not every timer tick); contact markers are only rebuilt when `contacts_updated` is set; added `TYPE_ICONS` import for tooltip icons
- 🔄 `gui/dashboard.py`: Added `self._map.update(data)` call in the `device_updated` block so the own-position marker updates when device info changes (e.g. GPS position update)
- 🔄 `config.py`: Version bumped to `1.9.10`
### Impact
- Map centering on own device now works correctly and updates only when position actually changes
- Contact markers are no longer needlessly destroyed and recreated on every UI timer tick — only on actual contact data changes
- Tooltips make it easy to identify nodes on the map without clicking
- No breaking changes — all existing map functionality preserved
### Credits
- Based on [PR #16](https://github.com/pe1hvh/meshcore-gui/pull/16) by [@rich257](https://github.com/rich257)
---
## [1.9.9] - 2026-02-18 — Variable Landing Page & Operator Callsign
### Added
- ✅ **Configurable operator callsign** — New `OPERATOR_CALLSIGN` constant in `config.py` (default: `"PE1HVH"`). Used in the landing page SVG and the drawer footer copyright label. Change this single value to personalize the entire GUI for a different operator
- ✅ **External landing page SVG** — The DOMCA splash screen is now loaded from a standalone file (`static/landing_default.svg`) instead of being hardcoded in `dashboard.py`. New `LANDING_SVG_PATH` constant in `config.py` points to the SVG file. The placeholder `{callsign}` in the SVG is replaced at runtime with `OPERATOR_CALLSIGN`
- ✅ **Landing page customization** — To use a custom landing page: copy `landing_default.svg` (or create your own SVG), use `{callsign}` wherever the operator callsign should appear, and point `LANDING_SVG_PATH` to your file. The default SVG includes an instructive comment block explaining the placeholder mechanism
### Changed
- 🔄 `config.py`: Added `OPERATOR_CALLSIGN` and `LANDING_SVG_PATH` constants in new **OPERATOR / LANDING PAGE** section; version bumped to `1.9.9`
- 🔄 `gui/dashboard.py`: Removed hardcoded `_DOMCA_SVG` string (~70 lines); added `_load_landing_svg()` helper that reads SVG from disk and replaces `{callsign}` placeholder; CSS variable `--pe1hvh` renamed to `--callsign`; drawer footer copyright label now uses `config.OPERATOR_CALLSIGN`
### Added (files)
- ✅ `static/landing_default.svg` — The original DOMCA splash SVG extracted as a standalone file, with `{callsign}` placeholder and `--callsign` CSS variable. Serves as both the default landing page and a reference template for custom SVGs
### Impact
- Out-of-the-box behavior is identical to v1.9.8 (same DOMCA branding, same PE1HVH callsign)
- Operators personalize by changing 1–2 lines in `config.py` — no code modifications needed
- Fallback: if the SVG file is missing, a minimal placeholder text is shown instead of a crash
- No breaking changes — all existing dashboard functionality (panels, menus, timer, theming) unchanged
---
## [1.9.8] - 2026-02-17 — Bugfix: Route Page Sender ID, Type & Location Not Populated
### Fixed
- 🛠 **Sender ID, Type and Location empty in Route Page** — After the v4.1 refactoring to `RouteBuilder`/`RouteNode`, the sender contact lookup relied solely on `SharedData.get_contact_by_prefix()` (live lock-based) and `get_contact_by_name()`. When both failed (e.g. empty `sender_pubkey` from RX_LOG decode, or name mismatch), `route['sender']` remained `None` and the route table fell through to a hardcoded fallback with `type: '-'`, `location: '-'`. The contact data was available in the snapshot `data['contacts']` but was never searched
- 🛠 **Route table fallback row ignored available contact data** — When `route['sender']` was `None`, the `_render_route_table` method used a static fallback row without attempting to find the contact in the data snapshot. Even when the contact was present in `data['contacts']` with valid type and location, these fields showed as `'-'`
### Changed
- 🔄 `services/route_builder.py`: Added two additional fallback strategies in `build()` after the existing SharedData lookups: (3) bidirectional pubkey prefix match against `data['contacts']` snapshot, (4) case-insensitive `adv_name` match against `data['contacts']` snapshot. Added helper methods `_find_contact_by_pubkey()` and `_find_contact_by_adv_name()` for snapshot-based lookups
- 🔄 `gui/route_page.py`: Added defensive fallback in `_render_route_table()` sender section — when `route['sender']` is `None`, attempts to find the contact in the snapshot via `_find_sender_contact()` before falling back to the static `'-'` row. Added `_find_sender_contact()` helper method
### Impact
- Sender ID (hash), Type and Location are now populated correctly in the route table when the contact is known
- Four-layer lookup chain ensures maximum resolution: (1) SharedData pubkey lookup, (2) SharedData name lookup, (3) snapshot pubkey lookup, (4) snapshot name lookup
- Defensive fallback in route_page guarantees data is shown even if RouteBuilder misses it
- No breaking changes — all existing route page behavior, styling and data flows unchanged
---
## [1.9.7] - 2026-02-17 — Layout Fix: Archive Filter Toggle & Route Page Styling
### Changed
- 🔄 `gui/archive_page.py`: Archive filter card now hidden by default; toggle visibility via a `filter_list` icon button placed right-aligned on the same row as the "📚 Archive" title. Header restructured from single label to `ui.row()` with `justify-between` layout
- 🔄 `gui/route_page.py`: Route page now uses DOMCA theme (imported from `dashboard.py`) with dark mode as default, consistent with the main dashboard. Header restyled from `bg-blue-600` to Quasar-themed header with JetBrains Mono font. Content container changed from `w-full max-w-4xl mx-auto` to `domca-panel` class for consistent responsive sizing
- 🔄 `gui/dashboard.py`: Added `domca-header-text` CSS class with `@media (max-width: 599px)` rule to hide header text on narrow viewports; applied to version label and status label
- 🔄 `gui/route_page.py`: Header label also uses `domca-header-text` class for consistent responsive behaviour
### Added
- ✅ **Archive filter toggle** — `filter_list` icon button in archive header row toggles the filter card visibility on click
- ✅ **Route page close button** — `X` (close) icon button added right-aligned in the route page header; calls `window.close()` to close the browser tab
- ✅ **Responsive header** — On viewports < 600px, header text labels are hidden; only icon buttons (menu, dark mode toggle, close) remain visible
### Impact
- Archive page is cleaner by default — filters only shown when needed
- Route page visually consistent with the main dashboard (DOMCA theme, dark mode, responsive panel width)
- Headers degrade gracefully on mobile (< 600px): only icon buttons visible, no text overflow
- No functional changes — all event handlers, callbacks, data bindings, logic and imports are identical to the input
---
## [1.9.6] - 2026-02-17 — Bugfix: Channel Discovery Reliability
### Fixed
- 🛠 **Channels not appearing (especially on mobile)** — Channel discovery aborted too early on slow BLE connections. The `_discover_channels()` probe used a single attempt per channel slot and stopped after just 2 consecutive empty responses. On mobile BLE stacks (WebBluetooth via NiceGUI) where GATT responses are slower, this caused discovery to abort before finding any channels, falling back to only `[0] Public`
- 🛠 **Race condition: channel update flag lost between threads** — `get_snapshot()` and `clear_update_flags()` were two separate calls, each acquiring the lock independently. If the BLE worker set `channels_updated = True` between these two calls, the GUI consumed the flag via `get_snapshot()` but then `clear_update_flags()` reset it — causing the channel submenu and dropdown to never populate
- 🛠 **Channels disappear on browser reconnect** — When a browser tab is closed and reopened, `render()` creates new (empty) NiceGUI containers for the drawer submenus, but did not reset `_last_channel_fingerprint`. The `_update_submenus()` method compared the new fingerprint against the stale one, found them equal, and skipped the rebuild — leaving the new containers permanently empty. Fixed by resetting both `_last_channel_fingerprint` and `_last_rooms_fingerprint` in `render()`
### Changed
- 🔄 `core/shared_data.py`: New atomic method `get_snapshot_and_clear_flags()` that reads the snapshot and resets all update flags in a single lock acquisition. Internally refactored to `_build_snapshot_unlocked()` helper. Existing `get_snapshot()` and `clear_update_flags()` retained for backward compatibility
- 🔄 `ble/worker.py`: `_discover_channels()` — `max_attempts` increased from 1 to 2 per channel slot; inter-attempt `delay` increased from 0.5s to 1.0s; consecutive error threshold raised from 2 to 3; inter-channel pause increased from 0.15s to 0.3s for mobile BLE stack breathing room
- 🔄 `gui/dashboard.py`: `_update_ui()` now uses `get_snapshot_and_clear_flags()` instead of separate `get_snapshot()` + `clear_update_flags()`; `render()` now resets `_last_channel_fingerprint` and `_last_rooms_fingerprint` to `None` so that `_update_submenus()` rebuilds into the freshly created containers; channel-dependent updates (`update_filters`, `update_channel_options`, `_update_submenus`) now run unconditionally when channel data exists — safe because each method has internal idempotency checks
- 🔄 `gui/panels/messages_panel.py`: `update_channel_options()` now includes an equality check on options dict to skip redundant `.update()` calls to the NiceGUI client on every 500ms timer tick
### Impact
- Channel discovery now survives transient BLE timeouts that are common on mobile connections
- Atomic snapshot eliminates the threading race condition that caused channels to silently never appear
- Browser close+reopen no longer loses channels — the single-instance timer race on the shared `DashboardPage` is fully mitigated
- No breaking changes — all existing API methods retained, all other functionality unchanged
---
## [1.9.5] - 2026-02-16 — Layout Fix: RX Log Table Responsive Sizing
### Fixed
- 🛠 **RX Log table did not adapt to panel/card size** — The table used `max-h-48` (a maximum height cap) instead of a responsive fixed height, causing it to remain small regardless of available space. Changed to `h-40` which is overridden by the existing dashboard CSS to `calc(100vh - 20rem)` — the same responsive pattern used by the Messages panel
- 🛠 **RX Log table did not fill card width** — Added `w-full` class to the table element so it stretches to the full width of the parent card
- 🛠 **RX Log card did not fill panel height** — Added `flex-grow` class to the card container so it expands to fill the available panel space
### Changed
- 🔄 `gui/panels/rxlog_panel.py`: Card classes `'w-full'` → `'w-full flex-grow'` (line 45); table classes `'text-xs max-h-48 overflow-y-auto'` → `'w-full text-xs h-40 overflow-y-auto'` (line 65)
### Impact
- RX Log table now fills the panel consistently on both desktop and mobile viewports
- Layout is consistent with other panels (Messages, Contacts) that use the same `h-40` responsive height pattern
- No functional changes — all event handlers, callbacks, data bindings, logica and imports are identical to the input
---
## [1.9.4] - 2026-02-16 — BLE Address Log Prefix & Entry Point Cleanup
### Added
- ✅ **BLE address prefix in log filename** — Log file is now named `<BLE_ADDRESS>_meshcore_gui.log` (e.g. `AA_BB_CC_DD_EE_FF_meshcore_gui.log`) instead of the generic `meshcore_gui.log`. Makes it easy to identify which device produced which log file when running multiple instances
- New helper `_sanitize_ble_address()` strips `literal:` prefix and replaces colons with underscores
- New function `configure_log_file(ble_address)` updates `LOG_FILE` at runtime before the logger is initialised
- Rotated backups follow the same naming pattern automatically
### Removed
- ❌ **`meshcore_gui/meshcore_gui.py`** — Redundant copy of `main()` that was never imported. All three entry points (`meshcore_gui.py` root, `__main__.py`, and `meshcore_gui/meshcore_gui.py`) contained near-identical copies of the same logic, causing changes to be missed (as demonstrated by this fix). `__main__.py` is now the single source of truth; root `meshcore_gui.py` is a thin wrapper that imports from it
### Changed
- 🔄 `config.py`: Added `_sanitize_ble_address()` and `configure_log_file()`; version bumped to `1.9.4`
- 🔄 `__main__.py`: Added `config.configure_log_file(ble_address)` call before any debug output
- 🔄 `meshcore_gui.py` (root): Reduced to 4-line wrapper importing `main` from `__main__`
### Impact
- Log files are now identifiable per BLE device
- Single source of truth for `main()` eliminates future sync issues between entry points
- Both startup methods (`python meshcore_gui.py` and `python -m meshcore_gui`) remain functional
- No breaking changes — defaults and all existing behaviour unchanged
---
## [1.9.3] - 2026-02-16 — Bugfix: Map Default Location & Payload Type Decoding
### Fixed
- 🛠 **Map centred on hardcoded Zwolle instead of device location** — All Leaflet maps used magic-number coordinates `(52.5, 6.0)` as initial centre and fallback. These are now replaced by a single configurable constant `DEFAULT_MAP_CENTER` in `config.py`. Once the device reports a valid `adv_lat`/`adv_lon`, maps re-centre on the actual device position (existing behaviour, unchanged)
- 🛠 **Payload type shown as raw integer** — Payload type is now retrieved from the decoded payload and translated to human-readable text using MeshCoreDecoder functions, instead of displaying the raw numeric type value
### Changed
- 🔄 `config.py`: Added `DEFAULT_MAP_CENTER` (default: `(52.5168, 6.0830)`) and `DEFAULT_MAP_ZOOM` (default: `9`) constants in new **MAP DEFAULTS** section. Version bumped to `1.9.2`
- 🔄 `gui/panels/map_panel.py`: Imports `DEFAULT_MAP_CENTER` and `DEFAULT_MAP_ZOOM` from config; `ui.leaflet(center=...)` uses config constants instead of hardcoded values
- 🔄 `gui/route_page.py`: Imports `DEFAULT_MAP_CENTER` and `DEFAULT_MAP_ZOOM` from config; fallback coordinates (`or 52.5` / `or 6.0`) replaced by `DEFAULT_MAP_CENTER[0]` / `[1]`; zoom uses `DEFAULT_MAP_ZOOM`
### Impact
- Map default location is now a single-point-of-change in `config.py`
- Payload type is displayed as readable text instead of a raw number
- No breaking changes — all existing map behaviour (re-centre on device position, contact markers) unchanged
## [1.9.2] - 2026-02-15 — CLI Parameters & Cleanup
### Added
- ✅ **`--port=PORT` CLI parameter** — Web server port is now configurable at startup (default: `8081`). Allows running multiple instances simultaneously on different ports
- ✅ **`--ble-pin=PIN` CLI parameter** — BLE pairing PIN is now configurable at startup (default: `123456`). Eliminates the need to edit `config.py` for devices with a non-default PIN, and works in systemd service files
- ✅ **Per-device log file** — Debug log file now includes the BLE address in its filename (e.g. `F0_9E_9E_75_A3_01_meshcore_gui.log`), so multiple instances log to separate files
### Fixed
- 🛠 **BLE PIN not applied from CLI** — `ble/worker.py` imported `BLE_PIN` as a constant at module load time (`from config import BLE_PIN`), capturing the default value `"123456"` before CLI parsing could override `config.BLE_PIN`. Changed to runtime access via `config.BLE_PIN` so the `--ble-pin` parameter is correctly passed to the BLE agent
### Removed
- ❌ **Redundant `meshcore_gui/meshcore_gui.py`** — This file was a near-identical copy of both `meshcore_gui.py` (top-level) and `meshcore_gui/__main__.py`, but was never imported or referenced. Removed to eliminate maintenance risk. The two remaining entry points cover all startup methods: `python meshcore_gui.py` and `python -m meshcore_gui`
### Impact
- Multiple instances can run side-by-side with different ports, PINs and log files
- Service deployments no longer require editing `config.py` — all runtime settings via CLI
- No breaking changes — all defaults are unchanged
---
## [1.9.1] - 2026-02-14 — Bugfix: Dual Reconnect Conflict
### Fixed
- 🛠 **Library reconnect interfered with application reconnect** — The meshcore library's internal `auto_reconnect` (visible in logs as `"Attempting reconnection 1/3"`) ran a fast 3-attempt reconnect cycle without bond cleanup. This prevented the application's own `reconnect_loop` (which does `remove_bond()` + backoff) from succeeding, because BlueZ retained a stale bond → `"failed to discover service"`
### Changed
- 🔄 `ble/worker.py`: Set `auto_reconnect=False` in both `MeshCore.create_ble()` call sites (`_connect()` and `_create_fresh_connection()`), so only the application's bond-aware `reconnect_loop` handles reconnection
- 🔄 `ble/worker.py`: Added `"failed to discover"` and `"service discovery"` to disconnect detection keywords for defensive coverage
### Impact
- Eliminates the ~9 second wasted library reconnect cycle after every BLE disconnect
- Application's `reconnect_loop` (with bond cleanup) now runs immediately after disconnect detection
- No breaking changes — the application reconnect logic was already fully functional
---
## [1.9.0] - 2026-02-14 — BLE Connection Stability
### Added
- ✅ **Built-in BLE PIN agent** — New `ble/ble_agent.py` registers a D-Bus agent with BlueZ to handle PIN pairing requests automatically. Eliminates the need for external `bt-agent.service` and `bluez-tools` package
- Uses `dbus_fast` (already a dependency of `bleak`, no new packages)
- Supports `RequestPinCode`, `RequestPasskey`, `DisplayPasskey`, `RequestConfirmation`, `AuthorizeService` callbacks
- Configurable PIN via `BLE_PIN` in `config.py` (default: `123456`)
- ✅ **Automatic bond cleanup** — New `ble/ble_reconnect.py` provides `remove_bond()` function that removes stale BLE bonds via D-Bus, equivalent to `bluetoothctl remove <address>`. Called automatically on startup and before each reconnect attempt
- ✅ **Automatic reconnect after disconnect** — BLEWorker main loop now detects BLE disconnects (via connection error exceptions) and automatically triggers a reconnect sequence: bond removal → linear backoff wait → fresh connection → re-wire handlers → reload device data
- Configurable via `RECONNECT_MAX_RETRIES` (default: 5) and `RECONNECT_BASE_DELAY` (default: 5.0s)
- After all retries exhausted: waits 60s then starts a new retry cycle (infinite recovery)
- ✅ **Generic install script** — `install_ble_stable.sh` auto-detects user, project directory, venv path and entry point to generate systemd service and D-Bus policy. Supports `--uninstall` flag
### Changed
- 🔄 **`ble/worker.py`** — `_async_main()` rewritten with three phases: (1) start PIN agent, (2) remove stale bond, (3) connect + main loop with disconnect detection. Reconnect logic re-wires all event handlers and reloads device data after successful reconnection
- 🔄 **`config.py`** — Added `BLE_PIN`, `RECONNECT_MAX_RETRIES`, `RECONNECT_BASE_DELAY` constants
### Removed
- ❌ **`bt-agent.service` dependency** — No longer needed; PIN pairing is handled by the built-in agent
- ❌ **`bluez-tools` system package** — No longer needed
- ❌ **`~/.meshcore-ble-pin` file** — No longer needed
- ❌ **Manual `bluetoothctl remove` before startup** — Handled automatically
- ❌ **`ExecStartPre` in systemd service** — Bond cleanup is internal
### Impact
- Zero external dependencies for BLE pairing on Linux
- Automatic recovery from the T1000e ~2 hour BLE disconnect issue
- No manual intervention needed after BLE connection loss
- Single systemd service (`meshcore-gui.service`) manages everything
- No breaking changes to existing functionality
---
## [1.8.0] - 2026-02-14 — DRY Message Construction & Archive Layout Unification
### Fixed
- 🛠 **Case-sensitive prefix matching** — `get_contact_name_by_prefix()` and `get_contact_by_prefix()` in `shared_data.py` failed to match path hashes (uppercase, e.g. `'B8'`) against contact pubkeys (lowercase, e.g. `'b8a3f2...'`). Added `.lower()` to both sides of the comparison, consistent with `_resolve_path_names()` which already had it
- 🛠 **Route page 404 from archive** — Archive page linked to `/route/{hash}` but route was registered as `/route/{msg_index:int}`, causing a JSON parse error for hex hash strings. Route parameter changed to `str` with 3-strategy lookup (index → memory hash → archive fallback)
- 🛠 **Three entry points out of sync** — `meshcore_gui.py` (root), `meshcore_gui/meshcore_gui.py` (inner) and `meshcore_gui/__main__.py` had diverging route registrations. All three now use identical `/route/{msg_key}` with `str` parameter
### Changed
- 🔄 **`core/models.py` — DRY factory methods and formatting**
- `Message.now_timestamp()`: static method replacing 7× hardcoded `datetime.now().strftime('%H:%M:%S')` across `events.py` and `commands.py`
- `Message.incoming()`: classmethod factory for received messages (`direction='in'`, auto-timestamp)
- `Message.outgoing()`: classmethod factory for sent messages (`sender='Me'`, `direction='out'`, auto-timestamp)
- `Message.format_line(channel_names)`: single-line display formatting (`"12:34:56 ← [Public] [2h✓] PE1ABC: Hello mesh!"`), replacing duplicate inline formatting in `messages_panel.py` and `archive_page.py`
- 🔄 **`ble/events.py`** — 4× `Message(...)` constructors replaced by `Message.incoming()`; `datetime` import removed
- 🔄 **`ble/commands.py`** — 3× `Message(...)` constructors replaced by `Message.outgoing()`; `datetime` import removed
- 🔄 **`gui/panels/messages_panel.py`** — 15 lines inline formatting replaced by single `msg.format_line(channel_names)` call
- 🔄 **`gui/archive_page.py` — Layout unified with main page**
- Multi-row card layout replaced by single-line `msg.format_line()` in monospace container (same style as main page)
- DM added to channel filter dropdown (post-filter on `channel is None`)
- Message click opens `/route/{message_hash}` in new tab (was: no click handler on archive messages)
- Removed `_render_message_card()` (98 lines) and `_render_archive_route()` (75 lines)
- Removed `RouteBuilder` dependency and `TYPE_LABELS` import
- File reduced from 445 to 267 lines
- 🔄 **`gui/route_page.py`** — `render(msg_index: int)` → `render(msg_key: str)` with 3-strategy message lookup: (1) numeric index from in-memory list, (2) hash match in memory, (3) `archive.get_message_by_hash()` fallback
- 🔄 **`services/message_archive.py`** — New method `get_message_by_hash(hash)` for single-message lookup by packet hash
- 🔄 **`__main__.py` + `meshcore_gui.py` (both)** — Route changed from `/route/{msg_index}` (int) to `/route/{msg_key}` (str)
### Impact
- DRY: timestamp formatting 7→1 definition, message construction 7→2 factories, line formatting 2→1 method
- Archive page visually consistent with main messages panel (single-line, monospace)
- Archive messages now clickable to open route visualization (was: only in-memory messages)
- Case-insensitive prefix matching fixes path name resolution for contacts with uppercase path hashes
- No breaking changes to BLE protocol handling, dedup, bot, or data storage
### Known Limitations
- DM filter in archive uses post-filtering (query without channel filter + filter on `channel is None`); becomes exact when `query_messages()` gets native DM support
### Parked for later
- Multi-path tracking (enrich RxLogEntry with multiple path observations)
- Events correlation improvements (only if proven data loss after `.lower()` fix)
---
## [1.7.0] - 2026-02-13 — Archive Channel Name Persistence
### Added
- ✅ **Channel name stored in archive** — Messages now persist `channel_name` alongside the numeric `channel` index in `<ADDRESS>_messages.json`, so archived messages retain their human-readable channel name even when the device is not connected
- `Message` dataclass: new field `channel_name: str` (default `""`, backward compatible)
- `SharedData.add_message()`: automatically resolves `channel_name` from the live channels list when not already set (new helper `_resolve_channel_name()`)
- `MessageArchive.add_message()`: writes `channel_name` to the JSON dict
- ✅ **Archive channel selector built from archived data** — Channel filter dropdown on `/archive` now populated via `SELECT DISTINCT channel_name` on the archive instead of the live BLE channels list
- New method `MessageArchive.get_distinct_channel_names()` returns sorted unique channel names from stored messages
- Selector shows only channels that actually have archived messages
- ✅ **Archive filter on channel name** — `MessageArchive.query_messages()` parameter changed from `channel: Optional[int]` to `channel_name: Optional[str]` (exact match on name string)
### Changed
- 🔄 `core/models.py`: Added `channel_name` field to `Message` dataclass and `from_dict()`
- 🔄 `core/shared_data.py`: `add_message()` resolves channel name; added `_resolve_channel_name()` helper
- 🔄 `services/message_archive.py`: `channel_name` persisted in JSON; `query_messages()` filters by name; new `get_distinct_channel_names()` method
- 🔄 `gui/archive_page.py`: Channel selector built from `archive.get_distinct_channel_names()`; filter state changed from `_channel_filter` (int) to `_channel_name_filter` (str); message cards show `channel_name` directly from archive
### Fixed
- 🛠 **Main page empty after startup** — After a restart the messages panel showed no messages until new live BLE traffic arrived. `SharedData.load_recent_from_archive()` now loads up to 100 recent archived messages during the cache-first startup phase, so historical messages are immediately visible
- New method `SharedData.load_recent_from_archive(limit)` — reads from `MessageArchive.query_messages()` and populates the in-memory list without re-archiving
- `BLEWorker._apply_cache()` calls `load_recent_from_archive()` at the end of cache loading
### Impact
- Archived messages now self-contained — channel name visible without live BLE connection
- Main page immediately shows historical messages after startup (no waiting for live BLE traffic)
- Backward compatible — old archive entries without `channel_name` fall back to `"Ch <idx>"`
- No breaking changes to existing functionality
---
## [1.6.0] - 2026-02-13 — Dashboard Layout Consolidation
### Changed
- 🔄 **Messages panel consolidated** — Filter checkboxes (DM + channels) and message input (text field, channel selector, Send button) are now integrated into the Messages panel, replacing the separate Filter and Input panels
- DM + channel checkboxes displayed centered in the Messages header row, between the "💬 Messages" label and the "📚 Archive" button
- Message input row (text field, channel selector, Send button) placed below the message list within the same card
- `messages_panel.py`: Constructor now accepts `put_command` callable; added `update_filters(data)`, `update_channel_options(channels)` methods and `channel_filters`, `last_channels` properties (all logic 1:1 from FilterPanel/InputPanel); `update()` signature unchanged
- 🔄 **Actions panel expanded** — BOT toggle checkbox moved from Filter panel to Actions panel, below the Refresh/Advert buttons
- `actions_panel.py`: Constructor now accepts `set_bot_enabled` callable; added `update(data)` method for BOT state sync; `_on_bot_toggle()` logic 1:1 from FilterPanel
- 🔄 **Dashboard layout simplified** — Centre column reduced from 4 panels (Map → Input → Filter → Messages) to 2 panels (Map → Messages)
- `dashboard.py`: FilterPanel and InputPanel no longer rendered; all dependencies rerouted to MessagesPanel and ActionsPanel; `_update_ui()` call-sites updated accordingly
### Removed (from layout, files retained)
- ❌ **Filter panel** no longer rendered as separate panel — `filter_panel.py` retained in codebase but not instantiated in dashboard
- ❌ **Input panel** no longer rendered as separate panel — `input_panel.py` retained in codebase but not instantiated in dashboard
### Impact
- Cleaner, more compact dashboard: 2 fewer panels in the centre column
- All functionality preserved — message filtering, send, BOT toggle, archive all work identically
- No breaking changes to BLE, services, core or other panels
---
<!-- ADDED: v1.5.0 feature + bugfix entry -->
## [1.5.0] - 2026-02-11 — Room Server Support, Dynamic Channel Discovery & Contact Management
### Added
- ✅ **Room Server panel** — Dedicated per-room-server message panel in the centre column below Messages. Each Room Server (type=3 contact) gets its own `ui.card()` with login/logout controls and message display
- Click a Room Server contact to open an add/login dialog with password field
- After login: messages are displayed in the room card; send messages directly from the room panel
- Password row + login button automatically replaced by Logout button after successful login
- Room Server author attribution via `signature` field (txt_type=2) — real message author is resolved from the 4-byte pubkey prefix, not the room server pubkey
- New panel: `gui/panels/room_server_panel.py` — per-room card management with login state tracking
- ✅ **Room Server password store** — Passwords stored outside the repository in `~/.meshcore-gui/room_passwords/<ADDRESS>.json`
- New service: `services/room_password_store.py` — JSON-backed persistent password storage per BLE device, analogous to `PinStore`
- Room panels are restored from stored passwords on app restart
- ✅ **Dynamic channel discovery** — Channels are now auto-discovered from the device at startup via `get_channel()` BLE probing, replacing the hardcoded `CHANNELS_CONFIG`
- Single-attempt probe per channel slot with early stop after 2 consecutive empty slots
- Channel name and encryption key extracted in a single pass (combined discovery + key loading)
- Configurable channel caching via `CHANNEL_CACHE_ENABLED` (default: `False` — always fresh from device)
- `MAX_CHANNELS` setting (default: 8) controls how many slots are probed
- ✅ **Individual contact deletion** — 🗑️ delete button per unpinned contact in the contacts list, with confirmation dialog
- New command: `remove_single_contact` in BLE command handler
- Pinned contacts are protected (no delete button shown)
- ✅ **"Also delete from history" option** — Checkbox in the Clean up confirmation dialog to also remove locally cached contact data
<!-- ADDED: Research document reference -->
- ✅ **Room Server protocol research** — `RoomServer_Companion_App_Onderzoek.md` documents the full companion app message flow (login, push protocol, signature mechanism, auto_message_fetching)
### Changed
- 🔄 `config.py`: Removed `CHANNELS_CONFIG` constant; added `MAX_CHANNELS` (default: 8) and `CHANNEL_CACHE_ENABLED` (default: `False`)
- 🔄 `ble/worker.py`: Replaced hardcoded channel loading with `_discover_channels()` method; added `_try_get_channel_info()` helper; `_apply_cache()` respects `CHANNEL_CACHE_ENABLED` setting; removed `_load_channel_keys()` (integrated into discovery pass)
- 🔄 `ble/commands.py`: Added `login_room`, `send_room_msg` and `remove_single_contact` command handlers
- 🔄 `gui/panels/contacts_panel.py`: Contact click now dispatches by type — type=3 (Room Server) opens room dialog, others open DM dialog; added `on_add_room` callback parameter; added 🗑️ delete button per unpinned contact
- 🔄 `gui/panels/messages_panel.py`: Room Server messages filtered from general message view via `_is_room_message()` with prefix matching; `update()` accepts `room_pubkeys` parameter
- 🔄 `gui/dashboard.py`: Added `RoomServerPanel` in centre column; `_update_ui()` passes `room_pubkeys` to Messages panel; added `_on_add_room_server` callback
- 🔄 `gui/panels/filter_panel.py`: Channel filter checkboxes now built dynamically from discovered channels (no hardcoded references)
- 🔄 `services/bot.py`: Removed stale comment referencing hardcoded channels
### Fixed
- 🛠 **Room Server messages appeared as DM** — Messages from Room Servers (txt_type=2) were displayed in the general Messages panel as direct messages. They are now filtered out and shown exclusively in the Room Server panel
- 🛠 **Historical room messages not shown after login** — Post-login fetch loop was polling `get_msg()` before room server had time to push messages over LoRa RF (10–75s per message). Removed redundant fetch loop; the library's `auto_message_fetching` handles `MESSAGES_WAITING` events correctly and event-driven
- 🛠 **Author attribution incorrect for room messages** — Room server messages showed the room server name as sender instead of the actual message author. Now correctly resolved from the `signature` field (4-byte pubkey prefix) via contact lookup
### Impact
- Room Servers are now first-class citizens in the GUI with dedicated panels
- Channel configuration no longer requires manual editing of `config.py`
- Contact list management is more granular with per-contact deletion
- No breaking changes to existing functionality (messages, DM, map, archive, bot, etc.)
---
## [1.4.0] - 2026-02-09 — SDK Event Race Condition Fix
### Fixed
- 🛠 **BLE startup delay of ~2 minutes eliminated** — The meshcore Python SDK (`commands/base.py`) dispatched device response events before `wait_for_events()` registered its subscription. On busy networks with frequent `RX_LOG_DATA` events, this caused `send_device_query()` and `get_channel()` to fail repeatedly with `no_event_received`, wasting 110+ seconds in timeouts
### Changed
- 📄 `meshcore` SDK `commands/base.py`: Rewritten `send()` method to subscribe to expected events **before** transmitting the BLE command (subscribe-before-send pattern), matching the approach used by the companion apps (meshcore.js, iOS, Android). Submitted upstream as [meshcore_py PR #52](https://github.com/meshcore-dev/meshcore_py/pull/52)
### Impact
- Startup time reduced from ~2+ minutes to ~10 seconds on busy networks
- All BLE commands (`send_device_query`, `get_channel`, `get_bat`, `send_appstart`, etc.) now succeed on first attempt instead of requiring multiple retries
- No changes to meshcore_gui code required — the fix is entirely in the meshcore SDK
### Temporary Installation
Until the fix is merged upstream, install the patched meshcore SDK:
```bash
pip install --force-reinstall git+https://github.com/PE1HVH/meshcore_py.git@fix/event-race-condition
```
---
<!-- ADDED: v1.3.2 bugfix entry -->
## [1.3.2] - 2026-02-09 — Bugfix: Bot Device Name Restoration After Restart
### Fixed
- 🛠 **Bot device name not properly restored after restart/crash** — After a restart or crash with bot mode previously active, the original device name was incorrectly stored as the bot name (e.g. `NL-OV-ZWL-STDSHGN-WKC Bot`) instead of the real device name (e.g. `PE1HVH T1000e`). The original device name is now correctly preserved and restored when bot mode is disabled
### Changed
- 🔄 `commands.py`: `set_bot_name` handler now verifies that the stored original name is not already the bot name before saving
- 🔄 `shared_data.py`: `original_device_name` is only written when it differs from `BOT_DEVICE_NAME` to prevent overwriting with the bot name on restart
---
<!-- ADDED: v1.3.1 bugfix entry -->
## [1.3.1] - 2026-02-09 — Bugfix: Auto-add AttributeError
### Fixed
- 🛠 **Auto-add error on first toggle** — Setting auto-add for the first time raised `AttributeError: 'telemetry_mode_base'`. The `set_manual_add_contacts()` SDK call now handles missing `telemetry_mode_base` attribute gracefully
### Changed
- 🔄 `commands.py`: `set_auto_add` handler wraps `set_manual_add_contacts()` call with attribute check and error handling for missing `telemetry_mode_base`
---
<!-- ADDED: New v1.3.0 entry at top -->
## [1.3.0] - 2026-02-08 — Bot Device Name Management
### Added
- ✅ **Bot device name switching** — When the BOT checkbox is enabled, the device name is automatically changed to a configurable bot name; when disabled, the original name is restored
- Original device name is saved before renaming so it can be restored on BOT disable
- Device name written to device via BLE `set_name()` SDK call
- Graceful handling of BLE failures during name change
- ✅ **`BOT_DEVICE_NAME` constant** in `config.py` — Configurable fixed device name used when bot mode is active (default: `;NL-OV-ZWL-STDSHGN-WKC Bot`)
### Changed
- 🔄 `config.py`: Added `BOT_DEVICE_NAME` constant for bot mode device name
- 🔄 `bot.py`: Removed hardcoded `BOT_NAME` prefix ("Zwolle Bot") from bot reply messages — bot replies no longer include a name prefix
- 🔄 `filter_panel.py`: BOT checkbox toggle now triggers device name save/rename via command queue
- 🔄 `commands.py`: Added `set_bot_name` and `restore_name` command handlers for device name switching
- 🔄 `shared_data.py`: Added `original_device_name` field for storing the pre-bot device name
### Removed
- ❌ `BOT_NAME` constant from `bot.py` — bot reply prefix removed; replies no longer prepend a bot display name
---
## [1.2.0] - 2026-02-08 — Contact Maintenance Feature
### Added
- ✅ **Pin/Unpin contacts** (Iteration A) — Toggle to pin individual contacts, protecting them from bulk deletion
- Persistent pin state stored in `~/.meshcore-gui/cache/<ADDRESS>_pins.json`
- Pinned contacts visually marked with yellow background
- Pinned contacts sorted to top of contact list
- Pin state survives app restart
- New service: `services/pin_store.py` — JSON-backed persistent pin storage
- ✅ **Bulk delete unpinned contacts** (Iteration B) — Remove all unpinned contacts from device in one action
- "🧹 Clean up" button in contacts panel with confirmation dialog
- Shows count of contacts to be removed vs. pinned contacts kept
- Progress status updates during removal
- Automatic device resync after completion
- New service: `services/contact_cleaner.py` — ContactCleanerService with purge statistics
- ✅ **Auto-add contacts toggle** (Iteration C) — Control whether device automatically adds new contacts from mesh adverts
- "📥 Auto-add" checkbox in contacts panel (next to Clean up button)
- Syncs with device via `set_manual_add_contacts()` SDK call
- Inverted logic handled internally (UI "Auto-add ON" = `set_manual_add_contacts(false)`)
- Optimistic update with automatic rollback on BLE failure
- State synchronized from device on each GUI update cycle
### Changed
- 🔄 `contacts_panel.py`: Added pin checkbox per contact, purge button, auto-add toggle, DM dialog (all existing functionality preserved)
- 🔄 `commands.py`: Added `purge_unpinned` and `set_auto_add` command handlers
- 🔄 `shared_data.py`: Added `auto_add_enabled` field with thread-safe getter/setter
- 🔄 `protocols.py`: Added `set_auto_add_enabled` and `is_auto_add_enabled` to Writer and Reader protocols
- 🔄 `dashboard.py`: Passes `PinStore` and `set_auto_add_enabled` callback to ContactsPanel
- 🔄 **UI language**: All Dutch strings in `contacts_panel.py` and `commands.py` translated to English
---
### Fixed
- 🛠 **Route table names and IDs not displayed** — Route tables in both current messages (RoutePage) and archive messages (ArchivePage) now correctly show node names and public key IDs for sender, repeaters and receiver
### Changed
- 🔄 **CHANGELOG.md**: Corrected version numbering to semantic versioning, fixed inaccurate references (archive button location, filter state persistence)
- 🔄 **README.md**: Added Message Archive feature, updated project structure, configuration table and architecture diagram
- 🔄 **MeshCore_GUI_Design.docx**: Added ArchivePage, MessageArchive, Models components; updated project structure, protocols, configuration and version history
---
## [1.1.0] - 2026-02-07 — Archive Viewer Feature
### Added
- ✅ **Archive Viewer Page** (`/archive`) — Full-featured message archive browser
- Pagination (50 messages per page, configurable)
- Channel filter dropdown (All + configured channels)
- Time range filter (24h, 7d, 30d, 90d, All time)
- Text search (case-insensitive)
- Filter state stored in instance variables (reset on page reload)
- Message cards with same styling as main messages panel
- Clickable messages for route visualization (where available)
- **💬 Reply functionality** — Expandable reply panel per message
- **🗺️ Inline route table** — Expandable route display per archive message with sender, repeaters and receiver (names, IDs, node types)
- *(Note: Reply panels and inline route tables removed in v1.8.0, replaced by click-to-route navigation via message hash)*
<!-- CHANGED: "Filter state persistence (app.storage.user)" replaced with "Filter state stored in
instance variables" — the code (archive_page.py:36-40) uses self._current_page etc.,
not app.storage.user. The comment in the code is misleading. -->
<!-- ADDED: "Inline route table" entry — _render_archive_route() in archive_page.py:333-407
was not documented. -->
- ✅ **MessageArchive.query_messages()** method
- Filter by: time range, channel, text search, sender
- Pagination support (limit, offset)
- Returns tuple: (messages, total_count)
- Sorting: Newest first
- ✅ **UI Integration**
- "📚 Archive" button in Messages panel header (opens in new tab)
- Back to Dashboard button in archive page
<!-- CHANGED: "📚 View Archive button in Actions panel" corrected — the button is in
MessagesPanel (messages_panel.py:25), not in ActionsPanel (actions_panel.py).
ActionsPanel only contains Refresh and Advert buttons. -->
- ✅ **Reply Panel**
- Expandable reply per message (💬 Reply button)
- Pre-filled with @sender mention
- Channel selector
- Send button with success notification
- Auto-close expansion after send
### Changed
- 🔄 `SharedData.get_snapshot()`: Now includes `'archive'` field
- 🔄 `MessagesPanel`: Added archive button in header row
- 🔄 Both entry points (`__main__.py` and `meshcore_gui.py`): Register `/archive` route
<!-- CHANGED: "ActionsPanel: Added archive button" corrected to "MessagesPanel" -->
### Performance
- Query: ~10ms for 10k messages with filters
- Memory: ~10KB per page (50 messages)
- No impact on main UI (separate page)
### Known Limitations
- ~~Route visualization only works for messages in recent buffer (last 100)~~ — Fixed in v1.8.0: archive messages now support click-to-route via `get_message_by_hash()` fallback
- Text search is linear scan (no indexing yet)
- Sender filter exists in API but not in UI yet
---
## [1.0.3] - 2026-02-07 — Critical Bugfix: Archive Overwrite Prevention
### Fixed
- 🛠 **CRITICAL**: Fixed bug where archive was overwritten instead of appended on restart
- 🛠 Archive now preserves existing data when read errors occur
- 🛠 Buffer is retained for retry if existing archive cannot be read
### Changed
- 🔄 `_flush_messages()`: Early return on read error instead of overwriting
- 🔄 `_flush_rxlog()`: Early return on read error instead of overwriting
- 🔄 Better error messages for version mismatch and JSON decode errors
### Details
**Problem:** If the existing archive file had a JSON parse error or version mismatch,
the flush operation would proceed with `existing_messages = []`, effectively
overwriting all historical data with only the new buffered messages.
**Solution:** The flush methods now:
1. Try to read existing archive first
2. If read fails (JSON error, version mismatch, IO error), abort the flush
3. Keep buffer intact for next retry
4. Only clear buffer after successful write
**Impact:** No data loss on restart or when archive files have issues.
### Testing
- ✅ Added `test_append_on_restart_not_overwrite()` integration test
- ✅ Verifies data is appended across multiple sessions
- ✅ All existing tests still pass
---
## [1.0.2] - 2026-02-07 — RxLog message_hash Enhancement
### Added
- ✅ `message_hash` field added to `RxLogEntry` model
- ✅ RxLog entries now include message_hash for correlation with messages
- ✅ Archive JSON includes message_hash in rxlog entries
### Changed
- 🔄 `events.py`: Restructured `on_rx_log()` to extract message_hash before creating RxLogEntry
- 🔄 `message_archive.py`: Updated rxlog archiving to include message_hash field
- 🔄 Tests updated to verify message_hash persistence
### Benefits
- **Correlation**: Link RX log entries to their corresponding messages
- **Analysis**: Track which packets resulted in messages
- **Debugging**: Better troubleshooting of packet processing
---
## [1.0.1] - 2026-02-07 — Entry Point Fix
### Fixed
- ✅ `meshcore_gui.py` (root entry point) now passes ble_address to SharedData
- ✅ Archive works correctly regardless of how application is started
### Changed
- 🔄 Both entry points (`meshcore_gui.py` and `meshcore_gui/__main__.py`) updated
---
## [1.0.0] - 2026-02-07 — Message & Metadata Persistence
### Added
- ✅ MessageArchive class for persistent storage
- ✅ Configurable retention periods (MESSAGE_RETENTION_DAYS, RXLOG_RETENTION_DAYS, CONTACT_RETENTION_DAYS)
- ✅ Automatic daily cleanup of old data
- ✅ Batch writes for performance
- ✅ Thread-safe with separate locks
- ✅ Atomic file writes
- ✅ Contact retention in DeviceCache
- ✅ Archive statistics API
- ✅ Comprehensive tests (20+ unit, 8+ integration)
- ✅ Full documentation
### Storage Locations
- `~/.meshcore-gui/archive/<ADDRESS>_messages.json`
- `~/.meshcore-gui/archive/<ADDRESS>_rxlog.json`
### Requirements Completed
- R1: All incoming messages persistent ✅
- R2: All incoming RxLog entries persistent ✅
- R3: Configurable retention ✅
- R4: Automatic cleanup ✅
- R5: Backward compatibility ✅
- R6: Contact retention ✅
- R7: Archive stats API ✅
- Fix3: Leaflet asset injection is now per page render instead of process-global, and browser bootstrap now retries until the host element, Leaflet runtime, and MeshCore panel runtime are all available. This fixes blank map containers caused by missing or late-loaded JS/CSS assets.
- Fix5: Removed per-snapshot map invalidate calls, stopped forcing a default dark theme during map bootstrap, and added client-side interaction/resize guards so zooming stays responsive and the theme no longer jumps back during status-loop updates.
## 2026-03-09 map hotfix v2
- regular map snapshots no longer carry theme state
- explicit theme changes are now handled only via the dedicated theme channel
- initial map render now sends an ensure_map command plus an immediate theme sync
- added no-op ensure_map handling in the Leaflet runtime to avoid accidental fallback behaviour

32
docs/DEV_RULES.md Normal file
View File

@@ -0,0 +1,32 @@
### Map developer rules
To avoid regressions in the map subsystem, follow these rules:
**Do**
- Keep the Leaflet map lifecycle **inside the browser runtime**
- Initialize Leaflet **exactly once per DOM container**
- Send **compact snapshots only** from Python
- Update markers **incrementally by node id**
- Keep theme handling in a **dedicated theme channel**
- Allow the browser runtime to maintain **viewport state**
- Define map/tile `maxZoom` **before** attaching clustering layers
**Do NOT**
- Recreate the map inside the 500 ms dashboard update loop
- Call `L.map(...)` from snapshot handlers, timers or retry loops
- Use `ui.leaflet()` or any NiceGUI map wrapper
- Embed theme state inside snapshot payloads
- Force map center/zoom during normal refresh cycles
- Call Leaflet APIs directly from Python
- Place the device marker inside the contact cluster layer
Breaking these rules will reintroduce:
- disappearing maps
- marker flicker
- viewport resets
- theme resets
- cluster bootstrap failures
- `Map container is already initialized` errors

332
docs/FEATURE_MESSAGE_PERSISTENCE.md Normal file
View File

@@ -0,0 +1,332 @@
# Message & Metadata Persistence
**Version:** 1.0
**Author:** PE1HVH
**Date:** 2026-02-07
## Overview
This feature implements persistent storage for all incoming messages, RX log entries, and contacts with configurable retention periods. The system uses a dual-layer architecture to balance real-time UI performance with comprehensive data retention.
## Architecture
```
┌─────────────────────────────────────┐
│ SharedData (in-memory buffer) │
│ - Last 100 messages (UI) │
│ - Last 50 rx_log (UI) │
│ - Thread-safe via Lock │
└──────────────┬──────────────────────┘
│ (on every add)
▼
┌─────────────────────────────────────┐
│ MessageArchive (persistent) │
│ - All messages (JSON) │
│ - All rx_log (JSON) │
│ - Retention filtering │
│ - Automatic cleanup (daily) │
│ - Separate Lock (no contention) │
└─────────────────────────────────────┘
```
### Design Principles
1. **Separation of Concerns**: SharedData handles real-time UI updates, MessageArchive handles persistence
2. **Thread Safety**: Independent locks prevent contention between UI and archiving
3. **Batch Writes**: Buffered writes reduce disk I/O (flushes every 10 items or 60 seconds)
4. **Configurable Retention**: Automatic cleanup based on configurable periods
5. **Backward Compatibility**: SharedData API unchanged, archive is optional
## Storage Format
### Messages Archive
**Location:** `~/.meshcore-gui/archive/<ADDRESS>_messages.json`
```json
{
"version": 1,
"address": "literal:AA:BB:CC:DD:EE:FF",
"last_updated": "2026-02-07T12:34:56.123456Z",
"messages": [
{
"time": "12:34:56",
"timestamp_utc": "2026-02-07T12:34:56.123456Z",
"sender": "PE1HVH",
"text": "Hello mesh!",
"channel": 0,
"direction": "in",
"snr": 8.5,
"path_len": 2,
"sender_pubkey": "abc123...",
"path_hashes": ["a1", "b2"],
"message_hash": "def456..."
}
]
}
```
### RX Log Archive
**Location:** `~/.meshcore-gui/archive/<ADDRESS>_rxlog.json`
```json
{
"version": 1,
"address": "literal:AA:BB:CC:DD:EE:FF",
"last_updated": "2026-02-07T12:34:56Z",
"entries": [
{
"time": "12:34:56",
"timestamp_utc": "2026-02-07T12:34:56Z",
"snr": 8.5,
"rssi": -95.0,
"payload_type": "MSG",
"hops": 2,
"message_hash": "def456..."
}
]
}
```
**Note:** The `message_hash` field enables correlation between RX log entries and messages. It will be empty for packets that are not messages (e.g., announcements, broadcasts).
## Configuration
Add to `meshcore_gui/config.py`:
```python
# Retention period for archived messages (in days)
MESSAGE_RETENTION_DAYS: int = 30
# Retention period for RX log entries (in days)
RXLOG_RETENTION_DAYS: int = 7
# Retention period for contacts (in days)
CONTACT_RETENTION_DAYS: int = 90
```
## Usage
### Basic Usage
The archive is automatically initialized when SharedData is created with a device identifier (serial port):
```python
from meshcore_gui.core.shared_data import SharedData
# With archive (normal use)
shared = SharedData("literal:AA:BB:CC:DD:EE:FF")
# Without archive (backward compatible)
shared = SharedData() # archive will be None
```
### Adding Data
All data added to SharedData is automatically archived:
```python
from meshcore_gui.core.models import Message, RxLogEntry
# Add message (goes to both SharedData and archive)
msg = Message(
time="12:34:56",
sender="PE1HVH",
text="Hello!",
channel=0,
direction="in",
)
shared.add_message(msg)
# Add RX log entry (goes to both SharedData and archive)
entry = RxLogEntry(
time="12:34:56",
snr=8.5,
rssi=-95.0,
payload_type="MSG",
hops=2,
)
shared.add_rx_log(entry)
```
### Getting Statistics
```python
# Get archive statistics
stats = shared.get_archive_stats()
if stats:
print(f"Total messages: {stats['total_messages']}")
print(f"Total RX log: {stats['total_rxlog']}")
print(f"Pending writes: {stats['pending_messages']}")
```
### Manual Flush
Archive writes are normally batched. To force immediate write:
```python
if shared.archive:
shared.archive.flush()
```
### Manual Cleanup
Cleanup runs automatically daily, but can be triggered manually:
```python
if shared.archive:
shared.archive.cleanup_old_data()
```
## Performance Characteristics
### Write Performance
- Batch writes: 10 messages or 60 seconds (whichever comes first)
- Write time: ~10ms for 1000 messages
- Memory overhead: Minimal (only buffer in memory, ~10 messages)
### Startup Performance
- Archive loading: <500ms for 10,000 messages
- Archive is counted, not loaded into memory
- No impact on UI responsiveness
### Storage Size
With default retention (30 days messages, 7 days rxlog):
- Typical message: ~200 bytes JSON
- 100 messages/day → ~6KB/day → ~180KB/month
- Expected archive size: <10MB
## Automatic Cleanup
The worker runs cleanup daily (every 86400 seconds):
1. **Message Cleanup**: Removes messages older than `MESSAGE_RETENTION_DAYS`
2. **RxLog Cleanup**: Removes entries older than `RXLOG_RETENTION_DAYS`
3. **Contact Cleanup**: Removes contacts not seen for `CONTACT_RETENTION_DAYS`
Cleanup is non-blocking and runs in the background worker thread.
## Thread Safety
### Lock Ordering
1. SharedData acquires its lock
2. SharedData calls MessageArchive methods
3. MessageArchive acquires its own lock
This ordering prevents deadlocks.
### Concurrent Access
- SharedData lock: Protects in-memory buffers
- MessageArchive lock: Protects file writes and batch buffers
- Independent locks prevent contention
## Error Handling
### Disk Write Failures
- Atomic writes using temp file + rename
- If write fails: buffer retained for retry
- Logged to debug output
- Application continues normally
### Corrupt Archives
- Version checking on load
- Invalid JSON → skip and start fresh
- Corrupted data → logged, not loaded
### Missing Directory
- Archive directory created automatically
- Parent directories created if needed
## Testing
### Unit Tests
```bash
python -m unittest tests.test_message_archive
```
Tests cover:
- Message and RxLog archiving
- Batch write behavior
- Retention cleanup
- Thread safety
- JSON serialization
### Integration Tests
```bash
python -m unittest tests.test_integration_archive
```
Tests cover:
- SharedData + Archive flow
- Buffer limits with archiving
- Persistence across restarts
- Backward compatibility
### Running All Tests
```bash
python -m unittest discover tests
```
## Migration Guide
### From v5.1 to v5.2
No migration needed! The feature is fully backward compatible:
1. Existing SharedData code works unchanged
2. Archive is optional (requires device identifier)
3. First run creates archive files automatically
4. No data loss from existing cache
### Upgrading Existing Installation
```bash
# No special steps needed
python meshcore_gui.py literal:AA:BB:CC:DD:EE:FF
```
Archive files will be created automatically on first message/rxlog.
## Future Enhancements (Out of Scope for v1.0)
- Full-text search in archive
- Export to CSV/JSON
- Compression of old messages
- Cloud sync / multi-device sync
- Web interface for archive browsing
- Advanced filtering and queries
## Troubleshooting
### Archive Not Created
**Problem:** No `~/.meshcore-gui/archive/` directory
**Solution:**
- Check that SharedData was initialized with device identifier
- Check disk permissions
- Enable debug mode: `--debug-on`
### Cleanup Not Running
**Problem:** Old messages not removed
**Solution:**
- Cleanup runs every 24 hours
- Manually trigger: `shared.archive.cleanup_old_data()`
- Check retention config values
### High Disk Usage
**Problem:** Archive files growing too large
**Solution:**
- Reduce `MESSAGE_RETENTION_DAYS` in config
- Run manual cleanup
- Check for misconfigured retention values
## Support
For issues or questions:
- GitHub: [PE1HVH/meshcore-gui](https://github.com/PE1HVH/meshcore-gui)
- Email: pe1hvh@example.com
## License
MIT License - Copyright (c) 2026 PE1HVH

165
docs/INSTALLATIE.md Normal file
View File

@@ -0,0 +1,165 @@
# MeshCore GUI — BLE Stabiliteit: Installatie-instructies (Legacy)
> **Let op:** Dit document is BLE-specifiek en wordt bewaard als referentie. De huidige GUI gebruikt USB-serieel; gebruik het handmatige systeemd-voorbeeld in de README.
## Wat is gewijzigd
### Nieuwe bestanden
| Bestand | Doel |
|---------|------|
| `meshcore_gui/ble/ble_agent.py` | Ingebouwde BlueZ D-Bus PIN agent (vervangt `bt-agent.service`) |
| `meshcore_gui/ble/ble_reconnect.py` | Bond-opruiming + automatische reconnect logica |
| `install_ble_stable.sh` | Generiek installatiescript (detecteert paden/user automatisch) |
### Gewijzigde bestanden
| Bestand | Wijziging |
|---------|-----------|
| `meshcore_gui/ble/worker.py` | Agent startup, disconnect detectie, auto-reconnect loop |
| `meshcore_gui/config.py` | Nieuwe constanten: `BLE_PIN`, `RECONNECT_MAX_RETRIES`, `RECONNECT_BASE_DELAY` |
---
## Snelle installatie (aanbevolen)
```bash
# 1. Verwijder eerst een eventuele kapotte service
sudo systemctl stop meshcore-gui 2>/dev/null
sudo systemctl disable meshcore-gui 2>/dev/null
sudo rm -f /etc/systemd/system/meshcore-gui.service
sudo systemctl daemon-reload
sudo systemctl reset-failed 2>/dev/null
# 2. Kopieer de nieuwe/gewijzigde bestanden naar je project
cp ble_agent.py ~/meshcore-gui/meshcore_gui/ble/
cp ble_reconnect.py ~/meshcore-gui/meshcore_gui/ble/
cp worker.py ~/meshcore-gui/meshcore_gui/ble/
cp config.py ~/meshcore-gui/meshcore_gui/
# 3. Ga naar je project directory en voer het installatiescript uit
cd ~/meshcore-gui
BLE_ADDRESS=FF:05:D6:71:83:8D bash install_ble_stable.sh
```
Het script detecteert automatisch:
- De juiste project directory (waar je het uitvoert)
- De huidige user
- Het pad naar de venv Python
- Het correcte entry point
---
## Handmatige installatie
Als je het script niet wilt gebruiken:
### 1. Kopieer Python bestanden
```bash
# Pas het pad aan naar jouw project directory
PROJECT=~/meshcore-gui
cp ble_agent.py $PROJECT/meshcore_gui/ble/
cp ble_reconnect.py $PROJECT/meshcore_gui/ble/
cp worker.py $PROJECT/meshcore_gui/ble/
cp config.py $PROJECT/meshcore_gui/
```
### 2. Upgrade meshcore library
```bash
cd $PROJECT
source venv/bin/activate
pip install --upgrade meshcore
```
### 3. D-Bus policy installeren
Maak `/etc/dbus-1/system.d/meshcore-ble.conf` met je eigen username:
```bash
sudo tee /etc/dbus-1/system.d/meshcore-ble.conf << 'EOF'
<!DOCTYPE busconfig PUBLIC "-//freedesktop//DTD D-BUS Bus Configuration 1.0//EN"
"http://www.freedesktop.org/standards/dbus/1.0/busconfig.dtd">
<busconfig>
<policy user="JOUW_USERNAME">
<allow send_destination="org.bluez"/>
<allow send_interface="org.bluez.Agent1"/>
<allow send_interface="org.bluez.AgentManager1"/>
</policy>
</busconfig>
EOF
```
### 4. Systemd service installeren
Maak `/etc/systemd/system/meshcore-gui.service` met je eigen paden:
```bash
sudo tee /etc/systemd/system/meshcore-gui.service << EOF
[Unit]
Description=MeshCore GUI (BLE)
After=bluetooth.target
Wants=bluetooth.target
[Service]
Type=simple
User=$(whoami)
WorkingDirectory=$PROJECT
ExecStart=$PROJECT/venv/bin/python meshcore_gui.py JOUW_BLE_ADRES --debug-on
Restart=on-failure
RestartSec=30
Environment=DBUS_SYSTEM_BUS_ADDRESS=unix:path=/var/run/dbus/system_bus_socket
[Install]
WantedBy=multi-user.target
EOF
sudo systemctl daemon-reload
sudo systemctl enable meshcore-gui
sudo systemctl start meshcore-gui
```
---
## Verwijderen
### Via het script
```bash
cd ~/meshcore-gui
bash install_ble_stable.sh --uninstall
```
### Handmatig
```bash
sudo systemctl stop meshcore-gui
sudo systemctl disable meshcore-gui
sudo rm -f /etc/systemd/system/meshcore-gui.service
sudo rm -f /etc/dbus-1/system.d/meshcore-ble.conf
sudo systemctl daemon-reload
sudo systemctl reset-failed
```
---
## Verificatie
```bash
# Service status
sudo systemctl status meshcore-gui
# Live logs
journalctl -u meshcore-gui -f
# Test PIN pairing (vanuit een andere terminal)
bluetoothctl remove <BLE_ADRES>
sudo systemctl restart meshcore-gui
# Test disconnect recovery
# Zet device uit → wacht 30s → zet weer aan → check logs
```
---
## Configuratie (config.py)
```python
BLE_PIN = "123456" # T1000e pairing PIN
RECONNECT_MAX_RETRIES = 5 # Max pogingen per disconnect
RECONNECT_BASE_DELAY = 5.0 # Wachttijd × poging nummer (5s, 10s, 15s...)
```
Pas deze waarden aan in `meshcore_gui/config.py` als je een ander device of andere timing nodig hebt.

108
docs/INTEGRATION_GUIDE.md Normal file
View File

@@ -0,0 +1,108 @@
# v5.5 Integration Guide — Subprocess BLE Connection
## Overzicht
Deze wijziging lost het BlueZ 5.82 probleem op door `meshcore-ble-connect`
als **persistent subprocess** te draaien dat de D-Bus/BLE connectie openhoudt,
terwijl `BleakClient` er alleen GATT service discovery overheen doet.
---
## Gewijzigde bestanden
### meshcore-ble-connect (5 bestanden)
| Bestand | Wijziging |
|---|---|
| `constants.py` | Versie → 1.1.0. Nieuwe constanten: `SERVICES_RESOLVED_TIMEOUT`, `DISCONNECT_POLL_INTERVAL`. Nieuwe exit codes: `CONNECT_FAILED` (5), `DISCONNECTED` (6). |
| `exceptions.py` | Nieuwe exception: `ConnectHoldError`. |
| `__main__.py` | Nieuw `--connect` flag. Mutual exclusion met `--check-only`. Doorgifte `connect_hold=` aan `BleConnectApp`. |
| `app.py` | Nieuw `connect_hold` parameter. Na bond OK → `_enter_connect_hold()` → `device.connect_and_hold()`. Werkt bij zowel bestaande bond als verse pairing. |
| `device.py` | Nieuwe methoden: `connect_and_hold()`, `is_connected()`, `is_services_resolved()`, `_wait_for_services_resolved()`, `_monitor_connection()`. Signal handling (SIGTERM/SIGINT) voor clean shutdown. |
### meshcore-gui (1 bestand)
| Bestand | Wijziging |
|---|---|
| `worker.py` | Nieuwe methoden: `_connect_via_subprocess()`, `_kill_connect_subprocess()`. Gewijzigd: `_connect()` gebruikt subprocess als primary path wanneer `_use_ble_connect=True`. Subprocess health check in main loop. Cleanup in finally block. |
---
## Architectuur
```
┌─────────────────────────────────────────────┐
│ meshcore-gui (worker.py) │
│ │
│ 1. ensure_bond() → bond OK │
│ 2. start subprocess: │
│ meshcore-ble-connect MAC --pin X │
│ --connect │
│ 3. wait for "READY" op stdout │
│ 4. BleakScanner.find_device_by_address() │
│ → populeert bleak's interne cache │
│ 5. BleakClient(addr).connect() │
│ → bleak ziet Connected=True in BlueZ │
│ → slaat Device1.Connect() over │
│ → doet alleen GATT service discovery │
│ 6. MeshCore.create_ble(client=client) │
│ 7. Main loop draait │
│ 8. subprocess health check elke 100ms │
└──────────────┬──────────────────────────────┘
│ subprocess (stdout PIPE)
│
┌──────────────▼──────────────────────────────┐
│ meshcore-ble-connect --connect │
│ │
│ 1. Bond flow (ensure/pair/trust) │
│ 2. Device1.Connect() via D-Bus │
│ 3. Poll ServicesResolved tot True │
│ 4. print("READY") → stdout │
│ 5. Monitor Connected property │
│ → print("DISCONNECTED") bij verlies │
│ 6. Wacht op SIGTERM of disconnect │
│ 7. Device1.Disconnect() bij shutdown │
└─────────────────────────────────────────────┘
```
## Installatie
```bash
# 1. Update meshcore-ble-connect
cd ~/meshcore-ble-connect
# Vervang de 5 gewijzigde bestanden in meshcore_ble_connect/
pip install -e . --break-system-packages
# 2. Test --connect mode standalone
meshcore-ble-connect FF:05:D6:71:83:8D --pin 123456 --connect --verbose
# Verwacht: READY op stdout, proces blijft draaien
# Ctrl+C om te stoppen
# 3. Update worker.py
cp worker.py ~/meshcore-gui/meshcore_gui/ble/worker.py
# 4. Start meshcore-gui
cd ~/meshcore-gui && python -m meshcore_gui
```
## Fallback gedrag
Als het subprocess faalt (bijv. op BlueZ < 5.78 waar het niet nodig is),
valt `_connect()` automatisch terug op de directe `MeshCore.create_ble(address)`
aanroep. Dit garandeert backwards compatibility.
## Risico-mitigatie
Het document noemde het risico dat bleak mogelijk opnieuw `Device1.Connect()`
aanroept. Dit is opgelost door:
1. **`BleakScanner.find_device_by_address()`** vóór `BleakClient.connect()` —
dit triggert bleak's `BlueZManager` singleton om bestaande BlueZ device
objecten te ontdekken via `GetManagedObjects()`.
2. De manager ziet `Connected=True` op het device (gezet door het subprocess)
→ `BleakClient.connect()` slaat `Device1.Connect()` over.
3. Bleak doet alleen GATT service resolution over de bestaande connectie.
Als de scanner het device niet vindt (het adverteert mogelijk niet terwijl
het connected is), probeert bleak alsnog. De `GetManagedObjects` call bij
manager initialisatie vangt dit op in de meeste gevallen.

225
docs/MAP_ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,225 @@
# Map Architecture — MeshCore GUI
## Overview
The MeshCore GUI map subsystem is implemented as a **browser-managed Leaflet runtime** embedded inside a NiceGUI container.
The key design decision is that the **map lifecycle is owned by the browser**, not by the Python UI update loop.
NiceGUI acts only as a container and data provider.
This architecture prevents map resets, marker flicker, and viewport jumps during the 500 ms dashboard refresh cycle.
---
# Architecture
```
NiceGUI Dashboard
│
│ snapshot (500 ms)
▼
MapPanel (Python)
│
│ JSON payload
▼
Leaflet Runtime (Browser)
│
├─ Map instance (persistent)
├─ Marker registry
├─ Theme state
└─ Viewport state
```
---
# Component Responsibilities
## MapPanel (Python)
Location:
```
meshcore_gui/gui/panels/map_panel.py
```
Responsibilities:
* provides the map container
* injects the Leaflet runtime assets
* sends compact map snapshots
* handles UI actions:
* theme toggle
* center on device
MapPanel **does NOT control the Leaflet map directly**.
It only sends data.
---
## MapSnapshotService
Location:
```
meshcore_gui/services/map_snapshot_service.py
```
Responsibilities:
* converts device/contact data into a compact JSON snapshot
* ensures stable node identifiers
* prepares payloads for the browser runtime
Example snapshot structure:
```json
{
"device": {...},
"contacts": [...],
"force_center": false
}
```
Snapshots are emitted every **500 ms** by the dashboard update loop.
---
## Leaflet Runtime
Location:
```
meshcore_gui/static/leaflet_map_panel.js
```
Responsibilities:
* initialize the Leaflet map once
* maintain persistent map instance
* manage marker registry
* apply snapshots incrementally
* manage map theme and viewport state
Key design rules:
```
map is created once
markers updated incrementally
snapshots never recreate the map
```
---
# Update Flow
```
SharedData
│
▼
Dashboard update loop (500 ms)
│
▼
MapSnapshotService
│
▼
MapPanel
│
▼
Leaflet Runtime
```
Snapshots are **coalesced** so the browser applies only the newest payload.
---
# Theme Handling
Theme changes are handled via a **dedicated theme channel**.
Snapshots do **not** carry theme information.
Reason:
Embedding theme state in snapshots caused race conditions where queued snapshots overwrote explicit user selections.
Theme state is managed in the browser runtime and restored on reconnect.
---
# Marker Model
Markers are keyed by **stable node id**.
```
device marker
contact markers
```
Updates are applied incrementally:
```
add marker
update marker
remove marker
```
This prevents marker flicker during the refresh loop.
---
# Important Constraints
Developers must **not**:
* recreate the Leaflet map inside the dashboard refresh loop
* embed theme state in snapshots
* call Leaflet APIs directly from Python
* force viewport resets during normal snapshot updates
Violating these rules will reintroduce:
* disappearing maps
* marker flicker
* viewport resets
* theme resets
---
# Reconnect Behaviour
When the NiceGUI connection temporarily drops:
1. the Leaflet runtime persists in the browser
2. the map instance remains intact
3. theme and viewport state are restored
4. snapshot updates resume once the connection returns
---
# Future Extensions
Possible improvements without breaking the architecture:
* marker clustering
* heatmap layers
* route overlays
* tile provider switching
All extensions must remain **browser-managed**.
---
# Summary
The MeshCore map subsystem follows a strict separation:
```
Python → data
Browser → map lifecycle
```
This prevents UI refresh cycles from interfering with map state and ensures smooth rendering even with frequent dashboard updates.

131
docs/MULTI_INSTANCE.md Normal file
View File

@@ -0,0 +1,131 @@
# Running Multiple MeshCore GUI Instances
> ⚠️ **WARNING: This guide has not been tested yet.** The configuration below is based on the application's architecture and should work, but has not been validated in practice. Please report any issues.
## Overview
MeshCore GUI supports running multiple instances simultaneously — for example, to monitor two different MeshCore devices from the same machine. Each instance gets its own web port, serial connection, and all persistent data (cache, archive, logs, pins, room passwords) is automatically separated by device identifier (serial port).
## Prerequisites
- MeshCore GUI v1.9.2 or later (with `--port` and serial CLI parameters)
## Quick Test (foreground)
Before creating services, verify that both instances start correctly:
**Terminal 1:**
```bash
cd ~/meshcore-gui
source venv/bin/activate
python meshcore_gui.py /dev/ttyUSB0 --debug-on --port=8081 --baud=115200
```
**Terminal 2:**
```bash
cd ~/meshcore-gui
source venv/bin/activate
python meshcore_gui.py /dev/ttyUSB1 --debug-on --port=8082 --baud=115200
```
Verify both are accessible at `http://localhost:8081` and `http://localhost:8082`.
## systemd Service Setup
### Service 1
```bash
sudo nano /etc/systemd/system/meshcore-gui-device1.service
```
```ini
[Unit]
Description=MeshCore GUI — Device 1 (/dev/ttyUSB0)
[Service]
Type=simple
User=your-username
WorkingDirectory=/home/your-username/meshcore-gui
ExecStart=/home/your-username/meshcore-gui/venv/bin/python meshcore_gui.py /dev/ttyUSB0 --debug-on --port=8081 --baud=115200
Restart=on-failure
RestartSec=30
[Install]
WantedBy=multi-user.target
```
### Service 2
```bash
sudo nano /etc/systemd/system/meshcore-gui-device2.service
```
```ini
[Unit]
Description=MeshCore GUI — Device 2 (/dev/ttyUSB1)
[Service]
Type=simple
User=your-username
WorkingDirectory=/home/your-username/meshcore-gui
ExecStart=/home/your-username/meshcore-gui/venv/bin/python meshcore_gui.py /dev/ttyUSB1 --debug-on --port=8082 --baud=115200
Restart=on-failure
RestartSec=30
[Install]
WantedBy=multi-user.target
```
Replace `your-username` and serial ports with your actual values.
### Enable and Start
```bash
sudo systemctl daemon-reload
sudo systemctl enable meshcore-gui-device1 meshcore-gui-device2
sudo systemctl start meshcore-gui-device1
sudo systemctl start meshcore-gui-device2
```
## Data Separation
All persistent data is automatically separated by device identifier. No additional configuration is needed.
| Data | Path example (device `/dev/ttyUSB0`) |
|------|------------------------------------------|
| Web interface | `http://host:8081` (via `--port`) |
| Cache | `~/.meshcore-gui/cache/_dev_ttyUSB0.json` |
| Message archive | `~/.meshcore-gui/archive/_dev_ttyUSB0_messages.json` |
| RX log archive | `~/.meshcore-gui/archive/_dev_ttyUSB0_rxlog.json` |
| Debug log | `~/.meshcore-gui/logs/_dev_ttyUSB0_meshcore_gui.log` |
| Pin state | `~/.meshcore-gui/pins/_dev_ttyUSB0_pins.json` |
| Room passwords | `~/.meshcore-gui/room_passwords/_dev_ttyUSB0_rooms.json` |
## Useful Commands
| Command | Description |
|---------|-------------|
| `sudo systemctl status meshcore-gui-device1` | Check status of device 1 |
| `sudo systemctl status meshcore-gui-device2` | Check status of device 2 |
| `sudo journalctl -u meshcore-gui-device1 -f` | Follow live log of device 1 |
| `sudo journalctl -u meshcore-gui-device2 -f` | Follow live log of device 2 |
| `sudo systemctl restart meshcore-gui-device1` | Restart device 1 (without affecting device 2) |
| `sudo systemctl stop meshcore-gui-device1` | Stop device 1 only |
| `sudo systemctl disable meshcore-gui-device1` | Prevent device 1 from starting on boot |
## Removing a Service
```bash
sudo systemctl stop meshcore-gui-device2
sudo systemctl disable meshcore-gui-device2
sudo rm /etc/systemd/system/meshcore-gui-device2.service
sudo systemctl daemon-reload
```
Optionally remove the device's persistent data:
```bash
rm ~/.meshcore-gui/cache/_dev_ttyUSB1.json
rm ~/.meshcore-gui/archive/_dev_ttyUSB1_*.json
rm ~/.meshcore-gui/logs/_dev_ttyUSB1_meshcore_gui.log
rm ~/.meshcore-gui/pins/_dev_ttyUSB1_pins.json
rm ~/.meshcore-gui/room_passwords/_dev_ttyUSB1_rooms.json
```

BIN
docs/MeshCore_GUI_Design.docx Normal file
View File

Binary file not shown.

219
docs/SOLID_ANALYSIS.md Normal file
View File

@@ -0,0 +1,219 @@
# SOLID Analysis — MeshCore GUI
## 1. Reference: standard Python OOP project conventions
| Convention | Norm | This project |
|-----------|------|-------------|
| Package with subpackage when widgets emerge | ✅ | ✅ `widgets/` subpackage (6 classes) |
| One class per module | ✅ | ✅ every module ≤1 class |
| Entry point outside package | ✅ | ✅ `meshcore_gui.py` beside package |
| `__init__.py` with version | ✅ | ✅ only `__version__` |
| Constants in own module | ✅ | ✅ `config.py` |
| No circular imports | ✅ | ✅ acyclic dependency tree |
| Type hints on public API | ✅ | ✅ 84/84 methods typed |
| Private methods with `_` prefix | ✅ | ✅ consistent |
| Docstrings on modules and classes | ✅ | ✅ present everywhere |
| PEP 8 import order | ✅ | ✅ stdlib → third-party → local |
### Dependency tree (acyclic)
```
config protocols
↑ ↑
shared_data worker
↑ main_page → widgets/*
↑ route_builder ← route_page
↑
meshcore_gui.py (only place that knows the concrete SharedData)
```
No circular dependencies. `config` and `protocols` are leaf nodes; everything points in one direction. Widgets depend only on `config` (for constants) and NiceGUI — they have zero knowledge of SharedData or protocols.
---
## 2. SOLID assessment per principle
### S — Single Responsibility Principle
> "A class should have only one reason to change."
| Module | Class | Responsibility | Verdict |
|--------|-------|---------------|---------|
| `config.py` | *(no class)* | Constants and debug helper | ✅ Single purpose |
| `protocols.py` | *(Protocol classes)* | Interface contracts | ✅ Single purpose |
| `shared_data.py` | SharedData | Thread-safe data store | ✅ See note |
| `ble/worker.py` | SerialWorker | Serial communication thread | ✅ Single purpose |
| `main_page.py` | DashboardPage | Dashboard layout orchestrator | ✅ See note |
| `route_builder.py` | RouteBuilder | Route data construction (pure logic) | ✅ Single purpose |
| `route_page.py` | RoutePage | Route page rendering | ✅ Single purpose |
| `widgets/device_panel.py` | DevicePanel | Header, device info, actions | ✅ Single purpose |
| `widgets/map_panel.py` | MapPanel | Leaflet map with markers | ✅ Single purpose |
| `widgets/contacts_panel.py` | ContactsPanel | Contacts list + DM dialog | ✅ Single purpose |
| `widgets/message_input.py` | MessageInput | Message input + channel select | ✅ Single purpose |
| `widgets/message_list.py` | MessageList | Message feed + channel filter | ✅ Single purpose |
| `widgets/rx_log_panel.py` | RxLogPanel | RX log table | ✅ Single purpose |
**SharedData:** 15 public methods in 5 categories (device updates, status, collections, snapshots, lookups). This is deliberate design: SharedData is the single source of truth between two threads. Splitting it would spread lock logic across multiple objects, making thread-safety harder. The responsibility is *"thread-safe data access"* — that is one reason to change.
**DashboardPage:** After the widget decomposition, DashboardPage is now 148 lines with only 4 methods. It is a thin orchestrator that composes six widgets into a layout and drives the update timer. All rendering and data-update logic has been extracted into the widget classes. The previous ⚠️ for DashboardPage is resolved.
**Conclusion SRP:** No violations. All classes have a single, well-defined responsibility.
---
### O — Open/Closed Principle
> "Open for extension, closed for modification."
| Scenario | How to extend | Existing code modified? |
|----------|--------------|------------------------|
| Add new page | New module + `@ui.page` in entry point | Only entry point (1 line) |
| Add new command | `_handle_command()` case | Only `ble/worker.py` |
| Add new contact type | `TYPE_ICONS/NAMES/LABELS` in config | Only `config.py` |
| Add new dashboard widget | New widget class + compose in DashboardPage | Only `main_page.py` |
| Add new route info | Extend RouteBuilder.build() | Only `route_builder.py` |
**Where not ideal:** `_handle_command()` in SerialWorker is an if/elif chain. In a larger project, a Command pattern or dict-dispatch would be more appropriate. For 4 commands this is pragmatically correct.
**Conclusion OCP:** Good. Extensions touch only one module.
---
### L — Liskov Substitution Principle
> "Subtypes must be substitutable for their base types."
There is **no inheritance** in this project. All classes are concrete and standalone. This is correct for the project scale — there is no reason for a class hierarchy.
**Where LSP does apply:** The Protocol interfaces (`SharedDataWriter`, `SharedDataReader`, `ContactLookup`, `SharedDataReadAndLookup`) define contracts that SharedData implements. Any object that satisfies these protocols can be substituted — for example a test stub. This is LSP via structural subtyping.
**Conclusion LSP:** Satisfied via Protocol interfaces. No violations.
---
### I — Interface Segregation Principle
> "Clients should not be forced to depend on interfaces they do not use."
| Client | Protocol | Methods visible | SharedData methods not visible |
|--------|----------|----------------|-------------------------------|
| SerialWorker | SharedDataWriter | 10 | 5 (snapshot, flags, GUI commands) |
| DashboardPage | SharedDataReader | 4 | 11 (all write methods) |
| RouteBuilder | ContactLookup | 1 | 14 (everything else) |
| RoutePage | SharedDataReadAndLookup | 5 | 10 (all write methods) |
| Widget classes | *(none — receive Dict/callback)* | 0 | 15 (all methods) |
Each consumer sees **only the methods it needs**. The protocols enforce this at the type level. Widget classes go even further: they have zero knowledge of SharedData and receive only plain dictionaries and callbacks.
**Conclusion ISP:** Satisfied. Each consumer depends on a narrow, purpose-built interface.
---
### D — Dependency Inversion Principle
> "Depend on abstractions, not on concretions."
| Dependency | Before (protocols) | After (protocols) |
|-----------|---------------|---------------|
| SerialWorker → SharedData | Concrete ⚠️ | Protocol (SharedDataWriter) ✅ |
| DashboardPage → SharedData | Concrete ⚠️ | Protocol (SharedDataReader) ✅ |
| RouteBuilder → SharedData | Concrete ⚠️ | Protocol (ContactLookup) ✅ |
| RoutePage → SharedData | Concrete ⚠️ | Protocol (SharedDataReadAndLookup) ✅ |
| Widget classes → SharedData | N/A | No dependency at all ✅ |
| meshcore_gui.py → SharedData | Concrete | Concrete ✅ (composition root) |
The **composition root** (`meshcore_gui.py`) is the only place that knows the concrete `SharedData` class. All other modules depend on protocols or receive plain data. This is standard DIP practice: the wiring layer knows the concretions, the business logic knows only abstractions.
**Conclusion DIP:** Satisfied. Constructor injection was already present; now the abstractions are explicit.
---
## 3. Protocol interface design
### Why `typing.Protocol` and not `abc.ABC`?
Python offers two approaches for defining interfaces:
| Aspect | `abc.ABC` (nominal) | `typing.Protocol` (structural) |
|--------|---------------------|-------------------------------|
| Subclassing required | Yes (`class Foo(MyABC)`) | No |
| Duck typing compatible | No | Yes |
| Runtime checkable | Yes | Optional (`@runtime_checkable`) |
| Python version | 3.0+ | 3.8+ |
Protocol was chosen because SharedData does not need to inherit from an abstract base class. Any object that has the right methods automatically satisfies the protocol — this is idiomatic Python (duck typing with type safety).
### Interface map
```
SharedDataWriter (SerialWorker)
├── update_from_appstart()
├── update_from_device_query()
├── set_status()
├── set_connected()
├── set_contacts()
├── set_channels()
├── add_message()
├── add_rx_log()
├── get_next_command()
└── get_contact_name_by_prefix()
SharedDataReader (DashboardPage)
├── get_snapshot()
├── clear_update_flags()
├── mark_gui_initialized()
└── put_command()
ContactLookup (RouteBuilder)
└── get_contact_by_prefix()
SharedDataReadAndLookup (RoutePage)
├── get_snapshot()
├── clear_update_flags()
├── mark_gui_initialized()
├── put_command()
└── get_contact_by_prefix()
```
---
## 4. Summary
| Principle | Before protocols | With protocols | With widgets | Change |
|----------|-----------------|----------------|--------------|--------|
| **SRP** | ✅ Good | ✅ Good | ✅ Good | Widget extraction resolved DashboardPage size |
| **OCP** | ✅ Good | ✅ Good | ✅ Good | Widgets are easy to add |
| **LSP** | ✅ N/A | ✅ Satisfied via Protocol | ✅ Satisfied via Protocol | — |
| **ISP** | ⚠️ Acceptable | ✅ Good | ✅ Good | Widgets have zero SharedData dependency |
| **DIP** | ⚠️ Acceptable | ✅ Good | ✅ Good | — |
### Changes: Protocol interfaces
| # | Change | Files affected |
|---|--------|---------------|
| 1 | Added `protocols.py` with 4 Protocol interfaces | New file |
| 2 | SerialWorker depends on `SharedDataWriter` | `ble/worker.py` |
| 3 | DashboardPage depends on `SharedDataReader` | `main_page.py` |
| 4 | RouteBuilder depends on `ContactLookup` | `route_builder.py` |
| 5 | RoutePage depends on `SharedDataReadAndLookup` | `route_page.py` |
| 6 | No consumer imports `shared_data.py` directly | All consumer modules |
### Changes: Widget decomposition
| # | Change | Files affected |
|---|--------|---------------|
| 1 | Added `widgets/` subpackage with 6 widget classes | New directory (7 files) |
| 2 | MeshCoreGUI (740 lines) replaced by DashboardPage (148 lines) + 6 widgets | `main_page.py`, `widgets/*.py` |
| 3 | DashboardPage is now a thin orchestrator | `main_page.py` |
| 4 | Widget classes depend only on `config` and NiceGUI | `widgets/*.py` |
| 5 | Maximum decoupling: widgets have zero SharedData knowledge | All widget modules |
### Metrics
| Metric | Monolith | With protocols | With widgets |
|--------|----------|----------------|--------------|
| Files | 1 | 8 | 16 |
| Total lines | 1,395 | ~1,500 | ~1,955 |
| Largest class (lines) | MeshCoreGUI (740) | MeshCoreGUI (740) | SharedData (263) |
| Typed methods | 51 (partial) | 51 (partial) | 90/90 |
| Protocol interfaces | 0 | 4 | 4 |

424
docs/TROUBLESHOOTING.md Normal file
View File

@@ -0,0 +1,424 @@
# MeshCore GUI - Legacy BLE Troubleshooting Guide
> **Note:** This guide applies to BLE connections only and is kept for historical reference. The current GUI uses USB serial; for serial issues, verify the correct port (e.g. `/dev/ttyUSB0`) and user permissions (e.g. `dialout` on Linux).
## Problem 1: EOFError during start_notify
BLE connection to MeshCore device fails with `EOFError` during `start_notify` on the UART TX characteristic. The error originates in `dbus_fast` (the D-Bus library used by `bleak`) and looks like this:
```
File "src/dbus_fast/_private/unmarshaller.py", line 395, in dbus_fast._private.unmarshaller.Unmarshaller._read_sock_with_fds
EOFError
```
Basic BLE connect works fine, but subscribing to notifications (`start_notify`) crashes.
## Problem 2: PIN or Key Missing / Authentication Failure
BLE connection fails immediately after connecting with `failed to discover services, device disconnected` or `le-connection-abort-by-local`. In `btmon`, the trace shows:
```
Encryption Change - Status: PIN or Key Missing (0x06)
Disconnect - Reason: Authentication Failure (0x05)
```
This happens when the MeshCore device requires BLE PIN pairing (e.g., PIN `123456`) but no BlueZ agent is running to handle the passkey exchange. Bleak cannot provide a PIN by itself — it relies on a BlueZ agent to handle pairing.
**Symptoms:**
- `bluetoothctl connect` fails with `le-connection-abort-by-local`
- `bluetoothctl pair` asks for a passkey and succeeds
- meshcore-gui still fails because bleak creates its own connection without an agent
- btmon shows repeated connect → encrypt → `PIN or Key Missing` → disconnect cycles
## Problem 3: Port already in use
meshcore-gui fails to start with:
```
ERROR: [Errno 98] error while attempting to bind on address ('0.0.0.0', 8081): address already in use
```
This means a previous meshcore-gui instance is still running (or the port hasn't been released yet).
---
## Diagnostic Steps
### 1. Check adapter status
```bash
hciconfig -a
```
Expected: `UP RUNNING`. If it shows `DOWN`, reset with:
```bash
sudo hciconfig hci0 down
sudo hciconfig hci0 up
```
### 2. Check if adapter is detected
```bash
lsusb | grep -i blue
```
### 3. Check power supply (Raspberry Pi)
```bash
vcgencmd get_throttled
```
Expected: `throttled=0x0`. Any other value indicates power issues that can cause BLE instability.
### 4. Test basic BLE connection (without notify)
```bash
python -c "
import asyncio
from bleak import BleakClient
async def test():
async with BleakClient('AA:BB:CC:DD:EE:FF') as c:
print('Connected:', c.is_connected)
asyncio.run(test())
"
```
If this works but meshcli/meshcore_gui fails, the problem is specifically `start_notify`.
### 5. Test start_notify in isolation
```bash
python -c "
import asyncio
from bleak import BleakClient
UART_TX = '6e400003-b5a3-f393-e0a9-e50e24dcca9e'
async def test():
async with BleakClient('AA:BB:CC:DD:EE:FF') as c:
def cb(s, d): print(f'RX: {d.hex()}')
await c.start_notify(UART_TX, cb)
print('Notify OK!')
await asyncio.sleep(2)
asyncio.run(test())
"
```
If this also fails with `EOFError`, the issue is confirmed at the BlueZ/D-Bus level.
### 6. Test notifications via bluetoothctl (outside Python)
```bash
bluetoothctl
scan on
# Wait for device to appear
connect AA:BB:CC:DD:EE:FF
# Wait for "Connection successful"
menu gatt
select-attribute 6e400003-b5a3-f393-e0a9-e50e24dcca9e
notify on
```
If `connect` fails with `le-connection-abort-by-local`, the problem is at the BlueZ or device level. No Python fix will help.
### 7. Check if pairing is required (PIN or Key Missing)
If `bluetoothctl connect` fails with `le-connection-abort-by-local`, try pairing instead:
```bash
bluetoothctl
scan on
pair AA:BB:CC:DD:EE:FF
# If it asks for a passkey, the device requires PIN pairing
```
If pairing succeeds but meshcore-gui still fails, the issue is a missing BlueZ agent (see Solution 2).
### 8. Use btmon for HCI-level debugging
```bash
sudo btmon
```
In another terminal, start meshcore-gui. Look for:
- `Encryption Change - Status: PIN or Key Missing (0x06)` → pairing/agent issue (Solution 2)
- Successful encryption but no service discovery → stale bond (Solution 1)
### 9. Check what is using port 8081
```bash
lsof -i :8081
```
If another process holds the port, see Solution 3.
---
## Solution 1: Stale BLE Pairing State (EOFError)
The root cause is a stale BLE pairing state between the Linux adapter and the MeshCore device. The fix requires a clean reconnect sequence:
### Step 1 - Remove the device from BlueZ
```bash
bluetoothctl
remove AA:BB:CC:DD:EE:FF
exit
```
### Step 2 - Hard power cycle the MeshCore device
Physically power off the T1000-e (not just a software reset). Wait 10 seconds, then power it back on.
### Step 3 - Scan and reconnect from scratch
```bash
bluetoothctl
scan on
```
Wait until the device appears: `[NEW] Device AA:BB:CC:DD:EE:FF MeshCore-...`
Then immediately connect:
```
connect AA:BB:CC:DD:EE:FF
```
### Step 4 - Verify notifications work
```
menu gatt
select-attribute 6e400003-b5a3-f393-e0a9-e50e24dcca9e
notify on
```
If this succeeds, disconnect cleanly:
```
notify off
back
disconnect AA:BB:CC:DD:EE:FF
exit
```
### Step 5 - Verify channels with meshcli
```bash
meshcli -d AA:BB:CC:DD:EE:FF
> get_channels
```
Confirm output matches `CHANNELS_CONFIG` in `meshcore_gui.py`, then:
```
> exit
```
### Step 6 - Start the GUI
```bash
cd ~/meshcore-gui
source venv/bin/activate
python meshcore_gui.py AA:BB:CC:DD:EE:FF
```
---
## Solution 2: Missing BlueZ Agent for PIN Pairing
When the MeshCore device requires BLE PIN pairing, bleak cannot provide the PIN by itself. BlueZ needs a running agent that responds to pairing requests with the correct passkey.
**Why this happens:** `bluetoothctl` acts as its own agent (which is why manual pairing works), but when bleak connects independently, there is no agent to handle the passkey exchange. Even if the device was previously paired via `bluetoothctl`, the bond can become invalid when:
- The MeshCore device is reset or firmware-updated
- Another device (e.g., companion app) pairs with the MeshCore device and overwrites its bond slot
- The bond keys get out of sync for any reason
### Step 1 - Install bluez-tools
```bash
sudo apt install bluez-tools
```
### Step 2 - Create a PIN file
```bash
echo "* 123456" > ~/.meshcore-ble-pin
chmod 600 ~/.meshcore-ble-pin
```
The format is `<address-or-wildcard> <pin>`. Use `*` to match any device, or specify a specific address:
```
FF:05:D6:71:83:8D 123456
```
### Step 3 - Remove any existing (corrupt) bond
```bash
bluetoothctl remove AA:BB:CC:DD:EE:FF
```
### Step 4 - Start the agent and meshcore-gui
```bash
bt-agent -c KeyboardOnly -p ~/.meshcore-ble-pin &
python meshcore_gui.py AA:BB:CC:DD:EE:FF
```
### Step 5 - Make the agent permanent (systemd service)
Create the service file:
```bash
sudo tee /etc/systemd/system/bt-agent.service << 'EOF'
[Unit]
Description=Bluetooth PIN Agent for MeshCore
After=bluetooth.service
Requires=bluetooth.service
[Service]
ExecStart=/usr/bin/bt-agent -c KeyboardOnly -p /home/hans/.meshcore-ble-pin
Restart=always
User=hans
[Install]
WantedBy=multi-user.target
EOF
```
Enable and start:
```bash
sudo systemctl enable bt-agent
sudo systemctl start bt-agent
```
Verify it is running:
```bash
sudo systemctl status bt-agent
```
Now meshcore-gui can connect at any time without manual pairing. The agent survives reboots.
**Important:** Only run ONE bt-agent instance. Multiple agents conflict with each other. If you have both a manual `bt-agent &` process and the systemd service running, kill the manual one:
```bash
pkill -f bt-agent
sudo systemctl start bt-agent
```
---
## Solution 3: Port 8081 Already in Use
This happens when a previous meshcore-gui instance is still running or hasn't fully released the port.
### Quick fix - Kill previous instance and free the port
```bash
pkill -9 -f meshcore_gui
sleep 3
```
Verify the port is free:
```bash
lsof -i :8081
```
If nothing shows up, the port is free. Start meshcore-gui:
```bash
nohup python meshcore_gui.py AA:BB:CC:DD:EE:FF --debug-on > ~/meshcore.log 2>&1 &
```
### If the port is still in use after killing
Sometimes TCP sockets linger in `TIME_WAIT` state. Wait 30 seconds or force it:
```bash
sleep 30
lsof -i :8081
```
### Running in background with nohup
To run meshcore-gui in the background (survives terminal close):
```bash
nohup python meshcore_gui.py AA:BB:CC:DD:EE:FF --debug-on > ~/meshcore.log 2>&1 &
```
Check if it started successfully:
```bash
sleep 5
tail -30 ~/meshcore.log
```
**Tip:** Always redirect output to a log file (not `/dev/null`) so you can diagnose problems:
```bash
# Good - keeps logs for debugging
nohup python meshcore_gui.py AA:BB:CC:DD:EE:FF --debug-on > ~/meshcore.log 2>&1 &
# Bad - hides all errors
nohup python meshcore_gui.py AA:BB:CC:DD:EE:FF --debug-on > /dev/null 2>&1 &
```
---
## Things That Did NOT Help
| Action | Result |
|---|---|
| `sudo systemctl restart bluetooth` | No effect |
| `sudo hciconfig hci0 down/up` | No effect |
| `sudo rmmod btusb && sudo modprobe btusb` | No effect |
| `sudo usbreset "8087:0026"` | No effect |
| `sudo reboot` | No effect |
| Clearing BlueZ cache (`/var/lib/bluetooth/*/cache`) | No effect |
| Recreating Python venv | No effect |
| Downgrading `dbus_fast` / `bleak` | No effect |
| Downgrading `linux-firmware` | No effect |
| Adding `pin="123456"` to `MeshCore.create_ble()` | Pairing fails — bleak's `pair()` cannot provide a passkey without a BlueZ agent |
| Pre-connecting via `bluetoothctl connect` before meshcore-gui | Bleak creates its own connection and doesn't reuse the existing one |
---
## Key Takeaways
### EOFError / stale bond
When `start_notify` fails with `EOFError` but basic BLE connect works, the issue is almost always a stale BLE state between the host adapter and the peripheral device. The fix is:
1. **Remove** the device from bluetoothctl
2. **Hard power cycle** the peripheral device
3. **Re-scan** and reconnect from scratch
### PIN or Key Missing / Authentication Failure
When btmon shows `PIN or Key Missing (0x06)` and connections drop immediately after encryption negotiation, the fix is:
1. **Remove** the corrupt bond from bluetoothctl
2. **Run `bt-agent`** with the correct PIN file so BlueZ can handle pairing requests
3. **Install as systemd service** for persistence across reboots
### Port already in use
When meshcore-gui fails with `[Errno 98] address already in use`:
1. **Kill** any existing meshcore-gui process: `pkill -9 -f meshcore_gui`
2. **Wait** a few seconds for the port to be released
3. **Verify** the port is free: `lsof -i :8081`
---
## Recommended Startup Sequence
For the most reliable BLE connection, always follow this order:
1. Ensure `bt-agent` is running (if device requires PIN pairing): `sudo systemctl status bt-agent`
2. Ensure no other meshcore-gui instance is running: `pkill -f meshcore_gui` and `lsof -i :8081`
3. Ensure no other application holds the BLE connection (BT manager, bluetoothctl, meshcli, companion app)
4. Verify the device is visible: `bluetoothctl scan on`
5. Check channels: `meshcli -d <BLE_ADDRESS>` → `get_channels` → `exit`
6. Start the GUI: `python meshcore_gui.py <BLE_ADDRESS>`

BIN
docs/ble/BLE_ARCHITECTURE.docx Normal file
View File

Binary file not shown.

494
docs/ble/BLE_ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,494 @@
# MeshCore GUI — BLE Architecture
## Overzicht
Dit document beschrijft hoe MeshCore GUI communiceert met een MeshCore T1000-E device via Bluetooth Low Energy (BLE), welke libraries daarbij betrokken zijn, en hoe de volledige stack van hardware tot applicatielogica in elkaar zit.
---
## 1. De BLE Stack
De communicatie loopt door 7 lagen, van hardware tot GUI:
```
┌─────────────────────────────────────────────────────┐
│ 7. meshcore_gui (applicatie) │
│ BLEWorker, EventHandler, CommandHandler │
├─────────────────────────────────────────────────────┤
│ 6. meshcore (meshcore_py) (protocol) │
│ MeshCore.connect(), commands.*, event callbacks │
├─────────────────────────────────────────────────────┤
│ 5. bleak (BLE abstractie) │
│ BleakClient.connect(), start_notify(), write() │
├─────────────────────────────────────────────────────┤
│ 4. dbus_fast (D-Bus async client) │
│ MessageBus, ServiceInterface, method calls │
├─────────────────────────────────────────────────────┤
│ 3. D-Bus system bus (IPC) │
│ /org/bluez/hci0, org.bluez.Device1, Agent1 │
├─────────────────────────────────────────────────────┤
│ 2. BlueZ (bluetoothd) (Bluetooth daemon) │
│ GATT, pairing, bonding, device management │
├─────────────────────────────────────────────────────┤
│ 1. Linux Kernel + Hardware (HCI driver + radio) │
│ hci0, Bluetooth 5.0 chip (RPi5 built-in / USB) │
└─────────────────────────────────────────────────────┘
```
---
## 2. Libraries en hun rol
### 2.1 bleak (Bluetooth Low Energy platform Agnostic Klient)
**Doel:** Cross-platform Python BLE library. Abstracteert de platform-specifieke BLE backends achter één API.
| Platform | Backend | Communicatie |
|----------|---------|-------------|
| Linux | BlueZ via D-Bus | `dbus_fast` → `bluetoothd` |
| macOS | CoreBluetooth | Objective-C bridge via `pyobjc` |
| Windows | WinRT | Windows Runtime BLE API |
**Hoe bleak werkt op Linux:**
Bleak praat *niet* rechtstreeks met de Bluetooth hardware. In plaats daarvan stuurt bleak D-Bus berichten naar de BlueZ daemon (`bluetoothd`), die op zijn beurt de kernel HCI driver aanstuurt. Elk bleak-commando wordt vertaald naar een D-Bus method call:
| bleak API | D-Bus call naar BlueZ |
|-----------|----------------------|
| `BleakClient.connect()` | `org.bluez.Device1.Connect()` |
| `BleakClient.disconnect()` | `org.bluez.Device1.Disconnect()` |
| `BleakClient.start_notify(uuid, callback)` | `org.bluez.GattCharacteristic1.StartNotify()` |
| `BleakClient.write_gatt_char(uuid, data)` | `org.bluez.GattCharacteristic1.WriteValue()` |
| `BleakScanner.discover()` | `org.bluez.Adapter1.StartDiscovery()` |
Bleak installeert automatisch `dbus_fast` als dependency.
### 2.2 dbus_fast
**Doel:** Async Python D-Bus library. Biedt twee functies:
1. **Client** — Bleak gebruikt `dbus_fast.aio.MessageBus` om D-Bus method calls naar BlueZ te sturen (connect, read, write, notify). Dit is intern aan bleak; onze code raakt dit niet direct aan.
2. **Server** — Onze `ble_agent.py` gebruikt `dbus_fast.service.ServiceInterface` om een D-Bus service te *exporteren*: de PIN agent die BlueZ aanroept wanneer het device pairing nodig heeft.
Doordat `dbus_fast` al een dependency van `bleak` is, hoeven we geen extra packages te installeren.
### 2.3 meshcore (meshcore_py)
**Doel:** MeshCore protocol implementatie. Vertaalt hoge-niveau commando's naar BLE GATT read/write operaties.
**GATT Service:** MeshCore devices gebruiken de **Nordic UART Service (NUS)** voor communicatie:
| Characteristic | UUID | Richting | Functie |
|---------------|------|----------|---------|
| RX | `6e400002-b5a3-f393-e0a9-e50e24dcca9e` | Host → Device | Commando's schrijven |
| TX | `6e400003-b5a3-f393-e0a9-e50e24dcca9e` | Device → Host | Responses/events ontvangen (notify) |
**Protocol:** De meshcore library:
- Serialiseert commando's (appstart, device_query, get_contacts, send_msg, etc.) naar binaire packets
- Schrijft deze naar de NUS RX characteristic via `bleak.write_gatt_char()`
- Luistert op de NUS TX characteristic via `bleak.start_notify()` voor responses en async events
- Deserialiseert binaire responses terug naar Python dicts met event types
**Communicatiepatroon:** Request-response met async events:
```
meshcore_gui → meshcore → bleak → D-Bus → BlueZ → HCI → Radio → T1000-E
│
meshcore_gui ← meshcore ← bleak ← D-Bus ← BlueZ ← HCI ← Radio ←──────┘
```
Commando's zijn *subscribe-before-send*: meshcore registreert eerst een notify handler op de TX characteristic, stuurt dan het commando via de RX characteristic, en wacht op de response via de notify callback. Dit voorkomt race conditions waarbij de response arriveert voordat de listener klaar is (gefixt in meshcore_py PR #52).
### 2.4 meshcoredecoder
**Doel:** Decodering van ruwe LoRa packets die via de RX log binnenkomen. Decrypts packets met channel keys en extraheert route-informatie (path hashes, hop data). Gebruikt door `PacketDecoder` in de BLE events layer.
### 2.5 Onze eigen BLE modules
| Module | Library | Functie |
|--------|---------|---------|
| `ble_agent.py` | `dbus_fast` (server) | Exporteert `org.bluez.Agent1` interface op D-Bus; beantwoordt PIN requests |
| `ble_reconnect.py` | `dbus_fast` (client) | `remove_bond()`: roept `org.bluez.Adapter1.RemoveDevice()` aan via D-Bus |
| `worker.py` | `meshcore` + `bleak` (indirect) | `MeshCore.connect()`, command loop, disconnect detection |
| `commands.py` | `meshcore` | `mc.commands.send_msg()`, `send_advert()`, etc. |
| `events.py` | `meshcore` | Callbacks: `CHANNEL_MSG_RECV`, `RX_LOG_DATA`, etc. |
---
## 3. De drie D-Bus gesprekken
Onze applicatie voert drie soorten D-Bus communicatie uit, elk met een ander doel:
### 3.1 PIN Agent (dbus_fast — server mode)
**Probleem:** Wanneer BlueZ een BLE device wil pairen dat een PIN vereist, zoekt het op de D-Bus naar een geregistreerde Agent die de PIN kan leveren. Zonder agent faalt de pairing met "failed to discover services".
**Oplossing:** `ble_agent.py` exporteert een `org.bluez.Agent1` service op D-Bus path `/meshcore/ble_agent`. BlueZ roept methodes aan op onze agent:
```
BlueZ (bluetoothd) Onze Agent (ble_agent.py)
│ │
│── RegisterAgent(/meshcore/ble_agent) ──→│ (bij startup)
│← OK ──────────────────────────────────│
│ │
│── RequestDefaultAgent() ──────────────→│
│← OK ──────────────────────────────────│
│ │
│ ... device wil pairen ... │
│ │
│── RequestPinCode(/org/bluez/.../dev) ─→│
│← "123456" ───────────────────────────│
│ │
│ ... pairing succesvol ... │
```
### 3.2 Bond Cleanup (dbus_fast — client mode)
**Probleem:** Na een disconnect slaat BlueZ de pairing keys op (een "bond"). Bij reconnectie gebruikt BlueZ deze oude keys, maar het device heeft ze verworpen → "PIN or Key Missing" error.
**Oplossing:** `ble_reconnect.py` stuurt een D-Bus method call naar BlueZ:
```python
# Equivalent van: bluetoothctl remove FF:05:D6:71:83:8D
bus.call(
destination="org.bluez",
path="/org/bluez/hci0", # Adapter
interface="org.bluez.Adapter1",
member="RemoveDevice",
signature="o",
body=["/org/bluez/hci0/dev_FF_05_D6_71_83_8D"] # Device object path
)
```
### 3.3 BLE Communicatie (bleak → dbus_fast — client mode)
Bleak stuurt intern D-Bus berichten voor alle BLE operaties. Dit is transparant voor onze code — wij roepen alleen de bleak API aan, bleak vertaalt naar D-Bus:
```python
# Onze code (via meshcore):
await mc.connect(ble_address)
# Wat bleak intern doet:
await bus.call("org.bluez.Device1.Connect()")
await bus.call("org.bluez.GattCharacteristic1.StartNotify()") # TX char
await bus.call("org.bluez.GattCharacteristic1.WriteValue()") # RX char
```
---
## 4. Sequence Diagram — Volledige BLE Lifecycle
Het onderstaande diagram toont de complete levenscyclus van een BLE sessie, van startup tot disconnect en reconnect.
```mermaid
sequenceDiagram
autonumber
participant GUI as GUI Thread<br/>(NiceGUI)
participant Worker as BLEWorker<br/>(asyncio thread)
participant Agent as BleAgentManager<br/>(ble_agent.py)
participant Reconnect as ble_reconnect.py
participant MC as meshcore<br/>(MeshCore)
participant Bleak as bleak<br/>(BleakClient)
participant DBus as D-Bus<br/>(system bus)
participant BZ as BlueZ<br/>(bluetoothd)
participant Dev as T1000-E<br/>(BLE device)
Note over Worker,Dev: ═══ FASE 1: PIN Agent Registratie ═══
Worker->>Agent: start(pin="123456")
Agent->>DBus: connect to system bus
Agent->>DBus: export /meshcore/ble_agent<br/>(org.bluez.Agent1)
Agent->>DBus: RegisterAgent(/meshcore/ble_agent, "KeyboardOnly")
DBus->>BZ: RegisterAgent
BZ-->>DBus: OK
Agent->>DBus: RequestDefaultAgent(/meshcore/ble_agent)
DBus->>BZ: RequestDefaultAgent
BZ-->>DBus: OK
Agent-->>Worker: Agent ready
Note over Worker,Dev: ═══ FASE 2: Bond Cleanup ═══
Worker->>Reconnect: remove_bond("FF:05:...")
Reconnect->>DBus: Adapter1.RemoveDevice(/org/bluez/hci0/dev_FF_05_...)
DBus->>BZ: RemoveDevice
BZ-->>DBus: OK (of "Does Not Exist" → genegeerd)
Reconnect-->>Worker: Bond removed
Note over Worker,Dev: ═══ FASE 3: Verbinding + GATT Discovery ═══
Worker->>MC: MeshCore.connect("FF:05:...")
MC->>Bleak: BleakClient.connect()
Bleak->>DBus: Device1.Connect()
DBus->>BZ: Connect
BZ->>Dev: BLE Connection Request
Dev-->>BZ: Connection Accepted
Note over BZ,Dev: Pairing vereist (PIN)
BZ->>DBus: Agent1.RequestPinCode(device_path)
DBus->>Agent: RequestPinCode()
Agent-->>DBus: "123456"
DBus-->>BZ: PIN
BZ->>Dev: Pairing met PIN
Dev-->>BZ: Pairing OK + Encryption active
BZ->>BZ: GATT Service Discovery
BZ-->>Bleak: Services resolved (NUS: 6e400001-...)
Bleak-->>MC: Connected
MC->>Bleak: start_notify(TX: 6e400003-...)
Bleak->>DBus: GattCharacteristic1.StartNotify()
DBus->>BZ: StartNotify
BZ-->>Bleak: Notifications enabled
MC->>Bleak: write(RX: 6e400002-..., appstart_cmd)
Bleak->>DBus: GattCharacteristic1.WriteValue(data)
DBus->>BZ: WriteValue
BZ->>Dev: BLE Write (appstart)
Dev-->>BZ: BLE Notify (response)
BZ-->>Bleak: Notification callback
Bleak-->>MC: Event: SELF_INFO
MC-->>Worker: self_info = {name, pubkey, freq, ...}
Note over Worker,Dev: ═══ FASE 4: Data Laden ═══
Worker->>MC: commands.send_device_query()
MC->>Bleak: write(RX, device_query_cmd)
Bleak->>DBus: WriteValue
DBus->>BZ: WriteValue
BZ->>Dev: device_query
Dev-->>BZ: notify(response)
BZ-->>Bleak: callback
Bleak-->>MC: Event: DEVICE_QUERY
MC-->>Worker: {firmware, tx_power, ...}
Worker->>MC: commands.get_channel(0..N)
MC-->>Worker: {name, channel_secret}
Worker->>MC: commands.get_contacts()
MC-->>Worker: [{pubkey, name, type, lat, lon}, ...]
Worker->>GUI: SharedData.set_channels(), set_contacts(), ...
GUI->>GUI: Timer 500ms → update UI
Note over Worker,Dev: ═══ FASE 5: Operationele Loop ═══
loop Elke 500ms
GUI->>GUI: _update_ui() → lees SharedData snapshot
end
loop Command Queue
GUI->>Worker: put_command("send_msg", {text, channel})
Worker->>MC: commands.send_msg(channel, text)
MC->>Bleak: write(RX, send_msg_packet)
Bleak->>DBus: WriteValue
DBus->>BZ: WriteValue
BZ->>Dev: BLE Write
end
loop Async Events (continu)
Dev-->>BZ: BLE Notify (incoming mesh message)
BZ-->>Bleak: Notification callback
Bleak-->>MC: raw data
MC-->>Worker: Event: CHANNEL_MSG_RECV
Worker->>Worker: EventHandler → dedup → SharedData.add_message()
Worker->>GUI: message_updated = True
end
Note over Worker,Dev: ═══ FASE 6: Disconnect + Auto-Reconnect ═══
Dev--xBZ: BLE link lost (~2 uur timeout)
BZ-->>Bleak: Disconnected callback
Bleak-->>MC: Connection lost
MC-->>Worker: Exception: "not connected" / "disconnected"
Worker->>Worker: Disconnect gedetecteerd
loop Reconnect (max 5 pogingen, lineaire backoff)
Worker->>Reconnect: remove_bond("FF:05:...")
Reconnect->>DBus: Adapter1.RemoveDevice
DBus->>BZ: RemoveDevice
BZ-->>Reconnect: OK
Worker->>Worker: wait(attempt × 5s)
Worker->>MC: MeshCore.connect("FF:05:...")
MC->>Bleak: BleakClient.connect()
Bleak->>DBus: Device1.Connect()
DBus->>BZ: Connect
BZ->>Dev: BLE Connection Request
alt Verbinding succesvol
Dev-->>BZ: Connected + Paired (PIN via Agent)
BZ-->>Bleak: Connected
Worker->>Worker: Re-wire event handlers + reload data
Worker->>GUI: set_status("✅ Reconnected")
else Verbinding mislukt
BZ-->>Bleak: Error
Worker->>Worker: Volgende poging...
end
end
Note over Worker,Dev: ═══ FASE 7: Cleanup ═══
Worker->>Agent: stop()
Agent->>DBus: UnregisterAgent(/meshcore/ble_agent)
Agent->>DBus: disconnect()
```
---
## 5. GATT Communicatie in Detail
### 5.1 Nordic UART Service (NUS)
Het MeshCore device adverteert één primaire BLE service: de **Nordic UART Service**. Dit is een de-facto standaard voor seriële communicatie over BLE, oorspronkelijk ontworpen door Nordic Semiconductor.
```
Service: Nordic UART Service
UUID: 6e400001-b5a3-f393-e0a9-e50e24dcca9e
├── RX Characteristic (Write Without Response)
│ UUID: 6e400002-b5a3-f393-e0a9-e50e24dcca9e
│ Richting: Host → Device
│ Gebruik: Commando's sturen naar het T1000-E
│ Max grootte: 20 bytes per write (MTU-afhankelijk)
│
└── TX Characteristic (Notify)
UUID: 6e400003-b5a3-f393-e0a9-e50e24dcca9e
Richting: Device → Host
Gebruik: Responses en async events ontvangen
Activatie: bleak.start_notify() → BlueZ StartNotify
```
### 5.2 Dataflow per commando
Een typisch commando (bijv. "stuur een mesh bericht") doorloopt deze stappen:
```
1. GUI: gebruiker typt bericht, klikt Send
2. GUI → SharedData: put_command("send_msg", {channel: 0, text: "Hello"})
3. BLEWorker: haalt command uit queue
4. meshcore: serialiseert naar binary packet
→ [header][cmd_type][channel_idx][payload_len][utf8_text]
5. bleak: write_gatt_char(NUS_RX_UUID, packet)
6. dbus_fast: GattCharacteristic1.WriteValue(packet_bytes, {})
7. BlueZ: schrijft naar HCI controller
8. HCI: stuurt BLE PDU via radio
9. T1000-E: ontvangt, verwerkt, stuurt via LoRa mesh
```
De response (of een inkomend mesh bericht) loopt de omgekeerde route:
```
1. T1000-E: ontvangt mesh bericht via LoRa
2. T1000-E → HCI: BLE notification met data
3. BlueZ: ontvangt notification, stuurt via D-Bus
4. dbus_fast: roept de notify callback in bleak aan
5. bleak: roept de registered callback in meshcore aan
6. meshcore: deserialiseert binary → Event(type, payload)
7. BLEWorker: EventHandler verwerkt het event
→ dedup check → naam resolutie → path hash extractie
8. SharedData: add_message(Message.incoming(...))
9. GUI: ziet message_updated flag bij volgende 500ms poll
```
### 5.3 Waarom subscribe-before-send?
BLE notifications zijn asynchroon. Als meshcore eerst het commando schrijft en *daarna* `start_notify()` aanroept, kan de response al verloren zijn gegaan voordat de listener klaar is. Dit was een bug in de originele meshcore_py die leidde tot ~2 minuten startup delay:
```
❌ Oud (race condition):
write(RX, command) → device antwoordt direct
start_notify(TX) → te laat, response is al weg
✅ Nieuw (PR #52):
start_notify(TX) → listener actief
write(RX, command) → device antwoordt
callback fired → response ontvangen
```
---
## 6. Pairing en Bonding
### 6.1 Waarom PIN pairing?
Het T1000-E device is geconfigureerd met BLE PIN `123456` (instelbaar via firmware). Dit voorkomt dat willekeurige BLE clients verbinden. BlueZ ondersteunt PIN pairing via het **Agent** mechanisme.
### 6.2 Agent interface
BlueZ definieert de `org.bluez.Agent1` D-Bus interface. Onze `BluezAgent` class implementeert deze callbacks:
| Methode | D-Bus Signature | Wanneer aangeroepen | Ons antwoord |
|---------|----------------|--------------------:|-------------|
| `RequestPinCode` | `o → s` | Device vraagt PIN | `"123456"` |
| `RequestPasskey` | `o → u` | Device vraagt numeriek passkey | `123456` (uint32) |
| `DisplayPasskey` | `oqu → ` | Passkey tonen (info only) | (log only) |
| `RequestConfirmation` | `ou → ` | Bevestig passkey match | (accept) |
| `AuthorizeService` | `os → ` | Service autorisatie | (accept) |
| `Cancel` | ` → ` | Pairing geannuleerd | (log only) |
| `Release` | ` → ` | Agent niet meer nodig | (cleanup) |
### 6.3 Het bonding probleem
Na succesvolle pairing slaat BlueZ de encryption keys op in `/var/lib/bluetooth/<adapter>/<device>/info`. Dit heet een "bond". Bij de volgende connectie probeert BlueZ deze keys te hergebruiken.
**Het probleem:** Het T1000-E verwerpt na ~2 uur de BLE verbinding (firmware timeout). BlueZ heeft nog de oude bond keys, maar het device heeft ze verworpen. Resultaat:
```
BlueZ: "Ik heb keys voor dit device, gebruik die"
T1000-E: "Ik ken deze keys niet → Reject (PIN or Key Missing)"
BlueZ: "Pairing failed"
```
**De oplossing:** Vóór elke reconnectie verwijderen we de bond:
```
remove_bond() → Adapter1.RemoveDevice() → BlueZ wist keys
connect() → BlueZ: "Geen keys, start verse pairing"
Agent → levert PIN → verse pairing succesvol
```
---
## 7. D-Bus Policy
Normale gebruikers mogen standaard niet alle BlueZ D-Bus interfaces aanspreken. De D-Bus policy file (`/etc/dbus-1/system.d/meshcore-ble.conf`) geeft de gebruiker die de service draait toestemming:
```xml
<busconfig>
<policy user="hans">
<allow send_destination="org.bluez"/>
<allow send_interface="org.bluez.Agent1"/>
<allow send_interface="org.bluez.AgentManager1"/>
</policy>
</busconfig>
```
Zonder deze policy:
- `bleak` kan nog steeds verbinden (bleak gebruikt een standaard D-Bus policy die al met BlueZ meekomt)
- Onze **agent** kan zich niet registreren → PIN pairing faalt
- Onze **bond cleanup** kan `RemoveDevice` niet aanroepen
---
## 8. Samenvatting Dependencies
```
meshcore-gui
├── nicegui → Web UI framework (onze GUI)
├── meshcore → MeshCore protocol (commando's, events)
│ └── bleak → BLE abstractie (connect, notify, write)
│ └── dbus_fast → D-Bus communicatie (naar BlueZ)
├── meshcoredecoder → LoRa packet decryptie + route extractie
└── (geen extra) → ble_agent.py en ble_reconnect.py
gebruiken dbus_fast die al via bleak
geïnstalleerd is
```
Alle BLE-gerelateerde functionaliteit draait op precies **vier Python packages**: `bleak`, `dbus_fast`, `meshcore`, en `meshcoredecoder`. Er zijn geen system-level dependencies meer nodig buiten `bluez` zelf (geen `bluez-tools`, geen `bt-agent`).
# Legacy BLE Document
> **Note:** This document describes the BLE architecture and is retained for historical reference. The current GUI uses USB serial.

491
docs/ble/BLE_ARCHITECTURE.txt Normal file
View File

@@ -0,0 +1,491 @@
# MeshCore GUI — BLE Architecture
## Overzicht
Dit document beschrijft hoe MeshCore GUI communiceert met een MeshCore T1000-E device via Bluetooth Low Energy (BLE), welke libraries daarbij betrokken zijn, en hoe de volledige stack van hardware tot applicatielogica in elkaar zit.
---
## 1. De BLE Stack
De communicatie loopt door 7 lagen, van hardware tot GUI:
```
┌─────────────────────────────────────────────────────┐
│ 7. meshcore_gui (applicatie) │
│ BLEWorker, EventHandler, CommandHandler │
├─────────────────────────────────────────────────────┤
│ 6. meshcore (meshcore_py) (protocol) │
│ MeshCore.connect(), commands.*, event callbacks │
├─────────────────────────────────────────────────────┤
│ 5. bleak (BLE abstractie) │
│ BleakClient.connect(), start_notify(), write() │
├─────────────────────────────────────────────────────┤
│ 4. dbus_fast (D-Bus async client) │
│ MessageBus, ServiceInterface, method calls │
├─────────────────────────────────────────────────────┤
│ 3. D-Bus system bus (IPC) │
│ /org/bluez/hci0, org.bluez.Device1, Agent1 │
├─────────────────────────────────────────────────────┤
│ 2. BlueZ (bluetoothd) (Bluetooth daemon) │
│ GATT, pairing, bonding, device management │
├─────────────────────────────────────────────────────┤
│ 1. Linux Kernel + Hardware (HCI driver + radio) │
│ hci0, Bluetooth 5.0 chip (RPi5 built-in / USB) │
└─────────────────────────────────────────────────────┘
```
---
## 2. Libraries en hun rol
### 2.1 bleak (Bluetooth Low Energy platform Agnostic Klient)
**Doel:** Cross-platform Python BLE library. Abstracteert de platform-specifieke BLE backends achter één API.
| Platform | Backend | Communicatie |
|----------|---------|-------------|
| Linux | BlueZ via D-Bus | `dbus_fast` → `bluetoothd` |
| macOS | CoreBluetooth | Objective-C bridge via `pyobjc` |
| Windows | WinRT | Windows Runtime BLE API |
**Hoe bleak werkt op Linux:**
Bleak praat *niet* rechtstreeks met de Bluetooth hardware. In plaats daarvan stuurt bleak D-Bus berichten naar de BlueZ daemon (`bluetoothd`), die op zijn beurt de kernel HCI driver aanstuurt. Elk bleak-commando wordt vertaald naar een D-Bus method call:
| bleak API | D-Bus call naar BlueZ |
|-----------|----------------------|
| `BleakClient.connect()` | `org.bluez.Device1.Connect()` |
| `BleakClient.disconnect()` | `org.bluez.Device1.Disconnect()` |
| `BleakClient.start_notify(uuid, callback)` | `org.bluez.GattCharacteristic1.StartNotify()` |
| `BleakClient.write_gatt_char(uuid, data)` | `org.bluez.GattCharacteristic1.WriteValue()` |
| `BleakScanner.discover()` | `org.bluez.Adapter1.StartDiscovery()` |
Bleak installeert automatisch `dbus_fast` als dependency.
### 2.2 dbus_fast
**Doel:** Async Python D-Bus library. Biedt twee functies:
1. **Client** — Bleak gebruikt `dbus_fast.aio.MessageBus` om D-Bus method calls naar BlueZ te sturen (connect, read, write, notify). Dit is intern aan bleak; onze code raakt dit niet direct aan.
2. **Server** — Onze `ble_agent.py` gebruikt `dbus_fast.service.ServiceInterface` om een D-Bus service te *exporteren*: de PIN agent die BlueZ aanroept wanneer het device pairing nodig heeft.
Doordat `dbus_fast` al een dependency van `bleak` is, hoeven we geen extra packages te installeren.
### 2.3 meshcore (meshcore_py)
**Doel:** MeshCore protocol implementatie. Vertaalt hoge-niveau commando's naar BLE GATT read/write operaties.
**GATT Service:** MeshCore devices gebruiken de **Nordic UART Service (NUS)** voor communicatie:
| Characteristic | UUID | Richting | Functie |
|---------------|------|----------|---------|
| RX | `6e400002-b5a3-f393-e0a9-e50e24dcca9e` | Host → Device | Commando's schrijven |
| TX | `6e400003-b5a3-f393-e0a9-e50e24dcca9e` | Device → Host | Responses/events ontvangen (notify) |
**Protocol:** De meshcore library:
- Serialiseert commando's (appstart, device_query, get_contacts, send_msg, etc.) naar binaire packets
- Schrijft deze naar de NUS RX characteristic via `bleak.write_gatt_char()`
- Luistert op de NUS TX characteristic via `bleak.start_notify()` voor responses en async events
- Deserialiseert binaire responses terug naar Python dicts met event types
**Communicatiepatroon:** Request-response met async events:
```
meshcore_gui → meshcore → bleak → D-Bus → BlueZ → HCI → Radio → T1000-E
│
meshcore_gui ← meshcore ← bleak ← D-Bus ← BlueZ ← HCI ← Radio ←──────┘
```
Commando's zijn *subscribe-before-send*: meshcore registreert eerst een notify handler op de TX characteristic, stuurt dan het commando via de RX characteristic, en wacht op de response via de notify callback. Dit voorkomt race conditions waarbij de response arriveert voordat de listener klaar is (gefixt in meshcore_py PR #52).
### 2.4 meshcoredecoder
**Doel:** Decodering van ruwe LoRa packets die via de RX log binnenkomen. Decrypts packets met channel keys en extraheert route-informatie (path hashes, hop data). Gebruikt door `PacketDecoder` in de BLE events layer.
### 2.5 Onze eigen BLE modules
| Module | Library | Functie |
|--------|---------|---------|
| `ble_agent.py` | `dbus_fast` (server) | Exporteert `org.bluez.Agent1` interface op D-Bus; beantwoordt PIN requests |
| `ble_reconnect.py` | `dbus_fast` (client) | `remove_bond()`: roept `org.bluez.Adapter1.RemoveDevice()` aan via D-Bus |
| `worker.py` | `meshcore` + `bleak` (indirect) | `MeshCore.connect()`, command loop, disconnect detection |
| `commands.py` | `meshcore` | `mc.commands.send_msg()`, `send_advert()`, etc. |
| `events.py` | `meshcore` | Callbacks: `CHANNEL_MSG_RECV`, `RX_LOG_DATA`, etc. |
---
## 3. De drie D-Bus gesprekken
Onze applicatie voert drie soorten D-Bus communicatie uit, elk met een ander doel:
### 3.1 PIN Agent (dbus_fast — server mode)
**Probleem:** Wanneer BlueZ een BLE device wil pairen dat een PIN vereist, zoekt het op de D-Bus naar een geregistreerde Agent die de PIN kan leveren. Zonder agent faalt de pairing met "failed to discover services".
**Oplossing:** `ble_agent.py` exporteert een `org.bluez.Agent1` service op D-Bus path `/meshcore/ble_agent`. BlueZ roept methodes aan op onze agent:
```
BlueZ (bluetoothd) Onze Agent (ble_agent.py)
│ │
│── RegisterAgent(/meshcore/ble_agent) ──→│ (bij startup)
│← OK ──────────────────────────────────│
│ │
│── RequestDefaultAgent() ──────────────→│
│← OK ──────────────────────────────────│
│ │
│ ... device wil pairen ... │
│ │
│── RequestPinCode(/org/bluez/.../dev) ─→│
│← "123456" ───────────────────────────│
│ │
│ ... pairing succesvol ... │
```
### 3.2 Bond Cleanup (dbus_fast — client mode)
**Probleem:** Na een disconnect slaat BlueZ de pairing keys op (een "bond"). Bij reconnectie gebruikt BlueZ deze oude keys, maar het device heeft ze verworpen → "PIN or Key Missing" error.
**Oplossing:** `ble_reconnect.py` stuurt een D-Bus method call naar BlueZ:
```python
# Equivalent van: bluetoothctl remove FF:05:D6:71:83:8D
bus.call(
destination="org.bluez",
path="/org/bluez/hci0", # Adapter
interface="org.bluez.Adapter1",
member="RemoveDevice",
signature="o",
body=["/org/bluez/hci0/dev_FF_05_D6_71_83_8D"] # Device object path
)
```
### 3.3 BLE Communicatie (bleak → dbus_fast — client mode)
Bleak stuurt intern D-Bus berichten voor alle BLE operaties. Dit is transparant voor onze code — wij roepen alleen de bleak API aan, bleak vertaalt naar D-Bus:
```python
# Onze code (via meshcore):
await mc.connect(ble_address)
# Wat bleak intern doet:
await bus.call("org.bluez.Device1.Connect()")
await bus.call("org.bluez.GattCharacteristic1.StartNotify()") # TX char
await bus.call("org.bluez.GattCharacteristic1.WriteValue()") # RX char
```
---
## 4. Sequence Diagram — Volledige BLE Lifecycle
Het onderstaande diagram toont de complete levenscyclus van een BLE sessie, van startup tot disconnect en reconnect.
```mermaid
sequenceDiagram
autonumber
participant GUI as GUI Thread<br/>(NiceGUI)
participant Worker as BLEWorker<br/>(asyncio thread)
participant Agent as BleAgentManager<br/>(ble_agent.py)
participant Reconnect as ble_reconnect.py
participant MC as meshcore<br/>(MeshCore)
participant Bleak as bleak<br/>(BleakClient)
participant DBus as D-Bus<br/>(system bus)
participant BZ as BlueZ<br/>(bluetoothd)
participant Dev as T1000-E<br/>(BLE device)
Note over Worker,Dev: ═══ FASE 1: PIN Agent Registratie ═══
Worker->>Agent: start(pin="123456")
Agent->>DBus: connect to system bus
Agent->>DBus: export /meshcore/ble_agent<br/>(org.bluez.Agent1)
Agent->>DBus: RegisterAgent(/meshcore/ble_agent, "KeyboardOnly")
DBus->>BZ: RegisterAgent
BZ-->>DBus: OK
Agent->>DBus: RequestDefaultAgent(/meshcore/ble_agent)
DBus->>BZ: RequestDefaultAgent
BZ-->>DBus: OK
Agent-->>Worker: Agent ready
Note over Worker,Dev: ═══ FASE 2: Bond Cleanup ═══
Worker->>Reconnect: remove_bond("FF:05:...")
Reconnect->>DBus: Adapter1.RemoveDevice(/org/bluez/hci0/dev_FF_05_...)
DBus->>BZ: RemoveDevice
BZ-->>DBus: OK (of "Does Not Exist" → genegeerd)
Reconnect-->>Worker: Bond removed
Note over Worker,Dev: ═══ FASE 3: Verbinding + GATT Discovery ═══
Worker->>MC: MeshCore.connect("FF:05:...")
MC->>Bleak: BleakClient.connect()
Bleak->>DBus: Device1.Connect()
DBus->>BZ: Connect
BZ->>Dev: BLE Connection Request
Dev-->>BZ: Connection Accepted
Note over BZ,Dev: Pairing vereist (PIN)
BZ->>DBus: Agent1.RequestPinCode(device_path)
DBus->>Agent: RequestPinCode()
Agent-->>DBus: "123456"
DBus-->>BZ: PIN
BZ->>Dev: Pairing met PIN
Dev-->>BZ: Pairing OK + Encryption active
BZ->>BZ: GATT Service Discovery
BZ-->>Bleak: Services resolved (NUS: 6e400001-...)
Bleak-->>MC: Connected
MC->>Bleak: start_notify(TX: 6e400003-...)
Bleak->>DBus: GattCharacteristic1.StartNotify()
DBus->>BZ: StartNotify
BZ-->>Bleak: Notifications enabled
MC->>Bleak: write(RX: 6e400002-..., appstart_cmd)
Bleak->>DBus: GattCharacteristic1.WriteValue(data)
DBus->>BZ: WriteValue
BZ->>Dev: BLE Write (appstart)
Dev-->>BZ: BLE Notify (response)
BZ-->>Bleak: Notification callback
Bleak-->>MC: Event: SELF_INFO
MC-->>Worker: self_info = {name, pubkey, freq, ...}
Note over Worker,Dev: ═══ FASE 4: Data Laden ═══
Worker->>MC: commands.send_device_query()
MC->>Bleak: write(RX, device_query_cmd)
Bleak->>DBus: WriteValue
DBus->>BZ: WriteValue
BZ->>Dev: device_query
Dev-->>BZ: notify(response)
BZ-->>Bleak: callback
Bleak-->>MC: Event: DEVICE_QUERY
MC-->>Worker: {firmware, tx_power, ...}
Worker->>MC: commands.get_channel(0..N)
MC-->>Worker: {name, channel_secret}
Worker->>MC: commands.get_contacts()
MC-->>Worker: [{pubkey, name, type, lat, lon}, ...]
Worker->>GUI: SharedData.set_channels(), set_contacts(), ...
GUI->>GUI: Timer 500ms → update UI
Note over Worker,Dev: ═══ FASE 5: Operationele Loop ═══
loop Elke 500ms
GUI->>GUI: _update_ui() → lees SharedData snapshot
end
loop Command Queue
GUI->>Worker: put_command("send_msg", {text, channel})
Worker->>MC: commands.send_msg(channel, text)
MC->>Bleak: write(RX, send_msg_packet)
Bleak->>DBus: WriteValue
DBus->>BZ: WriteValue
BZ->>Dev: BLE Write
end
loop Async Events (continu)
Dev-->>BZ: BLE Notify (incoming mesh message)
BZ-->>Bleak: Notification callback
Bleak-->>MC: raw data
MC-->>Worker: Event: CHANNEL_MSG_RECV
Worker->>Worker: EventHandler → dedup → SharedData.add_message()
Worker->>GUI: message_updated = True
end
Note over Worker,Dev: ═══ FASE 6: Disconnect + Auto-Reconnect ═══
Dev--xBZ: BLE link lost (~2 uur timeout)
BZ-->>Bleak: Disconnected callback
Bleak-->>MC: Connection lost
MC-->>Worker: Exception: "not connected" / "disconnected"
Worker->>Worker: Disconnect gedetecteerd
loop Reconnect (max 5 pogingen, lineaire backoff)
Worker->>Reconnect: remove_bond("FF:05:...")
Reconnect->>DBus: Adapter1.RemoveDevice
DBus->>BZ: RemoveDevice
BZ-->>Reconnect: OK
Worker->>Worker: wait(attempt × 5s)
Worker->>MC: MeshCore.connect("FF:05:...")
MC->>Bleak: BleakClient.connect()
Bleak->>DBus: Device1.Connect()
DBus->>BZ: Connect
BZ->>Dev: BLE Connection Request
alt Verbinding succesvol
Dev-->>BZ: Connected + Paired (PIN via Agent)
BZ-->>Bleak: Connected
Worker->>Worker: Re-wire event handlers + reload data
Worker->>GUI: set_status("✅ Reconnected")
else Verbinding mislukt
BZ-->>Bleak: Error
Worker->>Worker: Volgende poging...
end
end
Note over Worker,Dev: ═══ FASE 7: Cleanup ═══
Worker->>Agent: stop()
Agent->>DBus: UnregisterAgent(/meshcore/ble_agent)
Agent->>DBus: disconnect()
```
---
## 5. GATT Communicatie in Detail
### 5.1 Nordic UART Service (NUS)
Het MeshCore device adverteert één primaire BLE service: de **Nordic UART Service**. Dit is een de-facto standaard voor seriële communicatie over BLE, oorspronkelijk ontworpen door Nordic Semiconductor.
```
Service: Nordic UART Service
UUID: 6e400001-b5a3-f393-e0a9-e50e24dcca9e
├── RX Characteristic (Write Without Response)
│ UUID: 6e400002-b5a3-f393-e0a9-e50e24dcca9e
│ Richting: Host → Device
│ Gebruik: Commando's sturen naar het T1000-E
│ Max grootte: 20 bytes per write (MTU-afhankelijk)
│
└── TX Characteristic (Notify)
UUID: 6e400003-b5a3-f393-e0a9-e50e24dcca9e
Richting: Device → Host
Gebruik: Responses en async events ontvangen
Activatie: bleak.start_notify() → BlueZ StartNotify
```
### 5.2 Dataflow per commando
Een typisch commando (bijv. "stuur een mesh bericht") doorloopt deze stappen:
```
1. GUI: gebruiker typt bericht, klikt Send
2. GUI → SharedData: put_command("send_msg", {channel: 0, text: "Hello"})
3. BLEWorker: haalt command uit queue
4. meshcore: serialiseert naar binary packet
→ [header][cmd_type][channel_idx][payload_len][utf8_text]
5. bleak: write_gatt_char(NUS_RX_UUID, packet)
6. dbus_fast: GattCharacteristic1.WriteValue(packet_bytes, {})
7. BlueZ: schrijft naar HCI controller
8. HCI: stuurt BLE PDU via radio
9. T1000-E: ontvangt, verwerkt, stuurt via LoRa mesh
```
De response (of een inkomend mesh bericht) loopt de omgekeerde route:
```
1. T1000-E: ontvangt mesh bericht via LoRa
2. T1000-E → HCI: BLE notification met data
3. BlueZ: ontvangt notification, stuurt via D-Bus
4. dbus_fast: roept de notify callback in bleak aan
5. bleak: roept de registered callback in meshcore aan
6. meshcore: deserialiseert binary → Event(type, payload)
7. BLEWorker: EventHandler verwerkt het event
→ dedup check → naam resolutie → path hash extractie
8. SharedData: add_message(Message.incoming(...))
9. GUI: ziet message_updated flag bij volgende 500ms poll
```
### 5.3 Waarom subscribe-before-send?
BLE notifications zijn asynchroon. Als meshcore eerst het commando schrijft en *daarna* `start_notify()` aanroept, kan de response al verloren zijn gegaan voordat de listener klaar is. Dit was een bug in de originele meshcore_py die leidde tot ~2 minuten startup delay:
```
❌ Oud (race condition):
write(RX, command) → device antwoordt direct
start_notify(TX) → te laat, response is al weg
✅ Nieuw (PR #52):
start_notify(TX) → listener actief
write(RX, command) → device antwoordt
callback fired → response ontvangen
```
---
## 6. Pairing en Bonding
### 6.1 Waarom PIN pairing?
Het T1000-E device is geconfigureerd met BLE PIN `123456` (instelbaar via firmware). Dit voorkomt dat willekeurige BLE clients verbinden. BlueZ ondersteunt PIN pairing via het **Agent** mechanisme.
### 6.2 Agent interface
BlueZ definieert de `org.bluez.Agent1` D-Bus interface. Onze `BluezAgent` class implementeert deze callbacks:
| Methode | D-Bus Signature | Wanneer aangeroepen | Ons antwoord |
|---------|----------------|--------------------:|-------------|
| `RequestPinCode` | `o → s` | Device vraagt PIN | `"123456"` |
| `RequestPasskey` | `o → u` | Device vraagt numeriek passkey | `123456` (uint32) |
| `DisplayPasskey` | `oqu → ` | Passkey tonen (info only) | (log only) |
| `RequestConfirmation` | `ou → ` | Bevestig passkey match | (accept) |
| `AuthorizeService` | `os → ` | Service autorisatie | (accept) |
| `Cancel` | ` → ` | Pairing geannuleerd | (log only) |
| `Release` | ` → ` | Agent niet meer nodig | (cleanup) |
### 6.3 Het bonding probleem
Na succesvolle pairing slaat BlueZ de encryption keys op in `/var/lib/bluetooth/<adapter>/<device>/info`. Dit heet een "bond". Bij de volgende connectie probeert BlueZ deze keys te hergebruiken.
**Het probleem:** Het T1000-E verwerpt na ~2 uur de BLE verbinding (firmware timeout). BlueZ heeft nog de oude bond keys, maar het device heeft ze verworpen. Resultaat:
```
BlueZ: "Ik heb keys voor dit device, gebruik die"
T1000-E: "Ik ken deze keys niet → Reject (PIN or Key Missing)"
BlueZ: "Pairing failed"
```
**De oplossing:** Vóór elke reconnectie verwijderen we de bond:
```
remove_bond() → Adapter1.RemoveDevice() → BlueZ wist keys
connect() → BlueZ: "Geen keys, start verse pairing"
Agent → levert PIN → verse pairing succesvol
```
---
## 7. D-Bus Policy
Normale gebruikers mogen standaard niet alle BlueZ D-Bus interfaces aanspreken. De D-Bus policy file (`/etc/dbus-1/system.d/meshcore-ble.conf`) geeft de gebruiker die de service draait toestemming:
```xml
<busconfig>
<policy user="hans">
<allow send_destination="org.bluez"/>
<allow send_interface="org.bluez.Agent1"/>
<allow send_interface="org.bluez.AgentManager1"/>
</policy>
</busconfig>
```
Zonder deze policy:
- `bleak` kan nog steeds verbinden (bleak gebruikt een standaard D-Bus policy die al met BlueZ meekomt)
- Onze **agent** kan zich niet registreren → PIN pairing faalt
- Onze **bond cleanup** kan `RemoveDevice` niet aanroepen
---
## 8. Samenvatting Dependencies
```
meshcore-gui
├── nicegui → Web UI framework (onze GUI)
├── meshcore → MeshCore protocol (commando's, events)
│ └── bleak → BLE abstractie (connect, notify, write)
│ └── dbus_fast → D-Bus communicatie (naar BlueZ)
├── meshcoredecoder → LoRa packet decryptie + route extractie
└── (geen extra) → ble_agent.py en ble_reconnect.py
gebruiken dbus_fast die al via bleak
geïnstalleerd is
```
Alle BLE-gerelateerde functionaliteit draait op precies **vier Python packages**: `bleak`, `dbus_fast`, `meshcore`, en `meshcoredecoder`. Er zijn geen system-level dependencies meer nodig buiten `bluez` zelf (geen `bluez-tools`, geen `bt-agent`).

639
docs/ble/ble_capture_workflow_t_1000_e_explanation.md Normal file
View File

@@ -0,0 +1,639 @@
# BLE Capture Workflow T1000-e — Explanation & Background
> **Note:** This document is BLE-specific and kept for historical reference. The current GUI uses USB serial.
> **Source:** `ble_capture_workflow_t_1000_e.md`
>
> This document is a **companion guide** to the original technical working document. It provides:
> - Didactic explanation of BLE concepts and terminology
> - Background knowledge about GATT services and how they work
> - Context for better understanding future BLE projects
>
> **Intended audience:** Myself, as a long-term reference.
---
## 1. What is this document about?
This document explains the BLE concepts and terminology behind communicating with a **MeshCore T1000-e** radio from a Linux computer. It covers:
- How BLE connections work and how they differ from Classic Bluetooth
- The GATT service model and the Nordic UART Service (NUS) used by MeshCore
- Why BLE session ownership matters and how it can cause connection failures
**The key message in one sentence:**
> Only **one BLE client at a time** can be connected to the T1000-e. If something else is already connected, your connection will fail.
---
## 2. Terms and abbreviations explained
### 2.1 BLE — Bluetooth Low Energy
BLE is an **energy-efficient variant of Bluetooth**, designed for devices that need to run on a battery for months or years.
| Property | Classic Bluetooth | BLE |
|----------|-------------------|-----|
| Power consumption | High | Very low |
| Data rate | High | Low |
| Typical use | Audio, file transfer | Sensors, IoT, MeshCore |
**Analogy:** Classic Bluetooth is like a phone call (constantly connected, high energy). BLE is like sending text messages (brief contact when needed, low energy).
---
### 2.2 GATT — Generic Attribute Profile
GATT is the **structure** through which BLE devices expose their data. Think of it as a **digital bulletin board** with a fixed layout:
```
Service (category)
└── Characteristic (specific data point)
└── Descriptor (additional configuration)
```
**Example for MeshCore:**
```
Nordic UART Service (NUS)
├── RX Characteristic → messages from radio to computer
└── TX Characteristic → messages from computer to radio
```
---
### 2.3 NUS — Nordic UART Service
NUS is a **standard BLE service** developed by Nordic Semiconductor. It simulates an old-fashioned serial port (UART) over Bluetooth.
- **RX** (Receive): Data you **receive** from the device
- **TX** (Transmit): Data you **send** to the device
Note: RX/TX are from the computer's perspective, not the radio's.
#### Is NUS a protocol?
**No.** NUS is a **service specification**, not a protocol. This is an important distinction:
| Level | What is it | Example |
|-------|-----------|---------|
| **Protocol** | Rules for communication | BLE, ATT, GATT |
| **Service** | Collection of related characteristics | NUS, Heart Rate Service |
| **Characteristic** | Specific data point within a service | RX, TX |
**Restaurant analogy:**
| Concept | Restaurant analogy |
|---------|--------------------|
| **Protocol (GATT)** | The rules: you order from the waiter, food comes from the kitchen |
| **Service (NUS)** | A specific menu (e.g. "breakfast menu") |
| **Characteristics** | The individual dishes on that menu |
People often say "we're using the NUS protocol", but strictly speaking **GATT** is the protocol and **NUS** is a service offered via GATT.
---
### 2.4 Other GATT services (official and custom)
NUS is just one of many BLE services. The **Bluetooth SIG** (the organisation behind Bluetooth) defines dozens of official services. In addition, manufacturers can create their own (custom) services.
#### Official services (Bluetooth SIG)
These services have a **16-bit UUID** and are standardised for interoperability:
| Service | UUID | Application |
|---------|------|-------------|
| **Heart Rate Service** | 0x180D | Heart rate monitors, fitness devices |
| **Battery Service** | 0x180F | Reporting battery level |
| **Device Information** | 0x180A | Manufacturer, model number, firmware version |
| **Blood Pressure** | 0x1810 | Blood pressure monitors |
| **Health Thermometer** | 0x1809 | Medical thermometers |
| **Cycling Speed and Cadence** | 0x1816 | Bicycle sensors |
| **Environmental Sensing** | 0x181A | Temperature, humidity, pressure |
| **Glucose** | 0x1808 | Blood glucose meters |
| **HID over GATT** | 0x1812 | Keyboards, mice, gamepads |
| **Proximity** | 0x1802 | "Find My" functionality |
| **Generic Access** | 0x1800 | **Mandatory** — device name and appearance |
#### Custom/vendor-specific services
Manufacturers can define their own services with a **128-bit UUID**. Examples:
| Service | Manufacturer | Application |
|---------|--------------|-------------|
| **Nordic UART Service (NUS)** | Nordic Semiconductor | Serial port over BLE |
| **Apple Notification Center** | Apple | iPhone notifications to wearables |
| **Xiaomi Mi Band Service** | Xiaomi | Fitness tracker communication |
| **MeshCore Companion** | MeshCore | Radio communication (uses NUS) |
#### The difference: 16-bit vs. 128-bit UUID
| Type | Length | Example | Who can create it? |
|------|--------|---------|--------------------|
| **Official (SIG)** | 16-bit | `0x180D` | Bluetooth SIG only |
| **Custom** | 128-bit | `6e400001-b5a3-f393-e0a9-e50e24dcca9e` | Anyone |
The NUS service uses this 128-bit UUID:
```
6e400001-b5a3-f393-e0a9-e50e24dcca9e
```
#### Why this matters
In the MeshCore project we use **NUS** (a custom service) for communication. But when working with other BLE devices — such as a heart rate monitor or a smart thermostat — they typically use **official SIG services**.
The principle remains the same:
1. Discover which services the device offers
2. Find the right characteristic
3. Read, write, or subscribe to notify
---
### 2.5 Notify vs. Read
There are two ways to get data from a BLE device:
| Method | How it works | When to use |
|--------|-------------|-------------|
| **Read** | You actively request data | One-off values (e.g. battery status) |
| **Notify** | Device sends automatically when new data is available | Continuous data stream (e.g. messages) |
**Analogy:**
- **Read** = You call someone and ask "how are you?"
- **Notify** = You automatically receive a WhatsApp message when there's news
For MeshCore captures you use **Notify** — after all, you want to know when a message arrives.
---
### 2.6 CCCD — Client Characteristic Configuration Descriptor
The CCCD is the **on/off switch for Notify**. Technically:
1. Your computer writes a `1` to the CCCD
2. The device now knows: "this client wants notifications"
3. When new data arrives, the device automatically sends a message
**The crucial point:** Only **one client at a time** can activate the CCCD. A second client will receive the error:
```
Notify acquired
```
This means: "someone else has already enabled notify."
---
### 2.7 Pairing, Bonding and Trust
These are three separate steps in the BLE security process:
| Step | What happens | Analogy |
|------|-------------|---------|
| **Pairing** | Devices exchange cryptographic keys | You meet someone and exchange phone numbers |
| **Bonding** | The keys are stored permanently | You save the number in your contacts |
| **Trust** | The system trusts the device automatically | You add someone to your favourites |
After these three steps, you no longer need to enter the PIN code each time.
**Verification on Linux:**
```bash
bluetoothctl info AA:BB:CC:DD:EE:FF | egrep -i "Paired|Bonded|Trusted"
```
Expected output:
```
Paired: yes
Bonded: yes
Trusted: yes
```
---
### 2.8 Ownership — The core problem
**Ownership** is an informal term indicating: "which client currently holds the active GATT session with notify?"
**Analogy:** Think of a walkie-talkie where only one person can listen at a time:
- If GNOME Bluetooth Manager is already connected → it is the "owner"
- If your Python script then tries to connect → it won't get access
**Typical "owners" that cause problems:**
- GNOME Bluetooth GUI (often runs in the background)
- `bluetoothctl connect` (makes bluetoothctl the owner)
- Phone with Bluetooth enabled
- Other BLE apps
---
### 2.9 BlueZ
**BlueZ** is the official Bluetooth stack for Linux. It is the software that handles all Bluetooth communication between your applications and the hardware.
---
### 2.10 Bleak
**Bleak** is a Python library for BLE communication. It builds on top of BlueZ (Linux), Core Bluetooth (macOS) or WinRT (Windows).
---
## 3. BLE versus Classic Bluetooth
A common question: are BLE and "regular" Bluetooth the same thing? The answer is **no** — they are different technologies that happen to share the same name and frequency band.
### 3.1 Two flavours of Bluetooth
Since Bluetooth 4.0 (2010) there are **two separate radio systems** within the Bluetooth standard:
| Name | Technical term | Characteristics |
|------|---------------|-----------------|
| **Classic Bluetooth** | BR/EDR (Basic Rate / Enhanced Data Rate) | High data rate, continuous connection, more power |
| **Bluetooth Low Energy** | BLE (also: Bluetooth Smart) | Low data rate, short bursts, very efficient |
**Crucially:** These are **different radio protocols** that cannot communicate directly with each other.
### 3.2 Protocol and hardware
Bluetooth (both Classic and BLE) encompasses **multiple layers** — it is not just a protocol, but also hardware:
```
┌─────────────────────────────────────────┐
│ SOFTWARE │
│ ┌───────────────────────────────────┐ │
│ │ Application (your code) │ │
│ ├───────────────────────────────────┤ │
│ │ Profiles / GATT Services │ │
│ ├───────────────────────────────────┤ │
│ │ Protocols (ATT, L2CAP, etc.) │ │
│ ├───────────────────────────────────┤ │
│ │ Host Controller Interface (HCI) │ │ ← Software/firmware boundary
│ └───────────────────────────────────┘ │
├─────────────────────────────────────────┤
│ FIRMWARE │
│ ┌───────────────────────────────────┐ │
│ │ Link Layer / Controller │ │
│ └───────────────────────────────────┘ │
├─────────────────────────────────────────┤
│ HARDWARE │
│ ┌───────────────────────────────────┐ │
│ │ Radio (2.4 GHz transceiver) │ │
│ ├───────────────────────────────────┤ │
│ │ Antenna │ │
│ └───────────────────────────────────┘ │
└─────────────────────────────────────────┘
```
### 3.3 Where is the difference?
The difference exists across **multiple layers**, not just the protocol:
| Layer | Classic (BR/EDR) | BLE | Hardware difference? |
|-------|------------------|-----|---------------------|
| **Radio** | GFSK, π/4-DQPSK, 8DPSK | GFSK | **Yes** — different modulation |
| **Channels** | 79 channels, 1 MHz wide | 40 channels, 2 MHz wide | **Yes** — different layout |
| **Link Layer** | LMP (Link Manager Protocol) | LL (Link Layer) | **Yes** — different state machine |
| **Protocols** | L2CAP, RFCOMM, SDP | L2CAP, ATT, GATT | No — software |
### 3.4 Dual-mode devices
The overlap lies in devices that support **both**:
| Device type | Supports | Example |
|-------------|----------|---------|
| **Classic-only** | BR/EDR only | Old headsets, car audio |
| **BLE-only** (Bluetooth Smart) | BLE only | Fitness trackers, sensors, T1000-e |
| **Dual-Mode** (Bluetooth Smart Ready) | Both | Smartphones, laptops, ESP32 |
**Your smartphone** is dual-mode: it can talk to your classic Bluetooth headphones (BR/EDR) and to your MeshCore T1000-e (BLE).
### 3.5 Practical examples
| Scenario | What is used |
|----------|-------------|
| Music to your headphones | **Classic** (A2DP profile) |
| Heart rate from your smartwatch | **BLE** (Heart Rate Service) |
| Sending a file to a laptop | **Classic** (OBEX/FTP profile) |
| Reading the MeshCore T1000-e | **BLE** (NUS service) |
| Hands-free calling in the car | **Classic** (HFP profile) |
| Controlling a smart light | **BLE** (custom GATT service) |
---
## 4. BLE channel layout and frequency hopping
### 4.1 The 40 BLE channels
The 2.4 GHz ISM band runs from **2400 MHz to 2483.5 MHz** (83.5 MHz wide).
BLE divides this into **40 channels of 2 MHz each**:
```
2400 MHz 2480 MHz
│ │
▼ ▼
┌──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┐
│00│01│02│03│04│05│06│07│08│09│10│11│12│13│14│15│16│17│18│19│...→ 39
└──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┘
└──────────────────────────────────────────────────────┘
2 MHz per channel
```
**Total:** 40 × 2 MHz = **80 MHz** used
### 4.2 Advertising vs. data channels
The 40 channels are not all equal:
| Type | Channels | Function |
|------|----------|----------|
| **Advertising** | 3 (nos. 37, 38, 39) | Device discovery, initiating connections |
| **Data** | 37 (nos. 0-36) | Actual communication after connection |
The advertising channels are strategically chosen to **avoid Wi-Fi interference**:
```
Wi-Fi channel 1 Wi-Fi channel 6 Wi-Fi channel 11
│ │ │
▼ ▼ ▼
────████████─────────────█████████────────────████████────
BLE: ▲ ▲ ▲
Ch.37 Ch.38 Ch.39
(advertising channels sit between the Wi-Fi channels)
```
### 4.3 Comparison with Classic Bluetooth
| Aspect | Classic (BR/EDR) | BLE |
|--------|------------------|-----|
| **Number of channels** | 79 | 40 |
| **Channel width** | 1 MHz | 2 MHz |
| **Total bandwidth** | 79 MHz | 80 MHz |
| **Frequency hopping** | Yes, all 79 | Yes, 37 data channels |
Classic has **more but narrower** channels, BLE has **fewer but wider** channels.
### 4.4 Frequency hopping: one channel at a time
**Key insight:** You only ever use **one channel at a time**. The 40 channels exist for **frequency hopping** — alternately switching channels to avoid interference:
```
Time →
┌───┐ ┌───┐ ┌───┐ ┌───┐
Ch. 12 │ ▓ │ │ │ │ │ │ ▓ │
└───┘ └───┘ └───┘ └───┘
┌───┐ ┌───┐ ┌───┐ ┌───┐
Ch. 07 │ │ │ ▓ │ │ │ │ │
└───┘ └───┘ └───┘ └───┘
┌───┐ ┌───┐ ┌───┐ ┌───┐
Ch. 31 │ │ │ │ │ ▓ │ │ │
└───┘ └───┘ └───┘ └───┘
↑ ↑ ↑ ↑
Packet 1 Packet 2 Packet 3 Packet 4
```
This is **not parallel communication** — it is serial with alternating frequencies.
---
## 5. Two meanings of "serial"
When we say "NUS is serial", this can cause confusion. The word "serial" has **two different meanings** in this context.
### 5.1 Radio level: always serial
**All** wireless communication is serial at the physical level — you only have **one radio channel at a time** and bits go into the air **one after another**:
```
Radio wave: ▁▂▃▄▅▆▇█▇▆▅▄▃▂▁▂▃▄▅▆▇█▇▆▅▄▃▂▁
Bits: 0 1 1 0 1 0 0 1 1 1 0 1 0 0 1 0 → one by one
```
The 40 channels are for **frequency hopping**, not for parallel transmission. This applies to **all** BLE services — NUS, Heart Rate, Battery, all of them.
### 5.2 Data level: NUS simulates a serial port
When we say "NUS is a serial service", we mean something different:
**NUS simulates an old serial port (RS-232/UART):**
```
Historical (1980s-2000s):
Computer Device
┌──────┐ Serial cable ┌──────┐
│ COM1 │←────────────────→│ UART │
└──────┘ (RS-232) └──────┘
Bytes: 0x48 0x65 0x6C 0x6C 0x6F ("Hello")
└─────────────────────┘
No structure, just a stream of bytes
```
**NUS mimics this over BLE:**
```
Today:
Computer Device
┌──────┐ BLE (NUS) ┌──────┐
│ App │←~~~~~~~~~~~~~~~~~~~~→│ MCU │
└──────┘ (wireless) └──────┘
Behaves as if there were a serial cable
```
### 5.3 Comparison: serial vs. structured
| Aspect | NUS (serial) | Heart Rate (structured) |
|--------|-------------|------------------------|
| **Radio** | Serial, frequency hopping | Serial, frequency hopping |
| **Data** | Unstructured byte stream | Fixed fields with meaning |
| **Who determines the format?** | You (custom protocol) | Bluetooth SIG (specification) |
### 5.4 Analogy: motorway with lanes
Think of a **motorway with 40 lanes** (the channels):
- You may only use **one lane at a time**
- You regularly switch lanes (frequency hopping, to avoid collisions)
- The **cargo** you transport can differ:
| Service | Cargo analogy |
|---------|--------------|
| **NUS** | Loose items mixed together (flexible, but you need to figure out what's what) |
| **Heart Rate** | Standardised pallets (everyone knows what goes where) |
The **motorway works the same** — the difference lies in how you organise the cargo.
---
## 6. Serial vs. structured services (deep dive)
An important distinction that is often overlooked: **not all BLE services work the same way**. There are fundamentally two approaches.
### 6.1 Serial services (stream-based)
**NUS (Nordic UART Service)** is designed to **simulate a serial port**:
- Continuous stream of raw bytes
- No imposed structure
- You determine the format and meaning yourself
**Analogy:** A serial service is like a **phone line** — you can say whatever you want, in any language, without fixed rules.
```
Example NUS data (MeshCore):
0x01 0x0A 0x48 0x65 0x6C 0x6C 0x6F ...
└── Meaning determined by MeshCore protocol, not by BLE
```
### 6.2 Structured services (field-based)
Most official SIG services work **differently** — they define **exactly** which bytes mean what:
**Analogy:** A structured service is like a **tax form** — each field has a fixed meaning and a prescribed format.
#### Example: Heart Rate Measurement
```
Byte 0: Flags (bitfield)
├── Bit 0: 0 = heart rate in 1 byte, 1 = heart rate in 2 bytes
├── Bit 1-2: Sensor contact status
├── Bit 3: Energy expended present?
└── Bit 4: RR-interval present?
Byte 1(-2): Heart rate value
Byte N...: Optional additional fields (depending on flags)
```
**Concrete example:**
```
Received bytes: 0x00 0x73
0x00 = Flags: 8-bit format, no additional fields
0x73 = 115 decimal → heart rate is 115 bpm
```
So you don't receive the text "115", but a binary packet that you need to **parse** according to the specification.
#### Example: Battery Level
Simpler — just **1 byte**:
```
Received byte: 0x5A
0x5A = 90 decimal → battery is 90%
```
#### Example: Environmental Sensing (temperature)
```
Received bytes: 0x9C 0x08
Little-endian 16-bit signed integer: 0x089C = 2204
Resolution: 0.01°C
Temperature: 2204 × 0.01 = 22.04°C
```
### 6.3 Comparison table
| Aspect | Serial (NUS) | Structured (SIG) |
|--------|-------------|------------------|
| **Data format** | Free, self-determined | Fixed, by specification |
| **Who defines the format?** | You / the manufacturer | Bluetooth SIG |
| **Where to find the spec?** | Own documentation / source code | bluetooth.com/specifications |
| **Parsing** | Build your own parser | Standard parser possible |
| **Interoperability** | Own software only | Any conformant app/device |
| **Flexibility** | Maximum | Limited to spec |
| **Complexity** | Easy to get started | Reading the spec required |
### 6.4 Examples of structured services
| Service | Characteristic | Data format |
|---------|----------------|-------------|
| **Battery Service** | Battery Level | 1 byte: 0-100 (percentage) |
| **Heart Rate** | Heart Rate Measurement | Flags + 8/16-bit HR + optional fields |
| **Health Thermometer** | Temperature Measurement | IEEE-11073 FLOAT (4 bytes) |
| **Blood Pressure** | Blood Pressure Measurement | Compound: systolic, diastolic, MAP, pulse |
| **Cycling Speed & Cadence** | CSC Measurement | 32-bit counters + 16-bit time |
| **Environmental Sensing** | Temperature | 16-bit signed, resolution 0.01°C |
| **Environmental Sensing** | Humidity | 16-bit unsigned, resolution 0.01% |
| **Environmental Sensing** | Pressure | 32-bit unsigned, resolution 0.1 Pa |
### 6.5 When to use which approach?
| Situation | Recommended approach |
|-----------|---------------------|
| Custom protocol (MeshCore, custom IoT) | **Serial** (NUS or custom service) |
| Standard use case (heart rate, battery) | **Structured** (SIG service) |
| Interoperability with existing apps required | **Structured** (SIG service) |
| Complex, variable data structures | **Serial** with custom protocol |
| Quick prototype without studying specs | **Serial** (NUS) |
### 6.6 Why MeshCore uses NUS
MeshCore chose NUS (serial) because:
1. **Flexibility** — The Companion Protocol requires its own framing
2. **No suitable SIG service** — There is no "Mesh Radio Service" standard
3. **Bidirectional communication** — NUS offers both RX and TX characteristics
4. **Simplicity** — No need to implement a complex SIG specification
The downside: you can't just use any arbitrary BLE app to talk to MeshCore — you need software that understands the MeshCore Companion Protocol.
---
## 7. The OSI model in context
The document places the problem in a **layer model**. This helps understand *where* the problem lies:
| Layer | Name | In this project | Problem here? |
|-------|------|-----------------|---------------|
| 7 | Application | MeshCore Companion Protocol | No |
| 6 | Presentation | Frame encoding (hex) | No |
| **5** | **Session** | **GATT client ↔ server session** | **★ YES** |
| 4 | Transport | ATT / GATT | No |
| 2 | Data Link | BLE Link Layer | No |
| 1 | Physical | 2.4 GHz radio | No |
**Conclusion:** The ownership problem sits at **layer 5 (session)**. The firmware and protocol are not the problem — it's about who "owns" the session.
---
## 8. Conclusion
The key takeaways from this document:
- ✅ MeshCore BLE companion **works correctly** on Linux
- ✅ The firmware **does not block notify**
- ✅ The only requirement is: **exactly one active BLE client per radio**
Understanding the ownership model and BLE fundamentals described here is essential for working with any BLE-connected MeshCore device.
---
## 9. References
- MeshCore Companion Radio Protocol: [GitHub Wiki](https://github.com/meshcore-dev/MeshCore/wiki/Companion-Radio-Protocol)
- Bluetooth SIG Assigned Numbers (official services): [bluetooth.com/specifications/assigned-numbers](https://www.bluetooth.com/specifications/assigned-numbers/)
- Bluetooth SIG GATT Specifications: [bluetooth.com/specifications/specs](https://www.bluetooth.com/specifications/specs/)
- Nordic Bluetooth Numbers Database: [GitHub](https://github.com/NordicSemiconductor/bluetooth-numbers-database)
- GATT Explanation (Adafruit): [learn.adafruit.com](https://learn.adafruit.com/introduction-to-bluetooth-low-energy/gatt)
- Bleak documentation: [bleak.readthedocs.io](https://bleak.readthedocs.io/)
- BlueZ: [bluez.org](http://www.bluez.org/)
---
> **Document:** `ble_capture_workflow_t_1000_e_explanation.md`
> **Based on:** `ble_capture_workflow_t_1000_e.md`

639
docs/ble/ble_capture_workflow_t_1000_e_uitleg.md Normal file
View File

@@ -0,0 +1,639 @@
# BLE Capture Workflow T1000-e — Uitleg & Achtergrond
> **Note:** Dit document is BLE-specifiek en wordt bewaard als referentie. De huidige GUI gebruikt USB-serieel.
> **Bron:** `ble_capture_workflow_t_1000_e.md`
>
> Dit document is een **verdiepingsdocument** bij het originele technische werkdocument. Het biedt:
> - Didactische uitleg van BLE-concepten en terminologie
> - Achtergrondkennis over GATT-services en hun werking
> - Context om toekomstige BLE-projecten beter te begrijpen
>
> **Doelgroep:** Mezelf, als referentie voor de lange termijn.
---
## 1. Waar gaat dit document over?
Dit document legt de BLE-concepten en terminologie uit achter de communicatie met een **MeshCore T1000-e** radio vanaf een Linux-computer. Het behandelt:
- Hoe BLE-verbindingen werken en hoe ze verschillen van Classic Bluetooth
- Het GATT-servicemodel en de Nordic UART Service (NUS) die MeshCore gebruikt
- Waarom BLE-sessie-ownership belangrijk is en hoe het verbindingsproblemen kan veroorzaken
**De kernboodschap in één zin:**
> Er mag maar **één BLE-client tegelijk** verbonden zijn met de T1000-e. Als iets anders al verbonden is, faalt jouw verbinding.
---
## 2. Begrippen en afkortingen uitgelegd
### 2.1 BLE — Bluetooth Low Energy
BLE is een **zuinige variant van Bluetooth**, ontworpen voor apparaten die maanden of jaren op een batterij moeten werken.
| Eigenschap | Klassiek Bluetooth | BLE |
|------------|-------------------|-----|
| Stroomverbruik | Hoog | Zeer laag |
| Datasnelheid | Hoog | Laag |
| Typisch gebruik | Audio, bestanden | Sensoren, IoT, MeshCore |
**Analogie:** Klassiek Bluetooth is als een telefoongesprek (constant verbonden, veel energie). BLE is als SMS'jes sturen (kort contact wanneer nodig, weinig energie).
---
### 2.2 GATT — Generic Attribute Profile
GATT is de **structuur** waarmee BLE-apparaten hun data aanbieden. Zie het als een **digitaal prikbord** met een vaste indeling:
```
Service (categorie)
└── Characteristic (specifiek datapunt)
└── Descriptor (extra configuratie)
```
**Voorbeeld voor MeshCore:**
```
Nordic UART Service (NUS)
├── RX Characteristic → berichten van radio naar computer
└── TX Characteristic → berichten van computer naar radio
```
---
### 2.3 NUS — Nordic UART Service
NUS is een **standaard BLE-service** ontwikkeld door Nordic Semiconductor. Het simuleert een ouderwetse seriële poort (UART) over Bluetooth.
- **RX** (Receive): Data die je **ontvangt** van het apparaat
- **TX** (Transmit): Data die je **verstuurt** naar het apparaat
Let op: RX/TX zijn vanuit het perspectief van de computer, niet van de radio.
#### Is NUS een protocol?
**Nee.** NUS is een **servicespecificatie**, geen protocol. Dit is een belangrijk onderscheid:
| Niveau | Wat is het | Voorbeeld |
|--------|-----------|-----------|
| **Protocol** | Regels voor communicatie | BLE, ATT, GATT |
| **Service** | Verzameling van gerelateerde characteristics | NUS, Heart Rate Service |
| **Characteristic** | Specifiek datapunt binnen een service | RX, TX |
**Analogie met een restaurant:**
| Concept | Restaurant-analogie |
|---------|---------------------|
| **Protocol (GATT)** | De regels: je bestelt bij de ober, eten komt uit de keuken |
| **Service (NUS)** | Een specifieke menukaart (bijv. "ontbijtmenu") |
| **Characteristics** | De individuele gerechten op dat menu |
Mensen zeggen vaak "we gebruiken het NUS-protocol", maar strikt genomen is **GATT** het protocol en is **NUS** een service die via GATT wordt aangeboden.
---
### 2.4 Andere GATT-services (officieel en custom)
NUS is slechts één van vele BLE-services. De **Bluetooth SIG** (de organisatie achter Bluetooth) definieert tientallen officiële services. Daarnaast kunnen fabrikanten eigen (custom) services maken.
#### Officiële services (Bluetooth SIG)
Deze services hebben een **16-bit UUID** en zijn gestandaardiseerd voor interoperabiliteit:
| Service | UUID | Toepassing |
|---------|------|------------|
| **Heart Rate Service** | 0x180D | Hartslagmeters, fitnessapparaten |
| **Battery Service** | 0x180F | Batterijniveau rapporteren |
| **Device Information** | 0x180A | Fabrikant, modelnummer, firmwareversie |
| **Blood Pressure** | 0x1810 | Bloeddrukmeters |
| **Health Thermometer** | 0x1809 | Medische thermometers |
| **Cycling Speed and Cadence** | 0x1816 | Fietssensoren |
| **Environmental Sensing** | 0x181A | Temperatuur, luchtvochtigheid, druk |
| **Glucose** | 0x1808 | Bloedglucosemeters |
| **HID over GATT** | 0x1812 | Toetsenborden, muizen, gamepads |
| **Proximity** | 0x1802 | "Find My"-functionaliteit |
| **Generic Access** | 0x1800 | **Verplicht** — apparaatnaam en uiterlijk |
#### Custom/vendor-specific services
Fabrikanten kunnen eigen services definiëren met een **128-bit UUID**. Voorbeelden:
| Service | Fabrikant | Toepassing |
|---------|-----------|------------|
| **Nordic UART Service (NUS)** | Nordic Semiconductor | Seriële poort over BLE |
| **Apple Notification Center** | Apple | iPhone notificaties naar wearables |
| **Xiaomi Mi Band Service** | Xiaomi | Fitnesstracker communicatie |
| **MeshCore Companion** | MeshCore | Radio-communicatie (gebruikt NUS) |
#### Het verschil: 16-bit vs. 128-bit UUID
| Type | Lengte | Voorbeeld | Wie mag het maken? |
|------|--------|-----------|-------------------|
| **Officieel (SIG)** | 16-bit | `0x180D` | Alleen Bluetooth SIG |
| **Custom** | 128-bit | `6e400001-b5a3-f393-e0a9-e50e24dcca9e` | Iedereen |
De NUS-service gebruikt bijvoorbeeld deze 128-bit UUID:
```
6e400001-b5a3-f393-e0a9-e50e24dcca9e
```
#### Waarom dit relevant is
In het MeshCore-project gebruiken we **NUS** (een custom service) voor de communicatie. Maar als je met andere BLE-apparaten werkt — zoals een hartslagmeter of een slimme thermostaat — dan gebruiken die vaak **officiële SIG-services**.
Het principe blijft hetzelfde:
1. Ontdek welke services het apparaat aanbiedt
2. Zoek de juiste characteristic
3. Lees, schrijf, of abonneer op notify
---
### 2.5 Notify vs. Read
Er zijn twee manieren om data van een BLE-apparaat te krijgen:
| Methode | Werking | Wanneer gebruiken |
|---------|---------|-------------------|
| **Read** | Jij vraagt actief om data | Eenmalige waarden (bijv. batterijstatus) |
| **Notify** | Apparaat stuurt automatisch bij nieuwe data | Continue datastroom (bijv. berichten) |
**Analogie:**
- **Read** = Je belt iemand en vraagt "hoe gaat het?"
- **Notify** = Je krijgt automatisch een WhatsApp-bericht als er nieuws is
Voor MeshCore-captures gebruik je **Notify** — je wilt immers weten wanneer er een bericht binnenkomt.
---
### 2.6 CCCD — Client Characteristic Configuration Descriptor
De CCCD is de **aan/uit-schakelaar voor Notify**. Technisch gezien:
1. Jouw computer schrijft een `1` naar de CCCD
2. Het apparaat weet nu: "deze client wil notificaties"
3. Bij nieuwe data stuurt het apparaat automatisch een bericht
**Het cruciale punt:** Slechts **één client tegelijk** kan de CCCD activeren. Een tweede client krijgt de foutmelding:
```
Notify acquired
```
Dit betekent: "iemand anders heeft notify al ingeschakeld."
---
### 2.7 Pairing, Bonding en Trust
Dit zijn drie afzonderlijke stappen in het BLE-beveiligingsproces:
| Stap | Wat gebeurt er | Analogie |
|------|----------------|----------|
| **Pairing** | Apparaten wisselen cryptografische sleutels uit | Je maakt kennis en wisselt telefoonnummers |
| **Bonding** | De sleutels worden permanent opgeslagen | Je slaat het nummer op in je contacten |
| **Trust** | Het systeem vertrouwt het apparaat automatisch | Je zet iemand in je favorieten |
Na deze drie stappen hoef je niet elke keer opnieuw de pincode in te voeren.
**Controle in Linux:**
```bash
bluetoothctl info literal:AA:BB:CC:DD:EE:FF | egrep -i "Paired|Bonded|Trusted"
```
Verwachte output:
```
Paired: yes
Bonded: yes
Trusted: yes
```
---
### 2.8 Ownership — Het kernprobleem
**Ownership** is een informele term die aangeeft: "welke client heeft op dit moment de actieve GATT-sessie met notify?"
**Analogie:** Denk aan een walkietalkie waarbij maar één persoon tegelijk kan luisteren:
- Als GNOME Bluetooth Manager al verbonden is → die is de "eigenaar"
- Als jouw Python-script daarna probeert te verbinden → krijgt het geen toegang
**Typische "eigenaren" die problemen veroorzaken:**
- GNOME Bluetooth GUI (draait vaak op de achtergrond)
- `bluetoothctl connect` (maakt bluetoothctl de eigenaar)
- Telefoon met Bluetooth aan
- Andere BLE-apps
---
### 2.9 BlueZ
**BlueZ** is de officiële Bluetooth-stack voor Linux. Het is de software die alle Bluetooth-communicatie afhandelt tussen je applicaties en de hardware.
---
### 2.10 Bleak
**Bleak** is een Python-bibliotheek voor BLE-communicatie. Het bouwt voort op BlueZ (Linux), Core Bluetooth (macOS) of WinRT (Windows).
---
## 3. BLE versus Classic Bluetooth
Een veelvoorkomende vraag: zijn BLE en "gewone" Bluetooth hetzelfde? Het antwoord is **nee** — het zijn verschillende technologieën die wel dezelfde naam en frequentieband delen.
### 3.1 Twee smaken van Bluetooth
Sinds Bluetooth 4.0 (2010) zijn er **twee afzonderlijke radiosystemen** binnen de Bluetooth-standaard:
| Naam | Technische term | Kenmerken |
|------|-----------------|-----------|
| **Classic Bluetooth** | BR/EDR (Basic Rate / Enhanced Data Rate) | Hoge datasnelheid, continue verbinding, meer stroom |
| **Bluetooth Low Energy** | BLE (ook: Bluetooth Smart) | Lage datasnelheid, korte bursts, zeer zuinig |
**Cruciaal:** Dit zijn **verschillende radioprotocollen** die niet rechtstreeks met elkaar kunnen communiceren.
### 3.2 Protocol én hardware
Bluetooth (zowel Classic als BLE) omvat **meerdere lagen** — het is niet alleen een protocol, maar ook hardware:
```
┌─────────────────────────────────────────┐
│ SOFTWARE │
│ ┌───────────────────────────────────┐ │
│ │ Applicatie (jouw code) │ │
│ ├───────────────────────────────────┤ │
│ │ Profielen / GATT Services │ │
│ ├───────────────────────────────────┤ │
│ │ Protocollen (ATT, L2CAP, etc.) │ │
│ ├───────────────────────────────────┤ │
│ │ Host Controller Interface (HCI) │ │ ← Grens software/firmware
│ └───────────────────────────────────┘ │
├─────────────────────────────────────────┤
│ FIRMWARE │
│ ┌───────────────────────────────────┐ │
│ │ Link Layer / Controller │ │
│ └───────────────────────────────────┘ │
├─────────────────────────────────────────┤
│ HARDWARE │
│ ┌───────────────────────────────────┐ │
│ │ Radio (2.4 GHz transceiver) │ │
│ ├───────────────────────────────────┤ │
│ │ Antenne │ │
│ └───────────────────────────────────┘ │
└─────────────────────────────────────────┘
```
### 3.3 Waar zit het verschil?
Het verschil zit op **meerdere lagen**, niet alleen protocol:
| Laag | Classic (BR/EDR) | BLE | Verschil in hardware? |
|------|------------------|-----|----------------------|
| **Radio** | GFSK, π/4-DQPSK, 8DPSK | GFSK | **Ja** — andere modulatie |
| **Kanalen** | 79 kanalen, 1 MHz breed | 40 kanalen, 2 MHz breed | **Ja** — andere indeling |
| **Link Layer** | LMP (Link Manager Protocol) | LL (Link Layer) | **Ja** — andere state machine |
| **Protocollen** | L2CAP, RFCOMM, SDP | L2CAP, ATT, GATT | Nee — software |
### 3.4 Dual-mode apparaten
De overlap zit in apparaten die **beide** ondersteunen:
| Apparaattype | Ondersteunt | Voorbeeld |
|--------------|-------------|-----------|
| **Classic-only** | Alleen BR/EDR | Oude headsets, auto-audio |
| **BLE-only** (Bluetooth Smart) | Alleen BLE | Fitnesstrackers, sensoren, T1000-e |
| **Dual-Mode** (Bluetooth Smart Ready) | Beide | Smartphones, laptops, ESP32 |
**Jouw smartphone** is dual-mode: hij kan praten met je klassieke Bluetooth-koptelefoon (BR/EDR) én met je MeshCore T1000-e (BLE).
### 3.5 Praktijkvoorbeelden
| Scenario | Wat wordt gebruikt |
|----------|-------------------|
| Muziek naar je koptelefoon | **Classic** (A2DP profiel) |
| Hartslag van je smartwatch | **BLE** (Heart Rate Service) |
| Bestand naar laptop sturen | **Classic** (OBEX/FTP profiel) |
| MeshCore T1000-e uitlezen | **BLE** (NUS service) |
| Handsfree bellen in auto | **Classic** (HFP profiel) |
| Slimme lamp bedienen | **BLE** (eigen GATT service) |
---
## 4. BLE kanaalindeling en frequency hopping
### 4.1 De 40 BLE-kanalen
De 2.4 GHz ISM-band loopt van **2400 MHz tot 2483.5 MHz** (83.5 MHz breed).
BLE verdeelt dit in **40 kanalen van elk 2 MHz**:
```
2400 MHz 2480 MHz
│ │
▼ ▼
┌──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┐
│00│01│02│03│04│05│06│07│08│09│10│11│12│13│14│15│16│17│18│19│...→ 39
└──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┘
└──────────────────────────────────────────────────────┘
2 MHz per kanaal
```
**Totaal:** 40 × 2 MHz = **80 MHz** gebruikt
### 4.2 Advertising vs. data kanalen
De 40 kanalen zijn niet allemaal gelijk:
| Type | Kanalen | Functie |
|------|---------|---------|
| **Advertising** | 3 (nrs. 37, 38, 39) | Apparaten vinden, verbinding starten |
| **Data** | 37 (nrs. 0-36) | Daadwerkelijke communicatie na verbinding |
De advertising-kanalen zijn strategisch gekozen om **Wi-Fi-interferentie** te vermijden:
```
Wi-Fi kanaal 1 Wi-Fi kanaal 6 Wi-Fi kanaal 11
│ │ │
▼ ▼ ▼
────████████─────────────█████████────────────████████────
BLE: ▲ ▲ ▲
Ch.37 Ch.38 Ch.39
(advertising kanalen zitten tússen de Wi-Fi kanalen)
```
### 4.3 Vergelijking met Classic Bluetooth
| Aspect | Classic (BR/EDR) | BLE |
|--------|------------------|-----|
| **Aantal kanalen** | 79 | 40 |
| **Kanaalbreedte** | 1 MHz | 2 MHz |
| **Totale bandbreedte** | 79 MHz | 80 MHz |
| **Frequency hopping** | Ja, alle 79 | Ja, 37 datakanalen |
Classic heeft **meer maar smallere** kanalen, BLE heeft **minder maar bredere** kanalen.
### 4.4 Frequency hopping: één kanaal tegelijk
**Belangrijk inzicht:** Je gebruikt altijd maar **één kanaal tegelijk**. De 40 kanalen zijn er voor **frequency hopping** — het afwisselend wisselen van kanaal om interferentie te vermijden:
```
Tijd →
┌───┐ ┌───┐ ┌───┐ ┌───┐
Ch. 12 │ ▓ │ │ │ │ │ │ ▓ │
└───┘ └───┘ └───┘ └───┘
┌───┐ ┌───┐ ┌───┐ ┌───┐
Ch. 07 │ │ │ ▓ │ │ │ │ │
└───┘ └───┘ └───┘ └───┘
┌───┐ ┌───┐ ┌───┐ ┌───┐
Ch. 31 │ │ │ │ │ ▓ │ │ │
└───┘ └───┘ └───┘ └───┘
↑ ↑ ↑ ↑
Pakket 1 Pakket 2 Pakket 3 Pakket 4
```
Dit is **geen parallelle communicatie** — het is serieel met wisselende frequentie.
---
## 5. Twee betekenissen van "serieel"
Wanneer we zeggen "NUS is serieel", kan dit verwarring veroorzaken. Het woord "serieel" heeft namelijk **twee verschillende betekenissen** in deze context.
### 5.1 Radio-niveau: altijd serieel
**Alle** draadloze communicatie is serieel op fysiek niveau — je hebt maar **één radiokanaal tegelijk** en bits gaan **na elkaar** de lucht in:
```
Radiogolf: ▁▂▃▄▅▆▇█▇▆▅▄▃▂▁▂▃▄▅▆▇█▇▆▅▄▃▂▁
Bits: 0 1 1 0 1 0 0 1 1 1 0 1 0 0 1 0 → één voor één
```
De 40 kanalen zijn voor **frequency hopping**, niet voor parallel versturen. Dit geldt voor **alle** BLE-services — NUS, Heart Rate, Battery, allemaal.
### 5.2 Data-niveau: NUS simuleert een seriële poort
Wanneer we zeggen "NUS is een seriële service", bedoelen we iets anders:
**NUS simuleert een oude seriële poort (RS-232/UART):**
```
Historisch (jaren '80-'00):
Computer Apparaat
┌──────┐ Seriële kabel ┌──────┐
│ COM1 │←────────────────→│ UART │
└──────┘ (RS-232) └──────┘
Bytes: 0x48 0x65 0x6C 0x6C 0x6F ("Hello")
└─────────────────────┘
Geen structuur, gewoon een stroom bytes
```
**NUS bootst dit na over BLE:**
```
Vandaag:
Computer Apparaat
┌──────┐ BLE (NUS) ┌──────┐
│ App │←~~~~~~~~~~~~~~~~~~~~→│ MCU │
└──────┘ (draadloos) └──────┘
Gedraagt zich alsof er een seriële kabel zit
```
### 5.3 Vergelijking: serieel vs. gestructureerd
| Aspect | NUS (serieel) | Heart Rate (gestructureerd) |
|--------|---------------|----------------------------|
| **Radio** | Serieel, frequency hopping | Serieel, frequency hopping |
| **Data** | Ongestructureerde bytestroom | Vaste velden met betekenis |
| **Wie bepaalt formaat?** | Jij (eigen protocol) | Bluetooth SIG (specificatie) |
### 5.4 Analogie: snelweg met rijstroken
Denk aan een **snelweg met 40 rijstroken** (de kanalen):
- Je mag maar **één rijstrook tegelijk** gebruiken
- Je wisselt regelmatig van rijstrook (frequency hopping, om botsingen te vermijden)
- De **vracht** die je vervoert kan verschillen:
| Service | Vracht-analogie |
|---------|-----------------|
| **NUS** | Losse spullen door elkaar (flexibel, maar jij moet uitzoeken wat wat is) |
| **Heart Rate** | Gestandaardiseerde pallets (iedereen weet wat waar zit) |
De **snelweg werkt hetzelfde** — het verschil zit in hoe je de vracht organiseert.
---
## 6. Seriële vs. gestructureerde services (verdieping)
Een belangrijk onderscheid dat vaak over het hoofd wordt gezien: **niet alle BLE-services werken hetzelfde**. Er zijn fundamenteel twee benaderingen.
### 6.1 Seriële services (stream-gebaseerd)
**NUS (Nordic UART Service)** is ontworpen om een **seriële poort te simuleren**:
- Continue datastroom van ruwe bytes
- Geen opgelegde structuur
- Jij bepaalt zelf het formaat en de betekenis
**Analogie:** Een seriële service is als een **telefoonlijn** — je kunt alles zeggen wat je wilt, in elke taal, zonder vaste regels.
```
Voorbeeld NUS-data (MeshCore):
0x01 0x0A 0x48 0x65 0x6C 0x6C 0x6F ...
└── Betekenis bepaald door MeshCore protocol, niet door BLE
```
### 6.2 Gestructureerde services (veld-gebaseerd)
De meeste officiële SIG-services werken **anders** — ze definiëren **exact** welke bytes wat betekenen:
**Analogie:** Een gestructureerde service is als een **belastingformulier** — elk vakje heeft een vaste betekenis en een voorgeschreven formaat.
#### Voorbeeld: Heart Rate Measurement
```
Byte 0: Flags (bitfield)
├── Bit 0: 0 = hartslag in 1 byte, 1 = hartslag in 2 bytes
├── Bit 1-2: Sensor contact status
├── Bit 3: Energy expended aanwezig?
└── Bit 4: RR-interval aanwezig?
Byte 1(-2): Heart rate waarde
Byte N...: Optionele extra velden (afhankelijk van flags)
```
**Concreet voorbeeld:**
```
Ontvangen bytes: 0x00 0x73
0x00 = Flags: 8-bit formaat, geen extra velden
0x73 = 115 decimaal → hartslag is 115 bpm
```
Je krijgt dus niet de tekst "115", maar een binair pakket dat je moet **parsen** volgens de specificatie.
#### Voorbeeld: Battery Level
Eenvoudiger — slechts **1 byte**:
```
Ontvangen byte: 0x5A
0x5A = 90 decimaal → batterij is 90%
```
#### Voorbeeld: Environmental Sensing (temperatuur)
```
Ontvangen bytes: 0x9C 0x08
Little-endian 16-bit signed integer: 0x089C = 2204
Resolutie: 0.01°C
Temperatuur: 2204 × 0.01 = 22.04°C
```
### 6.3 Vergelijkingstabel
| Aspect | Serieel (NUS) | Gestructureerd (SIG) |
|--------|---------------|----------------------|
| **Data-indeling** | Vrij, zelf bepalen | Vast, door specificatie |
| **Wie definieert het formaat?** | Jij / de fabrikant | Bluetooth SIG |
| **Waar vind je de specificatie?** | Eigen documentatie / broncode | bluetooth.com/specifications |
| **Parsing** | Eigen parser bouwen | Standaard parser mogelijk |
| **Interoperabiliteit** | Alleen eigen software | Elke conforme app/device |
| **Flexibiliteit** | Maximaal | Beperkt tot spec |
| **Complexiteit** | Eenvoudig te starten | Spec lezen vereist |
### 6.4 Voorbeelden van gestructureerde services
| Service | Characteristic | Data-formaat |
|---------|----------------|--------------|
| **Battery Service** | Battery Level | 1 byte: 0-100 (percentage) |
| **Heart Rate** | Heart Rate Measurement | Flags + 8/16-bit HR + optionele velden |
| **Health Thermometer** | Temperature Measurement | IEEE-11073 FLOAT (4 bytes) |
| **Blood Pressure** | Blood Pressure Measurement | Compound: systolisch, diastolisch, MAP, pulse |
| **Cycling Speed & Cadence** | CSC Measurement | 32-bit tellers + 16-bit tijd |
| **Environmental Sensing** | Temperature | 16-bit signed, resolutie 0.01°C |
| **Environmental Sensing** | Humidity | 16-bit unsigned, resolutie 0.01% |
| **Environmental Sensing** | Pressure | 32-bit unsigned, resolutie 0.1 Pa |
### 6.5 Wanneer welke aanpak?
| Situatie | Aanbevolen aanpak |
|----------|-------------------|
| Eigen protocol (MeshCore, custom IoT) | **Serieel** (NUS of eigen service) |
| Standaard use-case (hartslag, batterij) | **Gestructureerd** (SIG-service) |
| Interoperabiliteit met bestaande apps vereist | **Gestructureerd** (SIG-service) |
| Complexe, variabele datastructuren | **Serieel** met eigen protocol |
| Snel prototype zonder spec-studie | **Serieel** (NUS) |
### 6.6 Waarom MeshCore NUS gebruikt
MeshCore koos voor NUS (serieel) omdat:
1. **Flexibiliteit** — Het Companion Protocol heeft eigen framing nodig
2. **Geen passende SIG-service** — Er is geen "Mesh Radio Service" standaard
3. **Bidirectionele communicatie** — NUS biedt RX én TX characteristics
4. **Eenvoud** — Geen complexe SIG-specificatie implementeren
Het nadeel: je kunt niet zomaar een willekeurige BLE-app gebruiken om met MeshCore te praten — je hebt software nodig die het MeshCore Companion Protocol begrijpt.
---
## 7. Het OSI-model in context
Het document plaatst het probleem in een **lagenmodel**. Dit helpt begrijpen *waar* het probleem zit:
| Laag | Naam | In dit project | Probleem hier? |
|------|------|----------------|----------------|
| 7 | Applicatie | MeshCore Companion Protocol | Nee |
| 6 | Presentatie | Frame-encoding (hex) | Nee |
| **5** | **Sessie** | **GATT client ↔ server sessie** | **★ JA** |
| 4 | Transport | ATT / GATT | Nee |
| 2 | Data Link | BLE Link Layer | Nee |
| 1 | Fysiek | 2.4 GHz radio | Nee |
**Conclusie:** Het ownership-probleem zit op **laag 5 (sessie)**. De firmware en het protocol zijn niet het probleem — het gaat om wie de sessie "bezit".
---
## 8. Conclusie
De belangrijkste inzichten uit dit document:
- ✅ MeshCore BLE companion **werkt correct** op Linux
- ✅ De firmware **blokkeert notify niet**
- ✅ Het enige vereiste is: **exact één actieve BLE-client per radio**
Het begrijpen van het ownership-model en de BLE-fundamenten uit dit document is essentieel voor het werken met elk BLE-verbonden MeshCore-apparaat.
---
## 9. Referenties
- MeshCore Companion Radio Protocol: [GitHub Wiki](https://github.com/meshcore-dev/MeshCore/wiki/Companion-Radio-Protocol)
- Bluetooth SIG Assigned Numbers (officiële services): [bluetooth.com/specifications/assigned-numbers](https://www.bluetooth.com/specifications/assigned-numbers/)
- Bluetooth SIG GATT Specifications: [bluetooth.com/specifications/specs](https://www.bluetooth.com/specifications/specs/)
- Nordic Bluetooth Numbers Database: [GitHub](https://github.com/NordicSemiconductor/bluetooth-numbers-database)
- GATT Uitleg (Adafruit): [learn.adafruit.com](https://learn.adafruit.com/introduction-to-bluetooth-low-energy/gatt)
- Bleak documentatie: [bleak.readthedocs.io](https://bleak.readthedocs.io/)
- BlueZ: [bluez.org](http://www.bluez.org/)
---
> **Document:** `ble_capture_workflow_t_1000_e_uitleg.md`
> **Gebaseerd op:** `ble_capture_workflow_t_1000_e.md`

293
install_scripts/install_ble_stable.sh Executable file
View File

@@ -0,0 +1,293 @@
#!/usr/bin/env bash
# ============================================================================
# MeshCore GUI — BLE Installer
# ============================================================================
#
# Installs the BLE PIN agent, reconnect module, systemd service
# and D-Bus policy. Automatically detects the correct paths and user.
#
# Usage:
# bash install_scripts/install_ble_stable.sh # from project root
# cd install_scripts && bash install_ble_stable.sh # from install_scripts/
#
# Optional:
# bash install_scripts/install_ble_stable.sh --uninstall
#
# Requirements:
# - meshcore-gui project with venv/ directory
# - sudo access (for systemd and D-Bus config)
# - Linux with BlueZ (Bluetooth)
#
# ============================================================================
set -euo pipefail
# ── Colors ──
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
info() { echo -e "${BLUE}[INFO]${NC} $*"; }
ok() { echo -e "${GREEN}[OK]${NC} $*"; }
warn() { echo -e "${YELLOW}[WARN]${NC} $*"; }
error() { echo -e "${RED}[ERROR]${NC} $*"; exit 1; }
# ── Resolve project root ──
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
if [[ "$(basename "${SCRIPT_DIR}")" == "install_scripts" ]]; then
PROJECT_DIR="$(dirname "${SCRIPT_DIR}")"
else
PROJECT_DIR="${SCRIPT_DIR}"
fi
# ── Uninstall mode ──
if [[ "${1:-}" == "--uninstall" ]]; then
info "Removing meshcore-gui service and configuration..."
sudo systemctl stop meshcore-gui 2>/dev/null || true
sudo systemctl disable meshcore-gui 2>/dev/null || true
sudo rm -f /etc/systemd/system/meshcore-gui.service
sudo rm -f /etc/dbus-1/system.d/meshcore-ble.conf
sudo systemctl daemon-reload
sudo systemctl reset-failed 2>/dev/null || true
ok "Service and configuration removed"
info "Python files in your project have NOT been removed."
info "Remove manually if desired:"
info " rm meshcore_gui/ble/ble_agent.py"
info " rm meshcore_gui/ble/ble_reconnect.py"
exit 0
fi
# ── Detect environment ──
info "Detecting environment..."
if [[ ! -f "${PROJECT_DIR}/meshcore_gui.py" ]] && [[ ! -d "${PROJECT_DIR}/meshcore_gui" ]]; then
error "Cannot find meshcore_gui.py or meshcore_gui/ in ${PROJECT_DIR}
Run this script from the project directory or from install_scripts/."
fi
CURRENT_USER="$(whoami)"
VENV_PYTHON="${PROJECT_DIR}/venv/bin/python"
# Check venv
if [[ ! -x "${VENV_PYTHON}" ]]; then
error "Virtual environment not found at: ${VENV_PYTHON}
Create it first:
python3 -m venv venv
source venv/bin/activate
pip install meshcore nicegui bleak meshcoredecoder"
fi
# Determine the entry point
if [[ -f "${PROJECT_DIR}/meshcore_gui.py" ]]; then
ENTRY_POINT="meshcore_gui.py"
elif [[ -d "${PROJECT_DIR}/meshcore_gui" ]]; then
ENTRY_POINT="-m meshcore_gui"
else
error "Cannot determine entry point."
fi
# Check BLE address argument
BLE_ADDRESS="${BLE_ADDRESS:-}"
if [[ -z "${BLE_ADDRESS}" ]]; then
echo ""
echo -e "${YELLOW}BLE MAC address not specified.${NC}"
echo "You can specify it in two ways:"
echo ""
echo " 1. As an environment variable:"
echo " BLE_ADDRESS=FF:05:D6:71:83:8D bash $0"
echo ""
echo " 2. Enter manually:"
read -rp " BLE MAC address (e.g. FF:05:D6:71:83:8D): " BLE_ADDRESS
echo ""
fi
if [[ -z "${BLE_ADDRESS}" ]]; then
error "No BLE MAC address specified. Aborted."
fi
# Summary
echo ""
echo "═══════════════════════════════════════════════════"
echo " MeshCore GUI — BLE Stability Installer"
echo "═══════════════════════════════════════════════════"
echo " Project dir: ${PROJECT_DIR}"
echo " User: ${CURRENT_USER}"
echo " Python: ${VENV_PYTHON}"
echo " Entry point: ${ENTRY_POINT}"
echo " BLE address: ${BLE_ADDRESS}"
echo "═══════════════════════════════════════════════════"
echo ""
read -rp "Continue? [y/N] " confirm
if [[ "${confirm}" != "y" && "${confirm}" != "Y" ]]; then
info "Aborted."
exit 0
fi
# ── Step 1: Upgrade meshcore library ──
info "Step 1/6: Upgrading meshcore library..."
"${PROJECT_DIR}/venv/bin/pip" install --upgrade meshcore --quiet 2>/dev/null || \
"${PROJECT_DIR}/venv/bin/pip" install --upgrade meshcore
MESHCORE_VERSION=$("${PROJECT_DIR}/venv/bin/pip" show meshcore 2>/dev/null | grep "^Version:" | awk '{print $2}')
ok "meshcore version: ${MESHCORE_VERSION:-unknown}"
# ── Step 2: Check that dbus_fast is available ──
info "Step 2/6: Checking dbus_fast dependency..."
if "${VENV_PYTHON}" -c "import dbus_fast" 2>/dev/null; then
ok "dbus_fast available (included with bleak)"
else
warn "dbus_fast not found, installing..."
"${PROJECT_DIR}/venv/bin/pip" install dbus-fast --quiet
ok "dbus_fast installed"
fi
# ── Step 3: Check Python files ──
info "Step 3/6: Checking Python files..."
BLE_DIR="${PROJECT_DIR}/meshcore_gui/ble"
if [[ ! -d "${BLE_DIR}" ]]; then
error "Directory ${BLE_DIR} not found."
fi
# Check if the files are already in place
AGENT_OK=false
RECONNECT_OK=false
[[ -f "${BLE_DIR}/ble_agent.py" ]] && AGENT_OK=true
[[ -f "${BLE_DIR}/ble_reconnect.py" ]] && RECONNECT_OK=true
if $AGENT_OK && $RECONNECT_OK; then
ok "ble_agent.py and ble_reconnect.py are already installed"
else
# Check if they are alongside the project files
if [[ -f "${PROJECT_DIR}/meshcore_gui/ble/ble_agent.py" ]]; then
ok "Files already present in project"
else
if ! $AGENT_OK; then
error "ble_agent.py not found in ${BLE_DIR}/
Copy this file manually to ${BLE_DIR}/"
fi
if ! $RECONNECT_OK; then
error "ble_reconnect.py not found in ${BLE_DIR}/
Copy this file manually to ${BLE_DIR}/"
fi
fi
fi
# Verify Python syntax
info "Verifying Python syntax..."
"${VENV_PYTHON}" -c "
import ast, sys
errors = []
for f in ['${BLE_DIR}/ble_agent.py', '${BLE_DIR}/ble_reconnect.py', '${BLE_DIR}/worker.py']:
try:
ast.parse(open(f).read())
except SyntaxError as e:
errors.append(f'{f}: {e}')
if errors:
print('SYNTAX ERRORS:')
for e in errors:
print(f' {e}')
sys.exit(1)
print('OK')
" || error "Syntax errors found in Python files"
ok "All Python files are syntactically correct"
# ── Step 4: Remove old bt-agent service ──
info "Step 4/6: Cleaning up old services..."
if systemctl is-active --quiet bt-agent 2>/dev/null; then
sudo systemctl stop bt-agent
sudo systemctl disable bt-agent
ok "bt-agent.service stopped and disabled"
elif systemctl list-unit-files | grep -q bt-agent 2>/dev/null; then
sudo systemctl disable bt-agent 2>/dev/null || true
ok "bt-agent.service disabled"
else
ok "bt-agent.service was already absent"
fi
# Stop existing meshcore-gui service if running
if systemctl is-active --quiet meshcore-gui 2>/dev/null; then
sudo systemctl stop meshcore-gui
ok "Existing meshcore-gui.service stopped"
fi
# ── Step 5: Install D-Bus policy ──
info "Step 5/6: Installing D-Bus policy..."
DBUS_CONF="/etc/dbus-1/system.d/meshcore-ble.conf"
sudo tee "${DBUS_CONF}" > /dev/null << DBUS_EOF
<!DOCTYPE busconfig PUBLIC "-//freedesktop//DTD D-BUS Bus Configuration 1.0//EN"
"http://www.freedesktop.org/standards/dbus/1.0/busconfig.dtd">
<busconfig>
<!-- Allow user '${CURRENT_USER}' to interact with BlueZ for BLE pairing agent -->
<policy user="${CURRENT_USER}">
<allow send_destination="org.bluez"/>
<allow send_interface="org.bluez.Agent1"/>
<allow send_interface="org.bluez.AgentManager1"/>
</policy>
</busconfig>
DBUS_EOF
ok "D-Bus policy installed for user '${CURRENT_USER}'"
# ── Step 6: Install systemd service ──
info "Step 6/6: Installing systemd service..."
SERVICE_FILE="/etc/systemd/system/meshcore-gui.service"
sudo tee "${SERVICE_FILE}" > /dev/null << SERVICE_EOF
[Unit]
Description=MeshCore GUI (BLE)
After=bluetooth.target
Wants=bluetooth.target
[Service]
Type=simple
User=${CURRENT_USER}
WorkingDirectory=${PROJECT_DIR}
ExecStart=${VENV_PYTHON} ${ENTRY_POINT} ${BLE_ADDRESS} --debug-on
Restart=on-failure
RestartSec=30
Environment=DBUS_SYSTEM_BUS_ADDRESS=unix:path=/var/run/dbus/system_bus_socket
[Install]
WantedBy=multi-user.target
SERVICE_EOF
sudo systemctl daemon-reload
sudo systemctl enable meshcore-gui
ok "meshcore-gui.service installed and enabled"
# ── Done ──
echo ""
echo "═══════════════════════════════════════════════════"
echo -e " ${GREEN}Installation complete!${NC}"
echo "═══════════════════════════════════════════════════"
echo ""
echo " Commands:"
echo " sudo systemctl start meshcore-gui # Start"
echo " sudo systemctl stop meshcore-gui # Stop"
echo " sudo systemctl restart meshcore-gui # Restart"
echo " sudo systemctl status meshcore-gui # Status"
echo " journalctl -u meshcore-gui -f # Live logs"
echo ""
echo " Uninstall:"
echo " bash install_scripts/install_ble_stable.sh --uninstall"
echo ""
echo "═══════════════════════════════════════════════════"
# Optionally start immediately
echo ""
read -rp "Start service now? [y/N] " start_now
if [[ "${start_now}" == "y" || "${start_now}" == "Y" ]]; then
sudo systemctl start meshcore-gui
sleep 2
if systemctl is-active --quiet meshcore-gui; then
ok "Service is running!"
echo ""
info "View live logs: journalctl -u meshcore-gui -f"
else
warn "Service could not start. Check logs:"
echo " journalctl -u meshcore-gui --no-pager -n 20"
fi
fi

238
install_scripts/install_bridge.sh Executable file
View File

@@ -0,0 +1,238 @@
#!/usr/bin/env bash
# =============================================================================
# MeshCore Bridge — systemd Service Installer
# =============================================================================
#
# Installs meshcore_bridge as a systemd daemon service.
#
# Usage:
# sudo bash install_scripts/install_bridge.sh # from project root
# cd install_scripts && sudo bash install_bridge.sh # from install_scripts/
# sudo bash install_scripts/install_bridge.sh --uninstall
#
# What this script does:
# 1. Copies meshcore_bridge.py and meshcore_bridge/ to /opt/meshcore-bridge/
# 2. Copies bridge_config.yaml to /etc/meshcore/ (if not already present)
# 3. Creates a systemd service unit file
# 4. Reloads systemd and enables the service
#
# After installation:
# sudo nano /etc/meshcore/bridge_config.yaml # edit configuration
# sudo systemctl start meshcore-bridge # start the service
# sudo systemctl enable meshcore-bridge # auto-start on boot
# sudo systemctl status meshcore-bridge # check status
# journalctl -u meshcore-bridge -f # follow logs
#
# Author: PE1HVH
# SPDX-License-Identifier: MIT
# Copyright: (c) 2026 PE1HVH
# =============================================================================
set -euo pipefail
SERVICE_NAME="meshcore-bridge"
INSTALL_DIR="/opt/meshcore-bridge"
CONFIG_DIR="/etc/meshcore"
CONFIG_FILE="${CONFIG_DIR}/bridge_config.yaml"
SERVICE_FILE="/etc/systemd/system/${SERVICE_NAME}.service"
# ── Resolve project root ──
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
if [[ "$(basename "${SCRIPT_DIR}")" == "install_scripts" ]]; then
PROJECT_DIR="$(dirname "${SCRIPT_DIR}")"
else
PROJECT_DIR="${SCRIPT_DIR}"
fi
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
info() { echo -e "${GREEN}[INFO]${NC} $*"; }
warn() { echo -e "${YELLOW}[WARN]${NC} $*"; }
error() { echo -e "${RED}[ERROR]${NC} $*" >&2; }
# ── Check root ──
if [[ $EUID -ne 0 ]]; then
error "This script must be run as root (sudo)."
exit 1
fi
# ── Uninstall mode ──
if [[ "${1:-}" == "--uninstall" ]]; then
info "Uninstalling ${SERVICE_NAME}..."
if systemctl is-active --quiet "${SERVICE_NAME}" 2>/dev/null; then
info "Stopping service..."
systemctl stop "${SERVICE_NAME}"
fi
if systemctl is-enabled --quiet "${SERVICE_NAME}" 2>/dev/null; then
info "Disabling service..."
systemctl disable "${SERVICE_NAME}"
fi
if [[ -f "${SERVICE_FILE}" ]]; then
info "Removing service file..."
rm -f "${SERVICE_FILE}"
systemctl daemon-reload
fi
if [[ -d "${INSTALL_DIR}" ]]; then
info "Removing installation directory..."
rm -rf "${INSTALL_DIR}"
fi
warn "Configuration preserved at ${CONFIG_DIR}/"
info "Uninstall complete."
exit 0
fi
# ── Install mode ──
info "Installing ${SERVICE_NAME}..."
# Verify source files exist
if [[ ! -f "${PROJECT_DIR}/meshcore_bridge.py" ]]; then
error "meshcore_bridge.py not found in ${PROJECT_DIR}"
error "Run this script from the project directory or from install_scripts/."
exit 1
fi
if [[ ! -d "${PROJECT_DIR}/meshcore_bridge" ]]; then
error "meshcore_bridge/ directory not found in ${PROJECT_DIR}"
exit 1
fi
# Detect Python interpreter
PYTHON_BIN=""
for candidate in python3.12 python3.11 python3.10 python3; do
if command -v "${candidate}" &>/dev/null; then
PYTHON_BIN="$(command -v "${candidate}")"
break
fi
done
if [[ -z "${PYTHON_BIN}" ]]; then
error "Python 3.10+ not found. Install Python first."
exit 1
fi
PYTHON_VERSION=$("${PYTHON_BIN}" -c "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}')")
info "Using Python ${PYTHON_VERSION} at ${PYTHON_BIN}"
# Check dependencies
info "Checking dependencies..."
"${PYTHON_BIN}" -c "import yaml" 2>/dev/null || {
error "pyyaml not installed. Run: pip install pyyaml"
exit 1
}
"${PYTHON_BIN}" -c "from meshcore_gui.core.shared_data import SharedData" 2>/dev/null || {
error "meshcore_gui not importable. Ensure it is installed or on PYTHONPATH."
exit 1
}
info "All dependencies satisfied."
# Create install directory
info "Copying files to ${INSTALL_DIR}/..."
mkdir -p "${INSTALL_DIR}"
cp "${PROJECT_DIR}/meshcore_bridge.py" "${INSTALL_DIR}/"
cp -r "${PROJECT_DIR}/meshcore_bridge" "${INSTALL_DIR}/"
chmod +x "${INSTALL_DIR}/meshcore_bridge.py"
# Copy config (preserve existing)
mkdir -p "${CONFIG_DIR}"
if [[ -f "${CONFIG_FILE}" ]]; then
warn "Config already exists at ${CONFIG_FILE} — not overwriting."
warn "New template saved as ${CONFIG_FILE}.new"
cp "${PROJECT_DIR}/bridge_config.yaml" "${CONFIG_FILE}.new"
else
info "Installing config template at ${CONFIG_FILE}"
cp "${PROJECT_DIR}/bridge_config.yaml" "${CONFIG_FILE}"
fi
# Detect meshcore_gui location for PYTHONPATH
MESHCORE_GUI_DIR=""
MESHCORE_GUI_PARENT=$("${PYTHON_BIN}" -c "
import meshcore_gui, os
print(os.path.dirname(os.path.dirname(meshcore_gui.__file__)))
" 2>/dev/null || true)
if [[ -n "${MESHCORE_GUI_PARENT}" ]]; then
MESHCORE_GUI_DIR="${MESHCORE_GUI_PARENT}"
info "meshcore_gui found at ${MESHCORE_GUI_DIR}"
fi
# Build PYTHONPATH
PYTHONPATH_VALUE="${INSTALL_DIR}"
if [[ -n "${MESHCORE_GUI_DIR}" ]]; then
PYTHONPATH_VALUE="${INSTALL_DIR}:${MESHCORE_GUI_DIR}"
fi
# Create systemd service file
info "Creating systemd service..."
cat > "${SERVICE_FILE}" << EOF
[Unit]
Description=MeshCore Bridge — Cross-Frequency Message Bridge Daemon
Documentation=file://${INSTALL_DIR}/BRIDGE.md
After=network.target
Wants=network.target
[Service]
Type=simple
ExecStart=${PYTHON_BIN} ${INSTALL_DIR}/meshcore_bridge.py --config=${CONFIG_FILE}
WorkingDirectory=${INSTALL_DIR}
Environment="PYTHONPATH=${PYTHONPATH_VALUE}"
# Restart policy
Restart=on-failure
RestartSec=10
StartLimitIntervalSec=300
StartLimitBurst=5
# Security hardening
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=read-only
ReadWritePaths=/home /var/log
PrivateTmp=true
# Logging
StandardOutput=journal
StandardError=journal
SyslogIdentifier=${SERVICE_NAME}
[Install]
WantedBy=multi-user.target
EOF
# Reload systemd
info "Reloading systemd daemon..."
systemctl daemon-reload
# Copy documentation
if [[ -f "${PROJECT_DIR}/BRIDGE.md" ]]; then
cp "${PROJECT_DIR}/BRIDGE.md" "${INSTALL_DIR}/"
fi
# ── Summary ──
echo
info "============================================="
info " Installation complete!"
info "============================================="
echo
info "Files installed:"
info " Application: ${INSTALL_DIR}/"
info " Config: ${CONFIG_FILE}"
info " Service: ${SERVICE_FILE}"
echo
info "Next steps:"
info " 1. Edit configuration: sudo nano ${CONFIG_FILE}"
info " 2. Start the service: sudo systemctl start ${SERVICE_NAME}"
info " 3. Enable auto-start: sudo systemctl enable ${SERVICE_NAME}"
info " 4. Check status: sudo systemctl status ${SERVICE_NAME}"
info " 5. Follow logs: journalctl -u ${SERVICE_NAME} -f"
info " 6. Open dashboard: http://localhost:9092"
echo
info "To uninstall: sudo bash install_scripts/install_bridge.sh --uninstall"

233
install_scripts/install_observer.sh Executable file
View File

@@ -0,0 +1,233 @@
#!/usr/bin/env bash
# ============================================================================
# MeshCore Observer — systemd Service Installer
# ============================================================================
#
# Installs a systemd service for the MeshCore Observer daemon.
# Automatically detects the venv and current user.
#
# Usage:
# bash install_scripts/install_observer.sh # from project root
# cd install_scripts && bash install_observer.sh # from install_scripts/
#
# Optional:
# bash install_scripts/install_observer.sh --uninstall
#
# Requirements:
# - meshcore-gui project with venv/ directory
# - nicegui and pyyaml installed in the venv
# - sudo access (for systemd)
#
# Author: PE1HVH
# SPDX-License-Identifier: MIT
# Copyright: (c) 2026 PE1HVH
# ============================================================================
set -euo pipefail
# ── Colors ──
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
info() { echo -e "${BLUE}[INFO]${NC} $*"; }
ok() { echo -e "${GREEN}[OK]${NC} $*"; }
warn() { echo -e "${YELLOW}[WARN]${NC} $*"; }
error() { echo -e "${RED}[ERROR]${NC} $*"; exit 1; }
SERVICE_NAME="meshcore-observer"
# ── Resolve project root ──
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
if [[ "$(basename "${SCRIPT_DIR}")" == "install_scripts" ]]; then
PROJECT_DIR="$(dirname "${SCRIPT_DIR}")"
else
PROJECT_DIR="${SCRIPT_DIR}"
fi
# ── Uninstall mode ──
if [[ "${1:-}" == "--uninstall" ]]; then
info "Removing ${SERVICE_NAME} service..."
sudo systemctl stop "${SERVICE_NAME}" 2>/dev/null || true
sudo systemctl disable "${SERVICE_NAME}" 2>/dev/null || true
sudo rm -f "/etc/systemd/system/${SERVICE_NAME}.service"
sudo systemctl daemon-reload
sudo systemctl reset-failed 2>/dev/null || true
ok "Service removed"
exit 0
fi
# ── Detect environment ──
info "Detecting environment..."
if [[ ! -f "${PROJECT_DIR}/meshcore_observer.py" ]] && [[ ! -d "${PROJECT_DIR}/meshcore_observer" ]]; then
error "Cannot find meshcore_observer.py or meshcore_observer/ in ${PROJECT_DIR}
Run this script from the project directory or from install_scripts/."
fi
CURRENT_USER="$(whoami)"
VENV_PYTHON="${PROJECT_DIR}/venv/bin/python"
# Check venv
if [[ ! -x "${VENV_PYTHON}" ]]; then
# Try parent directory venv (observer may be in meshcore-gui project)
PARENT_VENV="$(dirname "${PROJECT_DIR}")/venv/bin/python"
if [[ -x "${PARENT_VENV}" ]]; then
VENV_PYTHON="${PARENT_VENV}"
warn "Using parent directory venv: ${VENV_PYTHON}"
else
error "Virtual environment not found at: ${VENV_PYTHON}
Create it first:
python3 -m venv venv
source venv/bin/activate
pip install nicegui pyyaml"
fi
fi
# ── Check dependencies ──
info "Checking dependencies..."
"${VENV_PYTHON}" -c "import nicegui" 2>/dev/null || {
error "nicegui not installed in venv. Run:
source venv/bin/activate
pip install nicegui"
}
"${VENV_PYTHON}" -c "import yaml" 2>/dev/null || {
error "pyyaml not installed in venv. Run:
source venv/bin/activate
pip install pyyaml"
}
ok "All dependencies satisfied"
# ── Detect NODE_PATH for meshcore-decoder (MQTT auth) ──
NODE_PATH_VALUE=""
if command -v node &>/dev/null; then
NPM_GLOBAL="$(npm root -g 2>/dev/null || true)"
if [[ -n "${NPM_GLOBAL}" ]] && [[ -d "${NPM_GLOBAL}" ]]; then
NODE_PATH_VALUE="${NPM_GLOBAL}"
info "Node.js global modules: ${NPM_GLOBAL}"
fi
fi
# ── Optional settings ──
WEB_PORT="${WEB_PORT:-9093}"
ARCHIVE_DIR="${ARCHIVE_DIR:-~/.meshcore-gui/archive}"
DEBUG_ON="${DEBUG_ON:-}"
if [[ -z "${DEBUG_ON}" ]]; then
read -rp "Enable debug logging? [y/N] " dbg
if [[ "${dbg}" == "y" || "${dbg}" == "Y" ]]; then
DEBUG_ON="yes"
else
DEBUG_ON="no"
fi
fi
DEBUG_FLAG=""
if [[ "${DEBUG_ON}" == "yes" ]]; then
DEBUG_FLAG="--debug-on"
fi
# ── Config file ──
CONFIG_FLAG=""
CONFIG_FILE="${PROJECT_DIR}/observer_config.yaml"
if [[ -f "${CONFIG_FILE}" ]]; then
CONFIG_FLAG="--config=${CONFIG_FILE}"
info "Using config: ${CONFIG_FILE}"
else
info "No observer_config.yaml found — using defaults"
fi
# ── Summary ──
echo ""
echo "═══════════════════════════════════════════════════"
echo " MeshCore Observer — Service Installer"
echo "═══════════════════════════════════════════════════"
echo " Project dir: ${PROJECT_DIR}"
echo " User: ${CURRENT_USER}"
echo " Python: ${VENV_PYTHON}"
echo " Archive dir: ${ARCHIVE_DIR}"
echo " Web port: ${WEB_PORT}"
echo " Config: ${CONFIG_FILE}"
echo " Debug: ${DEBUG_ON}"
if [[ -n "${NODE_PATH_VALUE}" ]]; then
echo " NODE_PATH: ${NODE_PATH_VALUE}"
fi
echo "═══════════════════════════════════════════════════"
echo ""
read -rp "Continue? [y/N] " confirm
if [[ "${confirm}" != "y" && "${confirm}" != "Y" ]]; then
info "Aborted."
exit 0
fi
# ── Install systemd service ──
info "Installing systemd service..."
SERVICE_FILE="/etc/systemd/system/${SERVICE_NAME}.service"
# Build optional Environment line for NODE_PATH
ENV_LINE=""
if [[ -n "${NODE_PATH_VALUE}" ]]; then
ENV_LINE="Environment=\"NODE_PATH=${NODE_PATH_VALUE}\""
fi
sudo tee "${SERVICE_FILE}" > /dev/null << SERVICE_EOF
[Unit]
Description=MeshCore Observer — Read-Only Archive Monitor Dashboard
[Service]
Type=simple
User=${CURRENT_USER}
WorkingDirectory=${PROJECT_DIR}
ExecStart=${VENV_PYTHON} meshcore_observer.py ${CONFIG_FLAG} --port=${WEB_PORT} ${DEBUG_FLAG}
Restart=on-failure
RestartSec=30
${ENV_LINE}
[Install]
WantedBy=multi-user.target
SERVICE_EOF
sudo systemctl daemon-reload
sudo systemctl enable "${SERVICE_NAME}"
ok "${SERVICE_NAME}.service installed and enabled"
# ── Done ──
echo ""
echo "═══════════════════════════════════════════════════"
echo -e " ${GREEN}Installation complete!${NC}"
echo "═══════════════════════════════════════════════════"
echo ""
echo " Commands:"
echo " sudo systemctl start ${SERVICE_NAME} # Start"
echo " sudo systemctl stop ${SERVICE_NAME} # Stop"
echo " sudo systemctl restart ${SERVICE_NAME} # Restart"
echo " sudo systemctl status ${SERVICE_NAME} # Status"
echo " journalctl -u ${SERVICE_NAME} -f # Live logs"
echo ""
echo " Dashboard: http://localhost:${WEB_PORT}"
echo ""
echo " Uninstall:"
echo " bash install_scripts/install_observer.sh --uninstall"
echo ""
echo "═══════════════════════════════════════════════════"
# Optionally start immediately
echo ""
read -rp "Start service now? [y/N] " start_now
if [[ "${start_now}" == "y" || "${start_now}" == "Y" ]]; then
sudo systemctl start "${SERVICE_NAME}"
sleep 2
if systemctl is-active --quiet "${SERVICE_NAME}"; then
ok "Service is running!"
echo ""
info "View live logs: journalctl -u ${SERVICE_NAME} -f"
else
warn "Service could not start. Check logs:"
echo " journalctl -u ${SERVICE_NAME} --no-pager -n 20"
fi
fi

241
install_scripts/install_serial.sh Executable file
View File

@@ -0,0 +1,241 @@
#!/usr/bin/env bash
# ============================================================================
# MeshCore GUI — Serial Installer
# ============================================================================
#
# Installs a systemd service for the serial-based MeshCore GUI.
# Automatically detects paths and the current user.
#
# Usage:
# bash install_scripts/install_serial.sh # from project root
# cd install_scripts && bash install_serial.sh # from install_scripts/
#
# Optional:
# bash install_scripts/install_serial.sh --uninstall
#
# Requirements:
# - meshcore-gui project with venv/ directory
# - sudo access (for systemd)
#
# ============================================================================
set -euo pipefail
# ── Colors ──
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
info() { echo -e "${BLUE}[INFO]${NC} $*"; }
ok() { echo -e "${GREEN}[OK]${NC} $*"; }
warn() { echo -e "${YELLOW}[WARN]${NC} $*"; }
error() { echo -e "${RED}[ERROR]${NC} $*"; exit 1; }
# ── Resolve project root ──
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
if [[ "$(basename "${SCRIPT_DIR}")" == "install_scripts" ]]; then
PROJECT_DIR="$(dirname "${SCRIPT_DIR}")"
else
PROJECT_DIR="${SCRIPT_DIR}"
fi
# ── Uninstall mode ──
if [[ "${1:-}" == "--uninstall" ]]; then
info "Removing meshcore-gui service..."
sudo systemctl stop meshcore-gui 2>/dev/null || true
sudo systemctl disable meshcore-gui 2>/dev/null || true
sudo rm -f /etc/systemd/system/meshcore-gui.service
sudo systemctl daemon-reload
sudo systemctl reset-failed 2>/dev/null || true
ok "Service removed"
exit 0
fi
# ── Detect environment ──
info "Detecting environment..."
if [[ ! -f "${PROJECT_DIR}/meshcore_gui.py" ]] && [[ ! -d "${PROJECT_DIR}/meshcore_gui" ]]; then
error "Cannot find meshcore_gui.py or meshcore_gui/ in ${PROJECT_DIR}
Run this script from the project directory or from install_scripts/."
fi
CURRENT_USER="$(whoami)"
VENV_PYTHON="${PROJECT_DIR}/venv/bin/python"
# Check venv
if [[ ! -x "${VENV_PYTHON}" ]]; then
error "Virtual environment not found at: ${VENV_PYTHON}
Create it first:
python3 -m venv venv
source venv/bin/activate
pip install meshcore nicegui meshcoredecoder"
fi
# Determine the entry point
if [[ -f "${PROJECT_DIR}/meshcore_gui.py" ]]; then
ENTRY_POINT="meshcore_gui.py"
elif [[ -d "${PROJECT_DIR}/meshcore_gui" ]]; then
ENTRY_POINT="-m meshcore_gui"
else
error "Cannot determine entry point."
fi
# Serial port (env or prompt)
SERIAL_PORT="${SERIAL_PORT:-}"
if [[ -z "${SERIAL_PORT}" ]]; then
echo ""
echo -e "${YELLOW}Serial device not specified.${NC}"
echo "You can specify it in two ways:"
echo ""
echo " 1. As an environment variable:"
echo " SERIAL_PORT=/dev/ttyACM0 bash $0"
echo ""
echo " 2. Enter manually:"
read -rp " Serial device (e.g. /dev/ttyACM0 or /dev/ttyUSB0): " SERIAL_PORT
echo ""
fi
if [[ -z "${SERIAL_PORT}" ]]; then
error "No serial device specified. Aborted."
fi
# Optional settings
BAUD="${BAUD:-115200}"
SERIAL_CX_DLY="${SERIAL_CX_DLY:-0.1}"
WEB_PORT="${WEB_PORT:-8081}"
DEBUG_ON="${DEBUG_ON:-}"
if [[ -z "${DEBUG_ON}" ]]; then
read -rp "Enable debug logging? [y/N] " dbg
if [[ "${dbg}" == "y" || "${dbg}" == "Y" ]]; then
DEBUG_ON="yes"
else
DEBUG_ON="no"
fi
fi
DEBUG_FLAG=""
if [[ "${DEBUG_ON}" == "yes" ]]; then
DEBUG_FLAG="--debug-on"
fi
# Warn about dialout group (Linux)
if ! id -nG "${CURRENT_USER}" | grep -qw "dialout"; then
warn "User '${CURRENT_USER}' is not in the 'dialout' group."
warn "Serial access may fail. Fix with:"
warn " sudo usermod -aG dialout ${CURRENT_USER}"
warn " (then log out/in)"
fi
# Summary
echo ""
echo "═══════════════════════════════════════════════════"
echo " MeshCore GUI — Serial Installer"
echo "═══════════════════════════════════════════════════"
echo " Project dir: ${PROJECT_DIR}"
echo " User: ${CURRENT_USER}"
echo " Python: ${VENV_PYTHON}"
echo " Entry point: ${ENTRY_POINT}"
echo " Serial port: ${SERIAL_PORT}"
echo " Baudrate: ${BAUD}"
echo " CX delay: ${SERIAL_CX_DLY}"
echo " Web port: ${WEB_PORT}"
echo " Debug: ${DEBUG_ON}"
echo "═══════════════════════════════════════════════════"
echo ""
read -rp "Continue? [y/N] " confirm
if [[ "${confirm}" != "y" && "${confirm}" != "Y" ]]; then
info "Aborted."
exit 0
fi
# ── Step 1: Upgrade meshcore library ──
info "Step 1/3: Upgrading meshcore library..."
"${PROJECT_DIR}/venv/bin/pip" install --upgrade meshcore --quiet 2>/dev/null || \
"${PROJECT_DIR}/venv/bin/pip" install --upgrade meshcore
MESHCORE_VERSION=$("${PROJECT_DIR}/venv/bin/pip" show meshcore 2>/dev/null | grep "^Version:" | awk '{print $2}')
ok "meshcore version: ${MESHCORE_VERSION:-unknown}"
# ── Step 2: Verify Python syntax ──
info "Step 2/3: Verifying Python syntax..."
"${VENV_PYTHON}" -c "
import ast, sys
files = [
'${PROJECT_DIR}/meshcore_gui.py',
'${PROJECT_DIR}/meshcore_gui/ble/worker.py',
'${PROJECT_DIR}/meshcore_gui/ble/commands.py',
]
errors = []
for f in files:
try:
ast.parse(open(f).read())
except Exception as e:
errors.append(f'{f}: {e}')
if errors:
print('SYNTAX ERRORS:')
for e in errors:
print(f' {e}')
sys.exit(1)
print('OK')
" || error "Syntax errors found in Python files"
ok "Python files are syntactically correct"
# ── Step 3: Install systemd service ──
info "Step 3/3: Installing systemd service..."
SERVICE_FILE="/etc/systemd/system/meshcore-gui.service"
sudo tee "${SERVICE_FILE}" > /dev/null << SERVICE_EOF
[Unit]
Description=MeshCore GUI (Serial)
[Service]
Type=simple
User=${CURRENT_USER}
WorkingDirectory=${PROJECT_DIR}
ExecStart=${VENV_PYTHON} ${ENTRY_POINT} ${SERIAL_PORT} ${DEBUG_FLAG} --port=${WEB_PORT} --baud=${BAUD} --serial-cx-dly=${SERIAL_CX_DLY}
Restart=on-failure
RestartSec=30
[Install]
WantedBy=multi-user.target
SERVICE_EOF
sudo systemctl daemon-reload
sudo systemctl enable meshcore-gui
ok "meshcore-gui.service installed and enabled"
# ── Done ──
echo ""
echo "═══════════════════════════════════════════════════"
echo -e " ${GREEN}Installation complete!${NC}"
echo "═══════════════════════════════════════════════════"
echo ""
echo " Commands:"
echo " sudo systemctl start meshcore-gui # Start"
echo " sudo systemctl stop meshcore-gui # Stop"
echo " sudo systemctl restart meshcore-gui # Restart"
echo " sudo systemctl status meshcore-gui # Status"
echo " journalctl -u meshcore-gui -f # Live logs"
echo ""
echo " Uninstall:"
echo " bash install_scripts/install_serial.sh --uninstall"
echo ""
echo "═══════════════════════════════════════════════════"
# Optionally start immediately
echo ""
read -rp "Start service now? [y/N] " start_now
if [[ "${start_now}" == "y" || "${start_now}" == "Y" ]]; then
sudo systemctl start meshcore-gui
sleep 2
if systemctl is-active --quiet meshcore-gui; then
ok "Service is running!"
echo ""
info "View live logs: journalctl -u meshcore-gui -f"
else
warn "Service could not start. Check logs:"
echo " journalctl -u meshcore-gui --no-pager -n 20"
fi
fi

30
install_scripts/install_venv.sh Executable file
View File

@@ -0,0 +1,30 @@
#!/usr/bin/env bash
# ============================================================================
# MeshCore GUI — Virtual Environment Setup
# ============================================================================
#
# Creates a venv and installs core Python dependencies.
#
# Usage:
# bash install_scripts/install_venv.sh # from project root
# cd install_scripts && bash install_venv.sh # from install_scripts/
#
# ============================================================================
set -euo pipefail
# ── Resolve project root ──
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
if [[ "$(basename "${SCRIPT_DIR}")" == "install_scripts" ]]; then
PROJECT_DIR="$(dirname "${SCRIPT_DIR}")"
else
PROJECT_DIR="${SCRIPT_DIR}"
fi
cd "${PROJECT_DIR}"
echo "Creating virtual environment in ${PROJECT_DIR}/venv ..."
python3 -m venv venv
source venv/bin/activate
pip install nicegui meshcore bleak meshcoredecoder
echo "Done. Activate with: source ${PROJECT_DIR}/venv/bin/activate"

26
meshcore_bridge.py Executable file
View File

@@ -0,0 +1,26 @@
#!/usr/bin/env python3
"""
MeshCore Bridge — Cross-Frequency Message Bridge Daemon
========================================================
Standalone daemon that connects two meshcore_gui instances on
different frequencies by forwarding messages on a configurable
bridge channel. Requires zero modifications to the existing
meshcore_gui codebase.
Usage:
python meshcore_bridge.py
python meshcore_bridge.py --config=bridge_config.yaml
python meshcore_bridge.py --port=9092
python meshcore_bridge.py --debug-on
Author: PE1HVH
Version: 1.0.0
SPDX-License-Identifier: MIT
Copyright: (c) 2026 PE1HVH
"""
from meshcore_bridge.__main__ import main
if __name__ == "__main__":
main()

9
meshcore_bridge/__init__.py Normal file
View File

@@ -0,0 +1,9 @@
"""
MeshCore Bridge — Cross-Frequency Message Bridge Daemon.
Standalone daemon that connects two meshcore_gui instances on
different frequencies by forwarding messages on a configurable
bridge channel.
"""
__version__ = "1.0.0"

220
meshcore_bridge/__main__.py Normal file
View File

@@ -0,0 +1,220 @@
#!/usr/bin/env python3
"""
MeshCore Bridge — Entry Point
==============================
Parses command-line arguments, loads YAML configuration, creates two
SharedData/Worker pairs (one per device), initialises the BridgeEngine,
registers the NiceGUI dashboard page and starts the server.
Usage:
python meshcore_bridge.py
python meshcore_bridge.py --config=bridge_config.yaml
python meshcore_bridge.py --port=9092
python meshcore_bridge.py --debug-on
Author: PE1HVH
Version: 1.0.0
SPDX-License-Identifier: MIT
Copyright: (c) 2026 PE1HVH
"""
import asyncio
import sys
import threading
import time
from pathlib import Path
from nicegui import app, ui
# Allow overriding DEBUG before anything imports it
import meshcore_gui.config as gui_config
try:
from meshcore import MeshCore, EventType # noqa: F401
except ImportError:
print("ERROR: meshcore library not found")
print("Install with: pip install meshcore")
sys.exit(1)
from meshcore_gui.ble.worker import create_worker
from meshcore_gui.core.shared_data import SharedData
from meshcore_bridge.config import BridgeConfig, DEFAULT_CONFIG_PATH
from meshcore_bridge.bridge_engine import BridgeEngine
from meshcore_bridge.gui.dashboard import BridgeDashboard
# Global instances (needed by NiceGUI page decorators)
_dashboard: BridgeDashboard | None = None
@ui.page('/')
def _page_dashboard():
"""NiceGUI page handler — bridge dashboard."""
if _dashboard:
_dashboard.render()
def _print_usage():
"""Show usage information."""
print("MeshCore Bridge — Cross-Frequency Message Bridge Daemon")
print("=" * 58)
print()
print("Usage: python meshcore_bridge.py [OPTIONS]")
print()
print("Options:")
print(" --config=PATH Path to bridge_config.yaml (default: ./bridge_config.yaml)")
print(" --port=PORT Override GUI port from config (default: 9092)")
print(" --debug-on Enable verbose debug logging")
print(" --help Show this help message")
print()
print("Configuration:")
print(" All settings are defined in bridge_config.yaml.")
print(" See BRIDGE.md for full documentation.")
print()
print("Examples:")
print(" python meshcore_bridge.py")
print(" python meshcore_bridge.py --config=/etc/meshcore/bridge_config.yaml")
print(" python meshcore_bridge.py --port=9092 --debug-on")
def _parse_flags(argv):
"""Parse CLI arguments into a flag dict.
Handles ``--flag=value`` and boolean ``--flag``.
"""
flags = {}
for a in argv:
if '=' in a and a.startswith('--'):
key, value = a.split('=', 1)
flags[key] = value
elif a.startswith('--'):
flags[a] = True
return flags
def _bridge_poll_loop(engine: BridgeEngine, interval_ms: int):
"""Background thread that runs the bridge polling loop.
Args:
engine: BridgeEngine instance.
interval_ms: Polling interval in milliseconds.
"""
interval_s = interval_ms / 1000.0
while True:
try:
engine.poll_and_forward()
except Exception as e:
gui_config.debug_print(f"Bridge poll error: {e}")
time.sleep(interval_s)
def main():
"""Main entry point.
Loads configuration, creates dual workers, starts the bridge
engine and the NiceGUI dashboard.
"""
global _dashboard
flags = _parse_flags(sys.argv[1:])
if '--help' in flags:
_print_usage()
sys.exit(0)
# ── Load configuration ──
config_path = Path(flags.get('--config', str(DEFAULT_CONFIG_PATH)))
if config_path.exists():
print(f"Loading config from: {config_path}")
cfg = BridgeConfig.from_yaml(config_path)
else:
print(f"Config not found at {config_path}, using defaults.")
print(f"Run with --help for usage information.")
cfg = BridgeConfig()
# ── CLI overrides ──
if '--debug-on' in flags:
cfg.debug = True
gui_config.DEBUG = True
if '--port' in flags:
try:
cfg.gui_port = int(flags['--port'])
except ValueError:
print(f"ERROR: Invalid port: {flags['--port']}")
sys.exit(1)
cfg.config_path = str(config_path)
# ── Startup banner ──
print("=" * 58)
print("MeshCore Bridge — Cross-Frequency Message Bridge Daemon")
print("=" * 58)
print(f"Config: {config_path}")
print(f"Device A: {cfg.device_a.port} ({cfg.device_a.label})")
print(f"Device B: {cfg.device_b.port} ({cfg.device_b.label})")
print(f"Channel: #{cfg.channel_name} (A:idx={cfg.channel_idx_a}, B:idx={cfg.channel_idx_b})")
print(f"Poll interval:{cfg.poll_interval_ms}ms")
print(f"GUI port: {cfg.gui_port}")
print(f"Forward prefix: {'ON' if cfg.forward_prefix else 'OFF'}")
print(f"Debug mode: {'ON' if cfg.debug else 'OFF'}")
print("=" * 58)
# ── Create dual SharedData instances ──
shared_a = SharedData(f"bridge_a_{cfg.device_a.port.replace('/', '_')}")
shared_b = SharedData(f"bridge_b_{cfg.device_b.port.replace('/', '_')}")
# ── Create BridgeEngine ──
engine = BridgeEngine(shared_a, shared_b, cfg)
# ── Create workers (one per device) ──
gui_config.SERIAL_BAUDRATE = cfg.device_a.baud
worker_a = create_worker(
cfg.device_a.port,
shared_a,
baudrate=cfg.device_a.baud,
)
gui_config.SERIAL_BAUDRATE = cfg.device_b.baud
worker_b = create_worker(
cfg.device_b.port,
shared_b,
baudrate=cfg.device_b.baud,
)
# ── Start workers ──
print(f"Starting worker A ({cfg.device_a.port})...")
worker_a.start()
print(f"Starting worker B ({cfg.device_b.port})...")
worker_b.start()
# ── Start bridge polling thread ──
print(f"Starting bridge engine (poll every {cfg.poll_interval_ms}ms)...")
poll_thread = threading.Thread(
target=_bridge_poll_loop,
args=(engine, cfg.poll_interval_ms),
daemon=True,
)
poll_thread.start()
# ── Create dashboard ──
_dashboard = BridgeDashboard(shared_a, shared_b, engine, cfg)
# ── Start NiceGUI server (blocks) ──
print(f"Starting GUI on port {cfg.gui_port}...")
ui.run(
show=False,
host='0.0.0.0',
title=cfg.gui_title,
port=cfg.gui_port,
reload=False,
storage_secret='meshcore-bridge-secret',
)
if __name__ == "__main__":
main()

315
meshcore_bridge/bridge_engine.py Normal file
View File

@@ -0,0 +1,315 @@
"""
Core bridge logic: message monitoring, forwarding and loop prevention.
BridgeEngine polls two SharedData stores and forwards messages on the
configured bridge channel from one instance to the other. Loop
prevention is achieved via a bounded set of forwarded message hashes
and by filtering outbound (direction='out') messages.
Thread safety: all SharedData access goes through the existing lock
mechanism in SharedData. BridgeEngine itself is called from a single
asyncio task (the polling loop in __main__).
"""
import hashlib
import time
from collections import OrderedDict
from dataclasses import dataclass, field
from datetime import datetime
from typing import List, Optional
from meshcore_gui.core.models import Message
from meshcore_gui.core.shared_data import SharedData
from meshcore_bridge.config import BridgeConfig
@dataclass
class ForwardedEntry:
"""Record of a forwarded message for the bridge log."""
time: str
direction: str # "A→B" or "B→A"
sender: str
text: str
channel: Optional[int]
class BridgeEngine:
"""Core bridge logic: poll, filter, forward and deduplicate.
Monitors two SharedData instances for new incoming messages on the
configured bridge channel and forwards them to the opposite instance
via put_command().
Attributes:
stats: Runtime statistics dict exposed to the GUI dashboard.
"""
def __init__(
self,
shared_a: SharedData,
shared_b: SharedData,
config: BridgeConfig,
) -> None:
self._a = shared_a
self._b = shared_b
self._cfg = config
# Channel indices per device
self._ch_idx_a = config.channel_idx_a
self._ch_idx_b = config.channel_idx_b
# Loop prevention: bounded set of forwarded hashes
self._forwarded_hashes: OrderedDict = OrderedDict()
self._max_cache = config.max_forwarded_cache
# Tracking last seen message count per side
self._last_count_a: int = 0
self._last_count_b: int = 0
# Forwarded message log (for dashboard)
self._log: List[ForwardedEntry] = []
self._max_log: int = 200
# Runtime statistics
self.stats = {
"forwarded_a_to_b": 0,
"forwarded_b_to_a": 0,
"duplicates_blocked": 0,
"last_forward_time": "",
"started_at": datetime.now().strftime("%Y-%m-%d %H:%M:%S"),
"uptime_seconds": 0,
}
self._start_time = time.time()
# ------------------------------------------------------------------
# Public API
# ------------------------------------------------------------------
def poll_and_forward(self) -> int:
"""Check both stores for new bridge-channel messages and forward.
Returns:
Number of messages forwarded in this poll cycle.
"""
self.stats["uptime_seconds"] = int(time.time() - self._start_time)
count = 0
# A → B
count += self._poll_side(
source=self._a,
target=self._b,
source_ch=self._ch_idx_a,
target_ch=self._ch_idx_b,
direction_label="A→B",
last_count_attr="_last_count_a",
stat_key="forwarded_a_to_b",
)
# B → A
count += self._poll_side(
source=self._b,
target=self._a,
source_ch=self._ch_idx_b,
target_ch=self._ch_idx_a,
direction_label="B→A",
last_count_attr="_last_count_b",
stat_key="forwarded_b_to_a",
)
return count
def get_log(self) -> List[ForwardedEntry]:
"""Return a copy of the forwarded message log (newest first)."""
return list(reversed(self._log))
def get_total_forwarded(self) -> int:
"""Total number of messages forwarded since start."""
return (
self.stats["forwarded_a_to_b"]
+ self.stats["forwarded_b_to_a"]
)
# ------------------------------------------------------------------
# Internal
# ------------------------------------------------------------------
def _poll_side(
self,
source: SharedData,
target: SharedData,
source_ch: int,
target_ch: int,
direction_label: str,
last_count_attr: str,
stat_key: str,
) -> int:
"""Poll one side for new messages and forward to the other.
Args:
source: SharedData to read from.
target: SharedData to write to.
source_ch: Channel index on the source device.
target_ch: Channel index on the target device.
direction_label: "A→B" or "B→A" for logging.
last_count_attr: Name of the self._last_count_* attribute.
stat_key: Key in self.stats to increment.
Returns:
Number of messages forwarded.
"""
forwarded = 0
snapshot = source.get_snapshot()
msgs = snapshot["messages"]
last_count = getattr(self, last_count_attr)
# Detect list shrinkage (e.g. after reconnect/reload)
if len(msgs) < last_count:
setattr(self, last_count_attr, 0)
last_count = 0
new_msgs = msgs[last_count:]
setattr(self, last_count_attr, len(msgs))
for msg in new_msgs:
if self._should_forward(msg, source_ch):
self._forward(msg, target, target_ch, direction_label)
self.stats[stat_key] += 1
forwarded += 1
return forwarded
def _should_forward(self, msg: Message, expected_channel: int) -> bool:
"""Determine whether a message should be forwarded.
Filtering rules:
1. Channel must match the bridge channel for this side.
2. Outbound messages (direction='out') are never forwarded
— they are our own transmissions (including previous forwards).
3. Messages whose hash is already in the forwarded set are
duplicates (loop prevention).
Args:
msg: Message to evaluate.
expected_channel: Bridge channel index on this device.
Returns:
True if the message should be forwarded.
"""
# Rule 1: channel filter
if msg.channel != expected_channel:
return False
# Rule 2: never forward our own transmissions
if msg.direction == "out":
return False
# Rule 3: loop prevention via hash set
msg_hash = self._compute_hash(msg)
if msg_hash in self._forwarded_hashes:
self.stats["duplicates_blocked"] += 1
return False
return True
def _forward(
self,
msg: Message,
target: SharedData,
target_ch: int,
direction_label: str,
) -> None:
"""Forward a message to the target SharedData via put_command().
Args:
msg: Message to forward.
target: Target SharedData instance.
target_ch: Channel index on the target device.
direction_label: "A→B" or "B→A" for logging.
"""
msg_hash = self._compute_hash(msg)
# Register hash for loop prevention
self._forwarded_hashes[msg_hash] = True
if len(self._forwarded_hashes) > self._max_cache:
self._forwarded_hashes.popitem(last=False)
# Also register the hash of the text we're about to send so
# the *other* direction won't re-forward our forwarded message
# if it appears on the target device's bridge channel.
forward_text = self._build_forward_text(msg)
echo_hash = self._text_hash(forward_text)
self._forwarded_hashes[echo_hash] = True
if len(self._forwarded_hashes) > self._max_cache:
self._forwarded_hashes.popitem(last=False)
# Inject send command into the target's command queue
target.put_command({
"action": "send_message",
"channel": target_ch,
"text": forward_text,
"_bot": True, # suppress outgoing Message creation in CommandHandler
})
# Update stats and log
now = datetime.now().strftime("%H:%M:%S")
self.stats["last_forward_time"] = now
entry = ForwardedEntry(
time=now,
direction=direction_label,
sender=msg.sender,
text=msg.text,
channel=msg.channel,
)
self._log.append(entry)
if len(self._log) > self._max_log:
self._log.pop(0)
def _build_forward_text(self, msg: Message) -> str:
"""Build the text to transmit on the target device.
When forward_prefix is enabled, the original sender name is
prepended so recipients can identify the origin.
Args:
msg: Original message.
Returns:
Text string to send.
"""
if self._cfg.forward_prefix:
return f"[{msg.sender}] {msg.text}"
return msg.text
@staticmethod
def _compute_hash(msg: Message) -> str:
"""Compute a deduplication hash for a message.
Uses the message_hash field when available (deterministic
packet ID from MeshCore firmware). Falls back to a SHA-256
digest of channel + sender + text.
Args:
msg: Message to hash.
Returns:
Hash string.
"""
if msg.message_hash:
return f"mh:{msg.message_hash}"
raw = f"{msg.channel}:{msg.sender}:{msg.text}"
return f"ct:{hashlib.sha256(raw.encode()).hexdigest()[:16]}"
@staticmethod
def _text_hash(text: str) -> str:
"""Hash a plain text string for echo suppression.
Args:
text: Text to hash.
Returns:
Hash string.
"""
return f"tx:{hashlib.sha256(text.encode()).hexdigest()[:16]}"

140
meshcore_bridge/config.py Normal file
View File

@@ -0,0 +1,140 @@
"""
Bridge-specific configuration.
Loads settings from a YAML configuration file and provides typed
access to all bridge parameters. Falls back to sensible defaults
when keys are missing.
Dependencies:
pyyaml (6.x)
"""
from dataclasses import dataclass, field
from pathlib import Path
from typing import Optional
import yaml
# Default config file location (next to meshcore_bridge.py)
DEFAULT_CONFIG_PATH: Path = Path(__file__).parent.parent / "bridge_config.yaml"
@dataclass
class DeviceConfig:
"""Configuration for a single MeshCore device connection."""
port: str = "/dev/ttyUSB0"
baud: int = 115200
label: str = ""
@dataclass
class BridgeConfig:
"""Complete bridge daemon configuration."""
# Bridge channel settings
channel_name: str = "bridge"
channel_idx_a: int = 3
channel_idx_b: int = 3
poll_interval_ms: int = 200
forward_prefix: bool = True
max_forwarded_cache: int = 500
# Device connections
device_a: DeviceConfig = field(default_factory=lambda: DeviceConfig(
port="/dev/ttyUSB1", label="869.525 MHz",
))
device_b: DeviceConfig = field(default_factory=lambda: DeviceConfig(
port="/dev/ttyUSB2", label="868.000 MHz",
))
# GUI settings
gui_port: int = 9092
gui_title: str = "MeshCore Bridge"
# Runtime flags (set from CLI, not YAML)
debug: bool = False
config_path: str = ""
@classmethod
def from_yaml(cls, path: Path) -> "BridgeConfig":
"""Load configuration from a YAML file.
Missing keys fall back to dataclass defaults.
Args:
path: Path to the YAML configuration file.
Returns:
Populated BridgeConfig instance.
Raises:
FileNotFoundError: If the config file does not exist.
yaml.YAMLError: If the file contains invalid YAML.
"""
with open(path, "r", encoding="utf-8") as fh:
raw = yaml.safe_load(fh) or {}
bridge_section = raw.get("bridge", {})
device_a_section = raw.get("device_a", {})
device_b_section = raw.get("device_b", {})
gui_section = raw.get("gui", {})
dev_a = DeviceConfig(
port=device_a_section.get("port", "/dev/ttyUSB1"),
baud=device_a_section.get("baud", 115200),
label=device_a_section.get("label", "Device A"),
)
dev_b = DeviceConfig(
port=device_b_section.get("port", "/dev/ttyUSB2"),
baud=device_b_section.get("baud", 115200),
label=device_b_section.get("label", "Device B"),
)
return cls(
channel_name=bridge_section.get("channel_name", "bridge"),
channel_idx_a=bridge_section.get("channel_idx_a", 3),
channel_idx_b=bridge_section.get("channel_idx_b", 3),
poll_interval_ms=bridge_section.get("poll_interval_ms", 200),
forward_prefix=bridge_section.get("forward_prefix", True),
max_forwarded_cache=bridge_section.get("max_forwarded_cache", 500),
device_a=dev_a,
device_b=dev_b,
gui_port=gui_section.get("port", 9092),
gui_title=gui_section.get("title", "MeshCore Bridge"),
)
def to_yaml(self, path: Path) -> None:
"""Write the current configuration to a YAML file.
Args:
path: Destination file path.
"""
data = {
"bridge": {
"channel_name": self.channel_name,
"channel_idx_a": self.channel_idx_a,
"channel_idx_b": self.channel_idx_b,
"poll_interval_ms": self.poll_interval_ms,
"forward_prefix": self.forward_prefix,
"max_forwarded_cache": self.max_forwarded_cache,
},
"device_a": {
"port": self.device_a.port,
"baud": self.device_a.baud,
"label": self.device_a.label,
},
"device_b": {
"port": self.device_b.port,
"baud": self.device_b.baud,
"label": self.device_b.label,
},
"gui": {
"port": self.gui_port,
"title": self.gui_title,
},
}
path.parent.mkdir(parents=True, exist_ok=True)
with open(path, "w", encoding="utf-8") as fh:
yaml.dump(data, fh, default_flow_style=False, sort_keys=False)

1
meshcore_bridge/gui/__init__.py Normal file
View File

@@ -0,0 +1 @@
"""Bridge GUI package."""

193
meshcore_bridge/gui/dashboard.py Normal file
View File

@@ -0,0 +1,193 @@
"""
Bridge status dashboard — NiceGUI page with DOMCA theme.
Thin orchestrator that owns the layout, injects the DOMCA theme,
and runs a periodic update timer that refreshes all panels.
Visually consistent with the meshcore_gui dashboard.
"""
from nicegui import ui
from meshcore_gui.core.shared_data import SharedData
from meshcore_gui import config as gui_config
from meshcore_bridge.bridge_engine import BridgeEngine
from meshcore_bridge.config import BridgeConfig
from meshcore_bridge.gui.panels.status_panel import StatusPanel
from meshcore_bridge.gui.panels.log_panel import LogPanel
# ── DOMCA Theme (identical to meshcore_gui/gui/dashboard.py) ─────────
# Subset of the DOMCA theme CSS needed for the bridge dashboard.
_DOMCA_HEAD = '''
<link href="https://fonts.googleapis.com/css2?family=Exo+2:wght@800&family=JetBrains+Mono:wght@400;700&display=swap" rel="stylesheet">
<style>
/* ── DOMCA theme variables (dark) ── */
body.body--dark {
--bg: #0A1628;
--title: #48CAE4; --subtitle: #48CAE4;
}
/* ── DOMCA theme variables (light) ── */
body.body--light {
--bg: #FFFFFF;
--title: #0077B6; --subtitle: #0077B6;
}
/* ── DOMCA page background ── */
body.body--dark { background: #0A1628 !important; }
body.body--light { background: #f4f8fb !important; }
body.body--dark .q-page { background: #0A1628 !important; }
body.body--light .q-page { background: #f4f8fb !important; }
/* ── DOMCA header ── */
body.body--dark .q-header { background: #0d1f35 !important; }
body.body--light .q-header { background: #0077B6 !important; }
/* ── DOMCA cards — dark mode readable ── */
body.body--dark .q-card {
background: #112240 !important;
color: #e0f0f8 !important;
border: 1px solid rgba(0,119,182,0.15) !important;
}
body.body--dark .q-card .text-gray-600 { color: #48CAE4 !important; }
body.body--dark .q-card .text-xs { color: #c0dce8 !important; }
body.body--dark .q-card .text-sm { color: #d0e8f2 !important; }
/* ── Dark mode: fields ── */
body.body--dark .q-field__control { background: #0c1a2e !important; color: #e0f0f8 !important; }
body.body--dark .q-field__native { color: #e0f0f8 !important; }
/* ── Bridge-specific ── */
.bridge-header-text {
font-family: 'JetBrains Mono', monospace;
color: white;
}
</style>
'''
class BridgeDashboard:
"""Bridge status dashboard page.
Provides a NiceGUI-based status view showing both device
connections, bridge statistics and the forwarded message log.
"""
def __init__(
self,
shared_a: SharedData,
shared_b: SharedData,
engine: BridgeEngine,
config: BridgeConfig,
) -> None:
self._shared_a = shared_a
self._shared_b = shared_b
self._engine = engine
self._cfg = config
# Panels (created in render)
self._status: StatusPanel | None = None
self._log: LogPanel | None = None
# Header status label
self._header_status = None
def render(self) -> None:
"""Build the complete bridge dashboard layout and start the timer."""
# Create panel instances
self._status = StatusPanel(
self._shared_a, self._shared_b, self._engine, self._cfg,
)
self._log = LogPanel(self._engine)
# Inject DOMCA theme
ui.add_head_html(_DOMCA_HEAD)
# Default to dark mode
dark = ui.dark_mode(True)
# ── Header ────────────────────────────────────────────────
with ui.header().classes("items-center px-4 py-2 shadow-md"):
ui.icon("swap_horiz").classes("text-white text-2xl")
ui.label(
f"MeshCore Bridge v1.0.0"
).classes("text-lg font-bold ml-2 bridge-header-text")
ui.label(
f"({self._cfg.device_a.label} ↔ {self._cfg.device_b.label})"
).classes("text-xs ml-2 bridge-header-text").style("opacity: 0.65")
ui.space()
self._header_status = ui.label("Starting...").classes(
"text-sm opacity-70 bridge-header-text"
)
ui.button(
icon="brightness_6",
on_click=lambda: dark.toggle(),
).props("flat round dense color=white").tooltip("Toggle dark / light")
# ── Main Content ──────────────────────────────────────────
with ui.column().classes("w-full max-w-5xl mx-auto p-4 gap-4"):
# Config summary
with ui.card().classes("w-full"):
with ui.row().classes("items-center gap-2 mb-2"):
ui.icon("settings", color="primary").classes("text-lg")
ui.label("Bridge Configuration").classes(
"text-sm font-bold"
).style("font-family: 'JetBrains Mono', monospace")
with ui.row().classes("gap-4 flex-wrap"):
for lbl, val in [
("Channel", f"#{self._cfg.channel_name}"),
("Channel idx A", str(self._cfg.channel_idx_a)),
("Channel idx B", str(self._cfg.channel_idx_b)),
("Poll interval", f"{self._cfg.poll_interval_ms}ms"),
("Prefix", "ON" if self._cfg.forward_prefix else "OFF"),
("Loop cache", str(self._cfg.max_forwarded_cache)),
]:
with ui.column().classes("gap-0"):
ui.label(lbl).classes("text-xs opacity-50")
ui.label(val).classes("text-xs font-bold").style(
"font-family: 'JetBrains Mono', monospace"
)
# Status panel
self._status.render()
# Log panel
self._log.render()
# ── Update timer (500ms, same as meshcore_gui) ────────────
ui.timer(0.5, self._on_timer)
def _on_timer(self) -> None:
"""Periodic UI update callback."""
# Update header status
snap_a = self._shared_a.get_snapshot()
snap_b = self._shared_b.get_snapshot()
conn_a = snap_a.get("connected", False)
conn_b = snap_b.get("connected", False)
total = self._engine.get_total_forwarded()
if conn_a and conn_b:
status = f"✅ Both connected — {total} forwarded"
elif conn_a:
status = f"⚠️ Device B disconnected — {total} forwarded"
elif conn_b:
status = f"⚠️ Device A disconnected — {total} forwarded"
else:
status = "❌ Both devices disconnected"
if self._header_status:
self._header_status.set_text(status)
# Update panels
if self._status:
self._status.update()
if self._log:
self._log.update()

6
meshcore_bridge/gui/panels/__init__.py Normal file
View File

@@ -0,0 +1,6 @@
"""Bridge GUI panels."""
from meshcore_bridge.gui.panels.status_panel import StatusPanel
from meshcore_bridge.gui.panels.log_panel import LogPanel
__all__ = ["StatusPanel", "LogPanel"]

84
meshcore_bridge/gui/panels/log_panel.py Normal file
View File

@@ -0,0 +1,84 @@
"""
Log panel — forwarded message log for bridge troubleshooting.
Displays the last N forwarded messages with direction indicator,
sender, timestamp and message text. Layout follows the DOMCA
theme and message panel style used by meshcore_gui.
"""
from typing import Optional
from nicegui import ui
from meshcore_bridge.bridge_engine import BridgeEngine
class LogPanel:
"""Forwarded message log panel for the bridge dashboard.
Shows a scrollable list of forwarded messages, newest first,
with direction indicators (A→B / B→A).
"""
def __init__(self, engine: BridgeEngine) -> None:
self._engine = engine
self._log_container: Optional[ui.column] = None
self._last_count: int = 0
def render(self) -> None:
"""Build the log panel UI."""
with ui.card().classes("w-full"):
with ui.row().classes("items-center gap-2 mb-2"):
ui.icon("history", color="primary").classes("text-lg")
ui.label("Forwarded Messages").classes(
"text-sm font-bold"
).style("font-family: 'JetBrains Mono', monospace")
self._log_container = ui.column().classes(
"w-full gap-0 max-h-96 overflow-y-auto"
).style(
"font-family: 'JetBrains Mono', monospace; font-size: 0.75rem"
)
with self._log_container:
ui.label("Waiting for messages...").classes(
"text-xs opacity-40 py-2"
)
def update(self) -> None:
"""Refresh the log if new entries are available."""
log_entries = self._engine.get_log()
current_count = len(log_entries)
if current_count == self._last_count:
return
self._last_count = current_count
if not self._log_container:
return
self._log_container.clear()
with self._log_container:
if not log_entries:
ui.label("Waiting for messages...").classes(
"text-xs opacity-40 py-2"
)
return
for entry in log_entries[:200]:
direction_color = (
"text-blue-400" if "A→B" in entry.direction
else "text-green-400"
)
with ui.row().classes("w-full items-baseline gap-1 py-0.5"):
ui.label(entry.time).classes("text-xs opacity-50 shrink-0")
ui.label(entry.direction).classes(
f"text-xs font-bold shrink-0 {direction_color}"
)
ui.label(f"{entry.sender}:").classes(
"text-xs font-bold shrink-0"
)
ui.label(entry.text).classes(
"text-xs opacity-80 truncate"
)

192
meshcore_bridge/gui/panels/status_panel.py Normal file
View File

@@ -0,0 +1,192 @@
"""
Status panel — connection status for both bridge devices.
Shows connectivity state, device info, radio frequency and
bridge engine statistics in a layout consistent with the DOMCA
theme used by meshcore_gui.
"""
from typing import Dict, Optional
from nicegui import ui
from meshcore_gui.core.shared_data import SharedData
from meshcore_bridge.bridge_engine import BridgeEngine
from meshcore_bridge.config import BridgeConfig
class StatusPanel:
"""Connection status panel for the bridge dashboard.
Displays two device status cards (A and B) and a bridge
statistics summary card.
"""
def __init__(
self,
shared_a: SharedData,
shared_b: SharedData,
engine: BridgeEngine,
config: BridgeConfig,
) -> None:
self._a = shared_a
self._b = shared_b
self._engine = engine
self._cfg = config
# UI element references (populated by render)
self._status_a: Optional[ui.label] = None
self._status_b: Optional[ui.label] = None
self._device_a_name: Optional[ui.label] = None
self._device_b_name: Optional[ui.label] = None
self._freq_a: Optional[ui.label] = None
self._freq_b: Optional[ui.label] = None
self._connected_a: Optional[ui.icon] = None
self._connected_b: Optional[ui.icon] = None
# Stats labels
self._fwd_count: Optional[ui.label] = None
self._fwd_a_to_b: Optional[ui.label] = None
self._fwd_b_to_a: Optional[ui.label] = None
self._dupes_blocked: Optional[ui.label] = None
self._last_fwd: Optional[ui.label] = None
self._uptime: Optional[ui.label] = None
def render(self) -> None:
"""Build the status panel UI."""
with ui.row().classes("w-full gap-4 flex-wrap"):
self._render_device_card("A", self._cfg.device_a)
self._render_device_card("B", self._cfg.device_b)
self._render_stats_card()
def _render_device_card(self, side: str, dev_cfg) -> None:
"""Render a single device status card."""
with ui.card().classes("flex-1 min-w-[280px]"):
with ui.row().classes("items-center gap-2 mb-2"):
icon = ui.icon("link", color="green").classes("text-lg")
ui.label(f"Device {side}").classes(
"text-sm font-bold"
).style("font-family: 'JetBrains Mono', monospace")
ui.label(f"({dev_cfg.label})").classes(
"text-xs opacity-60"
).style("font-family: 'JetBrains Mono', monospace")
with ui.column().classes("gap-1"):
with ui.row().classes("items-center gap-2"):
ui.label("Port:").classes("text-xs opacity-60 w-20")
ui.label(dev_cfg.port).classes("text-xs").style(
"font-family: 'JetBrains Mono', monospace"
)
with ui.row().classes("items-center gap-2"):
ui.label("Status:").classes("text-xs opacity-60 w-20")
status_lbl = ui.label("Connecting...").classes("text-xs")
with ui.row().classes("items-center gap-2"):
ui.label("Device:").classes("text-xs opacity-60 w-20")
name_lbl = ui.label("-").classes("text-xs")
with ui.row().classes("items-center gap-2"):
ui.label("Frequency:").classes("text-xs opacity-60 w-20")
freq_lbl = ui.label("-").classes("text-xs")
# Store references for updates
if side == "A":
self._status_a = status_lbl
self._device_a_name = name_lbl
self._freq_a = freq_lbl
self._connected_a = icon
else:
self._status_b = status_lbl
self._device_b_name = name_lbl
self._freq_b = freq_lbl
self._connected_b = icon
def _render_stats_card(self) -> None:
"""Render the bridge statistics card."""
with ui.card().classes("flex-1 min-w-[280px]"):
with ui.row().classes("items-center gap-2 mb-2"):
ui.icon("swap_horiz", color="primary").classes("text-lg")
ui.label("Bridge Statistics").classes(
"text-sm font-bold"
).style("font-family: 'JetBrains Mono', monospace")
with ui.column().classes("gap-1"):
with ui.row().classes("items-center gap-2"):
ui.label("Total forwarded:").classes("text-xs opacity-60 w-32")
self._fwd_count = ui.label("0").classes("text-xs font-bold")
with ui.row().classes("items-center gap-2"):
ui.label("A → B:").classes("text-xs opacity-60 w-32")
self._fwd_a_to_b = ui.label("0").classes("text-xs")
with ui.row().classes("items-center gap-2"):
ui.label("B → A:").classes("text-xs opacity-60 w-32")
self._fwd_b_to_a = ui.label("0").classes("text-xs")
with ui.row().classes("items-center gap-2"):
ui.label("Dupes blocked:").classes("text-xs opacity-60 w-32")
self._dupes_blocked = ui.label("0").classes("text-xs")
with ui.row().classes("items-center gap-2"):
ui.label("Last forward:").classes("text-xs opacity-60 w-32")
self._last_fwd = ui.label("-").classes("text-xs")
with ui.row().classes("items-center gap-2"):
ui.label("Uptime:").classes("text-xs opacity-60 w-32")
self._uptime = ui.label("0s").classes("text-xs")
def update(self) -> None:
"""Refresh all status labels from current SharedData state."""
self._update_device("A", self._a)
self._update_device("B", self._b)
self._update_stats()
def _update_device(self, side: str, shared: SharedData) -> None:
"""Update device status labels for one side."""
snap = shared.get_snapshot()
if side == "A":
status_lbl = self._status_a
name_lbl = self._device_a_name
freq_lbl = self._freq_a
icon = self._connected_a
else:
status_lbl = self._status_b
name_lbl = self._device_b_name
freq_lbl = self._freq_b
icon = self._connected_b
if status_lbl:
status_lbl.set_text(snap.get("status", "Unknown"))
if name_lbl:
name_lbl.set_text(snap.get("name", "-") or "-")
if freq_lbl:
freq = snap.get("radio_freq", 0)
freq_lbl.set_text(f"{freq:.3f} MHz" if freq else "-")
if icon:
connected = snap.get("connected", False)
icon.props(f'name={"link" if connected else "link_off"}')
icon._props["color"] = "green" if connected else "red"
icon.update()
def _update_stats(self) -> None:
"""Update bridge statistics labels."""
s = self._engine.stats
total = self._engine.get_total_forwarded()
if self._fwd_count:
self._fwd_count.set_text(str(total))
if self._fwd_a_to_b:
self._fwd_a_to_b.set_text(str(s["forwarded_a_to_b"]))
if self._fwd_b_to_a:
self._fwd_b_to_a.set_text(str(s["forwarded_b_to_a"]))
if self._dupes_blocked:
self._dupes_blocked.set_text(str(s["duplicates_blocked"]))
if self._last_fwd:
self._last_fwd.set_text(s["last_forward_time"] or "-")
if self._uptime:
secs = s["uptime_seconds"]
h, rem = divmod(secs, 3600)
m, sec = divmod(rem, 60)
self._uptime.set_text(f"{h}h {m}m {sec}s" if h else f"{m}m {sec}s")

28
meshcore_gui.py Normal file
View File

@@ -0,0 +1,28 @@
#!/usr/bin/env python3
"""
MeshCore GUI — Dual Transport Edition (Serial + BLE)
=====================================================
Thin wrapper that delegates to the package entry point.
All application logic lives in :mod:`meshcore_gui.__main__`.
Usage — Serial:
python meshcore_gui.py /dev/ttyACM0
python meshcore_gui.py /dev/ttyACM0 --debug-on
Usage — BLE:
python meshcore_gui.py literal:AA:BB:CC:DD:EE:FF
python meshcore_gui.py literal:AA:BB:CC:DD:EE:FF --ble-pin 654321
python -m meshcore_gui <DEVICE>
Author: PE1HVH
Version: 5.0
SPDX-License-Identifier: MIT
Copyright: (c) 2026 PE1HVH
"""
from meshcore_gui.__main__ import main
if __name__ == "__main__":
main()

8
meshcore_gui/__init__.py Normal file
View File

@@ -0,0 +1,8 @@
"""
MeshCore GUI — Dual Transport Edition (Serial + BLE).
A graphical user interface for MeshCore mesh network devices,
communicating via USB serial or Bluetooth Low Energy (BLE).
"""
__version__ = "5.0"

290
meshcore_gui/__main__.py Normal file
View File

@@ -0,0 +1,290 @@
#!/usr/bin/env python3
"""
MeshCore GUI — Dual Transport Edition (Serial + BLE)
=====================================================
Entry point. Parses arguments, auto-detects the transport mode,
wires up the components, registers NiceGUI pages and starts the server.
Usage — Serial:
python meshcore_gui.py /dev/ttyACM0
python meshcore_gui.py /dev/ttyACM0 --debug-on
python meshcore_gui.py /dev/ttyACM0 --port=9090
python meshcore_gui.py /dev/ttyACM0 --baud=115200
python meshcore_gui.py /dev/ttyACM0 --serial-cx-dly=0.1
python meshcore_gui.py /dev/ttyACM0 --ssl
Usage — BLE:
python meshcore_gui.py literal:AA:BB:CC:DD:EE:FF
python meshcore_gui.py literal:AA:BB:CC:DD:EE:FF --ble-pin 654321
python meshcore_gui.py literal:AA:BB:CC:DD:EE:FF --debug-on
python -m meshcore_gui <DEVICE>
Author: PE1HVH
Version: 5.0
SPDX-License-Identifier: MIT
Copyright: (c) 2026 PE1HVH
"""
import sys
from nicegui import app, ui
# Allow overriding DEBUG before anything imports it
import meshcore_gui.config as config
try:
from meshcore import MeshCore, EventType # noqa: F401 — availability check
except ImportError:
print("ERROR: meshcore library not found")
print("Install with: pip install meshcore")
sys.exit(1)
from meshcore_gui.ble.worker import create_worker
from meshcore_gui.core.shared_data import SharedData
from meshcore_gui.gui.dashboard import DashboardPage
from meshcore_gui.gui.route_page import RoutePage
from meshcore_gui.gui.archive_page import ArchivePage
from meshcore_gui.services.pin_store import PinStore
from meshcore_gui.services.room_password_store import RoomPasswordStore
# Global instances (needed by NiceGUI page decorators)
_shared = None
_dashboard = None
_route_page = None
_archive_page = None
_pin_store = None
_room_password_store = None
@ui.page('/')
def _page_dashboard():
"""NiceGUI page handler — main dashboard."""
if _shared and _pin_store and _room_password_store:
DashboardPage(_shared, _pin_store, _room_password_store).render()
@ui.page('/route/{msg_key}')
def _page_route(msg_key: str):
"""NiceGUI page handler — route visualization."""
if _route_page:
_route_page.render(msg_key)
@ui.page('/archive')
def _page_archive():
"""NiceGUI page handler — message archive."""
if _archive_page:
_archive_page.render()
def _print_usage():
"""Show usage information for both serial and BLE modes."""
print("MeshCore GUI - Dual Transport Edition (Serial + BLE)")
print("=" * 55)
print()
print("Usage: python meshcore_gui.py <DEVICE> [OPTIONS]")
print()
print("The transport mode is auto-detected from the device argument:")
print(" /dev/ttyACM0 → Serial (USB)")
print(" literal:AA:BB:CC:DD:EE:FF → Bluetooth LE")
print()
print("Serial examples:")
print(" python meshcore_gui.py /dev/ttyACM0")
print(" python meshcore_gui.py /dev/ttyACM0 --debug-on")
print(" python meshcore_gui.py /dev/ttyACM0 --port=9090")
print(" python meshcore_gui.py /dev/ttyACM0 --baud=57600")
print(" python meshcore_gui.py /dev/ttyACM0 --serial-cx-dly=0.1")
print(" python meshcore_gui.py /dev/ttyACM0 --ssl")
print()
print("BLE examples:")
print(" python meshcore_gui.py literal:AA:BB:CC:DD:EE:FF")
print(" python meshcore_gui.py literal:AA:BB:CC:DD:EE:FF --debug-on")
print(" python meshcore_gui.py literal:AA:BB:CC:DD:EE:FF --ble-pin 654321")
print(" python meshcore_gui.py literal:AA:BB:CC:DD:EE:FF --ble-pin=654321")
print()
print("Common options:")
print(" --debug-on Enable verbose debug logging")
print(" --port=PORT Web server port (default: 8081)")
print(" --ssl Enable HTTPS with auto-generated certificate")
print()
print("Serial options:")
print(" --baud=BAUD Serial baudrate (default: 115200)")
print(" --serial-cx-dly=S Serial connection delay (default: 0.1)")
print()
print("BLE options:")
print(" --ble-pin PIN BLE pairing PIN (default: 123456)")
print()
print("Tips:")
print(" Serial: ls -l /dev/serial/by-id")
print(" BLE: bluetoothctl scan on")
def _parse_flags(argv):
"""Parse CLI arguments into positional args and a flag dict.
Handles ``--flag value``, ``--flag=value``, and boolean ``--flag``.
"""
args = []
flags = {}
i = 0
while i < len(argv):
a = argv[i]
if '=' in a and a.startswith('--'):
key, value = a.split('=', 1)
flags[key] = value
elif a == '--ble-pin':
if i + 1 < len(argv) and not argv[i + 1].startswith('--'):
flags['--ble-pin'] = argv[i + 1]
i += 1
else:
flags['--ble-pin'] = True
elif a.startswith('--'):
flags[a] = True
else:
args.append(a)
i += 1
return args, flags
def main():
"""Main entry point.
Parses CLI arguments, auto-detects the transport, initialises all
components and starts the NiceGUI server.
"""
global _shared, _dashboard, _route_page, _archive_page, _pin_store, _room_password_store
args, flags = _parse_flags(sys.argv[1:])
if not args:
_print_usage()
sys.exit(1)
device_id = args[0]
is_ble = config.is_ble_address(device_id)
config.TRANSPORT = "ble" if is_ble else "serial"
config.set_log_file_for_device(device_id)
# ── Common flags ──
if '--debug-on' in flags:
config.DEBUG = True
config.MESHCORE_LIB_DEBUG = True # sync: lib debug follows app debug
port = int(flags.get('--port', 8081))
# ── Serial-specific flags ──
if not is_ble:
if '--baud' in flags:
try:
config.SERIAL_BAUDRATE = int(flags['--baud'])
except ValueError:
print(f"ERROR: Invalid baudrate: {flags['--baud']}")
sys.exit(1)
if '--serial-cx-dly' in flags:
try:
config.SERIAL_CX_DELAY = float(flags['--serial-cx-dly'])
except ValueError:
print(f"ERROR: Invalid serial cx delay: {flags['--serial-cx-dly']}")
sys.exit(1)
# ── BLE-specific flags ──
if is_ble:
ble_pin = flags.get('--ble-pin')
if ble_pin and ble_pin is not True:
config.BLE_PIN = str(ble_pin)
# ── SSL ──
ssl_enabled = '--ssl' in flags
ssl_keyfile = None
ssl_certfile = None
if ssl_enabled:
import socket
import subprocess
from pathlib import Path
ssl_dir = config.DATA_DIR / 'ssl'
ssl_dir.mkdir(parents=True, exist_ok=True)
ssl_keyfile = str(ssl_dir / 'key.pem')
ssl_certfile = str(ssl_dir / 'cert.pem')
if not (ssl_dir / 'cert.pem').exists():
local_ip = '127.0.0.1'
try:
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.connect(('8.8.8.8', 80))
local_ip = s.getsockname()[0]
s.close()
except Exception:
pass
san = f"DNS:localhost,IP:127.0.0.1,IP:{local_ip}"
print(f"Generating self-signed SSL certificate (SAN: {san}) ...")
subprocess.run([
'openssl', 'req', '-x509',
'-newkey', 'rsa:2048',
'-keyout', ssl_keyfile,
'-out', ssl_certfile,
'-days', '3650',
'-nodes',
'-subj', '/CN=DOMCA MeshCore GUI',
'-addext', f'subjectAltName={san}',
], check=True, capture_output=True)
print(f"Certificate saved to {ssl_dir}/")
else:
print(f"Using existing certificate from {ssl_dir}/")
# ── Startup banner ──
transport_label = "BLE Edition" if is_ble else "Serial Edition"
print("=" * 55)
print(f"MeshCore GUI - {transport_label}")
print("=" * 55)
print(f"Device: {device_id}")
print(f"Transport: {'Bluetooth LE' if is_ble else 'USB Serial'}")
if is_ble:
print(f"BLE PIN: {config.BLE_PIN}")
else:
print(f"Baudrate: {config.SERIAL_BAUDRATE}")
print(f"CX delay: {config.SERIAL_CX_DELAY}")
print(f"Port: {port}")
print(f"SSL: {'ON (https)' if ssl_enabled else 'OFF (http)'}")
print(f"Debug mode: {'ON' if config.DEBUG else 'OFF'}")
print("=" * 55)
# ── Assemble components ──
_shared = SharedData(device_id)
_pin_store = PinStore(device_id)
_room_password_store = RoomPasswordStore(device_id)
_dashboard = DashboardPage(_shared, _pin_store, _room_password_store)
_route_page = RoutePage(_shared)
_archive_page = ArchivePage(_shared)
# ── Start worker ──
worker = create_worker(
device_id,
_shared,
baudrate=config.SERIAL_BAUDRATE,
cx_dly=config.SERIAL_CX_DELAY,
)
worker.start()
# ── Serve static PWA assets ──
from pathlib import Path
static_dir = Path(__file__).parent / 'static'
if static_dir.is_dir():
app.add_static_files('/static', str(static_dir))
# ── Start NiceGUI server (blocks) ──
run_kwargs = dict(
show=False, host='0.0.0.0', title='DOMCA MeshCore',
port=port, reload=False, storage_secret='meshcore-gui-secret',
)
if ssl_enabled:
run_kwargs['ssl_keyfile'] = ssl_keyfile
run_kwargs['ssl_certfile'] = ssl_certfile
ui.run(**run_kwargs)
if __name__ == "__main__":
main()

3
meshcore_gui/ble/__init__.py Normal file
View File

@@ -0,0 +1,3 @@
"""
Connection layer — device connection, commands and events.
"""

155
meshcore_gui/ble/ble_agent.py Normal file
View File

@@ -0,0 +1,155 @@
"""
Ingebouwde BlueZ D-Bus agent voor MeshCore BLE PIN pairing.
Vervangt de externe ``bt-agent.service`` (bluez-tools).
Gebruikt ``dbus_fast`` (async, al dependency van bleak).
De agent registreert zich bij BlueZ als default pairing agent en
beantwoordt PIN/passkey-verzoeken automatisch met de geconfigureerde
PIN (standaard ``123456`` voor T1000e).
Referentie
~~~~~~~~~~
- BlueZ Agent1 API: https://github.com/bluez/bluez/blob/master/doc/agent-api.txt
- mdphoto/meshecore-gui: https://github.com/mdphoto/meshecore-gui/blob/main/src/ble_agent.py
- dbus_fast: https://github.com/Bluetooth-Devices/dbus-fast
Author: PE1HVH / Claude
SPDX-License-Identifier: MIT
"""
import logging
from dbus_fast.aio import MessageBus
from dbus_fast import BusType
from dbus_fast.service import ServiceInterface, method
logger = logging.getLogger(__name__)
AGENT_PATH = "/meshcore/ble_agent"
CAPABILITY = "KeyboardOnly"
class BluezAgent(ServiceInterface):
"""BlueZ pairing agent die automatisch PIN afhandelt.
Implementeert de ``org.bluez.Agent1`` interface. Alle pairing-
gerelateerde callbacks geven de geconfigureerde PIN terug of
accepteren het verzoek stilzwijgend.
"""
def __init__(self, pin: str = "123456") -> None:
super().__init__("org.bluez.Agent1")
self.pin = pin
@method()
def Release(self) -> None:
logger.info("BLE Agent released")
@method()
def RequestPinCode(self, device: 'o') -> 's':
logger.info(f"PIN requested for {device}, providing: {self.pin}")
return self.pin
@method()
def RequestPasskey(self, device: 'o') -> 'u':
logger.info(f"Passkey requested for {device}, providing: {self.pin}")
return int(self.pin)
@method()
def DisplayPasskey(self, device: 'o', passkey: 'u', entered: 'q') -> None:
logger.info(f"Passkey display: {passkey} (entered: {entered})")
@method()
def DisplayPinCode(self, device: 'o', pincode: 's') -> None:
logger.info(f"PIN display: {pincode}")
@method()
def RequestConfirmation(self, device: 'o', passkey: 'u') -> None:
logger.info(f"Confirming passkey {passkey} for {device}")
@method()
def RequestAuthorization(self, device: 'o') -> None:
logger.info(f"Authorizing {device}")
@method()
def AuthorizeService(self, device: 'o', uuid: 's') -> None:
logger.info(f"Authorizing service {uuid} for {device}")
@method()
def Cancel(self) -> None:
logger.info("Pairing cancelled")
class BleAgentManager:
"""Beheert registratie/deregistratie van de BlueZ agent.
Gebruik::
agent = BleAgentManager(pin="123456")
await agent.start() # Registreer VOOR BLE connect
...
await agent.stop() # Deregistreer bij afsluiten
De manager verbindt met de system D-Bus, exporteert de agent op
``AGENT_PATH`` en registreert deze als default agent bij BlueZ.
"""
def __init__(self, pin: str = "123456") -> None:
self.pin = pin
self.bus: MessageBus | None = None
self.agent: BluezAgent | None = None
self._registered = False
@property
def is_registered(self) -> bool:
"""True als de agent succesvol geregistreerd is bij BlueZ."""
return self._registered
async def start(self) -> None:
"""Registreer agent bij BlueZ. Aanroepen VOOR BLE connect."""
try:
self.bus = await MessageBus(bus_type=BusType.SYSTEM).connect()
self.agent = BluezAgent(self.pin)
self.bus.export(AGENT_PATH, self.agent)
introspection = await self.bus.introspect("org.bluez", "/org/bluez")
proxy = self.bus.get_proxy_object(
"org.bluez", "/org/bluez", introspection
)
agent_manager = proxy.get_interface("org.bluez.AgentManager1")
await agent_manager.call_register_agent(AGENT_PATH, CAPABILITY)
await agent_manager.call_request_default_agent(AGENT_PATH)
self._registered = True
logger.info(f"BLE agent geregistreerd met PIN {self.pin}")
print(f"BLE: PIN agent geregistreerd (PIN {self.pin})")
except Exception as e:
logger.error(f"BLE agent registratie mislukt: {e}")
print(f"BLE: ⚠️ PIN agent registratie mislukt: {e}")
print(
"BLE: Tip — controleer D-Bus permissies of "
"installeer /etc/dbus-1/system.d/meshcore-ble.conf"
)
async def stop(self) -> None:
"""Deregistreer agent bij BlueZ."""
if self.bus and self._registered:
try:
introspection = await self.bus.introspect(
"org.bluez", "/org/bluez"
)
proxy = self.bus.get_proxy_object(
"org.bluez", "/org/bluez", introspection
)
agent_manager = proxy.get_interface("org.bluez.AgentManager1")
await agent_manager.call_unregister_agent(AGENT_PATH)
except Exception as e:
logger.warning(f"Agent deregistratie mislukt: {e}")
self._registered = False
if self.bus:
self.bus.disconnect()
self.bus = None
logger.info("BLE agent gestopt")
print("BLE: PIN agent gestopt")

169
meshcore_gui/ble/ble_connector.py Normal file
View File

@@ -0,0 +1,169 @@
"""
BLE bond manager via ``meshcore-ble-connect`` subprocess.
Wraps the standalone ``meshcore-ble-connect`` CLI tool (v1.1.0+) which
manages the full BLE bond lifecycle via D-Bus: discovery, pairing with
PIN (BLE SMP), trust, and bond verification. The tool is idempotent
and safe to call repeatedly.
This module replaces the built-in :class:`BleAgentManager` and the
``_ensure_paired()`` / ``remove_bond()`` flow in the BLE worker. The
``meshcore-ble-connect`` tool handles all BlueZ version differences
internally — no more ``NEEDS_PREPAIR`` branching needed.
Graceful degradation: if ``meshcore-ble-connect`` is not installed,
all functions log a warning and return success so the legacy flow
(BleAgentManager + bleak pairing) can still be used as fallback.
Exit codes from ``meshcore-ble-connect``:
+------+----------------------------+-------------------------------+
| Code | Meaning | GUI action |
+------+----------------------------+-------------------------------+
| 0 | Bond OK | Proceed with bleak connect |
| 1 | No bond (--check-only) | N/A (not used here) |
| 2 | Pairing failed | Show error, retry later |
| 3 | Adapter problem | Show "BT adapter not found" |
| 4 | D-Bus permission error | Show "Insufficient rights" |
+------+----------------------------+-------------------------------+
Author: PE1HVH / Claude
SPDX-License-Identifier: MIT
"""
import asyncio
import logging
import shutil
import subprocess
from typing import Optional, Tuple
from meshcore_gui.config import debug_print
logger = logging.getLogger(__name__)
# Timeout in seconds for the meshcore-ble-connect subprocess.
BLE_CONNECT_TIMEOUT: int = 60
# Human-readable error messages per exit code.
_EXIT_CODE_MESSAGES: dict[int, str] = {
2: "❌ BLE pairing failed — check PIN and device",
3: "❌ Bluetooth adapter not found",
4: "❌ Insufficient D-Bus permissions — check system config",
}
# Cache the availability check so we only look up the binary once.
_tool_available: Optional[bool] = None
def is_ble_connect_available() -> bool:
"""Check whether ``meshcore-ble-connect`` is on PATH.
The result is cached after the first call so repeated checks are
essentially free.
Returns:
True if the tool is found, False otherwise.
"""
global _tool_available
if _tool_available is None:
_tool_available = shutil.which("meshcore-ble-connect") is not None
if _tool_available:
logger.info("meshcore-ble-connect found on PATH")
print("BLE: ✅ meshcore-ble-connect found on PATH")
else:
logger.warning(
"meshcore-ble-connect NOT found on PATH — "
"falling back to legacy BLE agent"
)
print(
"BLE: ⚠️ meshcore-ble-connect not installed — "
"using legacy BLE agent fallback"
)
return _tool_available
async def ensure_bond(
mac: str,
pin: Optional[str] = None,
timeout: int = BLE_CONNECT_TIMEOUT,
) -> Tuple[bool, int, str]:
"""Ensure BLE bond is valid via ``meshcore-ble-connect``.
Calls the external tool as a subprocess. If the tool is not
installed, returns success immediately (graceful degradation).
Args:
mac: BLE MAC address (``literal:`` prefix is stripped).
pin: Optional pairing PIN. When ``None`` the tool runs
without ``--pin`` (interactive / agent-based pairing).
timeout: Subprocess timeout in seconds.
Returns:
Tuple of ``(success, exit_code, message)``:
- ``success``: True when the bond is valid (exit code 0) or
when the tool is not installed (fallback).
- ``exit_code``: The process exit code (0 on fallback).
- ``message``: Human-readable status or error message.
"""
if not is_ble_connect_available():
return True, 0, "meshcore-ble-connect not installed — skipped"
# Strip 'literal:' prefix if present (meshcore-ble-connect expects
# a plain MAC address).
clean_mac = mac.replace("literal:", "")
cmd = ["meshcore-ble-connect", clean_mac]
if pin:
cmd.extend(["--pin", pin])
debug_print(f"Running: {' '.join(cmd)} (timeout={timeout}s)")
logger.info("Calling meshcore-ble-connect for %s", clean_mac)
try:
result = await asyncio.to_thread(
subprocess.run,
cmd,
capture_output=True,
text=True,
timeout=timeout,
)
rc = result.returncode
if rc == 0:
msg = "BLE bond OK"
debug_print(f"meshcore-ble-connect: {msg}")
logger.info(msg)
return True, 0, msg
# Non-zero exit code — log stderr for diagnostics
stderr = result.stderr.strip()
if stderr:
debug_print(f"meshcore-ble-connect stderr: {stderr}")
logger.debug("meshcore-ble-connect stderr: %s", stderr)
msg = _EXIT_CODE_MESSAGES.get(rc, f"❌ meshcore-ble-connect failed (exit {rc})")
logger.warning("meshcore-ble-connect failed: rc=%d msg=%s", rc, msg)
print(f"BLE: {msg}")
return False, rc, msg
except subprocess.TimeoutExpired:
msg = f"❌ meshcore-ble-connect timed out after {timeout}s"
logger.error(msg)
print(f"BLE: {msg}")
return False, -1, msg
except FileNotFoundError:
# Race condition: tool was on PATH at startup but removed since.
global _tool_available
_tool_available = False
msg = "meshcore-ble-connect disappeared from PATH — skipped"
logger.warning(msg)
return True, 0, msg
except Exception as exc:
msg = f"❌ meshcore-ble-connect error: {exc}"
logger.error(msg)
debug_print(msg)
return False, -1, msg

125
meshcore_gui/ble/ble_reconnect.py Normal file
View File

@@ -0,0 +1,125 @@
"""
Automatische BLE reconnect met bond-opruiming via D-Bus.
Vervangt handmatige ``bluetoothctl remove`` stappen. Biedt twee
functies:
- :func:`remove_bond` — verwijdert een BLE bond via D-Bus
(equivalent van ``bluetoothctl remove <address>``)
- :func:`reconnect_loop` — exponential backoff reconnect met
automatische bond-opruiming
Beide functies zijn async en kunnen direct in de BLEWorker's
asyncio event loop worden aangeroepen.
Author: PE1HVH / Claude
SPDX-License-Identifier: MIT
"""
import asyncio
import logging
from typing import Any, Callable, Coroutine, Optional
from dbus_fast.aio import MessageBus
from dbus_fast import BusType
logger = logging.getLogger(__name__)
async def remove_bond(device_address: str) -> bool:
"""Verwijder BLE bond via D-Bus.
Equivalent van::
bluetoothctl remove <address>
Args:
device_address: BLE MAC-adres (bijv. ``"FF:05:D6:71:83:8D"``).
Het ``literal:`` prefix wordt automatisch verwijderd.
Returns:
True als de bond succesvol verwijderd is, False bij een fout
(bijv. als het device al verwijderd was).
"""
# Strip 'literal:' prefix als aanwezig
clean_address = device_address.replace("literal:", "")
dev_path = "/org/bluez/hci0/dev_" + clean_address.replace(":", "_")
bus = None
try:
bus = await MessageBus(bus_type=BusType.SYSTEM).connect()
introspection = await bus.introspect("org.bluez", "/org/bluez/hci0")
proxy = bus.get_proxy_object(
"org.bluez", "/org/bluez/hci0", introspection
)
adapter = proxy.get_interface("org.bluez.Adapter1")
await adapter.call_remove_device(dev_path)
logger.info(f"Bond verwijderd voor {clean_address}")
print(f"BLE: Bond verwijderd voor {clean_address}")
return True
except Exception as e:
# "Does Not Exist" is normaal als device al verwijderd was
error_str = str(e)
if "DoesNotExist" in error_str or "Does Not Exist" in error_str:
logger.debug(f"Bond al verwijderd voor {clean_address}")
print(f"BLE: Bond was al verwijderd voor {clean_address}")
else:
logger.warning(f"Bond verwijdering mislukt: {e}")
print(f"BLE: ⚠️ Bond verwijdering mislukt: {e}")
return False
finally:
if bus:
bus.disconnect()
async def reconnect_loop(
create_connection_func: Callable[[], Coroutine[Any, Any, Any]],
device_address: str,
max_retries: int = 5,
base_delay: float = 5.0,
) -> Optional[Any]:
"""Reconnect-loop: bond verwijderen, wachten, opnieuw verbinden.
Gebruikt exponential backoff: de wachttijd verdubbelt bij elke
mislukte poging (5s, 10s, 15s, 20s, 25s).
Args:
create_connection_func: Async functie die een nieuwe BLE-
verbinding opzet en het ``MeshCore`` object teruggeeft.
device_address: BLE MAC-adres.
max_retries: Maximaal aantal pogingen per disconnect.
base_delay: Basis wachttijd in seconden (vermenigvuldigt
met poging-nummer).
Returns:
Het nieuwe ``MeshCore`` object bij succes, of ``None`` als
alle pogingen mislukt zijn.
"""
for attempt in range(1, max_retries + 1):
delay = base_delay * attempt
logger.info(
f"Reconnect poging {attempt}/{max_retries} over {delay:.0f}s..."
)
print(
f"BLE: 🔄 Reconnect poging {attempt}/{max_retries} "
f"over {delay:.0f}s..."
)
await asyncio.sleep(delay)
# Stap 1: Verwijder de stale bond
await remove_bond(device_address)
await asyncio.sleep(2)
# Stap 2: Probeer opnieuw te verbinden
try:
connection = await create_connection_func()
logger.info(f"Herverbonden na poging {attempt}")
print(f"BLE: ✅ Herverbonden na poging {attempt}")
return connection
except Exception as e:
logger.error(f"Reconnect poging {attempt} mislukt: {e}")
print(f"BLE: ❌ Reconnect poging {attempt} mislukt: {e}")
logger.error(f"Reconnect mislukt na {max_retries} pogingen")
print(f"BLE: ❌ Reconnect mislukt na {max_retries} pogingen")
return None

562
meshcore_gui/ble/commands.py Normal file
View File

@@ -0,0 +1,562 @@
"""
Device command handlers for MeshCore GUI.
Extracted from ``SerialWorker`` so that each command is an isolated unit
of work. New commands can be registered without modifying existing
code (Open/Closed Principle).
"""
import asyncio
from typing import Dict, List, Optional
from meshcore import MeshCore, EventType
from meshcore_gui.config import BOT_DEVICE_NAME, DEVICE_NAME, debug_print
from meshcore_gui.core.models import Message
from meshcore_gui.core.protocols import SharedDataWriter
from meshcore_gui.services.cache import DeviceCache
class CommandHandler:
"""Dispatches and executes commands sent from the GUI.
Args:
mc: Connected MeshCore instance.
shared: SharedDataWriter for storing results.
cache: DeviceCache for persistent storage.
"""
def __init__(
self,
mc: MeshCore,
shared: SharedDataWriter,
cache: Optional[DeviceCache] = None,
) -> None:
self._mc = mc
self._shared = shared
self._cache = cache
# Handler registry — add new commands here (OCP)
self._handlers: Dict[str, object] = {
'send_message': self._cmd_send_message,
'send_dm': self._cmd_send_dm,
'send_advert': self._cmd_send_advert,
'refresh': self._cmd_refresh,
'purge_unpinned': self._cmd_purge_unpinned,
'set_auto_add': self._cmd_set_auto_add,
'set_device_name': self._cmd_set_device_name,
'login_room': self._cmd_login_room,
'logout_room': self._cmd_logout_room,
'send_room_msg': self._cmd_send_room_msg,
'load_room_history': self._cmd_load_room_history,
}
async def process_all(self) -> None:
"""Drain the command queue and dispatch each command."""
while True:
cmd = self._shared.get_next_command()
if cmd is None:
break
await self._dispatch(cmd)
async def _dispatch(self, cmd: Dict) -> None:
action = cmd.get('action')
handler = self._handlers.get(action)
if handler:
await handler(cmd)
else:
debug_print(f"Unknown command action: {action}")
# ------------------------------------------------------------------
# Individual command handlers
# ------------------------------------------------------------------
async def _cmd_send_message(self, cmd: Dict) -> None:
channel = cmd.get('channel', 0)
text = cmd.get('text', '')
is_bot = cmd.get('_bot', False)
if text:
await self._mc.commands.send_chan_msg(channel, text)
if not is_bot:
self._shared.add_message(Message.outgoing(
text, channel,
))
debug_print(
f"{'BOT' if is_bot else 'Sent'} message to "
f"channel {channel}: {text[:30]}"
)
async def _cmd_send_dm(self, cmd: Dict) -> None:
pubkey = cmd.get('pubkey', '')
text = cmd.get('text', '')
contact_name = cmd.get('contact_name', pubkey[:8])
if text and pubkey:
await self._mc.commands.send_msg(pubkey, text)
self._shared.add_message(Message.outgoing(
text, None, sender_pubkey=pubkey,
))
debug_print(f"Sent DM to {contact_name}: {text[:30]}")
async def _cmd_send_advert(self, cmd: Dict) -> None:
await self._mc.commands.send_advert(flood=True)
self._shared.set_status("📢 Advert sent")
debug_print("Advert sent")
async def _cmd_refresh(self, cmd: Dict) -> None:
debug_print("Refresh requested")
# Delegate to the worker's _load_data via a callback
if self._load_data_callback:
try:
self._shared.set_status("🔄 Refreshing...")
await self._load_data_callback()
self._shared.set_status("✅ Refreshed")
except Exception as exc:
self._shared.set_status(f"⚠️ Refresh error: {exc}")
debug_print(f"Refresh failed: {exc}")
async def _cmd_purge_unpinned(self, cmd: Dict) -> None:
"""Remove unpinned contacts from the MeshCore device.
Iterates the list of public keys, calls ``remove_contact``
for each one with a short delay between calls to avoid
overwhelming the link. After completion, triggers a
full refresh so the GUI reflects the new state.
If ``delete_from_history`` is True, also removes the
contacts from the local device cache on disk.
Expected command dict::
{
'action': 'purge_unpinned',
'pubkeys': ['aabbcc...', ...],
'delete_from_history': True/False,
}
"""
pubkeys: List[str] = cmd.get('pubkeys', [])
delete_from_history: bool = cmd.get('delete_from_history', False)
if not pubkeys:
self._shared.set_status("⚠️ No contacts to remove")
return
total = len(pubkeys)
removed = 0
errors = 0
self._shared.set_status(
f"🗑️ Removing {total} contacts..."
)
debug_print(f"Purge: starting removal of {total} contacts")
for i, pubkey in enumerate(pubkeys, 1):
try:
r = await self._mc.commands.remove_contact(pubkey)
if r.type == EventType.ERROR:
errors += 1
debug_print(
f"Purge: remove_contact({pubkey[:16]}) "
f"returned ERROR"
)
else:
removed += 1
debug_print(
f"Purge: removed {pubkey[:16]} "
f"({i}/{total})"
)
except Exception as exc:
errors += 1
debug_print(
f"Purge: remove_contact({pubkey[:16]}) "
f"exception: {exc}"
)
# Update status with progress
self._shared.set_status(
f"🗑️ Removing... {i}/{total}"
)
# Brief pause between calls to avoid congestion
if i < total:
await asyncio.sleep(0.5)
# Delete from local cache if requested
if delete_from_history and self._cache:
cache_removed = self._cache.remove_contacts(pubkeys)
debug_print(
f"Purge: removed {cache_removed} contacts "
f"from local history"
)
# Summary
if errors:
status = (
f"⚠️ {removed} contacts removed, "
f"{errors} failed"
)
else:
history_suffix = " and local history" if delete_from_history else ""
status = f"✅ {removed} contacts removed from device{history_suffix}"
self._shared.set_status(status)
print(f"Purge: {status}")
# Resync with device to confirm new state
if self._load_data_callback:
await self._load_data_callback()
async def _cmd_set_auto_add(self, cmd: Dict) -> None:
"""Toggle auto-add contacts on the MeshCore device.
The SDK function ``set_manual_add_contacts(true)`` means
*manual mode* (auto-add OFF). The UI toggle is inverted:
toggle ON = auto-add ON = ``set_manual_add_contacts(false)``.
On failure the SharedData flag is rolled back so the GUI
checkbox reverts on the next update cycle.
Note: some firmware/SDK versions raise ``KeyError`` (e.g.
``'telemetry_mode_base'``) when parsing the device response.
The command itself was already sent successfully in that
case, so we treat ``KeyError`` as *probable success* and keep
the requested state instead of rolling back.
Expected command dict::
{
'action': 'set_auto_add',
'enabled': True/False,
}
"""
enabled: bool = cmd.get('enabled', False)
# Invert: UI "auto-add ON" → manual_add = False
manual_add = not enabled
state = "ON" if enabled else "OFF"
try:
r = await self._mc.commands.set_manual_add_contacts(manual_add)
if r.type == EventType.ERROR:
# Rollback
self._shared.set_auto_add_enabled(not enabled)
self._shared.set_status(
"⚠️ Failed to change auto-add setting"
)
debug_print(
f"set_auto_add: ERROR response, rolled back to "
f"{'enabled' if not enabled else 'disabled'}"
)
else:
self._shared.set_auto_add_enabled(enabled)
self._shared.set_status(f"✅ Auto-add contacts: {state}")
debug_print(f"set_auto_add: success → {state}")
except KeyError as exc:
# SDK response-parsing error (e.g. missing 'telemetry_mode_base').
# The command was already transmitted; the device has likely
# accepted the new setting. Keep the requested state.
self._shared.set_auto_add_enabled(enabled)
self._shared.set_status(f"✅ Auto-add contacts: {state}")
debug_print(
f"set_auto_add: KeyError '{exc}' during response parse — "
f"command sent, treating as success → {state}"
)
except Exception as exc:
# Rollback
self._shared.set_auto_add_enabled(not enabled)
self._shared.set_status(
f"⚠️ Auto-add error: {exc}"
)
debug_print(f"set_auto_add exception: {exc}")
async def _cmd_set_device_name(self, cmd: Dict) -> None:
"""Set or restore the device name.
Uses the fixed names from config.py unless an explicit name is provided:
- Explicit name → set to that value
- BOT enabled → ``BOT_DEVICE_NAME`` (e.g. "NL-OV-ZWL-STDSHGN-WKC Bot")
- BOT disabled → ``DEVICE_NAME`` (e.g. "PE1HVH T1000e")
This avoids the previous bug where the dynamically read device
name could already be the bot name (e.g. after a restart while
BOT was active), causing the original name to be overwritten
with the bot name.
On failure the bot_enabled flag is rolled back so the GUI
checkbox reverts on the next update cycle.
Expected command dict::
{
'action': 'set_device_name',
'bot_enabled': True/False,
'name': 'optional explicit name',
}
"""
explicit_name = cmd.get('name')
has_explicit_name = explicit_name is not None and str(explicit_name).strip() != ""
if has_explicit_name:
target_name = str(explicit_name).strip()
bot_enabled = self._shared.is_bot_enabled()
else:
bot_enabled = bool(cmd.get('bot_enabled', False))
target_name = BOT_DEVICE_NAME if bot_enabled else DEVICE_NAME
try:
r = await self._mc.commands.set_name(target_name)
if r.type == EventType.ERROR:
# Rollback only when driven by BOT toggle
if not has_explicit_name:
self._shared.set_bot_enabled(not bot_enabled)
self._shared.set_status(
f"⚠️ Failed to set device name to '{target_name}'"
)
debug_print(
f"set_device_name: ERROR response for '{target_name}', "
f"{'rolled back bot_enabled to ' + str(not bot_enabled) if not has_explicit_name else 'no bot rollback'}"
)
return
self._shared.set_status(f"✅ Device name → {target_name}")
debug_print(f"set_device_name: success → '{target_name}'")
# Send advert so the network sees the new name
await self._mc.commands.send_advert(flood=True)
debug_print("set_device_name: advert sent")
except Exception as exc:
# Rollback on exception (BOT toggle only)
if not has_explicit_name:
self._shared.set_bot_enabled(not bot_enabled)
self._shared.set_status(f"⚠️ Device name error: {exc}")
debug_print(f"set_device_name exception: {exc}")
async def _cmd_login_room(self, cmd: Dict) -> None:
"""Login to a Room Server.
Follows the reference implementation (meshcore-cli):
1. ``send_login()`` → wait for ``MSG_SENT`` (companion radio sent LoRa packet)
2. ``wait_for_event(LOGIN_SUCCESS)`` → wait for room server confirmation
3. After LOGIN_SUCCESS, the room server starts pushing historical
messages over RF. ``auto_message_fetching`` handles those.
Expected command dict::
{
'action': 'login_room',
'pubkey': '<hex public key>',
'password': '<room password>',
'room_name': '<display name>',
}
"""
pubkey: str = cmd.get('pubkey', '')
password: str = cmd.get('password', '')
room_name: str = cmd.get('room_name', pubkey[:8])
if not pubkey:
self._shared.set_status("⚠️ Room login: no pubkey")
return
# Load archived room messages so the panel shows history
# while we wait for the LoRa login handshake.
self._shared.load_room_history(pubkey)
# Mark pending in SharedData so the panel can update
self._shared.set_room_login_state(pubkey, 'pending', 'Sending login…')
try:
# Step 1: Send login request to companion radio
self._shared.set_status(
f"🔄 Sending login to {room_name}…"
)
r = await self._mc.commands.send_login(pubkey, password)
if r.type == EventType.ERROR:
self._shared.set_room_login_state(
pubkey, 'fail', 'Login send failed',
)
self._shared.set_status(
f"⚠️ Room login failed: {room_name}"
)
debug_print(
f"login_room: send_login ERROR for {room_name} "
f"({pubkey[:16]})"
)
return
# Step 2: Wait for LOGIN_SUCCESS from room server via LoRa
# Use suggested_timeout from companion radio if available,
# otherwise default to 120 seconds (LoRa can be slow).
suggested = (r.payload or {}).get('suggested_timeout', 96000)
timeout_secs = max(suggested / 800, 30.0)
self._shared.set_status(
f"⏳ Waiting for room server response ({room_name})…"
)
debug_print(
f"login_room: MSG_SENT OK, waiting for LOGIN_SUCCESS "
f"(timeout={timeout_secs:.0f}s)"
)
login_event = await self._mc.wait_for_event(
EventType.LOGIN_SUCCESS, timeout=timeout_secs,
)
if login_event and login_event.type == EventType.LOGIN_SUCCESS:
is_admin = (login_event.payload or {}).get('is_admin', False)
self._shared.set_room_login_state(
pubkey, 'ok',
f"admin={is_admin}",
)
self._shared.set_status(
f"✅ Room login OK: {room_name} — "
f"history arriving over RF…"
)
debug_print(
f"login_room: LOGIN_SUCCESS for {room_name} "
f"(admin={is_admin})"
)
# Defensive: trigger one get_msg() to check for any
# messages already waiting in the companion radio's
# offline queue. auto_message_fetching handles the
# rest via MESSAGES_WAITING events.
try:
await self._mc.commands.get_msg()
debug_print("login_room: defensive get_msg() done")
except Exception as exc:
debug_print(f"login_room: defensive get_msg() error: {exc}")
else:
self._shared.set_room_login_state(
pubkey, 'fail',
'Timeout — no response from room server',
)
self._shared.set_status(
f"⚠️ Room login timeout: {room_name} "
f"(no response after {timeout_secs:.0f}s)"
)
debug_print(
f"login_room: LOGIN_SUCCESS timeout for "
f"{room_name} ({pubkey[:16]})"
)
except Exception as exc:
self._shared.set_room_login_state(
pubkey, 'fail', str(exc),
)
self._shared.set_status(
f"⚠️ Room login error: {exc}"
)
debug_print(f"login_room exception: {exc}")
async def _cmd_logout_room(self, cmd: Dict) -> None:
"""Logout from a Room Server.
Sends a logout command to the companion radio so it stops
keep-alive pings and the room server deregisters the client.
This resets the server-side ``sync_since`` state, ensuring
that the next login will receive the full message history.
Expected command dict::
{
'action': 'logout_room',
'pubkey': '<hex public key>',
'room_name': '<display name>',
}
"""
pubkey: str = cmd.get('pubkey', '')
room_name: str = cmd.get('room_name', pubkey[:8])
if not pubkey:
return
try:
r = await self._mc.commands.send_logout(pubkey)
if r.type == EventType.ERROR:
debug_print(
f"logout_room: ERROR for {room_name} "
f"({pubkey[:16]})"
)
else:
debug_print(
f"logout_room: OK for {room_name} "
f"({pubkey[:16]})"
)
except AttributeError:
# Library may not have send_logout — fall back to silent
debug_print(
f"logout_room: send_logout not available in library, "
f"skipping for {room_name}"
)
except Exception as exc:
debug_print(f"logout_room exception: {exc}")
self._shared.set_room_login_state(pubkey, 'logged_out')
self._shared.set_status(
f"Logged out from {room_name}"
)
async def _cmd_load_room_history(self, cmd: Dict) -> None:
"""Load archived room messages into the in-memory cache.
Called when a room card is rendered so the panel can display
historical messages even before login. Also safe to call
after login to refresh.
Expected command dict::
{
'action': 'load_room_history',
'pubkey': '<hex public key>',
}
"""
pubkey: str = cmd.get('pubkey', '')
if pubkey:
self._shared.load_room_history(pubkey)
async def _cmd_send_room_msg(self, cmd: Dict) -> None:
"""Send a message to a Room Server (post to room).
Uses ``send_msg`` with the Room Server's public key, which
is the standard way to post a message to a room after login.
Expected command dict::
{
'action': 'send_room_msg',
'pubkey': '<hex public key>',
'text': '<message text>',
'room_name': '<display name>',
}
"""
pubkey: str = cmd.get('pubkey', '')
text: str = cmd.get('text', '')
room_name: str = cmd.get('room_name', pubkey[:8])
if not text or not pubkey:
return
try:
await self._mc.commands.send_msg(pubkey, text)
self._shared.add_message(Message.outgoing(
text, None, sender_pubkey=pubkey,
))
debug_print(
f"send_room_msg: sent to {room_name}: "
f"{text[:30]}"
)
except Exception as exc:
self._shared.set_status(
f"⚠️ Room message error: {exc}"
)
debug_print(f"send_room_msg exception: {exc}")
# ------------------------------------------------------------------
# Callback for refresh (set by SerialWorker after construction)
# ------------------------------------------------------------------
_load_data_callback = None
def set_load_data_callback(self, callback) -> None:
"""Register the worker's ``_load_data`` coroutine for refresh."""
self._load_data_callback = callback

379
meshcore_gui/ble/events.py Normal file
View File

@@ -0,0 +1,379 @@
"""
Device event callbacks for MeshCore GUI.
Handles ``CHANNEL_MSG_RECV``, ``CONTACT_MSG_RECV`` and ``RX_LOG_DATA``
events from the MeshCore library. Extracted from ``SerialWorker`` so the
worker only deals with connection lifecycle.
"""
from typing import Dict, Optional
from meshcore_gui.config import debug_print
from meshcore_gui.core.models import Message, RxLogEntry
from meshcore_gui.core.protocols import SharedDataWriter
from meshcore_gui.ble.packet_decoder import PacketDecoder, PayloadType
from meshcore_gui.services.bot import MeshBot
from meshcore_gui.services.dedup import DualDeduplicator
class EventHandler:
"""Processes device events and writes results to shared data.
Args:
shared: SharedDataWriter for storing messages and RX log.
decoder: PacketDecoder for raw LoRa packet decryption.
dedup: DualDeduplicator for message deduplication.
bot: MeshBot for auto-reply logic.
"""
# Maximum entries in the path cache before oldest are evicted.
_PATH_CACHE_MAX = 200
def __init__(
self,
shared: SharedDataWriter,
decoder: PacketDecoder,
dedup: DualDeduplicator,
bot: MeshBot,
) -> None:
self._shared = shared
self._decoder = decoder
self._dedup = dedup
self._bot = bot
# Cache: message_hash → path_hashes (from RX_LOG decode).
# Used by on_channel_msg fallback to recover hashes that the
# CHANNEL_MSG_RECV event does not provide.
self._path_cache: Dict[str, list] = {}
# ------------------------------------------------------------------
# Helpers — resolve names at receive time
# ------------------------------------------------------------------
def _resolve_path_names(self, path_hashes: list) -> list:
"""Resolve 2-char path hashes to display names.
Performs a contact lookup for each hash *now* so the names are
captured at receive time and stored in the archive.
Args:
path_hashes: List of 2-char hex strings.
Returns:
List of display names (same length as *path_hashes*).
Unknown hashes become their uppercase hex value.
"""
names = []
for h in path_hashes:
if not h or len(h) < 2:
names.append('-')
continue
name = self._shared.get_contact_name_by_prefix(h)
# get_contact_name_by_prefix returns h[:8] as fallback,
# normalise to uppercase hex for 2-char hashes.
if name and name != h[:8]:
names.append(name)
else:
names.append(h.upper())
return names
# ------------------------------------------------------------------
# RX_LOG_DATA — the single source of truth for path info
# ------------------------------------------------------------------
def on_rx_log(self, event) -> None:
"""Handle RX log data events."""
payload = event.payload
# Extract basic RX log info
time_str = Message.now_timestamp()
snr = payload.get('snr', 0)
rssi = payload.get('rssi', 0)
payload_type = '?'
hops = payload.get('path_len', 0)
# Try to decode payload to get message_hash
message_hash = ""
rx_path_hashes: list = []
rx_path_names: list = []
rx_sender: str = ""
rx_receiver: str = self._shared.get_device_name() or ""
payload_hex = payload.get('payload', '')
decoded = None
if payload_hex:
decoded = self._decoder.decode(payload_hex)
if decoded is not None:
message_hash = decoded.message_hash
payload_type = self._decoder.get_payload_type_text(decoded.payload_type)
# Capture path info for all packet types
if decoded.path_hashes:
rx_path_hashes = decoded.path_hashes
rx_path_names = self._resolve_path_names(decoded.path_hashes)
# Use decoded path_length (from packet body) — more
# reliable than the frame-header path_len which can be 0.
if decoded.path_length:
hops = decoded.path_length
# Capture sender name when available (GroupText only)
if decoded.sender:
rx_sender = decoded.sender
# Cache path_hashes for correlation with on_channel_msg
if decoded.path_hashes and message_hash:
self._path_cache[message_hash] = decoded.path_hashes
# Evict oldest entries if cache is too large
if len(self._path_cache) > self._PATH_CACHE_MAX:
oldest = next(iter(self._path_cache))
del self._path_cache[oldest]
# Process decoded message if it's a group text
if decoded.payload_type == PayloadType.GroupText and decoded.is_decrypted:
if decoded.channel_idx is None:
# The channel hash could not be resolved to a channel index
# (PacketDecoder._hash_to_idx lookup returned None).
# Marking dedup here would suppress on_channel_msg, which
# carries a valid channel_idx from the device event — the only
# path through which the bot can pass Guard 2 and respond.
# Skip the entire block; on_channel_msg handles message + bot.
# Path info is already in _path_cache for on_channel_msg to use.
debug_print(
f"RX_LOG → GroupText decrypted but channel_idx unresolved "
f"(hash={decoded.message_hash}); deferring to on_channel_msg"
)
else:
self._dedup.mark_hash(decoded.message_hash)
self._dedup.mark_content(
decoded.sender, decoded.channel_idx, decoded.text,
)
sender_pubkey = ''
if decoded.sender:
match = self._shared.get_contact_by_name(decoded.sender)
if match:
sender_pubkey, _contact = match
snr_msg = self._extract_snr(payload)
self._shared.add_message(Message.incoming(
decoded.sender,
decoded.text,
decoded.channel_idx,
time=time_str,
snr=snr_msg,
path_len=decoded.path_length,
sender_pubkey=sender_pubkey,
path_hashes=decoded.path_hashes,
path_names=rx_path_names,
message_hash=decoded.message_hash,
))
debug_print(
f"RX_LOG → message: hash={decoded.message_hash}, "
f"sender={decoded.sender!r}, ch={decoded.channel_idx}, "
f"path={decoded.path_hashes}, "
f"path_names={rx_path_names}"
)
self._bot.check_and_reply(
sender=decoded.sender,
text=decoded.text,
channel_idx=decoded.channel_idx,
snr=snr_msg,
path_len=decoded.path_length,
path_hashes=decoded.path_hashes,
)
# Add RX log entry with message_hash and path info (if available)
# ── Fase 1 Observer: raw packet metadata ──
raw_packet_len = len(payload_hex) // 2 if payload_hex else 0
raw_payload_len = max(0, raw_packet_len - 1 - hops) if payload_hex else 0
raw_route_type = "D" if hops > 0 else ("F" if payload_hex else "")
raw_packet_type_num = -1
if payload_hex and decoded is not None:
try:
raw_packet_type_num = decoded.payload_type.value
except (AttributeError, ValueError):
pass
self._shared.add_rx_log(RxLogEntry(
time=time_str,
snr=snr,
rssi=rssi,
payload_type=payload_type,
hops=hops,
message_hash=message_hash,
path_hashes=rx_path_hashes,
path_names=rx_path_names,
sender=rx_sender,
receiver=rx_receiver,
raw_payload=payload_hex,
packet_len=raw_packet_len,
payload_len=raw_payload_len,
route_type=raw_route_type,
packet_type_num=raw_packet_type_num,
))
# ------------------------------------------------------------------
# CHANNEL_MSG_RECV — fallback when RX_LOG decode missed it
# ------------------------------------------------------------------
def on_channel_msg(self, event) -> None:
"""Handle channel message events."""
payload = event.payload
debug_print(f"Channel msg payload keys: {list(payload.keys())}")
# Dedup via hash
msg_hash = payload.get('message_hash', '')
if msg_hash and self._dedup.is_hash_seen(msg_hash):
debug_print(f"Channel msg suppressed (hash): {msg_hash}")
return
# Parse sender from "SenderName: message body" format
raw_text = payload.get('text', '')
sender, msg_text = '', raw_text
if ': ' in raw_text:
name_part, body_part = raw_text.split(': ', 1)
sender = name_part.strip()
msg_text = body_part
elif raw_text:
msg_text = raw_text
# Dedup via content
ch_idx = payload.get('channel_idx')
if self._dedup.is_content_seen(sender, ch_idx, msg_text):
debug_print(f"Channel msg suppressed (content): {sender!r}")
return
debug_print(
f"Channel msg (fallback): sender={sender!r}, "
f"text={msg_text[:40]!r}"
)
sender_pubkey = ''
if sender:
match = self._shared.get_contact_by_name(sender)
if match:
sender_pubkey, _contact = match
snr = self._extract_snr(payload)
# Recover path_hashes from RX_LOG cache (CHANNEL_MSG_RECV
# does not carry them, but the preceding RX_LOG decode does).
path_hashes = self._path_cache.pop(msg_hash, []) if msg_hash else []
path_names = self._resolve_path_names(path_hashes)
self._shared.add_message(Message.incoming(
sender,
msg_text,
ch_idx,
snr=snr,
path_len=payload.get('path_len', 0),
sender_pubkey=sender_pubkey,
path_hashes=path_hashes,
path_names=path_names,
message_hash=msg_hash,
))
self._bot.check_and_reply(
sender=sender,
text=msg_text,
channel_idx=ch_idx,
snr=snr,
path_len=payload.get('path_len', 0),
)
# ------------------------------------------------------------------
# CONTACT_MSG_RECV — DMs
# ------------------------------------------------------------------
def on_contact_msg(self, event) -> None:
"""Handle direct message and room message events.
Room Server messages arrive as ``CONTACT_MSG_RECV`` with
``txt_type == 2``. The ``pubkey_prefix`` is the Room Server's
key and the ``signature`` field contains the original author's
pubkey prefix. We resolve the author name from ``signature``
so the UI shows who actually wrote the message.
"""
payload = event.payload
pubkey = payload.get('pubkey_prefix', '')
txt_type = payload.get('txt_type', 0)
signature = payload.get('signature', '')
debug_print(f"DM payload keys: {list(payload.keys())}")
# Common fields for both Room and DM messages
msg_hash = payload.get('message_hash', '')
path_hashes = self._path_cache.pop(msg_hash, []) if msg_hash else []
path_names = self._resolve_path_names(path_hashes)
# DM payloads may report path_len=255 (0xFF) meaning "unknown";
# treat as 0 when no actual path data is available.
raw_path_len = payload.get('path_len', 0)
path_len = raw_path_len if raw_path_len < 255 else 0
if path_hashes:
# Trust actual decoded hashes over the raw header value
path_len = len(path_hashes)
# --- Room Server message (txt_type 2) ---
if txt_type == 2 and signature:
# Resolve actual author from signature (author pubkey prefix)
author = self._shared.get_contact_name_by_prefix(signature)
if not author:
author = signature[:8] if signature else '?'
self._shared.add_message(Message.incoming(
author,
payload.get('text', ''),
None,
snr=self._extract_snr(payload),
path_len=path_len,
sender_pubkey=pubkey,
path_hashes=path_hashes,
path_names=path_names,
message_hash=msg_hash,
))
debug_print(
f"Room msg from {author} (sig={signature}) "
f"via room {pubkey[:12]}: "
f"{payload.get('text', '')[:30]}"
)
return
# --- Regular DM ---
sender = ''
if pubkey:
sender = self._shared.get_contact_name_by_prefix(pubkey)
if not sender:
sender = pubkey[:8] if pubkey else ''
self._shared.add_message(Message.incoming(
sender,
payload.get('text', ''),
None,
snr=self._extract_snr(payload),
path_len=path_len,
sender_pubkey=pubkey,
path_hashes=path_hashes,
path_names=path_names,
message_hash=msg_hash,
))
debug_print(f"DM received from {sender}: {payload.get('text', '')[:30]}")
# ------------------------------------------------------------------
# Helpers
# ------------------------------------------------------------------
@staticmethod
def _extract_snr(payload: Dict) -> Optional[float]:
"""Extract SNR from a payload dict (handles 'SNR' and 'snr' keys)."""
raw = payload.get('SNR') or payload.get('snr')
if raw is not None:
try:
return float(raw)
except (ValueError, TypeError):
pass
return None

216
meshcore_gui/ble/packet_decoder.py Normal file
View File

@@ -0,0 +1,216 @@
"""
Packet decoder for MeshCore GUI — single-source approach.
Wraps ``meshcoredecoder`` to decode raw LoRa packets from RX_LOG_DATA
events. A single raw packet contains **everything**: message_hash,
path hashes, hop count, and (with channel keys) the decrypted text
and sender name.
No correlation with CHANNEL_MSG_RECV events is needed.
Channel decryption keys are loaded at startup (fetched from the device
via ``get_channel()`` or derived from the channel name as fallback).
"""
from dataclasses import dataclass, field
from hashlib import sha256
from typing import Dict, List, Optional
from meshcoredecoder import MeshCoreDecoder
from meshcoredecoder.crypto.channel_crypto import ChannelCrypto
from meshcoredecoder.crypto.key_manager import MeshCoreKeyStore
from meshcoredecoder.types.crypto import DecryptionOptions
from meshcoredecoder.types.enums import PayloadType
from meshcoredecoder.utils.enum_names import get_payload_type_name
from meshcore_gui.config import debug_print
# Re-export so other modules don't need to import meshcoredecoder
__all__ = ["PacketDecoder", "DecodedPacket", "PayloadType"]
# ---------------------------------------------------------------------------
# Decoded result
# ---------------------------------------------------------------------------
@dataclass
class DecodedPacket:
"""All data extracted from a single raw LoRa packet.
Attributes:
message_hash: Deterministic packet identifier (hex string).
payload_type: Enum (GroupText, Advert, Ack, …).
path_length: Number of repeater hashes in the path.
path_hashes: 2-char hex strings, one per repeater.
sender: Sender name (GroupText only, after decryption).
text: Message body (GroupText only, after decryption).
channel_idx: Channel index (GroupText only, via hash→idx map).
timestamp: Message timestamp (GroupText only).
is_decrypted: True if payload was successfully decrypted.
"""
message_hash: str
payload_type: PayloadType
path_length: int
path_hashes: List[str] = field(default_factory=list)
# GroupText-specific (populated after successful decryption)
sender: str = ""
text: str = ""
channel_idx: Optional[int] = None
timestamp: int = 0
is_decrypted: bool = False
# ---------------------------------------------------------------------------
# Decoder
# ---------------------------------------------------------------------------
class PacketDecoder:
"""Decode raw LoRa packets with channel-key decryption.
Usage::
decoder = PacketDecoder()
decoder.add_channel_key(0, secret_bytes) # from device
decoder.add_channel_key_from_name(1, "#test") # fallback
result = decoder.decode(payload_hex)
if result and result.is_decrypted:
print(result.sender, result.text, result.path_hashes)
"""
def __init__(self) -> None:
self._key_store = MeshCoreKeyStore()
self._options: Optional[DecryptionOptions] = None
# channel_hash (2-char lower hex) → channel_idx
self._hash_to_idx: Dict[str, int] = {}
# ------------------------------------------------------------------
# Key management
# ------------------------------------------------------------------
def add_channel_key(
self,
channel_idx: int,
secret_bytes: bytes,
source: str = "device",
) -> None:
"""Register a channel decryption key (16 raw bytes from device).
Args:
channel_idx: Channel index (0-based).
secret_bytes: 16-byte channel secret from ``get_channel()``.
source: Label for debug output (e.g. "device", "cache").
"""
secret_hex = secret_bytes.hex()
self._key_store.add_channel_secrets([secret_hex])
self._rebuild_options()
ch_hash = ChannelCrypto.calculate_channel_hash(secret_hex).lower()
self._hash_to_idx[ch_hash] = channel_idx
debug_print(
f"PacketDecoder: key for ch{channel_idx} "
f"(hash={ch_hash}, from {source})"
)
def add_channel_key_from_name(
self, channel_idx: int, channel_name: str,
) -> None:
"""Derive a channel key from the channel name (fallback).
MeshCore derives channel secrets as
``SHA-256(name.encode('utf-8'))[:16]``.
Args:
channel_idx: Channel index (0-based).
channel_name: Channel name string (e.g. ``"#test"``).
"""
secret_bytes = sha256(channel_name.encode("utf-8")).digest()[:16]
self.add_channel_key(channel_idx, secret_bytes, source=f"name '{channel_name}'")
@property
def has_keys(self) -> bool:
"""True if at least one channel key has been registered."""
return self._options is not None
# ------------------------------------------------------------------
# Decode
# ------------------------------------------------------------------
def decode(self, payload_hex: str) -> Optional[DecodedPacket]:
"""Decode a raw LoRa packet hex string.
Args:
payload_hex: Hex string from the RX_LOG_DATA event's
``payload`` field.
Returns:
:class:`DecodedPacket` on success, ``None`` if the data
is invalid or too short.
"""
if not payload_hex:
return None
try:
packet = MeshCoreDecoder.decode(payload_hex, self._options)
except Exception as exc:
debug_print(f"PacketDecoder: decode error: {exc}")
return None
if not packet.is_valid:
debug_print(f"PacketDecoder: invalid: {packet.errors}")
return None
result = DecodedPacket(
message_hash=packet.message_hash,
payload_type=packet.payload_type,
path_length=packet.path_length,
path_hashes=list(packet.path) if packet.path else [],
)
# --- GroupText decryption ---
if packet.payload_type == PayloadType.GroupText:
decoded_payload = packet.payload.get("decoded")
if decoded_payload and decoded_payload.decrypted:
d = decoded_payload.decrypted
result.sender = d.get("sender", "") or ""
result.text = d.get("message", "") or ""
result.timestamp = d.get("timestamp", 0)
result.is_decrypted = True
# Resolve channel_hash → channel_idx
ch_hash = decoded_payload.channel_hash.lower()
result.channel_idx = self._hash_to_idx.get(ch_hash)
debug_print(
f"PacketDecoder: GroupText OK — "
f"hash={result.message_hash}, "
f"sender={result.sender!r}, "
f"ch={result.channel_idx}, "
f"path={result.path_hashes}, "
f"text={result.text[:40]!r}"
)
else:
debug_print(
f"PacketDecoder: GroupText NOT decrypted "
f"(hash={result.message_hash})"
)
return result
def get_payload_type_text(self, payload_type: PayloadType) -> str:
"""Get human-friendly name for a PayloadType enum value."""
return get_payload_type_name(payload_type)
# ------------------------------------------------------------------
# Internal
# ------------------------------------------------------------------
def _rebuild_options(self) -> None:
"""Recreate DecryptionOptions after a key change."""
self._options = DecryptionOptions(key_store=self._key_store)

964
meshcore_gui/ble/worker.py Normal file
View File

@@ -0,0 +1,964 @@
"""
Communication worker for MeshCore GUI (Serial + BLE).
Runs in a separate thread with its own asyncio event loop. Connects
to the MeshCore device, wires up collaborators, and runs the command
processing loop.
Transport selection
~~~~~~~~~~~~~~~~~~~~
The :func:`create_worker` factory returns the appropriate worker class
based on the device identifier:
- ``/dev/ttyACM0`` → :class:`SerialWorker` (USB serial)
- ``literal:AA:BB:CC:DD:EE:FF`` → :class:`BLEWorker` (Bluetooth LE)
Both workers share the same base class (:class:`_BaseWorker`) which
implements the main loop, event wiring, data loading and caching.
Command execution → :mod:`meshcore_gui.ble.commands`
Event handling → :mod:`meshcore_gui.ble.events`
Packet decoding → :mod:`meshcore_gui.ble.packet_decoder`
PIN agent (BLE) → :mod:`meshcore_gui.ble.ble_agent`
Reconnect (BLE) → :mod:`meshcore_gui.ble.ble_reconnect`
Bot logic → :mod:`meshcore_gui.services.bot`
Deduplication → :mod:`meshcore_gui.services.dedup`
Cache → :mod:`meshcore_gui.services.cache`
Author: PE1HVH
SPDX-License-Identifier: MIT
"""
import abc
import asyncio
import threading
import time
from typing import Dict, List, Optional, Set
from meshcore import MeshCore, EventType
import meshcore_gui.config as _config
from meshcore_gui.config import (
DEFAULT_TIMEOUT,
CHANNEL_CACHE_ENABLED,
CONTACT_REFRESH_SECONDS,
MAX_CHANNELS,
RECONNECT_BASE_DELAY,
RECONNECT_MAX_RETRIES,
debug_data,
debug_print,
pp,
)
from meshcore_gui.core.protocols import SharedDataWriter
from meshcore_gui.ble.commands import CommandHandler
from meshcore_gui.ble.events import EventHandler
from meshcore_gui.ble.packet_decoder import PacketDecoder
from meshcore_gui.services.bot import BotConfig, MeshBot
from meshcore_gui.services.cache import DeviceCache
from meshcore_gui.services.dedup import DualDeduplicator
from meshcore_gui.services.device_identity import write_device_identity
# Seconds between background retry attempts for missing channel keys.
KEY_RETRY_INTERVAL: float = 30.0
# Seconds between periodic cleanup of old archived data (24 hours).
CLEANUP_INTERVAL: float = 86400.0
# ======================================================================
# Factory
# ======================================================================
def create_worker(device_id: str, shared: SharedDataWriter, **kwargs):
"""Return the appropriate worker for *device_id*.
Keyword arguments are forwarded to the worker constructor
(e.g. ``baudrate``, ``cx_dly`` for serial).
"""
from meshcore_gui.config import is_ble_address
if is_ble_address(device_id):
return BLEWorker(device_id, shared)
return SerialWorker(
device_id,
shared,
baudrate=kwargs.get("baudrate", _config.SERIAL_BAUDRATE),
cx_dly=kwargs.get("cx_dly", _config.SERIAL_CX_DELAY),
)
# ======================================================================
# Base worker (shared by BLE and Serial)
# ======================================================================
class _BaseWorker(abc.ABC):
"""Abstract base for transport-specific workers.
Subclasses must implement:
- :pyattr:`_log_prefix` — ``"BLE"`` or ``"SERIAL"``
- :meth:`_async_main` — transport-specific startup + main loop
- :meth:`_connect` — create the :class:`MeshCore` connection
- :meth:`_reconnect` — re-establish after a disconnect
- :pyattr:`_disconnect_keywords` — error substrings that signal
a broken connection
"""
def __init__(self, device_id: str, shared: SharedDataWriter) -> None:
self.device_id = device_id
self.shared = shared
self.mc: Optional[MeshCore] = None
self.running = True
self._disconnected = False
# Local cache (one file per device)
self._cache = DeviceCache(device_id)
# Collaborators (created eagerly, wired after connection)
self._decoder = PacketDecoder()
self._dedup = DualDeduplicator(max_size=200)
self._bot = MeshBot(
config=BotConfig(),
command_sink=shared.put_command,
enabled_check=shared.is_bot_enabled,
)
# Channel indices that still need keys from device
self._pending_keys: Set[int] = set()
# Dynamically discovered channels from device
self._channels: List[Dict] = []
# ── abstract properties / methods ─────────────────────────────
@property
@abc.abstractmethod
def _log_prefix(self) -> str:
"""Short label for log messages, e.g. ``"BLE"`` or ``"SERIAL"``."""
@property
@abc.abstractmethod
def _disconnect_keywords(self) -> tuple:
"""Lowercase substrings that indicate a transport disconnect."""
@abc.abstractmethod
async def _async_main(self) -> None:
"""Transport-specific startup + main loop."""
@abc.abstractmethod
async def _connect(self) -> None:
"""Create a fresh connection and wire collaborators."""
@abc.abstractmethod
async def _reconnect(self) -> Optional[MeshCore]:
"""Attempt to re-establish the connection after a disconnect."""
# ── thread lifecycle ──────────────────────────────────────────
def start(self) -> None:
"""Start the worker in a new daemon thread."""
thread = threading.Thread(target=self._run, daemon=True)
thread.start()
debug_print(f"{self._log_prefix} worker thread started")
def _run(self) -> None:
asyncio.run(self._async_main())
# ── shared main loop (called from subclass _async_main) ───────
async def _main_loop(self) -> None:
"""Command processing + periodic tasks.
Runs until ``self.running`` is cleared or a disconnect is
detected. Subclasses call this from their ``_async_main``.
"""
last_contact_refresh = time.time()
last_key_retry = time.time()
last_cleanup = time.time()
while self.running and not self._disconnected:
try:
await self._cmd_handler.process_all()
except Exception as e:
error_str = str(e).lower()
if any(kw in error_str for kw in self._disconnect_keywords):
print(f"{self._log_prefix}: ⚠️ Connection error detected: {e}")
self._disconnected = True
break
debug_print(f"Command processing error: {e}")
now = time.time()
if now - last_contact_refresh > CONTACT_REFRESH_SECONDS:
await self._refresh_contacts()
last_contact_refresh = now
if self._pending_keys and now - last_key_retry > KEY_RETRY_INTERVAL:
await self._retry_missing_keys()
last_key_retry = now
if now - last_cleanup > CLEANUP_INTERVAL:
await self._cleanup_old_data()
last_cleanup = now
await asyncio.sleep(0.1)
async def _handle_reconnect(self) -> bool:
"""Shared reconnect logic after a disconnect.
Returns True if reconnection succeeded, False otherwise.
"""
self.shared.set_connected(False)
self.shared.set_status("🔄 Verbinding verloren — herverbinden...")
print(f"{self._log_prefix}: Verbinding verloren, start reconnect...")
self.mc = None
new_mc = await self._reconnect()
if new_mc:
self.mc = new_mc
await asyncio.sleep(1)
self._wire_collaborators()
await self._load_data()
await self.mc.start_auto_message_fetching()
self._seed_dedup_from_messages()
self.shared.set_connected(True)
self.shared.set_status("✅ Herverbonden")
print(f"{self._log_prefix}: ✅ Herverbonden en operationeel")
return True
self.shared.set_status("❌ Herverbinding mislukt — herstart nodig")
print(
f"{self._log_prefix}: ❌ Kan niet herverbinden — "
"wacht 60s en probeer opnieuw..."
)
return False
# ── collaborator wiring ───────────────────────────────────────
def _wire_collaborators(self) -> None:
"""(Re-)create handlers and subscribe to MeshCore events."""
self._evt_handler = EventHandler(
shared=self.shared,
decoder=self._decoder,
dedup=self._dedup,
bot=self._bot,
)
self._cmd_handler = CommandHandler(
mc=self.mc, shared=self.shared, cache=self._cache,
)
self._cmd_handler.set_load_data_callback(self._load_data)
self.mc.subscribe(EventType.CHANNEL_MSG_RECV, self._evt_handler.on_channel_msg)
self.mc.subscribe(EventType.CONTACT_MSG_RECV, self._evt_handler.on_contact_msg)
self.mc.subscribe(EventType.RX_LOG_DATA, self._evt_handler.on_rx_log)
self.mc.subscribe(EventType.LOGIN_SUCCESS, self._on_login_success)
# ── LOGIN_SUCCESS handler (Room Server) ───────────────────────
def _on_login_success(self, event) -> None:
payload = event.payload or {}
pubkey = payload.get("pubkey_prefix", "")
is_admin = payload.get("is_admin", False)
debug_print(f"LOGIN_SUCCESS received: pubkey={pubkey}, admin={is_admin}")
self.shared.set_status("✅ Room login OK — messages arriving over RF…")
# ── apply cache ───────────────────────────────────────────────
def _apply_cache(self) -> None:
"""Push cached data to SharedData so GUI renders immediately."""
device = self._cache.get_device()
if device:
self.shared.update_from_appstart(device)
fw = device.get("firmware_version") or device.get("ver")
if fw:
self.shared.update_from_device_query({"ver": fw})
self.shared.set_status("📦 Loaded from cache")
debug_print(f"Cache → device info: {device.get('name', '?')}")
if CHANNEL_CACHE_ENABLED:
channels = self._cache.get_channels()
if channels:
self._channels = channels
self.shared.set_channels(channels)
debug_print(f"Cache → channels: {[c['name'] for c in channels]}")
else:
debug_print("Channel cache disabled — skipping cached channels")
contacts = self._cache.get_contacts()
if contacts:
self.shared.set_contacts(contacts)
debug_print(f"Cache → contacts: {len(contacts)}")
cached_keys = self._cache.get_channel_keys()
for idx_str, secret_hex in cached_keys.items():
try:
idx = int(idx_str)
secret_bytes = bytes.fromhex(secret_hex)
if len(secret_bytes) >= 16:
self._decoder.add_channel_key(idx, secret_bytes[:16], source="cache")
debug_print(f"Cache → channel key [{idx}]")
except (ValueError, TypeError) as exc:
debug_print(f"Cache → bad channel key [{idx_str}]: {exc}")
cached_orig_name = self._cache.get_original_device_name()
if cached_orig_name:
self.shared.set_original_device_name(cached_orig_name)
debug_print(f"Cache → original device name: {cached_orig_name}")
count = self.shared.load_recent_from_archive(limit=100)
if count:
debug_print(f"Cache → {count} recent messages from archive")
self._seed_dedup_from_messages()
# ── initial data loading ──────────────────────────────────────
async def _export_device_identity(self) -> None:
"""Export device keys and write identity file for Observer.
Calls ``export_private_key()`` on the device and writes the
result to ``~/.meshcore-gui/device_identity.json`` so the
MeshCore Observer can authenticate to the MQTT broker without
manual key configuration.
"""
pfx = self._log_prefix
try:
r = await self.mc.commands.export_private_key()
if r is None:
debug_print(f"{pfx}: export_private_key returned None")
return
if r.type == EventType.PRIVATE_KEY:
prv_bytes = r.payload.get("private_key", b"")
if len(prv_bytes) == 64:
# Gather device info for the identity file
pub_key = ""
dev_name = ""
fw_ver = ""
with self.shared.lock:
pub_key = self.shared.device.public_key
dev_name = self.shared.device.name
fw_ver = self.shared.device.firmware_version
write_device_identity(
public_key=pub_key,
private_key_bytes=prv_bytes,
device_name=dev_name,
firmware_version=fw_ver,
source_device=self.device_id,
)
else:
debug_print(
f"{pfx}: export_private_key: unexpected "
f"length {len(prv_bytes)} bytes"
)
elif r.type == EventType.DISABLED:
print(
f"{pfx}: ℹ️ Private key export is disabled on device "
f"— manual key setup required for Observer MQTT"
)
else:
debug_print(
f"{pfx}: export_private_key: unexpected "
f"response type {r.type}"
)
except Exception as exc:
debug_print(f"{pfx}: export_private_key failed: {exc}")
async def _load_data(self) -> None:
"""Load device info, channels and contacts from device."""
pfx = self._log_prefix
# send_appstart — reuse result from MeshCore.connect()
self.shared.set_status("🔄 Device info...")
cached_info = self.mc.self_info
if cached_info and cached_info.get("name"):
print(f"{pfx}: send_appstart OK (from connect): {cached_info.get('name')}")
self.shared.update_from_appstart(cached_info)
self._cache.set_device(cached_info)
else:
debug_print("self_info empty after connect(), falling back to manual send_appstart")
appstart_ok = False
for i in range(3):
debug_print(f"send_appstart fallback attempt {i + 1}/3")
try:
r = await self.mc.commands.send_appstart()
if r is None:
debug_print(f"send_appstart fallback {i + 1}: received None, retrying")
await asyncio.sleep(2.0)
continue
if r.type != EventType.ERROR:
print(f"{pfx}: send_appstart OK: {r.payload.get('name')} (fallback attempt {i + 1})")
self.shared.update_from_appstart(r.payload)
self._cache.set_device(r.payload)
appstart_ok = True
break
else:
debug_print(f"send_appstart fallback {i + 1}: ERROR — payload={pp(r.payload)}")
except Exception as exc:
debug_print(f"send_appstart fallback {i + 1} exception: {exc}")
await asyncio.sleep(2.0)
if not appstart_ok:
print(f"{pfx}: ⚠️ send_appstart failed after 3 fallback attempts")
# send_device_query
for i in range(5):
debug_print(f"send_device_query attempt {i + 1}/5")
try:
r = await self.mc.commands.send_device_query()
if r is None:
debug_print(f"send_device_query attempt {i + 1}: received None response, retrying")
await asyncio.sleep(2.0)
continue
if r.type != EventType.ERROR:
fw = r.payload.get("ver", "")
print(f"{pfx}: send_device_query OK: {fw} (attempt {i + 1})")
self.shared.update_from_device_query(r.payload)
if fw:
self._cache.set_firmware_version(fw)
break
else:
debug_print(f"send_device_query attempt {i + 1}: ERROR response — payload={pp(r.payload)}")
except Exception as exc:
debug_print(f"send_device_query attempt {i + 1} exception: {exc}")
await asyncio.sleep(2.0)
# Export device identity for MeshCore Observer
await self._export_device_identity()
# Channels
await self._discover_channels()
# Contacts
self.shared.set_status("🔄 Contacts...")
debug_print("get_contacts starting")
try:
r = await self._get_contacts_with_timeout()
debug_print(f"get_contacts result: type={r.type if r else None}")
if r and r.payload:
try:
payload_len = len(r.payload)
except Exception:
payload_len = None
if payload_len is not None and payload_len > 10:
debug_print(f"get_contacts payload size={payload_len} (omitted)")
else:
debug_data("get_contacts payload", r.payload)
if r is None:
debug_print(f"{pfx}: get_contacts returned None, keeping cached contacts")
elif r.type != EventType.ERROR:
merged = self._cache.merge_contacts(r.payload)
self.shared.set_contacts(merged)
print(f"{pfx}: Contacts — {len(r.payload)} from device, {len(merged)} total (with cache)")
else:
debug_print(f"{pfx}: get_contacts failed — payload={pp(r.payload)}, keeping cached contacts")
except Exception as exc:
debug_print(f"{pfx}: get_contacts exception: {exc}")
async def _get_contacts_with_timeout(self):
"""Fetch contacts with a bounded timeout to avoid hanging refresh."""
timeout = max(DEFAULT_TIMEOUT * 2, 10.0)
try:
return await asyncio.wait_for(
self.mc.commands.get_contacts(), timeout=timeout,
)
except asyncio.TimeoutError:
self.shared.set_status("⚠️ Contacts timeout — using cached contacts")
debug_print(f"get_contacts timeout after {timeout:.0f}s")
return None
# ── channel discovery ─────────────────────────────────────────
async def _discover_channels(self) -> None:
"""Discover channels and load their keys from the device."""
pfx = self._log_prefix
self.shared.set_status("🔄 Discovering channels...")
discovered: List[Dict] = []
cached_keys = self._cache.get_channel_keys()
confirmed: list[str] = []
from_cache: list[str] = []
derived: list[str] = []
consecutive_errors = 0
for idx in range(MAX_CHANNELS):
payload = await self._try_get_channel_info(idx, max_attempts=2, delay=1.0)
if payload is None:
consecutive_errors += 1
if consecutive_errors >= 3:
debug_print(
f"Channel discovery: {consecutive_errors} consecutive "
f"empty slots at idx {idx}, stopping"
)
break
continue
consecutive_errors = 0
name = payload.get("name") or payload.get("channel_name") or ""
if not name.strip():
debug_print(f"Channel [{idx}]: response OK but no name — skipping (undefined slot)")
continue
discovered.append({"idx": idx, "name": name})
secret = payload.get("channel_secret")
secret_bytes = self._extract_secret(secret)
if secret_bytes:
self._decoder.add_channel_key(idx, secret_bytes, source="device")
self._cache.set_channel_key(idx, secret_bytes.hex())
self._pending_keys.discard(idx)
confirmed.append(f"[{idx}] {name}")
elif str(idx) in cached_keys:
from_cache.append(f"[{idx}] {name}")
print(f"{pfx}: 📦 Channel [{idx}] '{name}' — using cached key")
else:
self._decoder.add_channel_key_from_name(idx, name)
self._pending_keys.add(idx)
derived.append(f"[{idx}] {name}")
print(f"{pfx}: ⚠️ Channel [{idx}] '{name}' — name-derived key (will retry)")
await asyncio.sleep(0.3)
if not discovered:
discovered = [{"idx": 0, "name": "Public"}]
print(f"{pfx}: ⚠️ No channels discovered, using default Public channel")
self._channels = discovered
self.shared.set_channels(discovered)
if CHANNEL_CACHE_ENABLED:
self._cache.set_channels(discovered)
debug_print("Channel list cached to disk")
print(f"{pfx}: Channels discovered: {[c['name'] for c in discovered]}")
print(f"{pfx}: PacketDecoder ready — has_keys={self._decoder.has_keys}")
if confirmed:
print(f"{pfx}: ✅ Keys from device: {', '.join(confirmed)}")
if from_cache:
print(f"{pfx}: 📦 Keys from cache: {', '.join(from_cache)}")
if derived:
print(f"{pfx}: ⚠️ Name-derived keys: {', '.join(derived)}")
async def _try_get_channel_info(
self, idx: int, max_attempts: int, delay: float,
) -> Optional[Dict]:
for attempt in range(max_attempts):
try:
r = await self.mc.commands.get_channel(idx)
if r is None:
debug_print(f"get_channel({idx}) attempt {attempt + 1}/{max_attempts}: received None response, retrying")
await asyncio.sleep(delay)
continue
if r.type == EventType.ERROR:
debug_print(f"get_channel({idx}) attempt {attempt + 1}/{max_attempts}: ERROR response — payload={pp(r.payload)}")
await asyncio.sleep(delay)
continue
debug_print(f"get_channel({idx}) attempt {attempt + 1}/{max_attempts}: OK — keys={list(r.payload.keys())}")
return r.payload
except Exception as exc:
debug_print(f"get_channel({idx}) attempt {attempt + 1}/{max_attempts} error: {exc}")
await asyncio.sleep(delay)
return None
async def _try_load_channel_key(
self, idx: int, name: str, max_attempts: int, delay: float,
) -> bool:
payload = await self._try_get_channel_info(idx, max_attempts, delay)
if payload is None:
return False
secret = payload.get("channel_secret")
secret_bytes = self._extract_secret(secret)
if secret_bytes:
self._decoder.add_channel_key(idx, secret_bytes, source="device")
self._cache.set_channel_key(idx, secret_bytes.hex())
print(f"{self._log_prefix}: ✅ Channel [{idx}] '{name}' — key from device (background retry)")
self._pending_keys.discard(idx)
return True
debug_print(f"get_channel({idx}): response OK but secret unusable")
return False
async def _retry_missing_keys(self) -> None:
if not self._pending_keys:
return
pending_copy = set(self._pending_keys)
ch_map = {ch["idx"]: ch["name"] for ch in self._channels}
debug_print(f"Background key retry: trying {len(pending_copy)} channels")
for idx in pending_copy:
name = ch_map.get(idx, f"ch{idx}")
loaded = await self._try_load_channel_key(idx, name, max_attempts=1, delay=0.5)
if loaded:
self._pending_keys.discard(idx)
await asyncio.sleep(1.0)
if not self._pending_keys:
print(f"{self._log_prefix}: ✅ All channel keys now loaded!")
else:
remaining = [f"[{idx}] {ch_map.get(idx, '?')}" for idx in sorted(self._pending_keys)]
debug_print(f"Background retry: still pending: {', '.join(remaining)}")
# ── helpers ────────────────────────────────────────────────────
def _seed_dedup_from_messages(self) -> None:
"""Seed the deduplicator with messages already in SharedData."""
snapshot = self.shared.get_snapshot()
messages = snapshot.get("messages", [])
seeded = 0
for msg in messages:
if msg.message_hash:
self._dedup.mark_hash(msg.message_hash)
seeded += 1
if msg.sender and msg.text:
self._dedup.mark_content(msg.sender, msg.channel, msg.text)
seeded += 1
debug_print(f"Dedup seeded with {seeded} entries from {len(messages)} messages")
@staticmethod
def _extract_secret(secret) -> Optional[bytes]:
if secret and isinstance(secret, bytes) and len(secret) >= 16:
return secret[:16]
if secret and isinstance(secret, str) and len(secret) >= 32:
try:
raw = bytes.fromhex(secret)
if len(raw) >= 16:
return raw[:16]
except ValueError:
pass
return None
# ── periodic tasks ────────────────────────────────────────────
async def _refresh_contacts(self) -> None:
try:
r = await self._get_contacts_with_timeout()
if r is None:
debug_print("Periodic refresh: get_contacts returned None, skipping")
return
if r.type != EventType.ERROR:
merged = self._cache.merge_contacts(r.payload)
self.shared.set_contacts(merged)
debug_print(
f"Periodic refresh: {len(r.payload)} from device, "
f"{len(merged)} total"
)
except Exception as exc:
debug_print(f"Periodic contact refresh failed: {exc}")
async def _cleanup_old_data(self) -> None:
try:
if self.shared.archive:
self.shared.archive.cleanup_old_data()
stats = self.shared.archive.get_stats()
debug_print(
f"Cleanup: archive now has {stats['total_messages']} messages, "
f"{stats['total_rxlog']} rxlog entries"
)
removed = self._cache.prune_old_contacts()
if removed > 0:
contacts = self._cache.get_contacts()
self.shared.set_contacts(contacts)
debug_print(f"Cleanup: pruned {removed} old contacts")
except Exception as exc:
debug_print(f"Periodic cleanup failed: {exc}")
# ======================================================================
# Serial worker
# ======================================================================
class SerialWorker(_BaseWorker):
"""Serial communication worker (USB/UART).
Args:
port: Serial device path (e.g. ``"/dev/ttyUSB0"``).
shared: SharedDataWriter for thread-safe communication.
baudrate: Serial baudrate (default from config).
cx_dly: Connection delay for meshcore serial transport.
"""
def __init__(
self,
port: str,
shared: SharedDataWriter,
baudrate: int = _config.SERIAL_BAUDRATE,
cx_dly: float = _config.SERIAL_CX_DELAY,
) -> None:
super().__init__(port, shared)
self.port = port
self.baudrate = baudrate
self.cx_dly = cx_dly
@property
def _log_prefix(self) -> str:
return "SERIAL"
@property
def _disconnect_keywords(self) -> tuple:
return (
"not connected", "disconnected", "connection reset",
"broken pipe", "i/o error", "read failed", "write failed",
"port is closed", "port closed",
)
async def _async_main(self) -> None:
try:
while self.running:
# ── Outer loop: (re)establish a fresh serial connection ──
self._disconnected = False
await self._connect()
if not self.mc:
print("SERIAL: Initial connection failed, retrying in 30s...")
self.shared.set_status("⚠️ Connection failed — retrying...")
await asyncio.sleep(30)
continue
# ── Inner loop: run + reconnect without calling _connect() again ──
# _handle_reconnect() already creates a fresh MeshCore and loads
# data — calling _connect() on top of that would attempt to open
# the serial port a second time, causing an immediate disconnect.
while self.running:
await self._main_loop()
if not self._disconnected or not self.running:
break
ok = await self._handle_reconnect()
if ok:
# Reconnected — reset flag and go back to _main_loop,
# NOT to the outer while (which would call _connect() again).
self._disconnected = False
else:
# All reconnect attempts exhausted — wait, then let the
# outer loop call _connect() for a clean fresh start.
await asyncio.sleep(60)
break
finally:
return
async def _connect(self) -> None:
if self._cache.load():
self._apply_cache()
print("SERIAL: Cache loaded — GUI populated from disk")
else:
print("SERIAL: No cache found — waiting for device data")
self.shared.set_status(f"🔄 Connecting to {self.port}...")
try:
print(f"SERIAL: Connecting to {self.port}...")
self.mc = await MeshCore.create_serial(
self.port,
baudrate=self.baudrate,
auto_reconnect=False,
default_timeout=DEFAULT_TIMEOUT,
debug=_config.MESHCORE_LIB_DEBUG,
cx_dly=self.cx_dly,
)
if self.mc is None:
raise RuntimeError("No response from device over serial")
print("SERIAL: Connected!")
await asyncio.sleep(1)
debug_print("Post-connection sleep done, wiring collaborators")
self._wire_collaborators()
await self._load_data()
await self.mc.start_auto_message_fetching()
self.shared.set_connected(True)
self.shared.set_status("✅ Connected")
print("SERIAL: Ready!")
if self._pending_keys:
pending_names = [
f"[{ch['idx']}] {ch['name']}"
for ch in self._channels
if ch["idx"] in self._pending_keys
]
print(
f"SERIAL: ⏳ Background retry active for: "
f"{', '.join(pending_names)} (every {KEY_RETRY_INTERVAL:.0f}s)"
)
except Exception as e:
print(f"SERIAL: Connection error: {e}")
self.mc = None # ensure _async_main sees connection as failed
if self._cache.has_cache:
self.shared.set_status(f"⚠️ Offline — using cached data ({e})")
else:
self.shared.set_status(f"❌ {e}")
async def _reconnect(self) -> Optional[MeshCore]:
for attempt in range(1, RECONNECT_MAX_RETRIES + 1):
delay = RECONNECT_BASE_DELAY * attempt
print(
f"SERIAL: 🔄 Reconnect attempt {attempt}/{RECONNECT_MAX_RETRIES} "
f"in {delay:.0f}s..."
)
await asyncio.sleep(delay)
try:
mc = await MeshCore.create_serial(
self.port,
baudrate=self.baudrate,
auto_reconnect=False,
default_timeout=DEFAULT_TIMEOUT,
debug=_config.MESHCORE_LIB_DEBUG,
cx_dly=self.cx_dly,
)
if mc is None:
raise RuntimeError("No response from device over serial")
return mc
except Exception as exc:
print(f"SERIAL: ❌ Reconnect attempt {attempt} failed: {exc}")
print(f"SERIAL: ❌ Reconnect failed after {RECONNECT_MAX_RETRIES} attempts")
return None
# ======================================================================
# BLE worker
# ======================================================================
class BLEWorker(_BaseWorker):
"""BLE communication worker (Bluetooth Low Energy).
Args:
address: BLE MAC address (e.g. ``"literal:AA:BB:CC:DD:EE:FF"``).
shared: SharedDataWriter for thread-safe communication.
"""
def __init__(self, address: str, shared: SharedDataWriter) -> None:
super().__init__(address, shared)
self.address = address
# BLE PIN agent — imported lazily so serial-only installs
# don't need dbus_fast / bleak.
from meshcore_gui.ble.ble_agent import BleAgentManager
self._agent = BleAgentManager(pin=_config.BLE_PIN)
@property
def _log_prefix(self) -> str:
return "BLE"
@property
def _disconnect_keywords(self) -> tuple:
return (
"not connected", "disconnected", "dbus",
"pin or key missing", "connection reset", "broken pipe",
"failed to discover", "service discovery",
)
async def _async_main(self) -> None:
from meshcore_gui.ble.ble_reconnect import remove_bond
# Step 1: Start PIN agent BEFORE any BLE connection
await self._agent.start()
# Step 2: Remove stale bond (clean slate)
await remove_bond(self.address)
await asyncio.sleep(1)
# Step 3: Connect + main loop
try:
while self.running:
# ── Outer loop: (re)establish a fresh BLE connection ──
self._disconnected = False
await self._connect()
if not self.mc:
print("BLE: Initial connection failed, retrying in 30s...")
self.shared.set_status("⚠️ Connection failed — retrying...")
await asyncio.sleep(30)
await remove_bond(self.address)
await asyncio.sleep(1)
continue
# ── Inner loop: run + reconnect without calling _connect() again ──
# _handle_reconnect() already creates a fresh MeshCore and loads
# data — calling _connect() on top would open a second BLE session,
# causing an immediate disconnect.
while self.running:
await self._main_loop()
if not self._disconnected or not self.running:
break
ok = await self._handle_reconnect()
if ok:
# Reconnected — reset flag and go back to _main_loop,
# NOT to the outer while (which would call _connect() again).
self._disconnected = False
else:
await asyncio.sleep(60)
await remove_bond(self.address)
await asyncio.sleep(1)
break
finally:
await self._agent.stop()
async def _connect(self) -> None:
if self._cache.load():
self._apply_cache()
print("BLE: Cache loaded — GUI populated from disk")
else:
print("BLE: No cache found — waiting for BLE data")
self.shared.set_status(f"🔄 Connecting to {self.address}...")
try:
print(f"BLE: Connecting to {self.address}...")
self.mc = await MeshCore.create_ble(
self.address,
auto_reconnect=False,
default_timeout=DEFAULT_TIMEOUT,
debug=_config.MESHCORE_LIB_DEBUG,
)
print("BLE: Connected!")
await asyncio.sleep(1)
debug_print("Post-connection sleep done, wiring collaborators")
self._wire_collaborators()
await self._load_data()
await self.mc.start_auto_message_fetching()
self.shared.set_connected(True)
self.shared.set_status("✅ Connected")
print("BLE: Ready!")
if self._pending_keys:
pending_names = [
f"[{ch['idx']}] {ch['name']}"
for ch in self._channels
if ch["idx"] in self._pending_keys
]
print(
f"BLE: ⏳ Background retry active for: "
f"{', '.join(pending_names)} (every {KEY_RETRY_INTERVAL:.0f}s)"
)
except Exception as e:
print(f"BLE: Connection error: {e}")
self.mc = None # ensure _async_main sees connection as failed
if self._cache.has_cache:
self.shared.set_status(f"⚠️ Offline — using cached data ({e})")
else:
self.shared.set_status(f"❌ {e}")
async def _reconnect(self) -> Optional[MeshCore]:
from meshcore_gui.ble.ble_reconnect import reconnect_loop
async def _create_fresh_connection() -> MeshCore:
return await MeshCore.create_ble(
self.address,
auto_reconnect=False,
default_timeout=DEFAULT_TIMEOUT,
debug=_config.MESHCORE_LIB_DEBUG,
)
return await reconnect_loop(
_create_fresh_connection,
self.address,
max_retries=RECONNECT_MAX_RETRIES,
base_delay=RECONNECT_BASE_DELAY,
)

390
meshcore_gui/config.py Normal file
View File

@@ -0,0 +1,390 @@
"""
Application configuration for MeshCore GUI.
Contains only global runtime settings.
Bot configuration lives in :mod:`meshcore_gui.services.bot`.
UI display constants live in :mod:`meshcore_gui.gui.constants`.
The ``DEBUG`` flag defaults to False and can be activated at startup
with the ``--debug-on`` command-line option.
Debug output is written to both stdout and a rotating log file at
``~/.meshcore-gui/logs/meshcore_gui.log``.
"""
import json
import logging
import sys
from logging.handlers import RotatingFileHandler
from pathlib import Path
from typing import Any, Dict, List
# ==============================================================================
# VERSION
# ==============================================================================
VERSION: str = "1.12.0"
# ==============================================================================
# OPERATOR / LANDING PAGE
# ==============================================================================
# Operator callsign shown on the landing page SVG and drawer footer.
# Change this to your own callsign (e.g. "PE1HVH", "PE1HVH/MIT").
OPERATOR_CALLSIGN: str = "PE1HVH"
# Path to the landing page SVG file.
# The placeholder ``{callsign}`` inside the SVG is replaced at runtime
# with ``OPERATOR_CALLSIGN``.
#
# Default: the bundled DOMCA splash (static/landing_default.svg).
# To use a custom SVG, point this to your own file, e.g.:
# LANDING_SVG_PATH = DATA_DIR / "landing.svg"
LANDING_SVG_PATH: Path = Path(__file__).parent / "static" / "landing_default.svg"
# ==============================================================================
# MAP DEFAULTS
# ==============================================================================
# Default map centre used as the initial view *before* the device reports
# its own GPS position. Once the device advertises a valid adv_lat/adv_lon
# pair, every map will re-centre on the device's actual location.
#
# Change these values to match the location of your device / station.
# Current default: Zwolle, The Netherlands (52.5168, 6.0830).
DEFAULT_MAP_CENTER: tuple[float, float] = (52.5168, 6.0830)
# Default zoom level for all Leaflet maps (higher = more zoomed in).
DEFAULT_MAP_ZOOM: int = 9
# ==============================================================================
# DIRECTORY STRUCTURE
# ==============================================================================
# Base data directory — all persistent data lives under this root.
# Existing services (cache, pins, archive) each define their own
# sub-directory; this constant centralises the root for new consumers.
DATA_DIR: Path = Path.home() / ".meshcore-gui"
# Log directory for debug and error log files.
LOG_DIR: Path = DATA_DIR / "logs"
# Log file path (rotating: max 5 MB per file, 3 backups = 20 MB total).
LOG_FILE: Path = LOG_DIR / "meshcore_gui.log"
def set_log_file_for_device(device_id: str) -> None:
"""Set the log file name based on the device identifier.
Transforms ``F0:9E:9E:75:A3:01`` into
``~/.meshcore-gui/logs/F0_9E_9E_75_A3_01_meshcore_gui.log`` and
``/dev/ttyUSB0`` into ``~/.meshcore-gui/logs/_dev_ttyUSB0_meshcore_gui.log``.
Must be called **before** the first ``debug_print()`` call so the
lazy logger initialisation picks up the correct path.
"""
global LOG_FILE
safe_name = (
device_id
.replace("literal:", "")
.replace(":", "_")
.replace("/", "_")
)
LOG_FILE = LOG_DIR / f"{safe_name}_meshcore_gui.log"
# Maximum size per log file in bytes (5 MB).
LOG_MAX_BYTES: int = 5 * 1024 * 1024
# Number of rotated backup files to keep.
LOG_BACKUP_COUNT: int = 3
# ==============================================================================
# DEBUG
# ==============================================================================
DEBUG: bool = False
# Internal file logger — initialised lazily on first debug_print() call.
_file_logger: logging.Logger | None = None
def _init_file_logger() -> logging.Logger:
"""Create and configure the rotating file logger (called once)."""
LOG_DIR.mkdir(parents=True, exist_ok=True)
logger = logging.getLogger("meshcore_gui.debug")
logger.setLevel(logging.DEBUG)
logger.propagate = False
handler = RotatingFileHandler(
LOG_FILE,
maxBytes=LOG_MAX_BYTES,
backupCount=LOG_BACKUP_COUNT,
encoding="utf-8",
)
handler.setFormatter(
logging.Formatter("%(asctime)s %(message)s", datefmt="%Y-%m-%d %H:%M:%S")
)
logger.addHandler(handler)
return logger
def _caller_module() -> str:
"""Return a short module label for the calling code.
Walks two frames up (debug_print -> caller) and extracts the
module ``__name__``. The common ``meshcore_gui.`` prefix is
stripped for brevity, e.g. ``ble.worker`` instead of
``meshcore_gui.ble.worker``.
"""
frame = sys._getframe(2) # 0=_caller_module, 1=debug_print, 2=actual caller
module = frame.f_globals.get("__name__", "<unknown>")
if module.startswith("meshcore_gui."):
module = module[len("meshcore_gui."):]
return module
def _init_meshcore_logger() -> None:
"""Route meshcore library debug output to our rotating log file.
The meshcore library uses ``logging.getLogger("meshcore")`` throughout,
but never attaches a handler. Without this function all library-level
debug output (raw send/receive, event dispatching, command flow)
is silently dropped because Python's root logger only forwards
WARNING and above.
Call once at startup (or lazily from ``debug_print``) so that
``MESHCORE_LIB_DEBUG=True`` actually produces visible output.
"""
LOG_DIR.mkdir(parents=True, exist_ok=True)
mc_logger = logging.getLogger("meshcore")
# Guard against duplicate handlers on repeated calls
if any(isinstance(h, RotatingFileHandler) for h in mc_logger.handlers):
return
handler = RotatingFileHandler(
LOG_FILE,
maxBytes=LOG_MAX_BYTES,
backupCount=LOG_BACKUP_COUNT,
encoding="utf-8",
)
handler.setFormatter(
logging.Formatter(
"%(asctime)s LIB [%(name)s]: %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
)
)
mc_logger.addHandler(handler)
# Also add a stdout handler so library output appears in the console
stdout_handler = logging.StreamHandler(sys.stdout)
stdout_handler.setFormatter(
logging.Formatter(
"%(asctime)s LIB [%(name)s]: %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
)
)
mc_logger.addHandler(stdout_handler)
def debug_print(msg: str) -> None:
"""Print a debug message when ``DEBUG`` is enabled.
Output goes to both stdout and the rotating log file.
The calling module name is automatically included so that
exception context is immediately clear, e.g.::
DEBUG [ble.worker]: send_appstart attempt 3 exception: TimeoutError
"""
global _file_logger
if not DEBUG:
return
module = _caller_module()
formatted = f"DEBUG [{module}]: {msg}"
# stdout (existing behaviour, now with module tag)
print(formatted)
# Rotating log file
if _file_logger is None:
_file_logger = _init_file_logger()
# Also wire up the meshcore library logger so MESHCORE_LIB_DEBUG
# output actually appears in the same log file + stdout.
_init_meshcore_logger()
_file_logger.debug(formatted)
def pp(obj: Any, indent: int = 2) -> str:
"""Pretty-format a dict, list, or other object for debug output.
Use inside f-strings::
debug_print(f"payload={pp(r.payload)}")
Dicts/lists get indented JSON; everything else falls back to repr().
"""
if isinstance(obj, (dict, list)):
try:
return json.dumps(obj, indent=indent, default=str, ensure_ascii=False)
except (TypeError, ValueError):
return repr(obj)
return repr(obj)
def debug_data(label: str, obj: Any) -> None:
"""Print a labelled data structure with pretty indentation.
Combines a header line with pretty-printed data below it::
debug_data("get_contacts result", r.payload)
Output::
DEBUG [worker]: get_contacts result ↓
{
"name": "PE1HVH",
"contacts": 629,
...
}
"""
if not DEBUG:
return
formatted = pp(obj)
# Single-line values stay on the same line
if '\n' not in formatted:
debug_print(f"{label}: {formatted}")
else:
# Multi-line: indent each line for readability
indented = '\n'.join(f" {line}" for line in formatted.splitlines())
debug_print(f"{label} ↓\n{indented}")
# ==============================================================================
# CHANNELS
# ==============================================================================
# Maximum number of channel slots to probe on the device.
# MeshCore supports up to 8 channels (indices 0-7).
MAX_CHANNELS: int = 8
# Enable or disable caching of the channel list to disk.
# When False (default), channels are always fetched fresh from the
# device at startup, guaranteeing the GUI always reflects the actual
# device configuration. When True, channels are loaded from cache
# for instant GUI population and then refreshed from the device.
# Note: channel *keys* (for packet decryption) are always cached
# regardless of this setting.
CHANNEL_CACHE_ENABLED: bool = False
# ==============================================================================
# BOT DEVICE NAME
# ==============================================================================
# Fixed device name applied when the BOT checkbox is enabled.
# The original device name is saved and restored when BOT is disabled.
BOT_DEVICE_NAME: str = "NL-OV-ZWL-STDSHGN-WKC Bot"
# Default device name used as fallback when restoring from BOT mode
# and no original name was saved (e.g. after a restart).
DEVICE_NAME: str = "PE1HVH T1000e"
# ==============================================================================
# CACHE / REFRESH
# ==============================================================================
# Default timeout (seconds) for meshcore command responses.
# Increase if you see frequent 'no_event_received' errors during startup.
DEFAULT_TIMEOUT: float = 10.0
# Enable debug logging inside the meshcore library itself.
# When True, raw send/receive data and event parsing are logged.
MESHCORE_LIB_DEBUG: bool = True
# ==============================================================================
# TRANSPORT MODE (auto-detected from CLI argument)
# ==============================================================================
# "serial" or "ble" — set at startup by main() based on the device argument.
TRANSPORT: str = "serial"
def is_ble_address(device_id: str) -> bool:
"""Detect whether *device_id* looks like a BLE MAC address.
Heuristic:
- Starts with ``literal:`` → BLE
- Matches ``XX:XX:XX:XX:XX:XX`` (6 colon-separated hex pairs) → BLE
- Everything else (``/dev/…``, ``COM…``) → Serial
"""
if device_id.lower().startswith("literal:"):
return True
parts = device_id.split(":")
if len(parts) == 6 and all(len(p) == 2 for p in parts):
try:
for p in parts:
int(p, 16)
return True
except ValueError:
pass
return False
TRANSPORT: str = "serial"
# Serial connection defaults.
SERIAL_BAUDRATE: int = 115200
SERIAL_CX_DELAY: float = 0.1
# BLE connection defaults.
# BLE pairing PIN for the MeshCore device (T1000e default: 123456).
# Used by the built-in D-Bus agent to answer pairing requests
# automatically — eliminates the need for bt-agent.service.
BLE_PIN: str = "123456"
# Maximum number of reconnect attempts after a disconnect.
RECONNECT_MAX_RETRIES: int = 5
# Base delay in seconds between reconnect attempts (multiplied by
# attempt number for linear backoff: 5s, 10s, 15s, 20s, 25s).
RECONNECT_BASE_DELAY: float = 5.0
# Interval in seconds between periodic contact refreshes from the device.
# Contacts are merged (new/changed contacts update the cache; contacts
# only present in cache are kept so offline nodes are preserved).
CONTACT_REFRESH_SECONDS: float = 300.0 # 5 minutes
# ==============================================================================
# EXTERNAL LINKS (drawer menu)
# ==============================================================================
EXT_LINKS = [
('MeshCore', 'https://meshcore.co.uk'),
('Handleiding', 'https://www.pe1hvh.nl/pdf/MeshCore_Complete_Handleiding.pdf'),
('Netwerk kaart', 'https://meshcore.co.uk/map'),
('LocalMesh NL', 'https://www.localmesh.nl/'),
]
# ==============================================================================
# ARCHIVE / RETENTION
# ==============================================================================
# Retention period for archived messages (in days).
# Messages older than this are automatically removed during cleanup.
MESSAGE_RETENTION_DAYS: int = 30
# Retention period for RX log entries (in days).
# RX log entries older than this are automatically removed during cleanup.
RXLOG_RETENTION_DAYS: int = 7
# Retention period for contacts (in days).
# Contacts not seen for longer than this are removed from cache.
CONTACT_RETENTION_DAYS: int = 90

16
meshcore_gui/core/__init__.py Normal file
View File

@@ -0,0 +1,16 @@
"""
Core domain layer — models, protocols and shared data store.
Re-exports the most commonly used names so consumers can write::
from meshcore_gui.core import SharedData, Message, RxLogEntry
"""
from meshcore_gui.core.models import ( # noqa: F401
Contact,
DeviceInfo,
Message,
RouteNode,
RxLogEntry,
)
from meshcore_gui.core.shared_data import SharedData # noqa: F401

351
meshcore_gui/core/models.py Normal file
View File

@@ -0,0 +1,351 @@
"""
Domain model for MeshCore GUI.
Typed dataclasses that replace untyped Dict objects throughout the
codebase. Each class represents a core domain concept. All classes
are immutable-friendly (frozen is not used because SharedData mutates
collections, but fields are not reassigned after construction).
Migration note
~~~~~~~~~~~~~~
``SharedData.get_snapshot()`` still returns a plain dict for backward
compatibility with the NiceGUI timer loop. Inside that dict, however,
``messages`` and ``rx_log`` are now lists of dataclass instances.
UI code can access attributes directly (``msg.sender``) or fall back
to ``dataclasses.asdict(msg)`` if a plain dict is needed.
"""
from dataclasses import dataclass, field
from datetime import datetime
from typing import Dict, List, Optional
# ---------------------------------------------------------------------------
# Message
# ---------------------------------------------------------------------------
@dataclass
class Message:
"""A channel message or direct message (DM).
Attributes:
time: Formatted timestamp (HH:MM:SS).
sender: Display name of the sender.
text: Message body.
channel: Channel index, or ``None`` for a DM.
direction: ``'in'`` for received, ``'out'`` for sent.
snr: Signal-to-noise ratio (dB), if available.
path_len: Hop count from the LoRa frame header.
sender_pubkey: Full public key of the sender (hex string).
path_hashes: List of 2-char hex strings, one per repeater.
path_names: List of resolved display names for each path hash,
captured at receive time so the archive is self-contained.
message_hash: Deterministic packet identifier (hex string).
channel_name: Human-readable channel name (resolved at add time).
"""
time: str
sender: str
text: str
channel: Optional[int]
direction: str
snr: Optional[float] = None
path_len: int = 0
sender_pubkey: str = ""
path_hashes: List[str] = field(default_factory=list)
path_names: List[str] = field(default_factory=list)
message_hash: str = ""
channel_name: str = ""
@staticmethod
def from_dict(d: dict) -> "Message":
"""Create a Message from an archive dictionary.
Args:
d: Dictionary as stored by MessageArchive.
Returns:
Message dataclass instance.
"""
return Message(
time=d.get("time", ""),
sender=d.get("sender", ""),
text=d.get("text", ""),
channel=d.get("channel"),
direction=d.get("direction", "in"),
snr=d.get("snr"),
path_len=d.get("path_len", 0),
sender_pubkey=d.get("sender_pubkey", ""),
path_hashes=d.get("path_hashes", []),
path_names=d.get("path_names", []),
message_hash=d.get("message_hash", ""),
channel_name=d.get("channel_name", ""),
)
# -- Timestamp helper ------------------------------------------------
@staticmethod
def now_timestamp() -> str:
"""Current time formatted as ``HH:MM:SS``."""
return datetime.now().strftime('%H:%M:%S')
# -- Factory methods -------------------------------------------------
@classmethod
def incoming(
cls,
sender: str,
text: str,
channel: Optional[int],
*,
time: str = "",
snr: Optional[float] = None,
path_len: int = 0,
sender_pubkey: str = "",
path_hashes: Optional[List[str]] = None,
path_names: Optional[List[str]] = None,
message_hash: str = "",
) -> "Message":
"""Create an incoming message with auto-generated timestamp.
Args:
sender: Display name of the sender.
text: Message body.
channel: Channel index, or ``None`` for a DM.
time: Optional pre-generated timestamp (default: now).
snr: Signal-to-noise ratio (dB).
path_len: Hop count from the LoRa frame header.
sender_pubkey: Full public key of the sender (hex string).
path_hashes: List of 2-char hex strings per repeater.
path_names: Resolved display names for each path hash.
message_hash: Deterministic packet identifier (hex string).
"""
return cls(
time=time or cls.now_timestamp(),
sender=sender,
text=text,
channel=channel,
direction='in',
snr=snr,
path_len=path_len,
sender_pubkey=sender_pubkey,
path_hashes=path_hashes or [],
path_names=path_names or [],
message_hash=message_hash,
)
@classmethod
def outgoing(
cls,
text: str,
channel: Optional[int],
*,
sender_pubkey: str = "",
) -> "Message":
"""Create an outgoing message (sender ``'Me'``, auto-timestamp).
Args:
text: Message body.
channel: Channel index, or ``None`` for a DM.
sender_pubkey: Recipient public key (hex string).
"""
return cls(
time=cls.now_timestamp(),
sender='Me',
text=text,
channel=channel,
direction='out',
sender_pubkey=sender_pubkey,
)
# -- Display formatting ----------------------------------------------
def format_line(
self,
channel_names: Optional[Dict[int, str]] = None,
show_channel: bool = True,
sender_prefix: str = '',
) -> str:
"""Format as a single display line for the messages panel.
Produces the same output as the original ``messages_panel.py``
inline formatting, e.g.::
12:34:56 ← [Public] [2h✓] PE1ABC: Hello mesh!
When *show_channel* is ``False`` the ``[channel]`` / ``[DM]``
tag is omitted (useful when the panel header already indicates
the active channel).
Args:
channel_names: Optional ``{channel_idx: name}`` lookup.
Falls back to ``self.channel_name``, then ``'ch<idx>'``.
show_channel: Include ``[channel]`` / ``[DM]`` prefix.
Defaults to ``True`` for backward compatibility.
sender_prefix: Optional prefix placed before the sender name,
e.g. a node-type icon from the map/contact view.
Returns:
Formatted single-line string.
"""
direction = '→' if self.direction == 'out' else '←'
ch_label = ''
if show_channel:
if self.channel is not None:
if channel_names and self.channel in channel_names:
ch_name = channel_names[self.channel]
elif self.channel_name:
ch_name = self.channel_name
else:
ch_name = f'ch{self.channel}'
ch_label = f'[{ch_name}] '
else:
ch_label = '[DM] '
if self.direction == 'in' and self.path_len > 0:
hop_tag = f'[{self.path_len}h{"✓" if self.path_hashes else ""}] '
else:
hop_tag = ''
sender_display = f"{sender_prefix}{self.sender}" if self.sender else ''
if self.sender:
return f"{self.time} {direction} {ch_label}{hop_tag}{sender_display}: {self.text}"
return f"{self.time} {direction} {ch_label}{hop_tag}{self.text}"
# ---------------------------------------------------------------------------
# Contact
# ---------------------------------------------------------------------------
@dataclass
class Contact:
"""A known mesh network node.
Attributes:
pubkey: Full public key (hex string).
adv_name: Advertised display name.
type: Node type (0=unknown, 1=CLI, 2=REP, 3=ROOM).
adv_lat: Advertised latitude (0.0 if unknown).
adv_lon: Advertised longitude (0.0 if unknown).
out_path: Hex string of stored route (2 hex chars per hop).
out_path_len: Number of hops in ``out_path``.
"""
pubkey: str
adv_name: str = ""
type: int = 0
adv_lat: float = 0.0
adv_lon: float = 0.0
out_path: str = ""
out_path_len: int = 0
@staticmethod
def from_dict(pubkey: str, d: dict) -> "Contact":
"""Create a Contact from a meshcore contacts dict entry."""
return Contact(
pubkey=pubkey,
adv_name=d.get("adv_name", ""),
type=d.get("type", 0),
adv_lat=d.get("adv_lat", 0.0),
adv_lon=d.get("adv_lon", 0.0),
out_path=d.get("out_path", ""),
out_path_len=d.get("out_path_len", 0),
)
# ---------------------------------------------------------------------------
# DeviceInfo
# ---------------------------------------------------------------------------
@dataclass
class DeviceInfo:
"""Radio device identification and configuration.
Attributes:
name: Device display name.
public_key: Device public key (hex string).
radio_freq: Radio frequency in MHz.
radio_sf: LoRa spreading factor.
radio_bw: Bandwidth in kHz.
tx_power: Transmit power in dBm.
adv_lat: Advertised latitude.
adv_lon: Advertised longitude.
firmware_version: Firmware version string.
"""
name: str = ""
public_key: str = ""
radio_freq: float = 0.0
radio_sf: int = 0
radio_bw: float = 0.0
tx_power: int = 0
adv_lat: float = 0.0
adv_lon: float = 0.0
firmware_version: str = ""
# ---------------------------------------------------------------------------
# RxLogEntry
# ---------------------------------------------------------------------------
@dataclass
class RxLogEntry:
"""A single RX log entry from the radio.
Attributes:
time: Formatted timestamp (HH:MM:SS).
snr: Signal-to-noise ratio (dB).
rssi: Received signal strength (dBm).
payload_type: Packet type identifier.
hops: Number of hops (path_len from frame header).
message_hash: Optional message hash for correlation with messages.
path_hashes: 2-char hex repeater hashes from decoded packet.
path_names: Resolved display names for each path hash.
"""
time: str
snr: float = 0.0
rssi: float = 0.0
payload_type: str = "?"
hops: int = 0
message_hash: str = ""
path_hashes: List[str] = field(default_factory=list)
path_names: List[str] = field(default_factory=list)
sender: str = ""
receiver: str = ""
# ── Fase 1 Observer fields (raw packet metadata) ──
raw_payload: str = "" # Raw hex packet data
packet_len: int = 0 # Total packet length (bytes)
payload_len: int = 0 # Payload length (bytes)
route_type: str = "" # "F" (flood) or "D" (direct)
packet_type_num: int = -1 # Numeric packet type (0-15)
# ---------------------------------------------------------------------------
# RouteNode
# ---------------------------------------------------------------------------
@dataclass
class RouteNode:
"""A node in a message route (sender, repeater or receiver).
Attributes:
name: Display name (or ``'-'`` if unknown).
lat: Latitude (0.0 if unknown).
lon: Longitude (0.0 if unknown).
type: Node type (0=unknown, 1=CLI, 2=REP, 3=ROOM).
pubkey: Public key or 2-char hash (hex string).
"""
name: str
lat: float = 0.0
lon: float = 0.0
type: int = 0
pubkey: str = ""
@property
def has_location(self) -> bool:
"""True if the node has GPS coordinates."""
return self.lat != 0 or self.lon != 0

115
meshcore_gui/core/protocols.py Normal file
View File

@@ -0,0 +1,115 @@
"""
Protocol interfaces for MeshCore GUI.
Defines the contracts between components using ``typing.Protocol``.
Each protocol captures the subset of SharedData that a specific
consumer needs, following the Interface Segregation Principle (ISP)
and the Dependency Inversion Principle (DIP).
Consumers depend on these protocols rather than on the concrete
SharedData class, which makes the contracts explicit and enables
testing with lightweight stubs.
v4.1 changes
~~~~~~~~~~~~~
- Added ``CommandSink`` protocol for bot and command dispatch.
- ``SharedDataWriter.add_message`` now accepts a ``Message`` dataclass.
- ``SharedDataWriter.add_rx_log`` now accepts an ``RxLogEntry`` dataclass.
"""
from typing import Dict, List, Optional, Protocol, runtime_checkable
from meshcore_gui.core.models import Message, RxLogEntry
# ----------------------------------------------------------------------
# CommandSink — used by MeshBot and GUI pages
# ----------------------------------------------------------------------
@runtime_checkable
class CommandSink(Protocol):
"""Enqueue commands for the worker."""
def put_command(self, cmd: Dict) -> None: ...
# ----------------------------------------------------------------------
# Writer — used by the worker
# ----------------------------------------------------------------------
@runtime_checkable
class SharedDataWriter(Protocol):
"""Write-side interface used by the worker.
The worker pushes data into the shared store: device info,
contacts, channels, messages, RX log entries and status updates.
It also reads commands enqueued by the GUI.
"""
def update_from_appstart(self, payload: Dict) -> None: ...
def update_from_device_query(self, payload: Dict) -> None: ...
def set_status(self, status: str) -> None: ...
def set_connected(self, connected: bool) -> None: ...
def set_contacts(self, contacts_dict: Dict) -> None: ...
def set_channels(self, channels: List[Dict]) -> None: ...
def add_message(self, msg: Message) -> None: ...
def add_rx_log(self, entry: RxLogEntry) -> None: ...
def get_next_command(self) -> Optional[Dict]: ...
def get_contact_name_by_prefix(self, pubkey_prefix: str) -> str: ...
def get_contact_by_name(self, name: str) -> Optional[tuple]: ...
def is_bot_enabled(self) -> bool: ...
def put_command(self, cmd: Dict) -> None: ...
def set_auto_add_enabled(self, enabled: bool) -> None: ...
def is_auto_add_enabled(self) -> bool: ...
def set_original_device_name(self, name: Optional[str]) -> None: ...
def get_original_device_name(self) -> Optional[str]: ...
def get_device_name(self) -> str: ...
def set_room_login_state(self, pubkey_prefix: str, state: str, detail: str = "") -> None: ...
def load_room_history(self, pubkey: str, limit: int = 50) -> None: ...
# ----------------------------------------------------------------------
# Reader — used by DashboardPage
# ----------------------------------------------------------------------
@runtime_checkable
class SharedDataReader(Protocol):
"""Read-side interface used by GUI pages.
GUI pages read snapshots of the shared data and manage
update flags. They also enqueue commands for the worker.
"""
def get_snapshot(self) -> Dict: ...
def clear_update_flags(self) -> None: ...
def mark_gui_initialized(self) -> None: ...
def put_command(self, cmd: Dict) -> None: ...
def set_bot_enabled(self, enabled: bool) -> None: ...
def set_auto_add_enabled(self, enabled: bool) -> None: ...
# ----------------------------------------------------------------------
# ContactLookup — used by RouteBuilder
# ----------------------------------------------------------------------
@runtime_checkable
class ContactLookup(Protocol):
"""Contact lookup interface used by RouteBuilder.
RouteBuilder needs to resolve public key prefixes and names
to contact records.
"""
def get_contact_by_prefix(self, pubkey_prefix: str) -> Optional[Dict]: ...
def get_contact_by_name(self, name: str) -> Optional[tuple]: ...
# ----------------------------------------------------------------------
# ReadAndLookup — used by RoutePage (needs both Reader + Lookup)
# ----------------------------------------------------------------------
@runtime_checkable
class SharedDataReadAndLookup(SharedDataReader, ContactLookup, Protocol):
"""Combined interface for RoutePage which reads snapshots and
delegates contact lookups to RouteBuilder."""
...

640
meshcore_gui/core/shared_data.py Normal file
View File

@@ -0,0 +1,640 @@
"""
Thread-safe shared data container for MeshCore GUI.
SharedData is the central data store shared between the worker thread
and the GUI main thread. All access goes through methods that acquire a
threading.Lock so both threads can safely read and write.
v4.1 changes
~~~~~~~~~~~~~
- ``messages`` is now ``List[Message]`` (was ``List[Dict]``).
- ``rx_log`` is now ``List[RxLogEntry]`` (was ``List[Dict]``).
- ``DeviceInfo`` dataclass replaces loose scalar fields.
- ``get_snapshot()`` returns typed objects; UI code accesses attributes
directly (``msg.sender``) instead of dict keys (``msg['sender']``).
"""
import queue
import threading
from dataclasses import asdict
from typing import Dict, List, Optional, Tuple
from meshcore_gui.config import debug_print
from meshcore_gui.core.models import DeviceInfo, Message, RxLogEntry
from meshcore_gui.services.message_archive import MessageArchive
class SharedData:
"""
Thread-safe container for shared data between worker and GUI.
Implements all four Protocol interfaces defined in ``protocols.py``.
"""
def __init__(self, device_id: Optional[str] = None) -> None:
self.lock = threading.Lock()
# Device info (typed)
self.device = DeviceInfo()
# Connection status
self.connected: bool = False
self.status: str = "Starting..."
# Data collections (typed)
self.contacts: Dict = {}
self.channels: List[Dict] = []
self.messages: List[Message] = []
self.rx_log: List[RxLogEntry] = []
# Dedup guard: fingerprints of messages already in self.messages.
# Acts as last-line-of-defence against duplicate inserts regardless
# of the source (archive reload, device event, reconnect).
self._message_fingerprints: set = set()
# Command queue (GUI → worker)
self.cmd_queue: queue.Queue = queue.Queue()
# Update flags — initially True so first GUI render shows data
self.device_updated: bool = True
self.contacts_updated: bool = True
self.channels_updated: bool = True
self.rxlog_updated: bool = True
self.messages_updated: bool = True
# Flag to track if GUI has done first render
self.gui_initialized: bool = False
# BOT enabled flag (toggled from GUI)
self.bot_enabled: bool = False
# Auto-add contacts flag (synced with device)
self.auto_add_enabled: bool = False
# Original device name (saved when BOT is enabled, restored when disabled)
self.original_device_name: Optional[str] = None
# Room Server login states: pubkey → {'state': 'ok'|'fail'|'pending'|'logged_out', 'detail': str}
self.room_login_states: Dict[str, Dict] = {}
# Room message cache: pubkey_prefix (12 hex) → List[Message]
# Populated from archive on first access per room, then kept in
# sync by add_message().
self._room_msg_cache: Dict[str, List[Message]] = {}
# Message archive (persistent storage)
self.archive: Optional[MessageArchive] = None
if device_id:
self.archive = MessageArchive(device_id)
debug_print(f"MessageArchive initialized for {device_id}")
# ------------------------------------------------------------------
# Device info updates
# ------------------------------------------------------------------
def update_from_appstart(self, payload: Dict) -> None:
"""Update device info from send_appstart response."""
with self.lock:
d = self.device
d.name = payload.get('name', d.name)
d.public_key = payload.get('public_key', d.public_key)
d.radio_freq = payload.get('radio_freq', d.radio_freq)
d.radio_sf = payload.get('radio_sf', d.radio_sf)
d.radio_bw = payload.get('radio_bw', d.radio_bw)
d.tx_power = payload.get('tx_power', d.tx_power)
d.adv_lat = payload.get('adv_lat', d.adv_lat)
d.adv_lon = payload.get('adv_lon', d.adv_lon)
self.device_updated = True
debug_print(f"Device info updated: {d.name}")
def update_from_device_query(self, payload: Dict) -> None:
"""Update firmware version from send_device_query response."""
with self.lock:
self.device.firmware_version = payload.get(
'ver', self.device.firmware_version,
)
self.device_updated = True
debug_print(f"Firmware version: {self.device.firmware_version}")
# ------------------------------------------------------------------
# Status
# ------------------------------------------------------------------
def set_status(self, status: str) -> None:
with self.lock:
self.status = status
def set_connected(self, connected: bool) -> None:
with self.lock:
self.connected = connected
# ------------------------------------------------------------------
# BOT
# ------------------------------------------------------------------
def set_bot_enabled(self, enabled: bool) -> None:
with self.lock:
self.bot_enabled = enabled
debug_print(f"BOT {'enabled' if enabled else 'disabled'}")
def is_bot_enabled(self) -> bool:
with self.lock:
return self.bot_enabled
# ------------------------------------------------------------------
# Auto-add contacts
# ------------------------------------------------------------------
def set_auto_add_enabled(self, enabled: bool) -> None:
"""Set auto-add contacts flag (thread-safe)."""
with self.lock:
self.auto_add_enabled = enabled
debug_print(f"Auto-add {'enabled' if enabled else 'disabled'}")
def is_auto_add_enabled(self) -> bool:
"""Get auto-add contacts flag (thread-safe)."""
with self.lock:
return self.auto_add_enabled
# ------------------------------------------------------------------
# Original device name (BOT feature)
# ------------------------------------------------------------------
def set_original_device_name(self, name: Optional[str]) -> None:
"""Store the original device name before BOT rename (thread-safe)."""
with self.lock:
self.original_device_name = name
debug_print(f"Original device name stored: {name}")
def get_original_device_name(self) -> Optional[str]:
"""Get the stored original device name (thread-safe)."""
with self.lock:
return self.original_device_name
def get_device_name(self) -> str:
"""Get the current device name (thread-safe)."""
with self.lock:
return self.device.name
# ------------------------------------------------------------------
# Room Server login state
# ------------------------------------------------------------------
def set_room_login_state(
self, pubkey_prefix: str, state: str, detail: str = "",
) -> None:
"""Update login state for a Room Server (thread-safe).
Cleans up any stale entries whose first 12 hex chars match the
new key. This prevents duplicate keys (e.g. a 12-char prefix
from the worker *and* a 64-char full pubkey from the command
handler) from coexisting and causing the UI to see stale state.
Args:
pubkey_prefix: Room server pubkey (full or prefix hex string).
state: One of 'pending', 'ok', 'fail', 'logged_out'.
detail: Human-readable detail string.
"""
with self.lock:
# Remove overlapping entries (different key length, same room)
norm = pubkey_prefix[:12]
stale = [
k for k in self.room_login_states
if k != pubkey_prefix and k[:12] == norm
]
for k in stale:
debug_print(
f"Room login state: removing stale key {k[:12]}…"
)
del self.room_login_states[k]
self.room_login_states[pubkey_prefix] = {
'state': state,
'detail': detail,
}
debug_print(
f"Room login state: {pubkey_prefix[:12]}… → {state}"
f"{(' (' + detail + ')') if detail else ''}"
)
def get_room_login_states(self) -> Dict[str, Dict]:
"""Return a copy of all room login states (thread-safe)."""
with self.lock:
return {k: v.copy() for k, v in self.room_login_states.items()}
# ------------------------------------------------------------------
# Room message cache (archive → UI)
# ------------------------------------------------------------------
def load_room_history(self, pubkey: str, limit: int = 50) -> None:
"""Load archived room messages into the in-memory cache.
Called by the command handler at room login and when a room
card is first created. Safe to call multiple times — subsequent
calls refresh the cache from the archive.
Args:
pubkey: Room server public key (full or prefix, ≥ 12 hex chars).
limit: Maximum number of archived messages to load.
"""
if not self.archive:
return
norm = pubkey[:12]
archived = self.archive.get_messages_by_sender_pubkey(norm, limit)
with self.lock:
messages = [Message.from_dict(d) for d in archived]
self._room_msg_cache[norm] = messages
debug_print(
f"Room history loaded: {norm}… → {len(messages)} messages"
)
def get_room_messages(self, pubkey: str) -> List[Message]:
"""Return cached room messages for a given room pubkey (thread-safe).
Args:
pubkey: Room server public key (full or prefix, ≥ 12 hex chars).
Returns:
List of Message objects (oldest first), or empty list.
"""
norm = pubkey[:12]
with self.lock:
return list(self._room_msg_cache.get(norm, []))
# ------------------------------------------------------------------
# Command queue
# ------------------------------------------------------------------
def put_command(self, cmd: Dict) -> None:
self.cmd_queue.put(cmd)
def get_next_command(self) -> Optional[Dict]:
try:
return self.cmd_queue.get_nowait()
except queue.Empty:
return None
# ------------------------------------------------------------------
# Collections
# ------------------------------------------------------------------
def set_contacts(self, contacts_dict: Dict) -> None:
with self.lock:
self.contacts = contacts_dict.copy()
self.contacts_updated = True
debug_print(f"Contacts updated: {len(self.contacts)} contacts")
def set_channels(self, channels: List[Dict]) -> None:
with self.lock:
self.channels = channels.copy()
self.channels_updated = True
debug_print(f"Channels updated: {[c['name'] for c in channels]}")
@staticmethod
def _message_fingerprint(msg: Message) -> str:
"""Build a dedup fingerprint for a message.
Uses message_hash when available (deterministic packet ID),
otherwise falls back to a composite of channel, sender and text.
Args:
msg: Message to fingerprint.
Returns:
String key suitable for set membership tests.
"""
if msg.message_hash:
return f"h:{msg.message_hash}"
return f"c:{msg.channel}:{msg.sender}:{msg.text}"
def add_message(self, msg: Message) -> None:
"""Add a Message to the store (max 100).
Skips the message if an identical fingerprint is already present,
preventing duplicates regardless of the insertion source (archive
reload, device event, reconnect).
Also resolves channel_name and path_names from the current
contacts/channels list if not already set, and appends to the
room-message cache if the sender matches a known room, keeping
archive and cache in sync.
"""
with self.lock:
# Dedup guard: skip if fingerprint already tracked
fps = {self._message_fingerprint(msg)}
# Also mark outgoing messages under device name to suppress echo
if msg.direction == 'out' and msg.sender == 'Me':
device_name = self.device.name
if device_name:
fps.add(f"c:{msg.channel}:{device_name}:{msg.text}")
if any(fp in self._message_fingerprints for fp in fps):
debug_print(
f"Message skipped (duplicate fingerprint): "
f"{msg.sender}: {msg.text[:30]}"
)
return
# Resolve channel_name if missing
if not msg.channel_name and msg.channel is not None:
msg.channel_name = self._resolve_channel_name(msg.channel)
# Resolve path_names if missing but path_hashes are present
if msg.path_hashes and not msg.path_names:
msg.path_names = self._resolve_path_names(msg.path_hashes)
self.messages.append(msg)
self.messages_updated = True
for fp in fps:
self._message_fingerprints.add(fp)
if len(self.messages) > 100:
removed = self.messages.pop(0)
# Evict fingerprint of removed message
removed_fps = {self._message_fingerprint(removed)}
if removed.direction == 'out' and removed.sender == 'Me':
device_name = self.device.name
if device_name:
removed_fps.add(
f"c:{removed.channel}:{device_name}:{removed.text}"
)
for fp in removed_fps:
self._message_fingerprints.discard(fp)
debug_print(
f"Message added: {msg.sender}: {msg.text[:30]}"
)
# Keep room message cache in sync
if msg.sender_pubkey:
norm = msg.sender_pubkey[:12]
if norm in self._room_msg_cache:
self._room_msg_cache[norm].append(msg)
# Archive message for persistent storage
if self.archive:
self.archive.add_message(msg)
def _resolve_channel_name(self, channel_idx: int) -> str:
"""Resolve a channel index to its display name.
MUST be called with self.lock held.
Args:
channel_idx: Numeric channel index.
Returns:
Channel name string, or ``'Ch <idx>'`` as fallback.
"""
for ch in self.channels:
ch_idx = ch.get('idx', ch.get('index', 0))
if ch_idx == channel_idx:
return ch.get('name', f'Ch {channel_idx}')
return f'Ch {channel_idx}'
def _resolve_path_names(self, path_hashes: list) -> list:
"""Resolve 2-char path hashes to display names.
MUST be called with self.lock held.
Safety-net for messages whose path_names were not resolved at
receive time (e.g. older code path, or contacts not yet loaded).
Args:
path_hashes: List of 2-char hex strings.
Returns:
List of display names (same length as *path_hashes*).
"""
names = []
for h in path_hashes:
if not h or len(h) < 2:
names.append('-')
continue
found_name = ''
for key, contact in self.contacts.items():
if key.lower().startswith(h.lower()):
found_name = contact.get('adv_name', '')
break
names.append(found_name if found_name else f'0x{h.upper()}')
return names
def add_rx_log(self, entry: RxLogEntry) -> None:
"""Add an RxLogEntry (max 50, newest first)."""
with self.lock:
self.rx_log.insert(0, entry)
if len(self.rx_log) > 50:
self.rx_log.pop()
self.rxlog_updated = True
# Archive entry for persistent storage
if self.archive:
self.archive.add_rx_log(entry)
def load_recent_from_archive(self, limit: int = 100) -> int:
"""Load the most recent archived messages into the in-memory list.
Intended for startup: populates ``self.messages`` from the
persistent archive so the main page shows historical messages
immediately, before any live device traffic arrives.
Safe to call multiple times (idempotent): clears the existing
message list and fingerprint set before loading, so reconnect
cycles do not produce duplicates.
Messages are inserted directly (not re-archived) to avoid
duplicating data on disk.
Args:
limit: Maximum number of messages to load.
Returns:
Number of messages loaded.
"""
if not self.archive:
return 0
recent, _ = self.archive.query_messages(limit=limit)
if not recent:
return 0
with self.lock:
# Clear existing messages and fingerprints to ensure
# idempotent behaviour on repeated calls (reconnect).
self.messages.clear()
self._message_fingerprints.clear()
# recent is newest-first; reverse so oldest is appended first
for msg_dict in reversed(recent):
msg = Message.from_dict(msg_dict)
fp = self._message_fingerprint(msg)
if fp not in self._message_fingerprints:
self.messages.append(msg)
self._message_fingerprints.add(fp)
# Cap at 100 (same as add_message)
if len(self.messages) > 100:
self.messages = self.messages[-100:]
# Rebuild fingerprint set from retained messages
self._message_fingerprints = {
self._message_fingerprint(m) for m in self.messages
}
debug_print(
f"Loaded {len(self.messages)} recent messages from archive"
)
return len(self.messages)
# ------------------------------------------------------------------
# Snapshot and flags
# ------------------------------------------------------------------
def get_snapshot(self) -> Dict:
"""Create a complete snapshot of all data for the GUI.
Returns a plain dict with typed objects inside. The
``messages`` and ``rx_log`` values are lists of dataclass
instances (not dicts).
"""
with self.lock:
return self._build_snapshot_unlocked()
def get_snapshot_and_clear_flags(self) -> Dict:
"""Atomically snapshot all data and reset update flags.
Combines ``get_snapshot()`` and ``clear_update_flags()`` in a
single lock acquisition. This eliminates the race condition
where the worker sets a flag between the two separate calls,
causing the GUI to miss an update (e.g. newly discovered
channels never appearing in the menu).
Returns:
Same dict structure as ``get_snapshot()``.
"""
with self.lock:
snapshot = self._build_snapshot_unlocked()
self.device_updated = False
self.contacts_updated = False
self.channels_updated = False
self.rxlog_updated = False
self.messages_updated = False
return snapshot
def _build_snapshot_unlocked(self) -> Dict:
"""Build the snapshot dict. MUST be called with self.lock held."""
d = self.device
return {
# DeviceInfo fields (flat for backward compat)
'name': d.name,
'public_key': d.public_key,
'radio_freq': d.radio_freq,
'radio_sf': d.radio_sf,
'radio_bw': d.radio_bw,
'tx_power': d.tx_power,
'adv_lat': d.adv_lat,
'adv_lon': d.adv_lon,
'firmware_version': d.firmware_version,
# Status
'connected': self.connected,
'status': self.status,
# Collections (typed copies)
'contacts': self.contacts.copy(),
'channels': self.channels.copy(),
'messages': self.messages.copy(),
'rx_log': self.rx_log.copy(),
# Flags
'device_updated': self.device_updated,
'contacts_updated': self.contacts_updated,
'channels_updated': self.channels_updated,
'rxlog_updated': self.rxlog_updated,
'messages_updated': self.messages_updated,
'gui_initialized': self.gui_initialized,
'bot_enabled': self.bot_enabled,
'auto_add_enabled': self.auto_add_enabled,
# Archive (for archive viewer)
'archive': self.archive,
# Room login states
'room_login_states': {
k: v.copy()
for k, v in self.room_login_states.items()
},
# Room message cache (archived + live)
'room_messages': {
k: list(v)
for k, v in self._room_msg_cache.items()
},
}
def clear_update_flags(self) -> None:
with self.lock:
self.device_updated = False
self.contacts_updated = False
self.channels_updated = False
self.rxlog_updated = False
self.messages_updated = False
def mark_gui_initialized(self) -> None:
with self.lock:
self.gui_initialized = True
debug_print("GUI marked as initialized")
# ------------------------------------------------------------------
# Contact lookups
# ------------------------------------------------------------------
def get_contact_by_prefix(self, pubkey_prefix: str) -> Optional[Dict]:
if not pubkey_prefix:
return None
with self.lock:
for key, contact in self.contacts.items():
if key.lower().startswith(pubkey_prefix.lower()) or pubkey_prefix.lower().startswith(key.lower()):
return contact.copy()
return None
def get_contact_name_by_prefix(self, pubkey_prefix: str) -> str:
if not pubkey_prefix:
return ""
with self.lock:
for key, contact in self.contacts.items():
if key.lower().startswith(pubkey_prefix.lower()):
name = contact.get('adv_name', '')
if name:
return name
return pubkey_prefix[:8]
def get_contact_by_name(self, name: str) -> Optional[Tuple[str, Dict]]:
if not name:
return None
with self.lock:
# Strategy 1: exact match
for key, contact in self.contacts.items():
if contact.get('adv_name', '') == name:
return (key, contact.copy())
# Strategy 2: case-insensitive
name_lower = name.lower()
for key, contact in self.contacts.items():
if contact.get('adv_name', '').lower() == name_lower:
return (key, contact.copy())
# Strategy 3: prefix match
for key, contact in self.contacts.items():
adv = contact.get('adv_name', '')
if not adv:
continue
if name.startswith(adv) or adv.startswith(name):
return (key, contact.copy())
return None
# ------------------------------------------------------------------
# Archive stats
# ------------------------------------------------------------------
def get_archive_stats(self) -> Optional[Dict]:
"""Get statistics from the message archive.
Returns:
Dict with archive stats, or None if archive not initialized.
"""
if self.archive:
return self.archive.get_stats()
return None

3
meshcore_gui/gui/__init__.py Normal file
View File

@@ -0,0 +1,3 @@
"""
Presentation layer — NiceGUI pages and panels.
"""

307
meshcore_gui/gui/archive_page.py Normal file
View File

@@ -0,0 +1,307 @@
"""
Archive viewer page for MeshCore GUI.
Displays archived messages with filters and pagination.
"""
from datetime import datetime, timedelta, timezone
from typing import Optional
from nicegui import ui
from meshcore_gui.core.models import Message
from meshcore_gui.gui.constants import resolve_contact_icon
from meshcore_gui.core.protocols import SharedDataReadAndLookup
class ArchivePage:
"""Archive viewer page with filters and pagination.
Shows archived messages in the same style as the main messages panel,
with filters (date range, text search) and pagination.
Channel filtering is driven by the drawer submenu via
:meth:`set_channel_filter`.
"""
def __init__(self, shared: SharedDataReadAndLookup, page_size: int = 50):
"""Initialize archive page.
Args:
shared: SharedData reader with contact lookup.
page_size: Number of messages per page.
"""
self._shared = shared
self._page_size = page_size
# Current page state
self._current_page = 0
self._channel_name_filter = None
self._text_filter = ""
self._days_back = 7 # Default: last 7 days
# UI references for inline refresh
self._channel_label = None
self._filter_card = None
self._msg_outer = None
self._text_input = None
self._days_select = None
# -- Channel filter (set by dashboard submenu) ---------------------
def set_channel_filter(self, channel) -> None:
"""Set the channel filter from the drawer submenu.
Args:
channel: None for all messages, 'DM' for DM only,
or str for a specific channel name.
"""
self._channel_name_filter = channel
self._current_page = 0
# Update channel label
if self._channel_label:
if channel is None:
self._channel_label.text = '\U0001f4da Archive — All'
elif channel == 'DM':
self._channel_label.text = '\U0001f4da Archive — DM'
else:
self._channel_label.text = f'\U0001f4da Archive — {channel}'
# Inline refresh
self._refresh_messages()
# -- Render --------------------------------------------------------
def render(self):
"""Render the archive page."""
with ui.column().classes('w-full p-4 gap-4').style(
'height: calc(100vh - 5rem); overflow: hidden'
):
# Header row: channel label (left) + filter icon (right)
with ui.row().classes('w-full items-center justify-between'):
self._channel_label = ui.label(
'\U0001f4da Archive — All'
).classes('text-2xl font-bold')
ui.button(
icon='filter_list',
on_click=lambda: self._filter_card.set_visibility(
not self._filter_card.visible
),
).props('flat round dense').tooltip('Toggle filters')
# Filters (days + text search — channel is driven by submenu)
self._render_filters()
# Messages container (refreshed inline)
self._msg_outer = ui.column().classes(
'w-full gap-2 flex-grow'
).style('overflow: hidden; min-height: 0')
self._refresh_messages()
def _render_filters(self):
"""Render filter controls (days + text search only)."""
self._filter_card = ui.card().classes('w-full')
self._filter_card.set_visibility(False)
with self._filter_card:
ui.label('Filters').classes('text-lg font-bold mb-2')
with ui.row().classes('w-full gap-4 items-end'):
# Days back filter
with ui.column().classes('flex-none'):
ui.label('Time Range').classes('text-sm')
self._days_select = ui.select(
options={
1: 'Last 24 hours',
7: 'Last 7 days',
30: 'Last 30 days',
90: 'Last 90 days',
9999: 'All time',
},
value=self._days_back,
).classes('w-48')
def on_days_change(e):
self._days_back = e.value
self._current_page = 0
self._refresh_messages()
self._days_select.on('update:model-value', on_days_change)
# Text search
with ui.column().classes('flex-1'):
ui.label('Search Text').classes('text-sm')
self._text_input = ui.input(
placeholder='Search in messages...',
value=self._text_filter,
).classes('w-full')
def on_text_change(e):
self._text_filter = e.value
self._current_page = 0
self._text_input.on('change', on_text_change)
# Search button (inline refresh — no page reload)
ui.button(
'Search', on_click=lambda: self._refresh_messages()
).props('flat color=primary')
# Clear filters
def clear_filters():
self._channel_name_filter = None
self._text_filter = ""
self._days_back = 7
self._current_page = 0
# Reset UI elements
if self._text_input:
self._text_input.value = ''
if self._days_select:
self._days_select.value = 7
if self._channel_label:
self._channel_label.text = '\U0001f4da Archive — All'
self._refresh_messages()
ui.button('Clear', on_click=clear_filters).props('flat')
def _refresh_messages(self):
"""Rebuild message list inline (no page reload)."""
if not self._msg_outer:
return
self._msg_outer.clear()
snapshot = self._shared.get_snapshot()
with self._msg_outer:
self._render_messages(snapshot)
def _render_messages(self, snapshot: dict):
"""Render messages with pagination.
Args:
snapshot: Current snapshot containing archive data.
"""
if not snapshot.get('archive'):
ui.label('Archive not available').classes('text-gray-500 italic')
return
archive = snapshot['archive']
contacts = snapshot.get('contacts', {})
# Calculate date range
now = datetime.now(timezone.utc)
after = None if self._days_back >= 9999 else now - timedelta(days=self._days_back)
# Handle DM filter separately (query_messages doesn't filter by channel=None)
is_dm_filter = (self._channel_name_filter == 'DM')
query_channel = None if is_dm_filter else self._channel_name_filter
# Query messages
messages, total_count = archive.query_messages(
after=after,
channel_name=query_channel,
text_search=self._text_filter if self._text_filter else None,
limit=self._page_size if not is_dm_filter else self._page_size * 5,
offset=self._current_page * self._page_size if not is_dm_filter else 0,
)
# Post-filter for DM (channel is None)
if is_dm_filter:
messages = [m for m in messages if m.get('channel') is None]
total_count = len(messages)
# Apply pagination manually
start = self._current_page * self._page_size
messages = messages[start:start + self._page_size]
# Pagination info
total_pages = (total_count + self._page_size - 1) // self._page_size
# Pagination header
with ui.row().classes('w-full items-center justify-between'):
ui.label(f'Showing {len(messages)} of {total_count} messages').classes('text-sm text-gray-600')
if total_pages > 1:
with ui.row().classes('gap-2'):
# Previous button
def go_prev():
if self._current_page > 0:
self._current_page -= 1
self._refresh_messages()
ui.button('Previous', on_click=go_prev).props(
f'flat {"disabled" if self._current_page == 0 else ""}'
)
# Page indicator
ui.label(f'Page {self._current_page + 1} / {total_pages}').classes('mx-2')
# Next button
def go_next():
if self._current_page < total_pages - 1:
self._current_page += 1
self._refresh_messages()
ui.button('Next', on_click=go_next).props(
f'flat {"disabled" if self._current_page >= total_pages - 1 else ""}'
)
# Messages list (single-line format, same as main page)
if not messages:
ui.label('No messages found').classes('text-gray-500 italic mt-4')
else:
with ui.column().classes(
'w-full flex-grow overflow-y-auto gap-0 text-sm font-mono '
'bg-gray-50 p-2 rounded'
):
# Hide channel tag when viewing a specific channel/DM
hide_ch = self._channel_name_filter is not None
for msg_dict in messages:
msg = Message.from_dict(msg_dict)
sender_icon = resolve_contact_icon(
contacts,
pubkey=msg.sender_pubkey,
name=msg.sender,
fallback_type=1 if msg.direction == 'out' else None,
)
line = msg.format_line(
show_channel=not hide_ch,
sender_prefix=f'{sender_icon} ',
)
msg_hash = msg_dict.get('message_hash', '')
ui.label(line).classes(
'text-xs leading-tight cursor-pointer '
'hover:bg-blue-50 rounded px-1'
).on('click', lambda e, h=msg_hash: self._open_route(h))
# Pagination footer
if total_pages > 1:
with ui.row().classes('w-full items-center justify-center mt-4'):
ui.button('Previous', on_click=go_prev).props(
f'flat {"disabled" if self._current_page == 0 else ""}'
)
ui.label(f'Page {self._current_page + 1} / {total_pages}').classes('mx-4')
ui.button('Next', on_click=go_next).props(
f'flat {"disabled" if self._current_page >= total_pages - 1 else ""}'
)
@staticmethod
def setup_route(shared: SharedDataReadAndLookup):
"""Setup the /archive route.
Args:
shared: SharedData reader with contact lookup.
"""
@ui.page('/archive')
def archive_page():
page = ArchivePage(shared)
page.render()
@staticmethod
def _open_route(msg_hash: str) -> None:
if msg_hash:
ui.navigate.to(f'/route/{msg_hash}')
else:
ui.notify('Route data not available for this message', type='warning')

64
meshcore_gui/gui/constants.py Normal file
View File

@@ -0,0 +1,64 @@
"""
Display constants for the GUI layer.
Contact type → icon/name/label mappings used by multiple panels.
"""
from typing import Any, Dict, Mapping, Optional
TYPE_ICONS: Dict[int, str] = {0: "○", 1: "📱", 2: "📡", 3: "🏠"}
TYPE_NAMES: Dict[int, str] = {0: "-", 1: "CLI", 2: "REP", 3: "ROOM"}
TYPE_LABELS: Dict[int, str] = {0: "-", 1: "Companion", 2: "Repeater", 3: "Room Server"}
def get_type_icon(contact_type: Optional[int]) -> str:
"""Return the display icon for a contact type."""
return TYPE_ICONS.get(contact_type or 0, TYPE_ICONS[0])
def get_type_label(contact_type: Optional[int]) -> str:
"""Return the human-readable label for a contact type."""
return TYPE_LABELS.get(contact_type or 0, TYPE_LABELS[0])
def get_type_display(contact_type: Optional[int]) -> str:
"""Return a combined icon + label string for a contact type."""
icon = get_type_icon(contact_type)
label = get_type_label(contact_type)
return f"{icon} {label}" if label != '-' else icon
def resolve_contact_type(
contacts: Mapping[str, Mapping[str, Any]],
pubkey: str = '',
name: str = '',
) -> int:
"""Resolve the contact type from snapshot contacts by pubkey or name."""
if pubkey:
pubkey_lower = pubkey.lower()
for key, contact in contacts.items():
key_lower = key.lower()
if key_lower.startswith(pubkey_lower) or pubkey_lower.startswith(key_lower):
return int(contact.get('type', 0) or 0)
if name:
name_lower = name.lower()
for contact in contacts.values():
adv_name = str(contact.get('adv_name', '') or '')
if adv_name and adv_name.lower() == name_lower:
return int(contact.get('type', 0) or 0)
return 0
def resolve_contact_icon(
contacts: Mapping[str, Mapping[str, Any]],
pubkey: str = '',
name: str = '',
fallback_type: Optional[int] = None,
) -> str:
"""Resolve the display icon for a contact using the shared type mapping."""
resolved_type = resolve_contact_type(contacts, pubkey=pubkey, name=name)
if resolved_type == 0 and fallback_type is not None:
resolved_type = fallback_type
return get_type_icon(resolved_type)

829
meshcore_gui/gui/dashboard.py Normal file
View File

@@ -0,0 +1,829 @@
"""
Main dashboard page for MeshCore GUI.
Thin orchestrator that owns the layout and the 500 ms update timer.
All visual content is delegated to individual panel classes in
:mod:`meshcore_gui.gui.panels`.
"""
import logging
from urllib.parse import urlencode
from nicegui import ui
from meshcore_gui import config
from meshcore_gui.core.protocols import SharedDataReader
from meshcore_gui.gui.panels import (
ActionsPanel,
ContactsPanel,
DevicePanel,
MapPanel,
MessagesPanel,
RoomServerPanel,
RxLogPanel,
)
from meshcore_gui.gui.archive_page import ArchivePage
from meshcore_gui.services.pin_store import PinStore
from meshcore_gui.services.room_password_store import RoomPasswordStore
# Suppress the harmless "Client has been deleted" warning that NiceGUI
# emits when a browser tab is refreshed while a ui.timer is active.
class _DeletedClientFilter(logging.Filter):
def filter(self, record: logging.LogRecord) -> bool:
return 'Client has been deleted' not in record.getMessage()
logging.getLogger('nicegui').addFilter(_DeletedClientFilter())
# ── DOMCA Theme ──────────────────────────────────────────────────────
# Fonts + CSS variables adapted from domca.nl style.css for NiceGUI/Quasar.
# Dark/light variable sets switch via Quasar's body--dark / body--light classes.
_DOMCA_HEAD = '''
<link rel="manifest" href="/static/manifest.json">
<meta name="theme-color" content="#0d1f35">
<meta name="mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
<meta name="apple-mobile-web-app-title" content="DOMCA">
<link rel="apple-touch-icon" href="/static/icon-192.png">
<link href="https://fonts.googleapis.com/css2?family=Exo+2:wght@800&family=JetBrains+Mono:wght@400;700&display=swap" rel="stylesheet">
<style>
/* ── DOMCA theme variables (dark) ── */
body.body--dark {
--bg: #0A1628;
--grid: #0077B6; --grid-op: 0.15;
--mesh-bg: #48CAE4; --mesh-bg-op: 0.08;
--line: #0077B6; --line-op: 0.6;
--wave: #48CAE4; --node: #00B4D8; --node-center: #CAF0F8;
--hub-text: #0A1628; --outer: #0077B6;
--title: #48CAE4; --subtitle: #48CAE4;
--tagline: #90E0EF; --tag-op: 0.5;
--badge-stroke: #0077B6; --badge-text: #48CAE4;
--callsign: #0077B6;
}
/* ── DOMCA theme variables (light) ── */
body.body--light {
--bg: #FFFFFF;
--grid: #023E8A; --grid-op: 0.04;
--mesh-bg: #0077B6; --mesh-bg-op: 0.05;
--line: #0096C7; --line-op: 0.35;
--wave: #0096C7; --node: #0077B6; --node-center: #FFFFFF;
--hub-text: #FFFFFF; --outer: #0096C7;
--title: #0077B6; --subtitle: #0077B6;
--tagline: #0096C7; --tag-op: 0.4;
--badge-stroke: #0077B6; --badge-text: #0077B6;
--callsign: #0096C7;
}
/* ── DOMCA page background ── */
body.body--dark { background: #0A1628 !important; }
body.body--light { background: #f4f8fb !important; }
body.body--dark .q-page { background: #0A1628 !important; }
body.body--light .q-page { background: #f4f8fb !important; }
/* ── DOMCA header ── */
body.body--dark .q-header { background: #0d1f35 !important; }
body.body--light .q-header { background: #0077B6 !important; }
/* ── DOMCA drawer — distinct from page background ── */
body.body--dark .domca-drawer { background: #0f2340 !important; border-right: 1px solid rgba(0,119,182,0.25) !important; }
body.body--light .domca-drawer { background: rgba(244,248,251,0.97) !important; }
.domca-drawer .q-btn__content { justify-content: flex-start !important; }
/* ── DOMCA cards — dark mode readable ── */
body.body--dark .q-card {
background: #112240 !important;
color: #e0f0f8 !important;
border: 1px solid rgba(0,119,182,0.15) !important;
}
body.body--dark .q-card .text-gray-600 { color: #48CAE4 !important; }
body.body--dark .q-card .text-gray-500 { color: #8badc4 !important; }
body.body--dark .q-card .text-gray-400 { color: #6a8fa8 !important; }
body.body--dark .q-card .text-xs { color: #c0dce8 !important; }
body.body--dark .q-card .text-sm { color: #d0e8f2 !important; }
body.body--dark .q-card .text-red-400 { color: #f87171 !important; }
/* ── Dark mode: message area, inputs, tables ── */
body.body--dark .bg-gray-50 { background: #0c1a2e !important; color: #c0dce8 !important; }
body.body--dark .bg-gray-100 { background: #152a45 !important; }
body.body--dark .hover\\:bg-gray-100:hover { background: #1a3352 !important; }
body.body--dark .hover\\:bg-blue-50:hover { background: #0d2a4a !important; }
body.body--dark .bg-yellow-50 { background: rgba(72,202,228,0.06) !important; }
body.body--dark .q-field__control { background: #0c1a2e !important; color: #e0f0f8 !important; }
body.body--dark .q-field__native { color: #e0f0f8 !important; }
body.body--dark .q-field__label { color: #8badc4 !important; }
body.body--dark .q-table { background: #112240 !important; color: #c0dce8 !important; }
body.body--dark .q-table thead th { color: #48CAE4 !important; }
body.body--dark .q-table tbody td { color: #c0dce8 !important; }
body.body--dark .q-checkbox__label { color: #c0dce8 !important; }
body.body--dark .q-btn--flat:not(.domca-menu-btn):not(.domca-sub-btn) { color: #48CAE4 !important; }
body.body--dark .q-separator { background: rgba(0,119,182,0.2) !important; }
/* ── DOMCA menu link styling ── */
body.body--dark .domca-menu-btn { color: #8badc4 !important; }
body.body--dark .domca-menu-btn:hover { color: #48CAE4 !important; }
body.body--light .domca-menu-btn { color: #3d6380 !important; }
body.body--light .domca-menu-btn:hover { color: #0077B6 !important; }
body.body--dark .domca-ext-link { color: #8badc4 !important; }
body.body--light .domca-ext-link { color: #3d6380 !important; }
/* ── DOMCA active menu item ── */
body.body--dark .domca-menu-active { color: #48CAE4 !important; background: rgba(72,202,228,0.1) !important; }
body.body--light .domca-menu-active { color: #0077B6 !important; background: rgba(0,119,182,0.08) !important; }
/* ── DOMCA submenu item styling ── */
body.body--dark .domca-sub-btn { color: #6a8fa8 !important; }
body.body--dark .domca-sub-btn:hover { color: #48CAE4 !important; }
body.body--light .domca-sub-btn { color: #5a7a90 !important; }
body.body--light .domca-sub-btn:hover { color: #0077B6 !important; }
/* ── DOMCA expansion panel in drawer ── */
.domca-drawer .q-expansion-item {
font-family: 'JetBrains Mono', monospace !important;
letter-spacing: 2px;
font-size: 0.8rem;
}
.domca-drawer .q-expansion-item .q-item {
padding: 0.35rem 1.2rem !important;
min-height: 32px !important;
}
.domca-drawer .q-expansion-item .q-expansion-item__content {
padding: 0 !important;
}
.domca-drawer .q-expansion-item + .q-expansion-item {
margin-top: 0 !important;
}
body.body--dark .domca-drawer .q-expansion-item { color: #8badc4 !important; }
body.body--dark .domca-drawer .q-expansion-item__container { background: transparent !important; }
body.body--dark .domca-drawer .q-item { color: #8badc4 !important; }
body.body--light .domca-drawer .q-expansion-item { color: #3d6380 !important; }
body.body--light .domca-drawer .q-item { color: #3d6380 !important; }
/* ── Landing page centering ── */
.domca-landing {
display: flex;
align-items: center;
justify-content: center;
min-height: calc(100vh - 64px);
padding: 0.5rem;
}
.domca-landing svg {
width: min(90vw, 800px);
height: auto;
display: block;
}
/* ── Panel container — responsive single column ── */
.domca-panel {
width: 100%;
max-width: 900px;
margin: 0 auto;
padding: 0.5rem;
}
/* ── Responsive heights — override fixed Tailwind heights in panels ── */
.domca-panel .h-40 { height: calc(100vh - 20rem) !important; min-height: 10rem; }
.domca-panel .h-32 { height: calc(100vh - 24rem) !important; min-height: 8rem; }
.domca-panel .h-72 { height: calc(100vh - 12rem) !important; min-height: 14rem; }
.domca-panel .max-h-48 { max-height: calc(100vh - 16rem) !important; min-height: 6rem; }
/* ── Allow narrow viewports down to 320px ── */
body, .q-layout, .q-page {
min-width: 0 !important;
}
.q-drawer { max-width: 80vw !important; width: 260px !important; min-width: 200px !important; }
/* ── Mobile optimisations ── */
@media (max-width: 640px) {
.domca-landing svg { width: 98vw; }
.domca-panel { padding: 0.25rem; }
.domca-panel .q-card { border-radius: 8px !important; }
}
@media (max-width: 400px) {
.domca-landing { padding: 0.25rem; }
.domca-landing svg { width: 100vw; }
.q-header { padding-left: 0.5rem !important; padding-right: 0.5rem !important; }
}
/* ── Footer label ── */
.domca-footer {
font-family: 'JetBrains Mono', monospace;
font-size: 0.65rem;
letter-spacing: 2px;
opacity: 0.3;
}
/* ── Header text: icon-only on narrow viewports ── */
@media (max-width: 599px) {
.domca-header-text { display: none !important; }
}
</style>
'''
# ── Landing SVG loader ────────────────────────────────────────────────
# Reads the SVG from config.LANDING_SVG_PATH and replaces {callsign}
# with config.OPERATOR_CALLSIGN. Falls back to a minimal placeholder
# when the file is missing.
def _load_landing_svg() -> str:
"""Load the landing page SVG from disk.
Returns:
SVG markup string with ``{callsign}`` replaced by the
configured operator callsign.
"""
path = config.LANDING_SVG_PATH
try:
raw = path.read_text(encoding="utf-8")
return raw.replace("{callsign}", config.OPERATOR_CALLSIGN)
except FileNotFoundError:
return (
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 400 100">'
'<text x="200" y="55" text-anchor="middle" '
'font-family="\'JetBrains Mono\',monospace" font-size="14" '
f'fill="var(--title)">Landing SVG not found: {path.name}</text>'
'</svg>'
)
# ── Standalone menu items (no submenus) ──────────────────────────────
_STANDALONE_ITEMS = [
('\U0001f465', 'CONTACTS', 'contacts'),
('\U0001f5fa\ufe0f', 'MAP', 'map'),
('\U0001f4e1', 'DEVICE', 'device'),
('\u26a1', 'ACTIONS', 'actions'),
('\U0001f4ca', 'RX LOG', 'rxlog'),
]
_EXT_LINKS = config.EXT_LINKS
# ── Shared button styles ─────────────────────────────────────────────
_SUB_BTN_STYLE = (
"font-family: 'JetBrains Mono', monospace; "
"letter-spacing: 1px; font-size: 0.72rem; "
"padding: 0.2rem 1.2rem 0.2rem 2.4rem"
)
_MENU_BTN_STYLE = (
"font-family: 'JetBrains Mono', monospace; "
"letter-spacing: 2px; font-size: 0.8rem; "
"padding: 0.35rem 1.2rem"
)
class DashboardPage:
"""Main dashboard rendered at ``/``.
Args:
shared: SharedDataReader for data access and command dispatch.
"""
def __init__(self, shared: SharedDataReader, pin_store: PinStore, room_password_store: RoomPasswordStore) -> None:
self._shared = shared
self._pin_store = pin_store
self._room_password_store = room_password_store
# Panels (created fresh on each render)
self._device: DevicePanel | None = None
self._contacts: ContactsPanel | None = None
self._map: MapPanel | None = None
self._messages: MessagesPanel | None = None
self._actions: ActionsPanel | None = None
self._rxlog: RxLogPanel | None = None
self._room_server: RoomServerPanel | None = None
# Header status label
self._status_label = None
# Local first-render flag
self._initialized: bool = False
# Panel switching state (layout)
self._panel_containers: dict = {}
self._active_panel: str = 'landing'
self._drawer = None
self._menu_buttons: dict = {}
# Submenu containers (for dynamic channel/room items)
self._msg_sub_container = None
self._archive_sub_container = None
self._rooms_sub_container = None
self._last_channel_fingerprint = None
self._last_rooms_fingerprint = None
# Archive page reference (for inline channel switching)
self._archive_page: ArchivePage | None = None
# ------------------------------------------------------------------
# Public
# ------------------------------------------------------------------
def render(self) -> None:
"""Build the complete dashboard layout and start the timer."""
self._initialized = False
# Reset fingerprints: render() creates new (empty) NiceGUI
# containers, so _update_submenus must rebuild into them even
# when the channel/room data hasn't changed since last session.
self._last_channel_fingerprint = None
self._last_rooms_fingerprint = None
# Create panel instances (UNCHANGED functional wiring)
put_cmd = self._shared.put_command
self._device = DevicePanel()
self._contacts = ContactsPanel(put_cmd, self._pin_store, self._shared.set_auto_add_enabled, self._on_add_room_server)
self._map = MapPanel()
self._messages = MessagesPanel(put_cmd)
self._actions = ActionsPanel(put_cmd, self._shared.set_bot_enabled)
self._rxlog = RxLogPanel()
self._room_server = RoomServerPanel(put_cmd, self._room_password_store)
# Inject DOMCA theme (fonts + CSS variables)
ui.add_head_html(_DOMCA_HEAD)
# Default to dark mode (DOMCA theme)
dark = ui.dark_mode(True)
dark.on_value_change(lambda e: self._map.set_ui_dark_mode(e.value))
self._map.set_ui_dark_mode(dark.value)
# ── Left Drawer (must be created before header for Quasar) ────
self._drawer = ui.left_drawer(value=False, bordered=True).classes(
'domca-drawer'
).style('padding: 0')
with self._drawer:
# DOMCA branding (clickable → landing page)
with ui.column().style('padding: 0.2rem 1.2rem 0'):
ui.button(
'DOMCA',
on_click=lambda: self._navigate_panel('landing'),
).props('flat no-caps').style(
"font-family: 'Exo 2', sans-serif; font-size: 1.4rem; "
"font-weight: 800; color: var(--title); letter-spacing: 4px; "
"margin-bottom: 0.3rem; padding: 0"
)
self._menu_buttons = {}
# ── 💬 MESSAGES (expandable with channel submenu) ──────
with ui.expansion(
'\U0001f4ac MESSAGES', icon=None, value=False,
).props('dense header-class="q-pa-none"').classes('w-full'):
self._msg_sub_container = ui.column().classes('w-full gap-0')
with self._msg_sub_container:
self._make_sub_btn(
'ALL', lambda: self._navigate_panel('messages', channel=None)
)
self._make_sub_btn(
'DM', lambda: self._navigate_panel('messages', channel='DM')
)
# Dynamic channel items populated by _update_submenus
# ── 🏠 ROOMS (expandable with room submenu) ───────────
with ui.expansion(
'\U0001f3e0 ROOMS', icon=None, value=False,
).props('dense header-class="q-pa-none"').classes('w-full'):
self._rooms_sub_container = ui.column().classes('w-full gap-0')
with self._rooms_sub_container:
self._make_sub_btn(
'ALL', lambda: self._navigate_panel('rooms')
)
# Pre-populate from persisted rooms
for entry in self._room_password_store.get_rooms():
short = entry.name or entry.pubkey[:12]
self._make_sub_btn(
f'\U0001f3e0 {short}',
lambda: self._navigate_panel('rooms'),
)
# ── 📚 ARCHIVE (expandable with channel submenu) ──────
with ui.expansion(
'\U0001f4da ARCHIVE', icon=None, value=False,
).props('dense header-class="q-pa-none"').classes('w-full'):
self._archive_sub_container = ui.column().classes('w-full gap-0')
with self._archive_sub_container:
self._make_sub_btn(
'ALL', lambda: self._navigate_panel('archive', channel=None)
)
self._make_sub_btn(
'DM', lambda: self._navigate_panel('archive', channel='DM')
)
# Dynamic channel items populated by _update_submenus
ui.separator().classes('my-1')
# ── Standalone menu items (MAP, DEVICE, ACTIONS, RX LOG)
for icon, label, panel_id in _STANDALONE_ITEMS:
btn = ui.button(
f'{icon} {label}',
on_click=lambda pid=panel_id: self._navigate_panel(pid),
).props('flat no-caps align=left').classes(
'w-full justify-start domca-menu-btn'
).style(_MENU_BTN_STYLE)
self._menu_buttons[panel_id] = btn
ui.separator().classes('my-2')
# External links (same as domca.nl navigation)
with ui.column().style('padding: 0 1.2rem'):
for label, url in _EXT_LINKS:
ui.link(label, url, new_tab=True).classes(
'domca-ext-link'
).style(
"font-family: 'JetBrains Mono', monospace; "
"letter-spacing: 2px; font-size: 0.72rem; "
"text-decoration: none; opacity: 0.6; "
"display: block; padding: 0.35rem 0"
)
# Footer in drawer
ui.space()
ui.label(f'\u00a9 2026 {config.OPERATOR_CALLSIGN}').classes('domca-footer').style('padding: 0 1.2rem 1rem')
# ── Header ────────────────────────────────────────────────
with ui.header().classes('items-center px-4 py-2 shadow-md'):
menu_btn = ui.button(
icon='menu',
on_click=lambda: self._drawer.toggle(),
).props('flat round dense color=white')
# Swap icon: menu ↔ close
self._drawer.on_value_change(
lambda e: menu_btn.props(f'icon={"close" if e.value else "menu"}')
)
ui.label(f'\U0001f517 MeshCore v{config.VERSION}').classes(
'text-lg font-bold ml-2 domca-header-text'
).style("font-family: 'JetBrains Mono', monospace")
# Transport mode badge
_is_ble = config.TRANSPORT == "ble"
_badge_icon = '🔵' if _is_ble else '🟢'
_badge_label = 'BLE' if _is_ble else 'Serial'
ui.label(f'{_badge_icon} {_badge_label}').classes(
'text-xs ml-2 domca-header-text'
).style(
"font-family: 'JetBrains Mono', monospace; "
"opacity: 0.65; letter-spacing: 1px"
)
ui.space()
_initial_status = self._shared.get_snapshot().get('status', 'Starting...')
self._status_label = ui.label(_initial_status).classes(
'text-sm opacity-70 domca-header-text'
)
ui.button(
icon='brightness_6',
on_click=lambda: dark.toggle(),
).props('flat round dense color=white').tooltip('Toggle dark / light')
# ── Main Content Area ─────────────────────────────────────
self._panel_containers = {}
# Landing page (SVG splash from file — visible by default)
landing = ui.column().classes('domca-landing w-full')
with landing:
ui.html(_load_landing_svg())
self._panel_containers['landing'] = landing
# Panel containers (hidden by default, shown on menu click)
panel_defs = [
('messages', self._messages),
('contacts', self._contacts),
('map', self._map),
('device', self._device),
('actions', self._actions),
('rxlog', self._rxlog),
('rooms', self._room_server),
]
for panel_id, panel_obj in panel_defs:
container = ui.column().classes('domca-panel')
container.set_visibility(False)
with container:
panel_obj.render()
self._panel_containers[panel_id] = container
# Archive panel (inline — replaces separate /archive page)
archive_container = ui.column().classes('domca-panel')
archive_container.set_visibility(False)
with archive_container:
self._archive_page = ArchivePage(self._shared)
self._archive_page.render()
self._panel_containers['archive'] = archive_container
self._active_panel = 'landing'
# Start update timer
self._apply_url_state()
ui.timer(0.5, self._update_ui)
# ------------------------------------------------------------------
# Submenu button helper (layout only)
# ------------------------------------------------------------------
@staticmethod
def _make_sub_btn(label: str, on_click) -> ui.button:
"""Create a submenu button in the drawer."""
return ui.button(
label,
on_click=on_click,
).props('flat no-caps align=left').classes(
'w-full justify-start domca-sub-btn'
).style(_SUB_BTN_STYLE)
# ------------------------------------------------------------------
# Dynamic submenu updates (layout — called from _update_ui)
# ------------------------------------------------------------------
def _update_submenus(self, data: dict) -> None:
"""Rebuild channel/room submenu items when data changes.
Only the dynamic items are rebuilt; the container is cleared and
ALL items (static + dynamic) are re-rendered.
"""
# ── Channel submenus (Messages + Archive) ──
channels = data.get('channels', [])
ch_fingerprint = tuple((ch['idx'], ch['name']) for ch in channels)
if ch_fingerprint != self._last_channel_fingerprint and channels:
self._last_channel_fingerprint = ch_fingerprint
# Rebuild Messages submenu
if self._msg_sub_container:
self._msg_sub_container.clear()
with self._msg_sub_container:
self._make_sub_btn(
'ALL', lambda: self._navigate_panel('messages', channel=None)
)
self._make_sub_btn(
'DM', lambda: self._navigate_panel('messages', channel='DM')
)
for ch in channels:
idx = ch['idx']
name = ch['name']
self._make_sub_btn(
f"[{idx}] {name}",
lambda i=idx: self._navigate_panel('messages', channel=i),
)
# Rebuild Archive submenu
if self._archive_sub_container:
self._archive_sub_container.clear()
with self._archive_sub_container:
self._make_sub_btn(
'ALL', lambda: self._navigate_panel('archive', channel=None)
)
self._make_sub_btn(
'DM', lambda: self._navigate_panel('archive', channel='DM')
)
for ch in channels:
idx = ch['idx']
name = ch['name']
self._make_sub_btn(
f"[{idx}] {name}",
lambda n=name: self._navigate_panel('archive', channel=n),
)
# ── Room submenus ──
rooms = self._room_password_store.get_rooms()
rooms_fingerprint = tuple((r.pubkey, r.name) for r in rooms)
if rooms_fingerprint != self._last_rooms_fingerprint:
self._last_rooms_fingerprint = rooms_fingerprint
if self._rooms_sub_container:
self._rooms_sub_container.clear()
with self._rooms_sub_container:
self._make_sub_btn(
'ALL', lambda: self._navigate_panel('rooms')
)
for entry in rooms:
short = entry.name or entry.pubkey[:12]
self._make_sub_btn(
f'\U0001f3e0 {short}',
lambda: self._navigate_panel('rooms'),
)
# ------------------------------------------------------------------
# Panel switching (layout helper — no functional logic)
# ------------------------------------------------------------------
def _apply_url_state(self) -> None:
"""Apply panel selection from URL query params on first render."""
try:
params = ui.context.client.request.query_params
except Exception:
return
panel = params.get('panel') or 'landing'
channel = params.get('channel')
if panel not in self._panel_containers:
panel = 'landing'
channel = None
if panel == 'messages':
if channel is None or channel.lower() == 'all':
channel = None
elif channel.upper() == 'DM':
channel = 'DM'
else:
channel = int(channel) if channel.isdigit() else None
elif panel == 'archive':
if channel is None or channel.lower() == 'all':
channel = None
elif channel.upper() == 'DM':
channel = 'DM'
else:
channel = None
self._show_panel(panel, channel)
def _build_panel_url(self, panel_id: str, channel=None) -> str:
params = {'panel': panel_id}
if channel is not None:
params['channel'] = str(channel)
return '/?' + urlencode(params)
def _navigate_panel(self, panel_id: str, channel=None) -> None:
"""Navigate with panel id in the URL so browser back restores state."""
ui.navigate.to(self._build_panel_url(panel_id, channel))
def _show_panel(self, panel_id: str, channel=None) -> None:
"""Show the selected panel, hide all others, close the drawer.
Args:
panel_id: Panel to show (e.g. 'messages', 'archive', 'rooms').
channel: Optional channel filter.
For messages: None=all, 'DM'=DM only, int=channel idx.
For archive: None=all, 'DM'=DM only, str=channel name.
"""
for pid, container in self._panel_containers.items():
container.set_visibility(pid == panel_id)
self._active_panel = panel_id
# Apply channel filter to messages panel
if panel_id == 'messages' and self._messages:
self._messages.set_active_channel(channel)
# Force immediate rebuild so the panel is populated the
# moment it becomes visible, instead of waiting for the
# next 500 ms timer tick (which caused the "empty on first
# click, populated on second click" symptom).
data = self._shared.get_snapshot()
self._messages.update(
data,
self._messages.channel_filters,
self._messages.last_channels,
room_pubkeys=(
self._room_server.get_room_pubkeys()
if self._room_server else None
),
)
# Apply channel filter to archive panel
if panel_id == 'archive' and self._archive_page:
self._archive_page.set_channel_filter(channel)
# Force map recenter when opening map panel (Leaflet may be hidden on load)
if panel_id == 'map' and self._map:
data = self._shared.get_snapshot()
data['force_center'] = True
self._map.update(data)
# Update active menu highlight (standalone buttons only)
for pid, btn in self._menu_buttons.items():
if pid == panel_id:
btn.classes('domca-menu-active', remove='')
else:
btn.classes(remove='domca-menu-active')
# Close drawer after selection
if self._drawer:
self._drawer.hide()
# ------------------------------------------------------------------
# Room Server callback (from ContactsPanel)
# ------------------------------------------------------------------
def _on_add_room_server(self, pubkey: str, name: str, password: str) -> None:
"""Handle adding a Room Server from the contacts panel.
Delegates to the RoomServerPanel which persists the entry,
creates the UI card and sends the login command.
"""
if self._room_server:
self._room_server.add_room(pubkey, name, password)
# ------------------------------------------------------------------
# Timer-driven UI update
# ------------------------------------------------------------------
def _update_ui(self) -> None:
try:
if not self._status_label:
return
# Atomic snapshot + flag clear: eliminates race condition
# where worker sets channels_updated between separate
# get_snapshot() and clear_update_flags() calls.
data = self._shared.get_snapshot_and_clear_flags()
is_first = not self._initialized
# Mark initialised immediately — even if a panel update
# crashes below, we must NOT retry the full first-render
# path every 500 ms (that causes the infinite rebuild).
if is_first:
self._initialized = True
# Always update status
self._status_label.text = data['status']
# Device info
if data['device_updated'] or is_first:
self._device.update(data)
# Map updates are intentionally limited to when the map panel
# is visible. Updating Leaflet every 500 ms while hidden can
# trigger excessive tile/layer work in the browser and make the
# rest of the UI feel unresponsive (for example the hamburger
# menu appearing to do nothing). The explicit update in
# _show_panel('map') still refreshes and recenters the map when
# the user opens it.
if self._active_panel == 'map' and (
data['device_updated'] or is_first
):
self._map.update(data)
# Channel-dependent UI: always ensure consistency when
# channels exist. Because a single DashboardPage instance
# is shared across browser sessions (render() is called on
# each new connection), the old session's timer can steal
# the is_first flag before the new timer fires. Running
# these unconditionally is safe because each method has an
# internal fingerprint/equality check that prevents
# unnecessary DOM updates.
if data['channels']:
self._messages.update_filters(data)
self._messages.update_channel_options(data['channels'])
self._update_submenus(data)
# BOT checkbox state (only on actual change or first render
# to avoid overwriting user interaction mid-toggle)
if data['channels_updated'] or is_first:
self._actions.update(data)
# Contacts
if data['contacts_updated'] or is_first:
self._contacts.update(data)
# Map
if (
self._active_panel == 'map'
and data['contacts']
and (
data['contacts_updated']
or not self._map.has_markers
or is_first
)
):
self._map.update(data)
# Messages (always — for live filter changes)
self._messages.update(
data,
self._messages.channel_filters,
self._messages.last_channels,
room_pubkeys=self._room_server.get_room_pubkeys() if self._room_server else None,
)
# Room Server panels (always — for live messages + contact changes)
self._room_server.update(data)
# RX Log
if data['rxlog_updated']:
self._rxlog.update(data)
# Signal worker that GUI is ready for data
if is_first and data['channels'] and data['contacts']:
self._shared.mark_gui_initialized()
except Exception as e:
err = str(e).lower()
if "deleted" not in err and "client" not in err:
import traceback
print(f"GUI update error: {e}")
traceback.print_exc()

17
meshcore_gui/gui/panels/__init__.py Normal file
View File

@@ -0,0 +1,17 @@
"""
Individual dashboard panels — each panel is a single-responsibility class.
Re-exports all panels for convenient importing::
from meshcore_gui.gui.panels import DevicePanel, ContactsPanel, ...
"""
from meshcore_gui.gui.panels.device_panel import DevicePanel # noqa: F401
from meshcore_gui.gui.panels.contacts_panel import ContactsPanel # noqa: F401
from meshcore_gui.gui.panels.map_panel import MapPanel # noqa: F401
from meshcore_gui.gui.panels.input_panel import InputPanel # noqa: F401
from meshcore_gui.gui.panels.filter_panel import FilterPanel # noqa: F401
from meshcore_gui.gui.panels.messages_panel import MessagesPanel # noqa: F401
from meshcore_gui.gui.panels.actions_panel import ActionsPanel # noqa: F401
from meshcore_gui.gui.panels.rxlog_panel import RxLogPanel # noqa: F401
from meshcore_gui.gui.panels.room_server_panel import RoomServerPanel # noqa: F401

80
meshcore_gui/gui/panels/actions_panel.py Normal file
View File

@@ -0,0 +1,80 @@
"""Actions panel — refresh, advertise buttons and bot toggle."""
from typing import Callable, Dict
from nicegui import ui
class ActionsPanel:
"""Action buttons and bot toggle in the right column.
Args:
put_command: Callable to enqueue a command dict for the worker.
set_bot_enabled: Callable to toggle the bot in SharedData.
"""
def __init__(self, put_command: Callable[[Dict], None], set_bot_enabled: Callable[[bool], None]) -> None:
self._put_command = put_command
self._set_bot_enabled = set_bot_enabled
self._bot_checkbox = None
self._name_input = None
self._suppress_bot_event = False
def render(self) -> None:
with ui.card().classes('w-full'):
ui.label('⚡ Actions').classes('font-bold text-gray-600')
with ui.row().classes('gap-2'):
ui.button('🔄 Refresh', on_click=self._refresh)
ui.button('📢 Advert', on_click=self._advert)
with ui.row().classes('w-full items-center gap-2'):
self._name_input = ui.input(
label='Device name',
placeholder='Set device name',
).classes('flex-grow')
ui.button('Set', on_click=self._set_name)
self._bot_checkbox = ui.checkbox(
'🤖 BOT',
value=False,
on_change=lambda e: self._on_bot_toggle(e.value),
)
self._bot_checkbox.tooltip('Enabling BOT changes the device name')
ui.label('⚠️ BOT changes device name').classes(
'text-xs text-amber-500'
)
def update(self, data: Dict) -> None:
"""Update BOT checkbox state from snapshot data."""
if self._bot_checkbox is not None:
desired = data.get('bot_enabled', False)
if self._bot_checkbox.value != desired:
self._suppress_bot_event = True
self._bot_checkbox.value = desired
self._suppress_bot_event = False
def _refresh(self) -> None:
self._put_command({'action': 'refresh'})
def _advert(self) -> None:
self._put_command({'action': 'send_advert'})
def _on_bot_toggle(self, value: bool) -> None:
"""Handle BOT checkbox toggle: update flag and queue name change."""
if self._suppress_bot_event:
return
self._set_bot_enabled(value)
self._put_command({
'action': 'set_device_name',
'bot_enabled': value,
})
def _set_name(self) -> None:
"""Send an explicit device name update."""
if self._name_input is None:
return
name = (self._name_input.value or "").strip()
if not name:
return
self._put_command({
'action': 'set_device_name',
'name': name,
})

313
meshcore_gui/gui/panels/contacts_panel.py Normal file
View File

@@ -0,0 +1,313 @@
"""Contacts panel — list of known mesh nodes with click-to-DM."""
from typing import Callable, Dict, Optional
from nicegui import ui
from meshcore_gui.gui.constants import TYPE_ICONS, TYPE_NAMES
from meshcore_gui.services.contact_cleaner import ContactCleanerService
from meshcore_gui.services.pin_store import PinStore
class ContactsPanel:
"""Displays contacts in the left column. Click opens a DM dialog.
Args:
put_command: Callable to enqueue a command dict for the worker.
pin_store: PinStore for persistent pin state.
"""
def __init__(
self,
put_command: Callable[[Dict], None],
pin_store: PinStore,
set_auto_add_enabled: Callable[[bool], None],
on_add_room: Optional[Callable[[str, str, str], None]] = None,
) -> None:
self._put_command = put_command
self._pin_store = pin_store
self._set_auto_add_enabled = set_auto_add_enabled
self._on_add_room = on_add_room
self._cleaner = ContactCleanerService(pin_store)
self._container = None
self._auto_add_checkbox = None
self._last_data: Optional[Dict] = None
def render(self) -> None:
with ui.card().classes('w-full'):
ui.label('👥 Contacts').classes('font-bold text-gray-600')
self._container = ui.column().classes(
'w-full gap-0 max-h-96 overflow-y-auto'
)
with ui.row().classes('w-full gap-2 mt-2 items-center'):
ui.button(
'🧹 Clean up',
on_click=self._open_purge_dialog,
)
self._auto_add_checkbox = ui.checkbox(
'📥 Auto-add',
value=False,
on_change=self._on_auto_add_change,
)
def update(self, data: Dict) -> None:
if not self._container:
return
self._last_data = data
# Sync auto-add checkbox with device state
if self._auto_add_checkbox is not None:
device_state = data.get('auto_add_enabled', False)
if self._auto_add_checkbox.value != device_state:
self._auto_add_checkbox.set_value(device_state)
self._container.clear()
# Sort: pinned contacts first, then alphabetical within each group
contacts_items = list(data['contacts'].items())
contacts_items.sort(
key=lambda item: (
0 if self._pin_store.is_pinned(item[0]) else 1,
item[1].get('adv_name', item[0][:12]).lower(),
)
)
with self._container:
for key, contact in contacts_items:
ctype = contact.get('type', 0)
icon = TYPE_ICONS.get(ctype, '○')
name = contact.get('adv_name', key[:12])
type_name = TYPE_NAMES.get(ctype, '-')
lat = contact.get('adv_lat', 0)
lon = contact.get('adv_lon', 0)
has_loc = lat != 0 or lon != 0
pinned = self._pin_store.is_pinned(key)
tooltip = (
f"{name}\nType: {type_name}\n"
f"Key: {key[:16]}...\nClick to send DM"
)
if has_loc:
tooltip += f"\nLat: {lat:.4f}\nLon: {lon:.4f}"
row_classes = (
'w-full items-center gap-1 py-0 px-1 '
'rounded no-wrap '
)
if pinned:
row_classes += 'bg-yellow-50'
# Outer row: checkbox + clickable contact info
with ui.row().classes(row_classes):
# Pin checkbox — click.stop prevents DM dialog opening
cb = ui.checkbox(
value=pinned,
).props('dense size=xs').on(
'click.stop', lambda e: None,
)
cb.on_value_change(
lambda e, k=key: self._toggle_pin(k)
)
# Clickable area for DM
with ui.row().classes(
'items-center gap-0.5 flex-grow '
'cursor-pointer hover:bg-gray-100 rounded py-0 px-1'
).on(
'click',
lambda e, k=key, n=name, t=ctype: self._on_contact_click(k, n, t),
):
ui.label(icon).classes('text-sm')
ui.label(name[:15]).classes(
'text-sm flex-grow truncate'
).tooltip(tooltip)
ui.label(type_name).classes('text-xs text-gray-500')
loc_icon = '📍' if has_loc else '✖'
loc_cls = 'text-xs w-4 text-center'
if not has_loc:
loc_cls += ' text-red-400'
ui.label(loc_icon).classes(loc_cls)
# ------------------------------------------------------------------
# Pin toggle
# ------------------------------------------------------------------
def _toggle_pin(self, pubkey: str) -> None:
"""Toggle pin state for a contact and refresh the list."""
if self._pin_store.is_pinned(pubkey):
self._pin_store.unpin(pubkey)
else:
self._pin_store.pin(pubkey)
# Re-render with last known data so sort order and visuals update
if self._last_data:
self.update(self._last_data)
# ------------------------------------------------------------------
# Auto-add toggle
# ------------------------------------------------------------------
def _on_auto_add_change(self, e) -> None:
"""Handle auto-add checkbox toggle.
Optimistically updates SharedData and sends the command.
On failure, the command handler rolls back SharedData and the
next GUI update cycle will revert the checkbox.
"""
enabled = e.value
self._set_auto_add_enabled(enabled)
self._put_command({
'action': 'set_auto_add',
'enabled': enabled,
})
# ------------------------------------------------------------------
# Purge unpinned contacts
# ------------------------------------------------------------------
def _open_purge_dialog(self) -> None:
"""Open confirmation dialog for bulk-deleting unpinned contacts."""
try:
if not self._last_data:
ui.notify('No contacts loaded', type='warning')
print("CleanUp: _last_data is None")
return
contacts = self._last_data.get('contacts', {})
if not contacts:
ui.notify('No contacts found', type='warning')
print("CleanUp: contacts dict is empty")
return
print(f"CleanUp: {len(contacts)} contacts found, calculating stats...")
stats = self._cleaner.get_purge_stats(contacts)
print(
f"CleanUp: unpinned={stats.unpinned_count}, "
f"pinned={stats.pinned_count}"
)
if stats.unpinned_count == 0:
ui.notify(
'All contacts are pinned — nothing to remove',
type='info',
)
return
with ui.dialog() as dialog, ui.card().classes('w-96'):
ui.label('🧹 Clean up contacts').classes(
'font-bold text-lg'
)
ui.label(
f'{stats.unpinned_count} contacts will be removed from device.\n'
f'{stats.pinned_count} pinned contacts will be kept.'
).classes('whitespace-pre-line my-2')
delete_history_cb = ui.checkbox(
'Also delete from local history',
).props('dense')
with ui.row().classes('w-full justify-end gap-2 mt-4'):
ui.button('Cancel', on_click=dialog.close).props(
'flat'
)
def confirm_purge():
self._put_command({
'action': 'purge_unpinned',
'pubkeys': stats.unpinned_keys,
'delete_from_history': delete_history_cb.value,
})
dialog.close()
ui.notify(
f'Removing {stats.unpinned_count} '
f'contacts...',
type='info',
)
ui.button(
'Remove',
on_click=confirm_purge,
).classes('bg-red-500 text-white')
dialog.open()
print("CleanUp: dialog opened successfully")
except Exception as exc:
print(f"CleanUp: EXCEPTION — {exc}")
ui.notify(
f'Error opening cleanup dialog: {exc}',
type='negative',
)
# ------------------------------------------------------------------
# Contact click dispatcher
# ------------------------------------------------------------------
def _on_contact_click(self, pubkey: str, name: str, ctype: int) -> None:
"""Route contact click to the appropriate dialog.
Type 3 (Room Server) opens a Room Server add/login dialog.
All other types open the standard DM dialog.
"""
if ctype == 3 and self._on_add_room:
self._open_room_dialog(pubkey, name)
else:
self._open_dm_dialog(pubkey, name)
# ------------------------------------------------------------------
# Room Server dialog
# ------------------------------------------------------------------
def _open_room_dialog(self, pubkey: str, contact_name: str) -> None:
"""Open dialog to add a Room Server panel with password."""
with ui.dialog() as dialog, ui.card().classes('w-96'):
ui.label(f'🏠 Add Room Server: {contact_name}').classes(
'font-bold text-lg'
)
pw_input = ui.input(
placeholder='Room password...',
password=True,
password_toggle_button=True,
).classes('w-full')
with ui.row().classes('w-full justify-end gap-2 mt-4'):
ui.button('Cancel', on_click=dialog.close).props('flat')
def add_and_login():
password = pw_input.value or ''
if self._on_add_room:
self._on_add_room(pubkey, contact_name, password)
dialog.close()
ui.button(
'Add & Login',
on_click=add_and_login,
).classes('bg-blue-500 text-white')
dialog.open()
# ------------------------------------------------------------------
# DM dialog
# ------------------------------------------------------------------
def _open_dm_dialog(self, pubkey: str, contact_name: str) -> None:
with ui.dialog() as dialog, ui.card().classes('w-96'):
ui.label(f'💬 DM to {contact_name}').classes('font-bold text-lg')
msg_input = ui.input(placeholder='Type your message...').classes('w-full')
with ui.row().classes('w-full justify-end gap-2 mt-4'):
ui.button('Cancel', on_click=dialog.close).props('flat')
def send_dm():
text = msg_input.value
if text:
self._put_command({
'action': 'send_dm',
'pubkey': pubkey,
'text': text,
'contact_name': contact_name,
})
dialog.close()
ui.button('Send', on_click=send_dm).classes('bg-blue-500 text-white')
dialog.open()

40
meshcore_gui/gui/panels/device_panel.py Normal file
View File

@@ -0,0 +1,40 @@
"""Device information panel — radio name, frequency, location, firmware."""
from typing import Dict
from nicegui import ui
class DevicePanel:
"""Displays device info in the left column."""
def __init__(self) -> None:
self._label = None
def render(self) -> None:
with ui.card().classes('w-full'):
ui.label('📡 Device').classes('font-bold text-gray-600')
self._label = ui.label('Connecting...').classes(
'text-sm whitespace-pre-line'
)
def update(self, data: Dict) -> None:
if not self._label:
return
lines = []
if data['name']:
lines.append(f"📡 {data['name']}")
if data['public_key']:
lines.append(f"🔑 {data['public_key'][:16]}...")
if data['radio_freq']:
lines.append(f"📻 {data['radio_freq']:.3f} MHz")
lines.append(f"⚙️ SF{data['radio_sf']} / {data['radio_bw']} kHz")
if data['tx_power']:
lines.append(f"⚡ TX: {data['tx_power']} dBm")
if data['adv_lat'] and data['adv_lon']:
lines.append(f"📍 {data['adv_lat']:.4f}, {data['adv_lon']:.4f}")
if data['firmware_version']:
lines.append(f"🏷️ {data['firmware_version']}")
self._label.text = "\n".join(lines) if lines else "Loading..."

88
meshcore_gui/gui/panels/filter_panel.py Normal file
View File

@@ -0,0 +1,88 @@
"""Filter panel — channel filter checkboxes and bot toggle."""
from typing import Callable, Dict, List
from nicegui import ui
class FilterPanel:
"""Channel filter checkboxes and bot on/off toggle.
Args:
set_bot_enabled: Callable to toggle the bot in SharedData.
put_command: Callable to enqueue a command.
"""
def __init__(
self,
set_bot_enabled: Callable[[bool], None],
put_command: Callable[[dict], None],
) -> None:
self._set_bot_enabled = set_bot_enabled
self._put_command = put_command
self._container = None
self._bot_checkbox = None
self._channel_filters: Dict = {}
self._last_channels: List[Dict] = []
self._suppress_bot_event = False
@property
def channel_filters(self) -> Dict:
"""Current filter checkboxes (key: channel idx or ``'DM'``)."""
return self._channel_filters
@property
def last_channels(self) -> List[Dict]:
"""Channel list from the most recent update."""
return self._last_channels
def render(self) -> None:
with ui.card().classes('w-full'):
with ui.row().classes('w-full items-center gap-4 justify-center'):
ui.label('📻 Filter:').classes('text-sm text-gray-600')
self._container = ui.row().classes('gap-4')
def _on_bot_toggle(self, value: bool) -> None:
"""Handle BOT checkbox toggle: update flag and queue name change."""
if self._suppress_bot_event:
return
self._set_bot_enabled(value)
self._put_command({
'action': 'set_device_name',
'bot_enabled': value,
})
def update(self, data: Dict) -> None:
"""Rebuild checkboxes when channel data changes."""
if not self._container or not data['channels']:
return
self._container.clear()
self._channel_filters = {}
with self._container:
self._bot_checkbox = ui.checkbox(
'🤖 BOT',
value=data.get('bot_enabled', False),
on_change=lambda e: self._on_bot_toggle(e.value),
)
self._bot_checkbox.tooltip('Enabling BOT changes the device name')
ui.label('⚠️ BOT changes device name').classes(
'text-xs text-amber-500'
)
ui.label('│').classes('text-gray-300')
cb_dm = ui.checkbox('DM', value=True)
self._channel_filters['DM'] = cb_dm
for ch in data['channels']:
cb = ui.checkbox(f"[{ch['idx']}] {ch['name']}", value=True)
self._channel_filters[ch['idx']] = cb
self._last_channels = data['channels']
if self._bot_checkbox is not None:
desired = data.get('bot_enabled', False)
if self._bot_checkbox.value != desired:
self._suppress_bot_event = True
self._bot_checkbox.value = desired
self._suppress_bot_event = False

59
meshcore_gui/gui/panels/input_panel.py Normal file
View File

@@ -0,0 +1,59 @@
"""Input panel — message input field, channel selector and send button."""
from typing import Callable, Dict, List
from nicegui import ui
class InputPanel:
"""Message composition panel in the centre column.
Args:
put_command: Callable to enqueue a command dict for the worker.
"""
def __init__(self, put_command: Callable[[Dict], None]) -> None:
self._put_command = put_command
self._msg_input = None
self._channel_select = None
@property
def channel_select(self):
"""Expose channel_select so FilterPanel can update its options."""
return self._channel_select
def render(self) -> None:
with ui.card().classes('w-full'):
with ui.row().classes('w-full items-center gap-2'):
self._msg_input = ui.input(
placeholder='Message...'
).classes('flex-grow')
self._channel_select = ui.select(
options={0: '[0] Public'}, value=0
).classes('w-32')
ui.button(
'Send', on_click=self._send_message
).classes('bg-blue-500 text-white')
def update_channel_options(self, channels: List[Dict]) -> None:
"""Update the channel dropdown options."""
if not self._channel_select or not channels:
return
opts = {ch['idx']: f"[{ch['idx']}] {ch['name']}" for ch in channels}
self._channel_select.options = opts
if self._channel_select.value not in opts:
self._channel_select.value = list(opts.keys())[0]
self._channel_select.update()
def _send_message(self) -> None:
text = self._msg_input.value
channel = self._channel_select.value
if text:
self._put_command({
'action': 'send_message',
'channel': channel,
'text': text,
})
self._msg_input.value = ''

202
meshcore_gui/gui/panels/map_panel.py Normal file
View File

@@ -0,0 +1,202 @@
"""Map panel — browser-managed Leaflet map hosted inside NiceGUI."""
from __future__ import annotations
import json
from uuid import uuid4
from typing import Dict
from nicegui import ui
from meshcore_gui.services.map_snapshot_service import MapSnapshotService
class MapPanel:
"""Interactive map panel hosted by NiceGUI and rendered by Leaflet."""
_persisted_theme_mode = 'auto'
def __init__(self) -> None:
self._map_theme_mode = self.__class__._persisted_theme_mode # auto | dark | light
self._ui_dark = True
self._theme_toggle = None
self._container_id = f'meshcore-leaflet-map-{uuid4().hex}'
self._snapshot_service = MapSnapshotService()
self._has_contacts = False
self._has_device = False
@property
def has_markers(self) -> bool:
"""Return whether the last rendered snapshot contained contacts."""
return self._has_contacts
def render(self) -> None:
"""Render the card and inject the browser-side Leaflet container."""
self._inject_assets()
with ui.card().classes('w-full'):
with ui.row().classes('w-full items-center justify-between'):
ui.label('🗺️ Map').classes('font-bold text-gray-600')
with ui.row().classes('items-center gap-2'):
ui.label('Theme').classes('text-xs text-gray-500')
self._theme_toggle = ui.toggle(
{'auto': 'Auto', 'dark': 'Dark', 'light': 'Light'},
value=self._map_theme_mode,
on_change=lambda e: self._set_map_theme_mode(e.value),
).props('dense')
ui.button('Center on Device', on_click=self._center_on_device)
ui.html(
f'<div id="{self._container_id}" class="meshcore-leaflet-host w-full h-72"></div>'
).classes('w-full h-72')
self._dispatch_to_browser(snapshot={'__command__': 'ensure_map'})
self._apply_theme_only()
def set_ui_dark_mode(self, value: bool | None) -> None:
"""Update the map theme when the NiceGUI dark mode changes."""
self._ui_dark = bool(value) if value is not None else True
if self._map_theme_mode == 'auto':
self._apply_theme_only()
def _set_map_theme_mode(self, mode: str) -> None:
"""Apply a new theme mode without recreating the Leaflet map."""
if mode not in ('auto', 'dark', 'light'):
return
self._map_theme_mode = mode
self.__class__._persisted_theme_mode = mode
self._apply_theme_only()
def _apply_theme_only(self) -> None:
"""Push only the effective theme to the browser map runtime."""
theme = self._snapshot_service.resolve_theme(
self._map_theme_mode,
self._ui_dark,
)
self._dispatch_to_browser(theme=theme)
def _center_on_device(self) -> None:
"""Center the browser map on the already-rendered device marker."""
if not self._has_device:
return
self._dispatch_to_browser(snapshot={'__command__': 'center_on_device'})
def update(self, data: Dict) -> None:
"""Send the latest compact map snapshot to the browser."""
snapshot = self._snapshot_service.build_snapshot(
data=data,
theme_mode=self._map_theme_mode,
ui_dark=self._ui_dark,
force_center=bool(data.get('force_center', False)),
)
payload = snapshot.to_dict()
self._has_contacts = bool(payload['contacts'])
self._has_device = payload['device'] is not None
# Theme updates are sent over a dedicated channel. Regular data snapshots
# must never carry theme state, otherwise the 500 ms refresh loop can
# overwrite a freshly selected browser theme with an older/default value.
payload.pop('theme', None)
self._dispatch_to_browser(snapshot=payload)
def _dispatch_to_browser(
self,
snapshot: Dict | None = None,
theme: str | None = None,
) -> None:
"""Send a boot/apply request to the browser runtime."""
command = (
'window.MeshCoreLeafletBoot && '
f'window.MeshCoreLeafletBoot({json.dumps(self._container_id)}, '
f'{json.dumps(snapshot)}, {json.dumps(theme)});'
)
ui.run_javascript(command)
@staticmethod
def _inject_assets() -> None:
"""Load Leaflet assets and the custom runtime exactly once per page."""
ui.add_head_html(
r'''
<script>
(function () {
const ASSET_STATE = window.__meshcoreLeafletAssets = window.__meshcoreLeafletAssets || {
panelRequested: false,
};
function ensureStylesheet(id, href) {
if (document.getElementById(id)) {
return;
}
const link = document.createElement('link');
link.id = id;
link.rel = 'stylesheet';
link.href = href;
document.head.appendChild(link);
}
function ensureScript(id, src, onload) {
const existing = document.getElementById(id);
if (existing) {
if (onload) {
if (existing.dataset.loaded === 'true') {
onload();
} else {
existing.addEventListener('load', onload, { once: true });
}
}
return;
}
const script = document.createElement('script');
script.id = id;
script.src = src;
script.async = false;
script.addEventListener('load', function () {
script.dataset.loaded = 'true';
if (onload) {
onload();
}
}, { once: true });
document.head.appendChild(script);
}
function ensurePanelRuntime() {
if (ASSET_STATE.panelRequested) {
return;
}
ASSET_STATE.panelRequested = true;
ensureScript(
'meshcore-leaflet-panel-js',
'/static/leaflet_map_panel.js'
);
}
ensureStylesheet(
'meshcore-leaflet-vendor-css',
'https://unpkg.com/leaflet@1.9.4/dist/leaflet.css'
);
ensureStylesheet(
'meshcore-leaflet-markercluster-css',
'https://unpkg.com/leaflet.markercluster@1.5.3/dist/MarkerCluster.css'
);
ensureStylesheet(
'meshcore-leaflet-markercluster-default-css',
'https://unpkg.com/leaflet.markercluster@1.5.3/dist/MarkerCluster.Default.css'
);
ensureStylesheet(
'meshcore-leaflet-panel-css',
'/static/leaflet_map_panel.css'
);
ensureScript(
'meshcore-leaflet-vendor-js',
'https://unpkg.com/leaflet@1.9.4/dist/leaflet.js',
function () {
ensureScript(
'meshcore-leaflet-markercluster-js',
'https://unpkg.com/leaflet.markercluster@1.5.3/dist/leaflet.markercluster.js',
ensurePanelRuntime
);
}
);
})();
</script>
'''
)

264
meshcore_gui/gui/panels/messages_panel.py Normal file
View File

@@ -0,0 +1,264 @@
"""Messages panel — filtered message display with channel selection and message input."""
from typing import Callable, Dict, List, Set
from nicegui import ui
from meshcore_gui.core.models import Message
from meshcore_gui.gui.constants import resolve_contact_icon
class MessagesPanel:
"""Displays filtered messages with channel selection and message input.
Channel filtering is driven by the drawer submenu via
:meth:`set_active_channel`. The message input, channel selector
and send button appear below the message list.
Args:
put_command: Callable to enqueue a command dict for the worker.
"""
def __init__(self, put_command: Callable[[Dict], None]) -> None:
self._put_command = put_command
self._container = None
self._channel_filters: Dict = {}
self._last_channels: List[Dict] = []
self._msg_input = None
self._channel_select = None
self._last_fingerprint = None # skip rebuild when unchanged
# Active channel set by drawer submenu (None = all)
self._active_channel = None
self._channel_label = None
# -- Properties (same as FilterPanel originals) --------------------
@property
def channel_filters(self) -> Dict:
"""Current filter checkboxes (key: channel idx or ``'DM'``)."""
return self._channel_filters
@property
def last_channels(self) -> List[Dict]:
"""Channel list from the most recent update."""
return self._last_channels
# -- Active channel (set by dashboard submenu) ---------------------
def set_active_channel(self, channel) -> None:
"""Set the active channel filter from the drawer submenu.
Args:
channel: None for all messages, 'DM' for DM only,
or int for a specific channel index.
"""
self._active_channel = channel
self._last_fingerprint = None # force rebuild on next update
# Update the header label
if self._channel_label:
if channel is None:
self._channel_label.text = '\U0001f4ac Messages — All'
elif channel == 'DM':
self._channel_label.text = '\U0001f4ac Messages — DM'
else:
# Find channel name from last_channels
name = str(channel)
for ch in self._last_channels:
if ch['idx'] == channel:
name = f"[{ch['idx']}] {ch['name']}"
break
self._channel_label.text = f'\U0001f4ac Messages — {name}'
# -- Render --------------------------------------------------------
def render(self) -> None:
with ui.card().classes('w-full'):
# Header row: Messages label with active channel indicator
with ui.row().classes('w-full items-center gap-2'):
self._channel_label = ui.label(
'\U0001f4ac Messages — All'
).classes('font-bold text-gray-600')
# Message container
self._container = ui.column().classes(
'w-full h-40 overflow-y-auto gap-0 text-sm font-mono '
'bg-gray-50 p-2 rounded'
)
# Send message row (moved from InputPanel)
with ui.row().classes('w-full items-center gap-2'):
self._msg_input = ui.input(
placeholder='Message...'
).classes('flex-grow')
self._channel_select = ui.select(
options={0: '[0] Public'}, value=0
).classes('w-32')
ui.button(
'Send', on_click=self._send_message
).classes('bg-blue-500 text-white')
# -- Filter data update (keeps channel list up to date) ------------
def update_filters(self, data: Dict) -> None:
"""Update channel data when channels change.
Note: filter checkboxes have been replaced by drawer submenu
selection. This method now only updates the internal channel
list used for display and the channel_filters compatibility
dict.
"""
if not data['channels']:
return
self._last_channels = data['channels']
# Update the header label if active channel is set to a channel idx
if self._active_channel is not None and self._active_channel != 'DM':
self.set_active_channel(self._active_channel)
# -- Channel selector (moved from InputPanel) ----------------------
def update_channel_options(self, channels: List[Dict]) -> None:
"""Update the channel dropdown options.
Includes an equality check to avoid sending redundant updates
to the NiceGUI client on every 500 ms timer tick.
"""
if not self._channel_select or not channels:
return
opts = {ch['idx']: f"[{ch['idx']}] {ch['name']}" for ch in channels}
if self._channel_select.options == opts:
return # unchanged — skip DOM update
self._channel_select.options = opts
if self._channel_select.value not in opts:
self._channel_select.value = list(opts.keys())[0]
self._channel_select.update()
# -- Send message (moved from InputPanel) --------------------------
def _send_message(self) -> None:
text = self._msg_input.value
channel = self._channel_select.value
if text:
self._put_command({
'action': 'send_message',
'channel': channel,
'text': text,
})
self._msg_input.value = ''
# -- Message display -----------------------------------------------
@staticmethod
def _is_room_message(msg: Message, room_pubkeys: Set[str]) -> bool:
"""Return True if *msg* belongs to a Room Server.
Matches when the message's ``sender_pubkey`` prefix-matches
any tracked room pubkey (same logic as RoomServerPanel).
"""
if not msg.sender_pubkey or not room_pubkeys:
return False
for rpk in room_pubkeys:
if (msg.sender_pubkey.startswith(rpk[:16])
or rpk.startswith(msg.sender_pubkey[:16])):
return True
return False
def update(
self,
data: Dict,
channel_filters: Dict,
last_channels: List[Dict],
room_pubkeys: Set[str] | None = None,
) -> None:
"""Refresh messages applying current filter state.
Filtering is driven by ``_active_channel`` (set via drawer
submenu). The ``channel_filters`` and ``last_channels``
parameters are kept for API compatibility but are not used
when ``_active_channel`` is set.
Args:
data: Snapshot dict from SharedData.
channel_filters: ``{channel_idx: checkbox, 'DM': checkbox}``
from filter checkboxes (legacy, unused when
_active_channel is set).
last_channels: Channel list from filter state.
room_pubkeys: Pubkeys of Room Servers to exclude from
the general message view (shown in
RoomServerPanel instead).
"""
if not self._container:
return
room_pks = room_pubkeys or set()
channel_names = {ch['idx']: ch['name'] for ch in last_channels}
contacts = data.get('contacts', {})
messages: List[Message] = data['messages']
# Apply filters
filtered = []
for orig_idx, msg in enumerate(messages):
# Skip room server messages (shown in RoomServerPanel)
if self._is_room_message(msg, room_pks):
continue
# Apply active channel filter (from drawer submenu)
if self._active_channel is not None:
if self._active_channel == 'DM':
# Show only DM messages (channel is None)
if msg.channel is not None:
continue
else:
# Show only messages for specific channel index
if msg.channel != self._active_channel:
continue
else:
# No active channel filter (ALL) — use checkbox filters
# as fallback for backwards compatibility
if msg.channel is None:
if channel_filters.get('DM') and not channel_filters['DM'].value:
continue
else:
if msg.channel in channel_filters and not channel_filters[msg.channel].value:
continue
filtered.append((orig_idx, msg))
# Rebuild only when content changed
fingerprint = tuple((orig_idx, id(msg)) for orig_idx, msg in filtered)
if fingerprint == self._last_fingerprint:
return
self._last_fingerprint = fingerprint
self._container.clear()
with self._container:
# Hide channel tag when viewing a specific channel/DM
hide_ch = self._active_channel is not None
for orig_idx, msg in reversed(filtered[-50:]):
sender_icon = resolve_contact_icon(
contacts,
pubkey=msg.sender_pubkey,
name=msg.sender,
fallback_type=1 if msg.direction == 'out' else None,
)
line = msg.format_line(
channel_names,
show_channel=not hide_ch,
sender_prefix=f'{sender_icon} ',
)
ui.label(line).classes(
'text-xs leading-tight cursor-pointer '
'hover:bg-blue-50 rounded px-1'
).on('click', lambda e, i=orig_idx: self._open_route(i))
@staticmethod
def _open_route(msg_index: int) -> None:
ui.navigate.to(f'/route/{msg_index}')

463
meshcore_gui/gui/panels/room_server_panel.py Normal file
View File

@@ -0,0 +1,463 @@
"""Room Server panel — per-room messaging with login and password storage."""
from typing import Callable, Dict, List, Set
from nicegui import ui
from meshcore_gui.core.models import Message
from meshcore_gui.services.room_password_store import RoomPasswordStore
class RoomServerPanel:
"""Displays one card per configured Room Server in the centre column.
Each card contains a password field, login/logout button, message
display and message input. Cards are created by calling
:meth:`add_room` (triggered from ContactsPanel when the user clicks
a type-3 contact).
Args:
put_command: Callable to enqueue a command dict for the worker.
room_password_store: Persistent store for room passwords.
"""
def __init__(
self,
put_command: Callable[[Dict], None],
room_password_store: RoomPasswordStore,
) -> None:
self._put_command = put_command
self._store = room_password_store
# Outer container that holds all room cards
self._container = None
# Per-room UI state keyed by pubkey
self._room_cards: Dict[str, Dict] = {}
# Login state tracked locally (not persisted)
self._logged_in: Set[str] = set()
# ------------------------------------------------------------------
# Render — restore persisted rooms on startup
# ------------------------------------------------------------------
def render(self) -> None:
"""Build the outer container and restore persisted rooms."""
self._container = ui.column().classes('w-full gap-2')
with self._container:
for entry in self._store.get_rooms():
self._render_room_card(
entry.pubkey, entry.name, entry.password,
)
# ------------------------------------------------------------------
# Public — add a room (called from Dashboard/ContactsPanel)
# ------------------------------------------------------------------
def add_room(self, pubkey: str, name: str, password: str) -> None:
"""Add a new Room Server card and persist it.
If the room is already shown, updates password and re-logins.
Args:
pubkey: Full public key (hex string).
name: Display name.
password: Room password.
"""
# Persist
self._store.add_room(pubkey, name, password)
if pubkey in self._room_cards:
# Already visible — update password field and re-login
card_state = self._room_cards[pubkey]
card_state['password'].value = password
self._login_room(card_state, pubkey)
return
# Create new card
if not self._container:
return
with self._container:
self._render_room_card(pubkey, name, password)
# Auto-login
if pubkey in self._room_cards:
self._login_room(self._room_cards[pubkey], pubkey)
def get_room_pubkeys(self) -> Set[str]:
"""Return the set of all room server pubkeys currently tracked.
Used by :class:`MessagesPanel` to filter out room messages from
the general DM view.
"""
return set(self._room_cards.keys())
# ------------------------------------------------------------------
# Update (called from dashboard timer)
# ------------------------------------------------------------------
def update(self, data: Dict) -> None:
"""Refresh messages and login state for each room card.
Args:
data: Snapshot dict from SharedData.
"""
if not self._container:
return
# Process room login state changes from the worker
login_states: Dict = data.get('room_login_states', {})
self._apply_login_states(login_states)
# Room messages from archive cache (keyed by 12-char pubkey prefix)
room_messages: Dict = data.get('room_messages', {})
# Live messages from current session's rolling buffer
live_messages: List[Message] = data.get('messages', [])
for pubkey, card_state in self._room_cards.items():
self._update_room_messages(
pubkey, card_state, room_messages, live_messages,
)
# ------------------------------------------------------------------
# Internal — login state feedback from worker
# ------------------------------------------------------------------
def _apply_login_states(self, login_states: Dict) -> None:
"""Apply server-confirmed login states to room cards.
Called every update tick. Matches login_states (keyed by
pubkey prefix from the device packet) against known room cards
(keyed by full pubkey) using prefix matching.
Args:
login_states: ``{pubkey_prefix: {'state': str, 'detail': str}}``
from SharedData.
"""
for pubkey, card_state in self._room_cards.items():
# Find matching login state (prefix match)
matched_state = None
for prefix, state_info in login_states.items():
if pubkey.startswith(prefix) or prefix.startswith(pubkey[:16]):
matched_state = state_info
break # Use first match only; prevents stale keys overriding
if matched_state is None:
continue
state = matched_state.get('state', '')
if state == 'ok' and pubkey not in self._logged_in:
# Server confirmed login
self._logged_in.add(pubkey)
card_state['status'].text = (
'✅ Logged in — history arriving over RF…'
)
card_state['pw_row'].set_visibility(False)
card_state['logout_btn'].set_visibility(True)
card_state['login_btn'].enable()
card_state['msg_input'].enable()
card_state['send_btn'].enable()
elif state == 'fail' and pubkey not in self._logged_in:
# Login failed or timed out — revert to login form
detail = matched_state.get('detail', 'Unknown error')
card_state['status'].text = f'❌ Login failed: {detail}'
card_state['pw_row'].set_visibility(True)
card_state['logout_btn'].set_visibility(False)
card_state['login_btn'].enable()
card_state['msg_input'].disable()
card_state['send_btn'].disable()
elif state == 'pending':
card_state['status'].text = '⏳ Logging in…'
elif state == 'logged_out' and pubkey in self._logged_in:
# Server confirmed logout — ensure UI is fully reset
# (catches edge cases where _logout_room UI update was
# overridden by a stale 'ok' state from previous tick)
self._logged_in.discard(pubkey)
card_state['status'].text = '⏳ Not logged in'
card_state['pw_row'].set_visibility(True)
card_state['logout_btn'].set_visibility(False)
card_state['login_btn'].enable()
card_state['msg_input'].disable()
card_state['send_btn'].disable()
# ------------------------------------------------------------------
# Internal — single room card
# ------------------------------------------------------------------
def _render_room_card(
self,
pubkey: str,
name: str,
password: str,
) -> None:
"""Render a single Room Server card.
Args:
pubkey: Public key of the room.
name: Display name.
password: Stored password.
"""
card_state: Dict = {}
is_logged_in = pubkey in self._logged_in
with ui.card().classes('w-full') as card:
card_state['card'] = card
# Header row: title + remove button
with ui.row().classes('w-full items-center justify-between'):
card_state['title'] = ui.label(
f'🏠 Room Server: {name}'
).classes('font-bold text-gray-600')
ui.button(
'✕',
on_click=lambda e, pk=pubkey: self._remove_room(pk),
).props('flat dense round size=sm')
# Password + Login row (hidden after login)
card_state['pw_row'] = ui.row().classes('w-full items-center gap-2')
with card_state['pw_row']:
card_state['password'] = ui.input(
placeholder='Password...',
value=password,
password=True,
password_toggle_button=True,
).classes('flex-grow')
card_state['login_btn'] = ui.button(
'Login',
on_click=lambda e, pk=pubkey: self._on_login_click(pk),
).classes('bg-blue-500 text-white')
# Logout button (hidden before login)
card_state['logout_btn'] = ui.button(
'Logout',
on_click=lambda e, pk=pubkey: self._on_login_click(pk),
).classes('bg-red-500 text-white')
# Set initial visibility
if is_logged_in:
card_state['pw_row'].set_visibility(False)
card_state['logout_btn'].set_visibility(True)
else:
card_state['pw_row'].set_visibility(True)
card_state['logout_btn'].set_visibility(False)
# Status label
card_state['status'] = ui.label(
'✅ Logged in' if is_logged_in
else '⏳ Not logged in'
).classes('text-xs text-gray-500')
# Messages container (scrollable)
card_state['msg_container'] = ui.column().classes(
'w-full h-32 overflow-y-auto gap-0 text-sm font-mono '
'bg-gray-50 p-2 rounded'
)
# Send row
with ui.row().classes('w-full items-center gap-2'):
card_state['msg_input'] = ui.input(
placeholder='Message...',
).classes('flex-grow')
card_state['send_btn'] = ui.button(
'Send',
on_click=lambda e, pk=pubkey: self._send_room_message(pk),
).classes('bg-blue-500 text-white')
# Disable send controls if not logged in
if not is_logged_in:
card_state['msg_input'].disable()
card_state['send_btn'].disable()
self._room_cards[pubkey] = card_state
# ------------------------------------------------------------------
# Internal — actions
# ------------------------------------------------------------------
def _on_login_click(self, pubkey: str) -> None:
"""Dispatch login or logout based on current state."""
card_state = self._room_cards.get(pubkey)
if not card_state:
return
if pubkey in self._logged_in:
self._logout_room(card_state, pubkey)
else:
self._login_room(card_state, pubkey)
def _login_room(self, card_state: Dict, pubkey: str) -> None:
"""Send login command to a Room Server.
Sets the UI to 'pending' state. The actual logged-in state
is updated later in :meth:`update` when the worker reports
LOGIN_SUCCESS via ``room_login_states`` in SharedData.
"""
password = card_state['password'].value or ''
name = card_state['title'].text.replace('🏠 Room Server: ', '')
# Persist password update
self._store.update_password(pubkey, password)
# Send login command via worker
self._put_command({
'action': 'login_room',
'pubkey': pubkey,
'password': password,
'room_name': name,
})
# Pending UI update — real state comes from SharedData
card_state['status'].text = '⏳ Logging in…'
card_state['login_btn'].disable()
ui.notify(f'Logging in to {name}...', type='info')
def _logout_room(self, card_state: Dict, pubkey: str) -> None:
"""Logout from a Room Server.
Sends a logout command via worker so the companion radio stops
keep-alive pings and the room server deregisters the client.
This ensures a clean ``sync_since`` reset on re-login.
"""
name = card_state['title'].text.replace('🏠 Room Server: ', '')
# Send logout command to companion radio / room server
self._put_command({
'action': 'logout_room',
'pubkey': pubkey,
'room_name': name,
})
self._logged_in.discard(pubkey)
# Clear messages — user should not see room history after logout
msg_container = card_state.get('msg_container')
if msg_container:
msg_container.clear()
card_state['status'].text = '⏳ Not logged in'
card_state['pw_row'].set_visibility(True)
card_state['logout_btn'].set_visibility(False)
card_state['login_btn'].enable()
card_state['msg_input'].disable()
card_state['send_btn'].disable()
ui.notify(f'Logged out from {name}', type='info')
def _send_room_message(self, pubkey: str) -> None:
"""Send a message to a Room Server."""
card_state = self._room_cards.get(pubkey)
if not card_state:
return
if pubkey not in self._logged_in:
ui.notify('Not logged in', type='warning')
return
msg_input = card_state.get('msg_input')
if not msg_input or not msg_input.value:
return
text = msg_input.value
name = card_state['title'].text.replace('🏠 Room Server: ', '')
self._put_command({
'action': 'send_room_msg',
'pubkey': pubkey,
'text': text,
'room_name': name,
})
msg_input.value = ''
def _remove_room(self, pubkey: str) -> None:
"""Remove a Room Server card and its stored data."""
self._store.remove_room(pubkey)
self._logged_in.discard(pubkey)
card_state = self._room_cards.pop(pubkey, None)
if card_state and card_state.get('card'):
self._container.remove(card_state['card'])
# ------------------------------------------------------------------
# Internal — message display
# ------------------------------------------------------------------
def _update_room_messages(
self,
pubkey: str,
card_state: Dict,
room_messages: Dict,
live_messages: List[Message],
) -> None:
"""Update the message display for a single room card.
Only shows messages when logged in. Merges archived room
messages (from ``room_messages`` cache) with live messages
from the current session. Displays newest-first so the most
recent message is always visible at the top without scrolling.
Args:
pubkey: Full public key of the room server.
card_state: UI state dict for this room card.
room_messages: ``{12-char-prefix: [Message, …]}`` from archive cache.
live_messages: Current session's rolling message buffer.
"""
msg_container = card_state.get('msg_container')
if not msg_container:
return
# Login gate — show nothing before login
if pubkey not in self._logged_in:
msg_container.clear()
return
norm = pubkey[:12]
# 1. Archived room messages (loaded from disk cache)
archived: List[Message] = room_messages.get(norm, [])
# 2. Live room messages from rolling buffer (current session)
live_room: List[Message] = []
for msg in live_messages:
if not msg.sender_pubkey:
continue
if (msg.sender_pubkey.startswith(norm)
or norm.startswith(msg.sender_pubkey[:12])):
live_room.append(msg)
# 3. Merge and dedup (archive may already contain live messages
# because add_message() appends to both)
seen = set()
merged: List[Message] = []
for msg in archived + live_room:
key = (msg.time, msg.text)
if key not in seen:
seen.add(key)
merged.append(msg)
# 4. Take last 30 then reverse: newest message at top
display = merged[-30:]
display.reverse()
msg_container.clear()
with msg_container:
for msg in display:
direction = '→' if msg.direction == 'out' else '←'
sender = msg.sender or '?'
line = f"{msg.time} {direction} {sender}: {msg.text}"
ui.label(line).classes(
'text-xs leading-tight px-1'
)

97
meshcore_gui/gui/panels/rxlog_panel.py Normal file
View File

@@ -0,0 +1,97 @@
"""RX log panel — table of recently received packets."""
from typing import Dict, List
from nicegui import ui
from meshcore_gui.core.models import RxLogEntry
class RxLogPanel:
"""RX log table in the right column."""
def __init__(self) -> None:
self._table = None
# ------------------------------------------------------------------
# Helpers
# ------------------------------------------------------------------
@staticmethod
def _build_path(entry: RxLogEntry) -> str:
"""Build a display path: Sender → [repeaters →] Receiver.
Falls back gracefully when sender or receiver is unknown.
"""
parts: list = []
if entry.sender:
parts.append(entry.sender)
# Repeater names (resolved or raw hex)
if entry.path_names:
parts.extend(entry.path_names)
if entry.receiver:
parts.append(entry.receiver)
return ' → '.join(parts) if parts else '-'
# ------------------------------------------------------------------
# Render / Update
# ------------------------------------------------------------------
def render(self) -> None:
with ui.card().classes('w-full flex-grow'):
ui.label('📊 RX Log').classes('font-bold text-gray-600')
self._table = ui.table(
columns=[
{'name': 'time', 'label': 'Time', 'field': 'time',
'align': 'left'},
{'name': 'snr', 'label': 'SNR', 'field': 'snr',
'align': 'right'},
{'name': 'rssi', 'label': 'RSSI', 'field': 'rssi',
'align': 'right'},
{'name': 'type', 'label': 'Type', 'field': 'type',
'align': 'left'},
{'name': 'hops', 'label': 'Hops', 'field': 'hops',
'align': 'right'},
{'name': 'path', 'label': 'Path', 'field': 'path',
'align': 'left',
'classes': 'rxlog-path-cell',
'headerClasses': 'rxlog-path-header'},
],
rows=[],
).props('dense flat').classes('w-full text-xs h-40 overflow-y-auto')
# Constrain the path column so it cannot push the table
# wider than the parent card.
ui.add_css('''
.rxlog-path-cell {
max-width: 160px;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.rxlog-path-header {
max-width: 160px;
}
''')
def update(self, data: Dict) -> None:
if not self._table:
return
entries: List[RxLogEntry] = data['rx_log'][:20]
rows = [
{
'time': e.time,
'snr': f"{e.snr:.1f}",
'rssi': f"{e.rssi:.0f}",
'type': e.payload_type,
'hops': str(e.hops),
'path': self._build_path(e),
}
for e in entries
]
self._table.rows = rows
self._table.update()

488
meshcore_gui/gui/route_page.py Normal file
View File

@@ -0,0 +1,488 @@
"""
Route visualization page for MeshCore GUI.
Standalone NiceGUI page that shows a Leaflet map with the message
route, a hop count summary, and a details table.
v4.1 changes
~~~~~~~~~~~~~
- Uses :class:`~meshcore_gui.models.Message` and
:class:`~meshcore_gui.models.RouteNode` instead of plain dicts.
"""
import json
from typing import Dict, List, Optional
from uuid import uuid4
from nicegui import ui
from meshcore_gui.gui.constants import (
get_type_display,
get_type_icon,
get_type_label,
resolve_contact_icon,
)
from meshcore_gui.gui.dashboard import _DOMCA_HEAD
from meshcore_gui.config import DEFAULT_MAP_CENTER, DEFAULT_MAP_ZOOM
from meshcore_gui.core.models import Message, RouteNode
from meshcore_gui.services.route_builder import RouteBuilder
from meshcore_gui.core.protocols import SharedDataReadAndLookup
_ROUTE_MAP_ASSETS = r"""
<script>
(function () {
if (window.__meshcoreLeafletAssetsRequested) {
return;
}
window.__meshcoreLeafletAssetsRequested = true;
function ensureStylesheet(id, href) {
if (document.getElementById(id)) {
return;
}
const link = document.createElement('link');
link.id = id;
link.rel = 'stylesheet';
link.href = href;
document.head.appendChild(link);
}
function ensureScript(id, src, onload) {
const existing = document.getElementById(id);
if (existing) {
if (onload) {
if (existing.dataset.loaded === 'true') {
onload();
} else {
existing.addEventListener('load', onload, { once: true });
}
}
return;
}
const script = document.createElement('script');
script.id = id;
script.src = src;
script.async = false;
script.addEventListener('load', function () {
script.dataset.loaded = 'true';
if (onload) {
onload();
}
}, { once: true });
document.head.appendChild(script);
}
ensureStylesheet('meshcore-leaflet-vendor-css', 'https://unpkg.com/leaflet@1.9.4/dist/leaflet.css');
ensureStylesheet('meshcore-leaflet-markercluster-css', 'https://unpkg.com/leaflet.markercluster@1.5.3/dist/MarkerCluster.css');
ensureStylesheet('meshcore-leaflet-markercluster-default-css', 'https://unpkg.com/leaflet.markercluster@1.5.3/dist/MarkerCluster.Default.css');
ensureStylesheet('meshcore-leaflet-panel-css', '/static/leaflet_map_panel.css');
ensureScript('meshcore-leaflet-vendor-js', 'https://unpkg.com/leaflet@1.9.4/dist/leaflet.js', function () {
ensureScript('meshcore-leaflet-markercluster-js', 'https://unpkg.com/leaflet.markercluster@1.5.3/dist/leaflet.markercluster.js', function () {
ensureScript('meshcore-leaflet-panel-js', '/static/leaflet_map_panel.js');
});
});
})();
</script>
"""
class RoutePage:
"""
Route visualization page rendered at ``/route/{msg_index}``.
Args:
shared: SharedDataReadAndLookup for data access and contact lookups
"""
def __init__(self, shared: SharedDataReadAndLookup) -> None:
self._shared = shared
self._builder = RouteBuilder(shared)
def render(self, msg_key: str) -> None:
"""Render the route page for a message."""
data = self._shared.get_snapshot()
messages: List[Message] = data['messages']
msg: Optional[Message] = None
try:
idx = int(msg_key)
if 0 <= idx < len(messages):
msg = messages[idx]
except (ValueError, TypeError):
pass
if msg is None and msg_key:
for message in messages:
if message.message_hash and message.message_hash == msg_key:
msg = message
break
if msg is None and msg_key:
archive = data.get('archive')
if archive:
msg_dict = archive.get_message_by_hash(msg_key)
if msg_dict:
msg = Message.from_dict(msg_dict)
if msg is None:
ui.label('❌ Message not found').classes('text-xl p-8')
ui.button('Back to Dashboard', on_click=lambda: ui.navigate.to('/')).classes(
'mt-4'
)
return
route = self._builder.build(msg, data)
route['message'] = msg
ui.page_title(f'Route — {msg.sender or "Unknown"}')
ui.add_head_html(_DOMCA_HEAD)
ui.add_head_html(_ROUTE_MAP_ASSETS)
ui.dark_mode(True)
with ui.header().classes('items-center px-4 py-2 shadow-md'):
ui.button(
icon='arrow_back',
on_click=lambda: ui.navigate.to('/'),
).props('flat round dense color=white').tooltip('Back to Dashboard')
ui.button(
icon='history',
on_click=lambda: ui.navigate.to('/archive'),
).props('flat round dense color=white').tooltip('Back to Archive')
ui.label('🗺️ MeshCore Route').classes(
'text-lg font-bold domca-header-text'
).style("font-family: 'JetBrains Mono', monospace")
ui.space()
ui.label('Route Detail').classes('text-sm opacity-70 domca-header-text')
with ui.column().classes('domca-panel gap-4').style('padding-top: 1rem'):
self._render_message_info(msg, data)
self._render_hop_summary(msg, route)
self._render_map(data, route)
self._render_send_panel(msg, data)
self._render_route_table(msg, data, route)
@staticmethod
def _render_message_info(msg: Message, data: Dict) -> None:
sender = msg.sender or 'Unknown'
direction = '→ Sent' if msg.direction == 'out' else '← Received'
sender_icon = resolve_contact_icon(
data.get('contacts', {}),
pubkey=msg.sender_pubkey,
name=msg.sender,
fallback_type=1 if msg.direction == 'out' else None,
)
ui.label(
f'Message Route — {sender_icon} {sender} ({direction})'
).classes('font-bold text-lg')
ui.label(
f"{msg.time} {sender_icon} {sender}: {msg.text[:120]}"
).classes('text-sm text-gray-600')
@staticmethod
def _render_hop_summary(msg: Message, route: Dict) -> None:
msg_path_len = route['msg_path_len']
path_nodes: List[RouteNode] = route['path_nodes']
resolved_hops = len(path_nodes)
path_source = route.get('path_source', 'none')
expected_repeaters = max(msg_path_len - 1, 0)
with ui.card().classes('w-full'):
with ui.row().classes('items-center gap-4'):
if msg.direction == 'in':
if msg_path_len == 0:
ui.label('📡 Direct (0 hops)').classes(
'text-lg font-bold text-green-600'
)
else:
hop_text = '1 hop' if msg_path_len == 1 else f'{msg_path_len} hops'
ui.label(f'📡 {hop_text}').classes(
'text-lg font-bold text-blue-600'
)
else:
ui.label('📡 Outgoing message').classes(
'text-lg font-bold text-gray-600'
)
if route['snr'] is not None:
ui.label(
f'📶 SNR: {route["snr"]:.1f} dB'
).classes('text-sm text-gray-600')
if expected_repeaters > 0 and resolved_hops > 0:
source_label = (
'from received packet'
if path_source == 'rx_log'
else 'from stored contact route'
)
rpt = 'repeater' if expected_repeaters == 1 else 'repeaters'
ui.label(
f'✅ {resolved_hops} of {expected_repeaters} '
f'{rpt} identified ({source_label})'
).classes('text-xs text-gray-500 mt-1')
elif msg_path_len > 0 and resolved_hops == 0:
ui.label(
f'ℹ️ {msg_path_len} '
f'hop{"s" if msg_path_len != 1 else ""} — '
f'repeater identities not resolved'
).classes('text-xs text-gray-500 mt-1')
@staticmethod
def _render_map(data: Dict, route: Dict) -> None:
"""Render the route map in browser JS using the shared MAP icons."""
with ui.card().classes('w-full'):
payload = RoutePage._build_route_map_payload(data, route)
if not payload['nodes']:
ui.label(
'📍 No location data available for map display'
).classes('text-gray-500 italic p-4')
return
container_id = f'route-map-{uuid4().hex}'
ui.html(
f'<div id="{container_id}" class="w-full h-96 rounded-lg overflow-hidden"></div>'
).classes('w-full')
boot_script = (
'(function bootRouteMap(retries){'
f'const id={json.dumps(container_id)};'
f'const payload={json.dumps(payload, ensure_ascii=False)};'
"if(typeof window.MeshCoreRouteMapBoot==='function'){window.MeshCoreRouteMapBoot(id,payload);return;}"
"if(retries>120){console.error('MeshCoreRouteMapBoot unavailable',{id});return;}"
'window.setTimeout(function(){bootRouteMap(retries+1);},60);'
'})(0);'
)
ui.timer(0.1, lambda script=boot_script: ui.run_javascript(script), once=True)
@staticmethod
def _build_route_map_payload(data: Dict, route: Dict) -> Dict:
"""Build the JS payload for the route map using shared node types."""
nodes = []
sender: RouteNode = route['sender']
if sender and sender.has_location:
nodes.append({
'name': sender.name or 'Unknown',
'lat': sender.lat,
'lon': sender.lon,
'node_type': int(sender.type or 0),
'short_key': sender.pubkey[:2].upper() if sender.pubkey else '-',
'role': get_type_label(sender.type),
})
elif sender is None:
fallback_contact = RoutePage._find_sender_contact(
route['message'],
data.get('contacts', {}),
)
if fallback_contact:
fb_key, fb_contact = fallback_contact
fb_lat = fb_contact.get('adv_lat', 0)
fb_lon = fb_contact.get('adv_lon', 0)
if fb_lat or fb_lon:
fb_type = int(fb_contact.get('type', 0) or 0)
nodes.append({
'name': fb_contact.get('adv_name') or route['message'].sender or 'Unknown',
'lat': fb_lat,
'lon': fb_lon,
'node_type': fb_type,
'short_key': fb_key[:2].upper() if fb_key else '-',
'role': get_type_label(fb_type),
})
for node in route['path_nodes']:
if not node.has_location:
continue
nodes.append({
'name': node.name or 'Unknown',
'lat': node.lat,
'lon': node.lon,
'node_type': int(node.type or 0),
'short_key': node.pubkey[:2].upper() if node.pubkey else '-',
'role': get_type_label(node.type),
})
self_node: RouteNode = route['self_node']
if self_node.has_location:
self_type = int(self_node.type or 1)
nodes.append({
'name': self_node.name or 'Local device',
'lat': self_node.lat,
'lon': self_node.lon,
'node_type': self_type,
'short_key': '-',
'role': get_type_label(self_type),
})
return {
'center': [
data['adv_lat'] or DEFAULT_MAP_CENTER[0],
data['adv_lon'] or DEFAULT_MAP_CENTER[1],
],
'zoom': DEFAULT_MAP_ZOOM,
'nodes': nodes,
}
@staticmethod
def _render_route_table(msg: Message, data: Dict, route: Dict) -> None:
msg_path_len = route['msg_path_len']
path_nodes: List[RouteNode] = route['path_nodes']
resolved_hops = len(path_nodes)
path_source = route.get('path_source', 'none')
with ui.card().classes('w-full'):
ui.label('📋 Route Details').classes('font-bold text-gray-600')
rows = []
sender: RouteNode = route['sender']
if sender:
rows.append({
'hop': 'Start',
'name': sender.name,
'hash': sender.pubkey[:2].upper() if sender.pubkey else '-',
'type': get_type_display(sender.type),
'location': f"{sender.lat:.4f}, {sender.lon:.4f}" if sender.has_location else '-',
'role': f'{get_type_icon(sender.type)} Sender',
})
else:
fallback_contact = RoutePage._find_sender_contact(msg, data.get('contacts', {}))
if fallback_contact:
fb_key, fb_contact = fallback_contact
fb_lat = fb_contact.get('adv_lat', 0)
fb_lon = fb_contact.get('adv_lon', 0)
fb_has_loc = fb_lat != 0 or fb_lon != 0
fb_type = int(fb_contact.get('type', 0) or 0)
rows.append({
'hop': 'Start',
'name': fb_contact.get('adv_name') or msg.sender or 'Unknown',
'hash': fb_key[:2].upper() if fb_key else '-',
'type': get_type_display(fb_type),
'location': f"{fb_lat:.4f}, {fb_lon:.4f}" if fb_has_loc else '-',
'role': f'{get_type_icon(fb_type)} Sender',
})
else:
rows.append({
'hop': 'Start',
'name': msg.sender or 'Unknown',
'hash': msg.sender_pubkey[:2].upper() if msg.sender_pubkey else '-',
'type': get_type_display(0),
'location': '-',
'role': f'{get_type_icon(0)} Sender',
})
for index, node in enumerate(path_nodes):
rows.append({
'hop': str(index + 1),
'name': node.name,
'hash': node.pubkey[:2].upper() if node.pubkey else '-',
'type': get_type_display(node.type),
'location': f"{node.lat:.4f}, {node.lon:.4f}" if node.has_location else '-',
'role': f'{get_type_icon(node.type)} Repeater',
})
if not path_nodes and 0 < msg_path_len < 255:
for index in range(msg_path_len):
rows.append({
'hop': str(index + 1),
'name': '-',
'hash': '-',
'type': get_type_display(0),
'location': '-',
'role': f'{get_type_icon(0)} Repeater',
})
self_node: RouteNode = route['self_node']
self_type = int(self_node.type or 1)
rows.append({
'hop': 'End',
'name': self_node.name,
'hash': '-',
'type': get_type_display(self_type),
'location': f"{self_node.lat:.4f}, {self_node.lon:.4f}" if self_node.has_location else '-',
'role': f'{get_type_icon(self_type)} Receiver' if msg.direction == 'in' else f'{get_type_icon(self_type)} Sender',
})
ui.table(
columns=[
{'name': 'hop', 'label': 'Hop', 'field': 'hop', 'align': 'center'},
{'name': 'role', 'label': 'Role', 'field': 'role'},
{'name': 'name', 'label': 'Name', 'field': 'name'},
{'name': 'hash', 'label': 'ID', 'field': 'hash', 'align': 'center'},
{'name': 'type', 'label': 'Type', 'field': 'type'},
{'name': 'location', 'label': 'Location', 'field': 'location'},
],
rows=rows,
).props('dense flat bordered').classes('w-full')
if msg_path_len == 0 and msg.direction == 'in':
ui.label(
'ℹ️ Direct message — no intermediate hops.'
).classes('text-xs text-gray-400 italic mt-2')
elif path_source == 'rx_log':
ui.label(
'ℹ️ Path extracted from received LoRa packet (RX_LOG). '
'Each ID is the first byte of a node\'s public key.'
).classes('text-xs text-gray-400 italic mt-2')
elif path_source == 'contact_out_path':
ui.label(
'ℹ️ Path from sender\'s stored contact route (out_path). '
'Last known route, not necessarily this message\'s path.'
).classes('text-xs text-gray-400 italic mt-2')
elif msg_path_len > 0 and resolved_hops == 0:
ui.label(
'ℹ️ Repeater identities could not be resolved.'
).classes('text-xs text-gray-400 italic mt-2')
elif msg.direction == 'out':
ui.label(
'ℹ️ Hop information is only available for received messages.'
).classes('text-xs text-gray-400 italic mt-2')
def _render_send_panel(self, msg: Message, data: Dict) -> None:
"""Send widget pre-filled with route acknowledgement message."""
path_hashes = msg.path_hashes
parts = [f"@[{msg.sender or 'Unknown'}] Received in Zwolle path({msg.path_len})"]
if path_hashes:
path_str = '>'.join(h.upper() for h in path_hashes)
parts.append(f'; {path_str}')
prefilled = ''.join(parts)
ch_options = {ch['idx']: f"[{ch['idx']}] {ch['name']}" for ch in data['channels']}
default_ch = data['channels'][0]['idx'] if data['channels'] else 0
with ui.card().classes('w-full'):
ui.label('📤 Reply').classes('font-bold text-gray-600')
with ui.row().classes('w-full items-center gap-2'):
msg_input = ui.input(value=prefilled).classes('flex-grow')
ch_select = ui.select(options=ch_options, value=default_ch).classes('w-32')
def send(inp=msg_input, sel=ch_select):
text = inp.value
if text:
self._shared.put_command({
'action': 'send_message',
'channel': sel.value,
'text': text,
})
inp.value = ''
ui.button('Send', on_click=send).classes('bg-blue-500 text-white')
@staticmethod
def _find_sender_contact(msg: Message, contacts: Dict) -> Optional[tuple]:
"""Defensive fallback: find sender contact data in snapshot."""
if msg.sender_pubkey:
pk_lower = msg.sender_pubkey.lower()
for key, contact in contacts.items():
key_lower = key.lower()
if key_lower.startswith(pk_lower) or pk_lower.startswith(key_lower):
return (key, contact)
if msg.sender:
name_lower = msg.sender.lower()
for key, contact in contacts.items():
adv_name = contact.get('adv_name', '')
if adv_name and adv_name.lower() == name_lower:
return (key, contact)
return None

3
meshcore_gui/services/__init__.py Normal file
View File

@@ -0,0 +1,3 @@
"""
Business logic services — bot, cache, deduplication and route building.
"""

188
meshcore_gui/services/bot.py Normal file
View File

@@ -0,0 +1,188 @@
"""
Keyword-triggered auto-reply bot for MeshCore GUI.
Extracted from SerialWorker to satisfy the Single Responsibility Principle.
The bot listens on a configured channel and replies to messages that
contain recognised keywords.
Open/Closed
~~~~~~~~~~~
New keywords are added via ``BotConfig.keywords`` (data) without
modifying the ``MeshBot`` class (code). Custom matching strategies
can be implemented by subclassing and overriding ``_match_keyword``.
"""
import time
from dataclasses import dataclass, field
from typing import Callable, Dict, List, Optional
from meshcore_gui.config import debug_print
# ==============================================================================
# Bot defaults (previously in config.py)
# ==============================================================================
# Channel indices the bot listens on (must match device channels).
BOT_CHANNELS: frozenset = frozenset({1, 4}) # #test, #bot
# Display name prepended to every bot reply.
BOT_NAME: str = "ZwolsBotje"
# Minimum seconds between two bot replies (prevents reply-storms).
BOT_COOLDOWN_SECONDS: float = 5.0
# Keyword → reply template mapping.
# Available variables: {bot}, {sender}, {snr}, {path}
# The bot checks whether the incoming message text *contains* the keyword
# (case-insensitive). First match wins.
BOT_KEYWORDS: Dict[str, str] = {
'test': '@[{sender}], rcvd | SNR {snr} | {path}',
'ping': 'Pong!',
'help': 'test, ping, help',
}
@dataclass
class BotConfig:
"""Configuration for :class:`MeshBot`.
Attributes:
channels: Channel indices to listen on.
name: Display name prepended to replies.
cooldown_seconds: Minimum seconds between replies.
keywords: Keyword → reply template mapping.
"""
channels: frozenset = field(default_factory=lambda: frozenset(BOT_CHANNELS))
name: str = BOT_NAME
cooldown_seconds: float = BOT_COOLDOWN_SECONDS
keywords: Dict[str, str] = field(default_factory=lambda: dict(BOT_KEYWORDS))
class MeshBot:
"""Keyword-triggered auto-reply bot.
The bot checks incoming messages against a set of keyword → template
pairs. When a keyword is found (case-insensitive substring match,
first match wins), the template is expanded and queued as a channel
message via *command_sink*.
Args:
config: Bot configuration.
command_sink: Callable that enqueues a command dict for the
worker (typically ``shared.put_command``).
enabled_check: Callable that returns ``True`` when the bot is
enabled (typically ``shared.is_bot_enabled``).
"""
def __init__(
self,
config: BotConfig,
command_sink: Callable[[Dict], None],
enabled_check: Callable[[], bool],
) -> None:
self._config = config
self._sink = command_sink
self._enabled = enabled_check
self._last_reply: float = 0.0
def check_and_reply(
self,
sender: str,
text: str,
channel_idx: Optional[int],
snr: Optional[float],
path_len: int,
path_hashes: Optional[List[str]] = None,
) -> None:
"""Evaluate an incoming message and queue a reply if appropriate.
Guards (in order):
1. Bot is enabled (checkbox in GUI).
2. Message is on the configured channel.
3. Sender is not the bot itself.
4. Sender name does not end with ``'Bot'`` (prevent loops).
5. Cooldown period has elapsed.
6. Message text contains a recognised keyword.
"""
# Guard 1: enabled?
if not self._enabled():
return
# Guard 2: correct channel?
if channel_idx not in self._config.channels:
return
# Guard 3: own messages?
if sender == "Me" or (text and text.startswith(self._config.name)):
return
# Guard 4: other bots?
if sender and sender.rstrip().lower().endswith("bot"):
debug_print(f"BOT: skipping message from other bot '{sender}'")
return
# Guard 5: cooldown?
now = time.time()
if now - self._last_reply < self._config.cooldown_seconds:
debug_print("BOT: cooldown active, skipping")
return
# Guard 6: keyword match
template = self._match_keyword(text)
if template is None:
return
# Build reply
path_str = self._format_path(path_len, path_hashes)
snr_str = f"{snr:.1f}" if snr is not None else "?"
reply = template.format(
bot=self._config.name,
sender=sender or "?",
snr=snr_str,
path=path_str,
)
self._last_reply = now
self._sink({
"action": "send_message",
"channel": channel_idx,
"text": reply,
"_bot": True,
})
debug_print(f"BOT: queued reply to '{sender}': {reply}")
# ------------------------------------------------------------------
# Extension point (OCP)
# ------------------------------------------------------------------
def _match_keyword(self, text: str) -> Optional[str]:
"""Return the reply template for the first matching keyword.
Override this method for custom matching strategies (regex,
exact match, priority ordering, etc.).
Returns:
Template string, or ``None`` if no keyword matched.
"""
text_lower = (text or "").lower()
for keyword, template in self._config.keywords.items():
if keyword in text_lower:
return template
return None
# ------------------------------------------------------------------
# Helpers
# ------------------------------------------------------------------
@staticmethod
def _format_path(
path_len: int,
path_hashes: Optional[List[str]],
) -> str:
"""Format path info as ``path(N); ``path(0)``."""
if not path_len:
return "path(0)"
return f"path({path_len})"

289
meshcore_gui/services/cache.py Normal file
View File

@@ -0,0 +1,289 @@
"""
Local JSON cache for device info, channels and contacts.
Loads instantly on startup so the GUI is immediately populated with
the last known state. Background refreshes update the cache
incrementally.
Cache location
~~~~~~~~~~~~~~
``~/.meshcore-gui/cache/<ADDRESS>.json``
One file per device identifier, so multiple devices are supported
without conflict.
Merge strategy (contacts)
~~~~~~~~~~~~~~~~~~~~~~~~~
- New contacts from device → added to cache with ``last_seen`` timestamp
- Existing contacts → updated (fresh data wins)
- Contacts only in cache (node offline) → kept
- Optional pruning of contacts not seen for > N days (not yet implemented)
"""
import json
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Dict, List, Optional
from meshcore_gui.config import CONTACT_RETENTION_DAYS, debug_print
CACHE_VERSION = 1
CACHE_DIR = Path.home() / ".meshcore-gui" / "cache"
class DeviceCache:
"""Read/write JSON cache for a single device.
Args:
device_id: Device identifier string (used to derive filename).
"""
def __init__(self, device_id: str) -> None:
self._address = device_id
safe_name = (
device_id
.replace("literal:", "")
.replace(":", "_")
.replace("/", "_")
)
self._path = CACHE_DIR / f"{safe_name}.json"
self._data: Dict = {}
@property
def path(self) -> Path:
"""Path to the cache file on disk."""
return self._path
@property
def has_cache(self) -> bool:
"""True if a cache file exists on disk."""
return self._path.exists()
# ------------------------------------------------------------------
# Load / Save
# ------------------------------------------------------------------
def load(self) -> bool:
"""Load cache from disk.
Returns:
True if a valid cache was loaded, False otherwise.
"""
if not self._path.exists():
debug_print(f"Cache: no file at {self._path}")
return False
try:
self._data = json.loads(self._path.read_text(encoding="utf-8"))
except (json.JSONDecodeError, OSError) as exc:
debug_print(f"Cache: load error: {exc}")
self._data = {}
return False
if self._data.get("version") != CACHE_VERSION:
debug_print("Cache: version mismatch, ignoring")
self._data = {}
return False
last = self._data.get("last_updated", "?")
debug_print(f"Cache: loaded from {self._path} (last_updated={last})")
return True
def save(self) -> None:
"""Write current state to disk."""
self._data["version"] = CACHE_VERSION
self._data["address"] = self._address
self._data["last_updated"] = datetime.now(timezone.utc).isoformat()
try:
CACHE_DIR.mkdir(parents=True, exist_ok=True)
self._path.write_text(
json.dumps(self._data, indent=2, ensure_ascii=False),
encoding="utf-8",
)
debug_print(f"Cache: saved to {self._path}")
except OSError as exc:
debug_print(f"Cache: save error: {exc}")
# ------------------------------------------------------------------
# Device info
# ------------------------------------------------------------------
def get_device(self) -> Optional[Dict]:
"""Return cached device info dict, or None."""
return self._data.get("device")
def set_device(self, payload: Dict) -> None:
"""Store device info and persist to disk."""
self._data["device"] = payload.copy()
self.save()
def set_firmware_version(self, version: str) -> None:
"""Update firmware version in the cached device info."""
device = self._data.get("device", {})
device["firmware_version"] = version
self._data["device"] = device
self.save()
# ------------------------------------------------------------------
# Channels
# ------------------------------------------------------------------
def get_channels(self) -> List[Dict]:
"""Return cached channel list (may be empty)."""
return self._data.get("channels", [])
def set_channels(self, channels: List[Dict]) -> None:
"""Store channel list and persist to disk."""
self._data["channels"] = [ch.copy() for ch in channels]
self.save()
# ------------------------------------------------------------------
# Channel keys
# ------------------------------------------------------------------
def get_channel_keys(self) -> Dict[int, str]:
"""Return cached channel keys as ``{idx: secret_hex}``."""
return self._data.get("channel_keys", {})
def set_channel_key(self, channel_idx: int, secret_hex: str) -> None:
"""Store a single channel key (hex string) and persist."""
keys = self._data.get("channel_keys", {})
keys[str(channel_idx)] = secret_hex
self._data["channel_keys"] = keys
self.save()
# ------------------------------------------------------------------
# Contacts (merge strategy)
# ------------------------------------------------------------------
def get_contacts(self) -> Dict:
"""Return cached contacts dict (may be empty)."""
return self._data.get("contacts", {})
def merge_contacts(self, fresh: Dict) -> Dict:
"""Merge fresh contacts into cache and persist.
Strategy:
- New contacts in ``fresh`` → added with ``last_seen``
- Existing contacts → updated (fresh data wins)
- Contacts only in cache → kept (node may be offline)
Args:
fresh: Contacts dict from ``get_contacts()`` device response.
Returns:
The merged contacts dict (superset of cached + fresh).
"""
cached = self._data.get("contacts", {})
now = datetime.now(timezone.utc).isoformat()
for key, contact in fresh.items():
contact_copy = contact.copy()
contact_copy["last_seen"] = now
cached[key] = contact_copy
self._data["contacts"] = cached
self.save()
debug_print(
f"Cache: contacts merged — "
f"{len(fresh)} fresh, {len(cached)} total"
)
return cached
def remove_contacts(self, pubkeys: List[str]) -> int:
"""Remove specific contacts from the local cache by public key.
Args:
pubkeys: List of public key hex strings to remove.
Returns:
Number of contacts actually removed from the cache.
"""
cached = self._data.get("contacts", {})
if not cached:
return 0
removed = 0
for key in pubkeys:
if key in cached:
del cached[key]
removed += 1
if removed > 0:
self._data["contacts"] = cached
self.save()
debug_print(
f"Cache: removed {removed} contacts from local history "
f"(remaining: {len(cached)})"
)
return removed
def prune_old_contacts(self) -> int:
"""Remove contacts not seen for longer than CONTACT_RETENTION_DAYS.
Returns:
Number of contacts removed.
"""
cached = self._data.get("contacts", {})
if not cached:
return 0
original_count = len(cached)
cutoff = datetime.now(timezone.utc) - timedelta(days=CONTACT_RETENTION_DAYS)
# Filter contacts based on last_seen timestamp
pruned = {}
for key, contact in cached.items():
last_seen_str = contact.get("last_seen")
# Keep contact if no last_seen (shouldn't happen) or if recent
if not last_seen_str:
pruned[key] = contact
continue
try:
last_seen = datetime.fromisoformat(last_seen_str)
if last_seen > cutoff:
pruned[key] = contact
except (ValueError, TypeError):
# Keep contact if timestamp is invalid
pruned[key] = contact
# Update and save if anything was removed
removed = original_count - len(pruned)
if removed > 0:
self._data["contacts"] = pruned
self.save()
debug_print(
f"Cache: pruned {removed} old contacts "
f"(retained: {len(pruned)})"
)
return removed
# ------------------------------------------------------------------
# Metadata
# ------------------------------------------------------------------
def get_last_updated(self) -> Optional[str]:
"""Return ISO timestamp of last cache update, or None."""
return self._data.get("last_updated")
# ------------------------------------------------------------------
# Original device name (BOT feature)
# ------------------------------------------------------------------
def get_original_device_name(self) -> Optional[str]:
"""Return cached original device name, or None."""
return self._data.get("original_device_name")
def set_original_device_name(self, name: Optional[str]) -> None:
"""Store or clear the original device name and persist to disk."""
if name is None:
self._data.pop("original_device_name", None)
else:
self._data["original_device_name"] = name
self.save()

75
meshcore_gui/services/contact_cleaner.py Normal file
View File

@@ -0,0 +1,75 @@
"""
Contact cleaner service for MeshCore GUI.
Provides business logic for bulk-deleting unpinned contacts from the
MeshCore device. All decision logic (which contacts to purge, counting
pinned vs unpinned) lives here — the GUI only calls this service and
displays results.
Thread safety
~~~~~~~~~~~~~
Methods read from SharedData (thread-safe) and PinStore (thread-safe).
No mutable state is stored in this service.
"""
from dataclasses import dataclass
from typing import Dict, List, Set
from meshcore_gui.services.pin_store import PinStore
@dataclass
class PurgeStats:
"""Statistics for a planned contact purge operation.
Attributes:
unpinned_keys: Public keys of contacts that will be removed.
pinned_count: Number of pinned contacts that will be kept.
total_count: Total number of contacts on the device.
"""
unpinned_keys: List[str]
pinned_count: int
total_count: int
@property
def unpinned_count(self) -> int:
"""Number of contacts that will be removed."""
return len(self.unpinned_keys)
class ContactCleanerService:
"""Business logic for bulk-deleting unpinned contacts.
Args:
pin_store: PinStore instance for checking pin status.
"""
def __init__(self, pin_store: PinStore) -> None:
self._pin_store = pin_store
def get_purge_stats(self, contacts: Dict) -> PurgeStats:
"""Calculate which contacts would be purged.
Iterates all contacts and separates them into pinned (kept)
and unpinned (to be removed).
Args:
contacts: Contacts dict from SharedData snapshot
(``{pubkey: contact_dict}``).
Returns:
PurgeStats with the list of unpinned keys and counts.
"""
pinned_keys: Set[str] = self._pin_store.get_pinned()
unpinned_keys: List[str] = []
for pubkey in contacts:
if pubkey not in pinned_keys:
unpinned_keys.append(pubkey)
return PurgeStats(
unpinned_keys=unpinned_keys,
pinned_count=len(contacts) - len(unpinned_keys),
total_count=len(contacts),
)

108
meshcore_gui/services/dedup.py Normal file
View File

@@ -0,0 +1,108 @@
"""
Message deduplication for MeshCore GUI.
Extracted from SerialWorker to satisfy the Single Responsibility Principle.
Provides bounded-size deduplication via message hash and content keys.
Two strategies are used because the two event sources carry different
identifiers:
1. **Hash-based** — ``RX_LOG_DATA`` events produce a deterministic
``message_hash``. When ``CHANNEL_MSG_RECV`` arrives for the same
packet, it is suppressed.
2. **Content-based** — ``CHANNEL_MSG_RECV`` events do *not* include
``message_hash``, so a composite key of ``channel:sender:text`` is
used as a fallback.
Both stores are bounded to prevent unbounded memory growth.
"""
from collections import OrderedDict
class MessageDeduplicator:
"""Bounded-size message deduplication store.
Uses an :class:`OrderedDict` as an LRU-style bounded set.
Oldest entries are evicted when the store exceeds ``max_size``.
Args:
max_size: Maximum number of keys to retain. 200 is generous
for the typical message rate of a mesh network.
"""
def __init__(self, max_size: int = 200) -> None:
self._max = max_size
self._seen: OrderedDict[str, None] = OrderedDict()
def is_seen(self, key: str) -> bool:
"""Check if a key has already been recorded."""
return key in self._seen
def mark(self, key: str) -> None:
"""Record a key. Evicts the oldest entry if at capacity."""
if key in self._seen:
# Move to end (most recent)
self._seen.move_to_end(key)
return
self._seen[key] = None
while len(self._seen) > self._max:
self._seen.popitem(last=False)
def clear(self) -> None:
"""Remove all recorded keys."""
self._seen.clear()
def __len__(self) -> int:
return len(self._seen)
class DualDeduplicator:
"""Combined hash-based and content-based deduplication.
Wraps two :class:`MessageDeduplicator` instances — one for
message hashes and one for content keys — behind a single
interface.
Args:
max_size: Maximum entries per store.
"""
def __init__(self, max_size: int = 200) -> None:
self._by_hash = MessageDeduplicator(max_size)
self._by_content = MessageDeduplicator(max_size)
# -- Hash-based --
def mark_hash(self, message_hash: str) -> None:
"""Record a message hash as processed."""
if message_hash:
self._by_hash.mark(message_hash)
def is_hash_seen(self, message_hash: str) -> bool:
"""Check if a message hash has already been processed."""
return bool(message_hash) and self._by_hash.is_seen(message_hash)
# -- Content-based --
def mark_content(self, sender: str, channel, text: str) -> None:
"""Record a content key as processed."""
key = self._content_key(sender, channel, text)
self._by_content.mark(key)
def is_content_seen(self, sender: str, channel, text: str) -> bool:
"""Check if a content key has already been processed."""
key = self._content_key(sender, channel, text)
return self._by_content.is_seen(key)
# -- Bulk --
def clear(self) -> None:
"""Clear both stores."""
self._by_hash.clear()
self._by_content.clear()
@staticmethod
def _content_key(sender: str, channel, text: str) -> str:
return f"{channel}:{sender}:{text}"

147
meshcore_gui/services/device_identity.py Normal file
View File

@@ -0,0 +1,147 @@
"""
Device identity file writer for MeshCore Observer integration.
After a successful connection, the worker calls
:func:`write_device_identity` with the device's public and private
keys. The resulting JSON file is placed outside the git repo at::
~/.meshcore-gui/device_identity.json
The MeshCore Observer reads this file automatically for MQTT
authentication — no manual key setup required.
File format::
{
"public_key": "64-char hex UPPERCASE (from send_appstart)",
"private_key": "128-char hex lowercase (full orlp/ed25519 expanded key)",
"device_name": "PE1HVH T1000e",
"firmware_version": "1.2.3",
"source_device": "/dev/ttyUSB1",
"updated_at": "2026-02-26T15:00:00+00:00"
}
Author: PE1HVH
Version: 1.0.0
SPDX-License-Identifier: MIT
Copyright: (c) 2026 PE1HVH
"""
import json
import stat
from datetime import datetime, timezone
from pathlib import Path
from typing import Optional
from meshcore_gui.config import DATA_DIR, debug_print
# Fixed output path — observer looks here by default
IDENTITY_FILE: Path = DATA_DIR / "device_identity.json"
def write_device_identity(
public_key: str,
private_key_bytes: bytes,
device_name: str = "",
firmware_version: str = "",
source_device: str = "",
) -> bool:
"""Write the device identity file for MeshCore Observer.
Args:
public_key: 64-char hex public key (from send_appstart).
This is the key shown in the GUI and registered
at LetsMesh. MUST be used for MQTT username.
private_key_bytes: 64 raw bytes from export_private_key() in
orlp/ed25519 expanded format. All 64 bytes
are needed for createAuthToken().
device_name: Device display name.
firmware_version: Firmware version string.
source_device: Device path (e.g. ``/dev/ttyUSB1``).
Returns:
True if the file was written successfully.
"""
try:
# The 64 bytes from export_private_key() are in orlp/ed25519
# *expanded* format:
# bytes 0..31 = clamped scalar (NOT the raw seed)
# bytes 32..63 = nonce prefix (NOT the public key)
#
# The public key is NOT contained in these 64 bytes — it must
# come from send_appstart() which returns the actual device
# public key as shown in the GUI and registered at LetsMesh.
#
# For MQTT auth via meshcore-decoder's createAuthToken(), the
# full 64 bytes are needed as privateKeyHex (128 hex chars).
if len(private_key_bytes) != 64:
debug_print(
f"DeviceIdentity: unexpected key length "
f"{len(private_key_bytes)}, expected 64 bytes"
)
return False
# Full 64-byte private key in MeshCore/orlp format
private_key_hex = private_key_bytes.hex()
if not public_key or len(public_key) != 64:
debug_print(
f"DeviceIdentity: no valid public key from appstart "
f"(got {public_key!r}), cannot write identity file"
)
return False
identity = {
"public_key": public_key.upper(),
"private_key": private_key_hex.lower(),
"device_name": device_name,
"firmware_version": firmware_version,
"source_device": source_device,
"updated_at": datetime.now(timezone.utc).isoformat(),
}
DATA_DIR.mkdir(parents=True, exist_ok=True)
IDENTITY_FILE.write_text(
json.dumps(identity, indent=2) + "\n",
encoding="utf-8",
)
# Restrictive permissions — file contains the private key
IDENTITY_FILE.chmod(stat.S_IRUSR | stat.S_IWUSR) # 600
debug_print(
f"DeviceIdentity: written to {IDENTITY_FILE} "
f"(pub={public_key[:12]}... priv={private_key_hex[:12]}...)"
)
print(f"📝 Device identity saved → {IDENTITY_FILE}")
return True
except Exception as exc:
debug_print(f"DeviceIdentity: write failed: {exc}")
print(f"⚠️ Could not save device identity: {exc}")
return False
def read_device_identity() -> Optional[dict]:
"""Read the device identity file.
Returns:
Dict with ``public_key`` and ``private_key`` (hex strings),
or None if the file does not exist or is invalid.
"""
if not IDENTITY_FILE.exists():
return None
try:
data = json.loads(IDENTITY_FILE.read_text(encoding="utf-8"))
pub = data.get("public_key", "")
priv = data.get("private_key", "")
if len(pub) == 64 and len(priv) in (64, 128):
return data
debug_print(
f"DeviceIdentity: invalid key lengths in {IDENTITY_FILE} "
f"(pub={len(pub)}, priv={len(priv)})"
)
return None
except (json.JSONDecodeError, OSError) as exc:
debug_print(f"DeviceIdentity: read error: {exc}")
return None

133
meshcore_gui/services/map_snapshot_service.py Normal file
View File

@@ -0,0 +1,133 @@
"""Service helpers for the browser-managed Leaflet map."""
from __future__ import annotations
from dataclasses import asdict, dataclass
from typing import Dict, List, Optional
@dataclass(frozen=True)
class MapNodeSnapshot:
"""Serializable snapshot for a single map node."""
id: str
name: str
short_key: str
node_type: int
lat: float
lon: float
@dataclass(frozen=True)
class MapDeviceSnapshot:
"""Serializable snapshot for the local device marker."""
name: str
lat: float
lon: float
@dataclass(frozen=True)
class MapSnapshot:
"""Serializable snapshot consumed by the browser Leaflet runtime."""
device: Optional[MapDeviceSnapshot]
contacts: List[MapNodeSnapshot]
theme: str
force_center: bool
def to_dict(self) -> Dict:
"""Return the snapshot as a plain JSON-serializable dict."""
return {
'device': asdict(self.device) if self.device else None,
'contacts': [asdict(contact) for contact in self.contacts],
'theme': self.theme,
'force_center': self.force_center,
}
class MapSnapshotService:
"""Build compact browser snapshots from SharedData payloads."""
def build_snapshot(
self,
data: Dict,
theme_mode: str,
ui_dark: bool,
force_center: bool = False,
) -> MapSnapshot:
"""Create a full map snapshot for the browser-side Leaflet runtime."""
return MapSnapshot(
device=self._build_device(data),
contacts=self._build_contacts(data),
theme=self.resolve_theme(theme_mode, ui_dark),
force_center=force_center or bool(data.get('force_center', False)),
)
def _build_device(self, data: Dict) -> Optional[MapDeviceSnapshot]:
"""Return the local device snapshot when valid coordinates exist."""
lat = self._coerce_coordinate(data.get('adv_lat'))
lon = self._coerce_coordinate(data.get('adv_lon'))
if lat is None or lon is None:
return None
return MapDeviceSnapshot(
name=str(data.get('name') or 'Device'),
lat=lat,
lon=lon,
)
def _build_contacts(self, data: Dict) -> List[MapNodeSnapshot]:
"""Return all valid contact marker snapshots sorted by display name."""
contacts: List[MapNodeSnapshot] = []
raw_contacts = data.get('contacts', {}) or {}
for key, contact in raw_contacts.items():
lat = self._coerce_coordinate(contact.get('adv_lat'))
lon = self._coerce_coordinate(contact.get('adv_lon'))
if lat is None or lon is None:
continue
key_str = str(key)
name = str(contact.get('adv_name') or key_str[:12])
node_type = self._coerce_node_type(contact.get('type'))
contacts.append(
MapNodeSnapshot(
id=key_str,
name=name,
short_key=key_str[:12],
node_type=node_type,
lat=lat,
lon=lon,
)
)
contacts.sort(key=lambda item: (item.name.lower(), item.short_key.lower()))
return contacts
@staticmethod
def resolve_theme(theme_mode: str, ui_dark: bool) -> str:
"""Resolve the effective tile theme from the configured mode."""
if theme_mode == 'dark':
return 'dark'
if theme_mode == 'light':
return 'light'
return 'dark' if ui_dark else 'light'
@staticmethod
def _coerce_coordinate(value: object) -> Optional[float]:
"""Normalize latitude/longitude values; return None for empty/zero."""
if value in (None, '', 0, 0.0):
return None
try:
numeric = float(value)
except (TypeError, ValueError):
return None
if numeric == 0.0:
return None
return numeric
@staticmethod
def _coerce_node_type(value: object) -> int:
"""Normalize node type values to the supported marker range."""
try:
node_type = int(value)
except (TypeError, ValueError):
return 0
return node_type if node_type in (0, 1, 2, 3) else 0

681
meshcore_gui/services/message_archive.py Normal file
View File

@@ -0,0 +1,681 @@
"""
Persistent message and RxLog archive for MeshCore GUI.
Stores all incoming messages and RX log entries with configurable retention.
Works alongside SharedData: SharedData holds the latest N items for UI display,
while MessageArchive persists everything to disk with automatic cleanup.
Storage location
~~~~~~~~~~~~~~~~
~/.meshcore-gui/archive/<ADDRESS>_messages.json
~/.meshcore-gui/archive/<ADDRESS>_rxlog.json
Retention strategy
~~~~~~~~~~~~~~~~~~
- Messages older than MESSAGE_RETENTION_DAYS are purged daily
- RxLog entries older than RXLOG_RETENTION_DAYS are purged daily
- Cleanup runs in background (non-blocking)
Thread safety
~~~~~~~~~~~~~~
All methods use an internal lock for thread-safe operation.
The lock is separate from SharedData's lock to avoid contention.
"""
import json
import threading
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Dict, List, Optional
from meshcore_gui.config import (
MESSAGE_RETENTION_DAYS,
RXLOG_RETENTION_DAYS,
debug_print,
)
from meshcore_gui.core.models import Message, RxLogEntry
ARCHIVE_VERSION = 1
ARCHIVE_DIR = Path.home() / ".meshcore-gui" / "archive"
class MessageArchive:
"""Persistent storage for messages and RX log entries.
Args:
device_id: Device identifier string (used to derive filenames).
"""
def __init__(self, device_id: str) -> None:
self._address = device_id
self._lock = threading.Lock()
# Sanitize address for filename
safe_name = (
device_id
.replace("literal:", "")
.replace(":", "_")
.replace("/", "_")
)
self._messages_path = ARCHIVE_DIR / f"{safe_name}_messages.json"
self._rxlog_path = ARCHIVE_DIR / f"{safe_name}_rxlog.json"
# In-memory batch buffers (flushed periodically)
self._message_buffer: List[Dict] = []
self._rxlog_buffer: List[Dict] = []
# Batch write thresholds
self._batch_size = 10
self._last_flush = datetime.now(timezone.utc)
self._flush_interval_seconds = 60
# Stats
self._total_messages = 0
self._total_rxlog = 0
# Load existing archives
self._load_archives()
# ------------------------------------------------------------------
# Initialization
# ------------------------------------------------------------------
def _load_archives(self) -> None:
"""Load existing archive files and count entries."""
with self._lock:
# Load messages
if self._messages_path.exists():
try:
data = json.loads(self._messages_path.read_text(encoding="utf-8"))
if data.get("version") == ARCHIVE_VERSION:
self._total_messages = len(data.get("messages", []))
debug_print(
f"Archive: loaded {self._total_messages} messages "
f"from {self._messages_path}"
)
except (json.JSONDecodeError, OSError) as exc:
debug_print(f"Archive: error loading messages: {exc}")
# Load rxlog
if self._rxlog_path.exists():
try:
data = json.loads(self._rxlog_path.read_text(encoding="utf-8"))
if data.get("version") == ARCHIVE_VERSION:
self._total_rxlog = len(data.get("entries", []))
debug_print(
f"Archive: loaded {self._total_rxlog} rxlog entries "
f"from {self._rxlog_path}"
)
except (json.JSONDecodeError, OSError) as exc:
debug_print(f"Archive: error loading rxlog: {exc}")
# ------------------------------------------------------------------
# Add operations (buffered)
# ------------------------------------------------------------------
def add_message(self, msg: Message) -> None:
"""Add a message to the archive (buffered write).
Args:
msg: Message dataclass instance.
"""
with self._lock:
# Convert to dict and add UTC timestamp
msg_dict = {
"time": msg.time,
"timestamp_utc": datetime.now(timezone.utc).isoformat(),
"sender": msg.sender,
"text": msg.text,
"channel": msg.channel,
"channel_name": msg.channel_name,
"direction": msg.direction,
"snr": msg.snr,
"path_len": msg.path_len,
"sender_pubkey": msg.sender_pubkey,
"path_hashes": msg.path_hashes,
"path_names": msg.path_names,
"message_hash": msg.message_hash,
}
self._message_buffer.append(msg_dict)
# Flush if batch size reached
if len(self._message_buffer) >= self._batch_size:
self._flush_messages()
# Also flush if interval exceeded
elif self._should_flush():
self._flush_all()
def add_rx_log(self, entry: RxLogEntry) -> None:
"""Add an RX log entry to the archive (buffered write).
Args:
entry: RxLogEntry dataclass instance.
"""
with self._lock:
# Convert to dict and add UTC timestamp
entry_dict = {
"time": entry.time,
"timestamp_utc": datetime.now(timezone.utc).isoformat(),
"snr": entry.snr,
"rssi": entry.rssi,
"payload_type": entry.payload_type,
"hops": entry.hops,
"message_hash": entry.message_hash,
"path_hashes": entry.path_hashes,
"path_names": entry.path_names,
"sender": entry.sender,
"receiver": entry.receiver,
"raw_payload": entry.raw_payload,
"packet_len": entry.packet_len,
"payload_len": entry.payload_len,
"route_type": entry.route_type,
"packet_type_num": entry.packet_type_num,
}
self._rxlog_buffer.append(entry_dict)
# Flush if batch size reached
if len(self._rxlog_buffer) >= self._batch_size:
self._flush_rxlog()
# Also flush if interval exceeded
elif self._should_flush():
self._flush_all()
# ------------------------------------------------------------------
# Flushing (write to disk)
# ------------------------------------------------------------------
def _should_flush(self) -> bool:
"""Check if flush interval has been exceeded."""
elapsed = (datetime.now(timezone.utc) - self._last_flush).total_seconds()
return elapsed >= self._flush_interval_seconds
def _flush_messages(self) -> None:
"""Flush message buffer to disk (MUST be called with lock held)."""
if not self._message_buffer:
return
# Read existing archive
existing_messages = []
if self._messages_path.exists():
try:
data = json.loads(self._messages_path.read_text(encoding="utf-8"))
if data.get("version") == ARCHIVE_VERSION:
existing_messages = data.get("messages", [])
else:
debug_print(
f"Archive: version mismatch in {self._messages_path}, "
f"expected {ARCHIVE_VERSION}, got {data.get('version')}"
)
# Don't overwrite if version mismatch - keep buffer for retry
return
except (json.JSONDecodeError, OSError) as exc:
debug_print(
f"Archive: error reading existing messages from {self._messages_path}: {exc}"
)
# Don't overwrite corrupted file - keep buffer for retry
return
# Append new messages
existing_messages.extend(self._message_buffer)
try:
# Write atomically (temp file + rename)
self._write_atomic(
self._messages_path,
{
"version": ARCHIVE_VERSION,
"address": self._address,
"last_updated": datetime.now(timezone.utc).isoformat(),
"messages": existing_messages,
}
)
self._total_messages = len(existing_messages)
debug_print(
f"Archive: flushed {len(self._message_buffer)} messages "
f"(total: {self._total_messages})"
)
# Clear buffer only after successful write
self._message_buffer.clear()
self._last_flush = datetime.now(timezone.utc)
except (OSError) as exc:
debug_print(f"Archive: error writing messages: {exc}")
# Keep buffer for retry
def _flush_rxlog(self) -> None:
"""Flush rxlog buffer to disk (MUST be called with lock held)."""
if not self._rxlog_buffer:
return
# Read existing archive
existing_entries = []
if self._rxlog_path.exists():
try:
data = json.loads(self._rxlog_path.read_text(encoding="utf-8"))
if data.get("version") == ARCHIVE_VERSION:
existing_entries = data.get("entries", [])
else:
debug_print(
f"Archive: version mismatch in {self._rxlog_path}, "
f"expected {ARCHIVE_VERSION}, got {data.get('version')}"
)
# Don't overwrite if version mismatch - keep buffer for retry
return
except (json.JSONDecodeError, OSError) as exc:
debug_print(
f"Archive: error reading existing rxlog from {self._rxlog_path}: {exc}"
)
# Don't overwrite corrupted file - keep buffer for retry
return
# Append new entries
existing_entries.extend(self._rxlog_buffer)
try:
# Write atomically (temp file + rename)
self._write_atomic(
self._rxlog_path,
{
"version": ARCHIVE_VERSION,
"address": self._address,
"last_updated": datetime.now(timezone.utc).isoformat(),
"entries": existing_entries,
}
)
self._total_rxlog = len(existing_entries)
debug_print(
f"Archive: flushed {len(self._rxlog_buffer)} rxlog entries "
f"(total: {self._total_rxlog})"
)
# Clear buffer only after successful write
self._rxlog_buffer.clear()
self._last_flush = datetime.now(timezone.utc)
except (OSError) as exc:
debug_print(f"Archive: error writing rxlog: {exc}")
# Keep buffer for retry
def _flush_all(self) -> None:
"""Flush all buffers to disk (MUST be called with lock held)."""
self._flush_messages()
self._flush_rxlog()
def flush(self) -> None:
"""Manually flush all pending writes to disk."""
with self._lock:
self._flush_all()
# ------------------------------------------------------------------
# Cleanup (retention)
# ------------------------------------------------------------------
def cleanup_old_data(self) -> None:
"""Remove messages and rxlog entries older than retention period.
This is intended to be called periodically (e.g., daily) as a
background task.
"""
with self._lock:
# Flush pending writes first
self._flush_all()
# Cleanup messages
self._cleanup_messages()
# Cleanup rxlog
self._cleanup_rxlog()
def _cleanup_messages(self) -> None:
"""Remove messages older than MESSAGE_RETENTION_DAYS."""
if not self._messages_path.exists():
return
try:
data = json.loads(self._messages_path.read_text(encoding="utf-8"))
if data.get("version") != ARCHIVE_VERSION:
return
messages = data.get("messages", [])
original_count = len(messages)
# Calculate cutoff date
cutoff = datetime.now(timezone.utc) - timedelta(days=MESSAGE_RETENTION_DAYS)
# Filter messages
filtered = [
msg for msg in messages
if self._is_newer_than(msg.get("timestamp_utc"), cutoff)
]
# Write back if anything was removed
if len(filtered) < original_count:
data["messages"] = filtered
data["last_updated"] = datetime.now(timezone.utc).isoformat()
self._write_atomic(self._messages_path, data)
removed = original_count - len(filtered)
self._total_messages = len(filtered)
debug_print(
f"Archive: cleanup removed {removed} old messages "
f"(retained: {len(filtered)})"
)
except (json.JSONDecodeError, OSError) as exc:
debug_print(f"Archive: error cleaning up messages: {exc}")
def _cleanup_rxlog(self) -> None:
"""Remove rxlog entries older than RXLOG_RETENTION_DAYS."""
if not self._rxlog_path.exists():
return
try:
data = json.loads(self._rxlog_path.read_text(encoding="utf-8"))
if data.get("version") != ARCHIVE_VERSION:
return
entries = data.get("entries", [])
original_count = len(entries)
# Calculate cutoff date
cutoff = datetime.now(timezone.utc) - timedelta(days=RXLOG_RETENTION_DAYS)
# Filter entries
filtered = [
entry for entry in entries
if self._is_newer_than(entry.get("timestamp_utc"), cutoff)
]
# Write back if anything was removed
if len(filtered) < original_count:
data["entries"] = filtered
data["last_updated"] = datetime.now(timezone.utc).isoformat()
self._write_atomic(self._rxlog_path, data)
removed = original_count - len(filtered)
self._total_rxlog = len(filtered)
debug_print(
f"Archive: cleanup removed {removed} old rxlog entries "
f"(retained: {len(filtered)})"
)
except (json.JSONDecodeError, OSError) as exc:
debug_print(f"Archive: error cleaning up rxlog: {exc}")
# ------------------------------------------------------------------
# Utilities
# ------------------------------------------------------------------
def _is_newer_than(self, timestamp_str: Optional[str], cutoff: datetime) -> bool:
"""Check if ISO timestamp is newer than cutoff date."""
if not timestamp_str:
return False
try:
timestamp = datetime.fromisoformat(timestamp_str)
return timestamp > cutoff
except (ValueError, TypeError):
return False
def _write_atomic(self, path: Path, data: Dict) -> None:
"""Write JSON data atomically using temp file + rename."""
# Ensure directory exists
ARCHIVE_DIR.mkdir(parents=True, exist_ok=True)
# Write to temp file
temp_path = path.with_suffix(".tmp")
temp_path.write_text(
json.dumps(data, indent=2, ensure_ascii=False),
encoding="utf-8",
)
# Atomic rename
temp_path.replace(path)
# ------------------------------------------------------------------
# Channel name discovery
# ------------------------------------------------------------------
def get_distinct_channel_names(self) -> list:
"""Return a sorted list of distinct channel names from archived messages.
Scans all stored messages and collects unique ``channel_name``
values. Empty or missing names are excluded.
Returns:
Sorted list of unique channel name strings.
"""
with self._lock:
# Flush pending writes so we don't miss recent messages
self._flush_messages()
if not self._messages_path.exists():
return []
try:
data = json.loads(
self._messages_path.read_text(encoding="utf-8")
)
if data.get("version") != ARCHIVE_VERSION:
return []
messages = data.get("messages", [])
names: set = set()
for msg in messages:
name = msg.get("channel_name", "")
if name:
names.add(name)
return sorted(names)
except (json.JSONDecodeError, OSError) as exc:
debug_print(
f"Archive: error reading distinct channel names: {exc}"
)
return []
# ------------------------------------------------------------------
# Single message lookup
# ------------------------------------------------------------------
def get_message_by_hash(self, message_hash: str) -> Optional[Dict]:
"""Return a single archived message by its message_hash.
Args:
message_hash: Hex string packet identifier.
Returns:
Message dict, or ``None`` if not found.
"""
if not message_hash:
return None
with self._lock:
# Flush pending writes so recent messages are searchable
self._flush_messages()
if not self._messages_path.exists():
return None
try:
data = json.loads(
self._messages_path.read_text(encoding="utf-8")
)
if data.get("version") != ARCHIVE_VERSION:
return None
for msg in data.get("messages", []):
if msg.get("message_hash") == message_hash:
return msg
except (json.JSONDecodeError, OSError) as exc:
debug_print(
f"Archive: error looking up hash {message_hash[:16]}: "
f"{exc}"
)
return None
# ------------------------------------------------------------------
# Stats
# ------------------------------------------------------------------
def get_stats(self) -> Dict:
"""Get archive statistics.
Returns:
Dict with 'total_messages' and 'total_rxlog' counts.
"""
with self._lock:
return {
"total_messages": self._total_messages,
"total_rxlog": self._total_rxlog,
"pending_messages": len(self._message_buffer),
"pending_rxlog": len(self._rxlog_buffer),
}
def get_messages_by_sender_pubkey(
self, pubkey_prefix: str, limit: int = 50,
) -> List[Dict]:
"""Return archived messages whose *sender_pubkey* starts with *pubkey_prefix*.
Useful for loading Room Server history: room messages are stored
with ``sender_pubkey`` equal to the room's public-key prefix.
Args:
pubkey_prefix: First N hex chars of the sender pubkey to match.
limit: Maximum number of messages to return (newest).
Returns:
List of message dicts (oldest-first), at most *limit* entries.
"""
with self._lock:
# Flush pending writes so we don't miss recent messages
self._flush_messages()
if not self._messages_path.exists():
return []
try:
data = json.loads(
self._messages_path.read_text(encoding="utf-8")
)
if data.get("version") != ARCHIVE_VERSION:
return []
messages = data.get("messages", [])
norm = pubkey_prefix[:12]
matched = [
msg for msg in messages
if (msg.get("sender_pubkey") or "").startswith(norm)
]
# Oldest-first, keep last *limit*
matched.sort(key=lambda m: m.get("timestamp_utc", ""))
return matched[-limit:]
except (json.JSONDecodeError, OSError) as exc:
debug_print(
f"Archive: error querying by pubkey {pubkey_prefix[:12]}: "
f"{exc}"
)
return []
def query_messages(
self,
after: Optional[datetime] = None,
before: Optional[datetime] = None,
channel_name: Optional[str] = None,
sender: Optional[str] = None,
text_search: Optional[str] = None,
limit: int = 100,
offset: int = 0,
) -> tuple:
"""Query archived messages with filters.
Args:
after: Only messages after this timestamp (UTC).
before: Only messages before this timestamp (UTC).
channel_name: Filter by channel name (exact match).
sender: Filter by sender name (case-insensitive substring match).
text_search: Search in message text (case-insensitive substring match).
limit: Maximum number of results to return.
offset: Skip this many results (for pagination).
Returns:
Tuple of (messages, total_count):
- messages: List of message dicts matching the filters, newest first
- total_count: Total number of messages matching filters (for pagination)
"""
with self._lock:
# Flush pending writes first
self._flush_messages()
if not self._messages_path.exists():
return [], 0
try:
data = json.loads(self._messages_path.read_text(encoding="utf-8"))
if data.get("version") != ARCHIVE_VERSION:
return [], 0
messages = data.get("messages", [])
# Apply filters
filtered = []
for msg in messages:
# Time filters
if after or before:
try:
msg_time = datetime.fromisoformat(msg.get("timestamp_utc", ""))
if after and msg_time < after:
continue
if before and msg_time > before:
continue
except (ValueError, TypeError):
continue
# Channel name filter (exact match)
if channel_name is not None:
if msg.get("channel_name", "") != channel_name:
continue
# Sender filter (case-insensitive substring)
if sender:
msg_sender = msg.get("sender", "")
if sender.lower() not in msg_sender.lower():
continue
# Text search (case-insensitive substring)
if text_search:
msg_text = msg.get("text", "")
if text_search.lower() not in msg_text.lower():
continue
filtered.append(msg)
# Sort newest first
filtered.sort(
key=lambda m: m.get("timestamp_utc", ""),
reverse=True
)
total_count = len(filtered)
# Apply pagination
paginated = filtered[offset:offset + limit]
return paginated, total_count
except (json.JSONDecodeError, OSError) as exc:
debug_print(f"Archive: error querying messages: {exc}")
return [], 0

125
meshcore_gui/services/pin_store.py Normal file
View File

@@ -0,0 +1,125 @@
"""
Persistent pin store for MeshCore GUI.
Stores a set of pinned contact public keys per device.
Pin status is purely app-side and is not stored on the device.
Storage location
~~~~~~~~~~~~~~~~
``~/.meshcore-gui/pins/<ADDRESS>.json``
Thread safety
~~~~~~~~~~~~~
All methods use an internal lock for thread-safe operation.
"""
import json
import threading
from pathlib import Path
from typing import Set
from meshcore_gui.config import debug_print
PINS_DIR = Path.home() / ".meshcore-gui" / "pins"
class PinStore:
"""Persistent storage for pinned contact public keys.
Args:
device_id: Device identifier string (used to derive filename).
"""
def __init__(self, device_id: str) -> None:
self._lock = threading.Lock()
safe_name = (
device_id
.replace("literal:", "")
.replace(":", "_")
.replace("/", "_")
)
self._path = PINS_DIR / f"{safe_name}_pins.json"
self._pinned: Set[str] = set()
self._load()
# ------------------------------------------------------------------
# Public API
# ------------------------------------------------------------------
def is_pinned(self, pubkey: str) -> bool:
"""Check if a contact is pinned.
Args:
pubkey: Full public key (hex string).
Returns:
True if the contact is pinned.
"""
with self._lock:
return pubkey in self._pinned
def pin(self, pubkey: str) -> None:
"""Pin a contact.
Args:
pubkey: Full public key (hex string).
"""
with self._lock:
self._pinned.add(pubkey)
self._save()
debug_print(f"PinStore: pinned {pubkey[:16]}")
def unpin(self, pubkey: str) -> None:
"""Unpin a contact.
Args:
pubkey: Full public key (hex string).
"""
with self._lock:
self._pinned.discard(pubkey)
self._save()
debug_print(f"PinStore: unpinned {pubkey[:16]}")
def get_pinned(self) -> Set[str]:
"""Return a copy of the set of pinned public keys.
Returns:
Set of pinned public key hex strings.
"""
with self._lock:
return self._pinned.copy()
# ------------------------------------------------------------------
# Persistence
# ------------------------------------------------------------------
def _load(self) -> None:
"""Load pinned contacts from disk."""
if not self._path.exists():
debug_print(f"PinStore: no file at {self._path}")
return
try:
data = json.loads(self._path.read_text(encoding="utf-8"))
self._pinned = set(data.get("pinned", []))
debug_print(
f"PinStore: loaded {len(self._pinned)} pinned contacts"
)
except (json.JSONDecodeError, OSError) as exc:
debug_print(f"PinStore: load error: {exc}")
self._pinned = set()
def _save(self) -> None:
"""Write pinned contacts to disk."""
try:
PINS_DIR.mkdir(parents=True, exist_ok=True)
data = {"pinned": sorted(self._pinned)}
self._path.write_text(
json.dumps(data, indent=2, ensure_ascii=False),
encoding="utf-8",
)
debug_print(f"PinStore: saved {len(self._pinned)} pins")
except OSError as exc:
debug_print(f"PinStore: save error: {exc}")

200
meshcore_gui/services/room_password_store.py Normal file
View File

@@ -0,0 +1,200 @@
"""
Persistent Room Server password store for MeshCore GUI.
Stores passwords and configuration for Room Server contacts per device.
Passwords are stored outside the repository under
``~/.meshcore-gui/room_passwords/<ADDRESS>.json``.
Thread safety
~~~~~~~~~~~~~
All methods use an internal lock for thread-safe operation.
"""
import json
import threading
from dataclasses import asdict, dataclass
from pathlib import Path
from typing import Dict, List, Optional
from meshcore_gui.config import debug_print
ROOM_PASSWORDS_DIR = Path.home() / ".meshcore-gui" / "room_passwords"
@dataclass
class RoomServerEntry:
"""Stored configuration for a single Room Server.
Attributes:
pubkey: Full public key (hex string).
name: Display name of the Room Server.
password: Stored password (plaintext — local file only).
"""
pubkey: str
name: str = ""
password: str = ""
class RoomPasswordStore:
"""Persistent storage for Room Server passwords.
Args:
device_id: Device identifier string (used to derive filename).
"""
def __init__(self, device_id: str) -> None:
self._lock = threading.Lock()
safe_name = (
device_id
.replace("literal:", "")
.replace(":", "_")
.replace("/", "_")
)
self._path = ROOM_PASSWORDS_DIR / f"{safe_name}_rooms.json"
self._rooms: Dict[str, RoomServerEntry] = {}
self._load()
# ------------------------------------------------------------------
# Public API
# ------------------------------------------------------------------
def get_rooms(self) -> List[RoomServerEntry]:
"""Return a list of all configured Room Server entries.
Returns:
List of RoomServerEntry instances (copies).
"""
with self._lock:
return list(self._rooms.values())
def get_room(self, pubkey: str) -> Optional[RoomServerEntry]:
"""Get a specific Room Server entry by public key.
Args:
pubkey: Full public key (hex string).
Returns:
RoomServerEntry if found, None otherwise.
"""
with self._lock:
entry = self._rooms.get(pubkey)
if entry:
return RoomServerEntry(
pubkey=entry.pubkey,
name=entry.name,
password=entry.password,
)
return None
def has_room(self, pubkey: str) -> bool:
"""Check if a Room Server is configured.
Args:
pubkey: Full public key (hex string).
Returns:
True if the Room Server is in the store.
"""
with self._lock:
return pubkey in self._rooms
def add_room(self, pubkey: str, name: str, password: str = "") -> None:
"""Add or update a Room Server entry.
Args:
pubkey: Full public key (hex string).
name: Display name.
password: Password (empty string if not yet set).
"""
with self._lock:
self._rooms[pubkey] = RoomServerEntry(
pubkey=pubkey,
name=name,
password=password,
)
self._save()
debug_print(
f"RoomPasswordStore: added/updated {name} "
f"({pubkey[:16]})"
)
def update_password(self, pubkey: str, password: str) -> None:
"""Update the password for an existing Room Server.
Args:
pubkey: Full public key (hex string).
password: New password.
"""
with self._lock:
if pubkey in self._rooms:
self._rooms[pubkey].password = password
self._save()
debug_print(
f"RoomPasswordStore: password updated for "
f"{pubkey[:16]}"
)
def remove_room(self, pubkey: str) -> None:
"""Remove a Room Server entry.
Args:
pubkey: Full public key (hex string).
"""
with self._lock:
if pubkey in self._rooms:
name = self._rooms[pubkey].name
del self._rooms[pubkey]
self._save()
debug_print(
f"RoomPasswordStore: removed {name} "
f"({pubkey[:16]})"
)
# ------------------------------------------------------------------
# Persistence
# ------------------------------------------------------------------
def _load(self) -> None:
"""Load Room Server entries from disk."""
if not self._path.exists():
debug_print(f"RoomPasswordStore: no file at {self._path}")
return
try:
data = json.loads(self._path.read_text(encoding="utf-8"))
rooms = data.get("rooms", {})
for pubkey, entry_dict in rooms.items():
self._rooms[pubkey] = RoomServerEntry(
pubkey=pubkey,
name=entry_dict.get("name", ""),
password=entry_dict.get("password", ""),
)
debug_print(
f"RoomPasswordStore: loaded {len(self._rooms)} rooms"
)
except (json.JSONDecodeError, OSError) as exc:
debug_print(f"RoomPasswordStore: load error: {exc}")
self._rooms = {}
def _save(self) -> None:
"""Write Room Server entries to disk."""
try:
ROOM_PASSWORDS_DIR.mkdir(parents=True, exist_ok=True)
data = {
"rooms": {
pubkey: asdict(entry)
for pubkey, entry in self._rooms.items()
}
}
self._path.write_text(
json.dumps(data, indent=2, ensure_ascii=False),
encoding="utf-8",
)
debug_print(
f"RoomPasswordStore: saved {len(self._rooms)} rooms"
)
except OSError as exc:
debug_print(f"RoomPasswordStore: save error: {exc}")

327
meshcore_gui/services/route_builder.py Normal file
View File

@@ -0,0 +1,327 @@
"""
Route data builder for MeshCore GUI.
Pure data logic — no UI code. Given a message and a data snapshot, this
module constructs a route dictionary that describes the path the message
has taken through the mesh network (sender → repeaters → receiver).
v4.1 changes
~~~~~~~~~~~~~
- ``build()`` now accepts a :class:`~meshcore_gui.models.Message`
dataclass instead of a plain dict.
- Route nodes returned as :class:`~meshcore_gui.models.RouteNode`.
"""
from typing import Dict, List, Optional
from meshcore_gui.config import debug_print
from meshcore_gui.core.models import Message, RouteNode
from meshcore_gui.core.protocols import ContactLookup
class RouteBuilder:
"""
Builds route data for a message from available contact information.
Uses only data already in memory — no extra commands are sent.
Args:
shared: ContactLookup for resolving pubkey prefixes to contacts
"""
def __init__(self, shared: ContactLookup) -> None:
self._shared = shared
def build(self, msg: Message, data: Dict) -> Dict:
"""
Build route data for a single message.
Args:
msg: Message dataclass instance.
data: Snapshot dictionary from SharedData.get_snapshot().
Returns:
Dictionary with keys:
sender: RouteNode or None
self_node: RouteNode
path_nodes: List[RouteNode]
snr: float or None
msg_path_len: int — hop count from the message itself
has_locations: bool — True if any node has GPS coords
path_source: str — 'rx_log', 'contact_out_path' or 'none'
"""
result: Dict = {
'sender': None,
'self_node': RouteNode(
name=data['name'] or 'Me',
lat=data['adv_lat'],
lon=data['adv_lon'],
),
'path_nodes': [],
'snr': msg.snr,
'msg_path_len': msg.path_len,
'has_locations': False,
'path_source': 'none',
}
# Look up sender in contacts
pubkey = msg.sender_pubkey
contact: Optional[Dict] = None
debug_print(
f"Route build: sender_pubkey={pubkey!r} "
f"(len={len(pubkey)}, first2={pubkey[:2]!r})"
)
if pubkey:
contact = self._shared.get_contact_by_prefix(pubkey)
debug_print(
f"Route build: contact lookup "
f"{'FOUND ' + contact.get('adv_name', '?') if contact else 'NOT FOUND'}"
)
if contact:
result['sender'] = RouteNode(
name=contact.get('adv_name') or pubkey[:8],
lat=contact.get('adv_lat', 0),
lon=contact.get('adv_lon', 0),
type=contact.get('type', 0),
pubkey=pubkey,
)
# Always try name-based fallback if sender still unresolved
if result['sender'] is None:
sender_name = msg.sender
if sender_name:
match = self._shared.get_contact_by_name(sender_name)
if match:
found_pubkey, contact_data = match
contact = contact_data
result['sender'] = RouteNode(
name=contact_data.get('adv_name') or found_pubkey[:8],
lat=contact_data.get('adv_lat', 0),
lon=contact_data.get('adv_lon', 0),
type=contact_data.get('type', 0),
pubkey=found_pubkey,
)
debug_print(
f"Route build: name fallback "
f"'{sender_name}' → pubkey={found_pubkey[:16]!r}"
)
# Fallback 3: direct lookup in snapshot contacts by pubkey
if result['sender'] is None and pubkey:
snapshot_contact = self._find_contact_by_pubkey(
pubkey, data['contacts'],
)
if snapshot_contact:
contact = snapshot_contact
result['sender'] = RouteNode(
name=snapshot_contact.get('adv_name') or pubkey[:8],
lat=snapshot_contact.get('adv_lat', 0),
lon=snapshot_contact.get('adv_lon', 0),
type=snapshot_contact.get('type', 0),
pubkey=pubkey,
)
debug_print(
f"Route build: snapshot pubkey fallback "
f"→ {snapshot_contact.get('adv_name', '?')}"
)
# Fallback 4: direct lookup in snapshot contacts by name
if result['sender'] is None and msg.sender:
name_match = self._find_contact_by_adv_name(
msg.sender, data['contacts'],
)
if name_match:
found_pubkey, snapshot_contact = name_match
contact = snapshot_contact
result['sender'] = RouteNode(
name=snapshot_contact.get('adv_name') or msg.sender,
lat=snapshot_contact.get('adv_lat', 0),
lon=snapshot_contact.get('adv_lon', 0),
type=snapshot_contact.get('type', 0),
pubkey=found_pubkey,
)
debug_print(
f"Route build: snapshot name fallback "
f"'{msg.sender}' → pubkey={found_pubkey[:16]!r}"
)
# --- Resolve path nodes (priority order) ---
# Priority 1: path_hashes from RX_LOG decode
rx_hashes = msg.path_hashes
if rx_hashes:
result['path_nodes'] = self._resolve_hashes(
rx_hashes, data['contacts'], msg.path_names,
)
result['path_source'] = 'rx_log'
debug_print(
f"Route from RX_LOG: {len(rx_hashes)} hashes → "
f"{len(result['path_nodes'])} nodes"
)
# Priority 2: out_path from sender's contact record
elif contact:
out_path = contact.get('out_path', '')
out_path_len = contact.get('out_path_len', 0)
debug_print(
f"Route: sender={contact.get('adv_name')}, "
f"out_path={out_path!r}, out_path_len={out_path_len}, "
f"msg_path_len={result['msg_path_len']}"
)
if out_path and out_path_len and out_path_len > 0:
result['path_nodes'] = self._parse_out_path(
out_path, out_path_len, data['contacts'],
)
result['path_source'] = 'contact_out_path'
# Determine if any node has GPS coordinates
all_nodes: List[RouteNode] = [result['self_node']]
if result['sender']:
all_nodes.append(result['sender'])
all_nodes.extend(result['path_nodes'])
result['has_locations'] = any(n.has_location for n in all_nodes)
return result
# ------------------------------------------------------------------
# Helpers
# ------------------------------------------------------------------
@staticmethod
def _resolve_hashes(
hashes: List[str],
contacts: Dict,
stored_names: Optional[List[str]] = None,
) -> List[RouteNode]:
"""Resolve a list of 1-byte path hashes into RouteNode objects.
Args:
hashes: List of 2-char hex strings.
contacts: Contact dict from snapshot.
stored_names: Pre-resolved names from the archive (same
length as *hashes*). Used as fallback when
the contact lookup fails (e.g. contact renamed
or not yet loaded).
"""
nodes: List[RouteNode] = []
for idx, hop_hash in enumerate(hashes):
if not hop_hash or len(hop_hash) < 2:
continue
hop_contact = RouteBuilder._find_contact_by_pubkey_hash(
hop_hash, contacts,
)
if hop_contact:
nodes.append(RouteNode(
name=hop_contact.get('adv_name') or f'0x{hop_hash}',
lat=hop_contact.get('adv_lat', 0),
lon=hop_contact.get('adv_lon', 0),
type=hop_contact.get('type', 0),
pubkey=hop_hash,
))
else:
# Fallback: use the name that was stored at receive time
fallback_name = '-'
if stored_names and idx < len(stored_names):
fallback_name = stored_names[idx] or '-'
if fallback_name == '-':
fallback_name = f'0x{hop_hash.upper()}'
nodes.append(RouteNode(
name=fallback_name,
pubkey=hop_hash,
))
return nodes
@staticmethod
def _parse_out_path(
out_path: str,
out_path_len: int,
contacts: Dict,
) -> List[RouteNode]:
"""Parse out_path hex string into a list of RouteNode objects."""
hashes: List[str] = []
hop_hex_len = 2
for i in range(0, min(len(out_path), out_path_len * 2), hop_hex_len):
hop_hash = out_path[i:i + hop_hex_len]
if hop_hash and len(hop_hash) == 2:
hashes.append(hop_hash)
return RouteBuilder._resolve_hashes(hashes, contacts)
@staticmethod
def _find_contact_by_pubkey_hash(
hash_hex: str, contacts: Dict,
) -> Optional[Dict]:
hash_hex = hash_hex.lower()
for pubkey, contact in contacts.items():
if pubkey.lower().startswith(hash_hex):
return contact
return None
@staticmethod
def _find_contact_by_pubkey(
pubkey_prefix: str, contacts: Dict,
) -> Optional[Dict]:
"""Find a contact by full or partial pubkey (bidirectional prefix match).
Mirrors the matching logic of
:meth:`SharedData.get_contact_by_prefix` but operates on the
snapshot contacts dict directly, avoiding a lock acquisition.
Args:
pubkey_prefix: Full or partial public key (hex string).
contacts: Contact dict from snapshot.
Returns:
Contact dict or ``None``.
"""
if not pubkey_prefix:
return None
prefix_lower = pubkey_prefix.lower()
for key, contact in contacts.items():
key_lower = key.lower()
if key_lower.startswith(prefix_lower) or prefix_lower.startswith(key_lower):
return contact
return None
@staticmethod
def _find_contact_by_adv_name(
name: str, contacts: Dict,
) -> Optional[tuple]:
"""Find a contact by advertised name (case-insensitive).
Mirrors the matching logic of
:meth:`SharedData.get_contact_by_name` but operates on the
snapshot contacts dict directly.
Args:
name: Display name to search for.
contacts: Contact dict from snapshot.
Returns:
``(pubkey, contact_dict)`` tuple or ``None``.
"""
if not name:
return None
name_lower = name.lower()
# Exact match first
for key, contact in contacts.items():
if contact.get('adv_name', '') == name:
return (key, contact)
# Case-insensitive fallback
for key, contact in contacts.items():
if contact.get('adv_name', '').lower() == name_lower:
return (key, contact)
return None

BIN
meshcore_gui/static/icon-192.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 7.8 KiB

BIN
meshcore_gui/static/icon-512.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 22 KiB

34
meshcore_gui/static/icon.svg Normal file
View File

@@ -0,0 +1,34 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512">
<rect width="512" height="512" rx="80" fill="#0A1628"/>
<g transform="translate(256,180)">
<!-- Mesh lines -->
<g stroke="#0077B6" stroke-width="8" opacity="0.6">
<line x1="0" y1="-80" x2="-100" y2="80"/>
<line x1="0" y1="-80" x2="100" y2="80"/>
<line x1="-100" y1="80" x2="100" y2="80"/>
<line x1="0" y1="-80" x2="0" y2="20"/>
<line x1="-100" y1="80" x2="0" y2="20"/>
<line x1="100" y1="80" x2="0" y2="20"/>
</g>
<!-- Outer nodes -->
<circle cx="0" cy="-80" r="18" fill="#00B4D8"/>
<circle cx="0" cy="-80" r="8" fill="#CAF0F8"/>
<circle cx="-100" cy="80" r="18" fill="#00B4D8"/>
<circle cx="-100" cy="80" r="8" fill="#CAF0F8"/>
<circle cx="100" cy="80" r="18" fill="#00B4D8"/>
<circle cx="100" cy="80" r="8" fill="#CAF0F8"/>
<!-- Hub node -->
<circle cx="0" cy="20" r="26" fill="#00B4D8"/>
<circle cx="0" cy="20" r="12" fill="#CAF0F8"/>
<text x="0" y="26" text-anchor="middle" font-family="monospace"
font-size="18" font-weight="700" fill="#0A1628">NL</text>
<!-- Radio waves -->
<g fill="none" stroke="#48CAE4" stroke-width="4">
<path d="M 30 5 Q 55 20 30 35" opacity="0.5"/>
<path d="M 42 -5 Q 75 20 42 45" opacity="0.35"/>
</g>
</g>
<!-- DOMCA text -->
<text x="256" y="380" text-anchor="middle" font-family="sans-serif"
font-size="72" font-weight="800" fill="#48CAE4" letter-spacing="6">DOMCA</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 1.5 KiB

77
meshcore_gui/static/landing_default.svg Normal file
View File

@@ -0,0 +1,77 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 800 400">
<!--
DOMCA default landing page SVG.
Uses CSS variables from the DOMCA theme (see dashboard.py _DOMCA_HEAD).
Placeholder:
{callsign} — replaced at runtime by config.OPERATOR_CALLSIGN
To create a custom landing page:
1. Copy this file (or create your own SVG)
2. Use {callsign} wherever you want the operator callsign inserted
3. Point config.LANDING_SVG_PATH to your file
-->
<defs>
<pattern id="gp" width="20" height="20" patternUnits="userSpaceOnUse">
<path d="M 20 0 L 0 0 0 20" fill="none" stroke="var(--grid)" stroke-width="0.3"/>
</pattern>
</defs>
<rect width="800" height="400" fill="var(--bg)"/>
<rect width="800" height="400" fill="url(#gp)" opacity="var(--grid-op)"/>
<g opacity="var(--mesh-bg-op)" stroke="var(--mesh-bg)" stroke-width="1">
<line x1="50" y1="80" x2="200" y2="150"/><line x1="200" y1="150" x2="150" y2="300"/>
<line x1="150" y1="300" x2="350" y2="250"/><line x1="350" y1="250" x2="200" y2="150"/>
<line x1="600" y1="100" x2="750" y2="180"/><line x1="750" y1="180" x2="680" y2="320"/>
<line x1="680" y1="320" x2="550" y2="280"/><line x1="550" y1="280" x2="600" y2="100"/>
</g>
<g transform="translate(80,80)">
<g stroke="var(--line)" stroke-width="2.5" opacity="var(--line-op)">
<line x1="100" y1="30" x2="45" y2="120"/><line x1="100" y1="30" x2="155" y2="120"/>
<line x1="45" y1="120" x2="155" y2="120"/><line x1="100" y1="30" x2="100" y2="95"/>
<line x1="45" y1="120" x2="100" y2="95"/><line x1="155" y1="120" x2="100" y2="95"/>
<line x1="100" y1="30" x2="100" y2="5"/><line x1="45" y1="120" x2="15" y2="145"/>
<line x1="155" y1="120" x2="185" y2="145"/><line x1="100" y1="95" x2="30" y2="60"/>
<line x1="100" y1="95" x2="170" y2="60"/><line x1="100" y1="95" x2="100" y2="160"/>
</g>
<g fill="none" stroke="var(--wave)" stroke-width="1.5">
<path d="M 115 80 Q 135 95 115 110" opacity="0.5"/>
<path d="M 125 72 Q 155 95 125 118" opacity="0.35"/>
<path d="M 135 64 Q 175 95 135 126" opacity="0.2"/>
</g>
<circle cx="100" cy="30" r="7" fill="var(--node)"/>
<circle cx="100" cy="30" r="3" fill="var(--node-center)"/>
<circle cx="45" cy="120" r="7" fill="var(--node)"/>
<circle cx="45" cy="120" r="3" fill="var(--node-center)"/>
<circle cx="155" cy="120" r="7" fill="var(--node)"/>
<circle cx="155" cy="120" r="3" fill="var(--node-center)"/>
<circle cx="100" cy="95" r="10" fill="var(--node)"/>
<circle cx="100" cy="95" r="4.5" fill="var(--node-center)"/>
<circle cx="100" cy="5" r="4" fill="var(--outer)" opacity="0.5"/>
<circle cx="15" cy="145" r="4" fill="var(--outer)" opacity="0.5"/>
<circle cx="185" cy="145" r="4" fill="var(--outer)" opacity="0.5"/>
<circle cx="30" cy="60" r="4" fill="var(--outer)" opacity="0.4"/>
<circle cx="170" cy="60" r="4" fill="var(--outer)" opacity="0.4"/>
<circle cx="100" cy="160" r="4" fill="var(--outer)" opacity="0.4"/>
<text x="100" y="99" text-anchor="middle" font-family="'JetBrains Mono',monospace"
font-size="9" font-weight="700" fill="var(--hub-text)" letter-spacing="1">NL</text>
</g>
<g transform="translate(310,110)">
<text x="0" y="65" font-family="'Exo 2',sans-serif" font-size="92"
font-weight="800" fill="var(--title)" letter-spacing="8">DOMCA</text>
<rect x="2" y="85" width="410" height="2" fill="var(--title)" rx="1"/>
<text x="3" y="112" font-family="'JetBrains Mono',monospace" font-size="14.5"
fill="var(--subtitle)" letter-spacing="4.5" opacity="0.9"
>DUTCH OPEN MESHCORE ACTIVITY</text>
<text x="3" y="140" font-family="'JetBrains Mono',monospace" font-size="11"
fill="var(--tagline)" letter-spacing="2" opacity="var(--tag-op)"
>CONNECTING THE MESH · SINCE 2025</text>
</g>
<g transform="translate(600,330)">
<rect width="140" height="32" rx="16" fill="none"
stroke="var(--badge-stroke)" stroke-width="1.5" opacity="0.5"/>
<text x="70" y="21" text-anchor="middle" font-family="'JetBrains Mono',monospace"
font-size="13" fill="var(--badge-text)" letter-spacing="2" opacity="0.7">domca.nl</text>
</g>
<text x="80" y="365" font-family="'JetBrains Mono',monospace" font-size="10"
fill="var(--callsign)" letter-spacing="2" opacity="0.3">{callsign}</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 4.3 KiB

64
meshcore_gui/static/leaflet_map_panel.css Normal file
View File

@@ -0,0 +1,64 @@
.meshcore-leaflet-host {
width: 100%;
height: 100%;
min-height: inherit;
}
.meshcore-leaflet-marker {
width: 34px;
height: 34px;
border-radius: 9999px;
display: flex;
align-items: center;
justify-content: center;
font-size: 16px;
border: 2px solid rgba(255, 255, 255, 0.92);
box-shadow: 0 3px 12px rgba(0, 0, 0, 0.28);
background: #2563eb;
color: #ffffff;
}
.meshcore-leaflet-marker.is-device {
width: 40px;
height: 40px;
font-size: 18px;
background: #dc2626;
}
.meshcore-leaflet-marker.is-unknown { background: #475569; }
.meshcore-leaflet-marker.is-companion { background: #2563eb; }
.meshcore-leaflet-marker.is-repeater { background: #059669; }
.meshcore-leaflet-marker.is-room { background: #7c3aed; }
.meshcore-leaflet-popup {
font-family: 'JetBrains Mono', monospace;
font-size: 0.75rem;
line-height: 1.45;
}
.meshcore-leaflet-popup strong {
display: block;
margin-bottom: 0.2rem;
}
.meshcore-marker-cluster {
background: transparent;
}
.meshcore-marker-cluster div {
width: 42px;
height: 42px;
border-radius: 9999px;
display: flex;
align-items: center;
justify-content: center;
background: rgba(37, 99, 235, 0.92);
border: 3px solid rgba(255, 255, 255, 0.92);
box-shadow: 0 3px 12px rgba(0, 0, 0, 0.28);
}
.meshcore-marker-cluster span {
color: #ffffff;
font-weight: 700;
font-size: 0.9rem;
}

637
meshcore_gui/static/leaflet_map_panel.js Normal file
View File

@@ -0,0 +1,637 @@
(function () {
const DEFAULT_CENTER = [52.5164, 6.083];
const DEFAULT_ZOOM = 13;
const RETRY_DELAY_MS = 60;
const MAX_RETRIES = 200;
const PANEL = window.MeshCoreLeafletPanel = window.MeshCoreLeafletPanel || {};
const maps = PANEL.maps = PANEL.maps || new Map();
const pending = PANEL.pending = PANEL.pending || new Map();
const watchers = PANEL.watchers = PANEL.watchers || new Map();
const preferences = PANEL.preferences = PANEL.preferences || new Map();
const THEME_STORAGE_KEY = 'meshcore_leaflet_theme';
function loadStoredTheme() {
try {
return window.localStorage ? window.localStorage.getItem(THEME_STORAGE_KEY) : null;
} catch (error) {
return null;
}
}
function storeTheme(theme) {
try {
if (!window.localStorage) {
return;
}
if (theme) {
window.localStorage.setItem(THEME_STORAGE_KEY, theme);
} else {
window.localStorage.removeItem(THEME_STORAGE_KEY);
}
} catch (error) {
// ignore storage errors
}
}
PANEL.ensureMap = function (containerId) {
const existing = maps.get(containerId);
const host = document.getElementById(containerId);
if (!host || typeof window.L === 'undefined' || typeof window.L.markerClusterGroup !== 'function') {
return null;
}
if (existing) {
if (existing.host !== host) {
if (existing.resizeObserver) {
existing.resizeObserver.disconnect();
}
existing.host = host;
host.__meshcoreLeafletState = existing;
}
PANEL.invalidate(containerId);
return existing;
}
if (host.__meshcoreLeafletState) {
maps.set(containerId, host.__meshcoreLeafletState);
PANEL.invalidate(containerId);
return host.__meshcoreLeafletState;
}
if (host._leaflet_id) {
throw new Error('Leaflet host already has a map but MeshCore runtime has no state; hard refresh required after previous failed init.');
}
const map = window.L.map(host, {
center: DEFAULT_CENTER,
zoom: DEFAULT_ZOOM,
minZoom: 2,
maxZoom: 19,
zoomControl: true,
preferCanvas: true,
});
const state = {
containerId,
map,
host,
theme: null,
layers: {
base: null,
contacts: null,
device: window.L.layerGroup().addTo(map),
},
contactMarkers: new Map(),
deviceMarker: null,
hasCentered: false,
pendingInvalidate: false,
userInteracting: false,
interactionCooldownTimer: null,
resizeObserver: null,
lastCenter: map.getCenter(),
lastZoom: map.getZoom(),
};
maps.set(containerId, state);
host.__meshcoreLeafletState = state;
try {
state.layers.base = window.L.tileLayer(
'https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png',
{
attribution:
'&copy; <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a> contributors',
maxZoom: 19,
}
).addTo(map);
state.theme = 'light';
state.layers.contacts = window.L.markerClusterGroup({
showCoverageOnHover: false,
spiderfyOnMaxZoom: true,
removeOutsideVisibleBounds: true,
animate: false,
chunkedLoading: true,
maxClusterRadius: 50,
iconCreateFunction(cluster) {
return window.L.divIcon({
html: '<div><span>' + cluster.getChildCount() + '</span></div>',
className: 'meshcore-marker-cluster',
iconSize: window.L.point(42, 42),
});
},
}).addTo(map);
} catch (error) {
maps.delete(containerId);
delete host.__meshcoreLeafletState;
try {
map.remove();
} catch (removeError) {
console.warn('Leaflet cleanup after failed init also failed', removeError);
}
throw error;
}
map.on('zoomstart movestart dragstart resize', () => {
state.userInteracting = true;
if (state.interactionCooldownTimer) {
window.clearTimeout(state.interactionCooldownTimer);
state.interactionCooldownTimer = null;
}
});
const endInteraction = function () {
state.lastCenter = state.map.getCenter();
state.lastZoom = state.map.getZoom();
if (state.interactionCooldownTimer) {
window.clearTimeout(state.interactionCooldownTimer);
}
state.interactionCooldownTimer = window.setTimeout(() => {
state.userInteracting = false;
state.interactionCooldownTimer = null;
}, 350);
};
map.on('zoomend moveend dragend', endInteraction);
if (window.ResizeObserver) {
state.resizeObserver = new window.ResizeObserver(() => {
PANEL.invalidate(containerId);
});
state.resizeObserver.observe(host);
}
const preference = preferences.get(containerId) || {};
if (!preference.theme) {
const storedTheme = loadStoredTheme();
if (storedTheme) {
preference.theme = storedTheme;
preferences.set(containerId, preference);
}
}
if (preference.theme) {
PANEL.setTheme(containerId, preference.theme);
}
PANEL.invalidate(containerId);
return state;
};
PANEL.invalidate = function (containerId) {
const state = maps.get(containerId);
if (!state || state.pendingInvalidate) {
return;
}
state.pendingInvalidate = true;
window.requestAnimationFrame(() => {
state.pendingInvalidate = false;
try {
state.map.invalidateSize({ pan: false, debounceMoveend: true });
} catch (error) {
console.warn('Leaflet invalidateSize failed', error);
}
});
};
PANEL.setTheme = function (containerId, theme) {
const state = maps.get(containerId);
if (!state || !theme || state.theme === theme || typeof window.L === 'undefined') {
return;
}
if (state.layers.base) {
state.map.removeLayer(state.layers.base);
state.layers.base = null;
}
const dark = theme === 'dark';
const url = dark
? 'https://{s}.basemaps.cartocdn.com/dark_all/{z}/{x}/{y}{r}.png'
: 'https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png';
const options = dark
? {
attribution:
'&copy; <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a> contributors &copy; <a href="https://carto.com/attributions">CARTO</a>',
maxZoom: 20,
}
: {
attribution:
'&copy; <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a> contributors',
maxZoom: 19,
};
state.layers.base = window.L.tileLayer(url, options).addTo(state.map);
state.theme = theme;
storeTheme(theme);
};
PANEL.centerOnDevice = function (containerId) {
const state = maps.get(containerId);
if (!state || !state.deviceMarker) {
return;
}
const latLng = state.deviceMarker.getLatLng();
state.map.setView(latLng, state.map.getZoom(), { animate: false });
state.lastCenter = latLng;
state.lastZoom = state.map.getZoom();
state.hasCentered = true;
};
PANEL.applySnapshot = function (containerId, snapshot) {
const state = maps.get(containerId);
if (!state || !snapshot) {
return;
}
const preference = preferences.get(containerId);
if (preference && preference.theme) {
PANEL.setTheme(containerId, preference.theme);
}
applyDevice(state, snapshot.device);
applyContacts(state, snapshot.contacts || []);
const shouldCenter = Boolean(
state.deviceMarker &&
!state.userInteracting &&
(snapshot.force_center || !state.hasCentered)
);
if (shouldCenter) {
const latLng = state.deviceMarker.getLatLng();
state.map.setView(latLng, state.map.getZoom(), { animate: false });
state.lastCenter = latLng;
state.lastZoom = state.map.getZoom();
state.hasCentered = true;
}
};
function applyDevice(state, device) {
if (!device) {
if (state.deviceMarker) {
state.layers.device.removeLayer(state.deviceMarker);
state.deviceMarker = null;
}
return;
}
const icon = buildIcon('📡', 'is-device', 'Device', [40, 40], [20, 20]);
const latLng = [device.lat, device.lon];
const popupHtml = popup(device.name, 'Device', 'local');
if (!state.deviceMarker) {
state.deviceMarker = window.L.marker(latLng, {
icon,
keyboard: false,
title: '📡 ' + device.name,
});
state.deviceMarker.bindPopup(popupHtml);
state.layers.device.addLayer(state.deviceMarker);
return;
}
state.deviceMarker.setLatLng(latLng);
state.deviceMarker.setIcon(icon);
state.deviceMarker.setPopupContent(popupHtml);
state.deviceMarker.options.title = '📡 ' + device.name;
}
function applyContacts(state, contacts) {
const nextIds = new Set();
for (const contact of contacts) {
nextIds.add(contact.id);
const existing = state.contactMarkers.get(contact.id);
const latLng = [contact.lat, contact.lon];
const markerIcon = buildTypeIcon(contact.node_type);
const markerTitle = markerTitlePrefix(contact.node_type) + ' ' + contact.name;
const popupHtml = popup(contact.name, labelForType(contact.node_type), contact.short_key);
if (!existing) {
const marker = window.L.marker(latLng, {
icon: markerIcon,
keyboard: false,
title: markerTitle,
});
marker.bindPopup(popupHtml);
state.layers.contacts.addLayer(marker);
state.contactMarkers.set(contact.id, marker);
continue;
}
existing.setLatLng(latLng);
existing.setIcon(markerIcon);
existing.setPopupContent(popupHtml);
existing.options.title = markerTitle;
if (!state.layers.contacts.hasLayer(existing)) {
state.layers.contacts.addLayer(existing);
}
}
for (const [contactId, marker] of state.contactMarkers.entries()) {
if (!nextIds.has(contactId)) {
state.layers.contacts.removeLayer(marker);
state.contactMarkers.delete(contactId);
}
}
}
function buildTypeIcon(nodeType) {
switch (nodeType) {
case 1:
return buildIcon('📱', 'is-companion', 'Companion');
case 2:
return buildIcon('📡', 'is-repeater', 'Repeater');
case 3:
return buildIcon('🏠', 'is-room', 'Room Server');
default:
return buildIcon('○', 'is-unknown', 'Unknown');
}
}
PANEL.buildTypeIcon = buildTypeIcon;
PANEL.markerTitlePrefix = markerTitlePrefix;
PANEL.labelForType = labelForType;
PANEL.buildPopupHtml = popup;
function markerTitlePrefix(nodeType) {
switch (nodeType) {
case 1:
return '📱';
case 2:
return '📡';
case 3:
return '🏠';
default:
return '○';
}
}
function labelForType(nodeType) {
switch (nodeType) {
case 1:
return 'Companion';
case 2:
return 'Repeater';
case 3:
return 'Room Server';
default:
return 'Unknown';
}
}
function buildIcon(symbol, extraClass, label, iconSize, iconAnchor) {
const resolvedSize = iconSize || [34, 34];
const resolvedAnchor = iconAnchor || [17, 17];
return window.L.divIcon({
className: '',
html:
'<div class="meshcore-leaflet-marker ' + extraClass + '" aria-label="' + label + '">' +
symbol +
'</div>',
iconSize: resolvedSize,
iconAnchor: resolvedAnchor,
popupAnchor: [0, -16],
});
}
function popup(name, label, shortKey) {
return (
'<div class="meshcore-leaflet-popup">' +
'<strong>' + escapeHtml(name) + '</strong>' +
'<div>Type: ' + escapeHtml(label) + '</div>' +
'<div>Key: ' + escapeHtml(shortKey) + '</div>' +
'</div>'
);
}
function escapeHtml(value) {
return String(value)
.replaceAll('&', '&amp;')
.replaceAll('<', '&lt;')
.replaceAll('>', '&gt;')
.replaceAll('"', '&quot;')
.replaceAll("'", '&#039;');
}
function scheduleProcess(containerId, retries) {
window.requestAnimationFrame(() => {
processPending(containerId, retries || 0);
});
}
function processPending(containerId, retries) {
const payload = pending.get(containerId);
if (!payload) {
return;
}
if (!isDomReady()) {
if (retries >= MAX_RETRIES) {
console.error('MeshCoreLeafletBoot timeout waiting for DOM readiness', { containerId });
return;
}
window.setTimeout(() => {
scheduleProcess(containerId, retries + 1);
}, RETRY_DELAY_MS);
return;
}
const host = document.getElementById(containerId);
if (!host) {
watchForHost(containerId, retries);
return;
}
if (typeof window.L === 'undefined' || typeof window.L.markerClusterGroup !== 'function') {
if (retries >= MAX_RETRIES) {
console.error('MeshCoreLeafletBoot timeout waiting for Leaflet markercluster', { containerId });
return;
}
window.setTimeout(() => {
scheduleProcess(containerId, retries + 1);
}, RETRY_DELAY_MS);
return;
}
try {
const state = PANEL.ensureMap(containerId);
if (!state) {
return;
}
const current = pending.get(containerId);
if (!current) {
return;
}
if (current.theme) {
PANEL.setTheme(containerId, current.theme);
}
if (current.snapshot && current.snapshot.__command__ === 'center_on_device') {
PANEL.centerOnDevice(containerId);
} else if (current.snapshot && current.snapshot.__command__ === 'ensure_map') {
// map has already been ensured above; no-op
} else if (current.snapshot) {
PANEL.applySnapshot(containerId, current.snapshot);
}
pending.delete(containerId);
} catch (error) {
console.error('MeshCoreLeafletBoot failed', error);
}
}
function watchForHost(containerId, retries) {
if (watchers.has(containerId)) {
return;
}
const observer = new MutationObserver(() => {
const host = document.getElementById(containerId);
if (!host) {
return;
}
observer.disconnect();
watchers.delete(containerId);
scheduleProcess(containerId, retries + 1);
});
observer.observe(document.body, {
childList: true,
subtree: true,
});
watchers.set(containerId, observer);
window.setTimeout(() => {
if (watchers.get(containerId) !== observer) {
return;
}
observer.disconnect();
watchers.delete(containerId);
if (retries >= MAX_RETRIES) {
console.error('MeshCoreLeafletBoot timeout waiting for host element', { containerId });
return;
}
scheduleProcess(containerId, retries + 1);
}, RETRY_DELAY_MS);
}
function isDomReady() {
return document.readyState === 'interactive' || document.readyState === 'complete';
}
window.MeshCoreRouteMapBoot = function (containerId, payload) {
if (!containerId || !payload) {
return;
}
const host = document.getElementById(containerId);
if (!host || typeof window.L === 'undefined') {
window.setTimeout(() => window.MeshCoreRouteMapBoot(containerId, payload), RETRY_DELAY_MS);
return;
}
if (host.__meshcoreRouteMap) {
try {
host.__meshcoreRouteMap.remove();
} catch (error) {
console.warn('MeshCoreRouteMap cleanup failed', error);
}
host.__meshcoreRouteMap = null;
host.innerHTML = '';
}
const map = window.L.map(host, {
center: payload.center || DEFAULT_CENTER,
zoom: payload.zoom || DEFAULT_ZOOM,
minZoom: 2,
maxZoom: 19,
zoomControl: true,
preferCanvas: true,
});
host.__meshcoreRouteMap = map;
window.L.tileLayer(
'https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png',
{
attribution:
'&copy; <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a> contributors',
maxZoom: 19,
}
).addTo(map);
const points = [];
for (const node of payload.nodes || []) {
if (typeof node.lat !== 'number' || typeof node.lon !== 'number') {
continue;
}
const latLng = [node.lat, node.lon];
points.push(latLng);
const nodeType = typeof node.node_type === 'number' ? node.node_type : 0;
const name = node.name || 'Unknown';
const shortKey = node.short_key || '-';
const role = node.role || labelForType(nodeType);
const marker = window.L.marker(latLng, {
icon: buildTypeIcon(nodeType),
keyboard: false,
title: markerTitlePrefix(nodeType) + ' ' + name,
});
marker.bindPopup(popup(name, role, shortKey));
marker.addTo(map);
}
if (points.length >= 2) {
window.L.polyline(points, { color: '#2563eb', weight: 3 }).addTo(map);
map.fitBounds(points, { padding: [24, 24], maxZoom: 16 });
} else if (points.length === 1) {
map.setView(points[0], payload.zoom || DEFAULT_ZOOM, { animate: false });
}
window.requestAnimationFrame(() => {
try {
map.invalidateSize({ pan: false, debounceMoveend: true });
} catch (error) {
console.warn('MeshCoreRouteMap invalidateSize failed', error);
}
});
};
window.MeshCoreLeafletBoot = function (containerId, snapshot, themeOnly) {
const current = pending.get(containerId) || { snapshot: null, theme: null };
const preference = preferences.get(containerId) || {};
if (!preference.theme) {
const storedTheme = loadStoredTheme();
if (storedTheme) {
preference.theme = storedTheme;
preferences.set(containerId, preference);
}
}
if (themeOnly) {
preference.theme = themeOnly;
preferences.set(containerId, preference);
current.theme = themeOnly;
storeTheme(themeOnly);
if (maps.has(containerId)) {
PANEL.setTheme(containerId, themeOnly);
}
} else if (!current.theme && preference.theme) {
current.theme = preference.theme;
}
if (snapshot) {
if (snapshot.__command__) {
current.snapshot = snapshot;
} else {
current.snapshot = { ...snapshot };
}
}
pending.set(containerId, current);
scheduleProcess(containerId, 0);
};
})();

24
meshcore_gui/static/manifest.json Normal file
View File

@@ -0,0 +1,24 @@
{
"name": "DOMCA MeshCore GUI",
"short_name": "DOMCA",
"description": "Dutch Open MeshCore Activity — Mesh network GUI",
"start_url": "/",
"display": "standalone",
"orientation": "any",
"background_color": "#0A1628",
"theme_color": "#0d1f35",
"icons": [
{
"src": "/static/icon-192.png",
"sizes": "192x192",
"type": "image/png",
"purpose": "any maskable"
},
{
"src": "/static/icon-512.png",
"sizes": "512x512",
"type": "image/png",
"purpose": "any maskable"
}
]
}

26
meshcore_observer.py Normal file
View File

@@ -0,0 +1,26 @@
#!/usr/bin/env python3
"""
MeshCore Observer — Read-Only Archive Monitor Dashboard
=========================================================
Standalone daemon that reads archive JSON files produced by
meshcore_gui and meshcore_bridge, aggregates them, and presents
a unified NiceGUI monitoring dashboard. 100% read-only — never
modifies archive files.
Usage:
python meshcore_observer.py
python meshcore_observer.py --config=observer_config.yaml
python meshcore_observer.py --port=9093
python meshcore_observer.py --debug-on
Author: PE1HVH
Version: 1.0.0
SPDX-License-Identifier: MIT
Copyright: (c) 2026 PE1HVH
"""
from meshcore_observer.__main__ import main
if __name__ == "__main__":
main()

9
meshcore_observer/__init__.py Normal file
View File

@@ -0,0 +1,9 @@
"""
MeshCore Observer — Read-Only Archive Monitor Dashboard.
Standalone daemon that reads archive JSON files produced by
meshcore_gui and meshcore_bridge, aggregates them, and presents
a unified NiceGUI monitoring dashboard.
"""
__version__ = "1.2.0"

243
meshcore_observer/__main__.py Normal file
View File

@@ -0,0 +1,243 @@
#!/usr/bin/env python3
"""
MeshCore Observer — Entry Point
=================================
Parses command-line arguments, loads YAML configuration, creates the
ArchiveWatcher, optionally starts the MQTT uplink, registers the
NiceGUI dashboard page and starts the server.
Usage:
python meshcore_observer.py
python meshcore_observer.py --config=observer_config.yaml
python meshcore_observer.py --port=9093
python meshcore_observer.py --debug-on
python meshcore_observer.py --mqtt-dry-run
Author: PE1HVH
Version: 1.1.0
SPDX-License-Identifier: MIT
Copyright: (c) 2026 PE1HVH
"""
import logging
import sys
from pathlib import Path
from nicegui import ui
from meshcore_observer import __version__
from meshcore_observer.config import ObserverConfig, DEFAULT_CONFIG_PATH
from meshcore_observer.archive_watcher import ArchiveWatcher
from meshcore_observer.gui.dashboard import ObserverDashboard
logger = logging.getLogger("meshcore_observer")
# Global instance (needed by NiceGUI page decorator)
_dashboard: ObserverDashboard | None = None
@ui.page("/")
def _page_dashboard():
"""NiceGUI page handler — observer dashboard."""
if _dashboard:
_dashboard.render()
def _print_usage():
"""Show usage information."""
print("MeshCore Observer — Read-Only Archive Monitor Dashboard")
print("=" * 58)
print()
print("Usage: python meshcore_observer.py [OPTIONS]")
print()
print("Options:")
print(" --config=PATH Path to observer_config.yaml (default: ./observer_config.yaml)")
print(" --port=PORT Override GUI port from config (default: 9093)")
print(" --debug-on Enable verbose debug logging")
print(" --mqtt-dry-run MQTT dry run: log payloads without publishing")
print(" --help Show this help message")
print()
print("Configuration:")
print(" All settings are defined in observer_config.yaml.")
print()
print("Examples:")
print(" python meshcore_observer.py")
print(" python meshcore_observer.py --config=/etc/meshcore/observer_config.yaml")
print(" python meshcore_observer.py --port=9093 --debug-on")
print(" python meshcore_observer.py --mqtt-dry-run")
def _parse_flags(argv):
"""Parse CLI arguments into a flag dict.
Handles ``--flag=value`` and boolean ``--flag``.
"""
flags = {}
for a in argv:
if "=" in a and a.startswith("--"):
key, value = a.split("=", 1)
flags[key] = value
elif a.startswith("--"):
flags[a] = True
return flags
def _setup_logging(debug: bool) -> None:
"""Configure logging for the observer process."""
level = logging.DEBUG if debug else logging.INFO
logging.basicConfig(
level=level,
format="%(asctime)s [%(name)s] %(levelname)s: %(message)s",
datefmt="%H:%M:%S",
)
def _create_mqtt_uplink(cfg: ObserverConfig):
"""Create and validate MQTT uplink if enabled.
Args:
cfg: Observer configuration.
Returns:
MqttUplink instance or None if disabled/invalid.
"""
if not cfg.mqtt.enabled:
logger.info("MQTT uplink: disabled")
return None
# Validate configuration
errors = cfg.mqtt.validate()
if errors:
for err in errors:
logger.error("MQTT config error: %s", err)
logger.error("MQTT uplink disabled due to configuration errors")
return None
try:
from meshcore_observer.mqtt_uplink import MqttUplink
except ImportError as exc:
logger.error("Cannot import MqttUplink: %s", exc)
logger.error(
"Install dependencies: pip install paho-mqtt PyNaCl"
)
return None
uplink = MqttUplink(cfg.mqtt, debug=cfg.debug)
mode = "DRY RUN" if cfg.mqtt.dry_run else "LIVE"
logger.info(
"MQTT uplink: enabled (%s) — IATA=%s, key=%s...",
mode, cfg.mqtt.iata, cfg.mqtt.resolve_public_key()[:12],
)
return uplink
def main():
"""Main entry point.
Loads configuration, creates ArchiveWatcher, optionally starts
MQTT uplink, starts the NiceGUI dashboard.
"""
global _dashboard
flags = _parse_flags(sys.argv[1:])
if "--help" in flags:
_print_usage()
sys.exit(0)
# ── Load configuration ──
config_path = Path(flags.get("--config", str(DEFAULT_CONFIG_PATH)))
if config_path.exists():
print(f"Loading config from: {config_path}")
cfg = ObserverConfig.from_yaml(config_path)
else:
print(f"Config not found at {config_path}, using defaults.")
print("Run with --help for usage information.")
cfg = ObserverConfig()
# ── CLI overrides ──
if "--debug-on" in flags:
cfg.debug = True
if "--port" in flags:
try:
cfg.gui_port = int(flags["--port"])
except ValueError:
print(f"ERROR: Invalid port: {flags['--port']}")
sys.exit(1)
if "--mqtt-dry-run" in flags:
cfg.mqtt.dry_run = True
# Also enable MQTT if not already
if not cfg.mqtt.enabled:
cfg.mqtt.enabled = True
cfg.config_path = str(config_path)
# ── Setup logging ──
_setup_logging(cfg.debug)
# ── Startup banner ──
print("=" * 58)
print("MeshCore Observer — Read-Only Archive Monitor Dashboard")
print("=" * 58)
print(f"Version: {__version__}")
print(f"Config: {config_path}")
print(f"Archive dir: {cfg.archive_dir}")
print(f"Poll interval:{cfg.poll_interval_s}s")
print(f"GUI port: {cfg.gui_port}")
print(f"Debug mode: {'ON' if cfg.debug else 'OFF'}")
print(f"MQTT uplink: {'ENABLED' if cfg.mqtt.enabled else 'DISABLED'}")
if cfg.mqtt.enabled:
mode = "DRY RUN" if cfg.mqtt.dry_run else "LIVE"
print(f"MQTT mode: {mode}")
print(f"MQTT IATA: {cfg.mqtt.iata}")
enabled_brokers = [b.name for b in cfg.mqtt.brokers if b.enabled]
print(f"MQTT brokers: {', '.join(enabled_brokers) or 'none'}")
print("=" * 58)
# ── Verify archive directory ──
archive_path = Path(cfg.archive_dir)
if not archive_path.exists():
logger.warning(
"Archive directory does not exist yet: %s — "
"will start scanning when it appears.",
cfg.archive_dir,
)
# ── Create ArchiveWatcher ──
watcher = ArchiveWatcher(cfg.archive_dir, debug=cfg.debug)
# ── Create MQTT uplink (if enabled) ──
mqtt_uplink = _create_mqtt_uplink(cfg)
if mqtt_uplink:
mqtt_uplink.start()
# ── Create dashboard ──
_dashboard = ObserverDashboard(watcher, cfg, mqtt_uplink=mqtt_uplink)
# ── Start NiceGUI server (blocks) ──
print(f"Starting GUI on port {cfg.gui_port}...")
try:
ui.run(
show=False,
host="0.0.0.0",
title=cfg.gui_title,
port=cfg.gui_port,
reload=False,
storage_secret="meshcore-observer-secret",
)
finally:
# Graceful MQTT shutdown
if mqtt_uplink:
mqtt_uplink.shutdown()
if __name__ == "__main__":
main()

274
meshcore_observer/archive_watcher.py Normal file
View File

@@ -0,0 +1,274 @@
"""
Archive file watcher for MeshCore Observer.
Scans the archive directory for ``*_messages.json`` and ``*_rxlog.json``
files, tracks their modification times, and returns only NEW entries
since the previous poll. 100% read-only — never writes to archive files.
Thread safety: all public methods acquire ``_lock`` before touching state.
"""
import json
import logging
import threading
from dataclasses import dataclass, field
from pathlib import Path
from typing import Dict, List, Optional, Tuple
logger = logging.getLogger(__name__)
ARCHIVE_VERSION = 1
@dataclass
class _FileState:
"""Tracking state for a single archive JSON file."""
path: Path
last_mtime: float = 0.0
last_entry_count: int = 0
address: str = ""
@dataclass
class PollResult:
"""New entries discovered during a single poll cycle.
Attributes:
new_messages: List of (source_address, message_dict) tuples.
new_rxlog: List of (source_address, entry_dict) tuples.
"""
new_messages: List[Tuple[str, dict]] = field(default_factory=list)
new_rxlog: List[Tuple[str, dict]] = field(default_factory=list)
class ArchiveWatcher:
"""Polls archive directory for new messages and RX log entries.
Designed for timer-based polling from the NiceGUI main thread.
Each call to :meth:`poll` scans the archive directory, detects
changed files via ``stat().st_mtime``, reads only changed files,
and returns new entries as a :class:`PollResult`.
Args:
archive_dir: Path to the archive directory.
debug: Enable verbose debug logging.
"""
def __init__(self, archive_dir: str, debug: bool = False) -> None:
self._archive_dir = Path(archive_dir).expanduser().resolve()
self._debug = debug
self._lock = threading.Lock()
# Tracking state: filepath_str → _FileState
self._msg_files: Dict[str, _FileState] = {}
self._rxlog_files: Dict[str, _FileState] = {}
# Aggregated totals
self._total_messages_seen: int = 0
self._total_rxlog_seen: int = 0
# ------------------------------------------------------------------
# Public API
# ------------------------------------------------------------------
def poll(self) -> PollResult:
"""Scan archive directory and return new entries since last poll.
Returns:
PollResult with new messages and RX log entries.
Empty lists when nothing has changed (no disk I/O).
"""
with self._lock:
result = PollResult()
if not self._archive_dir.exists():
return result
# Discover current files on disk
current_msg_paths = set()
current_rxlog_paths = set()
try:
for f in self._archive_dir.iterdir():
name = f.name
if name.endswith("_messages.json"):
current_msg_paths.add(str(f))
elif name.endswith("_rxlog.json"):
current_rxlog_paths.add(str(f))
except OSError as exc:
logger.error("Error scanning archive dir: %s", exc)
return result
# Prune vanished files
vanished_msg = set(self._msg_files.keys()) - current_msg_paths
for vp in vanished_msg:
del self._msg_files[vp]
vanished_rxlog = set(self._rxlog_files.keys()) - current_rxlog_paths
for vp in vanished_rxlog:
del self._rxlog_files[vp]
# Check message files
for fpath_str in current_msg_paths:
new_entries = self._check_file(
fpath_str, self._msg_files, "messages",
)
if new_entries:
result.new_messages.extend(new_entries)
self._total_messages_seen += len(new_entries)
# Check rxlog files
for fpath_str in current_rxlog_paths:
new_entries = self._check_file(
fpath_str, self._rxlog_files, "entries",
)
if new_entries:
result.new_rxlog.extend(new_entries)
self._total_rxlog_seen += len(new_entries)
return result
def get_sources(self) -> List[Dict]:
"""Return metadata about all tracked archive sources.
Returns:
List of dicts with keys: address, path, last_updated, message_count,
rxlog_count.
"""
with self._lock:
# Collect per-address info
sources: Dict[str, Dict] = {}
for fpath_str, state in self._msg_files.items():
addr = state.address or Path(fpath_str).stem
if addr not in sources:
sources[addr] = {
"address": addr,
"path": str(Path(fpath_str).parent),
"last_updated": "",
"message_count": 0,
"rxlog_count": 0,
}
sources[addr]["message_count"] = state.last_entry_count
sources[addr]["path"] = fpath_str
for fpath_str, state in self._rxlog_files.items():
addr = state.address or Path(fpath_str).stem
if addr not in sources:
sources[addr] = {
"address": addr,
"path": fpath_str,
"last_updated": "",
"message_count": 0,
"rxlog_count": 0,
}
sources[addr]["rxlog_count"] = state.last_entry_count
return list(sources.values())
def get_stats(self) -> Dict:
"""Return aggregate statistics.
Returns:
Dict with total_messages_seen, total_rxlog_seen, active_sources.
"""
with self._lock:
addresses = set()
for state in self._msg_files.values():
if state.address:
addresses.add(state.address)
for state in self._rxlog_files.values():
if state.address:
addresses.add(state.address)
return {
"total_messages_seen": self._total_messages_seen,
"total_rxlog_seen": self._total_rxlog_seen,
"active_sources": len(addresses),
}
# ------------------------------------------------------------------
# Internal
# ------------------------------------------------------------------
def _check_file(
self,
fpath_str: str,
tracking: Dict[str, _FileState],
entries_key: str,
) -> List[Tuple[str, dict]]:
"""Check a single file for new entries.
Args:
fpath_str: Absolute path string.
tracking: Dict of _FileState for this file category.
entries_key: JSON key containing the entry list ("messages" or "entries").
Returns:
List of (source_address, entry_dict) for new entries, or empty list.
"""
fpath = Path(fpath_str)
try:
current_mtime = fpath.stat().st_mtime
except OSError:
# File vanished between iterdir and stat
tracking.pop(fpath_str, None)
return []
# New file — start tracking
if fpath_str not in tracking:
tracking[fpath_str] = _FileState(path=fpath)
state = tracking[fpath_str]
# Unchanged file — no I/O needed
if current_mtime == state.last_mtime:
return []
# File changed — read and parse
state.last_mtime = current_mtime
try:
raw_text = fpath.read_text(encoding="utf-8")
data = json.loads(raw_text)
except (json.JSONDecodeError, OSError) as exc:
logger.error("Error reading %s: %s", fpath, exc)
return []
# Version check
if data.get("version") != ARCHIVE_VERSION:
if self._debug:
logger.debug(
"Skipping %s: version %s (expected %d)",
fpath.name, data.get("version"), ARCHIVE_VERSION,
)
return []
# Extract source address
address = data.get("address", fpath.stem)
state.address = address
entries = data.get(entries_key, [])
total_count = len(entries)
prev_count = state.last_entry_count
# Detect new entries (append-only assumption)
new_entries: List[Tuple[str, dict]] = []
if total_count > prev_count:
for entry in entries[prev_count:]:
# Tag each entry with source address
entry["_source"] = address
new_entries.append((address, entry))
state.last_entry_count = total_count
if new_entries and self._debug:
logger.debug(
"%s: %d new %s (total: %d)",
fpath.name, len(new_entries), entries_key, total_count,
)
return new_entries

348
meshcore_observer/auth_token.py Normal file
View File

@@ -0,0 +1,348 @@
"""
Ed25519 JWT authentication token for LetsMesh MQTT broker.
Generates tokens compatible with the ``@michaelhart/meshcore-decoder``
``createAuthToken()`` reference implementation.
Private key formats supported:
- **128 hex chars** (64 bytes): Full orlp/ed25519 expanded key as stored
in ``device_identity.json`` by ``device_identity.py``.
Format: ``[clamped_scalar(32)][nonce_prefix(32)]``.
- **64 hex chars** (32 bytes): Legacy Ed25519 seed. Works with PyNaCl
fallback and with Node.js (seed + pubkey concatenation).
Strategy:
1. **Node.js** — calls meshcore-decoder directly (reference impl)
2. **PyNaCl** — pure Python fallback (seed-only, 64-char keys)
Author: PE1HVH
Version: 2.1.0
SPDX-License-Identifier: MIT
Copyright: (c) 2026 PE1HVH
"""
import base64
import json
import logging
import os
import shutil
import subprocess
import time
from pathlib import Path
from typing import Optional
logger = logging.getLogger(__name__)
# ── Constants ────────────────────────────────────────────────────────
DEFAULT_TOKEN_LIFETIME_S = 3600 # 1 hour
TOKEN_REFRESH_MARGIN_S = 300 # Refresh 5 minutes before expiry
VALID_PRIVATE_KEY_LENGTHS = (64, 128) # 32-byte seed or 64-byte expanded
def _resolve_node_path() -> str:
"""Resolve NODE_PATH — check common global install locations."""
env_val = os.environ.get("NODE_PATH", "")
if env_val:
return env_val
# Common locations: Debian/Ubuntu, npm global on Pi/macOS, nvm
for candidate in (
"/usr/lib/node_modules",
"/usr/local/lib/node_modules",
Path.home() / ".npm-global" / "lib" / "node_modules",
):
if Path(candidate).is_dir():
return str(candidate)
return "/usr/lib/node_modules" # fallback
_NODE_ENV = {**os.environ, "NODE_PATH": _resolve_node_path()}
_node_available: Optional[bool] = None
# ── Key helpers ──────────────────────────────────────────────────────
def _is_valid_hex(value: str, allowed_lengths: tuple) -> bool:
"""Check if *value* is a hex string with one of *allowed_lengths*."""
try:
bytes.fromhex(value)
except ValueError:
return False
return len(value) in allowed_lengths
def _build_nodejs_private_key(
private_key_hex: str,
public_key_hex: str,
) -> str:
"""Return the 128-char hex private key meshcore-decoder expects.
- 128-char input → already complete orlp expanded key; pass through.
- 64-char input → legacy seed; concatenate ``seed + pubkey``.
"""
if len(private_key_hex) == 128:
return private_key_hex
return private_key_hex + public_key_hex.lower()
# ── Node.js strategy ────────────────────────────────────────────────
def _check_node_available() -> bool:
"""Check if Node.js and meshcore-decoder are available."""
global _node_available
if _node_available is not None:
return _node_available
if not shutil.which("node"):
logger.debug("Node.js not found in PATH")
_node_available = False
return False
try:
result = subprocess.run(
["node", "-e",
"require('@michaelhart/meshcore-decoder').createAuthToken"],
env=_NODE_ENV,
capture_output=True,
timeout=5,
)
_node_available = result.returncode == 0
if _node_available:
logger.info("Using Node.js meshcore-decoder for MQTT auth tokens")
else:
logger.debug(
"meshcore-decoder not available: %s",
result.stderr.decode().strip(),
)
except Exception as exc:
logger.debug("Node.js check failed: %s", exc)
_node_available = False
return _node_available
def _create_token_nodejs(
public_key_hex: str,
private_key_hex: str,
audience: str,
lifetime_s: int,
) -> str:
"""Create auth token via Node.js meshcore-decoder.
Handles both 64-char seeds (concatenated with pubkey) and
128-char orlp expanded keys (passed directly).
"""
full_priv = _build_nodejs_private_key(private_key_hex, public_key_hex)
pub_upper = public_key_hex.upper()
js_code = f"""
const {{ createAuthToken }} = require('@michaelhart/meshcore-decoder');
(async () => {{
const payload = {{
publicKey: '{pub_upper}',
aud: '{audience}',
iat: Math.floor(Date.now() / 1000),
exp: Math.floor(Date.now() / 1000) + {lifetime_s}
}};
const token = await createAuthToken(payload, '{full_priv}', '{pub_upper}');
process.stdout.write(token);
}})();
"""
result = subprocess.run(
["node", "-e", js_code],
env=_NODE_ENV,
capture_output=True,
timeout=10,
)
if result.returncode != 0:
stderr = result.stderr.decode().strip()
raise RuntimeError(f"Node.js token generation failed: {stderr}")
token = result.stdout.decode().strip()
if not token or token.count(".") != 2:
raise RuntimeError(
f"Node.js returned invalid token: {token[:50]}..."
)
return token
# ── PyNaCl strategy ──────────────────────────────────────────────────
def _base64url_encode(data: bytes) -> str:
"""Base64url encode without padding (JWT standard)."""
return base64.urlsafe_b64encode(data).rstrip(b"=").decode("ascii")
def _create_token_pynacl(
public_key_hex: str,
private_key_hex: str,
audience: str,
lifetime_s: int,
) -> str:
"""Create auth token via PyNaCl (fallback, 64-char seed only).
The orlp/ed25519 expanded format (128-char) cannot be used with
PyNaCl because the expanded key is not a seed — it is the result
of ``SHA-512(seed)`` with clamping applied. PyNaCl's ``SigningKey``
expects the original 32-byte seed.
"""
if len(private_key_hex) == 128:
raise ValueError(
"PyNaCl fallback requires a 64-char Ed25519 seed. "
"The 128-char orlp/ed25519 expanded key is only supported "
"via Node.js meshcore-decoder. "
"Install: npm install -g @michaelhart/meshcore-decoder"
)
try:
from nacl.signing import SigningKey
except ImportError:
raise ImportError(
"Neither Node.js meshcore-decoder nor PyNaCl are available. "
"Install one: npm install -g @michaelhart/meshcore-decoder "
"OR pip install PyNaCl"
)
signing_key = SigningKey(bytes.fromhex(private_key_hex))
header = {"alg": "Ed25519", "typ": "JWT"}
now = int(time.time())
payload = {
"publicKey": public_key_hex.upper(),
"aud": audience,
"iat": now,
"exp": now + lifetime_s,
}
header_b64 = _base64url_encode(
json.dumps(header, separators=(",", ":")).encode("utf-8")
)
payload_b64 = _base64url_encode(
json.dumps(payload, separators=(",", ":")).encode("utf-8")
)
message = f"{header_b64}.{payload_b64}".encode("utf-8")
signed = signing_key.sign(message)
signature_b64 = _base64url_encode(signed.signature)
return f"{header_b64}.{payload_b64}.{signature_b64}"
# ── Public API ───────────────────────────────────────────────────────
def create_auth_token(
public_key_hex: str,
private_key_hex: str,
audience: str,
lifetime_s: int = DEFAULT_TOKEN_LIFETIME_S,
) -> str:
"""Create a LetsMesh-compatible Ed25519 JWT authentication token.
Tries Node.js meshcore-decoder first (reference implementation),
falls back to PyNaCl if unavailable (seed-only).
Args:
public_key_hex: 64-char hex device public key (from appstart).
private_key_hex: Ed25519 private key — either 128-char hex
(orlp expanded, preferred) or 64-char hex (seed).
audience: Broker hostname (e.g. ``mqtt-eu-v1.letsmesh.net``).
lifetime_s: Token validity in seconds (default 3600).
Returns:
JWT-style token string: ``header.payload.signature``
Raises:
ValueError: If key format is invalid.
"""
if not _is_valid_hex(public_key_hex, (64,)):
raise ValueError(
f"Public key must be 64 hex chars, got {len(public_key_hex)}"
)
if not _is_valid_hex(private_key_hex, VALID_PRIVATE_KEY_LENGTHS):
raise ValueError(
f"Private key must be 64 or 128 hex chars, "
f"got {len(private_key_hex)}"
)
# Strategy 1: Node.js meshcore-decoder (reference implementation)
if _check_node_available():
try:
token = _create_token_nodejs(
public_key_hex, private_key_hex, audience, lifetime_s,
)
logger.debug("Token generated via Node.js meshcore-decoder")
return token
except Exception as exc:
logger.warning(
"Node.js token generation failed, falling back to PyNaCl: %s",
exc,
)
# Strategy 2: PyNaCl fallback (seed-only)
token = _create_token_pynacl(
public_key_hex, private_key_hex, audience, lifetime_s,
)
logger.debug("Token generated via PyNaCl (fallback)")
return token
class TokenManager:
"""Manages JWT token lifecycle with automatic refresh.
Args:
public_key_hex: 64-char hex device public key.
private_key_hex: 64- or 128-char hex Ed25519 private key.
lifetime_s: Token validity in seconds.
"""
def __init__(
self,
public_key_hex: str,
private_key_hex: str,
lifetime_s: int = DEFAULT_TOKEN_LIFETIME_S,
) -> None:
self._public_key = public_key_hex
self._private_key = private_key_hex
self._lifetime_s = lifetime_s
self._current_token: Optional[str] = None
self._token_expiry: float = 0.0
@property
def username(self) -> str:
"""MQTT username: ``v1_{PUBLIC_KEY}``."""
return f"v1_{self._public_key.upper()}"
def get_token(self, audience: str) -> str:
"""Get a valid token, refreshing if necessary."""
now = time.time()
if (
self._current_token is None
or now >= self._token_expiry - TOKEN_REFRESH_MARGIN_S
):
self._current_token = create_auth_token(
self._public_key,
self._private_key,
audience,
self._lifetime_s,
)
self._token_expiry = now + self._lifetime_s
logger.debug(
"Generated new auth token for %s (expires in %ds)",
audience,
self._lifetime_s,
)
return self._current_token
def invalidate(self) -> None:
"""Force token regeneration on next ``get_token()`` call."""
self._current_token = None
self._token_expiry = 0.0

372
meshcore_observer/config.py Normal file
View File

@@ -0,0 +1,372 @@
"""
Observer-specific configuration.
Loads settings from a YAML configuration file and provides typed
access to all observer parameters. Falls back to sensible defaults
when keys are missing.
Dependencies:
pyyaml (6.x)
"""
import json
import logging
import os
from dataclasses import dataclass, field
from pathlib import Path
from typing import List, Optional
import yaml
logger = logging.getLogger(__name__)
# Default config file location (next to meshcore_observer.py)
DEFAULT_CONFIG_PATH: Path = Path(__file__).parent.parent / "observer_config.yaml"
# Device identity file written by meshcore_gui (auto-detected)
DEFAULT_DEVICE_IDENTITY_PATH: Path = Path.home() / ".meshcore-gui" / "device_identity.json"
# Valid key lengths (hex chars)
VALID_PUBLIC_KEY_LENGTH = 64 # 32-byte Ed25519 public key
VALID_PRIVATE_KEY_LENGTHS = (64, 128) # 32-byte seed or 64-byte expanded
# ── Key validation helper ────────────────────────────────────────────
def _is_valid_key(hex_str: str, allowed_lengths: tuple) -> bool:
"""Check if *hex_str* is valid hex with one of the allowed lengths."""
if not hex_str or len(hex_str) not in allowed_lengths:
return False
try:
bytes.fromhex(hex_str)
return True
except ValueError:
return False
# ── MQTT Broker Configuration ────────────────────────────────────────
@dataclass
class MqttBrokerConfig:
"""Configuration for a single MQTT broker endpoint."""
name: str = "letsmesh-eu"
server: str = "mqtt-eu-v1.letsmesh.net"
port: int = 443
transport: str = "websockets"
tls: bool = True
enabled: bool = True
@classmethod
def from_dict(cls, data: dict) -> "MqttBrokerConfig":
return cls(
name=str(data.get("name", "letsmesh-eu")),
server=str(data.get("server", "mqtt-eu-v1.letsmesh.net")),
port=int(data.get("port", 443)),
transport=str(data.get("transport", "websockets")),
tls=bool(data.get("tls", True)),
enabled=bool(data.get("enabled", True)),
)
@dataclass
class MqttConfig:
"""MQTT uplink configuration.
Attributes:
enabled: Master MQTT enable switch (default OFF).
iata: 3-letter IATA airport code for topic path.
brokers: List of broker endpoints.
device_identity_file: Path to meshcore_gui device identity JSON.
private_key: Ed25519 private key (hex) — from config.
private_key_file: Path to file containing private key.
public_key: Device public key (hex) — for topics/auth.
device_name: Device display name for ``origin`` field.
upload_packet_types: Packet type filter (empty = all).
status_interval_s: Seconds between status republish.
reconnect_delay_s: Seconds between reconnect attempts.
max_reconnect_retries: Max reconnect retries (0 = infinite).
token_lifetime_s: JWT token validity in seconds.
dry_run: Log payloads but do not publish.
"""
enabled: bool = False
iata: str = "AMS"
brokers: List[MqttBrokerConfig] = field(default_factory=list)
device_identity_file: str = ""
private_key: str = ""
private_key_file: str = ""
public_key: str = ""
device_name: str = ""
upload_packet_types: List[int] = field(default_factory=list)
status_interval_s: int = 300
reconnect_delay_s: int = 10
max_reconnect_retries: int = 0
token_lifetime_s: int = 3600
dry_run: bool = False
# Cached identity data (not serialised)
_identity_cache: Optional[dict] = field(
default=None, repr=False, compare=False,
)
@classmethod
def from_dict(cls, data: dict) -> "MqttConfig":
brokers_raw = data.get("brokers", [])
brokers = [MqttBrokerConfig.from_dict(b) for b in brokers_raw]
return cls(
enabled=bool(data.get("enabled", False)),
iata=str(data.get("iata", "AMS")),
brokers=brokers,
device_identity_file=str(data.get("device_identity_file", "")),
private_key=str(data.get("private_key", "")),
private_key_file=str(data.get("private_key_file", "")),
public_key=str(data.get("public_key", "")),
device_name=str(data.get("device_name", "")),
upload_packet_types=list(data.get("upload_packet_types", [])),
status_interval_s=int(data.get("status_interval_s", 300)),
reconnect_delay_s=int(data.get("reconnect_delay_s", 10)),
max_reconnect_retries=int(data.get("max_reconnect_retries", 0)),
token_lifetime_s=int(data.get("token_lifetime_s", 3600)),
dry_run=bool(data.get("dry_run", False)),
)
# ── Device identity loading ──────────────────────────────────────
def _load_device_identity(self) -> Optional[dict]:
"""Load device identity JSON written by meshcore_gui.
Checks (in order):
1. ``device_identity_file`` from config (explicit path)
2. Default path ``~/.meshcore-gui/device_identity.json``
Accepts private keys in both 64-char (legacy seed) and 128-char
(orlp expanded) formats.
Returns:
Dict with ``public_key`` and ``private_key``, or None.
"""
if self._identity_cache is not None:
return self._identity_cache
paths_to_try = []
if self.device_identity_file:
paths_to_try.append(Path(self.device_identity_file).expanduser())
paths_to_try.append(DEFAULT_DEVICE_IDENTITY_PATH)
for id_path in paths_to_try:
if not id_path.exists():
continue
try:
data = json.loads(id_path.read_text(encoding="utf-8"))
pub = data.get("public_key", "")
priv = data.get("private_key", "")
if (_is_valid_key(pub, (VALID_PUBLIC_KEY_LENGTH,))
and _is_valid_key(priv, VALID_PRIVATE_KEY_LENGTHS)):
logger.info(
"Loaded device identity from %s "
"(key=%s..., priv=%d chars)",
id_path, pub[:12], len(priv),
)
self._identity_cache = data
return data
logger.warning(
"Device identity file %s has invalid key lengths "
"(pub=%d, priv=%d)",
id_path, len(pub), len(priv),
)
except (json.JSONDecodeError, OSError) as exc:
logger.warning(
"Cannot read device identity file %s: %s",
id_path, exc,
)
self._identity_cache = {} # Mark as tried (empty = not found)
return None
# ── Key resolution (Single Responsibility) ───────────────────────
def resolve_private_key(self) -> str:
"""Resolve the private key.
Priority:
1. ``MESHCORE_PRIVATE_KEY`` environment variable
2. Device identity file (auto from meshcore_gui)
3. ``private_key_file`` config path
4. ``private_key`` config value
Returns:
Hex private key string (64 or 128 chars), or empty string.
"""
# Priority 1: environment variable
env_key = os.environ.get("MESHCORE_PRIVATE_KEY", "").strip()
if env_key:
logger.debug("Using private key from MESHCORE_PRIVATE_KEY env var")
return env_key
# Priority 2: device identity file (written by meshcore_gui)
identity = self._load_device_identity()
if identity and identity.get("private_key"):
logger.debug("Using private key from device identity file")
return identity["private_key"]
# Priority 3: key file
if self.private_key_file:
key_path = Path(self.private_key_file).expanduser()
if key_path.exists():
try:
key_data = key_path.read_text(encoding="utf-8").strip()
if key_data:
logger.debug("Using private key from file: %s", key_path)
return key_data
except OSError as exc:
logger.error("Cannot read private key file %s: %s", key_path, exc)
else:
logger.warning("Private key file not found: %s", key_path)
# Priority 4: inline config value
if self.private_key:
logger.warning(
"Using private key from plain-text config — "
"consider using private_key_file or MESHCORE_PRIVATE_KEY env var instead"
)
return self.private_key
return ""
def resolve_public_key(self) -> str:
"""Resolve the public key.
Priority:
1. ``MESHCORE_PUBLIC_KEY`` environment variable
2. Device identity file (auto from meshcore_gui)
3. ``public_key`` config value
Returns:
64-char hex public key string, or empty string.
"""
env_key = os.environ.get("MESHCORE_PUBLIC_KEY", "").strip()
if env_key:
return env_key
identity = self._load_device_identity()
if identity and identity.get("public_key"):
logger.debug("Using public key from device identity file")
return identity["public_key"]
if self.public_key:
return self.public_key
return ""
def resolve_device_name(self) -> str:
"""Resolve the device display name.
Returns:
Device name string, or ``"MeshCore Observer"`` as default.
"""
if self.device_name:
return self.device_name
identity = self._load_device_identity()
if identity and identity.get("device_name"):
return identity["device_name"]
return "MeshCore Observer"
# ── Validation ───────────────────────────────────────────────────
def validate(self) -> List[str]:
"""Validate MQTT configuration and return list of errors."""
errors: List[str] = []
if not self.enabled:
return errors
# IATA code
if not self.iata or len(self.iata) != 3 or not self.iata.isalpha():
errors.append(
f"IATA code must be exactly 3 letters, got: '{self.iata}'"
)
# Public key
pub = self.resolve_public_key()
if not pub:
errors.append("Public key is required for MQTT authentication")
elif not _is_valid_key(pub, (VALID_PUBLIC_KEY_LENGTH,)):
errors.append(
f"Public key must be 64 hex chars, got {len(pub)}"
)
# Private key — accepts both 64 (seed) and 128 (expanded)
priv = self.resolve_private_key()
if not priv:
errors.append(
"Private key is required for MQTT authentication "
"(set via config, file, or MESHCORE_PRIVATE_KEY env var)"
)
elif not _is_valid_key(priv, VALID_PRIVATE_KEY_LENGTHS):
errors.append(
f"Private key must be 64 or 128 hex chars, got {len(priv)}"
)
# At least one enabled broker
enabled_brokers = [b for b in self.brokers if b.enabled]
if not enabled_brokers:
errors.append("At least one broker must be enabled")
return errors
# ── Main Observer Configuration ──────────────────────────────────────
@dataclass
class ObserverConfig:
"""Complete observer daemon configuration."""
archive_dir: str = str(Path.home() / ".meshcore-gui" / "archive")
poll_interval_s: float = 2.0
max_messages_display: int = 100
max_rxlog_display: int = 50
gui_port: int = 9093
gui_title: str = "MeshCore Observer"
debug: bool = False
config_path: str = ""
mqtt: MqttConfig = field(default_factory=MqttConfig)
@classmethod
def from_yaml(cls, path: Path) -> "ObserverConfig":
"""Load configuration from a YAML file.
Missing keys fall back to dataclass defaults.
"""
with open(path, "r", encoding="utf-8") as fh:
raw = yaml.safe_load(fh) or {}
observer_section = raw.get("observer", {})
gui_section = raw.get("gui", {})
mqtt_section = raw.get("mqtt", {})
archive_raw = observer_section.get(
"archive_dir",
str(Path.home() / ".meshcore-gui" / "archive"),
)
archive_dir = str(Path(archive_raw).expanduser().resolve())
mqtt_cfg = MqttConfig.from_dict(mqtt_section) if mqtt_section else MqttConfig()
return cls(
archive_dir=archive_dir,
poll_interval_s=float(observer_section.get("poll_interval_s", 2.0)),
max_messages_display=int(observer_section.get("max_messages_display", 100)),
max_rxlog_display=int(observer_section.get("max_rxlog_display", 50)),
gui_port=int(gui_section.get("port", 9093)),
gui_title=str(gui_section.get("title", "MeshCore Observer")),
mqtt=mqtt_cfg,
)

1
meshcore_observer/gui/__init__.py Normal file
View File

@@ -0,0 +1 @@
"""Observer GUI package."""

233
meshcore_observer/gui/dashboard.py Normal file
View File

@@ -0,0 +1,233 @@
"""
Observer dashboard — NiceGUI page with DOMCA theme.
Thin orchestrator that owns the layout, injects the DOMCA theme,
and runs a periodic update timer that polls the ArchiveWatcher
and refreshes all panels. Visually consistent with the
meshcore_gui and meshcore_bridge dashboards.
"""
from nicegui import ui
from meshcore_observer import __version__
from meshcore_observer.archive_watcher import ArchiveWatcher
from meshcore_observer.config import ObserverConfig
from meshcore_observer.gui.panels.sources_panel import SourcesPanel
from meshcore_observer.gui.panels.messages_panel import MessagesPanel
from meshcore_observer.gui.panels.rxlog_panel import RxLogPanel
from meshcore_observer.gui.panels.stats_panel import StatsPanel
from meshcore_observer.gui.panels.mqtt_panel import MqttPanel
# ── DOMCA Theme (identical to meshcore_bridge dashboard) ─────────────
_DOMCA_HEAD = '''
<link href="https://fonts.googleapis.com/css2?family=Exo+2:wght@800&family=JetBrains+Mono:wght@400;700&display=swap" rel="stylesheet">
<style>
/* ── DOMCA theme variables (dark) ── */
body.body--dark {
--bg: #0A1628;
--title: #48CAE4; --subtitle: #48CAE4;
}
/* ── DOMCA theme variables (light) ── */
body.body--light {
--bg: #FFFFFF;
--title: #0077B6; --subtitle: #0077B6;
}
/* ── DOMCA page background ── */
body.body--dark { background: #0A1628 !important; }
body.body--light { background: #f4f8fb !important; }
body.body--dark .q-page { background: #0A1628 !important; }
body.body--light .q-page { background: #f4f8fb !important; }
/* ── DOMCA header ── */
body.body--dark .q-header { background: #0d1f35 !important; }
body.body--light .q-header { background: #0077B6 !important; }
/* ── DOMCA cards — dark mode readable ── */
body.body--dark .q-card {
background: #112240 !important;
color: #e0f0f8 !important;
border: 1px solid rgba(0,119,182,0.15) !important;
}
body.body--dark .q-card .text-gray-600 { color: #48CAE4 !important; }
body.body--dark .q-card .text-xs { color: #c0dce8 !important; }
body.body--dark .q-card .text-sm { color: #d0e8f2 !important; }
/* ── Dark mode: fields ── */
body.body--dark .q-field__control { background: #0c1a2e !important; color: #e0f0f8 !important; }
body.body--dark .q-field__native { color: #e0f0f8 !important; }
/* ── Observer-specific ── */
.observer-header-text {
font-family: 'JetBrains Mono', monospace;
color: white;
}
</style>
'''
class ObserverDashboard:
"""Observer dashboard page.
Provides a NiceGUI-based monitoring view showing aggregated
messages, RX log entries, archive sources, statistics, and
MQTT uplink status from all detected archive files.
Args:
watcher: ArchiveWatcher for polling archive files.
config: ObserverConfig for display settings.
mqtt_uplink: MqttUplink instance (or None if MQTT disabled).
"""
def __init__(
self,
watcher: ArchiveWatcher,
config: ObserverConfig,
mqtt_uplink=None,
) -> None:
self._watcher = watcher
self._cfg = config
self._mqtt_uplink = mqtt_uplink
# Panels (created in render)
self._sources: SourcesPanel | None = None
self._messages: MessagesPanel | None = None
self._rxlog: RxLogPanel | None = None
self._stats: StatsPanel | None = None
self._mqtt: MqttPanel | None = None
# Header status label
self._header_status = None
def render(self) -> None:
"""Build the complete observer dashboard layout and start the timer."""
# Create panel instances
self._sources = SourcesPanel(self._watcher)
self._messages = MessagesPanel(self._cfg.max_messages_display)
self._rxlog = RxLogPanel(self._cfg.max_rxlog_display)
self._stats = StatsPanel(self._watcher)
self._mqtt = MqttPanel(self._mqtt_uplink)
# Inject DOMCA theme
ui.add_head_html(_DOMCA_HEAD)
# Default to dark mode
dark = ui.dark_mode(True)
# ── Header ────────────────────────────────────────────────
with ui.header().classes("items-center px-4 py-2 shadow-md"):
ui.icon("visibility").classes("text-white text-2xl")
ui.label(
f"MeshCore Observer v{__version__}"
).classes("text-lg font-bold ml-2 observer-header-text")
ui.space()
self._header_status = ui.label("Scanning...").classes(
"text-sm opacity-70 observer-header-text"
)
ui.button(
icon="brightness_6",
on_click=lambda: dark.toggle(),
).props("flat round dense color=white").tooltip("Toggle dark / light")
# ── Main Content ──────────────────────────────────────────
with ui.column().classes("w-full max-w-6xl mx-auto p-4 gap-4"):
# Config summary
with ui.card().classes("w-full"):
with ui.row().classes("items-center gap-2 mb-2"):
ui.icon("settings", color="primary").classes("text-lg")
ui.label("Configuration").classes(
"text-sm font-bold"
).style("font-family: 'JetBrains Mono', monospace")
with ui.row().classes("gap-4 flex-wrap"):
config_items = [
("Archive dir", self._cfg.archive_dir),
("Poll interval", f"{self._cfg.poll_interval_s}s"),
("Max messages", str(self._cfg.max_messages_display)),
("Max RX log", str(self._cfg.max_rxlog_display)),
]
# MQTT status in config summary
if self._cfg.mqtt.enabled:
mode = "DRY RUN" if self._cfg.mqtt.dry_run else "LIVE"
config_items.append(("MQTT", f"{mode} ({self._cfg.mqtt.iata})"))
else:
config_items.append(("MQTT", "Disabled"))
for lbl, val in config_items:
with ui.column().classes("gap-0"):
ui.label(lbl).classes("text-xs opacity-50")
ui.label(val).classes("text-xs font-bold").style(
"font-family: 'JetBrains Mono', monospace"
)
# Top row: Sources + Stats + MQTT
with ui.row().classes("w-full gap-4 flex-wrap"):
with ui.column().classes("flex-1 min-w-[300px]"):
self._sources.render()
with ui.column().classes("flex-1 min-w-[280px]"):
self._stats.render()
# MQTT panel (full width, only if enabled or for status)
self._mqtt.render()
# Messages panel (full width)
self._messages.render()
# RX log panel (full width)
self._rxlog.render()
# ── Update timer ──────────────────────────────────────────
ui.timer(self._cfg.poll_interval_s, self._on_timer)
def _on_timer(self) -> None:
"""Periodic UI update callback — poll watcher, publish MQTT, refresh panels."""
result = self._watcher.poll()
# Feed new data to panels
if result.new_messages and self._messages:
self._messages.add_messages(result.new_messages)
if result.new_rxlog and self._rxlog:
self._rxlog.add_entries(result.new_rxlog)
# Publish new RX log entries to MQTT (if enabled)
if result.new_rxlog and self._mqtt_uplink:
try:
self._mqtt_uplink.publish_entries(result.new_rxlog)
except Exception as exc:
import logging
logging.getLogger(__name__).error(
"MQTT publish error: %s", exc,
)
# Update header status
stats = self._watcher.get_stats()
n_sources = stats["active_sources"]
n_msg = stats["total_messages_seen"]
n_rx = stats["total_rxlog_seen"]
if n_sources > 0:
status = f"✅ {n_sources} source{'s' if n_sources != 1 else ''} — {n_msg} msg / {n_rx} rx"
else:
status = "⏳ Waiting for archive files..."
if self._header_status:
self._header_status.set_text(status)
# Refresh all panels
if self._sources:
self._sources.update()
if self._messages:
self._messages.update()
if self._rxlog:
self._rxlog.update()
if self._stats:
self._stats.update()
if self._mqtt:
self._mqtt.update()

1
meshcore_observer/gui/panels/__init__.py Normal file
View File

@@ -0,0 +1 @@
"""Observer dashboard panels."""

187
meshcore_observer/gui/panels/messages_panel.py Normal file
View File

@@ -0,0 +1,187 @@
"""
Messages panel — aggregated message feed from all archive sources.
Displays messages sorted by ``timestamp_utc`` (newest first),
with optional source and channel filtering.
"""
from typing import Dict, List, Optional, Tuple
from nicegui import ui
class MessagesPanel:
"""Aggregated message feed panel.
Maintains an in-memory buffer of messages from all sources,
sorted by UTC timestamp.
Args:
max_display: Maximum number of messages to display.
"""
def __init__(self, max_display: int = 100) -> None:
self._max_display = max_display
self._messages: List[dict] = []
self._table: Optional[ui.table] = None
# Filters
self._source_filter: str = ""
self._channel_filter: str = ""
self._source_select: Optional[ui.select] = None
self._channel_select: Optional[ui.select] = None
def render(self) -> None:
"""Build the messages panel UI."""
with ui.card().classes("w-full"):
with ui.row().classes("items-center gap-2 mb-2"):
ui.icon("chat", color="primary").classes("text-lg")
ui.label("Messages").classes(
"text-sm font-bold"
).style("font-family: 'JetBrains Mono', monospace")
ui.space()
# Source filter (REQ-10)
self._source_select = ui.select(
options={"": "All sources"},
value="",
on_change=lambda e: self._on_source_filter(e.value),
).props("dense outlined").classes("text-xs min-w-[140px]")
# Channel filter (REQ-11)
self._channel_select = ui.select(
options={"": "All channels"},
value="",
on_change=lambda e: self._on_channel_filter(e.value),
).props("dense outlined").classes("text-xs min-w-[140px]")
self._table = ui.table(
columns=[
{"name": "time", "label": "Time", "field": "time",
"align": "left", "sortable": True},
{"name": "source", "label": "Source", "field": "source",
"align": "left"},
{"name": "channel", "label": "Channel", "field": "channel",
"align": "left"},
{"name": "sender", "label": "Sender", "field": "sender",
"align": "left"},
{"name": "text", "label": "Text", "field": "text",
"align": "left",
"classes": "msg-text-cell",
"headerClasses": "msg-text-header"},
{"name": "snr", "label": "SNR", "field": "snr",
"align": "right"},
{"name": "hops", "label": "Hops", "field": "hops",
"align": "right"},
],
rows=[],
).props("dense flat").classes("w-full text-xs")
ui.add_css("""
.msg-text-cell {
max-width: 300px;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.msg-text-header { max-width: 300px; }
""")
# ------------------------------------------------------------------
# Data ingestion
# ------------------------------------------------------------------
def add_messages(self, new_messages: List[Tuple[str, dict]]) -> None:
"""Add new messages from a poll result.
Args:
new_messages: List of (source_address, message_dict) tuples.
"""
for _source, msg in new_messages:
self._messages.append(msg)
# Sort by timestamp (newest first) and trim
self._messages.sort(
key=lambda m: m.get("timestamp_utc", ""),
reverse=True,
)
self._messages = self._messages[:self._max_display * 2]
# ------------------------------------------------------------------
# Update UI
# ------------------------------------------------------------------
def update(self) -> None:
"""Refresh the messages table with current filter state."""
if not self._table:
return
# Update filter dropdowns with discovered values
self._update_filter_options()
# Apply filters
filtered = self._messages
if self._source_filter:
filtered = [m for m in filtered if m.get("_source") == self._source_filter]
if self._channel_filter:
filtered = [m for m in filtered if m.get("channel_name") == self._channel_filter]
display = filtered[:self._max_display]
rows = [
{
"time": m.get("time", m.get("timestamp_utc", "")[:19]),
"source": self._short_source(m.get("_source", "")),
"channel": m.get("channel_name", m.get("channel", "-")),
"sender": m.get("sender", ""),
"text": m.get("text", ""),
"snr": f"{m['snr']:.1f}" if m.get("snr") is not None else "-",
"hops": str(m.get("path_len", 0)),
}
for m in display
]
self._table.rows = rows
self._table.update()
# ------------------------------------------------------------------
# Helpers
# ------------------------------------------------------------------
def _on_source_filter(self, value: str) -> None:
"""Handle source filter change."""
self._source_filter = value
self.update()
def _on_channel_filter(self, value: str) -> None:
"""Handle channel filter change."""
self._channel_filter = value
self.update()
def _update_filter_options(self) -> None:
"""Update dropdown options from current message data."""
if self._source_select:
sources = sorted({m.get("_source", "") for m in self._messages if m.get("_source")})
options = {"": "All sources"}
options.update({s: self._short_source(s) for s in sources})
self._source_select.options = options
self._source_select.update()
if self._channel_select:
channels = sorted({m.get("channel_name", "") for m in self._messages if m.get("channel_name")})
options = {"": "All channels"}
options.update({c: c for c in channels})
self._channel_select.options = options
self._channel_select.update()
@staticmethod
def _short_source(address: str) -> str:
"""Shorten a source address for display."""
if not address:
return "-"
# Remove common prefixes for readability
short = address
for prefix in ("bridge_a_", "bridge_b_", "_dev_"):
if short.startswith(prefix):
short = short[len(prefix):]
return short[:20] if len(short) > 20 else short

165
meshcore_observer/gui/panels/mqtt_panel.py Normal file
View File

@@ -0,0 +1,165 @@
"""
MQTT status panel — broker connection status and publish statistics.
Displays MQTT uplink health: per-broker connection state, packet
counters, filter configuration, and any errors. Updates in real-time
via the dashboard timer.
"""
from typing import Dict, Optional
from nicegui import ui
class MqttPanel:
"""MQTT uplink status panel.
Args:
mqtt_uplink: MqttUplink instance (or None if MQTT disabled).
"""
def __init__(self, mqtt_uplink=None) -> None:
self._uplink = mqtt_uplink
# UI element references
self._status_icon: Optional[ui.icon] = None
self._status_label: Optional[ui.label] = None
self._topic_label: Optional[ui.label] = None
self._filter_label: Optional[ui.label] = None
self._published_label: Optional[ui.label] = None
self._filtered_label: Optional[ui.label] = None
self._skipped_label: Optional[ui.label] = None
self._brokers_container: Optional[ui.column] = None
def render(self) -> None:
"""Build the MQTT status panel UI."""
with ui.card().classes("w-full"):
with ui.row().classes("items-center gap-2 mb-2"):
ui.icon("cell_tower", color="primary").classes("text-lg")
ui.label("MQTT Uplink").classes(
"text-sm font-bold"
).style("font-family: 'JetBrains Mono', monospace")
ui.space()
self._status_icon = ui.icon("circle").classes("text-sm")
self._status_label = ui.label("").classes("text-xs")
if self._uplink is None:
ui.label("MQTT is disabled in configuration.").classes(
"text-xs opacity-40 py-1"
)
return
with ui.column().classes("gap-1 w-full"):
# Topic info
with ui.row().classes("items-center gap-2"):
ui.label("Topic:").classes("text-xs opacity-60 w-24")
self._topic_label = ui.label("-").classes(
"text-xs font-bold"
).style("font-family: 'JetBrains Mono', monospace")
# Filter info
with ui.row().classes("items-center gap-2"):
ui.label("Filter:").classes("text-xs opacity-60 w-24")
self._filter_label = ui.label("-").classes("text-xs")
# Counters
with ui.row().classes("items-center gap-2"):
ui.label("Published:").classes("text-xs opacity-60 w-24")
self._published_label = ui.label("0").classes(
"text-xs font-bold"
)
with ui.row().classes("items-center gap-2"):
ui.label("Filtered:").classes("text-xs opacity-60 w-24")
self._filtered_label = ui.label("0").classes("text-xs")
with ui.row().classes("items-center gap-2"):
ui.label("Skipped:").classes("text-xs opacity-60 w-24")
self._skipped_label = ui.label("0").classes("text-xs")
ui.separator().classes("my-2")
with ui.row().classes("items-center gap-2 mb-1"):
ui.icon("dns", color="primary").classes("text-sm")
ui.label("Brokers").classes("text-xs font-bold")
self._brokers_container = ui.column().classes("gap-1 w-full")
def update(self) -> None:
"""Refresh MQTT status from uplink instance."""
if self._uplink is None:
return
status = self._uplink.get_status()
# Global status indicator
if self._status_icon and self._status_label:
if status.get("dry_run"):
self._status_icon.props("color=yellow")
self._status_label.set_text("DRY RUN")
elif any(b["connected"] for b in status.get("brokers", [])):
self._status_icon.props("color=green")
self._status_label.set_text("Connected")
elif status.get("started"):
self._status_icon.props("color=orange")
self._status_label.set_text("Connecting...")
else:
self._status_icon.props("color=red")
self._status_label.set_text("Disconnected")
# Topic
if self._topic_label:
self._topic_label.set_text(status.get("topic_base", "-"))
# Filter
if self._filter_label:
filt = status.get("upload_filter", "ALL")
if isinstance(filt, list):
self._filter_label.set_text(", ".join(filt))
else:
self._filter_label.set_text(str(filt))
# Counters
if self._published_label:
self._published_label.set_text(str(status.get("total_published", 0)))
if self._filtered_label:
self._filtered_label.set_text(str(status.get("total_filtered", 0)))
if self._skipped_label:
self._skipped_label.set_text(
f"{status.get('total_skipped_no_raw', 0)} (no raw_payload)"
)
# Per-broker status
if self._brokers_container:
self._brokers_container.clear()
with self._brokers_container:
brokers = status.get("brokers", [])
if not brokers:
ui.label("No brokers configured.").classes(
"text-xs opacity-40 py-1"
)
else:
for b in brokers:
with ui.row().classes("items-center gap-2 py-0.5"):
# Connection dot
color = "green" if b["connected"] else "red"
ui.icon("circle", color=color).classes("text-xs")
ui.label(b["name"]).classes(
"text-xs opacity-70 w-28"
)
ui.label(f"{b['packets_published']} pkts").classes(
"text-xs w-20"
)
ui.label(
b.get("last_publish_time", "-")[-8:]
).classes("text-xs opacity-50 w-20")
# Show error if present
if b.get("last_error"):
with ui.row().classes("pl-6"):
ui.label(
f"⚠ {b['last_error']}"
).classes("text-xs text-red-400")

Some files were not shown because too many files have changed in this diff Show More