diff --git a/CHANGELOG.md b/CHANGELOG.md index 2c8d339..94004de 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,5 @@ + + # CHANGELOG - -All notable changes to MeshCore GUI are documented in this file. -Format follows [Keep a Changelog](https://keepachangelog.com/) and [Semantic Versioning](https://semver.org/). - ---- -## [1.13.0] - 2026-03-09 β€” Leaflet Map Runtime Stabilization - -### Added -- βœ… `meshcore_gui/static/leaflet_map_panel.js` β€” Dedicated browser-side Leaflet runtime responsible for map lifecycle, marker registry, clustering and theme handling independent from NiceGUI redraw cycles -- βœ… `meshcore_gui/static/leaflet_map_panel.css` β€” Styling for browser-side node markers, cluster icons and map container -- βœ… `meshcore_gui/services/map_snapshot_service.py` β€” Snapshot service that normalizes device/contact map data into a compact payload for the browser runtime -- βœ… Browser-side map state management for center, zoom and theme -- βœ… Theme persistence across reconnect events via browser storage fallback -- βœ… Browser-side contact clustering via `Leaflet.markercluster` -- βœ… Separate non-clustered device marker layer so the own device remains individually visible - -### Changed -- πŸ”„ `meshcore_gui/gui/panels/map_panel.py` β€” Replaced NiceGUI Leaflet wrapper usage with a pure browser-managed Leaflet container while preserving the existing card layout, theme toggle and center-on-device control -- πŸ”„ Leaflet bootstrap moved out of inline Python into a dedicated browser runtime loaded from `/static` -- πŸ”„ Asset loading order is now explicit: Leaflet first, then `Leaflet.markercluster`, then the MeshCore panel runtime -- πŸ”„ Map initialization now occurs only once per container; NiceGUI refresh cycles no longer recreate the map -- πŸ”„ Dashboard update loop now sends compact map snapshots instead of triggering redraws -- πŸ”„ Snapshot processing in the browser is coalesced so only the newest payload is applied -- πŸ”„ Map markers are managed in separate device/contact layers and updated incrementally by stable node id -- πŸ”„ Contact markers are rendered inside a persistent cluster layer while the device marker remains outside clustering -- πŸ”„ Theme switching moved to a dedicated theme channel instead of being embedded in snapshot data - -### Fixed -- πŸ›  **Map disappearing during dashboard refresh cycles** β€” prevented repeated map reinitialization caused by the 500 ms NiceGUI update loop -- πŸ›  **Markers disappearing between refreshes** β€” marker updates are now incremental and keyed by node id -- πŸ›  **Blank map container on load** β€” browser bootstrap now waits for DOM host, Leaflet runtime and panel runtime before initialization -- πŸ›  **Leaflet clustering bootstrap failure (`L is not defined`)** β€” resolved by enforcing correct script dependency order before the panel runtime starts -- πŸ›  **MarkerClusterGroup failure (`Map has no maxZoom specified`)** β€” the map now defines `maxZoom` during initial creation before the cluster layer is attached -- πŸ›  **Half-initialized map retry cascade (`Map container is already initialized`)** β€” map state is now registered safely during initialization so a failed attempt cannot trigger a second `L.map(...)` on the same container -- πŸ›  **Race condition between queued snapshot and theme selection** β€” explicit theme changes can no longer be overwritten by stale snapshot payloads -- πŸ›  **Viewport jumping back to default center/zoom** β€” stored viewport is no longer reapplied on each snapshot update -- πŸ›  **Theme reverting to default during reconnect** β€” effective map theme is restored before snapshot processing resumes - -### Impact -- Leaflet map is now managed entirely in the browser and is no longer recreated on each dashboard refresh -- Node markers remain stable and no longer flicker or disappear during the 500 ms update cycle -- Dense contact sets can now be rendered with clustering without violating the browser-owned map lifecycle -- Theme switching and viewport state persist reliably across reconnect events -- No breaking changes outside the map subsystem ---- -## [1.12.1] - 2026-03-08 β€” Minor change bot -### Changed -- πŸ”„ `meshcore_gui/services/bot.py`: remove path id's -### Impact -- No breaking changes β€” all existing functionality preserved serial. - ---- - -## [1.12.0] - 2026-02-26 β€” MeshCore Observer Fase 1 - -### Added -- βœ… **MeshCore Observer daemon** β€” New standalone read-only daemon (`meshcore_observer.py`) that reads archive JSON files produced by meshcore_gui and meshcore_bridge, aggregates them, and presents a unified NiceGUI monitoring dashboard on port 9093. -- βœ… **ArchiveWatcher** β€” Core component that polls `~/.meshcore-gui/archive/` for `*_messages.json` and `*_rxlog.json` files, tracks mtime changes, and returns only new entries since previous poll. Thread-safe, zero writes, graceful on corrupt JSON. -- βœ… **Observer dashboard panels** β€” Sources overview, aggregated messages feed (sorted by timestamp), aggregated RX log table, and statistics panel with uptime/counters/per-source breakdown. Full DOMCA theme (dark + light mode). -- βœ… **Source filter** β€” Dropdown to filter messages and RX log by archive source. -- βœ… **Channel filter** β€” Dropdown to filter messages by channel name. -- βœ… **ObserverConfig** β€” YAML-based configuration with `from_yaml()` classmethod, defaults work without config file. -- βœ… **observer_config.yaml** β€” Documented config template with all options. -- βœ… **install_observer.sh** β€” systemd installer (`/opt/meshcore-observer/`, `/etc/meshcore/observer_config.yaml`), with `--uninstall` option. -- βœ… **RxLogEntry raw packet fields** β€” 5 new fields on `RxLogEntry` dataclass: `raw_payload`, `packet_len`, `payload_len`, `route_type`, `packet_type_num` (all with defaults, backward compatible). -- βœ… **EventHandler.on_rx_log() metadata** β€” Raw payload hex and packet metadata now passed through to RxLogEntry and archived (preparation for Fase 2 LetsMesh uplink). - -### Changed -- πŸ”„ `meshcore_gui/core/models.py`: RxLogEntry +5 fields with defaults (backward compatible). -- πŸ”„ `meshcore_gui/ble/events.py`: on_rx_log() fills raw_payload and metadata (~10 lines added). -- πŸ”„ `meshcore_gui/services/message_archive.py`: add_rx_log() serializes the 5 new RxLogEntry fields. -- πŸ”„ `meshcore_gui/config.py`: Version bumped to `1.12.0`. - -### Impact -- **No breaking changes** β€” All new RxLogEntry fields have defaults; existing archives and code work identically. -- **New daemon** β€” meshcore_observer is fully standalone; no imports from meshcore_gui (reads only JSON files). - ---- - -### Added -- βœ… **Serial CLI flags** β€” `--baud=BAUD` and `--serial-cx-dly=SECONDS` for serial configuration at startup. - -### Changed -- πŸ”„ **Connection layer** β€” Switched from BLE to serial (`MeshCore.create_serial`) with serial reconnect handling. -- πŸ”„ `config.py`: Added `SERIAL_BAUDRATE`, `SERIAL_CX_DELAY`, `DEFAULT_TIMEOUT`, `MESHCORE_LIB_DEBUG`; removed BLE PIN settings; version bumped to `1.10.0`. -- πŸ”„ `meshcore_gui.py` / `meshcore_gui/__main__.py`: Updated usage, banners and defaults for serial ports. -- πŸ”„ Docs: Updated README and core docs for serial usage; BLE documents marked as legacy. - -### Impact -- No breaking changes β€” all existing functionality preserved serial. - ---- - -## [1.9.11] - 2026-02-19 β€” Message Dedup Hotfix - -### Fixed -- πŸ›  **Duplicate messages after (re)connect** β€” `load_recent_from_archive()` appended archived messages on every connect attempt without clearing existing entries; after N failed connects, each message appeared N times. Method is now idempotent: clears the in-memory list before loading. -- πŸ›  **Persistent duplicate messages** β€” Live BLE events for messages already loaded from archive were not suppressed because the `DualDeduplicator` was never seeded with archived content. Added `_seed_dedup_from_messages()` in `BLEWorker` after cache/archive load and after reconnect. -- πŸ›  **Last-line-of-defence dedup in SharedData** β€” `add_message()` now maintains a fingerprint set (`message_hash` or `channel:sender:text`) and silently skips messages whose fingerprint is already tracked. This guards against duplicates regardless of their source. -- πŸ›  **Messages panel empty on first click** β€” `_show_panel()` made the container visible but relied on the next 500 ms timer tick to populate it. Added an immediate `_messages.update()` call so content is rendered the moment the panel becomes visible. - -### Changed -- πŸ”„ `core/shared_data.py`: Added `_message_fingerprints` set and `_message_fingerprint()` static method; `add_message()` checks fingerprint before insert and evicts fingerprints when messages are rotated out; `load_recent_from_archive()` clears messages and fingerprints before loading (idempotent) -- πŸ”„ `ble/worker.py`: Added `_seed_dedup_from_messages()` helper; called after `_apply_cache()` and after reconnect `_load_data()` to seed `DualDeduplicator` with existing messages -- πŸ”„ `gui/dashboard.py`: `_show_panel()` now forces an immediate `_messages.update()` when the messages panel is shown, eliminating the stale-content flash -- πŸ”„ `config.py`: Version bumped to `1.9.11` - -### Impact -- Eliminates all duplicate message display scenarios: initial connect, failed retries, reconnect, and BLE event replay -- No breaking changes β€” all existing functionality preserved -- Fingerprint set is bounded to the same 100-message cap as the message list - ---- - -## [1.9.10] - 2026-02-19 β€” Map Tooltips & Separate Own-Position Marker - -### Added -- βœ… **Map marker tooltips** β€” All markers on the Leaflet map now show a tooltip on hover with the node name and type icon (πŸ“±, πŸ“‘, 🏠) from `TYPE_ICONS` -- βœ… **Separate own-position marker** β€” The device's own position is now tracked as a dedicated `_own_marker`, independent from contact markers. This prevents the own marker from being removed/recreated on every contact update cycle - -### Changed -- πŸ”„ `gui/panels/map_panel.py`: Renamed `_markers` to `_contacts_markers`; added `_own_marker` attribute; own position marker is only updated when `device_updated` flag is set (not every timer tick); contact markers are only rebuilt when `contacts_updated` is set; added `TYPE_ICONS` import for tooltip icons -- πŸ”„ `gui/dashboard.py`: Added `self._map.update(data)` call in the `device_updated` block so the own-position marker updates when device info changes (e.g. GPS position update) -- πŸ”„ `config.py`: Version bumped to `1.9.10` - -### Impact -- Map centering on own device now works correctly and updates only when position actually changes -- Contact markers are no longer needlessly destroyed and recreated on every UI timer tick β€” only on actual contact data changes -- Tooltips make it easy to identify nodes on the map without clicking -- No breaking changes β€” all existing map functionality preserved - -### Credits -- Based on [PR #16](https://github.com/pe1hvh/meshcore-gui/pull/16) by [@rich257](https://github.com/rich257) - ---- - -## [1.9.9] - 2026-02-18 β€” Variable Landing Page & Operator Callsign - -### Added -- βœ… **Configurable operator callsign** β€” New `OPERATOR_CALLSIGN` constant in `config.py` (default: `"PE1HVH"`). Used in the landing page SVG and the drawer footer copyright label. Change this single value to personalize the entire GUI for a different operator -- βœ… **External landing page SVG** β€” The DOMCA splash screen is now loaded from a standalone file (`static/landing_default.svg`) instead of being hardcoded in `dashboard.py`. New `LANDING_SVG_PATH` constant in `config.py` points to the SVG file. The placeholder `{callsign}` in the SVG is replaced at runtime with `OPERATOR_CALLSIGN` -- βœ… **Landing page customization** β€” To use a custom landing page: copy `landing_default.svg` (or create your own SVG), use `{callsign}` wherever the operator callsign should appear, and point `LANDING_SVG_PATH` to your file. The default SVG includes an instructive comment block explaining the placeholder mechanism - -### Changed -- πŸ”„ `config.py`: Added `OPERATOR_CALLSIGN` and `LANDING_SVG_PATH` constants in new **OPERATOR / LANDING PAGE** section; version bumped to `1.9.9` -- πŸ”„ `gui/dashboard.py`: Removed hardcoded `_DOMCA_SVG` string (~70 lines); added `_load_landing_svg()` helper that reads SVG from disk and replaces `{callsign}` placeholder; CSS variable `--pe1hvh` renamed to `--callsign`; drawer footer copyright label now uses `config.OPERATOR_CALLSIGN` - -### Added (files) -- βœ… `static/landing_default.svg` β€” The original DOMCA splash SVG extracted as a standalone file, with `{callsign}` placeholder and `--callsign` CSS variable. Serves as both the default landing page and a reference template for custom SVGs - -### Impact -- Out-of-the-box behavior is identical to v1.9.8 (same DOMCA branding, same PE1HVH callsign) -- Operators personalize by changing 1–2 lines in `config.py` β€” no code modifications needed -- Fallback: if the SVG file is missing, a minimal placeholder text is shown instead of a crash -- No breaking changes β€” all existing dashboard functionality (panels, menus, timer, theming) unchanged - ---- - -## [1.9.8] - 2026-02-17 β€” Bugfix: Route Page Sender ID, Type & Location Not Populated - -### Fixed -- πŸ›  **Sender ID, Type and Location empty in Route Page** β€” After the v4.1 refactoring to `RouteBuilder`/`RouteNode`, the sender contact lookup relied solely on `SharedData.get_contact_by_prefix()` (live lock-based) and `get_contact_by_name()`. When both failed (e.g. empty `sender_pubkey` from RX_LOG decode, or name mismatch), `route['sender']` remained `None` and the route table fell through to a hardcoded fallback with `type: '-'`, `location: '-'`. The contact data was available in the snapshot `data['contacts']` but was never searched -- πŸ›  **Route table fallback row ignored available contact data** β€” When `route['sender']` was `None`, the `_render_route_table` method used a static fallback row without attempting to find the contact in the data snapshot. Even when the contact was present in `data['contacts']` with valid type and location, these fields showed as `'-'` - -### Changed -- πŸ”„ `services/route_builder.py`: Added two additional fallback strategies in `build()` after the existing SharedData lookups: (3) bidirectional pubkey prefix match against `data['contacts']` snapshot, (4) case-insensitive `adv_name` match against `data['contacts']` snapshot. Added helper methods `_find_contact_by_pubkey()` and `_find_contact_by_adv_name()` for snapshot-based lookups -- πŸ”„ `gui/route_page.py`: Added defensive fallback in `_render_route_table()` sender section β€” when `route['sender']` is `None`, attempts to find the contact in the snapshot via `_find_sender_contact()` before falling back to the static `'-'` row. Added `_find_sender_contact()` helper method - -### Impact -- Sender ID (hash), Type and Location are now populated correctly in the route table when the contact is known -- Four-layer lookup chain ensures maximum resolution: (1) SharedData pubkey lookup, (2) SharedData name lookup, (3) snapshot pubkey lookup, (4) snapshot name lookup -- Defensive fallback in route_page guarantees data is shown even if RouteBuilder misses it -- No breaking changes β€” all existing route page behavior, styling and data flows unchanged - ---- - -## [1.9.7] - 2026-02-17 β€” Layout Fix: Archive Filter Toggle & Route Page Styling - -### Changed -- πŸ”„ `gui/archive_page.py`: Archive filter card now hidden by default; toggle visibility via a `filter_list` icon button placed right-aligned on the same row as the "πŸ“š Archive" title. Header restructured from single label to `ui.row()` with `justify-between` layout -- πŸ”„ `gui/route_page.py`: Route page now uses DOMCA theme (imported from `dashboard.py`) with dark mode as default, consistent with the main dashboard. Header restyled from `bg-blue-600` to Quasar-themed header with JetBrains Mono font. Content container changed from `w-full max-w-4xl mx-auto` to `domca-panel` class for consistent responsive sizing -- πŸ”„ `gui/dashboard.py`: Added `domca-header-text` CSS class with `@media (max-width: 599px)` rule to hide header text on narrow viewports; applied to version label and status label -- πŸ”„ `gui/route_page.py`: Header label also uses `domca-header-text` class for consistent responsive behaviour - -### Added -- βœ… **Archive filter toggle** β€” `filter_list` icon button in archive header row toggles the filter card visibility on click -- βœ… **Route page close button** β€” `X` (close) icon button added right-aligned in the route page header; calls `window.close()` to close the browser tab -- βœ… **Responsive header** β€” On viewports < 600px, header text labels are hidden; only icon buttons (menu, dark mode toggle, close) remain visible - -### Impact -- Archive page is cleaner by default β€” filters only shown when needed -- Route page visually consistent with the main dashboard (DOMCA theme, dark mode, responsive panel width) -- Headers degrade gracefully on mobile (< 600px): only icon buttons visible, no text overflow -- No functional changes β€” all event handlers, callbacks, data bindings, logic and imports are identical to the input - ---- - -## [1.9.6] - 2026-02-17 β€” Bugfix: Channel Discovery Reliability - -### Fixed -- πŸ›  **Channels not appearing (especially on mobile)** β€” Channel discovery aborted too early on slow BLE connections. The `_discover_channels()` probe used a single attempt per channel slot and stopped after just 2 consecutive empty responses. On mobile BLE stacks (WebBluetooth via NiceGUI) where GATT responses are slower, this caused discovery to abort before finding any channels, falling back to only `[0] Public` -- πŸ›  **Race condition: channel update flag lost between threads** β€” `get_snapshot()` and `clear_update_flags()` were two separate calls, each acquiring the lock independently. If the BLE worker set `channels_updated = True` between these two calls, the GUI consumed the flag via `get_snapshot()` but then `clear_update_flags()` reset it β€” causing the channel submenu and dropdown to never populate -- πŸ›  **Channels disappear on browser reconnect** β€” When a browser tab is closed and reopened, `render()` creates new (empty) NiceGUI containers for the drawer submenus, but did not reset `_last_channel_fingerprint`. The `_update_submenus()` method compared the new fingerprint against the stale one, found them equal, and skipped the rebuild β€” leaving the new containers permanently empty. Fixed by resetting both `_last_channel_fingerprint` and `_last_rooms_fingerprint` in `render()` - -### Changed -- πŸ”„ `core/shared_data.py`: New atomic method `get_snapshot_and_clear_flags()` that reads the snapshot and resets all update flags in a single lock acquisition. Internally refactored to `_build_snapshot_unlocked()` helper. Existing `get_snapshot()` and `clear_update_flags()` retained for backward compatibility -- πŸ”„ `ble/worker.py`: `_discover_channels()` β€” `max_attempts` increased from 1 to 2 per channel slot; inter-attempt `delay` increased from 0.5s to 1.0s; consecutive error threshold raised from 2 to 3; inter-channel pause increased from 0.15s to 0.3s for mobile BLE stack breathing room -- πŸ”„ `gui/dashboard.py`: `_update_ui()` now uses `get_snapshot_and_clear_flags()` instead of separate `get_snapshot()` + `clear_update_flags()`; `render()` now resets `_last_channel_fingerprint` and `_last_rooms_fingerprint` to `None` so that `_update_submenus()` rebuilds into the freshly created containers; channel-dependent updates (`update_filters`, `update_channel_options`, `_update_submenus`) now run unconditionally when channel data exists β€” safe because each method has internal idempotency checks -- πŸ”„ `gui/panels/messages_panel.py`: `update_channel_options()` now includes an equality check on options dict to skip redundant `.update()` calls to the NiceGUI client on every 500ms timer tick - -### Impact -- Channel discovery now survives transient BLE timeouts that are common on mobile connections -- Atomic snapshot eliminates the threading race condition that caused channels to silently never appear -- Browser close+reopen no longer loses channels β€” the single-instance timer race on the shared `DashboardPage` is fully mitigated -- No breaking changes β€” all existing API methods retained, all other functionality unchanged - ---- - -## [1.9.5] - 2026-02-16 β€” Layout Fix: RX Log Table Responsive Sizing - -### Fixed -- πŸ›  **RX Log table did not adapt to panel/card size** β€” The table used `max-h-48` (a maximum height cap) instead of a responsive fixed height, causing it to remain small regardless of available space. Changed to `h-40` which is overridden by the existing dashboard CSS to `calc(100vh - 20rem)` β€” the same responsive pattern used by the Messages panel -- πŸ›  **RX Log table did not fill card width** β€” Added `w-full` class to the table element so it stretches to the full width of the parent card -- πŸ›  **RX Log card did not fill panel height** β€” Added `flex-grow` class to the card container so it expands to fill the available panel space - -### Changed -- πŸ”„ `gui/panels/rxlog_panel.py`: Card classes `'w-full'` β†’ `'w-full flex-grow'` (line 45); table classes `'text-xs max-h-48 overflow-y-auto'` β†’ `'w-full text-xs h-40 overflow-y-auto'` (line 65) - -### Impact -- RX Log table now fills the panel consistently on both desktop and mobile viewports -- Layout is consistent with other panels (Messages, Contacts) that use the same `h-40` responsive height pattern -- No functional changes β€” all event handlers, callbacks, data bindings, logica and imports are identical to the input - ---- - -## [1.9.4] - 2026-02-16 β€” BLE Address Log Prefix & Entry Point Cleanup - -### Added -- βœ… **BLE address prefix in log filename** β€” Log file is now named `_meshcore_gui.log` (e.g. `AA_BB_CC_DD_EE_FF_meshcore_gui.log`) instead of the generic `meshcore_gui.log`. Makes it easy to identify which device produced which log file when running multiple instances - - New helper `_sanitize_ble_address()` strips `literal:` prefix and replaces colons with underscores - - New function `configure_log_file(ble_address)` updates `LOG_FILE` at runtime before the logger is initialised - - Rotated backups follow the same naming pattern automatically - -### Removed -- ❌ **`meshcore_gui/meshcore_gui.py`** β€” Redundant copy of `main()` that was never imported. All three entry points (`meshcore_gui.py` root, `__main__.py`, and `meshcore_gui/meshcore_gui.py`) contained near-identical copies of the same logic, causing changes to be missed (as demonstrated by this fix). `__main__.py` is now the single source of truth; root `meshcore_gui.py` is a thin wrapper that imports from it - -### Changed -- πŸ”„ `config.py`: Added `_sanitize_ble_address()` and `configure_log_file()`; version bumped to `1.9.4` -- πŸ”„ `__main__.py`: Added `config.configure_log_file(ble_address)` call before any debug output -- πŸ”„ `meshcore_gui.py` (root): Reduced to 4-line wrapper importing `main` from `__main__` - -### Impact -- Log files are now identifiable per BLE device -- Single source of truth for `main()` eliminates future sync issues between entry points -- Both startup methods (`python meshcore_gui.py` and `python -m meshcore_gui`) remain functional -- No breaking changes β€” defaults and all existing behaviour unchanged ---- - -## [1.9.3] - 2026-02-16 β€” Bugfix: Map Default Location & Payload Type Decoding - -### Fixed -- πŸ›  **Map centred on hardcoded Zwolle instead of device location** β€” All Leaflet maps used magic-number coordinates `(52.5, 6.0)` as initial centre and fallback. These are now replaced by a single configurable constant `DEFAULT_MAP_CENTER` in `config.py`. Once the device reports a valid `adv_lat`/`adv_lon`, maps re-centre on the actual device position (existing behaviour, unchanged) -- πŸ›  **Payload type shown as raw integer** β€” Payload type is now retrieved from the decoded payload and translated to human-readable text using MeshCoreDecoder functions, instead of displaying the raw numeric type value - -### Changed -- πŸ”„ `config.py`: Added `DEFAULT_MAP_CENTER` (default: `(52.5168, 6.0830)`) and `DEFAULT_MAP_ZOOM` (default: `9`) constants in new **MAP DEFAULTS** section. Version bumped to `1.9.2` -- πŸ”„ `gui/panels/map_panel.py`: Imports `DEFAULT_MAP_CENTER` and `DEFAULT_MAP_ZOOM` from config; `ui.leaflet(center=...)` uses config constants instead of hardcoded values -- πŸ”„ `gui/route_page.py`: Imports `DEFAULT_MAP_CENTER` and `DEFAULT_MAP_ZOOM` from config; fallback coordinates (`or 52.5` / `or 6.0`) replaced by `DEFAULT_MAP_CENTER[0]` / `[1]`; zoom uses `DEFAULT_MAP_ZOOM` - -### Impact -- Map default location is now a single-point-of-change in `config.py` -- Payload type is displayed as readable text instead of a raw number -- No breaking changes β€” all existing map behaviour (re-centre on device position, contact markers) unchanged - -## [1.9.2] - 2026-02-15 β€” CLI Parameters & Cleanup - -### Added -- βœ… **`--port=PORT` CLI parameter** β€” Web server port is now configurable at startup (default: `8081`). Allows running multiple instances simultaneously on different ports -- βœ… **`--ble-pin=PIN` CLI parameter** β€” BLE pairing PIN is now configurable at startup (default: `123456`). Eliminates the need to edit `config.py` for devices with a non-default PIN, and works in systemd service files -- βœ… **Per-device log file** β€” Debug log file now includes the BLE address in its filename (e.g. `F0_9E_9E_75_A3_01_meshcore_gui.log`), so multiple instances log to separate files - -### Fixed -- πŸ›  **BLE PIN not applied from CLI** β€” `ble/worker.py` imported `BLE_PIN` as a constant at module load time (`from config import BLE_PIN`), capturing the default value `"123456"` before CLI parsing could override `config.BLE_PIN`. Changed to runtime access via `config.BLE_PIN` so the `--ble-pin` parameter is correctly passed to the BLE agent - -### Removed -- ❌ **Redundant `meshcore_gui/meshcore_gui.py`** β€” This file was a near-identical copy of both `meshcore_gui.py` (top-level) and `meshcore_gui/__main__.py`, but was never imported or referenced. Removed to eliminate maintenance risk. The two remaining entry points cover all startup methods: `python meshcore_gui.py` and `python -m meshcore_gui` - -### Impact -- Multiple instances can run side-by-side with different ports, PINs and log files -- Service deployments no longer require editing `config.py` β€” all runtime settings via CLI -- No breaking changes β€” all defaults are unchanged - ---- - -## [1.9.1] - 2026-02-14 β€” Bugfix: Dual Reconnect Conflict - -### Fixed -- πŸ›  **Library reconnect interfered with application reconnect** β€” The meshcore library's internal `auto_reconnect` (visible in logs as `"Attempting reconnection 1/3"`) ran a fast 3-attempt reconnect cycle without bond cleanup. This prevented the application's own `reconnect_loop` (which does `remove_bond()` + backoff) from succeeding, because BlueZ retained a stale bond β†’ `"failed to discover service"` - -### Changed -- πŸ”„ `ble/worker.py`: Set `auto_reconnect=False` in both `MeshCore.create_ble()` call sites (`_connect()` and `_create_fresh_connection()`), so only the application's bond-aware `reconnect_loop` handles reconnection -- πŸ”„ `ble/worker.py`: Added `"failed to discover"` and `"service discovery"` to disconnect detection keywords for defensive coverage - -### Impact -- Eliminates the ~9 second wasted library reconnect cycle after every BLE disconnect -- Application's `reconnect_loop` (with bond cleanup) now runs immediately after disconnect detection -- No breaking changes β€” the application reconnect logic was already fully functional - ---- - -## [1.9.0] - 2026-02-14 β€” BLE Connection Stability - -### Added -- βœ… **Built-in BLE PIN agent** β€” New `ble/ble_agent.py` registers a D-Bus agent with BlueZ to handle PIN pairing requests automatically. Eliminates the need for external `bt-agent.service` and `bluez-tools` package - - Uses `dbus_fast` (already a dependency of `bleak`, no new packages) - - Supports `RequestPinCode`, `RequestPasskey`, `DisplayPasskey`, `RequestConfirmation`, `AuthorizeService` callbacks - - Configurable PIN via `BLE_PIN` in `config.py` (default: `123456`) -- βœ… **Automatic bond cleanup** β€” New `ble/ble_reconnect.py` provides `remove_bond()` function that removes stale BLE bonds via D-Bus, equivalent to `bluetoothctl remove
`. Called automatically on startup and before each reconnect attempt -- βœ… **Automatic reconnect after disconnect** β€” BLEWorker main loop now detects BLE disconnects (via connection error exceptions) and automatically triggers a reconnect sequence: bond removal β†’ linear backoff wait β†’ fresh connection β†’ re-wire handlers β†’ reload device data - - Configurable via `RECONNECT_MAX_RETRIES` (default: 5) and `RECONNECT_BASE_DELAY` (default: 5.0s) - - After all retries exhausted: waits 60s then starts a new retry cycle (infinite recovery) -- βœ… **Generic install script** β€” `install_ble_stable.sh` auto-detects user, project directory, venv path and entry point to generate systemd service and D-Bus policy. Supports `--uninstall` flag - -### Changed -- πŸ”„ **`ble/worker.py`** β€” `_async_main()` rewritten with three phases: (1) start PIN agent, (2) remove stale bond, (3) connect + main loop with disconnect detection. Reconnect logic re-wires all event handlers and reloads device data after successful reconnection -- πŸ”„ **`config.py`** β€” Added `BLE_PIN`, `RECONNECT_MAX_RETRIES`, `RECONNECT_BASE_DELAY` constants - -### Removed -- ❌ **`bt-agent.service` dependency** β€” No longer needed; PIN pairing is handled by the built-in agent -- ❌ **`bluez-tools` system package** β€” No longer needed -- ❌ **`~/.meshcore-ble-pin` file** β€” No longer needed -- ❌ **Manual `bluetoothctl remove` before startup** β€” Handled automatically -- ❌ **`ExecStartPre` in systemd service** β€” Bond cleanup is internal - -### Impact -- Zero external dependencies for BLE pairing on Linux -- Automatic recovery from the T1000e ~2 hour BLE disconnect issue -- No manual intervention needed after BLE connection loss -- Single systemd service (`meshcore-gui.service`) manages everything -- No breaking changes to existing functionality - ---- - -## [1.8.0] - 2026-02-14 β€” DRY Message Construction & Archive Layout Unification - -### Fixed -- πŸ›  **Case-sensitive prefix matching** β€” `get_contact_name_by_prefix()` and `get_contact_by_prefix()` in `shared_data.py` failed to match path hashes (uppercase, e.g. `'B8'`) against contact pubkeys (lowercase, e.g. `'b8a3f2...'`). Added `.lower()` to both sides of the comparison, consistent with `_resolve_path_names()` which already had it -- πŸ›  **Route page 404 from archive** β€” Archive page linked to `/route/{hash}` but route was registered as `/route/{msg_index:int}`, causing a JSON parse error for hex hash strings. Route parameter changed to `str` with 3-strategy lookup (index β†’ memory hash β†’ archive fallback) -- πŸ›  **Three entry points out of sync** β€” `meshcore_gui.py` (root), `meshcore_gui/meshcore_gui.py` (inner) and `meshcore_gui/__main__.py` had diverging route registrations. All three now use identical `/route/{msg_key}` with `str` parameter - -### Changed -- πŸ”„ **`core/models.py` β€” DRY factory methods and formatting** - - `Message.now_timestamp()`: static method replacing 7Γ— hardcoded `datetime.now().strftime('%H:%M:%S')` across `events.py` and `commands.py` - - `Message.incoming()`: classmethod factory for received messages (`direction='in'`, auto-timestamp) - - `Message.outgoing()`: classmethod factory for sent messages (`sender='Me'`, `direction='out'`, auto-timestamp) - - `Message.format_line(channel_names)`: single-line display formatting (`"12:34:56 ← [Public] [2hβœ“] PE1ABC: Hello mesh!"`), replacing duplicate inline formatting in `messages_panel.py` and `archive_page.py` -- πŸ”„ **`ble/events.py`** β€” 4Γ— `Message(...)` constructors replaced by `Message.incoming()`; `datetime` import removed -- πŸ”„ **`ble/commands.py`** β€” 3Γ— `Message(...)` constructors replaced by `Message.outgoing()`; `datetime` import removed -- πŸ”„ **`gui/panels/messages_panel.py`** β€” 15 lines inline formatting replaced by single `msg.format_line(channel_names)` call -- πŸ”„ **`gui/archive_page.py` β€” Layout unified with main page** - - Multi-row card layout replaced by single-line `msg.format_line()` in monospace container (same style as main page) - - DM added to channel filter dropdown (post-filter on `channel is None`) - - Message click opens `/route/{message_hash}` in new tab (was: no click handler on archive messages) - - Removed `_render_message_card()` (98 lines) and `_render_archive_route()` (75 lines) - - Removed `RouteBuilder` dependency and `TYPE_LABELS` import - - File reduced from 445 to 267 lines -- πŸ”„ **`gui/route_page.py`** β€” `render(msg_index: int)` β†’ `render(msg_key: str)` with 3-strategy message lookup: (1) numeric index from in-memory list, (2) hash match in memory, (3) `archive.get_message_by_hash()` fallback -- πŸ”„ **`services/message_archive.py`** β€” New method `get_message_by_hash(hash)` for single-message lookup by packet hash -- πŸ”„ **`__main__.py` + `meshcore_gui.py` (both)** β€” Route changed from `/route/{msg_index}` (int) to `/route/{msg_key}` (str) - -### Impact -- DRY: timestamp formatting 7β†’1 definition, message construction 7β†’2 factories, line formatting 2β†’1 method -- Archive page visually consistent with main messages panel (single-line, monospace) -- Archive messages now clickable to open route visualization (was: only in-memory messages) -- Case-insensitive prefix matching fixes path name resolution for contacts with uppercase path hashes -- No breaking changes to BLE protocol handling, dedup, bot, or data storage - -### Known Limitations -- DM filter in archive uses post-filtering (query without channel filter + filter on `channel is None`); becomes exact when `query_messages()` gets native DM support - -### Parked for later -- Multi-path tracking (enrich RxLogEntry with multiple path observations) -- Events correlation improvements (only if proven data loss after `.lower()` fix) - ---- - -## [1.7.0] - 2026-02-13 β€” Archive Channel Name Persistence - -### Added -- βœ… **Channel name stored in archive** β€” Messages now persist `channel_name` alongside the numeric `channel` index in `
_messages.json`, so archived messages retain their human-readable channel name even when the device is not connected - - `Message` dataclass: new field `channel_name: str` (default `""`, backward compatible) - - `SharedData.add_message()`: automatically resolves `channel_name` from the live channels list when not already set (new helper `_resolve_channel_name()`) - - `MessageArchive.add_message()`: writes `channel_name` to the JSON dict -- βœ… **Archive channel selector built from archived data** β€” Channel filter dropdown on `/archive` now populated via `SELECT DISTINCT channel_name` on the archive instead of the live BLE channels list - - New method `MessageArchive.get_distinct_channel_names()` returns sorted unique channel names from stored messages - - Selector shows only channels that actually have archived messages -- βœ… **Archive filter on channel name** β€” `MessageArchive.query_messages()` parameter changed from `channel: Optional[int]` to `channel_name: Optional[str]` (exact match on name string) - -### Changed -- πŸ”„ `core/models.py`: Added `channel_name` field to `Message` dataclass and `from_dict()` -- πŸ”„ `core/shared_data.py`: `add_message()` resolves channel name; added `_resolve_channel_name()` helper -- πŸ”„ `services/message_archive.py`: `channel_name` persisted in JSON; `query_messages()` filters by name; new `get_distinct_channel_names()` method -- πŸ”„ `gui/archive_page.py`: Channel selector built from `archive.get_distinct_channel_names()`; filter state changed from `_channel_filter` (int) to `_channel_name_filter` (str); message cards show `channel_name` directly from archive - -### Fixed -- πŸ›  **Main page empty after startup** β€” After a restart the messages panel showed no messages until new live BLE traffic arrived. `SharedData.load_recent_from_archive()` now loads up to 100 recent archived messages during the cache-first startup phase, so historical messages are immediately visible - - New method `SharedData.load_recent_from_archive(limit)` β€” reads from `MessageArchive.query_messages()` and populates the in-memory list without re-archiving - - `BLEWorker._apply_cache()` calls `load_recent_from_archive()` at the end of cache loading - -### Impact -- Archived messages now self-contained β€” channel name visible without live BLE connection -- Main page immediately shows historical messages after startup (no waiting for live BLE traffic) -- Backward compatible β€” old archive entries without `channel_name` fall back to `"Ch "` -- No breaking changes to existing functionality - ---- - -## [1.6.0] - 2026-02-13 β€” Dashboard Layout Consolidation - -### Changed -- πŸ”„ **Messages panel consolidated** β€” Filter checkboxes (DM + channels) and message input (text field, channel selector, Send button) are now integrated into the Messages panel, replacing the separate Filter and Input panels - - DM + channel checkboxes displayed centered in the Messages header row, between the "πŸ’¬ Messages" label and the "πŸ“š Archive" button - - Message input row (text field, channel selector, Send button) placed below the message list within the same card - - `messages_panel.py`: Constructor now accepts `put_command` callable; added `update_filters(data)`, `update_channel_options(channels)` methods and `channel_filters`, `last_channels` properties (all logic 1:1 from FilterPanel/InputPanel); `update()` signature unchanged -- πŸ”„ **Actions panel expanded** β€” BOT toggle checkbox moved from Filter panel to Actions panel, below the Refresh/Advert buttons - - `actions_panel.py`: Constructor now accepts `set_bot_enabled` callable; added `update(data)` method for BOT state sync; `_on_bot_toggle()` logic 1:1 from FilterPanel -- πŸ”„ **Dashboard layout simplified** β€” Centre column reduced from 4 panels (Map β†’ Input β†’ Filter β†’ Messages) to 2 panels (Map β†’ Messages) - - `dashboard.py`: FilterPanel and InputPanel no longer rendered; all dependencies rerouted to MessagesPanel and ActionsPanel; `_update_ui()` call-sites updated accordingly - -### Removed (from layout, files retained) -- ❌ **Filter panel** no longer rendered as separate panel β€” `filter_panel.py` retained in codebase but not instantiated in dashboard -- ❌ **Input panel** no longer rendered as separate panel β€” `input_panel.py` retained in codebase but not instantiated in dashboard - -### Impact -- Cleaner, more compact dashboard: 2 fewer panels in the centre column -- All functionality preserved β€” message filtering, send, BOT toggle, archive all work identically -- No breaking changes to BLE, services, core or other panels - ---- - - - -## [1.5.0] - 2026-02-11 β€” Room Server Support, Dynamic Channel Discovery & Contact Management - -### Added -- βœ… **Room Server panel** β€” Dedicated per-room-server message panel in the centre column below Messages. Each Room Server (type=3 contact) gets its own `ui.card()` with login/logout controls and message display - - Click a Room Server contact to open an add/login dialog with password field - - After login: messages are displayed in the room card; send messages directly from the room panel - - Password row + login button automatically replaced by Logout button after successful login - - Room Server author attribution via `signature` field (txt_type=2) β€” real message author is resolved from the 4-byte pubkey prefix, not the room server pubkey - - New panel: `gui/panels/room_server_panel.py` β€” per-room card management with login state tracking -- βœ… **Room Server password store** β€” Passwords stored outside the repository in `~/.meshcore-gui/room_passwords/
.json` - - New service: `services/room_password_store.py` β€” JSON-backed persistent password storage per BLE device, analogous to `PinStore` - - Room panels are restored from stored passwords on app restart -- βœ… **Dynamic channel discovery** β€” Channels are now auto-discovered from the device at startup via `get_channel()` BLE probing, replacing the hardcoded `CHANNELS_CONFIG` - - Single-attempt probe per channel slot with early stop after 2 consecutive empty slots - - Channel name and encryption key extracted in a single pass (combined discovery + key loading) - - Configurable channel caching via `CHANNEL_CACHE_ENABLED` (default: `False` β€” always fresh from device) - - `MAX_CHANNELS` setting (default: 8) controls how many slots are probed -- βœ… **Individual contact deletion** β€” πŸ—‘οΈ delete button per unpinned contact in the contacts list, with confirmation dialog - - New command: `remove_single_contact` in BLE command handler - - Pinned contacts are protected (no delete button shown) -- βœ… **"Also delete from history" option** β€” Checkbox in the Clean up confirmation dialog to also remove locally cached contact data - - -- βœ… **Room Server protocol research** β€” `RoomServer_Companion_App_Onderzoek.md` documents the full companion app message flow (login, push protocol, signature mechanism, auto_message_fetching) - -### Changed -- πŸ”„ `config.py`: Removed `CHANNELS_CONFIG` constant; added `MAX_CHANNELS` (default: 8) and `CHANNEL_CACHE_ENABLED` (default: `False`) -- πŸ”„ `ble/worker.py`: Replaced hardcoded channel loading with `_discover_channels()` method; added `_try_get_channel_info()` helper; `_apply_cache()` respects `CHANNEL_CACHE_ENABLED` setting; removed `_load_channel_keys()` (integrated into discovery pass) -- πŸ”„ `ble/commands.py`: Added `login_room`, `send_room_msg` and `remove_single_contact` command handlers -- πŸ”„ `gui/panels/contacts_panel.py`: Contact click now dispatches by type β€” type=3 (Room Server) opens room dialog, others open DM dialog; added `on_add_room` callback parameter; added πŸ—‘οΈ delete button per unpinned contact -- πŸ”„ `gui/panels/messages_panel.py`: Room Server messages filtered from general message view via `_is_room_message()` with prefix matching; `update()` accepts `room_pubkeys` parameter -- πŸ”„ `gui/dashboard.py`: Added `RoomServerPanel` in centre column; `_update_ui()` passes `room_pubkeys` to Messages panel; added `_on_add_room_server` callback -- πŸ”„ `gui/panels/filter_panel.py`: Channel filter checkboxes now built dynamically from discovered channels (no hardcoded references) -- πŸ”„ `services/bot.py`: Removed stale comment referencing hardcoded channels - -### Fixed -- πŸ›  **Room Server messages appeared as DM** β€” Messages from Room Servers (txt_type=2) were displayed in the general Messages panel as direct messages. They are now filtered out and shown exclusively in the Room Server panel -- πŸ›  **Historical room messages not shown after login** β€” Post-login fetch loop was polling `get_msg()` before room server had time to push messages over LoRa RF (10–75s per message). Removed redundant fetch loop; the library's `auto_message_fetching` handles `MESSAGES_WAITING` events correctly and event-driven -- πŸ›  **Author attribution incorrect for room messages** β€” Room server messages showed the room server name as sender instead of the actual message author. Now correctly resolved from the `signature` field (4-byte pubkey prefix) via contact lookup - -### Impact -- Room Servers are now first-class citizens in the GUI with dedicated panels -- Channel configuration no longer requires manual editing of `config.py` -- Contact list management is more granular with per-contact deletion -- No breaking changes to existing functionality (messages, DM, map, archive, bot, etc.) - ---- - -## [1.4.0] - 2026-02-09 β€” SDK Event Race Condition Fix - -### Fixed -- πŸ›  **BLE startup delay of ~2 minutes eliminated** β€” The meshcore Python SDK (`commands/base.py`) dispatched device response events before `wait_for_events()` registered its subscription. On busy networks with frequent `RX_LOG_DATA` events, this caused `send_device_query()` and `get_channel()` to fail repeatedly with `no_event_received`, wasting 110+ seconds in timeouts - -### Changed -- πŸ“„ `meshcore` SDK `commands/base.py`: Rewritten `send()` method to subscribe to expected events **before** transmitting the BLE command (subscribe-before-send pattern), matching the approach used by the companion apps (meshcore.js, iOS, Android). Submitted upstream as [meshcore_py PR #52](https://github.com/meshcore-dev/meshcore_py/pull/52) - -### Impact -- Startup time reduced from ~2+ minutes to ~10 seconds on busy networks -- All BLE commands (`send_device_query`, `get_channel`, `get_bat`, `send_appstart`, etc.) now succeed on first attempt instead of requiring multiple retries -- No changes to meshcore_gui code required β€” the fix is entirely in the meshcore SDK - -### Temporary Installation -Until the fix is merged upstream, install the patched meshcore SDK: -```bash -pip install --force-reinstall git+https://github.com/PE1HVH/meshcore_py.git@fix/event-race-condition -``` - ---- - - - -## [1.3.2] - 2026-02-09 β€” Bugfix: Bot Device Name Restoration After Restart - -### Fixed -- πŸ›  **Bot device name not properly restored after restart/crash** β€” After a restart or crash with bot mode previously active, the original device name was incorrectly stored as the bot name (e.g. `NL-OV-ZWL-STDSHGN-WKC Bot`) instead of the real device name (e.g. `PE1HVH T1000e`). The original device name is now correctly preserved and restored when bot mode is disabled - -### Changed -- πŸ”„ `commands.py`: `set_bot_name` handler now verifies that the stored original name is not already the bot name before saving -- πŸ”„ `shared_data.py`: `original_device_name` is only written when it differs from `BOT_DEVICE_NAME` to prevent overwriting with the bot name on restart - ---- - - - -## [1.3.1] - 2026-02-09 β€” Bugfix: Auto-add AttributeError - -### Fixed -- πŸ›  **Auto-add error on first toggle** β€” Setting auto-add for the first time raised `AttributeError: 'telemetry_mode_base'`. The `set_manual_add_contacts()` SDK call now handles missing `telemetry_mode_base` attribute gracefully - -### Changed -- πŸ”„ `commands.py`: `set_auto_add` handler wraps `set_manual_add_contacts()` call with attribute check and error handling for missing `telemetry_mode_base` - ---- - - - -## [1.3.0] - 2026-02-08 β€” Bot Device Name Management - -### Added -- βœ… **Bot device name switching** β€” When the BOT checkbox is enabled, the device name is automatically changed to a configurable bot name; when disabled, the original name is restored - - Original device name is saved before renaming so it can be restored on BOT disable - - Device name written to device via BLE `set_name()` SDK call - - Graceful handling of BLE failures during name change -- βœ… **`BOT_DEVICE_NAME` constant** in `config.py` β€” Configurable fixed device name used when bot mode is active (default: `;NL-OV-ZWL-STDSHGN-WKC Bot`) - -### Changed -- πŸ”„ `config.py`: Added `BOT_DEVICE_NAME` constant for bot mode device name -- πŸ”„ `bot.py`: Removed hardcoded `BOT_NAME` prefix ("Zwolle Bot") from bot reply messages β€” bot replies no longer include a name prefix -- πŸ”„ `filter_panel.py`: BOT checkbox toggle now triggers device name save/rename via command queue -- πŸ”„ `commands.py`: Added `set_bot_name` and `restore_name` command handlers for device name switching -- πŸ”„ `shared_data.py`: Added `original_device_name` field for storing the pre-bot device name - -### Removed -- ❌ `BOT_NAME` constant from `bot.py` β€” bot reply prefix removed; replies no longer prepend a bot display name - ---- - -## [1.2.0] - 2026-02-08 β€” Contact Maintenance Feature - -### Added -- βœ… **Pin/Unpin contacts** (Iteration A) β€” Toggle to pin individual contacts, protecting them from bulk deletion - - Persistent pin state stored in `~/.meshcore-gui/cache/
_pins.json` - - Pinned contacts visually marked with yellow background - - Pinned contacts sorted to top of contact list - - Pin state survives app restart - - New service: `services/pin_store.py` β€” JSON-backed persistent pin storage - -- βœ… **Bulk delete unpinned contacts** (Iteration B) β€” Remove all unpinned contacts from device in one action - - "🧹 Clean up" button in contacts panel with confirmation dialog - - Shows count of contacts to be removed vs. pinned contacts kept - - Progress status updates during removal - - Automatic device resync after completion - - New service: `services/contact_cleaner.py` β€” ContactCleanerService with purge statistics - -- βœ… **Auto-add contacts toggle** (Iteration C) β€” Control whether device automatically adds new contacts from mesh adverts - - "πŸ“₯ Auto-add" checkbox in contacts panel (next to Clean up button) - - Syncs with device via `set_manual_add_contacts()` SDK call - - Inverted logic handled internally (UI "Auto-add ON" = `set_manual_add_contacts(false)`) - - Optimistic update with automatic rollback on BLE failure - - State synchronized from device on each GUI update cycle - -### Changed -- πŸ”„ `contacts_panel.py`: Added pin checkbox per contact, purge button, auto-add toggle, DM dialog (all existing functionality preserved) -- πŸ”„ `commands.py`: Added `purge_unpinned` and `set_auto_add` command handlers -- πŸ”„ `shared_data.py`: Added `auto_add_enabled` field with thread-safe getter/setter -- πŸ”„ `protocols.py`: Added `set_auto_add_enabled` and `is_auto_add_enabled` to Writer and Reader protocols -- πŸ”„ `dashboard.py`: Passes `PinStore` and `set_auto_add_enabled` callback to ContactsPanel -- πŸ”„ **UI language**: All Dutch strings in `contacts_panel.py` and `commands.py` translated to English - ---- - -### Fixed -- πŸ›  **Route table names and IDs not displayed** β€” Route tables in both current messages (RoutePage) and archive messages (ArchivePage) now correctly show node names and public key IDs for sender, repeaters and receiver - -### Changed -- πŸ”„ **CHANGELOG.md**: Corrected version numbering to semantic versioning, fixed inaccurate references (archive button location, filter state persistence) -- πŸ”„ **README.md**: Added Message Archive feature, updated project structure, configuration table and architecture diagram -- πŸ”„ **MeshCore_GUI_Design.docx**: Added ArchivePage, MessageArchive, Models components; updated project structure, protocols, configuration and version history - ---- - -## [1.1.0] - 2026-02-07 β€” Archive Viewer Feature - - -### Added -- βœ… **Archive Viewer Page** (`/archive`) β€” Full-featured message archive browser - - Pagination (50 messages per page, configurable) - - Channel filter dropdown (All + configured channels) - - Time range filter (24h, 7d, 30d, 90d, All time) - - Text search (case-insensitive) - - Filter state stored in instance variables (reset on page reload) - - Message cards with same styling as main messages panel - - Clickable messages for route visualization (where available) - - **πŸ’¬ Reply functionality** β€” Expandable reply panel per message - - **πŸ—ΊοΈ Inline route table** β€” Expandable route display per archive message with sender, repeaters and receiver (names, IDs, node types) - - *(Note: Reply panels and inline route tables removed in v1.8.0, replaced by click-to-route navigation via message hash)* - - - - - -- βœ… **MessageArchive.query_messages()** method - - Filter by: time range, channel, text search, sender - - Pagination support (limit, offset) - - Returns tuple: (messages, total_count) - - Sorting: Newest first - -- βœ… **UI Integration** - - "πŸ“š Archive" button in Messages panel header (opens in new tab) - - Back to Dashboard button in archive page - - - -- βœ… **Reply Panel** - - Expandable reply per message (πŸ’¬ Reply button) - - Pre-filled with @sender mention - - Channel selector - - Send button with success notification - - Auto-close expansion after send - -### Changed -- πŸ”„ `SharedData.get_snapshot()`: Now includes `'archive'` field -- πŸ”„ `MessagesPanel`: Added archive button in header row -- πŸ”„ Both entry points (`__main__.py` and `meshcore_gui.py`): Register `/archive` route - - - -### Performance -- Query: ~10ms for 10k messages with filters -- Memory: ~10KB per page (50 messages) -- No impact on main UI (separate page) - -### Known Limitations -- ~~Route visualization only works for messages in recent buffer (last 100)~~ β€” Fixed in v1.8.0: archive messages now support click-to-route via `get_message_by_hash()` fallback -- Text search is linear scan (no indexing yet) -- Sender filter exists in API but not in UI yet - ---- - -## [1.0.3] - 2026-02-07 β€” Critical Bugfix: Archive Overwrite Prevention - - -### Fixed -- πŸ›  **CRITICAL**: Fixed bug where archive was overwritten instead of appended on restart -- πŸ›  Archive now preserves existing data when read errors occur -- πŸ›  Buffer is retained for retry if existing archive cannot be read - -### Changed -- πŸ”„ `_flush_messages()`: Early return on read error instead of overwriting -- πŸ”„ `_flush_rxlog()`: Early return on read error instead of overwriting -- πŸ”„ Better error messages for version mismatch and JSON decode errors - -### Details -**Problem:** If the existing archive file had a JSON parse error or version mismatch, -the flush operation would proceed with `existing_messages = []`, effectively -overwriting all historical data with only the new buffered messages. - -**Solution:** The flush methods now: -1. Try to read existing archive first -2. If read fails (JSON error, version mismatch, IO error), abort the flush -3. Keep buffer intact for next retry -4. Only clear buffer after successful write - -**Impact:** No data loss on restart or when archive files have issues. - -### Testing -- βœ… Added `test_append_on_restart_not_overwrite()` integration test -- βœ… Verifies data is appended across multiple sessions -- βœ… All existing tests still pass - ---- - -## [1.0.2] - 2026-02-07 β€” RxLog message_hash Enhancement - - -### Added -- βœ… `message_hash` field added to `RxLogEntry` model -- βœ… RxLog entries now include message_hash for correlation with messages -- βœ… Archive JSON includes message_hash in rxlog entries - -### Changed -- πŸ”„ `events.py`: Restructured `on_rx_log()` to extract message_hash before creating RxLogEntry -- πŸ”„ `message_archive.py`: Updated rxlog archiving to include message_hash field -- πŸ”„ Tests updated to verify message_hash persistence - -### Benefits -- **Correlation**: Link RX log entries to their corresponding messages -- **Analysis**: Track which packets resulted in messages -- **Debugging**: Better troubleshooting of packet processing - ---- - -## [1.0.1] - 2026-02-07 β€” Entry Point Fix - - -### Fixed -- βœ… `meshcore_gui.py` (root entry point) now passes ble_address to SharedData -- βœ… Archive works correctly regardless of how application is started - -### Changed -- πŸ”„ Both entry points (`meshcore_gui.py` and `meshcore_gui/__main__.py`) updated - ---- - -## [1.0.0] - 2026-02-07 β€” Message & Metadata Persistence - - -### Added -- βœ… MessageArchive class for persistent storage -- βœ… Configurable retention periods (MESSAGE_RETENTION_DAYS, RXLOG_RETENTION_DAYS, CONTACT_RETENTION_DAYS) -- βœ… Automatic daily cleanup of old data -- βœ… Batch writes for performance -- βœ… Thread-safe with separate locks -- βœ… Atomic file writes -- βœ… Contact retention in DeviceCache -- βœ… Archive statistics API -- βœ… Comprehensive tests (20+ unit, 8+ integration) -- βœ… Full documentation - -### Storage Locations -- `~/.meshcore-gui/archive/
_messages.json` -- `~/.meshcore-gui/archive/
_rxlog.json` - -### Requirements Completed -- R1: All incoming messages persistent βœ… -- R2: All incoming RxLog entries persistent βœ… -- R3: Configurable retention βœ… -- R4: Automatic cleanup βœ… -- R5: Backward compatibility βœ… -- R6: Contact retention βœ… -- R7: Archive stats API βœ… - -- Fix3: Leaflet asset injection is now per page render instead of process-global, and browser bootstrap now retries until the host element, Leaflet runtime, and MeshCore panel runtime are all available. This fixes blank map containers caused by missing or late-loaded JS/CSS assets. - -- Fix5: Removed per-snapshot map invalidate calls, stopped forcing a default dark theme during map bootstrap, and added client-side interaction/resize guards so zooming stays responsive and the theme no longer jumps back during status-loop updates. - - -## 2026-03-09 map hotfix v2 -- regular map snapshots no longer carry theme state -- explicit theme changes are now handled only via the dedicated theme channel -- initial map render now sends an ensure_map command plus an immediate theme sync -- added no-op ensure_map handling in the Leaflet runtime to avoid accidental fallback behaviour diff --git a/meshcore_gui/ble/commands.py b/meshcore_gui/ble/commands.py index 9203d93..027861f 100644 --- a/meshcore_gui/ble/commands.py +++ b/meshcore_gui/ble/commands.py @@ -330,13 +330,17 @@ class CommandHandler: debug_print(f"set_device_name exception: {exc}") async def _cmd_login_room(self, cmd: Dict) -> None: - """Login to a Room Server. + """Send a Room Server login request. - Follows the reference implementation (meshcore-cli): - 1. ``send_login()`` β†’ wait for ``MSG_SENT`` (companion radio sent LoRa packet) - 2. ``wait_for_event(LOGIN_SUCCESS)`` β†’ wait for room server confirmation - 3. After LOGIN_SUCCESS, the room server starts pushing historical - messages over RF. ``auto_message_fetching`` handles those. + This command handler owns only the *send* side of the login flow: + it queues archived room history for immediate UI display, marks the + room state as ``pending`` and sends the login packet to the companion + radio. + + The definitive ``LOGIN_SUCCESS`` handling is intentionally centralised + in :mod:`meshcore_gui.ble.worker`, which already subscribes to the + MeshCore event stream. Keeping the success path in one place avoids a + second competing wait/timeout path here in the command layer. Expected command dict:: @@ -355,20 +359,28 @@ class CommandHandler: self._shared.set_status("⚠️ Room login: no pubkey") return - # Load archived room messages so the panel shows history - # while we wait for the LoRa login handshake. + # Show archived room messages immediately while the radio/login path + # continues asynchronously. self._shared.load_room_history(pubkey) - - # Mark pending in SharedData so the panel can update self._shared.set_room_login_state(pubkey, 'pending', 'Sending login…') try: - # Step 1: Send login request to companion radio - self._shared.set_status( - f"πŸ”„ Sending login to {room_name}…" - ) + self._shared.set_status(f"πŸ”„ Sending login to {room_name}…") r = await self._mc.commands.send_login(pubkey, password) + if r is None: + self._shared.set_room_login_state( + pubkey, 'fail', 'Login send returned no response', + ) + self._shared.set_status( + f"⚠️ Room login failed: {room_name}" + ) + debug_print( + f"login_room: send_login returned None for {room_name} " + f"({pubkey[:16]})" + ) + return + if r.type == EventType.ERROR: self._shared.set_room_login_state( pubkey, 'fail', 'Login send failed', @@ -382,70 +394,20 @@ class CommandHandler: ) return - # Step 2: Wait for LOGIN_SUCCESS from room server via LoRa - # Use suggested_timeout from companion radio if available, - # otherwise default to 120 seconds (LoRa can be slow). suggested = (r.payload or {}).get('suggested_timeout', 96000) timeout_secs = max(suggested / 800, 30.0) - self._shared.set_status( f"⏳ Waiting for room server response ({room_name})…" ) debug_print( - f"login_room: MSG_SENT OK, waiting for LOGIN_SUCCESS " - f"(timeout={timeout_secs:.0f}s)" + f"login_room: login packet accepted for {room_name}; " + f"worker owns LOGIN_SUCCESS handling " + f"(suggested timeout {timeout_secs:.0f}s)" ) - login_event = await self._mc.wait_for_event( - EventType.LOGIN_SUCCESS, timeout=timeout_secs, - ) - - if login_event and login_event.type == EventType.LOGIN_SUCCESS: - is_admin = (login_event.payload or {}).get('is_admin', False) - self._shared.set_room_login_state( - pubkey, 'ok', - f"admin={is_admin}", - ) - self._shared.set_status( - f"βœ… Room login OK: {room_name} β€” " - f"history arriving over RF…" - ) - debug_print( - f"login_room: LOGIN_SUCCESS for {room_name} " - f"(admin={is_admin})" - ) - - # Defensive: trigger one get_msg() to check for any - # messages already waiting in the companion radio's - # offline queue. auto_message_fetching handles the - # rest via MESSAGES_WAITING events. - try: - await self._mc.commands.get_msg() - debug_print("login_room: defensive get_msg() done") - except Exception as exc: - debug_print(f"login_room: defensive get_msg() error: {exc}") - - else: - self._shared.set_room_login_state( - pubkey, 'fail', - 'Timeout β€” no response from room server', - ) - self._shared.set_status( - f"⚠️ Room login timeout: {room_name} " - f"(no response after {timeout_secs:.0f}s)" - ) - debug_print( - f"login_room: LOGIN_SUCCESS timeout for " - f"{room_name} ({pubkey[:16]})" - ) - except Exception as exc: - self._shared.set_room_login_state( - pubkey, 'fail', str(exc), - ) - self._shared.set_status( - f"⚠️ Room login error: {exc}" - ) + self._shared.set_room_login_state(pubkey, 'fail', str(exc)) + self._shared.set_status(f"⚠️ Room login error: {exc}") debug_print(f"login_room exception: {exc}") async def _cmd_logout_room(self, cmd: Dict) -> None: diff --git a/meshcore_gui/ble/events.py b/meshcore_gui/ble/events.py index 8fbf16f..5700d40 100644 --- a/meshcore_gui/ble/events.py +++ b/meshcore_gui/ble/events.py @@ -77,6 +77,16 @@ class EventHandler: names.append(h.upper()) return names + @staticmethod + def _looks_like_hex_identifier(value: str) -> bool: + """Return True when *value* looks like a pubkey/hash prefix.""" + if not value: + return False + probe = str(value).strip() + if len(probe) < 6: + return False + return all(ch in '0123456789abcdefABCDEF' for ch in probe) + # ------------------------------------------------------------------ # RX_LOG_DATA β€” the single source of truth for path info # ------------------------------------------------------------------ @@ -292,38 +302,83 @@ class EventHandler: def on_contact_msg(self, event) -> None: """Handle direct message and room message events. - Room Server messages arrive as ``CONTACT_MSG_RECV`` with - ``txt_type == 2``. The ``pubkey_prefix`` is the Room Server's - key and the ``signature`` field contains the original author's - pubkey prefix. We resolve the author name from ``signature`` - so the UI shows who actually wrote the message. + Room Server traffic also arrives as ``CONTACT_MSG_RECV``. + In practice the payload is not stable enough to rely only on + ``signature`` + ``pubkey_prefix``. Incoming room messages from + *other* participants may omit ``signature`` and may carry the + room key in receiver-style fields instead of ``pubkey_prefix``. + + To keep the rest of the GUI unchanged, room messages are stored + with ``sender`` = actual author name and ``sender_pubkey`` = room + public key. The Room Server panel already filters on + ``sender_pubkey`` to decide to which room a message belongs. """ - payload = event.payload + payload = event.payload or {} pubkey = payload.get('pubkey_prefix', '') txt_type = payload.get('txt_type', 0) signature = payload.get('signature', '') - debug_print(f"DM payload keys: {list(payload.keys())}") + debug_print( + "DM payload keys: " + f"{list(payload.keys())}; txt_type={txt_type}; " + f"pubkey_prefix={pubkey[:12]}; " + f"receiver={(payload.get('receiver') or '')[:12]}; " + f"room_pubkey={(payload.get('room_pubkey') or '')[:12]}; " + f"signature={(signature or '')[:12]}" + ) # Common fields for both Room and DM messages msg_hash = payload.get('message_hash', '') path_hashes = self._path_cache.pop(msg_hash, []) if msg_hash else [] path_names = self._resolve_path_names(path_hashes) - # DM payloads may report path_len=255 (0xFF) meaning "unknown"; - # treat as 0 when no actual path data is available. raw_path_len = payload.get('path_len', 0) path_len = raw_path_len if raw_path_len < 255 else 0 if path_hashes: - # Trust actual decoded hashes over the raw header value path_len = len(path_hashes) - # --- Room Server message (txt_type 2) --- - if txt_type == 2 and signature: - # Resolve actual author from signature (author pubkey prefix) - author = self._shared.get_contact_name_by_prefix(signature) + room_pubkey = ( + payload.get('room_pubkey') + or payload.get('receiver') + or payload.get('receiver_pubkey') + or payload.get('receiver_pubkey_prefix') + or pubkey + or '' + ) + + is_room_message = txt_type == 2 + + if is_room_message: + author = '' + explicit_name = ( + payload.get('author') + or payload.get('sender_name') + or payload.get('name') + or '' + ) + if explicit_name and not self._looks_like_hex_identifier(explicit_name): + author = explicit_name + + sender_field = str(payload.get('sender') or '').strip() + if not author and sender_field and not self._looks_like_hex_identifier(sender_field): + author = sender_field + + author_key = ( + signature + or payload.get('sender_pubkey') + or payload.get('author_pubkey') + or (sender_field if self._looks_like_hex_identifier(sender_field) else '') + or '' + ) + if not author and author_key: + author = self._shared.get_contact_name_by_prefix(author_key) if not author: - author = signature[:8] if signature else '?' + author = ( + explicit_name + or sender_field + or (author_key[:8] if author_key else '') + or '?' + ) self._shared.add_message(Message.incoming( author, @@ -331,14 +386,14 @@ class EventHandler: None, snr=self._extract_snr(payload), path_len=path_len, - sender_pubkey=pubkey, + sender_pubkey=room_pubkey, path_hashes=path_hashes, path_names=path_names, message_hash=msg_hash, )) debug_print( - f"Room msg from {author} (sig={signature}) " - f"via room {pubkey[:12]}: " + f"Room msg from {author} via room {room_pubkey[:12]} " + f"(sig={signature[:12] if signature else '-'}): " f"{payload.get('text', '')[:30]}" ) return @@ -348,7 +403,11 @@ class EventHandler: if pubkey: sender = self._shared.get_contact_name_by_prefix(pubkey) if not sender: - sender = pubkey[:8] if pubkey else '' + sender = ( + payload.get('name') + or payload.get('sender') + or (pubkey[:8] if pubkey else '') + ) self._shared.add_message(Message.incoming( sender, diff --git a/meshcore_gui/ble/events.py.bak b/meshcore_gui/ble/events.py.bak new file mode 100644 index 0000000..8fbf16f --- /dev/null +++ b/meshcore_gui/ble/events.py.bak @@ -0,0 +1,379 @@ +""" +Device event callbacks for MeshCore GUI. + +Handles ``CHANNEL_MSG_RECV``, ``CONTACT_MSG_RECV`` and ``RX_LOG_DATA`` +events from the MeshCore library. Extracted from ``SerialWorker`` so the +worker only deals with connection lifecycle. +""" + +from typing import Dict, Optional + +from meshcore_gui.config import debug_print +from meshcore_gui.core.models import Message, RxLogEntry +from meshcore_gui.core.protocols import SharedDataWriter +from meshcore_gui.ble.packet_decoder import PacketDecoder, PayloadType +from meshcore_gui.services.bot import MeshBot +from meshcore_gui.services.dedup import DualDeduplicator + + +class EventHandler: + """Processes device events and writes results to shared data. + + Args: + shared: SharedDataWriter for storing messages and RX log. + decoder: PacketDecoder for raw LoRa packet decryption. + dedup: DualDeduplicator for message deduplication. + bot: MeshBot for auto-reply logic. + """ + + # Maximum entries in the path cache before oldest are evicted. + _PATH_CACHE_MAX = 200 + + def __init__( + self, + shared: SharedDataWriter, + decoder: PacketDecoder, + dedup: DualDeduplicator, + bot: MeshBot, + ) -> None: + self._shared = shared + self._decoder = decoder + self._dedup = dedup + self._bot = bot + + # Cache: message_hash β†’ path_hashes (from RX_LOG decode). + # Used by on_channel_msg fallback to recover hashes that the + # CHANNEL_MSG_RECV event does not provide. + self._path_cache: Dict[str, list] = {} + + # ------------------------------------------------------------------ + # Helpers β€” resolve names at receive time + # ------------------------------------------------------------------ + + def _resolve_path_names(self, path_hashes: list) -> list: + """Resolve 2-char path hashes to display names. + + Performs a contact lookup for each hash *now* so the names are + captured at receive time and stored in the archive. + + Args: + path_hashes: List of 2-char hex strings. + + Returns: + List of display names (same length as *path_hashes*). + Unknown hashes become their uppercase hex value. + """ + names = [] + for h in path_hashes: + if not h or len(h) < 2: + names.append('-') + continue + name = self._shared.get_contact_name_by_prefix(h) + # get_contact_name_by_prefix returns h[:8] as fallback, + # normalise to uppercase hex for 2-char hashes. + if name and name != h[:8]: + names.append(name) + else: + names.append(h.upper()) + return names + + # ------------------------------------------------------------------ + # RX_LOG_DATA β€” the single source of truth for path info + # ------------------------------------------------------------------ + + def on_rx_log(self, event) -> None: + """Handle RX log data events.""" + payload = event.payload + + # Extract basic RX log info + time_str = Message.now_timestamp() + snr = payload.get('snr', 0) + rssi = payload.get('rssi', 0) + payload_type = '?' + hops = payload.get('path_len', 0) + + # Try to decode payload to get message_hash + message_hash = "" + rx_path_hashes: list = [] + rx_path_names: list = [] + rx_sender: str = "" + rx_receiver: str = self._shared.get_device_name() or "" + payload_hex = payload.get('payload', '') + decoded = None + if payload_hex: + decoded = self._decoder.decode(payload_hex) + if decoded is not None: + message_hash = decoded.message_hash + payload_type = self._decoder.get_payload_type_text(decoded.payload_type) + + # Capture path info for all packet types + if decoded.path_hashes: + rx_path_hashes = decoded.path_hashes + rx_path_names = self._resolve_path_names(decoded.path_hashes) + + # Use decoded path_length (from packet body) β€” more + # reliable than the frame-header path_len which can be 0. + if decoded.path_length: + hops = decoded.path_length + + # Capture sender name when available (GroupText only) + if decoded.sender: + rx_sender = decoded.sender + + # Cache path_hashes for correlation with on_channel_msg + if decoded.path_hashes and message_hash: + self._path_cache[message_hash] = decoded.path_hashes + # Evict oldest entries if cache is too large + if len(self._path_cache) > self._PATH_CACHE_MAX: + oldest = next(iter(self._path_cache)) + del self._path_cache[oldest] + + # Process decoded message if it's a group text + if decoded.payload_type == PayloadType.GroupText and decoded.is_decrypted: + if decoded.channel_idx is None: + # The channel hash could not be resolved to a channel index + # (PacketDecoder._hash_to_idx lookup returned None). + # Marking dedup here would suppress on_channel_msg, which + # carries a valid channel_idx from the device event β€” the only + # path through which the bot can pass Guard 2 and respond. + # Skip the entire block; on_channel_msg handles message + bot. + # Path info is already in _path_cache for on_channel_msg to use. + debug_print( + f"RX_LOG β†’ GroupText decrypted but channel_idx unresolved " + f"(hash={decoded.message_hash}); deferring to on_channel_msg" + ) + else: + self._dedup.mark_hash(decoded.message_hash) + self._dedup.mark_content( + decoded.sender, decoded.channel_idx, decoded.text, + ) + + sender_pubkey = '' + if decoded.sender: + match = self._shared.get_contact_by_name(decoded.sender) + if match: + sender_pubkey, _contact = match + + snr_msg = self._extract_snr(payload) + + self._shared.add_message(Message.incoming( + decoded.sender, + decoded.text, + decoded.channel_idx, + time=time_str, + snr=snr_msg, + path_len=decoded.path_length, + sender_pubkey=sender_pubkey, + path_hashes=decoded.path_hashes, + path_names=rx_path_names, + message_hash=decoded.message_hash, + )) + + debug_print( + f"RX_LOG β†’ message: hash={decoded.message_hash}, " + f"sender={decoded.sender!r}, ch={decoded.channel_idx}, " + f"path={decoded.path_hashes}, " + f"path_names={rx_path_names}" + ) + + self._bot.check_and_reply( + sender=decoded.sender, + text=decoded.text, + channel_idx=decoded.channel_idx, + snr=snr_msg, + path_len=decoded.path_length, + path_hashes=decoded.path_hashes, + ) + + # Add RX log entry with message_hash and path info (if available) + # ── Fase 1 Observer: raw packet metadata ── + raw_packet_len = len(payload_hex) // 2 if payload_hex else 0 + raw_payload_len = max(0, raw_packet_len - 1 - hops) if payload_hex else 0 + raw_route_type = "D" if hops > 0 else ("F" if payload_hex else "") + raw_packet_type_num = -1 + if payload_hex and decoded is not None: + try: + raw_packet_type_num = decoded.payload_type.value + except (AttributeError, ValueError): + pass + + self._shared.add_rx_log(RxLogEntry( + time=time_str, + snr=snr, + rssi=rssi, + payload_type=payload_type, + hops=hops, + message_hash=message_hash, + path_hashes=rx_path_hashes, + path_names=rx_path_names, + sender=rx_sender, + receiver=rx_receiver, + raw_payload=payload_hex, + packet_len=raw_packet_len, + payload_len=raw_payload_len, + route_type=raw_route_type, + packet_type_num=raw_packet_type_num, + )) + + # ------------------------------------------------------------------ + # CHANNEL_MSG_RECV β€” fallback when RX_LOG decode missed it + # ------------------------------------------------------------------ + + def on_channel_msg(self, event) -> None: + """Handle channel message events.""" + payload = event.payload + + debug_print(f"Channel msg payload keys: {list(payload.keys())}") + + # Dedup via hash + msg_hash = payload.get('message_hash', '') + if msg_hash and self._dedup.is_hash_seen(msg_hash): + debug_print(f"Channel msg suppressed (hash): {msg_hash}") + return + + # Parse sender from "SenderName: message body" format + raw_text = payload.get('text', '') + sender, msg_text = '', raw_text + if ': ' in raw_text: + name_part, body_part = raw_text.split(': ', 1) + sender = name_part.strip() + msg_text = body_part + elif raw_text: + msg_text = raw_text + + # Dedup via content + ch_idx = payload.get('channel_idx') + if self._dedup.is_content_seen(sender, ch_idx, msg_text): + debug_print(f"Channel msg suppressed (content): {sender!r}") + return + + debug_print( + f"Channel msg (fallback): sender={sender!r}, " + f"text={msg_text[:40]!r}" + ) + + sender_pubkey = '' + if sender: + match = self._shared.get_contact_by_name(sender) + if match: + sender_pubkey, _contact = match + + snr = self._extract_snr(payload) + + # Recover path_hashes from RX_LOG cache (CHANNEL_MSG_RECV + # does not carry them, but the preceding RX_LOG decode does). + path_hashes = self._path_cache.pop(msg_hash, []) if msg_hash else [] + path_names = self._resolve_path_names(path_hashes) + + self._shared.add_message(Message.incoming( + sender, + msg_text, + ch_idx, + snr=snr, + path_len=payload.get('path_len', 0), + sender_pubkey=sender_pubkey, + path_hashes=path_hashes, + path_names=path_names, + message_hash=msg_hash, + )) + + self._bot.check_and_reply( + sender=sender, + text=msg_text, + channel_idx=ch_idx, + snr=snr, + path_len=payload.get('path_len', 0), + ) + + # ------------------------------------------------------------------ + # CONTACT_MSG_RECV β€” DMs + # ------------------------------------------------------------------ + + def on_contact_msg(self, event) -> None: + """Handle direct message and room message events. + + Room Server messages arrive as ``CONTACT_MSG_RECV`` with + ``txt_type == 2``. The ``pubkey_prefix`` is the Room Server's + key and the ``signature`` field contains the original author's + pubkey prefix. We resolve the author name from ``signature`` + so the UI shows who actually wrote the message. + """ + payload = event.payload + pubkey = payload.get('pubkey_prefix', '') + txt_type = payload.get('txt_type', 0) + signature = payload.get('signature', '') + + debug_print(f"DM payload keys: {list(payload.keys())}") + + # Common fields for both Room and DM messages + msg_hash = payload.get('message_hash', '') + path_hashes = self._path_cache.pop(msg_hash, []) if msg_hash else [] + path_names = self._resolve_path_names(path_hashes) + + # DM payloads may report path_len=255 (0xFF) meaning "unknown"; + # treat as 0 when no actual path data is available. + raw_path_len = payload.get('path_len', 0) + path_len = raw_path_len if raw_path_len < 255 else 0 + if path_hashes: + # Trust actual decoded hashes over the raw header value + path_len = len(path_hashes) + + # --- Room Server message (txt_type 2) --- + if txt_type == 2 and signature: + # Resolve actual author from signature (author pubkey prefix) + author = self._shared.get_contact_name_by_prefix(signature) + if not author: + author = signature[:8] if signature else '?' + + self._shared.add_message(Message.incoming( + author, + payload.get('text', ''), + None, + snr=self._extract_snr(payload), + path_len=path_len, + sender_pubkey=pubkey, + path_hashes=path_hashes, + path_names=path_names, + message_hash=msg_hash, + )) + debug_print( + f"Room msg from {author} (sig={signature}) " + f"via room {pubkey[:12]}: " + f"{payload.get('text', '')[:30]}" + ) + return + + # --- Regular DM --- + sender = '' + if pubkey: + sender = self._shared.get_contact_name_by_prefix(pubkey) + if not sender: + sender = pubkey[:8] if pubkey else '' + + self._shared.add_message(Message.incoming( + sender, + payload.get('text', ''), + None, + snr=self._extract_snr(payload), + path_len=path_len, + sender_pubkey=pubkey, + path_hashes=path_hashes, + path_names=path_names, + message_hash=msg_hash, + )) + debug_print(f"DM received from {sender}: {payload.get('text', '')[:30]}") + + # ------------------------------------------------------------------ + # Helpers + # ------------------------------------------------------------------ + + @staticmethod + def _extract_snr(payload: Dict) -> Optional[float]: + """Extract SNR from a payload dict (handles 'SNR' and 'snr' keys).""" + raw = payload.get('SNR') or payload.get('snr') + if raw is not None: + try: + return float(raw) + except (ValueError, TypeError): + pass + return None diff --git a/meshcore_gui/ble/worker.py b/meshcore_gui/ble/worker.py index 6aac002..484ff0a 100644 --- a/meshcore_gui/ble/worker.py +++ b/meshcore_gui/ble/worker.py @@ -258,11 +258,38 @@ class _BaseWorker(abc.ABC): # ── LOGIN_SUCCESS handler (Room Server) ─────────────────────── def _on_login_success(self, event) -> None: + """Handle Room Server login confirmation. + + This worker callback is the *only* definitive success path for room + login. The command layer sends the login request and leaves the final + transition to ``ok`` to this subscriber so there is no competing + timeout/success logic elsewhere. + + The device event may expose the room key under different fields. + Update both the generic status line and the per-room login state, + then refresh archived room history for the matched room. + """ payload = event.payload or {} - pubkey = payload.get("pubkey_prefix", "") + pubkey = ( + payload.get("room_pubkey") + or payload.get("receiver") + or payload.get("receiver_pubkey") + or payload.get("pubkey_prefix") + or "" + ) is_admin = payload.get("is_admin", False) - debug_print(f"LOGIN_SUCCESS received: pubkey={pubkey}, admin={is_admin}") + debug_print( + f"LOGIN_SUCCESS received: pubkey={pubkey}, admin={is_admin}, " + f"keys={list(payload.keys())}" + ) self.shared.set_status("βœ… Room login OK β€” messages arriving over RF…") + if pubkey: + self.shared.set_room_login_state( + pubkey, 'ok', f'Server confirmed login (admin={is_admin})', + ) + self.shared.load_room_history(pubkey) + else: + debug_print('LOGIN_SUCCESS received without identifiable room pubkey') # ── apply cache ─────────────────────────────────────────────── diff --git a/meshcore_gui/ble/worker.py.bak b/meshcore_gui/ble/worker.py.bak new file mode 100644 index 0000000..6aac002 --- /dev/null +++ b/meshcore_gui/ble/worker.py.bak @@ -0,0 +1,964 @@ +""" +Communication worker for MeshCore GUI (Serial + BLE). + +Runs in a separate thread with its own asyncio event loop. Connects +to the MeshCore device, wires up collaborators, and runs the command +processing loop. + +Transport selection +~~~~~~~~~~~~~~~~~~~~ +The :func:`create_worker` factory returns the appropriate worker class +based on the device identifier: + +- ``/dev/ttyACM0`` β†’ :class:`SerialWorker` (USB serial) +- ``literal:AA:BB:CC:DD:EE:FF`` β†’ :class:`BLEWorker` (Bluetooth LE) + +Both workers share the same base class (:class:`_BaseWorker`) which +implements the main loop, event wiring, data loading and caching. + +Command execution β†’ :mod:`meshcore_gui.ble.commands` +Event handling β†’ :mod:`meshcore_gui.ble.events` +Packet decoding β†’ :mod:`meshcore_gui.ble.packet_decoder` +PIN agent (BLE) β†’ :mod:`meshcore_gui.ble.ble_agent` +Reconnect (BLE) β†’ :mod:`meshcore_gui.ble.ble_reconnect` +Bot logic β†’ :mod:`meshcore_gui.services.bot` +Deduplication β†’ :mod:`meshcore_gui.services.dedup` +Cache β†’ :mod:`meshcore_gui.services.cache` + + Author: PE1HVH + SPDX-License-Identifier: MIT +""" + +import abc +import asyncio +import threading +import time +from typing import Dict, List, Optional, Set + +from meshcore import MeshCore, EventType + +import meshcore_gui.config as _config +from meshcore_gui.config import ( + DEFAULT_TIMEOUT, + CHANNEL_CACHE_ENABLED, + CONTACT_REFRESH_SECONDS, + MAX_CHANNELS, + RECONNECT_BASE_DELAY, + RECONNECT_MAX_RETRIES, + debug_data, + debug_print, + pp, +) +from meshcore_gui.core.protocols import SharedDataWriter +from meshcore_gui.ble.commands import CommandHandler +from meshcore_gui.ble.events import EventHandler +from meshcore_gui.ble.packet_decoder import PacketDecoder +from meshcore_gui.services.bot import BotConfig, MeshBot +from meshcore_gui.services.cache import DeviceCache +from meshcore_gui.services.dedup import DualDeduplicator +from meshcore_gui.services.device_identity import write_device_identity + + +# Seconds between background retry attempts for missing channel keys. +KEY_RETRY_INTERVAL: float = 30.0 + +# Seconds between periodic cleanup of old archived data (24 hours). +CLEANUP_INTERVAL: float = 86400.0 + + +# ====================================================================== +# Factory +# ====================================================================== + +def create_worker(device_id: str, shared: SharedDataWriter, **kwargs): + """Return the appropriate worker for *device_id*. + + Keyword arguments are forwarded to the worker constructor + (e.g. ``baudrate``, ``cx_dly`` for serial). + """ + from meshcore_gui.config import is_ble_address + + if is_ble_address(device_id): + return BLEWorker(device_id, shared) + return SerialWorker( + device_id, + shared, + baudrate=kwargs.get("baudrate", _config.SERIAL_BAUDRATE), + cx_dly=kwargs.get("cx_dly", _config.SERIAL_CX_DELAY), + ) + + +# ====================================================================== +# Base worker (shared by BLE and Serial) +# ====================================================================== + +class _BaseWorker(abc.ABC): + """Abstract base for transport-specific workers. + + Subclasses must implement: + + - :pyattr:`_log_prefix` β€” ``"BLE"`` or ``"SERIAL"`` + - :meth:`_async_main` β€” transport-specific startup + main loop + - :meth:`_connect` β€” create the :class:`MeshCore` connection + - :meth:`_reconnect` β€” re-establish after a disconnect + - :pyattr:`_disconnect_keywords` β€” error substrings that signal + a broken connection + """ + + def __init__(self, device_id: str, shared: SharedDataWriter) -> None: + self.device_id = device_id + self.shared = shared + self.mc: Optional[MeshCore] = None + self.running = True + self._disconnected = False + + # Local cache (one file per device) + self._cache = DeviceCache(device_id) + + # Collaborators (created eagerly, wired after connection) + self._decoder = PacketDecoder() + self._dedup = DualDeduplicator(max_size=200) + self._bot = MeshBot( + config=BotConfig(), + command_sink=shared.put_command, + enabled_check=shared.is_bot_enabled, + ) + + # Channel indices that still need keys from device + self._pending_keys: Set[int] = set() + + # Dynamically discovered channels from device + self._channels: List[Dict] = [] + + # ── abstract properties / methods ───────────────────────────── + + @property + @abc.abstractmethod + def _log_prefix(self) -> str: + """Short label for log messages, e.g. ``"BLE"`` or ``"SERIAL"``.""" + + @property + @abc.abstractmethod + def _disconnect_keywords(self) -> tuple: + """Lowercase substrings that indicate a transport disconnect.""" + + @abc.abstractmethod + async def _async_main(self) -> None: + """Transport-specific startup + main loop.""" + + @abc.abstractmethod + async def _connect(self) -> None: + """Create a fresh connection and wire collaborators.""" + + @abc.abstractmethod + async def _reconnect(self) -> Optional[MeshCore]: + """Attempt to re-establish the connection after a disconnect.""" + + # ── thread lifecycle ────────────────────────────────────────── + + def start(self) -> None: + """Start the worker in a new daemon thread.""" + thread = threading.Thread(target=self._run, daemon=True) + thread.start() + debug_print(f"{self._log_prefix} worker thread started") + + def _run(self) -> None: + asyncio.run(self._async_main()) + + # ── shared main loop (called from subclass _async_main) ─────── + + async def _main_loop(self) -> None: + """Command processing + periodic tasks. + + Runs until ``self.running`` is cleared or a disconnect is + detected. Subclasses call this from their ``_async_main``. + """ + last_contact_refresh = time.time() + last_key_retry = time.time() + last_cleanup = time.time() + + while self.running and not self._disconnected: + try: + await self._cmd_handler.process_all() + except Exception as e: + error_str = str(e).lower() + if any(kw in error_str for kw in self._disconnect_keywords): + print(f"{self._log_prefix}: ⚠️ Connection error detected: {e}") + self._disconnected = True + break + debug_print(f"Command processing error: {e}") + + now = time.time() + + if now - last_contact_refresh > CONTACT_REFRESH_SECONDS: + await self._refresh_contacts() + last_contact_refresh = now + + if self._pending_keys and now - last_key_retry > KEY_RETRY_INTERVAL: + await self._retry_missing_keys() + last_key_retry = now + + if now - last_cleanup > CLEANUP_INTERVAL: + await self._cleanup_old_data() + last_cleanup = now + + await asyncio.sleep(0.1) + + async def _handle_reconnect(self) -> bool: + """Shared reconnect logic after a disconnect. + + Returns True if reconnection succeeded, False otherwise. + """ + self.shared.set_connected(False) + self.shared.set_status("πŸ”„ Verbinding verloren β€” herverbinden...") + print(f"{self._log_prefix}: Verbinding verloren, start reconnect...") + self.mc = None + + new_mc = await self._reconnect() + + if new_mc: + self.mc = new_mc + await asyncio.sleep(1) + self._wire_collaborators() + await self._load_data() + await self.mc.start_auto_message_fetching() + self._seed_dedup_from_messages() + self.shared.set_connected(True) + self.shared.set_status("βœ… Herverbonden") + print(f"{self._log_prefix}: βœ… Herverbonden en operationeel") + return True + + self.shared.set_status("❌ Herverbinding mislukt β€” herstart nodig") + print( + f"{self._log_prefix}: ❌ Kan niet herverbinden β€” " + "wacht 60s en probeer opnieuw..." + ) + return False + + # ── collaborator wiring ─────────────────────────────────────── + + def _wire_collaborators(self) -> None: + """(Re-)create handlers and subscribe to MeshCore events.""" + self._evt_handler = EventHandler( + shared=self.shared, + decoder=self._decoder, + dedup=self._dedup, + bot=self._bot, + ) + self._cmd_handler = CommandHandler( + mc=self.mc, shared=self.shared, cache=self._cache, + ) + self._cmd_handler.set_load_data_callback(self._load_data) + + self.mc.subscribe(EventType.CHANNEL_MSG_RECV, self._evt_handler.on_channel_msg) + self.mc.subscribe(EventType.CONTACT_MSG_RECV, self._evt_handler.on_contact_msg) + self.mc.subscribe(EventType.RX_LOG_DATA, self._evt_handler.on_rx_log) + self.mc.subscribe(EventType.LOGIN_SUCCESS, self._on_login_success) + + # ── LOGIN_SUCCESS handler (Room Server) ─────────────────────── + + def _on_login_success(self, event) -> None: + payload = event.payload or {} + pubkey = payload.get("pubkey_prefix", "") + is_admin = payload.get("is_admin", False) + debug_print(f"LOGIN_SUCCESS received: pubkey={pubkey}, admin={is_admin}") + self.shared.set_status("βœ… Room login OK β€” messages arriving over RF…") + + # ── apply cache ─────────────────────────────────────────────── + + def _apply_cache(self) -> None: + """Push cached data to SharedData so GUI renders immediately.""" + device = self._cache.get_device() + if device: + self.shared.update_from_appstart(device) + fw = device.get("firmware_version") or device.get("ver") + if fw: + self.shared.update_from_device_query({"ver": fw}) + self.shared.set_status("πŸ“¦ Loaded from cache") + debug_print(f"Cache β†’ device info: {device.get('name', '?')}") + + if CHANNEL_CACHE_ENABLED: + channels = self._cache.get_channels() + if channels: + self._channels = channels + self.shared.set_channels(channels) + debug_print(f"Cache β†’ channels: {[c['name'] for c in channels]}") + else: + debug_print("Channel cache disabled β€” skipping cached channels") + + contacts = self._cache.get_contacts() + if contacts: + self.shared.set_contacts(contacts) + debug_print(f"Cache β†’ contacts: {len(contacts)}") + + cached_keys = self._cache.get_channel_keys() + for idx_str, secret_hex in cached_keys.items(): + try: + idx = int(idx_str) + secret_bytes = bytes.fromhex(secret_hex) + if len(secret_bytes) >= 16: + self._decoder.add_channel_key(idx, secret_bytes[:16], source="cache") + debug_print(f"Cache β†’ channel key [{idx}]") + except (ValueError, TypeError) as exc: + debug_print(f"Cache β†’ bad channel key [{idx_str}]: {exc}") + + cached_orig_name = self._cache.get_original_device_name() + if cached_orig_name: + self.shared.set_original_device_name(cached_orig_name) + debug_print(f"Cache β†’ original device name: {cached_orig_name}") + + count = self.shared.load_recent_from_archive(limit=100) + if count: + debug_print(f"Cache β†’ {count} recent messages from archive") + + self._seed_dedup_from_messages() + + # ── initial data loading ────────────────────────────────────── + + async def _export_device_identity(self) -> None: + """Export device keys and write identity file for Observer. + + Calls ``export_private_key()`` on the device and writes the + result to ``~/.meshcore-gui/device_identity.json`` so the + MeshCore Observer can authenticate to the MQTT broker without + manual key configuration. + """ + pfx = self._log_prefix + try: + r = await self.mc.commands.export_private_key() + if r is None: + debug_print(f"{pfx}: export_private_key returned None") + return + + if r.type == EventType.PRIVATE_KEY: + prv_bytes = r.payload.get("private_key", b"") + if len(prv_bytes) == 64: + # Gather device info for the identity file + pub_key = "" + dev_name = "" + fw_ver = "" + with self.shared.lock: + pub_key = self.shared.device.public_key + dev_name = self.shared.device.name + fw_ver = self.shared.device.firmware_version + + write_device_identity( + public_key=pub_key, + private_key_bytes=prv_bytes, + device_name=dev_name, + firmware_version=fw_ver, + source_device=self.device_id, + ) + else: + debug_print( + f"{pfx}: export_private_key: unexpected " + f"length {len(prv_bytes)} bytes" + ) + + elif r.type == EventType.DISABLED: + print( + f"{pfx}: ℹ️ Private key export is disabled on device " + f"β€” manual key setup required for Observer MQTT" + ) + else: + debug_print( + f"{pfx}: export_private_key: unexpected " + f"response type {r.type}" + ) + + except Exception as exc: + debug_print(f"{pfx}: export_private_key failed: {exc}") + + async def _load_data(self) -> None: + """Load device info, channels and contacts from device.""" + pfx = self._log_prefix + + # send_appstart β€” reuse result from MeshCore.connect() + self.shared.set_status("πŸ”„ Device info...") + cached_info = self.mc.self_info + if cached_info and cached_info.get("name"): + print(f"{pfx}: send_appstart OK (from connect): {cached_info.get('name')}") + self.shared.update_from_appstart(cached_info) + self._cache.set_device(cached_info) + else: + debug_print("self_info empty after connect(), falling back to manual send_appstart") + appstart_ok = False + for i in range(3): + debug_print(f"send_appstart fallback attempt {i + 1}/3") + try: + r = await self.mc.commands.send_appstart() + if r is None: + debug_print(f"send_appstart fallback {i + 1}: received None, retrying") + await asyncio.sleep(2.0) + continue + if r.type != EventType.ERROR: + print(f"{pfx}: send_appstart OK: {r.payload.get('name')} (fallback attempt {i + 1})") + self.shared.update_from_appstart(r.payload) + self._cache.set_device(r.payload) + appstart_ok = True + break + else: + debug_print(f"send_appstart fallback {i + 1}: ERROR β€” payload={pp(r.payload)}") + except Exception as exc: + debug_print(f"send_appstart fallback {i + 1} exception: {exc}") + await asyncio.sleep(2.0) + if not appstart_ok: + print(f"{pfx}: ⚠️ send_appstart failed after 3 fallback attempts") + + # send_device_query + for i in range(5): + debug_print(f"send_device_query attempt {i + 1}/5") + try: + r = await self.mc.commands.send_device_query() + if r is None: + debug_print(f"send_device_query attempt {i + 1}: received None response, retrying") + await asyncio.sleep(2.0) + continue + if r.type != EventType.ERROR: + fw = r.payload.get("ver", "") + print(f"{pfx}: send_device_query OK: {fw} (attempt {i + 1})") + self.shared.update_from_device_query(r.payload) + if fw: + self._cache.set_firmware_version(fw) + break + else: + debug_print(f"send_device_query attempt {i + 1}: ERROR response β€” payload={pp(r.payload)}") + except Exception as exc: + debug_print(f"send_device_query attempt {i + 1} exception: {exc}") + await asyncio.sleep(2.0) + + # Export device identity for MeshCore Observer + await self._export_device_identity() + + # Channels + await self._discover_channels() + + # Contacts + self.shared.set_status("πŸ”„ Contacts...") + debug_print("get_contacts starting") + try: + r = await self._get_contacts_with_timeout() + debug_print(f"get_contacts result: type={r.type if r else None}") + if r and r.payload: + try: + payload_len = len(r.payload) + except Exception: + payload_len = None + if payload_len is not None and payload_len > 10: + debug_print(f"get_contacts payload size={payload_len} (omitted)") + else: + debug_data("get_contacts payload", r.payload) + if r is None: + debug_print(f"{pfx}: get_contacts returned None, keeping cached contacts") + elif r.type != EventType.ERROR: + merged = self._cache.merge_contacts(r.payload) + self.shared.set_contacts(merged) + print(f"{pfx}: Contacts β€” {len(r.payload)} from device, {len(merged)} total (with cache)") + else: + debug_print(f"{pfx}: get_contacts failed β€” payload={pp(r.payload)}, keeping cached contacts") + except Exception as exc: + debug_print(f"{pfx}: get_contacts exception: {exc}") + + async def _get_contacts_with_timeout(self): + """Fetch contacts with a bounded timeout to avoid hanging refresh.""" + timeout = max(DEFAULT_TIMEOUT * 2, 10.0) + try: + return await asyncio.wait_for( + self.mc.commands.get_contacts(), timeout=timeout, + ) + except asyncio.TimeoutError: + self.shared.set_status("⚠️ Contacts timeout β€” using cached contacts") + debug_print(f"get_contacts timeout after {timeout:.0f}s") + return None + + # ── channel discovery ───────────────────────────────────────── + + async def _discover_channels(self) -> None: + """Discover channels and load their keys from the device.""" + pfx = self._log_prefix + self.shared.set_status("πŸ”„ Discovering channels...") + discovered: List[Dict] = [] + cached_keys = self._cache.get_channel_keys() + + confirmed: list[str] = [] + from_cache: list[str] = [] + derived: list[str] = [] + + consecutive_errors = 0 + + for idx in range(MAX_CHANNELS): + payload = await self._try_get_channel_info(idx, max_attempts=2, delay=1.0) + + if payload is None: + consecutive_errors += 1 + if consecutive_errors >= 3: + debug_print( + f"Channel discovery: {consecutive_errors} consecutive " + f"empty slots at idx {idx}, stopping" + ) + break + continue + + consecutive_errors = 0 + name = payload.get("name") or payload.get("channel_name") or "" + if not name.strip(): + debug_print(f"Channel [{idx}]: response OK but no name β€” skipping (undefined slot)") + continue + + discovered.append({"idx": idx, "name": name}) + + secret = payload.get("channel_secret") + secret_bytes = self._extract_secret(secret) + + if secret_bytes: + self._decoder.add_channel_key(idx, secret_bytes, source="device") + self._cache.set_channel_key(idx, secret_bytes.hex()) + self._pending_keys.discard(idx) + confirmed.append(f"[{idx}] {name}") + elif str(idx) in cached_keys: + from_cache.append(f"[{idx}] {name}") + print(f"{pfx}: πŸ“¦ Channel [{idx}] '{name}' β€” using cached key") + else: + self._decoder.add_channel_key_from_name(idx, name) + self._pending_keys.add(idx) + derived.append(f"[{idx}] {name}") + print(f"{pfx}: ⚠️ Channel [{idx}] '{name}' β€” name-derived key (will retry)") + + await asyncio.sleep(0.3) + + if not discovered: + discovered = [{"idx": 0, "name": "Public"}] + print(f"{pfx}: ⚠️ No channels discovered, using default Public channel") + + self._channels = discovered + self.shared.set_channels(discovered) + if CHANNEL_CACHE_ENABLED: + self._cache.set_channels(discovered) + debug_print("Channel list cached to disk") + + print(f"{pfx}: Channels discovered: {[c['name'] for c in discovered]}") + print(f"{pfx}: PacketDecoder ready β€” has_keys={self._decoder.has_keys}") + if confirmed: + print(f"{pfx}: βœ… Keys from device: {', '.join(confirmed)}") + if from_cache: + print(f"{pfx}: πŸ“¦ Keys from cache: {', '.join(from_cache)}") + if derived: + print(f"{pfx}: ⚠️ Name-derived keys: {', '.join(derived)}") + + async def _try_get_channel_info( + self, idx: int, max_attempts: int, delay: float, + ) -> Optional[Dict]: + for attempt in range(max_attempts): + try: + r = await self.mc.commands.get_channel(idx) + if r is None: + debug_print(f"get_channel({idx}) attempt {attempt + 1}/{max_attempts}: received None response, retrying") + await asyncio.sleep(delay) + continue + if r.type == EventType.ERROR: + debug_print(f"get_channel({idx}) attempt {attempt + 1}/{max_attempts}: ERROR response β€” payload={pp(r.payload)}") + await asyncio.sleep(delay) + continue + debug_print(f"get_channel({idx}) attempt {attempt + 1}/{max_attempts}: OK β€” keys={list(r.payload.keys())}") + return r.payload + except Exception as exc: + debug_print(f"get_channel({idx}) attempt {attempt + 1}/{max_attempts} error: {exc}") + await asyncio.sleep(delay) + return None + + async def _try_load_channel_key( + self, idx: int, name: str, max_attempts: int, delay: float, + ) -> bool: + payload = await self._try_get_channel_info(idx, max_attempts, delay) + if payload is None: + return False + secret = payload.get("channel_secret") + secret_bytes = self._extract_secret(secret) + if secret_bytes: + self._decoder.add_channel_key(idx, secret_bytes, source="device") + self._cache.set_channel_key(idx, secret_bytes.hex()) + print(f"{self._log_prefix}: βœ… Channel [{idx}] '{name}' β€” key from device (background retry)") + self._pending_keys.discard(idx) + return True + debug_print(f"get_channel({idx}): response OK but secret unusable") + return False + + async def _retry_missing_keys(self) -> None: + if not self._pending_keys: + return + pending_copy = set(self._pending_keys) + ch_map = {ch["idx"]: ch["name"] for ch in self._channels} + debug_print(f"Background key retry: trying {len(pending_copy)} channels") + for idx in pending_copy: + name = ch_map.get(idx, f"ch{idx}") + loaded = await self._try_load_channel_key(idx, name, max_attempts=1, delay=0.5) + if loaded: + self._pending_keys.discard(idx) + await asyncio.sleep(1.0) + if not self._pending_keys: + print(f"{self._log_prefix}: βœ… All channel keys now loaded!") + else: + remaining = [f"[{idx}] {ch_map.get(idx, '?')}" for idx in sorted(self._pending_keys)] + debug_print(f"Background retry: still pending: {', '.join(remaining)}") + + # ── helpers ──────────────────────────────────────────────────── + + def _seed_dedup_from_messages(self) -> None: + """Seed the deduplicator with messages already in SharedData.""" + snapshot = self.shared.get_snapshot() + messages = snapshot.get("messages", []) + seeded = 0 + for msg in messages: + if msg.message_hash: + self._dedup.mark_hash(msg.message_hash) + seeded += 1 + if msg.sender and msg.text: + self._dedup.mark_content(msg.sender, msg.channel, msg.text) + seeded += 1 + debug_print(f"Dedup seeded with {seeded} entries from {len(messages)} messages") + + @staticmethod + def _extract_secret(secret) -> Optional[bytes]: + if secret and isinstance(secret, bytes) and len(secret) >= 16: + return secret[:16] + if secret and isinstance(secret, str) and len(secret) >= 32: + try: + raw = bytes.fromhex(secret) + if len(raw) >= 16: + return raw[:16] + except ValueError: + pass + return None + + # ── periodic tasks ──────────────────────────────────────────── + + async def _refresh_contacts(self) -> None: + try: + r = await self._get_contacts_with_timeout() + if r is None: + debug_print("Periodic refresh: get_contacts returned None, skipping") + return + if r.type != EventType.ERROR: + merged = self._cache.merge_contacts(r.payload) + self.shared.set_contacts(merged) + debug_print( + f"Periodic refresh: {len(r.payload)} from device, " + f"{len(merged)} total" + ) + except Exception as exc: + debug_print(f"Periodic contact refresh failed: {exc}") + + async def _cleanup_old_data(self) -> None: + try: + if self.shared.archive: + self.shared.archive.cleanup_old_data() + stats = self.shared.archive.get_stats() + debug_print( + f"Cleanup: archive now has {stats['total_messages']} messages, " + f"{stats['total_rxlog']} rxlog entries" + ) + removed = self._cache.prune_old_contacts() + if removed > 0: + contacts = self._cache.get_contacts() + self.shared.set_contacts(contacts) + debug_print(f"Cleanup: pruned {removed} old contacts") + except Exception as exc: + debug_print(f"Periodic cleanup failed: {exc}") + + +# ====================================================================== +# Serial worker +# ====================================================================== + +class SerialWorker(_BaseWorker): + """Serial communication worker (USB/UART). + + Args: + port: Serial device path (e.g. ``"/dev/ttyUSB0"``). + shared: SharedDataWriter for thread-safe communication. + baudrate: Serial baudrate (default from config). + cx_dly: Connection delay for meshcore serial transport. + """ + + def __init__( + self, + port: str, + shared: SharedDataWriter, + baudrate: int = _config.SERIAL_BAUDRATE, + cx_dly: float = _config.SERIAL_CX_DELAY, + ) -> None: + super().__init__(port, shared) + self.port = port + self.baudrate = baudrate + self.cx_dly = cx_dly + + @property + def _log_prefix(self) -> str: + return "SERIAL" + + @property + def _disconnect_keywords(self) -> tuple: + return ( + "not connected", "disconnected", "connection reset", + "broken pipe", "i/o error", "read failed", "write failed", + "port is closed", "port closed", + ) + + async def _async_main(self) -> None: + try: + while self.running: + # ── Outer loop: (re)establish a fresh serial connection ── + self._disconnected = False + await self._connect() + + if not self.mc: + print("SERIAL: Initial connection failed, retrying in 30s...") + self.shared.set_status("⚠️ Connection failed β€” retrying...") + await asyncio.sleep(30) + continue + + # ── Inner loop: run + reconnect without calling _connect() again ── + # _handle_reconnect() already creates a fresh MeshCore and loads + # data β€” calling _connect() on top of that would attempt to open + # the serial port a second time, causing an immediate disconnect. + while self.running: + await self._main_loop() + + if not self._disconnected or not self.running: + break + + ok = await self._handle_reconnect() + if ok: + # Reconnected β€” reset flag and go back to _main_loop, + # NOT to the outer while (which would call _connect() again). + self._disconnected = False + else: + # All reconnect attempts exhausted β€” wait, then let the + # outer loop call _connect() for a clean fresh start. + await asyncio.sleep(60) + break + finally: + return + + async def _connect(self) -> None: + if self._cache.load(): + self._apply_cache() + print("SERIAL: Cache loaded β€” GUI populated from disk") + else: + print("SERIAL: No cache found β€” waiting for device data") + + self.shared.set_status(f"πŸ”„ Connecting to {self.port}...") + try: + print(f"SERIAL: Connecting to {self.port}...") + self.mc = await MeshCore.create_serial( + self.port, + baudrate=self.baudrate, + auto_reconnect=False, + default_timeout=DEFAULT_TIMEOUT, + debug=_config.MESHCORE_LIB_DEBUG, + cx_dly=self.cx_dly, + ) + if self.mc is None: + raise RuntimeError("No response from device over serial") + print("SERIAL: Connected!") + + await asyncio.sleep(1) + debug_print("Post-connection sleep done, wiring collaborators") + self._wire_collaborators() + await self._load_data() + await self.mc.start_auto_message_fetching() + + self.shared.set_connected(True) + self.shared.set_status("βœ… Connected") + print("SERIAL: Ready!") + + if self._pending_keys: + pending_names = [ + f"[{ch['idx']}] {ch['name']}" + for ch in self._channels + if ch["idx"] in self._pending_keys + ] + print( + f"SERIAL: ⏳ Background retry active for: " + f"{', '.join(pending_names)} (every {KEY_RETRY_INTERVAL:.0f}s)" + ) + + except Exception as e: + print(f"SERIAL: Connection error: {e}") + self.mc = None # ensure _async_main sees connection as failed + if self._cache.has_cache: + self.shared.set_status(f"⚠️ Offline β€” using cached data ({e})") + else: + self.shared.set_status(f"❌ {e}") + + async def _reconnect(self) -> Optional[MeshCore]: + for attempt in range(1, RECONNECT_MAX_RETRIES + 1): + delay = RECONNECT_BASE_DELAY * attempt + print( + f"SERIAL: πŸ”„ Reconnect attempt {attempt}/{RECONNECT_MAX_RETRIES} " + f"in {delay:.0f}s..." + ) + await asyncio.sleep(delay) + try: + mc = await MeshCore.create_serial( + self.port, + baudrate=self.baudrate, + auto_reconnect=False, + default_timeout=DEFAULT_TIMEOUT, + debug=_config.MESHCORE_LIB_DEBUG, + cx_dly=self.cx_dly, + ) + if mc is None: + raise RuntimeError("No response from device over serial") + return mc + except Exception as exc: + print(f"SERIAL: ❌ Reconnect attempt {attempt} failed: {exc}") + print(f"SERIAL: ❌ Reconnect failed after {RECONNECT_MAX_RETRIES} attempts") + return None + + +# ====================================================================== +# BLE worker +# ====================================================================== + +class BLEWorker(_BaseWorker): + """BLE communication worker (Bluetooth Low Energy). + + Args: + address: BLE MAC address (e.g. ``"literal:AA:BB:CC:DD:EE:FF"``). + shared: SharedDataWriter for thread-safe communication. + """ + + def __init__(self, address: str, shared: SharedDataWriter) -> None: + super().__init__(address, shared) + self.address = address + + # BLE PIN agent β€” imported lazily so serial-only installs + # don't need dbus_fast / bleak. + from meshcore_gui.ble.ble_agent import BleAgentManager + self._agent = BleAgentManager(pin=_config.BLE_PIN) + + @property + def _log_prefix(self) -> str: + return "BLE" + + @property + def _disconnect_keywords(self) -> tuple: + return ( + "not connected", "disconnected", "dbus", + "pin or key missing", "connection reset", "broken pipe", + "failed to discover", "service discovery", + ) + + async def _async_main(self) -> None: + from meshcore_gui.ble.ble_reconnect import remove_bond + + # Step 1: Start PIN agent BEFORE any BLE connection + await self._agent.start() + + # Step 2: Remove stale bond (clean slate) + await remove_bond(self.address) + await asyncio.sleep(1) + + # Step 3: Connect + main loop + try: + while self.running: + # ── Outer loop: (re)establish a fresh BLE connection ── + self._disconnected = False + await self._connect() + + if not self.mc: + print("BLE: Initial connection failed, retrying in 30s...") + self.shared.set_status("⚠️ Connection failed β€” retrying...") + await asyncio.sleep(30) + await remove_bond(self.address) + await asyncio.sleep(1) + continue + + # ── Inner loop: run + reconnect without calling _connect() again ── + # _handle_reconnect() already creates a fresh MeshCore and loads + # data β€” calling _connect() on top would open a second BLE session, + # causing an immediate disconnect. + while self.running: + await self._main_loop() + + if not self._disconnected or not self.running: + break + + ok = await self._handle_reconnect() + if ok: + # Reconnected β€” reset flag and go back to _main_loop, + # NOT to the outer while (which would call _connect() again). + self._disconnected = False + else: + await asyncio.sleep(60) + await remove_bond(self.address) + await asyncio.sleep(1) + break + finally: + await self._agent.stop() + + async def _connect(self) -> None: + if self._cache.load(): + self._apply_cache() + print("BLE: Cache loaded β€” GUI populated from disk") + else: + print("BLE: No cache found β€” waiting for BLE data") + + self.shared.set_status(f"πŸ”„ Connecting to {self.address}...") + try: + print(f"BLE: Connecting to {self.address}...") + self.mc = await MeshCore.create_ble( + self.address, + auto_reconnect=False, + default_timeout=DEFAULT_TIMEOUT, + debug=_config.MESHCORE_LIB_DEBUG, + ) + print("BLE: Connected!") + + await asyncio.sleep(1) + debug_print("Post-connection sleep done, wiring collaborators") + self._wire_collaborators() + await self._load_data() + await self.mc.start_auto_message_fetching() + + self.shared.set_connected(True) + self.shared.set_status("βœ… Connected") + print("BLE: Ready!") + + if self._pending_keys: + pending_names = [ + f"[{ch['idx']}] {ch['name']}" + for ch in self._channels + if ch["idx"] in self._pending_keys + ] + print( + f"BLE: ⏳ Background retry active for: " + f"{', '.join(pending_names)} (every {KEY_RETRY_INTERVAL:.0f}s)" + ) + + except Exception as e: + print(f"BLE: Connection error: {e}") + self.mc = None # ensure _async_main sees connection as failed + if self._cache.has_cache: + self.shared.set_status(f"⚠️ Offline β€” using cached data ({e})") + else: + self.shared.set_status(f"❌ {e}") + + async def _reconnect(self) -> Optional[MeshCore]: + from meshcore_gui.ble.ble_reconnect import reconnect_loop + + async def _create_fresh_connection() -> MeshCore: + return await MeshCore.create_ble( + self.address, + auto_reconnect=False, + default_timeout=DEFAULT_TIMEOUT, + debug=_config.MESHCORE_LIB_DEBUG, + ) + + return await reconnect_loop( + _create_fresh_connection, + self.address, + max_retries=RECONNECT_MAX_RETRIES, + base_delay=RECONNECT_BASE_DELAY, + ) diff --git a/meshcore_gui/config.py b/meshcore_gui/config.py index 7c1b8fa..cfe4d5b 100644 --- a/meshcore_gui/config.py +++ b/meshcore_gui/config.py @@ -25,7 +25,7 @@ from typing import Any, Dict, List # ============================================================================== -VERSION: str = "1.13.2" +VERSION: str = "1.13.4" # ============================================================================== @@ -293,7 +293,7 @@ CHANNEL_CACHE_ENABLED: bool = False # Fixed device name applied when the BOT checkbox is enabled. # The original device name is saved and restored when BOT is disabled. -BOT_DEVICE_NAME: str = "NL-OV-ZWL-STDSHGN-WKC Bot" +BOT_DEVICE_NAME: str = "ZwolsBotje" # Default device name used as fallback when restoring from BOT mode # and no original name was saved (e.g. after a restart). diff --git a/meshcore_gui/core/shared_data.py b/meshcore_gui/core/shared_data.py index 7e3d1ca..c69cbc3 100644 --- a/meshcore_gui/core/shared_data.py +++ b/meshcore_gui/core/shared_data.py @@ -593,14 +593,58 @@ class SharedData: return None def get_contact_name_by_prefix(self, pubkey_prefix: str) -> str: + """Resolve a pubkey/prefix to the best available display name. + + The room server may report the author using different key fields: + a short prefix, a full public key, or a value copied into another + payload field. To keep sender display stable, match against both + the contact dict key and common pubkey-like fields stored inside + each contact record. + """ if not pubkey_prefix: return "" + + probe = str(pubkey_prefix).strip().lower() + if not probe: + return "" + + def _candidate_keys(contact_key: str, contact: Dict) -> List[str]: + values = [contact_key] + for field in ( + 'public_key', + 'pubkey', + 'pub_key', + 'publicKey', + 'sender_pubkey', + 'author_pubkey', + 'receiver_pubkey', + 'pubkey_prefix', + 'signature', + ): + value = contact.get(field) + if isinstance(value, str) and value.strip(): + values.append(value.strip()) + return values + with self.lock: + device_key = (self.device.public_key or '').strip().lower() + if device_key and ( + device_key.startswith(probe) + or probe.startswith(device_key) + ): + return self.device.name or 'Me' + for key, contact in self.contacts.items(): - if key.lower().startswith(pubkey_prefix.lower()): - name = contact.get('adv_name', '') - if name: - return name + for candidate in _candidate_keys(key, contact): + candidate_lower = candidate.lower() + if ( + candidate_lower.startswith(probe) + or probe.startswith(candidate_lower) + ): + name = str(contact.get('adv_name', '') or '').strip() + if name: + return name + return pubkey_prefix[:8] def get_contact_by_name(self, name: str) -> Optional[Tuple[str, Dict]]: diff --git a/meshcore_gui/gui/dashboard.py b/meshcore_gui/gui/dashboard.py index 6f86e63..d7cef43 100644 --- a/meshcore_gui/gui/dashboard.py +++ b/meshcore_gui/gui/dashboard.py @@ -680,30 +680,12 @@ class DashboardPage: # Apply channel filter to messages panel if panel_id == 'messages' and self._messages: self._messages.set_active_channel(channel) - # Force immediate rebuild so the panel is populated the - # moment it becomes visible, instead of waiting for the - # next 500 ms timer tick (which caused the "empty on first - # click, populated on second click" symptom). - data = self._shared.get_snapshot() - self._messages.update( - data, - self._messages.channel_filters, - self._messages.last_channels, - room_pubkeys=( - self._room_server.get_room_pubkeys() - if self._room_server else None - ), - ) # Apply channel filter to archive panel if panel_id == 'archive' and self._archive_page: self._archive_page.set_channel_filter(channel) - # Force map recenter when opening map panel (Leaflet may be hidden on load) - if panel_id == 'map' and self._map: - data = self._shared.get_snapshot() - data['force_center'] = True - self._map.update(data) + self._refresh_active_panel_now(force_map_center=(panel_id == 'map')) # Update active menu highlight (standalone buttons only) for pid, btn in self._menu_buttons.items(): @@ -716,6 +698,44 @@ class DashboardPage: if self._drawer: self._drawer.hide() + def _refresh_active_panel_now(self, force_map_center: bool = False) -> None: + """Refresh only the currently visible panel. + + This is used directly after a panel switch so the user does not + need to wait for the next 500 ms dashboard tick. + """ + data = self._shared.get_snapshot() + + if data.get('channels'): + self._messages.update_filters(data) + self._messages.update_channel_options(data['channels']) + self._update_submenus(data) + + if self._active_panel == 'device': + self._device.update(data) + elif self._active_panel == 'map': + if force_map_center: + data['force_center'] = True + self._map.update(data) + elif self._active_panel == 'actions': + self._actions.update(data) + elif self._active_panel == 'contacts': + self._contacts.update(data) + elif self._active_panel == 'messages': + self._messages.update( + data, + self._messages.channel_filters, + self._messages.last_channels, + room_pubkeys=( + self._room_server.get_room_pubkeys() + if self._room_server else None + ), + ) + elif self._active_panel == 'rooms': + self._room_server.update(data) + elif self._active_panel == 'rxlog': + self._rxlog.update(data) + # ------------------------------------------------------------------ # Room Server callback (from ContactsPanel) # ------------------------------------------------------------------ @@ -753,56 +773,49 @@ class DashboardPage: # Always update status self._status_label.text = data['status'] - # Device info - if data['device_updated'] or is_first: - self._device.update(data) - - # Map: always send a snapshot while the panel is active. - # The JS runtime coalesces pending payloads β€” only the newest - # is ever applied β€” so calling update() on every tick is cheap. - # This ensures the Leaflet runtime always gets at least one - # valid snapshot after it finishes loading, regardless of - # whether device_updated or is_first happened to be True - # on the tick that fired before MeshCoreLeafletBoot was defined. - if self._active_panel == 'map': - self._map.update(data) - - # Channel-dependent UI: always ensure consistency when - # channels exist. Because a single DashboardPage instance - # is shared across browser sessions (render() is called on - # each new connection), the old session's timer can steal - # the is_first flag before the new timer fires. Running - # these unconditionally is safe because each method has an - # internal fingerprint/equality check that prevents - # unnecessary DOM updates. + # Channel-dependent drawer/submenu state may stay global. + # The helpers below already contain equality checks, so this + # remains cheap while keeping navigation consistent. if data['channels']: self._messages.update_filters(data) self._messages.update_channel_options(data['channels']) self._update_submenus(data) - # BOT checkbox state (only on actual change or first render - # to avoid overwriting user interaction mid-toggle) - if data['channels_updated'] or is_first: - self._actions.update(data) + if self._active_panel == 'device': + if data['device_updated'] or is_first: + self._device.update(data) - # Contacts - if data['contacts_updated'] or is_first: - self._contacts.update(data) + elif self._active_panel == 'map': + # Keep sending snapshots while the map panel is active. + # The browser runtime coalesces pending payloads, so only + # the newest snapshot is applied. + self._map.update(data) - # Messages (always β€” for live filter changes) - self._messages.update( - data, - self._messages.channel_filters, - self._messages.last_channels, - room_pubkeys=self._room_server.get_room_pubkeys() if self._room_server else None, - ) + elif self._active_panel == 'actions': + if data['channels_updated'] or is_first: + self._actions.update(data) - # Room Server panels (always β€” for live messages + contact changes) - self._room_server.update(data) + elif self._active_panel == 'contacts': + if data['contacts_updated'] or is_first: + self._contacts.update(data) - # RX Log - if data['rxlog_updated']: - self._rxlog.update(data) + elif self._active_panel == 'messages': + self._messages.update( + data, + self._messages.channel_filters, + self._messages.last_channels, + room_pubkeys=( + self._room_server.get_room_pubkeys() + if self._room_server else None + ), + ) + + elif self._active_panel == 'rooms': + self._room_server.update(data) + + elif self._active_panel == 'rxlog': + if data['rxlog_updated'] or is_first: + self._rxlog.update(data) # Signal worker that GUI is ready for data if is_first and data['channels'] and data['contacts']: diff --git a/meshcore_gui/gui/panels/map_panel.py b/meshcore_gui/gui/panels/map_panel.py index 33a89f5..a3526dc 100644 --- a/meshcore_gui/gui/panels/map_panel.py +++ b/meshcore_gui/gui/panels/map_panel.py @@ -45,10 +45,9 @@ class MapPanel: on_change=lambda e: self._set_map_theme_mode(e.value), ).props('dense') ui.button('Center on Device', on_click=self._center_on_device) - ui.html( - f'
' - ).classes('w-full h-72') - self._dispatch_to_browser(snapshot={'__command__': 'ensure_map'}) + ui.element('div').props(f'id={self._container_id}').classes( + 'meshcore-leaflet-host w-full h-72' + ) self._apply_theme_only() def set_ui_dark_mode(self, value: bool | None) -> None: @@ -189,10 +188,10 @@ class MapPanel: 'meshcore-leaflet-vendor-js', 'https://unpkg.com/leaflet@1.9.4/dist/leaflet.js', function () { + ensurePanelRuntime(); ensureScript( 'meshcore-leaflet-markercluster-js', - 'https://unpkg.com/leaflet.markercluster@1.5.3/dist/leaflet.markercluster.js', - ensurePanelRuntime + 'https://unpkg.com/leaflet.markercluster@1.5.3/dist/leaflet.markercluster.js' ); } ); diff --git a/meshcore_gui/gui/panels/messages_panel.py b/meshcore_gui/gui/panels/messages_panel.py index 00fc9ee..a3f26e4 100644 --- a/meshcore_gui/gui/panels/messages_panel.py +++ b/meshcore_gui/gui/panels/messages_panel.py @@ -1,6 +1,6 @@ """Messages panel β€” filtered message display with channel selection and message input.""" -from typing import Callable, Dict, List, Set +from typing import Callable, Dict, Iterable, List, Set from nicegui import ui @@ -153,12 +153,30 @@ class MessagesPanel: # -- Message display ----------------------------------------------- + @staticmethod + def _merge_room_pubkeys( + ui_room_pubkeys: Set[str] | None, + known_room_pubkeys: Iterable[str] | None, + ) -> Set[str]: + """Merge UI-tracked and centrally known Room Server keys. + + The RoomServerPanel may not yet be fully restored when archived + messages are first shown. The SharedData registry provides a + second, UI-independent source of truth for room key prefixes. + """ + merged: Set[str] = set() + if ui_room_pubkeys: + merged.update(pk for pk in ui_room_pubkeys if pk) + if known_room_pubkeys: + merged.update(pk for pk in known_room_pubkeys if pk) + return merged + @staticmethod def _is_room_message(msg: Message, room_pubkeys: Set[str]) -> bool: """Return True if *msg* belongs to a Room Server. Matches when the message's ``sender_pubkey`` prefix-matches - any tracked room pubkey (same logic as RoomServerPanel). + any tracked or centrally known room pubkey. """ if not msg.sender_pubkey or not room_pubkeys: return False @@ -195,7 +213,10 @@ class MessagesPanel: if not self._container: return - room_pks = room_pubkeys or set() + room_pks = self._merge_room_pubkeys( + room_pubkeys, + data.get('known_room_pubkeys'), + ) channel_names = {ch['idx']: ch['name'] for ch in last_channels} contacts = data.get('contacts', {}) messages: List[Message] = data['messages'] diff --git a/meshcore_gui/gui/panels/room_server_panel.py b/meshcore_gui/gui/panels/room_server_panel.py index afd7e35..ab326a7 100644 --- a/meshcore_gui/gui/panels/room_server_panel.py +++ b/meshcore_gui/gui/panels/room_server_panel.py @@ -116,10 +116,12 @@ class RoomServerPanel: room_messages: Dict = data.get('room_messages', {}) # Live messages from current session's rolling buffer live_messages: List[Message] = data.get('messages', []) + # Contact dict for live sender-name resolution + contacts: Dict = data.get('contacts', {}) for pubkey, card_state in self._room_cards.items(): self._update_room_messages( - pubkey, card_state, room_messages, live_messages, + pubkey, card_state, room_messages, live_messages, contacts, ) # ------------------------------------------------------------------ @@ -389,6 +391,41 @@ class RoomServerPanel: if card_state and card_state.get('card'): self._container.remove(card_state['card']) + # ------------------------------------------------------------------ + # Internal β€” sender name resolution + # ------------------------------------------------------------------ + + @staticmethod + def _resolve_sender_name(sender: str, contacts: Dict) -> str: + """Resolve a sender field to a display name when possible. + + When ``msg.sender`` was stored as a raw hex prefix (because the + contact was not yet known at archive time), this method attempts + a live lookup against the current contacts snapshot so the UI + always shows a human-readable name instead of a hex code. + + Args: + sender: Value from ``Message.sender`` β€” may be a name or a hex string. + contacts: Current contacts snapshot from ``SharedData.get_snapshot()``. + + Returns: + Resolved display name, or the original sender value if no + match is found, or ``'?'`` when sender is empty. + """ + if not sender: + return '?' + probe = sender.strip().lower() + # Only resolve when the field looks like a hex identifier (6–64 hex chars) + if not (6 <= len(probe) <= 64 and all(ch in '0123456789abcdef' for ch in probe)): + return sender + for key, contact in contacts.items(): + candidate = key.strip().lower() + if candidate.startswith(probe) or probe.startswith(candidate[:len(probe)]): + name = str(contact.get('adv_name', '') or '').strip() + if name: + return name + return sender + # ------------------------------------------------------------------ # Internal β€” message display # ------------------------------------------------------------------ @@ -399,6 +436,7 @@ class RoomServerPanel: card_state: Dict, room_messages: Dict, live_messages: List[Message], + contacts: Dict, ) -> None: """Update the message display for a single room card. @@ -412,6 +450,7 @@ class RoomServerPanel: card_state: UI state dict for this room card. room_messages: ``{12-char-prefix: [Message, …]}`` from archive cache. live_messages: Current session's rolling message buffer. + contacts: Current contacts snapshot for live name resolution. """ msg_container = card_state.get('msg_container') if not msg_container: @@ -455,7 +494,7 @@ class RoomServerPanel: with msg_container: for msg in display: direction = 'β†’' if msg.direction == 'out' else '←' - sender = msg.sender or '?' + sender = self._resolve_sender_name(msg.sender or '', contacts) line = f"{msg.time} {direction} {sender}: {msg.text}" ui.label(line).classes( diff --git a/meshcore_gui/gui/route_page.py b/meshcore_gui/gui/route_page.py index 492346a..93e97e9 100644 --- a/meshcore_gui/gui/route_page.py +++ b/meshcore_gui/gui/route_page.py @@ -79,9 +79,8 @@ _ROUTE_MAP_ASSETS = r""" ensureStylesheet('meshcore-leaflet-panel-css', '/static/leaflet_map_panel.css'); ensureScript('meshcore-leaflet-vendor-js', 'https://unpkg.com/leaflet@1.9.4/dist/leaflet.js', function () { - ensureScript('meshcore-leaflet-markercluster-js', 'https://unpkg.com/leaflet.markercluster@1.5.3/dist/leaflet.markercluster.js', function () { - ensureScript('meshcore-leaflet-panel-js', '/static/leaflet_map_panel.js'); - }); + ensureScript('meshcore-leaflet-panel-js', '/static/leaflet_map_panel.js'); + ensureScript('meshcore-leaflet-markercluster-js', 'https://unpkg.com/leaflet.markercluster@1.5.3/dist/leaflet.markercluster.js'); }); })(); @@ -250,9 +249,9 @@ class RoutePage: ).classes('text-xs text-gray-400 italic px-2 pt-2') container_id = f'route-map-{uuid4().hex}' - ui.html( - f'
' - ).classes('w-full').style('height: 24rem') + ui.element('div').props(f'id={container_id}').classes( + 'w-full' + ).style('height:24rem;border-radius:0.5rem;overflow:hidden;') boot_script = ( '(function bootRouteMap(retries){' diff --git a/meshcore_gui/static/leaflet_map_panel.js b/meshcore_gui/static/leaflet_map_panel.js index 63c5979..a845f4d 100644 --- a/meshcore_gui/static/leaflet_map_panel.js +++ b/meshcore_gui/static/leaflet_map_panel.js @@ -38,7 +38,7 @@ const existing = maps.get(containerId); const host = document.getElementById(containerId); - if (!host || typeof window.L === 'undefined' || typeof window.L.markerClusterGroup !== 'function') { + if (!host || typeof window.L === 'undefined') { return null; } @@ -118,21 +118,7 @@ ).addTo(map); state.theme = 'light'; - state.layers.contacts = window.L.markerClusterGroup({ - showCoverageOnHover: false, - spiderfyOnMaxZoom: true, - removeOutsideVisibleBounds: true, - animate: false, - chunkedLoading: true, - maxClusterRadius: 50, - iconCreateFunction(cluster) { - return window.L.divIcon({ - html: '
' + cluster.getChildCount() + '
', - className: 'meshcore-marker-cluster', - iconSize: window.L.point(42, 42), - }); - }, - }).addTo(map); + state.layers.contacts = buildContactsLayer().addTo(map); } catch (error) { maps.delete(containerId); delete host.__meshcoreLeafletState; @@ -418,6 +404,29 @@ ); } + function buildContactsLayer() { + if (typeof window.L.markerClusterGroup === 'function') { + return window.L.markerClusterGroup({ + showCoverageOnHover: false, + spiderfyOnMaxZoom: true, + removeOutsideVisibleBounds: true, + animate: false, + chunkedLoading: true, + maxClusterRadius: 50, + iconCreateFunction(cluster) { + return window.L.divIcon({ + html: '
' + cluster.getChildCount() + '
', + className: 'meshcore-marker-cluster', + iconSize: window.L.point(42, 42), + }); + }, + }); + } + + console.warn('MeshCoreLeafletBoot markercluster unavailable; falling back to plain layer group'); + return window.L.layerGroup(); + } + function escapeHtml(value) { return String(value) .replaceAll('&', '&') @@ -456,9 +465,9 @@ return; } - if (typeof window.L === 'undefined' || typeof window.L.markerClusterGroup !== 'function') { + if (typeof window.L === 'undefined') { if (retries >= MAX_RETRIES) { - console.error('MeshCoreLeafletBoot timeout waiting for Leaflet markercluster', { containerId }); + console.error('MeshCoreLeafletBoot timeout waiting for Leaflet runtime', { containerId }); return; } window.setTimeout(() => { @@ -470,6 +479,13 @@ try { const state = PANEL.ensureMap(containerId); if (!state) { + if (retries >= MAX_RETRIES) { + console.error('MeshCoreLeafletBoot timeout waiting for visible map host', { containerId }); + return; + } + window.setTimeout(() => { + scheduleProcess(containerId, retries + 1); + }, RETRY_DELAY_MS); return; } const current = pending.get(containerId); @@ -497,35 +513,21 @@ return; } - const observer = new MutationObserver(() => { + const timer = window.setTimeout(() => { + watchers.delete(containerId); const host = document.getElementById(containerId); - if (!host) { + if (host) { + scheduleProcess(containerId, retries + 1); return; } - observer.disconnect(); - watchers.delete(containerId); - scheduleProcess(containerId, retries + 1); - }); - - observer.observe(document.body, { - childList: true, - subtree: true, - }); - - watchers.set(containerId, observer); - - window.setTimeout(() => { - if (watchers.get(containerId) !== observer) { - return; - } - observer.disconnect(); - watchers.delete(containerId); if (retries >= MAX_RETRIES) { console.error('MeshCoreLeafletBoot timeout waiting for host element', { containerId }); return; } scheduleProcess(containerId, retries + 1); }, RETRY_DELAY_MS); + + watchers.set(containerId, timer); } function isDomReady() { @@ -533,14 +535,28 @@ } - window.MeshCoreRouteMapBoot = function (containerId, payload) { + window.MeshCoreRouteMapBoot = function (containerId, payload, retries) { if (!containerId || !payload) { return; } + const attempt = typeof retries === 'number' ? retries : 0; const host = document.getElementById(containerId); if (!host || typeof window.L === 'undefined') { - window.setTimeout(() => window.MeshCoreRouteMapBoot(containerId, payload), RETRY_DELAY_MS); + if (attempt >= MAX_RETRIES) { + console.error('MeshCoreRouteMapBoot timeout waiting for host/runtime', { containerId }); + return; + } + window.setTimeout(() => window.MeshCoreRouteMapBoot(containerId, payload, attempt + 1), RETRY_DELAY_MS); + return; + } + + if (host.clientWidth === 0 && host.clientHeight === 0) { + if (attempt >= MAX_RETRIES) { + console.error('MeshCoreRouteMapBoot timeout waiting for visible route map host', { containerId }); + return; + } + window.setTimeout(() => window.MeshCoreRouteMapBoot(containerId, payload, attempt + 1), RETRY_DELAY_MS); return; } @@ -641,6 +657,11 @@ } } + if (!current.snapshot && current.theme && !maps.has(containerId)) { + pending.set(containerId, current); + return; + } + pending.set(containerId, current); scheduleProcess(containerId, 0); };