Merge pull request #2 from pe1hvh/feature/data-cache

feat: add local JSON cache for instant startup and offline resilience

README.md

@@ -19,11 +19,6 @@ Under the hood it uses `bleak` for Bluetooth Low Energy (which talks to BlueZ on
 
 > **Linux users:** BLE on Linux can be temperamental. BlueZ occasionally gets into a bad state, especially after repeated connect/disconnect cycles. If you run into connection issues, see the [Troubleshooting Guide](docs/TROUBLESHOOTING.md). On macOS and Windows, BLE is generally more stable out of the box.
 
-## TODO
-
-* **Message persistence** — Store sent and received messages to disk so chat history is preserved across sessions
-* **Automatic channel discovery** — Robustly detect and subscribe to available channels without manual configuration
-* **Auto-detect BLE address** — Automatically discover and store the BLE device address in config, eliminating manual entry
 
 ## Features
 
@@ -36,6 +31,8 @@ Under the hood it uses `bleak` for Bluetooth Low Energy (which talks to BlueZ on
 - **Keyword Bot** — Built-in auto-reply bot that responds to configurable keywords on selected channels, with cooldown and loop prevention
 - **Packet Decoding** — Raw LoRa packets from RX log are decoded and decrypted using channel keys, providing message hashes, path hashes and hop data
 - **Message Deduplication** — Dual-strategy dedup (hash-based and content-based) prevents duplicate messages from appearing
+- **Local Cache** — Device info, contacts and channel keys are cached to disk (`~/.meshcore-gui/cache/`) so the GUI is instantly populated on startup from the last known state, even before BLE connects. Contacts from the device are merged with cached contacts so offline nodes are preserved. Channel keys that fail to load at startup are retried in the background every 30 seconds
+- **Periodic Contact Refresh** — Contacts are automatically refreshed from the device at a configurable interval (default: 5 minutes) and merged with the cache
 - **Threaded Architecture** — BLE communication in separate thread for stable UI
 
 ## Screenshots
@@ -189,6 +186,8 @@ The GUI opens automatically in your browser at `http://localhost:8080`
 |---------|----------|-------------|
 | `DEBUG` | `meshcore_gui/config.py` | Set to `True` for verbose logging (or use `--debug-on`) |
 | `CHANNELS_CONFIG` | `meshcore_gui/config.py` | List of channels (hardcoded due to BLE timing issues) |
+| `CONTACT_REFRESH_SECONDS` | `meshcore_gui/config.py` | Interval between periodic contact refreshes (default: 300s / 5 minutes) |
+| `KEY_RETRY_INTERVAL` | `meshcore_gui/ble/worker.py` | Interval between background retry attempts for missing channel keys (default: 30s) |
 | `BOT_CHANNELS` | `meshcore_gui/services/bot.py` | Channel indices the bot listens on |
 | `BOT_NAME` | `meshcore_gui/services/bot.py` | Display name prepended to bot replies |
 | `BOT_COOLDOWN_SECONDS` | `meshcore_gui/services/bot.py` | Minimum seconds between bot replies |
@@ -233,6 +232,31 @@ Route data is resolved from two sources (in priority order):
 1. **RX log packet decode** — Path hashes extracted from the raw LoRa packet via `meshcoredecoder`
 2. **Contact out_path** — Stored route from the sender's contact record (fallback)
 
+### Local Cache
+
+Device info, contacts and channel keys are automatically cached to disk in `~/.meshcore-gui/cache/`. One JSON file is created per BLE device address.
+
+**Startup behaviour:**
+1. Cache is loaded first — GUI is immediately populated with the last known state
+2. BLE connection is established in the background
+3. Fresh data from the device updates both the GUI and the cache
+
+**Channel key loading:**
+
+BLE commands to the MeshCore device are fundamentally unreliable — `get_channel()`, `send_appstart()` and `send_device_query()` can all fail even after multiple retries. The channel key loading strategy handles this gracefully:
+
+1. Cached keys are loaded first and never overwritten by name-derived fallbacks
+2. Each channel gets 2 quick attempts at startup (non-blocking)
+3. Channels that fail are retried in the background every 30 seconds
+4. Successfully loaded keys are immediately written to the cache for next startup
+
+**Contact merge strategy:**
+- New contacts from the device are added to the cache with a `last_seen` timestamp
+- Existing contacts are updated (fresh data wins)
+- Contacts only in cache (node offline) are preserved
+
+If BLE connection fails, the GUI remains usable with cached data and shows an offline status.
+
 ### Keyword Bot
 
 The built-in bot automatically replies to messages containing recognised keywords. Enable or disable it via the 🤖 BOT checkbox in the filter bar.
@@ -279,22 +303,24 @@ The built-in bot automatically replies to messages containing recognised keyword
 │  ┌─────┴─────┐  │  │  │   ┌────┴────┐   │
 │  │  Panels   │  │  │  │   │   Bot   │   │
 │  │  RoutePage│  │  │  │   │  Dedup  │   │
-│  └───────────┘  │  │  │   └─────────┘   │
-└─────────────────┘  │  └─────────────────┘
-                     │
+│  └───────────┘  │  │  │   │  Cache  │   │
+└─────────────────┘  │  │   └─────────┘   │
+                     │  └─────────────────┘
               ┌──────┴──────┐
-              │ SharedData  │
-              │ (thread-    │
-              │  safe)      │
-              └─────────────┘
+              │ SharedData  │     ┌───────────────┐
+              │ (thread-    │     │ DeviceCache   │
+              │  safe)      │     │ (~/.meshcore- │
+              └─────────────┘     │  gui/cache/)  │
+                                  └───────────────┘
 ```
 
-- **BLEWorker**: Runs in separate thread with its own asyncio loop
+- **BLEWorker**: Runs in separate thread with its own asyncio loop, with background retry for missing channel keys
 - **CommandHandler**: Executes commands (send message, advert, refresh)
 - **EventHandler**: Processes incoming BLE events (messages, RX log)
 - **PacketDecoder**: Decodes raw LoRa packets and extracts route data
 - **MeshBot**: Keyword-triggered auto-reply on configured channels
 - **DualDeduplicator**: Prevents duplicate messages (hash-based + content-based)
+- **DeviceCache**: Local JSON cache per device for instant startup and offline resilience
 - **SharedData**: Thread-safe data sharing between BLE and GUI via Protocol interfaces
 - **DashboardPage**: Main GUI with modular panels (device, contacts, map, messages, etc.)
 - **RoutePage**: Standalone route visualization page opened per message
@@ -302,9 +328,9 @@ The built-in bot automatically replies to messages containing recognised keyword
 
 ## Known Limitations
 
-1. **Channels hardcoded** — The `get_channel()` function in meshcore-py is unreliable via BLE
-2. **send_appstart() sometimes fails** — Device info may remain empty with connection problems
-3. **Initial load time** — GUI waits for BLE data before the first render is complete
+1. **Channels hardcoded** — The `get_channel()` function in meshcore-py is unreliable via BLE (mitigated by background retry and disk caching of channel keys)
+2. **BLE command unreliability** — `send_appstart()`, `send_device_query()` and `get_channel()` can all fail intermittently. The application uses aggressive retries (10 attempts for device info, background retry every 30s for channel keys) and disk caching to compensate
+3. **Initial load time** — GUI waits for BLE data before the first render is complete (mitigated by cache: if cached data exists, the GUI populates instantly)
 
 ## Troubleshooting
 
@@ -358,6 +384,16 @@ Make sure the MeshCore device is powered on and in BLE Companion mode. Run the B
 - Check if your channels are correctly configured
 - Use `meshcli` to verify that messages are arriving
 
+#### Clearing the cache
+
+If cached data causes issues (e.g. stale contacts), delete the cache file:
+
+```bash
+rm ~/.meshcore-gui/cache/*.json
+```
+
+The cache will be recreated on the next successful BLE connection.
+
 ## Development
 
 ### Debug mode
@@ -378,10 +414,10 @@ meshcore-gui/
 ├── meshcore_gui/                    # Application package
 │   ├── __init__.py
 │   ├── __main__.py                  # Alternative entry: python -m meshcore_gui
-│   ├── config.py                    # DEBUG flag, channel configuration
+│   ├── config.py                    # DEBUG flag, channel configuration, refresh interval
 │   ├── ble/                         # BLE communication layer
 │   │   ├── __init__.py
-│   │   ├── worker.py                # BLE thread, connection lifecycle
+│   │   ├── worker.py                # BLE thread, connection lifecycle, cache-first startup, background key retry
 │   │   ├── commands.py              # Command execution (send, refresh, advert)
 │   │   ├── events.py                # Event callbacks (messages, RX log)
 │   │   └── packet_decoder.py        # Raw LoRa packet decoding via meshcoredecoder
@@ -408,6 +444,7 @@ meshcore-gui/
 │   └── services/                    # Business logic
 │       ├── __init__.py
 │       ├── bot.py                   # Keyword-triggered auto-reply bot
+│       ├── cache.py                 # Local JSON cache per BLE device
 │       ├── dedup.py                 # Message deduplication
 │       └── route_builder.py         # Route data construction
 ├── docs/

meshcore_gui/ble/packet_decoder.py

@@ -90,12 +90,18 @@ class PacketDecoder:
     # Key management
     # ------------------------------------------------------------------
 
-    def add_channel_key(self, channel_idx: int, secret_bytes: bytes) -> None:
+    def add_channel_key(
+        self,
+        channel_idx: int,
+        secret_bytes: bytes,
+        source: str = "device",
+    ) -> None:
         """Register a channel decryption key (16 raw bytes from device).
 
         Args:
             channel_idx:  Channel index (0-based).
             secret_bytes: 16-byte channel secret from ``get_channel()``.
+            source:       Label for debug output (e.g. "device", "cache").
         """
         secret_hex = secret_bytes.hex()
         self._key_store.add_channel_secrets([secret_hex])
@@ -105,7 +111,7 @@ class PacketDecoder:
         self._hash_to_idx[ch_hash] = channel_idx
         debug_print(
             f"PacketDecoder: key for ch{channel_idx} "
-            f"(hash={ch_hash}, from device)"
+            f"(hash={ch_hash}, from {source})"
         )
 
     def add_channel_key_from_name(
@@ -121,11 +127,7 @@ class PacketDecoder:
             channel_name: Channel name string (e.g. ``"#test"``).
         """
         secret_bytes = sha256(channel_name.encode("utf-8")).digest()[:16]
-        self.add_channel_key(channel_idx, secret_bytes)
-        debug_print(
-            f"PacketDecoder: key for ch{channel_idx} "
-            f"(derived from '{channel_name}')"
-        )
+        self.add_channel_key(channel_idx, secret_bytes, source=f"name '{channel_name}'")
 
     @property
     def has_keys(self) -> bool:

meshcore_gui/ble/worker.py

@@ -15,23 +15,43 @@ Event handling     → :mod:`meshcore_gui.ble.events`
 Packet decoding    → :mod:`meshcore_gui.ble.packet_decoder`
 Bot logic          → :mod:`meshcore_gui.services.bot`
 Deduplication      → :mod:`meshcore_gui.services.dedup`
+Cache              → :mod:`meshcore_gui.services.cache`
+
+v5.1 changes
+~~~~~~~~~~~~~
+- Cache-first startup: GUI is populated instantly from disk cache.
+- Background BLE refresh updates cache + SharedData incrementally.
+- Periodic contact refresh every ``CONTACT_REFRESH_SECONDS``.
+- Channel keys are cached to disk for instant packet decoding.
+- Background key retry: missing channel keys are retried every
+  ``KEY_RETRY_INTERVAL`` seconds until all keys are loaded.
 """
 
 import asyncio
 import threading
-from typing import Optional
+import time
+from typing import Optional, Set
 
 from meshcore import MeshCore, EventType
 
-from meshcore_gui.config import CHANNELS_CONFIG, debug_print
+from meshcore_gui.config import (
+    CHANNELS_CONFIG,
+    CONTACT_REFRESH_SECONDS,
+    debug_print,
+)
 from meshcore_gui.core.protocols import SharedDataWriter
 from meshcore_gui.ble.commands import CommandHandler
 from meshcore_gui.ble.events import EventHandler
 from meshcore_gui.ble.packet_decoder import PacketDecoder
 from meshcore_gui.services.bot import BotConfig, MeshBot
+from meshcore_gui.services.cache import DeviceCache
 from meshcore_gui.services.dedup import DualDeduplicator
 
 
+# Seconds between background retry attempts for missing channel keys.
+KEY_RETRY_INTERVAL: float = 30.0
+
+
 class BLEWorker:
     """BLE communication worker that runs in a separate thread.
 
@@ -46,6 +66,9 @@ class BLEWorker:
         self.mc: Optional[MeshCore] = None
         self.running = True
 
+        # Local cache (one file per device)
+        self._cache = DeviceCache(address)
+
         # Collaborators (created eagerly, wired after connection)
         self._decoder = PacketDecoder()
         self._dedup = DualDeduplicator(max_size=200)
@@ -55,6 +78,9 @@ class BLEWorker:
             enabled_check=shared.is_bot_enabled,
         )
 
+        # Channel indices that still need keys from device
+        self._pending_keys: Set[int] = set()
+
     # ------------------------------------------------------------------
     # Thread lifecycle
     # ------------------------------------------------------------------
@@ -71,15 +97,39 @@ class BLEWorker:
     async def _async_main(self) -> None:
         await self._connect()
         if self.mc:
+            last_contact_refresh = time.time()
+            last_key_retry = time.time()
+
             while self.running:
                 await self._cmd_handler.process_all()
+
+                now = time.time()
+
+                # Periodic contact refresh
+                if now - last_contact_refresh > CONTACT_REFRESH_SECONDS:
+                    await self._refresh_contacts()
+                    last_contact_refresh = now
+
+                # Background key retry for missing channels
+                if self._pending_keys and now - last_key_retry > KEY_RETRY_INTERVAL:
+                    await self._retry_missing_keys()
+                    last_key_retry = now
+
                 await asyncio.sleep(0.1)
 
     # ------------------------------------------------------------------
-    # Connection
+    # Connection (cache-first)
     # ------------------------------------------------------------------
 
     async def _connect(self) -> None:
+        # Phase 1: Load cache → GUI is instantly populated
+        if self._cache.load():
+            self._apply_cache()
+            print("BLE: Cache loaded — GUI populated from disk")
+        else:
+            print("BLE: No cache found — waiting for BLE data")
+
+        # Phase 2: Connect BLE
         self.shared.set_status(f"🔄 Connecting to {self.address}...")
         try:
             print(f"BLE: Connecting to {self.address}...")
@@ -103,6 +153,7 @@ class BLEWorker:
             self.mc.subscribe(EventType.CONTACT_MSG_RECV, self._evt_handler.on_contact_msg)
             self.mc.subscribe(EventType.RX_LOG_DATA, self._evt_handler.on_rx_log)
 
+            # Phase 3: Load data and keys from device (updates cache)
             await self._load_data()
             await self._load_channel_keys()
             await self.mc.start_auto_message_fetching()
@@ -111,89 +162,324 @@ class BLEWorker:
             self.shared.set_status("✅ Connected")
             print("BLE: Ready!")
 
+            if self._pending_keys:
+                pending_names = [
+                    f"[{ch['idx']}] {ch['name']}"
+                    for ch in CHANNELS_CONFIG
+                    if ch['idx'] in self._pending_keys
+                ]
+                print(
+                    f"BLE: ⏳ Background retry active for: "
+                    f"{', '.join(pending_names)} "
+                    f"(every {KEY_RETRY_INTERVAL:.0f}s)"
+                )
+
         except Exception as e:
             print(f"BLE: Connection error: {e}")
-            self.shared.set_status(f"❌ {e}")
+            if self._cache.has_cache:
+                self.shared.set_status(f"⚠️ Offline — using cached data ({e})")
+            else:
+                self.shared.set_status(f"❌ {e}")
 
     # ------------------------------------------------------------------
-    # Initial data loading
+    # Apply cache to SharedData
+    # ------------------------------------------------------------------
+
+    def _apply_cache(self) -> None:
+        """Push cached data to SharedData so GUI renders immediately."""
+        device = self._cache.get_device()
+        if device:
+            self.shared.update_from_appstart(device)
+            # Firmware version may be stored under 'ver' or 'firmware_version'
+            fw = device.get("firmware_version") or device.get("ver")
+            if fw:
+                self.shared.update_from_device_query({"ver": fw})
+            self.shared.set_status("📦 Loaded from cache")
+            debug_print(f"Cache → device info: {device.get('name', '?')}")
+
+        channels = self._cache.get_channels()
+        if channels:
+            self.shared.set_channels(channels)
+            debug_print(f"Cache → channels: {[c['name'] for c in channels]}")
+
+        contacts = self._cache.get_contacts()
+        if contacts:
+            self.shared.set_contacts(contacts)
+            debug_print(f"Cache → contacts: {len(contacts)}")
+
+        # Restore channel keys for instant packet decoding
+        cached_keys = self._cache.get_channel_keys()
+        for idx_str, secret_hex in cached_keys.items():
+            try:
+                idx = int(idx_str)
+                secret_bytes = bytes.fromhex(secret_hex)
+                if len(secret_bytes) >= 16:
+                    self._decoder.add_channel_key(idx, secret_bytes[:16], source="cache")
+                    debug_print(f"Cache → channel key [{idx}]")
+            except (ValueError, TypeError) as exc:
+                debug_print(f"Cache → bad channel key [{idx_str}]: {exc}")
+
+    # ------------------------------------------------------------------
+    # Initial data loading (refreshes cache)
     # ------------------------------------------------------------------
 
     async def _load_data(self) -> None:
-        """Load device info, channels and contacts."""
-        # send_appstart (retries)
+        """Load device info, channels and contacts from device.
+
+        Updates both SharedData (for GUI) and the disk cache.
+        Uses longer delays between retries because BLE command/response
+        over the meshcore library is unreliable with short intervals.
+        """
+        # send_appstart (retries with longer delays)
         self.shared.set_status("🔄 Device info...")
-        for i in range(5):
-            debug_print(f"send_appstart attempt {i + 1}")
-            r = await self.mc.commands.send_appstart()
-            if r.type != EventType.ERROR:
-                print(f"BLE: send_appstart OK: {r.payload.get('name')}")
-                self.shared.update_from_appstart(r.payload)
-                break
-            await asyncio.sleep(0.3)
+        appstart_ok = False
+        for i in range(10):
+            debug_print(f"send_appstart attempt {i + 1}/10")
+            try:
+                r = await self.mc.commands.send_appstart()
+                if r.type != EventType.ERROR:
+                    print(f"BLE: send_appstart OK: {r.payload.get('name')} (attempt {i + 1})")
+                    self.shared.update_from_appstart(r.payload)
+                    self._cache.set_device(r.payload)
+                    appstart_ok = True
+                    break
+            except Exception as exc:
+                debug_print(f"send_appstart attempt {i + 1} exception: {exc}")
+            await asyncio.sleep(1.0)
+
+        if not appstart_ok:
+            print("BLE: ⚠️  send_appstart failed after 10 attempts")
 
         # send_device_query (retries)
-        for i in range(5):
-            debug_print(f"send_device_query attempt {i + 1}")
-            r = await self.mc.commands.send_device_query()
-            if r.type != EventType.ERROR:
-                print(f"BLE: send_device_query OK: {r.payload.get('ver')}")
-                self.shared.update_from_device_query(r.payload)
-                break
-            await asyncio.sleep(0.3)
+        for i in range(10):
+            debug_print(f"send_device_query attempt {i + 1}/10")
+            try:
+                r = await self.mc.commands.send_device_query()
+                if r.type != EventType.ERROR:
+                    fw = r.payload.get("ver", "")
+                    print(f"BLE: send_device_query OK: {fw} (attempt {i + 1})")
+                    self.shared.update_from_device_query(r.payload)
+                    if fw:
+                        self._cache.set_firmware_version(fw)
+                    break
+            except Exception as exc:
+                debug_print(f"send_device_query attempt {i + 1} exception: {exc}")
+            await asyncio.sleep(1.0)
 
         # Channels (hardcoded — BLE get_channel is unreliable)
         self.shared.set_status("🔄 Channels...")
         self.shared.set_channels(CHANNELS_CONFIG)
+        self._cache.set_channels(CHANNELS_CONFIG)
         print(f"BLE: Channels loaded: {[c['name'] for c in CHANNELS_CONFIG]}")
 
-        # Contacts
+        # Contacts (merge with cache)
         self.shared.set_status("🔄 Contacts...")
-        r = await self.mc.commands.get_contacts()
-        if r.type != EventType.ERROR:
-            self.shared.set_contacts(r.payload)
-            print(f"BLE: Contacts loaded: {len(r.payload)} contacts")
+        try:
+            r = await self.mc.commands.get_contacts()
+            if r.type != EventType.ERROR:
+                merged = self._cache.merge_contacts(r.payload)
+                self.shared.set_contacts(merged)
+                print(
+                    f"BLE: Contacts — {len(r.payload)} from device, "
+                    f"{len(merged)} total (with cache)"
+                )
+            else:
+                debug_print("BLE: get_contacts failed, keeping cached contacts")
+        except Exception as exc:
+            debug_print(f"BLE: get_contacts exception: {exc}")
+
+    # ------------------------------------------------------------------
+    # Channel key loading (quick startup + background retry)
+    # ------------------------------------------------------------------
 
     async def _load_channel_keys(self) -> None:
-        """Load channel decryption keys from device or derive from name.
+        """Try to load channel keys from device — quick pass at startup.
 
-        Channels that cannot be confirmed on the device are logged with
-        a warning.  Sending and receiving on unconfirmed channels will
-        likely fail because the device does not know about them.
+        Each channel gets 2 quick attempts.  Channels that fail are
+        added to ``_pending_keys`` for background retry every
+        ``KEY_RETRY_INTERVAL`` seconds.
+
+        Priority:
+        1. Key from device (get_channel → channel_secret)
+        2. Key already in cache (preserved, not overwritten)
+        3. Key derived from channel name (last resort, only if no cache)
         """
         self.shared.set_status("🔄 Channel keys...")
+        cached_keys = self._cache.get_channel_keys()
+
         confirmed: list[str] = []
-        missing: list[str] = []
+        from_cache: list[str] = []
+        pending: list[str] = []
+        derived: list[str] = []
 
-        for ch in CHANNELS_CONFIG:
+        for ch_num, ch in enumerate(CHANNELS_CONFIG):
             idx, name = ch['idx'], ch['name']
-            loaded = False
 
-            for attempt in range(3):
-                try:
-                    r = await self.mc.commands.get_channel(idx)
-                    if r.type != EventType.ERROR:
-                        secret = r.payload.get('channel_secret')
-                        if secret and isinstance(secret, bytes) and len(secret) >= 16:
-                            self._decoder.add_channel_key(idx, secret[:16])
-                            print(f"BLE: ✅ Channel [{idx}] '{name}' — key loaded from device")
-                            confirmed.append(f"[{idx}] {name}")
-                            loaded = True
-                            break
-                except Exception as exc:
-                    debug_print(f"get_channel({idx}) attempt {attempt + 1} error: {exc}")
-                await asyncio.sleep(0.3)
+            # Quick startup attempt: 2 tries per channel
+            loaded = await self._try_load_channel_key(idx, name, max_attempts=2, delay=1.0)
 
-            if not loaded:
+            if loaded:
+                confirmed.append(f"[{idx}] {name}")
+            elif str(idx) in cached_keys:
+                # Cache has the key — don't overwrite with name-derived
+                from_cache.append(f"[{idx}] {name}")
+                print(f"BLE: 📦 Channel [{idx}] '{name}' — using cached key")
+            else:
+                # No device key, no cache key — derive from name as temporary fallback
                 self._decoder.add_channel_key_from_name(idx, name)
-                missing.append(f"[{idx}] {name}")
-                print(f"BLE: ⚠️  Channel [{idx}] '{name}' — NOT found on device (key derived from name)")
+                derived.append(f"[{idx}] {name}")
+                # Mark for background retry
+                self._pending_keys.add(idx)
+                print(f"BLE: ⚠️  Channel [{idx}] '{name}' — name-derived key (will retry)")
 
-        if missing:
-            print(f"BLE: ⚠️  Channels not confirmed on device: {', '.join(missing)}")
-            print(f"BLE: ⚠️  Sending/receiving on these channels may not work.")
-            print(f"BLE: ⚠️  Check your device config with: meshcli -d <BLE_ADDRESS> → get_channels")
+            # Brief pause between channels
+            if ch_num < len(CHANNELS_CONFIG) - 1:
+                await asyncio.sleep(0.5)
 
+        # Summary
         print(f"BLE: PacketDecoder ready — has_keys={self._decoder.has_keys}")
-        print(f"BLE: Confirmed: {', '.join(confirmed) if confirmed else 'none'}")
-        print(f"BLE: Unconfirmed: {', '.join(missing) if missing else 'none'}")
+        if confirmed:
+            print(f"BLE: ✅ From device: {', '.join(confirmed)}")
+        if from_cache:
+            print(f"BLE: 📦 From cache: {', '.join(from_cache)}")
+        if derived:
+            print(f"BLE: ⚠️  Name-derived: {', '.join(derived)}")
+
+    async def _try_load_channel_key(
+        self,
+        idx: int,
+        name: str,
+        max_attempts: int,
+        delay: float,
+    ) -> bool:
+        """Try to load a single channel key from the device.
+
+        Returns True if the key was successfully loaded and cached.
+        """
+        for attempt in range(max_attempts):
+            try:
+                r = await self.mc.commands.get_channel(idx)
+
+                if r.type == EventType.ERROR:
+                    debug_print(
+                        f"get_channel({idx}) attempt {attempt + 1}/{max_attempts}: "
+                        f"ERROR response"
+                    )
+                    await asyncio.sleep(delay)
+                    continue
+
+                secret = r.payload.get('channel_secret')
+                debug_print(
+                    f"get_channel({idx}) attempt {attempt + 1}/{max_attempts}: "
+                    f"type={type(secret).__name__}, "
+                    f"len={len(secret) if secret else 0}, "
+                    f"keys={list(r.payload.keys())}"
+                )
+
+                # Extract secret bytes (handles both bytes and hex string)
+                secret_bytes = self._extract_secret(secret)
+                if secret_bytes:
+                    self._decoder.add_channel_key(idx, secret_bytes, source="device")
+                    self._cache.set_channel_key(idx, secret_bytes.hex())
+                    print(
+                        f"BLE: ✅ Channel [{idx}] '{name}' — "
+                        f"key from device (attempt {attempt + 1})"
+                    )
+                    # Remove from pending if it was there
+                    self._pending_keys.discard(idx)
+                    return True
+
+                debug_print(
+                    f"get_channel({idx}) attempt {attempt + 1}/{max_attempts}: "
+                    f"response OK but secret unusable"
+                )
+
+            except Exception as exc:
+                debug_print(
+                    f"get_channel({idx}) attempt {attempt + 1}/{max_attempts} "
+                    f"error: {exc}"
+                )
+
+            await asyncio.sleep(delay)
+
+        return False
+
+    async def _retry_missing_keys(self) -> None:
+        """Background retry for channels that failed during startup.
+
+        Called periodically from the main loop.  Each missing channel
+        gets one attempt per cycle.  Successfully loaded keys are
+        removed from ``_pending_keys``.
+        """
+        if not self._pending_keys:
+            return
+
+        pending_copy = set(self._pending_keys)
+        ch_map = {ch['idx']: ch['name'] for ch in CHANNELS_CONFIG}
+
+        debug_print(
+            f"Background key retry: trying {len(pending_copy)} channels"
+        )
+
+        for idx in pending_copy:
+            name = ch_map.get(idx, f"ch{idx}")
+            loaded = await self._try_load_channel_key(
+                idx, name, max_attempts=1, delay=0.5,
+            )
+            if loaded:
+                self._pending_keys.discard(idx)
+            await asyncio.sleep(1.0)
+
+        if not self._pending_keys:
+            print("BLE: ✅ All channel keys now loaded!")
+        else:
+            remaining = [
+                f"[{idx}] {ch_map.get(idx, '?')}"
+                for idx in sorted(self._pending_keys)
+            ]
+            debug_print(f"Background retry: still pending: {', '.join(remaining)}")
+
+    # ------------------------------------------------------------------
+    # Helpers
+    # ------------------------------------------------------------------
+
+    @staticmethod
+    def _extract_secret(secret) -> Optional[bytes]:
+        """Extract 16-byte secret from various formats.
+
+        Handles:
+        - bytes (normal case from BLE)
+        - hex string (some firmware versions)
+
+        Returns 16-byte secret or None if unusable.
+        """
+        if secret and isinstance(secret, bytes) and len(secret) >= 16:
+            return secret[:16]
+
+        if secret and isinstance(secret, str) and len(secret) >= 32:
+            try:
+                raw = bytes.fromhex(secret)
+                if len(raw) >= 16:
+                    return raw[:16]
+            except ValueError:
+                pass
+
+        return None
+
+    # ------------------------------------------------------------------
+    # Periodic contact refresh
+    # ------------------------------------------------------------------
+
+    async def _refresh_contacts(self) -> None:
+        """Periodic background contact refresh — merge new/changed."""
+        try:
+            r = await self.mc.commands.get_contacts()
+            if r.type != EventType.ERROR:
+                merged = self._cache.merge_contacts(r.payload)
+                self.shared.set_contacts(merged)
+                debug_print(
+                    f"Periodic refresh: {len(r.payload)} from device, "
+                    f"{len(merged)} total"
+                )
+        except Exception as exc:
+            debug_print(f"Periodic contact refresh failed: {exc}")

meshcore_gui/config.py

@@ -41,3 +41,13 @@ CHANNELS_CONFIG: List[Dict] = [
     {'idx': 3, 'name': 'RahanSom'},
     {'idx': 4, 'name': '#bot'},
 ]
+
+
+# ==============================================================================
+# CACHE / REFRESH
+# ==============================================================================
+
+# Interval in seconds between periodic contact refreshes from the device.
+# Contacts are merged (new/changed contacts update the cache; contacts
+# only present in cache are kept so offline nodes are preserved).
+CONTACT_REFRESH_SECONDS: float = 300.0  # 5 minutes

meshcore_gui/services/__init__.py

@@ -1,3 +1,3 @@
 """
-Business logic services — bot, deduplication and route building.
+Business logic services — bot, cache, deduplication and route building.
 """

meshcore_gui/services/cache.py

@@ -0,0 +1,201 @@
+"""
+Local JSON cache for device info, channels and contacts.
+
+Loads instantly on startup so the GUI is immediately populated with
+the last known state.  Background BLE refreshes update the cache
+incrementally.
+
+Cache location
+~~~~~~~~~~~~~~
+``~/.meshcore-gui/cache/<ADDRESS>.json``
+
+One file per BLE device address, so multiple devices are supported
+without conflict.
+
+Merge strategy (contacts)
+~~~~~~~~~~~~~~~~~~~~~~~~~
+- New contacts from device → added to cache with ``last_seen`` timestamp
+- Existing contacts → updated (fresh data wins)
+- Contacts only in cache (node offline) → kept
+- Optional pruning of contacts not seen for > N days (not yet implemented)
+"""
+
+import json
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Dict, List, Optional
+
+from meshcore_gui.config import debug_print
+
+CACHE_VERSION = 1
+CACHE_DIR = Path.home() / ".meshcore-gui" / "cache"
+
+
+class DeviceCache:
+    """Read/write JSON cache for a single BLE device.
+
+    Args:
+        ble_address: BLE address string (used to derive filename).
+    """
+
+    def __init__(self, ble_address: str) -> None:
+        self._address = ble_address
+        safe_name = (
+            ble_address
+            .replace("literal:", "")
+            .replace(":", "_")
+            .replace("/", "_")
+        )
+        self._path = CACHE_DIR / f"{safe_name}.json"
+        self._data: Dict = {}
+
+    @property
+    def path(self) -> Path:
+        """Path to the cache file on disk."""
+        return self._path
+
+    @property
+    def has_cache(self) -> bool:
+        """True if a cache file exists on disk."""
+        return self._path.exists()
+
+    # ------------------------------------------------------------------
+    # Load / Save
+    # ------------------------------------------------------------------
+
+    def load(self) -> bool:
+        """Load cache from disk.
+
+        Returns:
+            True if a valid cache was loaded, False otherwise.
+        """
+        if not self._path.exists():
+            debug_print(f"Cache: no file at {self._path}")
+            return False
+
+        try:
+            self._data = json.loads(self._path.read_text(encoding="utf-8"))
+        except (json.JSONDecodeError, OSError) as exc:
+            debug_print(f"Cache: load error: {exc}")
+            self._data = {}
+            return False
+
+        if self._data.get("version") != CACHE_VERSION:
+            debug_print("Cache: version mismatch, ignoring")
+            self._data = {}
+            return False
+
+        last = self._data.get("last_updated", "?")
+        debug_print(f"Cache: loaded from {self._path} (last_updated={last})")
+        return True
+
+    def save(self) -> None:
+        """Write current state to disk."""
+        self._data["version"] = CACHE_VERSION
+        self._data["address"] = self._address
+        self._data["last_updated"] = datetime.now(timezone.utc).isoformat()
+
+        try:
+            CACHE_DIR.mkdir(parents=True, exist_ok=True)
+            self._path.write_text(
+                json.dumps(self._data, indent=2, ensure_ascii=False),
+                encoding="utf-8",
+            )
+            debug_print(f"Cache: saved to {self._path}")
+        except OSError as exc:
+            debug_print(f"Cache: save error: {exc}")
+
+    # ------------------------------------------------------------------
+    # Device info
+    # ------------------------------------------------------------------
+
+    def get_device(self) -> Optional[Dict]:
+        """Return cached device info dict, or None."""
+        return self._data.get("device")
+
+    def set_device(self, payload: Dict) -> None:
+        """Store device info and persist to disk."""
+        self._data["device"] = payload.copy()
+        self.save()
+
+    def set_firmware_version(self, version: str) -> None:
+        """Update firmware version in the cached device info."""
+        device = self._data.get("device", {})
+        device["firmware_version"] = version
+        self._data["device"] = device
+        self.save()
+
+    # ------------------------------------------------------------------
+    # Channels
+    # ------------------------------------------------------------------
+
+    def get_channels(self) -> List[Dict]:
+        """Return cached channel list (may be empty)."""
+        return self._data.get("channels", [])
+
+    def set_channels(self, channels: List[Dict]) -> None:
+        """Store channel list and persist to disk."""
+        self._data["channels"] = [ch.copy() for ch in channels]
+        self.save()
+
+    # ------------------------------------------------------------------
+    # Channel keys
+    # ------------------------------------------------------------------
+
+    def get_channel_keys(self) -> Dict[int, str]:
+        """Return cached channel keys as ``{idx: secret_hex}``."""
+        return self._data.get("channel_keys", {})
+
+    def set_channel_key(self, channel_idx: int, secret_hex: str) -> None:
+        """Store a single channel key (hex string) and persist."""
+        keys = self._data.get("channel_keys", {})
+        keys[str(channel_idx)] = secret_hex
+        self._data["channel_keys"] = keys
+        self.save()
+
+    # ------------------------------------------------------------------
+    # Contacts (merge strategy)
+    # ------------------------------------------------------------------
+
+    def get_contacts(self) -> Dict:
+        """Return cached contacts dict (may be empty)."""
+        return self._data.get("contacts", {})
+
+    def merge_contacts(self, fresh: Dict) -> Dict:
+        """Merge fresh contacts into cache and persist.
+
+        Strategy:
+        - New contacts in ``fresh`` → added with ``last_seen``
+        - Existing contacts → updated (fresh data wins)
+        - Contacts only in cache → kept (node may be offline)
+
+        Args:
+            fresh: Contacts dict from ``get_contacts()`` BLE response.
+
+        Returns:
+            The merged contacts dict (superset of cached + fresh).
+        """
+        cached = self._data.get("contacts", {})
+        now = datetime.now(timezone.utc).isoformat()
+
+        for key, contact in fresh.items():
+            contact_copy = contact.copy()
+            contact_copy["last_seen"] = now
+            cached[key] = contact_copy
+
+        self._data["contacts"] = cached
+        self.save()
+
+        debug_print(
+            f"Cache: contacts merged — "
+            f"{len(fresh)} fresh, {len(cached)} total"
+        )
+        return cached
+
+    # ------------------------------------------------------------------
+    # Metadata
+    # ------------------------------------------------------------------
+
+    def get_last_updated(self) -> Optional[str]:
+        """Return ISO timestamp of last cache update, or None."""
+        return self._data.get("last_updated")