fix(dm): delayed path backfill for FLOOD-delivered messages

When FLOOD delivery is confirmed, the PATH_UPDATE event payload often
has empty path data because firmware updates the contact's out_path
asynchronously. After 3s delay, read the contact's updated path from
the meshcore library's in-memory contacts dict and backfill the DB.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

.env.example

@@ -2,17 +2,25 @@
 # Copy this file to .env and adjust values for your setup
 
 # ============================================
-# MeshCore Device Configuration
+# MeshCore Device Connection
 # ============================================
+# Two transport options: Serial (USB) or TCP (network).
+# Set MC_TCP_HOST to use TCP; leave empty to use serial.
 
-# Serial port path
+# --- Option A: Serial (default) ---
 # Use "auto" for automatic detection (recommended if only one USB device)
 # Or specify manually: /dev/serial/by-id/usb-xxx or /dev/ttyUSB0
 # Find available devices: ls /dev/serial/by-id/
 MC_SERIAL_PORT=auto
 
+# --- Option B: TCP (e.g. remote device via ser2net, meshcore-proxy) ---
+# Set the IP/hostname of the device to connect via TCP instead of serial.
+# When MC_TCP_HOST is set, MC_SERIAL_PORT is ignored.
+# MC_TCP_HOST=192.168.1.100
+# MC_TCP_PORT=5555
+
 # Your MeshCore device name (used for .msgs file)
-# Use "auto" for automatic detection from meshcli (recommended)
+# Use "auto" for automatic detection from device (recommended)
 # Or specify manually: MarWoj, SP5XYZ, MyNode
 MC_DEVICE_NAME=auto
 

.gitignore

@@ -76,6 +76,7 @@ data/
 # ============================================
 *.log
 *.sql
+!app/schema.sql
 *.sqlite
 *.db
 
@@ -102,3 +103,8 @@ docs/UI-Contact-Management-MVP-v2.md
 docs/TEST-PLAN-Contact-Management-v2.md
 docs/github-discussion-*.md
 docs/github-response-spaces-in-device-name.md
+docs/check-compat-howto.md
+docs/v2/
+docs/PRD-mc-webui-2.md
+docs/PRD-mc-webui-2-en.html
+docs/PRD-mc-webui-2-pl.html

Dockerfile

@@ -1,10 +1,13 @@
-# mc-webui Dockerfile
-# Python 3.11+ with Flask (meshcore-cli runs in separate bridge container)
+# mc-webui v2 Dockerfile
+# Single container with direct MeshCore device access (serial/TCP)
 
 FROM python:3.11-slim
 
-# Install curl for testing
-RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/*
+# Install system deps: curl (healthcheck), udev (serial device support)
+RUN apt-get update && apt-get install -y \
+    curl \
+    udev \
+    && rm -rf /var/lib/apt/lists/*
 
 # Set working directory
 WORKDIR /app
@@ -17,7 +20,6 @@ RUN pip install --no-cache-dir -r requirements.txt
 
 # Copy application code
 # Note: Run 'python -m app.version freeze' before build to include version info
-# The version_frozen.py file will be copied automatically if it exists
 COPY app/ ./app/
 
 # Expose Flask port

README.md

@@ -1,28 +1,38 @@
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
 # mc-webui
 
-A lightweight web interface for meshcore-cli, providing browser-based access to MeshCore mesh network.
+A lightweight web interface providing browser-based access to MeshCore mesh network.
 
 [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/MarekWo/mc-webui)
 
 ## Overview
 
-**mc-webui** is a Flask-based web application that wraps `meshcore-cli`, eliminating the need for SSH/terminal access when using MeshCore chat on a LoRa device connected to a Debian VM via BLE or USB. Tested on Heltec V3 and Heltec V4.
+**mc-webui** is a Flask-based web application providing browser-based access to MeshCore mesh network. It communicates directly with your LoRa device (via USB, BLE, or TCP) using the `meshcore` Python library, eliminating the need for SSH/terminal access. Tested on Heltec V3 and Heltec V4.
 
 ![Diagram](images/diagram.jpeg)
 
 ## Key Features
 
-- **Mobile-first design** - Responsive UI optimized for small screens
+- **Mobile-first design** - Responsive UI optimized for small screens, with desktop sidebar for wide screens
 - **Channel management** - Create, join, share (QR code), and switch between encrypted channels
-- **Direct Messages (DM)** - Private messaging with delivery status tracking
+- **Direct Messages (DM)** - Private messaging with searchable contact selector, delivery tracking, configurable retry strategy, and multi-path routing
 - **Smart notifications** - Unread message counters per channel with cross-device sync
-- **Contact management** - Manual approval mode, filtering, protection, cleanup tools
-- **Contact map** - View contacts with GPS coordinates on OpenStreetMap (Leaflet)
+- **Contact management** - Manual approval, add via URI/QR, filtering, protection, ignoring, blocking, batch operations, and cleanup tools
+- **Global search** - Full-text search across all messages (channels and DMs) with FTS5 backend
+- **Database** - Fast and reliable SQLite storage for messages, contacts, and configurations
+- **Contact map** - View contacts and own device on OpenStreetMap (Leaflet) with last seen info
 - **Message archives** - Automatic daily archiving with browse-by-date selector
-- **Interactive Console** - Direct meshcli command execution via WebSocket
+- **Interactive Console** - Full MeshCore command suite via WebSocket — repeater, contact, device, and channel management
+- **Device dashboard** - Device info, statistics, and contact sharing (QR code / URI)
+- **Dark/Light theme** - Toggle between dark and light UI themes
+- **Settings** - Configurable DM retry parameters, message retention, quote length, and theme
+- **System Log** - Real-time log viewer with streaming
+- **Database backup** - Create, list, and download database backups from the UI
 - **@Mentions autocomplete** - Type @ to see contact suggestions with fuzzy search
-- **Echo tracking** - "Heard X repeats" with repeater IDs for sent messages, route path for incoming messages (persisted across restarts)
+- **Echo tracking** - "Heard X repeats" with repeater IDs for sent messages, all route paths for incoming messages with deterministic payload matching (persisted across restarts)
+- **MeshCore Analyzer** - View packet details on analyzer.letsmesh.net directly from channel messages
+- **DM delivery tracking** - ACK-based delivery confirmation with SNR and route info
+- **Multi-device support** - Database file named after device public key for easy multi-device setups
 - **PWA support** - Browser notifications and installable app (experimental)
 - **Full offline support** - Works without internet (local Bootstrap, icons, emoji picker)
 
@@ -41,9 +51,9 @@ For detailed feature documentation, see the [User Guide](docs/user-guide.md).
 - Docker and Docker Compose installed ([installation guide](docs/docker-install.md))
 
 **Important Notes:**
-- No meshcore-cli installation required on host - automatically installed inside Docker container
+- Powered by direct meshcore library integration (v2 architecture)
 - No manual directory setup needed - all data stored in `./data/` inside the project directory
-- meshcore-cli version 1.3.12+ is automatically installed for proper DM functionality
+- Uses a single-container architecture with a fast SQLite database
 
 ---
 
@@ -96,16 +106,16 @@ For detailed feature documentation, see the [User Guide](docs/user-guide.md).
 
     This will:
     - Download base images (Python, Alpine Linux)
-    - Install meshcore-cli inside containers
+    - Install the `meshcore` Python library
     - Create `./data/` directory structure automatically
-    - Start both containers (meshcore-bridge and mc-webui)
+    - Start the mc-webui container
 
 5. **Verify installation**
     ```bash
     docker compose ps
     ```
 
-    Both containers should show `Up` status. Check logs if needed:
+    The container should show `Up` status. Check logs if needed:
     ```bash
     docker compose logs -f
     ```
@@ -133,7 +143,9 @@ For detailed feature documentation, see the [User Guide](docs/user-guide.md).
 3. **Switch channels** - Use the dropdown in navbar
 4. **Direct Messages** - Access via menu (☰) → "Direct Messages"
 5. **Manage contacts** - Access via menu (☰) → "Contact Management"
-6. **Console** - Access via menu (☰) → "Console" for direct meshcli commands
+6. **Console** - Access via menu (☰) → "Console" for MeshCore commands
+7. **Search** - Access via menu (☰) → "Search" for full-text message search
+8. **Settings** - Access via menu (☰) → "Settings" for DM retry and other configuration
 
 For complete usage instructions, see the [User Guide](docs/user-guide.md).
 
@@ -263,6 +275,12 @@ sudo ~/mc-webui/scripts/updater/install.sh --uninstall
     <td align="center"><a href="gallery/approve_contact.png"><img src="gallery/approve_contact.png" width="150"><br>Approve Contact</a></td>
     <td align="center"><a href="gallery/channel_management.png"><img src="gallery/channel_management.png" width="150"><br>Channel Management</a></td>
   </tr>
+  <tr>
+    <td align="center"><a href="gallery/global_search.png"><img src="gallery/global_search.png" width="150"><br>Global Search</a></td>
+    <td align="center"><a href="gallery/message_filtering.png"><img src="gallery/message_filtering.png" width="150"><br>Message Filtering</a></td>
+    <td align="center"><a href="gallery/DM_Settings.png"><img src="gallery/DM_Settings.png" width="150"><br>Settings</a></td>
+    <td align="center"><a href="gallery/sytem_log.png"><img src="gallery/sytem_log.png" width="150"><br>System Log</a></td>
+  </tr>
   <tr>
     <td align="center"><a href="gallery/map.png"><img src="gallery/map.png" width="150"><br>Map</a></td>
     <td align="center"><a href="gallery/map_individual.png"><img src="gallery/map_individual.png" width="150"><br>Map (Individual)</a></td>
@@ -289,31 +307,39 @@ sudo ~/mc-webui/scripts/updater/install.sh --uninstall
 
 ### Completed Features
 
-- [x] Environment Setup & Docker Architecture
-- [x] Backend Basics (REST API, message parsing, CLI wrapper)
-- [x] Frontend Chat View (Bootstrap UI, message display)
-- [x] Message Sending (Send form, reply functionality)
+- [x] Environment Setup & Docker Architecture (single-container, direct device access)
+- [x] Backend Basics (REST API, SQLite database, meshcore library integration)
+- [x] Frontend Chat View (Bootstrap UI, message display, quote/reply)
+- [x] Message Sending (Send form, reply, quote with configurable length)
 - [x] Intelligent Auto-refresh (10s checks, UI updates only when needed)
-- [x] Contact Management (Cleanup modal with configurable threshold)
+- [x] Contact Management (Approval, add via URI/QR, filtering, protection, ignore/block, batch operations, cleanup)
 - [x] Channel Management (Create, join, share via QR, delete with auto-cleanup)
 - [x] Public Channels (# prefix support, auto-key generation)
 - [x] Message Archiving (Daily archiving with browse-by-date selector)
 - [x] Smart Notifications (Unread counters per channel and total)
-- [x] Direct Messages (DM) - Private messaging with delivery status tracking
-- [x] Advanced Contact Management - Multi-page interface with sorting, filtering
+- [x] Direct Messages (DM) - Searchable contact selector, delivery tracking, configurable retry, multi-path routing
+- [x] Global Message Search - Full-text search across channels and DMs (FTS5)
 - [x] Message Content Enhancements - Mention badges, clickable URLs, image previews
 - [x] @Mentions Autocomplete - Type @ to get contact suggestions with fuzzy search
 - [x] PWA Notifications (Experimental) - Browser notifications and app badge counters
 - [x] Full Offline Support - Local Bootstrap libraries and Service Worker caching
-- [x] Interactive Console - Direct meshcli access via WebSocket with command history
-- [x] Contact Map - View contacts with GPS coordinates on OpenStreetMap (Leaflet)
+- [x] Interactive Console - Full MeshCore command suite (repeater, contact, device, channel management)
+- [x] Contact Map - View contacts and own device on OpenStreetMap (Leaflet)
 - [x] Echo Tracking - "Heard X repeats" badge for sent channel messages
+- [x] MeshCore Analyzer - Packet analysis links on channel messages (analyzer.letsmesh.net)
+- [x] DM Delivery Tracking - ACK-based delivery checkmarks with SNR/route details
+- [x] Device Dashboard - Device info, statistics, and contact sharing (QR/URI)
+- [x] Settings Modal - DM retry parameters, message retention, and dark/light theme
+- [x] System Log - Real-time log viewer with streaming
+- [x] Database Backup - Create, list, and download backups from the UI
+- [x] Desktop Sidebar - Channel/contact sidebar for wide screens (tablet/desktop)
+- [x] Dark/Light Theme - Toggle between dark and light UI themes
+- [x] Multi-device Support - Database file named after device public key
 
 ### Next Steps
 
 - [ ] Performance Optimization - Frontend and backend improvements
 - [ ] Enhanced Testing - Unit and integration tests
-- [ ] Documentation Polish - API docs and usage guides
 
 ---
 
@@ -342,7 +368,7 @@ This is an open-source project. Contributions are welcome!
 ## References
 
 - [MeshCore Documentation](https://meshcore.org)
-- [meshcore-cli GitHub](https://github.com/meshcore-dev/meshcore-cli)
+- [meshcore Python library](https://pypi.org/project/meshcore/)
 
 ---
 

app/archiver/__init__.py

@@ -6,12 +6,16 @@ from app.archiver.manager import (
     archive_messages,
     list_archives,
     get_archive_path,
-    schedule_daily_archiving
+    schedule_daily_archiving,
+    schedule_retention,
+    init_retention_schedule
 )
 
 __all__ = [
     'archive_messages',
     'list_archives',
     'get_archive_path',
-    'schedule_daily_archiving'
+    'schedule_daily_archiving',
+    'schedule_retention',
+    'init_retention_schedule'
 ]

app/archiver/manager.py

@@ -20,6 +20,11 @@ _scheduler: Optional[BackgroundScheduler] = None
 
 # Job IDs
 CLEANUP_JOB_ID = 'daily_cleanup'
+RETENTION_JOB_ID = 'daily_retention'
+BACKUP_JOB_ID = 'daily_backup'
+
+# Module-level db reference (set by init_retention_schedule)
+_db = None
 
 
 def get_local_timezone_name() -> str:
@@ -291,7 +296,7 @@ def _cleanup_job():
             return
 
         # Convert to list format (same as preview-cleanup endpoint)
-        type_labels = {1: 'CLI', 2: 'REP', 3: 'ROOM', 4: 'SENS'}
+        type_labels = {1: 'COM', 2: 'REP', 3: 'ROOM', 4: 'SENS'}
         contacts = []
         for public_key, details in contacts_detailed.items():
             contacts.append({
@@ -460,6 +465,103 @@ def init_cleanup_schedule():
         logger.error(f"Error initializing cleanup schedule: {e}", exc_info=True)
 
 
+def _retention_job():
+    """Background job that runs daily to delete old messages from DB."""
+    logger.info("Running daily retention job...")
+
+    try:
+        from app.routes.api import get_retention_settings
+
+        settings = get_retention_settings()
+
+        if not settings.get('enabled'):
+            logger.info("Message retention is disabled, skipping")
+            return
+
+        if _db is None:
+            logger.error("Database not available for retention job")
+            return
+
+        days = settings.get('days', 90)
+        include_dms = settings.get('include_dms', False)
+        include_adverts = settings.get('include_adverts', False)
+
+        result = _db.cleanup_old_messages(
+            days=days,
+            include_dms=include_dms,
+            include_adverts=include_adverts
+        )
+
+        total = sum(result.values())
+        logger.info(f"Retention job completed: {total} rows deleted ({result})")
+
+    except Exception as e:
+        logger.error(f"Retention job failed: {e}", exc_info=True)
+
+
+def schedule_retention(enabled: bool, hour: int = 2) -> bool:
+    """Add or remove the retention job from the scheduler."""
+    global _scheduler
+
+    if _scheduler is None:
+        logger.warning("Scheduler not initialized, cannot schedule retention")
+        return False
+
+    try:
+        if enabled:
+            if not isinstance(hour, int) or hour < 0 or hour > 23:
+                hour = 2
+
+            trigger = CronTrigger(hour=hour, minute=30)
+
+            _scheduler.add_job(
+                func=_retention_job,
+                trigger=trigger,
+                id=RETENTION_JOB_ID,
+                name='Daily Message Retention',
+                replace_existing=True
+            )
+
+            tz_name = get_local_timezone_name()
+            logger.info(f"Retention job scheduled - will run daily at {hour:02d}:30 ({tz_name})")
+        else:
+            try:
+                _scheduler.remove_job(RETENTION_JOB_ID)
+                logger.info("Retention job removed from scheduler")
+            except Exception:
+                pass
+
+        return True
+
+    except Exception as e:
+        logger.error(f"Error scheduling retention: {e}", exc_info=True)
+        return False
+
+
+def init_retention_schedule(db=None):
+    """Initialize retention schedule from saved settings. Call at startup."""
+    global _db
+
+    if db is not None:
+        _db = db
+
+    try:
+        from app.routes.api import get_retention_settings
+
+        settings = get_retention_settings()
+
+        if settings.get('enabled'):
+            hour = settings.get('hour', 2)
+            schedule_retention(enabled=True, hour=hour)
+            tz_name = get_local_timezone_name()
+            logger.info(f"Message retention enabled from saved settings (hour={hour:02d}:30 {tz_name})")
+        else:
+            logger.info("Message retention is disabled in saved settings")
+
+    except Exception as e:
+        logger.error(f"Error initializing retention schedule: {e}", exc_info=True)
+
+
 def schedule_daily_archiving():
     """
     Initialize and start the background scheduler for daily archiving.
@@ -502,10 +604,60 @@ def schedule_daily_archiving():
         # Initialize cleanup schedule from saved settings
         init_cleanup_schedule()
 
+        # Initialize backup schedule
+        init_backup_schedule()
+
     except Exception as e:
         logger.error(f"Failed to start archive scheduler: {e}", exc_info=True)
 
 
+def init_backup_schedule():
+    """Initialize daily backup job from config."""
+    global _scheduler
+
+    if _scheduler is None:
+        return
+
+    if not config.MC_BACKUP_ENABLED:
+        logger.info("Backup is disabled in configuration")
+        return
+
+    try:
+        backup_hour = config.MC_BACKUP_HOUR
+        trigger = CronTrigger(hour=backup_hour, minute=0)
+        backup_dir = Path(config.MC_CONFIG_DIR) / 'backups'
+
+        _scheduler.add_job(
+            func=_backup_job,
+            trigger=trigger,
+            id=BACKUP_JOB_ID,
+            name='Daily Database Backup',
+            replace_existing=True,
+            args=[backup_dir]
+        )
+        logger.info(f"Backup schedule initialized: daily at {backup_hour:02d}:00")
+    except Exception as e:
+        logger.error(f"Error scheduling backup: {e}", exc_info=True)
+
+
+def _backup_job(backup_dir):
+    """Execute daily backup and cleanup old backups."""
+    global _db
+    if _db is None:
+        logger.warning("No database reference for backup")
+        return
+
+    try:
+        backup_path = _db.create_backup(backup_dir)
+        logger.info(f"Daily backup completed: {backup_path}")
+
+        removed = _db.cleanup_old_backups(backup_dir, config.MC_BACKUP_RETENTION_DAYS)
+        if removed > 0:
+            logger.info(f"Cleaned up {removed} old backup(s)")
+    except Exception as e:
+        logger.error(f"Backup job failed: {e}", exc_info=True)
+
+
 def stop_scheduler():
     """
     Stop the background scheduler.

app/config.py

@@ -18,14 +18,29 @@ class Config:
     MC_DEVICE_NAME = os.getenv('MC_DEVICE_NAME', 'MeshCore')
     MC_CONFIG_DIR = os.getenv('MC_CONFIG_DIR', '/root/.config/meshcore')
 
-    # MeshCore Bridge configuration
-    MC_BRIDGE_URL = os.getenv('MC_BRIDGE_URL', 'http://meshcore-bridge:5001/cli')
+    # MC_BRIDGE_URL removed in v2 (direct device communication)
 
-    # Archive configuration
+    # Archive configuration (v1 — archives move to SQLite in v2)
     MC_ARCHIVE_DIR = os.getenv('MC_ARCHIVE_DIR', '/root/.archive/meshcore')
     MC_ARCHIVE_ENABLED = os.getenv('MC_ARCHIVE_ENABLED', 'true').lower() == 'true'
     MC_ARCHIVE_RETENTION_DAYS = int(os.getenv('MC_ARCHIVE_RETENTION_DAYS', '7'))
 
+    # v2: Database
+    MC_DB_PATH = os.getenv('MC_DB_PATH', '')  # empty = auto: {MC_CONFIG_DIR}/mc_{pubkey_prefix}.db
+
+    # v2: TCP connection (alternative to serial, e.g. meshcore-proxy)
+    MC_TCP_HOST = os.getenv('MC_TCP_HOST', '')  # empty = use serial
+    MC_TCP_PORT = int(os.getenv('MC_TCP_PORT', '5555'))
+
+    # v2: Backup
+    MC_BACKUP_ENABLED = os.getenv('MC_BACKUP_ENABLED', 'true').lower() == 'true'
+    MC_BACKUP_HOUR = int(os.getenv('MC_BACKUP_HOUR', '2'))
+    MC_BACKUP_RETENTION_DAYS = int(os.getenv('MC_BACKUP_RETENTION_DAYS', '7'))
+
+    # v2: Connection
+    MC_AUTO_RECONNECT = os.getenv('MC_AUTO_RECONNECT', 'true').lower() == 'true'
+    MC_LOG_LEVEL = os.getenv('MC_LOG_LEVEL', 'INFO')
+
     # Flask server configuration
     FLASK_HOST = os.getenv('FLASK_HOST', '0.0.0.0')
     FLASK_PORT = int(os.getenv('FLASK_PORT', '5000'))
@@ -42,10 +57,23 @@ class Config:
         """Get the full path to archive directory"""
         return Path(self.MC_ARCHIVE_DIR)
 
+    @property
+    def db_path(self) -> Path:
+        """Get SQLite database path"""
+        if self.MC_DB_PATH:
+            return Path(self.MC_DB_PATH)
+        return Path(self.MC_CONFIG_DIR) / 'mc-webui.db'
+
+    @property
+    def use_tcp(self) -> bool:
+        """True if TCP transport should be used instead of serial"""
+        return bool(self.MC_TCP_HOST)
+
     def __repr__(self):
+        transport = f"tcp={self.MC_TCP_HOST}:{self.MC_TCP_PORT}" if self.use_tcp else f"serial={self.MC_SERIAL_PORT}"
         return (
             f"Config(device={self.MC_DEVICE_NAME}, "
-            f"port={self.MC_SERIAL_PORT}, "
+            f"{transport}, "
             f"config_dir={self.MC_CONFIG_DIR})"
         )
 

app/contacts_cache.py

@@ -0,0 +1,110 @@
+"""
+Contacts Cache - DB-backed contact name/key lookup.
+
+All contact data is stored in the SQLite contacts table.
+JSONL files are no longer used.
+
+Kept for backward compatibility: get_all_names(), get_all_contacts(),
+parse_advert_payload().
+"""
+
+import logging
+import math
+import struct
+from flask import current_app
+
+logger = logging.getLogger(__name__)
+
+_TYPE_LABELS = {0: 'COM', 1: 'COM', 2: 'REP', 3: 'ROOM', 4: 'SENS'}
+
+
+def _get_db():
+    """Get database instance from Flask app context."""
+    return getattr(current_app, 'db', None)
+
+
+def get_all_contacts() -> list:
+    """Get all known contacts from DB."""
+    try:
+        db = _get_db()
+        if db:
+            contacts = db.get_contacts()
+            return [{
+                'public_key': c.get('public_key', ''),
+                'name': c.get('name', ''),
+                'first_seen': c.get('first_seen', ''),
+                'last_seen': c.get('last_seen', ''),
+                'source': c.get('source', ''),
+                'lat': c.get('adv_lat', 0.0) or 0.0,
+                'lon': c.get('adv_lon', 0.0) or 0.0,
+                'type_label': _TYPE_LABELS.get(c.get('type', 1), 'UNKNOWN'),
+            } for c in contacts]
+    except Exception as e:
+        logger.error(f"Failed to get contacts: {e}")
+    return []
+
+
+def get_all_names() -> list:
+    """Get all unique non-empty contact names sorted alphabetically."""
+    try:
+        db = _get_db()
+        if db:
+            contacts = db.get_contacts()
+            return sorted(set(c.get('name', '') for c in contacts if c.get('name')))
+    except Exception as e:
+        logger.error(f"Failed to get contact names: {e}")
+    return []
+
+
+def parse_advert_payload(pkt_payload_hex: str):
+    """
+    Parse advert pkt_payload to extract public_key, node_name, and GPS coordinates.
+
+    Layout of pkt_payload (byte offsets):
+      [0:32]   Public Key (32 bytes = 64 hex chars)
+      [32:36]  Timestamp (4 bytes)
+      [36:100] Signature (64 bytes)
+      [100]    App Flags (1 byte) - bit 4: Location, bit 7: Name
+      [101+]   If Location (bit 4): Lat (4 bytes, LE int32/1e6) + Lon (4 bytes, LE int32/1e6)
+               If Name (bit 7): Node name (UTF-8, variable length)
+
+    Returns:
+        (public_key_hex, node_name, lat, lon) or (None, None, 0, 0) on failure
+    """
+    try:
+        raw = bytes.fromhex(pkt_payload_hex)
+        if len(raw) < 101:
+            return None, None, 0.0, 0.0
+
+        public_key = pkt_payload_hex[:64].lower()
+        app_flags = raw[100]
+
+        has_location = bool(app_flags & 0x10)  # bit 4
+        has_name = bool(app_flags & 0x80)      # bit 7
+
+        lat, lon = 0.0, 0.0
+        name_offset = 101
+
+        if has_location:
+            if len(raw) >= 109:
+                lat_i, lon_i = struct.unpack('<ii', raw[101:109])
+                lat, lon = lat_i / 1e6, lon_i / 1e6
+                # Validate: discard NaN, Infinity, and out-of-range values
+                if (math.isnan(lat) or math.isnan(lon) or
+                        math.isinf(lat) or math.isinf(lon) or
+                        not (-90 <= lat <= 90) or not (-180 <= lon <= 180)):
+                    lat, lon = 0.0, 0.0
+            name_offset += 8
+
+        if not has_name:
+            return public_key, None, lat, lon
+
+        if name_offset >= len(raw):
+            return public_key, None, lat, lon
+
+        name_bytes = raw[name_offset:]
+        node_name = name_bytes.decode('utf-8', errors='replace').rstrip('\x00')
+
+        return public_key, node_name if node_name else None, lat, lon
+    except Exception:
+        return None, None, 0.0, 0.0

app/log_handler.py

@@ -0,0 +1,84 @@
+"""
+In-memory ring buffer log handler with WebSocket broadcast.
+
+Captures Python log records into a fixed-size deque and optionally
+broadcasts them to connected SocketIO clients in real-time.
+"""
+
+import logging
+from collections import deque
+from datetime import datetime
+from threading import Lock
+
+
+class MemoryLogHandler(logging.Handler):
+    """Logging handler that stores records in a ring buffer and broadcasts via SocketIO."""
+
+    def __init__(self, capacity=2000, socketio=None):
+        super().__init__()
+        self.capacity = capacity
+        self.buffer = deque(maxlen=capacity)
+        self.socketio = socketio
+        self._lock = Lock()
+        self._seq = 0  # monotonic sequence for client catch-up
+
+    def emit(self, record):
+        try:
+            entry = self._format_record(record)
+            with self._lock:
+                self._seq += 1
+                entry['seq'] = self._seq
+                self.buffer.append(entry)
+
+            # Broadcast to connected clients
+            if self.socketio:
+                self.socketio.emit('log_entry', entry, namespace='/logs')
+        except Exception:
+            self.handleError(record)
+
+    def _format_record(self, record):
+        """Convert LogRecord to a serializable dict."""
+        return {
+            'timestamp': datetime.fromtimestamp(record.created).strftime('%Y-%m-%d %H:%M:%S.%f')[:-3],
+            'level': record.levelname,
+            'logger': record.name,
+            'message': record.getMessage(),
+        }
+
+    def get_entries(self, level=None, logger_filter=None, search=None, limit=None):
+        """Return filtered log entries from the buffer.
+
+        Args:
+            level: Minimum log level name (DEBUG, INFO, WARNING, ERROR, CRITICAL)
+            logger_filter: Logger name prefix filter (e.g. 'app.device_manager')
+            search: Text search in message (case-insensitive)
+            limit: Max entries to return (newest first before limit, returned in chronological order)
+
+        Returns:
+            List of log entry dicts
+        """
+        level_num = getattr(logging, level.upper(), 0) if level else 0
+        search_lower = search.lower() if search else None
+
+        with self._lock:
+            entries = list(self.buffer)
+
+        # Apply filters
+        if level_num > 0:
+            entries = [e for e in entries if getattr(logging, e['level'], 0) >= level_num]
+        if logger_filter:
+            entries = [e for e in entries if e['logger'].startswith(logger_filter)]
+        if search_lower:
+            entries = [e for e in entries if search_lower in e['message'].lower()]
+
+        # Limit (return newest N, in chronological order)
+        if limit and limit > 0 and len(entries) > limit:
+            entries = entries[-limit:]
+
+        return entries
+
+    def get_loggers(self):
+        """Return sorted list of unique logger names seen in the buffer."""
+        with self._lock:
+            loggers = sorted({e['logger'] for e in self.buffer})
+        return loggers

app/meshcore/parser.py

@@ -37,7 +37,8 @@ def parse_message(line: Dict, allowed_channels: Optional[List[int]] = None) -> O
         return None
 
     timestamp = line.get('timestamp', 0)
-    text = line.get('text', '').strip()
+    raw_text = line.get('text', '')
+    text = raw_text.strip()
 
     if not text:
         return None
@@ -69,7 +70,10 @@ def parse_message(line: Dict, allowed_channels: Optional[List[int]] = None) -> O
         'is_own': is_own,
         'snr': line.get('SNR'),
         'path_len': line.get('path_len'),
-        'channel_idx': channel_idx
+        'channel_idx': channel_idx,
+        'sender_timestamp': line.get('sender_timestamp'),
+        'txt_type': line.get('txt_type', 0),
+        'raw_text': raw_text
     }
 
 
@@ -440,7 +444,8 @@ def _parse_sent_msg(line: Dict) -> Optional[Dict]:
         'is_own': True,
         'txt_type': txt_type,
         'conversation_id': conversation_id,
-        'dedup_key': dedup_key
+        'dedup_key': dedup_key,
+        'expected_ack': line.get('expected_ack'),
     }
 
 
@@ -554,6 +559,26 @@ def read_dm_messages(
     return messages, pubkey_to_name
 
 
+def dedup_retry_messages(messages: List[Dict], window_seconds: int = 300) -> List[Dict]:
+    """Collapse outgoing messages with same text+recipient within a time window.
+
+    Auto-retry sends multiple SENT_MSG entries for the same message.
+    This keeps only the first occurrence and drops duplicates within the window.
+    """
+    deduped = []
+    seen_outgoing = {}  # (recipient, text) -> earliest timestamp
+    for msg in messages:
+        if msg.get('direction') == 'outgoing':
+            key = (msg.get('recipient', ''), msg.get('content', ''))
+            ts = msg.get('timestamp', 0)
+            prev_ts = seen_outgoing.get(key)
+            if prev_ts is not None and abs(ts - prev_ts) <= window_seconds:
+                continue
+            seen_outgoing[key] = ts
+        deduped.append(msg)
+    return deduped
+
+
 def get_dm_conversations(days: Optional[int] = 7) -> List[Dict]:
     """
     Get list of DM conversations with metadata.
@@ -577,6 +602,8 @@ def get_dm_conversations(days: Optional[int] = 7) -> List[Dict]:
     """
     messages, pubkey_to_name = read_dm_messages(days=days)
 
+    messages = dedup_retry_messages(messages)
+
     # Build reverse mapping: name -> pubkey_prefix
     name_to_pubkey = {name: pk for pk, name in pubkey_to_name.items()}
 

app/migrate_v1.py

@@ -0,0 +1,359 @@
+"""
+Migrate v1 data (.msgs JSONL) into v2 SQLite database.
+
+Runs automatically on startup if .msgs file exists and database is empty.
+Can also be run manually: python -m app.migrate_v1
+
+Migrates:
+- Live .msgs file (today's messages)
+- Archive .msgs files (historical messages)
+- Channel messages (CHAN, SENT_CHAN)
+- Direct messages (PRIV, SENT_MSG)
+"""
+
+import json
+import logging
+from pathlib import Path
+from typing import Optional, List
+
+logger = logging.getLogger(__name__)
+
+
+def _find_msgs_file(data_dir: Path, device_name: str) -> Optional[Path]:
+    """Find the live .msgs file for the given device name."""
+    msgs_file = data_dir / f"{device_name}.msgs"
+    if msgs_file.exists():
+        return msgs_file
+
+    # Try to find any .msgs file in the data dir
+    candidates = list(data_dir.glob("*.msgs"))
+    # Exclude archive files (pattern: name.YYYY-MM-DD.msgs)
+    live_files = [f for f in candidates if f.stem.count('.') == 0]
+    if len(live_files) == 1:
+        return live_files[0]
+
+    return None
+
+
+def _find_archive_files(data_dir: Path, device_name: str) -> List[Path]:
+    """Find all archive .msgs files, sorted oldest first."""
+    archive_files = []
+
+    # Check common archive locations
+    archive_dirs = [
+        data_dir / 'archive',          # /data/archive/
+        data_dir.parent / 'archive',   # sibling archive dir
+    ]
+
+    for archive_dir in archive_dirs:
+        if archive_dir.exists():
+            # Pattern: DeviceName.YYYY-MM-DD.msgs
+            for f in archive_dir.glob(f"{device_name}.*.msgs"):
+                # Validate it's an archive file (has date in name)
+                parts = f.stem.split('.')
+                if len(parts) >= 2:
+                    archive_files.append(f)
+
+    # Also check data_dir itself for archives
+    for f in data_dir.glob(f"{device_name}.*.msgs"):
+        parts = f.stem.split('.')
+        if len(parts) >= 2 and f not in archive_files:
+            archive_files.append(f)
+
+    # Sort by filename (which sorts by date since format is Name.YYYY-MM-DD)
+    archive_files.sort(key=lambda f: f.name)
+
+    return archive_files
+
+
+def migrate_v1_data(db, data_dir: Path, device_name: str) -> dict:
+    """
+    Import v1 .msgs data into v2 SQLite database.
+    Imports both live .msgs file and all archive files.
+
+    Args:
+        db: Database instance
+        data_dir: Path to meshcore config dir containing .msgs file
+        device_name: Device name (used for .msgs filename and own message detection)
+
+    Returns:
+        dict with migration stats
+    """
+    stats = {
+        'channel_messages': 0,
+        'direct_messages': 0,
+        'skipped': 0,
+        'errors': 0,
+        'files_processed': 0,
+    }
+
+    # Collect all files to import: archives first (oldest), then live
+    files_to_import = []
+
+    archive_files = _find_archive_files(data_dir, device_name)
+    if archive_files:
+        files_to_import.extend(archive_files)
+        logger.info(f"Found {len(archive_files)} archive files to migrate")
+
+    live_file = _find_msgs_file(data_dir, device_name)
+    if live_file:
+        files_to_import.append(live_file)
+
+    if not files_to_import:
+        logger.info("No .msgs files found, skipping v1 migration")
+        return {'status': 'skipped', 'reason': 'no_msgs_files'}
+
+    logger.info(f"Starting v1 data migration: {len(files_to_import)} files to process")
+
+    # Track seen timestamps+text to avoid duplicates across archive and live file
+    seen_channel = set()
+    seen_dm = set()
+
+    for msgs_file in files_to_import:
+        file_stats = _import_msgs_file(
+            db, msgs_file, device_name, seen_channel, seen_dm
+        )
+        stats['channel_messages'] += file_stats['channel_messages']
+        stats['direct_messages'] += file_stats['direct_messages']
+        stats['skipped'] += file_stats['skipped']
+        stats['errors'] += file_stats['errors']
+        stats['files_processed'] += 1
+
+    stats['status'] = 'completed'
+    logger.info(
+        f"v1 migration complete: {stats['files_processed']} files, "
+        f"{stats['channel_messages']} channel msgs, "
+        f"{stats['direct_messages']} DMs, {stats['skipped']} skipped, "
+        f"{stats['errors']} errors"
+    )
+    return stats
+
+
+def _import_msgs_file(db, msgs_file: Path, device_name: str,
+                      seen_channel: set, seen_dm: set) -> dict:
+    """Import a single .msgs file. Returns per-file stats."""
+    stats = {'channel_messages': 0, 'direct_messages': 0, 'skipped': 0, 'errors': 0}
+
+    logger.info(f"Importing {msgs_file.name}...")
+
+    try:
+        lines = msgs_file.read_text(encoding='utf-8', errors='replace').splitlines()
+    except Exception as e:
+        logger.error(f"Failed to read {msgs_file}: {e}")
+        stats['errors'] += 1
+        return stats
+
+    for line_num, raw_line in enumerate(lines, 1):
+        raw_line = raw_line.strip()
+        if not raw_line:
+            continue
+
+        try:
+            entry = json.loads(raw_line)
+        except json.JSONDecodeError:
+            stats['errors'] += 1
+            continue
+
+        msg_type = entry.get('type')
+
+        try:
+            if msg_type in ('CHAN', 'SENT_CHAN'):
+                # Dedup key: timestamp + first 50 chars of text
+                ts = entry.get('timestamp', 0)
+                text = entry.get('text', '')[:50]
+                dedup = (ts, text)
+                if dedup in seen_channel:
+                    stats['skipped'] += 1
+                    continue
+                seen_channel.add(dedup)
+
+                _migrate_channel_msg(db, entry, device_name)
+                stats['channel_messages'] += 1
+            elif msg_type == 'PRIV':
+                ts = entry.get('timestamp', 0)
+                text = entry.get('text', '')[:50]
+                dedup = (ts, text)
+                if dedup in seen_dm:
+                    stats['skipped'] += 1
+                    continue
+                seen_dm.add(dedup)
+
+                _migrate_dm_incoming(db, entry)
+                stats['direct_messages'] += 1
+            elif msg_type == 'SENT_MSG':
+                if entry.get('txt_type', 0) == 0:  # Only private messages
+                    ts = entry.get('timestamp', 0)
+                    text = entry.get('text', '')[:50]
+                    dedup = (ts, text)
+                    if dedup in seen_dm:
+                        stats['skipped'] += 1
+                        continue
+                    seen_dm.add(dedup)
+
+                    _migrate_dm_outgoing(db, entry, device_name)
+                    stats['direct_messages'] += 1
+                else:
+                    stats['skipped'] += 1
+            else:
+                stats['skipped'] += 1
+        except Exception as e:
+            stats['errors'] += 1
+            if stats['errors'] <= 5:
+                logger.warning(f"Migration error in {msgs_file.name} line {line_num}: {e}")
+
+    logger.info(
+        f"  {msgs_file.name}: {stats['channel_messages']} chan, "
+        f"{stats['direct_messages']} DMs, {stats['skipped']} skip, "
+        f"{stats['errors']} err"
+    )
+    return stats
+
+
+def _migrate_channel_msg(db, entry: dict, device_name: str):
+    """Migrate a CHAN or SENT_CHAN entry."""
+    raw_text = entry.get('text', '').strip()
+    if not raw_text:
+        return
+
+    is_own = entry.get('type') == 'SENT_CHAN'
+    channel_idx = entry.get('channel_idx', 0)
+    timestamp = entry.get('timestamp', 0)
+
+    if is_own:
+        sender = entry.get('sender', device_name)
+        content = raw_text
+    else:
+        # Parse sender from "SenderName: message" format
+        if ':' in raw_text:
+            sender, content = raw_text.split(':', 1)
+            sender = sender.strip()
+            content = content.strip()
+        else:
+            sender = 'Unknown'
+            content = raw_text
+
+    db.insert_channel_message(
+        channel_idx=channel_idx,
+        sender=sender,
+        content=content,
+        timestamp=timestamp,
+        sender_timestamp=entry.get('sender_timestamp'),
+        is_own=is_own,
+        txt_type=entry.get('txt_type', 0),
+        snr=entry.get('SNR'),
+        path_len=entry.get('path_len'),
+        pkt_payload=entry.get('pkt_payload'),
+        raw_json=json.dumps(entry, default=str),
+    )
+
+
+def _migrate_dm_incoming(db, entry: dict):
+    """Migrate a PRIV (incoming DM) entry."""
+    text = entry.get('text', '').strip()
+    if not text:
+        return
+
+    pubkey_prefix = entry.get('pubkey_prefix', '')
+    sender_name = entry.get('name', '')
+
+    # Resolve prefix to full key if contact exists
+    contact_key = pubkey_prefix if pubkey_prefix else None
+    if contact_key:
+        contact_key = _resolve_pubkey(db, contact_key)
+
+    # Create/update contact with sender name from v1 data
+    if contact_key and sender_name:
+        db.upsert_contact(
+            public_key=contact_key,
+            name=sender_name,
+            source='message',
+        )
+
+    db.insert_direct_message(
+        contact_pubkey=contact_key,
+        direction='in',
+        content=text,
+        timestamp=entry.get('timestamp', 0),
+        sender_timestamp=entry.get('sender_timestamp'),
+        txt_type=entry.get('txt_type', 0),
+        snr=entry.get('SNR'),
+        path_len=entry.get('path_len'),
+        pkt_payload=entry.get('pkt_payload'),
+        raw_json=json.dumps(entry, default=str),
+    )
+
+
+def _migrate_dm_outgoing(db, entry: dict, device_name: str):
+    """Migrate a SENT_MSG (outgoing DM) entry."""
+    text = entry.get('text', '').strip()
+    if not text:
+        return
+
+    # For outgoing DMs, we don't have recipient pubkey in v1 data.
+    # In v1, conversation_id was "name_{recipient}" — we store the name
+    # in raw_json for reference.
+    recipient = entry.get('recipient', entry.get('name', ''))
+
+    # Try to find pubkey from contacts table by recipient name
+    contact_pubkey = _lookup_pubkey_by_name(db, recipient)
+
+    db.insert_direct_message(
+        contact_pubkey=contact_pubkey,
+        direction='out',
+        content=text,
+        timestamp=entry.get('timestamp', 0),
+        sender_timestamp=entry.get('sender_timestamp'),
+        txt_type=entry.get('txt_type', 0),
+        expected_ack=entry.get('expected_ack'),
+        pkt_payload=entry.get('pkt_payload'),
+        raw_json=json.dumps(entry, default=str),
+    )
+
+
+def _resolve_pubkey(db, pubkey_prefix: str) -> Optional[str]:
+    """Check if a pubkey prefix matches a contact. Returns full key or None."""
+    if not pubkey_prefix:
+        return None
+    try:
+        contacts = db.get_contacts()
+        prefix = pubkey_prefix.lower()
+        for c in contacts:
+            pk = (c.get('public_key') or '').lower()
+            if pk and pk.startswith(prefix):
+                return pk
+    except Exception:
+        pass
+    return None
+
+
+def _lookup_pubkey_by_name(db, name: str) -> Optional[str]:
+    """Look up a contact's public_key by name. Returns None if not found."""
+    if not name:
+        return None
+    try:
+        contacts = db.get_contacts()
+        for c in contacts:
+            if c.get('name') == name:
+                return c.get('public_key')
+    except Exception:
+        pass
+    return None
+
+
+def should_migrate(db, data_dir: Path, device_name: str) -> bool:
+    """Check if migration is needed: .msgs files exist and DB has no messages."""
+    # Check for live file
+    has_live = _find_msgs_file(data_dir, device_name) is not None
+    # Check for archive files
+    has_archives = len(_find_archive_files(data_dir, device_name)) > 0
+
+    if not has_live and not has_archives:
+        return False
+
+    # Only migrate if DB is empty (no channel messages and no DMs)
+    try:
+        stats = db.get_stats()
+        total = stats.get('channel_messages', 0) + stats.get('direct_messages', 0)
+        return total == 0
+    except Exception:
+        return False

app/read_status.py

@@ -1,198 +1,143 @@
 """
-Read Status Manager - Server-side storage for message read status
+Read Status Manager - DB-backed storage for message read status
 
 Manages the last seen timestamps for channels and DM conversations,
 providing cross-device synchronization for unread message tracking.
+All data is stored in the read_status table of the SQLite database.
 """
 
-import json
 import logging
-import os
-from pathlib import Path
-from threading import Lock
-from app.config import config
+from flask import current_app
 
 logger = logging.getLogger(__name__)
 
-# Thread-safe lock for file operations
-_status_lock = Lock()
 
-# Path to read status file
-READ_STATUS_FILE = Path(config.MC_CONFIG_DIR) / '.read_status.json'
-
-
-def _get_default_status():
-    """Get default read status structure"""
-    return {
-        'channels': {},  # {"0": timestamp, "1": timestamp, ...}
-        'dm': {}         # {"name_User1": timestamp, "pk_abc123": timestamp, ...}
-    }
+def _get_db():
+    """Get database instance from Flask app context."""
+    return getattr(current_app, 'db', None)
 
 
 def load_read_status():
-    """
-    Load read status from disk.
+    """Load read status from database.
 
     Returns:
-        dict: Read status with 'channels' and 'dm' keys
+        dict: Read status with 'channels', 'dm', and 'muted_channels' keys
     """
-    with _status_lock:
-        try:
-            if not READ_STATUS_FILE.exists():
-                logger.info("Read status file does not exist, creating default")
-                return _get_default_status()
+    try:
+        db = _get_db()
+        rows = db.get_read_status()
 
-            with open(READ_STATUS_FILE, 'r', encoding='utf-8') as f:
-                status = json.load(f)
+        channels = {}
+        dm = {}
+        muted_channels = []
 
-            # Validate structure
-            if not isinstance(status, dict):
-                logger.warning("Invalid read status structure, resetting")
-                return _get_default_status()
+        for key, row in rows.items():
+            if key.startswith('chan_'):
+                chan_idx = key[5:]  # "chan_0" -> "0"
+                channels[chan_idx] = row['last_seen_ts']
+                if row.get('is_muted'):
+                    try:
+                        muted_channels.append(int(chan_idx))
+                    except ValueError:
+                        pass
+            elif key.startswith('dm_'):
+                conv_id = key[3:]  # "dm_name_User1" -> "name_User1"
+                dm[conv_id] = row['last_seen_ts']
 
-            # Ensure both keys exist
-            if 'channels' not in status:
-                status['channels'] = {}
-            if 'dm' not in status:
-                status['dm'] = {}
+        return {
+            'channels': channels,
+            'dm': dm,
+            'muted_channels': muted_channels,
+        }
 
-            logger.debug(f"Loaded read status: {len(status['channels'])} channels, {len(status['dm'])} DM conversations")
-            return status
-
-        except json.JSONDecodeError as e:
-            logger.error(f"Failed to parse read status file: {e}")
-            return _get_default_status()
-        except Exception as e:
-            logger.error(f"Error loading read status: {e}")
-            return _get_default_status()
+    except Exception as e:
+        logger.error(f"Error loading read status: {e}")
+        return {'channels': {}, 'dm': {}, 'muted_channels': []}
 
 
 def save_read_status(status):
-    """
-    Save read status to disk.
-
-    Args:
-        status (dict): Read status with 'channels' and 'dm' keys
-
-    Returns:
-        bool: True if successful, False otherwise
-    """
-    with _status_lock:
-        try:
-            # Ensure directory exists
-            READ_STATUS_FILE.parent.mkdir(parents=True, exist_ok=True)
-
-            # Write atomically (write to temp file, then rename)
-            temp_file = READ_STATUS_FILE.with_suffix('.tmp')
-            with open(temp_file, 'w', encoding='utf-8') as f:
-                json.dump(status, f, indent=2)
-
-            # Atomic rename
-            temp_file.replace(READ_STATUS_FILE)
-
-            logger.debug(f"Saved read status: {len(status['channels'])} channels, {len(status['dm'])} DM conversations")
-            return True
-
-        except Exception as e:
-            logger.error(f"Error saving read status: {e}")
-            return False
+    """No-op — data is written per-operation via mark_* functions."""
+    return True
 
 
 def mark_channel_read(channel_idx, timestamp):
-    """
-    Mark a channel as read up to a specific timestamp.
-
-    Args:
-        channel_idx (int or str): Channel index (will be converted to string)
-        timestamp (int or float): Unix timestamp of last read message
-
-    Returns:
-        bool: True if successful, False otherwise
-    """
+    """Mark a channel as read up to a specific timestamp."""
     try:
-        # Load current status
-        status = load_read_status()
-
-        # Update channel timestamp (ensure key is string for JSON compatibility)
-        channel_key = str(channel_idx)
-        status['channels'][channel_key] = int(timestamp)
-
-        # Save updated status
-        success = save_read_status(status)
-
-        if success:
-            logger.debug(f"Marked channel {channel_idx} as read at timestamp {timestamp}")
-
-        return success
-
+        db = _get_db()
+        db.mark_read(f"chan_{channel_idx}", int(timestamp))
+        logger.debug(f"Marked channel {channel_idx} as read at timestamp {timestamp}")
+        return True
     except Exception as e:
         logger.error(f"Error marking channel {channel_idx} as read: {e}")
         return False
 
 
 def mark_dm_read(conversation_id, timestamp):
-    """
-    Mark a DM conversation as read up to a specific timestamp.
-
-    Args:
-        conversation_id (str): Conversation identifier (e.g., "name_User1" or "pk_abc123")
-        timestamp (int or float): Unix timestamp of last read message
-
-    Returns:
-        bool: True if successful, False otherwise
-    """
+    """Mark a DM conversation as read up to a specific timestamp."""
     try:
-        # Load current status
-        status = load_read_status()
-
-        # Update DM timestamp
-        status['dm'][conversation_id] = int(timestamp)
-
-        # Save updated status
-        success = save_read_status(status)
-
-        if success:
-            logger.debug(f"Marked DM conversation {conversation_id} as read at timestamp {timestamp}")
-
-        return success
-
+        db = _get_db()
+        db.mark_read(f"dm_{conversation_id}", int(timestamp))
+        logger.debug(f"Marked DM conversation {conversation_id} as read at timestamp {timestamp}")
+        return True
     except Exception as e:
         logger.error(f"Error marking DM conversation {conversation_id} as read: {e}")
         return False
 
 
 def get_channel_last_seen(channel_idx):
-    """
-    Get last seen timestamp for a specific channel.
-
-    Args:
-        channel_idx (int or str): Channel index
-
-    Returns:
-        int: Unix timestamp, or 0 if never seen
-    """
+    """Get last seen timestamp for a specific channel."""
     try:
         status = load_read_status()
-        channel_key = str(channel_idx)
-        return status['channels'].get(channel_key, 0)
+        return status['channels'].get(str(channel_idx), 0)
     except Exception as e:
         logger.error(f"Error getting last seen for channel {channel_idx}: {e}")
         return 0
 
 
 def get_dm_last_seen(conversation_id):
-    """
-    Get last seen timestamp for a specific DM conversation.
-
-    Args:
-        conversation_id (str): Conversation identifier
-
-    Returns:
-        int: Unix timestamp, or 0 if never seen
-    """
+    """Get last seen timestamp for a specific DM conversation."""
     try:
         status = load_read_status()
         return status['dm'].get(conversation_id, 0)
     except Exception as e:
         logger.error(f"Error getting last seen for DM {conversation_id}: {e}")
         return 0
+
+
+def get_muted_channels():
+    """Get list of muted channel indices."""
+    try:
+        db = _get_db()
+        return db.get_muted_channels()
+    except Exception as e:
+        logger.error(f"Error getting muted channels: {e}")
+        return []
+
+
+def set_channel_muted(channel_idx, muted):
+    """Set mute state for a channel."""
+    try:
+        db = _get_db()
+        db.set_channel_muted(int(channel_idx), muted)
+        logger.info(f"Channel {channel_idx} {'muted' if muted else 'unmuted'}")
+        return True
+    except Exception as e:
+        logger.error(f"Error setting mute for channel {channel_idx}: {e}")
+        return False
+
+
+def mark_all_channels_read(channel_timestamps):
+    """Mark all channels as read in bulk.
+
+    Args:
+        channel_timestamps (dict): {"0": timestamp, "1": timestamp, ...}
+    """
+    try:
+        db = _get_db()
+        for channel_key, timestamp in channel_timestamps.items():
+            db.mark_read(f"chan_{channel_key}", int(timestamp))
+        logger.info(f"Marked {len(channel_timestamps)} channels as read")
+        return True
+    except Exception as e:
+        logger.error(f"Error marking all channels as read: {e}")
+        return False

app/routes/views.py

@@ -50,6 +50,17 @@ def contact_management():
     )
 
 
+@views_bp.route('/contacts/add')
+def contact_add():
+    """
+    Add Contact page - URI paste, QR scan, manual fields.
+    """
+    return render_template(
+        'contacts-add.html',
+        device_name=runtime_config.get_device_name()
+    )
+
+
 @views_bp.route('/contacts/pending')
 def contact_pending_list():
     """
@@ -85,6 +96,12 @@ def console():
     )
 
 
+@views_bp.route('/logs')
+def logs():
+    """System log viewer - real-time log streaming with filters."""
+    return render_template('logs.html')
+
+
 @views_bp.route('/health')
 def health():
     """

app/schema.sql

@@ -0,0 +1,237 @@
+-- mc-webui v2 SQLite Schema
+-- WAL mode and foreign keys are enabled programmatically in Database.__init__
+
+-- Schema versioning for future migrations
+CREATE TABLE IF NOT EXISTS schema_version (
+    version     INTEGER PRIMARY KEY,
+    applied_at  TEXT NOT NULL DEFAULT (datetime('now'))
+);
+INSERT OR IGNORE INTO schema_version (version) VALUES (1);
+
+-- Device identity and settings
+CREATE TABLE IF NOT EXISTS device (
+    id          INTEGER PRIMARY KEY CHECK (id = 1),  -- singleton row
+    public_key  TEXT NOT NULL DEFAULT '',
+    name        TEXT NOT NULL DEFAULT '',
+    created_at  TEXT NOT NULL DEFAULT (datetime('now')),
+    self_info   TEXT  -- JSON blob with full device info
+);
+
+-- All known contacts (replaces contacts_cache.jsonl)
+CREATE TABLE IF NOT EXISTS contacts (
+    public_key      TEXT PRIMARY KEY,       -- hex, lowercase
+    name            TEXT NOT NULL DEFAULT '',
+    type            INTEGER DEFAULT 0,      -- node type from device
+    flags           INTEGER DEFAULT 0,
+    out_path        TEXT DEFAULT '',         -- outgoing path string
+    out_path_len    INTEGER DEFAULT 0,
+    last_advert     TEXT,                   -- ISO 8601 timestamp
+    adv_lat         REAL,                   -- GPS latitude from advert
+    adv_lon         REAL,                   -- GPS longitude from advert
+    first_seen      TEXT NOT NULL DEFAULT (datetime('now')),
+    last_seen       TEXT NOT NULL DEFAULT (datetime('now')),
+    source          TEXT DEFAULT 'advert',  -- 'advert', 'device', 'manual'
+    is_protected    INTEGER DEFAULT 0,      -- 1 = protected from cleanup
+    lastmod         TEXT NOT NULL DEFAULT (datetime('now'))
+);
+
+-- Channel configuration
+CREATE TABLE IF NOT EXISTS channels (
+    idx         INTEGER PRIMARY KEY,        -- channel index (0-7)
+    name        TEXT NOT NULL DEFAULT '',
+    secret      TEXT,                       -- channel secret/key (hex)
+    updated_at  TEXT NOT NULL DEFAULT (datetime('now'))
+);
+
+-- Channel messages (replaces CHAN/SENT_CHAN from .msgs)
+CREATE TABLE IF NOT EXISTS channel_messages (
+    id                  INTEGER PRIMARY KEY AUTOINCREMENT,
+    channel_idx         INTEGER NOT NULL DEFAULT 0,
+    sender              TEXT NOT NULL DEFAULT '',
+    content             TEXT NOT NULL DEFAULT '',
+    timestamp           INTEGER NOT NULL DEFAULT 0,         -- unix epoch
+    sender_timestamp    INTEGER,                            -- sender's clock
+    is_own              INTEGER NOT NULL DEFAULT 0,         -- 1 = sent by us
+    txt_type            INTEGER DEFAULT 0,
+    snr                 REAL,
+    path_len            INTEGER,
+    pkt_payload         TEXT,                               -- for echo matching
+    raw_json            TEXT,                               -- original JSON line
+    created_at          TEXT NOT NULL DEFAULT (datetime('now'))
+);
+
+-- Direct messages (replaces PRIV/SENT_MSG from .msgs)
+CREATE TABLE IF NOT EXISTS direct_messages (
+    id                  INTEGER PRIMARY KEY AUTOINCREMENT,
+    contact_pubkey      TEXT,                               -- FK to contacts (nullable for unknown)
+    direction           TEXT NOT NULL CHECK (direction IN ('in', 'out')),
+    content             TEXT NOT NULL DEFAULT '',
+    timestamp           INTEGER NOT NULL DEFAULT 0,         -- unix epoch
+    sender_timestamp    INTEGER,
+    txt_type            INTEGER DEFAULT 0,
+    snr                 REAL,
+    path_len            INTEGER,
+    expected_ack        TEXT,                               -- ACK code for delivery tracking
+    pkt_payload         TEXT,                               -- raw packet payload for hash/analyzer
+    signature           TEXT,                               -- dedup signature
+    raw_json            TEXT,
+    created_at          TEXT NOT NULL DEFAULT (datetime('now')),
+    FOREIGN KEY (contact_pubkey) REFERENCES contacts(public_key) ON DELETE SET NULL
+);
+
+-- ACK tracking (replaces .acks.jsonl)
+CREATE TABLE IF NOT EXISTS acks (
+    id              INTEGER PRIMARY KEY AUTOINCREMENT,
+    expected_ack    TEXT NOT NULL,           -- ACK code to match
+    received_at     TEXT NOT NULL DEFAULT (datetime('now')),
+    snr             REAL,
+    rssi            REAL,
+    route_type      TEXT,                   -- 'direct', 'flood', etc.
+    is_retry        INTEGER DEFAULT 0,
+    dm_id           INTEGER,                -- FK to direct_messages (nullable)
+    FOREIGN KEY (dm_id) REFERENCES direct_messages(id) ON DELETE SET NULL
+);
+
+-- Echo tracking (replaces .echoes.jsonl)
+CREATE TABLE IF NOT EXISTS echoes (
+    id              INTEGER PRIMARY KEY AUTOINCREMENT,
+    pkt_payload     TEXT NOT NULL,           -- matches channel_messages.pkt_payload
+    path            TEXT,                   -- relay path string
+    snr             REAL,
+    received_at     TEXT NOT NULL DEFAULT (datetime('now')),
+    direction       TEXT DEFAULT 'incoming', -- 'sent' or 'incoming'
+    cm_id           INTEGER,                -- FK to channel_messages (nullable)
+    FOREIGN KEY (cm_id) REFERENCES channel_messages(id) ON DELETE SET NULL
+);
+
+-- Path tracking (replaces .path.jsonl)
+CREATE TABLE IF NOT EXISTS paths (
+    id              INTEGER PRIMARY KEY AUTOINCREMENT,
+    contact_pubkey  TEXT,
+    pkt_payload     TEXT,
+    path            TEXT,
+    snr             REAL,
+    path_len        INTEGER,
+    received_at     TEXT NOT NULL DEFAULT (datetime('now'))
+);
+
+-- User-configured paths for DM retry rotation
+CREATE TABLE IF NOT EXISTS contact_paths (
+    id              INTEGER PRIMARY KEY AUTOINCREMENT,
+    contact_pubkey  TEXT NOT NULL REFERENCES contacts(public_key) ON DELETE CASCADE,
+    path_hex        TEXT NOT NULL DEFAULT '',   -- raw hex path bytes (e.g. "5e34e761")
+    hash_size       INTEGER NOT NULL DEFAULT 1, -- bytes per hop: 1, 2, or 3
+    label           TEXT NOT NULL DEFAULT '',    -- friendly label (e.g. "via Zalesie")
+    is_primary      INTEGER NOT NULL DEFAULT 0, -- 1 = priority/default path
+    sort_order      INTEGER NOT NULL DEFAULT 0, -- lower = tried first during rotation
+    created_at      TEXT NOT NULL DEFAULT (datetime('now'))
+);
+
+-- Advertisements (replaces .adverts.jsonl)
+CREATE TABLE IF NOT EXISTS advertisements (
+    id              INTEGER PRIMARY KEY AUTOINCREMENT,
+    public_key      TEXT NOT NULL,
+    name            TEXT NOT NULL DEFAULT '',
+    type            INTEGER DEFAULT 0,
+    lat             REAL,
+    lon             REAL,
+    timestamp       INTEGER NOT NULL DEFAULT 0,
+    snr             REAL,
+    raw_payload     TEXT,
+    created_at      TEXT NOT NULL DEFAULT (datetime('now'))
+);
+
+-- Read status tracking (replaces .read_status.json)
+CREATE TABLE IF NOT EXISTS read_status (
+    key             TEXT PRIMARY KEY,       -- 'chan_0', 'dm_<pubkey>', etc.
+    last_seen_ts    INTEGER DEFAULT 0,      -- unix timestamp
+    is_muted        INTEGER DEFAULT 0,      -- 1 = muted (channels only)
+    updated_at      TEXT NOT NULL DEFAULT (datetime('now'))
+);
+
+-- Ignored contacts (adverts cached but not pending/auto-added)
+CREATE TABLE IF NOT EXISTS ignored_contacts (
+    public_key  TEXT PRIMARY KEY REFERENCES contacts(public_key),
+    created_at  TEXT NOT NULL DEFAULT (datetime('now'))
+);
+
+-- Blocked contacts (ignored + messages hidden from display)
+CREATE TABLE IF NOT EXISTS blocked_contacts (
+    public_key  TEXT PRIMARY KEY REFERENCES contacts(public_key),
+    created_at  TEXT NOT NULL DEFAULT (datetime('now'))
+);
+
+-- Blocked names (for bots/contacts without known public_key)
+CREATE TABLE IF NOT EXISTS blocked_names (
+    name        TEXT PRIMARY KEY,
+    created_at  TEXT NOT NULL DEFAULT (datetime('now'))
+);
+
+-- Application settings (key-value store, replaces .webui_settings.json)
+CREATE TABLE IF NOT EXISTS app_settings (
+    key         TEXT PRIMARY KEY,
+    value       TEXT NOT NULL DEFAULT '',       -- JSON-encoded value
+    updated_at  TEXT NOT NULL DEFAULT (datetime('now'))
+);
+
+-- ============================================================
+-- Indexes
+-- ============================================================
+
+CREATE INDEX IF NOT EXISTS idx_cm_channel_ts ON channel_messages(channel_idx, timestamp);
+CREATE INDEX IF NOT EXISTS idx_cm_pkt ON channel_messages(pkt_payload);
+CREATE INDEX IF NOT EXISTS idx_dm_contact ON direct_messages(contact_pubkey, timestamp);
+CREATE INDEX IF NOT EXISTS idx_dm_ack ON direct_messages(expected_ack);
+CREATE INDEX IF NOT EXISTS idx_acks_code ON acks(expected_ack);
+CREATE INDEX IF NOT EXISTS idx_echoes_pkt ON echoes(pkt_payload);
+CREATE INDEX IF NOT EXISTS idx_adv_pubkey ON advertisements(public_key, timestamp);
+CREATE INDEX IF NOT EXISTS idx_contacts_name ON contacts(name);
+CREATE INDEX IF NOT EXISTS idx_cp_contact ON contact_paths(contact_pubkey, sort_order);
+
+-- ============================================================
+-- Full-Text Search (FTS5)
+-- ============================================================
+
+CREATE VIRTUAL TABLE IF NOT EXISTS channel_messages_fts USING fts5(
+    content,
+    content=channel_messages,
+    content_rowid=id
+);
+
+CREATE VIRTUAL TABLE IF NOT EXISTS direct_messages_fts USING fts5(
+    content,
+    content=direct_messages,
+    content_rowid=id
+);
+
+-- FTS triggers: keep FTS index in sync with source tables
+
+CREATE TRIGGER IF NOT EXISTS cm_fts_insert AFTER INSERT ON channel_messages BEGIN
+    INSERT INTO channel_messages_fts(rowid, content) VALUES (new.id, new.content);
+END;
+
+CREATE TRIGGER IF NOT EXISTS cm_fts_delete AFTER DELETE ON channel_messages BEGIN
+    INSERT INTO channel_messages_fts(channel_messages_fts, rowid, content)
+        VALUES ('delete', old.id, old.content);
+END;
+
+CREATE TRIGGER IF NOT EXISTS cm_fts_update AFTER UPDATE OF content ON channel_messages BEGIN
+    INSERT INTO channel_messages_fts(channel_messages_fts, rowid, content)
+        VALUES ('delete', old.id, old.content);
+    INSERT INTO channel_messages_fts(rowid, content) VALUES (new.id, new.content);
+END;
+
+CREATE TRIGGER IF NOT EXISTS dm_fts_insert AFTER INSERT ON direct_messages BEGIN
+    INSERT INTO direct_messages_fts(rowid, content) VALUES (new.id, new.content);
+END;
+
+CREATE TRIGGER IF NOT EXISTS dm_fts_delete AFTER DELETE ON direct_messages BEGIN
+    INSERT INTO direct_messages_fts(direct_messages_fts, rowid, content)
+        VALUES ('delete', old.id, old.content);
+END;
+
+CREATE TRIGGER IF NOT EXISTS dm_fts_update AFTER UPDATE OF content ON direct_messages BEGIN
+    INSERT INTO direct_messages_fts(direct_messages_fts, rowid, content)
+        VALUES ('delete', old.id, old.content);
+    INSERT INTO direct_messages_fts(rowid, content) VALUES (new.id, new.content);
+END;

app/static/css/theme.css

@@ -0,0 +1,613 @@
+/* =============================================================================
+   mc-webui Theme System
+   Defines CSS custom properties for light/dark themes.
+   Bootstrap 5.3 data-bs-theme handles most component styling;
+   these variables cover custom app-specific elements.
+   ============================================================================= */
+
+/* =============================================================================
+   Light Theme (default)
+   ============================================================================= */
+:root {
+    /* Backgrounds */
+    --bg-body: #ffffff;
+    --bg-surface: #f8f9fa;
+    --bg-surface-alt: #f0f0f0;
+    --bg-hover: #e9ecef;
+    --bg-active: #e7f1ff;
+    --bg-messages: #ffffff;
+    --bg-dm-messages: #fafafa;
+
+    /* Text */
+    --text-primary: #212529;
+    --text-secondary: #495057;
+    --text-muted: #6c757d;
+    --text-meta: #adb5bd;
+
+    /* Borders */
+    --border-color: #dee2e6;
+    --border-light: #f0f0f0;
+
+    /* Messages */
+    --msg-own-bg: #e7f1ff;
+    --msg-other-bg: #f8f9fa;
+    --msg-border: #dee2e6;
+    --msg-own-border: #b8daff;
+
+    /* Sender */
+    --sender-color: #0d6efd;
+    --sender-own-color: #084298;
+
+    /* Navbar */
+    --navbar-bg: #0d6efd;
+    --navbar-border: transparent;
+
+    /* Scrollbar */
+    --scrollbar-track: #f1f1f1;
+    --scrollbar-thumb: #888;
+    --scrollbar-thumb-hover: #555;
+    --scrollbar-thumb-light: #ccc;
+    --scrollbar-thumb-light-hover: #aaa;
+
+    /* Filter */
+    --filter-bg: #ffffff;
+    --filter-highlight: #fff3cd;
+    --filter-input-border: #ced4da;
+    --filter-btn-me-bg: #e7f1ff;
+    --filter-btn-me-color: #0d6efd;
+    --filter-btn-me-hover: #cfe2ff;
+    --filter-btn-clear-bg: #f8f9fa;
+    --filter-btn-clear-color: #6c757d;
+    --filter-btn-clear-hover: #e9ecef;
+
+    /* Popup / Dropdown */
+    --popup-bg: #ffffff;
+    --popup-shadow: 0 0.5rem 1rem rgba(0, 0, 0, 0.15);
+
+    /* Quote */
+    --quote-color: #6c757d;
+    --quote-bg: rgba(108, 117, 125, 0.1);
+    --quote-border: #6c757d;
+    --quote-own-color: #495057;
+    --quote-own-bg: rgba(8, 66, 152, 0.1);
+    --quote-own-border: #084298;
+
+    /* Mention badge */
+    --mention-bg: #0d6efd;
+    --mention-own-bg: #084298;
+
+    /* Links */
+    --link-color: #0d6efd;
+    --link-hover: #0a58ca;
+    --link-own-color: #084298;
+    --link-own-hover: #052c65;
+
+    /* Channel link */
+    --channel-link-bg: #198754;
+    --channel-link-hover: #157347;
+    --channel-link-own-bg: #0f5132;
+    --channel-link-own-hover: #0d4429;
+
+    /* Echo badge */
+    --echo-color: #198754;
+    --echo-bg: rgba(25, 135, 84, 0.1);
+
+    /* Search */
+    --search-mark-bg: #fff3cd;
+
+    /* Offcanvas menu */
+    --offcanvas-item-border: #dee2e6;
+    --offcanvas-item-hover: #f8f9fa;
+    --offcanvas-icon-color: #0d6efd;
+
+    /* FAB */
+    --fab-shadow: 0 4px 8px rgba(0, 0, 0, 0.3);
+    --fab-shadow-hover: 0 6px 12px rgba(0, 0, 0, 0.4);
+
+    /* Conversation list */
+    --conversation-border: #dee2e6;
+    --conversation-hover: #f8f9fa;
+    --conversation-unread: #e7f1ff;
+
+    /* Map filter badges */
+    --map-badge-inactive-bg: white;
+
+    /* Mention autocomplete */
+    --mention-item-highlight: #e7f1ff;
+    --mention-item-border: #f0f0f0;
+
+    /* Image border */
+    --image-border: #dee2e6;
+
+    /* Actions border */
+    --actions-border: rgba(0, 0, 0, 0.1);
+
+    /* Cards */
+    --card-bg: #ffffff;
+    --card-shadow: 0 1px 3px rgba(0, 0, 0, 0.1);
+    --card-shadow-hover: 0 2px 8px rgba(0, 0, 0, 0.15);
+
+    /* Info badge */
+    --info-badge-bg: #e7f3ff;
+    --info-badge-color: #0c5460;
+
+    /* Contact key clickable */
+    --key-hover-color: #0d6efd;
+    --key-hover-bg: #e7f1ff;
+    --key-copied-color: #198754;
+    --key-copied-bg: #d1e7dd;
+
+    /* Path items (DM) */
+    --path-item-bg: #ffffff;
+    --path-item-border: #dee2e6;
+    --path-item-primary-bg: #f0f7ff;
+    --path-item-primary-border: #0d6efd;
+
+    /* DM contact dropdown */
+    --dropdown-bg: #ffffff;
+    --dropdown-separator-bg: #f8f9fa;
+    --dropdown-item-hover: #e9ecef;
+}
+
+/* =============================================================================
+   Dark Theme
+   Inspired by mc-webui demo landing page (https://mc-webui.marwoj.net/)
+   Color palette: deep navy backgrounds, slate surfaces, soft blue accents
+   ============================================================================= */
+[data-theme="dark"] {
+    /* Override Bootstrap 5.3 dark mode variables for our custom palette */
+    --bs-body-bg: #0f172a;
+    --bs-body-color: #f8fafc;
+    --bs-border-color: #334155;
+    --bs-tertiary-bg: #1e293b;
+    --bs-secondary-bg: #162032;
+
+    /* Backgrounds */
+    --bg-body: #0f172a;
+    --bg-surface: #1e293b;
+    --bg-surface-alt: #162032;
+    --bg-hover: #2d3a4e;
+    --bg-active: #1e3a5f;
+    --bg-messages: #0f172a;
+    --bg-dm-messages: #131c2e;
+
+    /* Text */
+    --text-primary: #f8fafc;
+    --text-secondary: #94a3b8;
+    --text-muted: #64748b;
+    --text-meta: #475569;
+
+    /* Borders */
+    --border-color: #334155;
+    --border-light: #1e293b;
+
+    /* Messages */
+    --msg-own-bg: #1e3a5f;
+    --msg-other-bg: #1e293b;
+    --msg-border: #334155;
+    --msg-own-border: #2563eb;
+
+    /* Sender */
+    --sender-color: #60a5fa;
+    --sender-own-color: #93c5fd;
+
+    /* Navbar */
+    --navbar-bg: #1e293b;
+    --navbar-border: #334155;
+
+    /* Scrollbar */
+    --scrollbar-track: #1e293b;
+    --scrollbar-thumb: #475569;
+    --scrollbar-thumb-hover: #64748b;
+    --scrollbar-thumb-light: #334155;
+    --scrollbar-thumb-light-hover: #475569;
+
+    /* Filter */
+    --filter-bg: #1e293b;
+    --filter-highlight: rgba(251, 191, 36, 0.2);
+    --filter-input-border: #334155;
+    --filter-btn-me-bg: #1e3a5f;
+    --filter-btn-me-color: #60a5fa;
+    --filter-btn-me-hover: #264a6f;
+    --filter-btn-clear-bg: #1e293b;
+    --filter-btn-clear-color: #94a3b8;
+    --filter-btn-clear-hover: #2d3a4e;
+
+    /* Popup / Dropdown */
+    --popup-bg: #1e293b;
+    --popup-shadow: 0 0.5rem 1rem rgba(0, 0, 0, 0.4);
+
+    /* Quote */
+    --quote-color: #94a3b8;
+    --quote-bg: rgba(148, 163, 184, 0.1);
+    --quote-border: #64748b;
+    --quote-own-color: #94a3b8;
+    --quote-own-bg: rgba(37, 99, 235, 0.15);
+    --quote-own-border: #2563eb;
+
+    /* Mention badge */
+    --mention-bg: #2563eb;
+    --mention-own-bg: #1d4ed8;
+
+    /* Links */
+    --link-color: #60a5fa;
+    --link-hover: #93c5fd;
+    --link-own-color: #93c5fd;
+    --link-own-hover: #bfdbfe;
+
+    /* Channel link */
+    --channel-link-bg: #059669;
+    --channel-link-hover: #10b981;
+    --channel-link-own-bg: #047857;
+    --channel-link-own-hover: #059669;
+
+    /* Echo badge */
+    --echo-color: #10b981;
+    --echo-bg: rgba(16, 185, 129, 0.15);
+
+    /* Search */
+    --search-mark-bg: rgba(251, 191, 36, 0.3);
+
+    /* Offcanvas menu */
+    --offcanvas-item-border: #334155;
+    --offcanvas-item-hover: #253347;
+    --offcanvas-icon-color: #60a5fa;
+
+    /* FAB */
+    --fab-shadow: 0 4px 8px rgba(0, 0, 0, 0.5);
+    --fab-shadow-hover: 0 6px 12px rgba(0, 0, 0, 0.6);
+
+    /* Conversation list */
+    --conversation-border: #334155;
+    --conversation-hover: #253347;
+    --conversation-unread: #1e3a5f;
+
+    /* Map filter badges */
+    --map-badge-inactive-bg: #1e293b;
+
+    /* Mention autocomplete */
+    --mention-item-highlight: #1e3a5f;
+    --mention-item-border: #334155;
+
+    /* Image border */
+    --image-border: #334155;
+
+    /* Actions border */
+    --actions-border: rgba(255, 255, 255, 0.1);
+
+    /* Cards */
+    --card-bg: #1e293b;
+    --card-shadow: 0 1px 3px rgba(0, 0, 0, 0.3);
+    --card-shadow-hover: 0 2px 8px rgba(0, 0, 0, 0.4);
+
+    /* Info badge */
+    --info-badge-bg: rgba(37, 99, 235, 0.15);
+    --info-badge-color: #60a5fa;
+
+    /* Contact key clickable */
+    --key-hover-color: #60a5fa;
+    --key-hover-bg: #1e3a5f;
+    --key-copied-color: #10b981;
+    --key-copied-bg: rgba(16, 185, 129, 0.15);
+
+    /* Path items (DM) */
+    --path-item-bg: #1e293b;
+    --path-item-border: #334155;
+    --path-item-primary-bg: #1e3a5f;
+    --path-item-primary-border: #2563eb;
+
+    /* DM contact dropdown */
+    --dropdown-bg: #1e293b;
+    --dropdown-separator-bg: #162032;
+    --dropdown-item-hover: #2d3a4e;
+}
+
+/* =============================================================================
+   Dark Theme - Bootstrap Component Overrides
+   Bootstrap 5.3 data-bs-theme="dark" handles most defaults; these overrides
+   customize colors to match our deep navy palette.
+   ============================================================================= */
+
+/* Navbar */
+[data-theme="dark"] .navbar.bg-primary {
+    background-color: var(--navbar-bg) !important;
+    border-bottom: 1px solid var(--navbar-border);
+}
+
+[data-theme="dark"] .navbar .btn-outline-light {
+    border-color: #475569;
+    color: #94a3b8;
+}
+
+[data-theme="dark"] .navbar .btn-outline-light:hover {
+    background-color: #334155;
+    border-color: #64748b;
+    color: #f8fafc;
+}
+
+/* Form controls */
+[data-theme="dark"] .form-control,
+[data-theme="dark"] .form-select {
+    background-color: var(--bg-body);
+    color: var(--text-primary);
+    border-color: var(--border-color);
+}
+
+[data-theme="dark"] .form-control:focus,
+[data-theme="dark"] .form-select:focus {
+    background-color: var(--bg-body);
+    color: var(--text-primary);
+    border-color: #3b82f6;
+    box-shadow: 0 0 0 0.2rem rgba(59, 130, 246, 0.25);
+}
+
+[data-theme="dark"] .form-control::placeholder {
+    color: var(--text-muted);
+}
+
+/* Modal */
+[data-theme="dark"] .modal-content {
+    background-color: var(--bg-surface);
+    color: var(--text-primary);
+    border-color: var(--border-color);
+}
+
+[data-theme="dark"] .modal-header {
+    border-bottom-color: var(--border-color);
+}
+
+[data-theme="dark"] .modal-footer {
+    border-top-color: var(--border-color);
+}
+
+[data-theme="dark"] .btn-close {
+    filter: invert(1) grayscale(100%) brightness(200%);
+}
+
+/* Offcanvas */
+[data-theme="dark"] .offcanvas {
+    background-color: var(--bg-surface);
+    color: var(--text-primary);
+}
+
+[data-theme="dark"] .offcanvas-header {
+    border-bottom-color: var(--border-color);
+}
+
+/* List group */
+[data-theme="dark"] .list-group-item {
+    background-color: transparent;
+    color: var(--text-primary);
+    border-color: var(--border-color);
+}
+
+[data-theme="dark"] .list-group-item-action:hover {
+    background-color: var(--bg-hover);
+    color: var(--text-primary);
+}
+
+/* Nav tabs */
+[data-theme="dark"] .nav-tabs {
+    border-bottom-color: var(--border-color);
+}
+
+[data-theme="dark"] .nav-tabs .nav-link {
+    color: var(--text-muted);
+}
+
+[data-theme="dark"] .nav-tabs .nav-link:hover {
+    border-color: var(--border-color);
+    color: var(--text-secondary);
+}
+
+[data-theme="dark"] .nav-tabs .nav-link.active {
+    background-color: var(--bg-surface);
+    color: var(--text-primary);
+    border-color: var(--border-color) var(--border-color) var(--bg-surface);
+}
+
+/* Tables */
+[data-theme="dark"] .table {
+    color: var(--text-primary);
+    border-color: var(--border-color);
+}
+
+/* Alerts */
+[data-theme="dark"] .alert-info {
+    background-color: rgba(59, 130, 246, 0.1);
+    color: #60a5fa;
+    border-color: rgba(59, 130, 246, 0.2);
+}
+
+[data-theme="dark"] .alert-light {
+    background-color: var(--bg-surface-alt);
+    color: var(--text-secondary);
+    border-color: var(--border-color);
+}
+
+/* Card (Bootstrap) */
+[data-theme="dark"] .card {
+    background-color: var(--bg-surface);
+    border-color: var(--border-color);
+    color: var(--text-primary);
+}
+
+/* Badge overrides for better dark mode contrast */
+[data-theme="dark"] .badge.bg-secondary {
+    background-color: #475569 !important;
+}
+
+/* Text utilities */
+[data-theme="dark"] .text-muted {
+    color: var(--text-muted) !important;
+}
+
+[data-theme="dark"] .text-dark {
+    color: var(--text-primary) !important;
+}
+
+[data-theme="dark"] .border-bottom {
+    border-bottom-color: var(--border-color) !important;
+}
+
+[data-theme="dark"] .border-top {
+    border-top-color: var(--border-color) !important;
+}
+
+/* bg-light override */
+[data-theme="dark"] .bg-light {
+    background-color: var(--bg-surface-alt) !important;
+}
+
+/* Toast */
+[data-theme="dark"] .toast {
+    background-color: var(--bg-surface);
+    color: var(--text-primary);
+    border-color: var(--border-color);
+}
+
+[data-theme="dark"] .toast-header {
+    background-color: var(--bg-surface-alt);
+    color: var(--text-primary);
+    border-bottom-color: var(--border-color);
+}
+
+/* Progress bar */
+[data-theme="dark"] .progress {
+    background-color: var(--bg-surface-alt);
+}
+
+/* Tooltip-like popups */
+[data-theme="dark"] .dm-delivery-popup,
+[data-theme="dark"] .path-popup {
+    background-color: #475569;
+    color: #f8fafc;
+}
+
+/* Form check / switch */
+[data-theme="dark"] .form-check-input {
+    background-color: var(--bg-surface-alt);
+    border-color: var(--border-color);
+}
+
+[data-theme="dark"] .form-check-input:checked {
+    background-color: #3b82f6;
+    border-color: #3b82f6;
+}
+
+/* Input group */
+[data-theme="dark"] .input-group-text {
+    background-color: var(--bg-surface-alt);
+    color: var(--text-secondary);
+    border-color: var(--border-color);
+}
+
+/* Accordion (if used) */
+[data-theme="dark"] .accordion-item {
+    background-color: var(--bg-surface);
+    border-color: var(--border-color);
+}
+
+/* Dropdown menu (Bootstrap) */
+[data-theme="dark"] .dropdown-menu {
+    background-color: var(--bg-surface);
+    border-color: var(--border-color);
+}
+
+[data-theme="dark"] .dropdown-item {
+    color: var(--text-primary);
+}
+
+[data-theme="dark"] .dropdown-item:hover {
+    background-color: var(--bg-hover);
+    color: var(--text-primary);
+}
+
+/* Spinner */
+[data-theme="dark"] .spinner-border {
+    color: #3b82f6;
+}
+
+/* Status bar (bottom) */
+[data-theme="dark"] .border-top {
+    border-color: var(--border-color) !important;
+}
+
+/* QR code container - keep white bg for readability */
+[data-theme="dark"] .qr-code-container,
+[data-theme="dark"] #shareChannelQR,
+[data-theme="dark"] #deviceShareContent .text-center img,
+[data-theme="dark"] #deviceShareContent canvas {
+    background-color: #ffffff;
+    padding: 8px;
+    border-radius: 0.5rem;
+}
+
+/* Emoji picker dark mode */
+[data-theme="dark"] emoji-picker {
+    --background: #1e293b;
+    --border-color: #334155;
+    --indicator-color: #3b82f6;
+    --input-border-color: #334155;
+    --input-font-color: #f8fafc;
+    --input-placeholder-color: #64748b;
+    --outline-color: #3b82f6;
+    --category-font-color: #94a3b8;
+    --button-active-background: #334155;
+    --button-hover-background: #2d3a4e;
+}
+
+/* =============================================================================
+   Theme Switcher UI
+   ============================================================================= */
+
+.theme-option {
+    display: flex;
+    align-items: center;
+    gap: 0.75rem;
+    padding: 0.75rem 1rem;
+    border: 2px solid var(--border-color);
+    border-radius: 0.75rem;
+    cursor: pointer;
+    transition: all 0.2s ease;
+    background-color: var(--card-bg);
+}
+
+.theme-option:hover {
+    border-color: #3b82f6;
+}
+
+.theme-option.active {
+    border-color: #3b82f6;
+    background-color: var(--bg-active);
+}
+
+.theme-option-preview {
+    width: 40px;
+    height: 40px;
+    border-radius: 0.5rem;
+    flex-shrink: 0;
+    display: flex;
+    align-items: center;
+    justify-content: center;
+    font-size: 1.25rem;
+}
+
+.theme-option-preview.light {
+    background: linear-gradient(135deg, #ffffff 50%, #e9ecef 50%);
+    border: 1px solid #dee2e6;
+}
+
+.theme-option-preview.dark {
+    background: linear-gradient(135deg, #1e293b 50%, #0f172a 50%);
+    border: 1px solid #334155;
+}
+
+.theme-option-label {
+    font-weight: 500;
+}
+
+.theme-option-desc {
+    font-size: 0.8rem;
+    color: var(--text-muted);
+}

app/static/js/logs.js

@@ -0,0 +1,276 @@
+/**
+ * System Log Viewer
+ *
+ * Real-time log streaming via WebSocket with filtering and search.
+ */
+(function () {
+    'use strict';
+
+    // --- DOM refs ---
+    const logEntries = document.getElementById('logEntries');
+    const loadingMsg = document.getElementById('loadingMsg');
+    const logCount = document.getElementById('logCount');
+    const statusDot = document.getElementById('statusDot');
+    const pauseBtn = document.getElementById('pauseBtn');
+    const pauseIcon = document.getElementById('pauseIcon');
+    const clearBtn = document.getElementById('clearBtn');
+    const levelFilter = document.getElementById('levelFilter');
+    const loggerFilter = document.getElementById('loggerFilter');
+    const searchFilter = document.getElementById('searchFilter');
+    const resetFilters = document.getElementById('resetFilters');
+
+    // --- State ---
+    let paused = false;
+    let autoScroll = true;
+    let entries = [];       // all received entries
+    let displayCount = 0;
+    const MAX_DISPLAY = 3000;  // max DOM entries before trimming
+    let searchDebounce = null;
+    let knownLoggers = new Set();
+
+    // --- Level ordering ---
+    const LEVEL_ORDER = { DEBUG: 0, INFO: 1, WARNING: 2, ERROR: 3, CRITICAL: 4 };
+
+    // --- WebSocket ---
+    const socket = io('/logs', {
+        transports: ['websocket', 'polling'],
+        reconnection: true,
+        reconnectionDelay: 2000,
+    });
+
+    socket.on('connect', () => {
+        setStatus('live');
+        // Load initial entries
+        loadInitialLogs();
+    });
+
+    socket.on('disconnect', () => {
+        setStatus('disconnected');
+    });
+
+    socket.on('log_entry', (entry) => {
+        addEntry(entry);
+    });
+
+    // --- Functions ---
+
+    function setStatus(state) {
+        statusDot.className = 'status-indicator ' + state;
+    }
+
+    function loadInitialLogs() {
+        const level = levelFilter.value;
+        const params = new URLSearchParams();
+        if (level) params.set('level', level);
+        params.set('limit', '1000');
+
+        fetch('/api/logs?' + params.toString())
+            .then(r => r.json())
+            .then(data => {
+                if (!data.success) return;
+                loadingMsg?.remove();
+
+                // Update logger filter options
+                if (data.loggers) {
+                    data.loggers.forEach(l => knownLoggers.add(l));
+                    updateLoggerOptions();
+                }
+
+                // Render entries
+                entries = data.entries || [];
+                renderAll();
+            })
+            .catch(err => {
+                if (loadingMsg) loadingMsg.textContent = 'Failed to load logs';
+                console.error('Failed to load logs:', err);
+            });
+    }
+
+    function addEntry(entry) {
+        entries.push(entry);
+
+        // Track new loggers
+        if (!knownLoggers.has(entry.logger)) {
+            knownLoggers.add(entry.logger);
+            updateLoggerOptions();
+        }
+
+        // If paused or filtered out, don't add to DOM
+        if (paused) {
+            updateCount();
+            return;
+        }
+
+        if (matchesFilter(entry)) {
+            appendEntryDOM(entry);
+            trimDOM();
+            if (autoScroll) scrollToBottom();
+        }
+        updateCount();
+    }
+
+    function matchesFilter(entry) {
+        // Level filter
+        const minLevel = levelFilter.value;
+        if (minLevel && (LEVEL_ORDER[entry.level] || 0) < (LEVEL_ORDER[minLevel] || 0)) {
+            return false;
+        }
+
+        // Logger filter
+        const loggerVal = loggerFilter.value;
+        if (loggerVal && !entry.logger.startsWith(loggerVal)) {
+            return false;
+        }
+
+        // Search filter
+        const search = searchFilter.value.trim().toLowerCase();
+        if (search && !entry.message.toLowerCase().includes(search) &&
+            !entry.logger.toLowerCase().includes(search)) {
+            return false;
+        }
+
+        return true;
+    }
+
+    function renderAll() {
+        logEntries.innerHTML = '';
+        displayCount = 0;
+
+        const filtered = entries.filter(e => matchesFilter(e));
+        // Only render last MAX_DISPLAY entries
+        const toRender = filtered.slice(-MAX_DISPLAY);
+        const fragment = document.createDocumentFragment();
+
+        for (const entry of toRender) {
+            fragment.appendChild(createEntryElement(entry));
+            displayCount++;
+        }
+
+        logEntries.appendChild(fragment);
+        updateCount();
+        scrollToBottom();
+    }
+
+    function appendEntryDOM(entry) {
+        logEntries.appendChild(createEntryElement(entry));
+        displayCount++;
+    }
+
+    function createEntryElement(entry) {
+        const div = document.createElement('div');
+        div.className = 'log-line';
+
+        // Shorten timestamp to HH:MM:SS.mmm
+        const ts = entry.timestamp.length > 11 ? entry.timestamp.substring(11) : entry.timestamp;
+
+        // Shorten logger name (remove 'app.' prefix)
+        let loggerName = entry.logger;
+        if (loggerName.startsWith('app.')) {
+            loggerName = loggerName.substring(4);
+        }
+
+        // Pad level to 5 chars
+        const levelPad = entry.level.padEnd(5);
+
+        // Build the line with color spans
+        const search = searchFilter.value.trim().toLowerCase();
+        let message = escapeHtml(entry.message);
+        if (search) {
+            message = highlightSearch(message, search);
+        }
+
+        div.innerHTML =
+            `<span class="log-ts">${escapeHtml(ts)}</span> ` +
+            `<span class="log-level-${entry.level}">${escapeHtml(levelPad)}</span> ` +
+            `<span class="log-logger">${escapeHtml(loggerName.padEnd(18).substring(0, 18))}</span> ` +
+            `<span class="log-msg-${entry.level}">${message}</span>`;
+
+        return div;
+    }
+
+    function trimDOM() {
+        while (logEntries.children.length > MAX_DISPLAY) {
+            logEntries.removeChild(logEntries.firstChild);
+            displayCount--;
+        }
+    }
+
+    function scrollToBottom() {
+        logEntries.scrollTop = logEntries.scrollHeight;
+    }
+
+    function updateCount() {
+        const total = entries.length;
+        const shown = logEntries.children.length;
+        logCount.textContent = shown === total
+            ? `${total} entries`
+            : `${shown} / ${total} entries`;
+    }
+
+    function updateLoggerOptions() {
+        const current = loggerFilter.value;
+        // Group loggers by top-level module
+        const sorted = Array.from(knownLoggers).sort();
+
+        loggerFilter.innerHTML = '<option value="">All modules</option>';
+        for (const name of sorted) {
+            const opt = document.createElement('option');
+            opt.value = name;
+            // Shorten display
+            opt.textContent = name.startsWith('app.') ? name.substring(4) : name;
+            if (name === current) opt.selected = true;
+            loggerFilter.appendChild(opt);
+        }
+    }
+
+    function escapeHtml(str) {
+        const div = document.createElement('div');
+        div.textContent = str;
+        return div.innerHTML;
+    }
+
+    function highlightSearch(html, search) {
+        if (!search) return html;
+        // Case-insensitive highlight (on already-escaped HTML)
+        const regex = new RegExp('(' + search.replace(/[.*+?^${}()|[\]\\]/g, '\\$&') + ')', 'gi');
+        return html.replace(regex, '<mark>$1</mark>');
+    }
+
+    // --- Auto-scroll detection ---
+    logEntries.addEventListener('scroll', () => {
+        const atBottom = logEntries.scrollHeight - logEntries.scrollTop - logEntries.clientHeight < 50;
+        autoScroll = atBottom;
+    });
+
+    // --- Controls ---
+    pauseBtn.addEventListener('click', () => {
+        paused = !paused;
+        pauseIcon.className = paused ? 'bi bi-play-fill' : 'bi bi-pause-fill';
+        setStatus(paused ? 'paused' : 'live');
+        if (!paused) {
+            // Resume: re-render to catch up
+            renderAll();
+        }
+    });
+
+    clearBtn.addEventListener('click', () => {
+        entries = [];
+        logEntries.innerHTML = '';
+        displayCount = 0;
+        updateCount();
+    });
+
+    // Filter handlers
+    levelFilter.addEventListener('change', () => renderAll());
+    loggerFilter.addEventListener('change', () => renderAll());
+    searchFilter.addEventListener('input', () => {
+        clearTimeout(searchDebounce);
+        searchDebounce = setTimeout(() => renderAll(), 250);
+    });
+    resetFilters.addEventListener('click', () => {
+        levelFilter.value = 'INFO';
+        loggerFilter.value = '';
+        searchFilter.value = '';
+        renderAll();
+    });
+})();

app/static/js/sw.js

@@ -1,4 +1,4 @@
-const CACHE_NAME = 'mc-webui-v4';
+const CACHE_NAME = 'mc-webui-v9';
 const ASSETS_TO_CACHE = [
     '/',
     '/static/css/style.css',
@@ -6,6 +6,7 @@ const ASSETS_TO_CACHE = [
     '/static/js/dm.js',
     '/static/js/contacts.js',
     '/static/js/message-utils.js',
+    '/static/js/filter-utils.js',
     '/static/js/console.js',
     '/static/images/android-chrome-192x192.png',
     '/static/images/android-chrome-512x512.png',

app/templates/base.html

@@ -1,5 +1,5 @@
 <!DOCTYPE html>
-<html lang="en">
+<html lang="en" data-theme="light" data-bs-theme="light">
 <head>
     <meta charset="UTF-8">
     <meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover">
@@ -12,6 +12,15 @@
     <link rel="manifest" href="{{ url_for('static', filename='manifest.json') }}">
     <link rel="shortcut icon" href="{{ url_for('static', filename='favicon.ico') }}">
 
+    <!-- Theme: apply saved preference before CSS loads to prevent flash -->
+    <script>
+        (function() {
+            var t = localStorage.getItem('mc-webui-theme') || 'light';
+            document.documentElement.setAttribute('data-theme', t);
+            document.documentElement.setAttribute('data-bs-theme', t);
+        })();
+    </script>
+
     <!-- Bootstrap 5 CSS (local) -->
     <link href="{{ url_for('static', filename='vendor/bootstrap/css/bootstrap.min.css') }}" rel="stylesheet">
     <!-- Bootstrap Icons (local) -->
@@ -24,6 +33,8 @@
 
     <!-- Custom CSS -->
     <link rel="stylesheet" href="{{ url_for('static', filename='css/style.css') }}">
+    <!-- Theme CSS (light/dark mode) -->
+    <link rel="stylesheet" href="{{ url_for('static', filename='css/theme.css') }}">
 
     {% block extra_head %}{% endblock %}
 </head>
@@ -38,14 +49,14 @@
                 {% endif %}
             </span>
             <div class="d-flex align-items-center gap-2">
-                <div id="notificationBell" class="btn btn-outline-light btn-sm position-relative" style="cursor: default;" title="Unread messages">
+                <div id="notificationBell" class="btn btn-outline-light position-relative navbar-touch-btn" style="cursor: pointer;" onclick="markAllChannelsRead()" title="Mark all as read">
                     <i class="bi bi-bell"></i>
                 </div>
-                <select id="channelSelector" class="form-select form-select-sm" style="width: auto; min-width: 100px;" title="Select channel">
+                <select id="channelSelector" class="form-select navbar-touch-select" style="width: auto; min-width: 100px;" title="Select channel">
                     <option value="0">Public</option>
                     <!-- Channels loaded dynamically via JavaScript -->
                 </select>
-                <button class="btn btn-outline-light btn-sm" data-bs-toggle="offcanvas" data-bs-target="#mainMenu" title="Menu">
+                <button class="btn btn-outline-light navbar-touch-btn" data-bs-toggle="offcanvas" data-bs-target="#mainMenu" title="Menu">
                     <i class="bi bi-list"></i>
                 </button>
             </div>
@@ -72,6 +83,7 @@
         </div>
         <div class="offcanvas-body">
             <div class="list-group list-group-flush">
+                <!-- Messages -->
                 <button class="list-group-item list-group-item-action d-flex align-items-center gap-3" id="refreshBtn">
                     <i class="bi bi-arrow-clockwise" style="font-size: 1.5rem;"></i>
                     <span>Refresh Messages</span>
@@ -80,7 +92,6 @@
                     <i class="bi bi-broadcast-pin" style="font-size: 1.5rem;"></i>
                     <span>Manage Channels</span>
                 </button>
-                <!-- Notifications Toggle -->
                 <button id="notificationsToggle" class="list-group-item list-group-item-action d-flex align-items-center gap-3" type="button">
                     <i class="bi bi-bell" style="font-size: 1.5rem;"></i>
                     <div class="flex-grow-1">
@@ -104,9 +115,9 @@
                     </select>
                 </div>
 
-                <!-- Network Commands Section -->
+                <!-- Network -->
                 <div class="list-group-item py-2 mt-2">
-                    <small class="text-muted fw-bold text-uppercase">Network Commands</small>
+                    <small class="text-muted fw-bold text-uppercase">Network</small>
                 </div>
                 <button class="list-group-item list-group-item-action d-flex align-items-center gap-3" id="advertBtn" title="Send single advertisement (recommended for normal operation)">
                     <i class="bi bi-megaphone" style="font-size: 1.5rem;"></i>
@@ -123,16 +134,10 @@
                     </div>
                 </button>
 
+                <!-- Tools -->
                 <div class="list-group-item py-2 mt-2">
-                    <small class="text-muted fw-bold text-uppercase">Configuration</small>
+                    <small class="text-muted fw-bold text-uppercase">Tools</small>
                 </div>
-                <button class="list-group-item list-group-item-action d-flex align-items-center gap-3" data-bs-toggle="modal" data-bs-target="#consoleModal" data-bs-dismiss="offcanvas">
-                    <i class="bi bi-terminal" style="font-size: 1.5rem;"></i>
-                    <div>
-                        <span>Console</span>
-                        <small class="d-block text-muted">Direct meshcli commands</small>
-                    </div>
-                </button>
                 <button class="list-group-item list-group-item-action d-flex align-items-center gap-3"
                         id="mapBtn" title="Show all contacts with GPS on map">
                     <i class="bi bi-map" style="font-size: 1.5rem;"></i>
@@ -141,10 +146,43 @@
                         <small class="d-block text-muted">All contacts with GPS</small>
                     </div>
                 </button>
+                <button class="list-group-item list-group-item-action d-flex align-items-center gap-3" data-bs-toggle="modal" data-bs-target="#consoleModal" data-bs-dismiss="offcanvas">
+                    <i class="bi bi-terminal" style="font-size: 1.5rem;"></i>
+                    <div>
+                        <span>Console</span>
+                        <small class="d-block text-muted">Direct meshcli commands</small>
+                    </div>
+                </button>
+
+                <!-- System -->
+                <div class="list-group-item py-2 mt-2">
+                    <small class="text-muted fw-bold text-uppercase">System</small>
+                </div>
                 <button class="list-group-item list-group-item-action d-flex align-items-center gap-3" data-bs-toggle="modal" data-bs-target="#deviceInfoModal" data-bs-dismiss="offcanvas">
                     <i class="bi bi-cpu" style="font-size: 1.5rem;"></i>
                     <span>Device Info</span>
                 </button>
+                <button class="list-group-item list-group-item-action d-flex align-items-center gap-3" data-bs-toggle="modal" data-bs-target="#logsModal" data-bs-dismiss="offcanvas">
+                    <i class="bi bi-journal-text" style="font-size: 1.5rem;"></i>
+                    <div>
+                        <span>System Log</span>
+                        <small class="d-block text-muted">Real-time application logs</small>
+                    </div>
+                </button>
+                <button class="list-group-item list-group-item-action d-flex align-items-center gap-3" data-bs-toggle="modal" data-bs-target="#backupModal" data-bs-dismiss="offcanvas">
+                    <i class="bi bi-database-down" style="font-size: 1.5rem;"></i>
+                    <div>
+                        <span>Backup</span>
+                        <small class="d-block text-muted">Database backup & restore</small>
+                    </div>
+                </button>
+                <button class="list-group-item list-group-item-action d-flex align-items-center gap-3" data-bs-toggle="modal" data-bs-target="#settingsModal" data-bs-dismiss="offcanvas">
+                    <i class="bi bi-gear" style="font-size: 1.5rem;"></i>
+                    <div>
+                        <span>Settings</span>
+                        <small class="d-block text-muted">Application settings</small>
+                    </div>
+                </button>
             </div>
         </div>
     </div>
@@ -274,13 +312,42 @@
         <div class="modal-dialog">
             <div class="modal-content">
                 <div class="modal-header">
-                    <h5 class="modal-title"><i class="bi bi-cpu"></i> Device Info</h5>
+                    <h5 class="modal-title"><i class="bi bi-cpu"></i> Device</h5>
                     <button type="button" class="btn-close" data-bs-dismiss="modal"></button>
                 </div>
                 <div class="modal-body">
-                    <div id="deviceInfoContent">
-                        <div class="text-center py-3">
-                            <div class="spinner-border spinner-border-sm"></div> Loading...
+                    <ul class="nav nav-tabs mb-3" role="tablist">
+                        <li class="nav-item" role="presentation">
+                            <button class="nav-link active" data-bs-toggle="tab" data-bs-target="#tabDeviceInfo" type="button">Info</button>
+                        </li>
+                        <li class="nav-item" role="presentation">
+                            <button class="nav-link" data-bs-toggle="tab" data-bs-target="#tabDeviceStats" type="button" id="statsTabBtn">Stats</button>
+                        </li>
+                        <li class="nav-item" role="presentation">
+                            <button class="nav-link" data-bs-toggle="tab" data-bs-target="#tabDeviceShare" type="button" id="shareTabBtn">Share</button>
+                        </li>
+                    </ul>
+                    <div class="tab-content">
+                        <div class="tab-pane fade show active" id="tabDeviceInfo">
+                            <div id="deviceInfoContent">
+                                <div class="text-center py-3">
+                                    <div class="spinner-border spinner-border-sm"></div> Loading...
+                                </div>
+                            </div>
+                        </div>
+                        <div class="tab-pane fade" id="tabDeviceStats">
+                            <div id="deviceStatsContent">
+                                <div class="text-center py-3 text-muted">
+                                    Click to load stats
+                                </div>
+                            </div>
+                        </div>
+                        <div class="tab-pane fade" id="tabDeviceShare">
+                            <div id="deviceShareContent">
+                                <div class="text-center py-3 text-muted">
+                                    Click to generate share code
+                                </div>
+                            </div>
                         </div>
                     </div>
                 </div>
@@ -288,6 +355,149 @@
         </div>
     </div>
 
+    <!-- Settings Modal -->
+    <div class="modal fade" id="settingsModal" tabindex="-1">
+        <div class="modal-dialog">
+            <div class="modal-content">
+                <div class="modal-header">
+                    <h5 class="modal-title"><i class="bi bi-gear"></i> Settings</h5>
+                    <button type="button" class="btn-close" data-bs-dismiss="modal"></button>
+                </div>
+                <div class="modal-body">
+                    <ul class="nav nav-tabs mb-3" role="tablist">
+                        <li class="nav-item" role="presentation">
+                            <button class="nav-link active" data-bs-toggle="tab" data-bs-target="#tabSettingsMessages" type="button">Messages</button>
+                        </li>
+                        <li class="nav-item" role="presentation">
+                            <button class="nav-link" data-bs-toggle="tab" data-bs-target="#tabSettingsChat" type="button">Group Chat</button>
+                        </li>
+                        <li class="nav-item" role="presentation">
+                            <button class="nav-link" data-bs-toggle="tab" data-bs-target="#tabSettingsAppearance" type="button">Appearance</button>
+                        </li>
+                    </ul>
+                    <div class="tab-content">
+                        <div class="tab-pane fade show active" id="tabSettingsMessages">
+                            <div id="settingsMessagesContent">
+                                <form id="dmRetrySettingsForm">
+                                    <p class="text-muted small mb-3">Retries are counted after the initial send, e.g. 3 retries = 4 total attempts.</p>
+                                    <h6 class="text-muted mb-2">When path is known (DIRECT)</h6>
+                                    <table class="table table-sm table-borderless mb-3 align-middle">
+                                        <tbody>
+                                            <tr>
+                                                <td class="ps-0">Direct retries <span class="badge rounded-pill text-muted" data-bs-toggle="tooltip" title="Attempts via known path before switching to flood"><i class="bi bi-info-circle"></i></span></td>
+                                                <td class="pe-0" style="width:5rem"><input type="number" class="form-control form-control-sm" id="settDirectMaxRetries" min="0" max="20" value="3"></td>
+                                            </tr>
+                                            <tr>
+                                                <td class="ps-0">Flood retries <span class="badge rounded-pill text-muted" data-bs-toggle="tooltip" title="Flood attempts after direct retries exhausted (when no configured paths)"><i class="bi bi-info-circle"></i></span></td>
+                                                <td class="pe-0"><input type="number" class="form-control form-control-sm" id="settDirectFloodRetries" min="0" max="5" value="1"></td>
+                                            </tr>
+                                            <tr>
+                                                <td class="ps-0">Interval (s) <span class="badge rounded-pill text-muted" data-bs-toggle="tooltip" title="Seconds to wait between direct retries"><i class="bi bi-info-circle"></i></span></td>
+                                                <td class="pe-0"><input type="number" class="form-control form-control-sm" id="settDirectInterval" min="5" max="300" value="30"></td>
+                                            </tr>
+                                        </tbody>
+                                    </table>
+
+                                    <h6 class="text-muted mb-2">When no path (FLOOD)</h6>
+                                    <table class="table table-sm table-borderless mb-3 align-middle">
+                                        <tbody>
+                                            <tr>
+                                                <td class="ps-0">Max retries <span class="badge rounded-pill text-muted" data-bs-toggle="tooltip" title="Flood retry attempts (also used after path rotation)"><i class="bi bi-info-circle"></i></span></td>
+                                                <td class="pe-0" style="width:5rem"><input type="number" class="form-control form-control-sm" id="settFloodMaxRetries" min="0" max="10" value="3"></td>
+                                            </tr>
+                                            <tr>
+                                                <td class="ps-0">Interval (s) <span class="badge rounded-pill text-muted" data-bs-toggle="tooltip" title="Seconds to wait between flood retries"><i class="bi bi-info-circle"></i></span></td>
+                                                <td class="pe-0"><input type="number" class="form-control form-control-sm" id="settFloodInterval" min="5" max="300" value="60"></td>
+                                            </tr>
+                                        </tbody>
+                                    </table>
+
+                                    <h6 class="text-muted mb-2">Other</h6>
+                                    <table class="table table-sm table-borderless mb-3 align-middle">
+                                        <tbody>
+                                            <tr>
+                                                <td class="ps-0">Grace period (s) <span class="badge rounded-pill text-muted" data-bs-toggle="tooltip" title="Wait for late ACKs after all retries exhausted"><i class="bi bi-info-circle"></i></span></td>
+                                                <td class="pe-0" style="width:5rem"><input type="number" class="form-control form-control-sm" id="settGracePeriod" min="10" max="300" value="60"></td>
+                                            </tr>
+                                        </tbody>
+                                    </table>
+
+                                    <div class="d-flex gap-2">
+                                        <button type="submit" class="btn btn-primary btn-sm">Save</button>
+                                        <button type="button" class="btn btn-outline-secondary btn-sm" id="settingsResetBtn">Reset to defaults</button>
+                                    </div>
+                                </form>
+                            </div>
+                        </div>
+                        <div class="tab-pane fade" id="tabSettingsChat">
+                            <form id="chatSettingsForm">
+                                <h6 class="text-muted mb-2">Quote</h6>
+                                <table class="table table-sm table-borderless mb-3 align-middle">
+                                    <tbody>
+                                        <tr>
+                                            <td class="ps-0">Quote length (bytes) <span class="badge rounded-pill text-muted" data-bs-toggle="tooltip" title="Default max UTF-8 bytes for truncated quotes"><i class="bi bi-info-circle"></i></span></td>
+                                            <td class="pe-0" style="width:5rem"><input type="number" class="form-control form-control-sm" id="settQuoteMaxBytes" min="5" max="120" value="20"></td>
+                                        </tr>
+                                    </tbody>
+                                </table>
+                                <div class="d-flex gap-2">
+                                    <button type="submit" class="btn btn-primary btn-sm">Save</button>
+                                    <button type="button" class="btn btn-outline-secondary btn-sm" id="chatSettingsResetBtn">Reset to defaults</button>
+                                </div>
+                            </form>
+                        </div>
+                        <div class="tab-pane fade" id="tabSettingsAppearance">
+                            <h6 class="text-muted mb-3">Theme</h6>
+                            <div class="d-flex flex-column gap-2">
+                                <div class="theme-option active" data-theme-value="light" onclick="setTheme('light')">
+                                    <div class="theme-option-preview light">
+                                        <i class="bi bi-sun"></i>
+                                    </div>
+                                    <div>
+                                        <div class="theme-option-label">Light</div>
+                                        <div class="theme-option-desc">Classic bright interface</div>
+                                    </div>
+                                </div>
+                                <div class="theme-option" data-theme-value="dark" onclick="setTheme('dark')">
+                                    <div class="theme-option-preview dark">
+                                        <i class="bi bi-moon-stars" style="color: #60a5fa;"></i>
+                                    </div>
+                                    <div>
+                                        <div class="theme-option-label">Dark</div>
+                                        <div class="theme-option-desc">Easy on the eyes, deep navy palette</div>
+                                    </div>
+                                </div>
+                            </div>
+                        </div>
+                    </div>
+                </div>
+            </div>
+        </div>
+    </div>
+
+    <!-- Quote Dialog Modal -->
+    <div class="modal fade" id="quoteModal" tabindex="-1">
+        <div class="modal-dialog modal-sm modal-dialog-centered">
+            <div class="modal-content">
+                <div class="modal-header py-2">
+                    <h6 class="modal-title"><i class="bi bi-quote"></i> Quote message</h6>
+                    <button type="button" class="btn-close btn-close-sm" data-bs-dismiss="modal"></button>
+                </div>
+                <div class="modal-body py-2">
+                    <p class="text-muted small mb-2" id="quotePreview"></p>
+                    <div class="d-flex gap-2 align-items-center mb-2">
+                        <label class="form-label mb-0 small text-nowrap" for="quoteBytesInput">Max bytes:</label>
+                        <input type="number" class="form-control form-control-sm" id="quoteBytesInput" min="5" max="120" style="width:5rem">
+                    </div>
+                </div>
+                <div class="modal-footer py-1">
+                    <button type="button" class="btn btn-outline-secondary btn-sm" id="quoteTruncatedBtn">Truncated</button>
+                    <button type="button" class="btn btn-primary btn-sm" id="quoteFullBtn">Full quote</button>
+                </div>
+            </div>
+        </div>
+    </div>
+
     <!-- Map Modal (Leaflet) -->
     <div class="modal fade" id="mapModal" tabindex="-1">
         <div class="modal-dialog modal-lg modal-dialog-centered">
@@ -300,11 +510,14 @@
                     <!-- Type filter (hidden for single contact view) -->
                     <div id="mapTypeFilter" class="d-none px-3 py-2 border-bottom bg-light">
                         <div class="d-flex flex-wrap gap-2 align-items-center">
-                            <span class="small text-muted me-1">Show:</span>
-                            <span class="map-filter-badge active" data-type="1" id="mapFilterCLI">CLI</span>
+                            <span class="map-filter-badge active" data-type="1" id="mapFilterCOM">COM</span>
                             <span class="map-filter-badge active" data-type="2" id="mapFilterREP">REP</span>
                             <span class="map-filter-badge active" data-type="3" id="mapFilterROOM">ROOM</span>
                             <span class="map-filter-badge active" data-type="4" id="mapFilterSENS">SENS</span>
+                            <div class="form-check form-switch ms-auto mb-0">
+                                <input class="form-check-input" type="checkbox" id="mapCachedSwitch">
+                                <label class="form-check-label small" for="mapCachedSwitch">Cached</label>
+                            </div>
                         </div>
                     </div>
                     <div id="leafletMap" style="height: 400px; width: 100%;"></div>
@@ -349,6 +562,72 @@
         </div>
     </div>
 
+    <!-- Backup Modal -->
+    <div class="modal fade" id="backupModal" tabindex="-1">
+        <div class="modal-dialog">
+            <div class="modal-content">
+                <div class="modal-header">
+                    <h5 class="modal-title"><i class="bi bi-database-down"></i> Database Backup</h5>
+                    <button type="button" class="btn-close" data-bs-dismiss="modal"></button>
+                </div>
+                <div class="modal-body">
+                    <div class="d-flex justify-content-between align-items-center mb-3">
+                        <button class="btn btn-primary" id="createBackupBtn" onclick="createBackup()">
+                            <i class="bi bi-plus-circle"></i> Create Backup
+                        </button>
+                        <span id="backupAutoStatus" class="text-muted small"></span>
+                    </div>
+                    <div id="backupList">
+                        <div class="text-center text-muted py-3">
+                            <div class="spinner-border spinner-border-sm"></div> Loading...
+                        </div>
+                    </div>
+                </div>
+            </div>
+        </div>
+    </div>
+
+    <!-- Search Modal -->
+    <div class="modal fade" id="searchModal" tabindex="-1">
+        <div class="modal-dialog modal-lg">
+            <div class="modal-content">
+                <div class="modal-header">
+                    <h5 class="modal-title"><i class="bi bi-search"></i> Search Messages</h5>
+                    <button type="button" class="btn-close" data-bs-dismiss="modal"></button>
+                </div>
+                <div class="modal-body">
+                    <div class="input-group mb-3">
+                        <input type="text" class="form-control" id="searchInput" placeholder="Search all messages..." autofocus>
+                        <button class="btn btn-primary" type="button" id="searchBtn">
+                            <i class="bi bi-search"></i>
+                        </button>
+                        <button class="btn btn-outline-secondary" type="button" id="searchHelpBtn" title="Search syntax help">
+                            <i class="bi bi-question-circle"></i>
+                        </button>
+                    </div>
+                    <div id="searchHelp" class="alert alert-light small mb-3" style="display:none;">
+                        <strong>Search tips:</strong>
+                        <ul class="mb-1 ps-3">
+                            <li><code>hello world</code> — messages containing both words</li>
+                            <li><code>"hello world"</code> — exact phrase</li>
+                            <li><code>hello OR world</code> — either word</li>
+                            <li><code>hello NOT world</code> — hello but not world</li>
+                            <li><code>hell*</code> — prefix match (hello,hellas...)</li>
+                        </ul>
+                        <div class="text-muted">Special characters (<code>. , - :</code>) should be wrapped in quotes.<br>
+                        <a href="https://www.sqlite.org/fts5.html#full_text_query_syntax" target="_blank" rel="noopener">Full FTS5 syntax reference <i class="bi bi-box-arrow-up-right"></i></a></div>
+                    </div>
+                    <div id="searchResults">
+                        <div class="text-center text-muted py-4">
+                            <i class="bi bi-search" style="font-size: 2rem;"></i>
+                            <p class="mt-2 mb-0">Search across all channel and direct messages</p>
+                        </div>
+                    </div>
+                </div>
+            </div>
+        </div>
+    </div>
+
     <!-- Toast container for notifications -->
     <div class="toast-container position-fixed top-0 start-0 p-3">
         <div id="notificationToast" class="toast" role="alert">
@@ -374,7 +653,13 @@
     <!-- Filter Utilities (must load before app.js) -->
     <script src="{{ url_for('static', filename='js/filter-utils.js') }}"></script>
 
+    <!-- SocketIO for real-time updates -->
+    <script src="{{ url_for('static', filename='vendor/socket.io/socket.io.min.js') }}"></script>
+
     <!-- Custom JS -->
+    <!-- QR Code generator (for Device Share) -->
+    <script src="https://cdn.jsdelivr.net/npm/qrcodejs@1.0.0/qrcode.min.js"></script>
+
     <script src="{{ url_for('static', filename='js/app.js') }}"></script>
 
     <!-- PWA Viewport Fix for Android -->
@@ -408,6 +693,27 @@
         }
     </script>
 
+    <!-- Theme Switching -->
+    <script>
+        function setTheme(theme) {
+            document.documentElement.setAttribute('data-theme', theme);
+            document.documentElement.setAttribute('data-bs-theme', theme);
+            localStorage.setItem('mc-webui-theme', theme);
+            // Update theme selector UI
+            document.querySelectorAll('.theme-option').forEach(function(el) {
+                el.classList.toggle('active', el.getAttribute('data-theme-value') === theme);
+            });
+        }
+
+        // Initialize theme selector UI on settings modal open
+        document.addEventListener('DOMContentLoaded', function() {
+            var current = localStorage.getItem('mc-webui-theme') || 'light';
+            document.querySelectorAll('.theme-option').forEach(function(el) {
+                el.classList.toggle('active', el.getAttribute('data-theme-value') === current);
+            });
+        });
+    </script>
+
     {% block extra_scripts %}{% endblock %}
 </body>
 </html>

app/templates/contacts-add.html

@@ -0,0 +1,128 @@
+{% extends "contacts_base.html" %}
+
+{% block title %}Add Contact - mc-webui{% endblock %}
+
+{% block extra_head %}
+<!-- html5-qrcode for QR scanning -->
+<script src="https://unpkg.com/html5-qrcode@2.3.8/html5-qrcode.min.js"></script>
+{% endblock %}
+
+{% block page_content %}
+<div id="addPageContent" class="p-3">
+    <!-- Page Header -->
+    <div class="mb-3">
+        <h4 class="mb-2">
+            <i class="bi bi-person-plus"></i> Add Contact
+        </h4>
+    </div>
+
+    <!-- Action Buttons -->
+    <div class="d-flex gap-2 mb-3">
+        <button class="btn btn-outline-secondary btn-sm" onclick="navigateTo('/contacts/manage');">
+            <i class="bi bi-arrow-left"></i> Back
+        </button>
+    </div>
+
+    <!-- Input Mode Tabs -->
+    <ul class="nav nav-tabs mb-3" role="tablist">
+        <li class="nav-item" role="presentation">
+            <button class="nav-link active" id="tab-uri" data-bs-toggle="tab" data-bs-target="#pane-uri" type="button" role="tab">
+                <i class="bi bi-link-45deg"></i> URI
+            </button>
+        </li>
+        <li class="nav-item" role="presentation">
+            <button class="nav-link" id="tab-qr" data-bs-toggle="tab" data-bs-target="#pane-qr" type="button" role="tab">
+                <i class="bi bi-qr-code-scan"></i> QR Code
+            </button>
+        </li>
+        <li class="nav-item" role="presentation">
+            <button class="nav-link" id="tab-manual" data-bs-toggle="tab" data-bs-target="#pane-manual" type="button" role="tab">
+                <i class="bi bi-pencil"></i> Manual
+            </button>
+        </li>
+    </ul>
+
+    <div class="tab-content">
+        <!-- URI Paste Tab -->
+        <div class="tab-pane fade show active" id="pane-uri" role="tabpanel">
+            <div class="mb-3">
+                <label for="uriInput" class="form-label">MeshCore URI:</label>
+                <textarea class="form-control font-monospace" id="uriInput" rows="3"
+                    placeholder="meshcore://contact/add?name=...&public_key=...&type=..."></textarea>
+                <small class="form-text text-muted">Paste a meshcore:// URI from the MeshCore mobile app</small>
+            </div>
+            <!-- URI Preview -->
+            <div id="uriPreview" class="alert alert-info d-none mb-3">
+                <strong>Preview:</strong>
+                <div><span class="text-muted">Name:</span> <span id="uriPreviewName"></span></div>
+                <div><span class="text-muted">Key:</span> <span id="uriPreviewKey" class="font-monospace small" style="word-break: break-all;"></span></div>
+                <div><span class="text-muted">Type:</span> <span id="uriPreviewType"></span></div>
+            </div>
+            <button class="btn btn-success" id="addFromUriBtn" disabled>
+                <i class="bi bi-plus-circle"></i> Add Contact
+            </button>
+        </div>
+
+        <!-- QR Code Tab -->
+        <div class="tab-pane fade" id="pane-qr" role="tabpanel">
+            <!-- Camera Scanner -->
+            <div id="qrScannerContainer" class="mb-3">
+                <div id="qrReader" style="width: 100%; max-width: 500px;"></div>
+                <div id="qrCameraButtons" class="d-flex gap-2 mt-2">
+                    <button class="btn btn-primary btn-sm" id="startCameraBtn">
+                        <i class="bi bi-camera-video"></i> Start Camera
+                    </button>
+                    <button class="btn btn-outline-secondary btn-sm d-none" id="stopCameraBtn">
+                        <i class="bi bi-stop-circle"></i> Stop Camera
+                    </button>
+                </div>
+            </div>
+            <!-- File Upload Fallback -->
+            <div class="mb-3">
+                <label for="qrFileInput" class="form-label">Or upload a QR code image:</label>
+                <input type="file" class="form-control" id="qrFileInput" accept="image/*">
+            </div>
+            <!-- QR Result -->
+            <div id="qrResult" class="alert alert-success d-none mb-3">
+                <strong>Scanned:</strong>
+                <div><span class="text-muted">Name:</span> <span id="qrResultName"></span></div>
+                <div><span class="text-muted">Key:</span> <span id="qrResultKey" class="font-monospace small" style="word-break: break-all;"></span></div>
+                <div><span class="text-muted">Type:</span> <span id="qrResultType"></span></div>
+            </div>
+            <div id="qrError" class="alert alert-danger d-none mb-3"></div>
+            <button class="btn btn-success d-none" id="addFromQrBtn">
+                <i class="bi bi-plus-circle"></i> Add Contact
+            </button>
+        </div>
+
+        <!-- Manual Entry Tab -->
+        <div class="tab-pane fade" id="pane-manual" role="tabpanel">
+            <div class="mb-3">
+                <label for="manualName" class="form-label">Name:</label>
+                <input type="text" class="form-control" id="manualName" placeholder="Contact name" maxlength="32">
+            </div>
+            <div class="mb-3">
+                <label for="manualKey" class="form-label">Public Key (64 hex chars):</label>
+                <input type="text" class="form-control font-monospace" id="manualKey"
+                    placeholder="e.g. a1b2c3d4..." maxlength="64" pattern="[0-9a-fA-F]{64}">
+                <small class="form-text text-muted" id="manualKeyCount">0 / 64 characters</small>
+            </div>
+            <div class="mb-3">
+                <label for="manualType" class="form-label">Contact Type:</label>
+                <select class="form-select" id="manualType">
+                    <option value="1" selected>COM (Companion)</option>
+                    <option value="2">REP (Repeater)</option>
+                    <option value="3">ROOM (Room Server)</option>
+                    <option value="4">SENS (Sensor)</option>
+                </select>
+            </div>
+            <button class="btn btn-success" id="addManualBtn" disabled>
+                <i class="bi bi-plus-circle"></i> Add Contact
+            </button>
+        </div>
+    </div>
+
+    <!-- Status Messages -->
+    <div id="addStatus" class="mt-3 d-none"></div>
+</div>
+{% endblock %}

app/templates/contacts-existing.html

@@ -29,26 +29,31 @@
 
     <!-- Filter and Sort Toolbar -->
     <div class="filter-sort-toolbar">
+        <!-- Source Filter -->
+        <select class="form-select" id="sourceFilter">
+            <option value="ALL">All sources</option>
+            <option value="DEVICE">On device</option>
+            <option value="CACHE">Cache only</option>
+            <option value="IGNORED">Ignored</option>
+            <option value="BLOCKED">Blocked</option>
+        </select>
+
         <!-- Type Filter -->
         <select class="form-select" id="typeFilter">
             <option value="ALL">All Types</option>
-            <option value="CLI">CLI</option>
+            <option value="COM">COM</option>
             <option value="REP">REP</option>
             <option value="ROOM">ROOM</option>
             <option value="SENS">SENS</option>
         </select>
 
-        <!-- Sort Buttons -->
-        <div class="sort-buttons">
-            <button class="sort-btn" data-sort="name" id="sortByName" title="Sort by contact name">
-                <span>Name</span>
-                <i class="bi bi-sort-down"></i>
-            </button>
-            <button class="sort-btn active" data-sort="last_advert" id="sortByLastAdvert" title="Sort by last advertisement time">
-                <span>Last advert</span>
-                <i class="bi bi-sort-down"></i>
-            </button>
-        </div>
+        <!-- Sort Dropdown -->
+        <select class="form-select" id="sortSelect">
+            <option value="last_advert_desc">Last advert ↓</option>
+            <option value="last_advert_asc">Last advert ↑</option>
+            <option value="name_asc">Name A→Z</option>
+            <option value="name_desc">Name Z→A</option>
+        </select>
     </div>
 
     <!-- Loading State -->

app/templates/contacts-manage.html

@@ -32,6 +32,15 @@
             <i class="bi bi-list-ul"></i> Manage Contacts
         </h5>
 
+        <!-- Add Contact Card -->
+        <div class="nav-card" onclick="navigateTo('/contacts/add');" style="border-left: 4px solid #198754;">
+            <div>
+                <h6><i class="bi bi-person-plus"></i> Add Contact</h6>
+                <small class="text-muted">Add from URI, QR code, or manual entry</small>
+            </div>
+            <i class="bi bi-chevron-right text-muted"></i>
+        </div>
+
         <!-- Pending Contacts Card -->
         <div class="nav-card" onclick="navigateTo('/contacts/pending');">
             <div>
@@ -85,8 +94,8 @@
                 <label class="form-label">Contact Types:</label>
                 <div class="d-flex flex-wrap gap-2">
                     <div class="form-check">
-                        <input class="form-check-input cleanup-type-filter" type="checkbox" value="1" id="cleanupTypeCLI" checked>
-                        <label class="form-check-label" for="cleanupTypeCLI">CLI</label>
+                        <input class="form-check-input cleanup-type-filter" type="checkbox" value="1" id="cleanupTypeCOM" checked>
+                        <label class="form-check-label" for="cleanupTypeCOM">COM</label>
                     </div>
                     <div class="form-check">
                         <input class="form-check-input cleanup-type-filter" type="checkbox" value="2" id="cleanupTypeREP" checked>

app/templates/contacts-pending.html

@@ -26,44 +26,36 @@
     <div class="mb-3">
         <div class="card">
             <div class="card-body p-3">
-                <h6 class="mb-3"><i class="bi bi-funnel"></i> Filters</h6>
-
-                <!-- Name Search -->
-                <div class="mb-3">
-                    <input type="text" class="form-control" id="pendingSearchInput"
-                           placeholder="Search by name or public key...">
-                </div>
+                <h6 class="mb-2">
+                    <i class="bi bi-funnel"></i> Filters
+                    <i class="bi bi-info-circle text-muted ms-1" role="button" tabindex="0"
+                       data-bs-toggle="tooltip" data-bs-placement="right"
+                       title="Filter contacts by type or name, then approve or ignore them in bulk."></i>
+                </h6>
 
                 <!-- Type Filter Badges -->
-                <div class="mb-3">
-                    <label class="form-label small text-muted">Contact Types:</label>
-                    <div class="d-flex flex-wrap gap-2">
-                        <span class="type-filter-badge active" data-type="CLI" id="typeFilterCLI">CLI</span>
-                        <span class="type-filter-badge" data-type="REP" id="typeFilterREP">REP</span>
-                        <span class="type-filter-badge" data-type="ROOM" id="typeFilterROOM">ROOM</span>
-                        <span class="type-filter-badge" data-type="SENS" id="typeFilterSENS">SENS</span>
-                    </div>
+                <div class="d-flex flex-wrap gap-2 mb-2">
+                    <span class="type-filter-badge active" data-type="COM" id="typeFilterCOM">COM</span>
+                    <span class="type-filter-badge" data-type="REP" id="typeFilterREP">REP</span>
+                    <span class="type-filter-badge" data-type="ROOM" id="typeFilterROOM">ROOM</span>
+                    <span class="type-filter-badge" data-type="SENS" id="typeFilterSENS">SENS</span>
                 </div>
 
-                <!-- Batch Approval Button -->
+                <!-- Search + Batch Action Buttons -->
                 <div class="d-flex gap-2">
-                    <button class="btn btn-success flex-grow-1" id="addFilteredBtn">
-                        <i class="bi bi-check-circle-fill"></i> Add Filtered
-                        <span class="badge bg-light text-dark ms-2" id="filteredCountBadge">0</span>
+                    <input type="text" class="form-control form-control-sm" id="pendingSearchInput"
+                           placeholder="Search..." style="min-width: 0;">
+                    <button class="btn btn-success btn-sm flex-shrink-0" id="addFilteredBtn">
+                        <i class="bi bi-check-circle-fill"></i> Approve
+                    </button>
+                    <button class="btn btn-outline-secondary btn-sm flex-shrink-0" id="ignoreFilteredBtn">
+                        <i class="bi bi-eye-slash"></i> Ignore
                     </button>
                 </div>
             </div>
         </div>
     </div>
 
-    <!-- Page Description -->
-    <div class="mb-3">
-        <p class="text-muted small mb-0">
-            <i class="bi bi-info-circle"></i>
-            Approve or reject contacts waiting for manual approval.
-        </p>
-    </div>
-
     <!-- Loading State -->
     <div id="pendingLoading" class="text-center py-5" style="display: none;">
         <div class="spinner-border text-primary" role="status">
@@ -122,4 +114,34 @@
         </div>
     </div>
 </div>
+
+<!-- Batch Ignore Confirmation Modal -->
+<div class="modal fade" id="batchIgnoreModal" tabindex="-1" aria-labelledby="batchIgnoreModalLabel" aria-hidden="true">
+    <div class="modal-dialog modal-dialog-scrollable">
+        <div class="modal-content">
+            <div class="modal-header bg-secondary text-white">
+                <h5 class="modal-title" id="batchIgnoreModalLabel">
+                    <i class="bi bi-eye-slash"></i> Confirm Batch Ignore
+                </h5>
+                <button type="button" class="btn-close btn-close-white" data-bs-dismiss="modal" aria-label="Close"></button>
+            </div>
+            <div class="modal-body">
+                <p class="mb-2">You are about to ignore <strong id="batchIgnoreCount">0</strong> contacts:</p>
+
+                <div class="list-group mb-3" id="batchIgnoreList" style="max-height: 300px; overflow-y: auto;">
+                </div>
+
+                <div class="alert alert-warning mb-0">
+                    <i class="bi bi-info-circle"></i> Ignored contacts will not trigger new pending requests. You can unignore them later from the Existing Contacts page.
+                </div>
+            </div>
+            <div class="modal-footer">
+                <button type="button" class="btn btn-secondary" data-bs-dismiss="modal">Cancel</button>
+                <button type="button" class="btn btn-outline-secondary" id="confirmBatchIgnoreBtn">
+                    <i class="bi bi-eye-slash"></i> Ignore All
+                </button>
+            </div>
+        </div>
+    </div>
+</div>
 {% endblock %}

app/templates/contacts.html

@@ -3,208 +3,6 @@
 {% block title %}Contact Management - mc-webui{% endblock %}
 
 {% block extra_head %}
-<style>
-    /* Mobile-first custom styles for Contact Management */
-    /* Compact manual approval section */
-    .compact-setting {
-        display: flex;
-        align-items: center;
-        gap: 0.75rem;
-        padding: 0.75rem;
-        background-color: #f8f9fa;
-        border-radius: 0.5rem;
-        margin-bottom: 1rem;
-    }
-
-    .info-icon {
-        color: #6c757d;
-        cursor: help;
-        font-size: 1.1rem;
-    }
-
-    .info-icon:hover {
-        color: #0d6efd;
-    }
-
-    .pending-contact-card {
-        background-color: white;
-        border: 1px solid #dee2e6;
-        border-radius: 0.5rem;
-        padding: 1rem;
-        margin-bottom: 0.75rem;
-        box-shadow: 0 1px 3px rgba(0, 0, 0, 0.1);
-    }
-
-    .contact-name {
-        font-size: 1.1rem;
-        font-weight: 600;
-        color: #212529;
-        margin-bottom: 0.5rem;
-        word-wrap: break-word;
-    }
-
-    .contact-key {
-        font-family: 'Courier New', monospace;
-        font-size: 0.85rem;
-        color: #6c757d;
-        word-break: break-all;
-        margin-bottom: 0.75rem;
-    }
-
-    .btn-action {
-        min-height: 44px; /* Touch-friendly size */
-        font-size: 1rem;
-    }
-
-    .empty-state {
-        text-align: center;
-        padding: 1.5rem 1rem;
-        color: #6c757d;
-    }
-
-    .empty-state i {
-        font-size: 2rem;
-        margin-bottom: 0.5rem;
-        opacity: 0.5;
-    }
-
-    .empty-state.compact {
-        padding: 1rem;
-    }
-
-    .empty-state.compact i {
-        font-size: 1.5rem;
-        margin-bottom: 0.25rem;
-    }
-
-    .info-badge {
-        display: inline-block;
-        background-color: #e7f3ff;
-        color: #0c5460;
-        padding: 0.5rem 1rem;
-        border-radius: 0.375rem;
-        font-size: 0.9rem;
-        margin-top: 0.5rem;
-    }
-
-    /* Existing Contacts Styles */
-    .existing-contact-card {
-        background-color: white;
-        border: 1px solid #dee2e6;
-        border-radius: 0.5rem;
-        padding: 1rem;
-        margin-bottom: 0.75rem;
-        box-shadow: 0 1px 3px rgba(0, 0, 0, 0.1);
-        transition: box-shadow 0.2s;
-    }
-
-    .existing-contact-card:hover {
-        box-shadow: 0 2px 8px rgba(0, 0, 0, 0.15);
-    }
-
-    .type-badge {
-        font-size: 0.75rem;
-        padding: 0.25rem 0.5rem;
-        font-weight: 600;
-    }
-
-    .contact-info-row {
-        display: flex;
-        align-items: center;
-        gap: 0.5rem;
-        margin-bottom: 0.5rem;
-        flex-wrap: wrap;
-    }
-
-    .counter-badge {
-        font-size: 1rem;
-        padding: 0.35rem 0.75rem;
-    }
-
-    .counter-ok {
-        background-color: #28a745;
-    }
-
-    .counter-warning {
-        background-color: #ffc107;
-        color: #212529;
-    }
-
-    .counter-alarm {
-        background-color: #dc3545;
-        animation: pulse 1.5s infinite;
-    }
-
-    @keyframes pulse {
-        0%, 100% { opacity: 1; }
-        50% { opacity: 0.7; }
-    }
-
-    .search-toolbar {
-        display: flex;
-        gap: 0.5rem;
-        margin-bottom: 1rem;
-        flex-wrap: wrap;
-    }
-
-    .search-toolbar input,
-    .search-toolbar select {
-        flex: 1;
-        min-width: 150px;
-    }
-
-    /* Scrollable contacts lists */
-    #pendingList {
-        max-height: 200px;
-        overflow-y: auto;
-        overflow-x: hidden;
-        -webkit-overflow-scrolling: touch;
-    }
-
-    #existingList {
-        /* No max-height limit - let it use available space */
-        overflow-y: auto;
-        overflow-x: hidden;
-        -webkit-overflow-scrolling: touch;
-        /* Dynamic height based on viewport */
-        max-height: calc(100vh - 400px);
-        min-height: 300px;
-    }
-
-    @media (max-width: 768px) {
-        #existingList {
-            max-height: calc(100vh - 450px);
-        }
-    }
-
-    /* Custom scrollbar styling */
-    #existingList::-webkit-scrollbar,
-    #pendingList::-webkit-scrollbar {
-        width: 8px;
-    }
-
-    #existingList::-webkit-scrollbar-track,
-    #pendingList::-webkit-scrollbar-track {
-        background: #f1f1f1;
-        border-radius: 4px;
-    }
-
-    #existingList::-webkit-scrollbar-thumb,
-    #pendingList::-webkit-scrollbar-thumb {
-        background: #888;
-        border-radius: 4px;
-    }
-
-    #existingList::-webkit-scrollbar-thumb:hover,
-    #pendingList::-webkit-scrollbar-thumb:hover {
-        background: #555;
-    }
-
-    /* Compact section headers */
-    .section-compact {
-        margin-bottom: 0.75rem;
-    }
-</style>
 {% endblock %}
 
 {% block content %}
@@ -286,7 +84,7 @@
                     <input type="text" class="form-control" id="searchInput" placeholder="Search by name or public key...">
                     <select class="form-select" id="typeFilter" style="max-width: 150px;">
                         <option value="ALL">All Types</option>
-                        <option value="CLI">CLI</option>
+                        <option value="COM">COM</option>
                         <option value="REP">REP</option>
                         <option value="ROOM">ROOM</option>
                         <option value="SENS">SENS</option>

app/templates/contacts_base.html

@@ -1,10 +1,19 @@
 <!DOCTYPE html>
-<html lang="en">
+<html lang="en" data-theme="light" data-bs-theme="light">
 <head>
     <meta charset="UTF-8">
     <meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover">
     <title>{% block title %}Contact Management - mc-webui{% endblock %}</title>
 
+    <!-- Theme: apply saved preference before CSS loads to prevent flash -->
+    <script>
+        (function() {
+            var t = localStorage.getItem('mc-webui-theme') || 'light';
+            document.documentElement.setAttribute('data-theme', t);
+            document.documentElement.setAttribute('data-bs-theme', t);
+        })();
+    </script>
+
     <!-- Favicon -->
     <link rel="apple-touch-icon" sizes="180x180" href="{{ url_for('static', filename='images/apple-touch-icon.png') }}">
     <link rel="icon" type="image/png" sizes="32x32" href="{{ url_for('static', filename='images/favicon-32x32.png') }}">
@@ -24,126 +33,11 @@
 
     <!-- Custom CSS -->
     <link rel="stylesheet" href="{{ url_for('static', filename='css/style.css') }}">
+    <!-- Theme CSS (light/dark mode) -->
+    <link rel="stylesheet" href="{{ url_for('static', filename='css/theme.css') }}">
 
     <style>
-        /* Mobile-first custom styles for Contact Management */
-        /* Compact manual approval section */
-        .compact-setting {
-            display: flex;
-            align-items: center;
-            gap: 0.75rem;
-            padding: 0.75rem;
-            background-color: #f8f9fa;
-            border-radius: 0.5rem;
-            margin-bottom: 1rem;
-        }
-
-        .info-icon {
-            color: #6c757d;
-            cursor: help;
-            font-size: 1.1rem;
-        }
-
-        .info-icon:hover {
-            color: #0d6efd;
-        }
-
-        .pending-contact-card {
-            background-color: white;
-            border: 1px solid #dee2e6;
-            border-radius: 0.5rem;
-            padding: 1rem;
-            margin-bottom: 0.75rem;
-            box-shadow: 0 1px 3px rgba(0, 0, 0, 0.1);
-        }
-
-        .contact-name {
-            font-size: 1.1rem;
-            font-weight: 600;
-            color: #212529;
-            margin-bottom: 0.5rem;
-            word-wrap: break-word;
-        }
-
-        .contact-key {
-            font-family: 'Courier New', monospace;
-            font-size: 0.85rem;
-            color: #6c757d;
-            word-break: break-all;
-            margin-bottom: 0.75rem;
-        }
-
-        .contact-key.clickable-key {
-            cursor: pointer;
-            transition: color 0.15s, background-color 0.15s;
-            padding: 0.15rem 0.3rem;
-            margin-left: -0.3rem;
-            border-radius: 0.25rem;
-        }
-
-        .contact-key.clickable-key:hover {
-            color: #0d6efd;
-            background-color: #e7f1ff;
-        }
-
-        .contact-key.clickable-key.copied {
-            color: #198754;
-            background-color: #d1e7dd;
-        }
-
-        .empty-state {
-            text-align: center;
-            padding: 1.5rem 1rem;
-            color: #6c757d;
-        }
-
-        .empty-state i {
-            font-size: 2rem;
-            margin-bottom: 0.5rem;
-            opacity: 0.5;
-        }
-
-        .empty-state.compact {
-            padding: 1rem;
-        }
-
-        .empty-state.compact i {
-            font-size: 1.5rem;
-            margin-bottom: 0.25rem;
-        }
-
-        .info-badge {
-            display: inline-block;
-            background-color: #e7f3ff;
-            color: #0c5460;
-            padding: 0.5rem 1rem;
-            border-radius: 0.375rem;
-            font-size: 0.9rem;
-            margin-top: 0.5rem;
-        }
-
-        /* Existing Contacts Styles */
-        .existing-contact-card {
-            background-color: white;
-            border: 1px solid #dee2e6;
-            border-radius: 0.5rem;
-            padding: 1rem;
-            margin-bottom: 0.75rem;
-            box-shadow: 0 1px 3px rgba(0, 0, 0, 0.1);
-            transition: box-shadow 0.2s;
-        }
-
-        .existing-contact-card:hover {
-            box-shadow: 0 2px 8px rgba(0, 0, 0, 0.15);
-        }
-
-        .type-badge {
-            font-size: 0.75rem;
-            padding: 0.25rem 0.5rem;
-            font-weight: 600;
-        }
-
-        /* Protected contact styling */
+        /* Contact Management page layout overrides */
         .protection-indicator {
             font-size: 0.85rem;
         }
@@ -165,19 +59,19 @@
             user-select: none;
         }
 
-        .type-filter-badge[data-type="CLI"] {
+        .type-filter-badge[data-type="COM"] {
             color: #0d6efd;
-            background-color: white;
+            background-color: var(--map-badge-inactive-bg);
             border: 2px solid #0d6efd;
         }
-        .type-filter-badge[data-type="CLI"].active {
+        .type-filter-badge[data-type="COM"].active {
             color: white;
             background-color: #0d6efd;
         }
 
         .type-filter-badge[data-type="REP"] {
             color: #198754;
-            background-color: white;
+            background-color: var(--map-badge-inactive-bg);
             border: 2px solid #198754;
         }
         .type-filter-badge[data-type="REP"].active {
@@ -187,7 +81,7 @@
 
         .type-filter-badge[data-type="ROOM"] {
             color: #0dcaf0;
-            background-color: white;
+            background-color: var(--map-badge-inactive-bg);
             border: 2px solid #0dcaf0;
         }
         .type-filter-badge[data-type="ROOM"].active {
@@ -197,7 +91,7 @@
 
         .type-filter-badge[data-type="SENS"] {
             color: #ffc107;
-            background-color: white;
+            background-color: var(--map-badge-inactive-bg);
             border: 2px solid #ffc107;
         }
         .type-filter-badge[data-type="SENS"].active {
@@ -205,108 +99,29 @@
             background-color: #ffc107;
         }
 
-        .contact-info-row {
-            display: flex;
-            align-items: center;
-            gap: 0.5rem;
-            margin-bottom: 0.5rem;
-            flex-wrap: wrap;
-        }
-
-        .counter-badge {
-            font-size: 1rem;
-            padding: 0.35rem 0.75rem;
-        }
-
-        .counter-ok {
-            background-color: #28a745;
-        }
-
-        .counter-warning {
-            background-color: #ffc107;
-            color: #212529;
-        }
-
-        .counter-alarm {
-            background-color: #dc3545;
-            animation: pulse 1.5s infinite;
-        }
-
-        @keyframes pulse {
-            0%, 100% { opacity: 1; }
-            50% { opacity: 0.7; }
-        }
-
-        .search-toolbar {
-            display: flex;
-            gap: 0.5rem;
-            margin-bottom: 1rem;
-            flex-wrap: wrap;
-        }
-
-        .search-toolbar input,
-        .search-toolbar select {
-            flex: 1;
-            min-width: 150px;
-        }
-
-        /* Scrollable contacts lists */
-        #pendingList {
-            height: calc(100vh - 280px);
-            overflow-y: auto;
-            overflow-x: hidden;
-            -webkit-overflow-scrolling: touch;
-            min-height: 300px;
-        }
-
+        /* Scrollable contacts lists - use flexbox to fill remaining space */
+        #pendingList,
         #existingList {
+            flex: 1 1 0;
+            min-height: 0;
             overflow-y: auto;
             overflow-x: hidden;
             -webkit-overflow-scrolling: touch;
-            height: calc(100vh - 260px);
-            min-height: 300px;
         }
 
-        /* Custom scrollbar styling */
-        #existingList::-webkit-scrollbar,
-        #pendingList::-webkit-scrollbar {
-            width: 8px;
-        }
-
-        #existingList::-webkit-scrollbar-track,
-        #pendingList::-webkit-scrollbar-track {
-            background: #f1f1f1;
-            border-radius: 4px;
-        }
-
-        #existingList::-webkit-scrollbar-thumb,
-        #pendingList::-webkit-scrollbar-thumb {
-            background: #888;
-            border-radius: 4px;
-        }
-
-        #existingList::-webkit-scrollbar-thumb:hover,
-        #pendingList::-webkit-scrollbar-thumb:hover {
-            background: #555;
-        }
-
-        /* Compact section headers */
-        .section-compact {
-            margin-bottom: 0.75rem;
-        }
-
-        /* NEW: Full-screen lists for dedicated pages */
+        /* Full-screen lists for dedicated pages */
         .contacts-list-fullscreen {
-            height: calc(100vh - 240px);
+            flex: 1 1 0;
+            min-height: 0;
             overflow-y: auto;
             -webkit-overflow-scrolling: touch;
             padding: 0;
         }
 
-        /* NEW: Navigation cards on manage page */
+        /* Navigation cards on manage page */
         .nav-card {
-            background: white;
-            border: 1px solid #dee2e6;
+            background: var(--card-bg);
+            border: 1px solid var(--border-color);
             border-radius: 0.5rem;
             padding: 1.25rem;
             margin-bottom: 1rem;
@@ -318,7 +133,7 @@
         }
 
         .nav-card:hover {
-            box-shadow: 0 4px 12px rgba(0, 0, 0, 0.15);
+            box-shadow: var(--card-shadow-hover);
         }
 
         .nav-card h6 {
@@ -340,40 +155,7 @@
             font-size: 0.85rem;
         }
 
-        .sort-buttons {
-            display: flex;
-            gap: 0.375rem;
-        }
-
-        .sort-btn {
-            display: flex;
-            align-items: center;
-            gap: 0.2rem;
-            padding: 0.35rem 0.5rem;
-            background: #f8f9fa;
-            border: 1px solid #dee2e6;
-            border-radius: 0.375rem;
-            cursor: pointer;
-            font-size: 0.85rem;
-            transition: all 0.2s;
-            white-space: nowrap;
-        }
-
-        .sort-btn:hover {
-            background: #e9ecef;
-        }
-
-        .sort-btn.active {
-            background-color: #0d6efd;
-            color: white;
-            border-color: #0d6efd;
-        }
-
-        .sort-btn i {
-            font-size: 0.85rem;
-        }
-
-        /* NEW: Back buttons */
+        /* Back buttons */
         .back-buttons {
             display: flex;
             gap: 0.5rem;
@@ -386,7 +168,7 @@
             min-height: 44px;
         }
 
-        /* NEW: Cleanup section on manage page */
+        /* Cleanup section on manage page */
         .cleanup-section {
             background-color: #fff3cd;
             border: 1px solid #ffc107;
@@ -395,6 +177,11 @@
             margin-bottom: 1.5rem;
         }
 
+        [data-theme="dark"] .cleanup-section {
+            background-color: rgba(255, 193, 7, 0.1);
+            border-color: rgba(255, 193, 7, 0.3);
+        }
+
         .cleanup-section h6 {
             color: #856404;
             margin-bottom: 0.75rem;
@@ -403,28 +190,46 @@
             gap: 0.5rem;
         }
 
+        [data-theme="dark"] .cleanup-section h6 {
+            color: #ffc107;
+        }
+
         /* Override global overflow: hidden from style.css for Contact Management pages */
         html, body {
             overflow: auto !important;
+            height: 100%;
+        }
+
+        body {
+            display: flex;
+            flex-direction: column;
+            background-color: var(--bg-body);
+            color: var(--text-primary);
         }
 
         main {
             overflow: auto !important;
+            flex: 1 1 0;
+            min-height: 0;
+            display: flex;
+            flex-direction: column;
         }
 
-        /* Mobile responsiveness */
-        @media (max-width: 768px) {
-            #existingList {
-                height: calc(100vh - 300px);
-            }
+        main > .container-fluid,
+        main > .container-fluid > .row,
+        main > .container-fluid > .row > .col-12 {
+            flex: 1 1 0;
+            min-height: 0;
+            display: flex;
+            flex-direction: column;
+        }
 
-            #pendingList {
-                height: calc(100vh - 320px);
-            }
-
-            .contacts-list-fullscreen {
-                height: calc(100vh - 300px);
-            }
+        #pendingPageContent,
+        #existingPageContent {
+            flex: 1 1 0;
+            min-height: 0;
+            display: flex;
+            flex-direction: column;
         }
     </style>
 
@@ -496,7 +301,7 @@
                     <div id="mapTypeFilter" class="d-none px-3 py-2 border-bottom bg-light">
                         <div class="d-flex flex-wrap gap-2 align-items-center">
                             <span class="small text-muted me-1">Show:</span>
-                            <span class="map-filter-badge active" data-type="1" id="mapFilterCLI">CLI</span>
+                            <span class="map-filter-badge active" data-type="1" id="mapFilterCOM">COM</span>
                             <span class="map-filter-badge active" data-type="2" id="mapFilterREP">REP</span>
                             <span class="map-filter-badge active" data-type="3" id="mapFilterROOM">ROOM</span>
                             <span class="map-filter-badge active" data-type="4" id="mapFilterSENS">SENS</span>

app/templates/dm.html

@@ -1,10 +1,19 @@
 <!DOCTYPE html>
-<html lang="en">
+<html lang="en" data-theme="light" data-bs-theme="light">
 <head>
     <meta charset="UTF-8">
     <meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover">
     <title>Direct Messages - mc-webui</title>
 
+    <!-- Theme: apply saved preference before CSS loads to prevent flash -->
+    <script>
+        (function() {
+            var t = localStorage.getItem('mc-webui-theme') || 'light';
+            document.documentElement.setAttribute('data-theme', t);
+            document.documentElement.setAttribute('data-bs-theme', t);
+        })();
+    </script>
+
     <!-- Favicon -->
     <link rel="apple-touch-icon" sizes="180x180" href="{{ url_for('static', filename='images/apple-touch-icon.png') }}">
     <link rel="icon" type="image/png" sizes="32x32" href="{{ url_for('static', filename='images/favicon-32x32.png') }}">
@@ -17,146 +26,308 @@
     <!-- Bootstrap Icons (local) -->
     <link rel="stylesheet" href="{{ url_for('static', filename='vendor/bootstrap-icons/bootstrap-icons.css') }}">
 
+    <!-- Leaflet CSS (for repeater map picker) -->
+    <link rel="stylesheet" href="https://unpkg.com/leaflet@1.9.4/dist/leaflet.css"
+          integrity="sha256-p4NxAoJBhIIN+hmNHrzRCf9tD/miZyoHS5obTRR9BMY="
+          crossorigin="" />
+
     <!-- Custom CSS -->
     <link rel="stylesheet" href="{{ url_for('static', filename='css/style.css') }}">
+    <!-- Theme CSS (light/dark mode) -->
+    <link rel="stylesheet" href="{{ url_for('static', filename='css/theme.css') }}">
 
     <!-- Emoji Picker (local) -->
     <script type="module" src="{{ url_for('static', filename='vendor/emoji-picker-element/index.js') }}"></script>
-    <style>
-        emoji-picker {
-            --emoji-size: 1.5rem;
-            --num-columns: 8;
-        }
-        .emoji-picker-container {
-            position: relative;
-        }
-        .emoji-picker-popup {
-            position: absolute;
-            bottom: 100%;
-            right: 0;
-            z-index: 1000;
-            margin-bottom: 0.5rem;
-            box-shadow: 0 0.5rem 1rem rgba(0, 0, 0, 0.15);
-            border-radius: 0.5rem;
-            overflow: hidden;
-        }
-        .emoji-picker-popup.hidden {
-            display: none;
-        }
-
-        /* Mobile responsive adjustments */
-        @media (max-width: 576px) {
-            emoji-picker {
-                --emoji-size: 1.25rem;
-                --num-columns: 6;
-            }
-            .emoji-picker-popup {
-                right: auto;
-                left: 0;
-                width: 100%;
-                max-width: 100%;
-            }
-        }
-    </style>
+    <!-- Inline styles removed - now in style.css -->
 </head>
 <body>
     <!-- Main Content -->
     <main>
         <div class="container-fluid d-flex flex-column" style="height: 100vh;">
-            <!-- Conversation Selector Bar -->
-            <div class="row border-bottom bg-light">
-                <div class="col-12 p-2">
-                    <select id="dmConversationSelector" class="form-select" title="Select conversation">
-                        <option value="">Select chat...</option>
-                        <!-- Conversations loaded dynamically via JavaScript -->
-                    </select>
-                </div>
-            </div>
-            <!-- Messages Container -->
-            <div class="row flex-grow-1 overflow-hidden" style="min-height: 0;">
-                <div class="col-12 position-relative" style="height: 100%;">
-                    <!-- Filter bar overlay -->
-                    <div id="dmFilterBar" class="filter-bar">
-                        <div class="filter-bar-inner">
-                            <input type="text" id="dmFilterInput" class="filter-bar-input" placeholder="Filter messages..." autocomplete="off">
-                            <span id="dmFilterMatchCount" class="filter-match-count"></span>
-                            <button type="button" id="dmFilterClearBtn" class="filter-bar-btn filter-bar-btn-clear" title="Clear">
-                                <i class="bi bi-x"></i>
-                            </button>
-                            <button type="button" id="dmFilterCloseBtn" class="filter-bar-btn filter-bar-btn-close" title="Close">
-                                <i class="bi bi-x-lg"></i>
-                            </button>
-                        </div>
+            <!-- Main content: sidebar + chat -->
+            <div class="d-flex flex-grow-1 overflow-hidden" style="min-height: 0;">
+                <!-- DM Sidebar (visible on lg+ screens) -->
+                <div id="dmSidebar" class="dm-sidebar">
+                    <div class="dm-sidebar-header">
+                        <input type="text"
+                               id="dmSidebarSearch"
+                               class="form-control form-control-sm"
+                               placeholder="Search contacts..."
+                               autocomplete="off">
                     </div>
-                    <div id="dmMessagesContainer" class="messages-container h-100 overflow-auto p-3">
-                        <div id="dmMessagesList">
-                            <!-- Placeholder shown when no conversation selected -->
-                            <div class="dm-empty-state">
-                                <i class="bi bi-envelope"></i>
-                                <p class="mb-1">Select a conversation</p>
-                                <small class="text-muted">Choose from the dropdown above or start a new chat from channel messages</small>
+                    <div class="dm-sidebar-list" id="dmSidebarList">
+                        <!-- Populated by JavaScript -->
+                    </div>
+                </div>
+                <!-- Chat Area -->
+                <div class="flex-grow-1 d-flex flex-column" style="min-width: 0;">
+                    <!-- Conversation Selector Bar (mobile only, hidden on lg+) -->
+                    <div class="dm-mobile-selector border-bottom bg-light">
+                        <div class="p-2">
+                            <div class="d-flex align-items-center gap-2">
+                                <!-- Searchable contact selector -->
+                                <div class="position-relative flex-grow-1" id="dmContactSearchWrapper">
+                                    <input type="text"
+                                           id="dmContactSearchInput"
+                                           class="form-control"
+                                           placeholder="Select chat..."
+                                           autocomplete="off">
+                                    <div id="dmContactDropdown" class="dm-contact-dropdown" style="display: none;"></div>
+                                </div>
+                                <!-- Clear search button -->
+                                <button type="button"
+                                        class="btn btn-outline-secondary flex-shrink-0"
+                                        id="dmClearSearchBtn"
+                                        title="Clear selection"
+                                        style="display: none;">
+                                    <i class="bi bi-x-lg"></i>
+                                </button>
+                                <!-- Contact info button -->
+                                <button type="button"
+                                        class="btn btn-outline-secondary flex-shrink-0"
+                                        id="dmContactInfoBtn"
+                                        title="Contact info"
+                                        disabled>
+                                    <i class="bi bi-info-circle"></i>
+                                </button>
                             </div>
                         </div>
                     </div>
-                    <!-- Scroll to bottom button -->
-                    <button id="dmScrollToBottomBtn" class="scroll-to-bottom-btn" title="Scroll to bottom">
-                        <i class="bi bi-chevron-double-down"></i>
-                    </button>
-                </div>
-            </div>
-
-            <!-- Send Message Form -->
-            <div class="row border-top bg-light">
-                <div class="col-12">
-                    <form id="dmSendForm" class="p-3">
-                        <div class="emoji-picker-container">
-                            <div class="input-group">
-                                <textarea
-                                    id="dmMessageInput"
-                                    class="form-control"
-                                    placeholder="Type a message..."
-                                    rows="2"
-                                    maxlength="500"
-                                    disabled
-                                ></textarea>
-                                <button type="button" class="btn btn-outline-secondary" id="dmEmojiBtn" title="Insert emoji">
-                                    <i class="bi bi-emoji-smile"></i>
+                    <!-- Desktop contact header (visible on lg+ when sidebar is shown) -->
+                    <div class="dm-desktop-header border-bottom bg-light">
+                        <div class="p-2 d-flex align-items-center gap-2">
+                            <span id="dmDesktopContactName" class="fw-medium flex-grow-1 text-truncate"></span>
+                            <button type="button"
+                                    class="btn btn-outline-secondary btn-sm flex-shrink-0"
+                                    id="dmDesktopInfoBtn"
+                                    title="Contact info"
+                                    disabled>
+                                <i class="bi bi-info-circle"></i>
+                            </button>
+                        </div>
+                    </div>
+                    <!-- Messages Container -->
+                    <div class="flex-grow-1 position-relative overflow-hidden" style="min-height: 0;">
+                        <!-- Filter bar overlay -->
+                        <div id="dmFilterBar" class="filter-bar">
+                            <div class="filter-bar-inner">
+                                <input type="text" id="dmFilterInput" class="filter-bar-input" placeholder="Filter messages..." autocomplete="off">
+                                <span id="dmFilterMatchCount" class="filter-match-count"></span>
+                                <button type="button" id="dmFilterClearBtn" class="filter-bar-btn filter-bar-btn-clear" title="Clear">
+                                    <i class="bi bi-x"></i>
                                 </button>
-                                <button type="submit" class="btn btn-success px-4" id="dmSendBtn" disabled>
-                                    <i class="bi bi-send"></i>
+                                <button type="button" id="dmFilterCloseBtn" class="filter-bar-btn filter-bar-btn-close" title="Close">
+                                    <i class="bi bi-x-lg"></i>
                                 </button>
                             </div>
-                            <!-- Emoji picker popup (hidden by default) -->
-                            <div id="dmEmojiPickerPopup" class="emoji-picker-popup hidden"></div>
                         </div>
-                        <div class="d-flex justify-content-end">
-                            <small class="text-muted"><span id="dmCharCounter">0</span> / 150</small>
+                        <div id="dmMessagesContainer" class="messages-container h-100 overflow-auto p-3">
+                            <div id="dmMessagesList">
+                                <!-- Placeholder shown when no conversation selected -->
+                                <div class="dm-empty-state">
+                                    <i class="bi bi-envelope"></i>
+                                    <p class="mb-1">Select a conversation</p>
+                                    <small class="text-muted">Choose from the list or start a new chat from channel messages</small>
+                                </div>
+                            </div>
+                        </div>
+                        <!-- Scroll to bottom button -->
+                        <button id="dmScrollToBottomBtn" class="scroll-to-bottom-btn" title="Scroll to bottom">
+                            <i class="bi bi-chevron-double-down"></i>
+                        </button>
+                    </div>
+                    <!-- Send Message Form -->
+                    <div class="border-top bg-light">
+                        <form id="dmSendForm" class="p-3">
+                            <div class="emoji-picker-container">
+                                <div class="input-group">
+                                    <textarea
+                                        id="dmMessageInput"
+                                        class="form-control"
+                                        placeholder="Type a message..."
+                                        rows="2"
+                                        maxlength="500"
+                                        disabled
+                                    ></textarea>
+                                    <button type="button" class="btn btn-outline-secondary" id="dmEmojiBtn" title="Insert emoji">
+                                        <i class="bi bi-emoji-smile"></i>
+                                    </button>
+                                    <button type="submit" class="btn btn-success px-4" id="dmSendBtn" disabled>
+                                        <i class="bi bi-send"></i>
+                                    </button>
+                                </div>
+                                <!-- Emoji picker popup (hidden by default) -->
+                                <div id="dmEmojiPickerPopup" class="emoji-picker-popup hidden"></div>
+                            </div>
+                            <div class="d-flex justify-content-end">
+                                <small class="text-muted"><span id="dmCharCounter">0</span> / 150</small>
+                            </div>
+                        </form>
+                    </div>
+                    <!-- Status Bar -->
+                    <div class="border-top">
+                        <div class="p-2 small text-muted d-flex justify-content-between align-items-center">
+                            <span id="dmStatusText">
+                                <i class="bi bi-circle-fill text-secondary"></i> Connecting...
+                            </span>
+                            <span id="dmLastRefresh">Updated: Never</span>
                         </div>
-                    </form>
-                </div>
-            </div>
-
-            <!-- Status Bar -->
-            <div class="row border-top">
-                <div class="col-12">
-                    <div class="p-2 small text-muted d-flex justify-content-between align-items-center">
-                        <span id="dmStatusText">
-                            <i class="bi bi-circle-fill text-secondary"></i> Connecting...
-                        </span>
-                        <span id="dmLastRefresh">Updated: Never</span>
                     </div>
                 </div>
             </div>
         </div>
 
-        <!-- Floating Action Button for Filter -->
-        <div class="fab-container">
+        <!-- Floating Action Buttons -->
+        <div class="fab-container" id="dmFabContainer">
+            <button class="fab fab-toggle" id="dmFabToggle" title="Hide buttons">
+                <i class="bi bi-chevron-right"></i>
+            </button>
             <button class="fab fab-filter" id="dmFilterFab" title="Filter Messages">
                 <i class="bi bi-funnel-fill"></i>
             </button>
         </div>
     </main>
 
+    <!-- Contact Info Modal -->
+    <div class="modal fade" id="dmContactInfoModal" tabindex="-1" aria-hidden="true">
+        <div class="modal-dialog">
+            <div class="modal-content">
+                <div class="modal-header">
+                    <h6 class="modal-title"><i class="bi bi-person-circle"></i> Contact Info</h6>
+                    <button type="button" class="btn-close" data-bs-dismiss="modal"></button>
+                </div>
+                <div class="modal-body" id="dmContactInfoBody"></div>
+                <!-- Path management section (populated dynamically) -->
+                <div class="modal-body border-top pt-2 pb-1" id="dmPathSection" style="display: none;">
+                    <div class="path-section-header">
+                        <h6><i class="bi bi-signpost-split"></i> Paths</h6>
+                        <button type="button" class="btn btn-outline-primary btn-sm" id="dmAddPathBtn" title="Add path">
+                            <i class="bi bi-plus-lg"></i>
+                        </button>
+                    </div>
+                    <div id="dmPathList"></div>
+                    <!-- Path action buttons -->
+                    <div class="d-flex justify-content-end gap-2 mt-1">
+                        <button type="button" class="btn btn-outline-secondary btn-sm" id="dmClearPathsBtn"
+                                title="Delete all configured paths from database">
+                            <i class="bi bi-trash"></i> Clear Paths
+                        </button>
+                        <button type="button" class="btn btn-outline-danger btn-sm" id="dmResetFloodBtn"
+                                title="Reset device path to FLOOD mode">
+                            <i class="bi bi-broadcast"></i> Reset to FLOOD
+                        </button>
+                    </div>
+                </div>
+                <div class="modal-footer">
+                    <div class="d-flex align-items-center justify-content-between w-100">
+                        <div class="d-flex gap-3">
+                            <div class="form-check form-switch" title="Auto Retry: resend DM if no ACK received">
+                                <input class="form-check-input" type="checkbox" id="dmAutoRetryToggle" checked>
+                                <label class="form-check-label small" for="dmAutoRetryToggle">Auto Retry</label>
+                            </div>
+                            <div class="form-check form-switch" title="Keep path: don't auto-reset to FLOOD after failed retries">
+                                <input class="form-check-input" type="checkbox" id="dmNoAutoFloodToggle">
+                                <label class="form-check-label small" for="dmNoAutoFloodToggle">Keep path</label>
+                            </div>
+                        </div>
+                        <button type="button" class="btn btn-secondary btn-sm" data-bs-dismiss="modal">Close</button>
+                    </div>
+                </div>
+            </div>
+        </div>
+    </div>
+
+    <!-- Repeater Map Picker Modal -->
+    <div class="modal fade" id="repeaterMapModal" tabindex="-1" style="z-index: 1080;">
+        <div class="modal-dialog modal-lg modal-dialog-centered">
+            <div class="modal-content">
+                <div class="modal-header py-2">
+                    <h6 class="modal-title"><i class="bi bi-geo-alt"></i> Select Repeater from Map</h6>
+                    <button type="button" class="btn-close" data-bs-dismiss="modal"></button>
+                </div>
+                <div class="modal-body p-0">
+                    <div class="d-flex align-items-center gap-2 px-3 py-2 border-bottom bg-light">
+                        <div class="form-check form-switch mb-0">
+                            <input class="form-check-input" type="checkbox" id="rptMapCachedSwitch">
+                            <label class="form-check-label small" for="rptMapCachedSwitch">Cached</label>
+                        </div>
+                        <span class="text-muted small ms-auto" id="rptMapCount"></span>
+                    </div>
+                    <div id="rptLeafletMap" style="height: 400px; width: 100%;"></div>
+                </div>
+                <div class="modal-footer py-2">
+                    <span class="me-auto small text-muted" id="rptMapSelected">Click a repeater on the map</span>
+                    <button type="button" class="btn btn-sm btn-secondary" data-bs-dismiss="modal">Close</button>
+                    <button type="button" class="btn btn-sm btn-primary" id="rptMapAddBtn" disabled>Add</button>
+                </div>
+            </div>
+        </div>
+    </div>
+
+    <!-- Add Path Modal -->
+    <div class="modal fade" id="addPathModal" tabindex="-1" style="z-index: 1070;">
+        <div class="modal-dialog modal-dialog-centered">
+            <div class="modal-content">
+                <div class="modal-header py-2">
+                    <h6 class="modal-title"><i class="bi bi-signpost-split"></i> Add Path</h6>
+                    <button type="button" class="btn-close" data-bs-dismiss="modal"></button>
+                </div>
+                <div class="modal-body" id="dmAddPathForm">
+                    <div class="mb-2">
+                        <label class="form-label small mb-1">Hash Size</label>
+                        <div class="btn-group btn-group-sm w-100" role="group">
+                            <input type="radio" class="btn-check" name="pathHashSize" id="pathHash1" value="1" checked>
+                            <label class="btn btn-outline-secondary" for="pathHash1">1B (max 64)</label>
+                            <input type="radio" class="btn-check" name="pathHashSize" id="pathHash2" value="2">
+                            <label class="btn btn-outline-secondary" for="pathHash2">2B (max 32)</label>
+                            <input type="radio" class="btn-check" name="pathHashSize" id="pathHash3" value="3">
+                            <label class="btn btn-outline-secondary" for="pathHash3">3B (max 21)</label>
+                        </div>
+                    </div>
+                    <div class="mb-2">
+                        <label class="form-label small mb-1">Path (hex)</label>
+                        <div class="input-group input-group-sm">
+                            <input type="text" class="form-control font-monospace" id="dmPathHexInput"
+                                   placeholder="e.g. 5e,e7 or 5e34,e761" autocomplete="off">
+                            <button type="button" class="btn btn-outline-secondary" id="dmPickRepeaterBtn"
+                                    title="Pick repeater from list">
+                                <i class="bi bi-plus-circle"></i>
+                            </button>
+                            <button type="button" class="btn btn-outline-secondary" id="dmPickRepeaterMapBtn"
+                                    title="Pick repeater from map">
+                                <i class="bi bi-geo-alt"></i>
+                            </button>
+                        </div>
+                        <div id="dmPathUniquenessWarning" class="path-uniqueness-warning mt-1" style="display: none;"></div>
+                    </div>
+                    <!-- Repeater picker (hidden by default) -->
+                    <div id="dmRepeaterPicker" style="display: none;" class="border rounded mb-2">
+                        <div class="d-flex border-bottom">
+                            <div class="btn-group btn-group-sm flex-shrink-0" role="group">
+                                <input type="radio" class="btn-check" name="repeaterSearchMode" id="rptSearchName" value="name" checked>
+                                <label class="btn btn-outline-secondary border-0 rounded-0" for="rptSearchName">Name</label>
+                                <input type="radio" class="btn-check" name="repeaterSearchMode" id="rptSearchId" value="id">
+                                <label class="btn btn-outline-secondary border-0 rounded-0" for="rptSearchId">ID</label>
+                            </div>
+                            <input type="text" class="form-control form-control-sm border-0"
+                                   id="dmRepeaterSearch" placeholder="Search by name..." autocomplete="off">
+                        </div>
+                        <div id="dmRepeaterList" style="max-height: 180px; overflow-y: auto;"></div>
+                    </div>
+                    <div class="mb-2">
+                        <label class="form-label small mb-1">Label (optional)</label>
+                        <input type="text" class="form-control form-control-sm" id="dmPathLabelInput"
+                               placeholder="e.g. via Mountain RPT" maxlength="50">
+                    </div>
+                </div>
+                <div class="modal-footer py-2">
+                    <button type="button" class="btn btn-sm btn-outline-secondary" id="dmCancelPathBtn" data-bs-dismiss="modal">Cancel</button>
+                    <button type="button" class="btn btn-sm btn-primary" id="dmSavePathBtn">Add Path</button>
+                </div>
+            </div>
+        </div>
+    </div>
+
     <!-- Toast container for notifications -->
     <div class="toast-container position-fixed top-0 start-0 p-3">
         <div id="notificationToast" class="toast" role="alert">
@@ -177,6 +348,14 @@
     <!-- Filter Utilities (must load before dm.js) -->
     <script src="{{ url_for('static', filename='js/filter-utils.js') }}"></script>
 
+    <!-- Leaflet JS (for repeater map picker) -->
+    <script src="https://unpkg.com/leaflet@1.9.4/dist/leaflet.js"
+            integrity="sha256-20nQCchB9co0qIjJZRGuk2/Z9VM+kNiyxNV1lvTlZBo="
+            crossorigin=""></script>
+
+    <!-- SocketIO for real-time updates -->
+    <script src="{{ url_for('static', filename='vendor/socket.io/socket.io.min.js') }}"></script>
+
     <!-- Custom JS -->
     <script src="{{ url_for('static', filename='js/dm.js') }}"></script>
 

app/templates/index.html

@@ -5,158 +5,119 @@
 {% block extra_head %}
 <!-- Emoji Picker (local) -->
 <script type="module" src="{{ url_for('static', filename='vendor/emoji-picker-element/index.js') }}"></script>
-<style>
-    emoji-picker {
-        --emoji-size: 1.5rem;
-        --num-columns: 8;
-    }
-    .emoji-picker-container {
-        position: relative;
-    }
-    .emoji-picker-popup {
-        position: absolute;
-        bottom: 100%;
-        right: 0;
-        z-index: 1000;
-        margin-bottom: 0.5rem;
-        box-shadow: 0 0.5rem 1rem rgba(0, 0, 0, 0.15);
-        border-radius: 0.5rem;
-        overflow: hidden;
-    }
-    .emoji-picker-popup.hidden {
-        display: none;
-    }
-
-    /* Mobile responsive adjustments */
-    @media (max-width: 576px) {
-        emoji-picker {
-            --emoji-size: 1.25rem;
-            --num-columns: 6;
-        }
-        .emoji-picker-popup {
-            right: auto;
-            left: 0;
-            width: 100%;
-            max-width: 100%;
-        }
-    }
-
-    /* Modal fullscreen - remove all margins and padding */
-    #dmModal .modal-dialog.modal-fullscreen,
-    #contactsModal .modal-dialog.modal-fullscreen,
-    #consoleModal .modal-dialog.modal-fullscreen {
-        margin: 0 !important;
-        width: 100vw !important;
-        max-width: 100vw !important;
-        height: 100vh !important;
-        max-height: 100vh !important;
-    }
-
-    #dmModal .modal-content,
-    #contactsModal .modal-content,
-    #consoleModal .modal-content {
-        border: none !important;
-        border-radius: 0 !important;
-        height: 100vh !important;
-    }
-
-    #dmModal .modal-body,
-    #contactsModal .modal-body,
-    #consoleModal .modal-body {
-        overflow: hidden !important;
-    }
-</style>
 {% endblock %}
 
 {% block content %}
 <div class="container-fluid d-flex flex-column" style="height: 100%;">
-    <!-- Messages Container -->
-    <div class="row flex-grow-1 overflow-hidden" style="min-height: 0;">
-        <div class="col-12 position-relative" style="height: 100%;">
-            <!-- Filter bar overlay -->
-            <div id="filterBar" class="filter-bar">
-                <div class="filter-bar-inner">
-                    <input type="text" id="filterInput" class="filter-bar-input" placeholder="Filter messages..." autocomplete="off">
-                    <span id="filterMatchCount" class="filter-match-count"></span>
-                    <button type="button" id="filterClearBtn" class="filter-bar-btn filter-bar-btn-clear" title="Clear">
-                        <i class="bi bi-x"></i>
-                    </button>
-                    <button type="button" id="filterCloseBtn" class="filter-bar-btn filter-bar-btn-close" title="Close">
-                        <i class="bi bi-x-lg"></i>
-                    </button>
-                </div>
+    <!-- Main content: sidebar + chat -->
+    <div class="d-flex flex-grow-1 overflow-hidden" style="min-height: 0;">
+        <!-- Channel Sidebar (visible on lg+ screens) -->
+        <div id="channelSidebar" class="channel-sidebar">
+            <div class="channel-sidebar-header">
+                <i class="bi bi-broadcast-pin"></i> Channels
             </div>
-            <div id="messagesContainer" class="messages-container h-100 overflow-auto p-3">
-                <div id="messagesList">
-                    <!-- Messages will be loaded here via JavaScript -->
-                    <div class="text-center text-muted py-5">
-                        <div class="spinner-border" role="status">
-                            <span class="visually-hidden">Loading...</span>
+            <div class="channel-sidebar-list" id="channelSidebarList">
+                <!-- Populated by JavaScript -->
+            </div>
+        </div>
+        <!-- Chat Area -->
+        <div class="flex-grow-1 d-flex flex-column" style="min-width: 0;">
+            <!-- Messages Container -->
+            <div class="flex-grow-1 position-relative overflow-hidden" style="min-height: 0;">
+                <!-- Filter bar overlay -->
+                <div id="filterBar" class="filter-bar">
+                    <div class="filter-bar-inner">
+                        <div class="filter-input-wrapper">
+                            <input type="text" id="filterInput" class="filter-bar-input" placeholder="Filter messages..." autocomplete="off">
+                            <!-- Filter mentions autocomplete popup -->
+                            <div id="filterMentionsPopup" class="mentions-popup filter-mentions-popup hidden">
+                                <div class="mentions-list" id="filterMentionsList"></div>
+                            </div>
                         </div>
-                        <p class="mt-3">Loading messages...</p>
+                        <button type="button" id="filterMeBtn" class="filter-bar-btn filter-bar-btn-me" title="Filter my messages">
+                            <i class="bi bi-person-fill"></i>
+                        </button>
+                        <span id="filterMatchCount" class="filter-match-count"></span>
+                        <button type="button" id="filterClearBtn" class="filter-bar-btn filter-bar-btn-clear" title="Clear">
+                            <i class="bi bi-x"></i>
+                        </button>
+                        <button type="button" id="filterCloseBtn" class="filter-bar-btn filter-bar-btn-close" title="Close">
+                            <i class="bi bi-x-lg"></i>
+                        </button>
                     </div>
                 </div>
+                <div id="messagesContainer" class="messages-container h-100 overflow-auto p-3">
+                    <div id="messagesList">
+                        <!-- Messages will be loaded here via JavaScript -->
+                        <div class="text-center text-muted py-5">
+                            <div class="spinner-border" role="status">
+                                <span class="visually-hidden">Loading...</span>
+                            </div>
+                            <p class="mt-3">Loading messages...</p>
+                        </div>
+                    </div>
+                </div>
+                <!-- Scroll to bottom button -->
+                <button id="scrollToBottomBtn" class="scroll-to-bottom-btn" title="Scroll to bottom">
+                    <i class="bi bi-chevron-double-down"></i>
+                </button>
             </div>
-            <!-- Scroll to bottom button -->
-            <button id="scrollToBottomBtn" class="scroll-to-bottom-btn" title="Scroll to bottom">
-                <i class="bi bi-chevron-double-down"></i>
-            </button>
-        </div>
-    </div>
-
-    <!-- Send Message Form -->
-    <div class="row border-top bg-light">
-        <div class="col-12">
-            <form id="sendMessageForm" class="p-3">
-                <div class="emoji-picker-container">
-                    <div class="input-group">
-                        <textarea
-                            id="messageInput"
-                            class="form-control"
-                            placeholder="Type a message..."
-                            rows="2"
-                            maxlength="500"
-                            required
-                        ></textarea>
-                        <button type="button" class="btn btn-outline-secondary" id="emojiBtn" title="Insert emoji">
-                            <i class="bi bi-emoji-smile"></i>
-                        </button>
-                        <button type="submit" class="btn btn-primary px-4" id="sendBtn">
-                            <i class="bi bi-send"></i>
-                        </button>
+            <!-- Send Message Form -->
+            <div class="border-top bg-light">
+                <form id="sendMessageForm" class="p-3">
+                    <div class="emoji-picker-container">
+                        <div class="input-group">
+                            <textarea
+                                id="messageInput"
+                                class="form-control"
+                                placeholder="Type a message..."
+                                rows="2"
+                                maxlength="500"
+                                required
+                            ></textarea>
+                            <button type="button" class="btn btn-outline-secondary" id="emojiBtn" title="Insert emoji">
+                                <i class="bi bi-emoji-smile"></i>
+                            </button>
+                            <button type="submit" class="btn btn-primary px-4" id="sendBtn">
+                                <i class="bi bi-send"></i>
+                            </button>
+                        </div>
+                        <!-- Emoji picker popup (hidden by default) -->
+                        <div id="emojiPickerPopup" class="emoji-picker-popup hidden"></div>
+                        <!-- Mentions autocomplete popup (hidden by default) -->
+                        <div id="mentionsPopup" class="mentions-popup hidden">
+                            <div class="mentions-list" id="mentionsList"></div>
+                        </div>
                     </div>
-                    <!-- Emoji picker popup (hidden by default) -->
-                    <div id="emojiPickerPopup" class="emoji-picker-popup hidden"></div>
-                    <!-- Mentions autocomplete popup (hidden by default) -->
-                    <div id="mentionsPopup" class="mentions-popup hidden">
-                        <div class="mentions-list" id="mentionsList"></div>
+                    <div class="d-flex justify-content-end">
+                        <small id="charCounter" class="text-muted">0 / 135</small>
                     </div>
+                </form>
+            </div>
+            <!-- Status Bar -->
+            <div class="border-top">
+                <div class="p-2 small text-muted d-flex justify-content-between align-items-center">
+                    <span id="statusText">
+                        <i class="bi bi-circle-fill text-secondary"></i> Connecting...
+                    </span>
+                    <span id="lastRefresh">Updated: Never</span>
                 </div>
-                <div class="d-flex justify-content-end">
-                    <small id="charCounter" class="text-muted">0 / 135</small>
-                </div>
-            </form>
-        </div>
-    </div>
-
-    <!-- Status Bar -->
-    <div class="row border-top">
-        <div class="col-12">
-            <div class="p-2 small text-muted d-flex justify-content-between align-items-center">
-                <span id="statusText">
-                    <i class="bi bi-circle-fill text-secondary"></i> Connecting...
-                </span>
-                <span id="lastRefresh">Updated: Never</span>
             </div>
         </div>
     </div>
 </div>
 
 <!-- Floating Action Buttons -->
-<div class="fab-container">
+<div class="fab-container" id="fabContainer">
+    <button class="fab fab-toggle" id="fabToggle" title="Hide buttons">
+        <i class="bi bi-chevron-right"></i>
+    </button>
     <button class="fab fab-filter" id="filterFab" title="Filter Messages">
         <i class="bi bi-funnel-fill"></i>
     </button>
+    <button class="fab fab-search" id="globalSearchBtn" data-bs-toggle="modal" data-bs-target="#searchModal" title="Search Messages">
+        <i class="bi bi-search"></i>
+    </button>
     <button class="fab fab-dm" data-bs-toggle="modal" data-bs-target="#dmModal" title="Direct Messages">
         <i class="bi bi-envelope-fill"></i>
     </button>
@@ -215,6 +176,23 @@
         </div>
     </div>
 </div>
+
+<!-- System Log Modal (Full Screen) -->
+<div class="modal fade" id="logsModal" tabindex="-1" aria-hidden="true">
+    <div class="modal-dialog modal-fullscreen">
+        <div class="modal-content" style="background-color: #1a1a2e;">
+            <div class="modal-header" style="background-color: #16213e; border-bottom: 1px solid #0f3460;">
+                <h5 class="modal-title text-white"><i class="bi bi-journal-text"></i> System Log</h5>
+                <button type="button" class="btn btn-outline-light" data-bs-dismiss="modal">
+                    <i class="bi bi-x-lg"></i> Close
+                </button>
+            </div>
+            <div class="modal-body p-0">
+                <iframe id="logsFrame" style="width: 100%; height: 100%; border: none;"></iframe>
+            </div>
+        </div>
+    </div>
+</div>
 {% endblock %}
 
 {% block extra_scripts %}
@@ -287,6 +265,23 @@
                 }
             });
         }
+
+        // System Log modal - load iframe on open, clear on close
+        const logsModal = document.getElementById('logsModal');
+        if (logsModal) {
+            logsModal.addEventListener('show.bs.modal', function () {
+                const logsFrame = document.getElementById('logsFrame');
+                if (logsFrame) {
+                    logsFrame.src = '/logs';
+                }
+            });
+            logsModal.addEventListener('hidden.bs.modal', function () {
+                const logsFrame = document.getElementById('logsFrame');
+                if (logsFrame) {
+                    logsFrame.src = '';  // disconnect WebSocket when closed
+                }
+            });
+        }
     });
 </script>
 {% endblock %}

app/templates/logs.html

@@ -0,0 +1,202 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover">
+    <title>System Log - mc-webui</title>
+
+    <!-- Favicon -->
+    <link rel="apple-touch-icon" sizes="180x180" href="{{ url_for('static', filename='images/apple-touch-icon.png') }}">
+    <link rel="icon" type="image/png" sizes="32x32" href="{{ url_for('static', filename='images/favicon-32x32.png') }}">
+    <link rel="icon" type="image/png" sizes="16x16" href="{{ url_for('static', filename='images/favicon-16x16.png') }}">
+    <link rel="manifest" href="{{ url_for('static', filename='manifest.json') }}">
+    <link rel="shortcut icon" href="{{ url_for('static', filename='favicon.ico') }}">
+
+    <!-- Bootstrap 5 CSS (local) -->
+    <link href="{{ url_for('static', filename='vendor/bootstrap/css/bootstrap.min.css') }}" rel="stylesheet">
+    <!-- Bootstrap Icons (local) -->
+    <link rel="stylesheet" href="{{ url_for('static', filename='vendor/bootstrap-icons/bootstrap-icons.css') }}">
+    <!-- Custom CSS -->
+    <link rel="stylesheet" href="{{ url_for('static', filename='css/style.css') }}">
+
+    <style>
+        html, body {
+            height: 100%;
+            margin: 0;
+            background-color: #1a1a2e;
+        }
+
+        .log-container {
+            display: flex;
+            flex-direction: column;
+            height: 100vh;
+        }
+
+        .log-header {
+            background-color: #16213e;
+            border-bottom: 1px solid #0f3460;
+            padding: 0.5rem 1rem;
+            flex-shrink: 0;
+        }
+
+        .log-filters {
+            background-color: #16213e;
+            border-bottom: 1px solid #0f3460;
+            padding: 0.5rem 1rem;
+            flex-shrink: 0;
+        }
+
+        .log-entries {
+            flex: 1;
+            overflow-y: auto;
+            padding: 0.5rem;
+            background-color: #1a1a2e;
+            min-height: 0;
+            font-family: 'Courier New', Consolas, monospace;
+            font-size: 0.82rem;
+            line-height: 1.4;
+        }
+
+        .log-line {
+            padding: 1px 4px;
+            white-space: pre-wrap;
+            word-break: break-word;
+            border-bottom: 1px solid rgba(255,255,255,0.03);
+        }
+
+        .log-line:hover {
+            background-color: rgba(255,255,255,0.05);
+        }
+
+        .log-ts { color: #6c757d; }
+        .log-logger { color: #4ecdc4; }
+
+        .log-level-DEBUG { color: #6c757d; }
+        .log-level-INFO { color: #e0e0e0; }
+        .log-level-WARNING { color: #ffd93d; }
+        .log-level-ERROR { color: #ff6b6b; font-weight: bold; }
+        .log-level-CRITICAL { color: #ff3333; font-weight: bold; background-color: rgba(255,0,0,0.1); }
+
+        .log-msg-DEBUG { color: #888; }
+        .log-msg-INFO { color: #c8c8c8; }
+        .log-msg-WARNING { color: #e8d44d; }
+        .log-msg-ERROR { color: #ff8888; }
+        .log-msg-CRITICAL { color: #ff6666; }
+
+        /* Filter controls */
+        .filter-select, .filter-input {
+            background-color: #0f3460;
+            border: 1px solid #1a1a2e;
+            color: #e0e0e0;
+            font-size: 0.82rem;
+            padding: 0.25rem 0.5rem;
+        }
+        .filter-select:focus, .filter-input:focus {
+            background-color: #0f3460;
+            border-color: #4ecdc4;
+            color: #e0e0e0;
+            box-shadow: 0 0 0 0.15rem rgba(78, 205, 196, 0.25);
+        }
+        .filter-select option {
+            background-color: #16213e;
+        }
+
+        .btn-log {
+            background-color: #0f3460;
+            border: 1px solid #1a1a2e;
+            color: #4ecdc4;
+            font-size: 0.82rem;
+            padding: 0.25rem 0.5rem;
+        }
+        .btn-log:hover, .btn-log:focus {
+            background-color: #1a1a4e;
+            border-color: #4ecdc4;
+            color: #4ecdc4;
+        }
+        .btn-log.active {
+            background-color: #4ecdc4;
+            color: #1a1a2e;
+        }
+
+        .status-indicator {
+            width: 8px;
+            height: 8px;
+            border-radius: 50%;
+            display: inline-block;
+        }
+        .status-indicator.live { background-color: #00ff88; }
+        .status-indicator.paused { background-color: #ffd93d; }
+        .status-indicator.disconnected { background-color: #ff6b6b; }
+
+        .log-count {
+            color: #6c757d;
+            font-size: 0.75rem;
+        }
+
+        /* Highlight search matches */
+        mark {
+            background-color: rgba(255, 217, 61, 0.3);
+            color: inherit;
+            padding: 0;
+        }
+
+        @media (max-width: 576px) {
+            .log-header { padding: 0.4rem 0.5rem; }
+            .log-filters { padding: 0.4rem 0.5rem; }
+            .log-entries { font-size: 0.75rem; padding: 0.25rem; }
+        }
+    </style>
+</head>
+<body>
+    <div class="log-container">
+        <!-- Header -->
+        <div class="log-header d-flex justify-content-between align-items-center">
+            <div class="d-flex align-items-center gap-2">
+                <span class="log-count" id="logCount">0 entries</span>
+                <span class="status-indicator" id="statusDot"></span>
+            </div>
+            <div class="d-flex align-items-center gap-2">
+                <button class="btn btn-sm btn-log" id="pauseBtn" title="Pause/Resume">
+                    <i class="bi bi-pause-fill" id="pauseIcon"></i>
+                </button>
+                <button class="btn btn-sm btn-log" id="clearBtn" title="Clear display">
+                    <i class="bi bi-trash"></i>
+                </button>
+            </div>
+        </div>
+
+        <!-- Filters -->
+        <div class="log-filters">
+            <div class="d-flex gap-2 flex-wrap align-items-center">
+                <select class="form-select form-select-sm filter-select" id="levelFilter" style="width: auto; min-width: 90px;">
+                    <option value="">All levels</option>
+                    <option value="DEBUG">DEBUG</option>
+                    <option value="INFO" selected>INFO</option>
+                    <option value="WARNING">WARNING</option>
+                    <option value="ERROR">ERROR</option>
+                </select>
+                <select class="form-select form-select-sm filter-select" id="loggerFilter" style="width: auto; min-width: 120px;">
+                    <option value="">All modules</option>
+                </select>
+                <input type="text" class="form-control form-control-sm filter-input" id="searchFilter"
+                       placeholder="Search..." style="width: auto; min-width: 150px; flex: 1;">
+                <button class="btn btn-sm btn-log" id="resetFilters" title="Reset filters">
+                    <i class="bi bi-x-circle"></i>
+                </button>
+            </div>
+        </div>
+
+        <!-- Log entries -->
+        <div class="log-entries" id="logEntries">
+            <div class="text-muted text-center py-3" id="loadingMsg">Loading logs...</div>
+        </div>
+    </div>
+
+    <!-- Socket.IO client -->
+    <script src="{{ url_for('static', filename='vendor/socket.io/socket.io.min.js') }}"></script>
+    <!-- Bootstrap JS Bundle (local) -->
+    <script src="{{ url_for('static', filename='vendor/bootstrap/js/bootstrap.bundle.min.js') }}"></script>
+    <!-- Log Viewer JS -->
+    <script src="{{ url_for('static', filename='js/logs.js') }}"></script>
+</body>
+</html>

docker-compose.yml

@@ -1,71 +1,38 @@
+# mc-webui v2 — single container with direct device access
 services:
-  # MeshCore Bridge - Handles USB communication with meshcli
-  meshcore-bridge:
-    build:
-      context: ./meshcore-bridge
-      dockerfile: Dockerfile
-    container_name: meshcore-bridge
+  mc-webui:
+    build: .
+    container_name: mc-webui
     restart: unless-stopped
+    ports:
+      - "${FLASK_PORT:-5000}:${FLASK_PORT:-5000}"
     # Grant access to serial devices for auto-detection
-    # This allows MC_SERIAL_PORT=auto to work without specifying device upfront
-    # Major 188 = ttyUSB (CP2102, CH340, etc.), Major 166 = ttyACM (ESP32-S3, etc.)
+    # Major 188 = ttyUSB (CP2102, CH340), Major 166 = ttyACM (ESP32-S3)
     device_cgroup_rules:
       - 'c 188:* rmw'
       - 'c 166:* rmw'
     volumes:
-      - "${MC_CONFIG_DIR}:/root/.config/meshcore:rw"
+      - "${MC_CONFIG_DIR:-./data}:/data:rw"
       - "/dev:/dev"
     environment:
       - MC_SERIAL_PORT=${MC_SERIAL_PORT:-auto}
-      - MC_CONFIG_DIR=/root/.config/meshcore
-      - MC_DEVICE_NAME=${MC_DEVICE_NAME:-auto}
-      - TZ=${TZ:-UTC}
-    networks:
-      - meshcore-net
-    healthcheck:
-      test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:5001/health')"]
-      interval: 30s
-      timeout: 10s
-      retries: 3
-      start_period: 10s
-
-  # Main Web UI - Communicates with bridge via HTTP
-  mc-webui:
-    build:
-      context: .
-      dockerfile: Dockerfile
-    container_name: mc-webui
-    restart: unless-stopped
-    ports:
-      - "${FLASK_PORT:-5000}:5000"
-    volumes:
-      - "${MC_CONFIG_DIR}:/root/.config/meshcore:rw"
-      - "${MC_ARCHIVE_DIR:-./archive}:/root/.archive/meshcore:rw"
-    environment:
-      - MC_BRIDGE_URL=http://meshcore-bridge:5001/cli
-      - MC_DEVICE_NAME=${MC_DEVICE_NAME}
-      - MC_CONFIG_DIR=/root/.config/meshcore
-      - MC_ARCHIVE_DIR=/root/.archive/meshcore
-      - MC_ARCHIVE_ENABLED=${MC_ARCHIVE_ENABLED:-true}
-      - MC_ARCHIVE_RETENTION_DAYS=${MC_ARCHIVE_RETENTION_DAYS:-7}
+      - MC_DEVICE_NAME=${MC_DEVICE_NAME:-MeshCore}
+      - MC_CONFIG_DIR=/data
+      - MC_TCP_HOST=${MC_TCP_HOST:-}
+      - MC_TCP_PORT=${MC_TCP_PORT:-5555}
+      - MC_BACKUP_ENABLED=${MC_BACKUP_ENABLED:-true}
+      - MC_BACKUP_HOUR=${MC_BACKUP_HOUR:-2}
+      - MC_BACKUP_RETENTION_DAYS=${MC_BACKUP_RETENTION_DAYS:-7}
       - FLASK_HOST=${FLASK_HOST:-0.0.0.0}
       - FLASK_PORT=${FLASK_PORT:-5000}
       - FLASK_DEBUG=${FLASK_DEBUG:-false}
       - TZ=${TZ:-UTC}
     env_file:
-      - .env
-    depends_on:
-      meshcore-bridge:
-        condition: service_healthy
-    networks:
-      - meshcore-net
+      - path: .env
+        required: false
     healthcheck:
-      test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:5000/api/status')"]
+      test: ["CMD", "curl", "-f", "http://localhost:5000/health"]
       interval: 30s
       timeout: 10s
       retries: 3
-      start_period: 10s
-
-networks:
-  meshcore-net:
-    driver: bridge
+      start_period: 15s

docs/MIGRATION-v1-to-v2.md

@@ -0,0 +1,45 @@
+# Migration Guide: v1 to v2
+
+## Overview
+
+v2 replaces the `meshcore-cli` bridge with direct `meshcore` library communication. This changes how contacts and DMs work at a fundamental level.
+
+## Breaking Changes
+
+### 1. DM contacts must exist on the device firmware
+
+**Symptom**: Sending a DM to an existing contact fails with "Contact not on device".
+
+**Why**: In v2, `meshcore` communicates directly with the device firmware, which **requires** the contact to exist in its internal contact table (max 350 entries) to send a DM. The mc-webui database may contain hundreds of contacts from advertisement history, but only a handful are actually present on the device.
+
+This can happen after:
+- **Firmware reflash** — wipes the device contact table while the DB retains all contacts
+- **Migration from v1** — v1 `meshcore-cli` bridge managed contacts independently; many DB contacts may have never been added to the device
+- **Device reset** — any factory reset clears the firmware contact table
+
+**How to verify**: Check the startup log for `Synced N contacts from device to database`. This N is the actual number of contacts on the device — likely much smaller than the total in the DB.
+
+**Fix**: For each contact you want to DM:
+1. Delete the contact from the Contacts page
+2. Wait for their next advertisement
+3. Approve the contact when it appears in the pending list
+
+This adds the contact to the device's firmware table, enabling DM sending.
+
+**Note**: Incoming DMs from any contact still work regardless — this only affects *sending* DMs.
+
+### 2. Contact soft-delete preserves DM history
+
+In v2, deleting a contact is a soft-delete (marked as `source='deleted'` in the database). This preserves DM conversation history. When the contact is re-added, it automatically "undeletes" and all previous DMs are visible again.
+
+### 3. Database schema
+
+v2 uses SQLite with WAL mode instead of flat JSON files. The migration from v1 data happens automatically on first startup (see `app/migrate_v1.py`). The v1 data files are preserved and not modified.
+
+## Post-Migration Checklist
+
+- [ ] Verify device connection (green "Connected" indicator)
+- [ ] Check that channel messages are flowing normally
+- [ ] Check startup log: `Synced N contacts from device to database` — this is your actual device contact count
+- [ ] For each DM contact you need: delete, wait for advert, re-approve
+- [ ] Verify DM sending works with a test message

docs/architecture.md

@@ -6,318 +6,271 @@ Technical documentation for mc-webui, covering system architecture, project stru
 
 - [Tech Stack](#tech-stack)
 - [Container Architecture](#container-architecture)
-- [Bridge Session Architecture](#bridge-session-architecture)
+- [DeviceManager Architecture](#devicemanager-architecture)
 - [Project Structure](#project-structure)
-- [Message File Format](#message-file-format)
+- [Database Architecture](#database-architecture)
 - [API Reference](#api-reference)
+- [WebSocket API](#websocket-api)
 - [Offline Support](#offline-support)
 
 ---
 
 ## Tech Stack
 
-- **Backend:** Python 3.11+, Flask, Flask-SocketIO (gevent)
+- **Backend:** Python 3.11+, Flask, Flask-SocketIO (gevent), SQLite
 - **Frontend:** HTML5, Bootstrap 5, vanilla JavaScript, Socket.IO client
-- **Deployment:** Docker / Docker Compose (2-container architecture)
-- **Communication:** HTTP bridge to meshcore-cli, WebSocket for interactive console
-- **Data source:** `~/.config/meshcore/<device_name>.msgs` (JSON Lines)
+- **Deployment:** Docker / Docker Compose (Single-container architecture)
+- **Communication:** Direct hardware access (USB, BLE, or TCP) via `meshcore` library
+- **Data source:** SQLite Database (`./data/meshcore/<pubkey_prefix>.db`)
 
 ---
 
 ## Container Architecture
 
-mc-webui uses a **2-container architecture** for improved USB stability:
+mc-webui uses a **single-container architecture** for simplified deployment and direct hardware communication:
 
-```
+```text
 ┌─────────────────────────────────────────────────────────────┐
 │                     Docker Network                           │
 │                                                              │
-│  ┌─────────────────────┐      ┌─────────────────────────┐  │
-│  │   meshcore-bridge   │      │       mc-webui          │  │
-│  │                     │      │                         │  │
-│  │  - USB device access│ HTTP │  - Flask web app        │  │
-│  │  - meshcli process  │◄────►│  - User interface       │  │
-│  │  - Port 5001        │      │  - Port 5000            │  │
-│  │                     │      │                         │  │
-│  └─────────┬───────────┘      └─────────────────────────┘  │
+│  ┌───────────────────────────────────────────────────────┐   │
+│  │                       mc-webui                        │   │
+│  │                                                       │   │
+│  │  - Flask web app (Port 5000)                          │   │
+│  │  - DeviceManager (Direct USB/TCP access)              │   │
+│  │  - Database (SQLite)                                  │   │
+│  │                                                       │   │
+│  └─────────┬─────────────────────────────────────────────┘   │
 │            │                                                 │
 └────────────┼─────────────────────────────────────────────────┘
              │
              ▼
       ┌──────────────┐
-      │  USB Device  │
-      │  (Heltec V4) │
+      │  USB/TCP     │
+      │  Device      │
       └──────────────┘
 ```
 
-### meshcore-bridge (Port 5001 - internal)
-
-Lightweight service with exclusive USB device access:
-
-- Maintains a **persistent meshcli session** (single long-lived process)
-- Multiplexes stdout: JSON adverts → `.adverts.jsonl` log, CLI commands → HTTP responses
-- Real-time message reception via `msgs_subscribe` (no polling)
-- Thread-safe command queue with event-based synchronization
-- Watchdog thread for automatic crash recovery
-- Exposes HTTP API on port 5001 (internal only)
-
-### mc-webui (Port 5000 - external)
-
-Main web application:
-
-- Flask-based web interface with Flask-SocketIO
-- Communicates with bridge via HTTP API
-- WebSocket support for interactive Console (`/console` namespace)
-- No direct USB access (prevents device locking)
-
-This separation solves USB timeout/deadlock issues common in Docker + VM environments.
+This v2 architecture eliminates the need for a separate bridge container and relies on the native `meshcore` Python library for direct communication, ensuring lower latency and greater stability.
 
 ---
 
-## Bridge Session Architecture
+## DeviceManager Architecture
 
-The meshcore-bridge maintains a **single persistent meshcli session** instead of spawning new processes per request:
+The `DeviceManager` handles the connection to the MeshCore device via a direct session:
 
-- **Single subprocess.Popen** - One long-lived meshcli process with stdin/stdout pipes
-- **Multiplexing** - Intelligently routes output:
-  - JSON adverts (with `payload_typename: "ADVERT"`) → logged to `{device_name}.adverts.jsonl`
-  - CLI command responses → returned via HTTP API
-- **Real-time messages** - `msgs_subscribe` command enables instant message reception without polling
-- **Thread-safe queue** - Commands are serialized through a queue.Queue for FIFO execution
-- **Timeout-based detection** - Response completion detected when no new lines arrive for 300ms
-- **Auto-restart watchdog** - Monitors process health and restarts on crash
-
-This architecture enables advanced features like pending contact management (`manual_add_contacts`) and provides better stability and performance.
+- **Single persistent session** - One long-lived connection utilizing the `meshcore` library
+- **Event-driven** - Subscribes to device events (e.g., incoming messages, advert receptions, ACKs) and triggers appropriate handlers
+- **Direct Database integration** - Seamlessly syncs contacts, messages, and device settings to the SQLite database
+- **Real-time messages** - Instant message processing via callback events without polling
+- **Thread-safe queue** - Commands are serialized to prevent device lockups
+- **Auto-restart watchdog** - Monitors connection health and restarts the session on crash
 
 ---
 
 ## Project Structure
 
-```
+```text
 mc-webui/
 ├── Dockerfile                      # Main app Docker image
-├── docker-compose.yml              # Multi-container orchestration
-├── meshcore-bridge/
-│   ├── Dockerfile                  # Bridge service image
-│   ├── bridge.py                   # HTTP API wrapper for meshcli
-│   └── requirements.txt            # Bridge dependencies (Flask only)
+├── docker-compose.yml              # Single-container orchestration
 ├── app/
 │   ├── __init__.py
-│   ├── main.py                     # Flask entry point
+│   ├── main.py                     # Flask entry point + Socket.IO handlers
 │   ├── config.py                   # Configuration from env vars
-│   ├── read_status.py              # Server-side read status manager
+│   ├── database.py                 # SQLite database models and CRUD operations
+│   ├── device_manager.py           # Core logic for meshcore communication
+│   ├── contacts_cache.py           # Persistent contacts cache (DB-backed)
+│   ├── read_status.py              # Server-side read status manager (DB-backed)
+│   ├── version.py                  # Git-based version management
+│   ├── migrate_v1.py               # Migration script from v1 flat files to v2 SQLite
 │   ├── meshcore/
 │   │   ├── __init__.py
-│   │   ├── cli.py                  # HTTP client for bridge API
-│   │   └── parser.py               # .msgs file parser
+│   │   ├── cli.py                  # Meshcore library wrapper interface
+│   │   └── parser.py               # Data parsers
 │   ├── archiver/
 │   │   └── manager.py              # Archive scheduler and management
 │   ├── routes/
 │   │   ├── __init__.py
 │   │   ├── api.py                  # REST API endpoints
 │   │   └── views.py                # HTML views
-│   ├── static/
-│   │   ├── css/
-│   │   │   └── style.css           # Custom styles
-│   │   ├── js/
-│   │   │   ├── app.js              # Main page frontend logic
-│   │   │   ├── dm.js               # Direct Messages page logic
-│   │   │   ├── contacts.js         # Contact Management logic
-│   │   │   ├── console.js          # Interactive console WebSocket client
-│   │   │   ├── message-utils.js    # Message content processing
-│   │   │   └── sw.js               # Service Worker for PWA
-│   │   ├── vendor/                 # Local vendor libraries (offline)
-│   │   │   ├── bootstrap/          # Bootstrap CSS/JS
-│   │   │   ├── bootstrap-icons/    # Icon fonts
-│   │   │   ├── socket.io/          # Socket.IO client library
-│   │   │   └── emoji-picker-element/
-│   │   └── manifest.json           # PWA manifest
-│   └── templates/
-│       ├── base.html               # Base template
-│       ├── index.html              # Main chat view
-│       ├── dm.html                 # Direct Messages view
-│       ├── console.html            # Interactive meshcli console
-│       ├── contacts_base.html      # Contact pages base template
-│       ├── contacts-manage.html    # Contact Management settings
-│       ├── contacts-pending.html   # Pending contacts view
-│       └── contacts-existing.html  # Existing contacts view
+│   ├── static/                     # Frontend assets (CSS, JS, images, vendors)
+│   └── templates/                  # HTML templates
 ├── docs/                           # Documentation
-├── images/                         # Screenshots and diagrams
-├── requirements.txt                # Python dependencies
-├── .env.example                    # Example environment config
+├── scripts/                        # Utility scripts (update, watchdog, updater)
 └── README.md
 ```
 
 ---
 
-## Message File Format
+## Database Architecture
 
-Location: `~/.config/meshcore/<device_name>.msgs` (JSON Lines)
+mc-webui v2 uses a robust **SQLite Database** with WAL (Write-Ahead Logging) enabled.
 
-### Message Types
+Location: `./data/meshcore/<pubkey_prefix>.db`
 
-**Channel messages:**
-```json
-{"type": "CHAN", "text": "User: message", "timestamp": 1766300846}
-{"type": "SENT_CHAN", "text": "my message", "name": "DeviceName", "timestamp": 1766309432}
-```
+Key tables:
+- `messages` - All channel and direct messages (with FTS5 index for full-text search)
+- `contacts` - Contact list with sync status, types, block/ignore flags
+- `channels` - Channel configuration and keys
+- `echoes` - Sent message tracking and repeater paths
+- `acks` - DM delivery status
+- `settings` - Application settings (migrated from .webui_settings.json)
 
-**Private messages:**
-```json
-{"type": "PRIV", "text": "message", "sender_timestamp": 1766300846, "pubkey_prefix": "abc123", "sender": "User"}
-{"type": "SENT_MSG", "text": "message", "recipient": "User", "expected_ack": "xyz", "suggested_timeout": 30000}
-```
-
-**Note on SENT_MSG:** Requires meshcore-cli >= 1.3.12 for correct format with both `recipient` and `sender` fields.
+The use of SQLite allows for fast queries, reliable data storage, full-text search, and complex filtering (such as contact ignoring/blocking) without the risk of file corruption inherent to flat JSON files.
 
 ---
 
 ## API Reference
 
-### Main Web UI Endpoints
+### Messages
 
 | Method | Endpoint | Description |
 |--------|----------|-------------|
-| GET | `/api/messages` | List messages (supports `?archive_date`, `?days`, `?channel_idx`) |
+| GET | `/api/messages` | List messages (`?archive_date`, `?days`, `?channel_idx`) |
 | POST | `/api/messages` | Send message (`{text, channel_idx, reply_to?}`) |
 | GET | `/api/messages/updates` | Check for new messages (smart refresh) |
-| GET | `/api/status` | Connection status |
+| GET | `/api/messages/<id>/meta` | Get message metadata (echoes, paths) |
+| GET | `/api/messages/search` | Full-text search (`?q=`, `?channel_idx=`, `?limit=`) |
+
+### Contacts
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
 | GET | `/api/contacts` | List contacts |
-| GET | `/api/contacts/detailed` | Full contact_info data |
-| POST | `/api/contacts/delete` | Delete contact by name |
-| GET | `/api/contacts/pending` | List pending contacts |
+| GET | `/api/contacts/detailed` | Full contact data (includes protection, ignore, block flags) |
+| GET | `/api/contacts/cached` | Get cached contacts (superset of device contacts) |
+| POST | `/api/contacts/delete` | Soft-delete contact (`{selector}`) |
+| POST | `/api/contacts/cached/delete` | Delete cached contact |
+| GET | `/api/contacts/protected` | List protected public keys |
+| POST | `/api/contacts/<key>/protect` | Toggle contact protection |
+| POST | `/api/contacts/<key>/ignore` | Toggle contact ignore |
+| POST | `/api/contacts/<key>/block` | Toggle contact block |
+| GET | `/api/contacts/blocked-names` | Get blocked names count |
+| POST | `/api/contacts/block-name` | Block a name pattern |
+| GET | `/api/contacts/blocked-names-list` | List blocked name patterns |
+| POST | `/api/contacts/preview-cleanup` | Preview cleanup criteria |
+| POST | `/api/contacts/cleanup` | Remove contacts by filter |
+| GET | `/api/contacts/cleanup-settings` | Get auto-cleanup settings |
+| POST | `/api/contacts/cleanup-settings` | Update auto-cleanup settings |
+| GET | `/api/contacts/pending` | Pending contacts (`?types=1&types=2`) |
 | POST | `/api/contacts/pending/approve` | Approve pending contact |
-| POST | `/api/contacts/preview-cleanup` | Preview cleanup matches |
-| POST | `/api/contacts/cleanup` | Execute contact cleanup |
+| POST | `/api/contacts/pending/reject` | Reject pending contact |
+| POST | `/api/contacts/pending/clear` | Clear all pending contacts |
+| POST | `/api/contacts/manual-add` | Add contact from URI or params |
+| POST | `/api/contacts/<key>/push-to-device` | Push cached contact to device |
+| POST | `/api/contacts/<key>/move-to-cache` | Move device contact to cache |
+| GET | `/api/contacts/repeaters` | List repeater contacts (for path picker) |
+| GET | `/api/contacts/<key>/paths` | Get contact paths |
+| POST | `/api/contacts/<key>/paths` | Add path to contact |
+| PUT | `/api/contacts/<key>/paths/<id>` | Update path (star, label) |
+| DELETE | `/api/contacts/<key>/paths/<id>` | Delete path |
+| POST | `/api/contacts/<key>/paths/reorder` | Reorder paths |
+| POST | `/api/contacts/<key>/paths/reset_flood` | Reset to FLOOD routing |
+| POST | `/api/contacts/<key>/paths/clear` | Clear all paths |
+| GET | `/api/contacts/<key>/no_auto_flood` | Get "Keep path" flag |
+| PUT | `/api/contacts/<key>/no_auto_flood` | Set "Keep path" flag |
+
+### Channels
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
 | GET | `/api/channels` | List all channels |
 | POST | `/api/channels` | Create new channel |
 | POST | `/api/channels/join` | Join existing channel |
 | DELETE | `/api/channels/<index>` | Remove channel |
-| GET | `/api/channels/<index>/qr` | Generate QR code |
-| GET | `/api/dm/conversations` | List DM conversations |
-| GET | `/api/dm/messages` | Get messages for conversation |
-| POST | `/api/dm/messages` | Send DM |
-| GET | `/api/dm/updates` | Check for new DMs |
-| GET | `/api/device/info` | Device information |
-| GET | `/api/device/settings` | Get device settings |
-| POST | `/api/device/settings` | Update device settings |
-| POST | `/api/device/command` | Execute special command |
-| GET | `/api/read_status` | Get server-side read status |
-| POST | `/api/read_status/mark_read` | Mark messages as read |
-| GET | `/api/archives` | List available archives |
-| POST | `/api/archive/trigger` | Manually trigger archiving |
+| GET | `/api/channels/<index>/qr` | QR code (`?format=json\|png`) |
+| GET | `/api/channels/muted` | Get muted channels |
+| POST | `/api/channels/<index>/mute` | Toggle channel mute |
 
-### WebSocket API (Console)
-
-Interactive meshcli console via Socket.IO WebSocket connection.
-
-**Namespace:** `/console`
-
-| Event | Direction | Description |
-|-------|-----------|-------------|
-| `send_command` | Client → Server | Execute command (`{command: "infos"}`) |
-| `console_status` | Server → Client | Connection status message |
-| `command_response` | Server → Client | Command result (`{success, command, output}`) |
-
-**Features:**
-- Command history navigation (up/down arrows)
-- Auto-reconnection on disconnect
-- Output cleaning (removes prompts like "DeviceName|*")
-- Slow command timeouts: `node_discover` (15s), `recv` (60s), `send` (15s)
-
-### Bridge Internal API (Port 5001)
+### Direct Messages
 
 | Method | Endpoint | Description |
 |--------|----------|-------------|
-| POST | `/cli` | Execute meshcli command (`{args: ["cmd"], timeout?}`) |
-| GET | `/health` | Bridge health check |
-| GET | `/pending_contacts` | List pending contacts |
-| POST | `/add_pending` | Approve pending contact |
-| GET | `/device/settings` | Get device settings |
-| POST | `/device/settings` | Update device settings |
+| GET | `/api/dm/conversations` | List DM conversations |
+| GET | `/api/dm/messages` | Get messages (`?conversation_id=`, `?limit=`) |
+| POST | `/api/dm/messages` | Send DM (`{recipient, text}`) |
+| GET | `/api/dm/updates` | Check for new DMs |
+| GET | `/api/dm/auto_retry` | Get DM retry configuration |
+| POST | `/api/dm/auto_retry` | Update DM retry configuration |
 
-**Base URL:** `http://meshcore-bridge:5001` (internal Docker network only)
+### Device & Settings
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| GET | `/api/status` | Connection status (device name, serial port) |
+| GET | `/api/device/info` | Device information |
+| GET | `/api/device/stats` | Device statistics |
+| GET | `/api/device/settings` | Get device settings |
+| POST | `/api/device/settings` | Update device settings |
+| POST | `/api/device/command` | Execute command (advert, floodadv) |
+| GET | `/api/device/commands` | List available special commands |
+| GET | `/api/chat/settings` | Get chat settings (quote length) |
+| POST | `/api/chat/settings` | Update chat settings |
+| GET | `/api/retention-settings` | Get message retention settings |
+| POST | `/api/retention-settings` | Update retention settings |
+
+### Archives & Backup
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| GET | `/api/archives` | List archives |
+| POST | `/api/archive/trigger` | Manual archive |
+| GET | `/api/backup/list` | List database backups |
+| POST | `/api/backup/create` | Create database backup |
+| GET | `/api/backup/download` | Download backup file |
+
+### Other
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| GET | `/api/read_status` | Get server-side read status |
+| POST | `/api/read_status/mark_read` | Mark messages as read |
+| POST | `/api/read_status/mark_all_read` | Mark all messages as read |
+| GET | `/api/version` | Get app version |
+| GET | `/api/check-update` | Check for available updates |
+| GET | `/api/updater/status` | Get updater service status |
+| POST | `/api/updater/trigger` | Trigger remote update |
+| GET | `/api/advertisements` | Get recent advertisements |
+| GET | `/api/console/history` | Get console command history |
+| POST | `/api/console/history` | Save console command |
+| DELETE | `/api/console/history` | Clear console history |
+| GET | `/api/logs` | Get application logs |
+
+---
+
+## WebSocket API
+
+### Console Namespace (`/console`)
+
+Interactive console via Socket.IO WebSocket connection.
+
+**Client → Server:**
+- `send_command` - Execute command (`{command: "infos"}`)
+
+**Server → Client:**
+- `console_status` - Connection status
+- `command_response` - Command result (`{success, command, output}`)
+
+### Chat Namespace (`/chat`)
+
+Real-time message delivery via Socket.IO.
+
+**Server → Client:**
+- `new_channel_message` - New channel message received
+- `new_dm_message` - New DM received
+- `message_echo` - Echo/ACK update for sent message
+- `dm_ack` - DM delivery confirmation
+
+### Logs Namespace (`/logs`)
+
+Real-time log streaming via Socket.IO.
+
+**Server → Client:**
+- `log_line` - New log line
 
 ---
 
 ## Offline Support
 
-The application works completely offline without internet connection - perfect for mesh networks in remote or emergency scenarios.
-
-### Local Vendor Libraries
-
-| Library | Size | Location |
-|---------|------|----------|
-| Bootstrap 5.3.2 CSS | ~227 KB | `static/vendor/bootstrap/css/` |
-| Bootstrap 5.3.2 JS | ~80 KB | `static/vendor/bootstrap/js/` |
-| Bootstrap Icons 1.11.2 | ~398 KB | `static/vendor/bootstrap-icons/` |
-| Emoji Picker Element | ~529 KB | `static/vendor/emoji-picker-element/` |
-
-**Total offline package size:** ~1.2 MB
-
-### Service Worker Caching
-
-- **Cache version:** `mc-webui-v3`
-- **Strategy:** Hybrid caching
-  - **Cache-first** for vendor libraries (static, unchanging)
-  - **Network-first** for app code (dynamic, needs updates)
-
-### How It Works
-
-1. On first visit (online), Service Worker installs and caches all assets
-2. Vendor libraries (Bootstrap, Icons, Emoji Picker) loaded from cache instantly
-3. App code checks network first, falls back to cache if offline
-4. Complete UI functionality available offline
-5. Only API calls (messages, channels, contacts) require connectivity
-
----
-
-## Environment Variables
-
-| Variable | Description | Default |
-|----------|-------------|---------|
-| `MC_SERIAL_PORT` | Serial device path | - |
-| `MC_DEVICE_NAME` | Device name (for files) | - |
-| `MC_CONFIG_DIR` | Configuration directory | `./data/meshcore` |
-| `MC_ARCHIVE_DIR` | Archive directory | `./data/archive` |
-| `MC_ARCHIVE_ENABLED` | Enable automatic archiving | `true` |
-| `MC_ARCHIVE_RETENTION_DAYS` | Days to show in live view | `7` |
-| `FLASK_HOST` | Listen address | `0.0.0.0` |
-| `FLASK_PORT` | Web server port | `5000` |
-| `FLASK_DEBUG` | Debug mode | `false` |
-| `TZ` | Timezone for logs | `UTC` |
-
----
-
-## Persistent Settings
-
-### Settings File
-
-**Location:** `MC_CONFIG_DIR/.webui_settings.json`
-
-```json
-{
-  "manual_add_contacts": false
-}
-```
-
-### Read Status File
-
-**Location:** `MC_CONFIG_DIR/.read_status.json`
-
-Stores per-channel and per-conversation read timestamps for cross-device synchronization.
-
-```json
-{
-  "channels": {"0": 1735900000, "1": 1735900100},
-  "dm": {"name_User1": 1735900200}
-}
-```
-
----
-
-## Related Documentation
-
-- [User Guide](user-guide.md) - How to use all features
-- [Troubleshooting](troubleshooting.md) - Common issues and solutions
-- [Docker Installation](docker-install.md) - How to install Docker
+The application works completely offline without internet connection. Vendor libraries (Bootstrap, Bootstrap Icons, Socket.IO, Emoji Picker) are bundled locally. A Service Worker provides hybrid caching to ensure functionality without connectivity.

docs/rpt-mgmt.md

@@ -19,7 +19,7 @@ This guide explains how to manage a MeshCore repeater device directly from the m
 
     <img src="../images/RPT-Mgmt-03-new-rpt-approve.png" alt="Approve repeater" width="200px">
 
-- Reset your search filter (it is recommended to leave CLI selected only) and return to the main chat view
+- Reset your search filter (it is recommended to leave COM selected only) and return to the main chat view
 
     <img src="../images/RPT-Mgmt-04-back-to-home.png" alt="Return to home" width="200px">
 

docs/troubleshooting.md

@@ -5,8 +5,8 @@ Common issues and solutions for mc-webui.
 ## Table of Contents
 
 - [Common Issues](#common-issues)
+- [Device Not Responding](#device-not-responding)
 - [Docker Commands](#docker-commands)
-- [Testing Bridge API](#testing-bridge-api)
 - [Backup and Restore](#backup-and-restore)
 - [Next Steps](#next-steps)
 - [Getting Help](#getting-help)
@@ -19,8 +19,7 @@ Common issues and solutions for mc-webui.
 
 **Check logs:**
 ```bash
-docker compose logs meshcore-bridge
-docker compose logs mc-webui
+docker compose logs -f mc-webui
 ```
 
 **Common causes:**
@@ -52,18 +51,19 @@ docker compose ps
 
 ### No messages appearing
 
-**Verify meshcli is working:**
+**Check device connection:**
 ```bash
-# Test meshcli directly in bridge container
-docker compose exec meshcore-bridge meshcli -s /dev/ttyUSB0 infos
+# Check container logs for device communication
+docker compose logs -f mc-webui
 ```
 
-**Check .msgs file:**
+**Check database:**
 ```bash
-docker compose exec mc-webui cat /root/.config/meshcore/YourDeviceName.msgs
+# Verify the database file exists
+ls -la data/meshcore/*.db
 ```
 
-Replace `YourDeviceName` with your `MC_DEVICE_NAME`.
+**Check System Log in the web UI** (Menu → System Log) for real-time device event information.
 
 ---
 
@@ -86,9 +86,9 @@ sudo chmod 666 /dev/serial/by-id/usb-Espressif*
 ls -l /dev/serial/by-id/
 ```
 
-**Restart bridge container:**
+**Restart container:**
 ```bash
-docker compose restart meshcore-bridge
+docker compose restart mc-webui
 ```
 
 **Check device permissions:**
@@ -100,40 +100,33 @@ Should show `crw-rw----` with group `dialout`.
 
 ---
 
-### USB Communication Issues
+### Device not responding
 
-The 2-container architecture resolves common USB timeout/deadlock problems:
-
-- **meshcore-bridge** has exclusive USB access
-- **mc-webui** uses HTTP (no direct device access)
-- Restarting `mc-webui` **does not** affect USB connection
-- If bridge has USB issues, restart only that service:
-  ```bash
-  docker compose restart meshcore-bridge
+**Symptoms:**
+- Container logs show repeated `no_event_received` errors and restarts:
   ```
+  ERROR:meshcore:Error while querying device: Event(type=<EventType.ERROR: 'command_error'>, payload={'reason': 'no_event_received'})
+  ```
+- Device name not detected (auto-detection fails)
+- All commands timeout in the Console
 
----
+**What this means:**
 
-### Bridge connection errors
+The serial connection to the USB adapter (e.g. CP2102) is working, but the MeshCore device firmware is not responding to protocol commands. The device boots (serial port connects), but the application code is not running properly.
 
-```bash
-# Check bridge health
-docker compose exec mc-webui curl http://meshcore-bridge:5001/health
+**What does NOT help:**
+- Restarting Docker containers
+- Restarting the host machine
+- USB reset or USB power cycle (only resets the USB-to-UART adapter, not the MeshCore radio module)
 
-# Bridge logs
-docker compose logs -f meshcore-bridge
+**Fix: Re-flash the firmware**
 
-# Test meshcli directly in bridge container
-docker compose exec meshcore-bridge meshcli -s /dev/ttyUSB0 infos
-```
+The MeshCore device firmware is likely corrupted. Re-flash the latest firmware using the MeshCore Flasher:
+1. Download the latest firmware from [MeshCore releases](https://github.com/ripplebiz/MeshCore/releases)
+2. Flash using [MeshCore Flasher](https://flasher.meshcore.co) or esptool
+3. Restart mc-webui: `docker compose up -d`
 
----
-
-### Messages not updating
-
-- Check that `.msgs` file exists in `MC_CONFIG_DIR`
-- Verify bridge service is healthy: `docker compose ps`
-- Check bridge logs for command errors
+This can happen after a power failure during OTA update, flash memory corruption, or other hardware anomalies.
 
 ---
 
@@ -143,15 +136,9 @@ docker compose exec meshcore-bridge meshcli -s /dev/ttyUSB0 infos
 ```bash
 # mc-webui container logs
 docker compose logs -f mc-webui
-
-# meshcore-bridge container logs (where settings are applied)
-docker compose logs -f meshcore-bridge
 ```
 
-**Look for:**
-- "Loaded webui settings" - confirms settings file is being read
-- "manual_add_contacts set to on/off" - confirms setting is applied to meshcli session
-- "Saved manual_add_contacts=..." - confirms setting is persisted to file
+You can also check the System Log in the web UI (Menu → System Log) for real-time information about contact events and settings changes.
 
 ---
 
@@ -160,17 +147,13 @@ docker compose logs -f meshcore-bridge
 ### View logs
 
 ```bash
-docker compose logs -f              # All services
-docker compose logs -f mc-webui     # Main app only
-docker compose logs -f meshcore-bridge  # Bridge only
+docker compose logs -f mc-webui
 ```
 
-### Restart services
+### Restart
 
 ```bash
-docker compose restart              # Restart both
-docker compose restart mc-webui     # Restart main app only
-docker compose restart meshcore-bridge  # Restart bridge only
+docker compose restart mc-webui
 ```
 
 ### Start / Stop
@@ -196,69 +179,22 @@ docker compose ps
 
 ```bash
 docker compose exec mc-webui sh
-docker compose exec meshcore-bridge sh
 ```
 
 ---
 
-## Testing Bridge API
-
-The `meshcore-bridge` container exposes HTTP endpoints for diagnostics.
-
-### Test endpoints
-
-```bash
-# List pending contacts (from inside mc-webui container or server)
-curl -s http://meshcore-bridge:5001/pending_contacts | jq
-
-# Add a pending contact
-curl -s -X POST http://meshcore-bridge:5001/add_pending \
-  -H 'Content-Type: application/json' \
-  -d '{"selector":"Skyllancer"}' | jq
-
-# Check bridge health
-docker compose exec mc-webui curl http://meshcore-bridge:5001/health
-```
-
-### Example responses
-
-**GET /pending_contacts:**
-```json
-{
-  "success": true,
-  "pending": [
-    {
-      "name": "Skyllancer",
-      "public_key": "f9ef..."
-    },
-    {
-      "name": "KRA Reksio mob2🐕",
-      "public_key": "41d5..."
-    }
-  ],
-  "raw_stdout": "Skyllancer: f9ef...\nKRA Reksio mob2🐕: 41d5..."
-}
-```
-
-**POST /add_pending:**
-```json
-{
-  "success": true,
-  "stdout": "Contact added successfully",
-  "stderr": "",
-  "returncode": 0
-}
-```
-
-**Note:** These endpoints require `manual_add_contacts` mode to be enabled.
-
----
-
 ## Backup and Restore
 
 **All important data is in the `data/` directory.**
 
-### Create backup
+### UI Backup (recommended)
+
+You can create and download database backups directly from the web UI:
+1. Click the menu icon (☰) → "Backup"
+2. Click "Create Backup" to create a timestamped backup
+3. Click "Download" to save a backup to your local machine
+
+### Manual backup (CLI)
 
 ```bash
 cd ~/mc-webui
@@ -296,7 +232,7 @@ After successful installation:
 
 1. **Join channels** - Create or join encrypted channels with other users
 2. **Configure contacts** - Enable manual approval if desired
-3. **Test Direct Messages** - Send DM to other CLI contacts
+3. **Test Direct Messages** - Send DM to other COM contacts
 4. **Set up backups** - Schedule regular backups of `data/` directory
 5. **Read full documentation** - See [User Guide](user-guide.md) for all features
 
@@ -309,9 +245,8 @@ After successful installation:
 - [Architecture](architecture.md) - Technical documentation
 - [README](../README.md) - Installation guide
 - MeshCore docs: https://meshcore.org
-- meshcore-cli docs: https://github.com/meshcore-dev/meshcore-cli
 
 **Issues:**
 - GitHub Issues: https://github.com/MarekWo/mc-webui/issues
 - Check existing issues before creating new ones
-- Include logs when reporting problems
+- Include logs when reporting problems (use Menu → System Log for easy access)

docs/user-guide.md

@@ -10,7 +10,15 @@ This guide covers all features and functionality of mc-webui. For installation i
 - [Sending Messages](#sending-messages)
 - [Message Content Features](#message-content-features)
 - [Direct Messages (DM)](#direct-messages-dm)
+- [Global Search](#global-search)
 - [Contact Management](#contact-management)
+- [Adding Contacts](#adding-contacts)
+- [DM Path Management](#dm-path-management)
+- [Interactive Console](#interactive-console)
+- [Device Dashboard](#device-dashboard)
+- [Settings](#settings)
+- [System Log](#system-log)
+- [Database Backup](#database-backup)
 - [Network Commands](#network-commands)
 - [PWA Notifications](#pwa-notifications)
 
@@ -29,6 +37,8 @@ The main page displays chat history from the currently selected channel. The app
 
 By default, the live view shows messages from the last 7 days. Older messages are automatically archived and can be accessed via the date selector.
 
+On wide screens (tablets/desktops), a sidebar shows the channel list on the left side for quick switching.
+
 ---
 
 ## Managing Channels
@@ -146,11 +156,12 @@ Access the Direct Messages feature:
 
 ### Using the DM Page
 
-1. **Select a recipient** from the dropdown at the top:
+1. **Select a recipient** using the searchable contact selector at the top:
+   - Type to search contacts by name (fuzzy matching)
    - **Existing conversations** are shown first (with message history)
-   - Separator: "--- Available contacts ---"
-   - **All client contacts** from your device (only CLI type, no repeaters/rooms)
-   - You can start a new conversation with anyone in your contacts list
+   - **All companion contacts** from your device (only COM type, no repeaters/rooms)
+   - Click the info icon next to a contact to view their details (public key, type, location)
+   - Use the (x) button to clear the search and select a different contact
 2. Type your message in the input field (max 140 bytes, same as channels)
 3. Use the emoji picker button to insert emojis
 4. Press Enter or click Send
@@ -162,12 +173,13 @@ Access the Direct Messages feature:
 - When you return to the DM page, it automatically opens the last conversation you were viewing
 - This works similarly to how the main page remembers your selected channel
 
-**Note:** Only client contacts (CLI) are shown in the dropdown. Repeaters (REP), rooms (ROOM), and sensors (SENS) are automatically filtered out.
+**Note:** Only companion contacts (COM) are shown in the selector. Repeaters (REP), rooms (ROOM), and sensors (SENS) are automatically filtered out.
 
 ### Message Status Indicators
 
+- ✓ **Delivered** (green checkmark) - Recipient confirmed receipt (ACK). Tap/hover for SNR and route details
+- ? **Unknown** (gray question mark) - No ACK received. Message may still have been delivered — ACK packets are often lost over multi-hop routes. Tap the icon for details
 - ⏳ **Pending** (clock icon, yellow) - Message sent, awaiting delivery confirmation
-- Note: Due to meshcore-cli limitations, we cannot track actual delivery status
 
 ### DM Notifications
 
@@ -175,6 +187,27 @@ Access the Direct Messages feature:
 - Each conversation shows unread indicator (*) in the dropdown
 - DM badge in the menu shows total unread DM count
 
+### Desktop Sidebar
+
+On wide screens (tablets/desktops), the DM page shows a sidebar with the contact list on the left side, making it easy to switch between conversations without using the dropdown selector.
+
+---
+
+## Global Search
+
+Search across all your messages (channels and DMs) using full-text search:
+
+1. Click the menu icon (☰) in the navbar
+2. Select "Search" from the menu
+3. Type your search query and press Enter or click the search button
+
+**Features:**
+- **Full-text search** powered by SQLite FTS5 for fast results
+- **FTS5 syntax support** - Use quotes for exact phrases (`"hello world"`), prefix matching (`mesh*`), boolean operators (`hello AND world`)
+- Results show message content, sender, channel/conversation, and timestamp
+- Click a result to navigate to that channel or DM conversation
+- Syntax help available via the (?) icon next to the search field
+
 ---
 
 ## Contact Management
@@ -205,8 +238,8 @@ When manual approval is enabled, new contacts appear in the Pending Contacts lis
 
 **View contact details:**
 - Contact name with emoji (if present)
-- Type badge (CLI, REP, ROOM, SENS) with color coding:
-  - CLI (blue): Regular clients
+- Type badge (COM, REP, ROOM, SENS) with color coding:
+  - COM (blue): Companions (clients)
   - REP (green): Repeaters
   - ROOM (cyan): Room servers
   - SENS (yellow): Sensors
@@ -215,7 +248,7 @@ When manual approval is enabled, new contacts appear in the Pending Contacts lis
 - Map button (when GPS coordinates are available)
 
 **Filter contacts:**
-- By type: Use checkboxes to show only specific contact types (default: CLI only)
+- By type: Use checkboxes to show only specific contact types (default: COM only)
 - By name or key: Search by partial contact name or public key prefix
 
 **Approve contacts:**
@@ -224,8 +257,12 @@ When manual approval is enabled, new contacts appear in the Pending Contacts lis
   - Confirmation modal shows list of contacts to be approved
   - Progress indicator during batch approval
 
+**Ignore contacts:**
+- **Batch ignore:** Click "Ignore Filtered" to ignore all filtered contacts at once
+- **Single ignore:** Click "Ignore" on individual contacts
+
 **Other actions:**
-- Click "Map" button to view contact location on Google Maps (when GPS data available)
+- Click "Map" button to view contact location on the map (when GPS data available)
 - Click "Copy Key" to copy full public key to clipboard
 - Click "Refresh" to reload pending contacts list
 
@@ -233,7 +270,7 @@ When manual approval is enabled, new contacts appear in the Pending Contacts lis
 
 ### Existing Contacts
 
-The Existing Contacts section displays all contacts currently stored on your device (CLI, REP, ROOM, SENS types).
+The Existing Contacts section displays all contacts currently stored on your device (COM, REP, ROOM, SENS types).
 
 **Features:**
 - **Counter badge** - Shows current contact count vs. 350 limit (MeshCore device max)
@@ -241,7 +278,7 @@ The Existing Contacts section displays all contacts currently stored on your dev
   - Yellow: Warning (300-339 contacts)
   - Red (pulsing): Alarm (≥ 340 contacts)
 - **Search** - Filter contacts by name or public key prefix
-- **Type filter** - Show only specific contact types (All / CLI / REP / ROOM / SENS)
+- **Type filter** - Show only specific contact types (All / COM / REP / ROOM / SENS)
 - **Contact cards** - Display name, type badge, public key prefix, path info, and last seen timestamp
 - **Last Seen** - Shows when each contact was last active with activity indicators:
   - 🟢 **Active** (seen < 5 minutes ago)
@@ -252,10 +289,16 @@ The Existing Contacts section displays all contacts currently stored on your dev
 
 **Managing contacts:**
 1. **Search contacts:** Type in the search box to filter by name or public key prefix
-2. **Filter by type:** Use the type dropdown to show only CLI, REP, ROOM, or SENS
+2. **Filter by type:** Use the type dropdown to show only COM, REP, ROOM, or SENS
 3. **Copy public key:** Click "Copy Key" button to copy the public key prefix to clipboard
 4. **Delete a contact:** Click the "Delete" button (red trash icon) and confirm
 
+**Ignoring and Blocking Contacts:**
+- **Ignore**: The contact is hidden from the main view and their messages do not trigger notifications.
+- **Block**: The contact is completely blocked. Their messages are dropped and will not appear anywhere.
+
+To ignore or block a contact, click the "Ignore" or "Block" button on their contact card. To restore them, switch the type filter to "Ignored" or "Blocked" and click the "Restore" button.
+
 **Contact capacity monitoring:**
 - MeshCore devices have a limit of 350 contacts
 - The counter badge changes color as you approach the limit:
@@ -263,6 +306,13 @@ The Existing Contacts section displays all contacts currently stored on your dev
   - **300-339**: Yellow warning (nearing limit)
   - **340-350**: Red alarm (critical - delete some contacts soon)
 
+### Contact Map
+
+Access the map from the main menu to view the GPS locations of your contacts.
+- Contacts with known GPS coordinates will be displayed as markers on OpenStreetMap.
+- Click a marker to see the contact name and details.
+- Use the **Cached** switch to toggle the display of cache-only contacts (contacts that are saved in your database but no longer present in the device's internal memory).
+
 ### Contact Cleanup Tool
 
 The advanced cleanup tool allows you to filter and remove contacts based on multiple criteria:
@@ -272,7 +322,7 @@ The advanced cleanup tool allows you to filter and remove contacts based on mult
 3. Configure filters:
    - **Name Filter:** Enter partial contact name to search (optional)
    - **Advanced Filters** (collapsible):
-     - **Contact Types:** Select which types to include (CLI, REP, ROOM, SENS)
+     - **Contact Types:** Select which types to include (COM, REP, ROOM, SENS)
      - **Date Field:** Choose between "Last Advert" (recommended) or "Last Modified"
      - **Days of Inactivity:** Contacts inactive for more than X days (0 = ignore)
 4. Click **Preview Cleanup** to see matching contacts
@@ -303,6 +353,206 @@ You can schedule automatic cleanup to run daily at a specified hour:
 
 ---
 
+## Adding Contacts
+
+Add new contacts to your device from the Contact Management page:
+
+1. Click the "Add Contact" button at the top of the Contact Management page
+2. Opens a dedicated page with three methods:
+
+### Paste URI
+
+1. Paste a MeshCore contact URI (`meshcore://...`) into the text field
+2. The contact details (name, public key, type) are automatically parsed and previewed
+3. Click "Add to Device" to add the contact
+
+### Scan QR Code
+
+1. Click "Scan QR" to open the camera
+2. Point at a MeshCore QR code (from another user's Share tab)
+3. The URI is decoded and contact details are previewed
+4. Click "Add to Device" to add the contact
+
+### Manual Entry
+
+1. Enter the contact's public key (64 hex characters)
+2. Optionally enter name, type (COM/REP/ROOM/SENS), and location
+3. Click "Add to Device"
+
+### Cache vs Device Contacts
+
+- **Device contacts** are stored on the MeshCore hardware (limit: 350)
+- **Cache contacts** are stored only in the database (unlimited)
+- Use "Push to Device" to promote a cache contact to the device
+- Use "Move to Cache" to free a device slot while keeping the contact in the database
+
+---
+
+## DM Path Management
+
+Configure message routing paths for individual contacts:
+
+1. Open a DM conversation
+2. Click the contact info icon next to the contact name
+3. In the Contact Info modal, navigate to the "Paths" section
+
+### Path Configuration
+
+- **Add Path** - Add a repeater to the routing path using:
+  - **Repeater picker** - Browse available repeaters by name or ID
+  - **Map picker** - Select repeaters from a map view showing their GPS locations
+  - **Import current path** - Import the path currently stored on the device
+- **Reorder** - Drag paths to change priority (starred path is used first)
+- **Star** - Mark a preferred primary path (used first in retry rotation)
+- **Delete** - Remove individual paths
+
+### Keep Path Toggle
+
+- Enable "Keep path" to prevent the device from automatically switching to FLOOD routing
+- When enabled, the device will always use the configured DIRECT path(s)
+- Useful when you know the optimal route and don't want the device to override it
+
+### Path Operations
+
+- **Reset to FLOOD** - Clear all paths and switch to FLOOD routing
+- **Clear Paths** - Remove all configured paths without changing routing mode
+
+---
+
+## Interactive Console
+
+Access the interactive console for direct MeshCore command execution:
+
+1. Click the menu icon (☰) in the navbar
+2. Select "Console" from the menu
+3. Opens in a fullscreen modal with a command prompt
+
+### Available Command Categories
+
+The console supports a comprehensive set of MeshCore commands organized into categories:
+
+**Repeater Management:**
+- `req_owner <name>` - Request repeater owner info
+- `req_regions <name>` - Request repeater regions
+- `req_clock <name>` - Request repeater clock
+- `req_neighbours <name>` - Request repeater neighbors list
+- `set_owner <name> <value>` - Set repeater owner
+- `set_regions <name> <value>` - Set repeater regions
+- `set_clock <name>` - Sync repeater clock
+
+**Contact Management:**
+- `contacts` - List all device contacts
+- `.contacts` - List contacts (JSON format)
+- `.pending_contacts` - List pending contacts
+- `add_pending <key>` - Approve pending contact
+- `remove_contact <name>` - Remove contact
+
+**Device & Channel Management:**
+- `infos` / `ver` - Device info / firmware version
+- `stats` - Device statistics
+- `self_telemetry` - Own device telemetry
+- `get_channels` - List channels
+- `get <param>` / `set <param> <value>` - Get/set device parameters
+- `trace <name>` - Trace route to contact
+- `neighbours` - Request neighbor list from device
+
+### Console Features
+
+- **Command history** - Navigate with up/down arrows, or use the history dropdown
+- **Persistent history** - Saved on server, accessible across sessions
+- **Auto-reconnect** - WebSocket reconnects automatically on disconnect
+- **Status indicator** - Green/yellow/red dot shows connection status
+- **Human-readable output** - Clock times, statistics, and telemetry formatted for readability
+
+---
+
+## Device Dashboard
+
+Access device information and statistics:
+
+1. Click the menu icon (☰) in the navbar
+2. Select "Device Info" from the menu
+
+### Info Tab
+
+Displays device parameters in a readable table:
+- Device name, type, public key
+- Location coordinates with map button
+- Radio parameters (frequency, bandwidth, spreading factor, coding rate)
+- TX power, multi-acks, location sharing settings
+
+### Stats Tab
+
+Shows live device statistics:
+- Uptime, free memory, battery voltage
+- Message counters (sent, received, forwarded)
+- Current airtime usage
+
+### Share Tab
+
+Share your device contact with others:
+- **QR Code** - Scannable QR code containing your contact URI
+- **URI** - Copyable `meshcore://` URI that others can paste into their Add Contact page
+
+---
+
+## Settings
+
+Access the Settings modal to configure application behavior:
+
+1. Click the menu icon (☰) in the navbar
+2. Select "Settings" from the menu
+
+### DM Retry Settings
+
+Configure how direct messages are retried when delivery is not confirmed:
+- **Retry count** - Number of retry attempts (includes initial send)
+- **Retry interval** - Seconds between retries
+- **Flood fallback attempt** - After which attempt to switch from DIRECT to FLOOD routing
+- **Grace period** - Seconds to wait for late ACKs after all retries complete
+
+### Quote Settings
+
+- **Max quote length** - Maximum number of bytes to include when quoting a message
+
+### Message Retention
+
+- **Live view days** - Number of days of messages shown in the live view (older messages are archived)
+
+### Theme
+
+- **Dark / Light** - Toggle between dark and light UI themes. The preference is saved in local browser storage
+
+---
+
+## System Log
+
+View real-time application logs:
+
+1. Click the menu icon (☰) in the navbar
+2. Select "System Log" from the menu
+3. Opens in a fullscreen modal with streaming log output
+
+The log viewer shows the most recent application log entries and streams new entries in real-time. Useful for monitoring device events, debugging issues, and verifying message delivery.
+
+---
+
+## Database Backup
+
+Create and manage database backups:
+
+1. Click the menu icon (☰) in the navbar
+2. Select "Backup" from the menu
+
+**Features:**
+- **Create backup** - Creates a timestamped copy of the current database
+- **List backups** - View all available backups with timestamps and file sizes
+- **Download** - Download any backup file to your local machine
+
+Backups are stored in the `./data/` directory alongside the main database.
+
+---
+
 ## Network Commands
 
 Access network commands from the slide-out menu under "Network Commands" section:
@@ -418,6 +668,6 @@ To get the full PWA experience with app badge counters:
 - **Repeater Management:** [rpt-mgmt.md](rpt-mgmt.md)
 - **Troubleshooting:** [troubleshooting.md](troubleshooting.md)
 - **Architecture:** [architecture.md](architecture.md)
+- **Container Watchdog:** [watchdog.md](watchdog.md)
 - **MeshCore docs:** https://meshcore.org
-- **meshcore-cli docs:** https://github.com/meshcore-dev/meshcore-cli
 - **GitHub Issues:** https://github.com/MarekWo/mc-webui/issues

docs/watchdog.md

@@ -1,14 +1,16 @@
 # Container Watchdog
 
-The Container Watchdog is a systemd service that monitors Docker containers and automatically restarts unhealthy or stopped ones. This is useful for ensuring reliability, especially on resource-constrained systems.
+The Container Watchdog is a systemd service that monitors the `mc-webui` Docker container and automatically restarts it if it becomes unhealthy or if the LoRa device becomes unresponsive. This is useful for ensuring reliability, especially on resource-constrained systems or when the LoRa hardware hangs.
 
 ## Features
 
 - **Health monitoring** - Checks container status every 30 seconds
-- **Automatic restart** - Restarts containers that become unhealthy
-- **Auto-start stopped containers** - Starts containers that have stopped (configurable)
+- **Log monitoring** - Monitors `mc-webui` logs for specific "unresponsive LoRa device" errors
+- **Automatic restart** - Restarts the container when issues are detected
+- **Auto-start stopped container** - Starts the container if it has stopped (configurable)
+- **Hardware USB reset** - Performs a low-level USB bus reset (unbind/bind or DTR/RTS) if the LoRa device freezes. *Note: USB reset is automatically skipped if a TCP connection is used.*
 - **Diagnostic logging** - Captures container logs before restart for troubleshooting
-- **HTTP status endpoint** - Query container status via HTTP API
+- **HTTP status endpoint** - Query watchdog status via HTTP API
 - **Restart history** - Tracks all automatic restarts with timestamps
 
 ## Installation
@@ -20,9 +22,9 @@ sudo ./scripts/watchdog/install.sh
 
 The installer will:
 - Create a systemd service `mc-webui-watchdog`
-- Start monitoring containers immediately
+- Start monitoring the container immediately
 - Enable automatic startup on boot
-- Create log file at `/var/log/mc-webui-watchdog.log`
+- Create a log file at `/var/log/mc-webui-watchdog.log`
 
 ## Usage
 
@@ -59,9 +61,9 @@ curl http://localhost:5051/history
 
 ### Diagnostic Files
 
-When a container is restarted, diagnostic information is saved to:
+When the container is restarted, diagnostic information is saved to:
 ```
-/tmp/mc-webui-watchdog-{container}-{timestamp}.log
+/tmp/mc-webui-watchdog-mc-webui-{timestamp}.log
 ```
 
 These files contain:
@@ -81,7 +83,8 @@ If you need to customize the behavior, the service supports these environment va
 | `CHECK_INTERVAL` | `30` | Seconds between health checks |
 | `LOG_FILE` | `/var/log/mc-webui-watchdog.log` | Path to log file |
 | `HTTP_PORT` | `5051` | HTTP status port (0 to disable) |
-| `AUTO_START` | `true` | Start stopped containers (set to `false` to disable) |
+| `AUTO_START` | `true` | Start stopped container (set to `false` to disable) |
+| `USB_DEVICE_PATH` | *(auto-detected)* | Path to the LoRa device for hardware USB bus reset |
 
 To modify defaults, create an override file:
 ```bash
@@ -105,29 +108,3 @@ Note: The log file is preserved after uninstall. Remove manually if needed:
 ```bash
 sudo rm /var/log/mc-webui-watchdog.log
 ```
-
-## Troubleshooting
-
-### Service won't start
-
-Check the logs:
-```bash
-journalctl -u mc-webui-watchdog -n 50
-```
-
-Common issues:
-- Docker not running
-- Python 3 not installed
-- Permission issues
-
-### Containers keep restarting
-
-Check the diagnostic files in `/tmp/mc-webui-watchdog-*.log` to see what's causing the containers to become unhealthy.
-
-### HTTP endpoint not responding
-
-Verify the service is running and check if port 5051 is available:
-```bash
-systemctl status mc-webui-watchdog
-ss -tlnp | grep 5051
-```

meshcore-bridge/Dockerfile

@@ -1,28 +0,0 @@
-# MeshCore Bridge Dockerfile
-FROM python:3.11-slim
-
-LABEL maintainer="mc-webui"
-LABEL description="MeshCore CLI Bridge - HTTP API wrapper for meshcli"
-
-WORKDIR /bridge
-
-# Install system dependencies
-RUN apt-get update && apt-get install -y \
-    gcc \
-    python3-dev \
-    && rm -rf /var/lib/apt/lists/*
-
-# Install meshcore-cli (from PyPI)
-RUN pip install --no-cache-dir meshcore-cli==1.3.21
-
-# Copy bridge application
-COPY requirements.txt .
-RUN pip install --no-cache-dir -r requirements.txt
-
-COPY bridge.py .
-
-# Expose bridge API port
-EXPOSE 5001
-
-# Run bridge
-CMD ["python", "bridge.py"]

meshcore-bridge/requirements.txt

@@ -1,10 +0,0 @@
-# MeshCore Bridge - Minimal dependencies
-Flask==3.0.0
-Werkzeug==3.0.1
-
-# WebSocket support for console
-flask-socketio==5.3.6
-python-socketio==5.10.0
-python-engineio==4.8.1
-gevent==23.9.1
-gevent-websocket==0.10.1

requirements.txt

@@ -23,7 +23,13 @@ Pillow==10.1.0
 # HTTP Client for MeshCore Bridge communication
 requests==2.31.0
 
+# Cryptography for pkt_payload computation (AES-128-ECB)
+pycryptodome==3.21.0
+
 # WebSocket support for console (threading mode - no gevent needed)
 flask-socketio==5.3.6
 python-socketio==5.10.0
 python-engineio==4.8.1
+
+# v2: Direct MeshCore device communication (replaces bridge subprocess)
+meshcore>=2.2.0

scripts/check_compat.py

@@ -0,0 +1,558 @@
+#!/usr/bin/env python3
+"""
+meshcore-cli compatibility checker for mc-webui
+
+Tests all meshcli commands and response formats used by mc-webui
+against the currently running meshcore-bridge instance.
+
+Usage (from host, piped into mc-webui container):
+    cd ~/mc-webui
+    cat scripts/check_compat.py | docker compose exec -T mc-webui python -
+
+    # Full mode (includes advert test):
+    cat scripts/check_compat.py | docker compose exec -T mc-webui env FULL=1 python -
+"""
+
+import json
+import os
+import re
+import sys
+import time
+import requests
+
+DEFAULT_BRIDGE_URL = "http://meshcore-bridge:5001"
+
+# Expected fields in .contacts JSON response (per contact entry)
+EXPECTED_CONTACT_FIELDS = {
+    "public_key", "type", "adv_name", "flags",
+    "out_path_len", "out_path", "last_advert",
+    "adv_lat", "adv_lon", "lastmod"
+}
+
+# Valid contact types in text format
+VALID_CONTACT_TYPES = {"COM", "REP", "ROOM", "SENS"}
+
+# Expected fields in /health response
+EXPECTED_HEALTH_FIELDS = {
+    "status", "serial_port", "device_name", "device_name_source"
+}
+
+# Channel line format: "0: Public [8b3387e9c5cdea6ac9e5edbaa115cd72]"
+CHANNEL_REGEX = re.compile(r'^(\d+):\s+(.+?)\s+\[([a-f0-9]{32})\]$')
+
+# Contacts text format: columns separated by 2+ spaces
+CONTACTS_SPLIT_REGEX = re.compile(r'\s{2,}')
+
+
+class CompatChecker:
+    """Checks meshcore-cli compatibility with mc-webui"""
+
+    PASS = "PASS"
+    WARN = "WARN"
+    FAIL = "FAIL"
+    SKIP = "SKIP"
+    ERROR = "ERROR"
+
+    def __init__(self, bridge_url, full_mode=False):
+        self.bridge_url = bridge_url.rstrip('/')
+        self.full_mode = full_mode
+        self.results = []
+
+    def run_command(self, args, timeout=10):
+        """Send command to bridge /cli endpoint. Returns parsed JSON response."""
+        resp = requests.post(
+            f"{self.bridge_url}/cli",
+            json={"args": args, "timeout": timeout},
+            headers={"Connection": "close"},
+            timeout=timeout + 5
+        )
+        resp.raise_for_status()
+        return resp.json()
+
+    def add(self, status, category, detail):
+        """Record a test result."""
+        self.results.append((status, category, detail))
+
+    # ── Test methods ──────────────────────────────────────────────
+
+    def test_health(self):
+        """Test GET /health endpoint"""
+        cat = "Bridge Health"
+        try:
+            resp = requests.get(f"{self.bridge_url}/health", timeout=5)
+            resp.raise_for_status()
+            data = resp.json()
+
+            missing = EXPECTED_HEALTH_FIELDS - set(data.keys())
+            if missing:
+                self.add(self.FAIL, cat, f"missing fields: {', '.join(sorted(missing))}")
+                return
+
+            if data["status"] != "healthy":
+                self.add(self.FAIL, cat, f"status={data['status']} (expected 'healthy')")
+                return
+
+            extra = set(data.keys()) - EXPECTED_HEALTH_FIELDS - {
+                "serial_port_source", "advert_log", "echoes_log"
+            }
+            detail = f"status=healthy, device={data['device_name']}"
+            if extra:
+                self.add(self.WARN, cat, f"{detail} (new fields: {', '.join(sorted(extra))})")
+            else:
+                self.add(self.PASS, cat, detail)
+
+        except Exception as e:
+            self.add(self.ERROR, cat, str(e))
+
+    def test_device_info(self):
+        """Test infos and .infos commands"""
+        for cmd in ["infos", ".infos"]:
+            cat = f"Device Info ({cmd})"
+            try:
+                data = self.run_command([cmd], timeout=5)
+                if not data.get("success"):
+                    self.add(self.FAIL, cat, f"command failed: {data.get('stderr', '')}")
+                    continue
+
+                stdout = data.get("stdout", "").strip()
+                if not stdout:
+                    self.add(self.FAIL, cat, "empty response")
+                    continue
+
+                # Try to parse JSON from output
+                json_obj = self._extract_json_object(stdout)
+                if json_obj is None:
+                    self.add(self.FAIL, cat, "no JSON object found in response")
+                    continue
+
+                if "name" not in json_obj:
+                    self.add(self.FAIL, cat, f"'name' field missing from JSON (keys: {', '.join(json_obj.keys())})")
+                else:
+                    self.add(self.PASS, cat, f"JSON valid, name='{json_obj['name']}'")
+
+            except Exception as e:
+                self.add(self.ERROR, cat, str(e))
+
+    def test_contacts_text(self):
+        """Test contacts command (text format)"""
+        cat = "Contacts (text)"
+        try:
+            data = self.run_command(["contacts"])
+            if not data.get("success"):
+                self.add(self.FAIL, cat, f"command failed: {data.get('stderr', '')}")
+                return
+
+            stdout = data.get("stdout", "").strip()
+            if not stdout:
+                self.add(self.WARN, cat, "empty response (no contacts on device)")
+                return
+
+            # Parse using same logic as cli.py parse_contacts()
+            type_counts = {"COM": 0, "REP": 0, "ROOM": 0, "SENS": 0}
+            parsed = 0
+            unparsed_lines = []
+
+            for line in stdout.split('\n'):
+                line_stripped = line.strip()
+                if not line_stripped or line_stripped.startswith('---') or \
+                   line.lower().startswith('contact') or line.startswith('INFO:') or \
+                   self._is_prompt_line(line_stripped):
+                    continue
+
+                parts = CONTACTS_SPLIT_REGEX.split(line)
+                if len(parts) >= 2:
+                    contact_type = parts[1].strip()
+                    if contact_type in VALID_CONTACT_TYPES:
+                        type_counts[contact_type] += 1
+                        parsed += 1
+                        continue
+
+                unparsed_lines.append(line_stripped[:60])
+
+            if parsed == 0:
+                self.add(self.FAIL, cat, "no contacts parsed - format may have changed")
+                if unparsed_lines:
+                    self.add(self.FAIL, cat, f"unparsed lines: {unparsed_lines[:3]}")
+                return
+
+            types_str = ", ".join(f"{k}={v}" for k, v in type_counts.items() if v > 0)
+            detail = f"{parsed} contacts parsed, types: {types_str}"
+            if unparsed_lines:
+                self.add(self.WARN, cat, f"{detail} ({len(unparsed_lines)} unparsed lines: {unparsed_lines[:3]})")
+            else:
+                self.add(self.PASS, cat, detail)
+
+        except Exception as e:
+            self.add(self.ERROR, cat, str(e))
+
+    def test_contacts_json(self):
+        """Test .contacts command (JSON format)"""
+        cat = "Contacts (JSON)"
+        try:
+            data = self.run_command([".contacts"])
+            if not data.get("success"):
+                self.add(self.FAIL, cat, f"command failed: {data.get('stderr', '')}")
+                return
+
+            stdout = data.get("stdout", "").strip()
+            if not stdout:
+                self.add(self.WARN, cat, "empty response (no contacts on device)")
+                return
+
+            # Parse JSON using brace-matching (same as cli.py)
+            json_obj = self._extract_json_object(stdout)
+            if json_obj is None:
+                self.add(self.FAIL, cat, "no JSON object found in response")
+                return
+
+            if not isinstance(json_obj, dict):
+                self.add(self.FAIL, cat, f"expected dict, got {type(json_obj).__name__}")
+                return
+
+            if len(json_obj) == 0:
+                self.add(self.WARN, cat, "JSON valid but empty (no contacts)")
+                return
+
+            # Check fields in first contact entry
+            first_key = next(iter(json_obj))
+            first_contact = json_obj[first_key]
+
+            if not isinstance(first_contact, dict):
+                self.add(self.FAIL, cat, f"contact entry is {type(first_contact).__name__}, expected dict")
+                return
+
+            actual_fields = set(first_contact.keys())
+            missing = EXPECTED_CONTACT_FIELDS - actual_fields
+            extra = actual_fields - EXPECTED_CONTACT_FIELDS
+
+            detail = f"{len(json_obj)} contacts, all expected fields present"
+            if missing:
+                self.add(self.FAIL, cat, f"missing fields: {', '.join(sorted(missing))}")
+            elif extra:
+                self.add(self.WARN, cat, f"{len(json_obj)} contacts OK (new fields: {', '.join(sorted(extra))})")
+            else:
+                self.add(self.PASS, cat, detail)
+
+        except Exception as e:
+            self.add(self.ERROR, cat, str(e))
+
+    def test_contact_info(self):
+        """Test apply_to t=1 contact_info command"""
+        cat = "Contact Info (apply_to)"
+        try:
+            data = self.run_command(["apply_to", "t=1", "contact_info"])
+            if not data.get("success"):
+                self.add(self.FAIL, cat, f"command failed: {data.get('stderr', '')}")
+                return
+
+            stdout = data.get("stdout", "").strip()
+            if not stdout:
+                self.add(self.WARN, cat, "empty response (no COM contacts)")
+                return
+
+            # contact_info returns multiple JSON objects (one per contact)
+            json_count = 0
+            for line in stdout.split('\n'):
+                line = line.strip()
+                if line.startswith('{'):
+                    try:
+                        json.loads(line)
+                        json_count += 1
+                    except json.JSONDecodeError:
+                        pass
+
+            if json_count > 0:
+                self.add(self.PASS, cat, f"{json_count} contact info entries parsed")
+            else:
+                # Try brace-matching for multi-line JSON
+                json_obj = self._extract_json_object(stdout)
+                if json_obj is not None:
+                    self.add(self.PASS, cat, "contact info JSON parsed (multi-line)")
+                else:
+                    self.add(self.WARN, cat, "command succeeded but no JSON found in output")
+
+        except Exception as e:
+            self.add(self.ERROR, cat, str(e))
+
+    def test_channels(self):
+        """Test get_channels command"""
+        cat = "Channels"
+        try:
+            data = self.run_command(["get_channels"])
+            if not data.get("success"):
+                self.add(self.FAIL, cat, f"command failed: {data.get('stderr', '')}")
+                return
+
+            stdout = data.get("stdout", "").strip()
+            if not stdout:
+                self.add(self.FAIL, cat, "empty response (device should have at least Public channel)")
+                return
+
+            channels = []
+            unparsed = []
+            for line in stdout.split('\n'):
+                line = line.strip()
+                if not line or self._is_prompt_line(line):
+                    continue
+                match = CHANNEL_REGEX.match(line)
+                if match:
+                    channels.append({
+                        'index': int(match.group(1)),
+                        'name': match.group(2),
+                        'key': match.group(3)
+                    })
+                else:
+                    unparsed.append(line[:60])
+
+            if not channels:
+                self.add(self.FAIL, cat, "no channels parsed - format may have changed")
+                if unparsed:
+                    self.add(self.FAIL, cat, f"unparsed lines: {unparsed[:3]}")
+                return
+
+            names = ", ".join(f"{c['name']}(#{c['index']})" for c in channels)
+            detail = f"{len(channels)} channels: {names}"
+            if unparsed:
+                self.add(self.WARN, cat, f"{detail} ({len(unparsed)} unparsed lines: {unparsed[:3]})")
+            else:
+                self.add(self.PASS, cat, detail)
+
+        except Exception as e:
+            self.add(self.ERROR, cat, str(e))
+
+    def test_recv(self):
+        """Test recv command (short timeout)"""
+        cat = "Recv"
+        try:
+            # Use short timeout - we just want to verify the command is accepted
+            data = self.run_command(["recv"], timeout=5)
+            if not data.get("success"):
+                stderr = data.get("stderr", "")
+                # Timeout is acceptable for recv (no new messages)
+                if "timeout" in stderr.lower():
+                    self.add(self.PASS, cat, "command accepted (timed out - no new messages)")
+                else:
+                    self.add(self.FAIL, cat, f"command failed: {stderr}")
+                return
+
+            stdout = data.get("stdout", "").strip()
+            if stdout:
+                self.add(self.PASS, cat, f"command accepted ({len(stdout.split(chr(10)))} lines)")
+            else:
+                self.add(self.PASS, cat, "command accepted (no new messages)")
+
+        except requests.exceptions.Timeout:
+            # Timeout is acceptable for recv
+            self.add(self.PASS, cat, "command accepted (HTTP timeout - normal for recv)")
+        except Exception as e:
+            self.add(self.ERROR, cat, str(e))
+
+    def test_settings(self):
+        """Test set commands used during bridge initialization"""
+        settings = [
+            (["set", "json_log_rx", "on"], "Settings (json_log_rx)"),
+            (["set", "print_adverts", "on"], "Settings (print_adverts)"),
+            (["msgs_subscribe"], "Settings (msgs_subscribe)"),
+        ]
+
+        for args, cat in settings:
+            try:
+                data = self.run_command(args, timeout=5)
+                if data.get("success"):
+                    self.add(self.PASS, cat, "accepted")
+                else:
+                    stderr = data.get("stderr", "")
+                    stdout = data.get("stdout", "")
+                    # Some settings return output but bridge marks as timeout
+                    if "timeout" in stderr.lower() and not stdout:
+                        self.add(self.WARN, cat, "possible timeout (no output)")
+                    else:
+                        self.add(self.FAIL, cat, f"failed: {stderr or stdout}")
+            except Exception as e:
+                self.add(self.ERROR, cat, str(e))
+
+    def test_pending_contacts(self):
+        """Test GET /pending_contacts bridge endpoint"""
+        cat = "Pending Contacts"
+        try:
+            resp = requests.get(f"{self.bridge_url}/pending_contacts", timeout=10)
+            resp.raise_for_status()
+            data = resp.json()
+
+            if "success" not in data:
+                self.add(self.FAIL, cat, "response missing 'success' field")
+                return
+
+            if data.get("success"):
+                contacts = data.get("contacts", data.get("pending", []))
+                self.add(self.PASS, cat, f"endpoint OK ({len(contacts)} pending)")
+            else:
+                self.add(self.WARN, cat, f"endpoint returned success=false: {data.get('error', '')}")
+
+        except Exception as e:
+            self.add(self.ERROR, cat, str(e))
+
+    def test_advert(self):
+        """Test advert command (has network side-effect)"""
+        cat = "Advert"
+        if not self.full_mode:
+            self.add(self.SKIP, cat, "skipped (use --full to enable)")
+            return
+
+        try:
+            data = self.run_command(["advert"], timeout=10)
+            if data.get("success"):
+                self.add(self.PASS, cat, "advertisement sent")
+            else:
+                self.add(self.FAIL, cat, f"failed: {data.get('stderr', '')}")
+        except Exception as e:
+            self.add(self.ERROR, cat, str(e))
+
+    # ── Helpers ───────────────────────────────────────────────────
+
+    @staticmethod
+    def _is_prompt_line(line):
+        """Check if line is a meshcli prompt or summary (not actual data)."""
+        # Prompt lines: "DeviceName|* command" or "DeviceName|*"
+        if '|*' in line:
+            return True
+        # Summary lines: "> 310 contacts in device"
+        if line.startswith('>'):
+            return True
+        return False
+
+    def _extract_json_object(self, text):
+        """Extract first complete JSON object from text using brace-matching."""
+        depth = 0
+        start_idx = None
+
+        for i, char in enumerate(text):
+            if char == '{':
+                if depth == 0:
+                    start_idx = i
+                depth += 1
+            elif char == '}':
+                depth -= 1
+                if depth == 0 and start_idx is not None:
+                    try:
+                        return json.loads(text[start_idx:i + 1])
+                    except json.JSONDecodeError:
+                        start_idx = None
+                        continue
+
+        return None
+
+    def _get_meshcli_version(self):
+        """Try to get meshcore-cli version from bridge container."""
+        try:
+            data = self.run_command(["version"], timeout=5)
+            if data.get("success") and data.get("stdout"):
+                return data["stdout"].strip()
+        except Exception:
+            pass
+        return "unknown"
+
+    # ── Main runner ───────────────────────────────────────────────
+
+    def run_all(self):
+        """Run all tests and print report. Returns exit code."""
+        print()
+        print("meshcore-cli Compatibility Report")
+        print("=" * 50)
+        print(f"Bridge URL: {self.bridge_url}")
+        print(f"Mode: {'full' if self.full_mode else 'safe (read-only)'}")
+        print(f"Timestamp: {time.strftime('%Y-%m-%d %H:%M:%S')}")
+        print()
+
+        # Check bridge is reachable first
+        try:
+            requests.get(f"{self.bridge_url}/health", timeout=3)
+        except Exception as e:
+            print(f"[ERROR] Cannot reach bridge at {self.bridge_url}: {e}")
+            print()
+            print("Make sure meshcore-bridge is running:")
+            print("  docker compose ps")
+            print("  docker compose logs meshcore-bridge")
+            return 1
+
+        # Run all tests
+        tests = [
+            self.test_health,
+            self.test_device_info,
+            self.test_contacts_text,
+            self.test_contacts_json,
+            self.test_contact_info,
+            self.test_channels,
+            self.test_recv,
+            self.test_settings,
+            self.test_pending_contacts,
+            self.test_advert,
+        ]
+
+        for test in tests:
+            test()
+
+        # Print results
+        for status, category, detail in self.results:
+            print(f"[{status:5s}] {category} - {detail}")
+
+        # Summary
+        counts = {s: 0 for s in [self.PASS, self.WARN, self.FAIL, self.SKIP, self.ERROR]}
+        for status, _, _ in self.results:
+            counts[status] += 1
+
+        total_tests = counts[self.PASS] + counts[self.FAIL] + counts[self.ERROR]
+        print()
+        print(f"Result: {counts[self.PASS]}/{total_tests} PASS", end="")
+        if counts[self.WARN]:
+            print(f", {counts[self.WARN]} WARN", end="")
+        if counts[self.FAIL]:
+            print(f", {counts[self.FAIL]} FAIL", end="")
+        if counts[self.ERROR]:
+            print(f", {counts[self.ERROR]} ERROR", end="")
+        if counts[self.SKIP]:
+            print(f", {counts[self.SKIP]} SKIP", end="")
+        print()
+
+        has_failures = counts[self.FAIL] > 0 or counts[self.ERROR] > 0
+        if has_failures:
+            print()
+            print("COMPATIBILITY ISSUES DETECTED - review FAIL/ERROR results above")
+
+        return 1 if has_failures else 0
+
+
+def main():
+    bridge_url = os.environ.get("BRIDGE_URL", DEFAULT_BRIDGE_URL)
+    full_mode = os.environ.get("FULL", "").lower() in ("1", "true", "yes")
+
+    # Support --bridge-url and --full from command line too
+    args = sys.argv[1:]
+    i = 0
+    while i < len(args):
+        if args[i] == "--bridge-url" and i + 1 < len(args):
+            bridge_url = args[i + 1]
+            i += 2
+        elif args[i] == "--full":
+            full_mode = True
+            i += 1
+        elif args[i] in ("-h", "--help"):
+            print("Usage: check_compat.py [--bridge-url URL] [--full]")
+            print(f"  --bridge-url  Bridge URL (default: {DEFAULT_BRIDGE_URL})")
+            print(f"                Or set BRIDGE_URL env var")
+            print(f"  --full        Include tests with network side-effects")
+            print(f"                Or set FULL=1 env var")
+            print()
+            print("Run from host:")
+            print("  cat scripts/check_compat.py | docker compose exec -T mc-webui python -")
+            print("  cat scripts/check_compat.py | docker compose exec -T mc-webui env FULL=1 python -")
+            sys.exit(0)
+        else:
+            i += 1
+
+    checker = CompatChecker(bridge_url, full_mode)
+    sys.exit(checker.run_all())
+
+
+if __name__ == "__main__":
+    main()

scripts/watchdog/README.md

@@ -0,0 +1,21 @@
+# mc-webui Container Watchdog
+
+The `watchdog` service is a utility designed to run on the host machine running the Docker containers for the `mc-webui` project. Its primary purpose is to continuously monitor the health of the application's containers, specifically the `mc-webui` container, which handles the physical connection to the LoRa device (like Heltec V3 or V4).
+
+## Key Capabilities
+
+- **Automated Restarts:** If a container becomes `unhealthy`, stops, or reports device connection issues in its logs, the watchdog automatically restarts it to restore service without human intervention.
+- **Hardware USB Bus Reset:** If the `mc-webui` container fails to recover after three successive restarts (e.g., due to a hardware freeze on the LoRa device itself), the watchdog will intelligently simulate a physical disconnection and reconnection of the device via a low-level USB bus reset, completely resolving hardware lockups.
+
+## Installation / Update
+
+You can easily install or update the watchdog by running the provided installer script with root privileges:
+
+```bash
+cd ~/mc-webui/scripts/watchdog
+sudo ./install.sh
+```
+
+## Detailed Documentation
+
+For full details on configuration, logs, troubleshooting, and more advanced features, please refer to the main [Container Watchdog Documentation](../../docs/watchdog.md) located in the `docs` folder.

scripts/watchdog/install.sh

@@ -98,6 +98,7 @@ Environment=CHECK_INTERVAL=30
 Environment=LOG_FILE=${LOG_FILE}
 Environment=HTTP_PORT=5051
 Environment=AUTO_START=true
+Environment=USB_DEVICE_PATH=${USB_DEVICE_PATH}
 ExecStart=/usr/bin/python3 -u ${SCRIPT_DIR}/watchdog.py
 Restart=always
 RestartSec=10
@@ -144,6 +145,7 @@ echo "Features:"
 echo "  - Checks container health every 30 seconds"
 echo "  - Automatically restarts unhealthy containers"
 echo "  - Saves diagnostic logs before restart"
+echo "  - Performs hardware USB bus reset if LoRa device is stuck"
 echo ""
 echo "Useful commands:"
 echo "  systemctl status $SERVICE_NAME        # Check service status"

scripts/watchdog/mc-webui-watchdog.service

@@ -11,6 +11,7 @@ Environment=MCWEBUI_DIR=/home/marek/mc-webui
 Environment=CHECK_INTERVAL=30
 Environment=LOG_FILE=/var/log/mc-webui-watchdog.log
 Environment=HTTP_PORT=5051
+Environment=USB_DEVICE_PATH=
 ExecStart=/usr/bin/python3 -u /home/marek/mc-webui/scripts/watchdog/watchdog.py
 Restart=always
 RestartSec=10

scripts/watchdog/watchdog.py

@@ -26,6 +26,7 @@ import json
 import subprocess
 import threading
 import time
+import fcntl
 from datetime import datetime
 from http.server import HTTPServer, BaseHTTPRequestHandler
 from pathlib import Path
@@ -37,8 +38,8 @@ LOG_FILE = os.environ.get('LOG_FILE', '/var/log/mc-webui-watchdog.log')
 HTTP_PORT = int(os.environ.get('HTTP_PORT', '5051'))
 AUTO_START = os.environ.get('AUTO_START', 'true').lower() != 'false'
 
-# Containers to monitor
-CONTAINERS = ['meshcore-bridge', 'mc-webui']
+# Containers to monitor (v2: single container, no meshcore-bridge)
+CONTAINERS = ['mc-webui']
 
 # Global state
 last_check_time = None
@@ -59,6 +60,218 @@ def log(message: str, level: str = 'INFO'):
         print(f"[{timestamp}] [ERROR] Failed to write to log file: {e}")
 
 
+# USB Device Reset Constant
+USBDEVFS_RESET = 21780 # 0x5514
+
+def auto_detect_serial_port() -> str:
+    """Detect the serial port (e.g., /dev/ttyACM0) from environment or by-id."""
+    env_file = os.path.join(MCWEBUI_DIR, '.env')
+    serial_port = 'auto'
+    
+    if os.path.exists(env_file):
+        try:
+            with open(env_file, 'r') as f:
+                for line in f:
+                    if line.startswith('MC_SERIAL_PORT='):
+                        serial_port = line.split('=', 1)[1].strip().strip('"\'')
+                        break
+        except Exception as e:
+            log(f"Failed to read .env file for serial port: {e}", "WARN")
+
+    if serial_port.lower() == 'auto':
+        by_id_path = Path('/dev/serial/by-id')
+        if by_id_path.exists():
+            devices = list(by_id_path.iterdir())
+            if len(devices) == 1:
+                serial_port = str(devices[0])
+            elif len(devices) > 1:
+                log("Multiple serial devices found, cannot auto-detect single port", "WARN")
+                return None
+            else:
+                log("No serial devices found in /dev/serial/by-id", "WARN")
+                return None
+        else:
+            log("/dev/serial/by-id does not exist", "WARN")
+            return None
+
+    if not serial_port or not os.path.exists(serial_port):
+        log(f"Serial port {serial_port} not found", "WARN")
+        return None
+        
+    try:
+        real_tty = os.path.realpath(serial_port)
+        return real_tty
+    except Exception as e:
+        log(f"Error resolving serial port {serial_port}: {e}", "ERROR")
+        return None
+
+def auto_detect_usb_device() -> str:
+    """Attempt to auto-detect the physical USB device path (e.g., /dev/bus/usb/001/002) for LoRa."""
+    real_tty = auto_detect_serial_port()
+    if not real_tty:
+        return None
+
+    try:
+        tty_name = os.path.basename(real_tty)
+        # Find USB bus and dev number via sysfs
+        sysfs_path = f"/sys/class/tty/{tty_name}/device"
+        if not os.path.exists(sysfs_path):
+            log(f"Sysfs path {sysfs_path} not found", "WARN")
+            return None
+
+        usb_dev_dir = os.path.dirname(os.path.realpath(sysfs_path))
+        busnum_file = os.path.join(usb_dev_dir, "busnum")
+        devnum_file = os.path.join(usb_dev_dir, "devnum")
+
+        if os.path.exists(busnum_file) and os.path.exists(devnum_file):
+            with open(busnum_file) as f:
+                busnum = int(f.read().strip())
+            with open(devnum_file) as f:
+                devnum = int(f.read().strip())
+            return f"/dev/bus/usb/{busnum:03d}/{devnum:03d}"
+            
+        log("Could not find busnum/devnum files in sysfs", "WARN")
+        return None
+    except Exception as e:
+        log(f"Error during USB device auto-detection: {e}", "ERROR")
+        return None
+
+def is_tcp_connection() -> bool:
+    """Check if the application is configured to use a TCP connection instead of a serial port."""
+    env_file = os.path.join(MCWEBUI_DIR, '.env')
+    
+    if os.path.exists(env_file):
+        try:
+            with open(env_file, 'r') as f:
+                for line in f:
+                    if line.startswith('MC_TCP_HOST='):
+                        val = line.split('=', 1)[1].strip().strip('"\'')
+                        if val:
+                            return True
+        except Exception as e:
+            log(f"Failed to read .env file for TCP host: {e}", "WARN")
+
+    return False
+
+def reset_esp32_device():
+    """Perform a hardware reset on ESP32/LoRa device using DTR/RTS lines via ioctl."""
+    serial_port = auto_detect_serial_port()
+    if not serial_port:
+        log("Cannot perform ESP32 reset: serial port could not be determined", "WARN")
+        return False
+        
+    log(f"Performing ESP32 hard reset via DTR/RTS on {serial_port}", "WARN")
+    try:
+        import struct
+        import termios
+        
+        fd = os.open(serial_port, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
+        
+        TIOCM_DTR = 0x002
+        TIOCM_RTS = 0x004
+        TIOCMBIS = getattr(termios, 'TIOCMBIS', 0x5416)
+        TIOCMBIC = getattr(termios, 'TIOCMBIC', 0x5417)
+        
+        # Reset sequence used by esptool
+        # DTR=False (Clear), RTS=True (Set) -> EN=Low
+        fcntl.ioctl(fd, TIOCMBIC, struct.pack('i', TIOCM_DTR))
+        fcntl.ioctl(fd, TIOCMBIS, struct.pack('i', TIOCM_RTS))
+        time.sleep(0.1)
+        
+        # DTR=False (Clear), RTS=False (Clear) -> EN=High
+        fcntl.ioctl(fd, TIOCMBIC, struct.pack('i', TIOCM_DTR | TIOCM_RTS))
+        time.sleep(0.05)
+        
+        os.close(fd)
+        log("ESP32 DTR/RTS reset sent successfully", "INFO")
+        return True
+    except Exception as e:
+        log(f"ESP32 DTR/RTS reset failed: {e}", "ERROR")
+        return False
+
+def reset_usb_device():
+    """Perform a hardware USB bus reset on the LoRa device."""
+    # First get the sysfs path if possible
+    real_tty = auto_detect_serial_port()
+    usb_sysfs_dir = None
+    usb_device_id = None
+    if real_tty:
+        try:
+            tty_name = os.path.basename(real_tty)
+            sysfs_path = f"/sys/class/tty/{tty_name}/device"
+            if os.path.exists(sysfs_path):
+                usb_intf_dir = os.path.realpath(sysfs_path)
+                usb_sysfs_dir = os.path.dirname(usb_intf_dir)
+                usb_device_id = os.path.basename(usb_sysfs_dir)
+        except Exception as e:
+            log(f"Error finding sysfs path: {e}", "WARN")
+
+    if usb_sysfs_dir:
+        # 1. Try toggling authorized flag
+        authorized_path = os.path.join(usb_sysfs_dir, "authorized")
+        if os.path.exists(authorized_path):
+            log(f"Toggling USB authorized flag on {usb_sysfs_dir}", "WARN")
+            try:
+                with open(authorized_path, 'w') as f:
+                    f.write("0")
+                time.sleep(2)
+                with open(authorized_path, 'w') as f:
+                    f.write("1")
+                log("USB authorized toggle successful", "INFO")
+                time.sleep(2)
+            except Exception as e:
+                log(f"USB authorized toggle failed: {e}", "ERROR")
+
+        # 2. Try unbind/bind from usb driver
+        unbind_path = "/sys/bus/usb/drivers/usb/unbind"
+        bind_path = "/sys/bus/usb/drivers/usb/bind"
+        if os.path.exists(unbind_path) and usb_device_id:
+            log(f"Unbinding USB device {usb_device_id} from usb driver", "WARN")
+            try:
+                with open(unbind_path, 'w') as f:
+                    f.write(usb_device_id)
+                time.sleep(2)
+                with open(bind_path, 'w') as f:
+                    f.write(usb_device_id)
+                log("USB unbind/bind successful", "INFO")
+                time.sleep(2)
+            except Exception as e:
+                log(f"USB unbind/bind failed: {e}", "ERROR")
+
+    # 3. Fallback to ioctl USBDEVFS_RESET
+    device_path = os.environ.get('USB_DEVICE_PATH')
+    if not device_path:
+        device_path = auto_detect_usb_device()
+
+    if not device_path:
+        log("Cannot perform USB ioctl reset: device path could not be determined", "WARN")
+        return False
+
+    log(f"Performing hardware USB bus reset via ioctl on {device_path}", "WARN")
+    try:
+        with open(device_path, 'w') as fd:
+            fcntl.ioctl(fd, USBDEVFS_RESET, 0)
+        log("USB ioctl bus reset successful", "INFO")
+        return True
+    except Exception as e:
+        log(f"USB ioctl reset failed: {e}", "ERROR")
+        return False
+
+def count_recent_restarts(container_name: str, minutes: int = 8) -> int:
+    """Count how many times a container was restarted in the last N minutes due to unhealthiness."""
+    cutoff_time = time.time() - (minutes * 60)
+    count = 0
+    for entry in restart_history:
+        if entry.get('container') == container_name and 'restart_success' in entry:
+            try:
+                dt = datetime.fromisoformat(entry['timestamp'])
+                if dt.timestamp() >= cutoff_time:
+                    count += 1
+            except ValueError:
+                pass
+    return count
+
+
 def run_docker_command(args: list, timeout: int = 30) -> tuple:
     """Run docker command and return (success, stdout, stderr)."""
     try:
@@ -216,8 +429,25 @@ def handle_unhealthy_container(container_name: str, status: dict):
     except Exception as e:
         log(f"Failed to save diagnostic info: {e}", 'ERROR')
 
-    # Restart the container
-    restart_success = restart_container(container_name)
+    # v2: mc-webui owns the device connection directly — USB reset if repeated failures
+    restart_success = False
+    if container_name == 'mc-webui':
+        recent_restarts = count_recent_restarts(container_name, minutes=8)
+        if recent_restarts >= 3 and not is_tcp_connection():
+            log(f"{container_name} has been restarted {recent_restarts} times in the last 8 minutes. Attempting hardware USB reset.", "WARN")
+            # Stop the container first so it releases the serial port
+            run_compose_command(['stop', container_name])
+            reset_esp32_device()
+            if reset_usb_device():
+                time.sleep(5)  # Give OS time to re-enumerate the device
+            restart_success = start_container(container_name)
+        else:
+            if recent_restarts >= 3 and is_tcp_connection():
+                log(f"{container_name} has been restarted {recent_restarts} times in the last 8 minutes. TCP connection used, skipping hardware USB reset.", "WARN")
+            restart_success = restart_container(container_name)
+    else:
+        # Restart the container
+        restart_success = restart_container(container_name)
 
     # Record in history
     restart_history.append({
@@ -233,6 +463,85 @@ def handle_unhealthy_container(container_name: str, status: dict):
         restart_history = restart_history[-50:]
 
 
+def check_device_unresponsive(container_name: str) -> bool:
+    """Check if the container logs indicate the USB device is unresponsive."""
+    success, stdout, stderr = run_compose_command([
+        'logs', '--since', '1m', container_name
+    ])
+    if not success:
+        return False
+        
+    error_patterns = [
+        "No response from meshcore node, disconnecting",
+        "Device connected but self_info is empty",
+        "Failed to connect after 10 attempts"
+    ]
+    
+    for pattern in error_patterns:
+        if pattern in stdout:
+            return True
+            
+    return False
+
+
+def handle_unresponsive_device(container_name: str, status: dict):
+    """Handle an unresponsive device - log details, possibly reset USB, and restart container."""
+    global restart_history
+
+    log(f"Container {container_name} device is unresponsive! Status: {status}", 'WARN')
+
+    # Capture logs before restart
+    log(f"Capturing logs from {container_name} before restart...")
+    logs = get_container_logs(container_name, lines=200)
+
+    # Save detailed diagnostic info
+    diag_file = f"/tmp/mc-webui-watchdog-{container_name}-unresponsive-{datetime.now().strftime('%Y%m%d-%H%M%S')}.log"
+    try:
+        with open(diag_file, 'w') as f:
+            f.write(f"=== Container Diagnostic Report (Unresponsive Device) ===\n")
+            f.write(f"Timestamp: {datetime.now().isoformat()}\n")
+            f.write(f"Container: {container_name}\n")
+            f.write(f"Status: {json.dumps(status, indent=2)}\n")
+            f.write(f"\n=== Recent Logs ===\n")
+            f.write(logs)
+        log(f"Diagnostic info saved to: {diag_file}")
+    except Exception as e:
+        log(f"Failed to save diagnostic info: {e}", 'ERROR')
+
+    # v2: mc-webui owns the device connection directly — USB reset if repeated failures
+    restart_success = False
+    if container_name == 'mc-webui':
+        recent_restarts = count_recent_restarts(container_name, minutes=8)
+        if recent_restarts >= 3 and not is_tcp_connection():
+            log(f"{container_name} has been restarted {recent_restarts} times in the last 8 minutes. Attempting hardware USB reset.", "WARN")
+            # Stop the container first so it releases the serial port
+            run_compose_command(['stop', container_name])
+            reset_esp32_device()
+            if reset_usb_device():
+                time.sleep(5)  # Give OS time to re-enumerate the device
+            restart_success = start_container(container_name)
+        else:
+            if recent_restarts >= 3 and is_tcp_connection():
+                log(f"{container_name} has been restarted {recent_restarts} times in the last 8 minutes. TCP connection used, skipping hardware USB reset.", "WARN")
+            restart_success = restart_container(container_name)
+    else:
+        # Restart the container
+        restart_success = restart_container(container_name)
+
+    # Record in history
+    restart_history.append({
+        'timestamp': datetime.now().isoformat(),
+        'container': container_name,
+        'reason': 'unresponsive_device',
+        'status_before': status,
+        'restart_success': restart_success,
+        'diagnostic_file': diag_file
+    })
+
+    # Keep only last 50 entries
+    if len(restart_history) > 50:
+        restart_history = restart_history[-50:]
+
 def check_containers():
     """Check all monitored containers."""
     global last_check_time, last_check_results
@@ -254,6 +563,8 @@ def check_containers():
                 log(f"Container {container_name} is not running (status: {status['status']}), AUTO_START disabled", 'WARN')
         elif status['health'] == 'unhealthy':
             handle_unhealthy_container(container_name, status)
+        elif container_name == 'mc-webui' and check_device_unresponsive(container_name):
+            handle_unresponsive_device(container_name, status)
 
     last_check_results = results
     return results

tests/test_database.py

@@ -0,0 +1,530 @@
+"""
+Integration tests for mc-webui v2 Database class.
+
+Run: python -m pytest tests/test_database.py -v
+"""
+
+import tempfile
+import time
+from pathlib import Path
+import pytest
+
+from app.database import Database
+
+
+@pytest.fixture
+def db():
+    """Create a temporary database for each test."""
+    with tempfile.TemporaryDirectory() as tmp:
+        yield Database(Path(tmp) / 'test.db')
+
+
+# ================================================================
+# Schema & Initialization
+# ================================================================
+
+class TestInitialization:
+    def test_creates_database_file(self, db):
+        assert db.db_path.exists()
+
+    def test_all_tables_exist(self, db):
+        stats = db.get_stats()
+        expected_tables = [
+            'device', 'contacts', 'channels', 'channel_messages',
+            'direct_messages', 'acks', 'echoes', 'paths',
+            'advertisements', 'read_status'
+        ]
+        for table in expected_tables:
+            assert table in stats, f"Missing table: {table}"
+            assert stats[table] == 0
+
+    def test_wal_mode_enabled(self, db):
+        import sqlite3
+        conn = sqlite3.connect(str(db.db_path))
+        mode = conn.execute("PRAGMA journal_mode").fetchone()[0]
+        conn.close()
+        assert mode == 'wal'
+
+    def test_db_size_in_stats(self, db):
+        stats = db.get_stats()
+        assert stats['db_size_bytes'] > 0
+
+
+# ================================================================
+# Device
+# ================================================================
+
+class TestDevice:
+    def test_set_and_get_device_info(self, db):
+        db.set_device_info(public_key='abc123', name='TestDevice')
+        info = db.get_device_info()
+        assert info is not None
+        assert info['public_key'] == 'abc123'
+        assert info['name'] == 'TestDevice'
+
+    def test_update_device_info(self, db):
+        db.set_device_info(public_key='key1', name='Name1')
+        db.set_device_info(public_key='key2', name='Name2')
+        info = db.get_device_info()
+        assert info['public_key'] == 'key2'
+        assert info['name'] == 'Name2'
+
+    def test_get_device_info_empty(self, db):
+        assert db.get_device_info() is None
+
+
+# ================================================================
+# Contacts
+# ================================================================
+
+class TestContacts:
+    def test_upsert_and_get(self, db):
+        db.upsert_contact('AABB', name='Alice')
+        contacts = db.get_contacts()
+        assert len(contacts) == 1
+        assert contacts[0]['public_key'] == 'aabb'  # lowercased
+        assert contacts[0]['name'] == 'Alice'
+
+    def test_upsert_updates_existing(self, db):
+        db.upsert_contact('AABB', name='Alice')
+        db.upsert_contact('AABB', name='Alice Updated', source='device')
+        contacts = db.get_contacts()
+        assert len(contacts) == 1
+        assert contacts[0]['name'] == 'Alice Updated'
+
+    def test_upsert_preserves_name_on_empty(self, db):
+        db.upsert_contact('AABB', name='Alice')
+        db.upsert_contact('AABB', name='')  # empty name should not overwrite
+        contact = db.get_contact('AABB')
+        assert contact['name'] == 'Alice'
+
+    def test_get_contact_by_key(self, db):
+        db.upsert_contact('AABB', name='Alice')
+        contact = db.get_contact('aabb')
+        assert contact is not None
+        assert contact['name'] == 'Alice'
+
+    def test_get_contact_not_found(self, db):
+        assert db.get_contact('nonexistent') is None
+
+    def test_delete_contact(self, db):
+        db.upsert_contact('AABB', name='Alice')
+        assert db.delete_contact('AABB') is True
+        assert db.get_contact('AABB') is None
+
+    def test_delete_nonexistent(self, db):
+        assert db.delete_contact('nonexistent') is False
+
+    def test_protect_contact(self, db):
+        db.upsert_contact('AABB', name='Alice')
+        db.set_contact_protected('AABB', True)
+        contact = db.get_contact('AABB')
+        assert contact['is_protected'] == 1
+
+    def test_protected_not_overwritten(self, db):
+        db.upsert_contact('AABB', name='Alice')
+        db.set_contact_protected('AABB', True)
+        db.upsert_contact('AABB', name='Alice', is_protected=0)
+        contact = db.get_contact('AABB')
+        assert contact['is_protected'] == 1  # stays protected
+
+    def test_contact_with_gps(self, db):
+        db.upsert_contact('CC', name='Bob', adv_lat=52.23, adv_lon=21.01)
+        contact = db.get_contact('CC')
+        assert abs(contact['adv_lat'] - 52.23) < 0.001
+        assert abs(contact['adv_lon'] - 21.01) < 0.001
+
+    def test_get_protected_keys(self, db):
+        db.upsert_contact('AA', name='Alice')
+        db.upsert_contact('BB', name='Bob')
+        db.set_contact_protected('AA', True)
+        keys = db.get_protected_keys()
+        assert 'aa' in keys
+        assert 'bb' not in keys
+
+
+# ================================================================
+# App Settings
+# ================================================================
+
+class TestAppSettings:
+    def test_set_and_get_setting(self, db):
+        db.set_setting('test_key', 'test_value')
+        assert db.get_setting('test_key') == 'test_value'
+
+    def test_get_nonexistent_setting(self, db):
+        assert db.get_setting('nonexistent') is None
+
+    def test_set_and_get_json(self, db):
+        db.set_setting_json('cleanup', {'enabled': True, 'days': 7})
+        result = db.get_setting_json('cleanup')
+        assert result == {'enabled': True, 'days': 7}
+
+    def test_get_json_default(self, db):
+        result = db.get_setting_json('missing', {'default': True})
+        assert result == {'default': True}
+
+    def test_setting_upsert(self, db):
+        db.set_setting_json('key', 'old')
+        db.set_setting_json('key', 'new')
+        assert db.get_setting_json('key') == 'new'
+
+
+# ================================================================
+# Channels
+# ================================================================
+
+class TestChannels:
+    def test_upsert_and_list(self, db):
+        db.upsert_channel(0, 'Public')
+        db.upsert_channel(1, 'Private', secret='abc123')
+        channels = db.get_channels()
+        assert len(channels) == 2
+        assert channels[0]['idx'] == 0
+        assert channels[1]['name'] == 'Private'
+
+    def test_delete_channel(self, db):
+        db.upsert_channel(0, 'Public')
+        assert db.delete_channel(0) is True
+        assert len(db.get_channels()) == 0
+
+
+# ================================================================
+# Channel Messages
+# ================================================================
+
+class TestChannelMessages:
+    def test_insert_and_get(self, db):
+        ts = int(time.time())
+        msg_id = db.insert_channel_message(
+            channel_idx=0, sender='Alice', content='Hello!',
+            timestamp=ts, snr=-5.5, path_len=2
+        )
+        assert msg_id > 0
+
+        messages = db.get_channel_messages(0)
+        assert len(messages) == 1
+        assert messages[0]['sender'] == 'Alice'
+        assert messages[0]['content'] == 'Hello!'
+        assert messages[0]['snr'] == -5.5
+
+    def test_limit_and_offset(self, db):
+        ts = int(time.time())
+        for i in range(10):
+            db.insert_channel_message(0, f'User{i}', f'Msg {i}', ts + i)
+
+        messages = db.get_channel_messages(0, limit=3)
+        assert len(messages) == 3
+        # Should be the last 3 messages
+        assert messages[0]['content'] == 'Msg 7'
+        assert messages[2]['content'] == 'Msg 9'
+
+    def test_filter_by_channel(self, db):
+        ts = int(time.time())
+        db.insert_channel_message(0, 'A', 'Chan 0 msg', ts)
+        db.insert_channel_message(1, 'B', 'Chan 1 msg', ts + 1)
+
+        ch0 = db.get_channel_messages(0)
+        ch1 = db.get_channel_messages(1)
+        assert len(ch0) == 1
+        assert len(ch1) == 1
+        assert ch0[0]['content'] == 'Chan 0 msg'
+
+    def test_delete_channel_messages(self, db):
+        ts = int(time.time())
+        db.insert_channel_message(0, 'A', 'Keep', ts)
+        db.insert_channel_message(1, 'B', 'Delete', ts)
+        deleted = db.delete_channel_messages(1)
+        assert deleted == 1
+        assert len(db.get_channel_messages(0)) == 1
+        assert len(db.get_channel_messages(1)) == 0
+
+    def test_own_message(self, db):
+        ts = int(time.time())
+        db.insert_channel_message(0, 'Me', 'My msg', ts, is_own=True)
+        messages = db.get_channel_messages(0)
+        assert messages[0]['is_own'] == 1
+
+
+# ================================================================
+# Direct Messages
+# ================================================================
+
+class TestDirectMessages:
+    def test_insert_and_get(self, db):
+        db.upsert_contact('aabb', name='Alice')
+        ts = int(time.time())
+        dm_id = db.insert_direct_message('aabb', 'in', 'Hello', ts)
+        assert dm_id > 0
+
+        messages = db.get_dm_messages('aabb')
+        assert len(messages) == 1
+        assert messages[0]['direction'] == 'in'
+        assert messages[0]['content'] == 'Hello'
+
+    def test_conversations_list(self, db):
+        db.upsert_contact('aa', name='Alice')
+        db.upsert_contact('bb', name='Bob')
+        ts = int(time.time())
+        db.insert_direct_message('aa', 'in', 'Hi from Alice', ts)
+        db.insert_direct_message('bb', 'out', 'Hi to Bob', ts + 1)
+
+        convos = db.get_dm_conversations()
+        assert len(convos) == 2
+        # Most recent first
+        assert convos[0]['display_name'] == 'Bob'
+        assert convos[1]['display_name'] == 'Alice'
+
+    def test_dm_with_ack(self, db):
+        db.upsert_contact('aa', name='Alice')
+        ts = int(time.time())
+        dm_id = db.insert_direct_message('aa', 'out', 'Test', ts, expected_ack='ACK123')
+        db.insert_ack('ACK123', snr=-3.0, dm_id=dm_id)
+
+        ack = db.get_ack_for_dm('ACK123')
+        assert ack is not None
+        assert ack['snr'] == -3.0
+
+    def test_dm_with_pkt_payload(self, db):
+        db.upsert_contact('cc', name='Charlie')
+        ts = int(time.time())
+        dm_id = db.insert_direct_message(
+            'cc', 'in', 'Hello', ts, pkt_payload='deadbeef01020304'
+        )
+        messages = db.get_dm_messages('cc')
+        assert len(messages) == 1
+        assert messages[0]['pkt_payload'] == 'deadbeef01020304'
+
+
+# ================================================================
+# Echoes
+# ================================================================
+
+class TestEchoes:
+    def test_insert_and_get(self, db):
+        ts = int(time.time())
+        cm_id = db.insert_channel_message(0, 'Me', 'Test', ts, pkt_payload='PKT1')
+        db.insert_echo('PKT1', path='Me>Node1>Node2', snr=-4.0, cm_id=cm_id)
+
+        echoes = db.get_echoes_for_message('PKT1')
+        assert len(echoes) == 1
+        assert echoes[0]['path'] == 'Me>Node1>Node2'
+
+
+# ================================================================
+# Full-Text Search (FTS5)
+# ================================================================
+
+class TestFTS:
+    def test_search_channel_messages(self, db):
+        ts = int(time.time())
+        db.insert_channel_message(0, 'Alice', 'MeshCore is awesome', ts)
+        db.insert_channel_message(0, 'Bob', 'Hello world', ts + 1)
+
+        results = db.search_messages('awesome')
+        assert len(results) == 1
+        assert results[0]['content'] == 'MeshCore is awesome'
+
+    def test_search_direct_messages(self, db):
+        db.upsert_contact('aa', name='Alice')
+        ts = int(time.time())
+        db.insert_direct_message('aa', 'in', 'Secret mesh network', ts)
+
+        results = db.search_messages('mesh network')
+        assert len(results) == 1
+        assert results[0]['msg_source'] == 'dm'
+
+    def test_search_combined(self, db):
+        db.upsert_contact('aa', name='Alice')
+        ts = int(time.time())
+        db.insert_channel_message(0, 'Bob', 'Testing mesh', ts)
+        db.insert_direct_message('aa', 'in', 'Testing mesh too', ts + 1)
+
+        results = db.search_messages('testing mesh')
+        assert len(results) == 2
+
+    def test_search_no_results(self, db):
+        results = db.search_messages('nonexistent')
+        assert len(results) == 0
+
+
+# ================================================================
+# Read Status
+# ================================================================
+
+class TestReadStatus:
+    def test_mark_and_get(self, db):
+        db.mark_read('chan_0', 1000)
+        status = db.get_read_status()
+        assert 'chan_0' in status
+        assert status['chan_0']['last_seen_ts'] == 1000
+
+    def test_mark_keeps_max_timestamp(self, db):
+        db.mark_read('chan_0', 2000)
+        db.mark_read('chan_0', 1000)  # older — should not downgrade
+        status = db.get_read_status()
+        assert status['chan_0']['last_seen_ts'] == 2000
+
+    def test_mute_channel(self, db):
+        db.set_channel_muted(0, True)
+        status = db.get_read_status()
+        assert status['chan_0']['is_muted'] == 1
+
+        db.set_channel_muted(0, False)
+        status = db.get_read_status()
+        assert status['chan_0']['is_muted'] == 0
+
+
+# ================================================================
+# Backup
+# ================================================================
+
+class TestBackup:
+    def test_create_backup(self, db):
+        db.insert_channel_message(0, 'Test', 'Backup test', int(time.time()))
+        backup_dir = db.db_path.parent / 'backups'
+        backup_path = db.create_backup(backup_dir)
+        assert backup_path.exists()
+        assert backup_path.stat().st_size > 0
+
+    def test_list_backups(self, db):
+        backup_dir = db.db_path.parent / 'backups'
+        db.create_backup(backup_dir)
+        backups = db.list_backups(backup_dir)
+        assert len(backups) == 1
+        assert backups[0]['filename'].endswith('.db')
+
+    def test_list_backups_empty_dir(self, db):
+        with tempfile.TemporaryDirectory() as tmp:
+            backups = db.list_backups(Path(tmp))
+            assert len(backups) == 0
+
+
+# ================================================================
+# Maintenance
+# ================================================================
+
+class TestMaintenance:
+    def test_cleanup_old_messages(self, db):
+        now = int(time.time())
+        old = now - 86400 * 10  # 10 days ago
+        db.insert_channel_message(0, 'Old', 'Old msg', old)
+        db.insert_channel_message(0, 'New', 'New msg', now)
+
+        deleted = db.cleanup_old_messages(days=5)
+        assert deleted == 1
+        remaining = db.get_channel_messages(0)
+        assert len(remaining) == 1
+        assert remaining[0]['content'] == 'New msg'
+
+    def test_stats(self, db):
+        db.upsert_contact('aa', name='Alice')
+        db.insert_channel_message(0, 'A', 'Test', int(time.time()))
+        stats = db.get_stats()
+        assert stats['contacts'] == 1
+        assert stats['channel_messages'] == 1
+
+
+# ================================================================
+# Advertisements
+# ================================================================
+
+class TestAdvertisements:
+    def test_insert(self, db):
+        db.insert_advertisement(
+            'AABB', 'Alice', type=1, lat=52.23, lon=21.01,
+            timestamp=int(time.time()), snr=-3.0
+        )
+        stats = db.get_stats()
+        assert stats['advertisements'] == 1
+
+
+# ================================================================
+# Paths
+# ================================================================
+
+class TestPaths:
+    def test_insert(self, db):
+        db.insert_path('aa', pkt_payload='PKT', path='A>B>C', snr=-5.0, path_len=3)
+        stats = db.get_stats()
+        assert stats['paths'] == 1
+
+
+# ================================================================
+# v1 Migration
+# ================================================================
+
+class TestV1Migration:
+    def _write_msgs(self, path, lines):
+        """Write JSONL lines to a .msgs file."""
+        import json
+        with open(path, 'w') as f:
+            for line in lines:
+                f.write(json.dumps(line) + '\n')
+
+    def test_migrate_channel_messages(self, db):
+        import tempfile, json
+        from app.migrate_v1 import migrate_v1_data, should_migrate
+
+        with tempfile.TemporaryDirectory() as tmp:
+            data_dir = Path(tmp)
+            self._write_msgs(data_dir / 'TestDevice.msgs', [
+                {'type': 'CHAN', 'channel_idx': 0, 'text': 'Alice: Hello world', 'timestamp': 1000, 'SNR': -5.0, 'path_len': 2},
+                {'type': 'SENT_CHAN', 'channel_idx': 0, 'text': 'My message', 'timestamp': 1001, 'sender': 'TestDevice'},
+                {'type': 'CHAN', 'channel_idx': 1, 'text': 'Bob: On channel 1', 'timestamp': 1002},
+            ])
+
+            assert should_migrate(db, data_dir, 'TestDevice')
+
+            result = migrate_v1_data(db, data_dir, 'TestDevice')
+            assert result['status'] == 'completed'
+            assert result['channel_messages'] == 3
+
+            msgs = db.get_channel_messages()
+            assert len(msgs) == 3
+            assert msgs[0]['sender'] == 'Alice'
+            assert msgs[0]['content'] == 'Hello world'
+            assert msgs[1]['sender'] == 'TestDevice'
+            assert msgs[1]['content'] == 'My message'
+            assert msgs[1]['is_own'] == 1
+            assert msgs[2]['sender'] == 'Bob'
+            assert msgs[2]['channel_idx'] == 1
+
+    def test_migrate_dm_messages(self, db):
+        import tempfile, json
+        from app.migrate_v1 import migrate_v1_data
+
+        with tempfile.TemporaryDirectory() as tmp:
+            data_dir = Path(tmp)
+            self._write_msgs(data_dir / 'TestDevice.msgs', [
+                {'type': 'PRIV', 'text': 'Hello from Alice', 'timestamp': 2000, 'pubkey_prefix': 'aabb', 'name': 'Alice'},
+                {'type': 'SENT_MSG', 'text': 'Reply to Alice', 'timestamp': 2001, 'recipient': 'Alice', 'txt_type': 0},
+                {'type': 'SENT_MSG', 'text': 'Channel sent', 'timestamp': 2002, 'txt_type': 1},  # should be skipped
+            ])
+
+            result = migrate_v1_data(db, data_dir, 'TestDevice')
+            assert result['status'] == 'completed'
+            assert result['direct_messages'] == 2
+            assert result['skipped'] == 1
+
+    def test_should_migrate_false_when_db_has_data(self, db):
+        import tempfile
+        from app.migrate_v1 import should_migrate
+
+        with tempfile.TemporaryDirectory() as tmp:
+            data_dir = Path(tmp)
+            self._write_msgs(data_dir / 'Dev.msgs', [
+                {'type': 'CHAN', 'text': 'Test: msg', 'timestamp': 1000},
+            ])
+
+            # Add a message to DB first
+            db.insert_channel_message(0, 'X', 'Existing', int(time.time()))
+
+            assert not should_migrate(db, data_dir, 'Dev')
+
+    def test_should_migrate_false_when_no_msgs_file(self, db):
+        import tempfile
+        from app.migrate_v1 import should_migrate
+
+        with tempfile.TemporaryDirectory() as tmp:
+            assert not should_migrate(db, Path(tmp), 'NoDevice')