49 Commits

Author SHA1 Message Date
MarekWo
3e8eb00e3e feat(contacts): add manual_add command for adding contacts from URI or params
Add support for adding contacts manually using the MeshCore mobile app URI
format (meshcore://contact/add?name=...&public_key=...&type=...) or raw
parameters (public_key, type, name). This enables contact sharing between
mc-webui and the MeshCore Android/iOS app via URI/QR codes.

- Add parse_meshcore_uri() helper to parse mobile app URIs
- Add DeviceManager.add_contact_manual() using CMD_ADD_UPDATE_CONTACT
- Update import_contact_uri() to handle both mobile app and hex blob URIs
- Add manual_add console command with two usage variants
- Update console help text

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-24 17:52:22 +01:00
MarekWo
d54d8f58dd fix(console): fix node_discover display using correct payload fields
The DISCOVER_RESPONSE payload uses 'pubkey' and 'node_type', not
'public_key'/'name'/'adv_name'. Now shows pubkey prefix, resolved
contact name, node type, SNR, and RSSI. Also rename CLI->COM type.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-24 14:17:42 +01:00
MarekWo
5ccd882c5a refactor: eliminate JSONL companion files, delegate to DB
Remove contacts_cache.jsonl and adverts.jsonl file I/O — all contact
data is already in the SQLite contacts/advertisements tables. Clean up
stale JSONL files (acks, echoes, path, dm_sent) at startup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-24 08:16:41 +01:00
MarekWo
2a9f90c01d refactor: migrate read_status from JSON file to SQLite database
Replace file-based .read_status.json with DB-backed read_status table.
One-time migration imports existing data at startup. The read_status.py
module keeps the same public API so route handlers need no changes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-24 08:13:26 +01:00
MarekWo
acfa5d3550 refactor: use public key prefix for DB filename instead of device name
DB filename changes from {device_name}.db to mc_{pubkey[:8]}.db,
making it stable across device renames and preparing for multi-device support.
Existing databases are auto-migrated at startup by probing the device table.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-24 08:11:20 +01:00
MarekWo
33a71bed17 refactor(ui): rename contact type label CLI to COM (companion)
The MeshCore community uses "companion" not "client" for type 1 nodes.
Rename the CLI label to COM across all UI, API, JS, and docs to align
with official terminology. Includes cache migration for old CLI entries.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 14:37:30 +01:00
MarekWo
a1f2a1c5ef feat: name database file after device name for multi-device support
Database file is now named {device_name}.db (e.g. MarWoj.db) instead of
the generic mc-webui.db. On first boot, mc-webui.db is automatically
renamed once the device name is detected. On subsequent boots, the
existing device-named DB is found by scanning the config directory.

This enables future multi-device support where each MeshCore device
has its own separate database file.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 21:07:58 +01:00
MarekWo
0110e65b97 feat: add System Log viewer with real-time streaming
In-memory ring buffer (2000 entries) captures all Python log records.
New /logs page streams entries via WebSocket in real-time with:
- Level filter (DEBUG/INFO/WARNING/ERROR)
- Module filter (auto-populated from seen loggers)
- Text search with highlighting
- Auto-scroll with pause/resume
- Dark theme matching Console style

Menu entry added under Configuration section.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 20:34:29 +01:00
MarekWo
4f25d244b1 refactor: migrate .webui_settings.json to database + fix NEW_CONTACT edge case
All settings (protected_contacts, cleanup_settings, retention_settings,
manual_add_contacts) moved from .webui_settings.json file to SQLite database.
Startup migration auto-imports existing file and renames it to .json.bak.

Added safeguard in _on_new_contact: if firmware fires NEW_CONTACT for a
contact already on the device, skip pending and log a warning. Also added
diagnostic logging showing previous DB state (source, protected) when
contacts reappear as pending.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 20:14:15 +01:00
MarekWo
39a0e944a7 fix(console): correct trace display order
SNR precedes the hop hash: 12.50 > [5e]12.25 > [d1]-8.25 > [e7]-3.00
(each SNR shows link quality, hash shows the next relay node)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 17:29:53 +01:00
MarekWo
4b4e71f5bd fix(console): correct trace output format
Format: 12.50 > [5e]12.25 > [d1]-8.25 > [e7]-3.00
(SNR first, then each hop shows [hash]SNR)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 17:17:40 +01:00
MarekWo
20924d134d fix(console): trace path support, stats field names, self_telemetry format
- trace: accepts comma-separated hex path (e.g. "trace 5e,d1,e7"),
  waits for TRACE_DATA response with proper timeout from device
- stats: fix field names (uptime_secs, queue_len, battery_mv, etc.),
  show all radio/packet stats with detail breakdown
- self_telemetry: format LPP sensor data nicely instead of raw dict

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 17:03:35 +01:00
MarekWo
019d351ab7 fix(console): req_regions and req_owner output formatting
- req_regions: library returns string, not dict — was crashing
  with "'str' object has no attribute 'items'"
- req_owner: format like meshcore-cli ("X is owned by Y")

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 16:17:42 +01:00
MarekWo
3057882f20 fix(console): get/set help formatting, fix get path_hash_mode
- get help / set help: detailed parameter descriptions with
  explanations, matching meshcore-cli style
- get path_hash_mode: library returns int not Event, fixed check
- set help: now reachable (was behind len(args)>=3 guard)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 13:42:31 +01:00
MarekWo
5a4c259c0b fix(console): ver command now queries firmware info properly
Was using self_info (which has no firmware data). Now uses
send_device_query() like meshcore-cli, showing model, version,
build date and repeat mode.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 13:23:56 +01:00
MarekWo
3acdc7a402 feat(console): fix req_clock format, add req_neighbours command
- req_clock: parse timestamp from binary hex data (little-endian)
  and display as human-readable datetime, matching meshcore-cli
- req_neighbours: new command that fetches neighbour list from
  repeater with formatted output (name resolution from device
  contacts and DB cache, time ago, SNR)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 12:25:28 +01:00
MarekWo
fe7c67ee9a fix(console): human-readable clock, fix repeater timeouts
- Clock command now shows datetime like meshcore-cli: "Current time: 2026-03-19 11:39:07 (1773916747)"
- Repeater req_* commands: pass timeout=0 to meshcore library so it uses
  device's suggested_timeout instead of hardcoded 30s (matching meshcore-cli behavior)
- Execute timeout raised to 120s to accommodate slow repeater responses

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 11:58:46 +01:00
MarekWo
4f64cc92e5 feat(console): add device/channel management commands (Etap 3)
Add device management: get/set params, clock/clock sync, time,
reboot, ver, scope, self_telemetry, node_discover.
Add channel management: get_channel, set_channel, add_channel,
remove_channel. Update help text with all command categories.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 08:10:46 +01:00
MarekWo
d80f9a7b3a feat(console): add contact management commands (Etap 2)
Add 14 console commands for contact management: contact_info,
path, disc_path, reset_path, change_path, advert_path,
share_contact, export_contact, import_contact, remove_contact,
change_flags, pending_contacts, add_pending, flush_pending.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 08:09:03 +01:00
MarekWo
d6b92e2754 feat(console): add repeater management commands (Etap 1)
Add 9 new console commands for repeater management:
login, logout, cmd, req_status, req_regions, req_owner,
req_acl, req_clock, req_mma. Add resolve_contact helper
and _parse_time_arg utility. Update help text with categories.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 08:07:22 +01:00
MarekWo
3622619ba4 fix(contacts): decode path correctly using MeshCore V1 encoding
Path buffer from firmware contains trailing garbage bytes beyond the
actual hop data. out_path_len encodes both hop count (lower 6 bits)
and hash size (upper 2 bits). Now we:
- Truncate out_path to meaningful bytes (hop_count * hash_size)
- Format as readable E7→DE→54→54→D8 instead of raw hex string
- Show hop count derived from actual path arrows

Example: out_path_len=5 with out_path="e7de5454d81c49dfb86f8a"
now correctly displays as "E7→DE→54→54→D8 (5 hops)" instead of
showing the full 11-byte buffer.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 21:26:06 +01:00
MarekWo
5f72f40742 feat(contacts): show path/route info in UI and split console commands
- Console `contacts` now shows device-only contacts with path info
  (matching meshcore-cli format: name, type, pubkey, path)
- New `contacts_all` command shows all contacts (device + cached from DB)
- Contact cards in UI now always show routing mode for device contacts
  (Flood, Direct 0 hop, or hex path with hop count)
- Fix path_or_mode computation: prioritize out_path over out_path_len
  to handle firmware edge case where out_path exists but out_path_len=-1

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 21:01:15 +01:00
MarekWo
e4a1e75cc0 fix(socketio): register /chat namespace handler to fix real-time message delivery
The /chat namespace had no server-side connect handler registered. With
python-socketio 5.x (always_connect=False), client connections to
unregistered namespaces are silently rejected. This caused all SocketIO
events (new_message, ack, echo) to never reach the frontend — messages
only appeared via the 60s polling fallback.

Fixes:
- Add @socketio.on('connect', namespace='/chat') handler in main.py
- Add optimistic message append: sent messages appear instantly before
  API round-trip (eliminates 3-4s serial command delay)
- Skip own-message SocketIO events to prevent duplicates
- Add connect_error handler for frontend debugging
- Bump SW cache to v6

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 08:15:43 +01:00
MarekWo
6fba37c609 feat(console): add stats, telemetry, neighbors, trace commands
- stats: device uptime, TX/RX air time, packet counts, errors
- telemetry <name>: request sensor data from remote node
- neighbors <name>: list neighbors of a remote node
- trace [tag]: send trace packet for mesh topology discovery

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 07:24:08 +01:00
MarekWo
d6a7354f06 fix(channels): use device-reported max_channels instead of hardcoded 8
Firmware reports MAX_GROUP_CHANNELS (typically 40 for companion builds)
in the DEVICE_INFO response. Fetch it at startup and use it in all
channel iteration loops. Previously hardcoded range(8) prevented
channels 8+ from appearing and blocked adding new channels.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 20:38:15 +01:00
MarekWo
b034a181ce feat(retention): add message retention scheduling (Task 2.6)
- Add daily retention job that deletes old channel messages, DMs, and
  advertisements based on configurable age threshold
- Add GET/POST /api/retention-settings endpoints
- Extend cleanup_old_messages() to optionally include DMs and adverts
- Wire up APScheduler in create_app() (also enables existing archiving
  and contact cleanup schedulers that were never started in v2)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 17:28:54 +01:00
MarekWo
5df10f0ab9 feat(v2): Expand console router with status, channels, help commands
- Add 'status' command: connection, name, battery, contacts count
- Add 'channels' command: list configured channels (0-7)
- Add 'help' command: list all available commands with descriptions
- Update unknown command message to suggest 'help'

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 17:23:40 +01:00
MarekWo
97a2014af2 feat(v2): Auto-migrate v1 .msgs data to SQLite on first startup
Reads the existing .msgs JSONL file and imports channel messages and DMs
into the v2 SQLite database. Runs automatically when device connects and
DB is empty. Handles sender parsing, pubkey resolution, and FK constraints.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 11:32:28 +01:00
MarekWo
badf67cf74 feat(v2): Rewrite main.py and cli.py for direct device communication
main.py: Initialize Database + DeviceManager in create_app(), replace
bridge-dependent startup code, simplified console command router.
cli.py: All functions now delegate to DeviceManager instead of HTTP
bridge calls. Same signatures preserved for api.py compatibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 07:23:59 +01:00
MarekWo
de0108d6aa feat: Add persistent contacts cache for @mention autocomplete
Contacts cache accumulates all known node names from device contacts
and adverts into a JSONL file, so @mentions work even after contacts
are removed from the device. Background thread scans adverts every
45s and parses advert payloads to extract public keys and node names.

Existing Contacts page now shows merged view with "Cache" badge for
contacts not on device, plus source filter (All/On device/Cache only).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 17:13:36 +01:00
MarekWo
f35b4ebe95 fix: Retry device name detection when bridge is not ready at startup
The background thread now retries with exponential backoff (5s→60s)
instead of giving up after 3 attempts. Also accepts detected device
name from bridge even when bridge health status is unhealthy.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 11:14:41 +01:00
MarekWo
ed8cab6dc5 feat: Add repeater commands to slow commands list
Added req_status, req_neighbours, and trace commands with 15s timeout
as they communicate with repeaters and need time for responses.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 17:18:07 +01:00
MarekWo
136bcd28e1 fix: Handle leading whitespace in JSON packet line filtering
Strip leading whitespace before checking if line starts with '{' to
ensure JSON packet lines are properly filtered regardless of indentation.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 15:55:05 +01:00
MarekWo
4709ecc771 fix: Suppress werkzeug WebSocket handshake errors in logs
The werkzeug development server produces "write() before start_response"
errors during WebSocket upgrade. These are cosmetic - the connection
still works via Socket.IO retry. Added a logging filter to suppress
these errors for cleaner logs.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 15:50:34 +01:00
MarekWo
251dbf0359 fix: Filter out JSON packet lines from console output
JSON lines with payload_typename (internal mesh protocol data like
CONTROL packets) are now filtered out from console output, showing
only the human-readable results.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 15:46:02 +01:00
MarekWo
7c78bef906 feat: Add branch info to version display and update checker
- version.py now captures and exports GIT_BRANCH
- Display branch badge next to version in menu (e.g., "2026.01.20+abc1234 [dev]")
- /api/version now returns branch field
- /api/check-update uses frozen branch instead of hardcoded "dev"
- Allows proper update checking for both dev and main branches

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-20 21:00:11 +01:00
MarekWo
7ca3f4d2dd feat: Add dynamic Git-based versioning system
- Add app/version.py module generating version from Git metadata
- Format: YYYY.MM.DD+<commit_hash> (e.g., 2025.01.18+576c8ca9)
- Add +dirty suffix for uncommitted changes (ignores .env, technotes/)
- Add /api/version endpoint for monitoring
- Display version in hamburger menu
- Add freeze mechanism for Docker builds

Deploy command updated:
git push && ssh ... "cd ~/mc-webui && git pull && python -m app.version freeze && docker compose up -d --build"

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-18 11:47:22 +01:00
MarekWo
c7163aa035 feat: Auto-detect device name from meshcli prompt
Bridge now detects device name from meshcli prompt ("DeviceName|*")
and exposes it via /health endpoint. mc-webui fetches this at startup
and uses RuntimeConfig for dynamic device name throughout the app.

Fallback chain: prompt detection → .infos command → MC_DEVICE_NAME env var

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-15 07:48:10 +01:00
MarekWo
672fb2d1ad fix: Add allow_unsafe_werkzeug for threading mode
Flask-SocketIO in threading mode requires explicit permission
to use Werkzeug dev server.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 14:49:45 +01:00
MarekWo
51f64be025 perf: Switch SocketIO from gevent to threading mode
gevent async_mode requires monkey-patching at startup and was causing
6-12s page load times (vs 1-2s before). Threading mode doesn't require
special setup and is sufficient for occasional Console commands.

- Change async_mode from 'gevent' to 'threading'
- Remove gevent/gevent-websocket from requirements (bridge has its own)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 14:44:43 +01:00
MarekWo
2412df1d01 fix: Strip leading whitespace from first output line
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 14:01:14 +01:00
MarekWo
1c33ea7b31 fix: Simpler regex to match any prompt line with |*
Previous regex was too specific with \[\d+\] pattern.
New pattern ^[^|]+\|\* matches any line starting with <name>|*

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 13:58:09 +01:00
MarekWo
dd8b174e20 fix: Clean console output and handle slow commands
1. Remove prompt lines (e.g., "MarWoj|*") from output
2. Remove echoed command from response
3. Strip leading/trailing whitespace
4. Longer timeout for slow commands: node_discover (15s),
   recv (60s), send/send_msg (15s)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 13:53:17 +01:00
MarekWo
30163173b0 fix: Use correct bridge API format (args list, not command string)
Bridge expects {"args": ["infos"], "timeout": 30}
Returns {"success": true, "stdout": "..."}

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 13:36:24 +01:00
MarekWo
024cbdd7f6 fix: Use MC_BRIDGE_URL config instead of non-existent host/port
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 13:33:02 +01:00
MarekWo
30f286a813 fix: Use socketio.emit with room=sid in background task
Background tasks lose socket context, so emit() doesn't work.
Fixed by capturing socket ID and using socketio.emit(room=sid).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 13:28:12 +01:00
MarekWo
c376ecff30 fix: Proxy console WebSocket through main app for HTTPS compatibility
Browser blocks mixed content (HTTPS page -> HTTP WebSocket on port 5001).
Solution: Route WebSocket through main Flask app which goes through
existing HTTPS reverse proxy.

- Add Flask-SocketIO to main mc-webui app
- WebSocket handler proxies commands to bridge via HTTP
- Remove port 5001 external exposure (no longer needed)
- Remove duplicate title from console header

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 13:18:56 +01:00
MarekWo
f5fedbc96c Feature: Add message archiving system with browse-by-date selector
Implements automatic daily archiving of messages to improve performance
and enable browsing historical chat by date.

Backend changes:
- Add APScheduler for daily archiving at midnight (00:00 UTC)
- Create app/archiver/manager.py with archive logic and scheduler
- Extend parser.py to read from archive files and filter by days
- Add archive configuration to config.py (MC_ARCHIVE_*)

API changes:
- Extend GET /api/messages with archive_date and days parameters
- Add GET /api/archives endpoint to list available archives
- Add POST /api/archive/trigger for manual archiving

Frontend changes:
- Add date selector dropdown in navbar for archive browsing
- Implement archive list loading and date selection
- Update formatTime() to show full dates in archive view
- Live view now shows only last 7 days (configurable)

Docker & Config:
- Add archive volume mount in docker-compose.yml
- Add MC_ARCHIVE_DIR, MC_ARCHIVE_ENABLED, MC_ARCHIVE_RETENTION_DAYS env vars
- Update .env.example with archive configuration section

Documentation:
- Update README.md with archive feature and usage instructions
- Update .claude/instructions.md with archive endpoints

Key features:
- Automatic daily archiving (midnight UTC)
- Live view filtered to last 7 days for better performance
- Browse historical messages by date via dropdown selector
- Archives stored as dated files: {device}.YYYY-MM-DD.msgs
- Original .msgs file never modified (safe, read-only approach)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-21 20:21:33 +01:00
MarekWo
cf456422e2 Phase 1: Backend basics - Complete Flask application with REST API
Implemented core backend functionality:
- Flask application structure with blueprints
- Configuration module loading from environment variables
- MeshCore CLI wrapper with subprocess execution and timeout handling
- Message parser for .msgs JSON Lines file format
- REST API endpoints (messages, status, sync, contacts cleanup)
- HTML views with Bootstrap 5 responsive design
- Frontend JavaScript with auto-refresh and live updates
- Custom CSS styling for chat interface

API Endpoints:
- GET  /api/messages - List messages with pagination
- POST /api/messages - Send message with optional reply-to
- GET  /api/status - Device connection status
- POST /api/sync - Trigger message sync
- POST /api/contacts/cleanup - Remove inactive contacts
- GET  /api/device/info - Device information

Features:
- Auto-refresh every 60s (configurable)
- Reply to messages with @[UserName] format
- Toast notifications for feedback
- Settings modal for contact management
- Responsive design (mobile-friendly)
- Message bubbles with sender, timestamp, SNR, hop count

Ready for testing on production server (192.168.131.80:5000)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-21 14:02:46 +01:00