Commit Graph

27 Commits

Author SHA1 Message Date
MarekWo
b9a9436271 feat: Enhance echo tracking with paths, incoming routes, and persistence
- Show repeater path codes in sent message echo badge (e.g., "2 (5e, d1)")
- Capture and display route path for incoming messages in message meta
- Persist all echo data to .echoes.jsonl (survives container restarts)
- Load echo data from disk on startup with 7-day retention and compaction
- Combine sent echo and incoming path data in single /echo_counts response

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 15:59:41 +01:00
MarekWo
33aad637a5 upg: Meshcore-cli upgrade to 1.3.21 2026-02-07 14:22:43 +01:00
MarekWo
a7af271d1f fix: Update meshcore-cli to version 1.3.16 2026-01-31 07:04:38 +01:00
MarekWo
07040dd6d0 feat: Add "Heard X repeats" echo tracking for sent messages
Track how many repeaters heard and forwarded sent channel messages,
similar to the Meshcore mobile app's "Heard X repeats" feature.

- Bridge: Detect GRP_TXT echoes from stdout, track unique paths
- Bridge: New /register_echo and /echo_counts API endpoints
- API: Register messages for echo tracking after send
- API: Merge echo counts into /api/messages response
- Frontend: Display green badge with broadcast icon next to Resend button
- CSS: Echo badge styling with dark mode support

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 20:57:10 +01:00
MarekWo
9740fe3746 fix: Wait for slow commands before declaring no output in console
Commands like node_discover that take 10-15 seconds without producing
any intermediate output were being marked as completed after 300ms of
silence, resulting in "(no output)" being shown to the user.

Now the monitor waits for at least 80% of the command timeout before
applying the silence detection, ensuring slow commands have time to
complete and return their results.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 15:35:40 +01:00
MarekWo
b2496b17b0 feat: Auto-detect serial port from /dev/serial/by-id/
When MC_SERIAL_PORT=auto, bridge scans /dev/serial/by-id/ and:
- Uses the device if exactly one found
- Fails with helpful message if multiple devices (list provided)
- Fails if no devices found

docker-compose.yml now uses device_cgroup_rules instead of explicit
device mapping, allowing auto-detection inside the container.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-15 15:18:07 +01:00
MarekWo
4814b43566 fix: Handle status prefixes in meshcli prompt detection
Meshcli outputs status lines like "Fetching channels ....DeviceName|*".
Updated detection to extract device name after dot sequences.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-15 12:16:07 +01:00
MarekWo
c7163aa035 feat: Auto-detect device name from meshcli prompt
Bridge now detects device name from meshcli prompt ("DeviceName|*")
and exposes it via /health endpoint. mc-webui fetches this at startup
and uses RuntimeConfig for dynamic device name throughout the app.

Fallback chain: prompt detection → .infos command → MC_DEVICE_NAME env var

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-15 07:48:10 +01:00
MarekWo
67724ed237 fix: Set COLUMNS/LINES env vars without TERM=dumb
Previous fix with TERM=dumb caused Public channel to disappear.
Only setting COLUMNS and LINES as fallback for os.get_terminal_size().

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 19:38:44 +01:00
MarekWo
85d42d9ffe fix: Set terminal env vars to prevent os.get_terminal_size() errors
meshcli calls os.get_terminal_size() which fails without a TTY.
Setting COLUMNS, LINES, and TERM=dumb provides fallback values.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 18:56:57 +01:00
MarekWo
a5f7fd59e6 feat: Add interactive meshcli console with WebSocket support
- Add Flask-SocketIO backend with gevent for real-time communication
- Create chat-style console UI showing only user's command responses
- WebSocket commands tracked separately from HTTP API (ws_ prefix)
- Console accessible from main menu as fullscreen modal
- Command history navigation with arrow keys
- Auto-reconnection on disconnect
- Update service worker cache for offline support

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 08:27:42 +01:00
MarekWo
bcc12fdd38 update: meshcore-cli updated to 1.3.15 2026-01-07 15:44:26 +01:00
MarekWo
fd2b9b1592 feat: Refactor pending contacts to JSON format with filtering and batch approval
- Change backend from 'pending_contacts' to '.pending_contacts' command
- Parse JSON response with enriched contact data (type, GPS, timestamps)
- Add type badges (CLI/REP/ROOM/SENS) with color coding
- Add Map button for contacts with GPS coordinates
- Add type filter (checkboxes, default: CLI only) and name search
- Add batch approval with confirmation modal
- Follow existing contacts UI pattern for consistency
- Mobile-first design with touch-friendly controls

Breaking change: /api/contacts/pending response format changed

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-03 09:56:56 +01:00
MarekWo
77c72ba62e feat: add contact management MVP (manual approval + pending)
Implements Contact Management UI with manual contact approval and pending contacts list.

**Backend changes (meshcore-bridge/bridge.py):**
- Remove forced manual_add_contacts on session init (was for testing only)
- Add _load_webui_settings() to read .webui_settings.json from MC_CONFIG_DIR
- Add /set_manual_add_contacts endpoint for persistent settings management
- Settings now persist across container restarts via .webui_settings.json

**Backend changes (app/meshcore/cli.py):**
- Add get_pending_contacts() - proxy to bridge /pending_contacts
- Add approve_pending_contact() - proxy to bridge /add_pending (always uses full public_key)
- Add get_device_settings() - read .webui_settings.json
- Add set_manual_add_contacts() - proxy to bridge /set_manual_add_contacts

**API changes (app/routes/api.py):**
- Add GET /api/contacts/pending - list pending contacts
- Add POST /api/contacts/pending/approve - approve contact by public_key
- Add GET /api/device/settings - get persistent settings
- Add POST /api/device/settings - update manual_add_contacts setting

**Frontend (app/routes/views.py, templates, js):**
- Add /contacts/manage route rendering contacts.html
- Add contacts.html template with mobile-first design
- Add contacts.js with settings toggle and pending list UI
- Add "Contact Management" menu item in base.html
- Features: manual approval toggle, pending list, approve/copy actions, toast notifications

**Documentation (README.md):**
- Add Contact Management section in Usage
- Add to Key Features list
- Add debugging instructions

**Key features:**
- Manual approval toggle (persists across restarts)
- Pending contacts list with name and public_key
- Approve button (always sends full public_key for compatibility)
- Copy full key button (clipboard API)
- Auto-refresh on page load
- Mobile-first responsive design
- Info badge when manual approval is disabled
- Toast notifications for user feedback

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-29 09:52:09 +01:00
MarekWo
2c8677b6e8 fix(bridge): Improve pending_contacts parser to filter JSON lines
Issue: Parser incorrectly matched JSON lines (adverts, GRP_TXT messages)
as pending contacts because they contained colons. This caused false
positives like {"raw_hex": "..."} being parsed as contact name.

Root cause: meshcli outputs JSON for received messages (due to
json_log_rx=on setting) during pending_contacts command execution.
Original parser only checked for ':' presence, matching any JSON.

Solution: Added robust filtering before parsing:
1. Skip empty lines
2. Skip JSON lines (starting with { or [)
3. Skip meshcli prompt lines (ending with |*)
4. Validate public_key contains only hex characters (0-9, a-f, A-F)

This ensures only valid "ContactName: <hex_pubkey>" format is parsed.

Example of filtered content:
- SKIP: {"raw_hex": "25bc...", "payload_typename": "GRP_TXT"}
- SKIP: MarWoj|*
- PARSE: Skyllancer: f9ef123abc456...

Testing: No false positives with JSON lines in output.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-29 08:29:59 +01:00
MarekWo
c0f6fd7dfc feat(bridge): Enable manual_add_contacts mode in session init
Enabled manual contact approval mode in meshcli session initialization.
This requires explicit approval for new contacts attempting to connect,
providing enhanced security and network access control.

Changes:
- Added 'set manual_add_contacts on' to _init_session_settings()
- Updates session init log message to include manual_add_contacts status
- Created comprehensive technical documentation (technotes/pending-contacts-api.md)

Benefits:
- DoS prevention - blocks flooding with fake contact requests
- Network privacy - control who can see your node
- Trust model - explicit approval for all new contacts
- Spam filtering - reject unwanted connection attempts

Technical notes document includes:
- Problem statement and solution overview
- API endpoint specifications and examples
- Testing procedures and expected workflows
- Future UI integration plans
- Security considerations and recommendations
- Meshcli command reference

When manual approval is enabled, new contacts appear in pending list
(accessible via GET /pending_contacts) and must be approved via
POST /add_pending before they can communicate with the node.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-29 08:20:49 +01:00
MarekWo
815adb5cfa feat(bridge): Add pending contacts management API endpoints
Added two new HTTP endpoints to meshcore-bridge for managing pending contacts
(contacts awaiting manual approval when manual_add_contacts mode is enabled):

New endpoints:
- GET /pending_contacts - List all pending contacts awaiting approval
  - Parses meshcli output format: "Name: <hex_public_key>"
  - Returns JSON array with {name, public_key} objects
  - Includes raw_stdout for debugging

- POST /add_pending - Approve and add a pending contact
  - Accepts JSON body: {"selector": "<name_or_pubkey>"}
  - Validates selector is non-empty string
  - Executes meshcli add_pending command via persistent session

Additional changes:
- Added curl to mc-webui Dockerfile for testing endpoints
- Updated README with Testing Bridge API section
- Included example curl commands and expected responses

Implementation notes:
- Uses existing MeshCLISession.execute_command() - no new processes
- Same persistent session and command queue architecture
- Consistent error handling with existing /cli endpoint

Enables future UI for manual contact approval workflow.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-29 08:09:35 +01:00
MarekWo
3a100e742d fix(bridge): Correct command name from msg_subscribe to msgs_subscribe 2025-12-28 17:14:09 +01:00
MarekWo
d720d6a263 fix(bridge): Replace polling with msg_subscribe for real-time messages
Critical fixes based on user feedback:

1. **Remove auto-recv polling (30s interval)**
   - Polling with 'recv' doesn't fetch NEW messages, only reads from .msgs
   - Wasteful - creates unnecessary command traffic

2. **Add msg_subscribe for real-time message reception**
   - meshcli's msg_subscribe enables automatic message events
   - Messages arrive via EventType.CHANNEL_MSG_RECV events
   - No polling needed - truly asynchronous

3. **Make TZ configurable via .env instead of hardcoded**
   - Changed docker-compose.yml: TZ=${TZ:-UTC}
   - Added TZ=Europe/Warsaw to .env.example
   - Users can now set their own timezone

4. **Remove unused shlex import**
   - Not needed after switching to manual double-quote wrapping

Technical details:
- msg_subscribe sends subscription command to meshcli at init
- meshcli then emits events when messages arrive
- Events trigger TTY errors (harmless - meshcli tries to print_above)
- Messages are still saved to .msgs file by meshcli core

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-28 17:10:33 +01:00
MarekWo
34c60515ad fix(bridge): Add auto-recv thread to sync messages every 30s
CRITICAL FIX: Messages stopped arriving after switching to persistent session.
Root cause: meshcli in interactive mode (Popen stdin/stdout) doesn't receive
messages automatically - requires explicit 'recv' command.

Changes:
- Add auto_recv_thread that calls 'recv' every 30 seconds in background
- This ensures continuous message synchronization without breaking /cli
- Messages are saved to .msgs file by meshcli as usual
- Add TZ=Europe/Warsaw to both containers for correct timestamps in logs

Technical details:
- auto-recv runs in separate thread via command queue (no blocking)
- First recv after 5s delay (let session initialize)
- Uses execute_command() internally, so respects command serialization
- Thread stops gracefully on shutdown_flag

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-28 16:59:32 +01:00
MarekWo
36badea5a2 fix(bridge): Use double quotes instead of shlex.quote for message args
Users reported single quotes appearing in sent messages.
shlex.quote() uses single quotes which meshcli treats literally.

Changes:
- Replace shlex.quote() with custom double-quote wrapping
- Only quote args containing spaces or special chars
- Escape internal double quotes with backslash
- Example: ['public', 'To jest test'] → 'public "To jest test"'

meshcli should strip double quotes but preserve content.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-28 15:55:42 +01:00
MarekWo
56b7c335e8 fix(bridge): Quote command arguments to preserve spaces in messages
Users reported messages with spaces were truncated to first word.
Root cause: ' '.join(args) didn't quote arguments with spaces.

Changes:
- Import shlex module
- Use shlex.quote() for each argument in execute_command()
- Example: ['public', 'To jest test'] → "public 'To jest test'"

This ensures meshcli receives multi-word messages correctly.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-28 15:52:06 +01:00
MarekWo
693b211a71 fix(bridge): Replace echo markers with timeout-based response detection
meshcli doesn't support 'echo' command, causing "Unknown command" errors.
Changed strategy from end-markers to timeout-based detection:

- Remove echo "___END_{cmd_id}___" markers
- Add _monitor_response_timeout() thread per command
- Track last_line_time timestamp for each response
- Complete command when no new lines received for 300ms
- Update _append_to_current_response() to update timestamps

This approach is more robust and doesn't depend on meshcli's command set.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-28 15:44:53 +01:00
MarekWo
34afa6908d feat(bridge): Implement persistent meshcli session with advert logging
Major refactor of meshcore-bridge to maintain a single long-lived meshcli
process instead of spawning new processes per request.

Changes:
- Add MeshCLISession class managing persistent subprocess.Popen session
- Implement thread-safe command queue with event-based synchronization
- Add stdout/stderr reader threads with JSON advert detection
- Log adverts automatically to {device_name}.adverts.jsonl with timestamp
- Add end-of-response markers (echo "___END_{cmd_id}___") for multiplexing
- Implement watchdog thread for auto-restart on meshcli crash
- Update /cli endpoint to delegate commands through persistent session
- Add MC_CONFIG_DIR and MC_DEVICE_NAME env vars to docker-compose.yml

Architecture benefits:
- No more USB port conflicts between concurrent requests
- Continuous advert logging without breaking /cli compatibility
- Better error recovery with automatic session restart
- Reduced overhead from process spawning

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-28 15:41:41 +01:00
MarekWo
ad4a7b3f49 build(bridge): Update meshcore-cli to version 1.3.12
Fix critical bug in private message logging that prevented proper DM tracking.

Previous version (1.3.11) logged sent private messages with incomplete data:
- Missing recipient information
- Only included sender name in 'name' field
- Format: {"type": "SENT_MSG", "name": "MarWoj", "text": "...", ...}

Updated version (1.3.12) includes both sender and recipient:
- Added 'recipient' field with recipient's device name
- Added 'sender' field with sender's name
- Format: {"type": "SENT_MSG", "recipient": "SP7UNR_tdeck", "sender": "MarWoj", "name": "SP7UNR_tdeck", ...}

This fix enables proper message tracking in the DM module, as it now correctly
identifies both parties in private message exchanges.

Fix requested from meshcore-cli maintainers and implemented in v1.3.12.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-27 18:52:26 +01:00
MarekWo
c19a5bac88 fix: Add request locking and caching to prevent USB conflicts
Root cause: Multiple concurrent meshcli calls were fighting for USB access,
causing "Protocol error" and 504 Gateway Timeouts.

Changes to meshcore-bridge:
- Add threading.Lock to serialize meshcli subprocess calls
- Prevent concurrent USB access that causes OSError [Errno 71]
- Reduce DEFAULT_TIMEOUT from 30s to 10s
- Add detailed logging for lock acquisition and release

Changes to main API:
- Implement 30s cache for get_channels() to reduce USB calls
- Cache invalidation after channel create/join/delete operations
- Use cached channels in /api/channels and /api/messages/updates
- Reduce HTTP timeout from 30s to 12s (10s bridge + 2s buffer)

Impact:
- Eliminates race conditions when page loads (multiple API calls)
- Prevents USB port conflicts and protocol errors
- Faster response times due to caching
- No need for manual USB resets after container restarts

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-23 22:02:57 +01:00
MarekWo
4608665e82 refactor: Implement 2-container architecture to solve USB stability issues
BREAKING CHANGE: Switched from single-container to multi-container setup

This commit introduces a meshcore-bridge service that isolates USB device
access from the main application, resolving persistent USB timeout and
deadlock issues in Docker + VM environments.

Changes:
- Add meshcore-bridge/ - Lightweight HTTP API wrapper for meshcli
  - Flask server exposes /cli endpoint (port 5001, internal only)
  - Exclusive USB device access via --device flag
  - Health check endpoint at /health

- Refactor app/meshcore/cli.py
  - Replace subprocess calls with HTTP requests to bridge
  - Add requests library dependency
  - Better error handling for bridge communication

- Update docker-compose.yml
  - Define meshcore-bridge and mc-webui services
  - Create meshcore-net Docker network
  - Add depends_on with health check condition
  - Bridge gets USB device, main app uses HTTP only

- Modify Dockerfile
  - Remove meshcore-cli installation from main app
  - Lighter image without gcc dependencies

- Update config.py
  - Add MC_BRIDGE_URL environment variable
  - Remove meshcli_command property (no longer needed)

- Update documentation (README.md, .claude/instructions.md)
  - Document 2-container architecture
  - Add troubleshooting section for bridge
  - Update prerequisites (no host meshcore-cli needed)
  - Add architecture diagram in project structure

Benefits:
 Solves USB device locking after container restarts
 Restartable main app without USB reset
 Better separation of concerns
 Easier debugging (isolated meshcli logs)
 No manual USB recovery scripts needed

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-23 08:39:10 +01:00