Commit Graph

218 Commits

Author SHA1 Message Date
MarekWo
752c60f02d feat(v2): Import archive .msgs files + DB-based message history
Migration now imports all archive files (oldest first) in addition to the
live .msgs file, with deduplication. Archives endpoint and message history
now query SQLite by date instead of reading .msgs files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 13:05:33 +01:00
MarekWo
97a2014af2 feat(v2): Auto-migrate v1 .msgs data to SQLite on first startup
Reads the existing .msgs JSONL file and imports channel messages and DMs
into the v2 SQLite database. Runs automatically when device connects and
DB is empty. Handles sender parsing, pubkey resolution, and FK constraints.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 11:32:28 +01:00
MarekWo
64860ba178 fix(v2): Parse sender name from channel message text format
Channel messages from meshcore arrive as "SenderName: message text".
The library doesn't provide sender name separately. Now parsing it
from the text (split on first colon), matching v1 parser behavior.

Also:
- Look up DM sender names from mc.contacts instead of event payload
- Fix SNR field name (uppercase 'SNR' from meshcore library)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 11:10:13 +01:00
MarekWo
65eb44d0ff fix(v2): Use event.payload instead of event.data throughout
The meshcore Event class has 'payload' not 'data'. All event handlers
were silently getting empty dicts, causing:
- Channel messages showing 'Unknown' sender
- Channel info not returning name/secret
- Sent message event data being lost

Also normalizes channel_name/channel_secret keys from CHANNEL_INFO
events and converts secret bytes to hex string.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 11:01:42 +01:00
MarekWo
a7b9b74fa2 fix(v2): Use mc.commands for meshcore command methods
MeshCore library exposes command methods (get_channel, send_msg,
send_advert, etc.) on mc.commands, not directly on the MeshCore
instance. Updated all DeviceManager calls accordingly.

Fixes: channels not loading, message sending, advert, battery, etc.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 10:49:04 +01:00
MarekWo
1b142b26f2 fix(v2): Use Flask current_app for DeviceManager lookup in cli.py
The module-level 'from app.main import device_manager' was returning
None in Flask request context even though device_manager was set.
Now tries current_app.device_manager first (Flask app context),
falling back to module import for non-request contexts.

Fixes 500 errors on /api/contacts, /api/contacts/pending,
/api/contacts/detailed, and /api/channels endpoints.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 10:33:07 +01:00
MarekWo
adf17d2d54 fix(v2): Add connection retry logic and self_info null guard
- Add _connect_with_retry() with exponential backoff (10 attempts)
- Guard against self_info being None after meshcore library disconnects
  due to unresponsive device
- Prevents crash when device is busy (e.g. held by orphan container)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 10:23:05 +01:00
MarekWo
2e95bbf9b5 fix(v2): Serial port auto-detection and channel_messages query
- Add _detect_serial_port() to DeviceManager — resolves 'auto' to
  actual device via /dev/serial/by-id with common path fallbacks
- Make channel_idx optional in get_channel_messages() so status and
  channel-updates endpoints can query across all channels

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 10:15:44 +01:00
MarekWo
e98acf6afa feat(v2): Add pkt_payload to DMs, update watchdog for single container
- Add pkt_payload column to direct_messages table for stable packet
  hash generation and Analyzer URL linking
- Update insert_direct_message() and DeviceManager to store pkt_payload
- Add test for DM pkt_payload storage (43 tests pass)
- Update watchdog to monitor only mc-webui (meshcore-bridge removed)
- USB reset trigger now fires for mc-webui container failures

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 10:01:43 +01:00
MarekWo
df8e2d2218 feat(v2): Route API endpoints through Database, remove bridge
- Update api.py: messages, contacts, DM endpoints read from SQLite DB
- Add DB fallback paths for parser.py backward compatibility
- Replace bridge echo registration with DeviceManager event handling
- Update status endpoint to use db.get_stats()
- Update channel updates/DM updates endpoints for DB queries
- Delete channel messages via DB instead of parser
- Remove meshcore-bridge/ directory (no longer needed in v2)
- Remove MC_BRIDGE_URL from config

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 09:28:14 +01:00
MarekWo
badf67cf74 feat(v2): Rewrite main.py and cli.py for direct device communication
main.py: Initialize Database + DeviceManager in create_app(), replace
bridge-dependent startup code, simplified console command router.
cli.py: All functions now delegate to DeviceManager instead of HTTP
bridge calls. Same signatures preserved for api.py compatibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 07:23:59 +01:00
MarekWo
a8a0becb13 feat(v2): Complete DeviceManager with event handlers and commands
Event handlers: channel messages, DMs, ACKs, adverts, path updates,
new contacts, disconnection — all write to Database + emit SocketIO.
Command methods: send_channel_message, send_dm, get/delete contacts,
get/set/remove channels, send_advert, check_connection, battery,
manual_add_contacts, pending contacts approval.
Auto message fetching and initial contact sync on connect.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 07:21:25 +01:00
MarekWo
c9cf37e8d5 feat(v2): Add DeviceManager skeleton with connect/disconnect
Background thread runs meshcore async event loop. Supports both
serial and TCP transports. Flask routes bridge sync→async via
execute() method. Event subscriptions marked as TODO for Phase 1.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 07:01:16 +01:00
MarekWo
68b14434ca feat(v2): Add Database class with full CRUD and backup
Sync SQLite wrapper with WAL mode, connection-per-call thread safety.
Methods for: device info, contacts (upsert/get/delete/protect),
channels, channel messages, DMs, ACKs, echoes, paths, advertisements,
read status, FTS5 search, stats, cleanup, and sqlite3.backup().

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 06:59:39 +01:00
MarekWo
9f9b6e7ed7 feat(v2): Add SQLite schema with 10 tables, indexes and FTS5
Tables: device, contacts, channels, channel_messages, direct_messages,
acks, echoes, paths, advertisements, read_status.
Includes schema_version for migrations, FTS5 virtual tables with
auto-sync triggers for full-text search on messages.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 06:58:19 +01:00
MarekWo
ebfc3c9845 feat(v2): Add v2 config settings (DB, TCP, backup)
New environment variables for Phase 0:
- MC_DB_PATH: SQLite database location
- MC_TCP_HOST/MC_TCP_PORT: TCP transport (meshcore-proxy)
- MC_BACKUP_ENABLED/HOUR/RETENTION_DAYS: automated backup
- MC_AUTO_RECONNECT, MC_LOG_LEVEL: connection management

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 06:57:08 +01:00
MarekWo
8ab19582cd fix(dm): Fix unread markers by deduping retries in all endpoints
Extracted dedup_retry_messages() helper (300s window, was 120s which
was too tight) and applied it in three places:
- GET /api/dm/messages - already had inline dedup, now uses helper
- get_dm_conversations() - fixes last_message_timestamp inflation
- GET /api/dm/updates - was missing dedup entirely, counted retries
  as unread messages (root cause of persistent unread markers)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 10:34:39 +01:00
MarekWo
c2acbb4ba1 fix(dm): Wait for msg JSON response and fix unread markers
Monitor: For .msg commands, keep waiting for expected_ack in response
instead of completing on silence timeout. Waits up to half of cmd_timeout
(5s) for the JSON to arrive before giving up.

Unread: Apply 120s time-window dedup to outgoing messages in conversation
listing, so retry messages don't inflate last_message_timestamp and cause
permanent unread markers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 10:19:28 +01:00
MarekWo
49159a888c fix(dm): Continue retry on command timeout and dedup retry messages
- Don't abort retry loop when msg command fails or times out - the device
  may be temporarily busy (especially during flood mode)
- Add 120s time-window dedup for outgoing messages with same text+recipient
  to prevent duplicate messages in chat when retry acks aren't tracked

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 09:58:05 +01:00
MarekWo
c0f93029cd fix(dm): Fix auto-retry not triggering and increase retry limits
- Fix JSON parsing: msg command outputs multi-line pretty-printed JSON
  (indent=4), but parser tried line-by-line. Now tries full-text parse
  first, then line-by-line, then regex fallback.
- Change retry limits: 5 direct + 3 flood attempts (was 3 total)
- Separate max_attempts (direct) and max_flood parameters
- Add debug logging when ack extraction fails

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 21:39:58 +01:00
MarekWo
c37a7d3b23 feat(dm): Auto-retry for undelivered DM messages
Implement automatic retry for DM messages when ACK is not received,
similar to the MeshCore mobile app's Auto Retry feature. The bridge
monitors for ACK after each send and retries up to 3 times, switching
to flood routing after 2 failed direct attempts via reset_path.

- Bridge: background retry engine with configurable max_attempts,
  flood_after; retry group tracking to prevent duplicate messages
- Bridge: enhanced ACK status checks retry groups so delivery is
  detected even if only a retry attempt's ACK arrives
- Backend: filter retry SENT_MSG duplicates from message list
- Frontend: extended ACK polling window, auto-retry toggle in DM bar

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 21:23:32 +01:00
MarekWo
fd4818cfad fix(ui): Remove CSS rule that stacked channel buttons vertically
Remove nested @media (max-width: 400px) rule that forced btn-group
to flex-direction: column, causing buttons to stack on mobile.
Also remove now-unused .list-group-item small styles (channel keys
no longer shown).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 07:35:56 +01:00
MarekWo
71f292d843 feat(ui): Mark-all-read confirmation dialog and compact channel list
Add confirm() dialog before marking all messages as read, showing
list of unread channels with counts. Remove channel key/ID from
Manage Channels modal to save vertical space on mobile.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 07:27:35 +01:00
MarekWo
7a4f4d3161 feat(notifications): Channel mute toggle and mark-all-as-read bell button
Add ability to mute notifications for individual channels via Manage
Channels modal (bell/bell-slash toggle button). Muted channels are
excluded from unread badge counts, browser notifications, and app icon
badge. Bell icon click now marks all channels as read in bulk.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 22:00:40 +01:00
MarekWo
ad478a8d47 feat(ui): Add @me filter button, DM filter push-down, and DM FAB toggle
- Add person icon button in filter bar that inserts the current device
  name into the search field, for filtering own messages
- DM filter bar already benefits from the CSS sibling push-down rule
  added in previous commit (same class names used)
- Add collapsible FAB toggle to DM view, same pattern as channel chat

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 09:05:47 +01:00
MarekWo
6310c41934 feat(ui): FAB toggle, filter bar layout fix, and filter @mentions
- Add collapsible FAB container with chevron toggle button to
  temporarily hide floating action buttons that overlap messages
- Make filter bar push messages down instead of overlaying the first
  matched message (CSS sibling selector adds padding-top)
- Add @mentions autocomplete to filter search bar - typing @ shows
  contact list dropdown, selecting inserts plain name (not @[] format)
  so all messages from/mentioning that user are found

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 08:47:00 +01:00
MarekWo
9ad3435609 fix: Always use attempt=0 payload for analyzer URL computation
The attempt loop (0-3) for matching incoming echo paths left
computed_payload at attempt=3 when no match was found, producing
wrong analyzer hashes. Combined with 1-hour incoming_paths cleanup
in bridge (vs 7-day .echoes.jsonl retention), this caused older
messages to lose both path info and correct analyzer links.

Two fixes:
- Compute base_payload at attempt=0 upfront for analyzer URL
- Extend incoming_paths memory cleanup from 1h to 7 days

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 17:15:12 +01:00
MarekWo
6d50391ea8 fix: Decode GPS coordinates as int32/1e6, not float
MeshCore encodes lat/lon as little-endian signed int32 divided by 1e6,
not as IEEE 754 floats. This caused all map pins to show at (0,0).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 13:45:43 +01:00
MarekWo
587bc8cb9f fix: Validate GPS coordinates from advert payloads
Discard NaN, Infinity, and out-of-range lat/lon values from
struct.unpack to prevent JSON parse errors in browser.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 13:10:34 +01:00
MarekWo
247b11e1e9 feat: Enrich contacts cache with GPS coordinates and node type
- Extract lat/lon from advert payloads (struct unpack from binary)
- Store type_label and lat/lon in cache from device seed and adverts
- Show Map button for cache contacts with GPS coordinates
- Show colored type badge (CLI/REP/ROOM/SENS) for typed cache contacts
- Type filter now works for cache contacts with known type
- Change counter label from "known" to "cached"

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 12:40:42 +01:00
MarekWo
a5e767e5bf fix: Replace sort buttons with dropdown for mobile-friendly contact filters
Replaces two sort toggle buttons with a single <select> dropdown (e-commerce style)
so all 3 filter/sort controls fit on mobile screens.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 07:19:56 +01:00
MarekWo
de0108d6aa feat: Add persistent contacts cache for @mention autocomplete
Contacts cache accumulates all known node names from device contacts
and adverts into a JSONL file, so @mentions work even after contacts
are removed from the device. Background thread scans adverts every
45s and parses advert payloads to extract public keys and node names.

Existing Contacts page now shows merged view with "Cache" badge for
contacts not on device, plus source filter (All/On device/Cache only).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 17:13:36 +01:00
MarekWo
0a73556c78 fix: Use bi-clipboard-data icon for analyzer (bi-flask unavailable)
bi-flask was added in Bootstrap Icons 1.12+, but project uses 1.11.2.
Replace with bi-clipboard-data which is available and conveys
"data analysis".

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 08:24:43 +01:00
MarekWo
5a7a9476f8 feat: Always show analyzer link for incoming msgs + flask icon
Generate analyzer_url from computed pkt_payload for all incoming
channel messages, not just those with echo path matches. This means
the analyzer button appears even when no route paths were captured.

Also change analyzer button icon from bi-search (magnifying glass)
to bi-flask (lab flask) to better convey "analysis/inspection".

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 08:20:10 +01:00
MarekWo
68b2166445 fix: Use unstripped raw_text for pkt_payload computation
The parser's .strip() was removing trailing whitespace from message
text, but the encrypted radio payload includes those trailing spaces.
This caused pkt_payload mismatches for messages ending with spaces
(e.g., "Dzień dobry "). Use original unstripped text for raw_text.

Also add debug logging for unmatched messages to help diagnose
remaining edge cases.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 08:09:28 +01:00
MarekWo
28148d32d8 feat: Deterministic echo-to-message matching via pkt_payload computation
Replace unreliable timestamp-based heuristic (±10s window) with exact
cryptographic matching for incoming channel message routes. Compute
pkt_payload by reconstructing the AES-128-ECB encrypted packet from
message data (sender_timestamp, txt_type, text) + channel secret, then
match against echo data by exact key lookup.

Also accumulate ALL route paths per message (previously only last path
was kept due to dict overwrite), and display them in a multi-path popup
showing SNR and hops for each route.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 07:29:49 +01:00
MarekWo
2ed3dc3758 feat: Add unknown delivery status indicator + update docs
Add clickable "?" icon on DMs without ACK, showing a popup
explaining that delivery is unknown (mobile-friendly).
Update README, user guide with new features (Analyzer links,
DM delivery tracking).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 08:14:58 +01:00
MarekWo
235c74338d fix: Skip redundant DM refreshes once delivery ACK is confirmed
Stop scheduling further post-send reloads as soon as the last
own message shows a delivery checkmark.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 07:51:36 +01:00
MarekWo
cdd28e66fc fix: Auto-refresh DM view after send to show delivery status
Add two extra delayed reloads (6s, 15s) after sending a DM,
matching the channel chat pattern, so ACK checkmarks appear
without needing to send another message.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 09:39:14 +01:00
MarekWo
7a960f2556 feat: Add DM delivery tracking via ACK packet detection
Bridge captures ACK packets from meshcli stdout (json_log_rx),
persists to .acks.jsonl, and exposes /ack_status endpoint.
Delivery status is merged server-side into DM messages and
displayed as a green checkmark with SNR/route tooltip.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 09:30:33 +01:00
MarekWo
cf537628cf feat: Add MeshCore Analyzer link button to channel messages
Compute packet_hash from pkt_payload (SHA-256 of type byte + payload)
and generate analyzer.letsmesh.net links. Button appears on both sent
and received messages when echo data is available.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 08:26:43 +01:00
MarekWo
f35b4ebe95 fix: Retry device name detection when bridge is not ready at startup
The background thread now retries with exponential backoff (5s→60s)
instead of giving up after 3 attempts. Also accepts detected device
name from bridge even when bridge health status is unhealthy.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 11:14:41 +01:00
MarekWo
01fc9edf24 feat: Add tap-to-show path popup for mobile devices
Tooltips don't work on touchscreens. Added a popup that appears on
tap/click, shows the full path, and auto-dismisses after 4 seconds.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 16:41:33 +01:00
MarekWo
ddb69f1a18 fix: Deduplicate echo paths and improve incoming path matching
- Deduplicate 2-char repeater codes in echo badge (same repeater via
  different routes was shown twice, e.g., "3 (d1, 5e, 5e)")
- Use deduplicated count for unique repeaters, not unique full paths
- Improve incoming path correlation: widen window to 10s, prefer
  path_len match but fall back to timestamp-only if needed
- Add debug logging for incoming path correlation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 16:29:23 +01:00
MarekWo
b9a9436271 feat: Enhance echo tracking with paths, incoming routes, and persistence
- Show repeater path codes in sent message echo badge (e.g., "2 (5e, d1)")
- Capture and display route path for incoming messages in message meta
- Persist all echo data to .echoes.jsonl (survives container restarts)
- Load echo data from disk on startup with 7-day retention and compaction
- Combine sent echo and incoming path data in single /echo_counts response

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 15:59:41 +01:00
MarekWo
a822d94317 fix: Align sender name baseline with timestamp in own messages
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 21:13:36 +01:00
MarekWo
db6915f53f feat: Improve chat message display
- Add sender name above outgoing messages in group chat
- Display emoji-only messages in larger font size
- Fix leading space before text in quoted replies

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-06 20:42:27 +01:00
MarekWo
2c17c83253 feat: Add clickable #channel links in chat messages
- Auto-detect #channel names in messages and convert to clickable links
- Click existing channel: switch to it via channel selector
- Click non-existing channel: join via API, then switch
- Green styling to distinguish from blue @mentions
- Only active in channel context (not in DMs)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 08:54:50 +01:00
MarekWo
d37326c261 feat: Add chat filter functionality for channel and DM messages
- Add filter-utils.js with diacritic-insensitive search and text highlighting
- Add FAB filter button (gray funnel icon) to channel chat and DM
- Filter bar slides in as overlay at top of chat area
- Real-time filtering with debounce (150ms) as user types
- Matched text highlighted with yellow background
- Support for Polish characters (wol matches wół)
- Keyboard shortcuts: Ctrl+F to open, Escape to close
- Match counter shows X / Y filtered messages
- Filter persists during auto-refresh

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 21:29:44 +01:00
MarekWo
2bb2d02476 fix: Force connection close to prevent stale connections in scheduler
Added 'Connection: close' header to bridge requests to prevent
connection pooling issues when running from APScheduler background
thread context.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 12:28:41 +01:00