Commit Graph

284 Commits

Author SHA1 Message Date
MarekWo 3a26da18fd fix(websocket): update message metadata in-place without full chat reload
Instead of reloading the entire message list when echo data arrives,
now updates only the affected message elements in the DOM:

- Add data-msg-id attribute to message wrappers for targeted lookup
- Add GET /api/messages/<id>/meta endpoint returning metadata for a
  single message (computes pkt_payload, looks up echoes, analyzer URL)
- Replace loadMessages() echo handler with refreshMessagesMeta() that
  finds messages missing metadata and updates them individually
- Fix path_len=0 treated as falsy (use ?? instead of ||)

Flow: message appears instantly via WebSocket (with SNR + hops), then
~2s later echo data triggers targeted meta fetch → route info and
analyzer button appear smoothly without any chat window reload.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 12:49:56 +01:00
MarekWo 0e15df430f fix(websocket): listen for echo events to update message metadata in real-time
The backend already emits 'echo' SocketIO events when RX_LOG_DATA arrives
with route/path data, but the frontend wasn't listening. Now the frontend
handles echo events with a debounced loadMessages() refresh (2s delay) to
pick up computed pkt_payload, analyzer_url, hops, and route info.

This fixes messages appearing without metadata until manual page refresh.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 12:38:46 +01:00
MarekWo e817181261 fix(websocket): include SNR, hops, route and analyzer URL in channel message events
The SocketIO new_message emit for channel messages was missing snr,
path_len, pkt_payload and analyzer_url fields, causing messages received
via WebSocket to render without metadata until a full page refresh.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 12:12:41 +01:00
MarekWo 9a0d05ae93 fix(search): remove double hash in channel names, add FTS5 syntax help
Channel names from device already include # prefix — removed hardcoded #
from search results badge. Added (?) help button with search syntax
examples and link to FTS5 docs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 07:38:37 +01:00
MarekWo d74a1572bb feat(dm): increase retry attempts to 10, delay flood fallback to attempt 8
Keep direct path retries longer before falling back to flood mode,
giving more time for ACK delivery on known routes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 21:26:58 +01:00
MarekWo 6fcbcb7d4f fix(ui): fix [object Object] in device info/stats, soften search button color
- Stats: battery fallback used ${bat} on an object — now uses battery_mv
  from core stats when dedicated get_bat returns null
- Info: remove old v1 regex JSON parsing, use v2 dict response directly
- Search FAB: change from bright orange to muted teal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 20:14:25 +01:00
MarekWo 65b33b4af6 fix(phase3): fix database/device access in search, backup, stats, map endpoints
- Search, backup, stats endpoints used current_app.config.get('DEVICE_MANAGER')
  which doesn't exist — replaced with _get_dm()/_get_db() helpers
- /api/device/info used old v1 CLI — replaced with DeviceManager.get_device_info()
  returning structured dict instead of string (fixes map own device marker)
- Moved search button from navbar to FAB menu (between filter and DM buttons)
- Bump SW cache to v7

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 20:02:23 +01:00
MarekWo e4a1e75cc0 fix(socketio): register /chat namespace handler to fix real-time message delivery
The /chat namespace had no server-side connect handler registered. With
python-socketio 5.x (always_connect=False), client connections to
unregistered namespaces are silently rejected. This caused all SocketIO
events (new_message, ack, echo) to never reach the frontend — messages
only appeared via the 60s polling fallback.

Fixes:
- Add @socketio.on('connect', namespace='/chat') handler in main.py
- Add optimistic message append: sent messages appear instantly before
  API round-trip (eliminates 3-4s serial command delay)
- Skip own-message SocketIO events to prevent duplicates
- Add connect_error handler for frontend debugging
- Bump SW cache to v6

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 08:15:43 +01:00
MarekWo c6a2444249 fix(backup): allow backup job scheduling before db reference is set
The _db reference is set by init_retention_schedule() which runs after
schedule_daily_archiving(). The backup job checks _db at runtime.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 07:29:30 +01:00
MarekWo 3f9d096ed0 chore(sw): bump cache to v5, add filter-utils.js to cached assets
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 07:28:02 +01:00
MarekWo ec383bf8e9 feat(map): add own device marker, last seen info, and formatTimeAgo
- Own device shown as red star marker on map (from self_info GPS)
- Contact popups now show "Last seen: X min ago" from last_advert
- New formatTimeAgo() utility for relative timestamps

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 07:27:37 +01:00
MarekWo ab01e6f17a feat(stats): add device statistics dashboard with Info/Stats tabs
- New GET /api/device/stats endpoint (core, radio, packets, DB stats)
- Device Info modal now has Info and Stats tabs
- Stats tab shows: battery, uptime, TX/RX air time, packet counts,
  DB row counts, and database size

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 07:25:45 +01:00
MarekWo 6fba37c609 feat(console): add stats, telemetry, neighbors, trace commands
- stats: device uptime, TX/RX air time, packet counts, errors
- telemetry <name>: request sensor data from remote node
- neighbors <name>: list neighbors of a remote node
- trace [tag]: send trace packet for mesh topology discovery

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 07:24:08 +01:00
MarekWo 4ecab9b307 feat(backup): add backup API endpoints and UI
- POST /api/backup/create — trigger immediate backup
- GET /api/backup/list — list backups with sizes
- GET /api/backup/download — download backup file
- Backup modal accessible from menu with create/download buttons
- Daily automatic backup via APScheduler (configurable hour/retention)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 07:22:50 +01:00
MarekWo d6e2a3472a feat(search): add global message search with FTS5 backend
- New GET /api/messages/search endpoint using existing FTS5 indexes
- Search modal accessible from navbar search icon
- Debounced search (300ms) across all channel and DM messages
- Results show source (channel/DM), sender, timestamp with highlights
- Click result navigates to the relevant channel or DM conversation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 07:20:04 +01:00
MarekWo a501da914a fix(chat): replace triple reload with SocketIO append after sending message
Instead of reloading all messages 3 times (1s, 6s, 15s) after sending,
the sent message now appears instantly via SocketIO new_message event.
Only one deferred reload remains at 15s to pick up echo data.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 07:17:39 +01:00
MarekWo 92b55d9bdb feat(map): add Cached switch to show cache-only contacts on map
The Map modal (from main menu) now has a "Cached" toggle switch that,
when enabled, also displays contacts stored in the DB cache alongside
device contacts. Cached markers are slightly smaller and more
transparent to visually distinguish them from device contacts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 07:01:53 +01:00
MarekWo 3fb1c09dc1 fix(chat): exclude blocked contacts from unread message counts
Both /api/messages/updates (channel) and /api/dm/updates endpoints
now filter out blocked contacts when computing unread counts, so
badge numbers no longer include messages from blocked users.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 06:45:38 +01:00
MarekWo 82b55d450e fix(chat): use showNotification instead of showToast in chat actions
showToast is only defined in contacts.js, not app.js. The chat page
uses showNotification. The ReferenceError was silently caught, preventing
loadBlockedNames() and loadMessages() from executing after blocking.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 21:31:58 +01:00
MarekWo e1ceff3a65 fix(chat): refresh messages after blocking contact from chat
Added client-side blocked name filtering in displayMessages() as
defense-in-depth alongside server-side filtering. This ensures blocked
sender messages are hidden immediately after blocking from chat.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 21:27:48 +01:00
MarekWo 833d01df9f fix(contacts): remove duplicate const declaration breaking contacts.js
applySortAndFilters() had duplicate const declarations for sourceFilter
and selectedSource, causing a SyntaxError that prevented the entire
contacts.js from loading. Both Pending and Existing pages were broken.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 21:14:10 +01:00
MarekWo b0076c3739 feat(contacts): name-based blocking, fix CSS breakpoint
- New blocked_names table for blocking bots without known public_key
- get_blocked_contact_names() returns union of pubkey-blocked + name-blocked
- POST /api/contacts/block-name endpoint for name-based blocking
- GET /api/contacts/blocked-names-list for management UI
- Block button always visible in chat (falls back to name-based block)
- Blocked Names section shown in Existing Contacts Blocked filter
- CSS breakpoint for icon-only buttons: 768px → 428px (iPhone-sized)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 21:03:19 +01:00
MarekWo 0d5c021e40 fix(contacts): type label bug, responsive buttons, remove Copy Key
- Fix pending contacts always showing "CLI" — compute type_label in API
- Remove Copy Key button from pending cards, make key clickable instead
- Responsive contact buttons: icon+text on desktop, icon-only on <=768px
- Add flex-wrap for button rows on small screens

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 07:16:04 +01:00
MarekWo 2a3a48ed5f feat(contacts): add ignored and blocked contact lists
- New DB tables: ignored_contacts, blocked_contacts (keyed by pubkey)
- Ignored contacts: cached but excluded from pending/auto-add
- Blocked contacts: ignored + messages hidden from chat (stored in DB)
- Backend: filter in _on_new_contact, _on_channel_message, _on_dm_received
- API: /contacts/<pk>/ignore, /contacts/<pk>/block toggle endpoints
- API: filter blocked from /api/messages and /dm/conversations
- Frontend: Ignore/Block buttons on pending cards, existing cards, chat messages
- Frontend: source filter dropdown with Ignored/Blocked options
- Frontend: status icons (eye-slash, slash-circle) on contact cards
- Frontend: real-time blocked message filtering via socketio
- Name→pubkey mapping for chat window block/ignore buttons

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 21:10:21 +01:00
MarekWo b709cc7b14 feat(contacts): add device/cache source icon to contact cards
Green chip icon for device contacts, grey cloud icon for cache-only.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 20:03:46 +01:00
MarekWo 09fbc56956 feat(contacts): complete cache functionality, fix display bugs
- _on_new_contact() in manual mode: upsert to DB as cache (source='advert')
  so contacts appear in @mentions and Cache filter before approval
- _on_advertisement(): check mc.pending_contacts for name/metadata fallback
- get_pending_contacts(): include last_advert in response
- /api/contacts/cached: return numeric last_advert timestamp
- contacts.js: fix adv_lat/adv_lon field names (was c.lat/c.lon),
  use last_advert timestamp instead of last_seen datetime string
- upsert_contact: source priority — never downgrade 'device' to 'advert'

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 08:51:25 +01:00
MarekWo 34b6e9b1ec fix(contacts): fallback add to mc.contacts after approve
ensure_contacts(follow=True) may not return the just-added contact.
Add manual fallback to mc.contacts so /api/contacts/detailed and
send_dm can find it immediately.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 13:37:51 +01:00
MarekWo 6c34ce85d8 fix(contacts): sync device↔DB contacts, restore contact cache
- get_contacts_with_last_seen() reads from mc.contacts (device firmware)
  instead of DB, so /api/contacts/detailed returns only device contacts
- _sync_contacts_to_db() now bidirectional: downgrades stale 'device'
  contacts to 'advert' (cache-only) when not on device anymore
- delete_contact() sets source='advert' (cache) instead of 'deleted',
  keeping contacts visible in @mentions and cache filter
- get_contacts() returns all contacts (no 'deleted' filter needed)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 13:10:13 +01:00
MarekWo 53928390c8 fix(contacts): soft-delete contacts to preserve DM history
- delete_contact() now sets source='deleted' instead of SQL DELETE
- get_contacts() filters out deleted contacts (hidden from UI)
- upsert_contact() on re-add overwrites source, auto-undeleting
- DM FK references stay intact, no more orphaned messages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 08:29:44 +01:00
MarekWo 66fa261151 fix(dm): prevent orphaned DMs on contact deletion, improve relinking
- Stop deleting contacts from DB on device removal (preserves DM history)
- Filter NULL contact_pubkey from DM conversations list
- Match outgoing DMs by contact name in raw_json during relinking

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 08:09:24 +01:00
MarekWo d1ce3ceb92 fix(dm): refresh mc.contacts on approve, DB name fallback, relink orphans
Three fixes for DM sending after contact delete/re-add:

1. approve_contact() now calls ensure_contacts() to refresh mc.contacts
   so send_dm can find newly added contacts immediately

2. cli.send_dm() falls back to DB name lookup when mc.contacts misses,
   preventing the contact name from being passed as a pubkey string

3. approve_contact() re-links orphaned DMs (NULL contact_pubkey from
   ON DELETE SET NULL) back to the re-added contact

New DB methods: get_contact_by_name(), relink_orphaned_dms()

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 07:41:15 +01:00
MarekWo 1a3a1e937c fix(dm): clear error when contact not on device, log error_code detail
Device firmware requires contacts in its table to send DMs. Passing
a raw pubkey string results in error_code=2. Show actionable error
message instead of generic "Device error".

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 07:32:56 +01:00
MarekWo 6a5fe98e32 debug(dm): add detailed logging for device send errors
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 07:31:28 +01:00
MarekWo 94f1bd98de fix(dm): don't require contact in mc.contacts to send DM
When the contact is not found in the in-memory mc.contacts dict,
fall back to passing the pubkey hex string directly to send_msg()
which handles it natively. Fixes "Contact not found" error.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 07:26:24 +01:00
MarekWo 8f31c27360 fix(dm): resolve short pubkey prefix to full key on incoming DM
When a DM arrives with only pubkey_prefix (short hex) and the sender
is not in mc.contacts, fall back to DB prefix lookup to get the full
64-char public key. Prevents ghost contact entries and "Unknown" DM
conversations.

Also adds get_contact_by_prefix() database helper.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 07:19:46 +01:00
MarekWo 5b757e9548 feat(dm): add delivery confirmation, retry, and receiver-side dedup
- Fix ACK handler bug: read 'code' field instead of 'expected_ack'
- Add DM retry (up to 3 attempts) with same timestamp for receiver dedup
- Add receiver-side dedup in _on_dm_received() (sender_timestamp or time-window)
- Add PATH_UPDATE as backup delivery signal for flood DMs
- Track pending acks with dm_id for proper ACK→DM linkage
- Return dm_id and expected_ack from POST /dm/messages API
- Add find_dm_duplicate() and get_dm_by_id() database helpers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 07:02:58 +01:00
MarekWo dc8c7ad1d6 fix(channels): pre-allocate reader.channels to prevent lib corruption
meshcore lib 2.2.21 bug: reader.py line 434 does
  self.channels = self.channels.extend([...])
which sets self.channels = None (extend returns None).
This corrupts ALL subsequent channel message processing.

Fix: after getting max_channels from device_info, pre-allocate
mc._reader.channels to max_channels slots so the extend path
is never triggered.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:05:17 +01:00
MarekWo 8821892b4c fix(channels): stop channel iteration on consecutive empty slots
meshcore lib 2.2.21 has a bug where list.extend() return value (None)
is assigned to self.channels, corrupting state for indices >= 20.
Stop iterating after 3 consecutive empty/failed channels to avoid
hitting this bug and reduce error log spam.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:02:43 +01:00
MarekWo 97323649c7 feat(tcp): document TCP connection in .env.example + fix defaults
- Add MC_TCP_HOST and MC_TCP_PORT options to .env.example with clear
  documentation for serial vs TCP transport selection
- Change default MC_TCP_PORT from 5000 to 5555 (avoids Flask conflict)
- Wire MC_TCP_HOST/MC_TCP_PORT through docker-compose.yml from .env
  (previously was commented-out hardcoded values)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 07:27:08 +01:00
MarekWo d6a7354f06 fix(channels): use device-reported max_channels instead of hardcoded 8
Firmware reports MAX_GROUP_CHANNELS (typically 40 for companion builds)
in the DEVICE_INFO response. Fetch it at startup and use it in all
channel iteration loops. Previously hardcoded range(8) prevented
channels 8+ from appearing and blocked adding new channels.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 20:38:15 +01:00
MarekWo 9b206beeac fix(echo): validate channel hash before correlating echo with sent message
Pending echo correlation was assigning ANY first echo to the sent message,
even if it came from a different channel. This caused cross-channel mismatches
(e.g., Public channel echo assigned to #krakow message).

Fix: check that pkt_payload's first byte (channel_hash = sha256(secret)[0])
matches the channel we sent on before accepting correlation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 20:18:32 +01:00
MarekWo ac1667bd01 fix(ui): fall back to echo SNR when message SNR is missing
meshcore library doesn't always provide SNR in CHANNEL_MSG_RECV events.
Use the first echo path's SNR as fallback for inline display.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 17:25:25 +01:00
MarekWo ba990b155f fix(ui): fix leading pipe separator when SNR is missing from message
Use array-based metaParts.join(' | ') instead of string concatenation
to avoid ugly leading "| Hops: 0" when meshcore lib doesn't provide SNR.
Also revert temporary INFO-level debug logging back to DEBUG.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 17:19:09 +01:00
MarekWo 7bcd6bd216 debug: add INFO logging for RX_LOG_DATA events and subscriptions 2026-03-02 16:56:43 +01:00
MarekWo 9f249a4521 feat(echoes): add RX_LOG_DATA echo tracking + sent message pkt_payload correlation
- Subscribe to RX_LOG_DATA events to capture repeated radio packets
- Parse GRP_TXT (0x05) payload to extract pkt_payload and path
- Classify echoes as sent (pending echo correlation) or incoming
- Register pending echo when sending channel messages for pkt_payload capture
- Add update_message_pkt_payload() DB method for sent message correlation
- Return echo_paths/echo_snrs for ALL messages (not just own) in GET /messages
- Frontend: build paths from echo_paths for incoming message route display
- Emit SocketIO 'echo' event for real-time badge updates

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 16:53:12 +01:00
MarekWo 44832ada5e fix(analyzer): fix pkt_payload computation for block-aligned text + trailing whitespace
Two bugs in Analyzer URL generation:
1. Firmware omits null+padding when header+text exactly fills AES block boundary
   (len % 16 == 0), but our code always added \0+padding → wrong MAC → wrong hash
2. content.strip() in event handler removed trailing whitespace that was part of
   the original packet. Now uses raw_text from raw_json (preserves original text)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 15:53:59 +01:00
MarekWo 2a3a00e654 fix(contacts): refresh contacts on unknown advert + cleanup fixes
When an ADVERTISEMENT arrives for a pubkey not in mc.contacts, the
firmware has auto-added a new contact. Trigger ensure_contacts() to
refresh the contact list and get the name. Also: remove contacts from
mc.contacts cache on delete, add reject/clear pending API endpoints,
promote advert logging to INFO level.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 08:30:07 +01:00
MarekWo 37694dde09 fix(contacts): remove from pending list after approve + add reject/clear
approve_contact now removes the contact from mc.pending_contacts after
successful approval. Added reject_contact (remove without adding) and
clear_pending_contacts methods with API endpoints.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 08:14:10 +01:00
MarekWo 2c547ee1fc fix(perf): disable auto_update_contacts to prevent serial blocking
auto_update_contacts=True triggers ensure_contacts() on every
ADVERTISEMENT event, fetching 324+ contacts over serial (several seconds).
This blocks the serial port and delays MESSAGES_WAITING processing,
causing 10-30s message reception delays. Contacts are synced at startup
and updated individually via NEW_CONTACT events.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 07:48:08 +01:00
MarekWo 63f2473933 fix(contacts): respect manual approval setting in NEW_CONTACT handler
When manual approval is enabled in settings, _on_new_contact now skips
DB upsert and emits SocketIO event for pending contacts. When auto mode
is on, contacts are added to DB immediately as before. Also enriched
approve_contact and get_pending_contacts with full contact data.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 07:45:44 +01:00