Files
meshcore-hub/docs/letsmesh.md
T
Louis King 76f3dfa7eb feat: raw packet capture, browse, and classification (v0.13.0)
Add a first-class Raw Packets feature that captures every inbound MeshCore
packet from the LetsMesh `packets` feed exactly as received, independent of
how the collector later classifies it.

Capture & storage
- New `RawPacket` model + migration (raw_packets table) with single and
  composite indexes for the dominant filter-then-sort-by-newest queries.
- Collector-side `RAW_PACKET_CAPTURE_ENABLED` flag (default off); capture hook
  reuses the decoder's per-hex cache (no second decode), one row per observer
  reception, never blocks event dispatch.
- Separate `RAW_PACKET_RETENTION_DAYS` (falls back to DATA_RETENTION_DAYS);
  cleanup runs regardless of capture so disabling drains the table. Raw-packet
  observers retained in the is_observer recompute union.

API
- `GET /packets` and `/packets/{id}` with rich filtering, role-aware Redis
  cache key, and channel-visibility redaction (restricted-channel packets are
  returned metadata-only, not hidden, so pagination counts stay stable).

Web
- `FEATURE_PACKETS` flag (default off). Responsive Packets page (table desktop,
  cards mobile) plus a Packet Detail page (breadcrumb nav, raw hex + decoded).
- Nav entry after Messages on all three surfaces; home.js reordered so Map
  precedes Members; new packets icon + colour.

Finer-grained classification
- Replace the single `letsmesh_packet` catch-all with per-payload-type event
  types (req, ack, encrypted_direct, encrypted_channel, grp_data, multipart,
  control, raw_custom, ...); letsmesh_packet kept only as the unresolved-type
  safety net.

Link from structured tables
- Add `packet_hash` to advertisements and messages (populated at ingest);
  exact `packet_hash` filter on /packets; cube-icon link on the Adverts and
  Messages lists -> /packets?packet_hash=<hash>, shown only when the feature is
  on and the row has a stored hash.

Docs/config: .env.example, docker-compose (collector + web), AGENTS.md,
SCHEMAS.md, docs/letsmesh.md, docs/upgrading.md (## v0.13.0), en/nl i18n, and a
plan/tasks doc under docs/plans/.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-12 22:40:31 +01:00

4.1 KiB

LetsMesh Packet Decoding

The collector subscribes to packets published by meshcore-packet-capture:

  • <prefix>/+/+/packets
  • <prefix>/+/+/status
  • <prefix>/+/+/internal

Normalization Behavior

  • status packets are stored as informational letsmesh_status events and are not mapped to advertisement rows.

  • Decoder payload type 4 is mapped to advertisement when node identity metadata is present.

  • Decoder payload type 11 (control discover response) is mapped to contact.

  • Decoder payload type 9 is mapped to trace_data.

  • Decoder payload type 8 is mapped to informational path_updated events.

  • Decoder payload type 1 can map to native response events (telemetry_response, battery, path_updated, status_response) when decrypted structured content is available.

  • packet_type=5 packets are mapped to channel_msg_recv.

  • packet_type=1, 2, and 7 packets are mapped to contact_msg_recv when decryptable text is available.

  • Packets that no structured handler claims are no longer all labelled letsmesh_packet. They are classified by their MeshCore payload type so the event_type is specific:

    Payload type event_type
    0x00 REQ req
    0x01 RESPONSE response
    0x02 TXT_MSG (undecryptable) encrypted_direct
    0x03 ACK ack
    0x04 ADVERT (no identity) advert
    0x05 GRP_TXT (unknown key) encrypted_channel
    0x06 GRP_DATA grp_data
    0x07 ANON_REQ anon_req
    0x08 PATH path
    0x09 TRACE trace
    0x0A MULTIPART multipart
    0x0B CONTROL control
    0x0F RAW_CUSTOM raw_custom

    letsmesh_packet is retained only as a safety net for packets whose payload type cannot be resolved. Reaching these fallbacks for TXT_MSG/GRP_TXT means the payload did not decrypt, hence the encrypted_* labels (decryptable ones become contact_msg_recv / channel_msg_recv).

Raw Packet Capture

  • When RAW_PACKET_CAPTURE_ENABLED is set (Compose derives it from FEATURE_PACKETS), every packet on the packets feed is also stored verbatim in the raw_packets table — one row per observer reception, independent of structured classification. The status and internal feeds carry no on-air raw hex and are not captured as raw packets.
  • Capture reuses the single decode the normalizer already performs (the decoder caches per raw hex), so it adds only an insert plus an observer upsert to the ingest path.
  • The /packets API and Packets page apply channel-visibility rules: channel-message packets on a channel above the viewer's role are returned metadata-only with the payload redacted (redacted=true, raw_hex/decoded nulled), not hidden. Non-channel and visible-channel packets are returned in full.

Channel Keys

  • For channel packets, if a channel key is available, a channel label is attached (for example Public or #test) for UI display.
  • In the messages feed and dashboard channel sections, known channel indexes are preferred for labels (17 -> Public, 217 -> #test) to avoid stale channel-name mismatches.
  • Additional channel names are loaded from the channels database table (managed via CLI, API, or seed YAML).
  • The collector keeps built-in keys for Public and #test, and merges any additional keys from enabled database channel rows.

Location and Messages

  • Decoder-advertisement packets with location metadata update node GPS (lat/lon) for map display.
  • This keeps advertisement listings focused on node advert traffic only, not observer status telemetry.
  • Packets without decryptable message text are kept as informational letsmesh_packet events and are not shown in the messages feed; when decode succeeds the decoded JSON is attached to those packet log events.
  • When decoder output includes a human sender (payload.decoded.decrypted.sender), message text is normalized to Name: Message before storage; receiver/observer names are never used as sender fallback.

Decoder Runtime

  • Docker runtime uses the native Python meshcoredecoder library (no external Node.js dependency).