Files
meshcore-hub/SCHEMAS.md
T
Louis King 76f3dfa7eb feat: raw packet capture, browse, and classification (v0.13.0)
Add a first-class Raw Packets feature that captures every inbound MeshCore
packet from the LetsMesh `packets` feed exactly as received, independent of
how the collector later classifies it.

Capture & storage
- New `RawPacket` model + migration (raw_packets table) with single and
  composite indexes for the dominant filter-then-sort-by-newest queries.
- Collector-side `RAW_PACKET_CAPTURE_ENABLED` flag (default off); capture hook
  reuses the decoder's per-hex cache (no second decode), one row per observer
  reception, never blocks event dispatch.
- Separate `RAW_PACKET_RETENTION_DAYS` (falls back to DATA_RETENTION_DAYS);
  cleanup runs regardless of capture so disabling drains the table. Raw-packet
  observers retained in the is_observer recompute union.

API
- `GET /packets` and `/packets/{id}` with rich filtering, role-aware Redis
  cache key, and channel-visibility redaction (restricted-channel packets are
  returned metadata-only, not hidden, so pagination counts stay stable).

Web
- `FEATURE_PACKETS` flag (default off). Responsive Packets page (table desktop,
  cards mobile) plus a Packet Detail page (breadcrumb nav, raw hex + decoded).
- Nav entry after Messages on all three surfaces; home.js reordered so Map
  precedes Members; new packets icon + colour.

Finer-grained classification
- Replace the single `letsmesh_packet` catch-all with per-payload-type event
  types (req, ack, encrypted_direct, encrypted_channel, grp_data, multipart,
  control, raw_custom, ...); letsmesh_packet kept only as the unresolved-type
  safety net.

Link from structured tables
- Add `packet_hash` to advertisements and messages (populated at ingest);
  exact `packet_hash` filter on /packets; cube-icon link on the Adverts and
  Messages lists -> /packets?packet_hash=<hash>, shown only when the feature is
  on and the row has a stored hash.

Docs/config: .env.example, docker-compose (collector + web), AGENTS.md,
SCHEMAS.md, docs/letsmesh.md, docs/upgrading.md (## v0.13.0), en/nl i18n, and a
plan/tasks doc under docs/plans/.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-12 22:40:31 +01:00

20 KiB

MeshCore Event Schemas

This document defines the complete JSON payload schemas for all MeshCore events supported by the API.

Event Categories

Events are categorized by how they're handled:

  • Persisted Events: Stored in database tables and available via REST API
  • Webhook Events: Trigger HTTP POST notifications when configured
  • Informational Events: Logged but not persisted to separate tables

Table of Contents


Persisted & Webhook Events

These events are both stored in the database and trigger webhooks when configured.

ADVERTISEMENT / NEW_ADVERT

Node advertisements announcing presence and metadata.

Database Table: advertisements

Payload Schema:

{
  "public_key": "string (64 hex chars)",
  "name": "string (optional)",
  "adv_type": "string (optional)",
  "flags": "integer (optional)",
  "lat": "number (optional)",
  "lon": "number (optional)",
  "route_type": "string (optional)",
  "advert_timestamp": "integer (optional)"
}

Field Descriptions:

  • public_key: Node's full 64-character hexadecimal public key (required)
  • name: Node name/alias (e.g., "Gateway-01", "Alice")
  • adv_type: Node type - common values: "chat", "repeater", "room", "companion" (other values may appear from upstream feeds and are normalized by the collector when possible)
  • flags: Node capability/status flags (bitmask)
  • lat: GPS latitude when provided by decoder metadata
  • lon: GPS longitude when provided by decoder metadata
  • route_type: Route type of the advertisement packet — "flood" (original flood), "transport_flood" (relayed flood), "direct" (zero-hop local), "transport_direct" (relayed direct). Only present for LetsMesh-decoded adverts; native mode adverts have route_type=NULL.
  • advert_timestamp: Node's own Unix timestamp (uint32) from the decoded advert payload. Used for deduplication when within ±4 hours of received_at. May be NULL for native mode adverts or when the decoder does not provide a timestamp.

Example:

{
  "public_key": "4767c2897c256df8d85a5fa090574284bfd15b92d47359741b0abd5098ed30c4",
  "name": "Gateway-01",
  "adv_type": "repeater",
  "flags": 218,
  "lat": 42.470001,
  "lon": -71.330001,
  "route_type": "flood",
  "advert_timestamp": 1747300000
}

Webhook Trigger: Yes REST API: GET /api/v1/advertisements


CONTACT_MSG_RECV

Direct/private messages between two nodes.

Database Table: messages

Payload Schema:

{
  "pubkey_prefix": "string (12 chars)",
  "path_len": "integer (optional)",
  "txt_type": "integer (optional)",
  "signature": "string (optional)",
  "text": "string",
  "SNR": "number (optional)",
  "sender_timestamp": "integer (optional)"
}

Field Descriptions:

  • pubkey_prefix: First 12 characters of the source public key prefix, used for message identification (or source hash prefix in compatibility ingest modes)
  • path_len: Number of hops message traveled
  • txt_type: Message type indicator (0=plain, 2=signed, etc.)
  • signature: Message signature (8 hex chars) when txt_type=2
  • text: Message content (required)
  • SNR: Signal-to-Noise Ratio in dB
  • sender_timestamp: Unix timestamp when message was sent

Example:

{
  "pubkey_prefix": "01ab2186c4d5",
  "path_len": 3,
  "txt_type": 0,
  "signature": null,
  "text": "Hello Bob!",
  "SNR": 15.5,
  "sender_timestamp": 1732820498
}

Webhook Trigger: Yes REST API: GET /api/v1/messages Webhook JSONPath Examples:

  • Send only text: $.data.text
  • Send text + SNR: $.data.[text,SNR]

CHANNEL_MSG_RECV

Group/broadcast messages on specific channels.

Database Table: messages

Payload Schema:

{
  "channel_idx": "integer (optional)",
  "channel_name": "string (optional)",
  "pubkey_prefix": "string (12 chars, optional)",
  "path_len": "integer (optional)",
  "txt_type": "integer (optional)",
  "signature": "string (optional)",
  "text": "string",
  "SNR": "number (optional)",
  "sender_timestamp": "integer (optional)"
}

Field Descriptions:

  • channel_idx: Channel number (0-255) when available
  • channel_name: Channel display label (e.g., "Public", "Community", "#test") when available
  • pubkey_prefix: First 12 characters of the source public key prefix, used for message identification when available
  • path_len: Number of hops message traveled
  • txt_type: Message type indicator (0=plain, 2=signed, etc.)
  • signature: Message signature (8 hex chars) when txt_type=2
  • text: Message content (required)
  • SNR: Signal-to-Noise Ratio in dB
  • sender_timestamp: Unix timestamp when message was sent

Example:

{
  "channel_idx": 4,
  "path_len": 10,
  "txt_type": 0,
  "signature": null,
  "text": "Hello from the mesh!",
  "SNR": 8.5,
  "sender_timestamp": 1732820498
}

Webhook Trigger: Yes REST API: GET /api/v1/messages Webhook JSONPath Examples:

  • Send only text: $.data.text
  • Send channel + text: $.data.[channel_idx,text]

Compatibility ingest note:

  • In LetsMesh upload compatibility mode, packet type 5 is normalized to CHANNEL_MSG_RECV and packet types 1, 2, and 7 are normalized to CONTACT_MSG_RECV when decryptable text is available.
  • LetsMesh packets without decryptable message text are treated as informational letsmesh_packet events instead of message events.
  • For UI labels, known channel indexes are mapped (17 -> Public, 217 -> #test) and preferred over ambiguous/stale channel-name hints.
  • Additional channel labels are loaded from the channels database table via the collector's periodic refresh.
  • When decoder output includes a human sender (payload.decoded.decrypted.sender), message text is normalized to Name: Message; sender identity remains unknown when only hash/prefix metadata is available.

Compatibility ingest note (advertisements):

  • In LetsMesh upload compatibility mode, status feed payloads are persisted as informational letsmesh_status events and are not normalized to ADVERTISEMENT.
  • In LetsMesh upload compatibility mode, decoded payload type 4 is normalized to ADVERTISEMENT when node identity metadata is present.
  • Payload type 4 location metadata (appData.location.latitude/longitude) is mapped to node lat/lon for map rendering.
  • This keeps advertisement persistence aligned with meshcore-packet-capture expectations (advertisement traffic only).

Compatibility ingest note (envelope fields):

  • The LetsMesh upload envelope carries SNR and path fields alongside the decoded packet payload. These are available on all packet types (messages, advertisements, traces, telemetry).
  • The normalizer extracts SNR (normalized to lowercase snr) and path (converted to path_len via hop count) from the envelope and includes them in the normalized payload.
  • Per-observer snr and path_len are stored in the event_observers junction table, allowing each observer to record its own signal strength and hop count.

Compatibility ingest note (non-message structured events):

  • Decoded payload type 9 is normalized to TRACE_DATA (traceTag, flags, auth, path hashes, and SNR values).
  • Decoded payload type 11 (Control/NodeDiscoverResp) is normalized to contact events for node upsert parity.
  • Decoded payload type 8 is normalized to informational PATH_UPDATED events (hop_count + path hashes).
  • Decoded payload type 1 can be normalized to TELEMETRY_RESPONSE, BATTERY, PATH_UPDATED, or STATUS_RESPONSE when decrypted response content is structured and parseable.

Persisted Events (Non-Webhook)

These events are stored in the database but do not trigger webhooks.

TRACE_DATA

Network trace path results showing route and signal strength.

Database Table: trace_paths

Payload Schema:

{
  "initiator_tag": "integer",
  "path_len": "integer (optional)",
  "flags": "integer (optional)",
  "auth": "integer (optional)",
  "path_hashes": "array of strings",
  "snr_values": "array of numbers",
  "hop_count": "integer (optional)"
}

Field Descriptions:

  • initiator_tag: Unique trace identifier (0-4294967295)
  • path_len: Length of the path
  • flags: Trace flags/options
  • auth: Authentication/validation data
  • path_hashes: Array of hex-encoded node hash identifiers, variable length (e.g., "4a" for single-byte, "b3fa" for multibyte), ordered by hops
  • snr_values: Array of SNR values corresponding to each hop
  • hop_count: Total number of hops

Example:

{
  "initiator_tag": 123456789,
  "path_len": 3,
  "flags": 0,
  "auth": 1,
  "path_hashes": ["4a", "b3fa", "02"],
  "snr_values": [25.3, 18.7, 12.4],
  "hop_count": 3
}

Note: MeshCore firmware v1.14+ supports multibyte path hashes. Older nodes use single-byte (2-character) hashes. Mixed-length hash arrays are expected in heterogeneous networks where nodes run different firmware versions.

Webhook Trigger: No REST API: GET /api/v1/trace-paths


TELEMETRY_RESPONSE

Sensor data from network nodes (temperature, humidity, battery, etc.).

Database Table: telemetry

Payload Schema:

{
  "node_public_key": "string (64 hex chars)",
  "lpp_data": "bytes (optional)",
  "parsed_data": "object (optional)"
}

Field Descriptions:

  • node_public_key: Full public key of the reporting node
  • lpp_data: Raw LPP-encoded sensor data (CayenneLPP format)
  • parsed_data: Decoded sensor readings as key-value pairs

Example:

{
  "node_public_key": "4767c2897c256df8d85a5fa090574284bfd15b92d47359741b0abd5098ed30c4",
  "lpp_data": null,
  "parsed_data": {
    "temperature": 22.5,
    "humidity": 65,
    "battery": 3.8,
    "pressure": 1013.25
  }
}

Webhook Trigger: No REST API: GET /api/v1/telemetry


CONTACTS

Contact sync event containing all known nodes.

Database Table: Updates nodes table

Payload Schema:

{
  "contacts": [
    {
      "public_key": "string (64 hex chars)",
      "name": "string (optional)",
      "node_type": "string (optional)"
    }
  ]
}

Field Descriptions:

  • contacts: Array of contact objects
  • public_key: Node's full public key
  • name: Node name/alias
  • node_type: One of: "chat", "repeater", "room", "none"

Example:

{
  "contacts": [
    {
      "public_key": "01ab2186c4d5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a6b7c8d9e0f1",
      "name": "Alice",
      "node_type": "chat"
    },
    {
      "public_key": "b3f4e5d6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4",
      "name": "Bob",
      "node_type": "chat"
    }
  ]
}

Webhook Trigger: No REST API: GET /api/v1/nodes


Informational Events

These events are logged to events_log table but not persisted to separate tables.

SEND_CONFIRMED

Confirmation that a sent message was delivered.

Payload Schema:

{
  "destination_public_key": "string (64 hex chars)",
  "round_trip_ms": "integer"
}

Field Descriptions:

  • destination_public_key: Recipient's full public key
  • round_trip_ms: Round-trip time in milliseconds

Example:

{
  "destination_public_key": "4767c2897c256df8d85a5fa090574284bfd15b92d47359741b0abd5098ed30c4",
  "round_trip_ms": 2500
}

STATUS_RESPONSE

Device status information.

Payload Schema:

{
  "node_public_key": "string (64 hex chars)",
  "status": "string (optional)",
  "uptime": "integer (optional)",
  "message_count": "integer (optional)"
}

Field Descriptions:

  • node_public_key: Node's full public key
  • status: Status description
  • uptime: Uptime in seconds
  • message_count: Total messages processed

Example:

{
  "node_public_key": "4767c2897c256df8d85a5fa090574284bfd15b92d47359741b0abd5098ed30c4",
  "status": "operational",
  "uptime": 86400,
  "message_count": 1523
}

BATTERY

Battery status information.

Payload Schema:

{
  "battery_voltage": "number",
  "battery_percentage": "integer"
}

Field Descriptions:

  • battery_voltage: Battery voltage (e.g., 3.7V)
  • battery_percentage: Battery level 0-100%

Example:

{
  "battery_voltage": 3.8,
  "battery_percentage": 75
}

PATH_UPDATED

Notification that routing path to a node has changed.

Payload Schema:

{
  "node_public_key": "string (64 hex chars)",
  "hop_count": "integer"
}

Field Descriptions:

  • node_public_key: Target node's full public key
  • hop_count: Number of hops in new path

Example:

{
  "node_public_key": "4767c2897c256df8d85a5fa090574284bfd15b92d47359741b0abd5098ed30c4",
  "hop_count": 3
}

Per-Payload-Type Classifications

Packets on the LetsMesh packets feed that no structured handler claims are classified by their MeshCore payload type rather than the generic letsmesh_packet. These are informational (logged to events_log, captured to raw_packets), with no dedicated table:

Payload type event_type Notes
0x00 REQ req Encrypted request, metadata only
0x01 RESPONSE response Response with no structured content
0x02 TXT_MSG encrypted_direct Direct text that did not decrypt
0x03 ACK ack Acknowledgment
0x04 ADVERT advert Advert lacking identity metadata
0x05 GRP_TXT encrypted_channel Channel text with unknown key
0x06 GRP_DATA grp_data Group datagram
0x07 ANON_REQ anon_req Anonymous request
0x08 PATH path Returned path (unparsed)
0x09 TRACE trace Trace (unparsed)
0x0A MULTIPART multipart Fragment of a sequence (no reassembly)
0x0B CONTROL control Control packet not matched to contact/status
0x0F RAW_CUSTOM raw_custom Custom-encrypted, opaque

letsmesh_packet remains as the safety-net label when the payload type cannot be resolved.

Other Informational Events

The following events are logged but have varying or device-specific payloads:

  • STATISTICS: Network statistics (varies by implementation)
  • DEVICE_INFO: Device hardware/firmware information
  • BINARY_RESPONSE: Binary data responses
  • CONTROL_DATA: Control/command responses
  • RAW_DATA: Raw protocol data
  • NEXT_CONTACT: Contact enumeration progress
  • ERROR: Error messages with description
  • MESSAGES_WAITING: Notification of queued messages
  • NO_MORE_MSGS: End of message queue
  • RX_LOG_DATA: Receive log data

Raw Packets

When RAW_PACKET_CAPTURE_ENABLED is set (Compose derives it from FEATURE_PACKETS), the collector stores every inbound packet from the LetsMesh packets feed exactly as received, independent of how it is classified. One row is written per observer reception (no deduplication), reusing the decode the normalizer already performs.

Database Table: raw_packets

Column Type Description
id UUID Primary key
observer_node_id FK nodes.id (SET NULL) Receiving interface, from the MQTT topic public key
packet_hash string(32) LetsMesh packet hash; links to structured records, groups multi-observer receptions
raw_hex text On-air bytes from payload["raw"]
packet_type integer Wire packet type
payload_type integer Decoder payload type
event_type string(50) How the collector classified the packet (advertisement, channel_msg_recv, letsmesh_packet, …)
channel_idx integer int(channelHash, 16) for channel-message packets, else NULL; drives visibility redaction
source_pubkey_prefix string(12) Sender prefix from decoder sourceHash / senderPublicKey
route_type string(20) flood, transport_flood, direct, transport_direct
path_len integer Hop count
snr float Signal-to-noise ratio reported by the observer
decoded JSON Decoder summary (so detail views need no re-decode)
received_at datetime When received by the observing interface

Linking from structured tables: advertisements and messages carry a nullable packet_hash (the LetsMesh wire hash) populated at ingest. Because the structured tables are deduplicated across observers, one row links to all the per-observer raw_packets sharing that hash — reachable via GET /api/v1/packets?packet_hash=<hash>. Only rows ingested while capture was enabled have a populated packet_hash (no backfill).

REST API: GET /api/v1/packets, GET /api/v1/packets/{id}

RawPacketRead shape mirrors the columns above plus a redacted: bool field and observer details (observed_by, observer_name, observer_tag_name). Channel-message packets on a channel above the requesting role's visibility level are returned with redacted=true and raw_hex/decoded/source_pubkey_prefix nulled; non-channel and visible-channel packets are returned in full. Responses are cached in Redis with a role-aware key.

Filters: search (hash/raw_hex/observer, bounded to a recent window when no since given), event_type, packet_type, channel_idx, route_type, public_key/pubkey_prefix, observed_by, decryptable, min_snr/max_snr, min_path_len/max_path_len, redacted, since/until, sort/order, limit/offset.


Webhook Payload Format

All webhook events are wrapped in a standard envelope before being sent:

{
  "event_type": "string",
  "timestamp": "string (ISO 8601)",
  "data": {
    // Event-specific payload (see schemas above)
  }
}

Example (Channel Message):

{
  "event_type": "CHANNEL_MSG_RECV",
  "timestamp": "2025-11-28T19:41:38.748379Z",
  "data": {
    "channel_idx": 4,
    "path_len": 10,
    "txt_type": 0,
    "text": "Hello from the mesh!",
    "SNR": 8.5,
    "sender_timestamp": 1732820498
  }
}

JSONPath Filtering

You can filter webhook payloads using JSONPath expressions:

JSONPath Result
$ Full payload (default)
$.data Event data only
$.data.text Message text only
$.data.[text,SNR] Multiple fields as array
$.event_type Event type string
$.timestamp Timestamp string

See AGENTS.md for webhook configuration details.


API Response: Observer Info

Events that support multi-observer tracking (messages, advertisements, trace paths, telemetry) include an observers array in API responses. Each observer entry contains:

Field Type Description
node_id string (UUID) Observer node UUID
public_key string (64 hex chars) Observer node public key
name string or null Observer node advertised name
tag_name string or null Observer name from node tags
snr number or null Signal-to-noise ratio at this observer (dB)
path_len integer or null Hop count at this observer
observed_at string (ISO 8601) When this observer captured the event

Event Flow

  1. Hardware/Mock MeshCore → Generates raw events
  2. Event Handler → Processes and persists to database
  3. Webhook Handler → Sends HTTP POST to configured URLs (if enabled)
  4. REST API → Query historical data from database

Most events are logged to the events_log table with full payloads for debugging and audit purposes. Some high-frequency informational events (e.g., NEXT_CONTACT) are intentionally excluded from logging to reduce database size.