mirror of
https://github.com/ajvpot/meshexplorer.git
synced 2026-07-06 17:50:58 +02:00
Alex Vanderpot
0265a900a0
The advert app_data flags byte (Core Protocol §2.8.3) uses split encoding: the low 4 bits are an integer node-type enum (1=chat/companion, 2=repeater, 3=room server, 4=sensor) and the high 4 bits are independent presence flags. The decode treated the type as independent bit flags, so a room server (type 3 = 0b0011) matched is_chat_node, is_repeater AND is_room_server at once, and a sensor (type 4) matched none and rendered as an untyped node. Migration 008 fixes the derivation in the meshcore_adverts / meshcore_adverts_latest views (read-time only, retroactive over all history), adds is_sensor and a node_type column, and carries neighbor_is_sensor onto the direct-neighbor graph. The UI gains a sensor badge across the node page, node cards, and advert details, and search/streaming/node queries expose is_sensor. Ingest: drop envelopes whose decoded packet has zero bytes instead of storing them. They decode to an empty payload and a degenerate packet_hash and carry no signal; dropped rows are counted and logged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MeshCore Ingest
A Go service that ingests MeshCore MQTT messages into ClickHouse, plus the ClickHouse image and SQL migrations for the schema.
This directory is normally run as part of the full stack via the
root docker compose. The notes below cover running and
developing it on its own.
Components
cmd/meshcoreingest— the ingest daemon. Subscribes to MeshCore MQTT topics and writes raw packets into themeshcore_packetstable.internal/ingestcommon— shared MQTT + ClickHouse connection/daemon logic.internal/migrate— a goose based migration runner (ClickHouse dialect).migrations/— the ClickHouse schema: themeshcore_packetstable, the decodedmeshcore_adverts/meshcore_adverts_latest/meshcore_public_channel_messagesviews, and theunified_latest_nodeinfoview consumed by the web app.clickhouse/— a thin ClickHouse server image plus the read-only user used by the web app.
Configuration
All configuration is via environment variables (no credentials are baked into the source):
| Variable | Description |
|---|---|
MQTT_BROKERS |
JSON array of brokers: [{"url","username","password","topics"}]. topics defaults to ["meshcore/#"]. Required; the daemon exits if unset. |
MQTT_CLIENT_ID |
MQTT client id prefix (default meshcore-ingest). |
CLICKHOUSE_HOST / CLICKHOUSE_PORT |
ClickHouse address (native protocol, default 127.0.0.1:9000). |
CLICKHOUSE_DB / CLICKHOUSE_USER / CLICKHOUSE_PASSWORD |
ClickHouse database and read/write credentials. |
Building
go build ./...
go test ./...
Running migrations
go run ./internal/migrate \
-host localhost -port 9000 \
-username default -password "$CLICKHOUSE_PASSWORD" \
-path migrations -action up
Actions: up, down, reset, status, version.
Running the ingest daemon
export MQTT_BROKERS='[{"url":"tcp://mqtt.example.com:1883","username":"u","password":"p","topics":["meshcore/#"]}]'
export CLICKHOUSE_HOST=localhost CLICKHOUSE_PORT=9000 CLICKHOUSE_PASSWORD=...
go run ./cmd/meshcoreingest