mirror of https://github.com/ajvpot/meshexplorer.git synced 2026-07-29 21:02:21 +02:00

Files

T

Alex Vanderpot 8afda3dbc3 Derive more neighbor edges: hash-size-aware paths, anchors, confidence tiers

Rework the meshcore_all_neighbor_edges materialized view (migration 008)
so the map's "show all neighbors" layer surfaces many more, and
higher-quality, edges:

- Parse routing paths at the packet's real hash_size (1/2/3 bytes per hop)
  instead of assuming 1-byte hops, so extended-hash packets are no longer
  mis-sliced, and wide (2-/3-byte) prefixes resolve uniquely.
- Add anchored edges: resolve a hop against a fully-known endpoint (the
  advert originator or the uploading gateway) when exactly one in-region
  repeater of the right prefix sits within a plausible LoRa-hop distance.
- Tag every edge with a derivation `method` and a `confidence` score, and
  aggregate undirected edges across observations.

Frontend: expose method/confidence on the all-neighbors API and color edges
by method / fade by confidence. The old "only MQTT neighbors" checkbox
becomes the top notch of a single confidence selector (MQTT-direct only ->
high -> standard -> all).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-06-19 20:40:07 -04:00

clickhouse

clickhouse: upgrade image 25.6.2.5 -> 26.5.1.882

2026-05-29 10:16:22 -04:00

cmd/meshcoreingest

Accept numeric SNR/len/RSSI in MeshCore packets messages

2026-06-19 03:07:06 -04:00

internal

ingest: batch ClickHouse inserts to stop MQTT flapping & packet loss (#41 )

2026-06-15 22:42:46 -04:00

migrations

Derive more neighbor edges: hash-size-aware paths, anchors, confidence tiers

2026-06-19 20:40:07 -04:00

.gitignore

ingest: meshcore MQTT->ClickHouse stack

2026-05-29 01:01:33 -04:00

Dockerfile

ingest: meshcore MQTT->ClickHouse stack

2026-05-29 01:01:33 -04:00

go.mod

ingest: fix go module path to match repo

2026-05-29 01:19:42 -04:00

go.sum

ingest: meshcore MQTT->ClickHouse stack

2026-05-29 01:01:33 -04:00

README.md

ingest: meshcore MQTT->ClickHouse stack

2026-05-29 01:01:33 -04:00

README.md

MeshCore Ingest

A Go service that ingests MeshCore MQTT messages into ClickHouse, plus the ClickHouse image and SQL migrations for the schema.

This directory is normally run as part of the full stack via the root docker compose. The notes below cover running and developing it on its own.

Components

cmd/meshcoreingest — the ingest daemon. Subscribes to MeshCore MQTT topics and writes raw packets into the meshcore_packets table.
internal/ingestcommon — shared MQTT + ClickHouse connection/daemon logic.
internal/migrate — a goose based migration runner (ClickHouse dialect).
migrations/ — the ClickHouse schema: the meshcore_packets table, the decoded meshcore_adverts / meshcore_adverts_latest / meshcore_public_channel_messages views, and the unified_latest_nodeinfo view consumed by the web app.
clickhouse/ — a thin ClickHouse server image plus the read-only user used by the web app.

Configuration

All configuration is via environment variables (no credentials are baked into the source):

Variable	Description
`MQTT_BROKERS`	JSON array of brokers: `[{"url","username","password","topics"}]`. `topics` defaults to `["meshcore/#"]`. Required; the daemon exits if unset.
`MQTT_CLIENT_ID`	MQTT client id prefix (default `meshcore-ingest`).
`CLICKHOUSE_HOST` / `CLICKHOUSE_PORT`	ClickHouse address (native protocol, default `127.0.0.1:9000`).
`CLICKHOUSE_DB` / `CLICKHOUSE_USER` / `CLICKHOUSE_PASSWORD`	ClickHouse database and read/write credentials.

Building

go build ./...
go test ./...

Running migrations

go run ./internal/migrate \
  -host localhost -port 9000 \
  -username default -password "$CLICKHOUSE_PASSWORD" \
  -path migrations -action up

Actions: up, down, reset, status, version.

Running the ingest daemon

export MQTT_BROKERS='[{"url":"tcp://mqtt.example.com:1883","username":"u","password":"p","topics":["meshcore/#"]}]'
export CLICKHOUSE_HOST=localhost CLICKHOUSE_PORT=9000 CLICKHOUSE_PASSWORD=...
go run ./cmd/meshcoreingest