mirror of
https://github.com/ajvpot/meshexplorer.git
synced 2026-07-28 20:32:23 +02:00
Alex Vanderpot
7528eac3a2
* Derive more neighbor edges: hash-size-aware paths, anchors, confidence tiers Rework the meshcore_all_neighbor_edges materialized view (migration 008) so the map's "show all neighbors" layer surfaces many more, and higher-quality, edges: - Parse routing paths at the packet's real hash_size (1/2/3 bytes per hop) instead of assuming 1-byte hops, so extended-hash packets are no longer mis-sliced, and wide (2-/3-byte) prefixes resolve uniquely. - Add anchored edges: resolve a hop against a fully-known endpoint (the advert originator or the uploading gateway) when exactly one in-region repeater of the right prefix sits within a plausible LoRa-hop distance. - Tag every edge with a derivation `method` and a `confidence` score, and aggregate undirected edges across observations. Frontend: expose method/confidence on the all-neighbors API and color edges by method / fade by confidence. The old "only MQTT neighbors" checkbox becomes the top notch of a single confidence selector (MQTT-direct only -> high -> standard -> all). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Tighten neighbor edges: 100km anchor cap + region-unique anchor binds Two precision fixes after reviewing cross-metro over-linking: - Cap anchored edges at 100 km (was 206 km). Anchors bind a hop to the single nearby repeater of a prefix, so a wrong bind is indistinguishable from a real hop -> a loose cap admitted ~200 km false links. path-uniq and direct edges (observed consecutive hops / zero-hop adverts) keep the 206 km backstop. - Require a region-unique prefix for anchor binds. Without it, a 1-byte hop whose true (unlocated) forwarder is out of set mis-binds to a coincidental same-prefix repeater that merely happens to be the lone nearby candidate (~22% of 1-byte anchors). Region-uniqueness removes the proximity guess; the 100 km check becomes a sanity confirm. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Demote anchored edges to substandard confidence Anchored edges are geographically inferred (resolve a hop to the region-unique repeater near a known endpoint), not observed consecutive hops, so they are less trustworthy than any path-uniq edge. Re-rank confidence to: direct 1.0 > path-uniq-3b 0.8 > path-uniq-2b 0.6 > anchor 0.45 > path-uniq-1b 0.4. Anchors now fall below the 0.5 "Standard" default (hidden by default, just above the noisy 1-byte tier). Add an "Include anchored (lower confidence)" UI tier at 0.45 so they can be opted into without the 1-byte noise. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Unify neighbor inference: global confidence setting, hover + node page - Move the neighbor-confidence preference from per-map-layer settings to global Settings (ConfigContext); the map all-neighbors layer, map hover, and node page all read it. Default "Standard" (0.5). - Map hover and the node detail page now derive neighbors from the unified graph (meshcore_all_neighbor_edges) via a new getMeshcoreNodeAllEdges + a mode=all option on the node neighbors route/hook, so they match "show all neighbors". - Node page shows direct and inferred neighbors in one list; inferred cards carry an "Inferred · <method>" + confidence indicator instead of direction arrows. - Color map hover lines by derivation method (purple/blue/teal), fade by confidence. - Remove the "Manage Channel Keys" button from the Settings popover (still reachable via the + on the messages page). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Keep neighbor lines when either endpoint is in view Both the server bbox filter (getAllNodeNeighbors) and the client filter (AllNeighborLines) required BOTH endpoints inside the viewport, so any line with one end off-screen vanished when zooming in. Switch both to "either endpoint in view": the server returns an edge if source OR target falls in the bbox, and the client keeps it if either endpoint is a visible node. Each edge carries both endpoints' coordinates, so the off-screen end still draws. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Alex Vanderpot <alex@Alexs-MacBook-Pro-2.local> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MeshCore Ingest
A Go service that ingests MeshCore MQTT messages into ClickHouse, plus the ClickHouse image and SQL migrations for the schema.
This directory is normally run as part of the full stack via the
root docker compose. The notes below cover running and
developing it on its own.
Components
cmd/meshcoreingest— the ingest daemon. Subscribes to MeshCore MQTT topics and writes raw packets into themeshcore_packetstable.internal/ingestcommon— shared MQTT + ClickHouse connection/daemon logic.internal/migrate— a goose based migration runner (ClickHouse dialect).migrations/— the ClickHouse schema: themeshcore_packetstable, the decodedmeshcore_adverts/meshcore_adverts_latest/meshcore_public_channel_messagesviews, and theunified_latest_nodeinfoview consumed by the web app.clickhouse/— a thin ClickHouse server image plus the read-only user used by the web app.
Configuration
All configuration is via environment variables (no credentials are baked into the source):
| Variable | Description |
|---|---|
MQTT_BROKERS |
JSON array of brokers: [{"url","username","password","topics"}]. topics defaults to ["meshcore/#"]. Required; the daemon exits if unset. |
MQTT_CLIENT_ID |
MQTT client id prefix (default meshcore-ingest). |
CLICKHOUSE_HOST / CLICKHOUSE_PORT |
ClickHouse address (native protocol, default 127.0.0.1:9000). |
CLICKHOUSE_DB / CLICKHOUSE_USER / CLICKHOUSE_PASSWORD |
ClickHouse database and read/write credentials. |
Building
go build ./...
go test ./...
Running migrations
go run ./internal/migrate \
-host localhost -port 9000 \
-username default -password "$CLICKHOUSE_PASSWORD" \
-path migrations -action up
Actions: up, down, reset, status, version.
Running the ingest daemon
export MQTT_BROKERS='[{"url":"tcp://mqtt.example.com:1883","username":"u","password":"p","topics":["meshcore/#"]}]'
export CLICKHOUSE_HOST=localhost CLICKHOUSE_PORT=9000 CLICKHOUSE_PASSWORD=...
go run ./cmd/meshcoreingest