mirror of
https://github.com/l5yth/potato-mesh.git
synced 2026-08-09 10:22:52 +02:00
data: meshcore rf metrics rssi/snr/hops/path (#849)
This commit is contained in:
l5y
+127
@@ -450,6 +450,22 @@ existing codebase, not to the change under review.
|
||||
the failures reproduce identically with and without any change under review
|
||||
and do not occur where the test domains resolve normally (CI). Attribute to
|
||||
the environment, not the codebase or the change.
|
||||
- **C2 — `tests/test_mesh.py` fails in isolation on Python 3.14.** Running
|
||||
the C2 command alone (`pytest -q tests/test_mesh.py`) fails 3 daemon
|
||||
reconnect-loop tests (`test_main_retries_interface_creation`,
|
||||
`test_main_reconnects_when_connection_event_clears`,
|
||||
`test_main_recreates_interface_after_snapshot_error`): their local
|
||||
`DummyEvent` helpers monkeypatch the global `threading.Event` with a
|
||||
`wait(self, timeout)` signature that requires an argument, and Python
|
||||
3.14's `Thread.start()` calls `self._started.wait()` with none →
|
||||
`TypeError`. Pre-existing and independent of any change under review
|
||||
(reproduces on a clean tree), and **the full suite (`pytest -q tests/`)
|
||||
passes** — the failure is an isolation/collection-order artifact of the
|
||||
global patch. A naive `timeout=None` default is *not* a fix: it lets the
|
||||
patched-Event path run further and breaks thread startup in the full suite
|
||||
too ("cannot join thread before it is started"). Needs a proper follow-up
|
||||
that stops monkeypatching the global `threading.Event` in those three
|
||||
tests; until then, judge C2 by the full-suite run.
|
||||
|
||||
---
|
||||
|
||||
@@ -2911,3 +2927,114 @@ locals, never the accumulators), the node detail page and chart suites
|
||||
position / neighbor aggregation keep their existing `SNAPSHOT_WINDOW`
|
||||
semantics — only telemetry aggregation changes. No Ruby/Python surface is
|
||||
touched (**C2** and the Python suite unaffected).
|
||||
|
||||
---
|
||||
|
||||
## Feature: MeshCore RF metrics (RSSI/SNR/hops/path) & roster-eviction assertion
|
||||
|
||||
Maps to SPEC decisions **RF1–RF8**. Ingestor-side logic lives in
|
||||
`data/mesh_ingestor/protocols/meshcore/` (runner, handlers, decode) and the
|
||||
Meshtastic hops computation in the packet store path; web-side, one additive
|
||||
migration adds `messages.hops`, `messages.path`, and `nodes.rssi`, mapped in
|
||||
`data_processing/` and serialized by the existing GET routes. Store + API only —
|
||||
no dashboard rendering (RF7). Unless a check says otherwise, Python commands
|
||||
assume the repo venv (`. .venv/bin/activate`).
|
||||
|
||||
### RF-A1 — hops-travelled stored on messages, both protocols — RF1
|
||||
```bash
|
||||
( . .venv/bin/activate && pytest -q tests/ -k "hops" )
|
||||
( cd web && bundle exec rspec spec -e "message hops" )
|
||||
```
|
||||
**Expected:** pass. MeshCore: a `CHANNEL_MSG_RECV`/`CONTACT_MSG_RECV` payload
|
||||
with `path_len: N` (N ≤ 63) yields a stored packet with `hops == N`; the `255`
|
||||
"direct" sentinel yields `hops == 0`; an absent `path_len` omits the field.
|
||||
Meshtastic: a packet carrying both `hopStart` and `hopLimit` yields
|
||||
`hops == hopStart − hopLimit`; either absent → field omitted. Web: the
|
||||
`messages` table has an additive `hops INTEGER` column (NULL for legacy rows),
|
||||
`POST /api/messages` accepts it, `GET /api/messages` serializes it, and the
|
||||
existing `hop_limit` column/semantics are untouched.
|
||||
|
||||
### RF-A2 — channel-message RSSI + path via the decrypt_channels join — RF2
|
||||
```bash
|
||||
( . .venv/bin/activate && pytest -q tests/test_provider_unit.py -k "decrypt or path or rssi" )
|
||||
( cd web && bundle exec rspec spec -e "message path" )
|
||||
```
|
||||
**Expected:** pass. `_run_meshcore` sets `mc.decrypt_channels = True` before
|
||||
`mc.connect()` returns. A channel-message payload carrying joined `RSSI`/`path`
|
||||
stores both (`rssi` → existing column; `path` → additive `messages.path TEXT`,
|
||||
lowercase hex, hashes in travel order); a payload **without** them (join miss,
|
||||
RX-log-less firmware) stores the message identically with the fields absent —
|
||||
never an error. DMs never carry `path`/`rssi` (E2E, no join — RF2's documented
|
||||
boundary). The message id (`_derive_message_id` inputs) is byte-identical with
|
||||
and without the new fields.
|
||||
|
||||
### RF-A3 — RX-log ADVERT frames upsert full node identity + signal — RF3
|
||||
```bash
|
||||
( . .venv/bin/activate && pytest -q tests/test_provider_unit.py -k "rx_log or advert" )
|
||||
( cd web && bundle exec rspec spec -e "node rssi" )
|
||||
```
|
||||
**Expected:** pass. An `RX_LOG_DATA` event with `payload_typename == "ADVERT"`
|
||||
upserts a node keyed by the canonical id derived from the full `adv_key`
|
||||
(`_meshcore_node_id`), carrying `adv_name` (long name), the
|
||||
`_MESHCORE_ADV_TYPE_ROLE` role for `adv_type`, a position when
|
||||
`adv_lat`/`adv_lon` are present, and per-reception `snr` → `nodes.snr`,
|
||||
`path_len` → `nodes.hops_away`, `rssi` → the additive `nodes.rssi INTEGER`
|
||||
column. A malformed advert (missing/short `adv_key`, absent parse fields) is
|
||||
tolerated without raising. Non-`ADVERT` RX-log frames produce **no** upsert and
|
||||
remain in the `DEBUG`-only capture; `RX_LOG_DATA` itself no longer lands in
|
||||
`ignored-meshcore.txt`. With **zero** RX-log frames the provider still passes
|
||||
RF-A1/RF-A4 behavior (graceful degradation). Web: `POST /api/nodes` accepts
|
||||
`rssi`, `GET /api/nodes` serializes it, and it stays `NULL` for Meshtastic
|
||||
nodes (no source).
|
||||
|
||||
### RF-A4 — roster-eviction assertion: read-modify-write, skip, tolerate — RF4
|
||||
```bash
|
||||
( . .venv/bin/activate && pytest -q tests/test_provider_unit.py -k "autoadd" )
|
||||
```
|
||||
**Expected:** pass. After connect the runner calls `get_autoadd_config`: when
|
||||
bit `0x01` is already set → **no** `set_autoadd_config` call (no flash write);
|
||||
when unset → exactly one `set_autoadd_config(config | 0x01)` (type-filter bits
|
||||
1–4 preserved, one-byte payload so `autoadd_max_hops` is untouched); when the
|
||||
query/set errors or times out (pre-1.16 firmware) → a warning is logged and
|
||||
startup **continues** (the connection still succeeds, mirroring
|
||||
`_ensure_channel_names` tolerance). No env/config knob gates the behavior
|
||||
(RF4: always-on, README-documented).
|
||||
|
||||
### RF-A5 — CONTACT_DELETED is an explicit debug no-op — RF5
|
||||
```bash
|
||||
( . .venv/bin/activate && pytest -q tests/test_provider_unit.py -k "contact_deleted" )
|
||||
```
|
||||
**Expected:** pass. `CONTACT_DELETED` appears in the subscribed handler map; on
|
||||
event it debug-logs and performs **no** node deletion, no POST, and no ignored-
|
||||
file write — the web DB retains evicted nodes (`retention.rb` remains the only
|
||||
data-expiry authority).
|
||||
|
||||
### RF-A6 — contract documented; migration additive; dedup frozen — RF6
|
||||
```bash
|
||||
git grep -nE 'hops|path|rssi' -- data/mesh_ingestor/CONTRACTS.md | head
|
||||
grep -nE 'ALTER TABLE (messages|nodes) ADD COLUMN' data/migrations/*rf_metric*.sql
|
||||
grep -nE 'hops|path' data/messages.sql; grep -n 'rssi' data/nodes.sql
|
||||
( . .venv/bin/activate && pytest -q tests/ -k "derive_message_id or dedup" )
|
||||
```
|
||||
**Expected:** `CONTRACTS.md` documents `messages.hops`/`messages.path` (with
|
||||
the `255`→direct rule and the path hex format) and `nodes.rssi` (advert→node
|
||||
mapping). The migration contains only additive `ALTER TABLE … ADD COLUMN`
|
||||
statements (no drops/rewrites); the base schema files carry the new columns for
|
||||
fresh databases. The dedup tests pass unchanged — the fingerprint inputs are
|
||||
byte-identical to pre-feature (MD-A1/MW-A1 hold).
|
||||
|
||||
### RF-R1 — Regression: prior acceptance still holds
|
||||
```bash
|
||||
( . .venv/bin/activate && pytest -q tests/ )
|
||||
( cd web && bundle exec rspec ) && ( cd web && npm test )
|
||||
```
|
||||
**Expected:** every prior check still passes. At risk and explicitly required
|
||||
to remain green: **A4e** (the MeshCore adverts-gap checks — the
|
||||
bare-`ADVERTISEMENT` minimal-upsert fallback must keep working alongside the
|
||||
new RX-log enrichment; its assertions are **updated**, not removed), **C2**
|
||||
(`test_mesh.py` POST shapes — all field additions are additive), **MD-A1 /
|
||||
MW-A1** (MeshCore dedup — id derivation byte-identical), **MC-A1 / LH-A1 /
|
||||
GH-A1** (MeshCore message/contact machinery — naming, `last_heard`, and
|
||||
stale-contact behavior unchanged), and **B1/B4/B5** (all suites, headers,
|
||||
formatters). The JS suite is exercised for regression only — RF7 adds no
|
||||
frontend behavior.
|
||||
|
||||
@@ -274,6 +274,24 @@ example `ALLOWED_CHANNELS="Chat,Ops"`); packets on other channels are discarded.
|
||||
Use `HIDDEN_CHANNELS` to block specific channels from the web UI even when they
|
||||
appear in the allowlist.
|
||||
|
||||
### MeshCore
|
||||
|
||||
Set `PROTOCOL=meshcore` to ingest from a MeshCore companion-firmware node
|
||||
instead (serial, TCP, or BLE via the same `CONNECTION` formats). Alongside
|
||||
contacts and messages, the ingestor captures RF metrics: per-message SNR and
|
||||
hop counts, per-channel-message RSSI and repeater path (decoded from the
|
||||
radio's RX log), and per-advert SNR/RSSI/hops for every node heard on air —
|
||||
including nodes the radio's contact roster has no room for.
|
||||
|
||||
**Note — the ingestor writes one radio setting.** At startup it enables the
|
||||
firmware's *overwrite-oldest-contact* flag (`autoadd_config` bit `0x01`,
|
||||
firmware ≥ 1.16) so a full contact roster evicts its oldest non-favourite
|
||||
entry instead of rejecting new nodes. The write happens only when the bit is
|
||||
not already set, preserves all other auto-add settings, persists on the
|
||||
device, and never evicts favourites. Older firmware without the command is
|
||||
left untouched. Evicted contacts remain in the dashboard's database — only
|
||||
the radio's local roster rotates.
|
||||
|
||||
### Passive UDP transport
|
||||
|
||||
The Meshtastic node radio accepts only **one** API client at a time (serial or
|
||||
|
||||
@@ -642,3 +642,56 @@ it rare and, when it occurs, near-invisible.
|
||||
| **BL3** | **Shared dark filter on the fallback tile (amends HT2).** `.map-tiles-fallback` no longer renders `filter:none`; it carries the **same** `grayscale(1) invert(1) brightness(0.9) contrast(1.08)` filter as `.map-tiles-hot`, expressed as one comma-grouped `base.css` rule (the single source of truth for the value). Both providers therefore converge to one coherent dark look, so a viewport mixing HOT and CARTO tiles is no longer a checkerboard — the second half of the fix. The per-tile class swap in `fallback-tile-layer.js` is **unchanged**; only what `.map-tiles-fallback` *does* in CSS changes. Offline placeholder tiles still carry neither class and stay unfiltered. | interview |
|
||||
| **BL4** | **Both maps, one rule; posture preserved (reaffirms HT5 / HT7).** The federation map shares the dashboard's `#map` container, so the single `base.css` filter rule and the shared `createBasemapLayer` factory land the blend on **both** maps identically (parity, Invariant IV). The filter value and the timeout stay **frontend constants** — no Ruby `tile_filters`, no `/version` / `data-app-config` key, no `/api/*` change, no version bump (HT7 / D8 unchanged). | proposed |
|
||||
|
||||
---
|
||||
|
||||
## Feature: MeshCore RF metrics (RSSI/SNR/hops/path) & roster-eviction assertion
|
||||
|
||||
Closes the MeshCore↔Meshtastic RF-metrics parity gap (issue #762 for MeshCore):
|
||||
messages gain RSSI/SNR/hops-travelled/path, and node records gain per-advert
|
||||
signal metrics, all sourced from data the `meshcore` library already delivers
|
||||
but the ingestor currently drops. Three mechanisms: (1) native `path_len`/`SNR`
|
||||
fields on the message sync events; (2) the library's built-in RX-log⇆message
|
||||
join enabled via `decrypt_channels`; (3) a new `RX_LOG_DATA` subscription
|
||||
handling on-air `ADVERT` frames (full node identity + signal metrics,
|
||||
roster-independent). Additionally the ingestor asserts the firmware's
|
||||
`AUTO_ADD_OVERWRITE_OLDEST` roster-eviction bit at startup so a full contact
|
||||
roster keeps rotating instead of rejecting new contacts. Integrates with
|
||||
`data/mesh_ingestor/protocols/meshcore/{handlers,runner,decode,interface,_constants}.py`,
|
||||
the packet store path in `data/mesh_ingestor/handlers/`, `CONTRACTS.md`, one
|
||||
additive SQLite migration (`messages.hops`, `messages.path`, `nodes.rssi`), the
|
||||
message/node mappings in `web/lib/potato_mesh/application/data_processing/`,
|
||||
and the serializers in `queries/`.
|
||||
|
||||
**Conflict check against existing decisions.** *Apex I (local LoRa)* —
|
||||
**consistent**: every new datum arrives over the existing local serial/BLE/TCP
|
||||
radio link; `decrypt_channels` is local crypto using channel keys the radio
|
||||
already holds; no broker, dependency, or egress (`guard-edits.py` untriggered).
|
||||
*Invariant II (privacy & consent)* — **consistent**: no new data class (channel
|
||||
messages already arrive decrypted via companion sync; adverts are public
|
||||
broadcasts already stored via the contact path); server-side opt-out,
|
||||
`PRIVATE`, and retention gates are untouched. The startup config write (RF4) is
|
||||
a **device** mutation, not a data exposure — treated as a §4.2 scope extension,
|
||||
documented operator-visibly in the README rather than silent. *Invariant IV
|
||||
(parity)* — **extends**: fills MeshCore's missing `rxSnr`/`rxRssi`/hops
|
||||
equivalents using the columns Meshtastic already populates; the new `hops`
|
||||
column is computed for **both** protocols. *D8/§3.4 (stable contract)* —
|
||||
**extends**: strictly additive fields (`hops`/`path` on messages, `rssi` on
|
||||
nodes; adverts otherwise reuse the already-accepted `snr`/`hops_away` node
|
||||
fields), no version bump. *A4e (MeshCore adverts gap)* — **extends**: RX-log
|
||||
adverts enrich non-roster nodes with full identity; the bare-`ADVERTISEMENT`
|
||||
minimal upsert stays as the fallback for builds without RX-log frames (A4e
|
||||
criteria updated, not removed). *MD-A1/MW-A1 (MeshCore dedup)* —
|
||||
**consistent**: `_derive_message_id` inputs stay byte-identical (RF6). No
|
||||
decision is contradicted.
|
||||
|
||||
| # | Decision | Source |
|
||||
| --- | --- | --- |
|
||||
| **RF1** | **Hops-travelled on messages, both protocols.** Message packets gain a `hops` field = repeater relays actually travelled: MeshCore reads the native `path_len` on `CONTACT_MSG_RECV`/`CHANNEL_MSG_RECV` (the `255` "direct" sentinel normalizes to `0`; the 2-bit `path_hash_mode` prefix is masked off); Meshtastic computes `hopStart − hopLimit` when both are present (else omits). Stored in a new **additive** `messages.hops INTEGER` column — deliberately distinct from `hop_limit` (remaining budget, a different semantic, left untouched). MeshCore V3 sync frames' native `SNR` continues to flow into the existing `messages.snr` column. | interview + code |
|
||||
| **RF2** | **Channel-message RSSI/path via the library's RX-log join.** The runner sets `mc.decrypt_channels = True`; the `meshcore` lib then matches each `CHANNEL_MSG_RECV` to its on-air frame by `SHA256(sender_timestamp + text)` and injects `RSSI`, `path`, `recv_time` (channel secrets are already registered by `_ensure_channel_names`, so no new key handling). The handler forwards `RSSI` → the existing `messages.rssi` column and the hop-hash route → a new **additive** `messages.path TEXT` column (lowercase hex, `path_hash_size`-byte hashes concatenated in travel order, last = the repeater heard directly; raw material for a future topology view, no hash→node resolution attempted now). **DM RSSI is explicitly out of scope** — direct messages are E2E-encrypted so no RX-log join exists; DMs still get native SNR + hops via RF1. | interview + code |
|
||||
| **RF3** | **RX-log `ADVERT` frames become full node upserts (roster-independent).** A new `RX_LOG_DATA` subscription handles frames with `payload_typename == "ADVERT"`: `adv_key` (full 32-byte pubkey) → canonical `!%08x` id, `adv_name` → long name, `adv_type` → role via `_MESHCORE_ADV_TYPE_ROLE`, `adv_lat`/`adv_lon` → position store, and per-reception `snr` → `nodes.snr`, `path_len` → `nodes.hops_away` (existing, already-accepted node fields), `rssi` → a new **additive** `nodes.rssi INTEGER` column (Meshtastic has no per-node RSSI source — `NodeInfo` carries only SNR — so it stays `NULL` there; the column itself is protocol-neutral). This restores full node identity even when the radio's roster is full, closing the anonymous-placeholder gap. **Only `ADVERT` frames are handled**; other RX-log payload types stay in the DEBUG-only catch-all, and `RX_LOG_DATA` no longer falls through to `ignored-meshcore.txt`. Degrades gracefully: companion firmware ≥ 1.16 pushes RX-log frames unconditionally while a client is connected, but if none arrive (other builds), RF1 metrics and the existing bare-`ADVERTISEMENT` fallback (A4e) still function — absent frames are never an error. | interview + code |
|
||||
| **RF4** | **Always-on roster-eviction assertion (extends §4.2 ingestor scope).** At startup (after connect, `_ensure_channel_names`-style error tolerance) the ingestor reads `get_autoadd_config` and, **only if** bit `0x01` (`AUTO_ADD_OVERWRITE_OLDEST`) is unset, writes `config \| 0x01` back — a read-modify-write that preserves the type-filter bits 1–4 and never touches `autoadd_max_hops`; when the bit is already set no write occurs (the firmware `savePrefs()`es every set — skipping avoids flash wear). Unconditional by deliberate choice — **no env/config knob**; the behavior is documented in the README (deliberate and operator-visible, not silent), favourites are never evicted (firmware guarantee), and the write persists in device flash. `ERROR`/timeout from pre-1.16 firmware logs a warning and continues. This is the ingestor's **first and only** radio-config write, bounded to this single bit. | interview |
|
||||
| **RF5** | **`CONTACT_DELETED` becomes an explicit no-op handler.** Roster eviction (RF4) makes the firmware emit `PUSH_CODE_CONTACT_DELETED` per evicted contact; the event moves from the DEBUG catch-all to an explicit debug-logged no-op in the handler map — the web DB **intentionally retains** evicted nodes (dashboard history is independent of roster capacity; `retention.rb` remains the only data-expiry authority). | interview |
|
||||
| **RF6** | **Additive contract; dedup fingerprint frozen.** `CONTRACTS.md` documents the new fields (`messages.hops`/`messages.path`, `nodes.rssi`), the `255`→direct rule, the path hex format, and the advert→node field mapping. All changes are **additive** (D8: no version bump): the POST routes accept and the GET routes serialize the new fields; absent fields stay `NULL`. The MeshCore dedup fingerprint (`_derive_message_id` inputs) and the `v1:` content-dedup scheme are **byte-identical** — new fields ride alongside, never inside, the id derivation (MD-A1/MW-A1 preserved). | interview + code |
|
||||
| **RF7** | **Store + API only; UI display deferred.** v1 ends at the JSON API — no dashboard rendering of the new metrics (chat hover, node popup, topology view from `path` are tracked follow-ups). Closes #762 for MeshCore at the data layer. | interview |
|
||||
| **RF8** | **Engineering bar (D9).** All new/changed units ship with 100% unit tests (including: RF1 hops normalization edge cases, RF2 join-absent fallback, RF3 malformed-advert tolerance, RF4 read-modify-write/skip/error paths, RF5 no-op), full PDoc/RDoc, the exact Apache header, `black`/`rufo` clean; every existing suite stays green. | CLAUDE.md |
|
||||
|
||||
|
||||
@@ -44,6 +44,7 @@ remains accepted. Per-field acceptance is nil-aware, so a camelCase value of
|
||||
- `num` (int node number)
|
||||
- `lastHeard` (int unix seconds)
|
||||
- `snr` (float)
|
||||
- `rssi` (int|nil) — per-advert reception RSSI (SPEC RF3). Sourced from MeshCore RX-log adverts; Meshtastic reports no per-node RSSI, so the field stays absent/NULL there. The web upsert keeps the last stored value when an update omits it (`COALESCE`), so contact-roster refreshes never wipe a per-advert reading.
|
||||
- `hopsAway` (int)
|
||||
- `isFavorite` (bool)
|
||||
- `user` (mapping; e.g. `shortName`, `longName`, `macaddr`, `hwModel`, `publicKey`, `isUnmessagable`)
|
||||
@@ -62,12 +63,16 @@ The web application applies the same normalisation as a safety net so legacy ing
|
||||
|
||||
**Wire-format note for federation peers (issue #782).** Position time is exposed **only** as `position_time` (unix seconds) on GET responses (`/api/nodes`, `/api/positions`); the redundant ISO twin (`pos_time_iso` on `/api/nodes`, `position_time_iso` on `/api/positions`) was **removed in 0.7.0** — clients format `position_time` themselves. Sentinel rows are compacted by **omitting** `position_time` rather than emitting `0` or `"1970-01-01T00:00:00Z"`. Federation peers consuming this API and any third-party clients SHOULD treat an *absent* `position_time` as "no GPS lock recorded" and not synthesise a zero or epoch value when re-serialising. Older peers that key on `position_time == 0` may need a small adjustment.
|
||||
|
||||
**MeshCore advert sourcing (capturing adverts from other nodes).** A MeshCore node announces itself by broadcasting an *advert* (public key + type + name + optional lat/lon). The ingestor surfaces heard adverts to `POST /api/nodes` through three complementary paths so coverage does not depend on the radio's auto-add setting:
|
||||
**MeshCore advert sourcing (capturing adverts from other nodes).** A MeshCore node announces itself by broadcasting an *advert* (public key + type + name + optional lat/lon). The ingestor surfaces heard adverts to `POST /api/nodes` through four complementary paths so coverage does not depend on the radio's auto-add setting or roster capacity:
|
||||
|
||||
- *Contact roster (rich).* The startup `ensure_contacts()` fetch plus live `NEW_CONTACT` / `NEXT_CONTACT` pushes carry the full advert (name, role, position) and upsert complete node rows. This covers every node the radio has added to its contact book.
|
||||
- *Auto-update re-fetch (freshness).* The provider sets `mc.auto_update_contacts = True`, so the meshcore library re-fetches **changed** contacts (incrementally, by `lastmod`) whenever an `ADVERTISEMENT` / `PATH_UPDATE` push arrives. A re-advert from a known node therefore refreshes its `last_advert` / position without waiting for a reconnect.
|
||||
- *Bare advert (reach).* The `ADVERTISEMENT` (pubkey-only) push is also handled directly: for a public key **not** in the contact roster it upserts a minimal "heard now" node (`lastHeard`, `protocol`, `user.shortName`/`publicKey` only — no name/type/position), so radios running with auto-add off still register the advertiser. Known keys are skipped (the auto-update path keeps them fresh). The Ruby web app preserves an existing long name on conflict, so this placeholder never clobbers a richer record, and a later full contact advertisement reconciles it. Reconciliation does not depend on timestamp ordering: the contact record carries `lastHeard = last_advert` (the **sender-stamped** advert-creation time), which is always older than the placeholder's wall-clock stamp — the web app's node upsert therefore fills identity fields (name, role, public key, …) that are still NULL even from an older-stamped record, while timestamps/telemetry stay freshness-guarded (ACCEPTANCE GH-A1).
|
||||
|
||||
- *RX-log advert (full identity + signal, roster-independent — SPEC RF3).* Companion firmware ≥ 1.16 pushes every received RF frame (`RX_LOG_DATA`) while a client is connected; the library parses `ADVERT` frames completely (full public key, name, type, optional lat/lon). The ingestor converts these to full node upserts carrying per-reception `snr` / `rssi` / `hopsAway`, so node identity and signal metrics no longer depend on the radio's contact roster at all — including when the roster is full. Absent RX-log frames (older/other builds) are never an error; the three paths above still function. Non-`ADVERT` RX-log frames are not ingested (DEBUG-only capture).
|
||||
|
||||
**MeshCore roster-eviction assertion (SPEC RF4).** At startup the provider asserts the firmware's `AUTO_ADD_OVERWRITE_OLDEST` bit (`autoadd_config` bit `0x01`): it reads the current config and, only when the bit is unset, writes `config | 0x01` back — preserving the type-filter bits and `autoadd_max_hops`, and skipping the write (and its flash `savePrefs()`) when already set. With the bit set, a full contact roster evicts its oldest non-favourite entry instead of rejecting new contacts, so `NEW_CONTACT` coverage keeps rotating; favourites are never evicted (firmware guarantee) and the resulting `CONTACT_DELETED` pushes are deliberately ignored (the web DB retains evicted nodes; server-side retention remains the only data-expiry authority). Unconditional, no configuration knob; pre-1.16 firmware answers `ERROR`/timeout, which logs a warning and never blocks startup.
|
||||
|
||||
New protocols SHOULD likewise treat "node was heard" as a first-class, name-optional upsert so peer discovery does not hinge on a roster being populated.
|
||||
|
||||
#### `POST /api/messages`
|
||||
@@ -77,7 +82,10 @@ Single message payload:
|
||||
- Required: `id` (int), `rx_time` (int), `rx_iso` (string)
|
||||
- Identity: `from_id` (string/int), `to_id` (string/int), `channel` (int), `portnum` (string|nil)
|
||||
- Payload: `text` (string|nil), `encrypted` (string|nil), `reply_id` (int|nil), `emoji` (string|nil)
|
||||
- RF: `snr` (float|nil), `rssi` (int|nil), `hop_limit` (int|nil)
|
||||
- RF: `snr` (float|nil), `rssi` (int|nil), `hop_limit` (int|nil), `hops` (int|nil), `path` (string|nil)
|
||||
- `hops` (SPEC RF1) — repeater relays actually travelled, distinct from `hop_limit`'s remaining-budget semantic. MeshCore: the native `path_len` with the `255` "direct" sentinel normalised to `0`. Meshtastic: `hopStart − hopLimit` when both are present, else absent. Additive; absent for legacy senders.
|
||||
- `path` (SPEC RF2) — MeshCore hop-hash route from the library's RX-log⇆message join (`decrypt_channels`): lowercase hex, `path_hash_size`-byte repeater hashes concatenated in travel order (last hash = the repeater heard directly). Absent on a join miss, on RX-log-less firmware, and on direct messages (E2E-encrypted, no join). Stored verbatim; no hash→node resolution is attempted. Additive.
|
||||
- Both fields are serialised back on `GET /api/messages`; neither participates in the dedup fingerprint below (the id derivation is byte-identical to pre-RF releases).
|
||||
- Meta: `channel_name` (string; only when not encrypted and known), `ingestor` (canonical host id), `lora_freq`, `modem_preset`
|
||||
- `protocol` (optional string; `"meshtastic"` or `"meshcore"`) — explicit per-record protocol stamp. Takes precedence over the value inherited from the registered ingestor; values outside the whitelist fall back to the ingestor lookup, then to `"meshtastic"`. Ingestors SHOULD stamp this on every message so the web app classifies senders correctly even before the ingestor heartbeat is processed.
|
||||
|
||||
|
||||
@@ -52,6 +52,8 @@ class MessageEvent(_MessageEventRequired, total=False):
|
||||
snr: float | None
|
||||
rssi: int | None
|
||||
hop_limit: int | None
|
||||
hops: int | None
|
||||
path: str | None
|
||||
reply_id: int | None
|
||||
emoji: str | None
|
||||
channel_name: str
|
||||
|
||||
@@ -243,6 +243,42 @@ def _is_encrypted_flag(value: object) -> bool:
|
||||
return bool(value)
|
||||
|
||||
|
||||
def _hops_travelled(packet: Mapping, hop_limit: object) -> int | None:
|
||||
"""Return the number of repeater hops a message actually travelled.
|
||||
|
||||
Prefers an explicit ``hops`` value stamped by a protocol handler (MeshCore
|
||||
normalizes its native ``path_len`` field this way, including the
|
||||
255-means-direct sentinel). Falls back to the Meshtastic derivation
|
||||
``hopStart - hopLimit``: ``hopStart`` is the sender's initial hop budget
|
||||
and ``hopLimit`` the budget remaining on receipt, so their difference is
|
||||
the relay count (SPEC RF1). Distinct from the stored ``hop_limit`` field,
|
||||
which keeps its remaining-budget semantic unchanged.
|
||||
|
||||
Parameters:
|
||||
packet: Normalized packet mapping possibly carrying ``hops`` and/or
|
||||
``hopStart``/``hop_start``.
|
||||
hop_limit: The already-extracted ``hopLimit``/``hop_limit`` value
|
||||
(passed in so the caller's single extraction is reused).
|
||||
|
||||
Returns:
|
||||
The hop count as an ``int``, or ``None`` when neither source is
|
||||
present or a value cannot be coerced to an integer.
|
||||
"""
|
||||
explicit = _first(packet, "hops", default=None)
|
||||
if explicit is not None:
|
||||
try:
|
||||
return int(explicit)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
hop_start = _first(packet, "hopStart", "hop_start", default=None)
|
||||
if hop_start is None or hop_limit is None:
|
||||
return None
|
||||
try:
|
||||
return int(hop_start) - int(hop_limit)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def upsert_node(node_id: object, node: object) -> None:
|
||||
"""Schedule an upsert for a single node.
|
||||
|
||||
@@ -481,6 +517,13 @@ def store_packet_dict(packet: Mapping) -> None:
|
||||
snr = _first(packet, "snr", "rx_snr", "rxSnr", default=None)
|
||||
rssi = _first(packet, "rssi", "rx_rssi", "rxRssi", default=None)
|
||||
hop = _first(packet, "hopLimit", "hop_limit", default=None)
|
||||
hops = _hops_travelled(packet, hop)
|
||||
# Hop-hash route stamped by the MeshCore handler (RF2); Meshtastic packets
|
||||
# never carry it. Guarded to strings so a malformed value is dropped
|
||||
# rather than serialized as garbage.
|
||||
path = _first(packet, "path", default=None)
|
||||
if not isinstance(path, str):
|
||||
path = None
|
||||
|
||||
to_id_normalized = str(to_id).strip() if to_id is not None else ""
|
||||
|
||||
@@ -538,6 +581,8 @@ def store_packet_dict(packet: Mapping) -> None:
|
||||
"snr": float(snr) if snr is not None else None,
|
||||
"rssi": int(rssi) if rssi is not None else None,
|
||||
"hop_limit": int(hop) if hop is not None else None,
|
||||
"hops": hops,
|
||||
"path": path,
|
||||
"reply_id": reply_id,
|
||||
"emoji": emoji,
|
||||
"ingestor": _state.host_node_id(),
|
||||
|
||||
@@ -78,6 +78,7 @@ from ... import queue as _queue # noqa: E402
|
||||
from ...connection import default_serial_targets # noqa: E402
|
||||
|
||||
from ._constants import ( # noqa: E402 - keep grouped with sibling re-exports.
|
||||
_AUTO_ADD_OVERWRITE_OLDEST,
|
||||
_CHANNEL_PROBE_FALLBACK_MAX,
|
||||
_CONNECT_TIMEOUT_SECS,
|
||||
_DEFAULT_BAUDRATE,
|
||||
@@ -101,6 +102,7 @@ from .decode import ( # noqa: E402
|
||||
_advert_to_node_dict,
|
||||
_contact_to_node_dict,
|
||||
_derive_modem_preset,
|
||||
_rx_advert_to_node_dict,
|
||||
_self_info_to_node_dict,
|
||||
)
|
||||
from .handlers import ( # noqa: E402
|
||||
@@ -120,12 +122,14 @@ from .interface import ClosedBeforeConnectedError, _MeshcoreInterface # noqa: E
|
||||
from .messages import ( # noqa: E402
|
||||
_derive_message_id,
|
||||
_extract_mention_names,
|
||||
_normalize_hops,
|
||||
_normalize_path,
|
||||
_parse_sender_name,
|
||||
_synthetic_node_dict,
|
||||
)
|
||||
from .position import _store_meshcore_position # noqa: E402
|
||||
from .provider import MeshcoreProvider # noqa: E402
|
||||
from .runner import _run_meshcore # noqa: E402
|
||||
from .runner import _ensure_autoadd_eviction, _run_meshcore # noqa: E402
|
||||
|
||||
__all__ = [
|
||||
"BLEConnection",
|
||||
@@ -135,6 +139,7 @@ __all__ = [
|
||||
"MeshcoreProvider",
|
||||
"SerialConnection",
|
||||
"TCPConnection",
|
||||
"_AUTO_ADD_OVERWRITE_OLDEST",
|
||||
"_CHANNEL_PROBE_FALLBACK_MAX",
|
||||
"_CONNECT_TIMEOUT_SECS",
|
||||
"_DEFAULT_BAUDRATE",
|
||||
@@ -150,6 +155,7 @@ __all__ = [
|
||||
"_derive_message_id",
|
||||
"_derive_modem_preset",
|
||||
"_derive_synthetic_node_id",
|
||||
"_ensure_autoadd_eviction",
|
||||
"_ensure_channel_names",
|
||||
"_extract_mention_names",
|
||||
"_log_unhandled_loop_exception",
|
||||
@@ -158,6 +164,8 @@ __all__ = [
|
||||
"_meshcore_adv_type_to_role",
|
||||
"_meshcore_node_id",
|
||||
"_meshcore_short_name",
|
||||
"_normalize_hops",
|
||||
"_normalize_path",
|
||||
"_parse_sender_name",
|
||||
"_process_contact_update",
|
||||
"_process_contacts",
|
||||
@@ -165,6 +173,7 @@ __all__ = [
|
||||
"_pubkey_prefix_to_node_id",
|
||||
"_record_meshcore_message",
|
||||
"_run_meshcore",
|
||||
"_rx_advert_to_node_dict",
|
||||
"_self_info_to_node_dict",
|
||||
"_store_meshcore_position",
|
||||
"_synthetic_node_dict",
|
||||
|
||||
@@ -62,5 +62,23 @@ _MESHCORE_ID_MASK = (1 << _MESHCORE_ID_BITS) - 1
|
||||
# or returns an older firmware version that omits ``max_channels``.
|
||||
_CHANNEL_PROBE_FALLBACK_MAX = 32
|
||||
|
||||
_AUTO_ADD_OVERWRITE_OLDEST = 0x01
|
||||
"""Firmware ``autoadd_config`` bit 0: evict the oldest non-favourite contact
|
||||
when the roster is full (``MyMesh::shouldOverwriteWhenFull``).
|
||||
|
||||
Bits 1–4 of the same byte are per-type auto-add filters (chat / repeater /
|
||||
room-server / sensor) that only apply in manual-add mode; the startup
|
||||
assertion (SPEC RF4) must preserve them via read-modify-write.
|
||||
"""
|
||||
|
||||
_DIRECT_PATH_LEN = 255
|
||||
"""Sentinel ``path_len`` value meaning the packet was received directly.
|
||||
|
||||
The companion protocol encodes a direct (zero-hop) reception as ``0xFF``
|
||||
instead of ``0``; any other value is the masked 6-bit hop count (the
|
||||
``meshcore`` library strips the 2-bit ``path_hash_mode`` prefix before
|
||||
dispatching, so handlers only ever see ``0``–``63`` or this sentinel).
|
||||
"""
|
||||
|
||||
# Matches @[Name] mention patterns in MeshCore message bodies.
|
||||
_MENTION_RE = re.compile(r"@\[([^\]]+)\]")
|
||||
|
||||
@@ -94,6 +94,61 @@ def _advert_to_node_dict(pub_key: str) -> dict:
|
||||
}
|
||||
|
||||
|
||||
def _rx_advert_to_node_dict(frame: dict) -> dict:
|
||||
"""Convert a parsed RX-log ``ADVERT`` frame into a ``POST /api/nodes`` dict.
|
||||
|
||||
Unlike the bare ``ADVERTISEMENT`` push (:func:`_advert_to_node_dict`), an
|
||||
on-air advert heard via ``RX_LOG_DATA`` is fully self-describing: the
|
||||
library's packet parser extracts the advertiser's full public key
|
||||
(``adv_key``), display name, node type, optional position, and the
|
||||
reception-side signal metrics. This restores full node identity — and
|
||||
per-advert SNR / RSSI / hop metrics — independently of the radio's contact
|
||||
roster (SPEC RF3).
|
||||
|
||||
Parameters:
|
||||
frame: Parsed RX-log frame from the ``meshcore`` library. Relevant
|
||||
keys: ``adv_key`` (64-hex public key), ``adv_name``, ``adv_type``
|
||||
(``ADV_TYPE_*``), ``adv_lat``/``adv_lon``, ``recv_time``, ``snr``,
|
||||
``rssi``, and ``path_len`` (hops travelled; RX-log frames carry a
|
||||
plain count, never the 255 message-sync sentinel).
|
||||
|
||||
Returns:
|
||||
Node dict compatible with the ``POST /api/nodes`` payload format,
|
||||
carrying top-level ``snr`` / ``rssi`` / ``hopsAway`` signal fields.
|
||||
``longName`` is included only when the advert carried a name, so a
|
||||
name-less advert never churns an existing richer record.
|
||||
"""
|
||||
pub_key = frame.get("adv_key", "")
|
||||
node_id = _meshcore_node_id(pub_key)
|
||||
name = (frame.get("adv_name") or "").strip()
|
||||
role = _meshcore_adv_type_to_role(frame.get("adv_type"))
|
||||
heard = frame.get("recv_time") or int(time.time())
|
||||
node: dict = {
|
||||
"lastHeard": heard,
|
||||
"protocol": "meshcore",
|
||||
"user": {
|
||||
**({"longName": name} if name else {}),
|
||||
"shortName": _meshcore_short_name(node_id),
|
||||
"publicKey": pub_key,
|
||||
**({"role": role} if role is not None else {}),
|
||||
},
|
||||
}
|
||||
# Reception-side signal metrics: top-level node fields, matching the keys
|
||||
# the web app already accepts (``snr``, ``hopsAway``) plus the additive
|
||||
# ``rssi`` column (RF3).
|
||||
if frame.get("snr") is not None:
|
||||
node["snr"] = frame["snr"]
|
||||
if frame.get("rssi") is not None:
|
||||
node["rssi"] = frame["rssi"]
|
||||
if frame.get("path_len") is not None:
|
||||
node["hopsAway"] = frame["path_len"]
|
||||
lat = frame.get("adv_lat")
|
||||
lon = frame.get("adv_lon")
|
||||
if lat is not None and lon is not None and (lat or lon):
|
||||
node["position"] = {"latitude": lat, "longitude": lon, "time": heard}
|
||||
return node
|
||||
|
||||
|
||||
def _derive_modem_preset(sf: object, bw: object, cr: object) -> str | None:
|
||||
"""Return a compact radio-parameter string from spreading factor, bandwidth, and coding rate.
|
||||
|
||||
|
||||
@@ -16,6 +16,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
import time
|
||||
|
||||
from ... import config, ingestors as _ingestors
|
||||
@@ -23,6 +24,7 @@ from .decode import (
|
||||
_advert_to_node_dict,
|
||||
_contact_to_node_dict,
|
||||
_derive_modem_preset,
|
||||
_rx_advert_to_node_dict,
|
||||
_self_info_to_node_dict,
|
||||
)
|
||||
from .identity import _derive_synthetic_node_id, _meshcore_node_id
|
||||
@@ -30,6 +32,8 @@ from .interface import _MeshcoreInterface
|
||||
from .messages import (
|
||||
_derive_message_id,
|
||||
_extract_mention_names,
|
||||
_normalize_hops,
|
||||
_normalize_path,
|
||||
_parse_sender_name,
|
||||
_synthetic_node_dict,
|
||||
)
|
||||
@@ -281,6 +285,10 @@ def _make_event_handlers(iface: _MeshcoreInterface, target: str | None) -> dict:
|
||||
"channel": channel_idx,
|
||||
"snr": payload.get("SNR"),
|
||||
"rssi": payload.get("RSSI"),
|
||||
"hops": _normalize_hops(payload.get("path_len")),
|
||||
# Injected by the decrypt_channels RX-log join (RF2); absent on a
|
||||
# join miss or RX-log-less firmware, never required.
|
||||
"path": _normalize_path(payload.get("path")),
|
||||
"protocol": "meshcore",
|
||||
"decoded": {
|
||||
"portnum": "TEXT_MESSAGE_APP",
|
||||
@@ -320,6 +328,7 @@ def _make_event_handlers(iface: _MeshcoreInterface, target: str | None) -> dict:
|
||||
"to_id": iface.host_node_id,
|
||||
"channel": 0,
|
||||
"snr": payload.get("SNR"),
|
||||
"hops": _normalize_hops(payload.get("path_len")),
|
||||
"protocol": "meshcore",
|
||||
"decoded": {
|
||||
"portnum": "TEXT_MESSAGE_APP",
|
||||
@@ -330,6 +339,65 @@ def _make_event_handlers(iface: _MeshcoreInterface, target: str | None) -> dict:
|
||||
_handlers._mark_packet_seen()
|
||||
_handlers.store_packet_dict(packet)
|
||||
|
||||
async def on_rx_log_data(evt) -> None:
|
||||
payload = evt.payload or {}
|
||||
if payload.get("payload_typename") != "ADVERT":
|
||||
# Non-ADVERT RF frames keep their DEBUG-only observability, routed
|
||||
# explicitly now that RX_LOG_DATA is a handled event and no longer
|
||||
# reaches the unhandled catch-all (RF3). Resolved via the parent
|
||||
# package so test fakes installed with monkeypatch apply.
|
||||
pkg = sys.modules["data.mesh_ingestor.protocols.meshcore"]
|
||||
pkg._record_meshcore_message(
|
||||
payload, source=f"{target or 'auto'}:RX_LOG_DATA"
|
||||
)
|
||||
return
|
||||
|
||||
pub_key = payload.get("adv_key", "")
|
||||
node_id = _meshcore_node_id(pub_key)
|
||||
if node_id is None:
|
||||
# Malformed advert (absent/short key, parse failure upstream) —
|
||||
# tolerated without raising (RF3).
|
||||
config._debug_log(
|
||||
"Malformed RX-log advert skipped",
|
||||
context="meshcore.rx_advert",
|
||||
severity="warning",
|
||||
)
|
||||
return
|
||||
|
||||
_handlers.upsert_node(node_id, _rx_advert_to_node_dict(payload))
|
||||
lat = payload.get("adv_lat")
|
||||
lon = payload.get("adv_lon")
|
||||
if lat is not None and lon is not None and (lat or lon):
|
||||
_store_meshcore_position(
|
||||
node_id,
|
||||
lat,
|
||||
lon,
|
||||
payload.get("recv_time"),
|
||||
_handlers.host_node_id(),
|
||||
)
|
||||
_handlers._mark_packet_seen()
|
||||
config._debug_log(
|
||||
"MeshCore RX-log advert",
|
||||
context="meshcore.rx_advert",
|
||||
node_id=node_id,
|
||||
name=payload.get("adv_name"),
|
||||
snr=payload.get("snr"),
|
||||
rssi=payload.get("rssi"),
|
||||
hops=payload.get("path_len"),
|
||||
)
|
||||
|
||||
async def on_contact_deleted(evt) -> None:
|
||||
# Deliberate no-op (SPEC RF5): the radio evicting a contact from its
|
||||
# roster (AUTO_ADD_OVERWRITE_OLDEST, RF4) must not delete anything
|
||||
# from the dashboard — the web DB intentionally retains evicted nodes
|
||||
# and ``retention.rb`` stays the only data-expiry authority.
|
||||
payload = evt.payload or {}
|
||||
config._debug_log(
|
||||
"MeshCore contact evicted from radio roster",
|
||||
context="meshcore.contact_deleted",
|
||||
node_id=_meshcore_node_id(payload.get("pubkey", "")),
|
||||
)
|
||||
|
||||
async def on_disconnected(evt) -> None:
|
||||
iface.isConnected = False
|
||||
config._debug_log(
|
||||
@@ -349,5 +417,7 @@ def _make_event_handlers(iface: _MeshcoreInterface, target: str | None) -> dict:
|
||||
"ADVERTISEMENT": on_advertisement,
|
||||
"CHANNEL_MSG_RECV": on_channel_msg,
|
||||
"CONTACT_MSG_RECV": on_contact_msg,
|
||||
"CONTACT_DELETED": on_contact_deleted,
|
||||
"RX_LOG_DATA": on_rx_log_data,
|
||||
"DISCONNECTED": on_disconnected,
|
||||
}
|
||||
|
||||
@@ -19,7 +19,58 @@ from __future__ import annotations
|
||||
import hashlib
|
||||
import time
|
||||
|
||||
from ._constants import _MENTION_RE, _MESHCORE_ID_MASK
|
||||
from ._constants import _DIRECT_PATH_LEN, _MENTION_RE, _MESHCORE_ID_MASK
|
||||
|
||||
|
||||
def _normalize_hops(path_len: object) -> int | None:
|
||||
"""Convert a MeshCore ``path_len`` event field to a hops-travelled count.
|
||||
|
||||
``CONTACT_MSG_RECV`` / ``CHANNEL_MSG_RECV`` payloads carry ``path_len`` as
|
||||
the number of repeater relays the message travelled, except that a direct
|
||||
(zero-hop) reception is encoded as the :data:`~._constants._DIRECT_PATH_LEN`
|
||||
sentinel (``255``) rather than ``0`` (SPEC RF1).
|
||||
|
||||
Parameters:
|
||||
path_len: Raw ``path_len`` value from the event payload, or ``None``
|
||||
when the firmware frame omitted it.
|
||||
|
||||
Returns:
|
||||
``0`` for the direct sentinel, the non-negative hop count otherwise,
|
||||
or ``None`` when the value is absent or unparseable (defensive: the
|
||||
library masks the field to 0–63 or 255, but payloads are untyped).
|
||||
"""
|
||||
if path_len is None:
|
||||
return None
|
||||
try:
|
||||
value = int(path_len)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
if value == _DIRECT_PATH_LEN:
|
||||
return 0
|
||||
if value < 0:
|
||||
return None
|
||||
return value
|
||||
|
||||
|
||||
def _normalize_path(path: object) -> str | None:
|
||||
"""Normalize a MeshCore hop-hash route string for storage.
|
||||
|
||||
The ``decrypt_channels`` RX-log join injects ``path`` into
|
||||
``CHANNEL_MSG_RECV`` payloads as a hex string of concatenated
|
||||
``path_hash_size``-byte repeater hashes in travel order (last hash = the
|
||||
repeater heard directly). Stored verbatim as raw material for a future
|
||||
topology view — no hash→node resolution is attempted (SPEC RF2).
|
||||
|
||||
Parameters:
|
||||
path: Raw ``path`` value from the event payload.
|
||||
|
||||
Returns:
|
||||
The lowercased hex string, or ``None`` when the value is absent,
|
||||
empty, or not a string (defensive: payloads are untyped).
|
||||
"""
|
||||
if not isinstance(path, str) or not path:
|
||||
return None
|
||||
return path.lower()
|
||||
|
||||
|
||||
def _derive_message_id(
|
||||
|
||||
@@ -21,13 +21,77 @@ import sys
|
||||
import threading
|
||||
|
||||
from ... import config
|
||||
from ._constants import _DEFAULT_BAUDRATE
|
||||
from ._constants import _AUTO_ADD_OVERWRITE_OLDEST, _DEFAULT_BAUDRATE
|
||||
from .channels import _ensure_channel_names
|
||||
from .connection import _make_connection
|
||||
from .handlers import _make_event_handlers
|
||||
from .interface import ClosedBeforeConnectedError, _MeshcoreInterface
|
||||
|
||||
|
||||
async def _ensure_autoadd_eviction(mc) -> None:
|
||||
"""Assert the firmware's roster-eviction bit at startup (SPEC RF4).
|
||||
|
||||
Reads the device's ``autoadd_config`` and, **only when** bit ``0x01``
|
||||
(:data:`~._constants._AUTO_ADD_OVERWRITE_OLDEST`) is unset, writes
|
||||
``config | 0x01`` back — a read-modify-write that preserves the
|
||||
type-filter bits 1–4, and a one-byte set so the firmware leaves
|
||||
``autoadd_max_hops`` untouched. When the bit is already set no write is
|
||||
issued: the firmware runs ``savePrefs()`` on every set, so skipping the
|
||||
no-op write avoids a flash write per ingestor restart.
|
||||
|
||||
Unconditional by design (no env/config knob); favourites are never
|
||||
evicted (firmware guarantee) and the setting persists in device flash.
|
||||
A device that does not support commands 58/59 (pre-1.16 firmware)
|
||||
answers with an ``ERROR`` event or times out — both are tolerated by the
|
||||
caller's ``try``/``except`` warning path, mirroring
|
||||
:func:`~.channels._ensure_channel_names`.
|
||||
|
||||
Parameters:
|
||||
mc: Connected ``MeshCore`` instance.
|
||||
"""
|
||||
evt = await mc.commands.get_autoadd_config()
|
||||
payload = getattr(evt, "payload", None) or {}
|
||||
current = payload.get("config")
|
||||
if current is None:
|
||||
# ERROR reply (unsupported command) or malformed payload — leave the
|
||||
# device untouched and surface a warning; startup continues.
|
||||
config._debug_log(
|
||||
"MeshCore autoadd config unavailable; eviction bit not asserted",
|
||||
context="meshcore.autoadd",
|
||||
severity="warning",
|
||||
always=True,
|
||||
)
|
||||
return
|
||||
|
||||
current = int(current)
|
||||
if current & _AUTO_ADD_OVERWRITE_OLDEST:
|
||||
config._debug_log(
|
||||
"MeshCore roster-eviction bit already set",
|
||||
context="meshcore.autoadd",
|
||||
autoadd_config=current,
|
||||
)
|
||||
return
|
||||
|
||||
desired = current | _AUTO_ADD_OVERWRITE_OLDEST
|
||||
set_evt = await mc.commands.set_autoadd_config(desired)
|
||||
if getattr(getattr(set_evt, "type", None), "name", "") == "ERROR":
|
||||
config._debug_log(
|
||||
"MeshCore rejected autoadd eviction config write",
|
||||
context="meshcore.autoadd",
|
||||
severity="warning",
|
||||
always=True,
|
||||
autoadd_config=desired,
|
||||
)
|
||||
return
|
||||
config._debug_log(
|
||||
"MeshCore roster-eviction bit asserted",
|
||||
context="meshcore.autoadd",
|
||||
severity="info",
|
||||
always=True,
|
||||
autoadd_config=desired,
|
||||
)
|
||||
|
||||
|
||||
async def _run_meshcore(
|
||||
iface: _MeshcoreInterface,
|
||||
target: str,
|
||||
@@ -68,6 +132,15 @@ async def _run_meshcore(
|
||||
mc = MeshCore(cx)
|
||||
iface._mc = mc
|
||||
|
||||
# Enable the library's RX-log⇆message join (SPEC RF2): with channel
|
||||
# secrets registered (``_ensure_channel_names`` fetches every channel,
|
||||
# and the reader auto-registers each secret into its packet parser),
|
||||
# the lib matches each CHANNEL_MSG_RECV to its on-air frame and injects
|
||||
# RSSI / path / recv_time. Purely local decryption with keys already
|
||||
# on the radio; a miss (no RX-log frame) simply leaves those fields
|
||||
# absent, so this degrades gracefully on firmware without RX logging.
|
||||
mc.decrypt_channels = True
|
||||
|
||||
handlers_map = _make_event_handlers(iface, target)
|
||||
for event_name, callback in handlers_map.items():
|
||||
mc.subscribe(EventType[event_name], callback)
|
||||
@@ -149,6 +222,20 @@ async def _run_meshcore(
|
||||
error=str(exc),
|
||||
)
|
||||
|
||||
# Assert the roster-eviction bit (RF4) after the readiness signal so a
|
||||
# slow or unsupported command never delays startup; errors and
|
||||
# timeouts are tolerated exactly like the channel-name fetch above.
|
||||
try:
|
||||
await _ensure_autoadd_eviction(mc)
|
||||
except Exception as exc:
|
||||
config._debug_log(
|
||||
"Failed to assert autoadd eviction config",
|
||||
context="meshcore.autoadd",
|
||||
severity="warning",
|
||||
always=True,
|
||||
error=str(exc),
|
||||
)
|
||||
|
||||
await mc.start_auto_message_fetching()
|
||||
|
||||
await stop_event.wait()
|
||||
|
||||
@@ -25,6 +25,8 @@ CREATE TABLE IF NOT EXISTS messages (
|
||||
snr REAL,
|
||||
rssi INTEGER,
|
||||
hop_limit INTEGER,
|
||||
hops INTEGER,
|
||||
path TEXT,
|
||||
lora_freq INTEGER,
|
||||
modem_preset TEXT,
|
||||
channel_name TEXT,
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
-- Copyright © 2025-26 l5yth & contributors
|
||||
--
|
||||
-- Licensed under the Apache License, Version 2.0 (the "License");
|
||||
-- you may not use this file except in compliance with the License.
|
||||
-- You may obtain a copy of the License at
|
||||
--
|
||||
-- http://www.apache.org/licenses/LICENSE-2.0
|
||||
--
|
||||
-- Unless required by applicable law or agreed to in writing, software
|
||||
-- distributed under the License is distributed on an "AS IS" BASIS,
|
||||
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
-- See the License for the specific language governing permissions and
|
||||
-- limitations under the License.
|
||||
|
||||
-- MeshCore RF metrics (SPEC RF1-RF3, RF6): additive columns only.
|
||||
--
|
||||
-- messages.hops: repeater relays actually travelled (MeshCore path_len with
|
||||
-- the 255-direct sentinel normalized to 0; Meshtastic hopStart - hopLimit).
|
||||
-- Distinct from hop_limit, which keeps its remaining-budget semantic.
|
||||
-- messages.path: MeshCore hop-hash route (lowercase hex, path_hash_size-byte
|
||||
-- hashes in travel order; last hash = repeater heard directly).
|
||||
-- nodes.rssi: per-advert reception RSSI (MeshCore RX-log adverts; NULL for
|
||||
-- Meshtastic, which reports no per-node RSSI).
|
||||
--
|
||||
-- The web app applies these conditionally at boot (database.rb); this file is
|
||||
-- the standalone mirror for CLI/manual migration of older installations.
|
||||
|
||||
ALTER TABLE messages ADD COLUMN hops INTEGER;
|
||||
ALTER TABLE messages ADD COLUMN path TEXT;
|
||||
ALTER TABLE nodes ADD COLUMN rssi INTEGER;
|
||||
@@ -27,6 +27,7 @@ CREATE TABLE IF NOT EXISTS nodes (
|
||||
is_favorite BOOLEAN,
|
||||
hops_away INTEGER,
|
||||
snr REAL,
|
||||
rssi INTEGER,
|
||||
last_heard INTEGER,
|
||||
first_heard INTEGER,
|
||||
battery_level REAL,
|
||||
|
||||
@@ -41,6 +41,8 @@ def test_message_event_schema():
|
||||
assert "from_id" in MessageEvent.__optional_keys__
|
||||
assert "snr" in MessageEvent.__optional_keys__
|
||||
assert "rssi" in MessageEvent.__optional_keys__
|
||||
assert "hops" in MessageEvent.__optional_keys__
|
||||
assert "path" in MessageEvent.__optional_keys__
|
||||
|
||||
|
||||
def test_message_event_requires_id_rx_time_rx_iso():
|
||||
|
||||
@@ -1476,3 +1476,133 @@ class TestStorePacketDictPrimaryChannelGuard:
|
||||
|
||||
assert any(path == "/api/messages" for path, _ in sent)
|
||||
assert "non-primary-channel" not in ignored
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# _hops_travelled — hops-travelled derivation (SPEC RF1)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestHopsTravelled:
|
||||
"""Unit tests for :func:`generic._hops_travelled`.
|
||||
|
||||
The helper prefers an explicit handler-stamped ``hops`` value (MeshCore's
|
||||
normalized ``path_len``) and falls back to the Meshtastic derivation
|
||||
``hopStart - hopLimit``; anything unparseable yields ``None``.
|
||||
"""
|
||||
|
||||
def test_explicit_hops_wins_over_derivation(self):
|
||||
"""A handler-stamped hops value beats the hopStart/hopLimit fallback."""
|
||||
assert generic_mod._hops_travelled({"hops": 4, "hopStart": 7}, 2) == 4
|
||||
|
||||
def test_explicit_zero_hops_is_preserved(self):
|
||||
"""hops == 0 (MeshCore direct sentinel already normalized) survives."""
|
||||
assert generic_mod._hops_travelled({"hops": 0, "hopStart": 7}, 2) == 0
|
||||
|
||||
def test_explicit_hops_uncoercible_returns_none(self):
|
||||
"""A non-integer explicit hops yields None, never a crash."""
|
||||
assert generic_mod._hops_travelled({"hops": "bogus"}, 2) is None
|
||||
|
||||
def test_meshtastic_derivation_camel_case(self):
|
||||
"""hopStart 7 with hopLimit 2 remaining means 5 relays travelled."""
|
||||
assert generic_mod._hops_travelled({"hopStart": 7}, 2) == 5
|
||||
|
||||
def test_meshtastic_derivation_snake_case(self):
|
||||
"""The snake_case hop_start alias is accepted too."""
|
||||
assert generic_mod._hops_travelled({"hop_start": 5}, 1) == 4
|
||||
|
||||
def test_missing_hop_start_returns_none(self):
|
||||
"""Without hopStart the derivation cannot run."""
|
||||
assert generic_mod._hops_travelled({}, 2) is None
|
||||
|
||||
def test_missing_hop_limit_returns_none(self):
|
||||
"""Without hopLimit the derivation cannot run."""
|
||||
assert generic_mod._hops_travelled({"hopStart": 3}, None) is None
|
||||
|
||||
def test_uncoercible_fallback_values_return_none(self):
|
||||
"""Unparseable hopStart/hopLimit values yield None, never a crash."""
|
||||
assert generic_mod._hops_travelled({"hopStart": "x"}, 2) is None
|
||||
assert generic_mod._hops_travelled({"hopStart": 7}, "y") is None
|
||||
|
||||
|
||||
class TestStorePacketDictHops:
|
||||
"""``store_packet_dict`` stamps the message payload with ``hops`` (RF1).
|
||||
|
||||
Mirrors the primary-channel-guard harness: capture the queued POST payload
|
||||
and assert the ``hops`` field alongside the untouched ``hop_limit``.
|
||||
"""
|
||||
|
||||
def _store(self, monkeypatch, packet: dict) -> dict:
|
||||
"""Run ``store_packet_dict`` and return the queued message payload."""
|
||||
import data.mesh_ingestor.queue as q
|
||||
|
||||
monkeypatch.setattr(config, "PRIMARY_CHANNEL_ONLY", False)
|
||||
monkeypatch.setattr(config, "ALLOWED_CHANNELS", ())
|
||||
monkeypatch.setattr(config, "HIDDEN_CHANNELS", ())
|
||||
monkeypatch.setattr(config, "DEBUG", False)
|
||||
|
||||
sent = []
|
||||
original = q._queue_post_json
|
||||
q._queue_post_json = lambda path, payload, *, priority, **kw: sent.append(
|
||||
(path, payload)
|
||||
)
|
||||
try:
|
||||
handlers.store_packet_dict(packet)
|
||||
finally:
|
||||
q._queue_post_json = original
|
||||
|
||||
assert len(sent) == 1 and sent[0][0] == "/api/messages"
|
||||
return sent[0][1]
|
||||
|
||||
def _make_packet(self, **extra) -> dict:
|
||||
packet = {
|
||||
"id": 555,
|
||||
"rxTime": 1_700_000_100,
|
||||
"from": "!sender",
|
||||
"to": "^all",
|
||||
"channel": 0,
|
||||
"decoded": {"text": "hops probe", "portnum": 1},
|
||||
}
|
||||
packet.update(extra)
|
||||
return packet
|
||||
|
||||
def test_meshtastic_hops_derived_from_hop_start_and_limit(self, monkeypatch):
|
||||
"""hopStart 7 / hopLimit 2 stores hops 5 with hop_limit untouched."""
|
||||
payload = self._store(monkeypatch, self._make_packet(hopStart=7, hopLimit=2))
|
||||
assert payload["hops"] == 5
|
||||
assert payload["hop_limit"] == 2
|
||||
|
||||
def test_explicit_hops_field_is_stored_verbatim(self, monkeypatch):
|
||||
"""A protocol-handler-stamped hops value (MeshCore) is stored as-is."""
|
||||
payload = self._store(monkeypatch, self._make_packet(hops=0))
|
||||
assert payload["hops"] == 0
|
||||
|
||||
def test_hops_none_when_no_source_present(self, monkeypatch):
|
||||
"""Neither hops nor hopStart present -> hops is None (legacy shape)."""
|
||||
payload = self._store(monkeypatch, self._make_packet(hopLimit=3))
|
||||
assert payload["hops"] is None
|
||||
assert payload["hop_limit"] == 3
|
||||
|
||||
|
||||
class TestStorePacketDictPath:
|
||||
"""``store_packet_dict`` forwards the MeshCore hop-hash route (RF2)."""
|
||||
|
||||
def test_meshcore_path_is_forwarded(self, monkeypatch):
|
||||
"""A handler-stamped path string reaches the queued payload."""
|
||||
harness = TestStorePacketDictHops()
|
||||
payload = harness._store(monkeypatch, harness._make_packet(path="f0bf44b53377"))
|
||||
assert payload["path"] == "f0bf44b53377"
|
||||
|
||||
def test_non_string_path_is_dropped(self, monkeypatch):
|
||||
"""A malformed (non-string) path value is dropped, not serialized."""
|
||||
harness = TestStorePacketDictHops()
|
||||
payload = harness._store(
|
||||
monkeypatch, harness._make_packet(path=["f0bf", "4453"])
|
||||
)
|
||||
assert payload["path"] is None
|
||||
|
||||
def test_path_none_when_absent(self, monkeypatch):
|
||||
"""Meshtastic packets carry no path -> payload path is None."""
|
||||
harness = TestStorePacketDictHops()
|
||||
payload = harness._store(monkeypatch, harness._make_packet())
|
||||
assert payload["path"] is None
|
||||
|
||||
+509
-1
@@ -70,6 +70,8 @@ from data.mesh_ingestor.protocols.meshcore import ( # noqa: E402 - path setup
|
||||
_make_connection,
|
||||
_make_event_handlers,
|
||||
_meshcore_adv_type_to_role,
|
||||
_normalize_hops,
|
||||
_normalize_path,
|
||||
_meshcore_node_id,
|
||||
_meshcore_short_name,
|
||||
_parse_sender_name,
|
||||
@@ -1726,6 +1728,382 @@ def test_on_contact_msg_queues_packet_with_from_id(monkeypatch):
|
||||
)
|
||||
|
||||
|
||||
def test_normalize_hops_maps_path_len_to_hops_travelled():
|
||||
"""_normalize_hops passes counts through, maps the 255 sentinel to 0, and
|
||||
rejects absent/unparseable/negative values (SPEC RF1)."""
|
||||
assert _normalize_hops(None) is None
|
||||
assert _normalize_hops(0) == 0
|
||||
assert _normalize_hops(3) == 3
|
||||
assert _normalize_hops("2") == 2
|
||||
assert _normalize_hops(255) == 0
|
||||
assert _normalize_hops("bogus") is None
|
||||
assert _normalize_hops(-1) is None
|
||||
|
||||
|
||||
def test_on_channel_msg_includes_hops_from_path_len(monkeypatch):
|
||||
"""A channel message with path_len carries the hop count on the packet."""
|
||||
import asyncio
|
||||
|
||||
captured, _upserted, _iface, hmap = _setup_channel_msg_handlers(monkeypatch)
|
||||
asyncio.run(
|
||||
hmap["CHANNEL_MSG_RECV"](
|
||||
_FakeEvt(
|
||||
{
|
||||
"sender_timestamp": 1_758_000_010,
|
||||
"text": "routed message",
|
||||
"channel_idx": 1,
|
||||
"path_len": 3,
|
||||
}
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
assert len(captured) == 1
|
||||
assert captured[0]["hops"] == 3
|
||||
|
||||
|
||||
def test_on_channel_msg_direct_sentinel_yields_zero_hops(monkeypatch):
|
||||
"""The 255 'direct' path_len sentinel normalizes to hops == 0."""
|
||||
import asyncio
|
||||
|
||||
captured, _upserted, _iface, hmap = _setup_channel_msg_handlers(monkeypatch)
|
||||
asyncio.run(
|
||||
hmap["CHANNEL_MSG_RECV"](
|
||||
_FakeEvt(
|
||||
{
|
||||
"sender_timestamp": 1_758_000_011,
|
||||
"text": "direct channel message",
|
||||
"channel_idx": 0,
|
||||
"path_len": 255,
|
||||
}
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
assert len(captured) == 1
|
||||
assert captured[0]["hops"] == 0
|
||||
|
||||
|
||||
def test_on_channel_msg_hops_none_when_path_len_absent(monkeypatch):
|
||||
"""A payload without path_len (older firmware) leaves hops unset."""
|
||||
import asyncio
|
||||
|
||||
captured, _upserted, _iface, hmap = _setup_channel_msg_handlers(monkeypatch)
|
||||
asyncio.run(
|
||||
hmap["CHANNEL_MSG_RECV"](
|
||||
_FakeEvt(
|
||||
{
|
||||
"sender_timestamp": 1_758_000_012,
|
||||
"text": "legacy message",
|
||||
"channel_idx": 0,
|
||||
}
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
assert len(captured) == 1
|
||||
assert captured[0]["hops"] is None
|
||||
|
||||
|
||||
def test_on_contact_msg_includes_hops_from_path_len(monkeypatch):
|
||||
"""Direct messages carry the normalized hop count too (native field, RF1)."""
|
||||
import asyncio
|
||||
import data.mesh_ingestor as _mesh_pkg
|
||||
import data.mesh_ingestor.protocols.meshcore as _mod
|
||||
|
||||
captured: list = []
|
||||
stub = _make_stub_handlers_module()
|
||||
stub.store_packet_dict = lambda pkt: captured.append(pkt)
|
||||
monkeypatch.setattr(_mod.config, "_debug_log", lambda *_a, **_k: None)
|
||||
monkeypatch.setattr(_mesh_pkg, "handlers", stub)
|
||||
|
||||
iface = _MeshcoreInterface(target=None)
|
||||
iface.host_node_id = "!deadbeef"
|
||||
|
||||
hmap = _make_event_handlers(iface, "/dev/ttyUSB0")
|
||||
asyncio.run(
|
||||
hmap["CONTACT_MSG_RECV"](
|
||||
_FakeEvt(
|
||||
{
|
||||
"sender_timestamp": 1_758_000_013,
|
||||
"text": "routed dm",
|
||||
"pubkey_prefix": "aabbccddee11",
|
||||
"path_len": 2,
|
||||
}
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
assert len(captured) == 1
|
||||
assert captured[0]["hops"] == 2
|
||||
|
||||
|
||||
def test_normalize_path_values():
|
||||
"""_normalize_path lowercases hex strings and rejects non-string/empty."""
|
||||
assert _normalize_path("F0BF44B53377") == "f0bf44b53377"
|
||||
assert _normalize_path("aabb") == "aabb"
|
||||
assert _normalize_path("") is None
|
||||
assert _normalize_path(None) is None
|
||||
assert _normalize_path(123) is None
|
||||
|
||||
|
||||
def test_on_channel_msg_includes_path_from_rx_log_join(monkeypatch):
|
||||
"""A channel message with a joined RX-log path stores it lowercased."""
|
||||
import asyncio
|
||||
|
||||
captured, _upserted, _iface, hmap = _setup_channel_msg_handlers(monkeypatch)
|
||||
asyncio.run(
|
||||
hmap["CHANNEL_MSG_RECV"](
|
||||
_FakeEvt(
|
||||
{
|
||||
"sender_timestamp": 1_758_000_014,
|
||||
"text": "joined message",
|
||||
"channel_idx": 0,
|
||||
"path": "F0BF44B53377",
|
||||
"path_len": 3,
|
||||
}
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
assert len(captured) == 1
|
||||
assert captured[0]["path"] == "f0bf44b53377"
|
||||
assert captured[0]["hops"] == 3
|
||||
|
||||
|
||||
def test_on_channel_msg_path_none_on_join_miss_or_invalid(monkeypatch):
|
||||
"""A join miss (absent path) or malformed path leaves the field None."""
|
||||
import asyncio
|
||||
|
||||
captured, _upserted, _iface, hmap = _setup_channel_msg_handlers(monkeypatch)
|
||||
asyncio.run(
|
||||
hmap["CHANNEL_MSG_RECV"](
|
||||
_FakeEvt(
|
||||
{
|
||||
"sender_timestamp": 1_758_000_015,
|
||||
"text": "no join",
|
||||
"channel_idx": 0,
|
||||
}
|
||||
)
|
||||
)
|
||||
)
|
||||
asyncio.run(
|
||||
hmap["CHANNEL_MSG_RECV"](
|
||||
_FakeEvt(
|
||||
{
|
||||
"sender_timestamp": 1_758_000_016,
|
||||
"text": "bad join",
|
||||
"channel_idx": 0,
|
||||
"path": 4711,
|
||||
}
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
assert len(captured) == 2
|
||||
assert captured[0]["path"] is None
|
||||
assert captured[1]["path"] is None
|
||||
|
||||
|
||||
def test_on_contact_deleted_is_debug_logged_no_op(monkeypatch):
|
||||
"""CONTACT_DELETED must log the evicted node and touch nothing else (RF5)."""
|
||||
import asyncio
|
||||
import data.mesh_ingestor.protocols.meshcore as _mod
|
||||
|
||||
captured, upserted, _iface, hmap = _setup_channel_msg_handlers(monkeypatch)
|
||||
logs: list = []
|
||||
monkeypatch.setattr(
|
||||
_mod.config, "_debug_log", lambda msg, **kw: logs.append((msg, kw))
|
||||
)
|
||||
|
||||
assert "CONTACT_DELETED" in hmap
|
||||
asyncio.run(hmap["CONTACT_DELETED"](_FakeEvt({"pubkey": "aabbccdd" + "00" * 28})))
|
||||
|
||||
# No packet stored, no node upserted, no exception — just the debug line.
|
||||
assert captured == []
|
||||
assert upserted == []
|
||||
assert any(
|
||||
kw.get("context") == "meshcore.contact_deleted"
|
||||
and kw.get("node_id") == "!aabbccdd"
|
||||
for _msg, kw in logs
|
||||
)
|
||||
|
||||
|
||||
def test_on_contact_deleted_tolerates_empty_payload(monkeypatch):
|
||||
"""A CONTACT_DELETED push with no payload must not raise."""
|
||||
import asyncio
|
||||
|
||||
captured, upserted, _iface, hmap = _setup_channel_msg_handlers(monkeypatch)
|
||||
asyncio.run(hmap["CONTACT_DELETED"](_FakeEvt(None)))
|
||||
|
||||
assert captured == []
|
||||
assert upserted == []
|
||||
|
||||
|
||||
def test_rx_advert_to_node_dict_full_frame():
|
||||
"""A fully-populated RX-log ADVERT frame maps every field (RF3)."""
|
||||
from data.mesh_ingestor.protocols.meshcore import _rx_advert_to_node_dict
|
||||
|
||||
pub_key = "511617e3" + "00" * 28
|
||||
node = _rx_advert_to_node_dict(
|
||||
{
|
||||
"adv_key": pub_key,
|
||||
"adv_name": "BER Drachentoeter",
|
||||
"adv_type": 2,
|
||||
"adv_lat": 52.516274,
|
||||
"adv_lon": 13.405612,
|
||||
"recv_time": 1_758_000_020,
|
||||
"snr": 12.0,
|
||||
"rssi": -69,
|
||||
"path_len": 2,
|
||||
}
|
||||
)
|
||||
|
||||
assert node["lastHeard"] == 1_758_000_020
|
||||
assert node["protocol"] == "meshcore"
|
||||
assert node["user"]["longName"] == "BER Drachentoeter"
|
||||
assert node["user"]["publicKey"] == pub_key
|
||||
assert node["user"]["role"] == "REPEATER"
|
||||
assert node["snr"] == 12.0
|
||||
assert node["rssi"] == -69
|
||||
assert node["hopsAway"] == 2
|
||||
assert node["position"] == {
|
||||
"latitude": 52.516274,
|
||||
"longitude": 13.405612,
|
||||
"time": 1_758_000_020,
|
||||
}
|
||||
|
||||
|
||||
def test_rx_advert_to_node_dict_minimal_frame():
|
||||
"""A name-less, position-less advert omits those keys instead of churning."""
|
||||
from data.mesh_ingestor.protocols.meshcore import _rx_advert_to_node_dict
|
||||
|
||||
pub_key = "aabbccdd" + "00" * 28
|
||||
node = _rx_advert_to_node_dict({"adv_key": pub_key, "path_len": 0})
|
||||
|
||||
assert node["user"]["publicKey"] == pub_key
|
||||
assert "longName" not in node["user"]
|
||||
assert "role" not in node["user"]
|
||||
assert "position" not in node
|
||||
assert "snr" not in node and "rssi" not in node
|
||||
# A zero-hop (directly heard) advert still records hopsAway == 0.
|
||||
assert node["hopsAway"] == 0
|
||||
assert isinstance(node["lastHeard"], int)
|
||||
|
||||
|
||||
def test_on_rx_log_data_advert_upserts_node_and_position(monkeypatch):
|
||||
"""An RX-log ADVERT frame upserts the full node and stores its position."""
|
||||
import asyncio
|
||||
import data.mesh_ingestor.protocols.meshcore.handlers as _handlers_mod
|
||||
|
||||
captured, upserted, _iface, hmap = _setup_channel_msg_handlers(monkeypatch)
|
||||
positions: list = []
|
||||
monkeypatch.setattr(
|
||||
_handlers_mod,
|
||||
"_store_meshcore_position",
|
||||
lambda *args: positions.append(args),
|
||||
)
|
||||
|
||||
pub_key = "511617e3" + "00" * 28
|
||||
asyncio.run(
|
||||
hmap["RX_LOG_DATA"](
|
||||
_FakeEvt(
|
||||
{
|
||||
"payload_typename": "ADVERT",
|
||||
"adv_key": pub_key,
|
||||
"adv_name": "BER Drachentoeter",
|
||||
"adv_type": 2,
|
||||
"adv_lat": 52.516274,
|
||||
"adv_lon": 13.405612,
|
||||
"recv_time": 1_758_000_021,
|
||||
"snr": 11.5,
|
||||
"rssi": -70,
|
||||
"path_len": 3,
|
||||
}
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
assert captured == [] # adverts never produce message packets
|
||||
assert len(upserted) == 1
|
||||
node_id, node = upserted[0]
|
||||
assert node_id == "!511617e3"
|
||||
assert node["user"]["longName"] == "BER Drachentoeter"
|
||||
assert node["snr"] == 11.5 and node["rssi"] == -70 and node["hopsAway"] == 3
|
||||
assert len(positions) == 1
|
||||
assert positions[0][0] == "!511617e3"
|
||||
assert positions[0][1] == 52.516274 and positions[0][2] == 13.405612
|
||||
|
||||
|
||||
def test_on_rx_log_data_advert_without_position_skips_position_store(monkeypatch):
|
||||
"""No adv_lat/adv_lon on the advert -> node upsert only, no position POST."""
|
||||
import asyncio
|
||||
import data.mesh_ingestor.protocols.meshcore.handlers as _handlers_mod
|
||||
|
||||
_captured, upserted, _iface, hmap = _setup_channel_msg_handlers(monkeypatch)
|
||||
positions: list = []
|
||||
monkeypatch.setattr(
|
||||
_handlers_mod,
|
||||
"_store_meshcore_position",
|
||||
lambda *args: positions.append(args),
|
||||
)
|
||||
|
||||
asyncio.run(
|
||||
hmap["RX_LOG_DATA"](
|
||||
_FakeEvt(
|
||||
{
|
||||
"payload_typename": "ADVERT",
|
||||
"adv_key": "aabbccdd" + "00" * 28,
|
||||
"snr": 4.0,
|
||||
}
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
assert len(upserted) == 1
|
||||
assert positions == []
|
||||
|
||||
|
||||
def test_on_rx_log_data_non_advert_routes_to_debug_capture(monkeypatch):
|
||||
"""Non-ADVERT RF frames go to the DEBUG-only capture, never upsert (RF3)."""
|
||||
import asyncio
|
||||
import data.mesh_ingestor.protocols.meshcore as _mod
|
||||
|
||||
captured, upserted, _iface, hmap = _setup_channel_msg_handlers(monkeypatch)
|
||||
recorded: list = []
|
||||
monkeypatch.setattr(
|
||||
_mod,
|
||||
"_record_meshcore_message",
|
||||
lambda message, *, source: recorded.append((message, source)),
|
||||
)
|
||||
|
||||
asyncio.run(
|
||||
hmap["RX_LOG_DATA"](
|
||||
_FakeEvt({"payload_typename": "GRP_TXT", "snr": 1.0, "rssi": -90})
|
||||
)
|
||||
)
|
||||
|
||||
assert captured == [] and upserted == []
|
||||
assert len(recorded) == 1
|
||||
message, source = recorded[0]
|
||||
assert message["payload_typename"] == "GRP_TXT"
|
||||
assert source.endswith(":RX_LOG_DATA")
|
||||
|
||||
|
||||
def test_on_rx_log_data_malformed_advert_tolerated(monkeypatch):
|
||||
"""A short/absent adv_key is skipped without raising (RF3)."""
|
||||
import asyncio
|
||||
|
||||
captured, upserted, _iface, hmap = _setup_channel_msg_handlers(monkeypatch)
|
||||
|
||||
asyncio.run(
|
||||
hmap["RX_LOG_DATA"](_FakeEvt({"payload_typename": "ADVERT", "adv_key": "ab"}))
|
||||
)
|
||||
asyncio.run(hmap["RX_LOG_DATA"](_FakeEvt({"payload_typename": "ADVERT"})))
|
||||
|
||||
assert captured == [] and upserted == []
|
||||
|
||||
|
||||
def test_on_channel_msg_id_identical_across_ingestors_with_different_rosters(
|
||||
monkeypatch,
|
||||
):
|
||||
@@ -2917,6 +3295,9 @@ def _make_fake_meshcore_mod(
|
||||
disconnect_raises: bool = False,
|
||||
connect_stall_event=None,
|
||||
on_ensure_contacts=None,
|
||||
autoadd_config: int | None = 0x1F,
|
||||
autoadd_get_raises: bool = False,
|
||||
autoadd_set_error: bool = False,
|
||||
):
|
||||
"""Build a minimal fake ``meshcore`` module for testing :func:`_run_meshcore`.
|
||||
|
||||
@@ -2947,6 +3328,8 @@ def _make_fake_meshcore_mod(
|
||||
"CHANNEL_MSG_RECV",
|
||||
"CONTACT_MSG_RECV",
|
||||
"ADVERTISEMENT",
|
||||
"CONTACT_DELETED",
|
||||
"RX_LOG_DATA",
|
||||
"DISCONNECTED",
|
||||
"CONNECTED",
|
||||
"ACK",
|
||||
@@ -2961,6 +3344,11 @@ def _make_fake_meshcore_mod(
|
||||
)
|
||||
|
||||
class _FakeCommands:
|
||||
def __init__(self):
|
||||
# Records every set_autoadd_config write so RF4 tests can assert
|
||||
# the read-modify-write / skip-when-set behavior.
|
||||
self.autoadd_set_calls: list[int] = []
|
||||
|
||||
async def send_device_query(self):
|
||||
# Return minimal DEVICE_INFO — channel probing is not under test here.
|
||||
return types.SimpleNamespace(
|
||||
@@ -2971,6 +3359,24 @@ def _make_fake_meshcore_mod(
|
||||
# Return ERROR for all channels — channel probing is not under test here.
|
||||
return types.SimpleNamespace(type=EventType.ERROR, payload={})
|
||||
|
||||
async def get_autoadd_config(self):
|
||||
# ``autoadd_config=None`` simulates pre-1.16 firmware: an ERROR
|
||||
# reply whose payload carries no ``config`` key.
|
||||
if autoadd_get_raises:
|
||||
raise TimeoutError("autoadd query timed out")
|
||||
if autoadd_config is None:
|
||||
return types.SimpleNamespace(type=EventType.ERROR, payload={})
|
||||
return types.SimpleNamespace(
|
||||
type=EventType.CHANNEL_INFO, # any non-ERROR type
|
||||
payload={"config": autoadd_config},
|
||||
)
|
||||
|
||||
async def set_autoadd_config(self, flag):
|
||||
self.autoadd_set_calls.append(flag)
|
||||
if autoadd_set_error:
|
||||
return types.SimpleNamespace(type=EventType.ERROR, payload={})
|
||||
return types.SimpleNamespace(type=EventType.OK, payload={})
|
||||
|
||||
class _FakeMeshCore:
|
||||
def __init__(self, cx):
|
||||
self._catch_all = None
|
||||
@@ -2978,6 +3384,9 @@ def _make_fake_meshcore_mod(
|
||||
# Mirrors the upstream property the runner flips on to keep the
|
||||
# contact roster live across re-adverts (meshcore adverts gap).
|
||||
self.auto_update_contacts = False
|
||||
# Mirrors the upstream property enabling the RX-log⇆message join
|
||||
# (SPEC RF2); the runner must flip it on before connecting.
|
||||
self.decrypt_channels = False
|
||||
# Records every non-catch-all subscription so tests can assert the
|
||||
# runner wires the ADVERTISEMENT handler.
|
||||
self.subscribed_events = []
|
||||
@@ -3511,7 +3920,9 @@ def test_on_advertisement_ignores_unmappable_pubkey(monkeypatch):
|
||||
|
||||
|
||||
def test_run_meshcore_enables_auto_update_and_subscribes_advert(monkeypatch):
|
||||
"""_run_meshcore must enable contact auto-update and subscribe the advert handler."""
|
||||
"""_run_meshcore must enable contact auto-update, enable the RX-log join
|
||||
(decrypt_channels, RF2), and subscribe the advert + contact-deleted
|
||||
handlers."""
|
||||
import asyncio
|
||||
import data.mesh_ingestor.protocols.meshcore as _mod
|
||||
|
||||
@@ -3526,4 +3937,101 @@ def test_run_meshcore_enables_auto_update_and_subscribes_advert(monkeypatch):
|
||||
|
||||
assert error_holder[0] is None
|
||||
assert iface._mc.auto_update_contacts is True
|
||||
assert iface._mc.decrypt_channels is True
|
||||
assert fake_mod.EventType.ADVERTISEMENT in iface._mc.subscribed_events
|
||||
assert fake_mod.EventType.CONTACT_DELETED in iface._mc.subscribed_events
|
||||
assert fake_mod.EventType.RX_LOG_DATA in iface._mc.subscribed_events
|
||||
|
||||
|
||||
def _run_meshcore_with_autoadd(monkeypatch, **factory_kwargs):
|
||||
"""Drive ``_run_meshcore`` with a fake lib and return ``(iface, error_holder, logs)``.
|
||||
|
||||
Shared harness for the RF4 roster-eviction assertion tests: captures
|
||||
``config._debug_log`` calls so tests can assert the warning/info paths.
|
||||
"""
|
||||
import asyncio
|
||||
import data.mesh_ingestor.protocols.meshcore as _mod
|
||||
|
||||
logs: list = []
|
||||
monkeypatch.setattr(
|
||||
_mod.config, "_debug_log", lambda msg, **kw: logs.append((msg, kw))
|
||||
)
|
||||
fake_mod = _make_fake_meshcore_mod(**factory_kwargs)
|
||||
_patch_meshcore_mod(monkeypatch, _mod, fake_mod)
|
||||
|
||||
iface = _MeshcoreInterface(target=None)
|
||||
connected, error_holder = asyncio.run(
|
||||
_run_until_connected(iface, "/dev/ttyUSB0", fake_mod, _mod)
|
||||
)
|
||||
assert connected.is_set()
|
||||
return iface, error_holder, logs
|
||||
|
||||
|
||||
def test_run_meshcore_asserts_eviction_bit_when_unset(monkeypatch):
|
||||
"""Bit 0x01 unset -> exactly one read-modify-write set preserving bits 1-4."""
|
||||
iface, error_holder, logs = _run_meshcore_with_autoadd(
|
||||
monkeypatch, autoadd_config=0x1E
|
||||
)
|
||||
|
||||
assert error_holder[0] is None
|
||||
assert iface._mc.commands.autoadd_set_calls == [0x1F]
|
||||
assert any(
|
||||
kw.get("context") == "meshcore.autoadd" and kw.get("autoadd_config") == 0x1F
|
||||
for _msg, kw in logs
|
||||
)
|
||||
|
||||
|
||||
def test_run_meshcore_skips_autoadd_write_when_bit_already_set(monkeypatch):
|
||||
"""Bit 0x01 already set -> no set call (no savePrefs flash write)."""
|
||||
iface, error_holder, _logs = _run_meshcore_with_autoadd(
|
||||
monkeypatch, autoadd_config=0x1F
|
||||
)
|
||||
|
||||
assert error_holder[0] is None
|
||||
assert iface._mc.commands.autoadd_set_calls == []
|
||||
|
||||
|
||||
def test_run_meshcore_autoadd_unsupported_firmware_continues(monkeypatch):
|
||||
"""Pre-1.16 firmware (ERROR reply, no config) -> warning, startup continues."""
|
||||
iface, error_holder, logs = _run_meshcore_with_autoadd(
|
||||
monkeypatch, autoadd_config=None
|
||||
)
|
||||
|
||||
assert error_holder[0] is None
|
||||
assert iface._mc.commands.autoadd_set_calls == []
|
||||
assert any(
|
||||
kw.get("context") == "meshcore.autoadd" and kw.get("severity") == "warning"
|
||||
for _msg, kw in logs
|
||||
)
|
||||
|
||||
|
||||
def test_run_meshcore_autoadd_query_timeout_continues(monkeypatch):
|
||||
"""A raising/timing-out query is swallowed with a warning; startup continues."""
|
||||
iface, error_holder, logs = _run_meshcore_with_autoadd(
|
||||
monkeypatch, autoadd_get_raises=True
|
||||
)
|
||||
|
||||
assert error_holder[0] is None
|
||||
assert iface._mc.commands.autoadd_set_calls == []
|
||||
assert any(
|
||||
kw.get("context") == "meshcore.autoadd"
|
||||
and kw.get("severity") == "warning"
|
||||
and "timed out" in str(kw.get("error", ""))
|
||||
for _msg, kw in logs
|
||||
)
|
||||
|
||||
|
||||
def test_run_meshcore_autoadd_set_rejected_logs_warning(monkeypatch):
|
||||
"""An ERROR reply to the set is logged as a warning; startup continues."""
|
||||
iface, error_holder, logs = _run_meshcore_with_autoadd(
|
||||
monkeypatch, autoadd_config=0x00, autoadd_set_error=True
|
||||
)
|
||||
|
||||
assert error_holder[0] is None
|
||||
assert iface._mc.commands.autoadd_set_calls == [0x01]
|
||||
assert any(
|
||||
kw.get("context") == "meshcore.autoadd"
|
||||
and kw.get("severity") == "warning"
|
||||
and kw.get("autoadd_config") == 0x01
|
||||
for _msg, kw in logs
|
||||
)
|
||||
|
||||
@@ -181,6 +181,10 @@ module PotatoMesh
|
||||
emoji = string_or_nil(message["emoji"])
|
||||
ingestor = string_or_nil(message["ingestor"])
|
||||
protocol = resolve_record_protocol(db, message, ingestor, cache: protocol_cache)
|
||||
# RF metrics (SPEC RF1/RF2): hops actually travelled and the MeshCore
|
||||
# hop-hash route; both additive and absent for legacy senders.
|
||||
hops = coerce_integer(message["hops"])
|
||||
path = string_or_nil(message["path"])
|
||||
|
||||
row = [
|
||||
msg_id,
|
||||
@@ -195,6 +199,8 @@ module PotatoMesh
|
||||
message["snr"],
|
||||
message["rssi"],
|
||||
message["hop_limit"],
|
||||
hops,
|
||||
path,
|
||||
lora_freq,
|
||||
modem_preset,
|
||||
channel_name,
|
||||
@@ -312,6 +318,8 @@ module PotatoMesh
|
||||
updates["snr"] = message["snr"] if message.key?("snr")
|
||||
updates["rssi"] = message["rssi"] if message.key?("rssi")
|
||||
updates["hop_limit"] = message["hop_limit"] if message.key?("hop_limit")
|
||||
updates["hops"] = hops unless hops.nil?
|
||||
updates["path"] = path if path
|
||||
updates["lora_freq"] = lora_freq unless lora_freq.nil?
|
||||
updates["modem_preset"] = modem_preset if modem_preset
|
||||
updates["channel_name"] = channel_name if channel_name
|
||||
@@ -381,8 +389,8 @@ module PotatoMesh
|
||||
|
||||
begin
|
||||
db.execute <<~SQL, row
|
||||
INSERT INTO messages(id,rx_time,rx_iso,from_id,to_id,channel,portnum,text,encrypted,snr,rssi,hop_limit,lora_freq,modem_preset,channel_name,reply_id,emoji,ingestor,protocol)
|
||||
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
|
||||
INSERT INTO messages(id,rx_time,rx_iso,from_id,to_id,channel,portnum,text,encrypted,snr,rssi,hop_limit,hops,path,lora_freq,modem_preset,channel_name,reply_id,emoji,ingestor,protocol)
|
||||
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
|
||||
SQL
|
||||
rescue SQLite3::ConstraintException
|
||||
existing_row = db.get_first_row(
|
||||
@@ -413,6 +421,8 @@ module PotatoMesh
|
||||
fallback_updates["snr"] = message["snr"] if message.key?("snr")
|
||||
fallback_updates["rssi"] = message["rssi"] if message.key?("rssi")
|
||||
fallback_updates["hop_limit"] = message["hop_limit"] if message.key?("hop_limit")
|
||||
fallback_updates["hops"] = hops unless hops.nil?
|
||||
fallback_updates["path"] = path if path
|
||||
fallback_updates["portnum"] = portnum if portnum
|
||||
fallback_updates["lora_freq"] = lora_freq unless lora_freq.nil?
|
||||
fallback_updates["modem_preset"] = modem_preset if modem_preset
|
||||
|
||||
@@ -268,6 +268,7 @@ module PotatoMesh
|
||||
coerce_bool(pick_alias(n, "isFavorite", "is_favorite")),
|
||||
pick_alias(n, "hopsAway", "hops_away"),
|
||||
n["snr"],
|
||||
n["rssi"],
|
||||
lh,
|
||||
lh,
|
||||
pick_alias(met, "batteryLevel", "battery_level"),
|
||||
@@ -294,9 +295,9 @@ module PotatoMesh
|
||||
db.transaction do
|
||||
db.execute(<<~SQL, row)
|
||||
INSERT INTO nodes(node_id,num,short_name,long_name,macaddr,hw_model,role,public_key,is_unmessagable,is_favorite,
|
||||
hops_away,snr,last_heard,first_heard,battery_level,voltage,channel_utilization,air_util_tx,uptime_seconds,
|
||||
hops_away,snr,rssi,last_heard,first_heard,battery_level,voltage,channel_utilization,air_util_tx,uptime_seconds,
|
||||
position_time,location_source,precision_bits,latitude,longitude,altitude,lora_freq,modem_preset,protocol,synthetic)
|
||||
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
|
||||
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
|
||||
ON CONFLICT(node_id) DO UPDATE SET
|
||||
num=COALESCE(excluded.num, nodes.num),
|
||||
short_name=COALESCE(excluded.short_name, nodes.short_name),
|
||||
@@ -307,6 +308,7 @@ module PotatoMesh
|
||||
public_key=COALESCE(excluded.public_key, nodes.public_key),
|
||||
is_unmessagable=COALESCE(excluded.is_unmessagable, nodes.is_unmessagable),
|
||||
is_favorite=excluded.is_favorite, hops_away=excluded.hops_away, snr=excluded.snr, last_heard=excluded.last_heard,
|
||||
rssi=COALESCE(excluded.rssi, nodes.rssi),
|
||||
first_heard=COALESCE(nodes.first_heard, excluded.first_heard, excluded.last_heard),
|
||||
battery_level=excluded.battery_level, voltage=excluded.voltage, channel_utilization=excluded.channel_utilization,
|
||||
air_util_tx=excluded.air_util_tx, uptime_seconds=excluded.uptime_seconds,
|
||||
|
||||
@@ -152,6 +152,12 @@ module PotatoMesh
|
||||
db.execute("ALTER TABLE nodes ADD COLUMN synthetic BOOLEAN NOT NULL DEFAULT 0")
|
||||
end
|
||||
|
||||
# RF metrics (SPEC RF3/RF6): per-advert reception RSSI. NULL for
|
||||
# Meshtastic nodes, which report no per-node RSSI.
|
||||
unless node_columns.include?("rssi")
|
||||
db.execute("ALTER TABLE nodes ADD COLUMN rssi INTEGER")
|
||||
end
|
||||
|
||||
if node_columns.include?("long_name")
|
||||
existing_indexes = db.execute("SELECT name FROM sqlite_master WHERE type='index' AND tbl_name='nodes'").flatten
|
||||
unless existing_indexes.include?("idx_nodes_long_name")
|
||||
@@ -259,6 +265,17 @@ module PotatoMesh
|
||||
db.execute("UPDATE messages SET protocol = 'meshtastic' WHERE protocol IS NULL OR TRIM(protocol) = ''")
|
||||
end
|
||||
|
||||
# RF metrics (SPEC RF1/RF2/RF6): hops actually travelled (distinct
|
||||
# from hop_limit's remaining-budget semantic) and the MeshCore
|
||||
# hop-hash route. Both additive, NULL for legacy rows.
|
||||
unless message_columns.include?("hops")
|
||||
db.execute("ALTER TABLE messages ADD COLUMN hops INTEGER")
|
||||
end
|
||||
|
||||
unless message_columns.include?("path")
|
||||
db.execute("ALTER TABLE messages ADD COLUMN path TEXT")
|
||||
end
|
||||
|
||||
reply_index_exists =
|
||||
db.get_first_value(
|
||||
"SELECT COUNT(*) FROM sqlite_master WHERE type='index' AND name='idx_messages_reply_id'",
|
||||
|
||||
@@ -71,6 +71,7 @@ module PotatoMesh
|
||||
sql = <<~SQL
|
||||
SELECT m.id, m.rx_time, m.rx_iso, m.from_id, m.to_id, m.channel,
|
||||
m.portnum, m.text, m.encrypted, m.rssi, m.hop_limit,
|
||||
m.hops, m.path,
|
||||
m.lora_freq, m.modem_preset, m.channel_name, m.snr,
|
||||
m.reply_id, m.emoji, m.ingestor, m.protocol
|
||||
FROM messages m
|
||||
|
||||
@@ -177,6 +177,7 @@ module PotatoMesh
|
||||
|
||||
sql = <<~SQL
|
||||
SELECT node_id, short_name, long_name, hw_model, role, snr,
|
||||
rssi, hops_away,
|
||||
battery_level, voltage, last_heard, first_heard,
|
||||
uptime_seconds, channel_utilization, air_util_tx,
|
||||
position_time, location_source, precision_bits,
|
||||
|
||||
@@ -0,0 +1,212 @@
|
||||
# Copyright © 2025-26 l5yth & contributors
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
# frozen_string_literal: true
|
||||
|
||||
require "spec_helper"
|
||||
require "sqlite3"
|
||||
require "json"
|
||||
|
||||
# RF-metrics ingest/read-side coverage (SPEC RF1-RF3, RF6; ACCEPTANCE RF-A1 -
|
||||
# RF-A3): the additive `messages.hops` / `messages.path` / `nodes.rssi`
|
||||
# columns round-trip through the authenticated POST ingest routes and surface
|
||||
# on the GET collection responses, while legacy payloads without the fields
|
||||
# keep storing NULL.
|
||||
RSpec.describe "RF metrics (hops/path/rssi) ingest and read-side" do
|
||||
let(:app) { Sinatra::Application }
|
||||
let(:api_token) { "spec-token" }
|
||||
let(:auth_headers) do
|
||||
{
|
||||
"CONTENT_TYPE" => "application/json",
|
||||
"HTTP_AUTHORIZATION" => "Bearer #{api_token}",
|
||||
}
|
||||
end
|
||||
|
||||
# Execute the provided block with a configured SQLite connection.
|
||||
#
|
||||
# @yieldparam db [SQLite3::Database] open database handle.
|
||||
# @return [void]
|
||||
def with_db
|
||||
db = SQLite3::Database.new(PotatoMesh::Config.db_path)
|
||||
db.busy_timeout = PotatoMesh::Config.db_busy_timeout_ms
|
||||
yield db
|
||||
ensure
|
||||
db&.close
|
||||
end
|
||||
|
||||
# Remove the rows this spec writes so examples stay independent.
|
||||
#
|
||||
# @return [void]
|
||||
def clear_rf_tables
|
||||
with_db do |db|
|
||||
db.execute("DELETE FROM messages")
|
||||
db.execute("DELETE FROM nodes")
|
||||
db.execute("DELETE FROM positions")
|
||||
end
|
||||
end
|
||||
|
||||
before do
|
||||
@original_token = ENV["API_TOKEN"]
|
||||
ENV["API_TOKEN"] = api_token
|
||||
clear_rf_tables
|
||||
PotatoMesh::App::ApiCache.invalidate_all
|
||||
end
|
||||
|
||||
after do
|
||||
if @original_token.nil?
|
||||
ENV.delete("API_TOKEN")
|
||||
else
|
||||
ENV["API_TOKEN"] = @original_token
|
||||
end
|
||||
end
|
||||
|
||||
# Build a minimal valid message payload the ingest route accepts.
|
||||
#
|
||||
# @param overrides [Hash] extra/overriding message fields.
|
||||
# @return [Hash] POST /api/messages payload.
|
||||
def message_payload(overrides = {})
|
||||
now = Time.now.to_i
|
||||
{
|
||||
"id" => 424_242,
|
||||
"rx_time" => now,
|
||||
"rx_iso" => Time.at(now).utc.iso8601,
|
||||
"from_id" => "!aabbccdd",
|
||||
"to_id" => "^all",
|
||||
"channel" => 0,
|
||||
"portnum" => "TEXT_MESSAGE_APP",
|
||||
"text" => "rf metrics probe",
|
||||
"snr" => 5.5,
|
||||
"rssi" => -80,
|
||||
"hop_limit" => 2,
|
||||
}.merge(overrides)
|
||||
end
|
||||
|
||||
describe "message hops" do
|
||||
it "stores hops on POST /api/messages and serves it on GET /api/messages" do
|
||||
post "/api/messages", message_payload("hops" => 5).to_json, auth_headers
|
||||
expect(last_response.status).to eq(201)
|
||||
|
||||
get "/api/messages"
|
||||
expect(last_response).to be_ok
|
||||
row = JSON.parse(last_response.body).find { |m| m["id"] == 424_242 }
|
||||
expect(row).not_to be_nil
|
||||
expect(row["hops"]).to eq(5)
|
||||
# hop_limit keeps its remaining-budget semantic untouched (RF1).
|
||||
expect(row["hop_limit"]).to eq(2)
|
||||
end
|
||||
|
||||
it "stores NULL hops for legacy messages without the field" do
|
||||
post "/api/messages", message_payload.to_json, auth_headers
|
||||
expect(last_response.status).to eq(201)
|
||||
|
||||
get "/api/messages"
|
||||
row = JSON.parse(last_response.body).find { |m| m["id"] == 424_242 }
|
||||
expect(row).not_to be_nil
|
||||
expect(row["hops"]).to be_nil
|
||||
end
|
||||
end
|
||||
|
||||
describe "message path" do
|
||||
it "stores the meshcore hop-hash path and serves it on GET /api/messages" do
|
||||
payload = message_payload(
|
||||
"id" => 424_243,
|
||||
"protocol" => "meshcore",
|
||||
"hops" => 3,
|
||||
"path" => "f0bf44b53377",
|
||||
)
|
||||
post "/api/messages", payload.to_json, auth_headers
|
||||
expect(last_response.status).to eq(201)
|
||||
|
||||
get "/api/messages"
|
||||
row = JSON.parse(last_response.body).find { |m| m["id"] == 424_243 }
|
||||
expect(row).not_to be_nil
|
||||
expect(row["path"]).to eq("f0bf44b53377")
|
||||
expect(row["hops"]).to eq(3)
|
||||
end
|
||||
|
||||
it "stores NULL path when the field is absent (join miss / Meshtastic)" do
|
||||
post "/api/messages", message_payload("id" => 424_244).to_json, auth_headers
|
||||
expect(last_response.status).to eq(201)
|
||||
|
||||
get "/api/messages"
|
||||
row = JSON.parse(last_response.body).find { |m| m["id"] == 424_244 }
|
||||
expect(row).not_to be_nil
|
||||
expect(row["path"]).to be_nil
|
||||
end
|
||||
end
|
||||
|
||||
describe "node rssi" do
|
||||
# Build a minimal MeshCore advert-style node payload (RF3).
|
||||
#
|
||||
# @param overrides [Hash] extra/overriding node fields.
|
||||
# @return [Hash] node dict for POST /api/nodes.
|
||||
def node_payload(overrides = {})
|
||||
{
|
||||
"lastHeard" => Time.now.to_i,
|
||||
"protocol" => "meshcore",
|
||||
"snr" => 12.0,
|
||||
"rssi" => -69,
|
||||
"hopsAway" => 2,
|
||||
"user" => {
|
||||
"longName" => "BER Drachentoeter",
|
||||
"shortName" => "5116",
|
||||
"publicKey" => "511617e3" + "00" * 28,
|
||||
"role" => "REPEATER",
|
||||
},
|
||||
}.merge(overrides)
|
||||
end
|
||||
|
||||
it "stores advert rssi and serves rssi and hops_away on GET /api/nodes" do
|
||||
post "/api/nodes", { "!511617e3" => node_payload }.to_json, auth_headers
|
||||
expect(last_response.status).to eq(201)
|
||||
|
||||
get "/api/nodes"
|
||||
expect(last_response).to be_ok
|
||||
row = JSON.parse(last_response.body).find { |n| n["node_id"] == "!511617e3" }
|
||||
expect(row).not_to be_nil
|
||||
expect(row["rssi"]).to eq(-69)
|
||||
expect(row["hops_away"]).to eq(2)
|
||||
expect(row["snr"]).to eq(12.0)
|
||||
end
|
||||
|
||||
it "preserves a stored rssi when a later upsert omits it (COALESCE)" do
|
||||
post "/api/nodes", { "!511617e3" => node_payload }.to_json, auth_headers
|
||||
expect(last_response.status).to eq(201)
|
||||
|
||||
# A contact-roster refresh carries no rssi; the stored per-advert value
|
||||
# must survive (nodes.rssi COALESCE upsert, RF3/RF6).
|
||||
refresh = node_payload("rssi" => nil, "lastHeard" => Time.now.to_i + 60)
|
||||
refresh.delete("rssi")
|
||||
post "/api/nodes", { "!511617e3" => refresh }.to_json, auth_headers
|
||||
expect(last_response.status).to eq(201)
|
||||
|
||||
get "/api/nodes"
|
||||
row = JSON.parse(last_response.body).find { |n| n["node_id"] == "!511617e3" }
|
||||
expect(row).not_to be_nil
|
||||
expect(row["rssi"]).to eq(-69)
|
||||
end
|
||||
|
||||
it "leaves rssi NULL for meshtastic nodes (no source)" do
|
||||
payload = node_payload("protocol" => "meshtastic")
|
||||
payload.delete("rssi")
|
||||
post "/api/nodes", { "!511617e3" => payload }.to_json, auth_headers
|
||||
expect(last_response.status).to eq(201)
|
||||
|
||||
get "/api/nodes"
|
||||
row = JSON.parse(last_response.body).find { |n| n["node_id"] == "!511617e3" }
|
||||
expect(row).not_to be_nil
|
||||
expect(row["rssi"]).to be_nil
|
||||
end
|
||||
end
|
||||
end
|
||||
Reference in New Issue
Block a user