adam/potato-mesh
1
0
Fork 0
mirror of https://github.com/l5yth/potato-mesh.git synced 2026-07-06 09:52:02 +02:00
Code Issues Packages Projects Releases Wiki Activity

web: breaking change on stats api (#801)

Browse Source 
* web: breaking change on stats api

* address review comments
This commit is contained in:
l5y
2026-06-21 13:41:41 +02:00
committed by GitHub
parent bb95b0792d
commit 5e0363a0ec
21 changed files with 717 additions and 152 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ACCEPTANCE.md
+118 -2
View File
@@ -43,9 +43,9 @@ then `kill` it afterward. Examples:
```bash
# Privacy checks (Layer A2): private mode, federation off
( cd web && API_TOKEN=acctest PRIVATE=1 FEDERATION=0 bundle exec rackup -p 41447 ) & SRV=$!
( cd web && API_TOKEN=acctest PRIVATE=1 FEDERATION=0 bundle exec ruby app.rb ) & SRV=$!
# Auth / contract checks (Layer C): public mode, known token
( cd web && API_TOKEN=acctest PRIVATE=0 FEDERATION=0 bundle exec rackup -p 41447 ) & SRV=$!
( cd web && API_TOKEN=acctest PRIVATE=0 FEDERATION=0 bundle exec ruby app.rb ) & SRV=$!
# ... run curl checks ...
kill "$SRV"
```
@@ -456,3 +456,119 @@ default-active tab stays the primary. Detection is by channel name, so a MeshCor
remain green: **A4c** (chat name resolution honors protocol — same render path)
and **B1** (all suites). The existing two-tier ordering assertions in
`chat-log-tabs.test.js` are **updated** to the three-tier order, not removed.
---
## Feature: /api/stats activity counts (messages & telemetry)
Maps to SPEC decisions **S1–S7**. The counts are produced by
`query_active_node_stats` (`web/lib/potato_mesh/application/queries/node_queries.rb`),
serialized by the `GET /api/stats` route (`application/routes/api.rb`), and
consumed for federation by `application/federation/crawl.rb`. Unless a check says
otherwise, start the server in **public** mode
(`API_TOKEN=acctest PRIVATE=0 FEDERATION=0 bundle exec ruby app.rb`).
### S-A1 — Breaking, versioned response shape (scope × metric tree) — S1, S2, S3
```bash
curl -s http://127.0.0.1:41447/api/stats \
| python3 -c 'import sys,json; d=json.load(sys.stdin); \
SC=("total","meshcore","meshtastic","reticulum"); ME=("nodes","messages","telemetry"); WI=("hour","day","week","month"); \
print(all(isinstance(d[s][m][w],int) for s in SC for m in ME for w in WI) and d["sampled"] is False and "active_nodes" not in d)'
git grep -nA2 'def version_fallback' -- web/lib/potato_mesh/config.rb
. .venv/bin/activate && pytest -q tests/test_version_sync.py
```
**Expected:** the Python check prints `True` — the payload is the tree
`{ total, meshcore, meshtastic, reticulum }`, each scope carrying
`{ nodes, messages, telemetry }`, each metric carrying integer
`{ hour, day, week, month }`, with `sampled` still present and `false`. The old
flat keys (`active_nodes`, integer-valued `meshcore`/`meshtastic`) are **gone** —
this is the intended, versioned break. `version_fallback` returns `"0.7.0"`, and
`test_version_sync.py` **passes** — the bump is applied in lockstep across all
five language manifests (`data.VERSION`, `Config.version_fallback`,
`web/package.json`, `app/pubspec.yaml`, `matrix/Cargo.toml`; `matrix/Cargo.lock`
is updated to match). The matching `git tag v0.7.0` is the maintainer release
step. `data/mesh_ingestor/CONTRACTS.md` documents the new `GET /api/stats` shape
and notes the 0.7.0 break. Full shape is asserted by the Ruby suite (S-A2/S-A3).
### S-A2 — `total` is unfiltered; protocol scopes are subsets; node counts preserved — S2
```bash
( cd web && bundle exec rspec spec/queries_spec.rb -e "active_node_stats" )
```
**Expected:** pass. With nodes seeded across protocols, `query_active_node_stats`
returns `total.<metric>` = counts over **all** rows and
`meshcore`/`meshtastic`/`reticulum` = `protocol = ?` subsets (so
`total ≥ Σ named protocols`). `total.nodes.{hour,day,week,month}` equals the
counts the prior `active_nodes` returned, and `meshcore.nodes`/`meshtastic.nodes`
equal the prior flat per-protocol counts (relocation, identical values). Every
metric honors the node opt-out marker using the filter appropriate to its table —
`opt_out_self_filter` for `nodes`, and `opt_out_node_id_filter` /
`opt_out_node_num_filter` for the message and telemetry-umbrella tables —
consistent with the existing list endpoints.
### S-A3 — `telemetry` umbrella + unchanged windows — S3, S4
```bash
( cd web && bundle exec rspec spec/queries_spec.rb -e "telemetry umbrella" )
```
**Expected:** pass. With one row inside the window in **each** of `positions`,
`telemetry`, `neighbors`, and `traces`, the `telemetry` metric counts **all four**
(positions + telemetry + neighbors + traces, by each table's `rx_time`); the
`messages` metric counts the `messages` table by `rx_time`; `nodes` counts
`nodes` by `last_heard`. Window cutoffs are unchanged — `hour` 3600s, `day`
86 400s, `week` `week_seconds`, `month` `four_weeks_seconds` — so a row older than
`four_weeks_seconds` is excluded from `month` (28-day floor, preserves C4).
### S-A4 — Privacy: messages zeroed in private mode — S5
*Run the server with `PRIVATE=1`.*
```bash
curl -s http://127.0.0.1:41447/api/stats \
| python3 -c 'import sys,json; d=json.load(sys.stdin); \
SC=("total","meshcore","meshtastic","reticulum"); WI=("hour","day","week","month"); \
print(all(d[s]["messages"][w]==0 for s in SC for w in WI))'
```
**Expected:** prints `True` — every `messages` count (in `total` and all protocol
scopes) is `0` under `PRIVATE=1`, mirroring the message-API 404 (A2a). `nodes` and
`telemetry` counts are unaffected by privacy mode (only `/api/messages*` is
gated). Behavior is also covered by a Ruby example
(`bundle exec rspec spec/app_spec.rb -e "/api/stats"` exercising private mode).
### S-A5 — `reticulum` forward-looking zero stub — S6
```bash
curl -s http://127.0.0.1:41447/api/stats \
| python3 -c 'import sys,json; d=json.load(sys.stdin); r=d["reticulum"]; \
ME=("nodes","messages","telemetry"); WI=("hour","day","week","month"); \
print(all(r[m][w]==0 for m in ME for w in WI))'
git grep -niE 'reticulum' -- web/lib/potato_mesh/application/queries/node_queries.rb
```
**Expected:** the Python check prints `True` — `reticulum` is present with every
count `0`. The grep shows the `reticulum` block carries an in-code comment marking
it a **stub** (always-zero until a Reticulum ingestor exists). `reticulum` is
**not** added to `KNOWN_PROTOCOLS` (still `meshcore` + `meshtastic`; verified by
A4a).
### S-A6 — One-way federation compatibility (new reads old) — S7
```bash
( cd web && bundle exec rspec spec/federation_spec.rb -e "stats" )
```
**Expected:** pass. The consumer resolves remote activity counts by trying the
**new** shape first (`total.nodes[window]`, `meshcore.nodes.day`,
`meshtastic.nodes.day`) and falling back to the **old** shape
(`active_nodes[window]`, `meshcore.day`, `meshtastic.day`), then to the existing
node-list fallback. The pre-existing federation specs that feed the **old** flat
shape continue to pass unchanged — they are the regression proof that a new
instance still reads an old peer. New unit coverage asserts
`remote_active_node_count_from_stats` handles both shapes (and prefers new).
### S-R1 — Regression: prior acceptance still holds
```bash
( cd web && npm test ) && ( cd web && bundle exec rspec )
( . .venv/bin/activate && pytest -q tests/ )
```
**Expected:** every prior check still passes. At risk and explicitly required to
remain green: **A3c** (federation specs — the old-shape stats specs must stay
green, proving one-way new-reads-old); **A2 / A2a** (privacy — `/api/messages`
still 404s in private mode **and** message counts are now zeroed, S-A4); and
**B1** (all suites). The JS stats assertions in `stats.test.js` /
`main-stats.test.js` (`normaliseActiveNodeStatsPayload`, `fetchActiveNodeStats`)
and the dashboard consumer (`stats.js`) are **updated** to read `total.nodes` from
the new shape, not removed. No POST/event contract changes, so **C2** and the
Python suite are unaffected.
README.md
+3 -3
View File
@@ -322,9 +322,9 @@ docker pull ghcr.io/l5yth/potato-mesh-matrix-bridge-linux-arm64:latest
docker pull ghcr.io/l5yth/potato-mesh-matrix-bridge-linux-armv7:latest
# version-pinned examples
docker pull ghcr.io/l5yth/potato-mesh-web-linux-amd64:v0.6.4
docker pull ghcr.io/l5yth/potato-mesh-ingestor-linux-amd64:v0.6.4
docker pull ghcr.io/l5yth/potato-mesh-matrix-bridge-linux-amd64:v0.6.4
docker pull ghcr.io/l5yth/potato-mesh-web-linux-amd64:v0.7.0
docker pull ghcr.io/l5yth/potato-mesh-ingestor-linux-amd64:v0.7.0
docker pull ghcr.io/l5yth/potato-mesh-matrix-bridge-linux-amd64:v0.7.0
```
Note: `latest` is only published for non-prerelease versions. Pre-release tags
SPEC.md
+27
View File
@@ -218,3 +218,30 @@ with the channel-ordering sort in
| **F2** | **Test-channel detection** is by the channel's resolved display **label**: the label contains the standalone word `ping`, `test`, or `bot`, case-insensitive, matched at **word boundaries**. So "Camping", "Robotics", "Contest", "Botswana" are **not** test channels; concatenated forms ("MyBot", "test2") are intentionally **not** matched either — the rule favors zero false positives over catching every variant. | interview |
| **F3** | **Default/primary channels are never demoted.** Test classification only reorders custom (index > 0) channels; an index-0 channel always leads even if its name matches a keyword, so the primary community feed is never hidden. | interview |
| **F4** | **Presentation-only & protocol-neutral.** Reorders tabs only — no change to channel membership, message contents/counts, the default-active tab (still the primary), or any data/API surface. Detection is by channel name and identical for MeshCore and Meshtastic, so the change **extends** Invariant IV (protocol parity) without privileging either protocol. | interview |
---
## Feature: /api/stats activity counts (messages & telemetry)
Extends `GET /api/stats` from active-node counts only to a uniform
`{ scope: { metric: { hour, day, week, month } } }` tree covering **nodes**,
**messages**, and **telemetry**, each as a grand `total` and a per-protocol
breakdown. The response shape changes incompatibly, so the change is a
**versioned breaking change** released as **0.7.0** with **one-way** federation
compatibility (new instances read old peers; old instances reading a new peer
degrade gracefully to their existing node-list fallback). Integrates with
`web/lib/potato_mesh/application/queries/node_queries.rb`
(`query_active_node_stats`), the `GET /api/stats` route in
`application/routes/api.rb`, the federation consumer in
`application/federation/crawl.rb`, `PotatoMesh::Config.version_fallback`, and
`data/mesh_ingestor/CONTRACTS.md`.
| # | Decision | Source |
| --- | --- | --- |
| **S1** | **Breaking, versioned response shape.** `/api/stats` returns `{ <scope>: { <metric>: { hour, day, week, month } }, sampled }` where `<scope>` ∈ {`total`, `meshcore`, `meshtastic`, `reticulum`} and `<metric>` ∈ {`nodes`, `messages`, `telemetry`}. This **breaks** the prior flat shape (`active_nodes` / flat `meshcore` / flat `meshtastic`) and is therefore a **versioned** break per D8: it ships under a minor bump to **0.7.0**, applied in lockstep across the five language manifests that `tests/test_version_sync.py` keeps in sync (`data.VERSION`, `Config.version_fallback`, `web/package.json`, `app/pubspec.yaml`, `matrix/Cargo.toml` + `Cargo.lock`), plus the maintainer's `git tag v0.7.0` release. **Explicitly amends D8's "evolve backward-compatibly" expectation for this route**; the apex (I) and privacy (II) invariants are untouched. | interview (D8 amendment) |
| **S2** | **`total` is unfiltered; protocol scopes are subsets.** `total.<metric>` counts all rows regardless of protocol; `meshcore` / `meshtastic` / `reticulum` are `WHERE protocol = ?` subsets (so `total` ≥ Σ named protocols). `total.nodes` reproduces the prior `active_nodes`, and `meshcore.nodes` / `meshtastic.nodes` reproduce the prior flat per-protocol node counts — identical values, relocated. | interview |
| **S3** | **`telemetry` is an umbrella metric.** The `telemetry` count aggregates **positions + telemetry + neighbors + traces** (every non-message, non-nodeinfo packet record), counted by each table's `rx_time`. `messages` counts the `messages` table by `rx_time`; `nodes` counts `nodes` by `last_heard` (unchanged from today). | interview |
| **S4** | **Activity windows unchanged.** Every count uses the existing cutoffs — `hour` (3600s), `day` (86 400s), `week` (`week_seconds`), `month` (`four_weeks_seconds`) — so no count can surface activity beyond the 28-day API visibility floor (preserves C4 / `MAX_QUERY_LIMIT` reasoning). | interview + code |
| **S5** | **Privacy: messages zeroed in private mode** (Invariant II). When `private_mode?`, every `messages` count (in `total` and all protocol scopes) is **0**, mirroring the `PRIVATE=1` message-API 404 (A2a) so stats never leak message volume that privacy hides. Node counts keep the `CLIENT_HIDDEN` exclusion; **all** metrics honor the node opt-out marker via the per-table opt-out filter (`opt_out_self_filter` for `nodes`; `opt_out_node_id_filter` / `opt_out_node_num_filter` for the message and telemetry-umbrella tables, matching the existing list endpoints). Telemetry/positions/neighbors/traces are not gated by `PRIVATE`, so those counts remain reported. | interview |
| **S6** | **`reticulum` is a forward-looking zero stub.** A `reticulum` scope is always emitted with all-zero counts and an in-code `# stub` comment, so the shape extends to future protocols without another break. It adds **no** ingest path (Invariant I), privileges no protocol (Invariant IV), and does **not** enter `KNOWN_PROTOCOLS` (which still gates the `?protocol=` query param at `meshcore` + `meshtastic`). | interview |
| **S7** | **One-way federation compatibility (new reads old).** Federation consumers (`crawl.rb`) try the new shape first (`total.nodes[window]`, `meshcore.nodes.day`, `meshtastic.nodes.day`) then fall back to the old shape (`active_nodes[window]`, `meshcore.day`, `meshtastic.day`), then to the existing node-list fallback. Detection is **structural** (key presence/shape) — no in-band version field. New instances read both old and new peers; old instances reading a new peer degrade gracefully (the accepted one-way limit). | interview |
app/ios/Flutter/AppFrameworkInfo.plist
+2 -2
View File
@@ -15,11 +15,11 @@
<key>CFBundlePackageType</key>
<string>FMWK</string>
<key>CFBundleShortVersionString</key>
<string>0.6.4</string>
<string>0.7.0</string>
<key>CFBundleSignature</key>
<string>????</string>
<key>CFBundleVersion</key>
<string>0.6.4</string>
<string>0.7.0</string>
<key>MinimumOSVersion</key>
<string>14.0</string>
</dict>
app/pubspec.yaml
+1 -1
View File
@@ -3,7 +3,7 @@
name: potato_mesh_reader
description: Meshtastic Reader — read-only view for PotatoMesh messages.
publish_to: "none"
version: 0.6.4
version: 0.7.0
environment:
sdk: ">=3.4.0 <4.0.0"
data/__init__.py
+1 -1
View File
@@ -18,7 +18,7 @@ The ``data.mesh`` module exposes helpers for reading Meshtastic node and
message information before forwarding it to the accompanying web application.
"""
VERSION = "0.6.4"
VERSION = "0.7.0"
"""Semantic version identifier shared with the dashboard and front-end."""
__version__ = VERSION
data/mesh_ingestor/CONTRACTS.md
+37 -1
View File
@@ -183,9 +183,45 @@ Every read endpoint enforces a server-side rolling-window floor on the data it r
| `GET /api/ingestors` | **28 days** | sparse heartbeats; same rationale |
| `GET /api/.../:id` (per-id lookup) | **28 days** | every per-id route uses the extended window so callers can backfill historical context for a specific node/conversation that has dropped out of the bulk view. The `since` clamp still applies. |
| `GET /api/telemetry/aggregated` | caller-controlled | `?windowSeconds=<N>` is mandatory; defaults to 86 400 (1 day). Bounded by `MAX_QUERY_LIMIT` on bucket count, not by a hard floor. |
| `GET /api/stats` | n/a | reports counts at fixed `hour`/`day`/`week`/`month` activity buckets. |
| `GET /api/stats` | n/a | reports activity counts at fixed `hour`/`day`/`week`/`month` buckets; response shape documented below. |
Federation peers should not assume an unbounded historical window: a peer that requests `/api/messages?since=0` from a partner expecting "everything" will only ever receive the last seven days. To pull older state, request the per-id endpoint (28 days) for the relevant nodes.
The constants live in `web/lib/potato_mesh/config.rb` (`week_seconds`, `four_weeks_seconds`).
### GET /api/stats response shape
> **Breaking change in 0.7.0.** Before 0.7.0 the payload was flat —
> `active_nodes: {hour,day,week,month}` plus integer-valued `meshcore`/`meshtastic`
> sub-hashes. From 0.7.0 it is the scope → metric → window tree below. The change
> is versioned (minor bump) per the backward-compat rule above. Federation
> consumers read the new shape and **fall back to the old shape** for pre-0.7.0
> peers (one-way compatibility); see `application/federation/crawl.rb`.
`GET /api/stats` returns counts as a `scope → metric → window` tree:
```jsonc
{
"total": { "nodes": {…}, "messages": {…}, "telemetry": {…} },
"meshcore": { "nodes": {…}, "messages": {…}, "telemetry": {…} },
"meshtastic": { "nodes": {…}, "messages": {…}, "telemetry": {…} },
"reticulum": { "nodes": {…}, "messages": {…}, "telemetry": {…} }, // stub: always 0
"sampled": false
}
```
- **Scopes.** `total` counts every visible row regardless of protocol; `meshcore`,
`meshtastic`, and `reticulum` are `protocol = ?` subsets, so
`total ≥ Σ named protocols`. `reticulum` is a forward-looking stub (no Reticulum
ingestor exists yet) and is always all-zero.
- **Metrics.** `nodes` counts `nodes` by `last_heard`; `messages` counts `messages`
by `rx_time`; `telemetry` is the umbrella over `positions` + `telemetry` +
`neighbors` + `traces` (every non-message packet record) by `rx_time`.
- **Windows.** Each metric maps to `{ "hour", "day", "week", "month" }` integer
counts at the fixed cutoffs (1 h / 24 h / `week_seconds` / `four_weeks_seconds`);
`month` cannot exceed the 28-day visibility floor.
- **Privacy.** Every metric honors the node opt-out marker. When `PRIVATE=1`, all
`messages` counts are forced to `0` (mirroring the disabled message API);
`nodes`/`telemetry` counts remain.
- **`sampled`** is unchanged: always `false` (the counts are exact, not sampled).
matrix/Cargo.lock
Generated
+1 -1
View File
@@ -968,7 +968,7 @@ checksum = "7edddbd0b52d732b21ad9a5fab5c704c14cd949e5e9a1ec5929a24fded1b904c"
[[package]]
name = "potatomesh-matrix-bridge"
version = "0.6.4"
version = "0.7.0"
dependencies = [
"anyhow",
"axum",
matrix/Cargo.toml
+1 -1
View File
@@ -14,7 +14,7 @@
[package]
name = "potatomesh-matrix-bridge"
version = "0.6.4"
version = "0.7.0"
edition = "2021"
[dependencies]
web/lib/potato_mesh/application/federation/crawl.rb
+40 -8
View File
@@ -25,8 +25,11 @@ module PotatoMesh
def remote_active_node_count_from_stats(payload, max_age_seconds:)
return nil unless payload.is_a?(Hash)
active_nodes = payload["active_nodes"]
return nil unless active_nodes.is_a?(Hash)
# Prefer the 0.7.0 shape (counts under total.nodes); fall back to the
# pre-0.7.0 flat active_nodes for older peers (one-way federation
# compatibility, SPEC S7).
node_windows = remote_stats_node_windows(payload)
return nil unless node_windows.is_a?(Hash)
age = coerce_integer(max_age_seconds) || 0
key = if age <= 3600
@@ -39,12 +42,40 @@ module PotatoMesh
"month"
end
value = coerce_integer(active_nodes[key])
value = coerce_integer(node_windows[key])
return nil unless value
[value, 0].max
end
# Resolve the total node-activity window hash from either /api/stats shape.
#
# @param payload [Hash] decoded /api/stats payload.
# @return [Hash, nil] +{ "hour", "day", "week", "month" }+ counts, or nil
# when neither the 0.7.0 (+total.nodes+) nor legacy (+active_nodes+) shape
# is present.
def remote_stats_node_windows(payload)
new_shape = payload.dig("total", "nodes")
return new_shape if new_shape.is_a?(Hash)
legacy = payload["active_nodes"]
legacy if legacy.is_a?(Hash)
end
# Resolve a protocol's 24h node count from either /api/stats shape.
#
# Reads the 0.7.0 +<protocol>.nodes.day+ value, falling back to the
# pre-0.7.0 flat +<protocol>.day+ for older peers.
#
# @param payload [Hash] decoded /api/stats payload.
# @param protocol [String] protocol scope name (e.g. "meshcore").
# @return [Integer, nil] node count active in the last day, or nil.
def remote_stats_protocol_day(payload, protocol)
value = payload.dig(protocol, "nodes", "day")
value = payload.dig(protocol, "day") if value.nil?
coerce_integer(value)
end
# Parse a remote federation instance payload into canonical attributes.
#
# @param payload [Hash] JSON object describing a remote instance.
@@ -293,12 +324,13 @@ module PotatoMesh
)
attributes[:nodes_count] = stats_count if stats_count
# Extract per-protocol 24h counts (informational, not signed).
# Extract per-protocol 24h counts (informational, not signed). Reads
# the 0.7.0 nodes sub-hash, falling back to the pre-0.7.0 flat shape.
if stats_payload.is_a?(Hash)
mc_day = stats_payload.dig("meshcore", "day")
mt_day = stats_payload.dig("meshtastic", "day")
attributes[:meshcore_nodes_count] = coerce_integer(mc_day) if mc_day
attributes[:meshtastic_nodes_count] = coerce_integer(mt_day) if mt_day
mc_day = remote_stats_protocol_day(stats_payload, "meshcore")
mt_day = remote_stats_protocol_day(stats_payload, "meshtastic")
attributes[:meshcore_nodes_count] = mc_day if mc_day
attributes[:meshtastic_nodes_count] = mt_day if mt_day
end
nodes_since_path = "/api/nodes?since=#{recent_cutoff}&limit=1000"
web/lib/potato_mesh/application/queries/node_queries.rb
+191 -61
View File
@@ -268,81 +268,211 @@ module PotatoMesh
db&.close
end
# Return exact active-node counts across common activity windows.
# Activity windows shared by every /api/stats metric, in a fixed order so
# generated column aliases (+total_hour+, +mc_day+, …) line up with their
# bind parameters.
STATS_WINDOWS = %w[hour day week month].freeze
# Per-protocol scopes counted alongside the unfiltered +total+, paired with
# the short column-alias prefix used in the generated SQL.
STATS_PROTOCOL_SCOPES = [["meshcore", "mc"], ["meshtastic", "mt"]].freeze
# Return exact activity counts for /api/stats as a scope → metric → window
# tree.
#
# Counts are resolved directly in SQL with COUNT(*) thresholds against
# +nodes.last_heard+ to avoid sampling bias from list endpoint limits.
# The shape is:
#
# { "total" => { "nodes" => {...}, "messages" => {...}, "telemetry" => {...} },
# "meshcore" => { ... }, "meshtastic" => { ... },
# "reticulum" => { ... all zero (stub) } }
#
# where each metric maps to a +{ "hour", "day", "week", "month" }+ window
# hash. +total+ counts every visible row regardless of protocol; the
# protocol scopes are +WHERE protocol = ?+ subsets, so
# +total ≥ Σ named protocols+. Counts are resolved directly in SQL with
# COUNT(*) thresholds (no sampling bias from list-endpoint limits) and honor
# the node opt-out marker on every metric. +messages+ counts are forced to
# zero in private mode (SPEC S5 / Invariant II).
#
# @param now [Integer] reference unix timestamp in seconds.
# @param db [SQLite3::Database, nil] optional open database handle to reuse.
# @return [Hash{String => Object}] counts keyed by hour/day/week/month plus
# per-protocol breakdowns under "meshcore" and "meshtastic" sub-hashes.
# @return [Hash{String => Hash}] scope → metric → window count tree.
def query_active_node_stats(now: Time.now.to_i, db: nil)
handle = db || open_database(readonly: true)
handle.results_as_hash = true
reference_now = coerce_integer(now) || Time.now.to_i
hour_cutoff = reference_now - 3600
day_cutoff = reference_now - 86_400
week_cutoff = reference_now - PotatoMesh::Config.week_seconds
# The "month" bucket reuses the four-week cap so no stats endpoint can
# surface activity from beyond the 28-day API visibility floor.
month_cutoff = reference_now - PotatoMesh::Config.four_weeks_seconds
private_clause = private_mode? ? " AND (role IS NULL OR role <> 'CLIENT_HIDDEN')" : ""
# Materialise the visible-nodes projection in a CTE so the opt-out
# LIKE predicate is evaluated once and the marker is bound twice in
# total instead of twice per subquery (44 → 22 binds). Every COUNT
# below then operates on the pre-filtered set, which also matches the
# rest of the read API's "no opt-out node ever leaves the database
# layer" invariant.
sql = <<~SQL
WITH visible_nodes AS (
SELECT last_heard, protocol
FROM nodes
WHERE #{opt_out_self_filter}#{private_clause}
)
SELECT
(SELECT COUNT(*) FROM visible_nodes WHERE last_heard >= ?) AS hour_count,
(SELECT COUNT(*) FROM visible_nodes WHERE last_heard >= ?) AS day_count,
(SELECT COUNT(*) FROM visible_nodes WHERE last_heard >= ?) AS week_count,
(SELECT COUNT(*) FROM visible_nodes WHERE last_heard >= ?) AS month_count,
(SELECT COUNT(*) FROM visible_nodes WHERE last_heard >= ? AND protocol = ?) AS mc_hour,
(SELECT COUNT(*) FROM visible_nodes WHERE last_heard >= ? AND protocol = ?) AS mc_day,
(SELECT COUNT(*) FROM visible_nodes WHERE last_heard >= ? AND protocol = ?) AS mc_week,
(SELECT COUNT(*) FROM visible_nodes WHERE last_heard >= ? AND protocol = ?) AS mc_month,
(SELECT COUNT(*) FROM visible_nodes WHERE last_heard >= ? AND protocol = ?) AS mt_hour,
(SELECT COUNT(*) FROM visible_nodes WHERE last_heard >= ? AND protocol = ?) AS mt_day,
(SELECT COUNT(*) FROM visible_nodes WHERE last_heard >= ? AND protocol = ?) AS mt_week,
(SELECT COUNT(*) FROM visible_nodes WHERE last_heard >= ? AND protocol = ?) AS mt_month
SQL
cutoffs = [hour_cutoff, day_cutoff, week_cutoff, month_cutoff]
params = opt_out_marker_params + cutoffs +
cutoffs.flat_map { |c| [c, "meshcore"] } +
cutoffs.flat_map { |c| [c, "meshtastic"] }
row = with_busy_retry do
handle.get_first_row(sql, params)
end || {}
{
"hour" => row["hour_count"].to_i,
"day" => row["day_count"].to_i,
"week" => row["week_count"].to_i,
"month" => row["month_count"].to_i,
"meshcore" => {
"hour" => row["mc_hour"].to_i,
"day" => row["mc_day"].to_i,
"week" => row["mc_week"].to_i,
"month" => row["mc_month"].to_i,
},
"meshtastic" => {
"hour" => row["mt_hour"].to_i,
"day" => row["mt_day"].to_i,
"week" => row["mt_week"].to_i,
"month" => row["mt_month"].to_i,
},
cutoffs = {
"hour" => reference_now - 3600,
"day" => reference_now - 86_400,
"week" => reference_now - PotatoMesh::Config.week_seconds,
"month" => reference_now - PotatoMesh::Config.four_weeks_seconds,
}
metrics = {
"nodes" => node_activity_counts(handle, cutoffs),
"messages" => message_activity_counts(handle, cutoffs),
"telemetry" => telemetry_activity_counts(handle, cutoffs),
}
assemble_stats_scopes(metrics)
ensure
handle&.close unless db
end
# Per-protocol node-activity counts keyed on +nodes.last_heard+, honoring
# the opt-out self-filter and, in private mode, the CLIENT_HIDDEN exclusion.
#
# @param handle [SQLite3::Database] open database handle.
# @param cutoffs [Hash{String => Integer}] window => lower-bound timestamp.
# @return [Hash{String => Hash}] scope => window counts.
def node_activity_counts(handle, cutoffs)
private_clause = private_mode? ? " AND (role IS NULL OR role <> 'CLIENT_HIDDEN')" : ""
projection = "SELECT last_heard AS t, protocol AS p FROM nodes " \
"WHERE #{opt_out_self_filter}#{private_clause}"
windowed_protocol_counts(
handle,
projection_sql: projection,
projection_params: opt_out_marker_params,
cutoffs: cutoffs,
)
end
# Per-protocol message-activity counts keyed on +messages.rx_time+. Privacy
# mode forces every count to zero, mirroring the +PRIVATE=1+ message-API 404
# so /api/stats never leaks message volume that privacy hides
# (SPEC S5 / Invariant II).
#
# @param handle [SQLite3::Database] open database handle.
# @param cutoffs [Hash{String => Integer}] window => lower-bound timestamp.
# @return [Hash{String => Hash}] scope => window counts.
def message_activity_counts(handle, cutoffs)
return zero_scope_counts if private_mode?
fragments = [opt_out_node_id_filter("from_id"), opt_out_node_id_filter("to_id")]
projection = "SELECT rx_time AS t, protocol AS p FROM messages " \
"WHERE #{fragments.join(" AND ")}"
windowed_protocol_counts(
handle,
projection_sql: projection,
projection_params: opt_out_marker_params * fragments.length,
cutoffs: cutoffs,
)
end
# Per-protocol telemetry-activity counts. "Telemetry" is the umbrella over
# every non-message packet record — positions + telemetry + neighbors +
# traces — unioned on +(rx_time, protocol)+, each table honoring the same
# opt-out filter its list endpoint applies (SPEC S3).
#
# @param handle [SQLite3::Database] open database handle.
# @param cutoffs [Hash{String => Integer}] window => lower-bound timestamp.
# @return [Hash{String => Hash}] scope => window counts.
def telemetry_activity_counts(handle, cutoffs)
sources = [
["positions", [opt_out_node_id_filter("node_id")]],
["telemetry", [opt_out_node_id_filter("node_id")]],
["neighbors", [opt_out_node_id_filter("node_id"), opt_out_node_id_filter("neighbor_id")]],
["traces", [opt_out_node_num_filter("src"), opt_out_node_num_filter("dest")]],
]
projections = []
params = []
sources.each do |table, fragments|
projections << "SELECT rx_time AS t, protocol AS p FROM #{table} WHERE #{fragments.join(" AND ")}"
params.concat(opt_out_marker_params * fragments.length)
end
windowed_protocol_counts(
handle,
projection_sql: projections.join("\nUNION ALL\n"),
projection_params: params,
cutoffs: cutoffs,
)
end
# Count visible rows from a +(t, p)+ projection across every window, as an
# unfiltered +total+ plus one subset per protocol scope. The projection is
# materialised in a CTE so its opt-out predicate is evaluated once; each
# COUNT then runs over the pre-filtered set.
#
# @param handle [SQLite3::Database] open database handle.
# @param projection_sql [String] SELECT yielding +t+ (activity time) and
# +p+ (protocol) columns for the visible rows of one metric.
# @param projection_params [Array] bind parameters for +projection_sql+.
# @param cutoffs [Hash{String => Integer}] window => lower-bound timestamp.
# @return [Hash{String => Hash}] +total+/+meshcore+/+meshtastic+ => window
# counts.
def windowed_protocol_counts(handle, projection_sql:, projection_params:, cutoffs:)
selects = []
window_params = []
STATS_WINDOWS.each do |window|
selects << "(SELECT COUNT(*) FROM visible WHERE t >= ?) AS total_#{window}"
window_params << cutoffs.fetch(window)
end
STATS_PROTOCOL_SCOPES.each do |protocol, prefix|
STATS_WINDOWS.each do |window|
selects << "(SELECT COUNT(*) FROM visible WHERE t >= ? AND p = ?) AS #{prefix}_#{window}"
window_params << cutoffs.fetch(window)
window_params << protocol
end
end
sql = "WITH visible AS (#{projection_sql})\nSELECT #{selects.join(",\n ")}"
row = with_busy_retry { handle.get_first_row(sql, projection_params + window_params) } || {}
scopes = { "total" => stats_window_hash(row, "total") }
STATS_PROTOCOL_SCOPES.each { |protocol, prefix| scopes[protocol] = stats_window_hash(row, prefix) }
scopes
end
# Extract one alias prefix's window-count hash from a result row.
#
# @param row [Hash] row returned by {windowed_protocol_counts}.
# @param prefix [String] column-alias prefix (+total+, +mc+, +mt+).
# @return [Hash{String => Integer}] window => count.
def stats_window_hash(row, prefix)
STATS_WINDOWS.each_with_object({}) do |window, acc|
acc[window] = row["#{prefix}_#{window}"].to_i
end
end
# Transpose metric → scope counts into the scope → metric tree returned by
# /api/stats and append the always-zero +reticulum+ stub.
#
# @param metrics [Hash{String => Hash}] metric => (scope => window counts).
# @return [Hash{String => Hash}] scope => (metric => window counts).
def assemble_stats_scopes(metrics)
scopes = {}
(["total"] + STATS_PROTOCOL_SCOPES.map(&:first)).each do |scope|
scopes[scope] = metrics.transform_values { |by_scope| by_scope[scope] }
end
# reticulum is a forward-looking stub: PotatoMesh has no Reticulum
# ingestor yet, so every count is zero. Emitting the scope now lets the
# response shape absorb the protocol later without another breaking change
# (SPEC S6).
scopes["reticulum"] = metrics.keys.each_with_object({}) do |metric, acc|
acc[metric] = zero_window_counts
end
scopes
end
# A fresh zero-filled window hash (+{ "hour" => 0, … }+), returned by value
# so callers never alias a shared mutable hash.
#
# @return [Hash{String => Integer}] zeroed window counts.
def zero_window_counts
STATS_WINDOWS.each_with_object({}) { |window, acc| acc[window] = 0 }
end
# A scope → window hash with every count zero (total + each protocol). Used
# for metrics suppressed by privacy mode.
#
# @return [Hash{String => Hash}] zeroed per-scope window counts.
def zero_scope_counts
scopes = { "total" => zero_window_counts }
STATS_PROTOCOL_SCOPES.each { |protocol, _| scopes[protocol] = zero_window_counts }
scopes
end
end
end
end
web/lib/potato_mesh/application/routes/api.rb
+3 -10
View File
@@ -110,16 +110,9 @@ module PotatoMesh
content_type :json
priv = private_mode? ? 1 : 0
cached = PotatoMesh::App::ApiCache.fetch("api:stats:#{priv}", ttl_seconds: 15) do
stats = query_active_node_stats
{
active_nodes: {
"hour" => stats["hour"], "day" => stats["day"],
"week" => stats["week"], "month" => stats["month"],
},
meshcore: stats["meshcore"],
meshtastic: stats["meshtastic"],
sampled: false,
}.to_json
# Scope → metric → window tree (SPEC S1). +sampled+ stays last and
# +false+ for backward continuity with the prior payload.
query_active_node_stats.merge("sampled" => false).to_json
end
etag cached[:etag], kind: :weak
web/lib/potato_mesh/config.rb
+4 -1
View File
@@ -278,9 +278,12 @@ module PotatoMesh
# Provide the fallback version string when git metadata is unavailable.
#
# Bumped to 0.7.0 for the breaking /api/stats response-shape change
# (SPEC S1); the matching +git tag v0.7.0+ is the maintainer release step.
#
# @return [String] semantic version identifier.
def version_fallback
"0.6.4"
"0.7.0"
end
# Default refresh interval for frontend polling routines.
web/package-lock.json
Generated
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "potato-mesh",
"version": "0.6.4",
"version": "0.7.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "potato-mesh",
"version": "0.6.4",
"version": "0.7.0",
"devDependencies": {
"istanbul-lib-coverage": "^3.2.2",
"istanbul-lib-report": "^3.0.1",
web/package.json
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "potato-mesh",
"version": "0.6.4",
"version": "0.7.0",
"type": "module",
"private": true,
"scripts": {
web/public/assets/js/app/__tests__/main-stats.test.js
+17 -15
View File
@@ -48,11 +48,13 @@ test('computeLocalActiveNodeStats calculates local hour/day/week/month counts wi
test('normaliseActiveNodeStatsPayload validates and normalizes API payload', () => {
const payload = {
active_nodes: {
hour: '11',
day: 22,
week: 33,
month: 44,
total: {
nodes: {
hour: '11',
day: 22,
week: 33,
month: 44,
},
},
sampled: false,
};
@@ -69,9 +71,9 @@ test('normaliseActiveNodeStatsPayload validates and normalizes API payload', ()
test('normaliseActiveNodeStatsPayload includes per-protocol buckets when present', () => {
const result = normaliseActiveNodeStatsPayload({
active_nodes: { hour: 10, day: 20, week: 30, month: 40 },
meshcore: { hour: 3, day: 8, week: 12, month: 15 },
meshtastic: { hour: 7, day: 12, week: 18, month: 25 },
total: { nodes: { hour: 10, day: 20, week: 30, month: 40 } },
meshcore: { nodes: { hour: 3, day: 8, week: 12, month: 15 } },
meshtastic: { nodes: { hour: 7, day: 12, week: 18, month: 25 } },
sampled: false,
});
assert.deepEqual(result.meshcore, { hour: 3, day: 8, week: 12, month: 15 });
@@ -80,11 +82,11 @@ test('normaliseActiveNodeStatsPayload includes per-protocol buckets when present
test('normaliseActiveNodeStatsPayload rejects malformed stat values', () => {
assert.equal(
normaliseActiveNodeStatsPayload({ active_nodes: { hour: 'x', day: 1, week: 1, month: 1 } }),
normaliseActiveNodeStatsPayload({ total: { nodes: { hour: 'x', day: 1, week: 1, month: 1 } } }),
null
);
assert.equal(
normaliseActiveNodeStatsPayload({ active_nodes: null }),
normaliseActiveNodeStatsPayload({ total: { nodes: null } }),
null
);
});
@@ -92,7 +94,7 @@ test('normaliseActiveNodeStatsPayload rejects malformed stat values', () => {
test('normaliseActiveNodeStatsPayload clamps negatives and truncates floats', () => {
assert.deepEqual(
normaliseActiveNodeStatsPayload({
active_nodes: { hour: -1.9, day: 2.8, week: 3.1, month: 4.9 },
total: { nodes: { hour: -1.9, day: 2.8, week: 3.1, month: 4.9 } },
sampled: 1
}),
{ hour: 0, day: 2, week: 3, month: 4, sampled: true }
@@ -107,7 +109,7 @@ test('fetchActiveNodeStats uses /api/stats when available', async () => {
ok: true,
async json() {
return {
active_nodes: { hour: 5, day: 15, week: 25, month: 35 },
total: { nodes: { hour: 5, day: 15, week: 25, month: 35 } },
sampled: false,
};
},
@@ -134,7 +136,7 @@ test('fetchActiveNodeStats reuses cached /api/stats response for repeated calls'
ok: true,
async json() {
return {
active_nodes: { hour: 2, day: 4, week: 6, month: 8 },
total: { nodes: { hour: 2, day: 4, week: 6, month: 8 } },
sampled: false,
};
},
@@ -185,7 +187,7 @@ test('fetchActiveNodeStats falls back to local counts on invalid payloads', asyn
fetchImpl: async () => ({
ok: true,
async json() {
return { active_nodes: { hour: 'bad' } };
return { total: { nodes: { hour: 'bad' } } };
}
})
});
@@ -198,5 +200,5 @@ test('formatActiveNodeStatsText emits compact day/week/month footer string', ()
stats: { day: 2, week: 3, month: 4, sampled: false },
});
assert.equal(text, '2/day \u00b7 3/week \u00b7 4/month');
assert.equal(text, '2/day · 3/week · 4/month');
});
web/public/assets/js/app/__tests__/stats.test.js
+20 -19
View File
@@ -111,12 +111,12 @@ test('computeLocalActiveNodeStats bins unknown protocols into meshtastic bucket'
});
// ---------------------------------------------------------------------------
// normaliseActiveNodeStatsPayload
// normaliseActiveNodeStatsPayload (0.7.0 scope → metric → window shape)
// ---------------------------------------------------------------------------
test('normaliseActiveNodeStatsPayload validates and normalises API payload', () => {
const result = normaliseActiveNodeStatsPayload({
active_nodes: { hour: '11', day: 22, week: 33, month: 44 },
total: { nodes: { hour: '11', day: 22, week: 33, month: 44 } },
sampled: false,
});
assert.equal(result.hour, 11);
@@ -128,9 +128,9 @@ test('normaliseActiveNodeStatsPayload validates and normalises API payload', ()
test('normaliseActiveNodeStatsPayload includes per-protocol buckets when present', () => {
const result = normaliseActiveNodeStatsPayload({
active_nodes: { hour: 10, day: 20, week: 30, month: 40 },
meshcore: { hour: 3, day: 8, week: 12, month: 15 },
meshtastic: { hour: 7, day: 12, week: 18, month: 25 },
total: { nodes: { hour: 10, day: 20, week: 30, month: 40 } },
meshcore: { nodes: { hour: 3, day: 8, week: 12, month: 15 } },
meshtastic: { nodes: { hour: 7, day: 12, week: 18, month: 25 } },
sampled: false,
});
assert.deepEqual(result.meshcore, { hour: 3, day: 8, week: 12, month: 15 });
@@ -139,7 +139,7 @@ test('normaliseActiveNodeStatsPayload includes per-protocol buckets when present
test('normaliseActiveNodeStatsPayload omits per-protocol buckets when absent', () => {
const result = normaliseActiveNodeStatsPayload({
active_nodes: { hour: 1, day: 2, week: 3, month: 4 },
total: { nodes: { hour: 1, day: 2, week: 3, month: 4 } },
sampled: false,
});
assert.equal(result.meshcore, undefined);
@@ -148,9 +148,9 @@ test('normaliseActiveNodeStatsPayload omits per-protocol buckets when absent', (
test('normaliseActiveNodeStatsPayload ignores malformed per-protocol buckets', () => {
const result = normaliseActiveNodeStatsPayload({
active_nodes: { hour: 1, day: 2, week: 3, month: 4 },
meshcore: { hour: 'bad', day: 1, week: 1, month: 1 },
meshtastic: 'not-an-object',
total: { nodes: { hour: 1, day: 2, week: 3, month: 4 } },
meshcore: { nodes: { hour: 'bad', day: 1, week: 1, month: 1 } },
meshtastic: { nodes: 'not-an-object' },
sampled: false,
});
assert.equal(result.hour, 1);
@@ -158,14 +158,15 @@ test('normaliseActiveNodeStatsPayload ignores malformed per-protocol buckets', (
assert.equal(result.meshtastic, undefined);
});
test('normaliseActiveNodeStatsPayload returns null for missing active_nodes', () => {
test('normaliseActiveNodeStatsPayload returns null for missing total.nodes', () => {
assert.equal(normaliseActiveNodeStatsPayload({}), null);
assert.equal(normaliseActiveNodeStatsPayload({ active_nodes: null }), null);
assert.equal(normaliseActiveNodeStatsPayload({ total: null }), null);
assert.equal(normaliseActiveNodeStatsPayload({ total: { nodes: null } }), null);
});
test('normaliseActiveNodeStatsPayload returns null when any stat is non-numeric', () => {
assert.equal(
normaliseActiveNodeStatsPayload({ active_nodes: { hour: 'x', day: 1, week: 1, month: 1 } }),
normaliseActiveNodeStatsPayload({ total: { nodes: { hour: 'x', day: 1, week: 1, month: 1 } } }),
null
);
});
@@ -173,7 +174,7 @@ test('normaliseActiveNodeStatsPayload returns null when any stat is non-numeric'
test('normaliseActiveNodeStatsPayload clamps negatives and truncates floats', () => {
assert.deepEqual(
normaliseActiveNodeStatsPayload({
active_nodes: { hour: -1.9, day: 2.8, week: 3.1, month: 4.9 },
total: { nodes: { hour: -1.9, day: 2.8, week: 3.1, month: 4.9 } },
sampled: 1,
}),
{ hour: 0, day: 2, week: 3, month: 4, sampled: true }
@@ -196,7 +197,7 @@ test('fetchActiveNodeStats returns remote stats when /api/stats succeeds', async
return {
ok: true,
async json() {
return { active_nodes: { hour: 5, day: 15, week: 25, month: 35 }, sampled: false };
return { total: { nodes: { hour: 5, day: 15, week: 25, month: 35 } }, sampled: false };
},
};
};
@@ -242,7 +243,7 @@ test('fetchActiveNodeStats falls back to local counts on invalid payload', async
nowSeconds: NOW,
fetchImpl: async () => ({
ok: true,
async json() { return { active_nodes: { hour: 'bad' } }; },
async json() { return { total: { nodes: { hour: 'bad' } } }; },
}),
});
assert.equal(stats.sampled, true);
@@ -256,7 +257,7 @@ test('fetchActiveNodeStats reuses cached result for repeated calls with same fet
calls.push(url);
return {
ok: true,
async json() { return { active_nodes: { hour: 1, day: 2, week: 3, month: 4 }, sampled: false }; },
async json() { return { total: { nodes: { hour: 1, day: 2, week: 3, month: 4 } }, sampled: false }; },
};
};
@@ -287,7 +288,7 @@ test('fetchActiveNodeStats concurrent calls share a single in-flight request', a
// Now let the response settle.
resolveResponse({
ok: true,
async json() { return { active_nodes: { hour: 9, day: 9, week: 9, month: 9 }, sampled: false }; },
async json() { return { total: { nodes: { hour: 9, day: 9, week: 9, month: 9 } }, sampled: false }; },
});
const [r1, r2] = await Promise.all([p1, p2]);
assert.deepEqual(r1, r2, 'concurrent requests should receive the same result');
@@ -302,11 +303,11 @@ test('formatActiveNodeStatsText emits compact day/week/month string', () => {
formatActiveNodeStatsText({
stats: { day: 2, week: 3, month: 4, sampled: false },
}),
'2/day \u00b7 3/week \u00b7 4/month'
'2/day · 3/week · 4/month'
);
});
test('formatActiveNodeStatsText handles missing or null stats gracefully', () => {
const text = formatActiveNodeStatsText({ stats: null });
assert.equal(text, '0/day \u00b7 0/week \u00b7 0/month', 'defaults to zero counts for null stats');
assert.equal(text, '0/day · 0/week · 0/month', 'defaults to zero counts for null stats');
});
web/public/assets/js/app/stats.js
+13 -14
View File
@@ -81,32 +81,31 @@ function normaliseProtocolBucket(bucket) {
}
/**
* Parse and validate the ``/api/stats`` payload.
* Parse and validate the ``/api/stats`` payload (0.7.0 scope → metric → window
* shape) into the flat node-count snapshot the dashboard renders.
*
* Node counts are read from ``total.nodes`` and the per-protocol
* ``<protocol>.nodes`` sub-buckets; the other metrics (messages/telemetry) are
* not surfaced in the header. The browser only ever calls its own same-version
* instance, so only the current shape is parsed.
*
* @param {*} payload Candidate JSON object from the stats endpoint.
* @returns {{hour: number, day: number, week: number, month: number, sampled: boolean, meshcore?: Object, meshtastic?: Object}|null} Normalized stats or null.
*/
export function normaliseActiveNodeStatsPayload(payload) {
const activeNodes = payload && typeof payload === 'object' ? payload.active_nodes : null;
if (!activeNodes || typeof activeNodes !== 'object') {
if (!payload || typeof payload !== 'object') {
return null;
}
const hour = Number(activeNodes.hour);
const day = Number(activeNodes.day);
const week = Number(activeNodes.week);
const month = Number(activeNodes.month);
if (![hour, day, week, month].every(Number.isFinite)) {
const nodes = normaliseProtocolBucket(payload.total?.nodes);
if (!nodes) {
return null;
}
const result = {
hour: Math.max(0, Math.trunc(hour)),
day: Math.max(0, Math.trunc(day)),
week: Math.max(0, Math.trunc(week)),
month: Math.max(0, Math.trunc(month)),
...nodes,
sampled: Boolean(payload.sampled)
};
const meshcore = normaliseProtocolBucket(payload.meshcore);
const meshtastic = normaliseProtocolBucket(payload.meshtastic);
const meshcore = normaliseProtocolBucket(payload.meshcore?.nodes);
const meshtastic = normaliseProtocolBucket(payload.meshtastic?.nodes);
if (meshcore) result.meshcore = meshcore;
if (meshtastic) result.meshtastic = meshtastic;
return result;
web/spec/app_spec.rb
+73 -3
View File
@@ -6319,24 +6319,94 @@ RSpec.describe "Potato Mesh Sinatra app" do
expect(last_response).to be_ok
payload = JSON.parse(last_response.body)
expect(payload["sampled"]).to eq(false)
expect(payload["active_nodes"]).to include(
expect(payload["total"]["nodes"]).to include(
"hour" => 1005,
"day" => 1005,
"week" => 1006,
"month" => 1007,
)
expect(payload["meshcore"]).to include(
expect(payload["meshcore"]["nodes"]).to include(
"hour" => 5,
"day" => 5,
"week" => 5,
"month" => 5,
)
expect(payload["meshtastic"]).to include(
expect(payload["meshtastic"]["nodes"]).to include(
"hour" => 1000,
"day" => 1000,
"week" => 1001,
"month" => 1002,
)
# The pre-0.7.0 flat key is gone (breaking change).
expect(payload).not_to have_key("active_nodes")
# reticulum is an always-zero forward-looking stub.
expect(payload["reticulum"]["nodes"].values).to all(eq(0))
expect(payload["reticulum"]["messages"].values).to all(eq(0))
expect(payload["reticulum"]["telemetry"].values).to all(eq(0))
end
it "counts messages and the telemetry umbrella with per-protocol breakdowns" do
clear_database
now = reference_time.to_i
allow(Time).to receive(:now).and_return(reference_time)
rx_iso = reference_time.utc.iso8601
with_db do |db|
# Seed the nodes the neighbor row references (neighbors has FK → nodes,
# and app_spec enables PRAGMA foreign_keys = ON).
db.execute("INSERT INTO nodes(node_id, num, last_heard, first_heard, role) VALUES (?,?,?,?,?)", ["!bbbb0001", 0xBBBB0001, now, now, "CLIENT"])
db.execute("INSERT INTO nodes(node_id, num, last_heard, first_heard, role) VALUES (?,?,?,?,?)", ["!cccc0001", 0xCCCC0001, now, now, "CLIENT"])
db.execute(
"INSERT INTO messages(id, rx_time, rx_iso, from_id, to_id, channel, text, protocol) VALUES (?,?,?,?,?,?,?,?)",
[1, now, rx_iso, "!aaaa0001", "!ffffffff", 0, "mc hi", "meshcore"],
)
db.execute(
"INSERT INTO messages(id, rx_time, rx_iso, from_id, to_id, channel, text, protocol) VALUES (?,?,?,?,?,?,?,?)",
[2, now, rx_iso, "!bbbb0001", "!ffffffff", 0, "mt hi", "meshtastic"],
)
# Telemetry umbrella: one row in each contributing table (meshtastic default).
db.execute("INSERT INTO positions(id, rx_time, rx_iso, node_id) VALUES (?,?,?,?)", [1, now, rx_iso, "!bbbb0001"])
db.execute("INSERT INTO telemetry(id, rx_time, rx_iso, node_id) VALUES (?,?,?,?)", [1, now, rx_iso, "!bbbb0001"])
db.execute("INSERT INTO neighbors(node_id, neighbor_id, rx_time) VALUES (?,?,?)", ["!bbbb0001", "!cccc0001", now])
db.execute("INSERT INTO traces(id, rx_time, rx_iso, src, dest) VALUES (?,?,?,?,?)", [1, now, rx_iso, 1, 2])
end
get "/api/stats"
payload = JSON.parse(last_response.body)
expect(payload["total"]["messages"]["hour"]).to eq(2)
expect(payload["meshcore"]["messages"]["hour"]).to eq(1)
expect(payload["meshtastic"]["messages"]["hour"]).to eq(1)
# positions + telemetry + neighbors + traces, all meshtastic-default → 4.
expect(payload["total"]["telemetry"]["hour"]).to eq(4)
expect(payload["meshtastic"]["telemetry"]["hour"]).to eq(4)
end
it "zeroes message counts but keeps node counts in private mode" do
ENV["PRIVATE"] = "1"
clear_database
now = reference_time.to_i
allow(Time).to receive(:now).and_return(reference_time)
rx_iso = reference_time.utc.iso8601
with_db do |db|
db.execute(
INSERT_NODE_WITH_METADATA_SQL,
["!node0001", 1, "n", "Node", "TBEAM", "CLIENT", now, now],
)
db.execute(
"INSERT INTO messages(id, rx_time, rx_iso, from_id, to_id, channel, text) VALUES (?,?,?,?,?,?,?)",
[1, now, rx_iso, "!node0001", "!ffffffff", 0, "secret"],
)
end
get "/api/stats"
payload = JSON.parse(last_response.body)
# PRIVATE=1 forces every message count to zero (SPEC S5 / Invariant II)...
expect(payload["total"]["messages"].values).to all(eq(0))
# ...while node counts remain visible.
expect(payload["total"]["nodes"]["hour"]).to eq(1)
end
end
web/spec/federation_spec.rb
+61
View File
@@ -507,6 +507,33 @@ RSpec.describe PotatoMesh::App::Federation do
expect(attributes_list.map { |attrs| attrs[:nodes_count] }).to all(eq(5))
end
it "reads the 0.7.0 stats shape for total and per-protocol counts" do
now = Time.at(1_700_000_000)
configure_remote_node_window(now)
new_shape = {
"total" => { "nodes" => { "hour" => 5, "day" => 7, "week" => 9, "month" => 11 } },
"meshcore" => { "nodes" => { "hour" => 1, "day" => 2, "week" => 3, "month" => 4 } },
"meshtastic" => { "nodes" => { "hour" => 4, "day" => 5, "week" => 6, "month" => 7 } },
"reticulum" => { "nodes" => { "hour" => 0, "day" => 0, "week" => 0, "month" => 0 } },
"sampled" => false,
}
mapping = stats_mapping(
now:,
stats_response: [new_shape, :stats],
full_nodes_response: [node_payload, :nodes],
)
stub_ingest_fetches(mapping)
federation_helpers.ingest_known_instances_from!(db, seed_domain)
# nodes_count comes from total.nodes (the 900s max age selects the hour window).
expect(attributes_list.map { |attrs| attrs[:nodes_count] }).to all(eq(5))
# Per-protocol 24h counts come from <protocol>.nodes.day.
expect(attributes_list.map { |attrs| attrs[:meshcore_nodes_count] }).to all(eq(2))
expect(attributes_list.map { |attrs| attrs[:meshtastic_nodes_count] }).to all(eq(5))
end
it "prefers recent node window counts when /api/stats is unavailable" do
now = Time.at(1_700_000_000)
configure_remote_node_window(now)
@@ -2104,6 +2131,40 @@ RSpec.describe PotatoMesh::App::Federation do
window = PotatoMesh::Config.week_seconds + 60
expect(federation_helpers.remote_active_node_count_from_stats(payload, max_age_seconds: window)).to eq(4)
end
it "reads the 0.7.0 total.nodes shape" do
payload = { "total" => { "nodes" => active } }
expect(federation_helpers.remote_active_node_count_from_stats(payload, max_age_seconds: 7_200)).to eq(2)
end
it "prefers the new shape over a legacy active_nodes block" do
payload = {
"total" => { "nodes" => { "hour" => 1, "day" => 8, "week" => 9, "month" => 10 } },
"active_nodes" => active,
}
expect(federation_helpers.remote_active_node_count_from_stats(payload, max_age_seconds: 7_200)).to eq(8)
end
it "returns nil when neither stats shape is present" do
payload = { "sampled" => false }
expect(federation_helpers.remote_active_node_count_from_stats(payload, max_age_seconds: 7_200)).to be_nil
end
end
describe ".remote_stats_protocol_day" do
it "reads the 0.7.0 nodes sub-hash" do
payload = { "meshcore" => { "nodes" => { "day" => 9 } } }
expect(federation_helpers.remote_stats_protocol_day(payload, "meshcore")).to eq(9)
end
it "falls back to the legacy flat day count" do
payload = { "meshtastic" => { "day" => 4 } }
expect(federation_helpers.remote_stats_protocol_day(payload, "meshtastic")).to eq(4)
end
it "returns nil when neither shape carries the protocol" do
expect(federation_helpers.remote_stats_protocol_day({}, "meshcore")).to be_nil
end
end
describe ".remote_instance_attributes_from_payload (extra branches)" do
web/spec/queries_spec.rb
+101 -6
View File
@@ -1146,15 +1146,68 @@ RSpec.describe PotatoMesh::App::Queries do
it "excludes opted-out nodes from active stats counters" do
stats = queries.query_active_node_stats(now: now)
# Only "!visible0" is visible — opted-out and short-marker nodes are dropped.
expect(stats["day"]).to eq(1)
expect(stats["week"]).to eq(1)
expect(stats["month"]).to eq(1)
# Only "!visible0" is visible — opted-out and short-marker nodes are dropped
# from node, message, and telemetry counts alike.
expect(stats["total"]["nodes"]["day"]).to eq(1)
expect(stats["total"]["nodes"]["month"]).to eq(1)
# Message id 11 (from opt-out) and id 12 (to opt-out) drop; only the
# visible → broadcast line survives.
expect(stats["total"]["messages"]["day"]).to eq(1)
# Telemetry umbrella = positions(1) + telemetry(1) + neighbors(0, peer
# opted out) + traces(1, the opted-out src dropped).
expect(stats["total"]["telemetry"]["day"]).to eq(3)
end
end
describe "#query_active_node_stats" do
it "caps the month bucket at four_weeks_seconds (28 days)" do
it "returns the full scope × metric × window tree" do
stats = queries.query_active_node_stats(now: now)
%w[total meshcore meshtastic reticulum].each do |scope|
expect(stats).to have_key(scope)
%w[nodes messages telemetry].each do |metric|
expect(stats[scope][metric].keys).to contain_exactly("hour", "day", "week", "month")
stats[scope][metric].each_value { |count| expect(count).to be_a(Integer) }
end
end
# The pre-0.7.0 flat keys are gone (intended breaking change).
expect(stats).not_to have_key("active_nodes")
expect(stats).not_to have_key("hour")
end
it "counts total across all protocols with per-protocol subsets" do
with_db do |db|
db.execute(
"INSERT INTO nodes(node_id, num, last_heard, first_heard, role, protocol) VALUES (?,?,?,?,?,?)",
["!core0001", 0xC0000001, now, now, "CLIENT", "meshcore"],
)
db.execute(
"INSERT INTO nodes(node_id, num, last_heard, first_heard, role, protocol) VALUES (?,?,?,?,?,?)",
["!tastic01", 0x70000001, now, now, "CLIENT", "meshtastic"],
)
end
stats = queries.query_active_node_stats(now: now)
expect(stats["meshcore"]["nodes"]["day"]).to eq(1)
expect(stats["meshtastic"]["nodes"]["day"]).to eq(1)
expect(stats["total"]["nodes"]["day"]).to eq(2)
expect(stats["total"]["nodes"]["day"]).to eq(
stats["meshcore"]["nodes"]["day"] + stats["meshtastic"]["nodes"]["day"],
)
end
it "aggregates positions, telemetry, neighbors, and traces into the telemetry umbrella" do
with_db do |db|
rx_iso = Time.at(now).utc.iso8601
db.execute("INSERT INTO positions(id, rx_time, rx_iso, node_id) VALUES (?,?,?,?)", [1, now, rx_iso, "!feed0001"])
db.execute("INSERT INTO telemetry(id, rx_time, rx_iso, node_id) VALUES (?,?,?,?)", [1, now, rx_iso, "!feed0001"])
db.execute("INSERT INTO neighbors(node_id, neighbor_id, rx_time) VALUES (?,?,?)", ["!feed0001", "!feed0002", now])
db.execute("INSERT INTO traces(id, rx_time, rx_iso, src, dest) VALUES (?,?,?,?,?)", [1, now, rx_iso, 0x11, 0x22])
end
stats = queries.query_active_node_stats(now: now)
# One row in each of the four umbrella tables → 4.
expect(stats["total"]["telemetry"]["hour"]).to eq(4)
end
it "caps the node month bucket at four_weeks_seconds (28 days)" do
twenty_nine_days_ago = now - (29 * 24 * 60 * 60)
with_db do |db|
db.execute(
@@ -1164,7 +1217,49 @@ RSpec.describe PotatoMesh::App::Queries do
end
stats = queries.query_active_node_stats(now: now)
# 28-day cap means this 29-day-old row falls outside the "month" bucket.
expect(stats["month"]).to eq(0)
expect(stats["total"]["nodes"]["month"]).to eq(0)
end
it "caps the telemetry umbrella month bucket at the visibility floor" do
old = now - (29 * 24 * 60 * 60)
with_db do |db|
db.execute("INSERT INTO positions(id, rx_time, rx_iso, node_id) VALUES (?,?,?,?)", [1, old, Time.at(old).utc.iso8601, "!feed0001"])
db.execute("INSERT INTO positions(id, rx_time, rx_iso, node_id) VALUES (?,?,?,?)", [2, now, Time.at(now).utc.iso8601, "!feed0001"])
end
stats = queries.query_active_node_stats(now: now)
# Only the recent row falls inside the month window (28-day floor; the spec
# stubs four_weeks_seconds to 7 days, so the 29-day-old row is excluded).
expect(stats["total"]["telemetry"]["month"]).to eq(1)
end
it "emits reticulum as an all-zero stub" do
with_db do |db|
db.execute("INSERT INTO nodes(node_id, num, last_heard, first_heard, role) VALUES (?,?,?,?,?)", ["!any00001", 1, now, now, "CLIENT"])
end
stats = queries.query_active_node_stats(now: now)
stats["reticulum"].each_value do |windows|
windows.each_value { |count| expect(count).to eq(0) }
end
end
it "zeroes message counts in private mode but keeps nodes and telemetry" do
private_queries = Class.new(harness_class) do
def private_mode?
true
end
end.new
with_db do |db|
rx_iso = Time.at(now).utc.iso8601
db.execute("INSERT INTO nodes(node_id, num, last_heard, first_heard, role) VALUES (?,?,?,?,?)", ["!priv0001", 1, now, now, "CLIENT"])
db.execute("INSERT INTO messages(id, rx_time, rx_iso, from_id, to_id, channel, text) VALUES (?,?,?,?,?,?,?)", [1, now, rx_iso, "!priv0001", "!ffffffff", 0, "hi"])
db.execute("INSERT INTO positions(id, rx_time, rx_iso, node_id) VALUES (?,?,?,?)", [1, now, rx_iso, "!priv0001"])
end
stats = private_queries.query_active_node_stats(now: now)
expect(stats["total"]["messages"]["day"]).to eq(0)
expect(stats["meshtastic"]["messages"]["day"]).to eq(0)
# Non-message metrics are unaffected by privacy mode.
expect(stats["total"]["nodes"]["day"]).to eq(1)
expect(stats["total"]["telemetry"]["day"]).to eq(1)
end
end