mirror of
https://github.com/l5yth/potato-mesh.git
synced 2026-10-30 19:41:58 +01:00
web: add pagination to all APIs
This commit is contained in:
10 files changed
+624
-32
No files matched your search
@@ -1176,3 +1176,102 @@ new helper file must carry the full Apache block); **D1** (`/version` config blo
|
||||
the shared layout's behavior is unchanged). Any existing view/app spec that
|
||||
asserted an exact *unversioned* asset string (e.g. `src="/assets/js/app/index.js"`)
|
||||
is **updated** to the `?v=` form, **not** removed.
|
||||
|
||||
---
|
||||
|
||||
## Feature: Uniform backward pagination (`?before=`) for bulk collection APIs
|
||||
|
||||
Maps to SPEC decisions **BP1–BP9**. `?before=<unix_seconds>` is added as an
|
||||
inclusive upper-bound keyset cursor to the six bulk collection GETs — `/api/nodes`,
|
||||
`/api/positions`, `/api/telemetry`, `/api/neighbors`, `/api/traces`,
|
||||
`/api/ingestors` — mirroring the existing `/api/messages` cursor (**C7**). The
|
||||
logic lives in `web/lib/potato_mesh/application/routes/api.rb` and the `query_*`
|
||||
helpers under `web/lib/potato_mesh/application/queries/`; the cursor is documented
|
||||
in `data/mesh_ingestor/CONTRACTS.md`. Unless a check says otherwise, start the
|
||||
server in public mode
|
||||
(`API_TOKEN=acctest PRIVATE=0 FEDERATION=0 bundle exec ruby app.rb`).
|
||||
|
||||
### BP-A1 — Every bulk collection pages backward through the full window — BP1, BP2, BP3
|
||||
```bash
|
||||
( cd web && bundle exec rspec spec/app_spec.rb -e "before pagination" )
|
||||
```
|
||||
**Expected:** pass. For **each** of `/api/nodes`, `/api/positions`,
|
||||
`/api/telemetry`, `/api/neighbors`, `/api/traces`, and `/api/ingestors`, seeding
|
||||
more than `MAX_QUERY_LIMIT` (1000) rows inside the route's window and walking
|
||||
newest → oldest — each page `limit=MAX_QUERY_LIMIT`, then `before=<oldest
|
||||
primary-sort value seen>`, de-duplicating by id — recovers **every** in-window row
|
||||
(the walk does not stall at the newest 1000). No single response exceeds
|
||||
`MAX_QUERY_LIMIT`. The cursor bounds the route's primary sort column inclusively:
|
||||
`rx_time` for positions/telemetry/neighbors/traces, `last_heard` for nodes,
|
||||
`last_seen_time` for ingestors.
|
||||
|
||||
### BP-A2 — `before` only narrows; the floor still bounds the window — BP2
|
||||
```bash
|
||||
( cd web && bundle exec rspec spec/app_spec.rb -e "before cannot widen the window" )
|
||||
```
|
||||
**Expected:** pass. A `before` newer than `now` returns the same rows as no
|
||||
`before` (a no-op upper bound). A `before` older than the route's floor, combined
|
||||
with the floor-clamped lower bound, returns **nothing beyond the floor** — a row
|
||||
older than the 7-day / 28-day floor stays excluded, so `before` cannot reach past
|
||||
it (preserves **C4**). A non-positive or non-integer `before` (`0`, `-5`, `abc`)
|
||||
is ignored as absent (parity with the messages `coerce_positive_or_nil`), so the
|
||||
unfiltered newest page is returned.
|
||||
|
||||
### BP-A3 — Inclusive boundary, protocol-neutral cursor — BP3, BP5
|
||||
```bash
|
||||
( cd web && bundle exec rspec spec/app_spec.rb -e "before pagination boundary" )
|
||||
```
|
||||
**Expected:** pass. Two rows sharing the exact boundary second are **both**
|
||||
returned when that second is passed as `before` (the inclusive `<=` ceiling never
|
||||
skips a boundary row — client dedup collapses the one-row overlap between pages).
|
||||
`?before=` composes with `?protocol=`: a backward walk filtered by
|
||||
`protocol=meshcore` returns only MeshCore rows and still recovers all of them,
|
||||
with neither protocol privileged.
|
||||
|
||||
### BP-A4 — History pages bypass the response cache — BP7
|
||||
```bash
|
||||
( cd web && bundle exec rspec spec/app_spec.rb -e "before bypasses the response cache" )
|
||||
```
|
||||
**Expected:** pass. A request carrying `before` is served from a fresh query, not
|
||||
the short-lived `ApiCache` newest-page entry, and issuing it does **not** overwrite
|
||||
or evict that hot entry — a subsequent no-`before` request still returns the cached
|
||||
newest page. Matches the established `/api/messages` behavior (a `since > 0` or
|
||||
`before` request skips the cache; the cache key for the default path is unchanged).
|
||||
|
||||
### BP-A5 — Privacy, opt-out, and apex are untouched — BP6
|
||||
```bash
|
||||
( cd web && bundle exec rspec spec/app_spec.rb -e "before pagination honors privacy" )
|
||||
git grep -niE 'mqtt|mosquitto|paho|amqp|kafka|broker' -- web/Gemfile web/Gemfile.lock | grep -viE 'via_?mqtt'
|
||||
```
|
||||
**Expected:** the rspec passes and the grep prints nothing. A backward walk over
|
||||
`/api/nodes` still excludes opted-out nodes (`NODE_OPT_OUT_MARKER`) and, in private
|
||||
mode, `CLIENT_HIDDEN` nodes — `before` only narrows, so it can never surface a row
|
||||
the route would otherwise hide (**A2c**, Invariant II). No manifest gains a broker
|
||||
dependency (Invariant I / **A1a**): the change is a read-side query param only.
|
||||
|
||||
### BP-A6 — Cursor documented; deferred scope recorded — BP1, BP8, BP9
|
||||
```bash
|
||||
git grep -n 'before' -- data/mesh_ingestor/CONTRACTS.md
|
||||
```
|
||||
**Expected:** the `CONTRACTS.md` "GET endpoint time windows" section documents the
|
||||
`?before=` inclusive upper-bound cursor and names the six collections that accept
|
||||
it. The deferred items in **BP9** are out of scope and must **not** appear in this
|
||||
change: `/api/instances` still lacks `limit`/`since`/`protocol`, and
|
||||
`/api/telemetry/aggregated` still uses camelCase `windowSeconds`/`bucketSeconds`
|
||||
(no snake_case alias) — these stay tracked follow-ups, not regressions.
|
||||
|
||||
### BP-R1 — Regression: prior acceptance still holds
|
||||
```bash
|
||||
( cd web && bundle exec rspec ) && ( cd web && npm test )
|
||||
( . .venv/bin/activate && pytest -q tests/ )
|
||||
```
|
||||
**Expected:** every prior check still passes. At risk and explicitly required to
|
||||
remain green: **C7** (messages backward pagination — its keyset mechanism is now
|
||||
shared by six more routes, but `/api/messages` behavior is unchanged); **C4**
|
||||
(window floors — `before` only narrows, never widens); **A2 / A2a / A2c** (privacy
|
||||
& opt-out — a narrowing upper bound exposes no hidden row, and `/api/messages`
|
||||
still 404s in private mode); **A4a** (`KNOWN_PROTOCOLS` unchanged); **PL-A1 /
|
||||
PL-A2** and **FC-A2** (the frontend message pager and cache seed-then-delta are
|
||||
untouched — frontend `before` adoption is deferred per **BP9**); and **B1** (all
|
||||
suites). No POST/event contract changes, so **C2** and the Python suite are
|
||||
unaffected (the only `data/` touch is the `CONTRACTS.md` GET-window documentation).
|
||||
@@ -342,3 +342,44 @@ helper + import-map builder) and the asset references in `views/layouts/app.erb`
|
||||
| **AV3** | **Full module-graph busting via import map.** Because `?v=` on an entry-point URL does **not** propagate to that module's relative `import './x.js'`, the layout `<head>` emits exactly one `<script type="importmap">` (before any module loads) mapping every served production `/assets/js/**/*.js` module (excluding `__tests__`) to its `?v=<APP_VERSION>` URL. This invalidates the whole transitive graph (e.g. `main.js` + its 33 imports), not just the entry points. **Safety property:** a module absent from the map degrades to today's unversioned-but-working load — a missing entry can never break a working import. Browsers without import-map support degrade to today's behavior (entry points still busted via AV2). | interview |
|
||||
| **AV4** | **Scope = JS + CSS only; native, no new egress.** Only executable/style assets are versioned (all JS + `base.css`). Images, favicons, and SVG icons keep today's `Last-Modified`/`ETag` revalidation (a stale logo is cosmetic, not behavioral). The mechanism uses the native browser import-map feature — **no new dependency, build step, external host, or analytics param** — so apex (I), privacy (II), federation (III), parity (IV), fixed-stack (D7), contract (D8), and no-phone-home (D11) all hold unchanged. The version query is not part of any `/api/*` contract (D8): Sinatra serves the same static file regardless of query string. | interview |
|
||||
| **AV5** | **Engineering bar (D9).** The helper and import-map builder ship with 100% unit tests + RDoc, Apache headers, `rufo`-clean; all existing suites stay green. View/app specs that assert exact asset markup are **updated** to the versioned form, not removed. | CLAUDE.md |
|
||||
|
||||
---
|
||||
|
||||
## Feature: Uniform backward pagination (`?before=`) for bulk collection APIs
|
||||
|
||||
Generalizes the `/api/messages` backward-pagination cursor (issue #796, **C7**) to
|
||||
the other five bulk collection GETs so a client can page **backward through** the
|
||||
visibility window instead of stalling at the newest `MAX_QUERY_LIMIT` (1000) rows.
|
||||
Motivated by an external consumer (`l5yth/meshint`) that could not retrieve more
|
||||
than 1000 nodes from `GET /api/nodes`. Read-side only; integrates with the GET
|
||||
routes in `web/lib/potato_mesh/application/routes/api.rb`, the query helpers in
|
||||
`web/lib/potato_mesh/application/queries/` (`node_queries.rb`,
|
||||
`telemetry_queries.rb`, `federation_queries.rb`; `common.rb` already provides the
|
||||
`coerce_positive_or_nil` cursor coercion), and the GET-window documentation in
|
||||
`data/mesh_ingestor/CONTRACTS.md`.
|
||||
|
||||
**Conflict check against existing decisions.** *D8 (stable contract + GET window
|
||||
floors)* — **extends**: `before` is a new optional, additive parameter (absent ⇒
|
||||
today's behavior) and only ever *narrows*, so it needs no version bump and no
|
||||
federation-compat fallback (unlike the 0.7.0 `/api/stats` break, **S1**). *C4
|
||||
(floors cannot be widened)* / *C7 (messages `before`)* — **extends**: the proven
|
||||
#796 keyset cursor is applied uniformly; the `MAX(since, floor)` clamp and
|
||||
`MAX_QUERY_LIMIT` cap are untouched. *Invariant I (apex)* — **consistent**:
|
||||
read-side query param, no broker/dependency/egress. *Invariant II (privacy)* —
|
||||
**consistent**: opt-out / `CLIENT_HIDDEN` / private-mode gates are unchanged and a
|
||||
narrowing upper bound can never surface a hidden row; no new on-disk retention.
|
||||
*Invariant IV (parity)* — **extends**: the cursor is protocol-neutral and composes
|
||||
with the existing `?protocol=` filter. No invariant is contradicted, so this
|
||||
feature adds new decisions without amending any prior one.
|
||||
|
||||
| # | Decision | Source |
|
||||
| --- | --- | --- |
|
||||
| **BP1** | **Uniform `?before=` on the six bulk collections.** `GET /api/{nodes, positions, telemetry, neighbors, traces, ingestors}` accept an optional `?before=<unix_seconds>` upper-bound cursor, mirroring the existing `GET /api/messages` behavior (**C7**). `before` is an **inclusive** (`<=`) ceiling on each route's **primary sort column** — the column it already `ORDER BY … DESC`: `rx_time` for `positions`/`telemetry`/`neighbors`/`traces`, `last_heard` for `nodes`, `last_seen_time` for `ingestors`. The per-id routes (`/api/.../:id`) and `/api/instances` are **out of scope**. | interview |
|
||||
| **BP2** | **Narrows only — never widens (preserves C4/D8).** `before` composes with the existing server-side window floor as an *additional upper bound*: the effective window is `MAX(since, floor) ≤ t ≤ before`. Because `before` only removes *newer* rows, no value can reach past the 7-day / 28-day floor (a `before` older than the floor simply yields fewer/zero rows; a `before` in the future is a no-op). It therefore needs **no clamp** of its own (unlike `since`, which is clamped *up* to the floor). A non-positive or non-integer `before` is ignored as absent via the existing `coerce_positive_or_nil`, identical to messages. `MAX_QUERY_LIMIT` per request is unchanged — `before` pages *through* the window, not *beyond* it. | interview + code |
|
||||
| **BP3** | **Keyset mechanics identical to messages (#796).** The caller walks newest → oldest: each page is `ORDER BY <sort_col> DESC LIMIT n`, then the **oldest `<sort_col>` value of the page** becomes the next `before`, de-duplicating by the collection's id client-side. The inclusive boundary deliberately overlaps consecutive pages by any rows sharing the boundary second so none is skipped; client dedup collapses the overlap. This is the established **C7 / PL-A2** walk applied uniformly. | interview + code |
|
||||
| **BP4** | **Additive & backward-compatible — no version bump.** `before` is a new optional query parameter; when absent every route behaves exactly as today, and existing consumers (including the federation crawl) are unaffected. Unlike **S1**, this is *not* a breaking change: no manifest version bump and no old/new shape fallback are required. **Extends** D8's backward-compatibility rule rather than amending it. | interview |
|
||||
| **BP5** | **Protocol-neutral (Invariant IV).** The cursor is identical for all rows regardless of protocol and composes with the existing `?protocol=` filter; neither Meshtastic nor MeshCore is privileged. | interview |
|
||||
| **BP6** | **Apex & privacy untouched (Invariants I/II).** Read-side only — no broker, dependency, or egress (I). The existing opt-out, `CLIENT_HIDDEN`, and private-mode behaviors are unchanged, and because `before` only narrows it can never expose a row a route would otherwise hide (II). No new on-disk retention is introduced. | interview + code |
|
||||
| **BP7** | **Cache-bypass parity with messages.** A request carrying `before` (like one carrying `since > 0`) **bypasses** the shared `ApiCache` response cache, which only memoises the default newest-page feed; the cache key for the cached (no-`before`, no-`since`) path is unchanged, so history pages never pollute or evict the hot newest-page entry. | code |
|
||||
| **BP8** | **Engineering bar (D9).** Ships with 100% unit tests across the route and query layers (mirroring the existing messages-`before` specs), RDoc on every edited method, Apache headers intact, `rufo`-clean; all existing Ruby/JS/Python suites stay green. | CLAUDE.md |
|
||||
| **BP9** | **Out of scope (deferred, tracked).** This feature ships the **server capability only**. (a) Wiring the browser data cache / JS data-fetchers to *backfill* collections via `before` (beyond the existing messages pager) is a tracked follow-up — the capability is unblocked, not wired. (b) The lone camelCase query params `windowSeconds` / `bucketSeconds` on `GET /api/telemetry/aggregated` and (c) the missing `limit` / `since` / `protocol` params on `GET /api/instances` are recorded here as known API-handling inconsistencies and tracked as **separate follow-ups**, deliberately excluded to keep this feature compartmentalized. | interview |
|
||||
@@ -197,6 +197,43 @@ Federation peers should not assume an unbounded historical window: a peer that r
|
||||
|
||||
The constants live in `web/lib/potato_mesh/config.rb` (`week_seconds`, `four_weeks_seconds`).
|
||||
|
||||
### GET endpoint backward pagination (`?before=`)
|
||||
|
||||
The six bulk collection endpoints — `GET /api/nodes`, `/api/positions`,
|
||||
`/api/telemetry`, `/api/neighbors`, `/api/traces`, and `/api/ingestors` — plus the
|
||||
pre-existing `GET /api/messages` cursor accept an optional `?before=<unix_seconds>`
|
||||
**inclusive upper-bound cursor** for backward pagination. It is the companion to
|
||||
`?since=`: where `since` raises the lower bound of the window, `before` lowers the
|
||||
upper bound. `before` bounds each route's **primary sort column** — the column it
|
||||
already orders by, newest first:
|
||||
|
||||
| Route | `before` bounds |
|
||||
| --- | --- |
|
||||
| `GET /api/nodes` | `last_heard` |
|
||||
| `GET /api/messages` | `rx_time` |
|
||||
| `GET /api/positions` | `rx_time` |
|
||||
| `GET /api/telemetry` | `rx_time` |
|
||||
| `GET /api/neighbors` | `rx_time` |
|
||||
| `GET /api/traces` | `rx_time` |
|
||||
| `GET /api/ingestors` | `last_seen_time` |
|
||||
|
||||
To page backward through more than one `limit`-sized response (the per-request cap
|
||||
is `MAX_QUERY_LIMIT` = 1000), walk newest → oldest: fetch a page, then re-request
|
||||
with `before` set to the **oldest sort-column value** in the page just received,
|
||||
de-duplicating rows by their id. The inclusive `<=` boundary intentionally repeats
|
||||
any row that shares the boundary second, so none is skipped across the page break;
|
||||
the client's id-dedup collapses the one-row overlap. Repeat until a short page
|
||||
(fewer than `limit` rows) signals the window is exhausted. This is how a client
|
||||
retrieves **every** in-window row instead of stalling at the newest 1000.
|
||||
|
||||
`before` **only ever narrows** the result set, so — exactly like `since` — it
|
||||
**cannot widen** the window past the route's floor in the table above: a `before`
|
||||
older than the floor merely returns fewer rows (the floor still clamps the lower
|
||||
bound), and a `before` newer than "now" is a no-op. A non-positive or non-integer
|
||||
`before` is ignored (treated as absent). The cursor composes with `?protocol=` and
|
||||
is protocol-neutral. The per-id routes (`GET /api/.../:id`) and `GET /api/instances`
|
||||
do **not** accept `before`.
|
||||
|
||||
### GET /api/stats response shape
|
||||
|
||||
> **Breaking change in 0.7.0.** Before 0.7.0 the payload was flat —
|
||||
|
||||
@@ -44,18 +44,11 @@ module PotatoMesh
|
||||
where_clauses << "m.rx_time >= ?"
|
||||
params << since_threshold
|
||||
|
||||
# Upper-bound cursor for backward pagination (issue #796). When set,
|
||||
# +before+ is an *inclusive* ceiling on +rx_time+: the client walks it
|
||||
# backward (newest -> oldest), passing the oldest +rx_time+ of each page
|
||||
# as the next cursor and de-duplicating by +id+ client-side, so rows that
|
||||
# share the boundary second are never skipped. Because the cursor only
|
||||
# ever *narrows* the result set, the seven-day floor above still bounds
|
||||
# the window — callers cannot use +before+ to reach further back.
|
||||
before_cursor = coerce_positive_or_nil(before)
|
||||
if before_cursor
|
||||
where_clauses << "m.rx_time <= ?"
|
||||
params << before_cursor
|
||||
end
|
||||
# Inclusive upper-bound cursor for backward pagination (issue #796;
|
||||
# SPEC BP1-BP3), shared with every bulk collection via the helper. It
|
||||
# only ever *narrows*, so the seven-day floor above still bounds the
|
||||
# window — callers cannot use +before+ to reach further back.
|
||||
append_before_filter(where_clauses, params, before, column: "m.rx_time")
|
||||
|
||||
unless include_encrypted
|
||||
where_clauses << "COALESCE(TRIM(m.encrypted), '') = ''"
|
||||
|
||||
@@ -142,6 +142,40 @@ module PotatoMesh
|
||||
params << protocol
|
||||
end
|
||||
|
||||
# Append an inclusive upper-bound (+before+) cursor to an in-flight WHERE
|
||||
# clause builder for backward pagination of a bulk collection endpoint
|
||||
# (SPEC BP1-BP3; the +/api/messages+ #796 cursor generalised to every
|
||||
# list route).
|
||||
#
|
||||
# +before+ is an *inclusive* (+<=+) ceiling on the route's primary sort
|
||||
# column — the column it already +ORDER BY ... DESC+ — so a client can
|
||||
# walk the feed newest -> oldest by passing the oldest sort value of each
|
||||
# page as the next +before+ and de-duplicating by id, never skipping a row
|
||||
# that shares the boundary second. Because the cursor only ever *narrows*
|
||||
# the result set it cannot widen the server-side window floor
|
||||
# (SPEC BP2 / acceptance C4): a +before+ older than the floor merely
|
||||
# returns fewer rows, and a +before+ in the future is a no-op.
|
||||
# Non-positive / non-integer values are ignored as absent via
|
||||
# {coerce_positive_or_nil}, matching the messages route.
|
||||
#
|
||||
# @param where_clauses [Array<String>] accumulating WHERE conditions (mutated).
|
||||
# @param params [Array] accumulating bind parameters (mutated).
|
||||
# @param before [Object] raw +before+ cursor from the request.
|
||||
# @param column [String] qualified sort column to bound (e.g. ``"last_heard"``,
|
||||
# ``"m.rx_time"``). Must match {SAFE_COLUMN_IDENTIFIER}; arbitrary user
|
||||
# input is not accepted.
|
||||
# @return [Integer, nil] the coerced cursor that was applied, or +nil+ when
|
||||
# +before+ was absent/invalid and no clause was added.
|
||||
def append_before_filter(where_clauses, params, before, column:)
|
||||
assert_safe_column_identifier!(column)
|
||||
cursor = coerce_positive_or_nil(before)
|
||||
return nil unless cursor
|
||||
|
||||
where_clauses << "#{column} <= ?"
|
||||
params << cursor
|
||||
cursor
|
||||
end
|
||||
|
||||
# Normalise a caller-provided limit to a sane, positive integer.
|
||||
#
|
||||
# @param limit [Object] value coerced to an integer.
|
||||
|
||||
@@ -22,8 +22,10 @@ module PotatoMesh
|
||||
# @param limit [Integer] maximum number of rows to return.
|
||||
# @param node_ref [String, Integer, nil] optional node reference to scope results.
|
||||
# @param since [Integer] unix timestamp threshold applied in addition to the rolling window.
|
||||
# @param before [Integer, nil] inclusive upper-bound +rx_time+ cursor for
|
||||
# backward pagination (SPEC BP1); rows newer than this are excluded.
|
||||
# @return [Array<Hash>] compacted position rows suitable for API responses.
|
||||
def query_positions(limit, node_ref: nil, since: 0, protocol: nil)
|
||||
def query_positions(limit, node_ref: nil, since: 0, before: nil, protocol: nil)
|
||||
limit = coerce_query_limit(limit)
|
||||
db = open_database(readonly: true)
|
||||
db.results_as_hash = true
|
||||
@@ -44,6 +46,10 @@ module PotatoMesh
|
||||
params.concat(clause.last)
|
||||
end
|
||||
|
||||
# Inclusive upper-bound cursor for backward pagination (SPEC BP1);
|
||||
# bounds the +rx_time+ sort column.
|
||||
append_before_filter(where_clauses, params, before, column: "rx_time")
|
||||
|
||||
append_opt_out_filter(where_clauses, params, opt_out_node_id_filter("node_id"))
|
||||
append_protocol_filter(where_clauses, params, protocol)
|
||||
|
||||
@@ -84,8 +90,10 @@ module PotatoMesh
|
||||
# @param limit [Integer] maximum number of rows to return.
|
||||
# @param node_ref [String, Integer, nil] optional node reference to scope results.
|
||||
# @param since [Integer] unix timestamp threshold applied in addition to the rolling window for collections.
|
||||
# @param before [Integer, nil] inclusive upper-bound +rx_time+ cursor for
|
||||
# backward pagination (SPEC BP1); rows newer than this are excluded.
|
||||
# @return [Array<Hash>] compacted neighbor rows suitable for API responses.
|
||||
def query_neighbors(limit, node_ref: nil, since: 0, protocol: nil)
|
||||
def query_neighbors(limit, node_ref: nil, since: 0, before: nil, protocol: nil)
|
||||
limit = coerce_query_limit(limit)
|
||||
db = open_database(readonly: true)
|
||||
db.results_as_hash = true
|
||||
@@ -110,6 +118,10 @@ module PotatoMesh
|
||||
# Either endpoint of the neighbour relationship may carry the
|
||||
# opt-out marker — filter both so a silenced node never appears as
|
||||
# a source or destination of an RF link.
|
||||
# Inclusive upper-bound cursor for backward pagination (SPEC BP1);
|
||||
# bounds the +rx_time+ sort column.
|
||||
append_before_filter(where_clauses, params, before, column: "rx_time")
|
||||
|
||||
append_opt_out_filter(where_clauses, params, opt_out_node_id_filter("node_id"))
|
||||
append_opt_out_filter(where_clauses, params, opt_out_node_id_filter("neighbor_id"))
|
||||
append_protocol_filter(where_clauses, params, protocol)
|
||||
@@ -141,8 +153,10 @@ module PotatoMesh
|
||||
# @param limit [Integer] maximum number of rows to return.
|
||||
# @param node_ref [String, Integer, nil] optional node reference to scope results.
|
||||
# @param since [Integer] unix timestamp threshold applied in addition to the rolling window.
|
||||
# @param before [Integer, nil] inclusive upper-bound +rx_time+ cursor for
|
||||
# backward pagination (SPEC BP1); rows newer than this are excluded.
|
||||
# @return [Array<Hash>] compacted trace rows suitable for API responses.
|
||||
def query_traces(limit, node_ref: nil, since: 0, protocol: nil)
|
||||
def query_traces(limit, node_ref: nil, since: 0, before: nil, protocol: nil)
|
||||
limit = coerce_query_limit(limit)
|
||||
db = open_database(readonly: true)
|
||||
db.results_as_hash = true
|
||||
@@ -173,6 +187,10 @@ module PotatoMesh
|
||||
# filtered separately at hydration time so a trace that only relays
|
||||
# through a silenced node still surfaces with the offending hop
|
||||
# removed.
|
||||
# Inclusive upper-bound cursor for backward pagination (SPEC BP1);
|
||||
# bounds the +rx_time+ sort column.
|
||||
append_before_filter(where_clauses, params, before, column: "rx_time")
|
||||
|
||||
append_opt_out_filter(where_clauses, params, opt_out_node_num_filter("src"))
|
||||
append_opt_out_filter(where_clauses, params, opt_out_node_num_filter("dest"))
|
||||
append_protocol_filter(where_clauses, params, protocol)
|
||||
|
||||
@@ -135,8 +135,10 @@ module PotatoMesh
|
||||
# @param limit [Integer] maximum number of rows to return.
|
||||
# @param node_ref [String, Integer, nil] optional node reference to narrow results.
|
||||
# @param since [Integer] unix timestamp threshold applied in addition to the rolling window for collections.
|
||||
# @param before [Integer, nil] inclusive upper-bound +last_heard+ cursor for
|
||||
# backward pagination (SPEC BP1); nodes newer than this are excluded.
|
||||
# @return [Array<Hash>] compacted node rows suitable for API responses.
|
||||
def query_nodes(limit, node_ref: nil, since: 0, protocol: nil)
|
||||
def query_nodes(limit, node_ref: nil, since: 0, before: nil, protocol: nil)
|
||||
limit = coerce_query_limit(limit)
|
||||
db = open_database(readonly: true)
|
||||
db.results_as_hash = true
|
||||
@@ -159,6 +161,13 @@ module PotatoMesh
|
||||
params << since_threshold
|
||||
end
|
||||
|
||||
# Inclusive upper-bound cursor for backward pagination (SPEC BP1-BP3).
|
||||
# Bounds the +last_heard+ sort column so callers can page newest ->
|
||||
# oldest past the +MAX_QUERY_LIMIT+ cap; only ever narrows (BP2). The
|
||||
# per-id route never supplies +before+, so single-node lookups are
|
||||
# unaffected.
|
||||
append_before_filter(where_clauses, params, before, column: "last_heard")
|
||||
|
||||
if private_mode?
|
||||
where_clauses << "(role IS NULL OR role <> 'CLIENT_HIDDEN')"
|
||||
end
|
||||
@@ -224,8 +233,10 @@ module PotatoMesh
|
||||
#
|
||||
# @param limit [Integer] maximum number of ingestors to return.
|
||||
# @param since [Integer] unix timestamp threshold applied in addition to the rolling window for collections.
|
||||
# @param before [Integer, nil] inclusive upper-bound +last_seen_time+ cursor
|
||||
# for backward pagination (SPEC BP1); ingestors newer than this are excluded.
|
||||
# @return [Array<Hash>] compacted ingestor rows suitable for API responses.
|
||||
def query_ingestors(limit, since: 0, protocol: nil)
|
||||
def query_ingestors(limit, since: 0, before: nil, protocol: nil)
|
||||
limit = coerce_query_limit(limit)
|
||||
db = open_database(readonly: true)
|
||||
db.results_as_hash = true
|
||||
@@ -237,6 +248,9 @@ module PotatoMesh
|
||||
since_threshold = normalize_since_threshold(since, floor: cutoff)
|
||||
where_clauses = ["last_seen_time >= ?"]
|
||||
params = [since_threshold]
|
||||
# Inclusive upper-bound cursor for backward pagination (SPEC BP1);
|
||||
# bounds the +last_seen_time+ sort column.
|
||||
append_before_filter(where_clauses, params, before, column: "last_seen_time")
|
||||
append_opt_out_filter(where_clauses, params, opt_out_node_id_filter("node_id"))
|
||||
append_protocol_filter(where_clauses, params, protocol)
|
||||
sql = <<~SQL
|
||||
|
||||
@@ -22,8 +22,10 @@ module PotatoMesh
|
||||
# @param limit [Integer] maximum number of rows to return.
|
||||
# @param node_ref [String, Integer, nil] optional node reference to scope results.
|
||||
# @param since [Integer] unix timestamp threshold applied in addition to the rolling window for collections.
|
||||
# @param before [Integer, nil] inclusive upper-bound +rx_time+ cursor for
|
||||
# backward pagination (SPEC BP1); rows newer than this are excluded.
|
||||
# @return [Array<Hash>] compacted telemetry rows suitable for API responses.
|
||||
def query_telemetry(limit, node_ref: nil, since: 0, protocol: nil)
|
||||
def query_telemetry(limit, node_ref: nil, since: 0, before: nil, protocol: nil)
|
||||
limit = coerce_query_limit(limit)
|
||||
db = open_database(readonly: true)
|
||||
db.results_as_hash = true
|
||||
@@ -44,6 +46,10 @@ module PotatoMesh
|
||||
params.concat(clause.last)
|
||||
end
|
||||
|
||||
# Inclusive upper-bound cursor for backward pagination (SPEC BP1);
|
||||
# bounds the +rx_time+ sort column.
|
||||
append_before_filter(where_clauses, params, before, column: "rx_time")
|
||||
|
||||
append_opt_out_filter(where_clauses, params, opt_out_node_id_filter("node_id"))
|
||||
append_protocol_filter(where_clauses, params, protocol)
|
||||
|
||||
|
||||
@@ -89,10 +89,15 @@ module PotatoMesh
|
||||
since = params["since"]
|
||||
protocol = sanitize_protocol(params["protocol"])
|
||||
since_val = coerce_integer(since) || 0
|
||||
# Inclusive upper-bound cursor for backward pagination (SPEC BP1). A
|
||||
# request carrying +before+ is a history page, so — like a +since+
|
||||
# query — it bypasses the shared cache (which only memoises the
|
||||
# default newest-page feed).
|
||||
before = coerce_positive_or_nil(params["before"])
|
||||
priv = private_mode? ? 1 : 0
|
||||
|
||||
if since_val > 0
|
||||
json_body = query_nodes(limit, since: since, protocol: protocol).to_json
|
||||
if since_val > 0 || before
|
||||
json_body = query_nodes(limit, since: since, before: before, protocol: protocol).to_json
|
||||
etag Digest::MD5.hexdigest(json_body), kind: :weak
|
||||
api_cache_control
|
||||
json_body
|
||||
@@ -139,9 +144,11 @@ module PotatoMesh
|
||||
protocol = sanitize_protocol(params["protocol"])
|
||||
since = params["since"]
|
||||
since_val = coerce_integer(since) || 0
|
||||
# Backward-pagination cursor (SPEC BP1); bypasses the cache like +since+.
|
||||
before = coerce_positive_or_nil(params["before"])
|
||||
|
||||
if since_val > 0
|
||||
json_body = query_ingestors(limit, since: since, protocol: protocol).to_json
|
||||
if since_val > 0 || before
|
||||
json_body = query_ingestors(limit, since: since, before: before, protocol: protocol).to_json
|
||||
etag Digest::MD5.hexdigest(json_body), kind: :weak
|
||||
api_cache_control
|
||||
json_body
|
||||
@@ -164,8 +171,7 @@ module PotatoMesh
|
||||
# Upper-bound cursor for backward pagination (issue #796). A request
|
||||
# carrying +before+ is a history page, so it bypasses the shared
|
||||
# response cache (which only memoises the default newest-page feed).
|
||||
before = coerce_integer(params["before"])
|
||||
before = nil if before && before <= 0
|
||||
before = coerce_positive_or_nil(params["before"])
|
||||
protocol = sanitize_protocol(params["protocol"])
|
||||
enc_key = include_encrypted ? "1" : "0"
|
||||
|
||||
@@ -210,9 +216,11 @@ module PotatoMesh
|
||||
since = params["since"]
|
||||
protocol = sanitize_protocol(params["protocol"])
|
||||
since_val = coerce_integer(since) || 0
|
||||
# Backward-pagination cursor (SPEC BP1); bypasses the cache like +since+.
|
||||
before = coerce_positive_or_nil(params["before"])
|
||||
|
||||
if since_val > 0
|
||||
json_body = query_positions(limit, since: since, protocol: protocol).to_json
|
||||
if since_val > 0 || before
|
||||
json_body = query_positions(limit, since: since, before: before, protocol: protocol).to_json
|
||||
etag Digest::MD5.hexdigest(json_body), kind: :weak
|
||||
api_cache_control
|
||||
json_body
|
||||
@@ -243,9 +251,11 @@ module PotatoMesh
|
||||
since = params["since"]
|
||||
protocol = sanitize_protocol(params["protocol"])
|
||||
since_val = coerce_integer(since) || 0
|
||||
# Backward-pagination cursor (SPEC BP1); bypasses the cache like +since+.
|
||||
before = coerce_positive_or_nil(params["before"])
|
||||
|
||||
if since_val > 0
|
||||
json_body = query_neighbors(limit, since: since, protocol: protocol).to_json
|
||||
if since_val > 0 || before
|
||||
json_body = query_neighbors(limit, since: since, before: before, protocol: protocol).to_json
|
||||
etag Digest::MD5.hexdigest(json_body), kind: :weak
|
||||
api_cache_control
|
||||
json_body
|
||||
@@ -276,9 +286,11 @@ module PotatoMesh
|
||||
since = params["since"]
|
||||
protocol = sanitize_protocol(params["protocol"])
|
||||
since_val = coerce_integer(since) || 0
|
||||
# Backward-pagination cursor (SPEC BP1); bypasses the cache like +since+.
|
||||
before = coerce_positive_or_nil(params["before"])
|
||||
|
||||
if since_val > 0
|
||||
json_body = query_telemetry(limit, since: since, protocol: protocol).to_json
|
||||
if since_val > 0 || before
|
||||
json_body = query_telemetry(limit, since: since, before: before, protocol: protocol).to_json
|
||||
etag Digest::MD5.hexdigest(json_body), kind: :weak
|
||||
api_cache_control
|
||||
json_body
|
||||
@@ -362,9 +374,11 @@ module PotatoMesh
|
||||
since = params["since"]
|
||||
protocol = sanitize_protocol(params["protocol"])
|
||||
since_val = coerce_integer(since) || 0
|
||||
# Backward-pagination cursor (SPEC BP1); bypasses the cache like +since+.
|
||||
before = coerce_positive_or_nil(params["before"])
|
||||
|
||||
if since_val > 0
|
||||
json_body = query_traces(limit, since: since, protocol: protocol).to_json
|
||||
if since_val > 0 || before
|
||||
json_body = query_traces(limit, since: since, before: before, protocol: protocol).to_json
|
||||
etag Digest::MD5.hexdigest(json_body), kind: :weak
|
||||
api_cache_control
|
||||
json_body
|
||||
|
||||
@@ -112,6 +112,39 @@ RSpec.describe "Potato Mesh Sinatra app" do
|
||||
ensure_self_instance_record!
|
||||
end
|
||||
|
||||
# Walk a bulk collection endpoint backward with the +before+ cursor the way an
|
||||
# API client must, returning the set of unique record ids recovered. Generic
|
||||
# over the route path, the id field used for client-side de-duplication, and
|
||||
# the primary sort field the cursor advances on (SPEC BP1-BP3).
|
||||
#
|
||||
# @param path [String] collection route, e.g. ``"/api/positions"``.
|
||||
# @param id_key [String] response field uniquely identifying a row.
|
||||
# @param sort_key [String] response field the +before+ cursor bounds.
|
||||
# @return [Array] the de-duplicated ids recovered across every page.
|
||||
def walk_before(path, id_key:, sort_key:)
|
||||
cap = PotatoMesh::App::Queries::MAX_QUERY_LIMIT
|
||||
seen = {}
|
||||
cursor = nil
|
||||
pages = 0
|
||||
loop do
|
||||
url = "#{path}?limit=#{cap}"
|
||||
url += "&before=#{cursor}" if cursor
|
||||
get url
|
||||
expect(last_response).to be_ok
|
||||
rows = JSON.parse(last_response.body)
|
||||
# The per-request cap is never exceeded by a single response.
|
||||
expect(rows.size).to be <= cap
|
||||
added = rows.reject { |row| seen.key?(row[id_key]) }
|
||||
added.each { |row| seen[row[id_key]] = true }
|
||||
pages += 1
|
||||
break if rows.size < cap # window exhausted
|
||||
break if added.empty? # no progress (server ignoring `before`)
|
||||
break if pages >= 10 # hard safety bound against an infinite loop
|
||||
cursor = rows.map { |row| row[sort_key] }.min
|
||||
end
|
||||
seen.keys
|
||||
end
|
||||
|
||||
# Retrieve the number of rows stored in the instances table.
|
||||
#
|
||||
# @return [Integer] count of stored instance records.
|
||||
@@ -6107,6 +6140,204 @@ RSpec.describe "Potato Mesh Sinatra app" do
|
||||
end
|
||||
|
||||
describe "GET /api/nodes" do
|
||||
# Regression-style coverage for SPEC BP1-BP6: more than MAX_QUERY_LIMIT
|
||||
# nodes inside the seven-day window must all remain reachable by paging
|
||||
# backward with an inclusive `before` cursor (the `meshint` 1000-node cap).
|
||||
it "exposes every in-window node through before pagination" do
|
||||
clear_database
|
||||
allow(Time).to receive(:now).and_return(reference_time)
|
||||
now = reference_time.to_i
|
||||
|
||||
cap = PotatoMesh::App::Queries::MAX_QUERY_LIMIT
|
||||
total = cap + 500
|
||||
# Seed more than one page of nodes, all inside the seven-day window, each
|
||||
# with a distinct last_heard so the keyset cursor is unambiguous.
|
||||
with_db do |db|
|
||||
db.transaction do
|
||||
total.times do |i|
|
||||
lh = now - 60 - i * 30
|
||||
db.execute(
|
||||
"INSERT INTO nodes(node_id, num, short_name, long_name, hw_model, role, last_heard, first_heard) VALUES(?,?,?,?,?,?,?,?)",
|
||||
["!%08x" % (0x1000 + i), 0x1000 + i, "n#{i}", "Node #{i}", "TBEAM", "CLIENT", lh, lh],
|
||||
)
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
# Walk the listing the way an external client (l5yth/meshint) must: pull a
|
||||
# page, then ask for everything at-or-before the oldest last_heard already
|
||||
# seen. Without an upper-bound cursor the server cannot return anything
|
||||
# past the newest `cap` rows, so the walk would stall before reaching them.
|
||||
recovered = walk_before("/api/nodes", id_key: "node_id", sort_key: "last_heard")
|
||||
|
||||
expect(recovered.size).to eq(total)
|
||||
end
|
||||
|
||||
it "before cannot widen the window past the floor and ignores a non-positive cursor" do
|
||||
clear_database
|
||||
allow(Time).to receive(:now).and_return(reference_time)
|
||||
now = reference_time.to_i
|
||||
stale_last = now - (PotatoMesh::Config.week_seconds + 4 * 60 * 60)
|
||||
fresh_last = now - 30
|
||||
|
||||
with_db do |db|
|
||||
db.execute(
|
||||
"INSERT INTO nodes(node_id, short_name, long_name, hw_model, role, last_heard, first_heard) VALUES(?,?,?,?,?,?,?)",
|
||||
["!stalenode", "stal", "Stale", "TBEAM", "CLIENT", stale_last, stale_last],
|
||||
)
|
||||
db.execute(
|
||||
"INSERT INTO nodes(node_id, short_name, long_name, hw_model, role, last_heard, first_heard) VALUES(?,?,?,?,?,?,?)",
|
||||
["!freshnode", "frsh", "Fresh", "TBEAM", "CLIENT", fresh_last, fresh_last],
|
||||
)
|
||||
end
|
||||
|
||||
# A `before` at "now" cannot reach the stale row: the seven-day floor still
|
||||
# clamps the lower bound, so `before` only ever narrows, never widens.
|
||||
get "/api/nodes?before=#{now}"
|
||||
expect(last_response).to be_ok
|
||||
ids = JSON.parse(last_response.body).map { |r| r["node_id"] }
|
||||
expect(ids).to include("!freshnode")
|
||||
expect(ids).not_to include("!stalenode")
|
||||
|
||||
# A `before` older than the floor returns nothing beyond the floor (the only
|
||||
# in-window row is newer than this ceiling, so the result is empty).
|
||||
get "/api/nodes?before=#{stale_last}"
|
||||
expect(last_response).to be_ok
|
||||
expect(JSON.parse(last_response.body)).to be_empty
|
||||
|
||||
# A non-positive / non-integer `before` is ignored (treated as absent), so
|
||||
# the fresh row still comes back rather than being filtered by a bogus
|
||||
# ceiling — parity with the /api/messages coercion.
|
||||
["0", "-5", "abc"].each do |bogus|
|
||||
get "/api/nodes?before=#{bogus}"
|
||||
expect(last_response).to be_ok
|
||||
ids = JSON.parse(last_response.body).map { |r| r["node_id"] }
|
||||
expect(ids).to include("!freshnode"), "before=#{bogus} should be ignored"
|
||||
end
|
||||
end
|
||||
|
||||
it "before pagination boundary is inclusive and protocol-neutral" do
|
||||
clear_database
|
||||
allow(Time).to receive(:now).and_return(reference_time)
|
||||
now = reference_time.to_i
|
||||
boundary = now - 100
|
||||
|
||||
with_db do |db|
|
||||
# Two nodes sharing the exact boundary last_heard, one per protocol, plus
|
||||
# a newer node above the boundary.
|
||||
db.execute(
|
||||
"INSERT INTO nodes(node_id, short_name, long_name, hw_model, role, last_heard, first_heard, protocol) VALUES(?,?,?,?,?,?,?,?)",
|
||||
["!edgecore0", "ec", "Edge Core", "TBEAM", "CLIENT", boundary, boundary, "meshcore"],
|
||||
)
|
||||
db.execute(
|
||||
"INSERT INTO nodes(node_id, short_name, long_name, hw_model, role, last_heard, first_heard, protocol) VALUES(?,?,?,?,?,?,?,?)",
|
||||
["!edgetast0", "et", "Edge Tastic", "TBEAM", "CLIENT", boundary, boundary, "meshtastic"],
|
||||
)
|
||||
db.execute(
|
||||
"INSERT INTO nodes(node_id, short_name, long_name, hw_model, role, last_heard, first_heard, protocol) VALUES(?,?,?,?,?,?,?,?)",
|
||||
["!newer0001", "nw", "Newer", "TBEAM", "CLIENT", now - 10, now - 10, "meshcore"],
|
||||
)
|
||||
end
|
||||
|
||||
# Inclusive <= ceiling: both rows sharing the boundary second are returned
|
||||
# when the boundary is used as `before` (the one-row overlap a client dedups).
|
||||
get "/api/nodes?before=#{boundary}"
|
||||
expect(last_response).to be_ok
|
||||
ids = JSON.parse(last_response.body).map { |r| r["node_id"] }
|
||||
expect(ids).to include("!edgecore0", "!edgetast0")
|
||||
expect(ids).not_to include("!newer0001")
|
||||
|
||||
# `before` composes with `protocol`, privileging neither protocol.
|
||||
get "/api/nodes?before=#{boundary}&protocol=meshcore"
|
||||
expect(last_response).to be_ok
|
||||
ids = JSON.parse(last_response.body).map { |r| r["node_id"] }
|
||||
expect(ids).to eq(["!edgecore0"])
|
||||
end
|
||||
|
||||
it "before pagination honors privacy and opt-out filters" do
|
||||
clear_database
|
||||
allow(Time).to receive(:now).and_return(reference_time)
|
||||
now = reference_time.to_i
|
||||
marker = PotatoMesh::Config.node_opt_out_marker
|
||||
|
||||
with_db do |db|
|
||||
db.execute(
|
||||
"INSERT INTO nodes(node_id, short_name, long_name, hw_model, role, last_heard, first_heard) VALUES(?,?,?,?,?,?,?)",
|
||||
["!plainnode", "pln", "Plain", "TBEAM", "CLIENT", now - 50, now - 50],
|
||||
)
|
||||
db.execute(
|
||||
"INSERT INTO nodes(node_id, short_name, long_name, hw_model, role, last_heard, first_heard) VALUES(?,?,?,?,?,?,?)",
|
||||
["!optoutnod", "opt", "Quiet #{marker} One", "TBEAM", "CLIENT", now - 60, now - 60],
|
||||
)
|
||||
db.execute(
|
||||
"INSERT INTO nodes(node_id, short_name, long_name, hw_model, role, last_heard, first_heard) VALUES(?,?,?,?,?,?,?)",
|
||||
["!hiddennod", "hid", "Hidden", "TBEAM", "CLIENT_HIDDEN", now - 70, now - 70],
|
||||
)
|
||||
end
|
||||
|
||||
# A backward-paginated request still applies the opt-out filter: a node
|
||||
# carrying the marker never appears even when within the `before` ceiling.
|
||||
get "/api/nodes?before=#{now}"
|
||||
expect(last_response).to be_ok
|
||||
ids = JSON.parse(last_response.body).map { |r| r["node_id"] }
|
||||
expect(ids).to include("!plainnode")
|
||||
expect(ids).not_to include("!optoutnod")
|
||||
|
||||
# In private mode the CLIENT_HIDDEN exclusion also still applies under a
|
||||
# `before` walk — narrowing can never surface a hidden row (Invariant II).
|
||||
allow(PotatoMesh::Config).to receive(:private_mode_enabled?).and_return(true)
|
||||
get "/api/nodes?before=#{now}"
|
||||
expect(last_response).to be_ok
|
||||
ids = JSON.parse(last_response.body).map { |r| r["node_id"] }
|
||||
expect(ids).to include("!plainnode")
|
||||
expect(ids).not_to include("!hiddennod")
|
||||
expect(ids).not_to include("!optoutnod")
|
||||
end
|
||||
|
||||
it "before bypasses the response cache" do
|
||||
clear_database
|
||||
allow(Time).to receive(:now).and_return(reference_time)
|
||||
now = reference_time.to_i
|
||||
# Start from a clean cache so a prior example's entry cannot mask the
|
||||
# behaviour under test (the store is process-global with a monotonic TTL).
|
||||
PotatoMesh::App::ApiCache.invalidate_all
|
||||
|
||||
with_db do |db|
|
||||
db.execute(
|
||||
"INSERT INTO nodes(node_id, last_heard, first_heard) VALUES(?,?,?)",
|
||||
["!cacheaaaa", now - 50, now - 50],
|
||||
)
|
||||
end
|
||||
|
||||
# Prime the newest-page cache with just node A.
|
||||
get "/api/nodes?limit=3"
|
||||
expect(last_response).to be_ok
|
||||
expect(JSON.parse(last_response.body).map { |r| r["node_id"] }).to eq(["!cacheaaaa"])
|
||||
|
||||
# Insert node B directly (no POST, so the cache is not invalidated).
|
||||
with_db do |db|
|
||||
db.execute(
|
||||
"INSERT INTO nodes(node_id, last_heard, first_heard) VALUES(?,?,?)",
|
||||
["!cachebbbb", now - 40, now - 40],
|
||||
)
|
||||
end
|
||||
|
||||
# A plain request is still served the cached page (proves the cache is live).
|
||||
get "/api/nodes?limit=3"
|
||||
expect(JSON.parse(last_response.body).map { |r| r["node_id"] }).to eq(["!cacheaaaa"])
|
||||
|
||||
# A `before` request bypasses the cache and sees the fresh node B.
|
||||
get "/api/nodes?limit=3&before=#{now}"
|
||||
expect(last_response).to be_ok
|
||||
before_ids = JSON.parse(last_response.body).map { |r| r["node_id"] }
|
||||
expect(before_ids).to include("!cacheaaaa", "!cachebbbb")
|
||||
|
||||
# Issuing the `before` request did not overwrite the hot newest-page cache
|
||||
# entry — a subsequent plain request still returns the cached page.
|
||||
get "/api/nodes?limit=3"
|
||||
expect(JSON.parse(last_response.body).map { |r| r["node_id"] }).to eq(["!cacheaaaa"])
|
||||
end
|
||||
|
||||
it "returns the stored nodes with derived timestamps" do
|
||||
import_nodes_fixture
|
||||
|
||||
@@ -6968,6 +7199,26 @@ RSpec.describe "Potato Mesh Sinatra app" do
|
||||
end
|
||||
|
||||
describe "GET /api/positions" do
|
||||
it "exposes every in-window row through before pagination" do
|
||||
clear_database
|
||||
allow(Time).to receive(:now).and_return(reference_time)
|
||||
now = reference_time.to_i
|
||||
total = PotatoMesh::App::Queries::MAX_QUERY_LIMIT + 5
|
||||
with_db do |db|
|
||||
db.transaction do
|
||||
total.times do |i|
|
||||
rx = now - 60 - i * 30
|
||||
db.execute(
|
||||
"INSERT INTO positions(id, node_id, rx_time, rx_iso, latitude, longitude) VALUES(?,?,?,?,?,?)",
|
||||
[1000 + i, "!pos00001", rx, Time.at(rx).utc.iso8601, 1.0, 2.0],
|
||||
)
|
||||
end
|
||||
end
|
||||
end
|
||||
recovered = walk_before("/api/positions", id_key: "id", sort_key: "rx_time")
|
||||
expect(recovered.size).to eq(total)
|
||||
end
|
||||
|
||||
it "returns stored positions ordered by receive time" do
|
||||
node_id = "!specfetch"
|
||||
rx_times = [reference_time.to_i - 50, reference_time.to_i - 10]
|
||||
@@ -7122,6 +7373,31 @@ RSpec.describe "Potato Mesh Sinatra app" do
|
||||
end
|
||||
|
||||
describe "GET /api/neighbors" do
|
||||
it "exposes every in-window row through before pagination" do
|
||||
clear_database
|
||||
allow(Time).to receive(:now).and_return(reference_time)
|
||||
now = reference_time.to_i
|
||||
total = PotatoMesh::App::Queries::MAX_QUERY_LIMIT + 5
|
||||
with_db do |db|
|
||||
db.transaction do
|
||||
# neighbors carries FK(node_id) and FK(neighbor_id) -> nodes(node_id),
|
||||
# so both endpoints must exist as nodes before the relationship rows.
|
||||
db.execute("INSERT INTO nodes(node_id, last_heard, first_heard) VALUES(?,?,?)", ["!nbr00001", now, now])
|
||||
total.times do |i|
|
||||
rx = now - 60 - i * 30
|
||||
nbr = "!%08x" % (0x20000 + i)
|
||||
db.execute("INSERT INTO nodes(node_id, last_heard, first_heard) VALUES(?,?,?)", [nbr, rx, rx])
|
||||
db.execute(
|
||||
"INSERT INTO neighbors(node_id, neighbor_id, rx_time) VALUES(?,?,?)",
|
||||
["!nbr00001", nbr, rx],
|
||||
)
|
||||
end
|
||||
end
|
||||
end
|
||||
recovered = walk_before("/api/neighbors", id_key: "neighbor_id", sort_key: "rx_time")
|
||||
expect(recovered.size).to eq(total)
|
||||
end
|
||||
|
||||
it "excludes neighbor records older than twenty-eight days from both bulk and per-id queries" do
|
||||
clear_database
|
||||
allow(Time).to receive(:now).and_return(reference_time)
|
||||
@@ -7261,6 +7537,26 @@ RSpec.describe "Potato Mesh Sinatra app" do
|
||||
end
|
||||
|
||||
describe "GET /api/telemetry" do
|
||||
it "exposes every in-window row through before pagination" do
|
||||
clear_database
|
||||
allow(Time).to receive(:now).and_return(reference_time)
|
||||
now = reference_time.to_i
|
||||
total = PotatoMesh::App::Queries::MAX_QUERY_LIMIT + 5
|
||||
with_db do |db|
|
||||
db.transaction do
|
||||
total.times do |i|
|
||||
rx = now - 60 - i * 30
|
||||
db.execute(
|
||||
"INSERT INTO telemetry(id, node_id, rx_time, rx_iso) VALUES(?,?,?,?)",
|
||||
[1000 + i, "!tel00001", rx, Time.at(rx).utc.iso8601],
|
||||
)
|
||||
end
|
||||
end
|
||||
end
|
||||
recovered = walk_before("/api/telemetry", id_key: "id", sort_key: "rx_time")
|
||||
expect(recovered.size).to eq(total)
|
||||
end
|
||||
|
||||
it "returns stored telemetry ordered by receive time" do
|
||||
post "/api/telemetry", telemetry_fixture.to_json, auth_headers
|
||||
expect(last_response.status).to eq(201)
|
||||
@@ -7680,6 +7976,26 @@ RSpec.describe "Potato Mesh Sinatra app" do
|
||||
end
|
||||
|
||||
describe "GET /api/traces" do
|
||||
it "exposes every in-window row through before pagination" do
|
||||
clear_database
|
||||
allow(Time).to receive(:now).and_return(reference_time)
|
||||
now = reference_time.to_i
|
||||
total = PotatoMesh::App::Queries::MAX_QUERY_LIMIT + 5
|
||||
with_db do |db|
|
||||
db.transaction do
|
||||
total.times do |i|
|
||||
rx = now - 60 - i * 30
|
||||
db.execute(
|
||||
"INSERT INTO traces(id, rx_time, rx_iso) VALUES(?,?,?)",
|
||||
[1000 + i, rx, Time.at(rx).utc.iso8601],
|
||||
)
|
||||
end
|
||||
end
|
||||
end
|
||||
recovered = walk_before("/api/traces", id_key: "id", sort_key: "rx_time")
|
||||
expect(recovered.size).to eq(total)
|
||||
end
|
||||
|
||||
it "returns stored traces ordered by receive time" do
|
||||
clear_database
|
||||
post "/api/traces", trace_fixture.to_json, auth_headers
|
||||
@@ -7785,6 +8101,26 @@ RSpec.describe "Potato Mesh Sinatra app" do
|
||||
end
|
||||
|
||||
describe "GET /api/ingestors" do
|
||||
it "exposes every in-window row through before pagination" do
|
||||
clear_database
|
||||
allow(Time).to receive(:now).and_return(reference_time)
|
||||
now = reference_time.to_i
|
||||
total = PotatoMesh::App::Queries::MAX_QUERY_LIMIT + 5
|
||||
with_db do |db|
|
||||
db.transaction do
|
||||
total.times do |i|
|
||||
ls = now - 60 - i * 30
|
||||
db.execute(
|
||||
"INSERT INTO ingestors(node_id, start_time, last_seen_time, version) VALUES(?,?,?,?)",
|
||||
["!%08x" % (0x30000 + i), ls, ls, "1.0.0"],
|
||||
)
|
||||
end
|
||||
end
|
||||
end
|
||||
recovered = walk_before("/api/ingestors", id_key: "node_id", sort_key: "last_seen_time")
|
||||
expect(recovered.size).to eq(total)
|
||||
end
|
||||
|
||||
it "uses the twenty-eight-day extended window so slow-tick ingestors stay visible" do
|
||||
clear_database
|
||||
allow(Time).to receive(:now).and_return(reference_time)
|
||||
|
||||
Reference in new issue
Block a user