Commit Graph

587 Commits

Author SHA1 Message Date
l5y cd771c4eea data: count every MeshCore RX-log frame; stamp position radio metadata (#861) 2026-07-27 11:21:16 +02:00
l5y 207150f235 web: mesh activity map card (#860)
* data,web: /api/stats packets rate as <scope>.packets.hour

* data,web: /api/stats packets as <scope>.packets.hour; total sums protocols

* web: mesh activity map card - packets/h total + per-protocol split

* web: /api/stats/activity bucketed packets/hour time-series

* web: mesh activity card draws the real 24h sparkline

* web: /charts mesh-activity figure, mobile card strip, legend-toggle fix

* web: mesh-activity design-review remediation

* web: /api/stats total sums rounded per-protocol rates + doc fixes
2026-07-26 20:59:26 +02:00
l5y b52f3949e4 data,web: mesh activity reporting & announcements (#859) 2026-07-26 09:09:28 +02:00
l5y 86f49506ff web: remediate frontend design & UX audit (#855)
* web: remediate frontend design & UX audit

* web: remediate design UX review follow-up

* web: remediate post-deploy design review (detail view, footer, marker recency)

* web: drop single-pass tag-strip in isReportedField

* web: legend shape key + pressed state + chat overflow
2026-07-25 22:06:59 +02:00
l5y 8d25dd42cd fix(docker): bump ingestor base to python 3.12.10 for cryptography's Rust MSRV (#858) 2026-07-25 19:34:22 +02:00
l5y b013faf13d web,data: reconcile MeshCore stale same-name node duplicates (#857)
* web,data: reconcile MeshCore stale same-name node duplicates

* data: MeshCore roster sync must not warm last_heard
v0.7.3
2026-07-24 23:25:48 +02:00
l5y 8b095669cf web: stack HOT over CARTO basemap layers, drop per-tile timeout (#856) 2026-07-24 22:06:37 +02:00
l5y 3d43061fff web: live relative-time tick - last-seen/updated ages count up in place (#851) 2026-07-23 12:03:13 +02:00
l5y 3878597c8d data,web: ingest every telemetry family from both protocols (#850)
* data,web: ingest every telemetry family from both protocols

* data: 24h per-node telemetry poll cooldown; RX_ONLY ingestor switch
2026-07-23 11:51:30 +02:00
l5y 22c470d9e6 data: meshcore rf metrics rssi/snr/hops/path (#849) 2026-07-22 21:41:57 +02:00
l5y 5b6d45e25e web: node table keeps latest telemetry of every type (per-field merge) (#847) 2026-07-22 16:14:17 +02:00
l5y 6fefce73e1 web: blend basemap providers to fix HOT/CARTO tile checkerboard (#846) 2026-07-22 13:02:39 +02:00
l5y 0cd2b2c4d4 Bump all packages to 0.7.3 (#845)
* Bump all packages to 0.7.3

* ci: disable Flutter mobile release workflow
2026-07-18 12:11:49 +02:00
l5y ebd500981d web: HOT primary basemap (dark-filtered) with per-tile CARTO fallback (#844) 2026-07-18 11:58:05 +02:00
l5y ea5560f9aa fix(docker): armv7 ingestor build toolchain; disable matrix fail-fast (#842) v0.7.2 2026-07-07 19:42:33 +02:00
l5y 211ae52d55 web: fix MeshCore ghost nodes (stale contact enrichment discarded) (#841)
* web: fix MeshCore ghost nodes (stale contact enrichment discarded)

* web: guard short_name gap-fill against empty strings; codify Known gaps
2026-07-06 09:27:36 +02:00
l5y 2479a6554f fix: UDP-transport hardening, bridge tracker coverage, CI repairs; release 0.7.2 (#840)
* fix: UDP-transport hardening, bridge tracker coverage, CI repairs; release 0.7.2

* fix: UDP-transport hardening, bridge tracker coverage, CI repairs; release 0.7.2
2026-07-06 00:30:52 +02:00
TJ Paulson df201f3ee4 fix: security review — dashboard XSS, private-mode federation gap, Matrix bridge resilience (#839)
* fix(web): escape untrusted mesh fields in dashboard to close DOM XSS

Security review found user-controlled mesh data reaching innerHTML unescaped
(upstream-originating; worth a PR back to l5yth/potato-mesh):

- High: channel_name was interpolated raw by formatChatChannelTag and rendered
  via innerHTML on the node-detail page and map overlays, so a crafted channel
  name (any radio can set one) executed script for every viewer. Now escaped
  with the canonical escapeHtml.
- Defense-in-depth: node_id, role, and hw_model in the node table are now
  escaped for consistency with sibling cells.

Regression test added for the channel-name escape. Full JS suite: 1448 pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(web): gate POST /api/instances by federation mode + hardening

Security review findings in the Sinatra app (upstream-originating; worth a PR
back to l5yth/potato-mesh):

- High: POST /api/instances performed outbound federation fetches and DB writes
  with no federation/private guard, so a PRIVATE=1 or FEDERATION=0 node still
  acted on unsolicited signed announcements. Added `halt 404 unless
  federation_enabled?`, mirroring the existing GET /api/instances guard.
- Low: the JSON-LD <script> block now escapes </script> breakout characters.
- Low: federation peer fetches now enforce a max response size
  (REMOTE_INSTANCE_MAX_RESPONSE_BYTES, default 8 MiB) to bound memory.

Regression spec added for the federation-gate (private + FEDERATION=0 cases).

NOTE: rspec could not be run locally (this host has only Ruby 2.6; the project
needs >=3.0 and no Docker was available). Changes are `ruby -c` syntax-checked
and reviewed against the mirrored guard; the ruby.yml CI workflow must confirm
the spec on push/PR before merge.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(matrix): stop silent message loss + add HTTP timeouts + log reg failures

Security/resilience review of the Matrix bridge (upstream-originating; worth a
PR back to l5yth/potato-mesh):

- High: within a poll batch, a later message that forwarded successfully
  advanced the in-memory rx_time watermark past an EARLIER message that failed,
  so the failed message was never refetched -> silent permanent loss. poll_once
  now breaks at the first failure, committing only the contiguous successful
  prefix; the failed message and everything after it are retried in order next
  poll (no reordering, no duplicates). Regression test added and proven to bite.
- High: the shared reqwest client had no timeouts, so a hung homeserver/API
  stalled the single-threaded poll loop (and startup health checks) forever.
  Added timeout(30s) + connect_timeout(10s).
- Medium: ensure_user_registered swallowed all non-success responses silently;
  it now warns (status + body) on anything other than the expected
  already-registered case, so a bad as_token is diagnosable.

cargo test: 118 passed; cargo fmt clean. Tradeoff: a permanently-failing
message now blocks its batch (retried each poll) rather than being lost -- safer
than silent loss; a skip-after-N/dead-letter follow-up is possible if needed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: address Copilot review on the security PR (Matrix errcode + spec hygiene)

- matrix.rs: ensure_user_registered now treats only the specific M_USER_IN_USE
  errcode as "already registered", instead of any 400 Bad Request, so real 400
  failures (malformed request, config issues) reach the diagnostic warn branch.
  Updated the M_USER_IN_USE test to send the errcode body and added a test that
  a non-M_USER_IN_USE 400 is not suppressed. cargo test: 119 passing.
- app_spec.rb: the private-mode POST /api/instances example now restores
  ENV["PRIVATE"] via an around/ensure block (mirroring the FEDERATION pattern),
  so private mode can't leak into later specs and cause order-dependent failures.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: implement Copilot follow-ups — streaming size cap + bounded skip

- instance_fetcher.rb: perform_single_http_request now uses the block form of
  Net::HTTP#request + response.read_body to enforce the response-size cap
  INCREMENTALLY, aborting as soon as the limit is exceeded, instead of letting
  the non-block form buffer the whole body into memory first. Updated the two
  federation_spec mocks to the block/streaming form and added a size-cap test.
- main.rs (Matrix bridge): a permanently-failing ("poison") message no longer
  blocks the batch forever. Consecutive per-message failures are counted
  in-memory; after MAX_FORWARD_ATTEMPTS (5) the message is skipped (advanced
  past, with a warning) so later messages make progress. Transient failures
  still stop-and-retry in order (no silent loss, no reordering). Added a
  skip-after-N regression test; the original watermark test still passes.
  cargo test: 120 passing, cargo fmt clean.

Ruby: instance_fetcher.rb is `ruby -c` clean; the spec exercises the new paths
but rspec/`ruby -c` can't run locally (Ruby 2.6 here can't parse the repo's
3.1+ syntax) — the ruby.yml CI is the gate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test(web): fix private-mode /api/instances spec to actually enable private mode

Running the suite locally (Ruby 4.0) revealed the private-mode POST /api/instances
example never entered private mode: the top-level `before` hook deletes
ENV["PRIVATE"] before each example, so toggling it (even via an around block)
had no effect and the request returned 201 instead of 404.

Switch to stubbing PotatoMesh::Config.private_mode_enabled? => true, the pattern
the rest of the suite uses for private-mode cases. This makes the test actually
exercise the guard AND removes the ENV manipulation entirely (so there is no
cross-spec ENV leak to worry about). Full web suite: 1470 examples, 0 failures;
rufo clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 22:27:37 +02:00
TJ Paulson 1b934ee6f7 feat(ingestor): passive UDP transport (Mesh via UDP), primary-channel only (#838)
Adds an optional passive UDP transport (TRANSPORT=udp) that reads the node's
"Mesh via UDP" LAN multicast instead of holding the radio's single API/serial
slot, filters to the PRIMARY channel by channel-hash (fail-closed), decrypts
with the default PSK, and enriches payloads to match the API-path packet shape
so the collector receives identical records. Implemented as a MeshProtocol
provider; 100% unit coverage on new modules; real-capture fixtures included
(node IDs/GPS anonymized). See PR description for details.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 22:24:26 +02:00
l5y 436bf34de6 web: chat-log retention and vertical scroll fix (#837)
* web: chat-log retention and vertical scroll fix

* web: suppress redundant advert when a node is heard via an encrypted message
v0.7.1
2026-07-02 20:50:27 +02:00
l5y 6d9d524e09 improve log wiring and capture all meshcore adverts (#836)
* data: fix meshcore adverts gap and capture adv from other nodes

* web: improve log wiring
2026-06-30 12:57:04 +02:00
l5y e633ff9d5e web: progressively backfill all bulk collections (#835) 2026-06-30 09:35:22 +02:00
l5y bc6d0187cc web: fix MeshCore chat duplicates and warm-cache message gap (#834) 2026-06-29 20:59:30 +02:00
l5y 8ed15f1142 web: initial-load module-graph waterfall (#832)
* web: initial-load module-graph waterfall

* web: prefetch initial API data on cold load (faster first paint)

Second phase of the initial-load fix (after the module-graph preload in
8915f8c). Even with the graph preloaded, the first /api/* fetch waited for the
~806KB bundle to download, parse, and boot. An early <script type="module"
async> boot module (main/boot-prefetch.js) now fires the first-load (since=0)
requests in parallel with the module graph (at priority:'high') and stashes the
in-flight Response promises on window.__PM_BOOT__; refresh() consumes them on its
first cold refresh via a new responsePromise option on the data-fetchers instead
of issuing its own requests.

Cold loads only: a synchronous localStorage marker (pm:cache-present, maintained
by the cache write-back / clear / disable paths) suppresses the prefetch on warm
revisits, leaving the FC2 seed-then-delta path untouched. Message endpoints are
skipped in private mode (data-pm-chat="false"), mirroring the /api/messages 404
(Invariant II / PS6). Pure pre-warm: an absent or rejected prefetch re-fetches,
so it is never load-bearing (FC7).

Read-side only; no API/DB/ingestor change, no new dependency. Adds ACCEPTANCE
EF-A1/EF-A2/EF-R1; 100% line/branch/func coverage on the new module (lcov);
rspec + npm test green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
v0.7.1-rc0
2026-06-29 14:50:33 +02:00
l5y 266f8ff8bd chore: bump version 0.7.1 (#833) 2026-06-29 14:45:37 +02:00
l5y b5d9249918 web: change tiles to carto (#831)
* web: change tiles to carto

* web: change tiles to carto
2026-06-28 20:59:52 +02:00
l5y c8d5a8a20f web: fix pubsub thread budget to not block entire app (#828)
* web: fix pubsub thread budget to not block entire app

* web: address review comments
2026-06-27 11:09:57 +02:00
l5y be5365105d web: fix api/events hang on shutdown (#827)
* web: fix api/events hang on shutdown

* web: address review comments
v0.7.0
2026-06-26 15:26:13 +02:00
l5y afb15ff48d web: fix live-update DOM handling (#826)
* web: fix live-update DOM handling

* web: live-update feedback upgrades (v2)

* web: more live-update feedback

* web: address review comments
2026-06-26 09:59:41 +02:00
l5y 554fa38f50 web: fix meshcore message duplication regression (#825) 2026-06-24 22:22:35 +02:00
l5y 1804a8a75e web: add visual feedback to pubsub events (#824) 2026-06-24 20:34:35 +02:00
l5y 4831e37a52 web: mark message author nodes as seen (#822) 2026-06-24 17:53:10 +02:00
l5y 8083206210 web: implement pubsub (#821) 2026-06-24 17:35:33 +02:00
l5y f5f9298746 web: add pagination to all APIs (#820) 2026-06-24 13:02:50 +02:00
l5y 7eccbfb89a fix: unify Meshtastic custom LoRa label with MeshCore format (#818) 2026-06-23 21:58:18 +02:00
l5y 3790686ab2 web: version JS/CSS assets to bust stale browser caches (#815) 2026-06-23 21:23:35 +02:00
George c8668a730c fix: report custom LoRa config as "Custom SF{sf}/BW{bw}/CR{cr}" when use_preset=False (#811) (#812) 2026-06-23 19:46:30 +02:00
l5y 9039fb6f95 web: implement local storage caching (#814)
* web: implement local storage caching

* web: address review comments
2026-06-23 19:35:51 +02:00
l5y 4107527eca web: render chat incrementally to reduce page load time (#813) 2026-06-23 12:18:00 +02:00
l5y 843dc85776 web: fix federation cycle and http response error handler (#810) v0.7.0-rc3 2026-06-22 17:06:56 +02:00
l5y be9e7c006f web: fix federation dns status 500 (#809) 2026-06-22 16:35:56 +02:00
l5y d323cab32a web: fix meshcore node synthisation, merging, and deduplication (#808) 2026-06-22 15:54:12 +02:00
l5y 088907345d web: progressively load messages in batches (#807)
* web: progressively load messages in batches

* web address review comments
2026-06-22 13:21:50 +02:00
l5y 027b6160df web: federation signature v2 migration (#805)
* web: federation signature v2 migration

* web: address review comments
v0.7.0-rc2
2026-06-22 10:01:45 +02:00
l5y af64b2c60f web: fix apis to use consistently use camel case (#802) v0.7.0-rc1 2026-06-21 20:17:40 +02:00
l5y 5e0363a0ec web: breaking change on stats api (#801)
* web: breaking change on stats api

* address review comments
v0.7.0-rc0
2026-06-21 13:41:41 +02:00
l5y bb95b0792d web: fix chat (#800)
* chore: add guardrails

* docs: fix meshcore badge

* web: life chat message cap at 1000 in frontend

* web: honor protocol in chat

* web: deprioritize test channels

* fix ci
2026-06-19 16:37:41 +02:00
dependabot[bot] efd2c59fb8 build(deps): bump openssl from 0.10.79 to 0.10.80 in /matrix (#795)
Bumps [openssl](https://github.com/rust-openssl/rust-openssl) from 0.10.79 to 0.10.80.
- [Release notes](https://github.com/rust-openssl/rust-openssl/releases)
- [Commits](https://github.com/rust-openssl/rust-openssl/compare/openssl-v0.10.79...openssl-v0.10.80)

---
updated-dependencies:
- dependency-name: openssl
  dependency-version: 0.10.80
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-28 23:24:54 +02:00
l5y 512b4f157b Fix regression where Meshcore chat senders show as Meshtastic (#794)
* Fix regression where Meshcore chat senders show as Meshtastic

* Address review feedback for protocol misclassification fix

- ingest.rb: exclude wrapper ``protocol`` key from /api/nodes batch-limit
  count so the documented 1000-node maximum still applies after the
  Python ingestor started stamping protocol at the wrapper level.
- Drop plan-file references from production and test comments per the
  repo guidelines; the why is already explained inline.

* Address protocol-fallback review feedback

- Neighbor placeholder now inherits the source node's protocol from the
  surrounding /api/neighbors entry, so the badge tracks the radio the
  peer lives on instead of collapsing to the neutral "Unknown" label
  (review item #1).
- resolve_record_protocol logs one warn_log line when an explicit
  protocol stamp is rejected as malformed, making misbehaving custom
  protocol adapters visible in the operator log instead of silently
  falling back (review item #3).

* Extract buildNodePlaceholder helper for testability

The neighbor placeholder logic in main.js lives inside an untested
closure, so codecov reported the protocol-propagation lines as
uncovered.  Extract the small placeholder builder into long-link-router
so it can be unit tested directly; the closure-internal call site stays
trivial (one factory call + one fallback call).
2026-05-24 09:49:45 +02:00
l5y d2fe4f8223 web: add node opt-out marker and data retention policies (#793)
* web: add node opt-out marker and data retention policies

* web: address review comments

* web: address review comments
2026-05-20 21:02:59 +02:00