149 Commits

Author SHA1 Message Date
l5y 211ae52d55 web: fix MeshCore ghost nodes (stale contact enrichment discarded) (#841)
* web: fix MeshCore ghost nodes (stale contact enrichment discarded)

* web: guard short_name gap-fill against empty strings; codify Known gaps
2026-07-06 09:27:36 +02:00
l5y 2479a6554f fix: UDP-transport hardening, bridge tracker coverage, CI repairs; release 0.7.2 (#840)
* fix: UDP-transport hardening, bridge tracker coverage, CI repairs; release 0.7.2

* fix: UDP-transport hardening, bridge tracker coverage, CI repairs; release 0.7.2
2026-07-06 00:30:52 +02:00
TJ Paulson df201f3ee4 fix: security review — dashboard XSS, private-mode federation gap, Matrix bridge resilience (#839)
* fix(web): escape untrusted mesh fields in dashboard to close DOM XSS

Security review found user-controlled mesh data reaching innerHTML unescaped
(upstream-originating; worth a PR back to l5yth/potato-mesh):

- High: channel_name was interpolated raw by formatChatChannelTag and rendered
  via innerHTML on the node-detail page and map overlays, so a crafted channel
  name (any radio can set one) executed script for every viewer. Now escaped
  with the canonical escapeHtml.
- Defense-in-depth: node_id, role, and hw_model in the node table are now
  escaped for consistency with sibling cells.

Regression test added for the channel-name escape. Full JS suite: 1448 pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(web): gate POST /api/instances by federation mode + hardening

Security review findings in the Sinatra app (upstream-originating; worth a PR
back to l5yth/potato-mesh):

- High: POST /api/instances performed outbound federation fetches and DB writes
  with no federation/private guard, so a PRIVATE=1 or FEDERATION=0 node still
  acted on unsolicited signed announcements. Added `halt 404 unless
  federation_enabled?`, mirroring the existing GET /api/instances guard.
- Low: the JSON-LD <script> block now escapes </script> breakout characters.
- Low: federation peer fetches now enforce a max response size
  (REMOTE_INSTANCE_MAX_RESPONSE_BYTES, default 8 MiB) to bound memory.

Regression spec added for the federation-gate (private + FEDERATION=0 cases).

NOTE: rspec could not be run locally (this host has only Ruby 2.6; the project
needs >=3.0 and no Docker was available). Changes are `ruby -c` syntax-checked
and reviewed against the mirrored guard; the ruby.yml CI workflow must confirm
the spec on push/PR before merge.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(matrix): stop silent message loss + add HTTP timeouts + log reg failures

Security/resilience review of the Matrix bridge (upstream-originating; worth a
PR back to l5yth/potato-mesh):

- High: within a poll batch, a later message that forwarded successfully
  advanced the in-memory rx_time watermark past an EARLIER message that failed,
  so the failed message was never refetched -> silent permanent loss. poll_once
  now breaks at the first failure, committing only the contiguous successful
  prefix; the failed message and everything after it are retried in order next
  poll (no reordering, no duplicates). Regression test added and proven to bite.
- High: the shared reqwest client had no timeouts, so a hung homeserver/API
  stalled the single-threaded poll loop (and startup health checks) forever.
  Added timeout(30s) + connect_timeout(10s).
- Medium: ensure_user_registered swallowed all non-success responses silently;
  it now warns (status + body) on anything other than the expected
  already-registered case, so a bad as_token is diagnosable.

cargo test: 118 passed; cargo fmt clean. Tradeoff: a permanently-failing
message now blocks its batch (retried each poll) rather than being lost -- safer
than silent loss; a skip-after-N/dead-letter follow-up is possible if needed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: address Copilot review on the security PR (Matrix errcode + spec hygiene)

- matrix.rs: ensure_user_registered now treats only the specific M_USER_IN_USE
  errcode as "already registered", instead of any 400 Bad Request, so real 400
  failures (malformed request, config issues) reach the diagnostic warn branch.
  Updated the M_USER_IN_USE test to send the errcode body and added a test that
  a non-M_USER_IN_USE 400 is not suppressed. cargo test: 119 passing.
- app_spec.rb: the private-mode POST /api/instances example now restores
  ENV["PRIVATE"] via an around/ensure block (mirroring the FEDERATION pattern),
  so private mode can't leak into later specs and cause order-dependent failures.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: implement Copilot follow-ups — streaming size cap + bounded skip

- instance_fetcher.rb: perform_single_http_request now uses the block form of
  Net::HTTP#request + response.read_body to enforce the response-size cap
  INCREMENTALLY, aborting as soon as the limit is exceeded, instead of letting
  the non-block form buffer the whole body into memory first. Updated the two
  federation_spec mocks to the block/streaming form and added a size-cap test.
- main.rs (Matrix bridge): a permanently-failing ("poison") message no longer
  blocks the batch forever. Consecutive per-message failures are counted
  in-memory; after MAX_FORWARD_ATTEMPTS (5) the message is skipped (advanced
  past, with a warning) so later messages make progress. Transient failures
  still stop-and-retry in order (no silent loss, no reordering). Added a
  skip-after-N regression test; the original watermark test still passes.
  cargo test: 120 passing, cargo fmt clean.

Ruby: instance_fetcher.rb is `ruby -c` clean; the spec exercises the new paths
but rspec/`ruby -c` can't run locally (Ruby 2.6 here can't parse the repo's
3.1+ syntax) — the ruby.yml CI is the gate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test(web): fix private-mode /api/instances spec to actually enable private mode

Running the suite locally (Ruby 4.0) revealed the private-mode POST /api/instances
example never entered private mode: the top-level `before` hook deletes
ENV["PRIVATE"] before each example, so toggling it (even via an around block)
had no effect and the request returned 201 instead of 404.

Switch to stubbing PotatoMesh::Config.private_mode_enabled? => true, the pattern
the rest of the suite uses for private-mode cases. This makes the test actually
exercise the guard AND removes the ENV manipulation entirely (so there is no
cross-spec ENV leak to worry about). Full web suite: 1470 examples, 0 failures;
rufo clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 22:27:37 +02:00
l5y bc6d0187cc web: fix MeshCore chat duplicates and warm-cache message gap (#834) 2026-06-29 20:59:30 +02:00
l5y 8ed15f1142 web: initial-load module-graph waterfall (#832)
* web: initial-load module-graph waterfall

* web: prefetch initial API data on cold load (faster first paint)

Second phase of the initial-load fix (after the module-graph preload in
8915f8c). Even with the graph preloaded, the first /api/* fetch waited for the
~806KB bundle to download, parse, and boot. An early <script type="module"
async> boot module (main/boot-prefetch.js) now fires the first-load (since=0)
requests in parallel with the module graph (at priority:'high') and stashes the
in-flight Response promises on window.__PM_BOOT__; refresh() consumes them on its
first cold refresh via a new responsePromise option on the data-fetchers instead
of issuing its own requests.

Cold loads only: a synchronous localStorage marker (pm:cache-present, maintained
by the cache write-back / clear / disable paths) suppresses the prefetch on warm
revisits, leaving the FC2 seed-then-delta path untouched. Message endpoints are
skipped in private mode (data-pm-chat="false"), mirroring the /api/messages 404
(Invariant II / PS6). Pure pre-warm: an absent or rejected prefetch re-fetches,
so it is never load-bearing (FC7).

Read-side only; no API/DB/ingestor change, no new dependency. Adds ACCEPTANCE
EF-A1/EF-A2/EF-R1; 100% line/branch/func coverage on the new module (lcov);
rspec + npm test green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 14:50:33 +02:00
l5y 266f8ff8bd chore: bump version 0.7.1 (#833) 2026-06-29 14:45:37 +02:00
l5y b5d9249918 web: change tiles to carto (#831)
* web: change tiles to carto

* web: change tiles to carto
2026-06-28 20:59:52 +02:00
l5y c8d5a8a20f web: fix pubsub thread budget to not block entire app (#828)
* web: fix pubsub thread budget to not block entire app

* web: address review comments
2026-06-27 11:09:57 +02:00
l5y be5365105d web: fix api/events hang on shutdown (#827)
* web: fix api/events hang on shutdown

* web: address review comments
2026-06-26 15:26:13 +02:00
l5y afb15ff48d web: fix live-update DOM handling (#826)
* web: fix live-update DOM handling

* web: live-update feedback upgrades (v2)

* web: more live-update feedback

* web: address review comments
2026-06-26 09:59:41 +02:00
l5y 554fa38f50 web: fix meshcore message duplication regression (#825) 2026-06-24 22:22:35 +02:00
l5y 1804a8a75e web: add visual feedback to pubsub events (#824) 2026-06-24 20:34:35 +02:00
l5y 4831e37a52 web: mark message author nodes as seen (#822) 2026-06-24 17:53:10 +02:00
l5y 8083206210 web: implement pubsub (#821) 2026-06-24 17:35:33 +02:00
l5y f5f9298746 web: add pagination to all APIs (#820) 2026-06-24 13:02:50 +02:00
l5y 3790686ab2 web: version JS/CSS assets to bust stale browser caches (#815) 2026-06-23 21:23:35 +02:00
l5y 843dc85776 web: fix federation cycle and http response error handler (#810) 2026-06-22 17:06:56 +02:00
l5y be9e7c006f web: fix federation dns status 500 (#809) 2026-06-22 16:35:56 +02:00
l5y d323cab32a web: fix meshcore node synthisation, merging, and deduplication (#808) 2026-06-22 15:54:12 +02:00
l5y 027b6160df web: federation signature v2 migration (#805)
* web: federation signature v2 migration

* web: address review comments
2026-06-22 10:01:45 +02:00
l5y af64b2c60f web: fix apis to use consistently use camel case (#802) 2026-06-21 20:17:40 +02:00
l5y 5e0363a0ec web: breaking change on stats api (#801)
* web: breaking change on stats api

* address review comments
2026-06-21 13:41:41 +02:00
l5y bb95b0792d web: fix chat (#800)
* chore: add guardrails

* docs: fix meshcore badge

* web: life chat message cap at 1000 in frontend

* web: honor protocol in chat

* web: deprioritize test channels

* fix ci
2026-06-19 16:37:41 +02:00
l5y 512b4f157b Fix regression where Meshcore chat senders show as Meshtastic (#794)
* Fix regression where Meshcore chat senders show as Meshtastic

* Address review feedback for protocol misclassification fix

- ingest.rb: exclude wrapper ``protocol`` key from /api/nodes batch-limit
  count so the documented 1000-node maximum still applies after the
  Python ingestor started stamping protocol at the wrapper level.
- Drop plan-file references from production and test comments per the
  repo guidelines; the why is already explained inline.

* Address protocol-fallback review feedback

- Neighbor placeholder now inherits the source node's protocol from the
  surrounding /api/neighbors entry, so the badge tracks the radio the
  peer lives on instead of collapsing to the neutral "Unknown" label
  (review item #1).
- resolve_record_protocol logs one warn_log line when an explicit
  protocol stamp is rejected as malformed, making misbehaving custom
  protocol adapters visible in the operator log instead of silently
  falling back (review item #3).

* Extract buildNodePlaceholder helper for testability

The neighbor placeholder logic in main.js lives inside an untested
closure, so codecov reported the protocol-propagation lines as
uncovered.  Extract the small placeholder builder into long-link-router
so it can be unit tested directly; the closure-internal call site stays
trivial (one factory call + one fallback call).
2026-05-24 09:49:45 +02:00
l5y d2fe4f8223 web: add node opt-out marker and data retention policies (#793)
* web: add node opt-out marker and data retention policies

* web: address review comments

* web: address review comments
2026-05-20 21:02:59 +02:00
l5y 8a89185fbe fix sentinel position data (#792)
* fix sentinel position data

* address review comments
2026-05-18 21:27:47 +02:00
l5y c32da2c541 chore: bump to 0.6.4 (#785) 2026-05-03 17:01:28 +02:00
l5y 73e161f432 web: fix liveliness of api data hydration bug (#783)
* web: fix liveliness of api data hydration bug

* web: address review comments
2026-05-03 13:05:37 +02:00
l5y 0fbff32535 web: refactor 2/7 federation (#773)
* web: refactor 2/7 federation

* web: close federation coverage gaps and apply review nits

Address Codecov patch coverage feedback by adding rspec examples for
the 51 lines flagged across the new federation shards (announce,
crawl, validation, http_client, self_instance, instance_metrics,
announcer_threads, lifecycle, signature). Per-shard line coverage in
the federation directory is now 100%.

Apply two review-comment changes: rename the awkwardly-named
http_client_get.rb to instance_fetcher.rb (matching its semantic
role rather than the HTTP verb), and declare PotatoMesh::App::Federation
explicitly in the federation.rb manifest so the namespace is owned by
this file rather than implicitly created by whichever shard happens to
load first.
2026-05-02 22:12:20 +02:00
l5y 03caf391e7 web: refactor 1/7 data processing (#772)
* web: refactor 1/7 data processing

* web: close coverage gaps in data_processing submodules

Bring every file under lib/potato_mesh/application/data_processing/ to
100% line coverage so codecov/patch passes on the 1/7 refactor PR. The
gap was a relocation of pre-existing untested branches; closing them
here keeps the subsequent refactor PRs in the series unblocked.

* Add unit tests covering canonical sender/recipient overrides,
  reply_id/emoji updates on existing rows, and the rare INSERT
  ConstraintException recovery path inside +insert_message+.
* Cover the non-canonical reporter and per-neighbour resolution
  branches in +insert_neighbors+.
* Cover the SQLException rescue in +upsert_ingestor+, the
  fallback_num branch in +touch_node_last_seen+, the limit fallback
  in +read_json_body+, the unrecognised-type branch in
  +store_decrypted_payload+, the +power+ telemetry_type fallback,
  the default-coercion path in +resolve_numeric_metric+, and the
  numeric/bare-hex paths in +canonical_node_parts+ and
  +coerce_trace_node_id+.

Drop dead code surfaced while pinning behaviour:

* +clear_encrypted+ in +insert_message+ has been initialised to
  +false+ and never reassigned since #633 dropped the
  decrypted-text override; remove it and the four dependent
  branches.
* The +rescue ArgumentError; nil+ tails in
  +identity.resolve_node_num+ and +traces.coerce_trace_node_id+ are
  unreachable because every +Integer(...)+ call inside is guarded by
  a regex pre-check.

Add a comment to the +data_processing.rb+ shim explaining that the
+require_relative+ list is ordered by dependency rather than
alphabetically, addressing review nit #5.
2026-05-02 22:08:21 +02:00
l5y 09d9e7be13 chore: bump version to 0.6.3 (#779)
* chore: bump version to 0.6.3

* chore: bump version to 0.6.3
2026-04-29 22:06:29 +02:00
l5y 43a5724b7f web: add seo improvements (#771)
* web: add seo improvements

* web: address review comments

* web: address review comments
2026-04-29 10:33:33 +02:00
l5y 161f22bf12 chore: bump version to 0.6.2 (#763)
Backfill v0.6.1 CHANGELOG entry (previously undocumented),
add v0.6.2 entry covering 9 commits since v0.6.1, and bump
the version string across data, web, matrix, and app along
with the README docker-pull examples.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 19:02:26 +02:00
l5y a4902548d3 web: fix emoji pattern render in short names (#760)
* web: fix emoji pattern render in short names

* web: address review comments
2026-04-21 13:02:36 +02:00
l5y db236d58e2 web: fix meshcore message duplication with 120s dupe protection (#758)
* web: fix meshcore message duplication with 120s dupe protection

* web: fix meshcore message duplication with 120s dupe protection

* web: address review comments

* web: address review comments
2026-04-21 10:13:39 +02:00
l5y a6cac6ced5 web: fix node duplication through message synthetization (#757)
* web: fix node duplication through message synthetization

* web: fix edge case

* web: address review comments
2026-04-21 08:53:09 +02:00
l5y 580c2fb6ea web: fix federation node counts (#749)
* web: fix federation node counts

* web: fix federation node counts

* web: address review comments
2026-04-15 15:43:34 +02:00
l5y 13b2ce9067 web: fix meshcore node misclassification (#748)
* web: fix meshcore node misclassification

* web: address review comments

* web: address review comments
2026-04-15 12:38:50 +02:00
l5y 5a73e212a3 web: optimize caching (#744)
* web: optimize caching

* web: address review comments

* web: address review comments

* web: run rufo
2026-04-14 23:29:54 +02:00
l5y 07c8e85caa web: fix federation resolver issue with multi addresses (#743)
* web: fix federation resolver issue with multi addresses

* web: add tests

* web: address review comments
2026-04-14 18:55:40 +02:00
l5y b951dbffeb web: per protocol active node counts (#735)
* web: per protocol active node counts

* web: address review comments
2026-04-13 18:26:16 +02:00
l5y aeb97477f0 chore: bump version to 0.6.1 (#726) 2026-04-09 13:14:20 +02:00
l5y 81e588e44c web: add markdown static pages (#723)
* web: add markdown static pages

* web: add tests and docker

* web: improve wording and configs

* web: add tests

* web: address review comments

* web: address review comments

* Potential fix for pull request finding 'CodeQL / Incomplete multi-character sanitization'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* web: address review comments

* web: address review comments

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-04-08 16:42:13 +02:00
l5y 083de6418f web: fix federation for multi protocol (#722)
* web: fix federation for multi protocol

* web: fix short name emojis

* web: address review comments

* ci: fix the codeql gap

* ci: fix the codeql gap

* ci: fix the codeql gap

* ci: remove swift
2026-04-08 14:36:43 +02:00
l5y 4d38ddd341 web: facelift (#716)
* web: facelift

* web: facelift

* web: facelift

* web: address review comments

* web: address review comments

* web: address review comments

* web: address review comments

* web: address review comments

* web: address review comments

* web: more css magic

* web: link parsing for chat contact

* web: remove one-letter fallback for shortnames

* Potential fix for pull request finding 'CodeQL / Incomplete multi-character sanitization'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* web: fix fallback for shortnames

* web: address review comments

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-04-07 21:38:43 +02:00
l5y d8b80c2a97 web: reference meshcore nodes in chat (#709)
* web: reference meshcore nodes in chat

* data: add adv_name to messages

* web: address review comments

* derive actual companion from name string

* derive actual companion from name string

* derive actual companion from name string

* web: address review comments

* web: address review comments
2026-04-06 13:39:00 +02:00
l5y 406fa80dd0 web: fix node disappearance role reset (#707)
* web: fix node disappearance role reset

* web: address review comments

* web: address review comments

* web: address review comments
2026-04-05 23:43:36 +02:00
l5y de1ccc5a2e release: v0.6.0 — remove deprecated env var aliases (#704)
* chore: bump version to 0.6.0 and remove deprecated env var aliases

BREAKING CHANGES:
- POTATOMESH_INSTANCE removed — use INSTANCE_DOMAIN
- PROVIDER removed — use PROTOCOL
- MESH_SERIAL removed — use CONNECTION
- PORT config alias removed — use CONNECTION

The _ConfigModule proxy class (which kept PROTOCOL/PROVIDER and
CONNECTION/PORT in sync) is deleted. docker-compose.yml now defaults
INSTANCE_DOMAIN to http://web:41447 so deployments without an explicit
value continue to work.

* tests: run black

* address review comments
2026-04-05 16:49:10 +02:00
l5y 0a479e4517 web: protect real node names from fallback (#702)
* web: protect real node names from fallback

* web: address review comments

* web: address review comments
2026-04-05 13:57:18 +02:00
l5y 06530f36ff web: add proper short names for meshcore companions (#693)
* web: add proper short names for meshcore companions

* web: address review comments
2026-04-05 09:01:43 +02:00