Compare commits

..

25 Commits

Author SHA1 Message Date
l5y 01665b6e3a Fixes POST request 403 errors on instances behind Cloudflare proxy (#368)
* Add full headers to ingestor POST requests to avoid CF bans

* run black

* Guard Authorization header when token absent

---------

Co-authored-by: varna9000 <milen@aeroisk.com>
2025-10-16 22:29:04 +02:00
l5y 1898a99789 Delay initial federation announcements (#366) 2025-10-16 21:50:43 +02:00
l5y 3eefda9205 Ensure well-known document stays in sync (#365) 2025-10-16 21:43:11 +02:00
l5y a6ba9a8227 Guard federation DNS resolution against restricted networks (#362)
* Guard federation DNS resolution against restricted networks

* Pin federation HTTP clients to vetted IPs
2025-10-16 21:15:34 +02:00
l5y 7055444c4b Add federation ingestion limits and tests (#364) 2025-10-16 21:15:18 +02:00
l5y 4bfc0e25cb Prefer reported primary channel names (#363) 2025-10-16 20:35:24 +02:00
l5y 81335cbf7b Decouple messages API from node joins (#360) 2025-10-16 13:19:29 +02:00
l5y 76b57c08c6 Fix ingestor reconnection detection (#361) 2025-10-16 13:06:32 +02:00
l5y 926b5591b0 Harden instance domain validation (#359) 2025-10-16 10:51:34 +02:00
l5y 957e597004 Ensure INSTANCE_DOMAIN propagates to containers (#358) 2025-10-15 23:22:46 +02:00
l5y 68cfbf139f chore: bump version to 0.5.2 (#356)
Co-authored-by: l5yth <d220195275+l5yth@users.noreply.github.com>
2025-10-15 23:16:30 +02:00
l5y b2f4fcaaa5 Gracefully retry federation announcements over HTTP (#355) 2025-10-15 23:11:59 +02:00
l5y dc2fa9d247 Recursively ingest federated instances (#353)
* Recursively ingest federated instances

* Keep absent is_private nil during signature verification
2025-10-15 21:35:37 +02:00
l5y a32125996c Remove federation timeout environment overrides (#352) 2025-10-15 20:04:19 +02:00
l5y 506a1ab5f6 Close unrelated short info overlays when opening short info (#351)
* Close unrelated overlays when opening short info

* Ensure map overlays respect nested short overlay closing
2025-10-15 16:35:38 +00:00
l5y db7b67d859 Improve federation instance error diagnostics (#350) 2025-10-15 18:35:22 +02:00
l5y 49f08a7f75 Harden federation domain validation and tests (#347)
* Harden federation domain validation and tests

* Preserve domain casing for signature verification

* Forward sanitize helper keyword argument

* Handle mixed-case domains during signature verification
2025-10-15 18:14:31 +02:00
l5y b2d35d3edf Handle malformed instance records (#348) 2025-10-15 17:08:24 +02:00
l5y a9d618cdbc Fix ingestor device mounting for non-serial connections (#346)
* Adjust ingestor device handling

* Restore serial device permissions for ingestor
2025-10-15 16:52:37 +02:00
l5y 6a65abd2e3 Persist instance config assets across Docker restarts (#345) 2025-10-15 16:14:59 +02:00
l5y a3aef8cadd Add modem preset display to node overlay (#340)
* Add modem metadata line to node overlays

* Ensure modem metadata loads for all overlays
2025-10-14 20:59:47 +02:00
l5y cff89a8c88 Display message frequency and channel in chat log (#339)
* Display message frequency and channel in chat log

* Ensure chat prefixes display consistent metadata brackets

* Ensure chat prefixes show non-breaking frequency placeholder

* Adjust chat channel tag placement
2025-10-14 20:56:42 +02:00
l5y 26c1366412 Bump fallback version to v0.5.1 (#338) 2025-10-14 16:51:04 +00:00
l5y 28f5b49f4d docs: update changelog for 0.5.0 (#337) 2025-10-14 16:48:36 +00:00
l5y a46da284e5 Fix ingestor package layout in Docker image (#336) 2025-10-14 18:47:54 +02:00
38 changed files with 3457 additions and 364 deletions
+5
View File
@@ -57,6 +57,11 @@ CONTACT_LINK='#potatomesh:dod.ngo'
# Debug mode (0=off, 1=on)
DEBUG=0
# Public domain name for this PotatoMesh instance
# Provide a hostname (with optional port) that resolves to the web service.
# Example: mesh.example.org or mesh.example.org:41447
INSTANCE_DOMAIN=mesh.example.org
# Docker image architecture (linux-amd64, linux-arm64, linux-armv7)
POTATOMESH_IMAGE_ARCH=linux-amd64
+86 -3
View File
@@ -1,12 +1,95 @@
# CHANGELOG
## Unreleased
## v0.5.1
* Preserve legacy configuration assets when migrating to XDG directories.
* Recursively ingest federated instances by @l5yth in <https://github.com/l5yth/potato-mesh/pull/353>
* Remove federation timeout environment overrides by @l5yth in <https://github.com/l5yth/potato-mesh/pull/352>
* Close unrelated short info overlays when opening short info by @l5yth in <https://github.com/l5yth/potato-mesh/pull/351>
* Improve federation instance error diagnostics by @l5yth in <https://github.com/l5yth/potato-mesh/pull/350>
* Harden federation domain validation and tests by @l5yth in <https://github.com/l5yth/potato-mesh/pull/347>
* Handle malformed instance records gracefully by @l5yth in <https://github.com/l5yth/potato-mesh/pull/348>
* Fix ingestor device mounting for non-serial connections by @l5yth in <https://github.com/l5yth/potato-mesh/pull/346>
* Ensure Docker deployments persist keyfile and well-known assets by @l5yth in <https://github.com/l5yth/potato-mesh/pull/345>
* Add modem preset display to node overlay by @l5yth in <https://github.com/l5yth/potato-mesh/pull/340>
* Display message frequency and channel in chat log by @l5yth in <https://github.com/l5yth/potato-mesh/pull/339>
* Bump fallback version string to v0.5.1 by @l5yth in <https://github.com/l5yth/potato-mesh/pull/338>
* Docs: update changelog for 0.5.0 by @l5yth in <https://github.com/l5yth/potato-mesh/pull/337>
* Fix ingestor docker import path by @l5yth in <https://github.com/l5yth/potato-mesh/pull/336>
## v0.5.0
* Add JavaScript configuration tests and coverage workflow
* Ensure node overlays appear above fullscreen map by @l5yth in <https://github.com/l5yth/potato-mesh/pull/333>
* Adjust node table columns responsively by @l5yth in <https://github.com/l5yth/potato-mesh/pull/332>
* Add LoRa metadata fields to nodes and messages by @l5yth in <https://github.com/l5yth/potato-mesh/pull/331>
* Add channel metadata capture for message tagging by @l5yth in <https://github.com/l5yth/potato-mesh/pull/329>
* Capture radio metadata for ingestor payloads by @l5yth in <https://github.com/l5yth/potato-mesh/pull/327>
* Fix FrozenError when filtering node query results by @l5yth in <https://github.com/l5yth/potato-mesh/pull/324>
* Ensure frontend reports git-aware version strings by @l5yth in <https://github.com/l5yth/potato-mesh/pull/321>
* Ensure web Docker image ships application sources by @l5yth in <https://github.com/l5yth/potato-mesh/pull/322>
* Refine stacked short info overlays on the map by @l5yth in <https://github.com/l5yth/potato-mesh/pull/319>
* Refine environment configuration defaults by @l5yth in <https://github.com/l5yth/potato-mesh/pull/318>
* Fix legacy configuration migration to XDG directories by @l5yth in <https://github.com/l5yth/potato-mesh/pull/317>
* Adopt XDG base directories for app data and config by @l5yth in <https://github.com/l5yth/potato-mesh/pull/316>
* Refactor: streamline ingestor environment variables by @l5yth in <https://github.com/l5yth/potato-mesh/pull/314>
* Adjust map auto-fit padding and default zoom by @l5yth in <https://github.com/l5yth/potato-mesh/pull/315>
* Ensure APIs filter stale data and refresh node details from latest sources by @l5yth in <https://github.com/l5yth/potato-mesh/pull/312>
* Improve offline tile fallback initialization by @l5yth in <https://github.com/l5yth/potato-mesh/pull/307>
* Add fallback for offline tile rendering errors by @l5yth in <https://github.com/l5yth/potato-mesh/pull/306>
* Fix map auto-fit handling and add controller by @l5yth in <https://github.com/l5yth/potato-mesh/pull/311>
* Fix map initialization bounds and add coverage by @l5yth in <https://github.com/l5yth/potato-mesh/pull/305>
* Increase coverage for configuration and sanitizer helpers by @l5yth in <https://github.com/l5yth/potato-mesh/pull/303>
* Add comprehensive theme and background front-end tests by @l5yth in <https://github.com/l5yth/potato-mesh/pull/302>
* Document sanitization and helper modules by @l5yth in <https://github.com/l5yth/potato-mesh/pull/301>
* Add in-repo Meshtastic protobuf stubs for tests by @l5yth in <https://github.com/l5yth/potato-mesh/pull/300>
* Handle CRL lookup failures during federation TLS by @l5yth in <https://github.com/l5yth/potato-mesh/pull/299>
* Ensure JavaScript workflow runs frontend tests by @l5yth in <https://github.com/l5yth/potato-mesh/pull/298>
* Unify structured logging across application and ingestor by @l5yth in <https://github.com/l5yth/potato-mesh/pull/296>
* Add Apache license headers to missing sources by @l5yth in <https://github.com/l5yth/potato-mesh/pull/297>
* Update workflows for ingestor, sinatra, and frontend by @l5yth in <https://github.com/l5yth/potato-mesh/pull/295>
* Fix IPv6 instance domain canonicalization by @l5yth in <https://github.com/l5yth/potato-mesh/pull/294>
* Handle federation HTTPS CRL verification failures by @l5yth in <https://github.com/l5yth/potato-mesh/pull/293>
* Adjust federation announcement interval to eight hours by @l5yth in <https://github.com/l5yth/potato-mesh/pull/292>
* Restore modular app functionality by @l5yth in <https://github.com/l5yth/potato-mesh/pull/291>
* Refactor config and metadata helpers into PotatoMesh modules by @l5yth in <https://github.com/l5yth/potato-mesh/pull/290>
* Update default site configuration defaults by @l5yth in <https://github.com/l5yth/potato-mesh/pull/288>
* Add regression test for queue drain concurrency by @l5yth in <https://github.com/l5yth/potato-mesh/pull/287>
* Ensure Docker config directories are created for non-root user by @l5yth in <https://github.com/l5yth/potato-mesh/pull/286>
* Clarify numeric address requirement for network target parsing by @l5yth in <https://github.com/l5yth/potato-mesh/pull/285>
* Ensure mesh ingestor queue resets active flag when idle by @l5yth in <https://github.com/l5yth/potato-mesh/pull/284>
* Clarify BLE connection description in README by @l5yth in <https://github.com/l5yth/potato-mesh/pull/283>
* Configure web container for production mode by @l5yth in <https://github.com/l5yth/potato-mesh/pull/282>
* Normalize INSTANCE_DOMAIN configuration to require hostnames by @l5yth in <https://github.com/l5yth/potato-mesh/pull/280>
* Avoid blocking startup on federation announcements by @l5yth in <https://github.com/l5yth/potato-mesh/pull/281>
* Fix production Docker builds for web and ingestor images by @l5yth in <https://github.com/l5yth/potato-mesh/pull/279>
* Improve instance domain detection logic by @l5yth in <https://github.com/l5yth/potato-mesh/pull/278>
* Implement federation announcements and instances API by @l5yth in <https://github.com/l5yth/potato-mesh/pull/277>
* Fix federation signature handling and IP guard by @l5yth in <https://github.com/l5yth/potato-mesh/pull/276>
* Add persistent federation metadata endpoint by @l5yth in <https://github.com/l5yth/potato-mesh/pull/274>
* Add configurable instance domain with reverse DNS fallback by @l5yth in <https://github.com/l5yth/potato-mesh/pull/272>
* Document production deployment configuration by @l5yth in <https://github.com/l5yth/potato-mesh/pull/273>
* Add targeted API endpoints and expose version metadata by @l5yth in <https://github.com/l5yth/potato-mesh/pull/271>
* Prometheus metrics updates on startup and for position/telemetry by @nicjansma in <https://github.com/l5yth/potato-mesh/pull/270>
* Add hourly reconnect handling for inactive mesh interface by @l5yth in <https://github.com/l5yth/potato-mesh/pull/267>
* Dockerfile fixes by @nicjansma in <https://github.com/l5yth/potato-mesh/pull/268>
* Added prometheus /metrics endpoint by @nicjansma in <https://github.com/l5yth/potato-mesh/pull/262>
* Add fullscreen toggle to map view by @l5yth in <https://github.com/l5yth/potato-mesh/pull/263>
* Relocate JS coverage export script into web directory by @l5yth in <https://github.com/l5yth/potato-mesh/pull/266>
* V0.4.0 version string in web UI by @nicjansma in <https://github.com/l5yth/potato-mesh/pull/265>
* Add energy saving cycle to ingestor daemon by @l5yth in <https://github.com/l5yth/potato-mesh/pull/256>
* Chore: restore apache headers by @l5yth in <https://github.com/l5yth/potato-mesh/pull/260>
* Docs: add matrix to readme by @l5yth in <https://github.com/l5yth/potato-mesh/pull/259>
* Force dark theme default based on sanitized cookie by @l5yth in <https://github.com/l5yth/potato-mesh/pull/252>
* Document mesh ingestor modules with PDoc-style docstrings by @l5yth in <https://github.com/l5yth/potato-mesh/pull/255>
* Handle missing node IDs in Meshtastic nodeinfo packets by @l5yth in <https://github.com/l5yth/potato-mesh/pull/251>
* Document Ruby helper methods with RDoc comments by @l5yth in <https://github.com/l5yth/potato-mesh/pull/254>
* Add JSDoc documentation across client scripts by @l5yth in <https://github.com/l5yth/potato-mesh/pull/253>
* Fix mesh ingestor telemetry and neighbor handling by @l5yth in <https://github.com/l5yth/potato-mesh/pull/249>
* Refactor front-end assets into external modules by @l5yth in <https://github.com/l5yth/potato-mesh/pull/245>
* Add tests for helper utilities and asset routes by @l5yth in <https://github.com/l5yth/potato-mesh/pull/243>
* Docs: add ingestor inline docstrings by @l5yth in <https://github.com/l5yth/potato-mesh/pull/244>
* Add comprehensive coverage tests for mesh ingestor by @l5yth in <https://github.com/l5yth/potato-mesh/pull/241>
* Add inline documentation to config helpers and frontend scripts by @l5yth in <https://github.com/l5yth/potato-mesh/pull/240>
* Update changelog by @l5yth in <https://github.com/l5yth/potato-mesh/pull/238>
## v0.4.0
+12 -3
View File
@@ -31,6 +31,7 @@ against the web API.
API_TOKEN=replace-with-a-strong-token
SITE_NAME=PotatoMesh Demo
CONNECTION=/dev/ttyACM0
INSTANCE_DOMAIN=mesh.example.org
```
Additional environment variables are optional:
@@ -43,6 +44,8 @@ Additional environment variables are optional:
the ingestor.
- `CHANNEL_INDEX` selects the LoRa channel when using serial or Bluetooth
connections.
- `INSTANCE_DOMAIN` pins the public hostname advertised by the web UI and API
responses, bypassing reverse DNS detection when set.
- `DEBUG` enables verbose logging across the stack.
## Docker Compose file
@@ -50,9 +53,15 @@ Additional environment variables are optional:
Use the `docker-compose.yml` file provided in the repository (or download the
[raw file from GitHub](https://raw.githubusercontent.com/l5yth/potato-mesh/main/docker-compose.yml)).
It already references the published GHCR images, defines persistent volumes for
data and logs, and includes optional bridge-profile services for environments
that require classic port mapping. Place this file in the same directory as
your `.env` file so Compose can pick up both.
data, configuration, and logs, and includes optional bridge-profile services for
environments that require classic port mapping. Place this file in the same
directory as your `.env` file so Compose can pick up both.
The dedicated configuration volume binds to `/app/.config/potato-mesh` inside
the container. This path stores the instance private key and staged
`/.well-known/potato-mesh` documents. Because the volume persists independently
of container lifecycle events, generated credentials are not replaced on reboot
or re-deploy.
## Start the stack
+22 -21
View File
@@ -1,10 +1,11 @@
# 🥔 PotatoMesh
[![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/l5yth/potato-mesh/ruby.yml?branch=main)](https://github.com/l5yth/potato-mesh/actions)
[![GitHub release (latest by date)](https://img.shields.io/github/v/release/l5yth/potato-mesh)](https://github.com/l5yth/potato-mesh/releases)
[![GitHub release](https://img.shields.io/github/v/release/l5yth/potato-mesh)](https://github.com/l5yth/potato-mesh/releases)
[![codecov](https://codecov.io/gh/l5yth/potato-mesh/branch/main/graph/badge.svg?token=FS7252JVZT)](https://codecov.io/gh/l5yth/potato-mesh)
[![Open-Source License](https://img.shields.io/github/license/l5yth/potato-mesh)](LICENSE)
[![Contributions Welcome](https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat)](https://github.com/l5yth/potato-mesh/issues)
[![Matrix Chat](https://img.shields.io/badge/matrix-%23potatomesh:dod.ngo-blue)](https://matrix.to/#/#potatomesh:dod.ngo)
A simple Meshtastic-powered node dashboard for your local community. _No MQTT clutter, just local LoRa aether._
@@ -24,7 +25,7 @@ Requires Ruby for the Sinatra web app and SQLite3 for the app's database.
```bash
pacman -S ruby sqlite3
gem install sinatra sqlite3 rackup puma rspec rack-test rufo
gem install sinatra sqlite3 rackup puma rspec rack-test rufo prometheus-client
cd ./web
bundle install
```
@@ -67,20 +68,6 @@ exec ruby app.rb -p 41447 -o 0.0.0.0
* Configure `INSTANCE_DOMAIN` with the public URL of your deployment so vanity
links and generated metadata resolve correctly.
### Configuration storage
PotatoMesh stores its runtime assets using the XDG base directory specification.
During startup the web application migrates existing configuration from
`web/.config` and `web/config` into the resolved `XDG_CONFIG_HOME` directory.
This preserves previously generated instance key material and
`/.well-known/potato-mesh` documents so upgrades do not create new credentials
unnecessarily. When XDG directories are not provided the application falls back
to the repository root.
The migrated key is written to `<XDG_CONFIG_HOME>/potato-mesh/keyfile` and the
well-known document is staged in
`<XDG_CONFIG_HOME>/potato-mesh/well-known/potato-mesh`.
The web app can be configured with environment variables (defaults shown):
* `SITE_NAME` - title and header shown in the UI (default: "PotatoMesh Demo")
@@ -90,6 +77,7 @@ The web app can be configured with environment variables (defaults shown):
* `MAX_DISTANCE` - hide nodes farther than this distance from the center (default: `42`)
* `CONTACT_LINK` - chat link or Matrix alias for footer and overlay (default: `#potatomesh:dod.ngo`)
* `PRIVATE` - set to `1` to hide the chat UI, disable message APIs, and exclude hidden clients (default: unset)
* `INSTANCE_DOMAIN` - public hostname (optionally with port) used for metadata, federation, and API links (default: auto-detected)
The application derives SEO-friendly document titles, descriptions, and social
preview tags from these existing configuration values and reuses the bundled
@@ -101,6 +89,18 @@ Example:
SITE_NAME="PotatoMesh Demo" MAP_CENTER=38.761944,-27.090833 MAX_DISTANCE=42 CONTACT_LINK="#potatomesh:dod.ngo" ./app.sh
```
### Configuration & Storage
PotatoMesh stores its runtime assets using the XDG base directory specification.
When XDG directories are not provided the application falls back
to the repository root.
The key is written to `$XDG_CONFIG_HOME/potato-mesh/keyfile` and the
well-known document is staged in
`$XDG_CONFIG_HOME/potato-mesh/well-known/potato-mesh`.
The database can be found in `$XDG_DATA_HOME/potato-mesh`.
### API
The web app contains an API:
@@ -110,7 +110,9 @@ The web app contains an API:
* GET `/api/messages?limit=100` - returns the latest 100 messages (disabled when `PRIVATE=1`)
* GET `/api/telemetry?limit=100` - returns the latest 100 telemetry data
* GET `/api/neighbors?limit=100` - returns the latest 100 neighbor tuples
* GET `/metrics`- prometheus endpoint
* GET `/api/instances` - returns known potato-mesh instances in other locations
* GET `/metrics`- metrics for the prometheus endpoint
* GET `/version`- information about the potato-mesh instance
* POST `/api/nodes` - upserts nodes provided as JSON object mapping node ids to node data (requires `Authorization: Bearer <API_TOKEN>`)
* POST `/api/positions` - appends positions provided as a JSON object or array (requires `Authorization: Bearer <API_TOKEN>`)
* POST `/api/messages` - appends messages provided as a JSON object or array (requires `Authorization: Bearer <API_TOKEN>`; disabled when `PRIVATE=1`)
@@ -162,10 +164,9 @@ interface. `CONNECTION` also accepts Bluetooth device addresses (e.g.,
## Demos
* <https://potatomesh.net/>
* <https://vrs.kdd2105.ru/>
* <https://potatomesh.stratospire.com/>
* <https://es1tem.uk/>
Post your nodes here:
* <https://github.com/l5yth/potato-mesh/discussions/258>
## Docker
+14
View File
@@ -75,6 +75,7 @@ MAX_DISTANCE=$(grep "^MAX_DISTANCE=" .env 2>/dev/null | cut -d'=' -f2- | tr -d '
CONTACT_LINK=$(grep "^CONTACT_LINK=" .env 2>/dev/null | cut -d'=' -f2- | tr -d '"' || echo "#potatomesh:dod.ngo")
API_TOKEN=$(grep "^API_TOKEN=" .env 2>/dev/null | cut -d'=' -f2- | tr -d '"' || echo "")
POTATOMESH_IMAGE_ARCH=$(grep "^POTATOMESH_IMAGE_ARCH=" .env 2>/dev/null | cut -d'=' -f2- | tr -d '"' || echo "linux-amd64")
INSTANCE_DOMAIN=$(grep "^INSTANCE_DOMAIN=" .env 2>/dev/null | cut -d'=' -f2- | tr -d '"' || echo "")
echo "📍 Location Settings"
echo "-------------------"
@@ -99,6 +100,13 @@ echo "------------------"
echo "Specify the Docker image architecture for your host (linux-amd64, linux-arm64, linux-armv7)."
read_with_default "Docker image architecture" "$POTATOMESH_IMAGE_ARCH" POTATOMESH_IMAGE_ARCH
echo ""
echo "🌐 Domain Settings"
echo "------------------"
echo "Provide the public hostname that clients should use to reach this PotatoMesh instance."
echo "Leave blank to allow automatic detection via reverse DNS."
read_with_default "Instance domain (e.g. mesh.example.org)" "$INSTANCE_DOMAIN" INSTANCE_DOMAIN
echo ""
echo "🔐 Security Settings"
echo "-------------------"
@@ -142,6 +150,11 @@ update_env "MAX_DISTANCE" "$MAX_DISTANCE"
update_env "CONTACT_LINK" "\"$CONTACT_LINK\""
update_env "API_TOKEN" "$API_TOKEN"
update_env "POTATOMESH_IMAGE_ARCH" "$POTATOMESH_IMAGE_ARCH"
if [ -n "$INSTANCE_DOMAIN" ]; then
update_env "INSTANCE_DOMAIN" "$INSTANCE_DOMAIN"
else
sed -i.bak '/^INSTANCE_DOMAIN=.*/d' .env
fi
# Migrate legacy connection settings and ensure defaults exist
if grep -q "^MESH_SERIAL=" .env; then
@@ -176,6 +189,7 @@ echo " Frequency: $FREQUENCY"
echo " Chat: ${CONTACT_LINK:-'Not set'}"
echo " API Token: ${API_TOKEN:0:8}..."
echo " Docker Image Arch: $POTATOMESH_IMAGE_ARCH"
echo " Instance Domain: ${INSTANCE_DOMAIN:-'Auto-detected'}"
echo ""
echo "🚀 You can now start PotatoMesh with:"
echo " docker-compose up -d"
+4 -4
View File
@@ -26,7 +26,7 @@ RUN set -eux; \
python -m pip install --no-cache-dir -r requirements.txt; \
apk del .build-deps
COPY data/ .
COPY data /app/data
RUN addgroup -S potatomesh && \
adduser -S potatomesh -G potatomesh && \
adduser potatomesh dialout && \
@@ -40,7 +40,7 @@ ENV CONNECTION=/dev/ttyACM0 \
POTATOMESH_INSTANCE="" \
API_TOKEN=""
CMD ["python", "mesh.py"]
CMD ["python", "-m", "data.mesh"]
# Windows production image
FROM python:${PYTHON_VERSION}-windowsservercore-ltsc2022 AS production-windows
@@ -55,7 +55,7 @@ WORKDIR /app
COPY data/requirements.txt ./
RUN python -m pip install --no-cache-dir -r requirements.txt
COPY data/ .
COPY data /app/data
USER ContainerUser
@@ -65,6 +65,6 @@ ENV CONNECTION=/dev/ttyACM0 \
POTATOMESH_INSTANCE="" \
API_TOKEN=""
CMD ["python", "mesh.py"]
CMD ["python", "-m", "data.mesh"]
FROM production-${TARGETOS} AS production
+26 -11
View File
@@ -78,17 +78,36 @@ def _iter_channel_objects(channels_obj: Any) -> Iterator[Any]:
def _primary_channel_name() -> str | None:
"""Return the name to use for the primary channel when available."""
"""Return the fallback name to use for the primary channel when needed."""
preset = getattr(config, "MODEM_PRESET", None)
if isinstance(preset, str) and preset.strip():
return preset
return preset.strip()
env_name = os.environ.get("CHANNEL", "").strip()
if env_name:
return env_name
return None
def _extract_channel_name(settings_obj: Any) -> str | None:
"""Normalise the configured channel name extracted from ``settings_obj``."""
if settings_obj is None:
return None
if isinstance(settings_obj, dict):
candidate = settings_obj.get("name")
else:
candidate = getattr(settings_obj, "name", None)
if isinstance(candidate, str):
candidate = candidate.strip()
if candidate:
return candidate
return None
def _normalize_role(role: Any) -> int | None:
"""Convert a channel role descriptor into an integer value."""
@@ -122,27 +141,23 @@ def _channel_tuple(channel_obj: Any) -> tuple[int, str] | None:
role_value = _normalize_role(getattr(channel_obj, "role", None))
if role_value == _ROLE_PRIMARY:
channel_index = 0
channel_name = _primary_channel_name()
channel_name = _extract_channel_name(getattr(channel_obj, "settings", None))
if channel_name is None:
channel_name = _primary_channel_name()
elif role_value == _ROLE_SECONDARY:
raw_index = getattr(channel_obj, "index", None)
try:
channel_index = int(raw_index)
except Exception:
channel_index = None
settings = getattr(channel_obj, "settings", None)
channel_name = getattr(settings, "name", None) if settings else None
channel_name = _extract_channel_name(getattr(channel_obj, "settings", None))
else:
return None
if not isinstance(channel_index, int):
return None
if isinstance(channel_name, str):
channel_name = channel_name.strip()
else:
channel_name = None
if not channel_name:
if not isinstance(channel_name, str) or not channel_name:
return None
return channel_index, channel_name
+54 -7
View File
@@ -167,6 +167,45 @@ def _is_ble_interface(iface_obj) -> bool:
return "ble_interface" in module_name
def _connected_state(candidate) -> bool | None:
"""Return the connection state advertised by ``candidate``.
Parameters:
candidate: Attribute returned from ``iface.isConnected`` on a
Meshtastic interface. The value may be a boolean, a callable that
yields a boolean, or a :class:`threading.Event` instance.
Returns:
``True`` when the interface is believed to be connected, ``False``
when it appears disconnected, and ``None`` when the state cannot be
determined from the provided attribute.
"""
if candidate is None:
return None
if isinstance(candidate, threading.Event):
return candidate.is_set()
is_set_method = getattr(candidate, "is_set", None)
if callable(is_set_method):
try:
return bool(is_set_method())
except Exception:
return None
if callable(candidate):
try:
return bool(candidate())
except Exception:
return None
try:
return bool(candidate)
except Exception: # pragma: no cover - defensive guard
return None
def main() -> None:
"""Run the mesh ingestion daemon until interrupted."""
@@ -411,13 +450,20 @@ def main() -> None:
connected_attr = getattr(iface, "isConnected", None)
believed_disconnected = False
if callable(connected_attr):
try:
believed_disconnected = not bool(connected_attr())
except Exception:
believed_disconnected = False
elif connected_attr is not None:
believed_disconnected = not bool(connected_attr)
connected_state = _connected_state(connected_attr)
if connected_state is None:
if callable(connected_attr):
try:
believed_disconnected = not bool(connected_attr())
except Exception:
believed_disconnected = False
elif connected_attr is not None:
try:
believed_disconnected = not bool(connected_attr)
except Exception: # pragma: no cover - defensive guard
believed_disconnected = False
else:
believed_disconnected = not connected_state
should_reconnect = believed_disconnected or (
inactivity_elapsed >= inactivity_reconnect_secs
@@ -468,5 +514,6 @@ __all__ = [
"_node_items_snapshot",
"_subscribe_receive_topics",
"_is_ble_interface",
"_connected_state",
"main",
]
+18 -4
View File
@@ -72,11 +72,25 @@ def _post_json(
return
url = f"{instance}{path}"
data = json.dumps(payload).encode("utf-8")
req = urllib.request.Request(
url, data=data, headers={"Content-Type": "application/json"}
)
# Add full headers to avoid Cloudflare blocks on instances behind cloudflare proxy
headers = {
"Content-Type": "application/json",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
"Accept": "application/json",
"Accept-Language": "en-US,en;q=0.9",
"Origin": f"{instance}",
"Referer": f"{instance}",
}
if api_token:
req.add_header("Authorization", f"Bearer {api_token}")
headers["Authorization"] = f"Bearer {api_token}"
req = urllib.request.Request(
url,
data=data,
headers=headers,
)
try:
with urllib.request.urlopen(req, timeout=10) as resp:
resp.read()
+6
View File
@@ -6,6 +6,7 @@ services:
volumes:
- ./web:/app
- ./data:/app/.local/share/potato-mesh
- ./.config/potato-mesh:/app/.config/potato-mesh
- /app/vendor/bundle
web-bridge:
@@ -14,6 +15,7 @@ services:
volumes:
- ./web:/app
- ./data:/app/.local/share/potato-mesh
- ./.config/potato-mesh:/app/.config/potato-mesh
- /app/vendor/bundle
ports:
- "41447:41447"
@@ -25,7 +27,9 @@ services:
volumes:
- ./data:/app
- ./data:/app/.local/share/potato-mesh
- ./.config/potato-mesh:/app/.config/potato-mesh
- /app/.local
- /dev:/dev
ingestor-bridge:
environment:
@@ -33,4 +37,6 @@ services:
volumes:
- ./data:/app
- ./data:/app/.local/share/potato-mesh
- ./.config/potato-mesh:/app/.config/potato-mesh
- /app/.local
- /dev:/dev
+11 -2
View File
@@ -10,10 +10,12 @@ x-web-base: &web-base
MAX_DISTANCE: ${MAX_DISTANCE:-42}
CONTACT_LINK: ${CONTACT_LINK:-#potatomesh:dod.ngo}
API_TOKEN: ${API_TOKEN}
INSTANCE_DOMAIN: ${INSTANCE_DOMAIN}
DEBUG: ${DEBUG:-0}
command: ["ruby", "app.rb", "-p", "41447", "-o", "0.0.0.0"]
volumes:
- potatomesh_data:/app/.local/share/potato-mesh
- potatomesh_config:/app/.config/potato-mesh
- potatomesh_logs:/app/logs
restart: unless-stopped
deploy:
@@ -32,12 +34,17 @@ x-ingestor-base: &ingestor-base
CHANNEL_INDEX: ${CHANNEL_INDEX:-0}
POTATOMESH_INSTANCE: ${POTATOMESH_INSTANCE:-http://web:41447}
API_TOKEN: ${API_TOKEN}
INSTANCE_DOMAIN: ${INSTANCE_DOMAIN}
DEBUG: ${DEBUG:-0}
volumes:
- potatomesh_data:/app/.local/share/potato-mesh
- potatomesh_config:/app/.config/potato-mesh
- potatomesh_logs:/app/logs
devices:
- ${CONNECTION:-/dev/ttyACM0}:${CONNECTION:-/dev/ttyACM0}
- /dev:/dev
device_cgroup_rules:
- 'c 166:* rwm' # ttyACM devices
- 'c 188:* rwm' # ttyUSB devices
- 'c 4:* rwm' # ttyS devices
privileged: false
restart: unless-stopped
deploy:
@@ -85,6 +92,8 @@ services:
volumes:
potatomesh_data:
driver: local
potatomesh_config:
driver: local
potatomesh_logs:
driver: local
+106 -4
View File
@@ -407,11 +407,14 @@ def test_capture_channels_from_interface_records_metadata(mesh_module, capsys):
mesh = mesh_module
mesh.config.MODEM_PRESET = "MediumFast"
mesh.channels._reset_channel_cache()
class DummyInterface:
def __init__(self) -> None:
self.wait_calls = 0
primary = SimpleNamespace(role=1, settings=SimpleNamespace())
primary = SimpleNamespace(
role=1, settings=SimpleNamespace(name=" radioamator ")
)
secondary = SimpleNamespace(
role="SECONDARY",
index="7",
@@ -428,19 +431,20 @@ def test_capture_channels_from_interface_records_metadata(mesh_module, capsys):
log_output = capsys.readouterr().out
assert iface.wait_calls == 1
assert mesh.channels.channel_mappings() == ((0, "MediumFast"), (7, "TestChannel"))
assert mesh.channels.channel_mappings() == ((0, "radioamator"), (7, "TestChannel"))
assert mesh.channels.channel_name(7) == "TestChannel"
assert "Captured channel metadata" in log_output
assert "channels=((0, 'MediumFast'), (7, 'TestChannel'))" in log_output
assert "channels=((0, 'radioamator'), (7, 'TestChannel'))" in log_output
mesh.channels.capture_from_interface(SimpleNamespace(localNode=None))
assert mesh.channels.channel_mappings() == ((0, "MediumFast"), (7, "TestChannel"))
assert mesh.channels.channel_mappings() == ((0, "radioamator"), (7, "TestChannel"))
def test_capture_channels_primary_falls_back_to_env(mesh_module, monkeypatch, capsys):
mesh = mesh_module
mesh.config.MODEM_PRESET = None
mesh.channels._reset_channel_cache()
monkeypatch.setenv("CHANNEL", "FallbackName")
class DummyInterface:
@@ -461,6 +465,29 @@ def test_capture_channels_primary_falls_back_to_env(mesh_module, monkeypatch, ca
assert "FallbackName" in log_output
def test_capture_channels_primary_falls_back_to_preset(mesh_module, capsys):
mesh = mesh_module
mesh.config.MODEM_PRESET = " MediumFast "
mesh.channels._reset_channel_cache()
class DummyInterface:
def __init__(self) -> None:
self.localNode = SimpleNamespace(
channels=[SimpleNamespace(role="PRIMARY", settings=SimpleNamespace())]
)
def waitForConfig(self) -> None: # noqa: D401 - matches interface contract
return None
mesh.channels.capture_from_interface(DummyInterface())
log_output = capsys.readouterr().out
assert mesh.channels.channel_mappings() == ((0, "MediumFast"),)
assert mesh.channels.channel_name(0) == "MediumFast"
assert "MediumFast" in log_output
def test_create_default_interface_falls_back_to_tcp(mesh_module, monkeypatch):
mesh = mesh_module
attempts = []
@@ -1201,6 +1228,81 @@ def test_main_retries_interface_creation(mesh_module, monkeypatch):
assert iface.closed is True
def test_connected_state_handles_threading_event(mesh_module):
mesh = mesh_module
event = mesh.threading.Event()
assert mesh._connected_state(event) is False
event.set()
assert mesh._connected_state(event) is True
def test_main_reconnects_when_connection_event_clears(mesh_module, monkeypatch):
mesh = mesh_module
attempts = []
interfaces = []
current_iface = {"obj": None}
import threading as real_threading_module
real_event_cls = real_threading_module.Event
class DummyInterface:
def __init__(self):
self.nodes = {}
self.isConnected = real_event_cls()
self.isConnected.set()
self.close_calls = 0
def close(self):
self.close_calls += 1
def fake_create(port):
iface = DummyInterface()
attempts.append(port)
interfaces.append(iface)
current_iface["obj"] = iface
return iface, port
class DummyStopEvent:
def __init__(self):
self._flag = False
self.wait_calls = 0
def is_set(self):
return self._flag
def set(self):
self._flag = True
def wait(self, timeout):
self.wait_calls += 1
if self.wait_calls == 1:
iface = current_iface["obj"]
assert iface is not None, "interface should be available"
iface.isConnected.clear()
return self._flag
self._flag = True
return True
monkeypatch.setattr(mesh, "PORT", "/dev/ttyTEST")
monkeypatch.setattr(mesh, "_create_serial_interface", fake_create)
monkeypatch.setattr(mesh.threading, "Event", DummyStopEvent)
monkeypatch.setattr(mesh.signal, "signal", lambda *_, **__: None)
monkeypatch.setattr(mesh, "SNAPSHOT_SECS", 0)
monkeypatch.setattr(mesh, "_RECONNECT_INITIAL_DELAY_SECS", 0)
monkeypatch.setattr(mesh, "_RECONNECT_MAX_DELAY_SECS", 0)
monkeypatch.setattr(mesh, "_CLOSE_TIMEOUT_SECS", 0)
mesh.main()
assert len(attempts) == 2
assert len(interfaces) == 2
assert interfaces[0].close_calls >= 1
assert interfaces[1].close_calls >= 1
def test_main_recreates_interface_after_snapshot_error(mesh_module, monkeypatch):
mesh = mesh_module
+4
View File
@@ -48,6 +48,7 @@ require_relative "application/prometheus"
require_relative "application/queries"
require_relative "application/data_processing"
require_relative "application/filesystem"
require_relative "application/instances"
require_relative "application/routes/api"
require_relative "application/routes/ingest"
require_relative "application/routes/root"
@@ -59,6 +60,7 @@ module PotatoMesh
extend App::Networking
extend App::Identity
extend App::Federation
extend App::Instances
extend App::Prometheus
extend App::Queries
extend App::DataProcessing
@@ -69,6 +71,7 @@ module PotatoMesh
include App::Networking
include App::Identity
include App::Federation
include App::Instances
include App::Prometheus
include App::Queries
include App::DataProcessing
@@ -166,6 +169,7 @@ SELF_INSTANCE_ID = PotatoMesh::Application::SELF_INSTANCE_ID unless defined?(SEL
PotatoMesh::App::Networking,
PotatoMesh::App::Identity,
PotatoMesh::App::Federation,
PotatoMesh::App::Instances,
PotatoMesh::App::Prometheus,
PotatoMesh::App::Queries,
PotatoMesh::App::DataProcessing,
+410 -34
View File
@@ -22,6 +22,25 @@ module PotatoMesh
raise "INSTANCE_DOMAIN could not be determined"
end
# Determine whether the local instance should persist its own record.
#
# @param domain [String, nil] candidate domain for the running instance.
# @return [Array(Boolean, String, nil)] tuple containing a decision flag and an optional reason.
def self_instance_registration_decision(domain)
source = app_constant(:INSTANCE_DOMAIN_SOURCE)
return [false, "INSTANCE_DOMAIN source is #{source}"] unless source == :environment
sanitized = sanitize_instance_domain(domain)
return [false, "INSTANCE_DOMAIN missing or invalid"] unless sanitized
ip = ip_from_domain(sanitized)
if ip && restricted_ip_address?(ip)
return [false, "INSTANCE_DOMAIN resolves to restricted IP"]
end
[true, nil]
end
def self_instance_attributes
domain = self_instance_domain
last_update = latest_node_update_timestamp || Time.now.to_i
@@ -68,43 +87,68 @@ module PotatoMesh
def ensure_self_instance_record!
attributes = self_instance_attributes
signature = sign_instance_attributes(attributes)
db = open_database
upsert_instance_record(db, attributes, signature)
debug_log(
"Registered self instance record",
context: "federation.instances",
domain: attributes[:domain],
instance_id: attributes[:id],
)
db = nil
allowed, reason = self_instance_registration_decision(attributes[:domain])
if allowed
db = open_database
upsert_instance_record(db, attributes, signature)
debug_log(
"Registered self instance record",
context: "federation.instances",
domain: attributes[:domain],
instance_id: attributes[:id],
)
else
debug_log(
"Skipped self instance registration",
context: "federation.instances",
domain: attributes[:domain],
reason: reason,
)
end
[attributes, signature]
ensure
db&.close
end
def federation_target_domains(self_domain)
domains = Set.new
normalized_self = sanitize_instance_domain(self_domain)&.downcase
ordered = []
seen = Set.new
PotatoMesh::Config.federation_seed_domains.each do |seed|
sanitized = sanitize_instance_domain(seed)
domains << sanitized.downcase if sanitized
sanitized = sanitize_instance_domain(seed)&.downcase
next unless sanitized
next if normalized_self && sanitized == normalized_self
next if seen.include?(sanitized)
ordered << sanitized
seen << sanitized
end
db = open_database(readonly: true)
db.results_as_hash = false
rows = with_busy_retry { db.execute("SELECT domain FROM instances WHERE domain IS NOT NULL AND TRIM(domain) != ''") }
rows = with_busy_retry {
db.execute("SELECT domain FROM instances WHERE domain IS NOT NULL AND TRIM(domain) != ''")
}
rows.flatten.compact.each do |raw_domain|
sanitized = sanitize_instance_domain(raw_domain)
domains << sanitized.downcase if sanitized
sanitized = sanitize_instance_domain(raw_domain)&.downcase
next unless sanitized
next if normalized_self && sanitized == normalized_self
next if seen.include?(sanitized)
ordered << sanitized
seen << sanitized
end
if self_domain
domains.delete(self_domain.downcase)
end
domains.to_a
ordered
rescue SQLite3::Exception
domains =
PotatoMesh::Config.federation_seed_domains.map do |seed|
sanitize_instance_domain(seed)&.downcase
end.compact
self_domain ? domains.reject { |domain| domain == self_domain.downcase } : domains
fallback = PotatoMesh::Config.federation_seed_domains.filter_map do |seed|
candidate = sanitize_instance_domain(seed)&.downcase
next if normalized_self && candidate == normalized_self
candidate
end
fallback.uniq
ensure
db&.close
end
@@ -112,6 +156,8 @@ module PotatoMesh
def announce_instance_to_domain(domain, payload_json)
return false unless domain && !domain.empty?
https_failures = []
instance_uri_candidates(domain, "/api/instances").each do |uri|
begin
http = build_remote_http_client(uri)
@@ -137,16 +183,51 @@ module PotatoMesh
status: response.code,
)
rescue StandardError => e
warn_log(
"Federation announcement raised exception",
metadata = {
context: "federation.announce",
target: uri.to_s,
error_class: e.class.name,
error_message: e.message,
}
if uri.scheme == "https" && https_connection_refused?(e)
debug_log(
"HTTPS federation announcement failed, retrying with HTTP",
**metadata,
)
https_failures << metadata
next
end
warn_log(
"Federation announcement raised exception",
**metadata,
)
end
end
https_failures.each do |metadata|
warn_log(
"Federation announcement raised exception",
**metadata,
)
end
false
end
# Determine whether an HTTPS announcement failure should fall back to HTTP.
#
# @param error [StandardError] failure raised while attempting HTTPS.
# @return [Boolean] true when the error corresponds to a refused TCP connection.
def https_connection_refused?(error)
current = error
while current
return true if current.is_a?(Errno::ECONNREFUSED)
current = current.respond_to?(:cause) ? current.cause : nil
end
false
end
@@ -192,12 +273,17 @@ module PotatoMesh
thread
end
# Launch a background thread responsible for the first federation broadcast.
#
# @return [Thread, nil] the thread handling the initial announcement.
def start_initial_federation_announcement!
existing = settings.respond_to?(:initial_federation_thread) ? settings.initial_federation_thread : nil
return existing if existing&.alive?
thread = Thread.new do
begin
delay = PotatoMesh::Config.initial_federation_delay_seconds
Kernel.sleep(delay) if delay.positive?
announce_instance_to_all_domains
rescue StandardError => e
warn_log(
@@ -266,7 +352,30 @@ module PotatoMesh
end
end
rescue StandardError => e
raise InstanceFetchError, e.message
raise_instance_fetch_error(e)
end
# Build a human readable error message for a failed instance request.
#
# @param error [StandardError] failure raised while performing the request.
# @return [String] description including the error class when necessary.
def instance_fetch_error_message(error)
message = error.message.to_s.strip
class_name = error.class.name || error.class.to_s
return class_name if message.empty?
message.include?(class_name) ? message : "#{class_name}: #{message}"
end
# Raise an InstanceFetchError that preserves the original context.
#
# @param error [StandardError] failure raised while performing the request.
# @return [void]
def raise_instance_fetch_error(error)
message = instance_fetch_error_message(error)
wrapped = InstanceFetchError.new(message)
wrapped.set_backtrace(error.backtrace)
raise wrapped
end
def fetch_instance_json(domain, path)
@@ -284,14 +393,254 @@ module PotatoMesh
[nil, errors]
end
# Parse a remote federation instance payload into canonical attributes.
#
# @param payload [Hash] JSON object describing a remote instance.
# @return [Array<(Hash, String), String>] tuple containing the attribute
# hash and signature when valid or a failure reason when invalid.
def remote_instance_attributes_from_payload(payload)
unless payload.is_a?(Hash)
return [nil, nil, "instance payload is not an object"]
end
id = string_or_nil(payload["id"])
return [nil, nil, "missing instance id"] unless id
domain = sanitize_instance_domain(payload["domain"])
return [nil, nil, "missing instance domain"] unless domain
pubkey = sanitize_public_key_pem(payload["pubkey"])
return [nil, nil, "missing instance public key"] unless pubkey
signature = string_or_nil(payload["signature"])
return [nil, nil, "missing instance signature"] unless signature
private_value = if payload.key?("isPrivate")
payload["isPrivate"]
else
payload["is_private"]
end
private_flag = coerce_boolean(private_value)
if private_flag.nil?
numeric_flag = coerce_integer(private_value)
private_flag = !numeric_flag.to_i.zero? if numeric_flag
end
attributes = {
id: id,
domain: domain,
pubkey: pubkey,
name: string_or_nil(payload["name"]),
version: string_or_nil(payload["version"]),
channel: string_or_nil(payload["channel"]),
frequency: string_or_nil(payload["frequency"]),
latitude: coerce_float(payload["latitude"]),
longitude: coerce_float(payload["longitude"]),
last_update_time: coerce_integer(payload["lastUpdateTime"]),
is_private: private_flag,
}
[attributes, signature, nil]
rescue StandardError => e
[nil, nil, e.message]
end
# Recursively ingest federation records exposed by the supplied domain.
#
# @param db [SQLite3::Database] open database connection used for writes.
# @param domain [String] remote domain to crawl for federation records.
# @param visited [Set<String>] domains processed during this crawl.
# @param per_response_limit [Integer, nil] maximum entries processed per response.
# @param overall_limit [Integer, nil] maximum unique domains visited.
# @return [Set<String>] updated set of visited domains.
def ingest_known_instances_from!(
db,
domain,
visited: nil,
per_response_limit: nil,
overall_limit: nil
)
sanitized = sanitize_instance_domain(domain)
return visited || Set.new unless sanitized
visited ||= Set.new
overall_limit ||= PotatoMesh::Config.federation_max_domains_per_crawl
per_response_limit ||= PotatoMesh::Config.federation_max_instances_per_response
if overall_limit && overall_limit.positive? && visited.size >= overall_limit
debug_log(
"Skipped remote instance crawl due to crawl limit",
context: "federation.instances",
domain: sanitized,
limit: overall_limit,
)
return visited
end
return visited if visited.include?(sanitized)
visited << sanitized
payload, metadata = fetch_instance_json(sanitized, "/api/instances")
unless payload.is_a?(Array)
warn_log(
"Failed to load remote federation instances",
context: "federation.instances",
domain: sanitized,
reason: Array(metadata).map(&:to_s).join("; "),
)
return visited
end
processed_entries = 0
payload.each do |entry|
if per_response_limit && per_response_limit.positive? && processed_entries >= per_response_limit
debug_log(
"Skipped remote instance entry due to response limit",
context: "federation.instances",
domain: sanitized,
limit: per_response_limit,
)
break
end
if overall_limit && overall_limit.positive? && visited.size >= overall_limit
debug_log(
"Skipped remote instance entry due to crawl limit",
context: "federation.instances",
domain: sanitized,
limit: overall_limit,
)
break
end
processed_entries += 1
attributes, signature, reason = remote_instance_attributes_from_payload(entry)
unless attributes && signature
warn_log(
"Discarded remote instance entry",
context: "federation.instances",
domain: sanitized,
reason: reason || "invalid payload",
)
next
end
if attributes[:is_private]
debug_log(
"Skipped private remote instance",
context: "federation.instances",
domain: attributes[:domain],
)
next
end
unless verify_instance_signature(attributes, signature, attributes[:pubkey])
warn_log(
"Discarded remote instance entry",
context: "federation.instances",
domain: attributes[:domain],
reason: "invalid signature",
)
next
end
attributes[:is_private] = false if attributes[:is_private].nil?
remote_nodes, node_metadata = fetch_instance_json(attributes[:domain], "/api/nodes")
unless remote_nodes
warn_log(
"Failed to load remote node data",
context: "federation.instances",
domain: attributes[:domain],
reason: Array(node_metadata).map(&:to_s).join("; "),
)
next
end
fresh, freshness_reason = validate_remote_nodes(remote_nodes)
unless fresh
warn_log(
"Discarded remote instance entry",
context: "federation.instances",
domain: attributes[:domain],
reason: freshness_reason || "stale node data",
)
next
end
begin
upsert_instance_record(db, attributes, signature)
ingest_known_instances_from!(
db,
attributes[:domain],
visited: visited,
per_response_limit: per_response_limit,
overall_limit: overall_limit,
)
rescue ArgumentError => e
warn_log(
"Failed to persist remote instance",
context: "federation.instances",
domain: attributes[:domain],
error_class: e.class.name,
error_message: e.message,
)
end
end
visited
end
# Resolve the host component of a remote URI and ensure the destination is
# safe for federation HTTP requests.
#
# The method performs a DNS lookup using Addrinfo to capture every
# available address for the supplied URI host. The resulting addresses are
# converted to {IPAddr} objects for consistent inspection via
# {restricted_ip_address?}. When all resolved addresses fall within
# restricted ranges, the method raises an ArgumentError so callers can
# abort the federation request before contacting the remote endpoint.
#
# @param uri [URI::Generic] remote endpoint candidate.
# @return [Array<IPAddr>] list of resolved, unrestricted IP addresses.
# @raise [ArgumentError] when +uri.host+ is blank or resolves solely to
# restricted addresses.
def resolve_remote_ip_addresses(uri)
host = uri&.host
raise ArgumentError, "URI missing host" unless host
addrinfo_records = Addrinfo.getaddrinfo(host, nil, Socket::AF_UNSPEC, Socket::SOCK_STREAM)
addresses = addrinfo_records.filter_map do |addr|
begin
IPAddr.new(addr.ip_address)
rescue IPAddr::InvalidAddressError
nil
end
end
unique_addresses = addresses.uniq { |ip| [ip.family, ip.to_s] }
unrestricted_addresses = unique_addresses.reject { |ip| restricted_ip_address?(ip) }
if unique_addresses.any? && unrestricted_addresses.empty?
raise ArgumentError, "restricted domain"
end
unrestricted_addresses
end
# Build an HTTP client configured for communication with a remote instance.
#
# @param uri [URI::Generic] target URI describing the remote endpoint.
# @return [Net::HTTP] HTTP client ready to execute the request.
def build_remote_http_client(uri)
remote_addresses = resolve_remote_ip_addresses(uri)
http = Net::HTTP.new(uri.host, uri.port)
if http.respond_to?(:ipaddr=) && remote_addresses.any?
http.ipaddr = remote_addresses.first.to_s
end
http.open_timeout = PotatoMesh::Config.remote_instance_http_timeout
http.read_timeout = PotatoMesh::Config.remote_instance_http_timeout
http.read_timeout = PotatoMesh::Config.remote_instance_read_timeout
http.use_ssl = uri.scheme == "https"
return http unless http.use_ssl?
@@ -435,14 +784,13 @@ module PotatoMesh
latest = nodes.filter_map do |node|
next unless node.is_a?(Hash)
timestamps = []
timestamps << coerce_integer(node["last_heard"])
timestamps << coerce_integer(node["position_time"])
timestamps << coerce_integer(node["first_heard"])
timestamps.compact.max
last_heard_values = []
last_heard_values << coerce_integer(node["last_heard"])
last_heard_values << coerce_integer(node["lastHeard"])
last_heard_values.compact.max
end.compact.max
return [false, "missing recent node updates"] unless latest
return [false, "missing last_heard data"] unless latest
cutoff = Time.now.to_i - PotatoMesh::Config.remote_instance_max_node_age
return [false, "node data is stale"] if latest < cutoff
@@ -451,6 +799,34 @@ module PotatoMesh
end
def upsert_instance_record(db, attributes, signature)
sanitized_domain = sanitize_instance_domain(attributes[:domain])
raise ArgumentError, "invalid domain" unless sanitized_domain
ip = ip_from_domain(sanitized_domain)
if ip && restricted_ip_address?(ip)
raise ArgumentError, "restricted domain"
end
normalized_domain = sanitized_domain
existing_id = with_busy_retry do
db.get_first_value(
"SELECT id FROM instances WHERE domain = ?",
normalized_domain,
)
end
if existing_id && existing_id != attributes[:id]
with_busy_retry do
db.execute("DELETE FROM instances WHERE id = ?", existing_id)
end
debug_log(
"Removed conflicting instance by domain",
context: "federation.instances",
domain: normalized_domain,
replaced_id: existing_id,
incoming_id: attributes[:id],
)
end
sql = <<~SQL
INSERT INTO instances (
id, domain, pubkey, name, version, channel, frequency,
@@ -472,7 +848,7 @@ module PotatoMesh
params = [
attributes[:id],
attributes[:domain],
normalized_domain,
attributes[:pubkey],
attributes[:name],
attributes[:version],
+3 -2
View File
@@ -53,9 +53,10 @@ module PotatoMesh
# Proxy for {PotatoMesh::Sanitizer.sanitize_instance_domain}.
#
# @param value [Object] candidate domain string.
# @param downcase [Boolean] whether to force lowercase normalisation.
# @return [String, nil] canonical domain or nil.
def sanitize_instance_domain(value)
PotatoMesh::Sanitizer.sanitize_instance_domain(value)
def sanitize_instance_domain(value, downcase: true)
PotatoMesh::Sanitizer.sanitize_instance_domain(value, downcase: downcase)
end
# Proxy for {PotatoMesh::Sanitizer.instance_domain_host}.
+17 -10
View File
@@ -170,11 +170,13 @@ module PotatoMesh
# @return [Array(String, String)] pair of JSON output and base64 signature.
def build_well_known_document
last_update = latest_node_update_timestamp
domain_value = sanitize_instance_domain(app_constant(:INSTANCE_DOMAIN))
payload = {
publicKey: app_constant(:INSTANCE_PUBLIC_KEY_PEM),
name: sanitized_site_name,
version: app_constant(:APP_VERSION),
domain: app_constant(:INSTANCE_DOMAIN),
domain: domain_value,
lastUpdate: last_update,
}
@@ -193,24 +195,31 @@ module PotatoMesh
[json_output, signature]
end
# Regenerate the well-known document when the on-disk copy is stale.
# Regenerate the well-known document when it is stale or when the existing
# content no longer matches the current instance configuration.
#
# @return [void]
def refresh_well_known_document_if_stale
FileUtils.mkdir_p(well_known_directory)
path = well_known_file_path
now = Time.now
json_output, signature = build_well_known_document
expected_contents = json_output.end_with?("\n") ? json_output : "#{json_output}\n"
needs_update = true
if File.exist?(path)
current_contents = File.binread(path)
mtime = File.mtime(path)
if (now - mtime) < PotatoMesh::Config.well_known_refresh_interval
return
if current_contents == expected_contents &&
(now - mtime) < PotatoMesh::Config.well_known_refresh_interval
needs_update = false
end
end
json_output, signature = build_well_known_document
return unless needs_update
File.open(path, File::WRONLY | File::CREAT | File::TRUNC, 0o644) do |file|
file.write(json_output)
file.write("\n") unless json_output.end_with?("\n")
file.write(expected_contents)
end
debug_log(
@@ -236,9 +245,7 @@ module PotatoMesh
return nil unless File.exist?(PotatoMesh::Config.db_path)
db = open_database(readonly: true)
value = db.get_first_value(
"SELECT MAX(COALESCE(last_heard, first_heard, position_time)) FROM nodes",
)
value = db.get_first_value("SELECT MAX(last_heard) FROM nodes")
value&.to_i
rescue SQLite3::Exception
nil
@@ -0,0 +1,199 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# frozen_string_literal: true
module PotatoMesh
module App
# Helper methods for maintaining and presenting instance records.
module Instances
# Remove duplicate instance records grouped by their canonical domain name
# while favouring the most recent entry.
#
# @return [void]
def clean_duplicate_instances!
db = open_database
rows = with_busy_retry do
db.execute(
<<~SQL
SELECT rowid, domain, last_update_time
FROM instances
WHERE domain IS NOT NULL AND TRIM(domain) != ''
SQL
)
end
grouped = rows.group_by do |row|
sanitize_instance_domain(row[1])&.downcase
rescue StandardError
nil
end
deletions = []
updates = {}
grouped.each do |canonical_domain, entries|
next if canonical_domain.nil?
next if entries.size <= 1
sorted_entries = entries.sort_by do |entry|
timestamp = coerce_integer(entry[2]) || -1
[timestamp, entry[0].to_i]
end
keeper = sorted_entries.last
next unless keeper
deletions.concat(sorted_entries[0...-1].map { |entry| entry[0].to_i })
current_domain = entries.find { |entry| entry[0] == keeper[0] }&.[](1)
if canonical_domain && current_domain != canonical_domain
updates[keeper[0].to_i] = canonical_domain
end
removed_count = sorted_entries.length - 1
warn_log(
"Removed duplicate instance records",
context: "instances.cleanup",
domain: canonical_domain,
removed: removed_count,
) if removed_count.positive?
end
unless deletions.empty?
placeholders = Array.new(deletions.size, "?").join(",")
with_busy_retry do
db.execute("DELETE FROM instances WHERE rowid IN (#{placeholders})", deletions)
end
end
updates.each do |rowid, canonical_domain|
with_busy_retry do
db.execute("UPDATE instances SET domain = ? WHERE rowid = ?", [canonical_domain, rowid])
end
end
rescue SQLite3::Exception => e
warn_log(
"Failed to clean duplicate instances",
context: "instances.cleanup",
error_class: e.class.name,
error_message: e.message,
)
ensure
db&.close
end
# Normalise and validate an instance database row for API presentation.
#
# @param row [Hash] raw database row with string keys.
# @return [Hash, nil] cleaned hash or +nil+ when the row is discarded.
def normalize_instance_row(row)
unless row.is_a?(Hash)
warn_log(
"Discarded malformed instance row",
context: "instances.normalize",
reason: "row not hash",
)
return nil
end
id = string_or_nil(row["id"])
domain = sanitize_instance_domain(row["domain"])&.downcase
pubkey = sanitize_public_key_pem(row["pubkey"])
signature = string_or_nil(row["signature"])
last_update_time = coerce_integer(row["last_update_time"])
is_private_raw = row["is_private"]
private_flag = coerce_boolean(is_private_raw)
if private_flag.nil?
numeric_private = coerce_integer(is_private_raw)
private_flag = !numeric_private.to_i.zero? if numeric_private
end
private_flag = false if private_flag.nil?
if id.nil? || domain.nil? || pubkey.nil?
warn_log(
"Discarded malformed instance row",
context: "instances.normalize",
instance_id: row["id"],
domain: row["domain"],
reason: "missing required fields",
)
return nil
end
payload = {
"id" => id,
"domain" => domain,
"pubkey" => pubkey,
"name" => string_or_nil(row["name"]),
"version" => string_or_nil(row["version"]),
"channel" => string_or_nil(row["channel"]),
"frequency" => string_or_nil(row["frequency"]),
"latitude" => coerce_float(row["latitude"]),
"longitude" => coerce_float(row["longitude"]),
"lastUpdateTime" => last_update_time,
"isPrivate" => private_flag,
"signature" => signature,
}
payload.reject { |_, value| value.nil? }
rescue StandardError => e
warn_log(
"Failed to normalise instance row",
context: "instances.normalize",
instance_id: row.respond_to?(:[]) ? row["id"] : nil,
domain: row.respond_to?(:[]) ? row["domain"] : nil,
error_class: e.class.name,
error_message: e.message,
)
nil
end
# Fetch all instance rows ready to be served by the API while handling
# malformed rows gracefully.
#
# @return [Array<Hash>] list of cleaned instance payloads.
def load_instances_for_api
clean_duplicate_instances!
db = open_database(readonly: true)
db.results_as_hash = true
rows = with_busy_retry do
db.execute(
<<~SQL
SELECT id, domain, pubkey, name, version, channel, frequency,
latitude, longitude, last_update_time, is_private, signature
FROM instances
WHERE domain IS NOT NULL AND TRIM(domain) != ''
AND pubkey IS NOT NULL AND TRIM(pubkey) != ''
ORDER BY LOWER(domain)
SQL
)
end
rows.each_with_object([]) do |row, memo|
normalized = normalize_instance_row(row)
memo << normalized if normalized
end
rescue SQLite3::Exception => e
warn_log(
"Failed to load instance records",
context: "instances.load",
error_class: e.class.name,
error_message: e.message,
)
[]
ensure
db&.close
end
end
end
end
@@ -62,6 +62,22 @@ module PotatoMesh
candidate = "#{candidate_host}:#{port}" if port_required?(uri, trimmed)
end
ipv6_with_port = candidate.match(/\A(?<address>.+):(?<port>\d+)\z/)
if ipv6_with_port
address = ipv6_with_port[:address]
port = ipv6_with_port[:port]
literal = ipv6_literal?(address)
if literal && PotatoMesh::Sanitizer.valid_port?(port)
candidate = "[#{literal}]:#{port}"
else
ipv6_literal = ipv6_literal?(candidate)
candidate = "[#{ipv6_literal}]" if ipv6_literal
end
else
ipv6_literal = ipv6_literal?(candidate)
candidate = "[#{ipv6_literal}]" if ipv6_literal
end
sanitized = sanitize_instance_domain(candidate)
unless sanitized
raise "INSTANCE_DOMAIN must be a bare hostname (optionally with a port) without schemes or paths: #{raw.inspect}"
+12 -73
View File
@@ -205,34 +205,8 @@ module PotatoMesh
sql = <<~SQL
SELECT m.id, m.rx_time, m.rx_iso, m.from_id, m.to_id, m.channel,
m.portnum, m.text, m.encrypted, m.rssi, m.hop_limit,
m.lora_freq AS msg_lora_freq, m.modem_preset AS msg_modem_preset,
m.channel_name AS msg_channel_name, m.snr AS msg_snr,
n.node_id AS node_node_id, n.num AS node_num,
n.short_name AS node_short_name, n.long_name AS node_long_name,
n.macaddr AS node_macaddr, n.hw_model AS node_hw_model,
n.role AS node_role, n.public_key AS node_public_key,
n.is_unmessagable AS node_is_unmessagable,
n.is_favorite AS node_is_favorite,
n.hops_away AS node_hops_away, n.snr AS node_snr,
n.last_heard AS node_last_heard, n.first_heard AS node_first_heard,
n.battery_level AS node_battery_level, n.voltage AS node_voltage,
n.channel_utilization AS node_channel_utilization,
n.air_util_tx AS node_air_util_tx,
n.uptime_seconds AS node_uptime_seconds,
n.position_time AS node_position_time,
n.location_source AS node_location_source,
n.precision_bits AS node_precision_bits,
n.latitude AS node_latitude, n.longitude AS node_longitude,
n.altitude AS node_altitude,
n.lora_freq AS node_lora_freq, n.modem_preset AS node_modem_preset
m.lora_freq, m.modem_preset, m.channel_name, m.snr
FROM messages m
LEFT JOIN nodes n ON (
m.from_id IS NOT NULL AND TRIM(m.from_id) <> '' AND (
m.from_id = n.node_id OR (
m.from_id GLOB '[0-9]*' AND CAST(m.from_id AS INTEGER) = n.num
)
)
)
SQL
sql += " WHERE #{where_clauses.join(" AND ")}\n"
sql += <<~SQL
@@ -243,56 +217,19 @@ module PotatoMesh
rows = db.execute(sql, params)
rows.each do |r|
r.delete_if { |key, _| key.is_a?(Integer) }
r["lora_freq"] = r.delete("msg_lora_freq")
r["modem_preset"] = r.delete("msg_modem_preset")
r["channel_name"] = r.delete("msg_channel_name")
snr_value = r.delete("msg_snr")
if PotatoMesh::Config.debug? && (r["from_id"].nil? || r["from_id"].to_s.empty?)
if PotatoMesh::Config.debug? && (r["from_id"].nil? || r["from_id"].to_s.strip.empty?)
raw = db.execute("SELECT * FROM messages WHERE id = ?", [r["id"]]).first
debug_log(
"Message join produced empty sender",
"Message query produced empty sender",
context: "queries.messages",
stage: "before_join",
stage: "raw_row",
row: raw,
)
debug_log(
"Message join produced empty sender",
context: "queries.messages",
stage: "after_join",
row: r,
)
end
node = {}
r.keys.grep(/^node_/).each do |k|
attribute = k.delete_prefix("node_")
node[attribute] = r.delete(k)
end
r["snr"] = snr_value
references = [r["from_id"]].compact
if references.any? && (node["node_id"].nil? || node["node_id"].to_s.empty?)
lookup_keys = []
canonical = normalize_node_id(db, r["from_id"])
lookup_keys << canonical if canonical
raw_ref = r["from_id"].to_s.strip
lookup_keys << raw_ref unless raw_ref.empty?
lookup_keys << raw_ref.to_i if raw_ref.match?(/\A[0-9]+\z/)
fallback = nil
lookup_keys.uniq.each do |ref|
sql = ref.is_a?(Integer) ? "SELECT * FROM nodes WHERE num = ?" : "SELECT * FROM nodes WHERE node_id = ?"
fallback = db.get_first_row(sql, [ref])
break if fallback
end
if fallback
fallback.each do |key, value|
next unless key.is_a?(String)
node[key] = value if node[key].nil?
end
end
end
node["role"] = "CLIENT" if node.key?("role") && (node["role"].nil? || node["role"].to_s.empty?)
r["node"] = node
canonical_from_id = string_or_nil(node["node_id"]) || string_or_nil(normalize_node_id(db, r["from_id"]))
canonical_from_id = string_or_nil(normalize_node_id(db, r["from_id"]))
node_id = canonical_from_id || string_or_nil(r["from_id"])
if canonical_from_id
raw_from_id = string_or_nil(r["from_id"])
if raw_from_id.nil? || raw_from_id.match?(/\A[0-9]+\z/)
@@ -302,11 +239,13 @@ module PotatoMesh
end
end
if PotatoMesh::Config.debug? && (r["from_id"].nil? || r["from_id"].to_s.empty?)
r["node_id"] = node_id if node_id
if PotatoMesh::Config.debug? && (r["from_id"].nil? || r["from_id"].to_s.strip.empty?)
debug_log(
"Message row missing sender after processing",
"Message query produced empty sender",
context: "queries.messages",
stage: "after_processing",
stage: "after_normalization",
row: r,
)
end
+1 -32
View File
@@ -127,39 +127,8 @@ module PotatoMesh
app.get "/api/instances" do
content_type :json
ensure_self_instance_record!
db = open_database(readonly: true)
db.results_as_hash = true
rows = with_busy_retry do
db.execute(
<<~SQL,
SELECT id, domain, pubkey, name, version, channel, frequency,
latitude, longitude, last_update_time, is_private, signature
FROM instances
WHERE domain IS NOT NULL AND TRIM(domain) != ''
AND pubkey IS NOT NULL AND TRIM(pubkey) != ''
ORDER BY LOWER(domain)
SQL
)
end
payload = rows.map do |row|
{
"id" => row["id"],
"domain" => row["domain"],
"pubkey" => row["pubkey"],
"name" => row["name"],
"version" => row["version"],
"channel" => row["channel"],
"frequency" => row["frequency"],
"latitude" => row["latitude"],
"longitude" => row["longitude"],
"lastUpdateTime" => row["last_update_time"]&.to_i,
"isPrivate" => row["is_private"].to_i == 1,
"signature" => row["signature"],
}.reject { |_, value| value.nil? }
end
payload = load_instances_for_api
JSON.generate(payload)
ensure
db&.close
end
end
end
@@ -84,7 +84,18 @@ module PotatoMesh
end
id = string_or_nil(payload["id"]) || string_or_nil(payload["instanceId"])
domain = sanitize_instance_domain(payload["domain"])
raw_domain_input = payload["domain"]
raw_domain = sanitize_instance_domain(raw_domain_input, downcase: false)
normalized_domain = raw_domain && sanitize_instance_domain(raw_domain)
unless raw_domain && normalized_domain
warn_log(
"Instance registration rejected",
context: "ingest.register",
domain: string_or_nil(raw_domain_input),
reason: "invalid domain",
)
halt 400, { error: "invalid domain" }.to_json
end
pubkey = sanitize_public_key_pem(payload["pubkey"])
name = string_or_nil(payload["name"])
version = string_or_nil(payload["version"])
@@ -99,7 +110,7 @@ module PotatoMesh
attributes = {
id: id,
domain: domain,
domain: normalized_domain,
pubkey: pubkey,
name: name,
version: version,
@@ -120,11 +131,21 @@ module PotatoMesh
halt 400, { error: "missing required fields" }.to_json
end
unless verify_instance_signature(attributes, signature, attributes[:pubkey])
signature_valid = verify_instance_signature(attributes, signature, attributes[:pubkey])
# Some remote peers sign payloads using a canonicalised lowercase
# domain while still sending a mixed-case domain. Retry signature
# verification with the original casing when the first attempt
# fails to maximise interoperability.
if !signature_valid && raw_domain && normalized_domain && raw_domain.casecmp?(normalized_domain) && raw_domain != normalized_domain
alternate_attributes = attributes.merge(domain: raw_domain)
signature_valid = verify_instance_signature(alternate_attributes, signature, attributes[:pubkey])
end
unless signature_valid
warn_log(
"Instance registration rejected",
context: "ingest.register",
domain: attributes[:domain],
domain: raw_domain || attributes[:domain],
reason: "invalid signature",
)
halt 400, { error: "invalid signature" }.to_json
@@ -152,6 +173,22 @@ module PotatoMesh
halt 400, { error: "restricted domain" }.to_json
end
begin
resolve_remote_ip_addresses(URI.parse("https://#{attributes[:domain]}"))
rescue ArgumentError => e
warn_log(
"Instance registration rejected",
context: "ingest.register",
domain: attributes[:domain],
reason: "restricted domain",
error_message: e.message,
)
halt 400, { error: "restricted domain" }.to_json
rescue SocketError
# DNS lookups that fail to resolve are handled later when the
# registration flow attempts to contact the remote instance.
end
well_known, well_known_meta = fetch_instance_json(attributes[:domain], "/.well-known/potato-mesh")
unless well_known
details_list = Array(well_known_meta).map(&:to_s)
@@ -204,6 +241,12 @@ module PotatoMesh
db = open_database
upsert_instance_record(db, attributes, signature)
ingest_known_instances_from!(
db,
attributes[:domain],
per_response_limit: PotatoMesh::Config.federation_max_instances_per_response,
overall_limit: PotatoMesh::Config.federation_max_domains_per_crawl,
)
debug_log(
"Registered remote instance",
context: "ingest.register",
+67 -4
View File
@@ -32,6 +32,11 @@ module PotatoMesh
DEFAULT_FREQUENCY = "915MHz"
DEFAULT_CONTACT_LINK = "#potatomesh:dod.ngo"
DEFAULT_MAX_DISTANCE_KM = 42.0
DEFAULT_REMOTE_INSTANCE_CONNECT_TIMEOUT = 5
DEFAULT_REMOTE_INSTANCE_READ_TIMEOUT = 12
DEFAULT_FEDERATION_MAX_INSTANCES_PER_RESPONSE = 64
DEFAULT_FEDERATION_MAX_DOMAINS_PER_CRAWL = 256
DEFAULT_INITIAL_FEDERATION_DELAY_SECONDS = 2
# Resolve the absolute path to the web application root directory.
#
@@ -129,7 +134,7 @@ module PotatoMesh
#
# @return [String] semantic version identifier.
def version_fallback
"v0.5.0"
"v0.5.2"
end
# Default refresh interval for frontend polling routines.
@@ -269,11 +274,38 @@ module PotatoMesh
"rsa-sha256"
end
# Timeout used when querying remote instances during federation.
# Connection timeout used when establishing federation HTTP sockets.
#
# @return [Integer] HTTP timeout in seconds.
# @return [Integer] connect timeout in seconds.
def remote_instance_http_timeout
5
DEFAULT_REMOTE_INSTANCE_CONNECT_TIMEOUT
end
# Read timeout used when streaming federation HTTP responses.
#
# @return [Integer] read timeout in seconds.
def remote_instance_read_timeout
DEFAULT_REMOTE_INSTANCE_READ_TIMEOUT
end
# Limit the number of remote instances processed from a single response.
#
# @return [Integer] maximum entries processed per /api/instances payload.
def federation_max_instances_per_response
fetch_positive_integer(
"FEDERATION_MAX_INSTANCES_PER_RESPONSE",
DEFAULT_FEDERATION_MAX_INSTANCES_PER_RESPONSE,
)
end
# Limit the total number of distinct domains crawled during one ingestion.
#
# @return [Integer] maximum unique domains visited per crawl.
def federation_max_domains_per_crawl
fetch_positive_integer(
"FEDERATION_MAX_DOMAINS_PER_CRAWL",
DEFAULT_FEDERATION_MAX_DOMAINS_PER_CRAWL,
)
end
# Maximum acceptable age for remote node data.
@@ -304,6 +336,16 @@ module PotatoMesh
8 * 60 * 60
end
# Determine the grace period before sending the initial federation announcement.
#
# @return [Integer] seconds to wait before the first broadcast cycle.
def initial_federation_delay_seconds
fetch_positive_integer(
"INITIAL_FEDERATION_DELAY_SECONDS",
DEFAULT_INITIAL_FEDERATION_DELAY_SECONDS,
)
end
# Retrieve the configured site name for presentation.
#
# @return [String] human friendly site label.
@@ -415,6 +457,27 @@ module PotatoMesh
trimmed.empty? ? default : trimmed
end
# Fetch and validate integer based configuration flags.
#
# @param key [String] environment variable to read.
# @param default [Integer] fallback value when unset or invalid.
# @return [Integer] positive integer sourced from configuration.
def fetch_positive_integer(key, default)
value = ENV[key]
return default if value.nil?
trimmed = value.strip
return default if trimmed.empty?
begin
parsed = Integer(trimmed, 10)
rescue ArgumentError
return default
end
parsed.positive? ? parsed : default
end
# Resolve the effective XDG directory honoring environment overrides.
#
# @param env_key [String] name of the environment variable to inspect.
+94 -3
View File
@@ -38,11 +38,17 @@ module PotatoMesh
end
# Ensure a value is a valid instance domain according to RFC 1035/3986
# rules. This rejects whitespace, path separators, and trailing dots.
# rules. Hostnames must include at least one dot-separated label and a
# top-level domain containing an alphabetic character. Literal IP
# addresses must be provided in standard dotted decimal form or enclosed in
# brackets when IPv6 notation is used. Optional ports must fall within the
# valid TCP/UDP range. Any opaque identifiers, URIs, or malformed hosts are
# rejected.
#
# @param value [String, Object, nil] candidate domain name.
# @param downcase [Boolean] whether to force the result to lowercase.
# @return [String, nil] canonical domain value or +nil+ when invalid.
def sanitize_instance_domain(value)
def sanitize_instance_domain(value, downcase: true)
host = string_or_nil(value)
return nil unless host
@@ -51,7 +57,92 @@ module PotatoMesh
return nil if trimmed.empty?
return nil if trimmed.match?(%r{[\s/\\@]})
trimmed
if trimmed.start_with?("[")
match = trimmed.match(/\A\[(?<address>[^\]]+)\](?::(?<port>\d+))?\z/)
return nil unless match
address = match[:address]
port = match[:port]
return nil if port && !valid_port?(port)
begin
IPAddr.new(address)
rescue IPAddr::InvalidAddressError
return nil
end
sanitized_address = downcase ? address.downcase : address
return "[#{sanitized_address}]#{port ? ":#{port}" : ""}"
end
domain = trimmed
port = nil
if domain.include?(":")
host_part, port_part = domain.split(":", 2)
return nil if host_part.nil? || host_part.empty?
return nil unless port_part && port_part.match?(/\A\d+\z/)
return nil unless valid_port?(port_part)
return nil if port_part.include?(":")
domain = host_part
port = port_part
end
unless valid_hostname?(domain) || valid_ipv4_literal?(domain)
return nil
end
sanitized_domain = downcase ? domain.downcase : domain
port ? "#{sanitized_domain}:#{port}" : sanitized_domain
end
# Determine whether the supplied hostname conforms to RFC 1035 label
# requirements and includes a valid top-level domain.
#
# @param hostname [String] host component without any port information.
# @return [Boolean] true when the hostname is valid.
def valid_hostname?(hostname)
return false if hostname.length > 253
labels = hostname.split(".")
return false if labels.length < 2
return false unless labels.all? { |label| valid_hostname_label?(label) }
top_level = labels.last
top_level.match?(/[a-z]/i)
end
# Validate a single hostname label ensuring the first and last characters
# are alphanumeric and that no unsupported symbols are present.
#
# @param label [String] hostname component between dots.
# @return [Boolean] true when the label is valid.
def valid_hostname_label?(label)
return false if label.empty?
return false if label.length > 63
label.match?(/\A[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\z/i)
end
# Validate whether a candidate represents a dotted decimal IPv4 literal.
#
# @param address [String] IP address string without port information.
# @return [Boolean] true when the address is a valid IPv4 literal.
def valid_ipv4_literal?(address)
return false unless address.match?(/\A\d{1,3}(?:\.\d{1,3}){3}\z/)
address.split(".").all? { |octet| octet.to_i.between?(0, 255) }
end
# Determine whether a port string represents a valid TCP/UDP port.
#
# @param port [String] numeric port representation.
# @return [Boolean] true when the port falls within the acceptable range.
def valid_port?(port)
value = port.to_i
value.positive? && value <= 65_535
end
# Extract the host component from a potentially bracketed domain literal.
@@ -0,0 +1,126 @@
/*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import test from 'node:test';
import assert from 'node:assert/strict';
import {
extractChatMessageMetadata,
formatChatMessagePrefix,
formatChatChannelTag,
formatNodeAnnouncementPrefix,
__test__
} from '../chat-format.js';
const {
firstNonNull,
normalizeString,
normalizeFrequency,
normalizeFrequencySlot,
FREQUENCY_PLACEHOLDER
} = __test__;
test('extractChatMessageMetadata prefers explicit region_frequency and channel_name', () => {
const payload = {
region_frequency: 868,
channel_name: ' Test Channel ',
lora_freq: 915,
channelName: 'Ignored'
};
const result = extractChatMessageMetadata(payload);
assert.deepEqual(result, { frequency: '868', channelName: 'Test Channel' });
});
test('extractChatMessageMetadata falls back to LoRa metadata', () => {
const payload = {
lora_freq: 915,
channelName: 'SpecChannel'
};
const result = extractChatMessageMetadata(payload);
assert.deepEqual(result, { frequency: '915', channelName: 'SpecChannel' });
});
test('extractChatMessageMetadata returns null metadata for invalid input', () => {
assert.deepEqual(extractChatMessageMetadata(null), { frequency: null, channelName: null });
assert.deepEqual(extractChatMessageMetadata(undefined), { frequency: null, channelName: null });
});
test('firstNonNull returns the first non-null candidate', () => {
assert.equal(firstNonNull(null, undefined, '', 'value'), '');
assert.equal(firstNonNull(undefined, null), null);
});
test('normalizeString trims strings and rejects empties', () => {
assert.equal(normalizeString(' Spec '), 'Spec');
assert.equal(normalizeString(' '), null);
assert.equal(normalizeString(123), '123');
assert.equal(normalizeString(Number.POSITIVE_INFINITY), null);
});
test('normalizeFrequency handles numeric and string inputs', () => {
assert.equal(normalizeFrequency(915), '915');
assert.equal(normalizeFrequency(868.125), '868.125');
assert.equal(normalizeFrequency(' 868MHz '), '868');
assert.equal(normalizeFrequency('n/a'), 'n/a');
assert.equal(normalizeFrequency(-5), null);
assert.equal(normalizeFrequency(null), null);
});
test('formatChatMessagePrefix preserves bracket placeholders', () => {
assert.equal(
formatChatMessagePrefix({ timestamp: '11:46:48', frequency: '868' }),
'[11:46:48][868]'
);
assert.equal(
formatChatMessagePrefix({ timestamp: '16:19:19', frequency: null }),
`[16:19:19][${FREQUENCY_PLACEHOLDER}]`
);
assert.equal(
formatChatMessagePrefix({ timestamp: '09:00:00', frequency: '' }),
`[09:00:00][${FREQUENCY_PLACEHOLDER}]`
);
});
test('formatChatChannelTag wraps channel names after the short name slot', () => {
assert.equal(
formatChatChannelTag({ channelName: 'TEST' }),
'[TEST]'
);
assert.equal(
formatChatChannelTag({ channelName: '' }),
'[]'
);
assert.equal(
formatChatChannelTag({ channelName: null }),
'[]'
);
});
test('formatNodeAnnouncementPrefix includes optional frequency bracket', () => {
assert.equal(
formatNodeAnnouncementPrefix({ timestamp: '12:34:56', frequency: '868' }),
'[12:34:56][868]'
);
assert.equal(
formatNodeAnnouncementPrefix({ timestamp: '01:02:03', frequency: null }),
`[01:02:03][${FREQUENCY_PLACEHOLDER}]`
);
});
test('normalizeFrequencySlot returns placeholder when frequency is missing', () => {
assert.equal(normalizeFrequencySlot(null), FREQUENCY_PLACEHOLDER);
assert.equal(normalizeFrequencySlot(''), FREQUENCY_PLACEHOLDER);
assert.equal(normalizeFrequencySlot(undefined), FREQUENCY_PLACEHOLDER);
assert.equal(normalizeFrequencySlot('915'), '915');
});
@@ -0,0 +1,123 @@
/*
* Copyright (C) 2025 l5yth
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import test from 'node:test';
import assert from 'node:assert/strict';
import { createMessageNodeHydrator } from '../message-node-hydrator.js';
/**
* Capture warning invocations produced during a test run.
*/
class LoggerStub {
constructor() {
this.messages = [];
}
/**
* Record a warning message for later inspection.
*
* @param {...*} args Warning arguments.
* @returns {void}
*/
warn(...args) {
this.messages.push(args);
}
}
test('hydrate attaches cached nodes without performing lookups', async () => {
const node = { node_id: '!abc', short_name: 'Node' };
const nodesById = new Map([[node.node_id, node]]);
const hydrator = createMessageNodeHydrator({
fetchNodeById: async () => {
throw new Error('fetch should not be called');
},
applyNodeFallback: () => {}
});
const messages = [{ node_id: '!abc', text: 'Hello' }];
const result = await hydrator.hydrate(messages, nodesById);
assert.equal(result.length, 1);
assert.strictEqual(result[0].node, node);
assert.equal(nodesById.size, 1);
});
test('hydrate fetches missing nodes once and caches the result', async () => {
let fetchCalls = 0;
const fetchedNode = { node_id: '!fetch', short_name: 'Fetched' };
const hydrator = createMessageNodeHydrator({
fetchNodeById: async id => {
fetchCalls += 1;
assert.equal(id, '!fetch');
return { ...fetchedNode };
},
applyNodeFallback: () => {}
});
const nodesById = new Map();
const messages = [{ from_id: '!fetch', text: 'one' }, { node_id: '!fetch', text: 'two' }];
const result = await hydrator.hydrate(messages, nodesById);
assert.equal(fetchCalls, 1);
assert.strictEqual(nodesById.get('!fetch').short_name, 'Fetched');
assert.strictEqual(result[0].node, nodesById.get('!fetch'));
assert.strictEqual(result[1].node, nodesById.get('!fetch'));
});
test('hydrate falls back to placeholders when lookups fail', async () => {
const logger = new LoggerStub();
let fallbackCalls = 0;
const hydrator = createMessageNodeHydrator({
fetchNodeById: async () => null,
applyNodeFallback: node => {
fallbackCalls += 1;
if (!node.short_name) {
node.short_name = 'Fallback';
}
},
logger
});
const nodesById = new Map();
const messages = [{ from_id: '!missing', text: 'hi' }];
const result = await hydrator.hydrate(messages, nodesById);
assert.equal(nodesById.has('!missing'), false);
assert.equal(fallbackCalls, 1);
assert.ok(result[0].node);
assert.equal(result[0].node.short_name, 'Fallback');
assert.equal(logger.messages.length, 0);
});
test('hydrate records warning when fetch rejects', async () => {
const logger = new LoggerStub();
const hydrator = createMessageNodeHydrator({
fetchNodeById: async () => {
throw new Error('network error');
},
applyNodeFallback: () => {},
logger
});
const nodesById = new Map();
const messages = [{ from_id: '!warn', text: 'warn' }];
const result = await hydrator.hydrate(messages, nodesById);
assert.equal(result[0].node.node_id, '!warn');
assert.ok(logger.messages.length >= 1);
assert.equal(nodesById.has('!warn'), false);
});
@@ -26,6 +26,7 @@ const {
extractNumber,
assignString,
assignNumber,
mergeModemMetadata,
mergeNodeFields,
mergeTelemetry,
mergePosition,
@@ -49,6 +50,8 @@ test('refreshNodeInformation merges telemetry metrics when the base node lacks t
short_name: 'TST',
battery_level: null,
last_heard: 1_000,
modem_preset: 'MediumFast',
lora_freq: '868.1',
})],
['/api/telemetry/!test?limit=1', createResponse(200, [{
node_id: '!test',
@@ -87,6 +90,8 @@ test('refreshNodeInformation merges telemetry metrics when the base node lacks t
assert.equal(node.battery, 73.5);
assert.equal(node.voltage, 4.1);
assert.equal(node.role, 'CLIENT');
assert.equal(node.modemPreset, 'MediumFast');
assert.equal(node.loraFreq, 868.1);
assert.equal(node.lastHeard, 1_200);
assert.equal(node.telemetryTime, 1_180);
assert.equal(node.latitude, 52.5);
@@ -123,7 +128,7 @@ test('refreshNodeInformation preserves fallback metrics when telemetry is unavai
return response ?? createResponse(404, { error: 'not found' });
};
const fallback = { nodeNum: 42, battery: 12.5, role: 'CLIENT' };
const fallback = { nodeNum: 42, battery: 12.5, role: 'CLIENT', modemPreset: 'FallbackPreset', loraFreq: 915 };
const node = await refreshNodeInformation({ nodeNum: 42, fallback }, { fetchImpl });
assert.equal(node.nodeId, '!num');
@@ -131,6 +136,8 @@ test('refreshNodeInformation preserves fallback metrics when telemetry is unavai
assert.equal(node.shortName, 'NUM');
assert.equal(node.battery, 12.5);
assert.equal(node.role, 'CLIENT');
assert.equal(node.modemPreset, 'FallbackPreset');
assert.equal(node.loraFreq, 915);
assert.equal(Array.isArray(node.neighbors) && node.neighbors.length, 0);
});
@@ -196,6 +203,21 @@ test('refreshNodeInformation enforces a fetch implementation', async () => {
}
});
test('mergeModemMetadata respects preference flags', () => {
const target = {};
mergeModemMetadata(target, { modem_preset: 'Base', lora_freq: '915.5' });
assert.equal(target.modemPreset, 'Base');
assert.equal(target.loraFreq, 915.5);
mergeModemMetadata(target, { modem_preset: 'New', lora_freq: '433' }, { preferExisting: true });
assert.equal(target.modemPreset, 'Base');
assert.equal(target.loraFreq, 915.5);
mergeModemMetadata(target, { modem_preset: 'Updated', lora_freq: '433' }, { preferExisting: false });
assert.equal(target.modemPreset, 'Updated');
assert.equal(target.loraFreq, 433);
});
test('helper utilities normalise primitive values', () => {
assert.equal(toTrimmedString(' hello '), 'hello');
assert.equal(toTrimmedString(''), null);
@@ -0,0 +1,65 @@
/*
* Copyright (C) 2025 l5yth
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import { describe, it } from 'node:test';
import assert from 'node:assert/strict';
import { extractModemMetadata, formatLoraFrequencyMHz, formatModemDisplay, __testUtils } from '../node-modem-metadata.js';
describe('node-modem-metadata', () => {
it('extracts modem preset and frequency from mixed payloads', () => {
const payload = {
modem_preset: ' MediumFast ',
lora_freq: '915',
};
assert.deepEqual(extractModemMetadata(payload), { modemPreset: 'MediumFast', loraFreq: 915 });
});
it('falls back across naming conventions when extracting metadata', () => {
const payload = {
modemPreset: 'LongSlow',
frequency: 868,
};
assert.deepEqual(extractModemMetadata(payload), { modemPreset: 'LongSlow', loraFreq: 868 });
});
it('ignores invalid modem metadata entries', () => {
assert.deepEqual(extractModemMetadata({ modem_preset: ' ', lora_freq: 'NaN' }), {
modemPreset: null,
loraFreq: null,
});
});
it('formats positive frequencies with MHz suffix', () => {
assert.equal(formatLoraFrequencyMHz(915), '915MHz');
assert.equal(formatLoraFrequencyMHz(867.5), '867.5MHz');
assert.equal(formatLoraFrequencyMHz('433.1234'), '433.123MHz');
assert.equal(formatLoraFrequencyMHz(null), null);
});
it('combines preset and frequency for overlay display', () => {
assert.equal(formatModemDisplay('MediumFast', 868), 'MediumFast (868MHz)');
assert.equal(formatModemDisplay('ShortSlow', null), 'ShortSlow');
assert.equal(formatModemDisplay(null, 433), '433MHz');
assert.equal(formatModemDisplay(undefined, undefined), null);
});
it('exposes trimmed string helper for targeted assertions', () => {
const { toTrimmedString } = __testUtils;
assert.equal(toTrimmedString(' hello '), 'hello');
assert.equal(toTrimmedString(''), null);
assert.equal(toTrimmedString(null), null);
});
});
+194
View File
@@ -0,0 +1,194 @@
/*
* Copyright (C) 2025 l5yth
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/**
* Extract channel metadata from a message payload for chat display.
*
* @param {Object} message Raw message payload from the API.
* @returns {{ frequency: string|null, channelName: string|null }}
* Normalized metadata values.
*/
export function extractChatMessageMetadata(message) {
if (!message || typeof message !== 'object') {
return { frequency: null, channelName: null };
}
const frequency = normalizeFrequency(
firstNonNull(
message.region_frequency,
message.regionFrequency,
message.lora_freq,
message.loraFreq,
message.frequency
)
);
const channelName = normalizeString(
firstNonNull(message.channel_name, message.channelName)
);
return { frequency, channelName };
}
/**
* Produce the formatted prefix for a chat message entry.
*
* Timestamp and frequency will each be wrapped in square brackets. Missing
* metadata values result in empty brackets (with the frequency replaced by the
* configured placeholder) to preserve the positional layout expected by
* operators.
*
* @param {{
* timestamp: string,
* frequency: string|null
* }} params Normalised and escaped display strings.
* @returns {string} Prefix string suitable for HTML insertion.
*/
export function formatChatMessagePrefix({ timestamp, frequency }) {
const ts = typeof timestamp === 'string' ? timestamp : '';
const freq = normalizeFrequencySlot(frequency);
return `[${ts}][${freq}]`;
}
/**
* Render the channel tag that follows the short name in a chat message entry.
*
* Empty channel names remain blank within the brackets, mirroring the original
* UI behaviour that reserves the slot without introducing placeholder text.
*
* @param {{ channelName: string|null }} params Normalised and escaped display strings.
* @returns {string} Channel tag suitable for HTML insertion.
*/
export function formatChatChannelTag({ channelName }) {
const channel = typeof channelName === 'string' ? channelName : channelName == null ? '' : String(channelName);
return `[${channel}]`;
}
/**
* Create the formatted prefix for node announcements in the chat log.
*
* Both the timestamp and the optional frequency will be wrapped in brackets,
* mirroring the chat message display while omitting the channel indicator.
*
* @param {{ timestamp: string, frequency: string|null }} params Display strings.
* @returns {string} Prefix string suitable for HTML insertion.
*/
export function formatNodeAnnouncementPrefix({ timestamp, frequency }) {
const ts = typeof timestamp === 'string' ? timestamp : '';
const freq = normalizeFrequencySlot(frequency);
return `[${ts}][${freq}]`;
}
/**
* Produce a consistently formatted frequency slot for chat prefixes.
*
* A missing or empty frequency is rendered as three HTML non-breaking spaces to
* ensure the UI maintains its expected alignment while clearly indicating the
* absence of data.
*
* @param {*} value Frequency value that has already been escaped for HTML.
* @returns {string} Frequency slot suitable for prefix rendering.
*/
function normalizeFrequencySlot(value) {
if (value == null) {
return FREQUENCY_PLACEHOLDER;
}
if (typeof value === 'string') {
return value.length > 0 ? value : FREQUENCY_PLACEHOLDER;
}
const strValue = String(value);
return strValue.length > 0 ? strValue : FREQUENCY_PLACEHOLDER;
}
/**
* HTML entity sequence inserted when a frequency is unavailable.
* @type {string}
*/
const FREQUENCY_PLACEHOLDER = '&nbsp;&nbsp;&nbsp;';
/**
* Return the first value in ``candidates`` that is not ``null`` or ``undefined``.
*
* @param {...*} candidates Candidate values.
* @returns {*} First present value or ``null`` when missing.
*/
function firstNonNull(...candidates) {
for (const value of candidates) {
if (value !== null && value !== undefined) {
return value;
}
}
return null;
}
/**
* Normalise potential channel name values to trimmed strings.
*
* @param {*} value Raw value.
* @returns {string|null} Sanitised channel name.
*/
function normalizeString(value) {
if (value == null) return null;
if (typeof value === 'string') {
const trimmed = value.trim();
return trimmed.length > 0 ? trimmed : null;
}
if (typeof value === 'number') {
if (!Number.isFinite(value)) return null;
return String(value);
}
return null;
}
/**
* Convert various frequency representations into clean strings.
*
* @param {*} value Raw frequency value.
* @returns {string|null} Frequency in MHz as a string, when available.
*/
function normalizeFrequency(value) {
if (value == null) return null;
if (typeof value === 'number') {
if (!Number.isFinite(value) || value <= 0) {
return null;
}
return Number.isInteger(value) ? String(value) : String(Number(value.toFixed(3)));
}
if (typeof value === 'string') {
const trimmed = value.trim();
if (!trimmed) return null;
const numericMatch = trimmed.match(/\d+(?:\.\d+)?/);
if (numericMatch) {
const parsed = Number(numericMatch[0]);
if (Number.isFinite(parsed) && parsed > 0) {
return Number.isInteger(parsed) ? String(Math.trunc(parsed)) : String(parsed);
}
}
return trimmed;
}
return null;
}
export const __test__ = {
firstNonNull,
normalizeString,
normalizeFrequency,
formatChatMessagePrefix,
formatNodeAnnouncementPrefix,
normalizeFrequencySlot,
FREQUENCY_PLACEHOLDER,
formatChatChannelTag
};
+86 -8
View File
@@ -19,6 +19,14 @@ import { createMapAutoFitController } from './map-auto-fit-controller.js';
import { attachNodeInfoRefreshToMarker, overlayToPopupNode } from './map-marker-node-info.js';
import { createShortInfoOverlayStack } from './short-info-overlay-manager.js';
import { refreshNodeInformation } from './node-details.js';
import { extractModemMetadata, formatModemDisplay } from './node-modem-metadata.js';
import { createMessageNodeHydrator } from './message-node-hydrator.js';
import {
extractChatMessageMetadata,
formatChatMessagePrefix,
formatChatChannelTag,
formatNodeAnnouncementPrefix
} from './chat-format.js';
/**
* Entry point for the interactive dashboard. Wires up event listeners,
@@ -103,6 +111,11 @@ export function initializeApp(config) {
let allNeighbors = [];
/** @type {Map<string, Object>} */
let nodesById = new Map();
const messageNodeHydrator = createMessageNodeHydrator({
fetchNodeById,
applyNodeFallback: applyNodeNameFallback,
logger: console,
});
/** @type {string|undefined} */
let lastChatDate;
const NODE_LIMIT = 1000;
@@ -115,6 +128,29 @@ export function initializeApp(config) {
/** @type {ReturnType<typeof setTimeout>|null} */
let refreshTimer = null;
/**
* Close any open short-info overlays that do not contain the provided anchor.
*
* The method preserves ancestor overlays that host nested short-name badges,
* ensuring context overlays (for example, neighbor listings) remain visible
* while unrelated overlays are dismissed.
*
* @param {?Element} anchorEl Short-name badge that triggered the interaction.
* @returns {void}
*/
function closeUnrelatedShortOverlays(anchorEl) {
if (!anchorEl) {
return;
}
const openOverlays = overlayStack.getOpenOverlays();
for (const entry of openOverlays) {
if (!entry || !entry.element || entry.element.contains(anchorEl)) {
continue;
}
overlayStack.close(entry.anchor);
}
}
/**
* Determine whether the provided value contains a non-empty string.
*
@@ -1512,6 +1548,7 @@ export function initializeApp(config) {
const nodeNum = Number.isFinite(fallbackDetails.nodeNum) ? fallbackDetails.nodeNum : null;
if (!nodeId && !nodeNum) {
closeUnrelatedShortOverlays(shortTarget);
openShortInfoOverlay(shortTarget, fallbackDetails);
return;
}
@@ -1526,6 +1563,7 @@ export function initializeApp(config) {
if (!overlayDetails.shortName && shortTarget.textContent) {
overlayDetails.shortName = shortTarget.textContent.replace(/\u00a0/g, ' ').trim();
}
closeUnrelatedShortOverlays(shortTarget);
openShortInfoOverlay(shortTarget, overlayDetails);
})
.catch(err => {
@@ -1535,6 +1573,7 @@ export function initializeApp(config) {
if (!overlayDetails.shortName && shortTarget.textContent) {
overlayDetails.shortName = shortTarget.textContent.replace(/\u00a0/g, ' ').trim();
}
closeUnrelatedShortOverlays(shortTarget);
openShortInfoOverlay(shortTarget, overlayDetails);
});
return;
@@ -1860,6 +1899,14 @@ export function initializeApp(config) {
normalized.hwModel = source.hwModel ?? source.hw_model;
}
const modemMetadata = extractModemMetadata(source);
if (modemMetadata.modemPreset) {
normalized.modemPreset = modemMetadata.modemPreset;
}
if (modemMetadata.loraFreq != null) {
normalized.loraFreq = modemMetadata.loraFreq;
}
const numericPairs = [
['battery', source.battery ?? source.battery_level],
['voltage', source.voltage],
@@ -1969,6 +2016,10 @@ export function initializeApp(config) {
if (shortParts.length) {
lines.push(shortParts.join(' '));
}
const modemDisplay = formatModemDisplay(overlayInfo.modemPreset, overlayInfo.loraFreq);
if (modemDisplay) {
lines.push(escapeHtml(modemDisplay));
}
const roleValue = shortInfoValueOrDash(overlayInfo.role || 'CLIENT');
if (roleValue !== '—') {
lines.push(`Role: ${escapeHtml(roleValue)}`);
@@ -2069,7 +2120,12 @@ export function initializeApp(config) {
div.className = 'chat-entry-node';
const short = renderShortHtml(n.short_name, n.role, n.long_name, n);
const longName = escapeHtml(n.long_name || '');
div.innerHTML = `[${ts}] ${short} <em>New node: ${longName}</em>`;
const metadata = extractChatMessageMetadata(n);
const prefix = formatNodeAnnouncementPrefix({
timestamp: escapeHtml(ts),
frequency: metadata.frequency ? escapeHtml(metadata.frequency) : ''
});
div.innerHTML = `${prefix} ${short} <em>New node: ${longName}</em>`;
return div;
}
@@ -2084,8 +2140,16 @@ export function initializeApp(config) {
const ts = formatTime(new Date(m.rx_time * 1000));
const short = renderShortHtml(m.node?.short_name, m.node?.role, m.node?.long_name, m.node);
const text = escapeHtml(m.text || '');
const metadata = extractChatMessageMetadata(m);
const prefix = formatChatMessagePrefix({
timestamp: escapeHtml(ts),
frequency: metadata.frequency ? escapeHtml(metadata.frequency) : ''
});
const channelTag = formatChatChannelTag({
channelName: metadata.channelName ? escapeHtml(metadata.channelName) : ''
});
div.className = 'chat-entry-msg';
div.innerHTML = `[${ts}] ${short} ${text}`;
div.innerHTML = `${prefix} ${short} ${channelTag} ${text}`;
return div;
}
@@ -2359,6 +2423,22 @@ export function initializeApp(config) {
return r.json();
}
/**
* Retrieve a single node record by identifier from the API.
*
* @param {string} nodeId Canonical node identifier.
* @returns {Promise<Object|null>} Parsed node payload or null when absent.
*/
async function fetchNodeById(nodeId) {
if (typeof nodeId !== 'string') return null;
const trimmed = nodeId.trim();
if (trimmed.length === 0) return null;
const r = await fetch(`/api/nodes/${encodeURIComponent(trimmed)}`, { cache: 'no-store' });
if (r.status === 404) return null;
if (!r.ok) throw new Error('HTTP ' + r.status);
return r.json();
}
/**
* Fetch recent messages from the JSON API.
*
@@ -2861,12 +2941,14 @@ export function initializeApp(config) {
},
showDetails: (anchor, info) => {
if (anchor) {
closeUnrelatedShortOverlays(anchor);
openShortInfoOverlay(anchor, info);
}
},
showError: (anchor, info, error) => {
console.warn('Failed to refresh node information for map marker', error);
if (anchor) {
closeUnrelatedShortOverlays(anchor);
openShortInfoOverlay(anchor, info);
}
},
@@ -3001,14 +3083,10 @@ export function initializeApp(config) {
mergePositionsIntoNodes(nodes, positions);
computeDistances(nodes);
mergeTelemetryIntoNodes(nodes, telemetryEntries);
if (Array.isArray(messages)) {
messages.forEach(message => {
if (message && message.node) applyNodeNameFallback(message.node);
});
}
renderChatLog(nodes, messages);
allNodes = nodes;
rebuildNodeIndex(allNodes);
const chatMessages = await messageNodeHydrator.hydrate(messages, nodesById);
renderChatLog(nodes, chatMessages);
allNeighbors = Array.isArray(neighborTuples) ? neighborTuples : [];
applyFilter();
statusEl.textContent = 'updated ' + new Date().toLocaleTimeString();
@@ -0,0 +1,150 @@
/*
* Copyright (C) 2025 l5yth
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/**
* Build a hydrator capable of attaching node metadata to chat messages.
*
* @param {{
* fetchNodeById: (nodeId: string) => Promise<object|null>,
* applyNodeFallback: (node: object) => void,
* logger?: { warn?: (message?: any, ...optionalParams: any[]) => void }
* }} options Factory configuration.
* @returns {{
* hydrate: (messages: Array<object>|null|undefined, nodesById: Map<string, object>) => Promise<Array<object>>
* }} Hydrator API.
*/
export function createMessageNodeHydrator({ fetchNodeById, applyNodeFallback, logger = console }) {
if (typeof fetchNodeById !== 'function') {
throw new TypeError('fetchNodeById must be a function');
}
if (typeof applyNodeFallback !== 'function') {
throw new TypeError('applyNodeFallback must be a function');
}
/** @type {Map<string, Promise<object|null>>} */
const inflightLookups = new Map();
/**
* Normalise potential node identifiers into canonical strings.
*
* @param {*} value Raw node identifier value.
* @returns {string} Trimmed identifier or empty string when invalid.
*/
function normalizeNodeId(value) {
if (value == null) return '';
const source = typeof value === 'string' ? value : String(value);
const trimmed = source.trim();
return trimmed.length > 0 ? trimmed : '';
}
/**
* Resolve the node metadata for the provided identifier.
*
* @param {string} nodeId Canonical node identifier.
* @param {Map<string, object>} nodesById Existing node cache.
* @returns {Promise<object|null>} Resolved node or null when unavailable.
*/
async function resolveNode(nodeId, nodesById) {
const id = normalizeNodeId(nodeId);
if (!id) return null;
if (nodesById instanceof Map && nodesById.has(id)) {
return nodesById.get(id);
}
if (inflightLookups.has(id)) {
return inflightLookups.get(id);
}
const promise = Promise.resolve()
.then(() => fetchNodeById(id))
.then(node => {
if (node && typeof node === 'object') {
applyNodeFallback(node);
if (nodesById instanceof Map) {
nodesById.set(id, node);
}
return node;
}
return null;
})
.catch(error => {
if (logger && typeof logger.warn === 'function') {
logger.warn('message node lookup failed', { nodeId: id, error });
}
return null;
})
.finally(() => {
inflightLookups.delete(id);
});
inflightLookups.set(id, promise);
return promise;
}
/**
* Attach node information to the provided message collection.
*
* @param {Array<object>|null|undefined} messages Message payloads from the API.
* @param {Map<string, object>} nodesById Lookup table of known nodes.
* @returns {Promise<Array<object>>} Hydrated message entries.
*/
async function hydrate(messages, nodesById) {
if (!Array.isArray(messages) || messages.length === 0) {
return Array.isArray(messages) ? messages : [];
}
const tasks = [];
for (const message of messages) {
if (!message || typeof message !== 'object') {
continue;
}
const explicitId = normalizeNodeId(message.node_id ?? message.nodeId ?? '');
const fallbackId = normalizeNodeId(message.from_id ?? message.fromId ?? '');
const targetId = explicitId || fallbackId;
if (!targetId) {
message.node = null;
continue;
}
message.node_id = targetId;
const existing = nodesById instanceof Map ? nodesById.get(targetId) : null;
if (existing) {
message.node = existing;
continue;
}
const task = resolveNode(targetId, nodesById).then(node => {
if (node) {
message.node = node;
} else {
const placeholder = { node_id: targetId };
applyNodeFallback(placeholder);
message.node = placeholder;
}
});
tasks.push(task);
}
if (tasks.length > 0) {
await Promise.all(tasks);
}
return messages;
}
return { hydrate };
}
+29
View File
@@ -14,6 +14,8 @@
* limitations under the License.
*/
import { extractModemMetadata } from './node-modem-metadata.js';
const DEFAULT_FETCH_OPTIONS = Object.freeze({ cache: 'no-store' });
const TELEMETRY_LIMIT = 1;
const POSITION_LIMIT = 1;
@@ -130,6 +132,30 @@ function assignNumber(target, key, value, { preferExisting = false } = {}) {
target[key] = numericValue;
}
/**
* Merge modem preset and frequency metadata into the aggregate node object.
*
* @param {Object} target Mutable aggregate node reference.
* @param {*} source Source record inspected for modem attributes.
* @param {{ preferExisting?: boolean }} [options] Behaviour modifiers.
* @returns {void}
*/
function mergeModemMetadata(target, source, { preferExisting = false } = {}) {
if (!isObject(target)) return;
if (!source || typeof source !== 'object') return;
const metadata = extractModemMetadata(source);
if (metadata.modemPreset) {
if (!preferExisting || toTrimmedString(target.modemPreset) == null) {
target.modemPreset = metadata.modemPreset;
}
}
if (metadata.loraFreq != null) {
if (!preferExisting || toFiniteNumber(target.loraFreq) == null) {
target.loraFreq = metadata.loraFreq;
}
}
}
/**
* Merge base node fields from an arbitrary record into the aggregate node object.
*
@@ -145,6 +171,7 @@ function mergeNodeFields(target, record) {
assignString(target, 'longName', extractString(record, ['longName', 'long_name']));
assignString(target, 'role', extractString(record, ['role']));
assignString(target, 'hwModel', extractString(record, ['hwModel', 'hw_model']));
mergeModemMetadata(target, record);
assignNumber(target, 'snr', extractNumber(record, ['snr']));
assignNumber(target, 'battery', extractNumber(record, ['battery', 'battery_level', 'batteryLevel']));
assignNumber(target, 'voltage', extractNumber(record, ['voltage']));
@@ -176,6 +203,7 @@ function mergeTelemetry(target, telemetry) {
target.telemetry = telemetry;
assignString(target, 'nodeId', extractString(telemetry, ['node_id', 'nodeId']), { preferExisting: true });
assignNumber(target, 'nodeNum', extractNumber(telemetry, ['node_num', 'nodeNum']), { preferExisting: true });
mergeModemMetadata(target, telemetry, { preferExisting: true });
assignNumber(target, 'battery', extractNumber(telemetry, ['battery_level', 'batteryLevel']), { preferExisting: true });
assignNumber(target, 'voltage', extractNumber(telemetry, ['voltage']), { preferExisting: true });
assignNumber(target, 'uptime', extractNumber(telemetry, ['uptime_seconds', 'uptimeSeconds']), { preferExisting: true });
@@ -408,6 +436,7 @@ export const __testUtils = {
extractNumber,
assignString,
assignNumber,
mergeModemMetadata,
mergeNodeFields,
mergeTelemetry,
mergePosition,
@@ -0,0 +1,95 @@
/*
* Copyright (C) 2025 l5yth
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/**
* Convert arbitrary input into a trimmed string representation.
*
* @param {*} value Candidate value.
* @returns {string|null} Trimmed string or ``null`` when empty.
*/
function toTrimmedString(value) {
if (value == null) return null;
const stringValue = String(value).trim();
return stringValue.length > 0 ? stringValue : null;
}
/**
* Normalize modem-related metadata from a node-shaped record.
*
* @param {*} source Arbitrary payload that may contain modem attributes.
* @returns {{ modemPreset: (string|null), loraFreq: (number|null) }} Normalized modem metadata.
*/
export function extractModemMetadata(source) {
if (!source || typeof source !== 'object') {
return { modemPreset: null, loraFreq: null };
}
const presetCandidate =
source.modemPreset ?? source.modem_preset ?? source.modempreset ?? source.ModemPreset ?? null;
const modemPreset = toTrimmedString(presetCandidate);
const freqCandidate = source.loraFreq ?? source.lora_freq ?? source.frequency ?? null;
const parsedFreq = Number(freqCandidate);
const loraFreq = Number.isFinite(parsedFreq) && parsedFreq > 0 ? parsedFreq : null;
return { modemPreset, loraFreq };
}
/**
* Format a numeric LoRa frequency in MHz with up to three fractional digits.
*
* @param {*} value Numeric frequency in MHz.
* @returns {string|null} Formatted frequency with units or ``null`` when invalid.
*/
export function formatLoraFrequencyMHz(value) {
const numeric = typeof value === 'number' ? value : Number(value);
if (!Number.isFinite(numeric) || numeric <= 0) {
return null;
}
const formatter = new Intl.NumberFormat('en-US', {
minimumFractionDigits: 0,
maximumFractionDigits: 3,
});
return `${formatter.format(numeric)}MHz`;
}
/**
* Produce a combined modem preset and frequency description suitable for overlays.
*
* @param {*} preset Raw modem preset value.
* @param {*} frequency Raw frequency value expressed in MHz.
* @returns {string|null} Human-readable description or ``null`` when no data available.
*/
export function formatModemDisplay(preset, frequency) {
const presetText = toTrimmedString(preset);
const freqText = formatLoraFrequencyMHz(frequency);
if (!presetText && !freqText) {
return null;
}
if (presetText && freqText) {
return `${presetText} (${freqText})`;
}
return presetText ?? freqText;
}
export const __testUtils = {
toTrimmedString,
};
+938 -126
View File
File diff suppressed because it is too large Load Diff
+80
View File
@@ -137,6 +137,86 @@ RSpec.describe PotatoMesh::Config do
end
end
describe ".remote_instance_http_timeout" do
it "returns the baked-in connect timeout" do
expect(described_class.remote_instance_http_timeout).to eq(
PotatoMesh::Config::DEFAULT_REMOTE_INSTANCE_CONNECT_TIMEOUT,
)
end
it "ignores environment overrides" do
within_env("REMOTE_INSTANCE_CONNECT_TIMEOUT" => "27") do
expect(described_class.remote_instance_http_timeout).to eq(
PotatoMesh::Config::DEFAULT_REMOTE_INSTANCE_CONNECT_TIMEOUT,
)
end
end
end
describe ".remote_instance_read_timeout" do
it "returns the baked-in read timeout" do
expect(described_class.remote_instance_read_timeout).to eq(
PotatoMesh::Config::DEFAULT_REMOTE_INSTANCE_READ_TIMEOUT,
)
end
it "ignores environment overrides" do
within_env("REMOTE_INSTANCE_READ_TIMEOUT" => "20") do
expect(described_class.remote_instance_read_timeout).to eq(
PotatoMesh::Config::DEFAULT_REMOTE_INSTANCE_READ_TIMEOUT,
)
end
end
end
describe ".federation_max_instances_per_response" do
it "returns the baked-in response limit when unset" do
within_env("FEDERATION_MAX_INSTANCES_PER_RESPONSE" => nil) do
expect(described_class.federation_max_instances_per_response).to eq(
PotatoMesh::Config::DEFAULT_FEDERATION_MAX_INSTANCES_PER_RESPONSE,
)
end
end
it "accepts positive overrides" do
within_env("FEDERATION_MAX_INSTANCES_PER_RESPONSE" => "7") do
expect(described_class.federation_max_instances_per_response).to eq(7)
end
end
it "rejects non-positive overrides" do
within_env("FEDERATION_MAX_INSTANCES_PER_RESPONSE" => "0") do
expect(described_class.federation_max_instances_per_response).to eq(
PotatoMesh::Config::DEFAULT_FEDERATION_MAX_INSTANCES_PER_RESPONSE,
)
end
end
end
describe ".federation_max_domains_per_crawl" do
it "returns the baked-in crawl limit when unset" do
within_env("FEDERATION_MAX_DOMAINS_PER_CRAWL" => nil) do
expect(described_class.federation_max_domains_per_crawl).to eq(
PotatoMesh::Config::DEFAULT_FEDERATION_MAX_DOMAINS_PER_CRAWL,
)
end
end
it "accepts positive overrides" do
within_env("FEDERATION_MAX_DOMAINS_PER_CRAWL" => "11") do
expect(described_class.federation_max_domains_per_crawl).to eq(11)
end
end
it "rejects invalid overrides" do
within_env("FEDERATION_MAX_DOMAINS_PER_CRAWL" => "-5") do
expect(described_class.federation_max_domains_per_crawl).to eq(
PotatoMesh::Config::DEFAULT_FEDERATION_MAX_DOMAINS_PER_CRAWL,
)
end
end
end
describe ".db_path" do
it "returns the default path inside the data directory" do
expect(described_class.db_path).to eq(described_class.default_db_path)
+245 -6
View File
@@ -15,7 +15,9 @@
require "spec_helper"
require "net/http"
require "openssl"
require "set"
require "uri"
require "socket"
RSpec.describe PotatoMesh::App::Federation do
subject(:federation_helpers) do
@@ -34,6 +36,18 @@ RSpec.describe PotatoMesh::App::Federation do
def reset_debug_messages
@debug_messages = []
end
def warn_messages
@warn_messages ||= []
end
def warn_log(message, **_metadata)
warn_messages << message
end
def reset_warn_messages
@warn_messages = []
end
end
end
end
@@ -42,6 +56,7 @@ RSpec.describe PotatoMesh::App::Federation do
federation_helpers.instance_variable_set(:@remote_instance_cert_store, nil)
federation_helpers.instance_variable_set(:@remote_instance_verify_callback, nil)
federation_helpers.reset_debug_messages
federation_helpers.reset_warn_messages
end
describe ".remote_instance_cert_store" do
@@ -113,10 +128,14 @@ RSpec.describe PotatoMesh::App::Federation do
end
describe ".build_remote_http_client" do
let(:timeout) { 15 }
let(:connect_timeout) { 5 }
let(:read_timeout) { 12 }
let(:public_addrinfo) { Addrinfo.ip("203.0.113.5") }
before do
allow(PotatoMesh::Config).to receive(:remote_instance_http_timeout).and_return(timeout)
allow(PotatoMesh::Config).to receive(:remote_instance_http_timeout).and_return(connect_timeout)
allow(PotatoMesh::Config).to receive(:remote_instance_read_timeout).and_return(read_timeout)
allow(Addrinfo).to receive(:getaddrinfo).and_return([public_addrinfo])
end
it "configures SSL settings for HTTPS endpoints" do
@@ -129,8 +148,8 @@ RSpec.describe PotatoMesh::App::Federation do
http = federation_helpers.build_remote_http_client(uri)
expect(http.use_ssl?).to be(true)
expect(http.open_timeout).to eq(timeout)
expect(http.read_timeout).to eq(timeout)
expect(http.open_timeout).to eq(connect_timeout)
expect(http.read_timeout).to eq(read_timeout)
expect(http.cert_store).to eq(store)
expect(http.verify_mode).to eq(OpenSSL::SSL::VERIFY_PEER)
expect(http.verify_callback).to eq(callback)
@@ -146,8 +165,8 @@ RSpec.describe PotatoMesh::App::Federation do
expect(http.use_ssl?).to be(false)
expect(http.cert_store).to be_nil
expect(http.open_timeout).to eq(timeout)
expect(http.read_timeout).to eq(timeout)
expect(http.open_timeout).to eq(connect_timeout)
expect(http.read_timeout).to eq(read_timeout)
end
it "leaves the certificate store unset when unavailable" do
@@ -160,5 +179,225 @@ RSpec.describe PotatoMesh::App::Federation do
expect(http.cert_store).to be_nil
expect(http.verify_callback).to be_nil
end
it "rejects URIs that resolve exclusively to restricted addresses" do
uri = URI.parse("https://loopback.mesh/api")
allow(Addrinfo).to receive(:getaddrinfo).and_return([Addrinfo.ip("127.0.0.1")])
expect do
federation_helpers.build_remote_http_client(uri)
end.to raise_error(ArgumentError, "restricted domain")
end
it "binds the HTTP client to the first unrestricted address" do
uri = URI.parse("https://remote.example.com/api")
allow(Addrinfo).to receive(:getaddrinfo).and_return([
Addrinfo.ip("127.0.0.1"),
public_addrinfo,
Addrinfo.ip("10.0.0.3"),
])
http = federation_helpers.build_remote_http_client(uri)
if http.respond_to?(:ipaddr)
expect(http.ipaddr).to eq("203.0.113.5")
else
skip "Net::HTTP#ipaddr accessor unavailable"
end
end
end
describe ".ingest_known_instances_from!" do
let(:db) { double(:db) }
let(:seed_domain) { "seed.mesh" }
let(:payload_entries) do
Array.new(3) do |index|
{
"id" => "remote-#{index}",
"domain" => "ally-#{index}.mesh",
"pubkey" => "ignored-pubkey-#{index}",
"signature" => "ignored-signature-#{index}",
}
end
end
let(:attributes_list) do
payload_entries.map do |entry|
{
id: entry["id"],
domain: entry["domain"],
pubkey: entry["pubkey"],
name: nil,
version: nil,
channel: nil,
frequency: nil,
latitude: nil,
longitude: nil,
last_update_time: nil,
is_private: false,
}
end
end
let(:node_payload) do
Array.new(PotatoMesh::Config.remote_instance_min_node_count) do |index|
{ "node_id" => "node-#{index}", "last_heard" => Time.now.to_i - index }
end
end
let(:response_map) do
mapping = { [seed_domain, "/api/instances"] => [payload_entries, :instances] }
attributes_list.each do |attributes|
mapping[[attributes[:domain], "/api/nodes"]] = [node_payload, :nodes]
mapping[[attributes[:domain], "/api/instances"]] = [[], :instances]
end
mapping
end
before do
allow(federation_helpers).to receive(:fetch_instance_json) do |host, path|
response_map.fetch([host, path]) { [nil, []] }
end
allow(federation_helpers).to receive(:verify_instance_signature).and_return(true)
allow(federation_helpers).to receive(:validate_remote_nodes).and_return([true, nil])
payload_entries.each_with_index do |entry, index|
allow(federation_helpers).to receive(:remote_instance_attributes_from_payload).with(entry).and_return([attributes_list[index], "signature-#{index}", nil])
end
end
it "stops processing once the per-response limit is exceeded" do
processed_domains = []
allow(federation_helpers).to receive(:upsert_instance_record) do |_db, attrs, _signature|
processed_domains << attrs[:domain]
end
allow(PotatoMesh::Config).to receive(:federation_max_instances_per_response).and_return(2)
allow(PotatoMesh::Config).to receive(:federation_max_domains_per_crawl).and_return(10)
visited = federation_helpers.ingest_known_instances_from!(db, seed_domain)
expect(processed_domains).to eq([
attributes_list[0][:domain],
attributes_list[1][:domain],
])
expect(visited).to include(seed_domain, attributes_list[0][:domain], attributes_list[1][:domain])
expect(visited).not_to include(attributes_list[2][:domain])
expect(federation_helpers.debug_messages).to include(a_string_including("response limit"))
end
it "halts recursion once the crawl limit would be exceeded" do
processed_domains = []
allow(federation_helpers).to receive(:upsert_instance_record) do |_db, attrs, _signature|
processed_domains << attrs[:domain]
end
allow(PotatoMesh::Config).to receive(:federation_max_instances_per_response).and_return(5)
allow(PotatoMesh::Config).to receive(:federation_max_domains_per_crawl).and_return(2)
visited = federation_helpers.ingest_known_instances_from!(db, seed_domain)
expect(processed_domains).to eq([attributes_list.first[:domain]])
expect(visited).to include(seed_domain, attributes_list.first[:domain])
expect(visited).not_to include(attributes_list[1][:domain], attributes_list[2][:domain])
expect(federation_helpers.debug_messages).to include(a_string_including("crawl limit"))
end
end
describe ".perform_instance_http_request" do
let(:uri) { URI.parse("https://remote.example.com/api") }
let(:http_client) { instance_double(Net::HTTP) }
before do
allow(federation_helpers).to receive(:build_remote_http_client).with(uri).and_return(http_client)
end
it "wraps errors that omit a message with the error class name" do
stub_const(
"RemoteTcpFailure",
Class.new(StandardError) do
def message
""
end
end,
)
allow(http_client).to receive(:start).and_raise(RemoteTcpFailure.new)
expect do
federation_helpers.send(:perform_instance_http_request, uri)
end.to raise_error(PotatoMesh::App::InstanceFetchError, "RemoteTcpFailure")
end
it "includes the error class name when the message omits it" do
allow(http_client).to receive(:start).and_raise(OpenSSL::SSL::SSLError.new("handshake failed"))
expect do
federation_helpers.send(:perform_instance_http_request, uri)
end.to raise_error(
PotatoMesh::App::InstanceFetchError,
"OpenSSL::SSL::SSLError: handshake failed",
)
end
it "preserves messages that already include the error class" do
allow(http_client).to receive(:start).and_raise(Net::ReadTimeout.new)
expect do
federation_helpers.send(:perform_instance_http_request, uri)
end.to raise_error(PotatoMesh::App::InstanceFetchError, "Net::ReadTimeout")
end
it "wraps restricted address resolution failures" do
allow(federation_helpers).to receive(:build_remote_http_client).and_call_original
allow(Addrinfo).to receive(:getaddrinfo).and_return([Addrinfo.ip("127.0.0.1")])
expect do
federation_helpers.send(:perform_instance_http_request, uri)
end.to raise_error(PotatoMesh::App::InstanceFetchError, "ArgumentError: restricted domain")
end
end
describe ".announce_instance_to_domain" do
let(:payload) { "{}" }
let(:https_uri) { URI.parse("https://remote.mesh/api/instances") }
let(:http_uri) { URI.parse("http://remote.mesh/api/instances") }
let(:http_connection) { instance_double("Net::HTTPConnection") }
let(:success_response) { Net::HTTPOK.new("1.1", "200", "OK") }
before do
allow(success_response).to receive(:code).and_return("200")
end
it "retries over HTTP when HTTPS connections are refused" do
https_client = instance_double(Net::HTTP)
http_client = instance_double(Net::HTTP)
allow(federation_helpers).to receive(:build_remote_http_client).with(https_uri).and_return(https_client)
allow(federation_helpers).to receive(:build_remote_http_client).with(http_uri).and_return(http_client)
allow(https_client).to receive(:start).and_raise(Errno::ECONNREFUSED.new("refused"))
allow(http_connection).to receive(:request).and_return(success_response)
allow(http_client).to receive(:start).and_yield(http_connection).and_return(success_response)
result = federation_helpers.announce_instance_to_domain("remote.mesh", payload)
expect(result).to be(true)
expect(federation_helpers.debug_messages).to include("HTTPS federation announcement failed, retrying with HTTP")
expect(federation_helpers.warn_messages).to be_empty
end
it "logs a warning when HTTPS refusal persists after HTTP fallback" do
https_client = instance_double(Net::HTTP)
http_client = instance_double(Net::HTTP)
allow(federation_helpers).to receive(:build_remote_http_client).with(https_uri).and_return(https_client)
allow(federation_helpers).to receive(:build_remote_http_client).with(http_uri).and_return(http_client)
allow(https_client).to receive(:start).and_raise(Errno::ECONNREFUSED.new("refused"))
allow(http_client).to receive(:start).and_raise(SocketError.new("dns failure"))
result = federation_helpers.announce_instance_to_domain("remote.mesh", payload)
expect(result).to be(false)
expect(federation_helpers.debug_messages).to include("HTTPS federation announcement failed, retrying with HTTP")
expect(
federation_helpers.warn_messages.count { |message| message.include?("Federation announcement raised exception") },
).to eq(2)
end
end
end
+55
View File
@@ -64,4 +64,59 @@ RSpec.describe PotatoMesh::App::Identity do
allow(PotatoMesh::Config).to receive(:legacy_keyfile_candidates).and_call_original
end
end
describe ".refresh_well_known_document_if_stale" do
let(:storage_dir) { Dir.mktmpdir }
let(:well_known_path) do
File.join(storage_dir, File.basename(PotatoMesh::Config.well_known_relative_path))
end
before do
allow(PotatoMesh::Config).to receive(:well_known_storage_root).and_return(storage_dir)
allow(PotatoMesh::Config).to receive(:well_known_relative_path).and_return(".well-known/potato-mesh")
allow(PotatoMesh::Config).to receive(:well_known_refresh_interval).and_return(86_400)
allow(PotatoMesh::Sanitizer).to receive(:sanitized_site_name).and_return("Test Instance")
allow(PotatoMesh::Sanitizer).to receive(:sanitize_instance_domain).and_return("example.com")
end
after do
FileUtils.remove_entry(storage_dir)
allow(PotatoMesh::Config).to receive(:well_known_storage_root).and_call_original
allow(PotatoMesh::Config).to receive(:well_known_relative_path).and_call_original
allow(PotatoMesh::Config).to receive(:well_known_refresh_interval).and_call_original
allow(PotatoMesh::Sanitizer).to receive(:sanitized_site_name).and_call_original
allow(PotatoMesh::Sanitizer).to receive(:sanitize_instance_domain).and_call_original
end
it "writes a well-known document when none exists" do
PotatoMesh::Application.refresh_well_known_document_if_stale
expect(File.exist?(well_known_path)).to be(true)
document = JSON.parse(File.read(well_known_path))
expect(document.fetch("version")).to eq(PotatoMesh::Application::APP_VERSION)
expect(document.fetch("domain")).to eq("example.com")
end
it "rewrites the document when configuration values change" do
PotatoMesh::Application.refresh_well_known_document_if_stale
original_contents = File.binread(well_known_path)
stub_const("PotatoMesh::Application::APP_VERSION", "9.9.9-test")
PotatoMesh::Application.refresh_well_known_document_if_stale
rewritten_contents = File.binread(well_known_path)
expect(rewritten_contents).not_to eq(original_contents)
document = JSON.parse(rewritten_contents)
expect(document.fetch("version")).to eq("9.9.9-test")
end
it "does not rewrite when content is current and within the refresh interval" do
PotatoMesh::Application.refresh_well_known_document_if_stale
original_contents = File.binread(well_known_path)
PotatoMesh::Application.refresh_well_known_document_if_stale
expect(File.binread(well_known_path)).to eq(original_contents)
end
end
end
+13 -2
View File
@@ -30,13 +30,24 @@ RSpec.describe PotatoMesh::Sanitizer do
it "rejects invalid domains" do
expect(described_class.sanitize_instance_domain(nil)).to be_nil
expect(described_class.sanitize_instance_domain(" ")).to be_nil
expect(described_class.sanitize_instance_domain("example")).to be_nil
expect(described_class.sanitize_instance_domain("example.org/")).to be_nil
expect(described_class.sanitize_instance_domain("example .org")).to be_nil
expect(described_class.sanitize_instance_domain("mesh_instance.example")).to be_nil
expect(described_class.sanitize_instance_domain("example.org:70000")).to be_nil
expect(described_class.sanitize_instance_domain("[::1")).to be_nil
end
it "normalises valid domains" do
expect(described_class.sanitize_instance_domain(" Example.Org. ")).to eq("Example.Org")
expect(described_class.sanitize_instance_domain("[::1]")).to eq("[::1]")
expect(described_class.sanitize_instance_domain(" Example.Org. ")).to eq("example.org")
expect(described_class.sanitize_instance_domain("Example.Org:443")).to eq("example.org:443")
expect(described_class.sanitize_instance_domain("[2001:DB8::1]")).to eq("[2001:db8::1]")
expect(described_class.sanitize_instance_domain("127.0.0.1:8080")).to eq("127.0.0.1:8080")
end
it "preserves case when requested" do
expect(described_class.sanitize_instance_domain("Mesh.Example", downcase: false)).to eq("Mesh.Example")
expect(described_class.sanitize_instance_domain("[2001:DB8::1]", downcase: false)).to eq("[2001:DB8::1]")
end
end
+1
View File
@@ -34,6 +34,7 @@ require "tmpdir"
require "fileutils"
ENV["RACK_ENV"] = "test"
ENV["INSTANCE_DOMAIN"] ||= "spec.mesh.test"
SPEC_TMPDIR = Dir.mktmpdir("potato-mesh-spec-")
ENV["XDG_DATA_HOME"] = File.join(SPEC_TMPDIR, "xdg-data")