data: track ingestors heartbeat (#549 )

* data: track ingestors heartbeat * data: address review comments * cover missing unit test vectors * cover missing unit test vectors
Harden instance selector navigation URLs (#550 )
2026-05-09 14:55:08 +02:00 · 2025-12-14 18:42:17 +01:00 · 2025-12-14 18:40:41 +01:00 · 2025-12-14 16:47:44 +01:00 · 2025-12-14 15:05:19 +01:00 · 2025-12-14 14:35:55 +01:00
37 changed files with 1989 additions and 96 deletions
@@ -53,6 +53,7 @@ Additional environment variables are optional:
 | `MAP_ZOOM` | _unset_ | Fixed Leaflet zoom (disables the auto-fit checkbox when set). |
 | `MAX_DISTANCE` | `42` | Maximum relationship distance (km) before edges are hidden. |
 | `DEBUG` | `0` | Enables verbose logging across services when set to `1`. |
+| `HIDDEN_CHANNELS` | _unset_ | Comma-separated channel names the ingestor skips when forwarding packets. |
 | `FEDERATION` | `1` | Controls whether the instance announces itself and crawls peers (`1`) or stays isolated (`0`). |
 | `PRIVATE` | `0` | Restricts public visibility and disables chat/message endpoints when set to `1`. |
 | `CONNECTION` | `/dev/ttyACM0` | Serial device, TCP endpoint, or Bluetooth target used by the ingestor to reach the radio. |
@@ -83,6 +83,7 @@ The web app can be configured with environment variables (defaults shown):
 | `MAP_ZOOM` | _unset_ | Fixed Leaflet zoom applied on first load; disables auto-fit when provided. |
 | `MAX_DISTANCE` | `42` | Maximum distance (km) before node relationships are hidden on the map. |
 | `DEBUG` | `0` | Set to `1` for verbose logging in the web and ingestor services. |
+| `HIDDEN_CHANNELS` | _unset_ | Comma-separated channel names the ingestor will ignore when forwarding packets. |
 | `FEDERATION` | `1` | Set to `1` to announce your instance and crawl peers, or `0` to disable federation. Private mode overrides this. |
 | `PRIVATE` | `0` | Set to `1` to hide the chat UI, disable message APIs, and exclude hidden clients from public listings. |

@@ -191,7 +192,10 @@ an IP address (for example `192.168.1.20:4403`) to use the Meshtastic TCP
 interface. `CONNECTION` also accepts Bluetooth device addresses (e.g.,
 `ED:4D:9E:95:CF:60`) and the script attempts a BLE connection if available. The
 ingestor will still honor the legacy `POTATOMESH_INSTANCE` variable when
-`INSTANCE_DOMAIN` is unset to ease upgrades from earlier deployments.
+`INSTANCE_DOMAIN` is unset to ease upgrades from earlier deployments. To keep
+private channels out of the web UI, set `HIDDEN_CHANNELS` to a comma-separated
+list of channel names (for example `HIDDEN_CHANNELS="Secret,Ops"`); packets on
+those channels are discarded instead of being sent to `/api/messages`.

 ## Docker

@@ -15,11 +15,11 @@
  <key>CFBundlePackageType</key>
  <string>FMWK</string>
  <key>CFBundleShortVersionString</key>
-  <string>1.0</string>
+  <string>0.5.7</string>
  <key>CFBundleSignature</key>
  <string>????</string>
  <key>CFBundleVersion</key>
-  <string>1.0</string>
+  <string>0.5.7</string>
  <key>MinimumOSVersion</key>
  <string>14.0</string>
 </dict>
@@ -1,7 +1,7 @@
 name: potato_mesh_reader
 description: Meshtastic Reader — read-only view for PotatoMesh messages.
 publish_to: "none"
-version: 0.5.6
+version: 0.5.7

 environment:
  sdk: ">=3.4.0 <4.0.0"
@@ -76,6 +76,7 @@ CHANNEL=$(grep "^CHANNEL=" .env 2>/dev/null | cut -d'=' -f2- | tr -d '"' || echo
 FREQUENCY=$(grep "^FREQUENCY=" .env 2>/dev/null | cut -d'=' -f2- | tr -d '"' || echo "915MHz")
 FEDERATION=$(grep "^FEDERATION=" .env 2>/dev/null | cut -d'=' -f2- | tr -d '"' || echo "1")
 PRIVATE=$(grep "^PRIVATE=" .env 2>/dev/null | cut -d'=' -f2- | tr -d '"' || echo "0")
+HIDDEN_CHANNELS=$(grep "^HIDDEN_CHANNELS=" .env 2>/dev/null | cut -d'=' -f2- | tr -d '"' || echo "")
 MAP_CENTER=$(grep "^MAP_CENTER=" .env 2>/dev/null | cut -d'=' -f2- | tr -d '"' || echo "38.761944,-27.090833")
 MAP_ZOOM=$(grep "^MAP_ZOOM=" .env 2>/dev/null | cut -d'=' -f2- | tr -d '"' || echo "")
 MAX_DISTANCE=$(grep "^MAX_DISTANCE=" .env 2>/dev/null | cut -d'=' -f2- | tr -d '"' || echo "42")
@@ -126,6 +127,8 @@ echo "-------------------"
 echo "Private mode hides public mesh messages from unauthenticated visitors."
 echo "Set to 1 to hide public feeds or 0 to keep them visible."
 read_with_default "Enable private mode (1=yes, 0=no)" "$PRIVATE" PRIVATE
+echo "Provide a comma-separated list of channel names to hide from the web UI (optional)."
+read_with_default "Hidden channels" "$HIDDEN_CHANNELS" HIDDEN_CHANNELS

 echo ""
 echo "🛠 Docker Settings"
@@ -196,6 +199,11 @@ update_env "POTATOMESH_IMAGE_TAG" "$POTATOMESH_IMAGE_TAG"
 update_env "FEDERATION" "$FEDERATION"
 update_env "PRIVATE" "$PRIVATE"
 update_env "CONNECTION" "$CONNECTION"
+if [ -n "$HIDDEN_CHANNELS" ]; then
+    update_env "HIDDEN_CHANNELS" "\"$HIDDEN_CHANNELS\""
+else
+    sed -i.bak '/^HIDDEN_CHANNELS=.*/d' .env
+fi
 if [ -n "$INSTANCE_DOMAIN" ]; then
    update_env "INSTANCE_DOMAIN" "$INSTANCE_DOMAIN"
 else
@@ -244,6 +252,7 @@ echo "   API Token: ${API_TOKEN:0:8}..."
 echo "   Docker Image Arch: $POTATOMESH_IMAGE_ARCH"
 echo "   Docker Image Tag: $POTATOMESH_IMAGE_TAG"
 echo "   Private Mode: ${PRIVATE}"
+echo "   Hidden Channels: ${HIDDEN_CHANNELS:-'None'}"
 echo "   Instance Domain: ${INSTANCE_DOMAIN:-'Auto-detected'}"
 if [ "${FEDERATION:-1}" = "0" ]; then
    echo "   Federation: Disabled"
@@ -18,7 +18,7 @@ The ``data.mesh`` module exposes helpers for reading Meshtastic node and
 message information before forwarding it to the accompanying web application.
 """

-VERSION = "0.5.6"
+VERSION = "0.5.7"
 """Semantic version identifier shared with the dashboard and front-end."""

 __version__ = VERSION
@@ -0,0 +1,26 @@
+-- Copyright © 2025-26 l5yth & contributors
+--
+-- Licensed under the Apache License, Version 2.0 (the "License");
+-- you may not use this file except in compliance with the License.
+-- You may obtain a copy of the License at
+--
+--     http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+
+PRAGMA journal_mode=WAL;
+
+CREATE TABLE IF NOT EXISTS ingestors (
+  node_id        TEXT PRIMARY KEY,
+  start_time     INTEGER NOT NULL,
+  last_seen_time INTEGER NOT NULL,
+  version        TEXT,
+  lora_freq      INTEGER,
+  modem_preset   TEXT
+);
+
+CREATE INDEX IF NOT EXISTS idx_ingestors_last_seen ON ingestors(last_seen_time);
@@ -26,6 +26,7 @@ CREATE TABLE IF NOT EXISTS instances (
  longitude         REAL,
  last_update_time  INTEGER,
  is_private        BOOLEAN NOT NULL DEFAULT 0,
+  nodes_count       INTEGER,
  contact_link      TEXT,
  signature         TEXT
 );
@@ -21,7 +21,17 @@ import threading as threading  # re-exported for compatibility
 import sys
 import types

-from . import channels, config, daemon, handlers, interfaces, queue, serialization
+from .. import VERSION as _PACKAGE_VERSION
+from . import (
+    channels,
+    config,
+    daemon,
+    handlers,
+    ingestors,
+    interfaces,
+    queue,
+    serialization,
+)

 __all__: list[str] = []

@@ -40,7 +50,15 @@ def _export_constants() -> None:
    __all__.extend(["json", "urllib", "glob", "threading", "signal"])


-for _module in (channels, daemon, handlers, interfaces, queue, serialization):
+for _module in (
+    channels,
+    daemon,
+    handlers,
+    interfaces,
+    queue,
+    serialization,
+    ingestors,
+):
    _reexport(_module)

 _export_constants()
@@ -52,11 +70,13 @@ _CONFIG_ATTRS = {
    "DEBUG",
    "INSTANCE",
    "API_TOKEN",
+    "HIDDEN_CHANNELS",
    "LORA_FREQ",
    "MODEM_PRESET",
    "_RECONNECT_INITIAL_DELAY_SECS",
    "_RECONNECT_MAX_DELAY_SECS",
    "_CLOSE_TIMEOUT_SECS",
+    "_INGESTOR_HEARTBEAT_SECS",
    "_debug_log",
 }

@@ -70,9 +90,16 @@ _HANDLER_ATTRS = set(handlers.__all__)
 _DAEMON_ATTRS = set(daemon.__all__)
 _SERIALIZATION_ATTRS = set(serialization.__all__)
 _INTERFACE_EXPORTS = set(interfaces.__all__)
+_INGESTOR_ATTRS = set(ingestors.__all__)
+
+# Re-export the package version for callers that previously referenced
+# data.mesh_ingestor.VERSION directly.
+VERSION = _PACKAGE_VERSION
+__all__.append("VERSION")

 __all__.extend(sorted(_CONFIG_ATTRS))
 __all__.extend(sorted(_INTERFACE_ATTRS))
+__all__.append("VERSION")


 class _MeshIngestorModule(types.ModuleType):
@@ -87,6 +114,10 @@ class _MeshIngestorModule(types.ModuleType):
            return getattr(interfaces, name)
        if name in _INTERFACE_EXPORTS:
            return getattr(interfaces, name)
+        if name in _INGESTOR_ATTRS:
+            return getattr(ingestors, name)
+        if name == "VERSION":
+            return VERSION
        raise AttributeError(name)

    def __setattr__(self, name: str, value):  # type: ignore[override]
@@ -121,6 +152,10 @@ class _MeshIngestorModule(types.ModuleType):
            setattr(serialization, name, value)
            super().__setattr__(name, getattr(serialization, name, value))
            handled = True
+        if name in _INGESTOR_ATTRS:
+            setattr(ingestors, name, value)
+            super().__setattr__(name, getattr(ingestors, name, value))
+            handled = True
        if handled:
            return
        super().__setattr__(name, value)
@@ -222,6 +222,27 @@ def channel_name(channel_index: int | None) -> str | None:
    return _CHANNEL_LOOKUP.get(int(channel_index))


+def hidden_channel_names() -> tuple[str, ...]:
+    """Return the configured set of hidden channel names."""
+
+    return tuple(getattr(config, "HIDDEN_CHANNELS", ()))
+
+
+def is_hidden_channel(channel_name_value: str | None) -> bool:
+    """Return ``True`` when ``channel_name_value`` is configured as hidden."""
+
+    if channel_name_value is None:
+        return False
+    normalized = channel_name_value.strip()
+    if not normalized:
+        return False
+    normalized_casefold = normalized.casefold()
+    for hidden in getattr(config, "HIDDEN_CHANNELS", ()):
+        if normalized_casefold == hidden.casefold():
+            return True
+    return False
+
+
 def _reset_channel_cache() -> None:
    """Clear cached channel data. Intended for use in tests only."""

@@ -234,5 +255,7 @@ __all__ = [
    "capture_from_interface",
    "channel_mappings",
    "channel_name",
+    "hidden_channel_names",
+    "is_hidden_channel",
    "_reset_channel_cache",
 ]
@@ -46,6 +46,9 @@ DEFAULT_ENERGY_ONLINE_DURATION_SECS = 300.0
 DEFAULT_ENERGY_SLEEP_SECS = float(6 * 60 * 60)
 """Sleep duration used when energy saving mode is active."""

+DEFAULT_INGESTOR_HEARTBEAT_SECS = float(60 * 60)
+"""Interval between ingestor heartbeat announcements."""
+
 CONNECTION = os.environ.get("CONNECTION") or os.environ.get("MESH_SERIAL")
 """Optional connection target for the mesh interface.

@@ -63,6 +66,40 @@ CHANNEL_INDEX = int(os.environ.get("CHANNEL_INDEX", str(DEFAULT_CHANNEL_INDEX)))
 DEBUG = os.environ.get("DEBUG") == "1"


+def _parse_hidden_channels(raw_value: str | None) -> tuple[str, ...]:
+    """Normalise a comma-separated list of hidden channel names.
+
+    Parameters:
+        raw_value: Raw environment string containing channel names separated by
+            commas. ``None`` and empty segments are ignored.
+
+    Returns:
+        A tuple of unique, non-empty channel names preserving input order while
+        deduplicating case-insensitively.
+    """
+
+    if not raw_value:
+        return ()
+
+    normalized_entries: list[str] = []
+    seen: set[str] = set()
+    for part in raw_value.split(","):
+        name = part.strip()
+        if not name:
+            continue
+        key = name.casefold()
+        if key in seen:
+            continue
+        seen.add(key)
+        normalized_entries.append(name)
+
+    return tuple(normalized_entries)
+
+
+HIDDEN_CHANNELS = _parse_hidden_channels(os.environ.get("HIDDEN_CHANNELS"))
+"""Channel names configured to be ignored by the ingestor."""
+
+
 def _resolve_instance_domain() -> str:
    """Resolve the configured instance domain from the environment.

@@ -100,6 +137,7 @@ _CLOSE_TIMEOUT_SECS = DEFAULT_CLOSE_TIMEOUT_SECS
 _INACTIVITY_RECONNECT_SECS = DEFAULT_INACTIVITY_RECONNECT_SECS
 _ENERGY_ONLINE_DURATION_SECS = DEFAULT_ENERGY_ONLINE_DURATION_SECS
 _ENERGY_SLEEP_SECS = DEFAULT_ENERGY_SLEEP_SECS
+_INGESTOR_HEARTBEAT_SECS = DEFAULT_INGESTOR_HEARTBEAT_SECS

 # Backwards compatibility shim for legacy imports.
 PORT = CONNECTION
@@ -144,6 +182,7 @@ __all__ = [
    "SNAPSHOT_SECS",
    "CHANNEL_INDEX",
    "DEBUG",
+    "HIDDEN_CHANNELS",
    "INSTANCE",
    "API_TOKEN",
    "ENERGY_SAVING",
@@ -155,6 +194,7 @@ __all__ = [
    "_INACTIVITY_RECONNECT_SECS",
    "_ENERGY_ONLINE_DURATION_SECS",
    "_ENERGY_SLEEP_SECS",
+    "_INGESTOR_HEARTBEAT_SECS",
    "_debug_log",
 ]

@@ -23,7 +23,7 @@ import time

 from pubsub import pub

-from . import config, handlers, interfaces
+from . import config, handlers, ingestors, interfaces

 _RECEIVE_TOPICS = (
    "meshtastic.receive",
@@ -169,6 +169,41 @@ def _is_ble_interface(iface_obj) -> bool:
    return "ble_interface" in module_name


+def _process_ingestor_heartbeat(iface, *, ingestor_announcement_sent: bool) -> bool:
+    """Send ingestor liveness heartbeats when a host id is known.
+
+    Parameters:
+        iface: Active mesh interface used to extract a host node id when absent.
+        ingestor_announcement_sent: Whether an initial heartbeat has already
+            been sent during the current session.
+
+    Returns:
+        Updated ``ingestor_announcement_sent`` flag reflecting whether an
+        initial heartbeat was transmitted.
+    """
+
+    host_id = handlers.host_node_id()
+    if host_id is None and iface is not None:
+        extracted = interfaces._extract_host_node_id(iface)
+        if extracted:
+            handlers.register_host_node_id(extracted)
+            host_id = handlers.host_node_id()
+
+    if host_id:
+        ingestors.set_ingestor_node_id(host_id)
+    heartbeat_sent = ingestors.queue_ingestor_heartbeat(
+        force=not ingestor_announcement_sent
+    )
+    if heartbeat_sent and not ingestor_announcement_sent:
+        return True
+    return ingestor_announcement_sent
+    iface_cls = getattr(iface_obj, "__class__", None)
+    if iface_cls is None:
+        return False
+    module_name = getattr(iface_cls, "__module__", "") or ""
+    return "ble_interface" in module_name
+
+
 def _connected_state(candidate) -> bool | None:
    """Return the connection state advertised by ``candidate``.

@@ -233,6 +268,7 @@ def main(existing_interface=None) -> None:
    inactivity_reconnect_secs = max(
        0.0, getattr(config, "_INACTIVITY_RECONNECT_SECS", 0.0)
    )
+    ingestor_announcement_sent = False

    energy_saving_enabled = config.ENERGY_SAVING
    energy_online_secs = max(0.0, config._ENERGY_ONLINE_DURATION_SECS)
@@ -288,6 +324,7 @@ def main(existing_interface=None) -> None:
                    handlers.register_host_node_id(
                        interfaces._extract_host_node_id(iface)
                    )
+                    ingestors.set_ingestor_node_id(handlers.host_node_id())
                    retry_delay = max(0.0, config._RECONNECT_INITIAL_DELAY_SECS)
                    initial_snapshot_sent = False
                    if not announced_target and resolved_target:
@@ -501,6 +538,10 @@ def main(existing_interface=None) -> None:
                        iface_connected_at = None
                        continue

+            ingestor_announcement_sent = _process_ingestor_heartbeat(
+                iface, ingestor_announcement_sent=ingestor_announcement_sent
+            )
+
            retry_delay = max(0.0, config._RECONNECT_INITIAL_DELAY_SECS)
            stop.wait(config.SNAPSHOT_SECS)
    except KeyboardInterrupt:  # pragma: no cover - interactive only
@@ -520,6 +561,7 @@ __all__ = [
    "_node_items_snapshot",
    "_subscribe_receive_topics",
    "_is_ble_interface",
+    "_process_ingestor_heartbeat",
    "_connected_state",
    "main",
 ]
@@ -1414,6 +1414,8 @@ def store_packet_dict(packet: Mapping) -> None:
    except Exception:
        channel = 0

+    channel_name_value = channels.channel_name(channel)
+
    pkt_id = _first(packet, "id", "packet_id", "packetId", default=None)
    if pkt_id is None:
        _record_ignored_packet(packet, reason="missing-packet-id")
@@ -1459,6 +1461,17 @@ def store_packet_dict(packet: Mapping) -> None:
        _record_ignored_packet(packet, reason="skipped-direct-message")
        return

+    if channels.is_hidden_channel(channel_name_value):
+        _record_ignored_packet(packet, reason="hidden-channel")
+        if config.DEBUG:
+            config._debug_log(
+                "Ignored packet on hidden channel",
+                context="handlers.store_packet_dict",
+                channel=channel,
+                channel_name=channel_name_value,
+            )
+        return
+
    message_payload = {
        "id": int(pkt_id),
        "rx_time": rx_time,
@@ -1476,11 +1489,8 @@ def store_packet_dict(packet: Mapping) -> None:
        "emoji": emoji,
    }

-    channel_name_value = None
-    if not encrypted_flag:
-        channel_name_value = channels.channel_name(channel)
-        if channel_name_value:
-            message_payload["channel_name"] = channel_name_value
+    if not encrypted_flag and channel_name_value:
+        message_payload["channel_name"] = channel_name_value
    _queue_post_json(
        "/api/messages",
        _apply_radio_metadata(message_payload),
@@ -0,0 +1,139 @@
+# Copyright © 2025-26 l5yth & contributors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Helpers for tracking ingestor identity and liveness announcements."""
+
+from __future__ import annotations
+
+import time
+from dataclasses import dataclass, field
+from typing import Callable
+
+from .. import VERSION as INGESTOR_VERSION
+from . import config, queue
+from .serialization import _canonical_node_id
+
+HEARTBEAT_INTERVAL_SECS = 60 * 60
+"""Default interval between ingestor heartbeat announcements."""
+
+
+@dataclass
+class _IngestorState:
+    """Mutable ingestor identity and heartbeat tracking data."""
+
+    start_time: int = field(default_factory=lambda: int(time.time()))
+    last_heartbeat: int | None = None
+    node_id: str | None = None
+
+
+STATE = _IngestorState()
+"""Shared ingestor identity state."""
+# Alias retained for clarity without exporting into the top-level mesh module to
+# avoid colliding with the HTTP queue state.
+INGESTOR_STATE = STATE
+
+
+def ingestor_start_time() -> int:
+    """Return the unix timestamp representing when the ingestor booted."""
+
+    return STATE.start_time
+
+
+def set_ingestor_node_id(node_id: str | None) -> str | None:
+    """Record the canonical host node identifier for the ingestor.
+
+    Parameters:
+        node_id: Raw node identifier reported by the connected device.
+
+    Returns:
+        Canonical node identifier in ``!xxxxxxxx`` form or ``None`` when the
+        provided value cannot be normalised.
+    """
+
+    canonical = _canonical_node_id(node_id)
+    if canonical is None:
+        return None
+
+    if STATE.node_id != canonical:
+        STATE.node_id = canonical
+        STATE.last_heartbeat = None
+
+    return canonical
+
+
+def queue_ingestor_heartbeat(
+    *,
+    force: bool = False,
+    send: Callable[[str, dict], None] | None = None,
+    node_id: str | None = None,
+) -> bool:
+    """Queue a heartbeat payload advertising ingestor liveness.
+
+    Parameters:
+        force: When ``True``, bypasses the heartbeat interval guard so an
+            announcement is queued immediately.
+        send: Optional transport callable used for tests; defaults to the queue
+            dispatcher.
+        node_id: Optional node identifier to register before sending. When
+            omitted the previously recorded identifier is reused.
+
+    Returns:
+        ``True`` when a heartbeat payload was queued, ``False`` otherwise.
+    """
+
+    canonical = _canonical_node_id(node_id) if node_id is not None else None
+    if canonical:
+        set_ingestor_node_id(canonical)
+    canonical = STATE.node_id
+
+    if canonical is None:
+        return False
+
+    now = int(time.time())
+    interval = max(
+        0, int(getattr(config, "_INGESTOR_HEARTBEAT_SECS", HEARTBEAT_INTERVAL_SECS))
+    )
+    last = STATE.last_heartbeat
+    if not force and last is not None and now - last < interval:
+        return False
+
+    payload = {
+        "node_id": canonical,
+        "start_time": STATE.start_time,
+        "last_seen_time": now,
+        "version": INGESTOR_VERSION,
+    }
+    if getattr(config, "LORA_FREQ", None) is not None:
+        payload["lora_freq"] = config.LORA_FREQ
+    if getattr(config, "MODEM_PRESET", None) is not None:
+        payload["modem_preset"] = config.MODEM_PRESET
+    queue._queue_post_json(
+        "/api/ingestors",
+        payload,
+        priority=getattr(
+            queue, "_INGESTOR_POST_PRIORITY", queue._DEFAULT_POST_PRIORITY
+        ),
+        send=send,
+    )
+    STATE.last_heartbeat = now
+    return True
+
+
+__all__ = [
+    "HEARTBEAT_INTERVAL_SECS",
+    "INGESTOR_STATE",
+    "ingestor_start_time",
+    "queue_ingestor_heartbeat",
+    "set_ingestor_node_id",
+]
@@ -74,6 +74,7 @@ def _payload_key_value_pairs(payload: Mapping[str, object]) -> str:


 _MESSAGE_POST_PRIORITY = 10
+_INGESTOR_POST_PRIORITY = 80
 _NEIGHBOR_POST_PRIORITY = 20
 _TRACE_POST_PRIORITY = 25
 _POSITION_POST_PRIORITY = 30
@@ -259,6 +260,7 @@ __all__ = [
    "QueueState",
    "_DEFAULT_POST_PRIORITY",
    "_MESSAGE_POST_PRIORITY",
+    "_INGESTOR_POST_PRIORITY",
    "_NEIGHBOR_POST_PRIORITY",
    "_NODE_POST_PRIORITY",
    "_POSITION_POST_PRIORITY",
@@ -49,6 +49,7 @@ x-ingestor-base: &ingestor-base
  environment:
    CONNECTION: ${CONNECTION:-/dev/ttyACM0}
    CHANNEL_INDEX: ${CHANNEL_INDEX:-0}
+    HIDDEN_CHANNELS: ${HIDDEN_CHANNELS:-""}
    API_TOKEN: ${API_TOKEN}
    INSTANCE_DOMAIN: ${INSTANCE_DOMAIN}
    POTATOMESH_INSTANCE: ${POTATOMESH_INSTANCE:-http://web:41447}
@@ -20,6 +20,7 @@ import re
 import sys
 import threading
 import types
+import time

 """End-to-end tests covering the mesh ingestion package."""

@@ -214,6 +215,9 @@ def mesh_module(monkeypatch):
        if attr in module.__dict__:
            delattr(module, attr)
    module.channels._reset_channel_cache()
+    module.ingestors.STATE.start_time = int(time.time())
+    module.ingestors.STATE.last_heartbeat = None
+    module.ingestors.STATE.node_id = None

    yield module

@@ -281,6 +285,25 @@ def test_instance_domain_infers_scheme_for_hostnames(mesh_module, monkeypatch):
        mesh_module.INSTANCE = mesh_module.config.INSTANCE


+def test_parse_hidden_channels_deduplicates_names(mesh_module):
+    """Ensure hidden channel parsing strips blanks and deduplicates."""
+
+    mesh = mesh_module
+    previous_hidden = mesh.HIDDEN_CHANNELS
+
+    try:
+        parsed = mesh.config._parse_hidden_channels(" Chat , ,Secret ,chat")
+        mesh.HIDDEN_CHANNELS = parsed
+
+        assert parsed == ("Chat", "Secret")
+        assert mesh.channels.hidden_channel_names() == ("Chat", "Secret")
+        assert mesh.channels.is_hidden_channel(" chat ")
+        assert not mesh.channels.is_hidden_channel("unknown")
+        assert mesh.config._parse_hidden_channels("") == ()
+    finally:
+        mesh.HIDDEN_CHANNELS = previous_hidden
+
+
 def test_subscribe_receive_topics_covers_all_handlers(mesh_module, monkeypatch):
    mesh = mesh_module
    daemon_mod = sys.modules["data.mesh_ingestor.daemon"]
@@ -1932,6 +1955,73 @@ def test_store_packet_dict_appends_channel_name(mesh_module, monkeypatch, capsys
    assert "channel_display='Chat'" in log_output


+def test_store_packet_dict_skips_hidden_channel(mesh_module, monkeypatch, capsys):
+    mesh = mesh_module
+    mesh.channels._reset_channel_cache()
+    mesh.config.MODEM_PRESET = None
+
+    class DummyInterface:
+        def __init__(self) -> None:
+            self.localNode = SimpleNamespace(
+                channels=[
+                    SimpleNamespace(
+                        role=1,
+                        settings=SimpleNamespace(name="Primary"),
+                    ),
+                    SimpleNamespace(
+                        role=2,
+                        index=5,
+                        settings=SimpleNamespace(name="Chat"),
+                    ),
+                ]
+            )
+
+        def waitForConfig(self):
+            return None
+
+    mesh.channels.capture_from_interface(DummyInterface())
+    capsys.readouterr()
+
+    captured: list[tuple[str, dict, int]] = []
+    ignored: list[str] = []
+    monkeypatch.setattr(
+        mesh,
+        "_queue_post_json",
+        lambda path, payload, *, priority: captured.append((path, payload, priority)),
+    )
+    monkeypatch.setattr(
+        mesh.handlers,
+        "_record_ignored_packet",
+        lambda packet, *, reason: ignored.append(reason),
+    )
+
+    previous_debug = mesh.config.DEBUG
+    previous_hidden = mesh.HIDDEN_CHANNELS
+    mesh.config.DEBUG = True
+    mesh.DEBUG = True
+    mesh.HIDDEN_CHANNELS = ("Chat",)
+
+    try:
+        packet = {
+            "id": "999",
+            "rxTime": 24_680,
+            "from": "!sender",
+            "to": "^all",
+            "channel": 5,
+            "decoded": {"text": "hidden msg", "portnum": 1},
+        }
+
+        mesh.store_packet_dict(packet)
+
+        assert captured == []
+        assert ignored == ["hidden-channel"]
+        assert "Ignored packet on hidden channel" in capsys.readouterr().out
+    finally:
+        mesh.HIDDEN_CHANNELS = previous_hidden
+        mesh.config.DEBUG = previous_debug
+        mesh.DEBUG = previous_debug
+
+
 def test_store_packet_dict_includes_encrypted_payload(mesh_module, monkeypatch):
    mesh = mesh_module
    captured = []
@@ -2575,6 +2665,133 @@ def test_queue_post_json_skips_when_active(mesh_module, monkeypatch):
    mesh._clear_post_queue()


+def test_process_ingestor_heartbeat_updates_flag(mesh_module, monkeypatch):
+    mesh = mesh_module
+    mesh.ingestors.STATE.last_heartbeat = None
+    mesh.ingestors.STATE.node_id = None
+    mesh.handlers.register_host_node_id(None)
+    recorded = {"force": None, "count": 0}
+
+    def fake_queue_ingestor_heartbeat(*, force):
+        recorded["force"] = force
+        recorded["count"] += 1
+        return True
+
+    monkeypatch.setattr(
+        mesh.ingestors, "queue_ingestor_heartbeat", fake_queue_ingestor_heartbeat
+    )
+
+    class DummyIface:
+        def __init__(self):
+            self.myNodeNum = 0xCAFEBABE
+
+    updated = mesh._process_ingestor_heartbeat(
+        DummyIface(), ingestor_announcement_sent=False
+    )
+
+    assert updated is True
+    assert recorded["force"] is True
+    assert recorded["count"] == 1
+    assert mesh.handlers.host_node_id() == "!cafebabe"
+
+
+def test_process_ingestor_heartbeat_skips_without_host(mesh_module, monkeypatch):
+    mesh = mesh_module
+    mesh.handlers.register_host_node_id(None)
+    mesh.ingestors.STATE.node_id = None
+    mesh.ingestors.STATE.last_heartbeat = None
+
+    monkeypatch.setattr(mesh.ingestors, "queue_ingestor_heartbeat", lambda **_: False)
+
+    updated = mesh._process_ingestor_heartbeat(None, ingestor_announcement_sent=False)
+
+    assert updated is False
+    assert mesh.ingestors.STATE.node_id is None
+    assert mesh.ingestors.STATE.last_heartbeat is None
+
+
+def test_ingestor_heartbeat_respects_interval_override(mesh_module, monkeypatch):
+    mesh = mesh_module
+    mesh.ingestors.STATE.start_time = 100
+    mesh.ingestors.STATE.last_heartbeat = 1_000
+    mesh.ingestors.STATE.node_id = "!abcd0001"
+    mesh._INGESTOR_HEARTBEAT_SECS = 10_000
+    monkeypatch.setattr(mesh.ingestors.time, "time", lambda: 2_000)
+    sent = mesh.ingestors.queue_ingestor_heartbeat()
+    assert sent is False
+    assert mesh.ingestors.STATE.last_heartbeat == 1_000
+
+
+def test_setting_ingestor_attr_propagates(mesh_module):
+    mesh = mesh_module
+    mesh._INGESTOR_HEARTBEAT_SECS = 123
+    assert mesh.config._INGESTOR_HEARTBEAT_SECS == 123
+
+
+def test_queue_ingestor_heartbeat_requires_node_id(mesh_module, monkeypatch):
+    mesh = mesh_module
+    captured = []
+
+    monkeypatch.setattr(
+        mesh.queue,
+        "_queue_post_json",
+        lambda path, payload, *, priority, send=None: captured.append(
+            (path, payload, priority)
+        ),
+    )
+
+    mesh.ingestors.STATE.node_id = None
+    mesh.ingestors.STATE.last_heartbeat = None
+
+    queued = mesh.ingestors.queue_ingestor_heartbeat(force=True)
+
+    assert queued is False
+    assert captured == []
+
+
+def test_queue_ingestor_heartbeat_enqueues_and_throttles(mesh_module, monkeypatch):
+    mesh = mesh_module
+    captured = []
+
+    monkeypatch.setattr(
+        mesh.queue,
+        "_queue_post_json",
+        lambda path, payload, *, priority, send=None: captured.append(
+            (path, payload, priority)
+        ),
+    )
+
+    mesh.ingestors.STATE.start_time = 1_700_000_000
+    mesh.ingestors.STATE.last_heartbeat = None
+    mesh.ingestors.STATE.node_id = None
+    mesh.config.LORA_FREQ = 915
+    mesh.config.MODEM_PRESET = "LongFast"
+
+    mesh.ingestors.set_ingestor_node_id("!CAFEBABE")
+    first = mesh.ingestors.queue_ingestor_heartbeat(force=True)
+    second = mesh.ingestors.queue_ingestor_heartbeat()
+
+    assert first is True
+    assert second is False
+    assert len(captured) == 1
+    path, payload, priority = captured[0]
+    assert path == "/api/ingestors"
+    assert payload["node_id"] == "!cafebabe"
+    assert payload["start_time"] == 1_700_000_000
+    assert payload["last_seen_time"] >= payload["start_time"]
+    assert payload["version"] == mesh.VERSION
+    assert payload["lora_freq"] == 915
+    assert payload["modem_preset"] == "LongFast"
+    assert priority == mesh.queue._INGESTOR_POST_PRIORITY
+
+
+def test_mesh_version_export_matches_package(mesh_module):
+    import data
+
+    mesh = mesh_module
+    assert mesh.VERSION == data.VERSION
+
+
 def test_node_to_dict_handles_proto_fallback(mesh_module, monkeypatch):
    mesh = mesh_module

@@ -199,6 +199,66 @@ module PotatoMesh
        updated
      end

+      # Insert or update an ingestor heartbeat payload.
+      #
+      # @param db [SQLite3::Database] open database handle.
+      # @param payload [Hash] ingestor payload from the collector.
+      # @return [Boolean] true when persistence succeeded.
+      def upsert_ingestor(db, payload)
+        return false unless payload.is_a?(Hash)
+
+        parts = canonical_node_parts(payload["node_id"] || payload["id"])
+        return false unless parts
+
+        node_id, = parts
+        now = Time.now.to_i
+
+        start_time = coerce_integer(payload["start_time"] || payload["startTime"]) || now
+        last_seen_time =
+          coerce_integer(payload["last_seen_time"] || payload["lastSeenTime"]) || start_time
+
+        start_time = 0 if start_time.negative?
+        last_seen_time = 0 if last_seen_time.negative?
+        start_time = now if start_time > now
+        last_seen_time = now if last_seen_time > now
+        last_seen_time = start_time if last_seen_time < start_time
+
+        version = string_or_nil(payload["version"] || payload["ingestorVersion"])
+        return false unless version
+        lora_freq = coerce_integer(payload["lora_freq"])
+        modem_preset = string_or_nil(payload["modem_preset"])
+
+        with_busy_retry do
+          db.execute <<~SQL, [node_id, start_time, last_seen_time, version, lora_freq, modem_preset]
+                       INSERT INTO ingestors(node_id, start_time, last_seen_time, version, lora_freq, modem_preset)
+                            VALUES(?,?,?,?,?,?)
+                       ON CONFLICT(node_id) DO UPDATE SET
+                         start_time = CASE
+                           WHEN excluded.start_time > ingestors.start_time THEN excluded.start_time
+                           ELSE ingestors.start_time
+                         END,
+                         last_seen_time = CASE
+                           WHEN excluded.last_seen_time > ingestors.last_seen_time THEN excluded.last_seen_time
+                           ELSE ingestors.last_seen_time
+                         END,
+                         version = COALESCE(excluded.version, ingestors.version),
+                         lora_freq = COALESCE(excluded.lora_freq, ingestors.lora_freq),
+                         modem_preset = COALESCE(excluded.modem_preset, ingestors.modem_preset)
+                     SQL
+        end
+
+        true
+      rescue SQLite3::SQLException => e
+        warn_log(
+          "Failed to upsert ingestor record",
+          context: "data_processing.ingestors",
+          node_id: node_id,
+          error_class: e.class.name,
+          error_message: e.message,
+        )
+        false
+      end
+
      def upsert_node(db, node_id, n)
        user = n["user"] || {}
        met = n["deviceMetrics"] || {}
@@ -1262,7 +1322,7 @@ module PotatoMesh
        rx_time = now if rx_time.nil? || rx_time > now
        rx_iso = string_or_nil(payload["rx_iso"]) || Time.at(rx_time).utc.iso8601

-        metrics = normalize_json_object(payload["metrics"])
+        metrics = normalize_json_object(payload["metrics"]) || {}
        src = coerce_integer(payload["src"] || payload["source"] || payload["from"])
        dest = coerce_integer(payload["dest"] || payload["destination"] || payload["to"])
        rssi = coerce_integer(payload["rssi"]) || coerce_integer(metrics["rssi"])
@@ -81,10 +81,10 @@ module PotatoMesh
        return false unless File.exist?(PotatoMesh::Config.db_path)

        db = open_database(readonly: true)
-        required = %w[nodes messages positions telemetry neighbors instances traces trace_hops]
+        required = %w[nodes messages positions telemetry neighbors instances traces trace_hops ingestors]
        tables =
          db.execute(
-            "SELECT name FROM sqlite_master WHERE type='table' AND name IN ('nodes','messages','positions','telemetry','neighbors','instances','traces','trace_hops')",
+            "SELECT name FROM sqlite_master WHERE type='table' AND name IN ('nodes','messages','positions','telemetry','neighbors','instances','traces','trace_hops','ingestors')",
          ).flatten
        (required - tables).empty?
      rescue SQLite3::Exception
@@ -99,7 +99,7 @@ module PotatoMesh
      def init_db
        FileUtils.mkdir_p(File.dirname(PotatoMesh::Config.db_path))
        db = open_database
-        %w[nodes messages positions telemetry neighbors instances traces].each do |schema|
+        %w[nodes messages positions telemetry neighbors instances traces ingestors].each do |schema|
          sql_file = File.expand_path("../../../../data/#{schema}.sql", __dir__)
          db.execute_batch(File.read(sql_file))
        end
@@ -167,6 +167,11 @@ module PotatoMesh
        instance_columns = db.execute("PRAGMA table_info(instances)").map { |row| row[1] }
        unless instance_columns.include?("contact_link")
          db.execute("ALTER TABLE instances ADD COLUMN contact_link TEXT")
+          instance_columns << "contact_link"
+        end
+
+        unless instance_columns.include?("nodes_count")
+          db.execute("ALTER TABLE instances ADD COLUMN nodes_count INTEGER")
        end

        telemetry_tables =
@@ -192,6 +197,24 @@ module PotatoMesh
          traces_schema = File.expand_path("../../../../data/traces.sql", __dir__)
          db.execute_batch(File.read(traces_schema))
        end
+
+        ingestor_tables =
+          db.execute("SELECT name FROM sqlite_master WHERE type='table' AND name='ingestors'").flatten
+        if ingestor_tables.empty?
+          ingestors_schema = File.expand_path("../../../../data/ingestors.sql", __dir__)
+          db.execute_batch(File.read(ingestors_schema))
+        else
+          ingestor_columns = db.execute("PRAGMA table_info(ingestors)").map { |row| row[1] }
+          unless ingestor_columns.include?("version")
+            db.execute("ALTER TABLE ingestors ADD COLUMN version TEXT")
+          end
+          unless ingestor_columns.include?("lora_freq")
+            db.execute("ALTER TABLE ingestors ADD COLUMN lora_freq INTEGER")
+          end
+          unless ingestor_columns.include?("modem_preset")
+            db.execute("ALTER TABLE ingestors ADD COLUMN modem_preset TEXT")
+          end
+        end
      rescue SQLite3::SQLException, Errno::ENOENT => e
        warn_log(
          "Failed to apply schema upgrade",
@@ -61,6 +61,7 @@ module PotatoMesh
      def self_instance_attributes
        domain = self_instance_domain
        last_update = latest_node_update_timestamp || Time.now.to_i
+        nodes_count = active_node_count_since(Time.now.to_i - PotatoMesh::Config.remote_instance_max_node_age)
        {
          id: app_constant(:SELF_INSTANCE_ID),
          domain: domain,
@@ -74,9 +75,36 @@ module PotatoMesh
          last_update_time: last_update,
          is_private: private_mode?,
          contact_link: sanitized_contact_link,
+          nodes_count: nodes_count,
        }
      end

+      # Count the number of nodes active since the supplied timestamp.
+      #
+      # @param cutoff [Integer] unix timestamp in seconds.
+      # @param db [SQLite3::Database, nil] optional open handle to reuse.
+      # @return [Integer, nil] node count or nil when unavailable.
+      def active_node_count_since(cutoff, db: nil)
+        return nil unless cutoff
+
+        handle = db || open_database(readonly: true)
+        count =
+          with_busy_retry do
+            handle.get_first_value("SELECT COUNT(*) FROM nodes WHERE last_heard >= ?", cutoff.to_i)
+          end
+        Integer(count)
+      rescue SQLite3::Exception, ArgumentError => e
+        warn_log(
+          "Failed to count active nodes",
+          context: "instances.nodes_count",
+          error_class: e.class.name,
+          error_message: e.message,
+        )
+        nil
+      ensure
+        handle&.close unless db
+      end
+
      def sign_instance_attributes(attributes)
        payload = canonical_instance_payload(attributes)
        Base64.strict_encode64(
@@ -723,6 +751,7 @@ module PotatoMesh
        end

        processed_entries = 0
+        recent_cutoff = Time.now.to_i - PotatoMesh::Config.remote_instance_max_node_age
        payload.each do |entry|
          if per_response_limit && per_response_limit.positive? && processed_entries >= per_response_limit
            debug_log(
@@ -777,13 +806,27 @@ module PotatoMesh

          attributes[:is_private] = false if attributes[:is_private].nil?

+          nodes_since_path = "/api/nodes?since=#{recent_cutoff}&limit=1000"
+          nodes_since_window, nodes_since_metadata = fetch_instance_json(attributes[:domain], nodes_since_path)
+          if nodes_since_window.is_a?(Array)
+            attributes[:nodes_count] = nodes_since_window.length
+          elsif nodes_since_metadata
+            warn_log(
+              "Failed to load remote node window",
+              context: "federation.instances",
+              domain: attributes[:domain],
+              reason: Array(nodes_since_metadata).map(&:to_s).join("; "),
+            )
+          end
+
          remote_nodes, node_metadata = fetch_instance_json(attributes[:domain], "/api/nodes")
+          remote_nodes ||= nodes_since_window if nodes_since_window.is_a?(Array)
          unless remote_nodes
            warn_log(
              "Failed to load remote node data",
              context: "federation.instances",
              domain: attributes[:domain],
-              reason: Array(node_metadata).map(&:to_s).join("; "),
+              reason: Array(node_metadata || nodes_since_metadata).map(&:to_s).join("; "),
            )
            next
          end
@@ -1059,8 +1102,8 @@ module PotatoMesh
        sql = <<~SQL
          INSERT INTO instances (
            id, domain, pubkey, name, version, channel, frequency,
-            latitude, longitude, last_update_time, is_private, contact_link, signature
-          ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+            latitude, longitude, last_update_time, is_private, nodes_count, contact_link, signature
+          ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
          ON CONFLICT(id) DO UPDATE SET
            domain=excluded.domain,
            pubkey=excluded.pubkey,
@@ -1072,10 +1115,12 @@ module PotatoMesh
            longitude=excluded.longitude,
            last_update_time=excluded.last_update_time,
            is_private=excluded.is_private,
+            nodes_count=excluded.nodes_count,
            contact_link=excluded.contact_link,
            signature=excluded.signature
        SQL

+        nodes_count = coerce_integer(attributes[:nodes_count])
        params = [
          attributes[:id],
          normalized_domain,
@@ -1088,6 +1133,7 @@ module PotatoMesh
          attributes[:longitude],
          attributes[:last_update_time],
          attributes[:is_private] ? 1 : 0,
+          nodes_count,
          attributes[:contact_link],
          signature,
        ]
@@ -143,6 +143,7 @@ module PotatoMesh
          "longitude" => coerce_float(row["longitude"]),
          "lastUpdateTime" => last_update_time,
          "isPrivate" => private_flag,
+          "nodesCount" => coerce_integer(row["nodes_count"]),
          "contactLink" => string_or_nil(row["contact_link"]),
          "signature" => signature,
        }
@@ -174,7 +175,7 @@ module PotatoMesh
        min_last_update_time = now - PotatoMesh::Config.week_seconds
        sql = <<~SQL
          SELECT id, domain, pubkey, name, version, channel, frequency,
-                 latitude, longitude, last_update_time, is_private, contact_link, signature
+                 latitude, longitude, last_update_time, is_private, nodes_count, contact_link, signature
          FROM instances
          WHERE domain IS NOT NULL AND TRIM(domain) != ''
            AND pubkey IS NOT NULL AND TRIM(pubkey) != ''
@@ -20,6 +20,7 @@ module PotatoMesh
      MAX_QUERY_LIMIT = 1000
      DEFAULT_TELEMETRY_WINDOW_SECONDS = 86_400
      DEFAULT_TELEMETRY_BUCKET_SECONDS = 300
+      TELEMETRY_ZERO_INVALID_COLUMNS = %w[battery_level voltage].freeze
      TELEMETRY_AGGREGATE_COLUMNS =
        %w[
          battery_level
@@ -48,6 +49,9 @@ module PotatoMesh
          soil_moisture
          soil_temperature
        ].freeze
+      TELEMETRY_AGGREGATE_SCALERS = {
+        "current" => 0.001,
+      }.freeze

      # Remove nil or empty values from an API response hash to reduce payload size
      # while preserving legitimate zero-valued measurements.
@@ -78,6 +82,19 @@ module PotatoMesh
        end
      end

+      # Treat zero-valued telemetry measurements that are known to be invalid
+      # (such as battery level or voltage) as missing data so they are omitted
+      # from API responses. Metrics that can legitimately be zero will remain
+      # untouched when routed through this helper.
+      #
+      # @param value [Numeric, nil] telemetry measurement.
+      # @return [Numeric, nil] nil when the value is zero, otherwise the original value.
+      def nil_if_zero(value)
+        return nil if value.respond_to?(:zero?) && value.zero?
+
+        value
+      end
+
      # Normalise a caller-provided limit to a sane, positive integer.
      #
      # @param limit [Object] value coerced to an integer.
@@ -245,6 +262,41 @@ module PotatoMesh
        db&.close
      end

+      def query_ingestors(limit)
+        limit = coerce_query_limit(limit)
+        db = open_database(readonly: true)
+        db.results_as_hash = true
+        now = Time.now.to_i
+        cutoff = now - PotatoMesh::Config.week_seconds
+        sql = <<~SQL
+          SELECT node_id, start_time, last_seen_time, version, lora_freq, modem_preset
+          FROM ingestors
+          WHERE last_seen_time >= ?
+          ORDER BY last_seen_time DESC
+          LIMIT ?
+        SQL
+
+        rows = db.execute(sql, [cutoff, limit])
+        rows.each do |row|
+          row.delete_if { |key, _| key.is_a?(Integer) }
+          start_time = coerce_integer(row["start_time"])
+          last_seen_time = coerce_integer(row["last_seen_time"])
+          start_time = now if start_time && start_time > now
+          last_seen_time = now if last_seen_time && last_seen_time > now
+          if start_time && last_seen_time && last_seen_time < start_time
+            last_seen_time = start_time
+          end
+          row["start_time"] = start_time
+          row["last_seen_time"] = last_seen_time
+          row["start_time_iso"] = Time.at(start_time).utc.iso8601 if start_time
+          row["last_seen_iso"] = Time.at(last_seen_time).utc.iso8601 if last_seen_time
+        end
+
+        rows.map { |row| compact_api_row(row) }
+      ensure
+        db&.close
+      end
+
      # Fetch chat messages with optional filtering.
      #
      # @param limit [Integer] maximum number of rows to return.
@@ -470,8 +522,8 @@ module PotatoMesh
          r["rssi"] = coerce_integer(r["rssi"])
          r["bitfield"] = coerce_integer(r["bitfield"])
          r["snr"] = coerce_float(r["snr"])
-          r["battery_level"] = coerce_float(r["battery_level"])
-          r["voltage"] = coerce_float(r["voltage"])
+          r["battery_level"] = sanitize_zero_invalid_metric("battery_level", coerce_float(r["battery_level"]))
+          r["voltage"] = sanitize_zero_invalid_metric("voltage", coerce_float(r["voltage"]))
          r["channel_utilization"] = coerce_float(r["channel_utilization"])
          r["air_util_tx"] = coerce_float(r["air_util_tx"])
          r["uptime_seconds"] = coerce_integer(r["uptime_seconds"])
@@ -479,7 +531,8 @@ module PotatoMesh
          r["relative_humidity"] = coerce_float(r["relative_humidity"])
          r["barometric_pressure"] = coerce_float(r["barometric_pressure"])
          r["gas_resistance"] = coerce_float(r["gas_resistance"])
-          r["current"] = coerce_float(r["current"])
+          current_ma = coerce_float(r["current"])
+          r["current"] = current_ma.nil? ? nil : current_ma / 1000.0
          r["iaq"] = coerce_integer(r["iaq"])
          r["distance"] = coerce_float(r["distance"])
          r["lux"] = coerce_float(r["lux"])
@@ -521,9 +574,10 @@ module PotatoMesh
        ]

        TELEMETRY_AGGREGATE_COLUMNS.each do |column|
-          select_clauses << "AVG(#{column}) AS #{column}_avg"
-          select_clauses << "MIN(#{column}) AS #{column}_min"
-          select_clauses << "MAX(#{column}) AS #{column}_max"
+          aggregate_source = telemetry_aggregate_source(column)
+          select_clauses << "AVG(#{aggregate_source}) AS #{column}_avg"
+          select_clauses << "MIN(#{aggregate_source}) AS #{column}_min"
+          select_clauses << "MAX(#{aggregate_source}) AS #{column}_max"
        end

        sql = <<~SQL
@@ -549,8 +603,18 @@ module PotatoMesh
            avg = coerce_float(row["#{column}_avg"])
            min_value = coerce_float(row["#{column}_min"])
            max_value = coerce_float(row["#{column}_max"])
+            scale = TELEMETRY_AGGREGATE_SCALERS[column]
+            if scale
+              avg *= scale unless avg.nil?
+              min_value *= scale unless min_value.nil?
+              max_value *= scale unless max_value.nil?
+            end

            metrics = {}
+            avg = sanitize_zero_invalid_metric(column, avg)
+            min_value = sanitize_zero_invalid_metric(column, min_value)
+            max_value = sanitize_zero_invalid_metric(column, max_value)
+
            metrics["avg"] = avg unless avg.nil?
            metrics["min"] = min_value unless min_value.nil?
            metrics["max"] = max_value unless max_value.nil?
@@ -578,6 +642,34 @@ module PotatoMesh
        db&.close
      end

+      # Normalise telemetry metrics that cannot legitimately be zero so API
+      # consumers do not mistake absent readings for valid measurements. Values
+      # for fields such as battery level and voltage are treated as missing data
+      # when they are zero.
+      #
+      # @param column [String] telemetry metric name.
+      # @param value [Numeric, nil] raw metric value.
+      # @return [Numeric, nil] metric value or nil when zero is invalid.
+      def sanitize_zero_invalid_metric(column, value)
+        return nil_if_zero(value) if TELEMETRY_ZERO_INVALID_COLUMNS.include?(column)
+
+        value
+      end
+
+      # Choose the SQL expression used to aggregate telemetry metrics. Metrics
+      # that cannot legitimately be zero are wrapped in a NULLIF to ensure
+      # invalid zero readings are ignored by aggregate functions such as AVG,
+      # MIN, and MAX, aligning the database semantics with API-level
+      # zero-as-missing handling.
+      #
+      # @param column [String] telemetry metric name.
+      # @return [String] SQL fragment used in aggregate expressions.
+      def telemetry_aggregate_source(column)
+        return "NULLIF(#{column}, 0)" if TELEMETRY_ZERO_INVALID_COLUMNS.include?(column)
+
+        column
+      end
+
      def query_traces(limit, node_ref: nil)
        limit = coerce_query_limit(limit)
        db = open_database(readonly: true)
@@ -77,6 +77,12 @@ module PotatoMesh
            rows.first.to_json
          end

+          app.get "/api/ingestors" do
+            content_type :json
+            limit = coerce_query_limit(params["limit"])
+            query_ingestors(limit).to_json
+          end
+
          app.get "/api/messages" do
            content_type :json
            limit = [params["limit"]&.to_i || 200, 1000].min
@@ -65,6 +65,25 @@ module PotatoMesh
            db&.close
          end

+          app.post "/api/ingestors" do
+            require_token!
+            content_type :json
+            begin
+              payload = JSON.parse(read_json_body)
+            rescue JSON::ParserError
+              halt 400, { error: "invalid JSON" }.to_json
+            end
+            unless payload.is_a?(Hash)
+              halt 400, { error: "invalid payload" }.to_json
+            end
+            db = open_database
+            stored = upsert_ingestor(db, payload)
+            halt 400, { error: "invalid payload" }.to_json unless stored
+            { status: "ok" }.to_json
+          ensure
+            db&.close
+          end
+
          app.post "/api/instances" do
            content_type :json
            begin
@@ -175,7 +175,7 @@ module PotatoMesh
    #
    # @return [String] semantic version identifier.
    def version_fallback
-      "0.5.6"
+      "0.5.7"
    end

    # Default refresh interval for frontend polling routines.
@@ -1,12 +1,12 @@
 {
  "name": "potato-mesh",
-  "version": "0.5.6",
+  "version": "0.5.7",
  "lockfileVersion": 3,
  "requires": true,
  "packages": {
    "": {
      "name": "potato-mesh",
-      "version": "0.5.6",
+      "version": "0.5.7",
      "devDependencies": {
        "istanbul-lib-coverage": "^3.2.2",
        "istanbul-lib-report": "^3.0.1",
@@ -1,6 +1,6 @@
 {
  "name": "potato-mesh",
-  "version": "0.5.6",
+  "version": "0.5.7",
  "type": "module",
  "private": true,
  "scripts": {
@@ -19,6 +19,7 @@ import assert from 'node:assert/strict';

 import { createDomEnvironment } from './dom-environment.js';
 import { initializeFederationPage } from '../federation-page.js';
+import { roleColors } from '../role-helpers.js';

 test('federation map centers on configured coordinates and follows theme filters', async () => {
  const env = createDomEnvironment({ includeBody: true, bodyHasDarkClass: true });
@@ -54,6 +55,7 @@ test('federation map centers on configured coordinates and follows theme filters
  tilePane.appendChild(tileImage);
  const mapSetViewCalls = [];
  const mapFitBoundsCalls = [];
+  const circleMarkerCalls = [];
  const tileLayerStub = {
    addTo() {
      return this;
@@ -94,7 +96,8 @@ test('federation map centers on configured coordinates and follows theme filters
        }
      };
    },
-    circleMarker() {
+    circleMarker(latlng, options) {
+      circleMarkerCalls.push({ latlng, options });
      return {
        bindPopup() {
          return this;
@@ -112,13 +115,15 @@ const fetchImpl = async () => ({
      version: '1.0.0',
      latitude: 10.12345,
      longitude: -20.98765,
-      lastUpdateTime: Math.floor(Date.now() / 1000) - 90
+      lastUpdateTime: Math.floor(Date.now() / 1000) - 90,
+      nodesCount: 12
    },
    {
      domain: 'bravo.mesh',
      contactLink: null,
      version: '2.0.0',
-      lastUpdateTime: Math.floor(Date.now() / 1000) - (2 * 86400)
+      lastUpdateTime: Math.floor(Date.now() / 1000) - (2 * 86400),
+      nodesCount: 2
    }
  ]
 });
@@ -150,14 +155,268 @@ const fetchImpl = async () => ({
    assert.match(firstRowHtml, /https:\/\/chat\.alpha/);
    assert.match(firstRowHtml, /10\.12345/);
    assert.match(firstRowHtml, /-20\.98765/);
+    assert.match(firstRowHtml, />12</);
    assert.match(firstRowHtml, /ago/);

    const secondRowHtml = rows[1].innerHTML;
    assert.match(secondRowHtml, /bravo\.mesh/);
    assert.match(secondRowHtml, /<em>—<\/em>/); // no contact link
    assert.match(secondRowHtml, /2\.0\.0/);
+    assert.match(secondRowHtml, />2</);
    assert.match(secondRowHtml, /d ago/);
    assert.deepEqual(mapFitBoundsCalls[0][0], [[10.12345, -20.98765]]);
+    assert.equal(circleMarkerCalls[0].options.fillColor, roleColors.CLIENT_HIDDEN);
+  } catch (error) {
+    console.error('federation sorting test error', error);
+    throw error;
+  } finally {
+    cleanup();
+  }
+});
+
+test('federation table sorting, contact rendering, and legend creation', async () => {
+  const env = createDomEnvironment({ includeBody: true, bodyHasDarkClass: false });
+  const { document, createElement, registerElement, cleanup } = env;
+
+  const mapEl = createElement('div', 'map');
+  registerElement('map', mapEl);
+  const statusEl = createElement('div', 'status');
+  registerElement('status', statusEl);
+
+  const tableEl = createElement('table', 'instances');
+  const tbodyEl = createElement('tbody');
+  registerElement('instances', tableEl);
+  tableEl.appendChild(tbodyEl);
+
+  const headerNameTh = createElement('th');
+  const headerName = createElement('span');
+  headerName.classList.add('sort-header');
+  headerName.dataset.sortKey = 'name';
+  headerName.dataset.sortLabel = 'Name';
+  headerNameTh.appendChild(headerName);
+
+  const headerDomainTh = createElement('th');
+  const headerDomain = createElement('span');
+  headerDomain.classList.add('sort-header');
+  headerDomain.dataset.sortKey = 'domain';
+  headerDomain.dataset.sortLabel = 'Domain';
+  headerDomainTh.appendChild(headerDomain);
+
+  const ths = [headerNameTh, headerDomainTh];
+  const headers = [headerName, headerDomain];
+  const headerHandlers = new Map();
+  headers.forEach(header => {
+    header.addEventListener = (event, handler) => {
+      const existing = headerHandlers.get(header) || {};
+      existing[event] = handler;
+      headerHandlers.set(header, existing);
+    };
+    header.closest = () => ths.find(th => th.childNodes.includes(header));
+    header.querySelector = selector => {
+      if (selector === '.sort-indicator') {
+        const span = createElement('span');
+        span.classList.add('sort-indicator');
+        return span;
+      }
+      return null;
+    };
+  });
+
+  tableEl.querySelectorAll = selector => {
+    if (selector === 'thead .sort-header[data-sort-key]') return headers;
+    if (selector === 'thead th') return ths;
+    return [];
+  };
+
+  const configPayload = {
+    mapCenter: { lat: 0, lon: 0 },
+    mapZoom: 3,
+    tileFilters: { light: 'none', dark: 'invert(1)' }
+  };
+  const configEl = createElement('div');
+  configEl.setAttribute('data-app-config', JSON.stringify(configPayload));
+
+  document.querySelector = selector => {
+    if (selector === '[data-app-config]') return configEl;
+    if (selector === '#instances tbody') return tbodyEl;
+    return null;
+  };
+
+  const legendContainers = [];
+  const mapSetViewCalls = [];
+  const mapFitBoundsCalls = [];
+  const circleMarkerCalls = [];
+
+  const DomUtil = {
+    create(tag, className, parent) {
+      const el = {
+        tagName: tag,
+        className,
+        children: [],
+        style: {},
+        textContent: '',
+        setAttribute() {},
+        appendChild(child) {
+          this.children.push(child);
+          return child;
+        },
+      };
+      if (parent && parent.appendChild) parent.appendChild(el);
+      return el;
+    }
+  };
+
+  const controlStub = () => {
+    const ctrl = {
+      onAdd: null,
+      container: null,
+      addTo(map) {
+        this.container = this.onAdd ? this.onAdd(map) : null;
+        legendContainers.push(this.container);
+        return this;
+      },
+      getContainer() {
+        return this.container;
+      }
+    };
+    return ctrl;
+  };
+
+  const markersLayer = {
+    layers: [],
+    addLayer(marker) {
+      this.layers.push(marker);
+      return marker;
+    },
+    addTo() {
+      return this;
+    }
+  };
+
+  const mapStub = {
+    addedControls: [],
+    setView(...args) {
+      mapSetViewCalls.push(args);
+    },
+    on() {},
+    fitBounds(...args) {
+      mapFitBoundsCalls.push(args);
+    },
+    addLayer(layer) {
+      this.addedControls.push(layer);
+      return layer;
+    }
+  };
+
+  const leafletStub = {
+    map() {
+      return mapStub;
+    },
+    tileLayer() {
+      return {
+        addTo() {
+          return this;
+        },
+        getContainer() {
+          return null;
+        },
+        on() {}
+      };
+    },
+    layerGroup() {
+      return markersLayer;
+    },
+    circleMarker(latlng, options) {
+      circleMarkerCalls.push({ latlng, options });
+      return {
+        bindPopup() {
+          return this;
+        },
+        addTo() {
+          return this;
+        }
+      };
+    },
+    control: controlStub,
+    DomUtil
+  };
+
+  const now = Math.floor(Date.now() / 1000);
+  const fetchImpl = async () => ({
+    ok: true,
+    json: async () => [
+      {
+        domain: 'c.mesh',
+        name: 'Charlie',
+        contactLink: 'https://charlie.example\nmatrix:#c:mesh',
+        version: '3.0.0',
+        latitude: 1,
+        longitude: 1,
+        lastUpdateTime: now - 10,
+        nodesCount: 0
+      },
+      {
+        domain: 'b.mesh',
+        contactLink: '',
+        version: '2.0.0',
+        latitude: 2,
+        longitude: 2,
+        lastUpdateTime: now - 60,
+        nodesCount: 650
+      },
+      {
+        domain: 'a.mesh',
+        name: 'Alpha',
+        contactLink: 'mailto:alpha@mesh',
+        version: '1.0.0',
+        latitude: 3,
+        longitude: 3,
+        lastUpdateTime: now - 30,
+        nodesCount: 5
+      }
+    ]
+  });
+
+  try {
+    await initializeFederationPage({ config: configPayload, fetchImpl, leaflet: leafletStub });
+
+    const rows = tbodyEl.childNodes.map(node => String(node.childNodes[0]));
+    assert.match(rows[0], /c\.mesh/);
+    assert.match(rows[0], /0</);
+    assert.match(rows[0], /https:\/\/charlie\.example/);
+    assert.match(rows[0], /matrix:#c:mesh/);
+    assert.match(rows[1], /a\.mesh/);
+    assert.match(rows[2], /b\.mesh/);
+
+    const nameHandlers = headerHandlers.get(headerName);
+    nameHandlers.click();
+    const afterNameSort = tbodyEl.childNodes.map(node => String(node.childNodes[0]));
+    assert.match(afterNameSort[0], /a\.mesh/);
+    assert.match(afterNameSort[1], /c\.mesh/);
+    assert.match(afterNameSort[2], /b\.mesh/);
+
+    nameHandlers.click();
+    const descSort = tbodyEl.childNodes.map(node => String(node.childNodes[0]));
+    assert.match(descSort[0], /c\.mesh/);
+    assert.match(descSort[1], /a\.mesh/);
+    assert.match(descSort[2], /b\.mesh/);
+    assert.equal(headerName.closest().attributes.get('aria-sort'), 'descending');
+
+    assert.equal(circleMarkerCalls[0].options.fillColor, roleColors.CLIENT_HIDDEN);
+    assert.equal(circleMarkerCalls[1].options.fillColor, roleColors.REPEATER);
+
+    assert.deepEqual(mapSetViewCalls[0], [[0, 0], 3]);
+    assert.equal(mapFitBoundsCalls[0][0].length, 3);
+
+    assert.equal(legendContainers.length, 1);
+    const legend = legendContainers[0];
+    assert.ok(legend.className.includes('legend'));
+    const legendHeader = legend.children.find(child => child.className === 'legend-header');
+    const legendTitle = legendHeader && Array.isArray(legendHeader.children)
+      ? legendHeader.children.find(child => child.className === 'legend-title')
+      : null;
+    assert.ok(legendTitle);
+    assert.equal(legendTitle.textContent, 'Active nodes');
  } finally {
    cleanup();
  }
@@ -90,10 +90,29 @@ test('resolveInstanceLabel falls back to the domain when the name is missing', (
 test('buildInstanceUrl normalises domains into navigable HTTPS URLs', () => {
  assert.equal(buildInstanceUrl('mesh.example'), 'https://mesh.example');
  assert.equal(buildInstanceUrl(' https://mesh.example '), 'https://mesh.example');
+  assert.equal(buildInstanceUrl('https://mesh.example/path?query#fragment'), 'https://mesh.example');
+  assert.equal(buildInstanceUrl('javascript:alert(1)'), null);
+  assert.equal(buildInstanceUrl('ftp://mesh.example'), null);
+  assert.equal(buildInstanceUrl('mesh.example:8080'), 'https://mesh.example:8080');
+  assert.equal(buildInstanceUrl('mesh.example<script>'), null);
  assert.equal(buildInstanceUrl(''), null);
  assert.equal(buildInstanceUrl(null), null);
 });

+test('buildInstanceUrl rejects malformed HTTP URLs safely', () => {
+  const originalWarn = console.warn;
+  const warnings = [];
+  console.warn = message => warnings.push(message);
+
+  try {
+    assert.equal(buildInstanceUrl('http://[::1'), null);
+    assert.equal(buildInstanceUrl('https://bad host.example'), null);
+    assert.ok(warnings.length >= 1);
+  } finally {
+    console.warn = originalWarn;
+  }
+});
+
 test('initializeInstanceSelector populates options alphabetically and selects the configured domain', async () => {
  const env = createDomEnvironment();
  const select = setupSelectElement(env.document);
@@ -16,6 +16,7 @@

 import { readAppConfig } from './config.js';
 import { mergeConfig } from './settings.js';
+import { roleColors } from './role-helpers.js';

 /**
 * Escape HTML special characters to prevent XSS.
@@ -78,6 +79,131 @@ function buildInstanceUrl(domain) {
  return `https://${trimmed}`;
 }

+const NODE_COUNT_COLOR_STOPS = [
+  { limit: 100, color: roleColors.CLIENT_HIDDEN },
+  { limit: 200, color: roleColors.SENSOR },
+  { limit: 300, color: roleColors.TRACKER },
+  { limit: 400, color: roleColors.CLIENT_MUTE },
+  { limit: 500, color: roleColors.CLIENT },
+  { limit: 600, color: roleColors.CLIENT_BASE },
+  { limit: 700, color: roleColors.REPEATER },
+  { limit: 800, color: roleColors.ROUTER_LATE },
+  { limit: 900, color: roleColors.ROUTER }
+];
+
+const DEFAULT_INSTANCE_COLOR = roleColors.LOST_AND_FOUND || '#3388ff';
+
+/**
+ * Determine the marker colour for an instance based on its active node count.
+ *
+ * @param {*} count Raw node count value from the API.
+ * @returns {string} CSS colour string.
+ */
+function colorForNodeCount(count) {
+  const numeric = Number(count);
+  if (!Number.isFinite(numeric) || numeric < 0) return DEFAULT_INSTANCE_COLOR;
+  const stop = NODE_COUNT_COLOR_STOPS.find(entry => numeric < entry.limit);
+  return stop && stop.color ? stop.color : DEFAULT_INSTANCE_COLOR;
+}
+
+/**
+ * Render arbitrary contact text while hyperlinking recognised URL-like segments.
+ *
+ * @param {*} contact Raw contact value from the API.
+ * @returns {string} HTML markup safe for insertion.
+ */
+function renderContactHtml(contact) {
+  if (typeof contact !== 'string') return '';
+  const trimmed = contact.trim();
+  if (!trimmed) return '';
+  const urlPattern = /(https?:\/\/[^\s]+|mailto:[^\s]+|matrix:[^\s]+)/gi;
+  const parts = [];
+  let lastIndex = 0;
+  let match;
+
+  while ((match = urlPattern.exec(trimmed)) !== null) {
+    const textBefore = trimmed.slice(lastIndex, match.index);
+    if (textBefore) {
+      parts.push(escapeHtml(textBefore));
+    }
+    const url = match[0];
+    const safeUrl = escapeHtml(url);
+    parts.push(`<a href="${safeUrl}" target="_blank" rel="noopener noreferrer">${safeUrl}</a>`);
+    lastIndex = match.index + url.length;
+  }
+
+  const trailing = trimmed.slice(lastIndex);
+  if (trailing) {
+    parts.push(escapeHtml(trailing));
+  }
+
+  const html = parts.join('');
+  return html.replace(/\r?\n/g, '<br>');
+}
+
+/**
+ * Convert a value into a finite number or null when invalid.
+ *
+ * @param {*} value Raw value to convert.
+ * @returns {number|null} Finite number or null.
+ */
+function toFiniteNumber(value) {
+  const num = Number(value);
+  return Number.isFinite(num) ? num : null;
+}
+
+/**
+ * Compare two string-like values ignoring case.
+ *
+ * @param {*} a Left-hand operand.
+ * @param {*} b Right-hand operand.
+ * @returns {number} Comparator result.
+ */
+function compareString(a, b) {
+  const left = typeof a === 'string' ? a.toLowerCase() : String(a ?? '').toLowerCase();
+  const right = typeof b === 'string' ? b.toLowerCase() : String(b ?? '').toLowerCase();
+  return left.localeCompare(right);
+}
+
+/**
+ * Compare two numeric values.
+ *
+ * @param {*} a Left-hand operand.
+ * @param {*} b Right-hand operand.
+ * @returns {number} Comparator result.
+ */
+function compareNumber(a, b) {
+  const left = toFiniteNumber(a);
+  const right = toFiniteNumber(b);
+  if (left == null && right == null) return 0;
+  if (left == null) return 1;
+  if (right == null) return -1;
+  if (left === right) return 0;
+  return left < right ? -1 : 1;
+}
+
+/**
+ * Determine whether a string-like value is present.
+ *
+ * @param {*} value Candidate value.
+ * @returns {boolean} true when present.
+ */
+function hasStringValue(value) {
+  if (value == null) return false;
+  if (typeof value === 'string') return value.trim() !== '';
+  return String(value).trim() !== '';
+}
+
+/**
+ * Determine whether a numeric value is present.
+ *
+ * @param {*} value Candidate value.
+ * @returns {boolean} true when present.
+ */
+function hasNumberValue(value) {
+  return toFiniteNumber(value) != null;
+}
+
 const TILE_LAYER_URL = 'https://{s}.tile.openstreetmap.fr/hot/{z}/{x}/{y}.png';

 /**
@@ -97,8 +223,12 @@ export async function initializeFederationPage(options = {}) {
  const fetchImpl = options.fetchImpl || fetch;
  const leaflet = options.leaflet || (typeof window !== 'undefined' ? window.L : null);
  const mapContainer = document.getElementById('map');
+  const tableEl = document.getElementById('instances');
  const tableBody = document.querySelector('#instances tbody');
  const statusEl = document.getElementById('status');
+  const sortHeaders = tableEl
+    ? Array.from(tableEl.querySelectorAll('thead .sort-header[data-sort-key]'))
+    : [];

  const hasLeaflet =
    typeof leaflet === 'object' &&
@@ -109,6 +239,154 @@ export async function initializeFederationPage(options = {}) {
  let map = null;
  let markersLayer = null;
  let tileLayer = null;
+  const tableSorters = {
+    name: { getValue: inst => inst.name ?? '', compare: compareString, hasValue: hasStringValue, defaultDirection: 'asc' },
+    domain: { getValue: inst => inst.domain ?? '', compare: compareString, hasValue: hasStringValue, defaultDirection: 'asc' },
+    contact: { getValue: inst => inst.contactLink ?? '', compare: compareString, hasValue: hasStringValue, defaultDirection: 'asc' },
+    version: { getValue: inst => inst.version ?? '', compare: compareString, hasValue: hasStringValue, defaultDirection: 'asc' },
+    channel: { getValue: inst => inst.channel ?? '', compare: compareString, hasValue: hasStringValue, defaultDirection: 'asc' },
+    frequency: { getValue: inst => inst.frequency ?? '', compare: compareString, hasValue: hasStringValue, defaultDirection: 'asc' },
+    nodesCount: {
+      getValue: inst => toFiniteNumber(inst.nodesCount ?? inst.nodes_count),
+      compare: compareNumber,
+      hasValue: hasNumberValue,
+      defaultDirection: 'desc'
+    },
+    latitude: { getValue: inst => toFiniteNumber(inst.latitude), compare: compareNumber, hasValue: hasNumberValue, defaultDirection: 'asc' },
+    longitude: { getValue: inst => toFiniteNumber(inst.longitude), compare: compareNumber, hasValue: hasNumberValue, defaultDirection: 'asc' },
+    lastUpdateTime: {
+      getValue: inst => toFiniteNumber(inst.lastUpdateTime),
+      compare: compareNumber,
+      hasValue: hasNumberValue,
+      defaultDirection: 'desc'
+    }
+  };
+  let sortState = {
+    key: 'lastUpdateTime',
+    direction: tableSorters.lastUpdateTime ? tableSorters.lastUpdateTime.defaultDirection : 'desc'
+  };
+
+  /**
+   * Sort instances using the active sort configuration.
+   *
+   * @param {Array<Object>} data Instance rows.
+   * @returns {Array<Object>} sorted rows.
+   */
+  const sortInstancesData = data => {
+    const sorter = tableSorters[sortState.key];
+    if (!sorter) return Array.isArray(data) ? [...data] : [];
+    const dir = sortState.direction === 'asc' ? 1 : -1;
+    return [...(data || [])].sort((a, b) => {
+      const aVal = sorter.getValue(a);
+      const bVal = sorter.getValue(b);
+      const aHas = sorter.hasValue ? sorter.hasValue(aVal) : hasStringValue(aVal);
+      const bHas = sorter.hasValue ? sorter.hasValue(bVal) : hasStringValue(bVal);
+      if (aHas && bHas) {
+        return sorter.compare(aVal, bVal) * dir;
+      }
+      if (aHas) return -1;
+      if (bHas) return 1;
+      return 0;
+    });
+  };
+
+  /**
+   * Update the visual sort indicators for the active column.
+   *
+   * @returns {void}
+   */
+  const syncSortIndicators = () => {
+    if (!tableEl || !sortHeaders.length) return;
+    tableEl.querySelectorAll('thead th').forEach(th => th.removeAttribute('aria-sort'));
+    sortHeaders.forEach(header => {
+      header.removeAttribute('data-sort-active');
+      const indicator = header.querySelector('.sort-indicator');
+      if (indicator) indicator.textContent = '';
+    });
+    const active = sortHeaders.find(header => header.dataset.sortKey === sortState.key);
+    if (!active) return;
+    const indicator = active.querySelector('.sort-indicator');
+    if (indicator) indicator.textContent = sortState.direction === 'asc' ? '▲' : '▼';
+    active.setAttribute('data-sort-active', 'true');
+    const th = active.closest('th');
+    if (th) {
+      th.setAttribute('aria-sort', sortState.direction === 'asc' ? 'ascending' : 'descending');
+    }
+  };
+
+  /**
+   * Render the instances table body with sorting applied.
+   *
+   * @param {Array<Object>} data Instance rows.
+   * @param {number} nowSec Reference timestamp for relative time rendering.
+   * @returns {void}
+   */
+  const renderTableRows = (data, nowSec) => {
+    if (!tableBody) return;
+    const frag = document.createDocumentFragment();
+    const sorted = sortInstancesData(data);
+
+    for (const instance of sorted) {
+      const tr = document.createElement('tr');
+      const url = buildInstanceUrl(instance.domain);
+      const nameHtml = instance.name ? escapeHtml(instance.name) : '<em>—</em>';
+      const domainHtml = url
+        ? `<a href="${escapeHtml(url)}" target="_blank" rel="noopener">${escapeHtml(instance.domain || '')}</a>`
+        : escapeHtml(instance.domain || '');
+      const contactHtml = renderContactHtml(instance.contactLink);
+      const nodesCountValue = toFiniteNumber(instance.nodesCount ?? instance.nodes_count);
+      const nodesCountText = nodesCountValue == null ? '<em>—</em>' : escapeHtml(String(nodesCountValue));
+
+      tr.innerHTML = `
+        <td class="instances-col instances-col--name">${nameHtml}</td>
+        <td class="instances-col instances-col--domain mono">${domainHtml}</td>
+        <td class="instances-col instances-col--contact">${contactHtml || '<em>—</em>'}</td>
+        <td class="instances-col instances-col--version mono">${escapeHtml(instance.version || '')}</td>
+        <td class="instances-col instances-col--channel">${escapeHtml(instance.channel || '')}</td>
+        <td class="instances-col instances-col--frequency">${escapeHtml(instance.frequency || '')}</td>
+        <td class="instances-col instances-col--nodes mono">${nodesCountText}</td>
+        <td class="instances-col instances-col--latitude mono">${fmtCoords(instance.latitude)}</td>
+        <td class="instances-col instances-col--longitude mono">${fmtCoords(instance.longitude)}</td>
+        <td class="instances-col instances-col--last-update mono">${timeAgo(instance.lastUpdateTime, nowSec)}</td>
+      `;
+
+      frag.appendChild(tr);
+    }
+
+    tableBody.replaceChildren(frag);
+    syncSortIndicators();
+  };
+
+  /**
+   * Wire up click and keyboard handlers for sortable headers.
+   *
+   * @param {Function} rerender Callback to refresh the table.
+   * @returns {void}
+   */
+  const attachSortHandlers = rerender => {
+    if (!sortHeaders.length) return;
+    const applySortKey = key => {
+      if (!key) return;
+      if (sortState.key === key) {
+        sortState = { key, direction: sortState.direction === 'asc' ? 'desc' : 'asc' };
+      } else {
+        const defaultDir = tableSorters[key]?.defaultDirection || 'asc';
+        sortState = { key, direction: defaultDir };
+      }
+      rerender();
+    };
+
+    sortHeaders.forEach(header => {
+      const key = header.dataset.sortKey;
+      header.addEventListener('click', () => applySortKey(key));
+      header.addEventListener('keydown', event => {
+        if (event.key === 'Enter' || event.key === ' ') {
+          event.preventDefault();
+          applySortKey(key);
+        }
+      });
+    });
+  };

  /**
   * Resolve the active theme based on the DOM state.
@@ -202,6 +480,38 @@ export async function initializeFederationPage(options = {}) {
  // Render map markers
  if (map && markersLayer && hasLeaflet && Array.isArray(instances)) {
    const bounds = [];
+    const canRenderLegend =
+      typeof leaflet.control === 'function' && leaflet.DomUtil && typeof leaflet.DomUtil.create === 'function';
+    if (canRenderLegend) {
+      const legendStops = NODE_COUNT_COLOR_STOPS.map((stop, index) => {
+        const lower = index === 0 ? 0 : NODE_COUNT_COLOR_STOPS[index - 1].limit;
+        const upper = stop.limit - 1;
+        const label = index === 0 ? `< ${stop.limit} nodes` : `${lower}-${upper} nodes`;
+        return { color: stop.color || DEFAULT_INSTANCE_COLOR, label };
+      });
+      const lastLimit = NODE_COUNT_COLOR_STOPS[NODE_COUNT_COLOR_STOPS.length - 1]?.limit || 900;
+      legendStops.push({ color: DEFAULT_INSTANCE_COLOR, label: `≥ ${lastLimit} nodes` });
+
+      const legend = leaflet.control({ position: 'bottomright' });
+      legend.onAdd = function onAdd() {
+        const container = leaflet.DomUtil.create('div', 'legend legend--instances');
+        container.setAttribute('aria-label', 'Active nodes legend');
+        const header = leaflet.DomUtil.create('div', 'legend-header', container);
+        const title = leaflet.DomUtil.create('span', 'legend-title', header);
+        title.textContent = 'Active nodes';
+        const items = leaflet.DomUtil.create('div', 'legend-items', container);
+        legendStops.forEach(stop => {
+          const item = leaflet.DomUtil.create('div', 'legend-item', items);
+          item.setAttribute('aria-hidden', 'true');
+          const swatch = leaflet.DomUtil.create('span', 'legend-swatch', item);
+          swatch.style.background = stop.color;
+          const label = leaflet.DomUtil.create('span', 'legend-label', item);
+          label.textContent = stop.label;
+        });
+        return container;
+      };
+      legend.addTo(map);
+    }

    for (const instance of instances) {
      const lat = Number(instance.latitude);
@@ -213,24 +523,28 @@ export async function initializeFederationPage(options = {}) {

      const name = instance.name || instance.domain || 'Unknown';
      const url = buildInstanceUrl(instance.domain);
-      const popupContent = url
-        ? `<strong><a href="${escapeHtml(url)}" target="_blank" rel="noopener">${escapeHtml(name)}</a></strong><br>
-           <span class="mono">${escapeHtml(instance.domain || '')}</span><br>
-           ${instance.channel ? `Channel: ${escapeHtml(instance.channel)}<br>` : ''}
-           ${instance.frequency ? `Frequency: ${escapeHtml(instance.frequency)}<br>` : ''}
-           ${instance.version ? `Version: ${escapeHtml(instance.version)}` : ''}`
-        : `<strong>${escapeHtml(name)}</strong>`;
+      const nodeCountValue = toFiniteNumber(instance.nodesCount ?? instance.nodes_count);
+      const popupLines = [
+        url
+          ? `<strong><a href="${escapeHtml(url)}" target="_blank" rel="noopener">${escapeHtml(name)}</a></strong>`
+          : `<strong>${escapeHtml(name)}</strong>`,
+        `<span class="mono">${escapeHtml(instance.domain || '')}</span>`,
+        instance.channel ? `Channel: ${escapeHtml(instance.channel)}` : '',
+        instance.frequency ? `Frequency: ${escapeHtml(instance.frequency)}` : '',
+        instance.version ? `Version: ${escapeHtml(instance.version)}` : '',
+        nodeCountValue != null ? `Active nodes (24h): ${escapeHtml(String(nodeCountValue))}` : ''
+      ].filter(Boolean);

      const marker = leaflet.circleMarker([lat, lon], {
-        radius: 8,
-        fillColor: '#4CAF50',
-        color: '#2E7D32',
-        weight: 2,
-        opacity: 1,
-        fillOpacity: 0.8
+        radius: 9,
+        fillColor: colorForNodeCount(nodeCountValue),
+        color: '#000',
+        weight: 1,
+        opacity: 0.8,
+        fillOpacity: 0.75
      });

-      marker.bindPopup(popupContent);
+      marker.bindPopup(popupLines.join('<br>'));
      markersLayer.addLayer(marker);
    }

@@ -245,35 +559,7 @@ export async function initializeFederationPage(options = {}) {

  // Render table
  if (tableBody && Array.isArray(instances)) {
-    const frag = document.createDocumentFragment();
-
-    for (const instance of instances) {
-      const tr = document.createElement('tr');
-      const url = buildInstanceUrl(instance.domain);
-      const nameHtml = instance.name
-        ? escapeHtml(instance.name)
-        : '<em>—</em>';
-      const domainHtml = url
-        ? `<a href="${escapeHtml(url)}" target="_blank" rel="noopener">${escapeHtml(instance.domain || '')}</a>`
-        : escapeHtml(instance.domain || '');
-      const contact = instance.contactLink ? escapeHtml(instance.contactLink) : '';
-      const contactHtml = contact ? `<span class="mono">${contact}</span>` : '<em>—</em>';
-
-      tr.innerHTML = `
-        <td class="instances-col instances-col--name">${nameHtml}</td>
-        <td class="instances-col instances-col--domain mono">${domainHtml}</td>
-        <td class="instances-col instances-col--contact">${contactHtml}</td>
-        <td class="instances-col instances-col--version mono">${escapeHtml(instance.version || '')}</td>
-        <td class="instances-col instances-col--channel">${escapeHtml(instance.channel || '')}</td>
-        <td class="instances-col instances-col--frequency">${escapeHtml(instance.frequency || '')}</td>
-        <td class="instances-col instances-col--latitude mono">${fmtCoords(instance.latitude)}</td>
-        <td class="instances-col instances-col--longitude mono">${fmtCoords(instance.longitude)}</td>
-        <td class="instances-col instances-col--last-update mono">${timeAgo(instance.lastUpdateTime, nowSec)}</td>
-      `;
-
-      frag.appendChild(tr);
-    }
-
-    tableBody.replaceChildren(frag);
+    attachSortHandlers(() => renderTableRows(instances, nowSec));
+    renderTableRows(instances, nowSec);
  }
 }
@@ -34,12 +34,15 @@ function resolveInstanceLabel(entry) {
  return domain;
 }

-/**
- * Construct a navigable URL for the provided instance domain.
- *
- * @param {string} domain Instance domain as returned by the federation catalog.
- * @returns {string|null} Navigable absolute URL or ``null`` when the domain is empty.
- */
+ /**
+  * Construct a navigable URL for the provided instance domain.
+  *
+  * The returned URL is guaranteed to use HTTP(S) and a host-only component to avoid
+  * interpreting arbitrary DOM-controlled text as executable content.
+  *
+  * @param {string} domain Instance domain as returned by the federation catalog.
+  * @returns {string|null} Navigable absolute URL or ``null`` when the domain is empty or unsafe.
+  */
 export function buildInstanceUrl(domain) {
  if (typeof domain !== 'string') {
    return null;
@@ -50,8 +53,29 @@ export function buildInstanceUrl(domain) {
    return null;
  }

-  if (/^[a-zA-Z][a-zA-Z\d+.-]*:\/\//.test(trimmed)) {
-    return trimmed;
+  const allowedHostPattern = /^[a-zA-Z0-9.-]+(?::\d{1,5})?$/;
+
+  if (/^https?:\/\//i.test(trimmed)) {
+    try {
+      const parsed = new URL(trimmed);
+      if (!['http:', 'https:'].includes(parsed.protocol)) {
+        return null;
+      }
+
+      const sanitizedHost = parsed.host.trim();
+      if (!allowedHostPattern.test(sanitizedHost)) {
+        return null;
+      }
+
+      return `${parsed.protocol}//${sanitizedHost}`;
+    } catch (error) {
+      console.warn('Rejected invalid instance URL', error);
+      return null;
+    }
+  }
+
+  if (!allowedHostPattern.test(trimmed)) {
+    return null;
  }

  return `https://${trimmed}`;
@@ -1373,6 +1373,19 @@ button:not(.chat-tab):not(.sort-button):hover {
  outline-offset: 2px;
 }

+.sort-header {
+  display: inline-flex;
+  align-items: center;
+  gap: 4px;
+  cursor: pointer;
+  user-select: none;
+}
+
+.sort-header:focus-visible {
+  outline: 2px solid #4a90e2;
+  outline-offset: 2px;
+}
+
 .sort-indicator {
  font-size: 0.75em;
  opacity: 0.6;
@@ -1850,6 +1863,10 @@ body.dark .sort-button {
  color: inherit;
 }

+body.dark .sort-header {
+  color: inherit;
+}
+
 body.dark .sort-button:hover {
  background: none;
 }
@@ -2075,6 +2092,12 @@ body.dark #map .leaflet-tile.map-tiles {
  min-width: 180px;
 }

+.instances-col--contact {
+  min-width: 160px;
+  white-space: pre-wrap;
+  word-break: break-word;
+}
+
 .instances-col--version {
  min-width: 80px;
 }
@@ -2084,6 +2107,10 @@ body.dark #map .leaflet-tile.map-tiles {
  min-width: 100px;
 }

+.instances-col--nodes {
+  min-width: 110px;
+}
+
 .instances-col--latitude,
 .instances-col--longitude {
  min-width: 100px;
@@ -103,6 +103,7 @@ RSpec.describe "Potato Mesh Sinatra app" do
      db.execute("DELETE FROM nodes")
      db.execute("DELETE FROM positions")
      db.execute("DELETE FROM telemetry")
+      db.execute("DELETE FROM ingestors")
    end
    ensure_self_instance_record!
  end
@@ -3467,6 +3468,43 @@ RSpec.describe "Potato Mesh Sinatra app" do
        end
      end

+      it "accepts traceroutes without metrics or RSSI fields" do
+        allow(Time).to receive(:now).and_return(reference_time)
+
+        payload = [
+          {
+            "id" => 9_003,
+            "request_id" => 42,
+            "src" => 0xAAAA0001,
+            "dest" => 0xAAAA0002,
+            "rx_time" => reference_time.to_i - 1,
+            "hops" => [0xAAAA0001, 0xAAAA0003, 0xAAAA0002],
+          },
+        ]
+
+        post "/api/traces", payload.to_json, auth_headers
+
+        expect(last_response).to be_ok
+        expect(JSON.parse(last_response.body)).to eq("status" => "ok")
+
+        with_db(readonly: true) do |db|
+          db.results_as_hash = true
+
+          stored = db.get_first_row("SELECT * FROM traces WHERE id = ?", [payload.first["id"]])
+          expect(stored["rx_time"]).to eq(payload.first["rx_time"])
+          expect(stored["rx_iso"]).to eq(Time.at(payload.first["rx_time"]).utc.iso8601)
+          expect(stored["rssi"]).to be_nil
+          expect(stored["snr"]).to be_nil
+          expect(stored["elapsed_ms"]).to be_nil
+
+          hops = db.execute(
+            "SELECT hop_index, node_id FROM trace_hops WHERE trace_id = ? ORDER BY hop_index",
+            [stored["id"]],
+          )
+          expect(hops.map { |row| row["node_id"] }).to eq(payload.first["hops"])
+        end
+      end
+
      it "returns 400 when the payload is not valid JSON" do
        post "/api/traces", "{", auth_headers

@@ -4434,7 +4472,8 @@ RSpec.describe "Potato Mesh Sinatra app" do
      expect(first_entry["telemetry_time_iso"]).to eq(Time.at(latest["telemetry_time"]).utc.iso8601)
      expect(first_entry).not_to have_key("device_metrics")
      expect_same_value(first_entry["battery_level"], telemetry_metric(latest, "battery_level"))
-      expect_same_value(first_entry["current"], telemetry_metric(latest, "current"))
+      expected_current = telemetry_metric(latest, "current")
+      expect_same_value(first_entry["current"], expected_current.nil? ? nil : expected_current / 1000.0)
      expect_same_value(first_entry["distance"], telemetry_metric(latest, "distance"))
      expect_same_value(first_entry["lux"], telemetry_metric(latest, "lux"))
      expect_same_value(first_entry["wind_direction"], telemetry_metric(latest, "wind_direction"))
@@ -4555,6 +4594,51 @@ RSpec.describe "Potato Mesh Sinatra app" do
      expect(filtered.first).not_to have_key("battery_level")
      expect(filtered.first).not_to have_key("portnum")
    end
+
+    it "omits zero-valued battery and voltage metrics from telemetry responses" do
+      clear_database
+      allow(Time).to receive(:now).and_return(reference_time)
+      now = reference_time.to_i
+
+      with_db do |db|
+        db.execute(
+          "INSERT INTO telemetry(id, node_id, rx_time, rx_iso, telemetry_time, battery_level, voltage, uptime_seconds, channel_utilization) VALUES(?,?,?,?,?,?,?,?,?)",
+          [
+            88,
+            "!tele-zero",
+            now,
+            Time.at(now).utc.iso8601,
+            now - 60,
+            0,
+            0,
+            0,
+            0.5,
+          ],
+        )
+      end
+
+      get "/api/telemetry"
+
+      expect(last_response).to be_ok
+      rows = JSON.parse(last_response.body)
+      expect(rows.length).to eq(1)
+      entry = rows.first
+      expect(entry["node_id"]).to eq("!tele-zero")
+      expect(entry["rx_time"]).to eq(now)
+      expect(entry["telemetry_time"]).to eq(now - 60)
+      expect(entry).not_to have_key("battery_level")
+      expect(entry).not_to have_key("voltage")
+      expect(entry["uptime_seconds"]).to eq(0)
+      expect(entry["channel_utilization"]).to eq(0.5)
+
+      get "/api/telemetry/!tele-zero"
+
+      expect(last_response).to be_ok
+      scoped_rows = JSON.parse(last_response.body)
+      expect(scoped_rows.length).to eq(1)
+      expect(scoped_rows.first).not_to have_key("battery_level")
+      expect(scoped_rows.first).not_to have_key("voltage")
+    end
  end

  describe "GET /api/telemetry/aggregated" do
@@ -4576,6 +4660,35 @@ RSpec.describe "Potato Mesh Sinatra app" do
      expect(a_bucket["aggregates"]).to have_key("battery_level")
      expect(a_bucket["aggregates"]["battery_level"]).to include("avg")
      expect(a_bucket).not_to have_key("device_metrics")
+
+      buckets_by_start = {}
+      buckets.each do |bucket|
+        start_time = bucket["bucket_start"]
+        buckets_by_start[start_time] = bucket if start_time
+      end
+      bucket_seconds = 300
+      current_by_bucket = Hash.new { |hash, key| hash[key] = [] }
+      telemetry_fixture.each do |entry|
+        timestamp = entry["rx_time"] || entry["telemetry_time"]
+        next unless timestamp
+
+        bucket_start = (timestamp / bucket_seconds) * bucket_seconds
+        current_value = telemetry_metric(entry, "current")
+        next if current_value.nil?
+
+        current_by_bucket[bucket_start] << current_value
+      end
+
+      current_by_bucket.each do |bucket_start, values|
+        bucket = buckets_by_start[bucket_start]
+        next unless bucket
+        aggregates = bucket.fetch("aggregates", {})
+        metrics = aggregates["current"]
+        expect(metrics).not_to be_nil
+        expect_same_value(metrics["avg"], values.sum / values.length / 1000.0)
+        expect_same_value(metrics["min"], values.min / 1000.0)
+        expect_same_value(metrics["max"], values.max / 1000.0)
+      end
    end

    it "applies default window and bucket sizes when parameters are omitted" do
@@ -4590,6 +4703,86 @@ RSpec.describe "Potato Mesh Sinatra app" do
      expect(buckets.first["bucket_seconds"]).to eq(PotatoMesh::App::Queries::DEFAULT_TELEMETRY_BUCKET_SECONDS)
    end

+    it "omits zero-valued battery and voltage aggregates" do
+      clear_database
+      allow(Time).to receive(:now).and_return(reference_time)
+      now = reference_time.to_i
+
+      with_db do |db|
+        db.execute(
+          "INSERT INTO telemetry(id, node_id, rx_time, rx_iso, telemetry_time, battery_level, voltage, channel_utilization) VALUES(?,?,?,?,?,?,?,?)",
+          [
+            991,
+            "!tele-agg-zero",
+            now,
+            Time.at(now).utc.iso8601,
+            now - 30,
+            0,
+            0,
+            0.25,
+          ],
+        )
+      end
+
+      get "/api/telemetry/aggregated?windowSeconds=3600&bucketSeconds=300"
+
+      expect(last_response).to be_ok
+      buckets = JSON.parse(last_response.body)
+      expect(buckets.length).to eq(1)
+      aggregates = buckets.first.fetch("aggregates")
+      expect(aggregates).not_to have_key("battery_level")
+      expect(aggregates).not_to have_key("voltage")
+      expect(aggregates.dig("channel_utilization", "avg")).to eq(0.25)
+    end
+
+    it "ignores zero-valued telemetry when aggregating mixed buckets" do
+      clear_database
+      allow(Time).to receive(:now).and_return(reference_time)
+      now = reference_time.to_i
+
+      with_db do |db|
+        db.execute(
+          "INSERT INTO telemetry(id, node_id, rx_time, rx_iso, telemetry_time, battery_level, voltage) VALUES(?,?,?,?,?,?,?)",
+          [
+            992,
+            "!tele-agg-mixed",
+            now,
+            Time.at(now).utc.iso8601,
+            now - 120,
+            0,
+            0,
+          ],
+        )
+
+        db.execute(
+          "INSERT INTO telemetry(id, node_id, rx_time, rx_iso, telemetry_time, battery_level, voltage) VALUES(?,?,?,?,?,?,?)",
+          [
+            993,
+            "!tele-agg-mixed",
+            now,
+            Time.at(now).utc.iso8601,
+            now - 60,
+            80.0,
+            3.7,
+          ],
+        )
+      end
+
+      get "/api/telemetry/aggregated?windowSeconds=3600&bucketSeconds=300"
+
+      expect(last_response).to be_ok
+      buckets = JSON.parse(last_response.body)
+      expect(buckets.length).to eq(1)
+      aggregates = buckets.first.fetch("aggregates")
+      expect(aggregates).to have_key("battery_level")
+      expect(aggregates.dig("battery_level", "avg")).to eq(80.0)
+      expect(aggregates.dig("battery_level", "min")).to eq(80.0)
+      expect(aggregates.dig("battery_level", "max")).to eq(80.0)
+      expect(aggregates.dig("voltage", "avg")).to eq(3.7)
+      expect(aggregates.dig("voltage", "min")).to eq(3.7)
+      expect(aggregates.dig("voltage", "max")).to eq(3.7)
+    end
+
    it "rejects invalid bucket and window parameters" do
      get "/api/telemetry/aggregated?windowSeconds=0&bucketSeconds=300"
      expect(last_response.status).to eq(400)
@@ -321,6 +321,39 @@ RSpec.describe PotatoMesh::App::Federation do
      expect(visited).not_to include(attributes_list[1][:domain], attributes_list[2][:domain])
      expect(federation_helpers.debug_messages).to include(a_string_including("crawl limit"))
    end
+
+    it "requests an expanded recent node window when counting remote activity" do
+      now = Time.at(1_700_000_000)
+      allow(Time).to receive(:now).and_return(now)
+      allow(PotatoMesh::Config).to receive(:remote_instance_max_node_age).and_return(900)
+      recent_cutoff = now.to_i - 900
+
+      mapping = { [seed_domain, "/api/instances"] => [payload_entries, :instances] }
+      attributes_list.each_with_index do |attributes, index|
+        mapping[[attributes[:domain], "/api/nodes?since=#{recent_cutoff}&limit=1000"]] = [node_payload, :nodes]
+        mapping[[attributes[:domain], "/api/nodes"]] = [node_payload, :nodes]
+        mapping[[attributes[:domain], "/api/instances"]] = [[], :instances]
+        allow(federation_helpers).to receive(:remote_instance_attributes_from_payload).with(payload_entries[index]).and_return([attributes, "signature-#{index}", nil])
+      end
+
+      captured_paths = []
+      allow(federation_helpers).to receive(:fetch_instance_json) do |host, path|
+        captured_paths << [host, path]
+        mapping.fetch([host, path]) { [nil, []] }
+      end
+      allow(federation_helpers).to receive(:verify_instance_signature).and_return(true)
+      allow(federation_helpers).to receive(:validate_remote_nodes).and_return([true, nil])
+      allow(federation_helpers).to receive(:upsert_instance_record)
+
+      federation_helpers.ingest_known_instances_from!(db, seed_domain)
+
+      expect(captured_paths).to include(
+        [attributes_list[0][:domain], "/api/nodes?since=#{recent_cutoff}&limit=1000"],
+        [attributes_list[1][:domain], "/api/nodes?since=#{recent_cutoff}&limit=1000"],
+        [attributes_list[2][:domain], "/api/nodes?since=#{recent_cutoff}&limit=1000"],
+      )
+      expect(attributes_list.map { |attrs| attrs[:nodes_count] }).to all(eq(node_payload.length))
+    end
  end

  describe ".upsert_instance_record" do
@@ -400,6 +433,34 @@ RSpec.describe PotatoMesh::App::Federation do
        expect(row[1]).to eq("sig-3")
      end
    end
+
+    it "stores the nodes_count for new records" do
+      with_db do |db|
+        federation_helpers.send(:upsert_instance_record, db, base_attributes.merge(nodes_count: 77), "sig-1")
+
+        stored = db.get_first_value("SELECT nodes_count FROM instances WHERE id = ?", base_attributes[:id])
+        expect(stored).to eq(77)
+      end
+    end
+
+    it "updates the nodes_count on conflict" do
+      with_db do |db|
+        federation_helpers.send(:upsert_instance_record, db, base_attributes.merge(nodes_count: 12), "sig-1")
+
+        federation_helpers.send(
+          :upsert_instance_record,
+          db,
+          base_attributes.merge(nodes_count: 99, name: "Renamed Mesh"),
+          "sig-2",
+        )
+
+        row =
+          db.get_first_row("SELECT nodes_count, name, signature FROM instances WHERE id = ?", base_attributes[:id])
+        expect(row[0]).to eq(99)
+        expect(row[1]).to eq("Renamed Mesh")
+        expect(row[2]).to eq("sig-2")
+      end
+    end
  end

  describe ".federation_user_agent_header" do
@@ -0,0 +1,182 @@
+# Copyright © 2025-26 l5yth & contributors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# frozen_string_literal: true
+
+require "spec_helper"
+require "json"
+require "time"
+
+RSpec.describe "Ingestor endpoints" do
+  let(:app) { Sinatra::Application }
+  let(:api_token) { "secret-token" }
+  let(:auth_headers) do
+    {
+      "CONTENT_TYPE" => "application/json",
+      "HTTP_AUTHORIZATION" => "Bearer #{api_token}",
+    }
+  end
+
+  before do
+    @original_token = ENV["API_TOKEN"]
+    ENV["API_TOKEN"] = api_token
+    clear_ingestors_table
+  end
+
+  after do
+    ENV["API_TOKEN"] = @original_token
+    clear_ingestors_table
+  end
+
+  def clear_ingestors_table
+    with_db do |db|
+      db.execute("DELETE FROM ingestors")
+      db.execute("VACUUM")
+    end
+  end
+
+  def with_db(readonly: false)
+    db = PotatoMesh::Application.open_database(readonly: readonly)
+    db.busy_timeout = PotatoMesh::Config.db_busy_timeout_ms
+    db.execute("PRAGMA foreign_keys = ON")
+    yield db
+  ensure
+    db&.close
+  end
+
+  def ingestor_payload(overrides = {})
+    now = Time.now.to_i
+    {
+      node_id: "!abc12345",
+      start_time: now - 120,
+      last_seen_time: now - 60,
+      version: "0.5.7",
+      lora_freq: 915,
+      modem_preset: "LongFast",
+    }.merge(overrides)
+  end
+
+  describe "POST /api/ingestors" do
+    it "requires a bearer token" do
+      post "/api/ingestors", ingestor_payload.to_json, { "CONTENT_TYPE" => "application/json" }
+
+      expect(last_response.status).to eq(403)
+    end
+
+    it "upserts ingestor state without regressing start time" do
+      payload = ingestor_payload
+      post "/api/ingestors", payload.to_json, auth_headers
+
+      expect(last_response.status).to eq(200)
+
+      newer_last_seen = payload[:last_seen_time] + 3_600
+      older_start = payload[:start_time] - 500
+      post "/api/ingestors",
+           payload.merge(last_seen_time: newer_last_seen, start_time: older_start).to_json,
+           auth_headers
+
+      expect(last_response.status).to eq(200)
+      with_db(readonly: true) do |db|
+        row = db.get_first_row(
+          "SELECT node_id, start_time, last_seen_time, version, lora_freq, modem_preset FROM ingestors WHERE node_id = ?",
+          [payload[:node_id]],
+        )
+        expect(row[0]).to eq(payload[:node_id])
+        expect(row[1]).to eq(payload[:start_time])
+        expect(row[2]).to be >= payload[:last_seen_time]
+        expect(row[2]).to be <= Time.now.to_i
+        expect(row[3]).to eq(payload[:version])
+        expect(row[4]).to eq(payload[:lora_freq])
+        expect(row[5]).to eq(payload[:modem_preset])
+      end
+    end
+
+    it "rejects payloads missing required fields" do
+      post "/api/ingestors", { node_id: "!abcd0001" }.to_json, auth_headers
+
+      expect(last_response.status).to eq(400)
+    end
+
+    it "rejects invalid JSON" do
+      post "/api/ingestors", "{", auth_headers
+
+      expect(last_response.status).to eq(400)
+    end
+
+    it "rejects payloads missing version" do
+      post "/api/ingestors", ingestor_payload(version: nil).to_json, auth_headers
+
+      expect(last_response.status).to eq(400)
+    end
+
+    it "rejects non-object payloads" do
+      post "/api/ingestors", [].to_json, auth_headers
+
+      expect(last_response.status).to eq(400)
+    end
+  end
+
+  describe "GET /api/ingestors" do
+    it "returns recent ingestors and omits stale rows" do
+      now = Time.now.to_i
+      with_db do |db|
+        db.execute(
+          "INSERT INTO ingestors(node_id, start_time, last_seen_time, version) VALUES(?,?,?,?)",
+          ["!fresh000", now - 100, now - 10, "0.5.7"],
+        )
+        db.execute(
+          "INSERT INTO ingestors(node_id, start_time, last_seen_time, version) VALUES(?,?,?,?)",
+          ["!stale000", now - (9 * 24 * 60 * 60), now - (9 * 24 * 60 * 60), "0.5.6"],
+        )
+        db.execute(
+          "INSERT INTO ingestors(node_id, start_time, last_seen_time, version, lora_freq, modem_preset) VALUES(?,?,?,?,?,?)",
+          ["!rich000", now - 200, now - 100, "0.5.8", 915, "MediumFast"],
+        )
+      end
+
+      get "/api/ingestors"
+
+      expect(last_response.status).to eq(200)
+      payload = JSON.parse(last_response.body)
+      expect(payload).to all(include("node_id", "start_time", "last_seen_time", "version"))
+      node_ids = payload.map { |entry| entry["node_id"] }
+      expect(node_ids).to include("!fresh000")
+      expect(node_ids).not_to include("!stale000")
+      rich = payload.find { |row| row["node_id"] == "!rich000" }
+      expect(rich["lora_freq"]).to eq(915)
+      expect(rich["modem_preset"]).to eq("MediumFast")
+      expect(rich["start_time_iso"]).to be_a(String)
+      expect(rich["last_seen_iso"]).to be_a(String)
+    end
+  end
+
+  describe "schema migrations" do
+    it "creates the ingestors table with frequency and modem columns" do
+      tmp_db = File.join(SPEC_TMPDIR, "ingestor-migrate.db")
+      FileUtils.rm_f(tmp_db)
+      original = PotatoMesh::Config.db_path
+      allow(PotatoMesh::Config).to receive(:db_path).and_return(tmp_db)
+
+      begin
+        PotatoMesh::Application.init_db
+        with_db(readonly: true) do |db|
+          columns = db.execute("PRAGMA table_info(ingestors)").map { |row| row[1] }
+          expect(columns).to include("lora_freq", "modem_preset", "version")
+        end
+      ensure
+        allow(PotatoMesh::Config).to receive(:db_path).and_return(original)
+      end
+    end
+  end
+end
@@ -38,6 +38,7 @@ RSpec.describe PotatoMesh::App::Instances do
  before do
    FileUtils.mkdir_p(File.dirname(PotatoMesh::Config.db_path))
    application_class.init_db unless application_class.db_schema_present?
+    application_class.ensure_schema_upgrades
    with_db do |db|
      db.execute("DELETE FROM instances")
    end
@@ -132,5 +133,48 @@ RSpec.describe PotatoMesh::App::Instances do
      expect(with_contact["contactLink"]).to eq("https://example.org/contact")
      expect(without_contact.key?("contactLink")).to be(false)
    end
+
+    it "includes nodesCount values, preserving zeros" do
+      fixed_time = Time.utc(2025, 2, 2, 8, 0, 0)
+      allow(Time).to receive(:now).and_return(fixed_time)
+
+      with_db do |db|
+        db.execute(
+          <<~SQL,
+          INSERT INTO instances (id, domain, pubkey, last_update_time, is_private, nodes_count)
+          VALUES (?, ?, ?, ?, ?, ?)
+        SQL
+          [
+            "instance-with-nodes",
+            "gamma.mesh.test",
+            PotatoMesh::Application::INSTANCE_PUBLIC_KEY_PEM,
+            fixed_time.to_i,
+            0,
+            42,
+          ],
+        )
+        db.execute(
+          <<~SQL,
+          INSERT INTO instances (id, domain, pubkey, last_update_time, is_private, nodes_count)
+          VALUES (?, ?, ?, ?, ?, ?)
+        SQL
+          [
+            "instance-with-zero",
+            "delta.mesh.test",
+            PotatoMesh::Application::INSTANCE_PUBLIC_KEY_PEM,
+            fixed_time.to_i,
+            0,
+            0,
+          ],
+        )
+      end
+
+      payload = application_class.load_instances_for_api
+      with_nodes = payload.find { |row| row["domain"] == "gamma.mesh.test" }
+      zero_nodes = payload.find { |row| row["domain"] == "delta.mesh.test" }
+
+      expect(with_nodes["nodesCount"]).to eq(42)
+      expect(zero_nodes["nodesCount"]).to eq(0)
+    end
  end
 end
@@ -17,15 +17,16 @@
  <table id="instances">
    <thead>
      <tr>
-        <th class="instances-col instances-col--name">Name</th>
-        <th class="instances-col instances-col--domain">Domain</th>
-        <th class="instances-col instances-col--contact">Contact</th>
-        <th class="instances-col instances-col--version">Version</th>
-        <th class="instances-col instances-col--channel">Channel</th>
-        <th class="instances-col instances-col--frequency">Frequency</th>
-        <th class="instances-col instances-col--latitude">Latitude</th>
-        <th class="instances-col instances-col--longitude">Longitude</th>
-        <th class="instances-col instances-col--last-update">Last Update</th>
+        <th class="instances-col instances-col--name" data-sort-key="name"><span class="sort-header" role="button" tabindex="0" data-sort-key="name" data-sort-label="Name">Name <span class="sort-indicator" aria-hidden="true"></span></span></th>
+        <th class="instances-col instances-col--domain" data-sort-key="domain"><span class="sort-header" role="button" tabindex="0" data-sort-key="domain" data-sort-label="Domain">Domain <span class="sort-indicator" aria-hidden="true"></span></span></th>
+        <th class="instances-col instances-col--contact" data-sort-key="contact"><span class="sort-header" role="button" tabindex="0" data-sort-key="contact" data-sort-label="Contact">Contact <span class="sort-indicator" aria-hidden="true"></span></span></th>
+        <th class="instances-col instances-col--version" data-sort-key="version"><span class="sort-header" role="button" tabindex="0" data-sort-key="version" data-sort-label="Version">Version <span class="sort-indicator" aria-hidden="true"></span></span></th>
+        <th class="instances-col instances-col--channel" data-sort-key="channel"><span class="sort-header" role="button" tabindex="0" data-sort-key="channel" data-sort-label="Channel">Channel <span class="sort-indicator" aria-hidden="true"></span></span></th>
+        <th class="instances-col instances-col--frequency" data-sort-key="frequency"><span class="sort-header" role="button" tabindex="0" data-sort-key="frequency" data-sort-label="Frequency">Frequency <span class="sort-indicator" aria-hidden="true"></span></span></th>
+        <th class="instances-col instances-col--nodes" data-sort-key="nodesCount"><span class="sort-header" role="button" tabindex="0" data-sort-key="nodesCount" data-sort-label="Active Nodes (24h)">Active Nodes (24h) <span class="sort-indicator" aria-hidden="true"></span></span></th>
+        <th class="instances-col instances-col--latitude" data-sort-key="latitude"><span class="sort-header" role="button" tabindex="0" data-sort-key="latitude" data-sort-label="Latitude">Latitude <span class="sort-indicator" aria-hidden="true"></span></span></th>
+        <th class="instances-col instances-col--longitude" data-sort-key="longitude"><span class="sort-header" role="button" tabindex="0" data-sort-key="longitude" data-sort-label="Longitude">Longitude <span class="sort-indicator" aria-hidden="true"></span></span></th>
+        <th class="instances-col instances-col--last-update" data-sort-key="lastUpdateTime"><span class="sort-header" role="button" tabindex="0" data-sort-key="lastUpdateTime" data-sort-label="Last Update">Last Update <span class="sort-indicator" aria-hidden="true"></span></span></th>
      </tr>
    </thead>
    <tbody></tbody>
Author	SHA1	Message	Date
l5y	06fb90513f	data: track ingestors heartbeat (#549 ) * data: track ingestors heartbeat * data: address review comments * cover missing unit test vectors * cover missing unit test vectors	2025-12-14 18:42:17 +01:00
l5y	b5eecb1ec1	Harden instance selector navigation URLs (#550 ) * Harden instance selector navigation URLs * Cover malformed instance URL handling	2025-12-14 18:40:41 +01:00
l5y	0e211aebdd	data: hide channels that have been flag for ignoring (#548 ) * data: hide channels that have been flag for ignoring * data: address review comments	2025-12-14 16:47:44 +01:00
l5y	96b62d7e14	web: fix limit when counting remote nodes (#547 )	2025-12-14 15:05:19 +01:00
l5y	baf6ffff0b	web: improve instances map and table view (#546 ) * web: improve instances map and table view * web: address review comments * run rufo	2025-12-14 14:35:55 +01:00
l5y	135de0863c	web: fix traces submission with optional fields on udp (#545 )	2025-12-14 13:27:07 +01:00
l5y	074a61baac	chore: bump version to 0.5.7 (#542 ) * chore: bump version to 0.5.7 * Change version to 0.5.7 in AppFrameworkInfo.plist Updated version numbers to 0.5.7.	2025-12-08 20:39:58 +01:00
l5y	209cc948bf	Handle zero telemetry aggregates (#538 ) * Handle zero telemetry aggregates * Fix telemetry aggregation to drop zero readings	2025-12-08 20:31:32 +01:00
l5y	cc108f2f49	web: fix telemetry api to return current in amperes (#541 ) * web: fix telemetry api to return current in amperes * web: address review comments	2025-12-08 20:18:10 +01:00