first cut (#651)

data/mesh_ingestor/CONTRACTS.md

@@ -0,0 +1,107 @@
+## Mesh ingestor contracts (stable interfaces)
+
+This repo’s ingestion pipeline is split into:
+
+- **Python collector** (`data/mesh_ingestor/*`) which normalizes packets/events and POSTs JSON to the web app.
+- **Sinatra web app** (`web/`) which accepts those payloads on `POST /api/*` ingest routes and persists them into SQLite tables defined under `data/*.sql`.
+
+This document records the **contracts that future providers must preserve**. The intent is to enable adding new providers (MeshCore, Reticulum, …) without changing the Ruby/DB/UI read-side.
+
+### Canonical node identity
+
+- **Canonical node id**: `nodes.node_id` is a `TEXT` primary key and is treated as canonical across the system.
+- **Format**: `!%08x` (lowercase hex, 8 chars), for example `!abcdef01`.
+- **Normalization**:
+  - Python currently normalizes via `data/mesh_ingestor/serialization.py:_canonical_node_id`.
+  - Ruby normalizes via `web/lib/potato_mesh/application/data_processing.rb:canonical_node_parts`.
+- **Dual addressing**: Ruby routes and queries accept either a canonical `!xxxxxxxx` string or a numeric node id; they normalize to `node_id`.
+
+Note: non-Meshtastic providers will need a strategy to map their native node identifiers into this `!%08x` space. That mapping is intentionally not standardized in code yet.
+
+### Ingest HTTP routes and payload shapes
+
+Future providers should emit payloads that match these shapes (keys + types), which are validated by existing tests (notably `tests/test_mesh.py`).
+
+#### `POST /api/nodes`
+
+Payload is a mapping keyed by canonical node id:
+
+- `{ "!abcdef01": { ... node fields ... } }`
+
+Node entry fields are “Meshtastic-ish” (camelCase) and may include:
+
+- `num` (int node number)
+- `lastHeard` (int unix seconds)
+- `snr` (float)
+- `hopsAway` (int)
+- `isFavorite` (bool)
+- `user` (mapping; e.g. `shortName`, `longName`, `macaddr`, `hwModel`, `role`, `publicKey`, `isUnmessagable`)
+- `deviceMetrics` (mapping; e.g. `batteryLevel`, `voltage`, `channelUtilization`, `airUtilTx`, `uptimeSeconds`)
+- `position` (mapping; `latitude`, `longitude`, `altitude`, `time`, `locationSource`, `precisionBits`, optional nested `raw`)
+- Optional radio metadata: `lora_freq`, `modem_preset`
+
+#### `POST /api/messages`
+
+Single message payload:
+
+- Required: `id` (int), `rx_time` (int), `rx_iso` (string)
+- Identity: `from_id` (string/int), `to_id` (string/int), `channel` (int), `portnum` (string|nil)
+- Payload: `text` (string|nil), `encrypted` (string|nil), `reply_id` (int|nil), `emoji` (string|nil)
+- RF: `snr` (float|nil), `rssi` (int|nil), `hop_limit` (int|nil)
+- Meta: `channel_name` (string; only when not encrypted and known), `ingestor` (canonical host id), `lora_freq`, `modem_preset`
+
+#### `POST /api/positions`
+
+Single position payload:
+
+- Required: `id` (int), `rx_time` (int), `rx_iso` (string)
+- Node: `node_id` (canonical string), `node_num` (int|nil), `num` (int|nil), `from_id` (canonical string), `to_id` (string|nil)
+- Position: `latitude`, `longitude`, `altitude` (floats|nil)
+- Position time: `position_time` (int|nil)
+- Quality: `location_source` (string|nil), `precision_bits` (int|nil), `sats_in_view` (int|nil), `pdop` (float|nil)
+- Motion: `ground_speed` (float|nil), `ground_track` (float|nil)
+- RF/meta: `snr`, `rssi`, `hop_limit`, `bitfield`, `payload_b64` (string|nil), `raw` (mapping|nil), `ingestor`, `lora_freq`, `modem_preset`
+
+#### `POST /api/telemetry`
+
+Single telemetry payload:
+
+- Required: `id` (int), `rx_time` (int), `rx_iso` (string)
+- Node: `node_id` (canonical string|nil), `node_num` (int|nil), `from_id`, `to_id`
+- Time: `telemetry_time` (int|nil)
+- Packet: `channel` (int), `portnum` (string|nil), `bitfield` (int|nil), `hop_limit` (int|nil)
+- RF: `snr` (float|nil), `rssi` (int|nil)
+- Raw: `payload_b64` (string; may be empty string when unknown)
+- Metrics: many optional snake_case keys (`battery_level`, `voltage`, `temperature`, etc.)
+- Meta: `ingestor`, `lora_freq`, `modem_preset`
+
+#### `POST /api/neighbors`
+
+Neighbors snapshot payload:
+
+- Node: `node_id` (canonical string), `node_num` (int|nil)
+- `neighbors`: list of entries with `neighbor_id` (canonical string), `neighbor_num` (int|nil), `snr` (float|nil), `rx_time` (int), `rx_iso` (string)
+- Snapshot time: `rx_time`, `rx_iso`
+- Optional: `node_broadcast_interval_secs` (int|nil), `last_sent_by_id` (canonical string|nil)
+- Meta: `ingestor`, `lora_freq`, `modem_preset`
+
+#### `POST /api/traces`
+
+Single trace payload:
+
+- Identity: `id` (int|nil), `request_id` (int|nil)
+- Endpoints: `src` (int|nil), `dest` (int|nil)
+- Path: `hops` (list[int])
+- Time: `rx_time` (int), `rx_iso` (string)
+- Metrics: `rssi` (int|nil), `snr` (float|nil), `elapsed_ms` (int|nil)
+- Meta: `ingestor`, `lora_freq`, `modem_preset`
+
+#### `POST /api/ingestors`
+
+Heartbeat payload:
+
+- `node_id` (canonical string)
+- `start_time` (int), `last_seen_time` (int)
+- `version` (string)
+- Optional: `lora_freq`, `modem_preset`
+

data/mesh_ingestor/daemon.py

@@ -24,6 +24,8 @@ import time
 from pubsub import pub
 
 from . import config, handlers, ingestors, interfaces
+from .provider import Provider
+from .providers.meshtastic import MeshtasticProvider
 
 _RECEIVE_TOPICS = (
     "meshtastic.receive",
@@ -243,10 +245,12 @@ def _connected_state(candidate) -> bool | None:
         return None
 
 
-def main(existing_interface=None) -> None:
+def main(existing_interface=None, *, provider: Provider | None = None) -> None:
     """Run the mesh ingestion daemon until interrupted."""
 
-    subscribed = _subscribe_receive_topics()
+    provider = provider or MeshtasticProvider()
+
+    subscribed = provider.subscribe()
     if subscribed:
         config._debug_log(
             "Subscribed to receive topics",
@@ -312,17 +316,11 @@ def main(existing_interface=None) -> None:
         while not stop.is_set():
             if iface is None:
                 try:
-                    if active_candidate:
+                    iface, resolved_target, active_candidate = provider.connect(
-                        iface, resolved_target = interfaces._create_serial_interface(
+                        active_candidate=active_candidate
-                            active_candidate
+                    )
-                        )
-                    else:
-                        iface, resolved_target = interfaces._create_default_interface()
-                        active_candidate = resolved_target
-                    interfaces._ensure_radio_metadata(iface)
-                    interfaces._ensure_channel_metadata(iface)
                     handlers.register_host_node_id(
-                        interfaces._extract_host_node_id(iface)
+                        provider.extract_host_node_id(iface)
                     )
                     ingestors.set_ingestor_node_id(handlers.host_node_id())
                     retry_delay = max(0.0, config._RECONNECT_INITIAL_DELAY_SECS)
@@ -415,8 +413,8 @@ def main(existing_interface=None) -> None:
 
             if not initial_snapshot_sent:
                 try:
-                    nodes = getattr(iface, "nodes", {}) or {}
+                    node_items = list(provider.node_snapshot_items(iface))
-                    node_items = _node_items_snapshot(nodes)
+                    node_items = _node_items_snapshot(dict(node_items))
                     if node_items is None:
                         config._debug_log(
                             "Skipping node snapshot due to concurrent modification",

data/mesh_ingestor/events.py

@@ -0,0 +1,164 @@
+# Copyright © 2025-26 l5yth & contributors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Protocol-agnostic event payload types for ingestion.
+
+The ingestor ultimately POSTs JSON to the web app's ingest routes. These types
+capture the *shape* of those payloads so multiple providers can emit the same
+events, regardless of how they source or decode packets.
+
+These are intentionally defined as ``TypedDict`` so existing code can continue
+to build plain dictionaries without a runtime dependency on dataclasses.
+"""
+
+from __future__ import annotations
+
+from typing import NotRequired, TypedDict
+
+
+class MessageEvent(TypedDict, total=False):
+    id: int
+    rx_time: int
+    rx_iso: str
+    from_id: object
+    to_id: object
+    channel: int
+    portnum: str | None
+    text: str | None
+    encrypted: str | None
+    snr: float | None
+    rssi: int | None
+    hop_limit: int | None
+    reply_id: int | None
+    emoji: str | None
+    channel_name: str
+    ingestor: str | None
+    lora_freq: int
+    modem_preset: str
+
+
+class PositionEvent(TypedDict, total=False):
+    id: int
+    node_id: str
+    node_num: int | None
+    num: int | None
+    from_id: str | None
+    to_id: object
+    rx_time: int
+    rx_iso: str
+    latitude: float | None
+    longitude: float | None
+    altitude: float | None
+    position_time: int | None
+    location_source: str | None
+    precision_bits: int | None
+    sats_in_view: int | None
+    pdop: float | None
+    ground_speed: float | None
+    ground_track: float | None
+    snr: float | None
+    rssi: int | None
+    hop_limit: int | None
+    bitfield: int | None
+    payload_b64: str | None
+    raw: dict
+    ingestor: str | None
+    lora_freq: int
+    modem_preset: str
+
+
+class TelemetryEvent(TypedDict, total=False):
+    id: int
+    node_id: str | None
+    node_num: int | None
+    from_id: object
+    to_id: object
+    rx_time: int
+    rx_iso: str
+    telemetry_time: int | None
+    channel: int
+    portnum: str | None
+    hop_limit: int | None
+    snr: float | None
+    rssi: int | None
+    bitfield: int | None
+    payload_b64: str
+    ingestor: str | None
+    lora_freq: int
+    modem_preset: str
+
+    # Metric keys are intentionally open-ended; the Ruby side is permissive and
+    # evolves over time.
+
+
+class NeighborEntry(TypedDict, total=False):
+    neighbor_id: str
+    neighbor_num: int | None
+    snr: float | None
+    rx_time: int
+    rx_iso: str
+
+
+class NeighborsSnapshot(TypedDict, total=False):
+    node_id: str
+    node_num: int | None
+    neighbors: list[NeighborEntry]
+    rx_time: int
+    rx_iso: str
+    node_broadcast_interval_secs: int | None
+    last_sent_by_id: str | None
+    ingestor: str | None
+    lora_freq: int
+    modem_preset: str
+
+
+class TraceEvent(TypedDict, total=False):
+    id: int | None
+    request_id: int | None
+    src: int | None
+    dest: int | None
+    rx_time: int
+    rx_iso: str
+    hops: list[int]
+    rssi: int | None
+    snr: float | None
+    elapsed_ms: int | None
+    ingestor: str | None
+    lora_freq: int
+    modem_preset: str
+
+
+class IngestorHeartbeat(TypedDict):
+    node_id: str
+    start_time: int
+    last_seen_time: int
+    version: str
+    lora_freq: NotRequired[int]
+    modem_preset: NotRequired[str]
+
+
+NodeUpsert = dict[str, dict]
+
+
+__all__ = [
+    "IngestorHeartbeat",
+    "MessageEvent",
+    "NeighborEntry",
+    "NeighborsSnapshot",
+    "NodeUpsert",
+    "PositionEvent",
+    "TelemetryEvent",
+    "TraceEvent",
+]
+

data/mesh_ingestor/node_identity.py

@@ -0,0 +1,116 @@
+# Copyright © 2025-26 l5yth & contributors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Node identity helpers shared across ingestor providers.
+
+The web application keys nodes by a canonical textual identifier of the form
+``!%08x`` (lowercase hex). Both the Python collector and Ruby server accept
+several input forms (ints, ``0x`` hex strings, ``!`` hex strings, decimal
+strings). This module centralizes that normalization.
+
+"""
+
+from __future__ import annotations
+
+from typing import Final
+
+
+_CANONICAL_PREFIX: Final[str] = "!"
+
+
+def canonical_node_id(value: object) -> str | None:
+    """Convert ``value`` into canonical ``!xxxxxxxx`` form.
+
+    Parameters:
+        value: Node reference which may be an int, float, or string.
+
+    Returns:
+        Canonical node id string or ``None`` when parsing fails.
+    """
+
+    if value is None:
+        return None
+    if isinstance(value, (int, float)):
+        try:
+            num = int(value)
+        except (TypeError, ValueError):
+            return None
+        if num < 0:
+            return None
+        return f"{_CANONICAL_PREFIX}{num & 0xFFFFFFFF:08x}"
+    if not isinstance(value, str):
+        return None
+
+    trimmed = value.strip()
+    if not trimmed:
+        return None
+    if trimmed.startswith("^"):
+        # Meshtastic special destinations like "^all" are not node ids; callers
+        # that already accept them should keep passing them through unchanged.
+        return trimmed
+    if trimmed.startswith(_CANONICAL_PREFIX):
+        body = trimmed[1:]
+    elif trimmed.lower().startswith("0x"):
+        body = trimmed[2:]
+    elif trimmed.isdigit():
+        try:
+            return f"{_CANONICAL_PREFIX}{int(trimmed, 10) & 0xFFFFFFFF:08x}"
+        except ValueError:
+            return None
+    else:
+        body = trimmed
+
+    if not body:
+        return None
+    try:
+        return f"{_CANONICAL_PREFIX}{int(body, 16) & 0xFFFFFFFF:08x}"
+    except ValueError:
+        return None
+
+
+def node_num_from_id(node_id: object) -> int | None:
+    """Extract the numeric node identifier from a canonical (or near-canonical) id."""
+
+    if node_id is None:
+        return None
+    if isinstance(node_id, (int, float)):
+        try:
+            num = int(node_id)
+        except (TypeError, ValueError):
+            return None
+        return num if num >= 0 else None
+    if not isinstance(node_id, str):
+        return None
+
+    trimmed = node_id.strip()
+    if not trimmed:
+        return None
+    if trimmed.startswith(_CANONICAL_PREFIX):
+        trimmed = trimmed[1:]
+    if trimmed.lower().startswith("0x"):
+        trimmed = trimmed[2:]
+    try:
+        return int(trimmed, 16)
+    except ValueError:
+        try:
+            return int(trimmed, 10)
+        except ValueError:
+            return None
+
+
+__all__ = [
+    "canonical_node_id",
+    "node_num_from_id",
+]
+

data/mesh_ingestor/provider.py

@@ -0,0 +1,65 @@
+# Copyright © 2025-26 l5yth & contributors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Provider interface for ingestion sources.
+
+Today the repo ships a Meshtastic provider only. This module defines the seam so
+future providers (MeshCore, Reticulum, ...) can be added without changing the
+web app ingest contract.
+"""
+
+from __future__ import annotations
+
+import enum
+from collections.abc import Iterable
+from typing import Protocol
+
+class ProviderCapability(enum.Flag):
+    """Feature flags describing what a provider can supply."""
+
+    NONE = 0
+    NODE_SNAPSHOT = enum.auto()
+    HEARTBEATS = enum.auto()
+
+
+class Provider(Protocol):
+    """Abstract source of mesh observations."""
+
+    name: str
+    capabilities: ProviderCapability
+
+    def subscribe(self) -> list[str]:
+        """Subscribe to any async receive callbacks and return topic names."""
+
+    def connect(
+        self, *, active_candidate: str | None
+    ) -> tuple[object, str | None, str | None]:
+        """Create an interface connection.
+
+        Returns:
+            (iface, resolved_target, next_active_candidate)
+        """
+
+    def extract_host_node_id(self, iface: object) -> str | None:
+        """Best-effort extraction of the connected host node id."""
+
+    def node_snapshot_items(self, iface: object) -> Iterable[tuple[str, object]]:
+        """Return iterable of (node_id, node_obj) for initial snapshot."""
+
+
+__all__ = [
+    "Provider",
+    "ProviderCapability",
+]
+

data/mesh_ingestor/providers/__init__.py

@@ -0,0 +1,26 @@
+# Copyright © 2025-26 l5yth & contributors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Provider implementations.
+
+This package contains protocol-specific provider implementations (Meshtastic
+today, others in the future).
+"""
+
+from __future__ import annotations
+
+from .meshtastic import MeshtasticProvider
+
+__all__ = ["MeshtasticProvider"]
+

data/mesh_ingestor/providers/meshtastic.py

@@ -0,0 +1,84 @@
+# Copyright © 2025-26 l5yth & contributors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Meshtastic provider implementation."""
+
+from __future__ import annotations
+
+from collections.abc import Iterable
+
+from .. import interfaces
+from ..provider import ProviderCapability
+
+
+class MeshtasticProvider:
+    """Meshtastic ingestion provider (current default)."""
+
+    name = "meshtastic"
+    capabilities = ProviderCapability.NODE_SNAPSHOT | ProviderCapability.HEARTBEATS
+
+    def __init__(self):
+        self._subscribed: list[str] = []
+
+    def subscribe(self) -> list[str]:
+        """Subscribe Meshtastic pubsub receive topics."""
+
+        if self._subscribed:
+            return list(self._subscribed)
+
+        # Delegate to the historical subscription helper in `daemon.py` so unit
+        # tests can monkeypatch the subscription mechanism via `daemon.pub`.
+        from .. import daemon as _daemon  # local import avoids module cycles
+
+        topics = _daemon._subscribe_receive_topics()
+
+        self._subscribed = topics
+        return list(topics)
+
+    def connect(
+        self, *, active_candidate: str | None
+    ) -> tuple[object, str | None, str | None]:
+        """Create a Meshtastic interface using the existing interface helpers."""
+
+        iface = None
+        resolved_target = None
+        next_candidate = active_candidate
+
+        if active_candidate:
+            iface, resolved_target = interfaces._create_serial_interface(active_candidate)
+        else:
+            iface, resolved_target = interfaces._create_default_interface()
+            next_candidate = resolved_target
+
+        interfaces._ensure_radio_metadata(iface)
+        interfaces._ensure_channel_metadata(iface)
+
+        return iface, resolved_target, next_candidate
+
+    def extract_host_node_id(self, iface: object) -> str | None:
+        return interfaces._extract_host_node_id(iface)
+
+    def node_snapshot_items(self, iface: object) -> Iterable[tuple[str, object]]:
+        nodes = getattr(iface, "nodes", {}) or {}
+        items_callable = getattr(nodes, "items", None)
+        if callable(items_callable):
+            return list(items_callable())
+        if hasattr(nodes, "__iter__") and hasattr(nodes, "__getitem__"):
+            keys = list(nodes)
+            return [(key, nodes[key]) for key in keys]
+        return []
+
+
+__all__ = ["MeshtasticProvider"]
+

data/mesh_ingestor/serialization.py

@@ -33,6 +33,9 @@ from google.protobuf.json_format import MessageToDict
 from google.protobuf.message import DecodeError
 from google.protobuf.message import Message as ProtoMessage
 
+from .node_identity import canonical_node_id as _canonical_node_id
+from .node_identity import node_num_from_id as _node_num_from_id
+
 _CLI_ROLE_MODULE_NAMES: tuple[str, ...] = (
     "meshtastic.cli.common",
     "meshtastic.cli.roles",
@@ -429,91 +432,6 @@ def _pkt_to_dict(packet) -> dict:
         return {"_unparsed": str(packet)}
 
 
-def _canonical_node_id(value) -> str | None:
-    """Convert node identifiers into the canonical ``!xxxxxxxx`` format.
-
-    Parameters:
-        value: Input identifier which may be an int, float or string.
-
-    Returns:
-        The canonical identifier or ``None`` if conversion fails.
-    """
-
-    if value is None:
-        return None
-    if isinstance(value, (int, float)):
-        try:
-            num = int(value)
-        except (TypeError, ValueError):
-            return None
-        if num < 0:
-            return None
-        return f"!{num & 0xFFFFFFFF:08x}"
-    if not isinstance(value, str):
-        return None
-
-    trimmed = value.strip()
-    if not trimmed:
-        return None
-    if trimmed.startswith("^"):
-        return trimmed
-    if trimmed.startswith("!"):
-        body = trimmed[1:]
-    elif trimmed.lower().startswith("0x"):
-        body = trimmed[2:]
-    elif trimmed.isdigit():
-        try:
-            return f"!{int(trimmed, 10) & 0xFFFFFFFF:08x}"
-        except ValueError:
-            return None
-    else:
-        body = trimmed
-
-    if not body:
-        return None
-    try:
-        return f"!{int(body, 16) & 0xFFFFFFFF:08x}"
-    except ValueError:
-        return None
-
-
-def _node_num_from_id(node_id) -> int | None:
-    """Extract the numeric node ID from a canonical identifier.
-
-    Parameters:
-        node_id: Identifier value accepted by :func:`_canonical_node_id`.
-
-    Returns:
-        The numeric node ID or ``None`` when parsing fails.
-    """
-
-    if node_id is None:
-        return None
-    if isinstance(node_id, (int, float)):
-        try:
-            num = int(node_id)
-        except (TypeError, ValueError):
-            return None
-        return num if num >= 0 else None
-    if not isinstance(node_id, str):
-        return None
-
-    trimmed = node_id.strip()
-    if not trimmed:
-        return None
-    if trimmed.startswith("!"):
-        trimmed = trimmed[1:]
-    if trimmed.lower().startswith("0x"):
-        trimmed = trimmed[2:]
-    try:
-        return int(trimmed, 16)
-    except ValueError:
-        try:
-            return int(trimmed, 10)
-        except ValueError:
-            return None
-
-
 def _merge_mappings(base, extra):
     """Merge two mapping-like objects recursively.
 

tests/test_decode_payload.py

@@ -19,8 +19,10 @@ import io
 import json
 import sys
 
-from meshtastic.protobuf import mesh_pb2
+import pytest
-from meshtastic.protobuf import telemetry_pb2
+
+mesh_pb2 = pytest.importorskip("meshtastic.protobuf.mesh_pb2")
+telemetry_pb2 = pytest.importorskip("meshtastic.protobuf.telemetry_pb2")
 
 from data.mesh_ingestor import decode_payload
 

tests/test_node_identity_unit.py

@@ -0,0 +1,54 @@
+# Copyright © 2025-26 l5yth & contributors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Unit tests for :mod:`data.mesh_ingestor.node_identity`."""
+
+from __future__ import annotations
+
+import sys
+from pathlib import Path
+
+REPO_ROOT = Path(__file__).resolve().parents[1]
+if str(REPO_ROOT) not in sys.path:
+    sys.path.insert(0, str(REPO_ROOT))
+
+from data.mesh_ingestor.node_identity import (  # noqa: E402 - path setup
+    canonical_node_id,
+    node_num_from_id,
+)
+
+
+def test_canonical_node_id_accepts_numeric():
+    assert canonical_node_id(1) == "!00000001"
+    assert canonical_node_id(0xABCDEF01) == "!abcdef01"
+    assert canonical_node_id(1.0) == "!00000001"
+
+
+def test_canonical_node_id_accepts_string_forms():
+    assert canonical_node_id("!ABCDEF01") == "!abcdef01"
+    assert canonical_node_id("0xABCDEF01") == "!abcdef01"
+    assert canonical_node_id("abcdef01") == "!abcdef01"
+    assert canonical_node_id("123") == "!0000007b"
+
+
+def test_canonical_node_id_passthrough_caret_destinations():
+    assert canonical_node_id("^all") == "^all"
+
+
+def test_node_num_from_id_parses_canonical_and_hex():
+    assert node_num_from_id("!abcdef01") == 0xABCDEF01
+    assert node_num_from_id("abcdef01") == 0xABCDEF01
+    assert node_num_from_id("0xabcdef01") == 0xABCDEF01
+    assert node_num_from_id(123) == 123
+
+

tests/test_provider_unit.py

@@ -0,0 +1,101 @@
+# Copyright © 2025-26 l5yth & contributors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Unit tests for :mod:`data.mesh_ingestor.provider` integration seams."""
+
+from __future__ import annotations
+
+import sys
+import types
+from pathlib import Path
+
+import pytest
+
+REPO_ROOT = Path(__file__).resolve().parents[1]
+if str(REPO_ROOT) not in sys.path:
+    sys.path.insert(0, str(REPO_ROOT))
+
+from data.mesh_ingestor import daemon  # noqa: E402 - path setup
+from data.mesh_ingestor.providers.meshtastic import (  # noqa: E402 - path setup
+    MeshtasticProvider,
+)
+
+
+def test_daemon_main_uses_provider_connect(monkeypatch):
+    calls = {"connect": 0}
+
+    class FakeProvider(MeshtasticProvider):
+        def subscribe(self):
+            return []
+
+        def connect(self, *, active_candidate):  # type: ignore[override]
+            calls["connect"] += 1
+            # Return a minimal iface and stop immediately via Event.
+            class Iface:
+                nodes = {}
+
+                def close(self):
+                    return None
+
+            return Iface(), "serial0", active_candidate
+
+        def extract_host_node_id(self, iface):  # type: ignore[override]
+            return "!host"
+
+        def node_snapshot_items(self, iface):  # type: ignore[override]
+            return []
+
+    # Make the loop exit quickly.
+    class AutoStopEvent:
+        def __init__(self):
+            self._set = False
+
+        def set(self):
+            self._set = True
+
+        def is_set(self):
+            return self._set
+
+        def wait(self, _timeout=None):
+            self._set = True
+            return True
+
+    monkeypatch.setattr(daemon.config, "SNAPSHOT_SECS", 0)
+    monkeypatch.setattr(daemon.config, "_RECONNECT_INITIAL_DELAY_SECS", 0)
+    monkeypatch.setattr(daemon.config, "_RECONNECT_MAX_DELAY_SECS", 0)
+    monkeypatch.setattr(daemon.config, "_CLOSE_TIMEOUT_SECS", 0)
+    monkeypatch.setattr(daemon.config, "_INGESTOR_HEARTBEAT_SECS", 0)
+    monkeypatch.setattr(daemon.config, "ENERGY_SAVING", False)
+    monkeypatch.setattr(daemon.config, "_INACTIVITY_RECONNECT_SECS", 0)
+    monkeypatch.setattr(daemon.config, "CONNECTION", "serial0")
+
+    monkeypatch.setattr(
+        daemon,
+        "threading",
+        types.SimpleNamespace(
+            Event=AutoStopEvent,
+            current_thread=daemon.threading.current_thread,
+            main_thread=daemon.threading.main_thread,
+        ),
+    )
+
+    monkeypatch.setattr(daemon.handlers, "register_host_node_id", lambda *_a, **_k: None)
+    monkeypatch.setattr(daemon.handlers, "host_node_id", lambda: "!host")
+    monkeypatch.setattr(daemon.handlers, "upsert_node", lambda *_a, **_k: None)
+    monkeypatch.setattr(daemon.handlers, "last_packet_monotonic", lambda: None)
+    monkeypatch.setattr(daemon.ingestors, "set_ingestor_node_id", lambda *_a, **_k: None)
+    monkeypatch.setattr(daemon.ingestors, "queue_ingestor_heartbeat", lambda *_a, **_k: True)
+
+    daemon.main(provider=FakeProvider())
+    assert calls["connect"] >= 1
+