refactor: streamline ingestor environment variables (#314)

* refactor: streamline ingestor environment variables

* fix: set connection env var in docker test
This commit is contained in:
l5y authored and GitHub committed 2025-10-13 11:02:33 +02:00
1 parent b63e5328b1
commit 5133e9d498
11 files changed
+115 -49

No files matched your search

+6 -7
View File
@@ -9,12 +9,14 @@
# Generate a secure token: openssl rand -hex 32
API_TOKEN=your-secure-api-token-here
# Meshtastic device path (required for ingestor)
# Common paths:
# Meshtastic connection target (required for ingestor)
# Common serial paths:
# - Linux: /dev/ttyACM0, /dev/ttyUSB0
# - macOS: /dev/cu.usbserial-*
# - Windows (WSL): /dev/ttyS*
MESH_SERIAL=/dev/ttyACM0
# You may also provide an IP:PORT pair (e.g. 192.168.1.20:4403) or a
# Bluetooth address (e.g. ED:4D:9E:95:CF:60).
CONNECTION=/dev/ttyACM0
# =============================================================================
# SITE CUSTOMIZATION
@@ -65,11 +67,8 @@ POTATOMESH_IMAGE_ARCH=linux-amd64
# is unavailable.
# COMPOSE_PROFILES=bridge
# Meshtastic snapshot interval (seconds)
MESH_SNAPSHOT_SECS=60
# Meshtastic channel index (0=primary, 1=secondary, etc.)
MESH_CHANNEL_INDEX=0
CHANNEL_INDEX=0
# Database settings
DB_BUSY_TIMEOUT_MS=5000
+1 -1
View File
@@ -131,7 +131,7 @@ jobs:
docker run --rm --name ingestor-test \
-e POTATOMESH_INSTANCE=http://localhost:41447 \
-e API_TOKEN=test-token \
-e MESH_SERIAL=mock \
-e CONNECTION=mock \
-e DEBUG=1 \
${{ env.REGISTRY }}/${{ env.IMAGE_PREFIX }}-ingestor-linux-amd64:${{ steps.version.outputs.version }} &
sleep 5
+6 -3
View File
@@ -30,7 +30,7 @@ against the web API.
```env
API_TOKEN=replace-with-a-strong-token
SITE_NAME=PotatoMesh Demo
MESH_SERIAL=/dev/ttyACM0
CONNECTION=/dev/ttyACM0
```
Additional environment variables are optional:
@@ -39,8 +39,11 @@ Additional environment variables are optional:
`MAX_NODE_DISTANCE_KM`, and `MATRIX_ROOM` customise the UI.
- `POTATOMESH_INSTANCE` (defaults to `http://web:41447`) lets the ingestor post
to a remote PotatoMesh instance if you do not run both services together.
- `MESH_CHANNEL_INDEX`, `MESH_SNAPSHOT_SECS`, and `DEBUG` adjust ingestor
behaviour.
- `CONNECTION` overrides the default serial device or network endpoint used by
the ingestor.
- `CHANNEL_INDEX` selects the LoRa channel when using serial or Bluetooth
connections.
- `DEBUG` enables verbose logging across the stack.
## Docker Compose file
+5 -5
View File
@@ -132,7 +132,7 @@ to the configured potato-mesh instance.
Check out `mesh.sh` ingestor script in the `./data` directory.
```bash
POTATOMESH_INSTANCE=http://127.0.0.1:41447 API_TOKEN=1eb140fd-cab4-40be-b862-41c607762246 MESH_SERIAL=/dev/ttyACM0 DEBUG=1 ./mesh.sh
POTATOMESH_INSTANCE=http://127.0.0.1:41447 API_TOKEN=1eb140fd-cab4-40be-b862-41c607762246 CONNECTION=/dev/ttyACM0 DEBUG=1 ./mesh.sh
[2025-02-20T12:34:56.789012Z] [potato-mesh] [info] channel=0 context=daemon.main port='41447' target='http://127.0.0.1' Mesh daemon starting
[...]
[2025-02-20T12:34:57.012345Z] [potato-mesh] [debug] context=handlers.upsert_node node_id=!849b7154 short_name='7154' long_name='7154' Queued node upsert payload
@@ -142,10 +142,10 @@ POTATOMESH_INSTANCE=http://127.0.0.1:41447 API_TOKEN=1eb140fd-cab4-40be-b862-41c
Run the script with `POTATOMESH_INSTANCE` and `API_TOKEN` to keep updating
node records and parsing new incoming messages. Enable debug output with `DEBUG=1`,
specify the serial port with `MESH_SERIAL` (default `/dev/ttyACM0`) or set it to an IP
address (for example `192.168.1.20:4403`) to use the Meshtastic TCP interface.
`MESH_SERIAL` also accepts Bluetooth device addresses (e.g., `ED:4D:9E:95:CF:60`)
and the script attempts a BLE connection if available.
specify the connection target with `CONNECTION` (default `/dev/ttyACM0`) or set it to
an IP address (for example `192.168.1.20:4403`) to use the Meshtastic TCP
interface. `CONNECTION` also accepts Bluetooth device addresses (e.g.,
`ED:4D:9E:95:CF:60`) and the script attempts a BLE connection if available.
## Demos
+12 -3
View File
@@ -146,9 +146,18 @@ update_env "MATRIX_ROOM" "\"$MATRIX_ROOM\""
update_env "API_TOKEN" "$API_TOKEN"
update_env "POTATOMESH_IMAGE_ARCH" "$POTATOMESH_IMAGE_ARCH"
# Add other common settings if they don't exist
if ! grep -q "^MESH_SERIAL=" .env; then
echo "MESH_SERIAL=/dev/ttyACM0" >> .env
# Migrate legacy connection settings and ensure defaults exist
if grep -q "^MESH_SERIAL=" .env; then
legacy_connection=$(grep "^MESH_SERIAL=" .env | head -n1 | cut -d'=' -f2-)
if [ -n "$legacy_connection" ] && ! grep -q "^CONNECTION=" .env; then
echo "♻️ Migrating legacy MESH_SERIAL value to CONNECTION"
update_env "CONNECTION" "$legacy_connection"
fi
sed -i.bak '/^MESH_SERIAL=.*/d' .env
fi
if ! grep -q "^CONNECTION=" .env; then
echo "CONNECTION=/dev/ttyACM0" >> .env
fi
if ! grep -q "^DEBUG=" .env; then
+4 -6
View File
@@ -34,9 +34,8 @@ RUN addgroup -S potatomesh && \
USER potatomesh
ENV MESH_SERIAL=/dev/ttyACM0 \
MESH_SNAPSHOT_SECS=60 \
MESH_CHANNEL_INDEX=0 \
ENV CONNECTION=/dev/ttyACM0 \
CHANNEL_INDEX=0 \
DEBUG=0 \
POTATOMESH_INSTANCE="" \
API_TOKEN=""
@@ -60,9 +59,8 @@ COPY data/ .
USER ContainerUser
ENV MESH_SERIAL=/dev/ttyACM0 \
MESH_SNAPSHOT_SECS=60 \
MESH_CHANNEL_INDEX=0 \
ENV CONNECTION=/dev/ttyACM0 \
CHANNEL_INDEX=0 \
DEBUG=0 \
POTATOMESH_INSTANCE="" \
API_TOKEN=""
+4 -1
View File
@@ -46,7 +46,7 @@ for _module in (daemon, handlers, interfaces, queue, serialization):
_export_constants()
_CONFIG_ATTRS = {
"PORT",
"CONNECTION",
"SNAPSHOT_SECS",
"CHANNEL_INDEX",
"DEBUG",
@@ -58,6 +58,9 @@ _CONFIG_ATTRS = {
"_debug_log",
}
# Legacy export maintained for backwards compatibility.
_CONFIG_ATTRS.add("PORT")
_INTERFACE_ATTRS = {"BLEInterface", "SerialInterface", "TCPInterface"}
_QUEUE_ATTRS = set(queue.__all__)
+66 -14
View File
@@ -17,27 +17,63 @@
from __future__ import annotations
import os
import sys
from datetime import datetime, timezone
from types import ModuleType
from typing import Any
PORT = os.environ.get("MESH_SERIAL")
SNAPSHOT_SECS = int(os.environ.get("MESH_SNAPSHOT_SECS", "60"))
CHANNEL_INDEX = int(os.environ.get("MESH_CHANNEL_INDEX", "0"))
DEFAULT_SNAPSHOT_SECS = 60
"""Default interval, in seconds, between state snapshot uploads."""
DEFAULT_CHANNEL_INDEX = 0
"""Default LoRa channel index used when none is specified."""
DEFAULT_RECONNECT_INITIAL_DELAY_SECS = 5.0
"""Initial reconnection delay applied after connection loss."""
DEFAULT_RECONNECT_MAX_DELAY_SECS = 60.0
"""Maximum reconnection backoff delay applied by the ingestor."""
DEFAULT_CLOSE_TIMEOUT_SECS = 5.0
"""Grace period for interface shutdown routines to complete."""
DEFAULT_INACTIVITY_RECONNECT_SECS = float(60 * 60)
"""Interval before forcing a reconnect when no packets are observed."""
DEFAULT_ENERGY_ONLINE_DURATION_SECS = 300.0
"""Duration to stay online before entering a low-power sleep cycle."""
DEFAULT_ENERGY_SLEEP_SECS = float(6 * 60 * 60)
"""Sleep duration used when energy saving mode is active."""
CONNECTION = os.environ.get("CONNECTION") or os.environ.get("MESH_SERIAL")
"""Optional connection target for the mesh interface.
When unset, platform-specific defaults will be inferred by the interface
implementations. The legacy :envvar:`MESH_SERIAL` environment variable is still
accepted for backwards compatibility.
"""
SNAPSHOT_SECS = DEFAULT_SNAPSHOT_SECS
"""Interval, in seconds, between state snapshot uploads."""
CHANNEL_INDEX = int(os.environ.get("CHANNEL_INDEX", str(DEFAULT_CHANNEL_INDEX)))
"""Index of the LoRa channel to select when connecting."""
DEBUG = os.environ.get("DEBUG") == "1"
INSTANCE = os.environ.get("POTATOMESH_INSTANCE", "").rstrip("/")
API_TOKEN = os.environ.get("API_TOKEN", "")
ENERGY_SAVING = os.environ.get("ENERGY_SAVING") == "1"
_RECONNECT_INITIAL_DELAY_SECS = float(os.environ.get("MESH_RECONNECT_INITIAL", "5"))
_RECONNECT_MAX_DELAY_SECS = float(os.environ.get("MESH_RECONNECT_MAX", "60"))
_CLOSE_TIMEOUT_SECS = float(os.environ.get("MESH_CLOSE_TIMEOUT", "5"))
_INACTIVITY_RECONNECT_SECS = float(
os.environ.get("MESH_INACTIVITY_RECONNECT_SECS", str(60 * 60))
)
_ENERGY_ONLINE_DURATION_SECS = float(
os.environ.get("ENERGY_ONLINE_DURATION_SECS", "300")
)
_ENERGY_SLEEP_SECS = float(os.environ.get("ENERGY_SLEEP_SECS", str(6 * 60 * 60)))
_RECONNECT_INITIAL_DELAY_SECS = DEFAULT_RECONNECT_INITIAL_DELAY_SECS
_RECONNECT_MAX_DELAY_SECS = DEFAULT_RECONNECT_MAX_DELAY_SECS
_CLOSE_TIMEOUT_SECS = DEFAULT_CLOSE_TIMEOUT_SECS
_INACTIVITY_RECONNECT_SECS = DEFAULT_INACTIVITY_RECONNECT_SECS
_ENERGY_ONLINE_DURATION_SECS = DEFAULT_ENERGY_ONLINE_DURATION_SECS
_ENERGY_SLEEP_SECS = DEFAULT_ENERGY_SLEEP_SECS
# Backwards compatibility shim for legacy imports.
PORT = CONNECTION
def _debug_log(
@@ -75,7 +111,7 @@ def _debug_log(
__all__ = [
"PORT",
"CONNECTION",
"SNAPSHOT_SECS",
"CHANNEL_INDEX",
"DEBUG",
@@ -90,3 +126,19 @@ __all__ = [
"_ENERGY_SLEEP_SECS",
"_debug_log",
]
class _ConfigModule(ModuleType):
"""Module proxy that keeps connection aliases synchronised."""
def __setattr__(self, name: str, value: Any) -> None: # type: ignore[override]
"""Propagate CONNECTION/PORT assignments to both attributes."""
if name in {"CONNECTION", "PORT"}:
super().__setattr__("CONNECTION", value)
super().__setattr__("PORT", value)
return
super().__setattr__(name, value)
sys.modules[__name__].__class__ = _ConfigModule
+1 -1
View File
@@ -219,7 +219,7 @@ def main() -> None:
signal.signal(signal.SIGTERM, handle_sigterm)
target = config.INSTANCE or "(no POTATOMESH_INSTANCE)"
configured_port = config.PORT
configured_port = config.CONNECTION
active_candidate = configured_port
announced_target = False
config._debug_log(
+3 -4
View File
@@ -29,9 +29,8 @@ x-web-base: &web-base
x-ingestor-base: &ingestor-base
image: ghcr.io/l5yth/potato-mesh-ingestor-${POTATOMESH_IMAGE_ARCH:-linux-amd64}:latest
environment:
MESH_SERIAL: ${MESH_SERIAL:-/dev/ttyACM0}
MESH_SNAPSHOT_SECS: ${MESH_SNAPSHOT_SECS:-60}
MESH_CHANNEL_INDEX: ${MESH_CHANNEL_INDEX:-0}
CONNECTION: ${CONNECTION:-/dev/ttyACM0}
CHANNEL_INDEX: ${CHANNEL_INDEX:-0}
POTATOMESH_INSTANCE: ${POTATOMESH_INSTANCE:-http://web:41447}
API_TOKEN: ${API_TOKEN}
DEBUG: ${DEBUG:-0}
@@ -39,7 +38,7 @@ x-ingestor-base: &ingestor-base
- potatomesh_data:/app/data
- potatomesh_logs:/app/logs
devices:
- ${MESH_SERIAL:-/dev/ttyACM0}:${MESH_SERIAL:-/dev/ttyACM0}
- ${CONNECTION:-/dev/ttyACM0}:${CONNECTION:-/dev/ttyACM0}
privileged: false
restart: unless-stopped
deploy:
+7 -4
View File
@@ -28,7 +28,10 @@ from meshtastic.mesh_interface import MeshInterface
from meshtastic.serial_interface import SerialInterface
from pubsub import pub
PORT = os.environ.get("MESH_SERIAL", "/dev/ttyACM0")
CONNECTION = os.environ.get("CONNECTION") or os.environ.get(
"MESH_SERIAL", "/dev/ttyACM0"
)
"""Connection target opened to capture Meshtastic traffic."""
OUT = os.environ.get("MESH_DUMP_FILE", "meshtastic-dump.ndjson")
# line-buffered append so you can tail -f safely
@@ -54,7 +57,7 @@ def write(kind: str, payload: dict) -> None:
# Connect to the node
iface: MeshInterface = SerialInterface(PORT)
iface: MeshInterface = SerialInterface(CONNECTION)
# Packet callback: every RF/Mesh packet the node receives/decodes lands here
@@ -92,12 +95,12 @@ try:
"meta",
{
"event": "started",
"port": PORT,
"port": CONNECTION,
"my_node_num": getattr(my, "my_node_num", None) if my else None,
},
)
except Exception as e:
write("meta", {"event": "started", "port": PORT, "error": str(e)})
write("meta", {"event": "started", "port": CONNECTION, "error": str(e)})
# Keep the process alive until Ctrl-C