12 Commits

Author SHA1 Message Date
Jack Kingsman
9e68544fe9 Add warning to MQTT section 2026-03-02 14:41:49 -08:00
Jack Kingsman
f059756064 Clearer labelling and page organization for MQTT 2026-03-02 14:25:44 -08:00
Jack Kingsman
95bacc4caf Split up community broker fields and reformat MQTT config page 2026-03-02 14:24:20 -08:00
Jack Kingsman
2581cc6af7 Show error toast on PK export failure 2026-03-02 14:24:20 -08:00
Jack Kingsman
05df314619 Refactor to combined base for MQTT 2026-03-02 14:24:19 -08:00
Jack Kingsman
00ca4afa8d Add support for community MQTT ingest 2026-03-02 14:24:19 -08:00
Jack Kingsman
2496d70c4b Add kofi link 2026-03-02 11:46:09 -08:00
Jack Kingsman
4b05dc2f41 Add clearer MQTT topics and payload shapes 2026-03-02 11:41:25 -08:00
Jack Kingsman
b8cdae8a03 Tag mesh-traffic-reliant tests with a warning 2026-03-02 10:52:48 -08:00
Jack Kingsman
3bad3cb21c Add clearer message for e2e test lags 2026-03-02 10:46:48 -08:00
Jack Kingsman
f118d5e222 Add debug log level info 2026-03-02 10:38:40 -08:00
Jack Kingsman
e0d87c4df3 Add licenses explicitly (probably should have been doing this for a while; oops! Apologies!) 2026-03-01 22:12:42 -08:00
35 changed files with 3668 additions and 332 deletions

View File

@@ -29,6 +29,7 @@ frontend/src/test/
# Docs
*.md
!README.md
!LICENSES.md
# Other
references/

View File

@@ -152,7 +152,9 @@ This message-layer echo/path handling is independent of raw-packet storage dedup
│ ├── event_handlers.py # Radio events
│ ├── decoder.py # Packet decryption
│ ├── websocket.py # Real-time broadcasts
── mqtt.py # Optional MQTT publisher
── mqtt_base.py # Shared MQTT publisher base class (lifecycle, reconnect, backoff)
│ ├── mqtt.py # Private MQTT publisher
│ └── community_mqtt.py # Community MQTT publisher (raw packet sharing)
├── frontend/ # React frontend
│ ├── AGENTS.md # Frontend documentation
│ ├── src/
@@ -233,6 +235,7 @@ Key test files:
- `tests/test_rx_log_data.py` - on_rx_log_data event handler integration
- `tests/test_ack_tracking_wiring.py` - DM ACK tracking extraction and wiring
- `tests/test_health_mqtt_status.py` - Health endpoint MQTT status field
- `tests/test_community_mqtt.py` - Community MQTT publisher (JWT, packet format, hash, broadcast)
### Frontend (Vitest)
@@ -357,10 +360,21 @@ Optional MQTT integration forwards mesh events to an external broker for home au
- `meshcore/raw/gm:<channel_key>` — raw packet attributed to a channel
- `meshcore/raw/unrouted` — raw packets that couldn't be attributed
**Architecture**: `broadcast_event()` in `websocket.py` calls `mqtt_broadcast()` — a single hook covering all message and raw_packet broadcasts. The `MqttPublisher` in `app/mqtt.py` manages a background connection loop with auto-reconnect and backoff. Publishes are fire-and-forget (silent drop if disconnected). Connection state changes trigger toasts via `broadcast_error`/`broadcast_success`. The health endpoint includes `mqtt_status`.
**Architecture**: `broadcast_event()` in `websocket.py` calls `mqtt_broadcast()` — a single hook covering all message and raw_packet broadcasts. The `MqttPublisher` in `app/mqtt.py` manages a background connection loop with auto-reconnect and backoff. Publishes are fire-and-forget (silent drop if disconnected). Connection state changes trigger toasts via `broadcast_error`/`broadcast_success`. The health endpoint includes `mqtt_status` (`disabled` when no broker host is set, or when both publish toggles are off).
**Security**: MQTT password stored in plaintext in SQLite, consistent with the project's trusted-network design.
### Community MQTT Sharing
Separate from private MQTT, the community publisher (`app/community_mqtt.py`) shares raw packets with the MeshCore community aggregator for coverage mapping and analysis. Only raw packets are shared — never decrypted messages.
- Connects to community broker (default `mqtt-us-v1.letsmesh.net:443`) via WebSockets over TLS.
- Authentication via Ed25519 JWT signed with the radio's private key. Tokens auto-renew before 24h expiry.
- Broker address: separate `community_mqtt_broker_host` and `community_mqtt_broker_port` fields; defaults to `mqtt-us-v1.letsmesh.net:443`.
- Topic: `meshcore/{IATA}/{pubkey}/packets` — IATA is a 3-letter region code.
- JWT `email` claim enables node claiming on the community aggregator.
- Config: `community_mqtt_enabled`, `community_mqtt_iata`, `community_mqtt_broker_host`, `community_mqtt_broker_port`, `community_mqtt_email` in `app_settings`.
### Server-Side Decryption
The server can decrypt packets using stored keys, both in real-time and for historical packets.
@@ -402,7 +416,7 @@ mc.subscribe(EventType.ACK, handler)
| `MESHCORE_LOG_LEVEL` | `INFO` | Logging level (`DEBUG`/`INFO`/`WARNING`/`ERROR`) |
| `MESHCORE_DATABASE_PATH` | `data/meshcore.db` | SQLite database location |
**Note:** Runtime app settings are stored in the database (`app_settings` table), not environment variables. These include `max_radio_contacts`, `auto_decrypt_dm_on_advert`, `sidebar_sort_order`, `advert_interval`, `last_advert_time`, `favorites`, `last_message_times`, `bots`, and all MQTT configuration (`mqtt_broker_host`, `mqtt_broker_port`, `mqtt_username`, `mqtt_password`, `mqtt_use_tls`, `mqtt_tls_insecure`, `mqtt_topic_prefix`, `mqtt_publish_messages`, `mqtt_publish_raw_packets`). They are configured via `GET/PATCH /api/settings` (and related settings endpoints).
**Note:** Runtime app settings are stored in the database (`app_settings` table), not environment variables. These include `max_radio_contacts`, `auto_decrypt_dm_on_advert`, `sidebar_sort_order`, `advert_interval`, `last_advert_time`, `favorites`, `last_message_times`, `bots`, all MQTT configuration (`mqtt_broker_host`, `mqtt_broker_port`, `mqtt_username`, `mqtt_password`, `mqtt_use_tls`, `mqtt_tls_insecure`, `mqtt_topic_prefix`, `mqtt_publish_messages`, `mqtt_publish_raw_packets`), and community MQTT configuration (`community_mqtt_enabled`, `community_mqtt_iata`, `community_mqtt_broker_host`, `community_mqtt_broker_port`, `community_mqtt_email`). They are configured via `GET/PATCH /api/settings` (and related settings endpoints).
Byte-perfect channel retries are user-triggered via `POST /api/messages/channel/{message_id}/resend` and are allowed for 30 seconds after the original send.

View File

@@ -29,6 +29,9 @@ RUN uv sync --frozen --no-dev
# Copy application code
COPY app/ ./app/
# Copy license attributions
COPY LICENSES.md ./
# Copy built frontend from first stage
COPY --from=frontend-builder /build/dist ./frontend/dist

1475
LICENSES.md Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -300,3 +300,15 @@ npx playwright test --headed # show the browser window
## API Documentation
With the backend running: http://localhost:8000/docs
## Debugging & Bug Reports
If you're experiencing issues or opening a bug report, please start the backend with debug logging enabled. Debug mode provides a much more detailed breakdown of radio communication, packet processing, and other internal operations, which makes it significantly easier to diagnose problems.
To start the server with debug logging:
```bash
MESHCORE_LOG_LEVEL=DEBUG uv run uvicorn app.main:app --host 0.0.0.0 --port 8000
```
Please include the relevant debug log output when filing an issue on GitHub.

View File

@@ -27,7 +27,9 @@ app/
├── packet_processor.py # Raw packet pipeline, dedup, path handling
├── event_handlers.py # MeshCore event subscriptions and ACK tracking
├── websocket.py # WS manager + broadcast helpers
├── mqtt.py # Optional MQTT publisher (fire-and-forget forwarding)
├── mqtt_base.py # Shared MQTT publisher base class (lifecycle, reconnect, backoff)
├── mqtt.py # Private MQTT publisher (fire-and-forget forwarding)
├── community_mqtt.py # Community MQTT publisher (raw packet sharing)
├── bot.py # Bot execution and outbound bot sends
├── dependencies.py # Shared FastAPI dependency providers
├── keystore.py # Ephemeral private/public key storage for DM decryption
@@ -104,16 +106,31 @@ app/
### MQTT publishing
- Optional forwarding of mesh events to an external MQTT broker.
- All config in `app_settings` (not env vars): `mqtt_broker_host`, `mqtt_broker_port`, `mqtt_username`, `mqtt_password`, `mqtt_use_tls`, `mqtt_topic_prefix`, `mqtt_publish_messages`, `mqtt_publish_raw_packets`.
- Disabled when `mqtt_broker_host` is empty.
- All config in `app_settings` (not env vars): `mqtt_broker_host`, `mqtt_broker_port`, `mqtt_username`, `mqtt_password`, `mqtt_use_tls`, `mqtt_tls_insecure`, `mqtt_topic_prefix`, `mqtt_publish_messages`, `mqtt_publish_raw_packets`.
- Disabled when `mqtt_broker_host` is empty, or when both publish toggles are off (`mqtt_publish_messages=false` and `mqtt_publish_raw_packets=false`).
- `broadcast_event()` in `websocket.py` calls `mqtt_broadcast()` — single hook covers all message and raw_packet events.
- `MqttPublisher` (`app/mqtt.py`) runs a background connection loop with auto-reconnect and exponential backoff (5s → 30s).
- Publishes are fire-and-forget; individual publish failures logged but not surfaced to users.
- Connection state changes surface via `broadcast_error`/`broadcast_success` toasts.
- Health endpoint includes `mqtt_status` field (`connected`, `disconnected`, `disabled`).
- Health endpoint includes `mqtt_status` field (`connected`, `disconnected`, `disabled`), where `disabled` covers both "no broker host configured" and "nothing enabled to publish".
- Settings changes trigger `mqtt_publisher.restart()` — no server restart needed.
- Topics: `{prefix}/dm:{key}`, `{prefix}/gm:{key}`, `{prefix}/raw/dm:{key}`, `{prefix}/raw/gm:{key}`, `{prefix}/raw/unrouted`.
### Community MQTT
- Separate publisher (`app/community_mqtt.py`) for sharing raw packets with the MeshCore community aggregator.
- Implementation intent: keep functional parity with the reference implementation at `https://github.com/agessaman/meshcore-packet-capture` unless this repository explicitly documents a deliberate deviation.
- Independent from the private `MqttPublisher` — different broker, authentication, and topic structure.
- Connects to the community broker (default `mqtt-us-v1.letsmesh.net:443`) via WebSockets over TLS.
- Authentication: Ed25519 JWT tokens signed with the radio's expanded "orlp" private key. Tokens expire after 24 hours; proactive renewal at 23 hours.
- Broker address: separate `community_mqtt_broker_host` and `community_mqtt_broker_port` fields; defaults to `mqtt-us-v1.letsmesh.net:443`.
- JWT claims include `publicKey`, `owner` (radio pubkey), `client` (app identifier), and optional `email` (for node claiming on the community aggregator).
- Topic: `meshcore/{IATA}/{pubkey}/packets` — IATA is a 3-letter region code (required to enable; no default).
- Only raw packets are published — never decrypted messages.
- Publishes are fire-and-forget. The connection loop detects publish failures via `connected` flag and reconnects within 60 seconds.
- Health endpoint includes `community_mqtt_status` field.
- Settings: `community_mqtt_enabled`, `community_mqtt_iata`, `community_mqtt_broker_host`, `community_mqtt_broker_port`, `community_mqtt_email`.
## API Surface (all under `/api`)
### Health
@@ -222,6 +239,7 @@ Main tables:
- `bots`
- `mqtt_broker_host`, `mqtt_broker_port`, `mqtt_username`, `mqtt_password`
- `mqtt_use_tls`, `mqtt_tls_insecure`, `mqtt_topic_prefix`, `mqtt_publish_messages`, `mqtt_publish_raw_packets`
- `community_mqtt_enabled`, `community_mqtt_iata`, `community_mqtt_broker_host`, `community_mqtt_broker_port`, `community_mqtt_email`
## Security Posture (intentional)
@@ -260,6 +278,7 @@ tests/
├── test_message_pagination.py # Cursor-based message pagination
├── test_message_prefix_claim.py # Message prefix claim logic
├── test_migrations.py # Schema migration system
├── test_community_mqtt.py # Community MQTT publisher (JWT, packet format, hash, broadcast)
├── test_mqtt.py # MQTT publisher topic routing and lifecycle
├── test_packet_pipeline.py # End-to-end packet processing
├── test_packets_router.py # Packets router endpoints (decrypt, maintenance)

402
app/community_mqtt.py Normal file
View File

@@ -0,0 +1,402 @@
"""Community MQTT publisher for sharing raw packets with the MeshCore community.
Publishes raw packet data to mqtt-us-v1.letsmesh.net using the protocol
defined by meshcore-packet-capture (https://github.com/agessaman/meshcore-packet-capture).
Authentication uses Ed25519 JWT tokens signed with the radio's private key.
This module is independent from the private MqttPublisher in app/mqtt.py.
"""
from __future__ import annotations
import asyncio
import base64
import hashlib
import json
import logging
import re
import ssl
import time
from datetime import datetime
from typing import Any
import nacl.bindings
from app.models import AppSettings
from app.mqtt_base import BaseMqttPublisher
logger = logging.getLogger(__name__)
_DEFAULT_BROKER = "mqtt-us-v1.letsmesh.net"
_DEFAULT_PORT = 443 # Community protocol uses WSS on port 443 by default
_CLIENT_ID = "RemoteTerm (github.com/jkingsman/Remote-Terminal-for-MeshCore)"
# Proactive JWT renewal: reconnect 1 hour before the 24h token expires
_TOKEN_LIFETIME = 86400 # 24 hours (must match _generate_jwt_token exp)
_TOKEN_RENEWAL_THRESHOLD = _TOKEN_LIFETIME - 3600 # 23 hours
# Ed25519 group order
_L = 2**252 + 27742317777372353535851937790883648493
_IATA_RE = re.compile(r"^[A-Z]{3}$")
# Route type mapping: bottom 2 bits of first byte
_ROUTE_MAP = {0: "F", 1: "F", 2: "D", 3: "T"}
def _base64url_encode(data: bytes) -> str:
"""Base64url encode without padding."""
return base64.urlsafe_b64encode(data).rstrip(b"=").decode("ascii")
def _ed25519_sign_expanded(
message: bytes, scalar: bytes, prefix: bytes, public_key: bytes
) -> bytes:
"""Sign a message using MeshCore's expanded Ed25519 key format.
MeshCore stores 64-byte "orlp" format keys: scalar(32) || prefix(32).
Standard Ed25519 libraries expect seed format and would re-SHA-512 the key.
This performs the signing manually using the already-expanded key material.
Port of meshcore-packet-capture's ed25519_sign_with_expanded_key().
"""
# r = SHA-512(prefix || message) mod L
r = int.from_bytes(hashlib.sha512(prefix + message).digest(), "little") % _L
# R = r * B (base point multiplication)
R = nacl.bindings.crypto_scalarmult_ed25519_base_noclamp(r.to_bytes(32, "little"))
# k = SHA-512(R || public_key || message) mod L
k = int.from_bytes(hashlib.sha512(R + public_key + message).digest(), "little") % _L
# s = (r + k * scalar) mod L
s = (r + k * int.from_bytes(scalar, "little")) % _L
return R + s.to_bytes(32, "little")
def _generate_jwt_token(
private_key: bytes,
public_key: bytes,
*,
audience: str = _DEFAULT_BROKER,
email: str = "",
) -> str:
"""Generate a JWT token for community MQTT authentication.
Creates a token with Ed25519 signature using MeshCore's expanded key format.
Token format: header_b64.payload_b64.signature_hex
Optional ``email`` embeds a node-claiming identity so the community
aggregator can associate this radio with an owner.
"""
header = {"alg": "Ed25519", "typ": "JWT"}
now = int(time.time())
pubkey_hex = public_key.hex().upper()
payload: dict[str, object] = {
"publicKey": pubkey_hex,
"iat": now,
"exp": now + _TOKEN_LIFETIME,
"aud": audience,
"owner": pubkey_hex,
"client": _CLIENT_ID,
}
if email:
payload["email"] = email
header_b64 = _base64url_encode(json.dumps(header, separators=(",", ":")).encode())
payload_b64 = _base64url_encode(json.dumps(payload, separators=(",", ":")).encode())
signing_input = f"{header_b64}.{payload_b64}".encode()
scalar = private_key[:32]
prefix = private_key[32:]
signature = _ed25519_sign_expanded(signing_input, scalar, prefix, public_key)
return f"{header_b64}.{payload_b64}.{signature.hex()}"
def _calculate_packet_hash(raw_bytes: bytes) -> str:
"""Calculate packet hash matching MeshCore's Packet::calculatePacketHash().
Parses the packet structure to extract payload type and payload data,
then hashes: payload_type(1 byte) [+ path_len(2 bytes LE) for TRACE] + payload_data.
Returns first 16 hex characters (uppercase).
"""
if not raw_bytes:
return "0" * 16
try:
header = raw_bytes[0]
payload_type = (header >> 2) & 0x0F
route_type = header & 0x03
# Transport codes present for TRANSPORT_FLOOD (0) and TRANSPORT_DIRECT (3)
has_transport = route_type in (0x00, 0x03)
offset = 1 # Past header
if has_transport:
offset += 4 # Skip 4 bytes of transport codes
# Read path_len (1 byte on wire). Invalid/truncated packets map to zero hash.
if offset >= len(raw_bytes):
return "0" * 16
path_len = raw_bytes[offset]
offset += 1
# Skip past path to get to payload. Invalid/truncated packets map to zero hash.
if len(raw_bytes) < offset + path_len:
return "0" * 16
payload_start = offset + path_len
payload_data = raw_bytes[payload_start:]
# Hash: payload_type(1 byte) [+ path_len as uint16_t LE for TRACE] + payload_data
hash_obj = hashlib.sha256()
hash_obj.update(bytes([payload_type]))
if payload_type == 9: # PAYLOAD_TYPE_TRACE
hash_obj.update(path_len.to_bytes(2, byteorder="little"))
hash_obj.update(payload_data)
return hash_obj.hexdigest()[:16].upper()
except Exception:
return "0" * 16
def _decode_packet_fields(raw_bytes: bytes) -> tuple[str, str, str, list[str], int | None]:
"""Decode packet fields used by the community uploader payload format.
Returns:
(route_letter, packet_type_str, payload_len_str, path_values, payload_type_int)
"""
# Reference defaults when decode fails
route = "U"
packet_type = "0"
payload_len = "0"
path_values: list[str] = []
payload_type: int | None = None
try:
if len(raw_bytes) < 2:
return route, packet_type, payload_len, path_values, payload_type
header = raw_bytes[0]
payload_version = (header >> 6) & 0x03
if payload_version != 0:
return route, packet_type, payload_len, path_values, payload_type
route_type = header & 0x03
has_transport = route_type in (0x00, 0x03)
offset = 1
if has_transport:
offset += 4
if len(raw_bytes) <= offset:
return route, packet_type, payload_len, path_values, payload_type
path_len = raw_bytes[offset]
offset += 1
if len(raw_bytes) < offset + path_len:
return route, packet_type, payload_len, path_values, payload_type
path_bytes = raw_bytes[offset : offset + path_len]
offset += path_len
payload_type = (header >> 2) & 0x0F
route = _ROUTE_MAP.get(route_type, "U")
packet_type = str(payload_type)
payload_len = str(max(0, len(raw_bytes) - offset))
path_values = [f"{b:02x}" for b in path_bytes]
return route, packet_type, payload_len, path_values, payload_type
except Exception:
return route, packet_type, payload_len, path_values, payload_type
def _format_raw_packet(data: dict[str, Any], device_name: str, public_key_hex: str) -> dict:
"""Convert a RawPacketBroadcast dict to meshcore-packet-capture format."""
raw_hex = data.get("data", "")
raw_bytes = bytes.fromhex(raw_hex) if raw_hex else b""
route, packet_type, payload_len, path_values, _payload_type = _decode_packet_fields(raw_bytes)
# Reference format uses local "now" timestamp and derived time/date fields.
current_time = datetime.now()
ts_str = current_time.isoformat()
# SNR/RSSI are always strings in reference output.
snr_val = data.get("snr")
rssi_val = data.get("rssi")
snr = str(snr_val) if snr_val is not None else "Unknown"
rssi = str(rssi_val) if rssi_val is not None else "Unknown"
packet_hash = _calculate_packet_hash(raw_bytes)
packet = {
"origin": device_name or "MeshCore Device",
"origin_id": public_key_hex.upper(),
"timestamp": ts_str,
"type": "PACKET",
"direction": "rx",
"time": current_time.strftime("%H:%M:%S"),
"date": current_time.strftime("%d/%m/%Y"),
"len": str(len(raw_bytes)),
"packet_type": packet_type,
"route": route,
"payload_len": payload_len,
"raw": raw_hex.upper(),
"SNR": snr,
"RSSI": rssi,
"hash": packet_hash,
}
if route == "D":
packet["path"] = ",".join(path_values)
return packet
class CommunityMqttPublisher(BaseMqttPublisher):
"""Manages the community MQTT connection and publishes raw packets."""
_backoff_max = 60
_log_prefix = "Community MQTT"
_not_configured_timeout: float | None = 30
def __init__(self) -> None:
super().__init__()
self._key_unavailable_warned: bool = False
async def start(self, settings: AppSettings) -> None:
self._key_unavailable_warned = False
await super().start(settings)
def _on_not_configured(self) -> None:
from app.keystore import has_private_key
from app.websocket import broadcast_error
if (
self._settings
and self._settings.community_mqtt_enabled
and not has_private_key()
and not self._key_unavailable_warned
):
broadcast_error(
"Community MQTT unavailable",
"Radio firmware does not support private key export.",
)
self._key_unavailable_warned = True
def _is_configured(self) -> bool:
"""Check if community MQTT is enabled and keys are available."""
from app.keystore import has_private_key
return bool(self._settings and self._settings.community_mqtt_enabled and has_private_key())
def _build_client_kwargs(self, settings: AppSettings) -> dict[str, Any]:
from app.keystore import get_private_key, get_public_key
private_key = get_private_key()
public_key = get_public_key()
assert private_key is not None and public_key is not None # guaranteed by _pre_connect
pubkey_hex = public_key.hex().upper()
broker_host = settings.community_mqtt_broker_host or _DEFAULT_BROKER
broker_port = settings.community_mqtt_broker_port or _DEFAULT_PORT
jwt_token = _generate_jwt_token(
private_key,
public_key,
audience=broker_host,
email=settings.community_mqtt_email or "",
)
tls_context = ssl.create_default_context()
return {
"hostname": broker_host,
"port": broker_port,
"transport": "websockets",
"tls_context": tls_context,
"websocket_path": "/",
"username": f"v1_{pubkey_hex}",
"password": jwt_token,
}
def _on_connected(self, settings: AppSettings) -> tuple[str, str]:
broker_host = settings.community_mqtt_broker_host or _DEFAULT_BROKER
broker_port = settings.community_mqtt_broker_port or _DEFAULT_PORT
return ("Community MQTT connected", f"{broker_host}:{broker_port}")
def _on_error(self) -> tuple[str, str]:
return (
"Community MQTT connection failure",
"Check your internet connection or try again later.",
)
def _should_break_wait(self, elapsed: float) -> bool:
if not self.connected:
logger.info("Community MQTT publish failure detected, reconnecting")
return True
if elapsed >= _TOKEN_RENEWAL_THRESHOLD:
logger.info("Community MQTT JWT nearing expiry, reconnecting")
return True
return False
async def _pre_connect(self, settings: AppSettings) -> bool:
from app.keystore import get_private_key, get_public_key
private_key = get_private_key()
public_key = get_public_key()
if private_key is None or public_key is None:
# Keys not available yet, wait for settings change or key export
self.connected = False
self._version_event.clear()
try:
await asyncio.wait_for(self._version_event.wait(), timeout=30)
except asyncio.TimeoutError:
pass
return False
return True
# Module-level singleton
community_publisher = CommunityMqttPublisher()
def community_mqtt_broadcast(event_type: str, data: dict[str, Any]) -> None:
"""Fire-and-forget community MQTT publish for raw packets only."""
if event_type != "raw_packet":
return
if not community_publisher.connected or community_publisher._settings is None:
return
asyncio.create_task(_community_maybe_publish(data))
async def _community_maybe_publish(data: dict[str, Any]) -> None:
"""Format and publish a raw packet to the community broker."""
settings = community_publisher._settings
if settings is None or not settings.community_mqtt_enabled:
return
try:
from app.keystore import get_public_key
from app.radio import radio_manager
public_key = get_public_key()
if public_key is None:
return
pubkey_hex = public_key.hex().upper()
# Get device name from radio
device_name = ""
if radio_manager.meshcore and radio_manager.meshcore.self_info:
device_name = radio_manager.meshcore.self_info.get("name", "")
packet = _format_raw_packet(data, device_name, pubkey_hex)
iata = settings.community_mqtt_iata.upper().strip()
if not _IATA_RE.fullmatch(iata):
logger.debug("Community MQTT: skipping publish — no valid IATA code configured")
return
topic = f"meshcore/{iata}/{pubkey_hex}/packets"
await community_publisher.publish(topic, packet)
except Exception as e:
logger.warning("Community MQTT broadcast error: %s", e)

View File

@@ -56,19 +56,22 @@ async def lifespan(app: FastAPI):
# Always start connection monitor (even if initial connection failed)
await radio_manager.start_connection_monitor()
# Start MQTT publisher if configured
# Start MQTT publishers if configured
from app.community_mqtt import community_publisher
from app.mqtt import mqtt_publisher
from app.repository import AppSettingsRepository
try:
mqtt_settings = await AppSettingsRepository.get()
await mqtt_publisher.start(mqtt_settings)
await community_publisher.start(mqtt_settings)
except Exception as e:
logger.warning("Failed to start MQTT publisher: %s", e)
logger.warning("Failed to start MQTT publisher(s): %s", e)
yield
logger.info("Shutting down")
await community_publisher.stop()
await mqtt_publisher.stop()
await radio_manager.stop_connection_monitor()
await stop_message_polling()

View File

@@ -254,6 +254,13 @@ async def run_migrations(conn: aiosqlite.Connection) -> int:
await set_version(conn, 31)
applied += 1
# Migration 32: Add community MQTT columns to app_settings
if version < 32:
logger.info("Applying migration 32: add community MQTT columns to app_settings")
await _migrate_032_add_community_mqtt_columns(conn)
await set_version(conn, 32)
applied += 1
if applied > 0:
logger.info(
"Applied %d migration(s), schema now at version %d", applied, await get_version(conn)
@@ -1891,3 +1898,31 @@ async def _migrate_031_add_mqtt_columns(conn: aiosqlite.Connection) -> None:
await conn.execute(f"ALTER TABLE app_settings ADD COLUMN {col_name} {col_def}")
await conn.commit()
async def _migrate_032_add_community_mqtt_columns(conn: aiosqlite.Connection) -> None:
"""Add community MQTT configuration columns to app_settings."""
# Guard: app_settings may not exist in partial-schema test setups
cursor = await conn.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name='app_settings'"
)
if not await cursor.fetchone():
await conn.commit()
return
cursor = await conn.execute("PRAGMA table_info(app_settings)")
columns = {row[1] for row in await cursor.fetchall()}
new_columns = [
("community_mqtt_enabled", "INTEGER DEFAULT 0"),
("community_mqtt_iata", "TEXT DEFAULT ''"),
("community_mqtt_broker_host", "TEXT DEFAULT 'mqtt-us-v1.letsmesh.net'"),
("community_mqtt_broker_port", "INTEGER DEFAULT 443"),
("community_mqtt_email", "TEXT DEFAULT ''"),
]
for col_name, col_def in new_columns:
if col_name not in columns:
await conn.execute(f"ALTER TABLE app_settings ADD COLUMN {col_name} {col_def}")
await conn.commit()

View File

@@ -463,6 +463,26 @@ class AppSettings(BaseModel):
default=False,
description="Whether to publish raw packets to MQTT",
)
community_mqtt_enabled: bool = Field(
default=False,
description="Whether to publish raw packets to the community MQTT broker (letsmesh.net)",
)
community_mqtt_iata: str = Field(
default="",
description="IATA region code for community MQTT topic routing (3 alpha chars)",
)
community_mqtt_broker_host: str = Field(
default="mqtt-us-v1.letsmesh.net",
description="Community MQTT broker hostname",
)
community_mqtt_broker_port: int = Field(
default=443,
description="Community MQTT broker port",
)
community_mqtt_email: str = Field(
default="",
description="Email address for node claiming on the community aggregator (optional)",
)
class BusyChannel(BaseModel):

View File

@@ -3,148 +3,44 @@
from __future__ import annotations
import asyncio
import json
import logging
import ssl
from typing import Any
import aiomqtt
from app.models import AppSettings
from app.mqtt_base import BaseMqttPublisher
logger = logging.getLogger(__name__)
# Reconnect backoff: start at 5s, cap at 30s
_BACKOFF_MIN = 5
_BACKOFF_MAX = 30
class MqttPublisher:
class MqttPublisher(BaseMqttPublisher):
"""Manages an MQTT connection and publishes mesh network events."""
def __init__(self) -> None:
self._client: aiomqtt.Client | None = None
self._task: asyncio.Task[None] | None = None
self._settings: AppSettings | None = None
self._settings_version: int = 0
self._version_event: asyncio.Event = asyncio.Event()
self.connected: bool = False
_backoff_max = 30
_log_prefix = "MQTT"
async def start(self, settings: AppSettings) -> None:
"""Start the background connection loop."""
self._settings = settings
self._settings_version += 1
self._version_event.set()
if self._task is None or self._task.done():
self._task = asyncio.create_task(self._connection_loop())
def _is_configured(self) -> bool:
"""Check if MQTT is configured and has something to publish."""
return bool(
self._settings
and self._settings.mqtt_broker_host
and (self._settings.mqtt_publish_messages or self._settings.mqtt_publish_raw_packets)
)
async def stop(self) -> None:
"""Cancel the background task and disconnect."""
if self._task and not self._task.done():
self._task.cancel()
try:
await self._task
except asyncio.CancelledError:
pass
self._task = None
self._client = None
self.connected = False
def _build_client_kwargs(self, settings: AppSettings) -> dict[str, Any]:
return {
"hostname": settings.mqtt_broker_host,
"port": settings.mqtt_broker_port,
"username": settings.mqtt_username or None,
"password": settings.mqtt_password or None,
"tls_context": self._build_tls_context(settings),
}
async def restart(self, settings: AppSettings) -> None:
"""Called when MQTT settings change — stop + start."""
await self.stop()
await self.start(settings)
def _on_connected(self, settings: AppSettings) -> tuple[str, str]:
return ("MQTT connected", f"{settings.mqtt_broker_host}:{settings.mqtt_broker_port}")
async def publish(self, topic: str, payload: dict[str, Any]) -> None:
"""Publish a JSON payload. Drops silently if not connected."""
if self._client is None or not self.connected:
return
try:
await self._client.publish(topic, json.dumps(payload))
except Exception as e:
logger.warning("MQTT publish failed on %s: %s", topic, e)
self.connected = False
# Wake the connection loop so it exits the wait and reconnects
self._settings_version += 1
self._version_event.set()
def _mqtt_configured(self) -> bool:
"""Check if MQTT is configured (broker host is set)."""
return bool(self._settings and self._settings.mqtt_broker_host)
async def _connection_loop(self) -> None:
"""Background loop: connect, wait, reconnect on failure."""
from app.websocket import broadcast_error, broadcast_success
backoff = _BACKOFF_MIN
while True:
if not self._mqtt_configured():
self.connected = False
self._client = None
# Wait until settings change (which might configure MQTT)
self._version_event.clear()
try:
await self._version_event.wait()
except asyncio.CancelledError:
return
continue
settings = self._settings
assert settings is not None # guaranteed by _mqtt_configured()
version_at_connect = self._settings_version
try:
tls_context = self._build_tls_context(settings)
async with aiomqtt.Client(
hostname=settings.mqtt_broker_host,
port=settings.mqtt_broker_port,
username=settings.mqtt_username or None,
password=settings.mqtt_password or None,
tls_context=tls_context,
) as client:
self._client = client
self.connected = True
backoff = _BACKOFF_MIN
broadcast_success(
"MQTT connected",
f"{settings.mqtt_broker_host}:{settings.mqtt_broker_port}",
)
_broadcast_mqtt_health()
# Wait until cancelled or settings version changes.
# The 60s timeout is a housekeeping wake-up; actual connection
# liveness is handled by paho-mqtt's keepalive mechanism.
while self._settings_version == version_at_connect:
self._version_event.clear()
try:
await asyncio.wait_for(self._version_event.wait(), timeout=60)
except asyncio.TimeoutError:
continue
except asyncio.CancelledError:
self.connected = False
self._client = None
return
except Exception as e:
self.connected = False
self._client = None
broadcast_error(
"MQTT connection failure",
"Please correct the settings or disable.",
)
_broadcast_mqtt_health()
logger.warning("MQTT connection error: %s (reconnecting in %ds)", e, backoff)
try:
await asyncio.sleep(backoff)
except asyncio.CancelledError:
return
backoff = min(backoff * 2, _BACKOFF_MAX)
def _on_error(self) -> tuple[str, str]:
return ("MQTT connection failure", "Please correct the settings or disable.")
@staticmethod
def _build_tls_context(settings: AppSettings) -> ssl.SSLContext | None:
@@ -162,14 +58,6 @@ class MqttPublisher:
mqtt_publisher = MqttPublisher()
def _broadcast_mqtt_health() -> None:
"""Push updated health (including mqtt_status) to all WS clients."""
from app.radio import radio_manager
from app.websocket import broadcast_health
broadcast_health(radio_manager.is_connected, radio_manager.connection_info)
def mqtt_broadcast(event_type: str, data: dict[str, Any]) -> None:
"""Fire-and-forget MQTT publish, matching broadcast_event's pattern."""
if event_type not in ("message", "raw_packet"):

216
app/mqtt_base.py Normal file
View File

@@ -0,0 +1,216 @@
"""Shared base class for MQTT publisher lifecycle management.
Both ``MqttPublisher`` (private broker) and ``CommunityMqttPublisher``
(community aggregator) inherit from ``BaseMqttPublisher``, which owns
the connection-loop skeleton, reconnect/backoff logic, and publish method.
Subclasses override a small set of hooks to control configuration checks,
client construction, toast messages, and optional wait-loop behavior.
"""
from __future__ import annotations
import asyncio
import json
import logging
import time
from abc import ABC, abstractmethod
from typing import Any
import aiomqtt
from app.models import AppSettings
logger = logging.getLogger(__name__)
_BACKOFF_MIN = 5
def _broadcast_health() -> None:
"""Push updated health (including MQTT status) to all WS clients."""
from app.radio import radio_manager
from app.websocket import broadcast_health
broadcast_health(radio_manager.is_connected, radio_manager.connection_info)
class BaseMqttPublisher(ABC):
"""Base class for MQTT publishers with shared lifecycle management.
Subclasses implement the abstract hooks to control configuration checks,
client construction, toast messages, and optional wait-loop behavior.
"""
_backoff_max: int = 30
_log_prefix: str = "MQTT"
_not_configured_timeout: float | None = None # None = block forever
def __init__(self) -> None:
self._client: aiomqtt.Client | None = None
self._task: asyncio.Task[None] | None = None
self._settings: AppSettings | None = None
self._settings_version: int = 0
self._version_event: asyncio.Event = asyncio.Event()
self.connected: bool = False
# ── Lifecycle ──────────────────────────────────────────────────────
async def start(self, settings: AppSettings) -> None:
"""Start the background connection loop."""
self._settings = settings
self._settings_version += 1
self._version_event.set()
if self._task is None or self._task.done():
self._task = asyncio.create_task(self._connection_loop())
async def stop(self) -> None:
"""Cancel the background task and disconnect."""
if self._task and not self._task.done():
self._task.cancel()
try:
await self._task
except asyncio.CancelledError:
pass
self._task = None
self._client = None
self.connected = False
async def restart(self, settings: AppSettings) -> None:
"""Called when settings change — stop + start."""
await self.stop()
await self.start(settings)
async def publish(self, topic: str, payload: dict[str, Any]) -> None:
"""Publish a JSON payload. Drops silently if not connected."""
if self._client is None or not self.connected:
return
try:
await self._client.publish(topic, json.dumps(payload))
except Exception as e:
logger.warning("%s publish failed on %s: %s", self._log_prefix, topic, e)
self.connected = False
# Wake the connection loop so it exits the wait and reconnects
self._settings_version += 1
self._version_event.set()
# ── Abstract hooks ─────────────────────────────────────────────────
@abstractmethod
def _is_configured(self) -> bool:
"""Return True when this publisher should attempt to connect."""
@abstractmethod
def _build_client_kwargs(self, settings: AppSettings) -> dict[str, Any]:
"""Return the keyword arguments for ``aiomqtt.Client(...)``."""
@abstractmethod
def _on_connected(self, settings: AppSettings) -> tuple[str, str]:
"""Return ``(title, detail)`` for the success toast on connect."""
@abstractmethod
def _on_error(self) -> tuple[str, str]:
"""Return ``(title, detail)`` for the error toast on connect failure."""
# ── Optional hooks ─────────────────────────────────────────────────
def _should_break_wait(self, elapsed: float) -> bool:
"""Return True to break the inner wait (e.g. token expiry)."""
return False
async def _pre_connect(self, settings: AppSettings) -> bool:
"""Called before connecting. Return True to proceed, False to retry."""
return True
def _on_not_configured(self) -> None:
"""Called each time the loop finds the publisher not configured."""
return # no-op by default; subclasses may override
# ── Connection loop ────────────────────────────────────────────────
async def _connection_loop(self) -> None:
"""Background loop: connect, wait for version change, reconnect on failure."""
from app.websocket import broadcast_error, broadcast_success
backoff = _BACKOFF_MIN
while True:
if not self._is_configured():
self._on_not_configured()
self.connected = False
self._client = None
self._version_event.clear()
try:
if self._not_configured_timeout is None:
await self._version_event.wait()
else:
await asyncio.wait_for(
self._version_event.wait(),
timeout=self._not_configured_timeout,
)
except asyncio.TimeoutError:
continue
except asyncio.CancelledError:
return
continue
settings = self._settings
assert settings is not None # guaranteed by _is_configured()
version_at_connect = self._settings_version
try:
if not await self._pre_connect(settings):
continue
client_kwargs = self._build_client_kwargs(settings)
connect_time = time.monotonic()
async with aiomqtt.Client(**client_kwargs) as client:
self._client = client
self.connected = True
backoff = _BACKOFF_MIN
title, detail = self._on_connected(settings)
broadcast_success(title, detail)
_broadcast_health()
# Wait until cancelled or settings version changes.
# The 60s timeout is a housekeeping wake-up; actual connection
# liveness is handled by paho-mqtt's keepalive mechanism.
while self._settings_version == version_at_connect:
self._version_event.clear()
try:
await asyncio.wait_for(self._version_event.wait(), timeout=60)
except asyncio.TimeoutError:
elapsed = time.monotonic() - connect_time
if self._should_break_wait(elapsed):
break
continue
# async with exited — client is now closed
self._client = None
self.connected = False
_broadcast_health()
except asyncio.CancelledError:
self.connected = False
self._client = None
return
except Exception as e:
self.connected = False
self._client = None
title, detail = self._on_error()
broadcast_error(title, detail)
_broadcast_health()
logger.warning(
"%s connection error: %s (reconnecting in %ds)",
self._log_prefix,
e,
backoff,
)
try:
await asyncio.sleep(backoff)
except asyncio.CancelledError:
return
backoff = min(backoff * 2, self._backoff_max)

View File

@@ -29,7 +29,10 @@ class AppSettingsRepository:
advert_interval, last_advert_time, bots,
mqtt_broker_host, mqtt_broker_port, mqtt_username, mqtt_password,
mqtt_use_tls, mqtt_tls_insecure, mqtt_topic_prefix,
mqtt_publish_messages, mqtt_publish_raw_packets
mqtt_publish_messages, mqtt_publish_raw_packets,
community_mqtt_enabled, community_mqtt_iata,
community_mqtt_broker_host, community_mqtt_broker_port,
community_mqtt_email
FROM app_settings WHERE id = 1
"""
)
@@ -103,6 +106,12 @@ class AppSettingsRepository:
mqtt_topic_prefix=row["mqtt_topic_prefix"] or "meshcore",
mqtt_publish_messages=bool(row["mqtt_publish_messages"]),
mqtt_publish_raw_packets=bool(row["mqtt_publish_raw_packets"]),
community_mqtt_enabled=bool(row["community_mqtt_enabled"]),
community_mqtt_iata=row["community_mqtt_iata"] or "",
community_mqtt_broker_host=row["community_mqtt_broker_host"]
or "mqtt-us-v1.letsmesh.net",
community_mqtt_broker_port=row["community_mqtt_broker_port"] or 443,
community_mqtt_email=row["community_mqtt_email"] or "",
)
@staticmethod
@@ -125,6 +134,11 @@ class AppSettingsRepository:
mqtt_topic_prefix: str | None = None,
mqtt_publish_messages: bool | None = None,
mqtt_publish_raw_packets: bool | None = None,
community_mqtt_enabled: bool | None = None,
community_mqtt_iata: str | None = None,
community_mqtt_broker_host: str | None = None,
community_mqtt_broker_port: int | None = None,
community_mqtt_email: str | None = None,
) -> AppSettings:
"""Update app settings. Only provided fields are updated."""
updates = []
@@ -204,6 +218,26 @@ class AppSettingsRepository:
updates.append("mqtt_publish_raw_packets = ?")
params.append(1 if mqtt_publish_raw_packets else 0)
if community_mqtt_enabled is not None:
updates.append("community_mqtt_enabled = ?")
params.append(1 if community_mqtt_enabled else 0)
if community_mqtt_iata is not None:
updates.append("community_mqtt_iata = ?")
params.append(community_mqtt_iata)
if community_mqtt_broker_host is not None:
updates.append("community_mqtt_broker_host = ?")
params.append(community_mqtt_broker_host)
if community_mqtt_broker_port is not None:
updates.append("community_mqtt_broker_port = ?")
params.append(community_mqtt_broker_port)
if community_mqtt_email is not None:
updates.append("community_mqtt_email = ?")
params.append(community_mqtt_email)
if updates:
query = f"UPDATE app_settings SET {', '.join(updates)} WHERE id = 1"
await db.conn.execute(query, params)

View File

@@ -17,6 +17,7 @@ class HealthResponse(BaseModel):
database_size_mb: float
oldest_undecrypted_timestamp: int | None
mqtt_status: str | None = None
community_mqtt_status: str | None = None
async def build_health_data(radio_connected: bool, connection_info: str | None) -> dict:
@@ -39,13 +40,25 @@ async def build_health_data(radio_connected: bool, connection_info: str | None)
try:
from app.mqtt import mqtt_publisher
if mqtt_publisher._mqtt_configured():
if mqtt_publisher._is_configured():
mqtt_status = "connected" if mqtt_publisher.connected else "disconnected"
else:
mqtt_status = "disabled"
except Exception:
pass
# Community MQTT status
community_mqtt_status: str | None = None
try:
from app.community_mqtt import community_publisher
if community_publisher._is_configured():
community_mqtt_status = "connected" if community_publisher.connected else "disconnected"
else:
community_mqtt_status = "disabled"
except Exception:
pass
return {
"status": "ok" if radio_connected else "degraded",
"radio_connected": radio_connected,
@@ -53,6 +66,7 @@ async def build_health_data(radio_connected: bool, connection_info: str | None)
"database_size_mb": db_size_mb,
"oldest_undecrypted_timestamp": oldest_ts,
"mqtt_status": mqtt_status,
"community_mqtt_status": community_mqtt_status,
}

View File

@@ -1,5 +1,6 @@
import asyncio
import logging
import re
from typing import Literal
from fastapi import APIRouter, HTTPException
@@ -97,6 +98,28 @@ class AppSettingsUpdate(BaseModel):
default=None,
description="Whether to publish raw packets to MQTT",
)
community_mqtt_enabled: bool | None = Field(
default=None,
description="Whether to publish raw packets to the community MQTT broker",
)
community_mqtt_iata: str | None = Field(
default=None,
description="IATA region code for community MQTT topic routing (3 alpha chars)",
)
community_mqtt_broker_host: str | None = Field(
default=None,
description="Community MQTT broker hostname",
)
community_mqtt_broker_port: int | None = Field(
default=None,
ge=1,
le=65535,
description="Community MQTT broker port",
)
community_mqtt_email: str | None = Field(
default=None,
description="Email address for node claiming on the community aggregator",
)
class FavoriteRequest(BaseModel):
@@ -181,6 +204,47 @@ async def update_settings(update: AppSettingsUpdate) -> AppSettings:
kwargs[field] = value
mqtt_changed = True
# Community MQTT fields
community_mqtt_changed = False
if update.community_mqtt_enabled is not None:
kwargs["community_mqtt_enabled"] = update.community_mqtt_enabled
community_mqtt_changed = True
if update.community_mqtt_iata is not None:
iata = update.community_mqtt_iata.upper().strip()
if iata and not re.fullmatch(r"[A-Z]{3}", iata):
raise HTTPException(
status_code=400,
detail="IATA code must be exactly 3 uppercase alphabetic characters",
)
kwargs["community_mqtt_iata"] = iata
community_mqtt_changed = True
if update.community_mqtt_broker_host is not None:
kwargs["community_mqtt_broker_host"] = update.community_mqtt_broker_host
community_mqtt_changed = True
if update.community_mqtt_broker_port is not None:
kwargs["community_mqtt_broker_port"] = update.community_mqtt_broker_port
community_mqtt_changed = True
if update.community_mqtt_email is not None:
kwargs["community_mqtt_email"] = update.community_mqtt_email
community_mqtt_changed = True
# Require IATA when enabling community MQTT
if kwargs.get("community_mqtt_enabled", False):
# Check the IATA value being set, or fall back to current settings
iata_value = kwargs.get("community_mqtt_iata")
if iata_value is None:
current = await AppSettingsRepository.get()
iata_value = current.community_mqtt_iata
if not iata_value or not re.fullmatch(r"[A-Z]{3}", iata_value):
raise HTTPException(
status_code=400,
detail="A valid IATA region code is required to enable community sharing",
)
if kwargs:
result = await AppSettingsRepository.update(**kwargs)
@@ -190,6 +254,12 @@ async def update_settings(update: AppSettingsUpdate) -> AppSettings:
await mqtt_publisher.restart(result)
# Restart community MQTT publisher if any community settings changed
if community_mqtt_changed:
from app.community_mqtt import community_publisher
await community_publisher.restart(result)
return result
return await AppSettingsRepository.get()

View File

@@ -104,6 +104,10 @@ def broadcast_event(event_type: str, data: dict) -> None:
mqtt_broadcast(event_type, data)
from app.community_mqtt import community_mqtt_broadcast
community_mqtt_broadcast(event_type, data)
def broadcast_error(message: str, details: str | None = None) -> None:
"""Broadcast an error notification to all connected clients.

View File

@@ -228,8 +228,10 @@ LocalStorage migration helpers for favorites; canonical favorites are server-sid
- `bots`
- `mqtt_broker_host`, `mqtt_broker_port`, `mqtt_username`, `mqtt_password`
- `mqtt_use_tls`, `mqtt_tls_insecure`, `mqtt_topic_prefix`, `mqtt_publish_messages`, `mqtt_publish_raw_packets`
- `community_mqtt_enabled`, `community_mqtt_iata`, `community_mqtt_broker_host`, `community_mqtt_broker_port`, `community_mqtt_email`
`HealthStatus` includes `mqtt_status` (`"connected"`, `"disconnected"`, `"disabled"`, or `null`).
`HealthStatus` also includes `community_mqtt_status` with the same status values.
`RawPacket.decrypted_info` includes `channel_key` and `contact_key` for MQTT topic routing.

View File

@@ -45,6 +45,17 @@ export function SettingsAboutSection({ className }: { className?: string }) {
MIT License
</a>
</p>
<p>
This code is free, and ad-free, forever. If you love my work,{' '}
<a
href="https://ko-fi.com/jackkingsman"
target="_blank"
rel="noopener noreferrer"
className="text-primary hover:underline"
>
buy me a coffee!
</a>
</p>
</div>
<Separator />

View File

@@ -27,6 +27,16 @@ export function SettingsMqttSection({
const [mqttPublishMessages, setMqttPublishMessages] = useState(false);
const [mqttPublishRawPackets, setMqttPublishRawPackets] = useState(false);
// Community MQTT state
const [communityMqttEnabled, setCommunityMqttEnabled] = useState(false);
const [communityMqttIata, setCommunityMqttIata] = useState('');
const [communityMqttBrokerHost, setCommunityMqttBrokerHost] = useState('mqtt-us-v1.letsmesh.net');
const [communityMqttBrokerPort, setCommunityMqttBrokerPort] = useState('443');
const [communityMqttEmail, setCommunityMqttEmail] = useState('');
const [privateExpanded, setPrivateExpanded] = useState(false);
const [communityExpanded, setCommunityExpanded] = useState(false);
const [busy, setBusy] = useState(false);
const [error, setError] = useState<string | null>(null);
@@ -40,6 +50,11 @@ export function SettingsMqttSection({
setMqttTopicPrefix(appSettings.mqtt_topic_prefix ?? 'meshcore');
setMqttPublishMessages(appSettings.mqtt_publish_messages ?? false);
setMqttPublishRawPackets(appSettings.mqtt_publish_raw_packets ?? false);
setCommunityMqttEnabled(appSettings.community_mqtt_enabled ?? false);
setCommunityMqttIata(appSettings.community_mqtt_iata ?? '');
setCommunityMqttBrokerHost(appSettings.community_mqtt_broker_host ?? 'mqtt-us-v1.letsmesh.net');
setCommunityMqttBrokerPort(String(appSettings.community_mqtt_broker_port ?? 443));
setCommunityMqttEmail(appSettings.community_mqtt_email ?? '');
}, [appSettings]);
const handleSave = async () => {
@@ -57,6 +72,11 @@ export function SettingsMqttSection({
mqtt_topic_prefix: mqttTopicPrefix || 'meshcore',
mqtt_publish_messages: mqttPublishMessages,
mqtt_publish_raw_packets: mqttPublishRawPackets,
community_mqtt_enabled: communityMqttEnabled,
community_mqtt_iata: communityMqttIata,
community_mqtt_broker_host: communityMqttBrokerHost || 'mqtt-us-v1.letsmesh.net',
community_mqtt_broker_port: parseInt(communityMqttBrokerPort, 10) || 443,
community_mqtt_email: communityMqttEmail,
};
await onSaveAppSettings(update);
toast.success('MQTT settings saved');
@@ -69,143 +89,319 @@ export function SettingsMqttSection({
return (
<div className={className}>
<div className="space-y-2">
<Label>Status</Label>
{health?.mqtt_status === 'connected' ? (
<div className="flex items-center gap-2">
<div className="w-2 h-2 rounded-full bg-green-500" />
<span className="text-sm text-green-400">Connected</span>
</div>
) : health?.mqtt_status === 'disconnected' ? (
<div className="flex items-center gap-2">
<div className="w-2 h-2 rounded-full bg-red-500" />
<span className="text-sm text-red-400">Disconnected</span>
</div>
) : (
<div className="flex items-center gap-2">
<div className="w-2 h-2 rounded-full bg-gray-500" />
<span className="text-sm text-muted-foreground">Disabled</span>
<div className="rounded-md border border-yellow-600/50 bg-yellow-950/30 px-4 py-3 text-sm text-yellow-200">
MQTT support is an experimental feature in open beta. All publishing uses QoS 0
(at-most-once delivery). Please report any bugs on the{' '}
<a
href="https://github.com/jkingsman/Remote-Terminal-for-MeshCore/issues"
target="_blank"
rel="noopener noreferrer"
className="underline hover:text-yellow-100"
>
GitHub issues page
</a>
.
</div>
{/* Private MQTT Broker */}
<div className="border border-input rounded-md overflow-hidden">
<button
type="button"
className="w-full flex items-center gap-2 px-4 py-3 text-left hover:bg-muted/40"
onClick={() => setPrivateExpanded(!privateExpanded)}
>
<span className="text-muted-foreground">{privateExpanded ? '▼' : '▶'}</span>
<h4 className="text-sm font-medium">Private MQTT Broker</h4>
{health?.mqtt_status === 'connected' ? (
<>
<div className="w-2 h-2 rounded-full bg-green-500" />
<span className="text-xs text-green-400">Connected</span>
</>
) : health?.mqtt_status === 'disconnected' ? (
<>
<div className="w-2 h-2 rounded-full bg-red-500" />
<span className="text-xs text-red-400">Disconnected</span>
</>
) : (
<>
<div className="w-2 h-2 rounded-full bg-gray-500" />
<span className="text-xs text-muted-foreground">Disabled</span>
</>
)}
</button>
{privateExpanded && (
<div className="px-4 pb-4 space-y-3 border-t border-input">
<p className="text-xs text-muted-foreground pt-3">
Forward mesh data to your own MQTT broker for home automation, logging, or alerting.
</p>
<label className="flex items-center gap-3 cursor-pointer">
<input
type="checkbox"
checked={mqttPublishMessages}
onChange={(e) => setMqttPublishMessages(e.target.checked)}
className="h-4 w-4 rounded border-border"
/>
<span className="text-sm">Publish Messages</span>
</label>
<p className="text-xs text-muted-foreground ml-7">
Forward decrypted DM and channel messages
</p>
<label className="flex items-center gap-3 cursor-pointer">
<input
type="checkbox"
checked={mqttPublishRawPackets}
onChange={(e) => setMqttPublishRawPackets(e.target.checked)}
className="h-4 w-4 rounded border-border"
/>
<span className="text-sm">Publish Raw Packets</span>
</label>
<p className="text-xs text-muted-foreground ml-7">Forward all RF packets</p>
{(mqttPublishMessages || mqttPublishRawPackets) && (
<div className="space-y-3">
<Separator />
<div className="grid grid-cols-1 sm:grid-cols-2 gap-4">
<div className="space-y-2">
<Label htmlFor="mqtt-host">Broker Host</Label>
<Input
id="mqtt-host"
type="text"
placeholder="e.g. 192.168.1.100"
value={mqttBrokerHost}
onChange={(e) => setMqttBrokerHost(e.target.value)}
/>
</div>
<div className="space-y-2">
<Label htmlFor="mqtt-port">Broker Port</Label>
<Input
id="mqtt-port"
type="number"
min="1"
max="65535"
value={mqttBrokerPort}
onChange={(e) => setMqttBrokerPort(e.target.value)}
/>
</div>
</div>
<div className="grid grid-cols-1 sm:grid-cols-2 gap-4">
<div className="space-y-2">
<Label htmlFor="mqtt-username">Username</Label>
<Input
id="mqtt-username"
type="text"
placeholder="Optional"
value={mqttUsername}
onChange={(e) => setMqttUsername(e.target.value)}
/>
</div>
<div className="space-y-2">
<Label htmlFor="mqtt-password">Password</Label>
<Input
id="mqtt-password"
type="password"
placeholder="Optional"
value={mqttPassword}
onChange={(e) => setMqttPassword(e.target.value)}
/>
</div>
</div>
<label className="flex items-center gap-3 cursor-pointer">
<input
type="checkbox"
checked={mqttUseTls}
onChange={(e) => setMqttUseTls(e.target.checked)}
className="h-4 w-4 rounded border-border"
/>
<span className="text-sm">Use TLS</span>
</label>
{mqttUseTls && (
<>
<label className="flex items-center gap-3 cursor-pointer ml-7">
<input
type="checkbox"
checked={mqttTlsInsecure}
onChange={(e) => setMqttTlsInsecure(e.target.checked)}
className="h-4 w-4 rounded border-border"
/>
<span className="text-sm">Skip certificate verification</span>
</label>
<p className="text-xs text-muted-foreground ml-7">
Allow self-signed or untrusted broker certificates
</p>
</>
)}
<Separator />
<div className="space-y-2">
<Label htmlFor="mqtt-prefix">Topic Prefix</Label>
<Input
id="mqtt-prefix"
type="text"
value={mqttTopicPrefix}
onChange={(e) => setMqttTopicPrefix(e.target.value)}
/>
<div className="text-xs text-muted-foreground space-y-2">
<div>
<p className="font-medium">
Decrypted messages{' '}
<span className="font-mono font-normal opacity-75">
{'{'}id, type, conversation_key, text, sender_timestamp, received_at,
paths, outgoing, acked{'}'}
</span>
</p>
<div className="font-mono ml-2 space-y-0.5">
<div>{mqttTopicPrefix || 'meshcore'}/dm:&lt;contact_key&gt;</div>
<div>{mqttTopicPrefix || 'meshcore'}/gm:&lt;channel_key&gt;</div>
</div>
</div>
<div>
<p className="font-medium">
Raw packets{' '}
<span className="font-mono font-normal opacity-75">
{'{'}id, observation_id, timestamp, data, payload_type, snr, rssi,
decrypted, decrypted_info{'}'}
</span>
</p>
<div className="font-mono ml-2 space-y-0.5">
<div>{mqttTopicPrefix || 'meshcore'}/raw/dm:&lt;contact_key&gt;</div>
<div>{mqttTopicPrefix || 'meshcore'}/raw/gm:&lt;channel_key&gt;</div>
<div>{mqttTopicPrefix || 'meshcore'}/raw/unrouted</div>
</div>
</div>
</div>
</div>
</div>
)}
</div>
)}
</div>
<Separator />
{/* Community Analytics */}
<div className="border border-input rounded-md overflow-hidden">
<button
type="button"
className="w-full flex items-center gap-2 px-4 py-3 text-left hover:bg-muted/40"
onClick={() => setCommunityExpanded(!communityExpanded)}
>
<span className="text-muted-foreground">{communityExpanded ? '▼' : '▶'}</span>
<h4 className="text-sm font-medium">Community Analytics</h4>
{health?.community_mqtt_status === 'connected' ? (
<>
<div className="w-2 h-2 rounded-full bg-green-500" />
<span className="text-xs text-green-400">Connected</span>
</>
) : health?.community_mqtt_status === 'disconnected' ? (
<>
<div className="w-2 h-2 rounded-full bg-red-500" />
<span className="text-xs text-red-400">Disconnected</span>
</>
) : (
<>
<div className="w-2 h-2 rounded-full bg-gray-500" />
<span className="text-xs text-muted-foreground">Disabled</span>
</>
)}
</button>
<div className="space-y-2">
<Label htmlFor="mqtt-host">Broker Host</Label>
<Input
id="mqtt-host"
type="text"
placeholder="e.g. 192.168.1.100"
value={mqttBrokerHost}
onChange={(e) => setMqttBrokerHost(e.target.value)}
/>
{communityExpanded && (
<div className="px-4 pb-4 space-y-3 border-t border-input">
<p className="text-xs text-muted-foreground pt-3">
Share raw packet data with the MeshCore community for coverage mapping and network
analysis. Only raw RF packets are shared never decrypted messages.
</p>
<label className="flex items-center gap-3 cursor-pointer">
<input
type="checkbox"
checked={communityMqttEnabled}
onChange={(e) => setCommunityMqttEnabled(e.target.checked)}
className="h-4 w-4 rounded border-border"
/>
<span className="text-sm">Enable Community Analytics</span>
</label>
{communityMqttEnabled && (
<div className="space-y-3">
<div className="grid grid-cols-1 sm:grid-cols-2 gap-4">
<div className="space-y-2">
<Label htmlFor="community-broker-host">Broker Host</Label>
<Input
id="community-broker-host"
type="text"
placeholder="mqtt-us-v1.letsmesh.net"
value={communityMqttBrokerHost}
onChange={(e) => setCommunityMqttBrokerHost(e.target.value)}
/>
</div>
<div className="space-y-2">
<Label htmlFor="community-broker-port">Broker Port</Label>
<Input
id="community-broker-port"
type="number"
min="1"
max="65535"
value={communityMqttBrokerPort}
onChange={(e) => setCommunityMqttBrokerPort(e.target.value)}
/>
</div>
</div>
<div className="space-y-2">
<Label htmlFor="community-iata">Region Code (IATA)</Label>
<Input
id="community-iata"
type="text"
maxLength={3}
placeholder="e.g. DEN, LAX, NYC"
value={communityMqttIata}
onChange={(e) => setCommunityMqttIata(e.target.value.toUpperCase())}
className="w-32"
/>
<p className="text-xs text-muted-foreground">
Your nearest airport&apos;s{' '}
<a
href="https://en.wikipedia.org/wiki/List_of_airports_by_IATA_airport_code:_A"
target="_blank"
rel="noopener noreferrer"
className="underline hover:text-foreground"
>
IATA code
</a>{' '}
(required)
</p>
{communityMqttIata && (
<p className="text-xs text-muted-foreground">
Topic: meshcore/{communityMqttIata}/&lt;pubkey&gt;/packets
</p>
)}
</div>
<div className="space-y-2">
<Label htmlFor="community-email">Owner Email (optional)</Label>
<Input
id="community-email"
type="email"
placeholder="you@example.com"
value={communityMqttEmail}
onChange={(e) => setCommunityMqttEmail(e.target.value)}
/>
<p className="text-xs text-muted-foreground">
Used to claim your node on the community aggregator
</p>
</div>
</div>
)}
</div>
)}
</div>
<div className="space-y-2">
<Label htmlFor="mqtt-port">Broker Port</Label>
<Input
id="mqtt-port"
type="number"
min="1"
max="65535"
value={mqttBrokerPort}
onChange={(e) => setMqttBrokerPort(e.target.value)}
/>
</div>
<div className="space-y-2">
<Label htmlFor="mqtt-username">Username</Label>
<Input
id="mqtt-username"
type="text"
placeholder="Optional"
value={mqttUsername}
onChange={(e) => setMqttUsername(e.target.value)}
/>
</div>
<div className="space-y-2">
<Label htmlFor="mqtt-password">Password</Label>
<Input
id="mqtt-password"
type="password"
placeholder="Optional"
value={mqttPassword}
onChange={(e) => setMqttPassword(e.target.value)}
/>
</div>
<label className="flex items-center gap-3 cursor-pointer">
<input
type="checkbox"
checked={mqttUseTls}
onChange={(e) => setMqttUseTls(e.target.checked)}
className="h-4 w-4 rounded border-border"
/>
<span className="text-sm">Use TLS</span>
</label>
{mqttUseTls && (
<>
<label className="flex items-center gap-3 cursor-pointer ml-7">
<input
type="checkbox"
checked={mqttTlsInsecure}
onChange={(e) => setMqttTlsInsecure(e.target.checked)}
className="h-4 w-4 rounded border-border"
/>
<span className="text-sm">Skip certificate verification</span>
</label>
<p className="text-xs text-muted-foreground ml-7">
Allow self-signed or untrusted broker certificates
</p>
</>
)}
<Separator />
<div className="space-y-2">
<Label htmlFor="mqtt-prefix">Topic Prefix</Label>
<Input
id="mqtt-prefix"
type="text"
value={mqttTopicPrefix}
onChange={(e) => setMqttTopicPrefix(e.target.value)}
/>
<p className="text-xs text-muted-foreground">
Topics: {mqttTopicPrefix || 'meshcore'}/dm:&lt;key&gt;, {mqttTopicPrefix || 'meshcore'}
/gm:&lt;key&gt;, {mqttTopicPrefix || 'meshcore'}
/raw/...
</p>
</div>
<Separator />
<label className="flex items-center gap-3 cursor-pointer">
<input
type="checkbox"
checked={mqttPublishMessages}
onChange={(e) => setMqttPublishMessages(e.target.checked)}
className="h-4 w-4 rounded border-border"
/>
<span className="text-sm">Publish Messages</span>
</label>
<p className="text-xs text-muted-foreground ml-7">
Forward decrypted DM and channel messages
</p>
<label className="flex items-center gap-3 cursor-pointer">
<input
type="checkbox"
checked={mqttPublishRawPackets}
onChange={(e) => setMqttPublishRawPackets(e.target.checked)}
className="h-4 w-4 rounded border-border"
/>
<span className="text-sm">Publish Raw Packets</span>
</label>
<p className="text-xs text-muted-foreground ml-7">Forward all RF packets</p>
<Button onClick={handleSave} disabled={busy} className="w-full">
{busy ? 'Saving...' : 'Save MQTT Settings'}
</Button>

View File

@@ -39,6 +39,7 @@ const baseHealth: HealthStatus = {
database_size_mb: 1.2,
oldest_undecrypted_timestamp: null,
mqtt_status: null,
community_mqtt_status: null,
};
const baseSettings: AppSettings = {
@@ -60,6 +61,11 @@ const baseSettings: AppSettings = {
mqtt_topic_prefix: 'meshcore',
mqtt_publish_messages: false,
mqtt_publish_raw_packets: false,
community_mqtt_enabled: false,
community_mqtt_iata: '',
community_mqtt_broker_host: 'mqtt-us-v1.letsmesh.net',
community_mqtt_broker_port: 443,
community_mqtt_email: '',
};
function renderModal(overrides?: {
@@ -149,6 +155,14 @@ function openMqttSection() {
fireEvent.click(mqttToggle);
}
function expandPrivateMqtt() {
fireEvent.click(screen.getByText('Private MQTT Broker'));
}
function expandCommunityMqtt() {
fireEvent.click(screen.getByText('Community Analytics'));
}
function openDatabaseSection() {
const databaseToggle = screen.getByRole('button', { name: /Database/i });
fireEvent.click(databaseToggle);
@@ -408,19 +422,30 @@ describe('SettingsModal', () => {
it('renders MQTT section with form inputs', () => {
renderModal();
openMqttSection();
expandPrivateMqtt();
// Publish checkboxes always visible
expect(screen.getByText('Publish Messages')).toBeInTheDocument();
expect(screen.getByText('Publish Raw Packets')).toBeInTheDocument();
// Broker config hidden until a publish option is enabled
expect(screen.queryByLabelText('Broker Host')).not.toBeInTheDocument();
// Enable one publish option to reveal broker config
fireEvent.click(screen.getByText('Publish Messages'));
expect(screen.getByLabelText('Broker Host')).toBeInTheDocument();
expect(screen.getByLabelText('Broker Port')).toBeInTheDocument();
expect(screen.getByLabelText('Username')).toBeInTheDocument();
expect(screen.getByLabelText('Password')).toBeInTheDocument();
expect(screen.getByLabelText('Topic Prefix')).toBeInTheDocument();
expect(screen.getByText('Publish Messages')).toBeInTheDocument();
expect(screen.getByText('Publish Raw Packets')).toBeInTheDocument();
});
it('saves MQTT settings through onSaveAppSettings', async () => {
const { onSaveAppSettings } = renderModal();
const { onSaveAppSettings } = renderModal({
appSettings: { ...baseSettings, mqtt_publish_messages: true },
});
openMqttSection();
expandPrivateMqtt();
const hostInput = screen.getByLabelText('Broker Host');
fireEvent.change(hostInput, { target: { value: 'mqtt.example.com' } });
@@ -446,7 +471,9 @@ describe('SettingsModal', () => {
});
openMqttSection();
expect(screen.getByText('Disabled')).toBeInTheDocument();
// Both MQTT and community MQTT show "Disabled" when null status
const disabledElements = screen.getAllByText('Disabled');
expect(disabledElements.length).toBeGreaterThanOrEqual(1);
});
it('shows MQTT connected status badge', () => {
@@ -465,6 +492,75 @@ describe('SettingsModal', () => {
expect(screen.getByText('Connected')).toBeInTheDocument();
});
it('renders community sharing section in MQTT tab', () => {
renderModal();
openMqttSection();
expandCommunityMqtt();
expect(screen.getByText('Community Analytics')).toBeInTheDocument();
expect(screen.getByText('Enable Community Analytics')).toBeInTheDocument();
});
it('shows IATA input only when community sharing is enabled', () => {
renderModal({
appSettings: {
...baseSettings,
community_mqtt_enabled: false,
},
});
openMqttSection();
expandCommunityMqtt();
expect(screen.queryByLabelText('Region Code (IATA)')).not.toBeInTheDocument();
// Enable community sharing
fireEvent.click(screen.getByText('Enable Community Analytics'));
expect(screen.getByLabelText('Region Code (IATA)')).toBeInTheDocument();
});
it('includes community MQTT fields in save payload', async () => {
const { onSaveAppSettings } = renderModal({
appSettings: {
...baseSettings,
community_mqtt_enabled: true,
community_mqtt_iata: 'DEN',
},
});
openMqttSection();
fireEvent.click(screen.getByRole('button', { name: 'Save MQTT Settings' }));
await waitFor(() => {
expect(onSaveAppSettings).toHaveBeenCalledWith(
expect.objectContaining({
community_mqtt_enabled: true,
community_mqtt_iata: 'DEN',
})
);
});
});
it('shows community MQTT connected status badge', () => {
renderModal({
appSettings: {
...baseSettings,
community_mqtt_enabled: true,
},
health: {
...baseHealth,
community_mqtt_status: 'connected',
},
});
openMqttSection();
// Community Analytics sub-section should show Connected
const communitySection = screen.getByText('Community Analytics').closest('div');
expect(communitySection).not.toBeNull();
// Both MQTT and community could show "Connected" — check count
const connectedElements = screen.getAllByText('Connected');
expect(connectedElements.length).toBeGreaterThanOrEqual(1);
});
it('fetches statistics when expanded in mobile external-nav mode', async () => {
const mockStats: StatisticsResponse = {
busiest_channels_24h: [],

View File

@@ -30,6 +30,7 @@ export interface HealthStatus {
database_size_mb: number;
oldest_undecrypted_timestamp: number | null;
mqtt_status: string | null;
community_mqtt_status: string | null;
}
export interface MaintenanceResult {
@@ -192,6 +193,11 @@ export interface AppSettings {
mqtt_topic_prefix: string;
mqtt_publish_messages: boolean;
mqtt_publish_raw_packets: boolean;
community_mqtt_enabled: boolean;
community_mqtt_iata: string;
community_mqtt_broker_host: string;
community_mqtt_broker_port: number;
community_mqtt_email: string;
}
export interface AppSettingsUpdate {
@@ -209,6 +215,11 @@ export interface AppSettingsUpdate {
mqtt_topic_prefix?: string;
mqtt_publish_messages?: boolean;
mqtt_publish_raw_packets?: boolean;
community_mqtt_enabled?: boolean;
community_mqtt_iata?: string;
community_mqtt_broker_host?: string;
community_mqtt_broker_port?: number;
community_mqtt_email?: string;
}
export interface MigratePreferencesRequest {

View File

@@ -69,6 +69,7 @@ reportAttributeAccessIssue = "warning"
[dependency-groups]
dev = [
"httpx>=0.28.1",
"pip-licenses>=5.0.0",
"pytest>=9.0.2",
"pytest-asyncio>=1.3.0",
"pytest-xdist>=3.0",

View File

@@ -35,9 +35,24 @@ PID_BACKEND_LINT=$!
) &
PID_FRONTEND_LINT=$!
(
echo -e "${BLUE}[licenses]${NC} Checking LICENSES.md freshness..."
cd "$SCRIPT_DIR"
TMPLIC=$(mktemp)
trap "rm -f \$TMPLIC" EXIT
bash scripts/collect_licenses.sh "$TMPLIC"
if ! diff -q "$TMPLIC" LICENSES.md > /dev/null 2>&1; then
echo -e "${RED}[licenses]${NC} LICENSES.md is stale — run scripts/collect_licenses.sh"
exit 1
fi
echo -e "${GREEN}[licenses]${NC} Passed!"
) &
PID_LICENSES=$!
FAIL=0
wait $PID_BACKEND_LINT || FAIL=1
wait $PID_FRONTEND_LINT || FAIL=1
wait $PID_LICENSES || FAIL=1
if [ $FAIL -ne 0 ]; then
echo -e "${RED}Phase 1 failed — aborting.${NC}"
exit 1

124
scripts/collect_licenses.sh Executable file
View File

@@ -0,0 +1,124 @@
#!/bin/bash
set -euo pipefail
# Collect third-party license texts into LICENSES.md
# Usage: scripts/collect_licenses.sh [output-path]
# output-path defaults to LICENSES.md at the repo root
REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
OUT="${1:-$REPO_ROOT/LICENSES.md}"
# ── Backend (Python) — uses pip-licenses ─────────────────────────────
backend_licenses() {
cd "$REPO_ROOT"
# Extract direct dependency names from pyproject.toml
local packages
packages=$(uv run python3 -c "
import re, tomllib
with open('pyproject.toml', 'rb') as f:
deps = tomllib.load(f)['project']['dependencies']
names = [re.split(r'[\[><=!;]', d)[0].strip() for d in deps]
print(' '.join(names))
")
# shellcheck disable=SC2086
uv run pip-licenses \
--packages $packages \
--with-license-file \
--no-license-path \
--format=json \
| uv run python3 -c "
import json, sys
data = sorted(json.load(sys.stdin), key=lambda d: d['Name'].lower())
for d in data:
name = d['Name']
version = d['Version']
lic = d.get('License', 'Unknown')
text = d.get('LicenseText', '').strip()
print(f'### {name} ({version}) — {lic}\n')
if text and text != 'UNKNOWN':
print('<details>')
print('<summary>Full license text</summary>')
print()
print('\`\`\`')
print(text)
print('\`\`\`')
print()
print('</details>')
else:
print('*License file not found in package metadata.*')
print()
"
}
# ── Frontend (npm) ───────────────────────────────────────────────────
frontend_licenses() {
cd "$REPO_ROOT/frontend"
node -e "
const fs = require('fs');
const path = require('path');
const pkg = JSON.parse(fs.readFileSync('package.json', 'utf8'));
const depNames = Object.keys(pkg.dependencies || {}).sort((a, b) =>
a.toLowerCase().localeCompare(b.toLowerCase())
);
for (const name of depNames) {
const pkgDir = path.join('node_modules', name);
let version = 'unknown';
let licenseType = 'Unknown';
let licenseText = null;
// Read package.json for version + license type
try {
const depPkg = JSON.parse(fs.readFileSync(path.join(pkgDir, 'package.json'), 'utf8'));
version = depPkg.version || version;
licenseType = depPkg.license || licenseType;
} catch {}
// Find license file (case-insensitive search)
try {
const files = fs.readdirSync(pkgDir);
const licFile = files.find(f => /^(licen[sc]e|copying)/i.test(f));
if (licFile) {
licenseText = fs.readFileSync(path.join(pkgDir, licFile), 'utf8').trim();
}
} catch {}
console.log('### ' + name + ' (' + version + ') — ' + licenseType + '\n');
if (licenseText) {
console.log('<details>');
console.log('<summary>Full license text</summary>');
console.log();
console.log('\`\`\`');
console.log(licenseText);
console.log('\`\`\`');
console.log();
console.log('</details>');
} else {
console.log('*License file not found in package.*');
}
console.log();
}
"
}
# ── Assemble ─────────────────────────────────────────────────────────
{
echo "# Third-Party Licenses"
echo
echo "Auto-generated by \`scripts/collect_licenses.sh\` — do not edit by hand."
echo
echo "## Backend (Python) Dependencies"
echo
backend_licenses
echo "## Frontend (npm) Dependencies"
echo
frontend_licenses
} > "$OUT"
echo "Wrote $OUT" >&2

View File

@@ -3,6 +3,6 @@ set -e
SCRIPT_DIR="$(cd "$(dirname "$0")/.." && pwd)"
echo "Starting E2E tests..."
echo "Starting E2E tests (kicks off a build; this may take a few minutes..."
cd "$SCRIPT_DIR/tests/e2e"
npx playwright test "$@"

View File

@@ -0,0 +1,33 @@
/**
* Extended Playwright test fixture for tests that depend on receiving
* messages from other nodes on the mesh network.
*
* Usage:
* import { test, expect } from '../helpers/meshTrafficTest';
* test('my test', { tag: '@mesh-traffic' }, async ({ page }) => { ... });
*
* When a @mesh-traffic-tagged test fails, an advisory annotation is added
* to the HTML report and a console message is printed, letting the user
* know the failure may be due to low mesh traffic rather than a real bug.
*/
import { test as base, expect } from '@playwright/test';
export { expect };
const TRAFFIC_ADVISORY =
'This test depends on receiving messages from other nodes on the mesh ' +
'network. Failure may indicate insufficient mesh traffic rather than a bug.';
export const test = base.extend<{ _meshTrafficAdvisory: void }>({
_meshTrafficAdvisory: [
async ({}, use, testInfo) => {
await use();
if (testInfo.status !== 'passed' && testInfo.tags.includes('@mesh-traffic')) {
testInfo.annotations.push({ type: 'notice', description: TRAFFIC_ADVISORY });
// Also print to console so it's visible in terminal output
console.log(`\n⚠ ${TRAFFIC_ADVISORY}\n`);
}
},
{ auto: true },
],
});

View File

@@ -1,4 +1,4 @@
import { test, expect } from '@playwright/test';
import { test, expect } from '../helpers/meshTrafficTest';
import { createChannel, getChannels, getMessages } from '../helpers/api';
/**
@@ -54,7 +54,7 @@ test.describe('Incoming mesh messages', () => {
}
});
test('receive an incoming message in any room', async ({ page }) => {
test('receive an incoming message in any room', { tag: '@mesh-traffic' }, async ({ page }) => {
await page.goto('/');
await expect(page.getByText('Connected')).toBeVisible();
@@ -102,7 +102,7 @@ test.describe('Incoming mesh messages', () => {
});
});
test('incoming message with path shows hop badge and path modal', async ({ page }) => {
test('incoming message with path shows hop badge and path modal', { tag: '@mesh-traffic' }, async ({ page }) => {
await page.goto('/');
await expect(page.getByText('Connected')).toBeVisible();

View File

@@ -1,4 +1,4 @@
import { test, expect } from '@playwright/test';
import { test, expect } from '../helpers/meshTrafficTest';
test.describe('Packet Feed page', () => {
test('packet feed page loads and shows header', async ({ page }) => {
@@ -7,7 +7,7 @@ test.describe('Packet Feed page', () => {
await expect(page.getByText('Raw Packet Feed')).toBeVisible({ timeout: 10_000 });
});
test('a packet appears in the raw packet feed', async ({ page }) => {
test('a packet appears in the raw packet feed', { tag: '@mesh-traffic' }, async ({ page }) => {
// This test waits for real RF traffic — needs 180s timeout
test.setTimeout(180_000);

View File

@@ -0,0 +1,434 @@
"""Tests for community MQTT publisher."""
import json
from unittest.mock import MagicMock, patch
import nacl.bindings
import pytest
from app.community_mqtt import (
_CLIENT_ID,
_DEFAULT_BROKER,
CommunityMqttPublisher,
_base64url_encode,
_calculate_packet_hash,
_ed25519_sign_expanded,
_format_raw_packet,
_generate_jwt_token,
community_mqtt_broadcast,
)
from app.models import AppSettings
def _make_test_keys() -> tuple[bytes, bytes]:
"""Generate a test MeshCore-format key pair.
Returns (private_key_64_bytes, public_key_32_bytes).
MeshCore format: scalar(32) || prefix(32), where scalar is already clamped.
"""
import hashlib
import os
seed = os.urandom(32)
expanded = hashlib.sha512(seed).digest()
scalar = bytearray(expanded[:32])
# Clamp scalar (standard Ed25519 clamping)
scalar[0] &= 248
scalar[31] &= 127
scalar[31] |= 64
scalar = bytes(scalar)
prefix = expanded[32:]
private_key = scalar + prefix
public_key = nacl.bindings.crypto_scalarmult_ed25519_base_noclamp(scalar)
return private_key, public_key
class TestBase64UrlEncode:
def test_encodes_without_padding(self):
result = _base64url_encode(b"\x00\x01\x02")
assert "=" not in result
def test_uses_url_safe_chars(self):
# Bytes that would produce + and / in standard base64
result = _base64url_encode(b"\xfb\xff\xfe")
assert "+" not in result
assert "/" not in result
class TestJwtGeneration:
def test_token_has_three_parts(self):
private_key, public_key = _make_test_keys()
token = _generate_jwt_token(private_key, public_key)
parts = token.split(".")
assert len(parts) == 3
def test_header_contains_ed25519_alg(self):
private_key, public_key = _make_test_keys()
token = _generate_jwt_token(private_key, public_key)
header_b64 = token.split(".")[0]
# Add padding for base64 decoding
import base64
padded = header_b64 + "=" * (4 - len(header_b64) % 4)
header = json.loads(base64.urlsafe_b64decode(padded))
assert header["alg"] == "Ed25519"
assert header["typ"] == "JWT"
def test_payload_contains_required_fields(self):
private_key, public_key = _make_test_keys()
token = _generate_jwt_token(private_key, public_key)
payload_b64 = token.split(".")[1]
import base64
padded = payload_b64 + "=" * (4 - len(payload_b64) % 4)
payload = json.loads(base64.urlsafe_b64decode(padded))
assert payload["publicKey"] == public_key.hex().upper()
assert "iat" in payload
assert "exp" in payload
assert payload["exp"] - payload["iat"] == 86400
assert payload["aud"] == _DEFAULT_BROKER
assert payload["owner"] == public_key.hex().upper()
assert payload["client"] == _CLIENT_ID
assert "email" not in payload # omitted when empty
def test_payload_includes_email_when_provided(self):
private_key, public_key = _make_test_keys()
token = _generate_jwt_token(private_key, public_key, email="test@example.com")
payload_b64 = token.split(".")[1]
import base64
padded = payload_b64 + "=" * (4 - len(payload_b64) % 4)
payload = json.loads(base64.urlsafe_b64decode(padded))
assert payload["email"] == "test@example.com"
def test_payload_uses_custom_audience(self):
private_key, public_key = _make_test_keys()
token = _generate_jwt_token(private_key, public_key, audience="custom.broker.net")
payload_b64 = token.split(".")[1]
import base64
padded = payload_b64 + "=" * (4 - len(payload_b64) % 4)
payload = json.loads(base64.urlsafe_b64decode(padded))
assert payload["aud"] == "custom.broker.net"
def test_signature_is_valid_hex(self):
private_key, public_key = _make_test_keys()
token = _generate_jwt_token(private_key, public_key)
sig_hex = token.split(".")[2]
sig_bytes = bytes.fromhex(sig_hex)
assert len(sig_bytes) == 64
def test_signature_verifies(self):
"""Verify the JWT signature using nacl.bindings.crypto_sign_open."""
private_key, public_key = _make_test_keys()
token = _generate_jwt_token(private_key, public_key)
parts = token.split(".")
signing_input = f"{parts[0]}.{parts[1]}".encode()
signature = bytes.fromhex(parts[2])
# crypto_sign_open expects signature + message concatenated
signed_message = signature + signing_input
# This will raise if the signature is invalid
verified = nacl.bindings.crypto_sign_open(signed_message, public_key)
assert verified == signing_input
class TestEddsaSignExpanded:
def test_produces_64_byte_signature(self):
private_key, public_key = _make_test_keys()
message = b"test message"
sig = _ed25519_sign_expanded(message, private_key[:32], private_key[32:], public_key)
assert len(sig) == 64
def test_signature_verifies_with_nacl(self):
private_key, public_key = _make_test_keys()
message = b"hello world"
sig = _ed25519_sign_expanded(message, private_key[:32], private_key[32:], public_key)
signed_message = sig + message
verified = nacl.bindings.crypto_sign_open(signed_message, public_key)
assert verified == message
def test_different_messages_produce_different_signatures(self):
private_key, public_key = _make_test_keys()
sig1 = _ed25519_sign_expanded(b"msg1", private_key[:32], private_key[32:], public_key)
sig2 = _ed25519_sign_expanded(b"msg2", private_key[:32], private_key[32:], public_key)
assert sig1 != sig2
class TestPacketFormatConversion:
def test_basic_field_mapping(self):
data = {
"id": 1,
"observation_id": 100,
"timestamp": 1700000000,
"data": "0a1b2c3d",
"payload_type": "ADVERT",
"snr": 5.5,
"rssi": -90,
"decrypted": False,
"decrypted_info": None,
}
result = _format_raw_packet(data, "TestNode", "AABBCCDD" * 8)
assert result["origin"] == "TestNode"
assert result["origin_id"] == "AABBCCDD" * 8
assert result["raw"] == "0A1B2C3D"
assert result["SNR"] == "5.5"
assert result["RSSI"] == "-90"
assert result["type"] == "PACKET"
assert result["direction"] == "rx"
assert result["len"] == "4"
def test_timestamp_is_iso8601(self):
data = {"timestamp": 1700000000, "data": "00", "snr": None, "rssi": None}
result = _format_raw_packet(data, "Node", "AA" * 32)
assert result["timestamp"]
assert "T" in result["timestamp"]
def test_snr_rssi_unknown_when_none(self):
data = {"timestamp": 0, "data": "00", "snr": None, "rssi": None}
result = _format_raw_packet(data, "Node", "AA" * 32)
assert result["SNR"] == "Unknown"
assert result["RSSI"] == "Unknown"
def test_packet_type_extraction(self):
# Header 0x14 = type 5, route 0 (TRANSPORT_FLOOD): header + 4 transport + path_len.
data = {"timestamp": 0, "data": "140102030400", "snr": None, "rssi": None}
result = _format_raw_packet(data, "Node", "AA" * 32)
assert result["packet_type"] == "5"
assert result["route"] == "F"
def test_route_mapping(self):
# Test all 4 route types (matches meshcore-packet-capture)
# TRANSPORT_FLOOD=0 -> "F", FLOOD=1 -> "F", DIRECT=2 -> "D", TRANSPORT_DIRECT=3 -> "T"
samples = [
("000102030400", "F"), # TRANSPORT_FLOOD: header + transport + path_len
("0100", "F"), # FLOOD: header + path_len
("0200", "D"), # DIRECT: header + path_len
("030102030400", "T"), # TRANSPORT_DIRECT: header + transport + path_len
]
for raw_hex, expected in samples:
data = {"timestamp": 0, "data": raw_hex, "snr": None, "rssi": None}
result = _format_raw_packet(data, "Node", "AA" * 32)
assert result["route"] == expected
def test_hash_is_16_uppercase_hex_chars(self):
data = {"timestamp": 0, "data": "aabb", "snr": None, "rssi": None}
result = _format_raw_packet(data, "Node", "AA" * 32)
assert len(result["hash"]) == 16
assert result["hash"] == result["hash"].upper()
def test_empty_data_handled(self):
data = {"timestamp": 0, "data": "", "snr": None, "rssi": None}
result = _format_raw_packet(data, "Node", "AA" * 32)
assert result["raw"] == ""
assert result["len"] == "0"
assert result["packet_type"] == "0"
assert result["route"] == "U"
def test_includes_reference_time_fields(self):
data = {"timestamp": 0, "data": "0100aabb", "snr": 1.0, "rssi": -70}
result = _format_raw_packet(data, "Node", "AA" * 32)
assert result["time"]
assert result["date"]
assert result["payload_len"] == "2"
def test_adds_path_for_direct_route(self):
# route=2 (DIRECT), path_len=2, path=aa bb, payload=cc
data = {"timestamp": 0, "data": "0202AABBCC", "snr": 1.0, "rssi": -70}
result = _format_raw_packet(data, "Node", "AA" * 32)
assert result["route"] == "D"
assert result["path"] == "aa,bb"
def test_direct_route_includes_empty_path_field(self):
data = {"timestamp": 0, "data": "0200", "snr": 1.0, "rssi": -70}
result = _format_raw_packet(data, "Node", "AA" * 32)
assert result["route"] == "D"
assert "path" in result
assert result["path"] == ""
def test_unknown_version_uses_defaults(self):
# version=1 in high bits, type=5, route=1
header = (1 << 6) | (5 << 2) | 1
data = {"timestamp": 0, "data": f"{header:02x}00", "snr": 1.0, "rssi": -70}
result = _format_raw_packet(data, "Node", "AA" * 32)
assert result["packet_type"] == "0"
assert result["route"] == "U"
assert result["payload_len"] == "0"
class TestCalculatePacketHash:
def test_empty_bytes_returns_zeroes(self):
result = _calculate_packet_hash(b"")
assert result == "0" * 16
def test_returns_16_uppercase_hex_chars(self):
# Simple flood packet: header(1) + path_len(1) + payload
raw = bytes([0x01, 0x00, 0xAA, 0xBB]) # FLOOD, no path, payload=0xAABB
result = _calculate_packet_hash(raw)
assert len(result) == 16
assert result == result.upper()
def test_flood_packet_hash(self):
"""FLOOD route (0x01): no transport codes, header + path_len + payload."""
import hashlib
# Header 0x11 = route=FLOOD(1), payload_type=4(ADVERT): (4<<2)|1 = 0x11
payload = b"\xde\xad"
raw = bytes([0x11, 0x00]) + payload # header, path_len=0, payload
result = _calculate_packet_hash(raw)
# Expected: sha256(payload_type_byte + payload_data)[:16].upper()
expected = hashlib.sha256(bytes([4]) + payload).hexdigest()[:16].upper()
assert result == expected
def test_transport_flood_skips_transport_codes(self):
"""TRANSPORT_FLOOD (0x00): has 4 bytes of transport codes after header."""
import hashlib
# Header 0x10 = route=TRANSPORT_FLOOD(0), payload_type=4: (4<<2)|0 = 0x10
transport_codes = b"\x01\x02\x03\x04"
payload = b"\xca\xfe"
raw = bytes([0x10]) + transport_codes + bytes([0x00]) + payload
result = _calculate_packet_hash(raw)
expected = hashlib.sha256(bytes([4]) + payload).hexdigest()[:16].upper()
assert result == expected
def test_transport_direct_skips_transport_codes(self):
"""TRANSPORT_DIRECT (0x03): also has 4 bytes of transport codes."""
import hashlib
# Header 0x13 = route=TRANSPORT_DIRECT(3), payload_type=4: (4<<2)|3 = 0x13
transport_codes = b"\x05\x06\x07\x08"
payload = b"\xbe\xef"
raw = bytes([0x13]) + transport_codes + bytes([0x00]) + payload
result = _calculate_packet_hash(raw)
expected = hashlib.sha256(bytes([4]) + payload).hexdigest()[:16].upper()
assert result == expected
def test_trace_packet_includes_path_len_in_hash(self):
"""TRACE packets (type 9) include path_len as uint16_t LE in the hash."""
import hashlib
# Header for TRACE with FLOOD route: (9<<2)|1 = 0x25
path_len = 3
path_data = b"\xaa\xbb\xcc"
payload = b"\x01\x02"
raw = bytes([0x25, path_len]) + path_data + payload
result = _calculate_packet_hash(raw)
expected_hash = (
hashlib.sha256(bytes([9]) + path_len.to_bytes(2, byteorder="little") + payload)
.hexdigest()[:16]
.upper()
)
assert result == expected_hash
def test_with_path_data(self):
"""Packet with non-zero path_len should skip path bytes to reach payload."""
import hashlib
# FLOOD route, payload_type=2 (TXT_MSG): (2<<2)|1 = 0x09
path_data = b"\xaa\xbb" # 2 bytes of path
payload = b"\x48\x65\x6c\x6c\x6f" # "Hello"
raw = bytes([0x09, 0x02]) + path_data + payload
result = _calculate_packet_hash(raw)
expected = hashlib.sha256(bytes([2]) + payload).hexdigest()[:16].upper()
assert result == expected
def test_truncated_packet_returns_zeroes(self):
# Header says TRANSPORT_FLOOD, but missing path_len at required offset.
raw = bytes([0x10, 0x01, 0x02])
assert _calculate_packet_hash(raw) == "0" * 16
class TestCommunityMqttPublisher:
def test_initial_state(self):
pub = CommunityMqttPublisher()
assert pub.connected is False
assert pub._client is None
assert pub._task is None
@pytest.mark.asyncio
async def test_publish_drops_when_disconnected(self):
pub = CommunityMqttPublisher()
# Should not raise
await pub.publish("topic", {"key": "value"})
@pytest.mark.asyncio
async def test_stop_resets_state(self):
pub = CommunityMqttPublisher()
pub.connected = True
pub._client = MagicMock()
await pub.stop()
assert pub.connected is False
assert pub._client is None
def test_is_configured_false_when_disabled(self):
pub = CommunityMqttPublisher()
pub._settings = AppSettings(community_mqtt_enabled=False)
with patch("app.keystore.has_private_key", return_value=True):
assert pub._is_configured() is False
def test_is_configured_false_when_no_private_key(self):
pub = CommunityMqttPublisher()
pub._settings = AppSettings(community_mqtt_enabled=True)
with patch("app.keystore.has_private_key", return_value=False):
assert pub._is_configured() is False
def test_is_configured_true_when_enabled_with_key(self):
pub = CommunityMqttPublisher()
pub._settings = AppSettings(community_mqtt_enabled=True)
with patch("app.keystore.has_private_key", return_value=True):
assert pub._is_configured() is True
class TestCommunityMqttBroadcast:
def test_filters_non_raw_packet(self):
"""Non-raw_packet events should be ignored."""
with patch("app.community_mqtt.community_publisher") as mock_pub:
mock_pub.connected = True
mock_pub._settings = AppSettings(community_mqtt_enabled=True)
community_mqtt_broadcast("message", {"text": "hello"})
# No asyncio.create_task should be called for non-raw_packet events
# Since we're filtering, we just verify no exception
def test_skips_when_disconnected(self):
"""Should not publish when disconnected."""
with (
patch("app.community_mqtt.community_publisher") as mock_pub,
patch("app.community_mqtt.asyncio.create_task") as mock_task,
):
mock_pub.connected = False
mock_pub._settings = AppSettings(community_mqtt_enabled=True)
community_mqtt_broadcast("raw_packet", {"data": "00"})
mock_task.assert_not_called()
def test_skips_when_settings_none(self):
"""Should not publish when settings are None."""
with (
patch("app.community_mqtt.community_publisher") as mock_pub,
patch("app.community_mqtt.asyncio.create_task") as mock_task,
):
mock_pub.connected = True
mock_pub._settings = None
community_mqtt_broadcast("raw_packet", {"data": "00"})
mock_task.assert_not_called()
class TestPublishFailureSetsDisconnected:
@pytest.mark.asyncio
async def test_publish_error_sets_connected_false(self):
"""A publish error should set connected=False so the loop can detect it."""
pub = CommunityMqttPublisher()
pub.connected = True
mock_client = MagicMock()
mock_client.publish = MagicMock(side_effect=Exception("broker gone"))
pub._client = mock_client
await pub.publish("topic", {"data": "test"})
assert pub.connected is False

View File

@@ -34,6 +34,30 @@ class TestHealthMqttStatus:
mqtt_publisher._settings = original_settings
mqtt_publisher.connected = original_connected
@pytest.mark.asyncio
async def test_mqtt_disabled_when_nothing_to_publish(self, test_db):
"""MQTT status is 'disabled' when broker host is set but no publish options enabled."""
from app.mqtt import mqtt_publisher
original_settings = mqtt_publisher._settings
original_connected = mqtt_publisher.connected
try:
from app.models import AppSettings
mqtt_publisher._settings = AppSettings(
mqtt_broker_host="broker.local",
mqtt_publish_messages=False,
mqtt_publish_raw_packets=False,
)
mqtt_publisher.connected = False
data = await build_health_data(True, "TCP: 1.2.3.4:4000")
assert data["mqtt_status"] == "disabled"
finally:
mqtt_publisher._settings = original_settings
mqtt_publisher.connected = original_connected
@pytest.mark.asyncio
async def test_mqtt_connected_when_publisher_connected(self, test_db):
"""MQTT status is 'connected' when publisher is connected."""
@@ -44,7 +68,9 @@ class TestHealthMqttStatus:
try:
from app.models import AppSettings
mqtt_publisher._settings = AppSettings(mqtt_broker_host="broker.local")
mqtt_publisher._settings = AppSettings(
mqtt_broker_host="broker.local", mqtt_publish_messages=True
)
mqtt_publisher.connected = True
data = await build_health_data(True, "TCP: 1.2.3.4:4000")
@@ -64,7 +90,9 @@ class TestHealthMqttStatus:
try:
from app.models import AppSettings
mqtt_publisher._settings = AppSettings(mqtt_broker_host="broker.local")
mqtt_publisher._settings = AppSettings(
mqtt_broker_host="broker.local", mqtt_publish_raw_packets=True
)
mqtt_publisher.connected = False
data = await build_health_data(False, None)

View File

@@ -100,8 +100,8 @@ class TestMigration001:
# Run migrations
applied = await run_migrations(conn)
assert applied == 31 # All migrations run
assert await get_version(conn) == 31
assert applied == 32 # All migrations run
assert await get_version(conn) == 32
# Verify columns exist by inserting and selecting
await conn.execute(
@@ -183,9 +183,9 @@ class TestMigration001:
applied1 = await run_migrations(conn)
applied2 = await run_migrations(conn)
assert applied1 == 31 # All migrations run
assert applied1 == 32 # All migrations run
assert applied2 == 0 # No migrations on second run
assert await get_version(conn) == 31
assert await get_version(conn) == 32
finally:
await conn.close()
@@ -246,8 +246,8 @@ class TestMigration001:
applied = await run_migrations(conn)
# All migrations applied (version incremented) but no error
assert applied == 31
assert await get_version(conn) == 31
assert applied == 32
assert await get_version(conn) == 32
finally:
await conn.close()
@@ -374,10 +374,10 @@ class TestMigration013:
)
await conn.commit()
# Run migration 13 (plus 14-27 which also run)
# Run migration 13 (plus 14-33 which also run)
applied = await run_migrations(conn)
assert applied == 19
assert await get_version(conn) == 31
assert applied == 20
assert await get_version(conn) == 32
# Verify bots array was created with migrated data
cursor = await conn.execute("SELECT bots FROM app_settings WHERE id = 1")
@@ -497,7 +497,7 @@ class TestMigration018:
assert await cursor.fetchone() is not None
await run_migrations(conn)
assert await get_version(conn) == 31
assert await get_version(conn) == 32
# Verify autoindex is gone
cursor = await conn.execute(
@@ -575,8 +575,8 @@ class TestMigration018:
await conn.commit()
applied = await run_migrations(conn)
assert applied == 14 # Migrations 18-31 run (18+19 skip internally)
assert await get_version(conn) == 31
assert applied == 15 # Migrations 18-32 run (18+19 skip internally)
assert await get_version(conn) == 32
finally:
await conn.close()
@@ -648,7 +648,7 @@ class TestMigration019:
assert await cursor.fetchone() is not None
await run_migrations(conn)
assert await get_version(conn) == 31
assert await get_version(conn) == 32
# Verify autoindex is gone
cursor = await conn.execute(
@@ -714,8 +714,8 @@ class TestMigration020:
assert (await cursor.fetchone())[0] == "delete"
applied = await run_migrations(conn)
assert applied == 12 # Migrations 20-31
assert await get_version(conn) == 31
assert applied == 13 # Migrations 20-32
assert await get_version(conn) == 32
# Verify WAL mode
cursor = await conn.execute("PRAGMA journal_mode")
@@ -745,7 +745,7 @@ class TestMigration020:
await set_version(conn, 20)
applied = await run_migrations(conn)
assert applied == 11 # Migrations 21-31 still run
assert applied == 12 # Migrations 21-32 still run
# Still WAL + INCREMENTAL
cursor = await conn.execute("PRAGMA journal_mode")
@@ -803,8 +803,8 @@ class TestMigration028:
await conn.commit()
applied = await run_migrations(conn)
assert applied == 4
assert await get_version(conn) == 31
assert applied == 5
assert await get_version(conn) == 32
# Verify payload_hash column is now BLOB
cursor = await conn.execute("PRAGMA table_info(raw_packets)")
@@ -873,8 +873,8 @@ class TestMigration028:
await conn.commit()
applied = await run_migrations(conn)
assert applied == 4 # Version still bumped
assert await get_version(conn) == 31
assert applied == 5 # Version still bumped
assert await get_version(conn) == 32
# Verify data unchanged
cursor = await conn.execute("SELECT payload_hash FROM raw_packets")
@@ -882,3 +882,62 @@ class TestMigration028:
assert bytes(row["payload_hash"]) == b"\xab" * 32
finally:
await conn.close()
class TestMigration032:
"""Test migration 032: add community MQTT columns to app_settings."""
@pytest.mark.asyncio
async def test_migration_adds_all_community_mqtt_columns(self):
"""Migration adds enabled, iata, broker, and email columns."""
conn = await aiosqlite.connect(":memory:")
conn.row_factory = aiosqlite.Row
try:
await set_version(conn, 31)
# Create app_settings without community columns (pre-migration schema)
await conn.execute("""
CREATE TABLE app_settings (
id INTEGER PRIMARY KEY,
max_radio_contacts INTEGER DEFAULT 200,
favorites TEXT DEFAULT '[]',
auto_decrypt_dm_on_advert INTEGER DEFAULT 0,
sidebar_sort_order TEXT DEFAULT 'recent',
last_message_times TEXT DEFAULT '{}',
preferences_migrated INTEGER DEFAULT 0,
advert_interval INTEGER DEFAULT 0,
last_advert_time INTEGER DEFAULT 0,
bots TEXT DEFAULT '[]',
mqtt_broker_host TEXT DEFAULT '',
mqtt_broker_port INTEGER DEFAULT 1883,
mqtt_username TEXT DEFAULT '',
mqtt_password TEXT DEFAULT '',
mqtt_use_tls INTEGER DEFAULT 0,
mqtt_tls_insecure INTEGER DEFAULT 0,
mqtt_topic_prefix TEXT DEFAULT 'meshcore',
mqtt_publish_messages INTEGER DEFAULT 0,
mqtt_publish_raw_packets INTEGER DEFAULT 0
)
""")
await conn.execute("INSERT INTO app_settings (id) VALUES (1)")
await conn.commit()
applied = await run_migrations(conn)
assert applied == 1
assert await get_version(conn) == 32
# Verify all columns exist with correct defaults
cursor = await conn.execute(
"""SELECT community_mqtt_enabled, community_mqtt_iata,
community_mqtt_broker_host, community_mqtt_broker_port,
community_mqtt_email
FROM app_settings WHERE id = 1"""
)
row = await cursor.fetchone()
assert row["community_mqtt_enabled"] == 0
assert row["community_mqtt_iata"] == ""
assert row["community_mqtt_broker_host"] == "mqtt-us-v1.letsmesh.net"
assert row["community_mqtt_broker_port"] == 443
assert row["community_mqtt_email"] == ""
finally:
await conn.close()

View File

@@ -81,12 +81,12 @@ class TestMqttPublisher:
def test_not_configured_when_host_empty(self):
pub = MqttPublisher()
pub._settings = _make_settings(mqtt_broker_host="")
assert pub._mqtt_configured() is False
assert pub._is_configured() is False
def test_configured_when_host_set(self):
pub = MqttPublisher()
pub._settings = _make_settings(mqtt_broker_host="broker.local")
assert pub._mqtt_configured() is True
assert pub._is_configured() is True
@pytest.mark.asyncio
async def test_publish_drops_silently_when_disconnected(self):
@@ -300,8 +300,8 @@ class TestConnectionLoop:
mock_client.__aenter__ = AsyncMock(side_effect=side_effect_aenter)
with (
patch("app.mqtt.aiomqtt.Client", return_value=mock_client),
patch("app.mqtt._broadcast_mqtt_health"),
patch("app.mqtt_base.aiomqtt.Client", return_value=mock_client),
patch("app.mqtt_base._broadcast_health"),
patch("app.websocket.broadcast_success"),
patch("app.websocket.broadcast_health"),
):
@@ -321,7 +321,7 @@ class TestConnectionLoop:
"""Connection loop should retry after a connection error with backoff."""
import asyncio
from app.mqtt import _BACKOFF_MIN
from app.mqtt_base import _BACKOFF_MIN
pub = MqttPublisher()
settings = _make_settings()
@@ -354,12 +354,12 @@ class TestConnectionLoop:
return factory
with (
patch("app.mqtt.aiomqtt.Client", side_effect=make_client_factory()),
patch("app.mqtt._broadcast_mqtt_health"),
patch("app.mqtt_base.aiomqtt.Client", side_effect=make_client_factory()),
patch("app.mqtt_base._broadcast_health"),
patch("app.websocket.broadcast_success"),
patch("app.websocket.broadcast_error"),
patch("app.websocket.broadcast_health"),
patch("app.mqtt.asyncio.sleep", new_callable=AsyncMock) as mock_sleep,
patch("app.mqtt_base.asyncio.sleep", new_callable=AsyncMock) as mock_sleep,
):
await pub.start(settings)
@@ -375,10 +375,10 @@ class TestConnectionLoop:
@pytest.mark.asyncio
async def test_backoff_increases_on_repeated_failures(self):
"""Backoff should double after each failure, capped at _BACKOFF_MAX."""
"""Backoff should double after each failure, capped at _backoff_max."""
import asyncio
from app.mqtt import _BACKOFF_MAX, _BACKOFF_MIN
from app.mqtt_base import _BACKOFF_MIN
pub = MqttPublisher()
settings = _make_settings()
@@ -408,11 +408,11 @@ class TestConnectionLoop:
raise asyncio.CancelledError
with (
patch("app.mqtt.aiomqtt.Client", side_effect=factory),
patch("app.mqtt._broadcast_mqtt_health"),
patch("app.mqtt_base.aiomqtt.Client", side_effect=factory),
patch("app.mqtt_base._broadcast_health"),
patch("app.websocket.broadcast_error"),
patch("app.websocket.broadcast_health"),
patch("app.mqtt.asyncio.sleep", side_effect=capture_sleep),
patch("app.mqtt_base.asyncio.sleep", side_effect=capture_sleep),
):
await pub.start(settings)
try:
@@ -423,8 +423,8 @@ class TestConnectionLoop:
assert sleep_args[0] == _BACKOFF_MIN
assert sleep_args[1] == _BACKOFF_MIN * 2
assert sleep_args[2] == _BACKOFF_MIN * 4
# Fourth should be capped at _BACKOFF_MAX (5*8=40 > 30)
assert sleep_args[3] == _BACKOFF_MAX
# Fourth should be capped at _backoff_max (5*8=40 > 30)
assert sleep_args[3] == MqttPublisher._backoff_max
@pytest.mark.asyncio
async def test_waits_for_settings_when_unconfigured(self):
@@ -449,8 +449,8 @@ class TestConnectionLoop:
return mock
with (
patch("app.mqtt.aiomqtt.Client", side_effect=make_success_client),
patch("app.mqtt._broadcast_mqtt_health"),
patch("app.mqtt_base.aiomqtt.Client", side_effect=make_success_client),
patch("app.mqtt_base._broadcast_health"),
patch("app.websocket.broadcast_success"),
patch("app.websocket.broadcast_health"),
):
@@ -472,7 +472,7 @@ class TestConnectionLoop:
@pytest.mark.asyncio
async def test_health_broadcast_on_connect_and_failure(self):
"""_broadcast_mqtt_health should be called on connect and on failure."""
"""_broadcast_health should be called on connect and on failure."""
import asyncio
pub = MqttPublisher()
@@ -497,8 +497,8 @@ class TestConnectionLoop:
return mock
with (
patch("app.mqtt.aiomqtt.Client", side_effect=make_client),
patch("app.mqtt._broadcast_mqtt_health", side_effect=track_health),
patch("app.mqtt_base.aiomqtt.Client", side_effect=make_client),
patch("app.mqtt_base._broadcast_health", side_effect=track_health),
patch("app.websocket.broadcast_success"),
patch("app.websocket.broadcast_health"),
):
@@ -512,7 +512,7 @@ class TestConnectionLoop:
@pytest.mark.asyncio
async def test_health_broadcast_on_connection_error(self):
"""_broadcast_mqtt_health should be called when connection fails."""
"""_broadcast_health should be called when connection fails."""
import asyncio
pub = MqttPublisher()
@@ -534,11 +534,11 @@ class TestConnectionLoop:
return mock
with (
patch("app.mqtt.aiomqtt.Client", side_effect=make_failing_client),
patch("app.mqtt._broadcast_mqtt_health", side_effect=track_health),
patch("app.mqtt_base.aiomqtt.Client", side_effect=make_failing_client),
patch("app.mqtt_base._broadcast_health", side_effect=track_health),
patch("app.websocket.broadcast_error"),
patch("app.websocket.broadcast_health"),
patch("app.mqtt.asyncio.sleep", side_effect=cancel_on_sleep),
patch("app.mqtt_base.asyncio.sleep", side_effect=cancel_on_sleep),
):
await pub.start(settings)
try:

View File

@@ -502,6 +502,11 @@ class TestAppSettingsRepository:
"mqtt_topic_prefix": "meshcore",
"mqtt_publish_messages": 0,
"mqtt_publish_raw_packets": 0,
"community_mqtt_enabled": 0,
"community_mqtt_iata": "",
"community_mqtt_broker_host": "mqtt-us-v1.letsmesh.net",
"community_mqtt_broker_port": 443,
"community_mqtt_email": "",
}
)
mock_conn.execute = AsyncMock(return_value=mock_cursor)

View File

@@ -117,6 +117,81 @@ class TestUpdateSettings:
assert settings.mqtt_publish_messages is False
assert settings.mqtt_publish_raw_packets is False
@pytest.mark.asyncio
async def test_community_mqtt_fields_round_trip(self, test_db):
"""Community MQTT settings should be saved and retrieved correctly."""
mock_community = type("MockCommunity", (), {"restart": AsyncMock()})()
with patch("app.community_mqtt.community_publisher", mock_community):
result = await update_settings(
AppSettingsUpdate(
community_mqtt_enabled=True,
community_mqtt_iata="DEN",
community_mqtt_broker_host="custom-broker.example.com",
community_mqtt_broker_port=8883,
community_mqtt_email="test@example.com",
)
)
assert result.community_mqtt_enabled is True
assert result.community_mqtt_iata == "DEN"
assert result.community_mqtt_broker_host == "custom-broker.example.com"
assert result.community_mqtt_broker_port == 8883
assert result.community_mqtt_email == "test@example.com"
# Verify persistence
fresh = await AppSettingsRepository.get()
assert fresh.community_mqtt_enabled is True
assert fresh.community_mqtt_iata == "DEN"
assert fresh.community_mqtt_broker_host == "custom-broker.example.com"
assert fresh.community_mqtt_broker_port == 8883
assert fresh.community_mqtt_email == "test@example.com"
# Verify restart was called
mock_community.restart.assert_called_once()
@pytest.mark.asyncio
async def test_community_mqtt_iata_validation_rejects_invalid(self, test_db):
"""Invalid IATA codes should be rejected."""
with pytest.raises(HTTPException) as exc:
await update_settings(AppSettingsUpdate(community_mqtt_iata="A"))
assert exc.value.status_code == 400
with pytest.raises(HTTPException) as exc:
await update_settings(AppSettingsUpdate(community_mqtt_iata="ABCDE"))
assert exc.value.status_code == 400
with pytest.raises(HTTPException) as exc:
await update_settings(AppSettingsUpdate(community_mqtt_iata="12"))
assert exc.value.status_code == 400
with pytest.raises(HTTPException) as exc:
await update_settings(AppSettingsUpdate(community_mqtt_iata="ABCD"))
assert exc.value.status_code == 400
@pytest.mark.asyncio
async def test_community_mqtt_enable_requires_iata(self, test_db):
"""Enabling community MQTT without a valid IATA code should be rejected."""
with pytest.raises(HTTPException) as exc:
await update_settings(AppSettingsUpdate(community_mqtt_enabled=True))
assert exc.value.status_code == 400
assert "IATA" in exc.value.detail
@pytest.mark.asyncio
async def test_community_mqtt_iata_uppercased(self, test_db):
"""IATA codes should be uppercased."""
mock_community = type("MockCommunity", (), {"restart": AsyncMock()})()
with patch("app.community_mqtt.community_publisher", mock_community):
result = await update_settings(AppSettingsUpdate(community_mqtt_iata="den"))
assert result.community_mqtt_iata == "DEN"
@pytest.mark.asyncio
async def test_community_mqtt_defaults_on_fresh_db(self, test_db):
"""Community MQTT fields should have correct defaults on a fresh database."""
settings = await AppSettingsRepository.get()
assert settings.community_mqtt_enabled is False
assert settings.community_mqtt_iata == ""
assert settings.community_mqtt_email == ""
class TestToggleFavorite:
@pytest.mark.asyncio

36
uv.lock generated
View File

@@ -420,6 +420,19 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/c4/cb/00451c3cf31790287768bb12c6bec834f5d292eaf3022afc88e14b8afc94/paho_mqtt-2.1.0-py3-none-any.whl", hash = "sha256:6db9ba9b34ed5bc6b6e3812718c7e06e2fd7444540df2455d2c51bd58808feee", size = 67219 },
]
[[package]]
name = "pip-licenses"
version = "5.5.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "prettytable" },
{ name = "tomli", marker = "python_full_version < '3.11'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/44/4c/b4be9024dae3b5b3c0a6c58cc1d4a35fffe51c3adb835350cb7dcd43b5cd/pip_licenses-5.5.1.tar.gz", hash = "sha256:7df370e6e5024a3f7449abf8e4321ef868ba9a795698ad24ab6851f3e7fc65a7", size = 49108 }
wheels = [
{ url = "https://files.pythonhosted.org/packages/a0/a3/0b369cdffef3746157712804f1ded9856c75aa060217ee206f742c74e753/pip_licenses-5.5.1-py3-none-any.whl", hash = "sha256:ed5e229a93760e529cfa7edaec6630b5a2cd3874c1bddb8019e5f18a723fdead", size = 22108 },
]
[[package]]
name = "pluggy"
version = "1.6.0"
@@ -429,6 +442,18 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538 },
]
[[package]]
name = "prettytable"
version = "3.17.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "wcwidth" },
]
sdist = { url = "https://files.pythonhosted.org/packages/79/45/b0847d88d6cfeb4413566738c8bbf1e1995fad3d42515327ff32cc1eb578/prettytable-3.17.0.tar.gz", hash = "sha256:59f2590776527f3c9e8cf9fe7b66dd215837cca96a9c39567414cbc632e8ddb0", size = 67892 }
wheels = [
{ url = "https://files.pythonhosted.org/packages/ee/8c/83087ebc47ab0396ce092363001fa37c17153119ee282700c0713a195853/prettytable-3.17.0-py3-none-any.whl", hash = "sha256:aad69b294ddbe3e1f95ef8886a060ed1666a0b83018bbf56295f6f226c43d287", size = 34433 },
]
[[package]]
name = "pycayennelpp"
version = "2.4.0"
@@ -922,6 +947,7 @@ test = [
[package.dev-dependencies]
dev = [
{ name = "httpx" },
{ name = "pip-licenses" },
{ name = "pyright" },
{ name = "pytest" },
{ name = "pytest-asyncio" },
@@ -949,6 +975,7 @@ provides-extras = ["test"]
[package.metadata.requires-dev]
dev = [
{ name = "httpx", specifier = ">=0.28.1" },
{ name = "pip-licenses", specifier = ">=5.0.0" },
{ name = "pyright", specifier = ">=1.1.390" },
{ name = "pytest", specifier = ">=9.0.2" },
{ name = "pytest-asyncio", specifier = ">=1.3.0" },
@@ -1237,6 +1264,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/6e/d4/ed38dd3b1767193de971e694aa544356e63353c33a85d948166b5ff58b9e/watchfiles-1.1.1-pp311-pypy311_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:3e6f39af2eab0118338902798b5aa6664f46ff66bc0280de76fca67a7f262a49", size = 457546 },
]
[[package]]
name = "wcwidth"
version = "0.6.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/35/a2/8e3becb46433538a38726c948d3399905a4c7cabd0df578ede5dc51f0ec2/wcwidth-0.6.0.tar.gz", hash = "sha256:cdc4e4262d6ef9a1a57e018384cbeb1208d8abbc64176027e2c2455c81313159", size = 159684 }
wheels = [
{ url = "https://files.pythonhosted.org/packages/68/5a/199c59e0a824a3db2b89c5d2dade7ab5f9624dbf6448dc291b46d5ec94d3/wcwidth-0.6.0-py3-none-any.whl", hash = "sha256:1a3a1e510b553315f8e146c54764f4fb6264ffad731b3d78088cdb1478ffbdad", size = 94189 },
]
[[package]]
name = "websockets"
version = "15.0.1"