- Updated byte representations in tests to use lowercase hex format for consistency.
- Reformatted code for better readability, including line breaks and indentation adjustments.
- Consolidated multiple lines into single lines where appropriate to enhance clarity.
- Ensured that all test cases maintain consistent formatting and style across the test suite.
Also extend the compat scanner to catch datetime.UTC used as an
attribute (datetime.datetime.now(datetime.UTC)) in addition to
direct imports, so this form cannot be reintroduced undetected.
Co-Authored-By: Zindello <josh@zindello.com.au>
Replace the try/except UTC shim in mqtt_handler.py with timezone.utc
directly, consistent with the api_endpoints.py fix. Add a static AST
scan that fails if datetime.UTC is reintroduced anywhere in the codebase,
guarding against future regressions on Python 3.10 (LuckFox Pico Ultra).
Co-Authored-By: Zindello <josh@zindello.com.au>
- Introduced tests for TraceHelper and DiscoveryHelper to validate packet forwarding and discovery request handling.
- Implemented tests for LoginHelper to ensure identity registration and login packet processing.
- Added tests for IdentityManager to cover identity registration, lookup, and filtering.
- Created tests for MeshCLI to verify command handling, configuration setting, and error paths.
Lets a fresh repeater install pick the pymc_usb (USB-CDC) or pymc_tcp
(Wi-Fi/Ethernet) external modem from the first-run /setup wizard
instead of requiring the user to hand-edit config.yaml after install.
radio-settings.json gets two new hardware entries; setup_wizard()
in api_endpoints.py handles them in dedicated branches that mirror
the existing KISS pattern (placeholders if the SPA doesn't yet send
modem-specific inputs, request body overrides if it does).
For pymc_tcp the wizard writes a sentinel host placeholder
('REPLACE_WITH_MODEM_HOST') so the YAML stays valid; on startup
get_radio_for_board() then errors with a clear pointer at
pymc_tcp.host (existing behavior from the PR #240 branch). pymc_usb
defaults to /dev/ttyACM0 at 921600 baud — matches the USB-CDC
device path documented in pymc_usb's README + pymc_driver.
Five new tests in tests/test_setup_wizard_pymc.py verify both
default and overridden code paths plus a KISS regression guard.
- Updated the __call__ method to return a boolean indicating transmission success.
- Adjusted handling of transmission failures to increment dropped_count appropriately.
- Enhanced logging for transmission failures in PacketRouter to ensure visibility of issues.
- Added tests to verify correct behavior during transmission exceptions and failures.
Adds a new read-only endpoint that serves the bundled `repeater/presets/*.yaml`
catalogue so the admin UI can render a network picker without bundling its own
copy of the broker dicts. The UI side of this is paired with
pyMC-dev/pyMC-RepeaterUI#TBD which retires src/assets/broker-templates.json
in favour of authClient.get('/api/broker_presets').
Why
The UI previously shipped a separate JSON snapshot of every supported MC2MQTT
network. The JSON and these YAML files drifted: the Waev entry on the UI side
pointed at mqtt-a.waev.app with audience mqtt.waev.app (single primary, no
failover) while the YAML side here listed two brokers (A + B). The result was
that operators picking 'Waev' from the dropdown silently lost the redundancy
this preset is meant to provide.
What changes
repeater/presets/*.yaml
- Add optional top-level `display_name` and `website` fields. The loader
treats them as advisory metadata for the UI; the runtime connection code
never reads them. `display_name` falls back to the titlecased filename
stem if absent so existing third-party presets keep rendering.
repeater/presets/waev.yaml
- Collapse from two broker entries (waev-a, waev-b) to a single broker on
`mqtt.waev.app`. The Waev edge Worker (see waev/src/router.ts:
MQTT_PRIMARY_FAILOVER_TIMEOUT_MS) already does server-side A/B failover on
the alias host with a 1500 ms timeout. Two independent client connections
would defeat the dedup-on-pubkey-hash contract on the waev ingest side.
Operators who want to pin to a specific container can edit host/audience
after import.
repeater/presets/meshmapper.yaml (new)
- Port of the historical MeshMapper entry from the UI's deprecated JSON.
Single broker on mqtt.meshmapper.cc, format: letsmesh (matches the
published wire contract; bump to a dedicated value if/when wire-level
differentiation lands).
repeater/web/api_endpoints.py
- New `broker_presets` CherryPy handler at `GET /api/broker_presets`.
Unauthenticated to match the existing `mqtt_status` precedent — the
response carries only public hostnames + TLS hints, no PII. Imports the
presets module lazily so a broken YAML never blocks process startup.
Response shape:
{
success: true,
data: [{ id, name, website?, brokers: [ ... raw YAML dicts ... ] }, …]
}
tests/test_presets.py
- Locks the new metadata fields (display_name, website) on all three presets.
- Locks the Waev single-alias-broker design with an explicit comment tying
the test to the waev Worker failover code.
- Adds MeshMapper coverage parallel to the other public-network presets.
- Adds a stub-instance test that drives the new `broker_presets` method on
an APIEndpoints stand-in (bypassing the heavyweight `__init__`) and
asserts the UI-ready response shape.
Verification
- New endpoint serves the expected three presets (letsmesh: 2 brokers,
meshmapper: 1, waev: 1) when exercised end-to-end against a local mock
that imports the real preset loader.
- Existing legacy-config migration tests (broker_index 0/1/-1 → preset +
overrides) still pass — the override pipeline is untouched.
Co-Authored-By: Oz <oz-agent@warp.dev>
Wires the TCPLoRaRadio and USBLoRaRadio drivers that landed in pyMC_core
on 2026-05-13 (PR pyMC-dev/pyMC_core#68) into get_radio_for_board() so
they can be selected from a repeater config file without any code change
in main.py / api_endpoints.
Both branches follow the existing pattern: read host/port (TCP) or
serial port (USB) plus auth/LBT options from their own config section,
share the LoRa parameters from the common `radio` section, fall back to
the firmware-default sync word 0x12, and surface ImportError as a clear
RuntimeError if the installed pymc_core is too old to ship the drivers.
config.yaml.example documents both sections and updates the radio_type
header comment with the full supported list. Five new tests in
tests/test_radio_config.py monkeypatch the radio classes and verify the
section/parameter wiring + missing-required-field errors.
No web UI / endpoint changes — the deployment this targets edits the
config file directly. A GUI wizard for these radio types can land
separately if there's appetite.
Every MQTT-published packet has shipped with duration="0" since the
PacketRecord factory was introduced. The repeater already computes LoRa
time-on-air via AirtimeManager.calculate_airtime() (the canonical
Semtech reference formula) for duty-cycle gating and TX delay, but the
result was thrown away after each packet - never stored on the
packet_record dict that flows to MQTT/SQLite/Glass/websocket.
What changes
- engine.py: RepeaterHandler._build_packet_record() now computes
airtime_ms once per packet (Semtech formula via AirtimeManager) and
stores it as packet_record['airtime_ms']. Single source of truth for
every downstream consumer.
- storage_utils.py: PacketRecord.from_packet_record() reads the new
airtime_ms field and serializes it as a rounded integer in the
'duration' field of the published JSON. Falls back to 0 if the field
is missing (backward compatibility for any older code path).
- storage_collector.py: _publish_packet_to_mqtt() simplified - no
recomputation, no helper. The publish path is now a passthrough.
Why
MQTT consumers (firmware-compatible analyzers, dashboards, the upstream
meshcoretomqtt project) expect the same time-on-air value the firmware
emits. Hard-coded "0" makes airtime/utilization charts derived from the
mqtt stream useless and silently diverges from firmware behavior.
Plumbing the value through packet_record (instead of recomputing in the
publish path) means any future consumer - SQLite schema, web UI charts,
Glass telemetry - reads the same number without separate calculations.
Tests
tests/test_packet_duration.py - 5 tests covering:
- backward compat (legacy packet_record without airtime_ms => '0')
- airtime_ms field flows through to duration as rounded integer string
- explicit zero stays '0'
- AirtimeManager output matches an independently-implemented Semtech
reference for typical MeshCore EU settings (SF8/62.5kHz/CR4-8)
- low-data-rate optimization branch (SF12/125kHz triggers DE=1)
Co-Authored-By: Oz <oz-agent@warp.dev>
Introduces a 'set format and forget' workflow for MQTT brokers. Users
reference a bundled preset by name inside the existing brokers: list,
and the package supplies the endpoints, audiences, and TLS settings.
Endpoint changes ship via 'pip install -U' instead of manual edits.
What changes
- New repeater/presets/ package with a tiny lazy YAML loader and two
bundled presets: waev (mqtt-{a,b}.waev.app) and letsmesh (EU + US).
- New format-family constant MC2MQTT_FORMATS = ('meshcoretomqtt',
'letsmesh', 'waev') replaces the inline tuple in topic resolution.
The legacy 'mqtt' format keeps its custom-topic semantics unchanged.
- Two-pass broker assembly in mqtt_handler.py: pass 1 expands every
{preset: <name>} entry inline; pass 2 collapses duplicates by name
with later-wins semantics. Place override entries AFTER preset
entries.
- Hard-coded LETSMESH_BROKERS constant deleted; its data now lives in
repeater/presets/letsmesh.yaml.
- convert_letsmesh_to_broker_config() collapsed from ~70 to ~25 lines
by emitting {preset: letsmesh} plus disable overrides for unwanted
brokers. Honors broker_index in (-1, 0, 1), additional_brokers, and
enabled flag exactly as before.
- update_mqtt_config API endpoint accepts {preset: <name>} entries and
passes them through unchanged so the web UI can author them when the
frontend is updated.
- config.yaml.example documents the preset entry shape, the override
rule, and the format family hierarchy.
- pyproject.toml ships presets/*.yaml as package data.
How to use
mqtt_brokers:
iata_code: "LAX"
brokers:
- preset: waev
# Override a single preset broker:
brokers:
- preset: waev
- name: waev-b
enabled: false
Tests
- tests/test_presets.py: 9 tests covering loader, expand/merge,
MC2MQTT topic-family parity, and parametrized legacy migration.
Co-Authored-By: Oz <oz-agent@warp.dev>
- Introduced options for using GPS coordinates for repeater location fields in config.
- Implemented precision control for GPS coordinates.
- Added a new API endpoint for a Server-Sent Events stream of GPS diagnostics.
- Updated GPSService to handle new configuration options and fallback logic.
- Enhanced unit tests for GPS location handling.
- Introduced `en_pin` and `en_pins` parameters in radio configuration.
- Updated `get_radio_for_board` to handle new configuration options.
- Added unit tests to verify correct handling of `en_pins`.
Addresses PR 191 reviewer feedback:
1. Shutdown drain
stop() now waits up to 5 s for in-flight _route_packet tasks to finish,
then cancels any that remain. Previously only the queue-consumer loop was
cancelled; created tasks were abandoned with no guarantee they completed.
Mechanism: _route_tasks set tracks live tasks (added on create, discarded
in the done-callback). stop() takes a snapshot and calls asyncio.wait()
with timeout=5.0, then cancels the still-pending subset.
2. Drop counter
_cap_drop_count increments each time a packet is dropped at the cap.
The running total is included in every WARNING log line and also printed
at shutdown so operators can tell at a glance whether the safety valve is
actually firing in production.
3. Tests (tests/test_packet_router.py)
test_cap_drops_packets_when_full — cap=3, send 8 → 5 drops, 3 in-flight
test_cap_drop_count_increments — count increments by 1 per drop
test_cap_drop_count_zero_... — count stays 0 when cap never reached
test_stop_waits_for_in_flight_tasks — slow task (0.2 s) completes, not cancelled
test_stop_cancels_tasks_...timeout — hanging task cancelled after timeout
test_route_tasks_set_cleaned_up — set empty after all tasks finish
test_counter_matches_set_size — _in_flight == len(_route_tasks) at cap
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Reviewer concern (PR 190):
The 1-second backoff sleep for local_transmission retry happened inside
`async with self._tx_lock`, blocking all other queued TX tasks for the
full second — hurting latency and throughput under load.
Fix — tighten lock scope to one attempt per acquisition:
Before: acquire lock → [attempt 0 → sleep(1) → attempt 1] → release
After: for each attempt:
[sleep(1) if retry] ← OUTSIDE the lock
acquire lock
re-check can_transmit ← fresh check every acquisition
attempt single send
record_tx on success
release lock
The duty-cycle gate now runs on every lock acquisition (not just the first),
which is correct: airtime state may change during the backoff sleep.
Tests added (tests/test_tx_lock.py):
1. test_concurrent_sends_do_not_interleave — two tasks racing to the same
delay timer must never overlap inside send_packet.
2. test_duty_cycle_toctou_is_fixed — second packet is dropped when the
first consumes the budget inside the lock.
3. test_local_retry_releases_lock_during_backoff — a concurrent relayed
packet fires at ~0.1s while local retry sleeps 1s; confirms it is not
blocked by the backoff.
4. test_non_local_failure_propagates — relayed send failure raises
immediately with exactly one attempt.
5. test_duty_cycle_rechecked_on_retry — if the budget is exhausted during
backoff, the retry is dropped by the in-lock gate (not sent).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Updated record_packet_only method to skip logging for TRACE packets, as TraceHelper manages trace paths.
- Enhanced documentation to clarify the handling of TRACE packets in the web UI.
- Added tests to ensure TRACE packets are not recorded, maintaining data integrity.
- Added local_identity parameter to RepeaterDaemon for improved trace path matching.
- Refactored trace path handling in TraceHelper to support multi-byte hashes and structured hops.
- Updated methods to ensure compatibility with new trace data formats and improved logging.
- Enhanced tests to validate new trace processing logic and path handling.
- Added functionality to heal companion registration names with empty values.
- Improved handling of identity keys and public key derivation for companions.
- Updated API endpoints to support companion identity lookups using name, lookup_identity_key, or public_key_prefix.
- Enhanced OpenAPI documentation to clarify requirements for identity creation, updates, and deletions, including trimming whitespace from names.
- Modify TX modes: forward, monitor, and add no_tx, allowing for flexible packet handling.
- Updated configuration and API endpoints to support the new modes.
- Adjusted logic in RepeaterHandler to manage packet processing based on the selected mode.
- Enhanced CLI commands to reflect the new mode settings.
- Added tests for each TX mode to ensure correct behavior.
- Functionality of Packet.apply_path_hash_mode and get_path_hashes
- Engine flood_forward and direct_forward with real multi-byte encoded packets
- PacketBuilder.create_trace payload structure and TraceHandler parsing
- Enforcement of max-hop boundaries per hash size
- Removed redundant original_path assignment in `RepeaterHandler` to streamline packet processing.
- Introduced `_is_direct_final_hop` helper method in `PacketRouter` to determine if a packet is the final destination for direct routes with an empty path.
- Updated comments in `PacketRouter` to clarify the handling of packets during routing, especially for direct forwarding scenarios.
- Adjusted logic to ensure packets are correctly processed or delivered based on their routing status, enhancing overall packet management.
- Introduced `path_hash_mode` setting in `config.yaml.example` to specify the hash size for flood packets.
- Updated `ConfigManager` to re-apply the path hash mode when the mesh section changes, with validation for acceptable values (0, 1, 2).
- Enhanced `RepeaterDaemon` to set the default path hash mode during initialization, ensuring consistent handling of flood packets.
- Removed redundant call to mark_seen() for duplicate packets.
- Added validation to ensure hop count does not exceed the maximum limit before appending to the packet path.
- Updated logic to check for path size constraints when appending hash bytes, improving packet processing efficiency.
Rightup