The PATH and protocol-response paths recorded the companion dedupe
unconditionally after the bridge fan-out, so a delivery where every
bridge raised was still suppressed for the full dedupe TTL — the client
lost the packet even though later mesh copies arrived. Have the fan-out
report delivery (at least one bridge completed without raising) next to
authentication, and mark the dedupe only on delivery, which is the
behaviour the marking helper's contract already documented. One healthy
bridge still counts as delivered, so duplicate suppression of repeated
mesh copies is unchanged.
_consume_via_local_candidates awaited the targeted companion bridge with
no exception guard, so a raising bridge aborted the whole candidate loop:
the hash-colliding room-server or repeater identity registered at the
same one-byte dest hash was never offered the packet, and an unclaimed
packet never fell through to the forwarding engine. Wrap the bridge call
with the same log-and-continue handling the fan-out path already uses.
Replace the eight hand-rolled companion-bridge delivery loops in the
packet router with a single _fan_out_to_bridges helper that reads
HandlerResult.authenticated directly, so a broken handler contract
surfaces instead of being hidden by getattr hedges.
Introduce a DropReason str-Enum in the engine for the seventeen fixed
drop reasons (with __str__ returning the value, since str() of a
str-Enum changed on Python 3.11+) and derive the router's
expected-drop check from it, retiring the string-prefix tuple.
Detailed variants keep their suffixes embedded in the reason string.
Split the companion-delivery dedupe cache into a pure check and an
explicit mark so PATH and protocol responses record delivery only
after the fan-out runs — a copy where every bridge raised is retried
on the next copy instead of being suppressed for the full TTL.
Match MeshCore's forwardMultipartDirect: a multipart packet is never
relayed as an ordinary directed packet. At an intermediate direct hop a
multipart ACK is re-emitted as a plain DIRECT ACK with the wrapper byte
stripped and this node removed from the path, deduped on the regenerated
ACK form and spaced by (remaining + 1) * 300ms. Flood routing, final
hops, and non-ACK embedded types are dropped rather than forwarded.
Coverage includes literal firmware-format wire vectors for one- and
two-byte hash widths and transport-direct input, end-to-end receive-to-
transmit relaying, and the drop cases.
MeshCore never relays control packets: the direct high-bit discovery
subset is explicitly released and any other control payload hits the
switch default that does not flood-route unknown types. Mark control
packets do-not-retransmit unconditionally instead of only when discovery
is enabled, so a repeater with discovery disabled no longer forwards
control/discovery traffic.
Extend the authenticated-ownership model to the PATH and RESPONSE
routing branches so a packet is consumed (do-not-retransmit) only when
a local identity MAC-verifies it. Prefix-only collisions and forged
traffic stay eligible for the forwarding engine.
- PathHelper marks do-not-retransmit and reports authenticated only
after a successful MAC decrypt with a valid path envelope; invalid or
truncated envelopes remain forwardable.
- PacketRouter aggregates authenticated results across the path helper
and companion bridges for PATH and RESPONSE, skipping the engine only
on authenticated ownership while preserving empty-path DIRECT release
hygiene.
Route login, text, and protocol-request packets through a single
_consume_via_local_candidates fan-out that offers each one to every local
candidate sharing the one-byte destination hash — the companion bridge and the
room-server/repeater identity — and consumes it only when one MAC-verifies it.
A prefix collision with a remote node (or a forged packet) is left for the
forwarding engine instead of being swallowed.
- gate processed_by_injection on the core HandlerResult.authenticated
- protocol-request now forwards on collision instead of always marking handled
- add real-crypto integration tests (text, room-server login, protocol-request)
and router fan-out tests covering companion + room-server collisions
On-air dest hashes are one byte, so a companion and a room-server (or
repeater) identity can share one. The login and text dispatch used
`if dest_hash in companion_bridges: ... elif login_helper/text_helper`,
so a companion silently shadowed a same-hash room server: its login
never reached the room-server handler and failed with Invalid HMAC
(e.g. Lake Stevens pubkey f55d.. / hash 0xf5 colliding with a companion).
Offer the packet to both handlers when both are registered at the hash;
decryption disambiguates — the key owner replies, the other fails HMAC
and no-ops. Non-colliding paths are unchanged. Applied to both the
LoginServerHandler and TextMessageHandler branches.
Adds regression tests covering collision (both invoked), companion-only
(server helper skipped), and server-without-companion for each path.
Merge the maintainer's fix-general-tidy branch (neighbor discovery,
keygen, API endpoints, web-asset rebuild, HTTP server config/control
commands, and an independent #286 room-server push/ACK/guest fix) into
the companion cleanup branch.
Both branches fixed#286 in parallel with byte-identical push-ACK CRC
logic. In the six overlapping files the maintainer's implementation is
kept (ACL replay-detection/session helpers, encoded path-len with legacy
fallback, expected_crc/ack_timeout_s injector API, dispatcher ACK
helpers); this branch's unique companion work is preserved on top
(sender_prefix persistence + migration, boot-state hardening /
CompanionStateLoadError, MessageQueue.max_size, older-core fallbacks).
Conflict resolution took the maintainer's side across the overlap, then
fixed two integration seams the merge introduced and updated this
branch's tests to the maintainer's API:
- room_server: timeout used undefined `hops`; aligned to `path_len`.
- packet_router: PATH helper was invoked twice (maintainer's
unconditional call plus this branch's conditional local-identity
call); dropped the now-redundant conditional block.
Pin openhop_core to @dev (was @feature/publish-workflow-message-handling)
so this can merge to the repeater's dev; core dev carries the required
sender_prefix and PathUtils.is_valid_path_len APIs.
Full suite green (1040 passed) against openhop_core dev and
refactor/companion-housekeeping; ruff clean.
Room server pushes waited on dispatcher.wait_for_ack, but nothing in the
repeater could ever resolve it, so every push timed out and re-pushed on
the backoff schedule (issue #286's duplicate floods — worst for virtual
companions on the same instance, whose ACKs never even cross the air):
- no core AckHandler is registered (all RX lands in the router fallback),
so received ACK CRCs were never fed to dispatcher ACK matching
- inject_packet waited on packet.get_crc(), a packet-hash CRC that no ACK
sender produces; the crypto CRC only the room server knows was ignored
- PATH returns (how a flood-received DM is ACKed) were decrypted by
PathHelper, but it read the encoded path_len wire byte as a raw count —
an empty 3-byte-hash path (0x80) parsed as a 128-byte truncated
payload — and the router's PATH branch never ran PathHelper for
room/repeater destinations when any companion bridge existed
Fixes:
- router ACK branch feeds discrete ACK CRCs (RF and locally injected) to
dispatcher._register_ack_received
- PATH packets addressed to a local server identity are processed by
PathHelper regardless of companion bridges; PathHelper decodes the
encoded path_len via PathUtils, keeps the encoded byte in out_path_len,
and registers the embedded ACK via ack_received_fn
- inject_packet accepts expected_ack_crc/ack_timeout; the room server
passes its crypto CRC and a hop-count-based timeout (the encoded byte
would have produced a ~4-minute direct-push wait)
Verified live on a real mesh: push to a same-instance virtual companion
resolves via the PATH-embedded ACK (encoded path_len 0x80) and push to a
firmware client resolves via the discrete RF ACK, no re-push loops.
Fixes#286Fixes#341
- Changed blank-password login behavior to create/manage guest sessions (when read-only is enabled), including replay protection and session timestamp tracking.
- Preserved and centralized `sync_since` handling during successful auth/session updates.
- Updated login handling to return explicit `(success, permissions)` and reset room-server sync guard state in SQLite on successful room-server login.
- Updated identity/banner output in mesh CLI from `pyMC_*` to `openHop_*` and adjusted default version fallback.
- Extended path handling to support encoded path-length semantics via `PathUtils`, with legacy fallback support for older/malformed packets.
- Added bundled ACK extraction from PATH payload extras and callback-based ACK propagation.
- Hardened room-server push logic: max-failure early skip, corrected expected ACK CRC calculation, path encoding compatibility, persisted sync/last-activity fields, and passed expected CRC + timeout into packet injection.
- Enhanced packet router ACK flow with dispatcher ACK registration helper, support for multipart ACK wrapper handling, configurable ACK wait timeout/CRC in injection, and ensured PATH helper processing always runs.
- Updated daemon wiring to pass ACK callback into `PathHelper`,
- make dispatcher dedupe configurable (prevent dbl deduping), and only register duplicate logging hook when dedupe is enabled.
- Expanded tests to cover guest blank-password replay tracking, updated room-server injector ACK expectations, and new duplicate-logging-hook gating behavior.
- Updated LogBuffer to support log entry IDs, enhanced log entry structure with additional metadata, and implemented subscriber management for real-time log streaming.
- Added OpenAPI specifications for new endpoints related to policy management, including retrieval, updating, validation, and group management for network policies.
- Implemented comprehensive tests for new policy endpoints, ensuring correct behavior for creating, updating, validating, and deleting policy groups and entries.
- Introduced policy evaluation tests to validate the functionality of the PolicyEngine, including various scenarios for action decisions based on defined rules.
- Enhanced packet routing tests to ensure proper handling of policy decisions in packet processing.
Push locally-injected TX packets to connected companion frame server
clients as PUSH_CODE_LOG_RX_DATA (0x88) with snr=0/rssi=0, so apps that
decrypt locally from raw RX (e.g. RemoteTerm) see companion-originated
channel traffic. The originating companion is excluded so a node never
hears its own transmission, matching physical firmware behavior.
inject_packet now takes an origin_hash (threaded per-companion via the
packet_injector partial); _on_raw_rx_for_companions gains exclude_hash
to skip that companion's frame server. OTA RX is unaffected.
- Updated byte representations in tests to use lowercase hex format for consistency.
- Reformatted code for better readability, including line breaks and indentation adjustments.
- Consolidated multiple lines into single lines where appropriate to enhance clarity.
- Ensured that all test cases maintain consistent formatting and style across the test suite.
- Updated the __call__ method to return a boolean indicating transmission success.
- Adjusted handling of transmission failures to increment dropped_count appropriately.
- Enhanced logging for transmission failures in PacketRouter to ensure visibility of issues.
- Added tests to verify correct behavior during transmission exceptions and failures.
Addresses PR 191 reviewer feedback:
1. Shutdown drain
stop() now waits up to 5 s for in-flight _route_packet tasks to finish,
then cancels any that remain. Previously only the queue-consumer loop was
cancelled; created tasks were abandoned with no guarantee they completed.
Mechanism: _route_tasks set tracks live tasks (added on create, discarded
in the done-callback). stop() takes a snapshot and calls asyncio.wait()
with timeout=5.0, then cancels the still-pending subset.
2. Drop counter
_cap_drop_count increments each time a packet is dropped at the cap.
The running total is included in every WARNING log line and also printed
at shutdown so operators can tell at a glance whether the safety valve is
actually firing in production.
3. Tests (tests/test_packet_router.py)
test_cap_drops_packets_when_full — cap=3, send 8 → 5 drops, 3 in-flight
test_cap_drop_count_increments — count increments by 1 per drop
test_cap_drop_count_zero_... — count stays 0 when cap never reached
test_stop_waits_for_in_flight_tasks — slow task (0.2 s) completes, not cancelled
test_stop_cancels_tasks_...timeout — hanging task cancelled after timeout
test_route_tasks_set_cleaned_up — set empty after all tasks finish
test_counter_matches_set_size — _in_flight == len(_route_tasks) at cap
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace the _route_tasks set in PacketRouter with a simple integer counter
(_in_flight / _max_in_flight=30) and add an early-drop guard in _process_queue.
Problems solved:
1. No cap on concurrent sleeping tasks: burst arrivals (multi-hop amplification,
collision retries) could stack unbounded _route_packet tasks, each holding a
packet closure and asyncio Task overhead, before the duty-cycle gate fired.
2. _route_tasks set held a strong reference to every Task object for the full
duration of its sleep — unnecessary in Python 3.12+ where the event loop
already holds tasks alive.
3. stop() iterated the full set to cancel tasks on shutdown — O(n) where n is
the in-flight count at shutdown time.
Fix: _in_flight counter increments before create_task and decrements in the
_on_route_done callback. The cap check (>= 30) in _process_queue is a last-resort
safety valve — LoRa airtime and the duty-cycle gate keep _in_flight in the
low single digits under normal load.
Also lower companion dedup prune threshold from 1000 to 200: the original 1000
allowed stale entries to accumulate for hundreds of PATH packets before the
O(n) dict comprehension sweep ran.
Trade-off documented: explicit task cancellation on shutdown is removed; tasks
are cancelled implicitly by event loop shutdown with identical outcome (no packet
transmits after the radio is closed regardless).
Docs: docs/pr_in_flight_cap.md — full problem analysis, alternative approaches
(semaphore, keep set + add cap), proof of counter sufficiency, rationale for
cap=30, and unit + field test plan.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Updated record_packet_only method to skip logging for TRACE packets, as TraceHelper manages trace paths.
- Enhanced documentation to clarify the handling of TRACE packets in the web UI.
- Added tests to ensure TRACE packets are not recorded, maintaining data integrity.
- Introduced `total_rx_airtime_ms` in `AirtimeManager` to track received packet airtime.
- Added `record_rx` method to log received airtime in `AirtimeManager`.
- Updated `RepeaterHandler` to count received packets and log RX airtime using the new method.
- Enhanced statistics reporting in `get_stats` to include total received airtime.
- Updated `ProtocolRequestHelper` to include total RX airtime in the RepeaterStats structure for better monitoring.
- Removed redundant original_path assignment in `RepeaterHandler` to streamline packet processing.
- Introduced `_is_direct_final_hop` helper method in `PacketRouter` to determine if a packet is the final destination for direct routes with an empty path.
- Updated comments in `PacketRouter` to clarify the handling of packets during routing, especially for direct forwarding scenarios.
- Adjusted logic to ensure packets are correctly processed or delivered based on their routing status, enhancing overall packet management.
- Introduced helper methods `_path_hash_display` and `_packet_record_src_dst` in `RepeaterHandler` to streamline path hash and source/destination hash extraction.
- Updated `record_packet` method to utilize a new `_build_packet_record` method for improved readability and maintainability.
- Enhanced `PacketRouter` comments for clarity on handling remote destinations and packet processing, ensuring better understanding of the routing logic.
- Introduced `record_packet_only` method in `RepeaterHandler` to log packets for UI/storage without forwarding or duplicate checks, specifically for injection-only types like ANON_REQ and ACK.
- Updated `PacketRouter` to call `_record_for_ui` method, ensuring that relevant metadata is recorded for packets processed by various handlers.
- Enhanced handling of packet metadata, including RSSI and SNR values, to improve the visibility of packet information in the web UI.
- Updated PacketRouter to deliver packets to all companion bridges when the destination is not recognized, ensuring better handling of ephemeral destinations.
- Refactored CompanionFrameServer to separate contact serialization and persistence logic, allowing for non-blocking database operations.
- Introduced a unique index for companion contacts in SQLite to support upsert functionality, enhancing data integrity and performance.
- Improved AdvertHelper to run database operations in a separate thread, preventing event loop blocking and maintaining responsiveness.
- Added logic to ensure that PATH and protocol-response packets are delivered to companions at most once per logical packet, preventing duplicate telemetry delivery.
- Introduced a deduplication key generation function and a mechanism to track delivery timestamps.
- Updated the enqueue method to utilize the new deduplication logic for companion bridges.
- Adjusted timeout for telemetry requests in CompanionAPIEndpoints to improve response handling.
- Improved local transmission handling by deferring local TX when duty cycle limits are exceeded, instead of dropping packets.
- Added LBT metadata extraction and logging for better monitoring of transmission attempts and delays.
- Refactored `schedule_retransmit` to support retrying local transmissions on failure, enhancing reliability.
- Introduced a lock in PacketRouter to serialize local TX operations, preventing race conditions during packet processing.
- Added a new method `deliver_control_data` in `RepeaterDaemon` to push CONTROL payloads to companion clients.
- Updated `PacketRouter` to invoke the new delivery method for control data packets.
- Introduced `push_control_data` in `CompanionFrameServer` to handle the sending of control data to clients.
- Enhanced `DiscoveryHelper` to support optional debug logging for control handling.
These changes improve the communication and control flow between the repeater and companion clients, facilitating better discovery response handling.
- Added raw RX and trace completion handlers to push data to connected companion clients.
- Enhanced PacketRouter to deliver ACK packets to all companion bridges.
- Introduced methods for retrieving companion statistics based on different types.
- Updated constants for handling new commands and responses in the companion protocol.
This update improves the interaction between the repeater and companion clients, enabling better data flow and monitoring capabilities.
- Add repeater/companion with frame server and constants
- Extend config, sqlite_handler, mesh_cli, packet_router for companion
- Update api_endpoints and auth_endpoints; adjust main entry
Co-authored-by: Cursor <cursoragent@cursor.com>
- Added security settings in config.yaml.example for managing authenticated clients.
- Introduced ACL class for handling access control and client authentication in acl.py.
- Created LoginHelper class for processing login requests and managing authentication in login.py.
- Added IdentityManager class for managing multiple identities in identity_manager.py.
- Updated main.py to initialize IdentityManager and LoginHelper, and register identities.
- Added packet_router to process ANON_REQ login packets through the LoginHelper.
agessaman