Merge the maintainer's fix-general-tidy branch (neighbor discovery,
keygen, API endpoints, web-asset rebuild, HTTP server config/control
commands, and an independent #286 room-server push/ACK/guest fix) into
the companion cleanup branch.
Both branches fixed#286 in parallel with byte-identical push-ACK CRC
logic. In the six overlapping files the maintainer's implementation is
kept (ACL replay-detection/session helpers, encoded path-len with legacy
fallback, expected_crc/ack_timeout_s injector API, dispatcher ACK
helpers); this branch's unique companion work is preserved on top
(sender_prefix persistence + migration, boot-state hardening /
CompanionStateLoadError, MessageQueue.max_size, older-core fallbacks).
Conflict resolution took the maintainer's side across the overlap, then
fixed two integration seams the merge introduced and updated this
branch's tests to the maintainer's API:
- room_server: timeout used undefined `hops`; aligned to `path_len`.
- packet_router: PATH helper was invoked twice (maintainer's
unconditional call plus this branch's conditional local-identity
call); dropped the now-redundant conditional block.
Pin openhop_core to @dev (was @feature/publish-workflow-message-handling)
so this can merge to the repeater's dev; core dev carries the required
sender_prefix and PathUtils.is_valid_path_len APIs.
Full suite green (1040 passed) against openhop_core dev and
refactor/companion-housekeeping; ruff clean.
Room server pushes waited on dispatcher.wait_for_ack, but nothing in the
repeater could ever resolve it, so every push timed out and re-pushed on
the backoff schedule (issue #286's duplicate floods — worst for virtual
companions on the same instance, whose ACKs never even cross the air):
- no core AckHandler is registered (all RX lands in the router fallback),
so received ACK CRCs were never fed to dispatcher ACK matching
- inject_packet waited on packet.get_crc(), a packet-hash CRC that no ACK
sender produces; the crypto CRC only the room server knows was ignored
- PATH returns (how a flood-received DM is ACKed) were decrypted by
PathHelper, but it read the encoded path_len wire byte as a raw count —
an empty 3-byte-hash path (0x80) parsed as a 128-byte truncated
payload — and the router's PATH branch never ran PathHelper for
room/repeater destinations when any companion bridge existed
Fixes:
- router ACK branch feeds discrete ACK CRCs (RF and locally injected) to
dispatcher._register_ack_received
- PATH packets addressed to a local server identity are processed by
PathHelper regardless of companion bridges; PathHelper decodes the
encoded path_len via PathUtils, keeps the encoded byte in out_path_len,
and registers the embedded ACK via ack_received_fn
- inject_packet accepts expected_ack_crc/ack_timeout; the room server
passes its crypto CRC and a hop-count-based timeout (the encoded byte
would have produced a ~4-minute direct-push wait)
Verified live on a real mesh: push to a same-instance virtual companion
resolves via the PATH-embedded ACK (encoded path_len 0x80) and push to a
firmware client resolves via the discrete RF ACK, no re-push loops.
Fixes#286Fixes#341
Installed openhop_core releases predating the sender_prefix change
(paired with fd43d86) reject the kwarg, so restoring queued messages
raised TypeError and aborted companion init. Detect support via the
QueuedMessage signature and drop persisted prefixes with a warning on
older cores instead of failing the boot.
A transient SQLite error during boot made companion_load_channels/
contacts/messages swallow the exception and return [], which is
indistinguishable from "no data". The Public-channel backfill then ran
over the empty store, so clients saw their channels wiped and later
saves could overwrite the persisted state.
- companion_load_{contacts,channels,messages} now return None on error
vs [] for genuinely empty, and log the companion hash
- add companion_count_{channels,messages} helpers
- extract shared _restore_companion_state used by both the boot and
hot-reload companion paths; each load is cross-checked against the
table's row count for the hash, retried once after a short delay,
and raises CompanionStateLoadError if it still cannot load, aborting
companion init (and the Public backfill) instead of starting empty
- log restored row counts per companion; log when the channel store
rejects a persisted channel (unchecked channels.set return)
- trim_companion_contacts_to_fit refuses to trim on a failed load
TXT_TYPE_SIGNED_PLAIN room posts carry a 4-byte author pubkey prefix
(QueuedMessage.sender_prefix, added in openhop_core). The SQLite
persistence path dropped it, so posts replayed from persistence showed
a zero-padded author prefix in the app frame while posts synced from
the live in-memory queue were correct.
- add sender_prefix column (hex text, default '') to companion_messages
with an ALTER TABLE migration for existing databases
- store/return the prefix in companion_push/pop/load_messages
- pass sender_prefix when rebuilding QueuedMessage from persistence in
the frame server and the startup preload paths
- add round-trip, default, migration, and rebuild tests
- Changed blank-password login behavior to create/manage guest sessions (when read-only is enabled), including replay protection and session timestamp tracking.
- Preserved and centralized `sync_since` handling during successful auth/session updates.
- Updated login handling to return explicit `(success, permissions)` and reset room-server sync guard state in SQLite on successful room-server login.
- Updated identity/banner output in mesh CLI from `pyMC_*` to `openHop_*` and adjusted default version fallback.
- Extended path handling to support encoded path-length semantics via `PathUtils`, with legacy fallback support for older/malformed packets.
- Added bundled ACK extraction from PATH payload extras and callback-based ACK propagation.
- Hardened room-server push logic: max-failure early skip, corrected expected ACK CRC calculation, path encoding compatibility, persisted sync/last-activity fields, and passed expected CRC + timeout into packet injection.
- Enhanced packet router ACK flow with dispatcher ACK registration helper, support for multipart ACK wrapper handling, configurable ACK wait timeout/CRC in injection, and ensured PATH helper processing always runs.
- Updated daemon wiring to pass ACK callback into `PathHelper`,
- make dispatcher dedupe configurable (prevent dbl deduping), and only register duplicate logging hook when dedupe is enabled.
- Expanded tests to cover guest blank-password replay tracking, updated room-server injector ACK expectations, and new duplicate-logging-hook gating behavior.
- Introduced `enforce_companion_contact_capacity` to manage contact limits during companion loading, with an option to trim non-favourite contacts when exceeding capacity.
- Updated `SQLiteHandler` to support message retention limits, allowing for automatic trimming of older messages based on `offline_queue_size`.
- Enhanced API endpoints to handle contact trimming on overflow, providing feedback on trimmed contacts during updates.
- Added utility functions for selecting and trimming contacts while preserving favourites.
- Improved logging for contact management actions and errors related to capacity.
Integrate latest dev while preserving per-companion bridge settings
and contact capacity validation. Resolve import conflicts in main.py
and api_endpoints.py.
Co-authored-by: Cursor <cursoragent@cursor.com>
- Introduced `CompanionContactCapacityError` to handle cases where persisted contacts exceed configured limits.
- Added utility functions for parsing companion bridge settings and validating contact capacity.
- Updated `RepeaterDaemon` to check contact capacity during companion loading and initialization.
- Enhanced API endpoints to validate companion settings and manage contact limits effectively.
- Implemented logging for bridge limits and errors related to contact capacity.
Push locally-injected TX packets to connected companion frame server
clients as PUSH_CODE_LOG_RX_DATA (0x88) with snr=0/rssi=0, so apps that
decrypt locally from raw RX (e.g. RemoteTerm) see companion-originated
channel traffic. The originating companion is excluded so a node never
hears its own transmission, matching physical firmware behavior.
inject_packet now takes an origin_hash (threaded per-companion via the
packet_injector partial); _on_raw_rx_for_companions gains exclude_hash
to skip that companion's frame server. OTA RX is unaffected.
- Updated the `LoginHelper` class to wrap the login handler in an `AnonRequestHandler`, allowing for proper handling of anonymous requests.
- Introduced methods for formatting region names based on flood policies and added support for retrieving transport keys from SQLite storage.
- Enhanced the constructor to accept additional parameters for SQLite handler and configuration, improving flexibility for owner-info and feature-flag replies.
- Added tests to validate the new functionality and ensure correct behavior of region name formatting and owner/features callbacks.
- Updated byte representations in tests to use lowercase hex format for consistency.
- Reformatted code for better readability, including line breaks and indentation adjustments.
- Consolidated multiple lines into single lines where appropriate to enhance clarity.
- Ensured that all test cases maintain consistent formatting and style across the test suite.
- Updated the __call__ method to return a boolean indicating transmission success.
- Adjusted handling of transmission failures to increment dropped_count appropriately.
- Enhanced logging for transmission failures in PacketRouter to ensure visibility of issues.
- Added tests to verify correct behavior during transmission exceptions and failures.
- Introduced options for using GPS coordinates for repeater location fields in config.
- Implemented precision control for GPS coordinates.
- Added a new API endpoint for a Server-Sent Events stream of GPS diagnostics.
- Updated GPSService to handle new configuration options and fallback logic.
- Enhanced unit tests for GPS location handling.
- Added record_duplicate method to RepeaterHandler to log known duplicate packets without forwarding.
- Enhanced RepeaterDaemon to subscribe to raw packets for deduplication logging, ensuring all path variants are visible in the UI.
- Updated recent_packets management to group duplicates under their original packets for better tracking.
- Added local_identity parameter to RepeaterDaemon for improved trace path matching.
- Refactored trace path handling in TraceHelper to support multi-byte hashes and structured hops.
- Updated methods to ensure compatibility with new trace data formats and improved logging.
- Enhanced tests to validate new trace processing logic and path handling.
- Modify TX modes: forward, monitor, and add no_tx, allowing for flexible packet handling.
- Updated configuration and API endpoints to support the new modes.
- Adjusted logic in RepeaterHandler to manage packet processing based on the selected mode.
- Enhanced CLI commands to reflect the new mode settings.
- Added tests for each TX mode to ensure correct behavior.
- Added `tcp_timeout` setting to `config.yaml.example` for companion configurations, allowing customization of TCP connection timeouts.
- Updated `RepeaterDaemon` to retrieve and apply the `tcp_timeout` setting, with a default value of 120 seconds.
- Modified `CompanionFrameServer` to accept `client_idle_timeout_sec` parameter, ensuring proper handling of idle timeouts.
- Enhanced API endpoints to include `tcp_timeout` in companion settings, improving configuration visibility and management.
- Introduced `normalize_companion_identity_key` function to standardize identity key formatting by stripping whitespace and removing the optional 0x prefix.
- Updated `RepeaterDaemon` to utilize the new normalization function when processing identity keys, improving consistency and error handling.
- Added a `stop` method in `CompanionFrameServer` to persist contacts and channels before stopping, ensuring data survives daemon restarts.
- Introduced `path_hash_mode` setting in `config.yaml.example` to specify the hash size for flood packets.
- Updated `ConfigManager` to re-apply the path hash mode when the mesh section changes, with validation for acceptable values (0, 1, 2).
- Enhanced `RepeaterDaemon` to set the default path hash mode during initialization, ensuring consistent handling of flood packets.
- Added backup functionality for config.yaml during save operations.
- Improved YAML saving to support Unicode characters and increased line width.
- Introduced a mechanism to sync companion node names to the configuration upon updates.
- Updated API endpoints to validate companion node names and persist changes to the configuration.
- Enhanced the RepeaterCompanionBridge to load and save preferences, ensuring consistent state management.
- Updated IdentityManager to include public key in identity details when retrieving identities.
- Introduced a new method in RepeaterDaemon for adding companions from configuration, supporting hot-reload functionality.
- Enhanced error handling for companion registration, ensuring proper validation of identity keys and settings.
- Updated API endpoints to include configured companions in the response, improving visibility of companion status and configuration.
agessaman