Commit Graph

265 Commits

Author SHA1 Message Date
Lloyd e19ec79b49 Implement advertisement packet deduplication with configurable TTL and max hashes 2026-03-06 16:47:38 +00:00
agessaman 5b0359b74b Enhance configuration management and companion node handling
- Added backup functionality for config.yaml during save operations.
- Improved YAML saving to support Unicode characters and increased line width.
- Introduced a mechanism to sync companion node names to the configuration upon updates.
- Updated API endpoints to validate companion node names and persist changes to the configuration.
- Enhanced the RepeaterCompanionBridge to load and save preferences, ensuring consistent state management.
2026-03-05 20:04:17 -08:00
agessaman 0217a49ed2 Refactor RepeaterHandler path management and enhance packet validation
- Removed redundant call to mark_seen() for duplicate packets.
- Added validation to ensure hop count does not exceed the maximum limit before appending to the packet path.
- Updated logic to check for path size constraints when appending hash bytes, improving packet processing efficiency.
2026-03-05 16:52:43 -08:00
agessaman c150b9a9bf Merge upstream/feat/newRadios into dev-companion-v2-cleanup
- Keep our Vite-built assets and index.html script (index-DyUIpN7m.js)
- Remove upstream-only asset chunks and RoomServers-BxQ-0q-x.js
- README: keep two-backend intro, add upstream CAUTION/compatibility table
- manage.sh: keep dialog/gauge UX and .[hardware]; add CH341 udev, sudoers, libusb, polkit, silent upgrade
- sqlite_handler: add crc_errors table, index, and cleanup from upstream
- engine: add validate_packet and mark_seen in direct_forward; keep our path hash_size/hop_count logic
- advert: keep comment, use current_time = now
- api_endpoints: use restart_service() from service_utils
- config merge: strip user config comments before yq merge (upstream)

Made-with: Cursor
2026-03-05 16:43:14 -08:00
agessaman a6170f70ed Add public key to identity details and enhance companion configuration handling
- Updated IdentityManager to include public key in identity details when retrieving identities.
- Introduced a new method in RepeaterDaemon for adding companions from configuration, supporting hot-reload functionality.
- Enhanced error handling for companion registration, ensuring proper validation of identity keys and settings.
- Updated API endpoints to include configured companions in the response, improving visibility of companion status and configuration.
2026-03-05 16:26:50 -08:00
agessaman b6757a0ca0 Refactor path handling in RepeaterHandler to utilize hash-based representations
- Replaced list-based path storage with hash-based methods for original and forwarded paths, improving efficiency and consistency.
- Updated display logic to format path hashes correctly, ensuring compatibility with new hash size management.
- Adjusted local transmission handling to align with the new hash representation, enhancing clarity in packet processing.
2026-03-05 14:06:43 -08:00
agessaman 0271aa9455 Add support for multi-byte hashes via local_hash_bytes
- Updated the RepeaterHandler constructor to accept local_hash_bytes, improving path handling.
- Implemented checks to ensure packet paths do not exceed MAX_PATH_SIZE when appending hash bytes.
- Refactored direct_forward method to utilize local_hash_bytes for next hop validation and path manipulation.
- Adjusted path length encoding to accommodate changes in path management logic.
2026-03-05 09:39:11 -08:00
Lloyd 136af19178 add loop detection configuration and tests for flood routing 2026-03-05 14:06:28 +00:00
Lloyd c54efa3412 add side bar display of advert blocks etc 2026-03-05 12:56:01 +00:00
Lloyd 2b7d394938 add logging for tracking advert blocking 2026-03-05 11:26:40 +00:00
Lloyd 7eeba4d569 add debugs to advert settings 2026-03-05 11:13:44 +00:00
Lloyd 7dccb7457f Add advertisement configuration options for rate limiting and adaptive settings 2026-03-05 11:12:28 +00:00
Lloyd d5e1b0e714 update stats display 2026-03-05 10:53:34 +00:00
Lloyd 4285023670 Implement adaptive advert rate limiting and configuration updates 2026-03-05 10:35:15 +00:00
agessaman 1fbd99d52c Add JSON serialization support for companion preferences
- Introduced a new utility function, _to_json_safe, to ensure companion preferences are JSON-serializable, handling various data types including enums and dataclasses.
- Updated the RepeaterCompanionBridge to use the new serialization method when saving preferences to SQLite.
- Modified SQLiteHandler to ensure companion_hash is consistently converted to a string before database operations.
- Enhanced error handling for preference persistence in the RepeaterCompanionBridge.
2026-03-03 16:20:52 -08:00
agessaman 4f94b343cc Modify CompanionBridge integration to support persisting NodePrefs
- Added new methods to SQLiteHandler for loading and saving companion preferences as JSON, improving data persistence.
- Introduced a migration to create a companion_prefs table for storing preferences, ensuring compatibility with existing data.
- Refactored main.py to utilize RepeaterCompanionBridge instead of CompanionBridge, aligning with the new architecture.
2026-03-02 21:28:15 -08:00
agessaman e54d79d7c2 Refactor message handling in CompanionFrameServer to use a dedicated frame building method
- Removed redundant code for building message frames based on message type.
- Introduced a new method, _build_message_frame, to streamline the process of constructing message frames, improving code readability and maintainability.
2026-03-02 07:52:36 -08:00
Lloyd 1f14dcd9fc Ui Chnages for CRC display 2026-03-02 13:53:46 +00:00
Lloyd 4a05e20172 Add CRC error tracking and API endpoints for error count and history
- Create a new table for storing CRC errors in SQLite.
- Implement methods to store and retrieve CRC error counts and history.
- Update StorageCollector to record CRC errors and expose relevant methods.
- Enhance RepeaterHandler to track and record CRC error deltas from the radio hardware.
- Add API endpoints to fetch CRC error count and history.
2026-03-02 12:36:08 +00:00
Lloyd c2f57c3d0f add tests and more validation to packets, remove crc setting from config as hardcoded. 2026-03-02 11:35:50 +00:00
agessaman d82ebc59b0 Normalize companion_hash format in SQLite migrations to include 0x prefix
- Updated the SQLiteHandler to apply a migration that prefixes companion_hash values with '0x' for consistency with room_hash patterns.
- Adjusted the main.py file to reflect the new formatting for companion_hash when generating its string representation.
2026-02-28 09:06:24 -08:00
agessaman d5fe1e637c Restore deleted AdvertHelper. 2026-02-27 22:18:56 -08:00
agessaman 6fa85a832f Update packet type labels in rrdtool_handler and sqlite_handler for consistency
- Enhanced readability by adding descriptive suffixes to packet type labels in both rrdtool_handler.py and sqlite_handler.py.
- Aligned packet type definitions with the new naming conventions from pyMC_core, ensuring consistency across the codebase.
2026-02-27 21:18:29 -08:00
Adam Gessaman c78133f175 Refactor CompanionFrameServer for non-blocking message synchronization
- Changed the _get_companion_stats method to be asynchronous, improving responsiveness.
- Added a new async method _cmd_sync_next_message to handle message synchronization without blocking the event loop, enhancing performance during message processing.
- Introduced constants for response codes to streamline message handling in the CompanionFrameServer.
2026-02-25 15:39:18 -08:00
agessaman 789a2f27ea Enhance PacketRouter and CompanionFrameServer for improved packet delivery and contact persistence
- Updated PacketRouter to deliver packets to all companion bridges when the destination is not recognized, ensuring better handling of ephemeral destinations.
- Refactored CompanionFrameServer to separate contact serialization and persistence logic, allowing for non-blocking database operations.
- Introduced a unique index for companion contacts in SQLite to support upsert functionality, enhancing data integrity and performance.
- Improved AdvertHelper to run database operations in a separate thread, preventing event loop blocking and maintaining responsiveness.
2026-02-24 21:51:56 -08:00
Lloyd 0df94a0a91 updated convert_firmware_key.sh to support output format options and improve usage instructions; update _load_or_create_identity_key to prioritize system-wide identity key location. 2026-02-24 20:41:15 +00:00
Lloyd 74914541f2 Enhance installation and uninstallation scripts with user-friendly messages, add polkit and sudoers configuration for service management, and improve service restart handling in API endpoints. 2026-02-24 11:33:21 +00:00
Lloyd 4d2943087f Refactor user group management and add container detection warnings for udev rules 2026-02-24 09:15:21 +00:00
agessaman 27bbaf80ac Implement deduplication for companion delivery in PacketRouter
- Added logic to ensure that PATH and protocol-response packets are delivered to companions at most once per logical packet, preventing duplicate telemetry delivery.
- Introduced a deduplication key generation function and a mechanism to track delivery timestamps.
- Updated the enqueue method to utilize the new deduplication logic for companion bridges.
- Adjusted timeout for telemetry requests in CompanionAPIEndpoints to improve response handling.
2026-02-23 20:04:54 -08:00
Lloyd 8f90cf71b0 Reference for E22p branch 2026-02-23 21:58:15 +00:00
agessaman f1a944222f Update login, path and req response types.
- Simplified KISS modem setup instructions in README.md by removing unnecessary details.
- Refactored ConfigManager in config_manager.py to improve code clarity and efficiency, including changes to the save_to_file method and live_update_daemon method.
- Updated logging and error handling for better debugging and maintenance.
- Adjusted method signatures for consistency and clarity across the ConfigManager class.
- Modified device_version in frame_server.py to use FIRMWARE_VER_CODE from pyMC_core for better version management.
- Enhanced login.py and protocol_request.py with additional payload type handling and logging improvements.
- Cleaned up auth_endpoints.py for better readability and consistency in response formatting.
2026-02-21 21:19:22 -08:00
agessaman e9841adeff Merge upstream feat/newRadios (radios, MQTT, SPI)
- Take upstream radio-presets.json and radio-settings.json (Femtofox, AIO v2, CH341).
- config.py: upstream get_radio_for_board (_parse_int, sx1262_ch341, GPIO) + re-add KISS branch.
- config.yaml.example: upstream radio/ch341 + our identities.companions.
- manage.sh: upstream SPI warning prompt.
- main.py: single try/finally with companion shutdown and _shutdown().
- letsmesh_handler: upstream UTC fallback and MQTT v5 reason code handling.
- storage_collector, api_endpoints: upstream storage_dir_cfg; apply_setup_wizard supports KISS and sx1262_ch341/ch341.
- airtime.py: upstream bw_hz fix for symbol time.
- pyproject.toml: keep pymc_core dependency (no git pin).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-21 15:59:01 -08:00
agessaman e14fd3feea Refactor noise floor retrieval in RepeaterHandler to use asyncio executor
- Updated the noise floor retrieval method to run in an executor, preventing blocking of the event loop during the KISS modem's command execution.
- This change enhances responsiveness by allowing the process to handle other tasks while waiting for the noise floor measurement.
2026-02-21 15:38:04 -08:00
agessaman 65164fffb7 Improve retransmission logic and duty cycle handling in RepeaterHandler
- Improved local transmission handling by deferring local TX when duty cycle limits are exceeded, instead of dropping packets.
- Added LBT metadata extraction and logging for better monitoring of transmission attempts and delays.
- Refactored `schedule_retransmit` to support retrying local transmissions on failure, enhancing reliability.
- Introduced a lock in PacketRouter to serialize local TX operations, preventing race conditions during packet processing.
2026-02-21 15:38:01 -08:00
agessaman c2f8a2e3cd refactor: companion FrameServer and related (substantive only, no Black)
Reapply refactor from ce8381a (replace monolithic FrameServer with thin
pymc_core subclass, re-export constants, SQLite persistence hooks) while
preserving pre-refactor whitespace where patch applied cleanly. Remaining
files match refactor commit exactly. Diff vs ce8381a is whitespace-only.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-21 15:35:47 -08:00
agessaman 22e337a707 Add CompanionAPIEndpoints integration in APIEndpoints class
- Introduced CompanionAPIEndpoints to handle routes under /api/companion/*.
- Enhanced the APIEndpoints class to create a nested companion object for improved modularity and organization of companion-related API functionality.
2026-02-15 22:10:23 -08:00
agessaman a3f96962ff Refactor advert frame handling in CompanionFrameServer
- Improved the `_build_advert_push_frames` function to handle optional fields and ensure thread safety.
- Enhanced the `on_advert_received` method to robustly process incoming contact data, including better handling of public keys and optional fields for advert details.
- Added error handling to log exceptions during advert processing, improving reliability.
2026-02-15 19:33:04 -08:00
agessaman c07d24d387 Fix frame building functionality for advert push in CompanionFrameServer
- Introduced a new method `_build_advert_push_frames` to construct short and full advert frames from provided data.
- Refactored the handling of contact data to build and send advert frames asynchronously, improving performance and thread safety.
- Enhanced data structure for advert frame construction, including optional fields for full advert details.
2026-02-14 21:07:18 -08:00
agessaman 2360660a6b fix message persistence in CompanionFrameServer to use async I/O
- Changed `_persist_companion_message` to an asynchronous method to prevent blocking the event loop during SQLite operations.
- Updated calls to `_persist_companion_message` in message handling functions to use `await`.
- Enhanced error handling to include `BrokenPipeError` alongside `ConnectionResetError` for improved robustness.
2026-02-14 20:58:01 -08:00
agessaman d5cabe831c Fix message persistence and deduplication in CompanionFrameServer
- Updated `_persist_companion_message` to clarify deduplication of messages in SQLite.
- Modified `on_message_received` and `on_channel_message_received` to include `packet_hash` for message identification.
- Enhanced `SQLiteHandler` to support deduplication by `packet_hash` when pushing messages, preventing duplicates in the database.
- Added `packet_hash` column to the `companion_messages` table and created an index for efficient lookups.
2026-02-14 20:53:25 -08:00
agessaman 49f412acb0 Enhance control data delivery and discovery handling
- Added a new method `deliver_control_data` in `RepeaterDaemon` to push CONTROL payloads to companion clients.
- Updated `PacketRouter` to invoke the new delivery method for control data packets.
- Introduced `push_control_data` in `CompanionFrameServer` to handle the sending of control data to clients.
- Enhanced `DiscoveryHelper` to support optional debug logging for control handling.

These changes improve the communication and control flow between the repeater and companion clients, facilitating better discovery response handling.
2026-02-14 20:24:00 -08:00
agessaman c0dec9e80a Implement companion client communication and statistics retrieval
- Added raw RX and trace completion handlers to push data to connected companion clients.
- Enhanced PacketRouter to deliver ACK packets to all companion bridges.
- Introduced methods for retrieving companion statistics based on different types.
- Updated constants for handling new commands and responses in the companion protocol.

This update improves the interaction between the repeater and companion clients, enabling better data flow and monitoring capabilities.
2026-02-13 22:26:05 -08:00
agessaman 15299bf374 Add companion module and API integration
- Add repeater/companion with frame server and constants
- Extend config, sqlite_handler, mesh_cli, packet_router for companion
- Update api_endpoints and auth_endpoints; adjust main entry

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-13 16:07:43 -08:00
Lloyd f1a81e1732 Add optional GPIO parameters to get_radio_for_board function 2026-02-12 16:39:39 +00:00
Lloyd 0465450dbf Fix integer conversion by stripping trailing commas in get_radio_for_board function 2026-02-11 11:57:27 +00:00
Lloyd f50b571b86 Refactor SPI configuration parsing to ensure integer values are correctly processed 2026-02-11 11:54:01 +00:00
Lloyd bc23001cd6 Enhance MQTT error handling with extended v5 error codes and improved logging 2026-02-11 09:34:56 +00:00
agessaman 878ff8dc51 Enhance configuration and setup for KISS modem support
- Added support for Meshcore KISS modem firmware in configuration, allowing users to set `radio_type: kiss` and configure serial port and baud rate.
- Updated `config.yaml.example` to include KISS modem settings.
- Modified `manage.sh` to install with hardware extras for KISS support.
- Enhanced `setup-radio-config.sh` to prompt for radio type and KISS modem settings.
- Updated API endpoints to handle KISS modem configurations and hardware options.
- Improved error handling for missing configuration sections.

This update improves flexibility for users utilizing KISS modems alongside SX1262 hardware.
2026-02-03 16:20:51 -08:00
theshaun 57a25a2dc0 Add support for Femtofox
- Introduce GPIO chip selection and GPIO backend selection
- Add default Femtofox settings (for SX1262 as LR1121 support may come later)
- Add support for Python 3.10 UTC to reduce footprint
- Fix airtime calculation bandwidth variable naming
- Add service auto-enable when starting
- Add Australia: SA, WA, QLD radio preset
- Add Snyk security instructions

Co-authored-by: theshaun <theshaun@users.noreply.github.com>
Based on PR #69
2026-02-03 22:34:14 +00:00
Lloyd f7c4e2b4a8 Add support for new radio hardware configurations and CH341 USB adapter 2026-02-03 22:28:53 +00:00