Commit Graph

543 Commits

Author SHA1 Message Date
TJ Downes 7d1aa57321 fix(router): drain in-flight tasks on shutdown; add drop counter; add tests
Addresses PR 191 reviewer feedback:

1. Shutdown drain
   stop() now waits up to 5 s for in-flight _route_packet tasks to finish,
   then cancels any that remain.  Previously only the queue-consumer loop was
   cancelled; created tasks were abandoned with no guarantee they completed.

   Mechanism: _route_tasks set tracks live tasks (added on create, discarded
   in the done-callback).  stop() takes a snapshot and calls asyncio.wait()
   with timeout=5.0, then cancels the still-pending subset.

2. Drop counter
   _cap_drop_count increments each time a packet is dropped at the cap.
   The running total is included in every WARNING log line and also printed
   at shutdown so operators can tell at a glance whether the safety valve is
   actually firing in production.

3. Tests (tests/test_packet_router.py)
   test_cap_drops_packets_when_full     — cap=3, send 8 → 5 drops, 3 in-flight
   test_cap_drop_count_increments       — count increments by 1 per drop
   test_cap_drop_count_zero_...         — count stays 0 when cap never reached
   test_stop_waits_for_in_flight_tasks  — slow task (0.2 s) completes, not cancelled
   test_stop_cancels_tasks_...timeout   — hanging task cancelled after timeout
   test_route_tasks_set_cleaned_up      — set empty after all tasks finish
   test_counter_matches_set_size        — _in_flight == len(_route_tasks) at cap

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 05:46:49 -07:00
TJ Downes cadec00117 perf: replace _route_tasks set with bounded in-flight counter
Replace the _route_tasks set in PacketRouter with a simple integer counter
(_in_flight / _max_in_flight=30) and add an early-drop guard in _process_queue.

Problems solved:
1. No cap on concurrent sleeping tasks: burst arrivals (multi-hop amplification,
   collision retries) could stack unbounded _route_packet tasks, each holding a
   packet closure and asyncio Task overhead, before the duty-cycle gate fired.
2. _route_tasks set held a strong reference to every Task object for the full
   duration of its sleep — unnecessary in Python 3.12+ where the event loop
   already holds tasks alive.
3. stop() iterated the full set to cancel tasks on shutdown — O(n) where n is
   the in-flight count at shutdown time.

Fix: _in_flight counter increments before create_task and decrements in the
_on_route_done callback. The cap check (>= 30) in _process_queue is a last-resort
safety valve — LoRa airtime and the duty-cycle gate keep _in_flight in the
low single digits under normal load.

Also lower companion dedup prune threshold from 1000 to 200: the original 1000
allowed stale entries to accumulate for hundreds of PATH packets before the
O(n) dict comprehension sweep ran.

Trade-off documented: explicit task cancellation on shutdown is removed; tasks
are cancelled implicitly by event loop shutdown with identical outcome (no packet
transmits after the radio is closed regardless).

Docs: docs/pr_in_flight_cap.md — full problem analysis, alternative approaches
(semaphore, keep set + add cap), proof of counter sufficiency, rationale for
cap=30, and unit + field test plan.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-21 18:47:16 -07:00
Lloyd c82f0cfce6 feat:add ui websockets teardown. 2026-04-21 14:47:18 +01:00
Lloyd be56e919fd feat: add server-side airtime bucket aggregation for optimized chart rendering 2026-04-21 14:46:30 +01:00
Lloyd 81a3b70415 feat: implement graceful shutdown handling and version cache optimizations 2026-04-21 12:07:08 +01:00
Lloyd 9797e08421 feat: implement background scheduling for deferred network publishing tasks, tidy shutdown process 2026-04-21 10:07:15 +01:00
Lloyd 3df4b03fd9 feat: implement deferred network publishing for packets, adverts, and noise floor records 2026-04-21 09:49:12 +01:00
Lloyd c5fd41f28a feat: enhance task management in handlers with tracking and error logging 2026-04-21 09:38:03 +01:00
Lloyd 1883bc47be refactor: centralize database connection handling with WAL mode and busy timeout 2026-04-20 16:17:34 +01:00
Lloyd b26ebeb807 fix: optimize memory tracing by reducing overhead and filtering snapshots 2026-04-20 16:04:19 +01:00
Lloyd 68a461b965 feat: add memory debug to gui 2026-04-20 15:41:27 +01:00
Lloyd 5eb1fc47ca feat: add memory_debug endpoint for memory leak diagnostics and improve SSL context handling for GitHub requests 2026-04-20 14:51:48 +01:00
Rightup 799a85ddf9 fix: remove --no-index from R2 pip install so pyyaml resolves from PyPI 2026-04-19 19:34:26 +01:00
Rightup dfe9ba20f3 Fix R2 wheels installation path for improved dependency resolution 2026-04-18 23:15:34 +01:00
Rightup d336c72625 Enhance installation process with R2 wheels support for ARM devices 2026-04-18 23:15:13 +01:00
Lloyd 083ad2bc7a Merge pull request #184 from zindello/feat/luckfoxInstallSupport 2026-04-18 13:00:09 +01:00
Joshua Mesilane a9590fac01 Fix the headless install option 2026-04-18 21:09:01 +10:00
Lloyd 8f2888f2d5 Merge pull request #183 from zindello/feat/luckfoxInstallSupport
Fix for polkit version detection
2026-04-18 09:05:12 +01:00
Joshua Mesilane 7ba26b72cb Fix for polkit version detection 2026-04-18 17:39:35 +10:00
Lloyd 56e5a93699 Merge pull request #182 from zindello/feat/luckfoxInstallSupport 2026-04-18 08:32:30 +01:00
Joshua Mesilane 8ebcb09eff Headless install fix 2026-04-18 17:12:06 +10:00
Joshua Mesilane 62d6627fab Fix readme 2026-04-18 17:09:26 +10:00
Joshua Mesilane 4e3b2bbc9a Updates to support installs on the LuckFox platform 2026-04-18 16:50:44 +10:00
Rightup 9442c51225 feat: update logo in ui 2026-04-17 23:51:43 +01:00
Rightup ffaaa76ea0 feat: add glass to repeater. 2026-04-17 23:51:04 +01:00
Lloyd 4d9c560b11 Merge pull request #178 from rightup/feat-remove-sys-packages
feat: migrate to virtual environment and clean up system-level packages
2026-04-15 10:40:58 +01:00
Lloyd f2a72eb203 feat: migrate to virtual environment and clean up system-level packages 2026-04-15 09:37:26 +01:00
Lloyd 4d49eb701b feat: add owner_info field to repeater configuration and add getter for protocol request handling 2026-04-13 16:49:02 +01:00
Lloyd 9d971d03b7 UI fix for air time 2026-04-11 21:14:39 +01:00
Lloyd 110d7c2aec feat: add airtime data retrieval functionality with API endpoint 2026-04-11 20:42:04 +01:00
Lloyd 178eaf5b4b Merge pull request #175 from rightup/feat/fix-regions-unscoped-flood
fix: rename global flood to unscoped flood, fix region handling
2026-04-09 09:43:04 +01:00
Lloyd a53012ba35 test: enhance transport flood handling in tests with policy checks 2026-04-09 09:17:59 +01:00
Lloyd 19e0f5d3dd build: update bundled web UI assets
Rebuilt from pyMC-RepeaterUI dev branch — includes unscoped flood policy
UI changes (rename from global flood, updated API endpoint and labels).
2026-04-09 09:16:43 +01:00
Lloyd 2386da2857 Merge pull request #170 from zindello/feat/fixRegions
Feat/fix regions: rename global_flood_allow to unscoped_flood_allow,
fix region handling to correctly separate unscoped traffic from scoped
regions, maintain backward compat with existing config files.
2026-04-09 08:45:29 +01:00
Joshua Mesilane 3851055b65 Fix tests 2026-04-09 09:03:02 +10:00
Joshua Mesilane 2c95c0db0a Update example config files 2026-04-09 08:59:52 +10:00
Joshua Mesilane 7370cdc688 Update openapi and fix the test script 2026-04-07 16:28:49 +10:00
Joshua Mesilane 38e1fbe3f9 Changing from 'Global' Flood to 'Unscoped' flood as '*' doesn't actually mean wildcard, it means unscoped. Region keys should still only be forwaded if they're whitelisted. UI changes pending 2026-04-06 22:32:05 +10:00
Lloyd 3010703e1b Merge pull request #83 from pinztrek/iplog
Log system IP address (updated)
2026-03-30 08:53:54 +01:00
Lloyd dd8d3577cd Merge pull request #163 from rightup/feat/companion
Feat/companion
2026-03-27 22:38:10 +00:00
Lloyd d83d3e07aa fix: update pymc_core dependency to point to dev branch 2026-03-27 22:35:31 +00:00
Lloyd a92708f9d5 update UI to reflect keygen changes 2026-03-27 12:36:25 +00:00
Lloyd 59c6c772d4 feat: add JSON input handling for generate_vanity_key endpoint 2026-03-27 12:21:59 +00:00
Lloyd 50c19be498 Add gen tool for repeater prefix 2026-03-27 12:17:35 +00:00
Lloyd fdc621f64d feat: add sanitization for bytes values to hex for JSON serialization 2026-03-27 11:24:27 +00:00
Lloyd f5dbd83cda feat: add backup and restore and DB man 2026-03-27 11:15:53 +00:00
Lloyd 031f7b5e47 feat: add identity_key support in repeater configuration and update related logic in config handling 2026-03-27 10:16:10 +00:00
Lloyd 8babc375f2 Merge pull request #160 from agessaman/feat/companion-timeout-change
fix: update default client idle timeout to 8 hours in RepeaterDaemon …
2026-03-26 09:21:26 +00:00
agessaman 3ca9ac56d8 fix: update default client idle timeout to 8 hours in RepeaterDaemon and CompanionFrameServer for improved session management 2026-03-25 16:14:30 -07:00
Lloyd 07a47523ab feat: enhance event loop handling for thread-safe scheduling in MeshCLI 2026-03-24 14:18:12 +00:00