iarv/Remote-Terminal-for-MeshCore
1
0
Fork 1
mirror of https://github.com/jkingsman/Remote-Terminal-for-MeshCore.git synced 2026-05-02 03:23:00 +02:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #204 from jkingsman/extended-contact-fetch-timeout

Browse Source 
Work better with radios that are flakey around providing current contact load state (BLE?)
This commit is contained in:
Jack Kingsman
2026-04-19 14:12:35 -07:00
committed by GitHub
parent 6c1b8bd7e9 f7a311d74b
commit 476adf393f
5 changed files with 636 additions and 76 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
AGENTS.md
View File

@@ -503,6 +503,7 @@ mc.subscribe(EventType.ACK, handler)
| `MESHCORE_BASIC_AUTH_PASSWORD` | *(none)* | Optional app-wide HTTP Basic auth password; must be set together with `MESHCORE_BASIC_AUTH_USERNAME` |
| `MESHCORE_ENABLE_MESSAGE_POLL_FALLBACK` | `false` | Switch the always-on radio audit task from hourly checks to aggressive 10-second polling; the audit checks both missed message drift and channel-slot cache drift |
| `MESHCORE_FORCE_CHANNEL_SLOT_RECONFIGURE` | `false` | Disable channel-slot reuse and force `set_channel(...)` before every channel send, even on serial/BLE |
| `MESHCORE_LOAD_WITH_AUTOEVICT` | `false` | Enable autoevict contact loading: sets `AUTO_ADD_OVERWRITE_OLDEST` on the radio so adds never fail with TABLE_FULL, skips the removal phase during reconcile, and allows blind loading when `get_contacts` fails. Loaded contacts are not radio-favorited and may be evicted by new adverts when the table is full. |
**Note:** Runtime app settings are stored in the database (`app_settings` table), not environment variables. These include `max_radio_contacts`, `auto_decrypt_dm_on_advert`, `advert_interval`, `last_advert_time`, `last_message_times`, `flood_scope`, `blocked_keys`, `blocked_names`, `discovery_blocked_types`, `tracked_telemetry_repeaters`, `auto_resend_channel`, and `telemetry_interval_hours`. `max_radio_contacts` is the configured radio contact capacity baseline used by background maintenance: favorites reload first, non-favorite fill targets about 80% of that value, and full offload/reload triggers around 95% occupancy. They are configured via `GET/PATCH /api/settings`. MQTT, bot, webhook, Apprise, and SQS configs are stored in the `fanout_configs` table, managed via `/api/fanout`. If the radio's channel slots appear unstable or another client is mutating them underneath this app, operators can force the old always-reconfigure send path with `MESHCORE_FORCE_CHANNEL_SLOT_RECONFIGURE=true`.

24
README_ADVANCED.md
View File

@@ -8,6 +8,7 @@ These are intended for diagnosing or working around radios that behave oddly.
|----------|---------|-------------|
| `MESHCORE_ENABLE_MESSAGE_POLL_FALLBACK` | false | Run aggressive 10-second `get_msg()` fallback polling to check for messages |
| `MESHCORE_FORCE_CHANNEL_SLOT_RECONFIGURE` | false | Disable channel-slot reuse and force `set_channel(...)` before every channel send |
| `MESHCORE_LOAD_WITH_AUTOEVICT` | false | Enable autoevict mode for contact loading (see [Contact Loading Issues](#contact-loading-issues) below) |
| `__CLOWNTOWN_DO_CLOCK_WRAPAROUND` | false | Highly experimental: if the radio clock is ahead of system time, try forcing the clock to `0xFFFFFFFF`, wait for uint32 wraparound, and then retry normal time sync before falling back to reboot |
By default the app relies on radio events plus MeshCore auto-fetch for incoming messages, and also runs a low-frequency hourly audit poll. That audit checks both:
@@ -19,6 +20,29 @@ If the audit finds a mismatch, you'll see an error in the application UI and you
`__CLOWNTOWN_DO_CLOCK_WRAPAROUND=true` is a last-resort clock remediation for nodes whose RTC is stuck in the future and where rescue-mode time setting or GPS-based time is not available. It intentionally relies on the clock rolling past the 32-bit epoch boundary, which is board-specific behavior and may not be safe or effective on all MeshCore targets. Treat it as highly experimental.
## Contact Loading Issues
RemoteTerm loads favorite and recently active contacts onto the radio so that the radio can automatically acknowledge incoming DMs on your behalf. To do this, it first enumerates the radio's existing contact table, then reconciles it with the desired working set.
On BLE connections with many contacts (or radios with large contact tables from organic advertisements), the initial contact enumeration may time out. If this happens, the app will still attempt to load your favorites and recent contacts onto the radio on a best-effort basis, but without a full snapshot of what's already on the radio, some adds may be redundant or fail.
If the radio's contact table is already full (from contacts added by advertisements or another client), the app may not be able to load all desired contacts. In this case you'll see a warning that auto-DM acking may not work for all contacts. To resolve this:
- **Clear the radio's contact table** using another MeshCore client (e.g., the official companion app), then restart RemoteTerm
- **Lower the contact fill target** in Radio Settings to reduce how many contacts the app tries to load
- **Enable autoevict mode** (see below) to let the radio automatically make room
- If you don't need auto-DM acking, you can safely ignore these warnings — **sending and receiving messages is never affected**
### Autoevict Mode
Setting `MESHCORE_LOAD_WITH_AUTOEVICT=true` enables an alternative contact loading strategy that avoids TABLE_FULL errors entirely. On connect, the app enables the radio's `AUTO_ADD_OVERWRITE_OLDEST` preference, which makes the radio automatically evict the oldest non-favorite contact when the contact table is full. This means:
- Contact adds never fail — the radio always makes room by evicting stale contacts
- The app can load contacts even when it can't enumerate the radio's existing contact table (e.g., on slow BLE connections)
- No contact removal step is needed during reconciliation
**Trade-off:** Contacts loaded by the app are not marked as radio-side favorites, so they are eviction candidates if the radio receives a new advertisement while full. In practice, freshly-loaded contacts have a recent `lastmod` timestamp and will be among the last to be evicted. If you disconnect the radio from RemoteTerm and use it standalone, your contacts will not be protected from eviction by newer advertisements.
## Sub-Path Reverse Proxy
RemoteTerm works behind a reverse proxy that serves it under a sub-path (e.g. `/meshcore/` or Home Assistant ingress). All frontend asset and API paths are relative, so they resolve correctly under any prefix.

1
app/config.py
View File

@@ -26,6 +26,7 @@ class Settings(BaseSettings):
default=False,
validation_alias="__CLOWNTOWN_DO_CLOCK_WRAPAROUND",
)
load_with_autoevict: bool = False
skip_post_connect_sync: bool = False
basic_auth_username: str = ""
basic_auth_password: str = ""

369
app/radio_sync.py
View File

@@ -43,9 +43,41 @@ from app.websocket import broadcast_error, broadcast_event
logger = logging.getLogger(__name__)
DEFAULT_MAX_CHANNELS = 40
_GET_CONTACTS_TIMEOUT = 10
AdvertMode = Literal["flood", "zero_hop"]
_AUTO_ADD_OVERWRITE_OLDEST = 0x01
_RADIO_CONTACT_FAVORITE = 0x01
async def _enable_autoevict_on_radio(mc: MeshCore) -> bool:
"""Ensure the radio's AUTO_ADD_OVERWRITE_OLDEST preference bit is set."""
try:
current = await mc.commands.get_autoadd_config()
if current is None or current.type == EventType.ERROR:
logger.warning("Could not read autoadd config from radio: %s", current)
return False
current_flags = current.payload.get("config", 0)
if current_flags & _AUTO_ADD_OVERWRITE_OLDEST:
logger.debug("Radio autoevict already enabled (autoadd_config=0x%02x)", current_flags)
return True
new_flags = current_flags | _AUTO_ADD_OVERWRITE_OLDEST
result = await mc.commands.set_autoadd_config(new_flags)
if result is not None and result.type == EventType.OK:
logger.info(
"Enabled radio autoevict (autoadd_config 0x%02x -> 0x%02x)",
current_flags,
new_flags,
)
return True
else:
logger.warning("Failed to enable radio autoevict: %s", result)
return False
except Exception as exc:
logger.warning("Error enabling radio autoevict: %s", exc)
return False
def _contact_sync_debug_fields(contact: Contact) -> dict[str, object]:
"""Return key contact fields for sync failure diagnostics."""
@@ -239,7 +271,7 @@ async def should_run_full_periodic_sync(mc: MeshCore) -> bool:
capacity = _effective_radio_capacity(app_settings.max_radio_contacts)
refill_target, full_sync_trigger = _compute_radio_contact_limits(capacity)
result = await mc.commands.get_contacts()
result = await mc.commands.get_contacts(timeout=_GET_CONTACTS_TIMEOUT)
if result is None or result.type == EventType.ERROR:
logger.warning("Periodic sync occupancy check failed: %s", result)
return False
@@ -430,6 +462,16 @@ async def ensure_default_channels() -> None:
async def sync_and_offload_all(mc: MeshCore) -> dict:
"""Run fast startup sync, then background contact reconcile."""
autoevict_requested = settings.load_with_autoevict
autoevict = False
if autoevict_requested:
autoevict = await _enable_autoevict_on_radio(mc)
if not autoevict:
logger.warning(
"Autoevict requested but unavailable; falling back to snapshot-based "
"background contact reconcile"
)
# Contact on_radio is legacy/stale metadata. Clear it during the offload/reload
# cycle so old rows stop claiming radio residency we do not actively track.
@@ -441,9 +483,25 @@ async def sync_and_offload_all(mc: MeshCore) -> dict:
# Ensure default channels exist
await ensure_default_channels()
snapshot_failed = "error" in contacts_result
if snapshot_failed and not autoevict:
logger.warning(
"Radio contact snapshot failed — attempting best-effort contact "
"loading without a full picture of what's already on the radio"
)
broadcast_error(
"Could not enumerate radio contacts",
"Loading favorites and recent contacts on a best-effort basis — "
"some adds may be redundant or fail if the radio's contact table "
"is already full. Set MESHCORE_LOAD_WITH_AUTOEVICT=true for more "
"reliable loading without needing to read the radio first. "
"See 'Contact Loading Issues' in the Advanced Setup documentation.",
)
start_background_contact_reconciliation(
initial_radio_contacts=contacts_result.get("radio_contacts", {}),
expected_mc=mc,
autoevict=autoevict,
)
return {
@@ -1045,7 +1103,7 @@ async def sync_contacts_from_radio(mc: MeshCore) -> dict:
synced = 0
try:
result = await mc.commands.get_contacts()
result = await mc.commands.get_contacts(timeout=_GET_CONTACTS_TIMEOUT)
if result is None or result.type == EventType.ERROR:
logger.error(
@@ -1108,12 +1166,24 @@ async def _reconcile_radio_contacts_in_background(
*,
initial_radio_contacts: dict[str, dict],
expected_mc: MeshCore,
autoevict: bool = False,
) -> None:
"""Converge radio contacts toward the desired favorites+recents working set."""
"""Converge radio contacts toward the desired favorites+recents working set.
When *autoevict* is ``True`` the removal phase is skipped entirely and the
desired working set is blind-refreshed. Re-adding the full desired list
refreshes each contact's recency on supported firmware, so one successful
full pass converges the radio toward the desired working set without relying
on a stale contact snapshot.
"""
radio_contacts = dict(initial_radio_contacts)
removed = 0
loaded = 0
failed = 0
table_full = False
autoevict_next_index = 0
autoevict_full_pass_retries = 0
_MAX_AUTOEVICT_RETRIES = 3
try:
while True:
@@ -1121,18 +1191,32 @@ async def _reconcile_radio_contacts_in_background(
logger.info("Stopping background contact reconcile: radio transport changed")
break
# Pre-lock snapshot for quick-exit checks; authoritative list is
# re-fetched inside the radio lock below.
selected_contacts = await get_contacts_selected_for_radio_sync()
desired_fill_contacts = [
contact for contact in selected_contacts if len(contact.public_key) >= 64
]
if autoevict:
if not desired_fill_contacts:
logger.info(
"Background contact blind fill complete: no desired contacts selected"
)
break
if autoevict_next_index >= len(desired_fill_contacts):
autoevict_next_index = 0
desired_contacts = {
contact.public_key.lower(): contact
for contact in selected_contacts
if len(contact.public_key) >= 64
contact.public_key.lower(): contact for contact in desired_fill_contacts
}
removable_keys = [key for key in radio_contacts if key not in desired_contacts]
removable_keys = (
[] if autoevict else [key for key in radio_contacts if key not in desired_contacts]
)
missing_contacts = [
contact for key, contact in desired_contacts.items() if key not in radio_contacts
]
if not removable_keys and not missing_contacts:
if not autoevict and not removable_keys and not missing_contacts:
logger.info(
"Background contact reconcile complete: %d contacts on radio working set",
len(radio_contacts),
@@ -1140,6 +1224,8 @@ async def _reconcile_radio_contacts_in_background(
break
progressed = False
autoevict_pass_complete = False
autoevict_pass_failed = False
try:
async with radio_manager.radio_operation(
"background_contact_reconcile",
@@ -1153,100 +1239,232 @@ async def _reconcile_radio_contacts_in_background(
budget = CONTACT_RECONCILE_BATCH_SIZE
selected_contacts = await get_contacts_selected_for_radio_sync()
desired_fill_contacts = [
contact for contact in selected_contacts if len(contact.public_key) >= 64
]
if autoevict and autoevict_next_index >= len(desired_fill_contacts):
autoevict_next_index = 0
desired_contacts = {
contact.public_key.lower(): contact
for contact in selected_contacts
if len(contact.public_key) >= 64
contact.public_key.lower(): contact for contact in desired_fill_contacts
}
for public_key in list(radio_contacts):
if budget <= 0:
break
if public_key in desired_contacts:
continue
remove_payload = (
mc.get_contact_by_key_prefix(public_key[:12])
or radio_contacts.get(public_key)
or {"public_key": public_key}
)
try:
remove_result = await mc.commands.remove_contact(remove_payload)
except Exception as exc:
failed += 1
budget -= 1
logger.warning(
"Error removing contact %s during background reconcile: %s",
public_key[:12],
exc,
)
continue
budget -= 1
if remove_result.type == EventType.OK:
radio_contacts.pop(public_key, None)
_evict_removed_contact_from_library_cache(mc, public_key)
removed += 1
progressed = True
else:
failed += 1
logger.warning(
"Failed to remove contact %s during background reconcile: %s",
public_key[:12],
remove_result.payload,
)
if budget > 0:
for public_key, contact in desired_contacts.items():
if not autoevict:
for public_key in list(radio_contacts):
if budget <= 0:
break
if public_key in radio_contacts:
continue
if mc.get_contact_by_key_prefix(public_key[:12]):
radio_contacts[public_key] = {"public_key": public_key}
if public_key in desired_contacts:
continue
remove_payload = (
mc.get_contact_by_key_prefix(public_key[:12])
or radio_contacts.get(public_key)
or {"public_key": public_key}
)
try:
add_payload = contact.to_radio_dict()
add_result = await mc.commands.add_contact(add_payload)
remove_result = await mc.commands.remove_contact(remove_payload)
except Exception as exc:
failed += 1
budget -= 1
logger.warning(
"Error adding contact %s during background reconcile: %s",
"Error removing contact %s during background reconcile: %s",
public_key[:12],
exc,
exc_info=True,
)
continue
budget -= 1
if add_result.type == EventType.OK:
radio_contacts[public_key] = add_payload
loaded += 1
if remove_result.type == EventType.OK:
radio_contacts.pop(public_key, None)
_evict_removed_contact_from_library_cache(mc, public_key)
removed += 1
progressed = True
else:
failed += 1
reason = add_result.payload
hint = ""
if reason is None:
hint = (
" (no response from radio — if this repeats, check for "
"serial port contention from another process or try a "
"power cycle)"
)
logger.warning(
"Failed to add contact %s during background reconcile: %s%s",
"Failed to remove contact %s during background reconcile: %s",
public_key[:12],
reason,
hint,
remove_result.payload,
)
if budget > 0:
if autoevict:
# Budget is consumed by the slice bound rather than
# per-operation decrement — autoevict skips the
# removal phase so the full budget is always available.
batch_contacts = desired_fill_contacts[
autoevict_next_index : autoevict_next_index + budget
]
processed_contacts = 0
for contact in batch_contacts:
public_key = contact.public_key.lower()
try:
add_payload = contact.to_radio_dict()
# In autoevict mode, app-loaded contacts should
# remain evictable by the radio even if the
# stored contact record carries the favorite bit.
add_payload["flags"] = (
int(add_payload.get("flags", 0)) & ~_RADIO_CONTACT_FAVORITE
)
add_result = await mc.commands.add_contact(add_payload)
except Exception as exc:
failed += 1
logger.warning(
"Error blind-filling contact %s during background reconcile: %s",
public_key[:12],
exc,
exc_info=True,
)
autoevict_pass_failed = True
processed_contacts += 1
continue
if add_result.type == EventType.OK:
radio_contacts[public_key] = add_payload
loaded += 1
progressed = True
else:
failed += 1
autoevict_pass_failed = True
reason = add_result.payload
if isinstance(reason, dict) and reason.get("error_code") == 3:
logger.warning(
"Radio contact table full — stopping "
"contact reconcile (loaded %d this cycle)",
loaded,
)
table_full = True
break
hint = ""
if reason is None:
hint = (
" (no response from radio — if this repeats, check for "
"serial port contention from another process or try a "
"power cycle)"
)
logger.warning(
"Failed to blind-fill contact %s during background reconcile: %s%s",
public_key[:12],
reason,
hint,
)
processed_contacts += 1
autoevict_next_index += processed_contacts
autoevict_pass_complete = autoevict_next_index >= len(
desired_fill_contacts
)
else:
for public_key, contact in desired_contacts.items():
if budget <= 0:
break
if public_key in radio_contacts:
continue
if mc.get_contact_by_key_prefix(public_key[:12]):
radio_contacts[public_key] = {"public_key": public_key}
continue
try:
add_payload = contact.to_radio_dict()
add_result = await mc.commands.add_contact(add_payload)
except Exception as exc:
failed += 1
budget -= 1
logger.warning(
"Error adding contact %s during background reconcile: %s",
public_key[:12],
exc,
exc_info=True,
)
continue
budget -= 1
if add_result.type == EventType.OK:
radio_contacts[public_key] = add_payload
loaded += 1
progressed = True
else:
failed += 1
reason = add_result.payload
if isinstance(reason, dict) and reason.get("error_code") == 3:
logger.warning(
"Radio contact table full — stopping "
"contact reconcile (loaded %d this cycle)",
loaded,
)
table_full = True
break
hint = ""
if reason is None:
hint = (
" (no response from radio — if this repeats, check for "
"serial port contention from another process or try a "
"power cycle)"
)
logger.warning(
"Failed to add contact %s during background reconcile: %s%s",
public_key[:12],
reason,
hint,
)
except RadioOperationBusyError:
logger.debug("Background contact reconcile yielding: radio busy")
await asyncio.sleep(CONTACT_RECONCILE_BUSY_BACKOFF_SECONDS)
continue
if table_full:
if autoevict:
logger.error(
"We're expecting the radio to be in AUTO_ADD_OVERWRITE_OLDEST mode, "
"so a full-table error means we have no idea what is going on with "
"this radio; it is misbehaving. You should consider DM auto-acking "
"to be unreliable and/or not working for this radio. Sending and "
"receiving messages are not impacted by this error unless other "
"things are broken on your radio."
)
broadcast_error(
"Could not load all desired contacts onto the radio for auto-DM ack",
"Despite having auto-evict enabled, we got a contact-table-full error "
"from your radio. DM auto-ack is likely unavailable.",
)
else:
normal_table_full_message = (
"The radio's contact table is full. Clearing your radio contacts "
"using another client, lowering your contact fill target in "
"settings, or setting MESHCORE_LOAD_WITH_AUTOEVICT=true may "
"relieve this. See 'Contact Loading Issues' in the Advanced "
"README.md"
)
logger.error(
"Contact reconcile hit TABLE_FULL. %s",
normal_table_full_message,
)
broadcast_error(
"Could not load all desired contacts onto the radio for auto-DM ack",
normal_table_full_message,
)
break
if autoevict and autoevict_pass_complete:
if autoevict_pass_failed:
autoevict_full_pass_retries += 1
if autoevict_full_pass_retries >= _MAX_AUTOEVICT_RETRIES:
logger.warning(
"Background contact blind fill giving up after %d full passes "
"with persistent failures (loaded %d, failed %d)",
autoevict_full_pass_retries,
loaded,
failed,
)
break
autoevict_next_index = 0
else:
logger.info(
"Background contact blind fill complete: refreshed %d desired contacts",
len(desired_fill_contacts),
)
break
await asyncio.sleep(CONTACT_RECONCILE_YIELD_SECONDS)
if not progressed:
continue
@@ -1269,6 +1487,7 @@ def start_background_contact_reconciliation(
*,
initial_radio_contacts: dict[str, dict],
expected_mc: MeshCore,
autoevict: bool = False,
) -> None:
"""Start or replace the background contact reconcile task for the current radio."""
global _contact_reconcile_task
@@ -1280,11 +1499,13 @@ def start_background_contact_reconciliation(
_reconcile_radio_contacts_in_background(
initial_radio_contacts=initial_radio_contacts,
expected_mc=expected_mc,
autoevict=autoevict,
)
)
logger.info(
"Started background contact reconcile for %d radio contact(s)",
"Started background contact reconcile for %d radio contact(s)%s",
len(initial_radio_contacts),
" (autoevict mode)" if autoevict else "",
)

317
tests/test_radio_sync.py
View File

@@ -15,6 +15,7 @@ from meshcore.events import Event
import app.radio_sync as radio_sync
from app.radio import RadioManager, radio_manager
from app.radio_sync import (
_enable_autoevict_on_radio,
_message_poll_loop,
_periodic_advert_loop,
_periodic_sync_loop,
@@ -76,6 +77,7 @@ async def _insert_contact(
name="Alice",
on_radio=False,
contact_type=0,
flags=0,
last_contacted=None,
last_advert=None,
direct_path=None,
@@ -88,7 +90,7 @@ async def _insert_contact(
"public_key": public_key,
"name": name,
"type": contact_type,
"flags": 0,
"flags": flags,
"direct_path": direct_path,
"direct_path_len": direct_path_len,
"direct_path_hash_mode": direct_path_hash_mode,
@@ -516,10 +518,101 @@ class TestSyncAndOffloadAll:
result = await sync_and_offload_all(mock_mc)
mock_start.assert_called_once_with(
initial_radio_contacts=radio_contacts, expected_mc=mock_mc
initial_radio_contacts=radio_contacts, expected_mc=mock_mc, autoevict=False
)
assert result["contact_reconcile_started"] is True
@pytest.mark.asyncio
async def test_falls_back_to_snapshot_reconcile_when_autoevict_enable_fails(self, test_db):
mock_mc = MagicMock()
radio_contacts = {KEY_A: {"public_key": KEY_A}}
with (
patch.object(radio_sync.settings, "load_with_autoevict", True),
patch(
"app.radio_sync._enable_autoevict_on_radio",
new=AsyncMock(return_value=False),
),
patch(
"app.radio_sync.sync_contacts_from_radio",
new=AsyncMock(return_value={"synced": 1, "radio_contacts": radio_contacts}),
),
patch(
"app.radio_sync.sync_and_offload_channels",
new=AsyncMock(return_value={"synced": 0, "cleared": 0}),
),
patch("app.radio_sync.ensure_default_channels", new=AsyncMock()),
patch("app.radio_sync.start_background_contact_reconciliation") as mock_start,
):
result = await sync_and_offload_all(mock_mc)
mock_start.assert_called_once_with(
initial_radio_contacts=radio_contacts,
expected_mc=mock_mc,
autoevict=False,
)
assert result["contact_reconcile_started"] is True
@pytest.mark.asyncio
async def test_autoevict_success_passes_flag_to_reconcile(self, test_db):
mock_mc = MagicMock()
radio_contacts = {KEY_A: {"public_key": KEY_A}}
with (
patch.object(radio_sync.settings, "load_with_autoevict", True),
patch(
"app.radio_sync._enable_autoevict_on_radio",
new=AsyncMock(return_value=True),
),
patch(
"app.radio_sync.sync_contacts_from_radio",
new=AsyncMock(return_value={"synced": 1, "radio_contacts": radio_contacts}),
),
patch(
"app.radio_sync.sync_and_offload_channels",
new=AsyncMock(return_value={"synced": 0, "cleared": 0}),
),
patch("app.radio_sync.ensure_default_channels", new=AsyncMock()),
patch("app.radio_sync.start_background_contact_reconciliation") as mock_start,
):
result = await sync_and_offload_all(mock_mc)
mock_start.assert_called_once_with(
initial_radio_contacts=radio_contacts,
expected_mc=mock_mc,
autoevict=True,
)
assert result["contact_reconcile_started"] is True
@pytest.mark.asyncio
async def test_best_effort_reconcile_when_snapshot_fails(self, test_db):
"""When sync_contacts_from_radio errors, reconcile still starts with empty snapshot."""
mock_mc = MagicMock()
with (
patch(
"app.radio_sync.sync_contacts_from_radio",
new=AsyncMock(return_value={"synced": 0, "radio_contacts": {}, "error": "timeout"}),
),
patch(
"app.radio_sync.sync_and_offload_channels",
new=AsyncMock(return_value={"synced": 0, "cleared": 0}),
),
patch("app.radio_sync.ensure_default_channels", new=AsyncMock()),
patch("app.radio_sync.start_background_contact_reconciliation") as mock_start,
patch("app.radio_sync.broadcast_error") as mock_broadcast,
):
result = await sync_and_offload_all(mock_mc)
mock_start.assert_called_once_with(
initial_radio_contacts={},
expected_mc=mock_mc,
autoevict=False,
)
assert result["contact_reconcile_started"] is True
mock_broadcast.assert_called_once()
assert "best-effort" in mock_broadcast.call_args.args[1]
@pytest.mark.asyncio
async def test_advert_fill_skips_repeaters(self, test_db):
"""Recent advert fallback only considers non-repeaters."""
@@ -798,6 +891,81 @@ class TestSyncAndOffloadAll:
assert payload["public_key"] == KEY_A
class TestEnableAutoevictOnRadio:
"""Test _enable_autoevict_on_radio read-modify-write flow."""
@pytest.mark.asyncio
async def test_sets_flag_when_not_already_set(self):
mc = MagicMock()
mc.commands.get_autoadd_config = AsyncMock(
return_value=MagicMock(type=EventType.OK, payload={"config": 0x00})
)
mc.commands.set_autoadd_config = AsyncMock(return_value=MagicMock(type=EventType.OK))
result = await _enable_autoevict_on_radio(mc)
assert result is True
mc.commands.set_autoadd_config.assert_awaited_once_with(0x01)
@pytest.mark.asyncio
async def test_noop_when_already_enabled(self):
mc = MagicMock()
mc.commands.get_autoadd_config = AsyncMock(
return_value=MagicMock(type=EventType.OK, payload={"config": 0x01})
)
mc.commands.set_autoadd_config = AsyncMock()
result = await _enable_autoevict_on_radio(mc)
assert result is True
mc.commands.set_autoadd_config.assert_not_awaited()
@pytest.mark.asyncio
async def test_preserves_other_flags(self):
mc = MagicMock()
mc.commands.get_autoadd_config = AsyncMock(
return_value=MagicMock(type=EventType.OK, payload={"config": 0x04})
)
mc.commands.set_autoadd_config = AsyncMock(return_value=MagicMock(type=EventType.OK))
result = await _enable_autoevict_on_radio(mc)
assert result is True
mc.commands.set_autoadd_config.assert_awaited_once_with(0x05)
@pytest.mark.asyncio
async def test_returns_false_on_get_error(self):
mc = MagicMock()
mc.commands.get_autoadd_config = AsyncMock(
return_value=MagicMock(type=EventType.ERROR, payload=None)
)
result = await _enable_autoevict_on_radio(mc)
assert result is False
@pytest.mark.asyncio
async def test_returns_false_on_set_failure(self):
mc = MagicMock()
mc.commands.get_autoadd_config = AsyncMock(
return_value=MagicMock(type=EventType.OK, payload={"config": 0x00})
)
mc.commands.set_autoadd_config = AsyncMock(return_value=MagicMock(type=EventType.ERROR))
result = await _enable_autoevict_on_radio(mc)
assert result is False
@pytest.mark.asyncio
async def test_returns_false_on_exception(self):
mc = MagicMock()
mc.commands.get_autoadd_config = AsyncMock(side_effect=RuntimeError("timeout"))
result = await _enable_autoevict_on_radio(mc)
assert result is False
class TestBackgroundContactReconcile:
"""Test the yielding background contact reconcile loop."""
@@ -844,6 +1012,151 @@ class TestBackgroundContactReconcile:
payload = mock_mc.commands.add_contact.call_args.args[0]
assert payload["public_key"] == KEY_B
@pytest.mark.asyncio
async def test_autoevict_blind_fill_readds_full_desired_set(self, test_db):
await _insert_contact(KEY_A, "Alice", flags=0x01, last_contacted=2000)
await _insert_contact(KEY_B, "Bob", last_contacted=1000)
alice = await ContactRepository.get_by_key(KEY_A)
bob = await ContactRepository.get_by_key(KEY_B)
assert alice is not None
assert bob is not None
mock_mc = MagicMock()
mock_mc.is_connected = True
mock_mc.get_contact_by_key_prefix = MagicMock(return_value=None)
mock_mc.commands.remove_contact = AsyncMock(return_value=MagicMock(type=EventType.OK))
mock_mc.commands.add_contact = AsyncMock(return_value=MagicMock(type=EventType.OK))
radio_manager._meshcore = mock_mc
@asynccontextmanager
async def _radio_operation(*args, **kwargs):
del args, kwargs
yield mock_mc
with (
patch.object(
radio_sync.radio_manager,
"radio_operation",
side_effect=lambda *args, **kwargs: _radio_operation(*args, **kwargs),
),
patch("app.radio_sync.CONTACT_RECONCILE_BATCH_SIZE", 10),
patch(
"app.radio_sync.get_contacts_selected_for_radio_sync",
side_effect=[[alice, bob], [alice, bob]],
),
patch("app.radio_sync.asyncio.sleep", new=AsyncMock()),
):
await radio_sync._reconcile_radio_contacts_in_background(
initial_radio_contacts={KEY_A: {"public_key": KEY_A}},
expected_mc=mock_mc,
autoevict=True,
)
mock_mc.commands.remove_contact.assert_not_called()
assert mock_mc.commands.add_contact.await_count == 2
loaded_keys = [
call.args[0]["public_key"] for call in mock_mc.commands.add_contact.call_args_list
]
assert loaded_keys == [KEY_A, KEY_B]
loaded_flags = [
call.args[0]["flags"] for call in mock_mc.commands.add_contact.call_args_list
]
assert loaded_flags == [0, 0]
@pytest.mark.asyncio
async def test_autoevict_table_full_breaks_with_error(self, test_db):
"""TABLE_FULL during autoevict stops the loop and broadcasts an error."""
await _insert_contact(KEY_A, "Alice", last_contacted=2000)
alice = await ContactRepository.get_by_key(KEY_A)
assert alice is not None
mock_mc = MagicMock()
mock_mc.is_connected = True
mock_mc.get_contact_by_key_prefix = MagicMock(return_value=None)
table_full_result = MagicMock(type=EventType.ERROR, payload={"error_code": 3})
mock_mc.commands.add_contact = AsyncMock(return_value=table_full_result)
radio_manager._meshcore = mock_mc
@asynccontextmanager
async def _radio_operation(*args, **kwargs):
del args, kwargs
yield mock_mc
with (
patch.object(
radio_sync.radio_manager,
"radio_operation",
side_effect=lambda *args, **kwargs: _radio_operation(*args, **kwargs),
),
patch("app.radio_sync.CONTACT_RECONCILE_BATCH_SIZE", 10),
patch(
"app.radio_sync.get_contacts_selected_for_radio_sync",
side_effect=[[alice], [alice]],
),
patch("app.radio_sync.asyncio.sleep", new=AsyncMock()),
patch("app.radio_sync.broadcast_error") as mock_broadcast,
):
await radio_sync._reconcile_radio_contacts_in_background(
initial_radio_contacts={},
expected_mc=mock_mc,
autoevict=True,
)
mock_broadcast.assert_called_once()
assert "auto-evict" in mock_broadcast.call_args.args[1].lower()
@pytest.mark.asyncio
async def test_autoevict_retry_cap_stops_after_max_retries(self, test_db):
"""Autoevict gives up after _MAX_AUTOEVICT_RETRIES full passes with failures."""
await _insert_contact(KEY_A, "Alice", last_contacted=2000)
alice = await ContactRepository.get_by_key(KEY_A)
assert alice is not None
mock_mc = MagicMock()
mock_mc.is_connected = True
mock_mc.get_contact_by_key_prefix = MagicMock(return_value=None)
# Every add fails with a non-TABLE_FULL error
fail_result = MagicMock(type=EventType.ERROR, payload={"error_code": 99})
mock_mc.commands.add_contact = AsyncMock(return_value=fail_result)
radio_manager._meshcore = mock_mc
@asynccontextmanager
async def _radio_operation(*args, **kwargs):
del args, kwargs
yield mock_mc
call_count = 0
async def _get_selected():
nonlocal call_count
call_count += 1
return [alice]
with (
patch.object(
radio_sync.radio_manager,
"radio_operation",
side_effect=lambda *args, **kwargs: _radio_operation(*args, **kwargs),
),
patch("app.radio_sync.CONTACT_RECONCILE_BATCH_SIZE", 10),
patch(
"app.radio_sync.get_contacts_selected_for_radio_sync",
side_effect=_get_selected,
),
patch("app.radio_sync.asyncio.sleep", new=AsyncMock()),
):
await radio_sync._reconcile_radio_contacts_in_background(
initial_radio_contacts={},
expected_mc=mock_mc,
autoevict=True,
)
# 2 calls per iteration (pre-lock + in-lock), 3 retries = 6 calls,
# plus 1 pre-lock call on the initial iteration = at most 8.
# The key assertion: it terminates rather than looping forever.
assert mock_mc.commands.add_contact.await_count <= 4
assert call_count <= 8
@pytest.mark.asyncio
async def test_yields_radio_lock_every_two_contact_operations(self, test_db):
await _insert_contact(KEY_A, "Alice", last_contacted=3000)