fix(repeater): retry failed stats poll up to a configurable limit

WHAT
Repeater statistics polls now retry within the same poll instead of
recording a failure after a single session. REPEATER_POLL_MAX_ATTEMPTS
(default 5) and REPEATER_POLL_RETRY_DELAY (default 5.0) control the
budget. The login/status/logout sequence moved into a private
_attempt_session(); _poll_one() drives the loop and writes exactly one
archive record carrying the new "attempts" field and the last error.
A missing password is not retried. Version bumped to 1.24.2; the stale
__version__ = "5.0" in __init__.py was aligned with config.VERSION.

WHY
Polls fail at irregular intervals with no established root cause. This
is an explicit workaround: retrying recovers readings now, while the
attempt count makes the underlying failure rate measurable so the real
cause can be identified from data instead of guessed at.

NOTES
No API or schema break — add_measurement() gained a keyword argument
with a default, and "attempts" is an added field that older records
simply lack. A poll blocks the worker loop for its full duration, so
worst-case command latency rises from roughly 60 seconds to roughly
three minutes for a repeater that never answers. Airtime rises
proportionally: up to five login/logout round trips per failing poll.
This commit is contained in:
pe1hvh committed 2026-09-02 08:05:13 +02:00
1 parent d5e9331d4b
commit fd3d0d2f35
5 files changed
+200 -26

No files matched your search

+55
View File
@@ -11,6 +11,61 @@ Format follows [Keep a Changelog](https://keepachangelog.com/) and [Semantic Ver
---
## [1.24.2] - 2026-09-02 — Repeater poll retries
### Added
- ✨ **`REPEATER_POLL_MAX_ATTEMPTS`** (`config.py`, default `5`): maximum
number of attempts per poll of a single repeater. One attempt is a
complete session — login, status request, logout. Set to `1` to
restore the previous single-shot behaviour.
- ✨ **`REPEATER_POLL_RETRY_DELAY`** (`config.py`, default `5.0`): seconds
between two attempts within the same poll.
- ✨ **`attempts` field in the repeater stats record**
(`services/repeater_stats_archive.py`): number of sessions used for
this result, so a repeater that only answers on the third try is
distinguishable from one that answers immediately. Zero when no
session was started, such as when no password is configured.
### Changed
- 🔁 **`RepeaterPoller._poll_one()` now retries a failed session**
(`services/repeater_poller.py`). The login/status/logout sequence moved
into a new private `_attempt_session()`; `_poll_one()` drives the retry
loop and writes exactly one archive record for the poll as a whole,
carrying the attempt count and the error of the last attempt. A missing
password is not retried — that is a configuration error, not a
transient one. The path reset after an unconfirmed login still runs per
attempt, so a second attempt goes out as a flood.
- 🔢 Version bumped to 1.24.2 in `config.py`. `__init__.py` carried a
stale `__version__ = "5.0"` that matched no release and is now aligned.
### Impact
- No API or schema breaks. `add_measurement()` gained a keyword argument
with a default, and `attempts` is an added field — records written
before 1.24.2 simply lack it and should be read as unknown, not as one.
- A poll now blocks the worker loop for longer. `_poll_repeaters()` is
awaited inline in the worker main loop, so with the defaults the worst
case rises from roughly 60 seconds to roughly 3 minutes for a repeater
that never answers. During that window queued commands are not
processed. Lower `REPEATER_POLL_MAX_ATTEMPTS` or
`REPEATER_LOGIN_TIMEOUT` if that latency matters more than the extra
chance of a reading.
- Airtime rises proportionally: up to five login/logout round trips per
failing poll instead of one.
- The manual poll from the REPEATERS panel goes through the same path and
inherits the retries. Its notification still clears after 3 seconds, so
it disappears before a retrying poll finishes.
### Rationale
Repeater statistics polls fail at irregular intervals and the cause is
not established — it could be a stale path, radio congestion, repeater
firmware or a timeout that is too tight. This release is an explicit
workaround, not a fix: retrying buys readings now, while the `attempts`
count makes the underlying failure rate measurable so the real cause can
be identified from data rather than guessed at.
---
## [1.24.1] - 2026-09-01 — Manual poll notification never disappeared
### Fixed
+1 -1
View File
@@ -5,4 +5,4 @@ A graphical user interface for MeshCore mesh network devices,
communicating via USB serial or Bluetooth Low Energy (BLE).
"""
__version__ = "5.0"
__version__ = "1.24.2"
+14 -1
View File
@@ -25,7 +25,7 @@ from typing import Any, Dict, List
# ==============================================================================
VERSION: str = "1.24.1"
VERSION: str = "1.24.2"
# ==============================================================================
@@ -470,6 +470,19 @@ REPEATER_LOGIN_TIMEOUT: float = 30.0
# Minimum seconds to wait for the STATUS_RESPONSE after a status request.
REPEATER_STATUS_TIMEOUT: float = 30.0
# Maximum number of attempts per poll of a single repeater.
# One attempt is a complete session: login, status request, logout. A
# repeater that fails to answer is retried up to this many times before
# the poll is recorded as failed. Set to 1 to disable retries entirely.
# Note that a poll blocks the worker loop for its whole duration, so the
# worst case is roughly MAX_ATTEMPTS x (LOGIN_TIMEOUT + RETRY_DELAY).
REPEATER_POLL_MAX_ATTEMPTS: int = 5
# Seconds to wait between two attempts within the same poll.
# Deliberately short: a longer pause does not make a stale path recover
# any faster, and the worker loop is blocked while the poll runs.
REPEATER_POLL_RETRY_DELAY: float = 5.0
# ==============================================================================
# CHANNEL LIST SORT
+113 -19
View File
@@ -29,6 +29,28 @@ relearns a route from the ACK. This mirrors what the library itself does
in ``send_msg_with_retry`` after repeated failures. ``reset_path`` is a
device-local command and costs no airtime.
Retries
~~~~~~~
A repeater that does not answer is retried within the same poll, up to
``REPEATER_POLL_MAX_ATTEMPTS`` times with ``REPEATER_POLL_RETRY_DELAY``
seconds in between. An attempt is the complete sequence above, logout
included: the login has to be redone because a session that never
confirmed cannot be assumed to exist. Combined with the path reset this
means the second attempt usually goes out as a flood, which is what
recovers a repeater whose stored route went stale.
Only one record is written per poll, carrying the attempt count and the
error of the last attempt. Recording every individual attempt would
skew the success ratio in the archive and grow the file several times
faster without adding information the count does not already give.
A missing password is not retried — that is a configuration error, not
a transient one.
This retry is a workaround, not a fix. It masks whatever makes the
repeater unreachable rather than addressing it; the attempt count in the
archive is there to make that underlying failure rate measurable.
Spreading
~~~~~~~~~
At most one repeater is polled per call, and each repeater gets its own
@@ -48,11 +70,14 @@ The poller never logs, returns or stores the password.
SPDX-License-Identifier: MIT
"""
import asyncio
import time
from typing import Any, Dict, Optional
from typing import Any, Dict, Optional, Tuple
from meshcore_gui.config import (
REPEATER_LOGIN_TIMEOUT,
REPEATER_POLL_MAX_ATTEMPTS,
REPEATER_POLL_RETRY_DELAY,
REPEATER_STATUS_TIMEOUT,
debug_print,
)
@@ -190,7 +215,11 @@ class RepeaterPoller:
# ------------------------------------------------------------------
async def _poll_one(self, mc, info: RepeaterInfo) -> None:
"""Run one full session against a single repeater.
"""Poll a single repeater, retrying a failed session.
Runs :meth:`_attempt_session` until it yields a status or the
configured attempt budget is exhausted, then writes exactly one
archive record for the poll as a whole.
Args:
mc: Connected ``MeshCore`` instance.
@@ -201,13 +230,83 @@ class RepeaterPoller:
if password is None:
self._archive.add_measurement(
info.pubkey, info.name, error="no_password_configured"
info.pubkey,
info.name,
error="no_password_configured",
attempts=0,
)
return
# A misconfigured zero or negative budget still buys one attempt;
# silently polling nothing at all would be harder to diagnose.
max_attempts = max(1, int(REPEATER_POLL_MAX_ATTEMPTS))
last_error = "no_attempt_made"
for attempt in range(1, max_attempts + 1):
status, last_error = await self._attempt_session(
mc, info, password, label, attempt, max_attempts
)
if status is not None:
if attempt > 1:
debug_print(
f"RepeaterPoller: {label} answered on attempt "
f"{attempt}/{max_attempts}"
)
self._archive.add_measurement(
info.pubkey,
info.name,
status=status,
attempts=attempt,
)
return
if attempt < max_attempts and REPEATER_POLL_RETRY_DELAY > 0:
await asyncio.sleep(REPEATER_POLL_RETRY_DELAY)
debug_print(
f"RepeaterPoller: {label} gave up after {max_attempts} "
f"attempt(s) — last error: {last_error}"
)
self._archive.add_measurement(
info.pubkey,
info.name,
error=last_error,
attempts=max_attempts,
)
async def _attempt_session(
self,
mc,
info: RepeaterInfo,
password: str,
label: str,
attempt: int,
max_attempts: int,
) -> Tuple[Optional[Dict[str, Any]], str]:
"""Run one complete session against a repeater.
Logs in, requests the status and always logs out again. Writes
nothing to the archive: the caller decides what a failed attempt
means once the whole poll is done.
Args:
mc: Connected ``MeshCore`` instance.
info: Repeater to poll.
password: Login password for this repeater.
label: Display label used in debug output.
attempt: Number of this attempt, starting at one.
max_attempts: Attempt budget, for debug output only.
Returns:
Tuple of the cleaned status and an empty string on success,
or ``None`` and a short failure reason on failure.
"""
suffix = f" (attempt {attempt}/{max_attempts})"
login_attempted = False
try:
debug_print(f"RepeaterPoller: login → {label}")
debug_print(f"RepeaterPoller: login → {label}{suffix}")
login_attempted = True
login_event = await mc.commands.send_login_sync(
info.pubkey,
@@ -216,34 +315,29 @@ class RepeaterPoller:
)
if login_event is None:
debug_print(f"RepeaterPoller: no login confirmation from {label}")
self._archive.add_measurement(
info.pubkey, info.name, error="login_failed_or_timeout"
debug_print(
f"RepeaterPoller: no login confirmation from {label}{suffix}"
)
await self._reset_path(mc, info.pubkey, label)
return
return None, "login_failed_or_timeout"
debug_print(f"RepeaterPoller: status request → {label}")
debug_print(f"RepeaterPoller: status request → {label}{suffix}")
status = await mc.commands.req_status_sync(
info.pubkey,
min_timeout=REPEATER_STATUS_TIMEOUT,
)
if not status:
self._archive.add_measurement(
info.pubkey, info.name, error="no_status_response"
debug_print(
f"RepeaterPoller: no status response from {label}{suffix}"
)
return
return None, "no_status_response"
self._archive.add_measurement(
info.pubkey, info.name, status=_clean_status(status)
)
return _clean_status(status), ""
except Exception as exc: # noqa: BLE001 — one repeater must not stop the loop
debug_print(f"RepeaterPoller: {label} failed: {exc}")
self._archive.add_measurement(
info.pubkey, info.name, error=f"exception: {type(exc).__name__}"
)
debug_print(f"RepeaterPoller: {label} failed{suffix}: {exc}")
return None, f"exception: {type(exc).__name__}"
finally:
if login_attempted:
@@ -19,6 +19,7 @@ Record schema
"name": "Repeater display name",
"ok": true,
"error": null,
"attempts": 1, # sessions used, since 1.24.2
"status": { ... raw fields as reported ... }
}
@@ -27,6 +28,13 @@ an empty ``status``. That is deliberate: a gap in the message archive
can then be matched against the poll moments to see whether it coincided
with a session.
``attempts`` counts how many complete sessions the poller needed before
it gave up or succeeded, so a repeater that only ever answers on the
third try is distinguishable from one that answers immediately. It is
zero when no session was started at all, such as when no password is
configured. Records written before 1.24.2 do not carry the field;
consumers should treat a missing value as unknown rather than as one.
Values in ``status`` are stored exactly as the repeater reports them —
no scaling, no rounding, no smoothing. Interpretation happens elsewhere.
@@ -103,14 +111,17 @@ class RepeaterStatsArchive:
name: str,
status: Optional[Dict[str, Any]] = None,
error: Optional[str] = None,
attempts: int = 1,
) -> Dict[str, Any]:
"""Append one poll result to the archive.
Args:
pubkey: Full public key of the polled repeater.
name: Display name of the repeater.
status: Raw status fields as reported, or ``None`` on failure.
error: Short failure reason, or ``None`` on success.
pubkey: Full public key of the polled repeater.
name: Display name of the repeater.
status: Raw status fields as reported, or ``None`` on failure.
error: Short failure reason, or ``None`` on success.
attempts: Number of complete sessions the poller used for this
result. Zero when no session was started at all.
Returns:
The record that was written.
@@ -121,6 +132,7 @@ class RepeaterStatsArchive:
"name": name,
"ok": error is None and status is not None,
"error": error,
"attempts": attempts,
"status": status or {},
}
@@ -144,7 +156,7 @@ class RepeaterStatsArchive:
debug_print(
f"RepeaterStatsArchive: recorded {name or pubkey[:16]} — "
f"ok={record['ok']}"
f"ok={record['ok']}, attempts={attempts}"
+ (f", error={error}" if error else "")
)
return record