Stored entries now record when they last logged in, so the web UI can
show whether a provisioned grant is still in use after a restart.
Firmware keeps this in RAM only; last_activity still starts at 0 on
load, so "not active since restart" keeps its meaning.
- acl_entries gains a nullable last_login column, added beside the
table's CREATE so a failed earlier migration cannot lock stored
admins out of a load that reads it.
- A login by a stored entry writes its time, best-effort, and not when
the stored second already matches it.
- A guest promoted by setperm keeps its login time; a row taken over
from another identity's label does not inherit that identity's.
- Key changes, renames and label adoption carry the time; a retried
load keeps the later of the live and stored times and writes a login
made while the store was unreadable.
/api/acl_clients adds client_name and client_type for each entry: the
name the node advertises, from an advert the repeater heard, or else a
companion's contact for it (a companion's own admin is often never heard
directly). Rebuilds the bundled UI (openHop_RepeaterUI feat/acl-management),
which shows the name in place of the one-byte address.
Codex confirmation round: the sweep of rows already under a name was
best-effort, so when it failed the rename or creation went ahead and the
leftovers became the room's grants. The sweep is now required: a rename
fails before the config is saved, and create_identity refuses before
adding the room.
Codex confirmation round: after a delete whose ACL cleanup failed, a room
created on that key and renamed to the deleted room's name took over its
leftover rows. Names are unique, so no live room holds the label a
rename takes or a new room gets; a rename (before taking the label) and
create_identity now drop every row already stored under it.
Codex confirmation round: rows a rekey's failed cleanup left at an old key
survived a later rename under the old label, which the rename did not
touch and the delete (by the new label) did not reach, so a room given
the old name and key could load them. A room owns only the rows under
its label at its current key; a rename, a rekey and a delete now drop
every other row under the labels involved. A failed sweep is retried by
the next one. Loading does not sweep: a room whose key was edited by hand
would otherwise lose rows that setting the key back recovers.
Codex confirmation round: deleting a room removed only its rows at its
current key. Rows a rekey's failed cleanup left at an old key kept its
label, so a room re-created with that name on that key loaded them. A
delete now drops every row under the room's label; names are unique, so
no other room holds it.
Codex round 5 (after the usage limit reset):
- A rekeyed or deleted room's old key kept answering logins, messages and
requests until a restart: its login, text and protocol handlers stayed
registered. The daemon now unregisters an identity from every packet
helper (_unregister_identity_everywhere), and update_identity and
delete_identity call it for the retired identity. Each helper removes
only its own handler, never another identity's on the same hash byte.
- Rows left at a key by a failed cleanup could grant their admins to a
later room on that key. A room loads only rows under its own label and
logs any others; moves copy, relabel and delete only the room's own
label's rows. A same-key rename relabels before the config is saved
and puts the label back if the save fails, so a room's own rows follow
its name. delete_identity reports an ACL cleanup that failed.
- Two overlapping hot reloads could start a superseded replacement
RoomServer whose sync loop then ran untracked. A swap starts its room
only if it is still the registered one.
A hot reload from PUT /api/update_identity re-runs TextHelper.register_identity,
which replaced self.room_servers[hash] and started a new sync loop without
stopping the old one, so the room pushed posts twice (the old loop with its
stale ACL). Take the RoomServer under the same hash, and any registered under
another hash by the same or previous name (a key change), out of room_servers,
and stop them before starting the replacement, on the running loop or via
run_coroutine_threadsafe from a request thread. update_identity passes the
original name through _register_identity_everywhere as previous_name.
Sixth review round (opencode: design still clean):
- A room whose passwords were cleared, then set again in a settings-only
update, was told no reload was needed, yet had no login handler or ACL
until a restart. Turning a password on counts as a reload.
- A restore refused because the new entry already holds the hash is logged.
- The no-login-helper move path says why it keeps its own store guard.
Fifth review round (opencode: "the ACL design is clean"; residual nits):
- The old identity entry is put back whenever the new registration did not
take its place, including when registration raises.
- A room whose update leaves it with no password is not hot-reloaded: it
is not registered for logins, so its ACL would stay under the old name.
The update waits for a restart and says so.
- delete_identity releases the room's name and hash through
unregister_identity, so the room can be re-created live rather than
conflicting with a room that no longer exists until a restart.
- LoginHelper.move_room_acl checks for a store before any path, the
superseded-ACL loop again skips the ACL being registered, and the
test daemon accepts the previous_name argument of the RoomServer
teardown fix on feat/quirky-lamarr-dfeae2, which should land with this
branch: the hot reload that now applies is what reaches that bug.
Fourth review round (opencode; codex hit its usage limit):
- update_identity's hot reload was always refused: the identity manager
rejects a name or hash already registered, which the running room's own
entry always is. The ACL name index then lagged the config, so a renamed
room was unmanageable by name, and a later rekey left a second live,
writable ACL. The old entry is now unregistered first and restored if
the new registration is refused (IdentityManager.unregister_identity),
and a move falls back to the store key when the name misses.
- A key change whose save and cleanup both failed left a copy at the new
key that blocked every retry. A copy under the room's own label is
replaced on retry; another identity's entries still refuse the move.
- A registration now detaches the ACL it replaces before loading, warns
when it displaces another identity sharing its hash byte, and the no-store
guard for a key change lives in LoginHelper.move_room_acl.
- /acl_stats is documented as it has always answered; store_error and
store_errors say they clear once a later read succeeds; the example
config says room servers keep their own security settings.
Third review round (codex, opencode):
- A failed save whose rollback also failed left the ACL under a key the
config did not name. A key change now copies the entries, saves the
config, then drops the old ones, all under the ACL's lock: any failure
leaves the key the saved config names holding its entries.
- Moves and deletes found the live ACL by scanning for its store key, and a
rekey left two ACL objects for one identity, so a second move or a delete
could act on the stale one. LoginHelper finds ACLs by name, and a new ACL
detaches and drops the one it replaces.
- A room deleted and re-added with the same key reused the detached ACL and
never stored again. A detached ACL is never reused.
- A failed read at startup was never retried; the next login or change
retries it, and acl_clients reports it as store_errors.
- A key change with no store anywhere was treated as moved. It is refused.
- The owner lookup fell back to the 1-byte hash for a room with no ACL,
attributing another identity's entries to it. acl_stats used the hash too.
- A refused update_identity left its merged settings in the live config;
every refusal now restores the entry. A key that yields no public key is
refused rather than skipping the shared-key check.
- Removing from every ACL stopped at the first failed write; it now reports
which failed and which succeeded. Room eviction carries on past one.
- Settings on a reused ACL are applied under its lock.
Second review round (codex, opencode):
- Moving a room's rows and pointing its live ACL at them happened in two
steps, so a grant could land under the old key in between.
ACL.move_store now does both under the ACL's lock.
- update_identity accepted a key another identity already uses, and the
move merged the two ACLs. It now refuses such a key before changing
anything, and the store refuses to move onto a key that has entries.
- update_identity saved the new key before moving the ACL, so a failed move
left a saved key with an empty ACL. The move now runs first, and a failed
save moves it back. It also runs when the daemon has no login helper.
- A failed read at startup looked like an empty ACL. The ACL records the
error and /api/acl_info reports it as store_error.
- A deleted room's handlers stay live until a restart, and a login there
wrote its entries back. Deleting now detaches the live ACL from storage.
- /api/acl_info looked ACLs up by the 1-byte hash, so identities sharing one
could show each other's counts. It uses the name lookup acl_clients does.
- The acl_clients filter is an object in the spec, as returned.
Review (codex, opencode) found:
- A room renamed and rekeyed in one update lost its ACL, and a room later
created under the old name could adopt it. Room servers are no longer
adopted by label; update_identity moves their rows to the new key and
name in one transaction and points the live ACL at them. Only the
repeater, which is unique, still adopts by label after a key change.
- A removal racing a grant could leave the grant stored but not listed.
Each ACL change and its write now happen under one lock.
- A failed write was reported as OK. The store now raises; setperm answers
"Err - failed to save" and leaves the table as it was, and the API
returns an error. A login still succeeds, as its session is valid.
- Re-registering an identity (a rename) built a new ACL, zeroing every
session's activity and replay watermark, so the room sync loop stopped
pushing to logged-in clients. The live ACL is now reused.
- The repeater's max_clients default rises to firmware's 32: admins are
never evicted and now persist, so 5 or 10 filled permanently. Stored
entries past max_clients all load, with a warning.
- atoi takes ASCII digits only; non-repeater identity types get their own
label; the sync-loop skip logs at debug; `get acl` has help.
API for the web UI:
- POST /api/acl_set_permissions sets an entry by identity name.
- /api/acl_clients lists entries by identity name rather than the 1-byte
hash two identities can share, with permissions_value, persisted and
identity_pubkey. /api/acl_info adds acl_entries and stored_entries, and
authenticated_clients counts entries that have been active.
- /api/acl_remove_client reads client_pubkey, as its schema always said
(the UI sent it, and the endpoint read only public_key), and takes
identity_name.
Firmware keeps its ACL in /s_contacts, so a key provisioned over serial with
`setperm <pubkey> 3` logs in with a blank password forever after. Ours was
RAM-only and setperm was a stub, so that workflow could not work.
- New acl_entries table, keyed by the local identity's full 32-byte public
key rather than its 1-byte hash, which collides between identities. An
identity_label ("repeater", "room_server:<name>") lets an identity adopt
its rows after a key change, as firmware's ACL survives a new private key.
Shared secrets are derived on load, never stored.
- The repeater stores every entry with permissions, a room server admins
only (firmware saveFilter). A write happens only when an entry's stored
permissions change, so a returning admin costs no I/O.
- setperm follows ClientACL::applyPermissions and firmware's replies: atoi
permissions stored as the whole byte, a guest role deletes by key prefix,
other roles need the full key. It refuses an empty prefix and a key that
is not a valid ed25519 point, both of which firmware accepts.
- `get acl` works from the web CLI (the serial stand-in, passed local=True)
and still errors remotely. It was unreachable: "get " matched it first.
- A full table evicts the least active non-admin, as firmware putClient,
instead of refusing; it still refuses when every entry is an admin.
- Room servers keep a stored admin through inactivity eviction and do not
push to entries that have not logged in since load, as firmware skips
clients with no activity.
- Deleting a room server drops its stored entries.
With the contact book full and overwrite-oldest on, a new advert evicts
the oldest contact in memory and _persist_contact upserts the newcomer
to SQLite, but the evicted row is never deleted. The stored count grows
by one per eviction until the next full save. If the daemon exits
without one (a crash, a kill, or a shutdown step that times out), the
companion then refuses to start because its stored contacts exceed
max_contacts.
Delete the evicted contact's row when contact_deleted fires, next to
the single-row upsert.
The event stream always served the first companion bridge. Every REST
endpoint that acts on one companion takes companion_name or
companion_hash, but events ignored its query string: a client asking for
another companion was served the first one's events, with nothing in
them to say whose they were, and a name that did not exist opened a
stream all the same.
Resolve the stream's companion the way the REST endpoints do, subscribe
that bridge, and route each event only to the clients of its companion.
Every event, the opening "connected" frame included, carries the
companion_hash it came from, so a client can confirm what it hears.
With no parameter the stream still follows the first companion. An
unknown companion now answers 404, and a repeater with no companion
loaded 503, as the REST endpoints do; the latter used to open a stream
that could stay silent.
_broadcast_sse drops a client whose queue is full, but the client's own
generator kept waiting on that queue and sending keepalives. The
connection looked healthy while no event could reach it again, and the
client had no reason to reconnect.
When the queue runs dry, end the stream if the client is no longer
registered. Everything already queued is still delivered first; the
client then sees the stream close and reconnects, which re-subscribes it.
CherryPy mounts StatsApp at / but sensor endpoints are on APIEndpoints.
Add @cherrypy.expose delegation methods so /sensors_types, /sensors_config,
/sensors_config_update, and /sensors_read resolve correctly instead of
falling through to the SPA catch-all.
The default method was raising 404 for all /api/* requests, blocking
sensor endpoints and other API routes from reaching the mounted
APIEndpoints object. CherryPy handles /api/* routing automatically.
- Add _settings_schema to all 9 sensor classes (bme280, ens210, ina219, shtc3,
lafvin_ups_3s, waveshare_ups_d, waveshare_ups_e, hardware_stats, openhop_modem)
- Implement GET /sensors/types, GET /sensors/config, POST /sensors/config,
POST /sensors/read endpoints in api_endpoints.py
- Update openapi.yaml with Sensors tag and endpoint schemas
- Add underscore-to-slash path mapping in contract checker
- Add sensors to contract allowlist
The three readers of the CRC scalar all answered for one receiver. On a bridge
that is half the node: a backhaul going deaf moved no number an operator or an
observer could see, while the sum of radios[].errors in the same status message
already said so.
All three now read RepeaterHandler.get_crc_error_count(), which sums the
radios -- stats.errors in the MQTT status message, crc_error_count in /stats,
and n_recv_errors in the MeshCore wire response, which previously read
self.radio directly and so described the default endpoint. stats.errors and the
per-radio entries beside it now reconcile.
This matches what node_stats() already does with the airtime counters that sit
next to it: a node-level counter describes the node, and which radio
contributed what stays available per radio, in radios[] and in the crc_errors
table.
The sum prefers each radio's live hardware counter and falls back to the last
count the sampler read from a radio the fabric cannot hand back. These are
lifetime counters, so dropping a radio would walk the node's total backwards
and hand an observer reading deltas a negative interval. A single-radio node
still reports its one unlabelled counter, read live, exactly as before.
The snapshot that keeps non-default radios reporting the band they are
actually on was seeded lazily, on the first status publish. That publish
waits for a broker to connect, so a node whose MQTT is briefly unreachable
could take a live radio save first and bake the pending settings in as
though they were what the hardware got -- reproducing the wrong-band
attribution the snapshot exists to prevent.
Capture it in MeshCoreToMqttPusher.__init__ instead, together with the
default radio id. That id was being recomputed from live config on every
publish, but fabric.default_radio is itself restart-required: a pending
change would refresh the radio that was not retuned and freeze the one that
was, reporting both wrongly where rebuilding everything got one right.
Seeding at construction also removes the unlocked check-then-set on
_applied_radio_map, which the status-publish timer thread shared with
whatever triggered the first publish.
The figures in stats are one set for the whole node, so on a two-radio bridge
they cannot say which side is carrying the traffic or which side has gone deaf.
Airtime is summed across channels and utilisation is the highest of them, which
is right for a node-level number and useless for telling the local radio from
the backhaul. Noise floor and errors are the default radio's outright, so the
link radio is simply unreported.
Every radio now carries its own tx_air_secs, rx_air_secs, current_airtime_ms,
utilization_percent, noise_floor and errors in its radios[] entry, under the
same field names stats uses so there is no second vocabulary to learn. The node
already computes all three -- AirtimeBudgets.per_radio_stats, the per-radio
noise floor cache, and the radios' CRC counters -- and /stats has reported the
first two as airtime_radios and noise_floor_radios since per-radio analytics
landed. This is the observer feed catching up.
Additive by construction: radios[] is published only where it already was, on a
node with more than one band, and a consumer that ignores it reads exactly what
it read before. Telemetry is merged into the map rather than gating it, and
each source contributes on its own, so a radio the fabric cannot hand back
keeps its air settings and still attributes its packets.
Two figures worth knowing before comparing them:
- stats.errors is the *default radio's* CRC count, matching /stats and the
MeshCore wire response, so on a bridge it does not equal the sum of
radios[].errors. Making the node scalar a sum would change three readers
that have always meant one radio, which is a decision of its own.
- tx/rx_air_secs truncate to whole seconds per radio and again for the node,
so the per-radio values can sum to one second less than the node's.
stats.errors was a literal 0 on every heartbeat, so an observer could not tell
a quiet node from a deaf one. It was hardcoded twice: once where the collector
builds the live stats, and again in publish_status, where the defaults were
spread *after* live_stats and so overrode anything the provider reported. The
second one would have pinned the field to 0 even once the first was fixed.
The node does count receive errors -- the radio's CRC failure counter, already
sampled on a timer, persisted as per-radio deltas and reported on the MeshCore
wire response as n_recv_errors. Report that, and let the defaults be defaults.
get_crc_error_count() on the engine is the single reader of the hardware
counter, replacing the inline read in get_stats so /stats and the status
message cannot drift apart. It reports the default radio's count rather than
the node's sum, matching the wire response, /stats, the noise floor beside it
in the same payload, and record_crc_errors' rule that only the default radio's
delta is published. A bridge's link radio errors are stored against their radio
id and queryable from the crc_errors table; they are not in this scalar.
queue_len stays 0. The Python repeater has no TX queue to measure, which is
what it has always reported -- the ordering fix just stops that being a
coincidence.
Drives AirtimeBudgets rather than stopping at the profile dict: a radios[]
entry stating only its frequency is charged at the modulation it inherits,
lands on the channel that bandwidth puts it on, and is no longer metered
5x faster than it transmits.
Per-radio sampling made the status message describe the wrong radio on a Fabric
node. _get_live_stats read the newest row in the noise_floor table with no
radio filter, which was right while only the default radio was ever stored.
_record_noise_floor_async now stores every radio's sample, default first, so
the newest row is the *other* radio's -- a bridge published its backhaul's
noise floor as the node's.
That is the opposite of the contract the sampler documents: _sampling_radios
puts the default first precisely because /stats, the MQTT status message and
companion stats all report that radio, and record_noise_floor honours it by
publishing only the default's sample. Reading the table directly went around
the rule.
Read the engine's cached reading instead, which is default-only by
construction, is what /stats reports, and agrees with the collector's own
cached value on the WebSocket path -- all three now say the same thing. It also
drops a SQLite query per heartbeat, which matters on an SD-card-bound Pi.
One behaviour change: the old query looked back 30 minutes, so a node restarted
within that window published a pre-restart reading as current. Now the key is
omitted until the first sample lands, 30 seconds in. The field was already
optional, and a reading from before the restart was not a current one.
Single-radio nodes are unaffected: their sole sample is the default's either
way.
Multi-radio entries currently replace radio, sx1262, and transport
configuration sections wholesale. Partial sx1262 entries therefore discard
board settings supplied by the top-level hardware preset, including TCXO
control, DIO2 RF switching, and radio enable pins.
Merge mapping sections key-by-key instead, treating the top-level sections as
defaults and radios[] values as per-radio overrides. Keep radio_type as a
scalar override so heterogeneous fabrics continue to work.
Overlay makes two spellings of one setting reachable together, where the
entry's could lose to an inherited default it never wrote: en_pins beats
en_pin in SX1262Radio, and address/serial_number beat device_address/serial
in get_radio_for_board. An entry naming either spelling now displaces the
other, so the failure mode is not a radio that silently never powers up.
This also corrects duty-cycle metering. It builds its air profiles through
_merge_radio_entry, so a radios[] entry that set only its frequency was
metered on library defaults -- 62.5 kHz / SF8 / CR8 -- rather than on the
node's own top-level radio block, and reported no tx_power at all.
_inherited_air_profile's docstring already described the overlay semantics
this commit implements.
Inheritance reaches the MQTT radio map the same way, and there it needed a
guard. The map is rebuilt from config on every status publish, so once a
partial entry inherits the top-level radio block, saving the default radio's
bandwidth moved the reported bandwidth of every radio that omits one --
while only the default radio is retuned, the rest keeping their band until a
restart. Observers join that map against rx_radio_id to decide which band
carried a packet, so it would have attributed those radios' packets to a band
they were never on. build_radio_status_entries now takes the map built when
the radios were last really configured and keeps every non-default entry from
it, which is the line AirtimeBudgets._effective_air already holds for
metering; MeshCoreToMqttPusher seeds it on its first build.
Log each resolved radio config at debug level, redacting modem_tcp.token via
redact_modem_tokens_in_place, since what an entry inherited is otherwise
invisible and a wrong answer surfaces only as a radio that will not transmit.
Add regression coverage for TCXO/RF-switch inheritance, explicit false
overrides, partial radio air settings, CH341 device selection, per-radio
radio_type overrides, alias displacement, metering inheritance, token
redaction, and a live default-radio save leaving other radios' published
air settings alone.
Fixes#485
An observer derives channel utilisation from the change in transmitted plus
received airtime between two status messages, so a node that publishes only
tx_air_secs cannot be metered: LetsMesh reports no utilisation for an openHop
bridge, and a consumer that defaults the missing key to zero sees TX-only
occupancy, which badly understates a repeater that hears far more than it
repeats.
utilization_percent, already published, is not that figure and cannot stand in
for it. It is this node's spend against its own duty-cycle budget over a
60-second window, so at the default 3600 ms/min it reads 100% at 6% of wall
clock -- a "how close am I to my TX cap" gauge, not channel occupancy.
AirtimeBudgets already tracks received airtime per channel and node_stats()
sums it, so this is a reporting gap rather than a missing measurement. Both
counters go out as whole seconds, matching the firmware's uint32 fields and the
two paths that already report them that way -- the MeshCore wire response in
handler_helpers/protocol_request.py and companion stats in main.py.
Worth knowing when reading the figure: RX airtime is estimated per packet from
the modulation that carried it, over packets that decoded. CRC failures,
preamble-only detections and traffic on other channels are not counted, so it
is a floor on receiver busy time rather than a measurement of it. Firmware's
counter has the same character, so the two remain comparable.
Built from openHop_RepeaterUI feat/per-radio-airtime at fea8f8b, so the shipped
bundle carries the fabric fan-out controls the backend validation in 8f16a2c
expects a caller to drive.
publish() skipped a connection whose disallowed_packet_types covered the packet
without recording anything, and the empty results list is how the same method
decides no broker was reachable. A node with healthy, connected brokers that all
filter a type therefore warned "No active broker connections" for every packet
of it -- loudest for exactly the common types someone would choose to filter.
Record the deliberate skip, so the warning fires only when nothing was
reachable. publish_mqtt already does this for its own intentional skips; this is
the same idiom one method up rather than a new mechanism.
The disabled-connection branch was never affected, which is why the warning
looked correct in a node with a broker switched off.
agessaman