The event stream always served the first companion bridge. Every REST
endpoint that acts on one companion takes companion_name or
companion_hash, but events ignored its query string: a client asking for
another companion was served the first one's events, with nothing in
them to say whose they were, and a name that did not exist opened a
stream all the same.
Resolve the stream's companion the way the REST endpoints do, subscribe
that bridge, and route each event only to the clients of its companion.
Every event, the opening "connected" frame included, carries the
companion_hash it came from, so a client can confirm what it hears.
With no parameter the stream still follows the first companion. An
unknown companion now answers 404, and a repeater with no companion
loaded 503, as the REST endpoints do; the latter used to open a stream
that could stay silent.
_broadcast_sse drops a client whose queue is full, but the client's own
generator kept waiting on that queue and sending keepalives. The
connection looked healthy while no event could reach it again, and the
client had no reason to reconnect.
When the queue runs dry, end the stream if the client is no longer
registered. Everything already queued is still delivered first; the
client then sees the stream close and reconnects, which re-subscribes it.
The three readers of the CRC scalar all answered for one receiver. On a bridge
that is half the node: a backhaul going deaf moved no number an operator or an
observer could see, while the sum of radios[].errors in the same status message
already said so.
All three now read RepeaterHandler.get_crc_error_count(), which sums the
radios -- stats.errors in the MQTT status message, crc_error_count in /stats,
and n_recv_errors in the MeshCore wire response, which previously read
self.radio directly and so described the default endpoint. stats.errors and the
per-radio entries beside it now reconcile.
This matches what node_stats() already does with the airtime counters that sit
next to it: a node-level counter describes the node, and which radio
contributed what stays available per radio, in radios[] and in the crc_errors
table.
The sum prefers each radio's live hardware counter and falls back to the last
count the sampler read from a radio the fabric cannot hand back. These are
lifetime counters, so dropping a radio would walk the node's total backwards
and hand an observer reading deltas a negative interval. A single-radio node
still reports its one unlabelled counter, read live, exactly as before.
The snapshot that keeps non-default radios reporting the band they are
actually on was seeded lazily, on the first status publish. That publish
waits for a broker to connect, so a node whose MQTT is briefly unreachable
could take a live radio save first and bake the pending settings in as
though they were what the hardware got -- reproducing the wrong-band
attribution the snapshot exists to prevent.
Capture it in MeshCoreToMqttPusher.__init__ instead, together with the
default radio id. That id was being recomputed from live config on every
publish, but fabric.default_radio is itself restart-required: a pending
change would refresh the radio that was not retuned and freeze the one that
was, reporting both wrongly where rebuilding everything got one right.
Seeding at construction also removes the unlocked check-then-set on
_applied_radio_map, which the status-publish timer thread shared with
whatever triggered the first publish.
The figures in stats are one set for the whole node, so on a two-radio bridge
they cannot say which side is carrying the traffic or which side has gone deaf.
Airtime is summed across channels and utilisation is the highest of them, which
is right for a node-level number and useless for telling the local radio from
the backhaul. Noise floor and errors are the default radio's outright, so the
link radio is simply unreported.
Every radio now carries its own tx_air_secs, rx_air_secs, current_airtime_ms,
utilization_percent, noise_floor and errors in its radios[] entry, under the
same field names stats uses so there is no second vocabulary to learn. The node
already computes all three -- AirtimeBudgets.per_radio_stats, the per-radio
noise floor cache, and the radios' CRC counters -- and /stats has reported the
first two as airtime_radios and noise_floor_radios since per-radio analytics
landed. This is the observer feed catching up.
Additive by construction: radios[] is published only where it already was, on a
node with more than one band, and a consumer that ignores it reads exactly what
it read before. Telemetry is merged into the map rather than gating it, and
each source contributes on its own, so a radio the fabric cannot hand back
keeps its air settings and still attributes its packets.
Two figures worth knowing before comparing them:
- stats.errors is the *default radio's* CRC count, matching /stats and the
MeshCore wire response, so on a bridge it does not equal the sum of
radios[].errors. Making the node scalar a sum would change three readers
that have always meant one radio, which is a decision of its own.
- tx/rx_air_secs truncate to whole seconds per radio and again for the node,
so the per-radio values can sum to one second less than the node's.
stats.errors was a literal 0 on every heartbeat, so an observer could not tell
a quiet node from a deaf one. It was hardcoded twice: once where the collector
builds the live stats, and again in publish_status, where the defaults were
spread *after* live_stats and so overrode anything the provider reported. The
second one would have pinned the field to 0 even once the first was fixed.
The node does count receive errors -- the radio's CRC failure counter, already
sampled on a timer, persisted as per-radio deltas and reported on the MeshCore
wire response as n_recv_errors. Report that, and let the defaults be defaults.
get_crc_error_count() on the engine is the single reader of the hardware
counter, replacing the inline read in get_stats so /stats and the status
message cannot drift apart. It reports the default radio's count rather than
the node's sum, matching the wire response, /stats, the noise floor beside it
in the same payload, and record_crc_errors' rule that only the default radio's
delta is published. A bridge's link radio errors are stored against their radio
id and queryable from the crc_errors table; they are not in this scalar.
queue_len stays 0. The Python repeater has no TX queue to measure, which is
what it has always reported -- the ordering fix just stops that being a
coincidence.
Drives AirtimeBudgets rather than stopping at the profile dict: a radios[]
entry stating only its frequency is charged at the modulation it inherits,
lands on the channel that bandwidth puts it on, and is no longer metered
5x faster than it transmits.
Per-radio sampling made the status message describe the wrong radio on a Fabric
node. _get_live_stats read the newest row in the noise_floor table with no
radio filter, which was right while only the default radio was ever stored.
_record_noise_floor_async now stores every radio's sample, default first, so
the newest row is the *other* radio's -- a bridge published its backhaul's
noise floor as the node's.
That is the opposite of the contract the sampler documents: _sampling_radios
puts the default first precisely because /stats, the MQTT status message and
companion stats all report that radio, and record_noise_floor honours it by
publishing only the default's sample. Reading the table directly went around
the rule.
Read the engine's cached reading instead, which is default-only by
construction, is what /stats reports, and agrees with the collector's own
cached value on the WebSocket path -- all three now say the same thing. It also
drops a SQLite query per heartbeat, which matters on an SD-card-bound Pi.
One behaviour change: the old query looked back 30 minutes, so a node restarted
within that window published a pre-restart reading as current. Now the key is
omitted until the first sample lands, 30 seconds in. The field was already
optional, and a reading from before the restart was not a current one.
Single-radio nodes are unaffected: their sole sample is the default's either
way.
Multi-radio entries currently replace radio, sx1262, and transport
configuration sections wholesale. Partial sx1262 entries therefore discard
board settings supplied by the top-level hardware preset, including TCXO
control, DIO2 RF switching, and radio enable pins.
Merge mapping sections key-by-key instead, treating the top-level sections as
defaults and radios[] values as per-radio overrides. Keep radio_type as a
scalar override so heterogeneous fabrics continue to work.
Overlay makes two spellings of one setting reachable together, where the
entry's could lose to an inherited default it never wrote: en_pins beats
en_pin in SX1262Radio, and address/serial_number beat device_address/serial
in get_radio_for_board. An entry naming either spelling now displaces the
other, so the failure mode is not a radio that silently never powers up.
This also corrects duty-cycle metering. It builds its air profiles through
_merge_radio_entry, so a radios[] entry that set only its frequency was
metered on library defaults -- 62.5 kHz / SF8 / CR8 -- rather than on the
node's own top-level radio block, and reported no tx_power at all.
_inherited_air_profile's docstring already described the overlay semantics
this commit implements.
Inheritance reaches the MQTT radio map the same way, and there it needed a
guard. The map is rebuilt from config on every status publish, so once a
partial entry inherits the top-level radio block, saving the default radio's
bandwidth moved the reported bandwidth of every radio that omits one --
while only the default radio is retuned, the rest keeping their band until a
restart. Observers join that map against rx_radio_id to decide which band
carried a packet, so it would have attributed those radios' packets to a band
they were never on. build_radio_status_entries now takes the map built when
the radios were last really configured and keeps every non-default entry from
it, which is the line AirtimeBudgets._effective_air already holds for
metering; MeshCoreToMqttPusher seeds it on its first build.
Log each resolved radio config at debug level, redacting modem_tcp.token via
redact_modem_tokens_in_place, since what an entry inherited is otherwise
invisible and a wrong answer surfaces only as a radio that will not transmit.
Add regression coverage for TCXO/RF-switch inheritance, explicit false
overrides, partial radio air settings, CH341 device selection, per-radio
radio_type overrides, alias displacement, metering inheritance, token
redaction, and a live default-radio save leaving other radios' published
air settings alone.
Fixes#485
An observer derives channel utilisation from the change in transmitted plus
received airtime between two status messages, so a node that publishes only
tx_air_secs cannot be metered: LetsMesh reports no utilisation for an openHop
bridge, and a consumer that defaults the missing key to zero sees TX-only
occupancy, which badly understates a repeater that hears far more than it
repeats.
utilization_percent, already published, is not that figure and cannot stand in
for it. It is this node's spend against its own duty-cycle budget over a
60-second window, so at the default 3600 ms/min it reads 100% at 6% of wall
clock -- a "how close am I to my TX cap" gauge, not channel occupancy.
AirtimeBudgets already tracks received airtime per channel and node_stats()
sums it, so this is a reporting gap rather than a missing measurement. Both
counters go out as whole seconds, matching the firmware's uint32 fields and the
two paths that already report them that way -- the MeshCore wire response in
handler_helpers/protocol_request.py and companion stats in main.py.
Worth knowing when reading the figure: RX airtime is estimated per packet from
the modulation that carried it, over packets that decoded. CRC failures,
preamble-only detections and traffic on other channels are not counted, so it
is a floor on receiver busy time rather than a measurement of it. Firmware's
counter has the same character, so the two remain comparable.
publish() skipped a connection whose disallowed_packet_types covered the packet
without recording anything, and the empty results list is how the same method
decides no broker was reachable. A node with healthy, connected brokers that all
filter a type therefore warned "No active broker connections" for every packet
of it -- loudest for exactly the common types someone would choose to filter.
Record the deliberate skip, so the warning fires only when nothing was
reachable. publish_mqtt already does this for its own intentional skips; this is
the same idiom one method up rather than a new mechanism.
The disabled-connection branch was never affected, which is why the warning
looked correct in a node with a broker switched off.
fabric.repeat_on_ingress and fabric.origin_tx are only checked when
build_radio_stack assembles the radios. Everything that writes config -- the
UI, a backup restore, a hand edit -- ran long before that, so a combination
with no defined meaning was accepted, written to disk, and refused only at the
next restart, by which point the caller that could have fixed it was gone.
validate_fabric_config runs those same checks against a whole config mapping.
config_import calls it before the section-merge loop touches self.config, and
validate_config reports the result as a fabric error, so the pre-restart check
names the thing that will block the restart; it covered no fabric or radios key
at all before.
Checking before the loop rather than inside it is the point. The loop merges
section by section, so a rejection partway through would leave half an import
applied to a running node. A malformed radios section is refused the same way,
ahead of everything, since the loop's own check for it ran only once the
earlier sections were already in. That also keeps the message about the real
problem: a non-list radios previously reached the fan-out check as "one radio"
and came back as a radio-count error.
The merge the preflight models has to match the loop exactly, or it rejects
configs the daemon would accept. A dict fabric merges by update; anything else
replaces the section wholesale, including fabric: null. That last case is how
an operator clears fan-out left invalid by an older build, so validating the
old section there would reject the one import that repairs the node.
tx_mode is checked only where the daemon applies it. _apply_fabric_tx_mode --
the thing that raises on an unknown mode -- runs for a non-empty radios list or
use_fabric and nowhere else, so a legacy single-radio node carrying a stale or
misspelled tx_mode boots fine today. Refusing its unrelated imports would be
this check inventing a failure rather than reporting one.
A radio id resolved to its frequency and modulation, which is enough to say
which band a packet was on but not how hard it was pushed. Two radios on one
node routinely run different power -- a wide local radio turned down to keep
off its neighbour, a narrow backhaul at full -- and nothing in the air settings
says so, which leaves an observer comparing their coverage with a term missing.
Each entry in the status `radios` map gains `tx_power`, in dBm.
The value is read through build_radio_profiles, so it inherits the same merge
and per-radio override rules as the rest of the profile rather than a second
reading of the config. get_radio_for_board applies a different default per
radio type -- 14 for kiss, 22 for the modems, and none at all for sx1262, which
requires the key -- so those defaults now live beside the other air defaults and
a config that is silent reports what is actually on the air.
Where the power cannot be known it is omitted rather than guessed, and unlike
the air settings it never voids the map: power does not decide which band a
packet was on, so an entry without it still attributes the packet correctly.
Single-radio nodes publish no map at all, as before.
Everything the map reports is what the node is configured with, not a reading
from the hardware, and the docstring now says so: a driver may clamp what it was
given (an SX1262 holds -9..22 dBm), and editing a radios[] entry does not
reconfigure that radio live, because config_manager only applies the top-level
radio section. That gap predates this field -- it applies to the frequency and
modulation already in the map -- but a power figure invites the comparison, so
it is written down rather than left to be discovered.
A boolean reads as unknown rather than as int(True)'s plausible-looking 1 dBm,
and an unreadable optional field logs at debug: this runs once per status
publish, so a warning there would repeat for the life of the node.
The filter added in the previous commit asked whether the record carried
a drop_reason. Every TRACE record carries one by design - trace_received
on arrival, trace_forwarded once relayed - and TraceHelper reaches
storage through log_trace_record, which took the default. Traces
therefore stopped reaching the brokers entirely.
Classifying a packet as invalid is the caller's job, and the engine
already does it; drop_reason is not that classification, since
duplicates, policy drops and traces all carry one and all belong on the
observer feed. So record_packet takes the caller's judgement as final
and defaults to publishing: skip_mqtt_if_invalid (default True, meaning
"decide for me") becomes skip_mqtt (default False, meaning "this one is
invalid").
TRACE also builds its own record rather than going through
_build_packet_record, so it never picked up the ingress radio. Read it
off the packet there. There is no egress counterpart: a forwarded trace
is injected into the router later and only mutates the in-memory record,
so what reaches the brokers is always the reception.
Neither gate has ever blocked anything.
disallowed_packet_types resolves configured names to integer payload-type
codes, but the publish path compared them against the payload's `type`
field, which the MC2MQTT schema fills with the literal string "PACKET".
The code lives in `packet_type`, serialised as a decimal string. Read it
from there as an int. A payload with no packet type (status, diagnostic
streams) or an unparseable one is published rather than dropped: not
being classifiable is not evidence of being disallowed.
skip_mqtt_if_invalid was computed in the engine, passed through
record_packet, and never read. Honour it for the broker publish only:
a packet this node could not parse is still stored and still reaches
Glass and the dashboard, which are the surfaces an operator debugs
their own RF from. Only malformed adverts, empty payloads and over-long
paths are affected - duplicates and policy drops are valid packets this
node chose not to repeat, and observers still see those.
A two-radio bridge is one MeshCore node with one pubkey, and the path
byte it appends on repeat is the same whichever radio carried the
packet. Nothing on the air says which band a hop used, and an observer
receiving on one side can only infer the other by already knowing its
neighbour's frequency. The node itself does know, so say it.
Two halves, both needed:
- Packets gain rx_radio_id (the radio that heard it) and tx_radio_ids
(every radio that retransmitted it - a bridge repeating on both sides
lists both). tx_radio_ids is always a list, so a consumer never has to
branch on the JSON type, and is absent rather than empty when the
packet was not repeated.
- Status gains radios: [{id, radio}], mapping each id to its
MHz,kHz,SF,CR settings. Without it a radio id is just an
operator-chosen label. Ids are unique only within a node, so consumers
key on (origin_id, id). Rebuilt per publish, since air settings can
change from the web UI while the node runs.
Single-radio nodes publish neither and their payload stays byte-for-byte
what it was: the ids would say nothing the status message doesn't
already. A node whose radio settings cannot all be read publishes no map
at all rather than a partial one that would place a packet on the wrong
band.
On a Fabric node the top-level status radio field now reports the
default radio rather than the top-level radio: block, which is the base
entries inherit from and can describe a band no radio is on.
The daemon samples each radio every 30 seconds, but /stats carried only one
figure, the default radio's. A client drawing a curve per radio could keep the
default one current and had to wait on a history refetch for the rest, so on a
bridge one curve moved live and the others lagged by minutes.
Keep the existing scalar exactly as it was — the observer feed and Glass still
carry one noise floor per node, and that is a separate change — and report the
other radios beside it in noise_floor_radios. A single-radio node has no such
list and its response is unchanged.
A radio that leaves the fabric drops out rather than leaving a stale figure
behind; one whose read times out keeps its last, so a curve that was drawing
fine does not blank on a single missed sample.
Two independent adversarial reviews of this branch. Every defect below was
reproduced before being fixed, and each fix is pinned by a test that fails
against the code it replaces.
A channel's ledger now lives as long as the channel does, instead of being
rebuilt from copies of its neighbours' history on every config reload. The
copying was wrong in both directions at once:
- A channel whose radios all moved away lost its ledger, so the next radio to
tune there started from zero. 3000 ms already on the air, then a fresh
3600 ms allowance inside the same minute: 6600 ms/min against a 3600 ms/min
limit, reachable with two applies in the web UI.
- A split handed the same history to both sides and the next merge added the
copies together, so retuning a radio back and forth doubled the ledger each
cycle. Six applies turned 500 ms of real airtime into 4000 ms and stopped the
node forwarding for a minute; total_airtime_ms is a lifetime counter, so the
inflation also reached the MeshCore wire field, the RRD/SQLite history,
companion stats and /stats permanently.
Nothing is carried between channels now. A channel keeps what was transmitted
on it, and a radio arriving from a busy band brings none of that band's spend
with it, because duty cycle is a property of spectrum. Spectrum the node has
left keeps its ledger until its window empties, and its lifetime total joins the
node's running total rather than vanishing. The one exception is the ``None``
key used when radio profiles cannot be read: it stands for "some channel, we
cannot say which", so it inherits the busiest window rather than starting clean.
Three more:
- A config reload lands on another thread, and the gate computed the airtime
before the send while the debit recomputed it after. A retune in that window
charged the modulation the radio has now for a transmission that went out on
the modulation it had a moment ago -- 213.5 ms recorded as 26.7 ms. The debit
now reuses the figure the gate computed for that radio.
- Only the default radio can be retuned without a restart, but a rebuild adopted
every radio's configured modulation, so an unrelated duty-cycle save started
metering a pending change the hardware had not been given. refresh() now
adopts a new modulation only for radios that really got one; limits are policy
rather than hardware and are always adopted.
- POST /set_radio_config with radio_id: "" fell past the radios[] lookup without
matching or erroring, wrote the top-level block alone, and left the default
radio's own entry behind -- hardware on one bandwidth, its meter on another. A
blank id is now read as the request not naming one.
Holding every radio the fabric could pick to its own budget was safe against
overrun but was not the behaviour these nodes had: against an older openhop-core
an exhausted local radio refused sends the fabric was about to put on an idle
link radio, black-holing that band for the rest of the window. The gate asks the
fabric which radio it is about to use -- under the TX lock, an instant before
the send -- and holds that one to its budget. The candidate set remains the
answer only when the fabric will not say.
Suite: 2163 pass, 3 skipped against openhop_core dev; 2160 pass, 6 skipped
against c95a684, which predates the selector change.
The tests that cover per-channel metering ask a fabric which radio will carry a
packet, which an openhop_core from before the selector change cannot answer. Run
against one, fourteen of them failed -- on a repeater whose whole point is that
the core change is optional.
Tests that genuinely need the newer core (they drive build_radio_stack through a
real FabricRadio) are skipped when it is absent. The engine tests instead use a
fabric double that implements core's resolve rule and nothing else: explicit
radio_id wins, then the selector, then default_radio. That keeps the engine
measured against the contract rather than against a convenience, and the real
RFFabric is still what runs wherever the installed core supports it.
One fan-out test asserted the engine names the egress radio. What must not
change is which radio transmits; whether the engine names it or leaves the
choice to the fabric depends on the core underneath, so it accepts either.
Against openhop_core c95a684: 2154 pass, 6 skipped. Against dev with the
selector change: 2157 pass, 3 skipped.
A duty cycle is a limit on a channel, not on a node, and one AirtimeManager
charging every radio at the top-level radio block's modulation is wrong twice
over on a bridge. The same packet occupies a 62.5 kHz channel about eight times
longer than a 500 kHz one, so whichever bandwidth that block names, the other
radio is mis-charged by that ratio; and the two share one allowance, so a busy
local radio spends the backhaul's airtime on a band it never transmits on.
There is now one budget per channel, built from each radio's own profile and
its own band's limit -- 868.0-868.6 MHz allows 1% where 869.4-869.65 allows
10%, and one number cannot describe both. Two radios tuned alike share a
budget, because they contend for the same spectrum. duty_cycle.shared_budget
restores a single node-wide allowance for anyone who wants the conservative
reading, and a single-radio node is metered exactly as it was.
Metering a channel means knowing which radio will carry a packet, and the
engine used to work that out by restating openhop_core's dispatch rules. It
asks now. openhop_core passes a packet's ingress radio to the TX policy, so the
choice is a function of the packet rather than of whatever arrived most
recently; the repeater asks before scheduling and gets the answer the send will
reach. That also closes a race that predates this: sticky and bridge read the
node's last RX at send time, which after a retransmit delay on a busy node is
another packet's arrival, and moves a reply onto the wrong band.
The core change is optional. The keyword in resolve_tx_radio_id's signature is
the version check, and against a core without it the tx_mode selectors are
registered in their original one-argument form and behave exactly as they did.
What changes there is the gate: with no way to know which radio will transmit,
every radio the fabric could pick is held to its own budget, so a node cannot
transmit past a legal limit by guessing the quiet channel. Which mode is in
force is logged at startup.
Also fixed, each reproduced by an adversarial review of the earlier attempt:
- A node with one radios[] entry had its modulation and limit overwritten by
the stale top-level block on every radio save from the web UI. The refresh
path now branches on whether profiles were read, not on how many radios
there are.
- build_radio_profiles is all-or-nothing by design -- it feeds the UI -- so one
radio_type: none collapsed metering onto the top-level block for the radio
still transmitting. Metering has its own builder that degrades per radio.
- Carrying the whole node's spend onto an unrecognised channel compounded over
repeated rebuilds (100 -> 400 -> 1600 ms) and wedged transmission for the
rest of the window. Carry-over is keyed by channel, which makes an unchanged
rebuild idempotent, and an unrecognised one starts from the busiest single
channel rather than the sum of them.
- A rebuild worked the previous default manager out from the config again,
which by then already named the new default -- preserving the wrong object
and orphaning the one NeighborScopeHelper holds for the life of the process.
It is remembered rather than recomputed.
- Every caller that reports one airtime figure was reporting the default
radio's channel: the MeshCore wire field total_air_time_secs, companion
stats, the RRD and SQLite history, and /stats. They read a node total again
-- totals summed across channels, utilisation the highest of them, because a
node at its limit on one channel is at a limit however quiet the other is.
tests/test_fabric_fanout.py's rig never bound a tx_mode selector, so all 49 of
those tests ran against a fabric with no TX policy at all. It binds the real
one now; that gap is why an on-air change went unnoticed through two reviews.
A bridge sends one logical packet through both radios, and the two do not fare
alike: 500 kHz and 62.5 kHz are different channels with different neighbours and
different contention. The packet row holds one set of LBT figures, from the
primary egress, so the other radio's attempts and backoffs were dropped on the
floor -- and an egress that failed was never stored at all, leaving the half of
the story an operator most wants to see only in the log.
A new packet_egress table keeps one row per physical send: packet id, timestamp,
radio, whether it went out, and its LBT attempts, total backoff and channel-busy
flag. Written only on a node with two or more radios; with one radio the packet
row already says all of this, and nothing about single-radio behaviour changes.
The rows are written on the storage writer thread beside the packet, addressed
by the id the insert returns, and are kept off packet_record itself so they do
not ride along on every websocket, Glass and MQTT publish. Retention ages them
out on the packet cutoff, and purging packets takes them along.
/lbt_diagnostics gains radios[]: the same summary and bucket shape per radio,
read from the new table over a covering index, so one radio's contention is
never read as the other's. The combined figures above it are unchanged and still
count each packet once. Per-radio entries carry no rf block -- the RRD series
behind it is node-wide, and hanging it off one radio would read as that radio's
own RSSI and loss -- and no packet-type breakdown, which would cost a heap
lookup per egress.
The fan-out now reads a send's LBT metadata whether or not it succeeded. The
single-egress path has always done so, and the figures on a failed send are
exactly the ones that explain it -- as far as they reach us: a radio that raises
instead of returning metadata (an SX1262 giving up on a busy channel) reports
nothing to read, so that row records the failure with no attempts rather than
inventing them.
Redundancy copies are recorded too. A multi-ack burst is real airtime on both
radios, and leaving it out would undercount exactly the traffic multi-ack adds
to a channel already busy enough to need it.
/db_stats lists packet_egress only once something has been written to it, so a
single-radio node reports the tables it always reported and never pays for
counting rows it does not have.
The bucket and summary maths moved out of get_lbt_diagnostics into three module
functions so the per-radio series is the same code as the combined one rather
than a second copy of 150 lines. The combined answer is unchanged.
A bridge has one receiver per band, and until now one of them stood for the
node: the background timer read the default radio and stored the answer with no
radio attached. On the 910.100/910.525 pair that reported the wide radio's noise
floor as the whole node's, and charged its CRC counter with both radios' errors.
Every physical radio is now sampled, one after another, and stored against its
own id. Migration 19 adds a nullable radio_id to noise_floor and crc_errors with
a covering index each, so a per-radio history reads only its own rows. Rows
written before this stay NULL: they were never measured per radio, and
labelling them would claim an attribution nobody made.
CRC baselines are per radio. A shared one charges every radio's errors to
whichever was sampled last, which reports swings that never happened.
Reads take an optional radio_id: /noise_floor_history, /noise_floor_stats,
/crc_error_count and /crc_error_history. Naming a radio also makes offset paging
stable, which the RF Health page depends on -- paging the merged series shifts a
page's contents every time the other radio samples. History rows name their
radio only on a node with two or more radios, so a single-radio response is
byte-for-byte what it was.
The default radio is sampled first and is the only one published. /stats, the
MQTT status message and companion stats read its cached value, and publishing
per radio doubles the observer-feed cadence -- its own change, not this one.
The StorageCollector facade carries the new arguments too. The web endpoints
reach the database through it, so the tests that drive them autospec it: a bare
mock accepts any keyword and would have let a facade that had not kept up
answer happily in tests and raise TypeError on a real node.
The self-echo guard registered a packet's hash and forwarded path when the
engine decided to forward it and kept it for 30 s. Two things could then
hide a real neighbour from the link tracker:
- A neighbour whose 1-byte path hash collides with ours, repeating the
same packet from the same upstream path, carries exactly the path we
would have sent.
- The registration happened before the duty-cycle gate, so a packet that
was never transmitted still suppressed matching receptions.
Registration now happens at the physical send, inside the TX lock after
the duty-cycle gate, and records which radio is sending. A send that
fails is forgotten. A reception counts as our echo only when it arrives on
a radio other than the one that sent it, and while that send is on the
air or within 2 s of it finishing, which covers a remote modem's delivery
delay. Single-radio nodes keep no records at all, since a lone radio
cannot hear itself.
A colliding neighbour is now mistaken for our echo only if it repeats the
same packet on the same path, on the other radio, inside that window.
The engine now passes each reception's radio to the neighbour link
tracker, from both the forwarding path and the duplicate-recording path,
so /neighbor_links can report how each radio hears a neighbour.
Two radios sharing one frequency hear each other transmit. The echo of a
relayed flood ends in this node's own path hash, so the tracker recorded
the node as its own neighbour, at near-zero-dB signal. The engine now
remembers the packet hash and exact path of each transmission for 30 s,
registered before the first send because an echo can arrive while a
later egress of the same fan-out is still on the air, and skips tracker
observation for a reception that matches. A neighbour repeating our copy
appends its own hash and still counts. The echo is still stored and
published as the reception it is; only the neighbour table ignores it.
Matching the exact path rather than the last hop's hash keeps a real
neighbour whose 1-byte hash collides with ours.
On a Fabric node the analytics endpoints merged every radio together, with
one transmission per logical packet and nothing to tell a reception on one
radio from the other. Every packet row already records its ingress radio
and each radio that sent it, so the split needs no new data.
The attribution rules get_airtime_buckets introduced move into shared
helpers: packet_carriers() says which stored radio ids a row occupied, and
RadioResolver maps them onto the configured radios (one radio owns every
row; with two, an unknown id is unattributed, never guessed).
get_airtime_buckets now uses them with no change in output.
With two or more radio profiles:
- /packet_stats adds radios[] (received, duplicates, dropped, physical
transmissions, average RSSI/SNR) and unattributed counts. The legacy
totals come from the same grouped pass, so the 24 h aggregate the
WebSocket broadcasts does not gain a second scan. Totals still count
one transmission per packet.
- /route_stats adds the route mix of each radio's receptions, plus
originated for packets no radio received.
- /radio_packet_rates (new) returns receptions and transmissions per
radio per bucket, grouped in SQL on columns idx_packets_airtime covers.
A week-long window is an index-only scan instead of a Python loop over
every row: 0.17 s against 1.11 s for the airtime loop on a synthetic
1.55M-row table.
- /neighbor_links gives each link radios[], its stats as each radio heard
it. The tracker keeps them beside the merged figures without changing
its keys or entry count.
- /neighbor_link_history rows carry rx_radio_id, and the endpoint takes a
radio_id filter and a by_radio bucket split.
A single-radio node gets exactly the answers it had: the legacy queries
still run for it and every new field is omitted.
get_airtime_buckets selects rx_radio_id, tx_radio_id and tx_radio_ids to
charge airtime per radio, but idx_packets_airtime only covers timestamp,
length, payload_length and transmitted. SQLite therefore fell back to
idx_packets_timestamp with a heap lookup for every row in the window:
the access pattern the covering index was added to remove, and the one
that pushes the chart past the client timeout on an SD card. The test
meant to guard this asserted the plain timestamp index.
Migration 18 drops idx_packets_airtime and rebuilds it with the three
radio id columns. CREATE INDEX IF NOT EXISTS never reshapes an existing
index, so the base schema cannot do this. It keeps creating the original
four-column shape, which is safe on a database that predates the radio
id columns, and the migration widens it once migrations 16 and 17 have
added them. The rebuild reads the packets table once, at startup.
The bucket query is now a module constant shared with the test, which
asserts COVERING INDEX idx_packets_airtime for both airtime queries, so
a column added to the query cannot slip out of the index unnoticed.
openhop-dev/openhop_core#123 settled on fabric.origin_tx: default | all
as the name for sending locally originated packets out on every radio,
and openhop-dev/openhop_core#124 with openhop-dev/openhop_repeater#412
implement that knob in core. Use the same key here so a Fabric config
means one thing whichever implementation lands.
local_tx_mode is still read, so a config written against the first
version of this branch keeps its behaviour. If both keys are set to
different modes, startup rejects the config before any hardware is
opened, since there is no way to tell which one the operator meant.
While running, the engine reads the option through the same helper and
lets origin_tx win. Validation errors name the key the operator actually
wrote.
The radio stack meta and the startup log report origin_tx.
A two-radio Fabric bridge runs each side at its own SF, so one combined
utilization line computed from a single caller-supplied profile describes
neither radio. Report airtime per radio instead.
build_radio_stack now records the effective air settings of every radio it
builds, in configured order, as radio_stack_meta["radio_profiles"]. The
same builder is exposed as build_radio_profiles() so /stats and the airtime
endpoint can read current settings from the live config rather than a boot
snapshot. Air settings are read the way get_radio_for_board reads them, and
a field that cannot be read is reported as null rather than defaulted, so
nothing is charted from settings that were never on the air.
get_airtime_buckets takes those profiles and charges each stored packet to
the radios that actually carried it: its ingress rx_radio_id and every
successful egress in tx_radio_ids, falling back to the scalar tx_radio_id
for pre-fan-out rows. A relayed packet therefore appears as RX on the radio
that heard it and TX on each radio that sent it, each at that radio's own
time-on-air. Duplicate receptions are included; they occupied the receiver.
With one profile every packet belongs to that radio, so single-radio and
pre-Fabric databases are attributed exactly as before. With two or more, a
packet whose radio cannot be identified is counted as unattributed rather
than guessed onto the default radio.
The response is additive: legacy buckets/rx_total/tx_total remain as the
combined series, and radios[] carries one series per radio alongside
unattributed_rx_count/unattributed_tx_count. Airtime comes from the shared
core estimator instead of a second inline Semtech formula, memoised per
(radio, length). The sf/bw_hz/cr/preamble query parameters are kept only as
a fallback; server profiles are authoritative and are never applied across
both radios of a bridge.
The existing guard stopped at the bridge's injector, so it proved inheritance
but not that the override survives what the repeater actually does with a
packet. This adds the missing leg: a real core Dispatcher, a real
RepeaterHandler engine and a real PacketRouter, with only the radio and the
daemon container substituted. The assertion is on the bytes read back off the
radio.
The conflict is made real rather than notional. Both the companion's default
scope and the dispatcher's are set to the repeater's own region, so without
the override that region is what goes out -- asserted by a control test that
sends the same message on the same channel with no override and finds the
repeater's code on the air. Only the override differs between the two.
Verified by mutation rather than by assuming: neutering
`_apply_explicit_flood_scope` so it falls back to the node's resolver fails
the integration test while leaving the control passing, so the test is
sensitive to the thing it names and not merely to "something got scoped".
A second mutation settles the marker question the previous commit reworded.
Clearing `_flood_scope_applied` while keeping the scoping leaves the
integration test green and fails only the assertion on the mark itself,
confirming the transport code is protected by the route change alone.
The fake radio returns a metadata dict because that is what the shipped
wrappers do. A bare True crashes the engine at `tx_metadata.get(...)` after
the bytes are already on the air -- harmless here since sx1262 and kiss_modem
both return dicts, but it would have masked a real failure behind an
apparently successful send.
Follow-up to review.
The dependency pin is `openhop_core@dev`, so CI installs a Core without the
per-message flood scope override until that change lands there. The guard
imported symbols from it directly, which meant the module could not import at
all and this branch could not be green on its own before Core merged.
The constants are now reached through the module and a `skipif` covers the
whole file, so the branch stands alone and the guard resumes the moment Core
lands. The skip reason names the cause, so a permanent silent skip would be
visible in the report rather than reading as a pass.
Also corrects the `_flood_scope_applied` comment. It claimed losing the mark
would let the dispatcher re-scope the packet to the repeater's own region. It
would not: `scope_packet` has already set TRANSPORT_FLOOD and the dispatcher
re-scopes only ROUTE_TYPE_FLOOD. The assertion stays, as the contract Core
states, but it is no longer described as load-bearing.
Still outstanding, deliberately: this stops at the bridge's injector and does
not drive PacketRouter.inject_packet, the engine, and the final dispatcher
with a conflicting repeater region configured. That is the test worth adding
next if this guard is ever to carry real weight.
openHop Core owns the per-message channel flood scope end to end, including
its protocol tests. What Core cannot check is that this package's subclasses
still reach it: RepeaterCompanionBridge and CompanionFrameServer both sit
between a client and Core's send path, and either could shadow a method or
drop a keyword without any Core test noticing.
The second test is the one that matters on a repeater specifically. The
dispatcher re-resolves flood scope for everything it sends, so a scoped
companion packet only survives because Core marks it decided. Were that mark
lost, the message would go out under the repeater's own region instead of the
one the client asked for, and nothing would report an error.
No feature code is needed here -- the behaviour is inherited from Core.
Add two Fabric options for an exactly-two-radio node:
- fabric.repeat_on_ingress (tx_mode: bridge): a relayed packet is sent
on the other radio first, then back out on the radio that heard it.
- fabric.local_tx_mode: all: locally originated packets (companions,
adverts, room/protocol replies) go out once on every radio, default
radio first.
MeshCore processing (policy, dedupe, path mutation) still runs once per
logical packet; only the physical TX is duplicated, each egress with its
own Packet, serialised through the TX lock and gated independently for
duty cycle and link health. One successful egress is a successful
forward. Multi-ack extras share the egress set.
inject_packet registers its single ACK waiter before the first TX so an
ACK heard mid-fan-out outlives the dispatcher's ACK cache. Fan-out fixes
flood scope and path-hash mode once before copying and metering, so all
radios carry identical bytes and the duty-cycle gates see them.
Packet records gain tx_radio_ids (JSON column + migration); tx_radio_id
remains the primary successful egress. Invalid option combinations are
rejected before hardware is opened. Defaults preserve existing behaviour.
The companion frame proxy only accepted an operator JWT in `?token=`,
while `/ws/packets` also accepts a long-lived API token (X-API-Key
header, or the same query parameter). Apply the identical rule to the
frame proxy so a client that already holds a repeater API key can open
the companion link on the HTTP port without keeping a JWT session alive.
No behaviour change for JWT clients. Same fallback order, close codes
and log lines as PacketWebSocket.
After mesh.default_region was wired to the core dispatcher's send-time
scope resolver, a relayed plain flood was indistinguishable from a
locally originated one, so Dispatcher._apply_flood_scope() converted it
to the repeater's default transport scope and sent it out-of-region.
Successful flood forwards now mark their existing scope decision as
authoritative before entering Dispatcher.send_packet(): relayed plain
floods stay plain and relayed transport floods retain the originator's
codes. Locally originated floods and REPLY_SCOPE_DEFAULT fallback
replies still pick up the configured default scope.
Marking only on the flood_forward() success path leaves packets rejected
by validation, policy, duplicate, or loop detection untouched, and
leaves locally injected transmissions (which bypass forwarding) alone.
Rightup