is_mqtt_gateway never set when a node gateways before its Node row exists (one-shot UPDATE + permanent cache) #62

New Issue

2026-10-05T06:04:25+02:00

adam commented

2026-10-05 06:04:25 +02:00

Summary

A node that acts as an MQTT gateway before meshview has a Node row for it is never marked is_mqtt_gateway = True, so it permanently shows MQTT Gateway: No in the UI even though it is a fully working gateway and publishes correct ServiceEnvelopes (with its own gateway_id) to the broker meshview is reading.

Root cause

In meshview/mqtt_store.py the gateway is marked with a one-shot UPDATE guarded by a permanent in-memory cache:

node_id = int(env.gateway_id[1:], 16)

if node_id not in MQTT_GATEWAY_CACHE:
    MQTT_GATEWAY_CACHE.add(node_id)                 # added unconditionally
    await session.execute(
        update(Node).where(Node.node_id == node_id) # UPDATE, not upsert
        .values(is_mqtt_gateway=True)
    )

If, at the moment the first gateway packet from node_id is processed, no Node row exists yet (its NodeInfo hasn't been received/decoded), the UPDATE ... WHERE node_id = X affects 0 rows. But node_id has already been added to MQTT_GATEWAY_CACHE, so the block never runs again for that node. Later, when the NodeInfo finally creates the Node row, is_mqtt_gateway stays NULL forever.

The startup cache is loaded only from already-True nodes (select(Node.node_id).where(Node.is_mqtt_gateway == True)), so a restart clears the stale cache entry and lets the node get marked on its next gateway packet — which is exactly the observed workaround (see below).

Who hits this

Gateways whose own NodeInfo reaches meshview late or not at all relative to their first relayed packet, e.g.:

Gateways that start relaying traffic immediately on boot (busy mesh) while their own NodeInfo is infrequent.
Nodes with channel encryption enabled for MQTT, where meshview can't decode their NodeInfo quickly to create the row.

In my setup, 3 Ethernet gateways (publishing correctly to a local broker, verified by subscribing) all showed MQTT Gateway: No while a different node showed Yes, purely due to packet/NodeInfo ordering.

Reproduction

Point meshview at a broker.
Have a gateway start publishing relayed packets (gateway_id = itself) before meshview has decoded that node's NodeInfo.
The node never gets is_mqtt_gateway = True, even after its NodeInfo arrives and its Node row exists.

Workaround

Restart meshview (clears the in-memory cache; the node is re-marked on its next gateway packet now that its row exists), or
UPDATE node SET is_mqtt_gateway = 1 WHERE node_id = <id>; directly.

Suggested fix

Make the marking robust to ordering, e.g. one of:

Upsert the node (insert a stub row, or ON CONFLICT DO UPDATE) so the flag is set even if the row doesn't exist yet, or
Only add to MQTT_GATEWAY_CACHE when the UPDATE actually affected ≥1 row (check result.rowcount), so it retries on a later packet once the row exists, or
Set is_mqtt_gateway = True at the same place the Node row is created/updated from NodeInfo, keyed on the gateway-seen set.

The second option is the smallest change and removes the dependency on NodeInfo/gateway-packet ordering.

### Summary A node that acts as an MQTT gateway **before meshview has a `Node` row for it** is never marked `is_mqtt_gateway = True`, so it permanently shows **MQTT Gateway: No** in the UI even though it is a fully working gateway and publishes correct `ServiceEnvelope`s (with its own `gateway_id`) to the broker meshview is reading. ### Root cause In `meshview/mqtt_store.py` the gateway is marked with a **one-shot `UPDATE`** guarded by a **permanent in-memory cache**: ```python node_id = int(env.gateway_id[1:], 16) if node_id not in MQTT_GATEWAY_CACHE: MQTT_GATEWAY_CACHE.add(node_id) # added unconditionally await session.execute( update(Node).where(Node.node_id == node_id) # UPDATE, not upsert .values(is_mqtt_gateway=True) ) ``` If, at the moment the first gateway packet from `node_id` is processed, **no `Node` row exists yet** (its `NodeInfo` hasn't been received/decoded), the `UPDATE ... WHERE node_id = X` affects **0 rows**. But `node_id` has already been added to `MQTT_GATEWAY_CACHE`, so the block **never runs again** for that node. Later, when the `NodeInfo` finally creates the `Node` row, `is_mqtt_gateway` stays `NULL` forever. The startup cache is loaded only from already-`True` nodes (`select(Node.node_id).where(Node.is_mqtt_gateway == True)`), so a restart clears the stale cache entry and lets the node get marked on its next gateway packet — which is exactly the observed workaround (see below). ### Who hits this Gateways whose **own `NodeInfo` reaches meshview late or not at all relative to their first relayed packet**, e.g.: - Gateways that start relaying traffic immediately on boot (busy mesh) while their own `NodeInfo` is infrequent. - Nodes with **channel encryption enabled** for MQTT, where meshview can't decode their `NodeInfo` quickly to create the row. In my setup, 3 Ethernet gateways (publishing correctly to a local broker, verified by subscribing) all showed **MQTT Gateway: No** while a different node showed **Yes**, purely due to packet/`NodeInfo` ordering. ### Reproduction 1. Point meshview at a broker. 2. Have a gateway start publishing relayed packets (`gateway_id = itself`) **before** meshview has decoded that node's `NodeInfo`. 3. The node never gets `is_mqtt_gateway = True`, even after its `NodeInfo` arrives and its `Node` row exists. ### Workaround - Restart meshview (clears the in-memory cache; the node is re-marked on its next gateway packet now that its row exists), **or** - `UPDATE node SET is_mqtt_gateway = 1 WHERE node_id = <id>;` directly. ### Suggested fix Make the marking robust to ordering, e.g. one of: - **Upsert** the node (insert a stub row, or `ON CONFLICT DO UPDATE`) so the flag is set even if the row doesn't exist yet, or - **Only add to `MQTT_GATEWAY_CACHE` when the `UPDATE` actually affected ≥1 row** (check `result.rowcount`), so it retries on a later packet once the row exists, or - Set `is_mqtt_gateway = True` at the same place the `Node` row is created/updated from `NodeInfo`, keyed on the gateway-seen set. The second option is the smallest change and removes the dependency on `NodeInfo`/gateway-packet ordering.

adam closed this issue

2026-10-05 06:04:25 +02:00

adam commented

2026-10-05 06:04:26 +02:00

I came across this issue while trying to research why I was persistently seeing "MQTT Gateway: No" on Meshview even though I clearly have a functioning MQTT gateway. In my case, the node was originally deployed without MQTT function and it was later added. It appears the record is similarly not being updated in this scenario, though I lack the time and technical skill to provide much more than my own observations.

I came across this issue while trying to research why I was persistently seeing "MQTT Gateway: No" on Meshview even though I clearly have a functioning MQTT gateway. In my case, the node was originally deployed without MQTT function and it was later added. It appears the record is similarly not being updated in this scenario, though I lack the time and technical skill to provide much more than my own observations.

adam commented

2026-10-05 06:04:27 +02:00

I have make this changes into the develop branch, thank you for your report.

I have make this changes into the develop branch, thank you for your report.

adam commented

2026-10-05 06:04:27 +02:00

Reproduced this today on a brand-new gateway, and I think the fix may not have
landed on develop — apologies if I am looking in the wrong place.

Reproduction, which matches the original report exactly:

A new gateway began publishing at 22:52, forwarding packets it had relayed for
other nodes, each with gateway_id = !f6fb8e00.
Its own NodeInfo did not arrive until 23:19, 27 minutes later.
It has shown MQTT Gateway: No ever since, while publishing valid
ServiceEnvelopes. Verified at the broker with a QoS 1 MQTT 5 publish
returning PUBACK RC:0, and verified in another meshview instance which
records packet_seen.node_id = 4143681024 for every packet.

The ordering in the report is not an edge case, it is the normal case. A gateway
starts relaying immediately on boot, while NodeInfo defaults to every 3 hours —
the Arizona community's (azmsh.net) recommended settings raise that to 12 hours, which
widens the window considerably.

On develop at time of writing, meshview/mqtt_store.py still has:

if node_id not in MQTT_GATEWAY_CACHE:
    MQTT_GATEWAY_CACHE.add(node_id)
    await session.execute(
        update(Node).where(Node.node_id == node_id).values(is_mqtt_gateway=True)
    )

None of the three suggested fixes appears present: there is no rowcount check
anywhere in the file, and the only on_conflict uses are for daily_snapshot,
packet and packet_seen, not Node. The same block is in the 3.0.8 tag and
in master.

I also checked whether the new ensure_node_exists() on develop closes it
incidentally, since it creates a Node row for an unknown node. It does not: its
only call site is after this block, and it is passed the packet's sender
rather than its gateway, so for a relayed packet it cannot create the row the
UPDATE needs.

Worth reopening? Happy to send a PR for the rowcount variant if that is the
option you prefer — it is a two-line change and keeps the cache's benefit:

if node_id not in MQTT_GATEWAY_CACHE:
    result = await session.execute(
        update(Node).where(Node.node_id == node_id).values(is_mqtt_gateway=True)
    )
    if result.rowcount:
        MQTT_GATEWAY_CACHE.add(node_id)

The documented workaround does work — restarting ingest clears the stale cache
entry and the flag is set on the next relayed packet.

Reproduced this today on a brand-new gateway, and I think the fix may not have landed on `develop` — apologies if I am looking in the wrong place. **Reproduction**, which matches the original report exactly: - A new gateway began publishing at 22:52, forwarding packets it had relayed for *other* nodes, each with `gateway_id = !f6fb8e00`. - Its own NodeInfo did not arrive until 23:19, 27 minutes later. - It has shown `MQTT Gateway: No` ever since, while publishing valid `ServiceEnvelope`s. Verified at the broker with a QoS 1 MQTT 5 publish returning `PUBACK RC:0`, and verified in another meshview instance which records `packet_seen.node_id = 4143681024` for every packet. The ordering in the report is not an edge case, it is the normal case. A gateway starts relaying immediately on boot, while NodeInfo defaults to every 3 hours — the Arizona community's (azmsh.net) recommended settings raise that to 12 hours, which widens the window considerably. **On `develop` at time of writing**, `meshview/mqtt_store.py` still has: ```python if node_id not in MQTT_GATEWAY_CACHE: MQTT_GATEWAY_CACHE.add(node_id) await session.execute( update(Node).where(Node.node_id == node_id).values(is_mqtt_gateway=True) ) ``` None of the three suggested fixes appears present: there is no `rowcount` check anywhere in the file, and the only `on_conflict` uses are for `daily_snapshot`, `packet` and `packet_seen`, not `Node`. The same block is in the `3.0.8` tag and in `master`. I also checked whether the new `ensure_node_exists()` on `develop` closes it incidentally, since it creates a `Node` row for an unknown node. It does not: its only call site is after this block, and it is passed the packet's **sender** rather than its **gateway**, so for a relayed packet it cannot create the row the `UPDATE` needs. Worth reopening? Happy to send a PR for the `rowcount` variant if that is the option you prefer — it is a two-line change and keeps the cache's benefit: ```python if node_id not in MQTT_GATEWAY_CACHE: result = await session.execute( update(Node).where(Node.node_id == node_id).values(is_mqtt_gateway=True) ) if result.rowcount: MQTT_GATEWAY_CACHE.add(node_id) ``` The documented workaround does work — restarting ingest clears the stale cache entry and the flag is set on the next relayed packet.

adam commented

2026-10-05 06:04:28 +02:00

Thanks @JeffreyPeacock, confirmed on my side. develop (at bb8da4c) and master both still add the node to MQTT_GATEWAY_CACHE before the UPDATE and never check whether it matched a row. Neither commit touching mqtt_store.py since this was closed changes that block, and no PR is linked to the close.

@pablorevilla-meshtastic could you reopen this? As the author I can't, since it was closed by a maintainer.

The rowcount variant is my preference too: smallest change, keeps the cache, and the only extra cost is one UPDATE per packet from that gateway until its Node row exists. Please go ahead with the PR.

Thanks @JeffreyPeacock, confirmed on my side. `develop` (at `bb8da4c`) and `master` both still add the node to `MQTT_GATEWAY_CACHE` before the `UPDATE` and never check whether it matched a row. Neither commit touching `mqtt_store.py` since this was closed changes that block, and no PR is linked to the close. @pablorevilla-meshtastic could you reopen this? As the author I can't, since it was closed by a maintainer. The `rowcount` variant is my preference too: smallest change, keeps the cache, and the only extra cost is one `UPDATE` per packet from that gateway until its `Node` row exists. Please go ahead with the PR.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: adam/meshview#62