incoming grp_data issue #161

New Issue

2026-10-27T09:35:53+01:00

adam commented

2026-10-27 09:35:53 +01:00

In developing a plug-in for openhop that relies on the GRP_DATA packet type, it appears that the incoming queue process strips the packet of some information that my plugin needs:

Impact: every companion-protocol app that listens on a binary channel-data type is
deaf. In our case the plugin's clients can send refresh requests to
the host, but the host never receives them.

Root cause

openhop_core's frame server pushes every received channel-data message through the
_persist_companion_message(msg_dict) hook and then pops it from the in-memory
queue — on this box, every inbound message goes through a SQLite round-trip. That
round-trip drops the two fields that make the message channel data:

repeater/data_acquisition/sqlite_handler.py — companion_push_message()
(~line 2666): the INSERT does not store channel_data_type /
channel_data_payload (the companion_messages table has no such columns).
repeater/data_acquisition/sqlite_handler.py — companion_pop_message()
(~line 2728): the SELECT doesn't return them either.
repeater/companion/frame_server.py — _sync_next_from_persistence()
(~line 82): rebuilds QueuedMessage without them, so channel_data_type
defaults to 0 and channel_data_payload to empty.
openhop_core/companion/frame_server.py — _build_message_frame()
(~line 1344) then sees channel_data_type == 0 and emits the message as an
empty group-TEXT frame instead of a channel-data frame.

Net effect: the payload and its type are silently discarded between persist and
pop. (The same round-trip also drops snr, rssi and packet_hash for all
message types — they're persisted but never restored. Minor, but same family.)

There is a proposed fix: (two actually)

Keep binary channel data out of persistence; it is transient signal data, and
losing it across a repeater restart is acceptable.

In repeater/companion/frame_server.py, at the top of
_persist_companion_message:

if msg_dict.get("channel_data_type"):
    return  # binary channel data: leave in memory for sync_next_message

i.e. don't persist it and don't pop_last() it — the bridge's in-memory queue
delivers it on the next sync_next_message poll. Group text keeps the existing
offline-storage behavior. ~2 lines, no schema change.

Option B

Make the round-trip lossless:

New migration adding to companion_messages:
- channel_data_type INTEGER NOT NULL DEFAULT 0
- channel_data_payload BLOB
companion_push_message(): include both in the INSERT.
companion_pop_message(): include both in the SELECT.
_sync_next_from_persistence(): pass both into the QueuedMessage(...)
constructor (channel_data_type=..., channel_data_payload=...), and
ideally also restore snr/rssi/packet_hash while touching it.

Thank you

In developing a plug-in for openhop that relies on the GRP_DATA packet type, it appears that the incoming queue process strips the packet of some information that my plugin needs: Impact: every companion-protocol app that listens on a binary channel-data type is deaf. In our case the plugin's clients can send refresh requests to the host, but the host never receives them. ## Root cause `openhop_core`'s frame server pushes every received channel-data message through the `_persist_companion_message(msg_dict)` hook and then pops it from the in-memory queue — on this box, every inbound message goes through a SQLite round-trip. That round-trip drops the two fields that make the message channel data: 1. `repeater/data_acquisition/sqlite_handler.py` — `companion_push_message()` (~line 2666): the INSERT does not store `channel_data_type` / `channel_data_payload` (the `companion_messages` table has no such columns). 2. `repeater/data_acquisition/sqlite_handler.py` — `companion_pop_message()` (~line 2728): the SELECT doesn't return them either. 3. `repeater/companion/frame_server.py` — `_sync_next_from_persistence()` (~line 82): rebuilds `QueuedMessage` without them, so `channel_data_type` defaults to 0 and `channel_data_payload` to empty. 4. `openhop_core/companion/frame_server.py` — `_build_message_frame()` (~line 1344) then sees `channel_data_type == 0` and emits the message as an empty group-TEXT frame instead of a channel-data frame. Net effect: the payload and its type are silently discarded between persist and pop. (The same round-trip also drops `snr`, `rssi` and `packet_hash` for all message types — they're persisted but never restored. Minor, but same family.) There is a proposed fix: (two actually) Keep binary channel data out of persistence; it is transient signal data, and losing it across a repeater restart is acceptable. In `repeater/companion/frame_server.py`, at the top of `_persist_companion_message`: ```python if msg_dict.get("channel_data_type"): return # binary channel data: leave in memory for sync_next_message ``` i.e. don't persist it and don't `pop_last()` it — the bridge's in-memory queue delivers it on the next `sync_next_message` poll. Group text keeps the existing offline-storage behavior. ~2 lines, no schema change. ### Option B Make the round-trip lossless: 1. New migration adding to `companion_messages`: - `channel_data_type INTEGER NOT NULL DEFAULT 0` - `channel_data_payload BLOB` 2. `companion_push_message()`: include both in the INSERT. 3. `companion_pop_message()`: include both in the SELECT. 4. `_sync_next_from_persistence()`: pass both into the `QueuedMessage(...)` constructor (`channel_data_type=...`, `channel_data_payload=...`), and ideally also restore `snr`/`rssi`/`packet_hash` while touching it. Thank you

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: adam/pyMC_Repeater#161