mirror of
https://github.com/l5yth/potato-mesh.git
synced 2026-08-12 11:53:16 +02:00
web: fix meshcore message duplication regression (#825)
This commit is contained in:
@@ -1608,3 +1608,43 @@ remain green:**
|
||||
- **B1** (all suites). The only contract change is the additive `nodes` publish on
|
||||
message ingest (a new SSE event, documented in `CONTRACTS.md`); no POST/GET shape
|
||||
changes, so **C2** and the Python suite are unaffected.
|
||||
|
||||
---
|
||||
|
||||
## Bugfix: MeshCore cross-ingestor dedup keys on the stable channel name
|
||||
|
||||
A single physical MeshCore channel message heard by two ingestors that store the
|
||||
same logical channel at **different local channel-slot indices** was stored twice.
|
||||
The per-receiver `channel` index is not stable across ingestors (e.g. `#bot` sits
|
||||
at slot 4 on one device and slot 6 on another), yet it fed both the ingestor
|
||||
fingerprint discriminator (`c<N>` → two different `messages.id` values) and the
|
||||
#756 web content-dedup SELECT (`AND channel = ?` → no match), so neither dedup
|
||||
layer collapsed the duplicate. Fix (web-only, no wire change): the content-dedup
|
||||
matches on the sender-stable `channel_name` (NULL-safe) instead of the local
|
||||
`channel` index, so the safety net collapses the duplicate at the system of record
|
||||
regardless of differing ids/slots. Strengthens **C5**.
|
||||
|
||||
### MD-A1 — Same message on different local channel slots collapses to one row
|
||||
```bash
|
||||
( cd web && bundle exec rspec spec/data_processing_spec.rb -e "meshcore content dedup" )
|
||||
```
|
||||
**Expected:** pass, including "collapses the same meshcore channel message heard on
|
||||
different local channel indices": two meshcore messages with identical `from_id` /
|
||||
`to_id` / `text` / in-window `rx_time` and the **same `channel_name`** ("#bot") but
|
||||
**different `channel` indices** (4 vs 6) and different ids collapse to a **single**
|
||||
stored row. Companion examples still hold: messages with a **different
|
||||
`channel_name`** are kept separate (the legitimate distinct-channel case), and
|
||||
different `text` / `to_id` / beyond-window `rx_time` stay separate.
|
||||
|
||||
### MD-R1 — Regression: prior acceptance still holds
|
||||
```bash
|
||||
( cd web && bundle exec rspec ) && ( cd web && npm test )
|
||||
( . .venv/bin/activate && pytest -q tests/ )
|
||||
```
|
||||
**Expected:** all green. At risk and required to remain green: **C5** (cross-ingestor
|
||||
dedup by id — now strengthened), the other #756 content-dedup examples (window
|
||||
inclusivity, different text/recipient), and **B1**. The pre-existing "does not
|
||||
collapse two meshcore messages on different channels" example is **updated** to use
|
||||
different channel *names* (the stable identifier) rather than different local
|
||||
indices — it is updated, not removed. No POST/GET/event contract change and no
|
||||
ingestor change, so **C2**, `CONTRACTS.md`, and the Python suite are unaffected.
|
||||
|
||||
@@ -225,23 +225,28 @@ module PotatoMesh
|
||||
# ``db.transaction(:immediate)`` is a future tightening if the race
|
||||
# is ever observed in production.
|
||||
if protocol == "meshcore" && from_id && channel_index && text && !text.to_s.empty?
|
||||
# ``channel = ?`` matches the ``channel_index`` bind cleanly
|
||||
# because the guard above rejects nil; ``to_id`` may legitimately
|
||||
# be nil (rare meshcore fallback), so it keeps ``IS ?`` for a
|
||||
# NULL-safe compare.
|
||||
# Match on the sender-stable ``channel_name`` (NULL-safe ``IS ?``)
|
||||
# rather than the per-receiver ``channel`` slot index. Two
|
||||
# ingestors store the same logical channel at different local
|
||||
# indices (e.g. ``#bot`` at slot 4 on one device, 6 on another), so
|
||||
# keying the dedup on the index lets the same physical transmission
|
||||
# through twice — the reported duplication. The channel *name* is
|
||||
# carried in the message text/contact roster identically across
|
||||
# receivers, so it is the stable discriminator. ``to_id`` is also
|
||||
# ``IS ?`` (rare meshcore nil fallback).
|
||||
duplicate_id = db.get_first_value(
|
||||
<<~SQL,
|
||||
SELECT id FROM messages
|
||||
WHERE protocol = 'meshcore'
|
||||
AND from_id = ?
|
||||
AND to_id IS ?
|
||||
AND channel = ?
|
||||
AND channel_name IS ?
|
||||
AND text = ?
|
||||
AND rx_time BETWEEN ? AND ?
|
||||
AND id != ?
|
||||
LIMIT 1
|
||||
SQL
|
||||
[from_id, to_id, channel_index, text,
|
||||
[from_id, to_id, channel_name, text,
|
||||
rx_time - MESHCORE_CONTENT_DEDUP_WINDOW_SECONDS,
|
||||
rx_time + MESHCORE_CONTENT_DEDUP_WINDOW_SECONDS, msg_id],
|
||||
)
|
||||
|
||||
@@ -1356,15 +1356,39 @@ RSpec.describe PotatoMesh::App::DataProcessing do
|
||||
db&.close
|
||||
end
|
||||
|
||||
it "does not collapse two meshcore messages on different channels" do
|
||||
it "does not collapse two meshcore messages on different named channels" do
|
||||
# Genuinely distinct channels are now distinguished by the stable
|
||||
# channel *name*, not the per-receiver local slot index.
|
||||
db = open_db
|
||||
meshcore_harness.insert_message(db, base_message.merge("id" => 1_000_005, "channel" => 5))
|
||||
meshcore_harness.insert_message(db, base_message.merge("id" => 1_000_006, "channel" => 6))
|
||||
meshcore_harness.insert_message(
|
||||
db, base_message.merge("id" => 1_000_005, "channel" => 5, "channel_name" => "#alpha"),
|
||||
)
|
||||
meshcore_harness.insert_message(
|
||||
db, base_message.merge("id" => 1_000_006, "channel" => 6, "channel_name" => "#beta"),
|
||||
)
|
||||
expect(message_count(db)).to eq(2)
|
||||
ensure
|
||||
db&.close
|
||||
end
|
||||
|
||||
it "collapses the same meshcore channel message heard on different local channel indices" do
|
||||
# One physical #bot transmission heard by two ingestors that store it at
|
||||
# different LOCAL channel slots (4 vs 6) — so each computes a different
|
||||
# fingerprint id. The channel *name* ("#bot") is identical across
|
||||
# receivers, so the content-dedup must collapse it to a single row.
|
||||
# Regression for the cross-ingestor duplication in the bug report.
|
||||
db = open_db
|
||||
meshcore_harness.insert_message(
|
||||
db, base_message.merge("id" => 1_000_201, "channel" => 4, "channel_name" => "#bot"),
|
||||
)
|
||||
meshcore_harness.insert_message(
|
||||
db, base_message.merge("id" => 1_000_202, "channel" => 6, "channel_name" => "#bot"),
|
||||
)
|
||||
expect(message_count(db)).to eq(1)
|
||||
ensure
|
||||
db&.close
|
||||
end
|
||||
|
||||
it "does not collapse two meshcore messages with different text" do
|
||||
db = open_db
|
||||
meshcore_harness.insert_message(db, base_message.merge("id" => 1_000_007, "text" => "first"))
|
||||
|
||||
Reference in New Issue
Block a user