diff --git a/ACCEPTANCE.md b/ACCEPTANCE.md index e746473..4c637d8 100644 --- a/ACCEPTANCE.md +++ b/ACCEPTANCE.md @@ -1608,3 +1608,43 @@ remain green:** - **B1** (all suites). The only contract change is the additive `nodes` publish on message ingest (a new SSE event, documented in `CONTRACTS.md`); no POST/GET shape changes, so **C2** and the Python suite are unaffected. + +--- + +## Bugfix: MeshCore cross-ingestor dedup keys on the stable channel name + +A single physical MeshCore channel message heard by two ingestors that store the +same logical channel at **different local channel-slot indices** was stored twice. +The per-receiver `channel` index is not stable across ingestors (e.g. `#bot` sits +at slot 4 on one device and slot 6 on another), yet it fed both the ingestor +fingerprint discriminator (`c` → two different `messages.id` values) and the +#756 web content-dedup SELECT (`AND channel = ?` → no match), so neither dedup +layer collapsed the duplicate. Fix (web-only, no wire change): the content-dedup +matches on the sender-stable `channel_name` (NULL-safe) instead of the local +`channel` index, so the safety net collapses the duplicate at the system of record +regardless of differing ids/slots. Strengthens **C5**. + +### MD-A1 — Same message on different local channel slots collapses to one row +```bash +( cd web && bundle exec rspec spec/data_processing_spec.rb -e "meshcore content dedup" ) +``` +**Expected:** pass, including "collapses the same meshcore channel message heard on +different local channel indices": two meshcore messages with identical `from_id` / +`to_id` / `text` / in-window `rx_time` and the **same `channel_name`** ("#bot") but +**different `channel` indices** (4 vs 6) and different ids collapse to a **single** +stored row. Companion examples still hold: messages with a **different +`channel_name`** are kept separate (the legitimate distinct-channel case), and +different `text` / `to_id` / beyond-window `rx_time` stay separate. + +### MD-R1 — Regression: prior acceptance still holds +```bash +( cd web && bundle exec rspec ) && ( cd web && npm test ) +( . .venv/bin/activate && pytest -q tests/ ) +``` +**Expected:** all green. At risk and required to remain green: **C5** (cross-ingestor +dedup by id — now strengthened), the other #756 content-dedup examples (window +inclusivity, different text/recipient), and **B1**. The pre-existing "does not +collapse two meshcore messages on different channels" example is **updated** to use +different channel *names* (the stable identifier) rather than different local +indices — it is updated, not removed. No POST/GET/event contract change and no +ingestor change, so **C2**, `CONTRACTS.md`, and the Python suite are unaffected. diff --git a/web/lib/potato_mesh/application/data_processing/messages.rb b/web/lib/potato_mesh/application/data_processing/messages.rb index 67abab6..17782ea 100644 --- a/web/lib/potato_mesh/application/data_processing/messages.rb +++ b/web/lib/potato_mesh/application/data_processing/messages.rb @@ -225,23 +225,28 @@ module PotatoMesh # ``db.transaction(:immediate)`` is a future tightening if the race # is ever observed in production. if protocol == "meshcore" && from_id && channel_index && text && !text.to_s.empty? - # ``channel = ?`` matches the ``channel_index`` bind cleanly - # because the guard above rejects nil; ``to_id`` may legitimately - # be nil (rare meshcore fallback), so it keeps ``IS ?`` for a - # NULL-safe compare. + # Match on the sender-stable ``channel_name`` (NULL-safe ``IS ?``) + # rather than the per-receiver ``channel`` slot index. Two + # ingestors store the same logical channel at different local + # indices (e.g. ``#bot`` at slot 4 on one device, 6 on another), so + # keying the dedup on the index lets the same physical transmission + # through twice — the reported duplication. The channel *name* is + # carried in the message text/contact roster identically across + # receivers, so it is the stable discriminator. ``to_id`` is also + # ``IS ?`` (rare meshcore nil fallback). duplicate_id = db.get_first_value( <<~SQL, SELECT id FROM messages WHERE protocol = 'meshcore' AND from_id = ? AND to_id IS ? - AND channel = ? + AND channel_name IS ? AND text = ? AND rx_time BETWEEN ? AND ? AND id != ? LIMIT 1 SQL - [from_id, to_id, channel_index, text, + [from_id, to_id, channel_name, text, rx_time - MESHCORE_CONTENT_DEDUP_WINDOW_SECONDS, rx_time + MESHCORE_CONTENT_DEDUP_WINDOW_SECONDS, msg_id], ) diff --git a/web/spec/data_processing_spec.rb b/web/spec/data_processing_spec.rb index c961b9e..c495075 100644 --- a/web/spec/data_processing_spec.rb +++ b/web/spec/data_processing_spec.rb @@ -1356,15 +1356,39 @@ RSpec.describe PotatoMesh::App::DataProcessing do db&.close end - it "does not collapse two meshcore messages on different channels" do + it "does not collapse two meshcore messages on different named channels" do + # Genuinely distinct channels are now distinguished by the stable + # channel *name*, not the per-receiver local slot index. db = open_db - meshcore_harness.insert_message(db, base_message.merge("id" => 1_000_005, "channel" => 5)) - meshcore_harness.insert_message(db, base_message.merge("id" => 1_000_006, "channel" => 6)) + meshcore_harness.insert_message( + db, base_message.merge("id" => 1_000_005, "channel" => 5, "channel_name" => "#alpha"), + ) + meshcore_harness.insert_message( + db, base_message.merge("id" => 1_000_006, "channel" => 6, "channel_name" => "#beta"), + ) expect(message_count(db)).to eq(2) ensure db&.close end + it "collapses the same meshcore channel message heard on different local channel indices" do + # One physical #bot transmission heard by two ingestors that store it at + # different LOCAL channel slots (4 vs 6) — so each computes a different + # fingerprint id. The channel *name* ("#bot") is identical across + # receivers, so the content-dedup must collapse it to a single row. + # Regression for the cross-ingestor duplication in the bug report. + db = open_db + meshcore_harness.insert_message( + db, base_message.merge("id" => 1_000_201, "channel" => 4, "channel_name" => "#bot"), + ) + meshcore_harness.insert_message( + db, base_message.merge("id" => 1_000_202, "channel" => 6, "channel_name" => "#bot"), + ) + expect(message_count(db)).to eq(1) + ensure + db&.close + end + it "does not collapse two meshcore messages with different text" do db = open_db meshcore_harness.insert_message(db, base_message.merge("id" => 1_000_007, "text" => "first"))