Pass 1 on PATH integration

app/AGENTS.md

@@ -104,8 +104,10 @@ app/
 - Channel sends use a session-local LRU slot cache after startup channel offload clears the radio. Repeated sends to the same room reuse the loaded slot; new rooms fill free slots up to the discovered channel capacity, then evict the least recently used cached room.
 - TCP radios do not reuse cached slot contents. For TCP, channel sends still force `set_channel(...)` before every send because this backend does not have exclusive device access.
 - `MESHCORE_FORCE_CHANNEL_SLOT_RECONFIGURE=true` disables slot reuse on all transports and forces the old always-`set_channel(...)` behavior before every channel send.
-- Contacts persist `out_path_hash_mode` in the database so contact sync and outbound DM routing reuse the exact stored mode instead of inferring from path bytes.
-- Contacts may also persist `route_override_path`, `route_override_len`, and `route_override_hash_mode`. `Contact.to_radio_dict()` gives these override fields precedence over learned `last_path*`, while advert processing still updates the learned route for telemetry/fallback.
+- Contacts persist canonical direct-route fields (`direct_path`, `direct_path_len`, `direct_path_hash_mode`) so contact sync and outbound DM routing reuse the exact stored hop width instead of inferring from path bytes.
+- Direct-route sources are limited to radio contact sync (`out_path`) and PATH/path-discovery updates. This mirrors firmware `onContactPathRecv(...)`, which replaces `ContactInfo.out_path` when a new returned path is heard.
+- `route_override_path`, `route_override_len`, and `route_override_hash_mode` take precedence over the learned direct route for radio-bound sends.
+- Advertisement paths are stored only in `contact_advert_paths` for analytics/visualization. They are not part of `Contact.to_radio_dict()` or DM route selection.
 - `contact_advert_paths` identity is `(public_key, path_hex, path_len)` because the same hex bytes can represent different routes at different hop widths.
 
 ### Read/unread state
@@ -120,8 +122,10 @@ app/
 - DM ACK tracking is an in-memory pending/buffered map in `services/dm_ack_tracker.py`, with periodic expiry from `radio_sync.py`.
 - Outgoing DMs send once inline, store/broadcast immediately after the first successful `MSG_SENT`, then may retry up to 2 more times in the background if still unacked.
 - DM retry timing follows the firmware-provided `suggested_timeout` from `PACKET_MSG_SENT`; do not replace it with a fixed app timeout unless you intentionally want more aggressive duplicate-prone retries.
-- The final DM retry is intentionally sent as flood via `reset_path(...)`, even when a routing override exists.
+- Direct-message send behavior is intended to emulate `meshcore_py.commands.send_msg_with_retry(...)`: stage the effective contact route on the radio, send, wait for ACK, and on the final retry force flood via `reset_path(...)`.
+- Non-final DM attempts use the contact's effective route (`override > direct > flood`). The final retry is intentionally sent as flood even when a routing override exists.
 - DM ACK state is terminal on first ACK. Retry attempts may register multiple expected ACK codes for the same message, but sibling pending codes are cleared once one ACK wins so a DM should not accrue multiple delivery confirmations from retries.
+- ACKs are delivery state, not routing state. Bundled ACKs inside PATH packets still satisfy pending DM sends, but ACK history does not feed contact route learning.
 
 ### Echo/repeat dedup
 
@@ -258,7 +262,7 @@ Client sends `"ping"` text; server replies `{"type":"pong"}`.
 ## Data Model Notes
 
 Main tables:
-- `contacts` (includes `first_seen` for contact age tracking and `out_path_hash_mode` for route round-tripping)
+- `contacts` (includes `first_seen` for contact age tracking and `direct_path_hash_mode` / `route_override_*` for DM routing)
 - `channels`
   Includes optional `flood_scope_override` for channel-specific regional sends.
 - `messages` (includes `sender_name`, `sender_key` for per-contact channel message attribution)

app/database.py

@@ -13,9 +13,10 @@ CREATE TABLE IF NOT EXISTS contacts (
     name TEXT,
     type INTEGER DEFAULT 0,
     flags INTEGER DEFAULT 0,
-    last_path TEXT,
-    last_path_len INTEGER DEFAULT -1,
-    out_path_hash_mode INTEGER DEFAULT 0,
+    direct_path TEXT,
+    direct_path_len INTEGER,
+    direct_path_hash_mode INTEGER,
+    direct_path_updated_at INTEGER,
     route_override_path TEXT,
     route_override_len INTEGER,
     route_override_hash_mode INTEGER,
@@ -25,7 +26,8 @@ CREATE TABLE IF NOT EXISTS contacts (
     last_seen INTEGER,
     on_radio INTEGER DEFAULT 0,
     last_contacted INTEGER,
-    first_seen INTEGER
+    first_seen INTEGER,
+    last_read_at INTEGER
 );
 
 CREATE TABLE IF NOT EXISTS channels (
@@ -33,7 +35,8 @@ CREATE TABLE IF NOT EXISTS channels (
     name TEXT NOT NULL,
     is_hashtag INTEGER DEFAULT 0,
     on_radio INTEGER DEFAULT 0,
-    flood_scope_override TEXT
+    flood_scope_override TEXT,
+    last_read_at INTEGER
 );
 
 CREATE TABLE IF NOT EXISTS messages (

app/decoder.py

@@ -60,6 +60,19 @@ class DecryptedDirectMessage:
     src_hash: str  # First byte of sender pubkey as hex
 
 
+@dataclass
+class DecryptedPathPayload:
+    """Result of decrypting a PATH payload."""
+
+    dest_hash: str
+    src_hash: str
+    returned_path: bytes
+    returned_path_len: int
+    returned_path_hash_mode: int
+    extra_type: int
+    extra: bytes
+
+
 @dataclass
 class ParsedAdvertisement:
     """Result of parsing an advertisement packet."""
@@ -563,3 +576,88 @@ def try_decrypt_dm(
         return None
 
     return decrypt_direct_message(packet_info.payload, shared_secret)
+
+
+def decrypt_path_payload(payload: bytes, shared_secret: bytes) -> DecryptedPathPayload | None:
+    """Decrypt a PATH payload using the ECDH shared secret."""
+    if len(payload) < 4:
+        return None
+
+    dest_hash = format(payload[0], "02x")
+    src_hash = format(payload[1], "02x")
+    mac = payload[2:4]
+    ciphertext = payload[4:]
+
+    if len(ciphertext) == 0 or len(ciphertext) % 16 != 0:
+        return None
+
+    calculated_mac = hmac.new(shared_secret, ciphertext, hashlib.sha256).digest()[:2]
+    if calculated_mac != mac:
+        return None
+
+    try:
+        cipher = AES.new(shared_secret[:16], AES.MODE_ECB)
+        decrypted = cipher.decrypt(ciphertext)
+    except Exception as e:
+        logger.debug("AES decryption failed for PATH payload: %s", e)
+        return None
+
+    if len(decrypted) < 2:
+        return None
+
+    from app.path_utils import decode_path_byte
+
+    packed_len = decrypted[0]
+    try:
+        returned_path_len, hash_size = decode_path_byte(packed_len)
+    except ValueError:
+        return None
+
+    path_byte_len = returned_path_len * hash_size
+    if len(decrypted) < 1 + path_byte_len + 1:
+        return None
+
+    offset = 1
+    returned_path = decrypted[offset : offset + path_byte_len]
+    offset += path_byte_len
+    extra_type = decrypted[offset] & 0x0F
+    offset += 1
+    extra = decrypted[offset:]
+
+    return DecryptedPathPayload(
+        dest_hash=dest_hash,
+        src_hash=src_hash,
+        returned_path=returned_path,
+        returned_path_len=returned_path_len,
+        returned_path_hash_mode=hash_size - 1,
+        extra_type=extra_type,
+        extra=extra,
+    )
+
+
+def try_decrypt_path(
+    raw_packet: bytes,
+    our_private_key: bytes,
+    their_public_key: bytes,
+    our_public_key: bytes,
+) -> DecryptedPathPayload | None:
+    """Try to decrypt a raw packet as a PATH packet."""
+    packet_info = parse_packet(raw_packet)
+    if packet_info is None or packet_info.payload_type != PayloadType.PATH:
+        return None
+
+    if len(packet_info.payload) < 4:
+        return None
+
+    dest_hash = packet_info.payload[0]
+    src_hash = packet_info.payload[1]
+    if dest_hash != our_public_key[0] or src_hash != their_public_key[0]:
+        return None
+
+    try:
+        shared_secret = derive_shared_secret(our_private_key, their_public_key)
+    except Exception as e:
+        logger.debug("Failed to derive shared secret for PATH payload: %s", e)
+        return None
+
+    return decrypt_path_payload(packet_info.payload, shared_secret)

app/event_handlers.py

@@ -14,11 +14,11 @@ from app.services.contact_reconciliation import (
     promote_prefix_contacts_for_contact,
     record_contact_name_and_reconcile,
 )
+from app.services.dm_ack_apply import apply_dm_ack_code
 from app.services.dm_ingest import (
     ingest_fallback_direct_message,
     resolve_fallback_direct_message_context,
 )
-from app.services.messages import increment_ack_and_broadcast
 from app.websocket import broadcast_event
 
 if TYPE_CHECKING:
@@ -197,11 +197,12 @@ async def on_path_update(event: "Event") -> None:
             )
             normalized_path_hash_mode = None
 
-    await ContactRepository.update_path(
+    await ContactRepository.update_direct_path(
         contact.public_key,
         str(path),
         normalized_path_len,
         normalized_path_hash_mode,
+        updated_at=int(time.time()),
     )
 
 
@@ -268,19 +269,10 @@ async def on_ack(event: "Event") -> None:
         return
 
     logger.debug("Received ACK with code %s", ack_code)
-
-    cleanup_expired_acks()
-
-    message_id = dm_ack_tracker.pop_pending_ack(ack_code)
-    if message_id is not None:
-        dm_ack_tracker.clear_pending_acks_for_message(message_id)
-        logger.info("ACK received for message %d", message_id)
-        # DM ACKs don't carry path data, so paths is intentionally omitted.
-        # The frontend's mergePendingAck handles the missing field correctly,
-        # preserving any previously known paths.
-        await increment_ack_and_broadcast(message_id=message_id, broadcast_fn=broadcast_event)
+    matched = await apply_dm_ack_code(ack_code, broadcast_fn=broadcast_event)
+    if matched:
+        logger.info("ACK received for code %s", ack_code)
     else:
-        dm_ack_tracker.buffer_unmatched_ack(ack_code)
         logger.debug("ACK code %s does not match any pending messages", ack_code)
 
 

app/migrations.py

@@ -346,6 +346,13 @@ async def run_migrations(conn: aiosqlite.Connection) -> int:
         await set_version(conn, 44)
         applied += 1
 
+    # Migration 45: Replace legacy contact route columns with direct-route columns
+    if version < 45:
+        logger.info("Applying migration 45: rebuild contacts direct-route columns")
+        await _migrate_045_rebuild_contacts_direct_route_columns(conn)
+        await set_version(conn, 45)
+        applied += 1
+
     if applied > 0:
         logger.info(
             "Applied %d migration(s), schema now at version %d", applied, await get_version(conn)
@@ -2635,3 +2642,140 @@ async def _migrate_044_dedupe_incoming_direct_messages(conn: aiosqlite.Connectio
            WHERE type = 'PRIV' AND outgoing = 0"""
     )
     await conn.commit()
+
+
+async def _migrate_045_rebuild_contacts_direct_route_columns(conn: aiosqlite.Connection) -> None:
+    """Replace legacy contact route columns with canonical direct-route columns."""
+    cursor = await conn.execute(
+        "SELECT name FROM sqlite_master WHERE type='table' AND name='contacts'"
+    )
+    if await cursor.fetchone() is None:
+        await conn.commit()
+        return
+
+    cursor = await conn.execute("PRAGMA table_info(contacts)")
+    columns = {row[1] for row in await cursor.fetchall()}
+
+    target_columns = {
+        "public_key",
+        "name",
+        "type",
+        "flags",
+        "direct_path",
+        "direct_path_len",
+        "direct_path_hash_mode",
+        "direct_path_updated_at",
+        "route_override_path",
+        "route_override_len",
+        "route_override_hash_mode",
+        "last_advert",
+        "lat",
+        "lon",
+        "last_seen",
+        "on_radio",
+        "last_contacted",
+        "first_seen",
+        "last_read_at",
+    }
+    if (
+        target_columns.issubset(columns)
+        and "last_path" not in columns
+        and "out_path_hash_mode" not in columns
+    ):
+        await conn.commit()
+        return
+
+    await conn.execute(
+        """
+        CREATE TABLE contacts_new (
+            public_key TEXT PRIMARY KEY,
+            name TEXT,
+            type INTEGER DEFAULT 0,
+            flags INTEGER DEFAULT 0,
+            direct_path TEXT,
+            direct_path_len INTEGER,
+            direct_path_hash_mode INTEGER,
+            direct_path_updated_at INTEGER,
+            route_override_path TEXT,
+            route_override_len INTEGER,
+            route_override_hash_mode INTEGER,
+            last_advert INTEGER,
+            lat REAL,
+            lon REAL,
+            last_seen INTEGER,
+            on_radio INTEGER DEFAULT 0,
+            last_contacted INTEGER,
+            first_seen INTEGER,
+            last_read_at INTEGER
+        )
+        """
+    )
+
+    select_expr = {
+        "public_key": "public_key",
+        "name": "NULL",
+        "type": "0",
+        "flags": "0",
+        "direct_path": "NULL",
+        "direct_path_len": "NULL",
+        "direct_path_hash_mode": "NULL",
+        "direct_path_updated_at": "NULL",
+        "route_override_path": "NULL",
+        "route_override_len": "NULL",
+        "route_override_hash_mode": "NULL",
+        "last_advert": "NULL",
+        "lat": "NULL",
+        "lon": "NULL",
+        "last_seen": "NULL",
+        "on_radio": "0",
+        "last_contacted": "NULL",
+        "first_seen": "NULL",
+        "last_read_at": "NULL",
+    }
+    for name in ("name", "type", "flags"):
+        if name in columns:
+            select_expr[name] = name
+
+    if "direct_path" in columns:
+        select_expr["direct_path"] = "direct_path"
+    elif "last_path" in columns:
+        select_expr["direct_path"] = "last_path"
+
+    if "direct_path_len" in columns:
+        select_expr["direct_path_len"] = "direct_path_len"
+    elif "last_path_len" in columns:
+        select_expr["direct_path_len"] = "last_path_len"
+
+    if "direct_path_hash_mode" in columns:
+        select_expr["direct_path_hash_mode"] = "direct_path_hash_mode"
+    elif "out_path_hash_mode" in columns:
+        select_expr["direct_path_hash_mode"] = "out_path_hash_mode"
+
+    for name in (
+        "route_override_path",
+        "route_override_len",
+        "route_override_hash_mode",
+        "last_advert",
+        "lat",
+        "lon",
+        "last_seen",
+        "on_radio",
+        "last_contacted",
+        "first_seen",
+        "last_read_at",
+    ):
+        if name in columns:
+            select_expr[name] = name
+
+    ordered_columns = list(select_expr.keys())
+    await conn.execute(
+        f"""
+        INSERT INTO contacts_new ({", ".join(ordered_columns)})
+        SELECT {", ".join(select_expr[name] for name in ordered_columns)}
+        FROM contacts
+        """
+    )
+
+    await conn.execute("DROP TABLE contacts")
+    await conn.execute("ALTER TABLE contacts_new RENAME TO contacts")
+    await conn.commit()

app/models.py

@@ -1,8 +1,19 @@
-from typing import Literal
+from collections.abc import Mapping
+from typing import Any, Literal
 
-from pydantic import BaseModel, Field
+from pydantic import BaseModel, Field, model_validator
 
-from app.path_utils import normalize_contact_route
+from app.path_utils import normalize_contact_route, normalize_route_override
+
+
+class ContactRoute(BaseModel):
+    """A normalized contact route."""
+
+    path: str = Field(description="Hex-encoded path bytes (empty string for direct/flood)")
+    path_len: int = Field(description="Hop count (-1=flood, 0=direct, >0=explicit route)")
+    path_hash_mode: int = Field(
+        description="Path hash mode (-1=flood, 0=1-byte, 1=2-byte, 2=3-byte hop identifiers)"
+    )
 
 
 class ContactUpsert(BaseModel):
@@ -12,9 +23,10 @@ class ContactUpsert(BaseModel):
     name: str | None = None
     type: int = 0
     flags: int = 0
-    last_path: str | None = None
-    last_path_len: int = -1
-    out_path_hash_mode: int | None = None
+    direct_path: str | None = None
+    direct_path_len: int | None = None
+    direct_path_hash_mode: int | None = None
+    direct_path_updated_at: int | None = None
     route_override_path: str | None = None
     route_override_len: int | None = None
     route_override_hash_mode: int | None = None
@@ -26,6 +38,20 @@ class ContactUpsert(BaseModel):
     last_contacted: int | None = None
     first_seen: int | None = None
 
+    @model_validator(mode="before")
+    @classmethod
+    def _translate_legacy_route_fields(cls, data: Any) -> Any:
+        if not isinstance(data, Mapping):
+            return data
+        translated = dict(data)
+        if "direct_path" not in translated and "last_path" in translated:
+            translated["direct_path"] = translated.get("last_path")
+        if "direct_path_len" not in translated and "last_path_len" in translated:
+            translated["direct_path_len"] = translated.get("last_path_len")
+        if "direct_path_hash_mode" not in translated and "out_path_hash_mode" in translated:
+            translated["direct_path_hash_mode"] = translated.get("out_path_hash_mode")
+        return translated
+
     @classmethod
     def from_contact(cls, contact: "Contact", **changes) -> "ContactUpsert":
         return cls.model_validate(
@@ -40,7 +66,7 @@ class ContactUpsert(BaseModel):
         cls, public_key: str, radio_data: dict, on_radio: bool = False
     ) -> "ContactUpsert":
         """Convert radio contact data to the contact-row write shape."""
-        last_path, last_path_len, out_path_hash_mode = normalize_contact_route(
+        direct_path, direct_path_len, direct_path_hash_mode = normalize_contact_route(
             radio_data.get("out_path"),
             radio_data.get("out_path_len", -1),
             radio_data.get(
@@ -53,9 +79,9 @@ class ContactUpsert(BaseModel):
             name=radio_data.get("adv_name"),
             type=radio_data.get("type", 0),
             flags=radio_data.get("flags", 0),
-            last_path=last_path,
-            last_path_len=last_path_len,
-            out_path_hash_mode=out_path_hash_mode,
+            direct_path=direct_path,
+            direct_path_len=direct_path_len,
+            direct_path_hash_mode=direct_path_hash_mode,
             lat=radio_data.get("adv_lat"),
             lon=radio_data.get("adv_lon"),
             last_advert=radio_data.get("last_advert"),
@@ -68,9 +94,10 @@ class Contact(BaseModel):
     name: str | None = None
     type: int = 0  # 0=unknown, 1=client, 2=repeater, 3=room, 4=sensor
     flags: int = 0
-    last_path: str | None = None
-    last_path_len: int = -1
-    out_path_hash_mode: int = 0
+    direct_path: str | None = None
+    direct_path_len: int = -1
+    direct_path_hash_mode: int = -1
+    direct_path_updated_at: int | None = None
     route_override_path: str | None = None
     route_override_len: int | None = None
     route_override_hash_mode: int | None = None
@@ -82,38 +109,125 @@ class Contact(BaseModel):
     last_contacted: int | None = None  # Last time we sent/received a message
     last_read_at: int | None = None  # Server-side read state tracking
     first_seen: int | None = None
+    effective_route: ContactRoute | None = None
+    effective_route_source: Literal["override", "direct", "flood"] = "flood"
+    direct_route: ContactRoute | None = None
+    route_override: ContactRoute | None = None
+
+    @model_validator(mode="before")
+    @classmethod
+    def _translate_legacy_route_fields(cls, data: Any) -> Any:
+        if not isinstance(data, Mapping):
+            return data
+        translated = dict(data)
+        if "direct_path" not in translated and "last_path" in translated:
+            translated["direct_path"] = translated.get("last_path")
+        if "direct_path_len" not in translated and "last_path_len" in translated:
+            translated["direct_path_len"] = translated.get("last_path_len")
+        if "direct_path_hash_mode" not in translated and "out_path_hash_mode" in translated:
+            translated["direct_path_hash_mode"] = translated.get("out_path_hash_mode")
+        return translated
+
+    def model_post_init(self, __context) -> None:
+        direct_path, direct_path_len, direct_path_hash_mode = normalize_contact_route(
+            self.direct_path,
+            self.direct_path_len,
+            self.direct_path_hash_mode,
+        )
+        self.direct_path = direct_path or None
+        self.direct_path_len = direct_path_len
+        self.direct_path_hash_mode = direct_path_hash_mode
+
+        route_override_path, route_override_len, route_override_hash_mode = (
+            normalize_route_override(
+                self.route_override_path,
+                self.route_override_len,
+                self.route_override_hash_mode,
+            )
+        )
+        self.route_override_path = route_override_path or None
+        self.route_override_len = route_override_len
+        self.route_override_hash_mode = route_override_hash_mode
+        if (
+            route_override_path is not None
+            and route_override_len is not None
+            and route_override_hash_mode is not None
+        ):
+            self.route_override = ContactRoute(
+                path=route_override_path,
+                path_len=route_override_len,
+                path_hash_mode=route_override_hash_mode,
+            )
+        else:
+            self.route_override = None
+
+        if direct_path_len >= 0:
+            self.direct_route = ContactRoute(
+                path=direct_path,
+                path_len=direct_path_len,
+                path_hash_mode=direct_path_hash_mode,
+            )
+        else:
+            self.direct_route = None
+
+        path, path_len, path_hash_mode = self.effective_route_tuple()
+        if self.has_route_override():
+            self.effective_route_source = "override"
+        elif self.direct_route is not None:
+            self.effective_route_source = "direct"
+        else:
+            self.effective_route_source = "flood"
+        self.effective_route = ContactRoute(
+            path=path,
+            path_len=path_len,
+            path_hash_mode=path_hash_mode,
+        )
 
     def has_route_override(self) -> bool:
         return self.route_override_len is not None
 
-    def effective_route(self) -> tuple[str, int, int]:
+    @property
+    def last_path(self) -> str | None:
+        return self.direct_path
+
+    @property
+    def last_path_len(self) -> int:
+        return self.direct_path_len
+
+    @property
+    def out_path_hash_mode(self) -> int:
+        return self.direct_path_hash_mode
+
+    def effective_route_tuple(self) -> tuple[str, int, int]:
         if self.has_route_override():
             return normalize_contact_route(
                 self.route_override_path,
                 self.route_override_len,
                 self.route_override_hash_mode,
             )
-        return normalize_contact_route(
-            self.last_path,
-            self.last_path_len,
-            self.out_path_hash_mode,
-        )
+        if self.direct_path_len >= 0:
+            return normalize_contact_route(
+                self.direct_path,
+                self.direct_path_len,
+                self.direct_path_hash_mode,
+            )
+        return "", -1, -1
 
     def to_radio_dict(self) -> dict:
         """Convert to the dict format expected by meshcore radio commands.
 
         The radio API uses different field names (adv_name, out_path, etc.)
-        than our database schema (name, last_path, etc.).
+        than our database schema (name, direct_path, etc.).
         """
-        last_path, last_path_len, out_path_hash_mode = self.effective_route()
+        effective_path, effective_path_len, effective_path_hash_mode = self.effective_route_tuple()
         return {
             "public_key": self.public_key,
             "adv_name": self.name or "",
             "type": self.type,
             "flags": self.flags,
-            "out_path": last_path,
-            "out_path_len": last_path_len,
-            "out_path_hash_mode": out_path_hash_mode,
+            "out_path": effective_path,
+            "out_path_len": effective_path_len,
+            "out_path_hash_mode": effective_path_hash_mode,
             "adv_lat": self.lat if self.lat is not None else 0.0,
             "adv_lon": self.lon if self.lon is not None else 0.0,
             "last_advert": self.last_advert if self.last_advert is not None else 0,
@@ -149,7 +263,7 @@ class ContactRoutingOverrideRequest(BaseModel):
 
     route: str = Field(
         description=(
-            "Blank clears the override and resets learned routing to flood, "
+            "Blank clears the override, "
             '"-1" forces flood, "0" forces direct, and explicit routes are '
             "comma-separated 1/2/3-byte hop hex values"
         )

app/packet_processor.py

@@ -26,6 +26,7 @@ from app.decoder import (
     parse_packet,
     try_decrypt_dm,
     try_decrypt_packet_with_channel_key,
+    try_decrypt_path,
 )
 from app.keystore import get_private_key, get_public_key, has_private_key
 from app.models import (
@@ -44,6 +45,7 @@ from app.services.contact_reconciliation import (
     promote_prefix_contacts_for_contact,
     record_contact_name_and_reconcile,
 )
+from app.services.dm_ack_apply import apply_dm_ack_code
 from app.services.messages import (
     create_dm_message_from_decrypted as _create_dm_message_from_decrypted,
 )
@@ -318,8 +320,7 @@ async def process_raw_packet(
 
     elif payload_type == PayloadType.ADVERT:
         # Process all advert arrivals (even payload-hash duplicates) so the
-        # path-freshness logic in _process_advertisement can pick the shortest
-        # path heard within the freshness window.
+        # advert-history table retains recent path observations.
         await _process_advertisement(raw_bytes, ts, packet_info)
 
     elif payload_type == PayloadType.TEXT_MESSAGE:
@@ -328,6 +329,9 @@ async def process_raw_packet(
         if decrypt_result:
             result.update(decrypt_result)
 
+    elif payload_type == PayloadType.PATH:
+        await _process_path_packet(raw_bytes, ts, packet_info)
+
     # Always broadcast raw packet for the packet feed UI (even duplicates)
     # This enables the frontend cracker to see all incoming packets in real-time
     broadcast_payload = RawPacketBroadcast(
@@ -430,51 +434,20 @@ async def _process_advertisement(
         logger.debug("Failed to parse advertisement payload")
         return
 
-    # Extract path info from packet
     new_path_len = packet_info.path_length
     new_path_hex = packet_info.path.hex() if packet_info.path else ""
 
     # Try to find existing contact
     existing = await ContactRepository.get_by_key(advert.public_key.lower())
 
-    # Determine which path to use: keep shorter path if heard recently (within 60s)
-    # This handles advertisement echoes through different routes
-    PATH_FRESHNESS_SECONDS = 60
-    use_existing_path = False
-
-    if existing and existing.last_advert:
-        path_age = timestamp - existing.last_advert
-        existing_path_len = existing.last_path_len if existing.last_path_len >= 0 else float("inf")
-
-        # Keep existing path if it's fresh and shorter (or equal)
-        if path_age <= PATH_FRESHNESS_SECONDS and existing_path_len <= new_path_len:
-            use_existing_path = True
-            logger.debug(
-                "Keeping existing shorter path for %s (existing=%d, new=%d, age=%ds)",
-                advert.public_key[:12],
-                existing_path_len,
-                new_path_len,
-                path_age,
-            )
-
-    if use_existing_path:
-        assert existing is not None  # Guaranteed by the conditions that set use_existing_path
-        path_len = existing.last_path_len if existing.last_path_len is not None else -1
-        path_hex = existing.last_path or ""
-        out_path_hash_mode = existing.out_path_hash_mode
-    else:
-        path_len = new_path_len
-        path_hex = new_path_hex
-        out_path_hash_mode = packet_info.path_hash_size - 1
-
     logger.debug(
-        "Parsed advertisement from %s: %s (role=%d, lat=%s, lon=%s, path_len=%d)",
+        "Parsed advertisement from %s: %s (role=%d, lat=%s, lon=%s, advert_path_len=%d)",
         advert.public_key[:12],
         advert.name,
         advert.device_role,
         advert.lat,
         advert.lon,
-        path_len,
+        new_path_len,
     )
 
     # Use device_role from advertisement for contact type (1=Chat, 2=Repeater, 3=Room, 4=Sensor).
@@ -501,9 +474,6 @@ async def _process_advertisement(
         lon=advert.lon,
         last_advert=timestamp,
         last_seen=timestamp,
-        last_path=path_hex,
-        last_path_len=path_len,
-        out_path_hash_mode=out_path_hash_mode,
         first_seen=timestamp,  # COALESCE in upsert preserves existing value
     )
 
@@ -667,3 +637,90 @@ async def _process_direct_message(
     # Couldn't decrypt with any known contact
     logger.debug("Could not decrypt DM with any of %d candidate contacts", len(candidate_contacts))
     return None
+
+
+async def _process_path_packet(
+    raw_bytes: bytes,
+    timestamp: int,
+    packet_info: PacketInfo | None,
+) -> None:
+    """Process a PATH packet and update the learned direct route."""
+    if not has_private_key():
+        return
+
+    private_key = get_private_key()
+    our_public_key = get_public_key()
+    if private_key is None or our_public_key is None:
+        return
+
+    if packet_info is None:
+        packet_info = parse_packet(raw_bytes)
+    if packet_info is None or packet_info.payload is None or len(packet_info.payload) < 4:
+        return
+
+    dest_hash = format(packet_info.payload[0], "02x").lower()
+    src_hash = format(packet_info.payload[1], "02x").lower()
+    our_first_byte = format(our_public_key[0], "02x").lower()
+    if dest_hash != our_first_byte:
+        return
+
+    candidate_contacts = await ContactRepository.get_by_pubkey_first_byte(src_hash)
+    if not candidate_contacts:
+        logger.debug("No contacts found matching hash %s for PATH decryption", src_hash)
+        return
+
+    for contact in candidate_contacts:
+        if len(contact.public_key) != 64:
+            continue
+        try:
+            contact_public_key = bytes.fromhex(contact.public_key)
+        except ValueError:
+            continue
+
+        result = try_decrypt_path(
+            raw_packet=raw_bytes,
+            our_private_key=private_key,
+            their_public_key=contact_public_key,
+            our_public_key=our_public_key,
+        )
+        if result is None:
+            continue
+
+        await ContactRepository.update_direct_path(
+            contact.public_key,
+            result.returned_path.hex(),
+            result.returned_path_len,
+            result.returned_path_hash_mode,
+            updated_at=timestamp,
+        )
+
+        if result.extra_type == PayloadType.ACK and len(result.extra) >= 4:
+            ack_code = result.extra[:4].hex()
+            matched = await apply_dm_ack_code(ack_code, broadcast_fn=broadcast_event)
+            if matched:
+                logger.info(
+                    "Applied bundled PATH ACK for %s via contact %s",
+                    ack_code,
+                    contact.public_key[:12],
+                )
+            else:
+                logger.debug(
+                    "Buffered bundled PATH ACK %s via contact %s",
+                    ack_code,
+                    contact.public_key[:12],
+                )
+        elif result.extra_type == PayloadType.RESPONSE and len(result.extra) > 0:
+            logger.debug(
+                "Observed bundled PATH RESPONSE from %s (%d bytes)",
+                contact.public_key[:12],
+                len(result.extra),
+            )
+
+        refreshed_contact = await ContactRepository.get_by_key(contact.public_key)
+        if refreshed_contact is not None:
+            broadcast_event("contact", refreshed_contact.model_dump())
+        return
+
+    logger.debug(
+        "Could not decrypt PATH packet with any of %d candidate contacts", len(candidate_contacts)
+    )

app/path_utils.py

@@ -153,12 +153,12 @@ def first_hop_hex(path_hex: str, hop_count: int) -> str | None:
 def normalize_contact_route(
     path_hex: str | None,
     path_len: int | None,
-    out_path_hash_mode: int | None,
+    path_hash_mode: int | None,
 ) -> tuple[str, int, int]:
     """Normalize stored contact route fields.
 
     Handles legacy/bad rows where the packed wire path byte was stored directly
-    in `last_path_len` (sometimes as a signed byte, e.g. `-125` for `0x83`).
+    in the hop-count column (sometimes as a signed byte, e.g. `-125` for `0x83`).
     Returns `(path_hex, hop_count, hash_mode)`.
     """
     normalized_path = path_hex or ""
@@ -169,7 +169,7 @@ def normalize_contact_route(
         normalized_len = -1
 
     try:
-        normalized_mode = int(out_path_hash_mode) if out_path_hash_mode is not None else None
+        normalized_mode = int(path_hash_mode) if path_hash_mode is not None else None
     except (TypeError, ValueError):
         normalized_mode = None
 
@@ -207,7 +207,7 @@ def normalize_contact_route(
 def normalize_route_override(
     path_hex: str | None,
     path_len: int | None,
-    out_path_hash_mode: int | None,
+    path_hash_mode: int | None,
 ) -> tuple[str | None, int | None, int | None]:
     """Normalize optional route-override fields while preserving the unset state."""
     if path_len is None:
@@ -216,7 +216,7 @@ def normalize_route_override(
     normalized_path, normalized_len, normalized_mode = normalize_contact_route(
         path_hex,
         path_len,
-        out_path_hash_mode,
+        path_hash_mode,
     )
     return normalized_path, normalized_len, normalized_mode
 

app/radio_sync.py

@@ -46,9 +46,10 @@ def _contact_sync_debug_fields(contact: Contact) -> dict[str, object]:
     return {
         "type": contact.type,
         "flags": contact.flags,
-        "last_path": contact.last_path,
-        "last_path_len": contact.last_path_len,
-        "out_path_hash_mode": contact.out_path_hash_mode,
+        "direct_path": contact.direct_path,
+        "direct_path_len": contact.direct_path_len,
+        "direct_path_hash_mode": contact.direct_path_hash_mode,
+        "direct_path_updated_at": contact.direct_path_updated_at,
         "route_override_path": contact.route_override_path,
         "route_override_len": contact.route_override_len,
         "route_override_hash_mode": contact.route_override_hash_mode,

app/repository/contacts.py

@@ -36,11 +36,20 @@ class ContactRepository:
     @staticmethod
     async def upsert(contact: ContactUpsert | Contact | Mapping[str, Any]) -> None:
         contact_row = ContactRepository._coerce_contact_upsert(contact)
-        last_path, last_path_len, out_path_hash_mode = normalize_contact_route(
-            contact_row.last_path,
-            contact_row.last_path_len,
-            contact_row.out_path_hash_mode,
-        )
+        if (
+            contact_row.direct_path is None
+            and contact_row.direct_path_len is None
+            and contact_row.direct_path_hash_mode is None
+        ):
+            direct_path = None
+            direct_path_len = None
+            direct_path_hash_mode = None
+        else:
+            direct_path, direct_path_len, direct_path_hash_mode = normalize_contact_route(
+                contact_row.direct_path,
+                contact_row.direct_path_len,
+                contact_row.direct_path_hash_mode,
+            )
         route_override_path, route_override_len, route_override_hash_mode = (
             normalize_route_override(
                 contact_row.route_override_path,
@@ -51,20 +60,25 @@ class ContactRepository:
 
         await db.conn.execute(
             """
-            INSERT INTO contacts (public_key, name, type, flags, last_path, last_path_len,
-                                  out_path_hash_mode,
+            INSERT INTO contacts (public_key, name, type, flags, direct_path, direct_path_len,
+                                  direct_path_hash_mode, direct_path_updated_at,
                                   route_override_path, route_override_len,
                                   route_override_hash_mode,
                                   last_advert, lat, lon, last_seen,
                                   on_radio, last_contacted, first_seen)
-            VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+            VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
             ON CONFLICT(public_key) DO UPDATE SET
                 name = COALESCE(excluded.name, contacts.name),
                 type = CASE WHEN excluded.type = 0 THEN contacts.type ELSE excluded.type END,
                 flags = excluded.flags,
-                last_path = COALESCE(excluded.last_path, contacts.last_path),
-                last_path_len = excluded.last_path_len,
-                out_path_hash_mode = excluded.out_path_hash_mode,
+                direct_path = COALESCE(excluded.direct_path, contacts.direct_path),
+                direct_path_len = COALESCE(excluded.direct_path_len, contacts.direct_path_len),
+                direct_path_hash_mode = COALESCE(
+                    excluded.direct_path_hash_mode, contacts.direct_path_hash_mode
+                ),
+                direct_path_updated_at = COALESCE(
+                    excluded.direct_path_updated_at, contacts.direct_path_updated_at
+                ),
                 route_override_path = COALESCE(
                     excluded.route_override_path, contacts.route_override_path
                 ),
@@ -87,9 +101,10 @@ class ContactRepository:
                 contact_row.name,
                 contact_row.type,
                 contact_row.flags,
-                last_path,
-                last_path_len,
-                out_path_hash_mode,
+                direct_path,
+                direct_path_len,
+                direct_path_hash_mode,
+                contact_row.direct_path_updated_at,
                 route_override_path,
                 route_override_len,
                 route_override_hash_mode,
@@ -107,12 +122,12 @@ class ContactRepository:
     @staticmethod
     def _row_to_contact(row) -> Contact:
         """Convert a database row to a Contact model."""
-        last_path, last_path_len, out_path_hash_mode = normalize_contact_route(
-            row["last_path"],
-            row["last_path_len"],
-            row["out_path_hash_mode"],
-        )
         available_columns = set(row.keys())
+        direct_path, direct_path_len, direct_path_hash_mode = normalize_contact_route(
+            row["direct_path"] if "direct_path" in available_columns else None,
+            row["direct_path_len"] if "direct_path_len" in available_columns else None,
+            row["direct_path_hash_mode"] if "direct_path_hash_mode" in available_columns else None,
+        )
         route_override_path = (
             row["route_override_path"] if "route_override_path" in available_columns else None
         )
@@ -136,9 +151,14 @@ class ContactRepository:
             name=row["name"],
             type=row["type"],
             flags=row["flags"],
-            last_path=last_path,
-            last_path_len=last_path_len,
-            out_path_hash_mode=out_path_hash_mode,
+            direct_path=direct_path,
+            direct_path_len=direct_path_len,
+            direct_path_hash_mode=direct_path_hash_mode,
+            direct_path_updated_at=(
+                row["direct_path_updated_at"]
+                if "direct_path_updated_at" in available_columns
+                else None
+            ),
             route_override_path=route_override_path,
             route_override_len=route_override_len,
             route_override_hash_mode=route_override_hash_mode,
@@ -286,42 +306,63 @@ class ContactRepository:
         return [ContactRepository._row_to_contact(row) for row in rows]
 
     @staticmethod
-    async def update_path(
+    async def update_direct_path(
         public_key: str,
         path: str,
         path_len: int,
-        out_path_hash_mode: int | None = None,
+        path_hash_mode: int | None = None,
+        updated_at: int | None = None,
     ) -> None:
         normalized_path, normalized_path_len, normalized_hash_mode = normalize_contact_route(
             path,
             path_len,
-            out_path_hash_mode,
+            path_hash_mode,
         )
+        ts = updated_at if updated_at is not None else int(time.time())
         await db.conn.execute(
-            """UPDATE contacts SET last_path = ?, last_path_len = ?,
-               out_path_hash_mode = COALESCE(?, out_path_hash_mode),
+            """UPDATE contacts SET direct_path = ?, direct_path_len = ?,
+               direct_path_hash_mode = COALESCE(?, direct_path_hash_mode),
+               direct_path_updated_at = ?,
                last_seen = ? WHERE public_key = ?""",
             (
                 normalized_path,
                 normalized_path_len,
                 normalized_hash_mode,
-                int(time.time()),
+                ts,
+                ts,
                 public_key.lower(),
             ),
         )
         await db.conn.commit()
 
+    @staticmethod
+    async def update_path(
+        public_key: str,
+        path: str,
+        path_len: int,
+        path_hash_mode: int | None = None,
+        updated_at: int | None = None,
+    ) -> None:
+        """Compatibility shim for legacy callers/tests."""
+        await ContactRepository.update_direct_path(
+            public_key,
+            path,
+            path_len,
+            path_hash_mode,
+            updated_at=updated_at,
+        )
+
     @staticmethod
     async def set_routing_override(
         public_key: str,
         path: str | None,
         path_len: int | None,
-        out_path_hash_mode: int | None = None,
+        path_hash_mode: int | None = None,
     ) -> None:
         normalized_path, normalized_len, normalized_hash_mode = normalize_route_override(
             path,
             path_len,
-            out_path_hash_mode,
+            path_hash_mode,
         )
         await db.conn.execute(
             """

app/routers/contacts.py

@@ -304,7 +304,6 @@ async def create_contact(
     contact_upsert = ContactUpsert(
         public_key=lower_key,
         name=request.name,
-        out_path_hash_mode=-1,
         on_radio=False,
     )
     await ContactRepository.upsert(contact_upsert)
@@ -474,7 +473,7 @@ async def request_path_discovery(public_key: str) -> PathDiscoveryResponse:
         return_len = int(payload.get("in_path_len") or 0)
         return_mode = _path_hash_mode_from_hop_width(payload.get("in_path_hash_len"))
 
-        await ContactRepository.update_path(
+        await ContactRepository.update_direct_path(
             contact.public_key,
             forward_path,
             forward_len,
@@ -524,9 +523,8 @@ async def set_contact_routing_override(
     route_text = request.route.strip()
     if route_text == "":
         await ContactRepository.clear_routing_override(contact.public_key)
-        await ContactRepository.update_path(contact.public_key, "", -1, -1)
         logger.info(
-            "Cleared routing override and reset learned path to flood for %s",
+            "Cleared routing override for %s",
             contact.public_key[:12],
         )
     elif route_text == "-1":

app/services/dm_ack_apply.py

@@ -0,0 +1,26 @@
+"""Shared direct-message ACK application logic."""
+
+from collections.abc import Callable
+from typing import Any
+
+from app.services import dm_ack_tracker
+from app.services.messages import increment_ack_and_broadcast
+
+BroadcastFn = Callable[..., Any]
+
+
+async def apply_dm_ack_code(ack_code: str, *, broadcast_fn: BroadcastFn) -> bool:
+    """Apply a DM ACK code using the shared pending/buffered state machine.
+
+    Returns True when the ACK matched a pending message, False when it was buffered.
+    """
+    dm_ack_tracker.cleanup_expired_acks()
+
+    message_id = dm_ack_tracker.pop_pending_ack(ack_code)
+    if message_id is None:
+        dm_ack_tracker.buffer_unmatched_ack(ack_code)
+        return False
+
+    dm_ack_tracker.clear_pending_acks_for_message(message_id)
+    await increment_ack_and_broadcast(message_id=message_id, broadcast_fn=broadcast_fn)
+    return True

app/services/dm_ingest.py

@@ -92,7 +92,6 @@ async def resolve_fallback_direct_message_context(
             last_contacted=received_at,
             first_seen=received_at,
             on_radio=False,
-            out_path_hash_mode=-1,
         )
         await contact_repository.upsert(placeholder_upsert)
         contact = await contact_repository.get_by_key(normalized_sender)