updatge readme

* Update CONTRIBUTING.md

* Update CONTRIBUTING.md

* Bound /api/coverage parameters and run the ITM model off the event loop

The coverage endpoint took radius_km, step_km and the other model inputs
straight from the query string with only a float() conversion. The ITM
loop advances by step_km until it passes radius_km, so step_km=0 (or a
negative value) never ends and radius_km=inf never ends either; a very
small step with a large radius means millions of model calls. All of this
ran synchronously inside the request handler, so one such request stalls
every other request the server is handling.

- Reject non-finite values and anything outside a sane range for each
  parameter (400 with a message saying the allowed range).
- Cap radius_km / step_km at 4000 steps.
- Run compute_perimeter / compute_coverage in a worker thread so the event
  loop keeps serving other requests while the model runs.
- Make the loops themselves refuse a non-positive step or non-finite radius,
  so a future caller cannot reintroduce the hang, and shrink the result
  cache (64 entries) since bounded inputs make large results possible.

Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Pablo Revilla <pablorevilla@gmail.com>
Co-authored-by: Nathan <nathan@stonekitty.net>
Co-authored-by: Claude Mythos 5.1 <noreply@anthropic.com>

.gitignore

@@ -22,6 +22,7 @@ meshview-web.pid
 /table_details.py
 config.ini
 *.log
+*.status.json
 
 # Screenshots
 screenshots/*

CONTRIBUTING.md

@@ -52,7 +52,7 @@ Unacceptable behavior includes harassment, insults, hate speech, personal attack
 
 ---
 
-## I Want to Contribute
+## I Want to Contribute!
 
 ### Legal Notice
 By contributing to Meshview, you agree that:
@@ -79,8 +79,6 @@ How to report:
 - Use a **clear and descriptive title**.  
 - Include reproduction steps and expected vs. actual behavior.  
 
-⚠️ Security issues should **not** be reported in public issues. Instead, email us at **meshview-maintainers@proton.me**.  
-
 ---
 
 ### Suggesting Enhancements
@@ -103,12 +101,11 @@ When submitting:
 We love first-time contributors! 🚀
 
 If you’d like to start coding:  
-1. Look for issues tagged with [good first issue](../../issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22).  
-2. Fork the repository and clone it locally.  
-3. Set up the development environment:  
-4. Run the app locally
-5. Create a new branch, make your changes, commit, and push.  
-6. Open a pull request!  
+1. Fork the repository and clone it locally.  
+2. Set up the development environment:  
+3. Run the app locally
+4. Create a new branch from `develop`, make your changes, commit, and push.  
+5. Open a pull request against the `develop` branch.  
 
 ---
 
@@ -130,4 +127,4 @@ Start by contributing regularly, engaging in issues/PRs, and helping others.
 
 ---
 
-✨ That’s it! Thanks again for being part of Meshview. Every contribution matters.  
+✨ That’s it! Thanks again for being part of Meshview. Every contribution matters.  

README.md

@@ -4,8 +4,35 @@
 
 The project serves as a real-time monitoring and diagnostic tool for the Meshtastic mesh network. It provides detailed insights into network activity, including message traffic, node positions, and telemetry data.
 
+### Version 3.0.9 — October 2026
+
+- Live map: added animated packet traces from senders to reporting gateways and configurable channel colors through the optional `[map_channel_colors]` section in `config.ini`.
+- Daily snapshots: new snapshots report activity for the previous complete UTC day, including newly imported packets, unique packet senders observed that day, and gateways that reported packets that day. Existing historical snapshots keep their original counts.
+- Node tracking: record nodes from packet activity before their node information arrives, and preserve known positions when map reports have missing or invalid coordinates.
+- Neighbor SNR chart: show the last seven days and the 15 neighbors with the most data points, distinguish neighbors with identical short names, and fix CSV export.
+- MQTT reliability: skip malformed messages and invalid node IDs without stopping packet ingestion.
+- Security and privacy: escape mesh-provided text in web pages, restrict language and static file requests to their intended directories, and prevent payload searches from matching redacted direct messages.
+- Coverage API: validate and bound request parameters, and run coverage calculations off the event loop to keep the web server responsive.
+
+### Version 3.0.8 — September 2026
+- Map updates: added OpenTopoMap as a selectable base map, improved map controls, and fixed several map/node display edge cases.
+- Traceroute: added a dedicated traceroute page, corrected route path orientation in graph edges, and improved traceroute API/documentation support.
+- Routing display: show ROUTING ACK/NAK packets clearly instead of reporting misleading `error_reason: NONE` values.
+- MQTT/decryption: expanded Meshtastic one-byte PSK shorthand support in `mqtt.secondary_keys`.
+- Health/maintenance: added cleanup status reporting to the health page and fixed timezone handling in health reporting.
+- UI/API fixes: improved Firehose, Node, and Top channel handling, including showing all channels where expected.
+- Contributors: thanks to [giannoug](https://github.com/giannoug) for fixes to ROUTING ACK/NAK display, `mqtt.secondary_keys` PSK shorthand handling, and traceroute path orientation, plus jschrempp and the Meshview community for testing and feedback included in this release.
+
+### Version 3.0.7 — May 2026
+- Reliability report: added a dedicated reliability page for checking how well a selected node is heard across the mesh, including heard/missed status, packet counts, hop counts, gateway tables, and a map view.
+- Node list/map UX: added node list filtering for wider active-node ranges, included all database nodes where appropriate, and made map node colors persistent across reboots.
+- QR/API fixes: corrected QR code generation when a node role is missing or unavailable.
+- Database cleanup: removed stored node public key data and added the Alembic migration to drop `node.public_key`.
+- Protobufs: refreshed  Meshtastic protobuf Python modules to upstream commit 
+- Tooling: improved the protobuf updater so it supports raw commit SHAs, the newer upstream `meshtastic/*.proto` layout, generated `.pyi` stubs, and the vendored `nanopb`/`serial_hal` outputs.
+
 ### Version 3.0.6 — March 2026
-- Daily snapshots: added the `daily_snapshot` table, scheduled snapshot capture, and `/api/snapshots/daily` for historical node, packet, and gateway counts.
+- Daily snapshots: added the `daily_snapshot` table, scheduled snapshot capture, and `/api/snapshots/daily` for historical node, packet, and gateway counts. These originally stored database totals; version 3.0.9 changes new snapshots to daily activity counts.
 - Stats/UI: expanded stats views with snapshot data and added channel filters to Chat and Firehose.
 - Map/node fixes: corrected node location handling, map popup behavior, packet page display, and node list sorting.
 - Ingestion/API reliability: improved duplicate `PacketSeen` handling, guarded invalid node info packets, and made traceroute/edge APIs more tolerant of decode failures.

docs/API_Documentation.md

@@ -203,6 +203,12 @@ Returns network edges (connections between nodes) based on traceroutes and neigh
 Traceroute edges are collected over the last 12 hours. Neighbor edges are based on
 port 71 packets.
 
+A completed traceroute is observed as a response packet, whose `from`/`to` are reversed
+with respect to the traced path, so edges are built as `to -> route[] -> from`. `route[]`
+may contain `0xFFFFFFFF` placeholders for hops the mesh could not identify; the edges on
+either side of such a placeholder are omitted, because the nodes around an unknown hop
+are not neighbours.
+
 Query Parameters
 - `type` (optional, string): `traceroute` or `neighbor`. If omitted, returns both.
 - `node_id` (optional, int): Filter edges to only those touching a node.
@@ -221,9 +227,37 @@ Response Example
 
 ## 6. Config API
 
+### GET `/api/snapshots/daily`
+
+Returns stored daily snapshots. For new snapshots, `packet_count` is the number of
+packet records imported during `date` (00:00 UTC inclusive to the next midnight
+exclusive). The scheduled capture records the previous complete UTC day. Repeated
+observations of the same packet are not counted separately. `node_count` counts distinct
+packet senders whose packets were imported or observed during that UTC day (including
+observations of previously imported packets). `gateway_count` counts distinct MQTT
+gateways with packet observations imported during that day. These are activity counts,
+not database totals; recipients alone do not count as active nodes. Activity uses local
+import timestamps, not device timestamps. Older snapshots retain their originally stored
+counts and are not automatically converted. Counts depend on the packet and observation
+history still retained at capture time.
+
 ### GET `/api/config`
 Returns a safe subset of server configuration.
 
+`site.map_channel_colors` contains channel color overrides from the optional
+`[map_channel_colors]` INI section. Each entry is `channel = #RRGGBB`; invalid colors
+are omitted. Channel matching on the map ignores case, spaces, hyphens, and underscores.
+Unlisted channels use generated colors. For example:
+
+```ini
+[map_channel_colors]
+LongFast = #4363d8
+LocalChat = #18aa18
+```
+
+Restart the web server and reload the map after changing these settings. Omitting
+the section uses generated colors for every channel.
+
 Response Example
 ```json
 {
@@ -324,13 +358,30 @@ Returns traceroute details and derived paths for a packet.
 Path Parameters
 - `packet_id` (required, int): Packet ID.
 
+A completed traceroute is observed as a response packet, whose `from`/`to` are reversed
+with respect to the traced path. `initiator` and `target` resolve this: the forward path
+runs `initiator -> forward_hops -> target` and the return path runs
+`target -> reverse_hops -> initiator`, in both directions of travel. `forward_hops` and
+`reverse_hops` contain only the intermediate hops, never the endpoints.
+
+A gateway usually reports a response that is still travelling home, so `reverse_hops` is
+often incomplete. `reverse_complete` says whether that observation had reached the
+initiator; only then does the corresponding `winning_paths.reverse` entry end at the
+initiator.
+
+`winning_paths` is built from response observations, since a response proves the request
+reached the target. `complete` on a `unique_forward_paths` entry says whether some
+response reported that hop list, and so whether it is known to end at the target.
+
 Response Example
 ```json
 {
   "packet": {
     "id": 123,
-    "from": 111,
-    "to": 222,
+    "from": 222,
+    "to": 111,
+    "initiator": 111,
+    "target": 222,
     "channel": "main"
   },
   "traceroute_packets": [
@@ -338,19 +389,21 @@ Response Example
       "index": 0,
       "gateway_node_id": 333,
       "done": true,
-      "forward_hops": [111, 444, 222],
-      "reverse_hops": [222, 444, 111]
+      "forward_hops": [444, 555],
+      "reverse_hops": [555, 444],
+      "reverse_complete": true
     }
   ],
   "unique_forward_paths": [
-    { "path": [111, 444, 222], "count": 2 }
+    { "path": [444, 555], "count": 2, "complete": true }
   ],
   "unique_reverse_paths": [
-    [222, 444, 111]
+    [555, 444]
   ],
-  "winning_paths": [
-    [111, 444, 222]
-  ]
+  "winning_paths": {
+    "forward": [[111, 444, 555, 222]],
+    "reverse": [[222, 555, 444, 111]]
+  }
 }
 ```
 
@@ -368,6 +421,49 @@ Response Example
   "timestamp": "2025-07-22T12:45:00+00:00",
   "version": "3.0.3",
   "git_revision": "abc1234",
+  "cleanup": {
+    "enabled": true,
+    "days_to_keep": 14,
+    "scheduled_time": "02:00",
+    "vacuum": false,
+    "status": "ok",
+    "status_file": "dbcleanup.status.json",
+    "last_run": {
+      "status": "ok",
+      "started_at": "2026-06-04T09:00:00+00:00",
+      "completed_at": "2026-06-04T09:00:03+00:00",
+      "cutoff_at": "2026-05-21T09:00:00+00:00",
+      "days_to_keep": 14,
+      "vacuum_requested": false,
+      "vacuum_completed": false,
+      "rows_deleted": {
+        "packet": 1200,
+        "packet_seen": 3400,
+        "traceroute": 42,
+        "node": 3
+      },
+      "error": null
+    }
+  },
+  "backup": {
+    "enabled": true,
+    "backup_dir": "./backups",
+    "scheduled_time": "02:00",
+    "status": "ok",
+    "status_file": "dbbackup.status.json",
+    "last_run": {
+      "status": "ok",
+      "started_at": "2026-06-04T09:00:00+00:00",
+      "completed_at": "2026-06-04T09:00:02+00:00",
+      "backup_dir": "./backups",
+      "database_path": "packets.db",
+      "backup_file": "backups/packets_backup_20260604_090000.db.gz",
+      "original_size_bytes": 12939444,
+      "compressed_size_bytes": 4211560,
+      "compression_percent": 67.5,
+      "error": null
+    }
+  },
   "database": "connected",
   "database_size": "12.34 MB",
   "database_size_bytes": 12939444

meshview/__version__.py

@@ -3,7 +3,7 @@
 import subprocess
 from pathlib import Path
 
-__version__ = "3.0.7"
+__version__ = "3.0.9"
 __release_date__ = "2026-5-12"
 
 

meshview/lang/en.json

@@ -159,6 +159,7 @@
       "times_seen": "Seen (24h)",
       "avg_gateways": "Avg Gateways",
       "showing_nodes": "Showing",
+      "all_channels": "All Channels",
       "nodes_suffix": "nodes"
     },
 
@@ -220,6 +221,8 @@
         "air_channel": "Air & Channel Utilization",
         "environment": "Environment Metrics",
         "neighbors_chart": "Neighbors (Signal-to-Noise)",
+        "neighbors_chart_subtitle": "Last {days} days — {n} neighbors",
+        "neighbors_chart_subtitle_capped": "Last {days} days — the {n} neighbors with the most data points, of {total}",
         "expand": "Expand",
         "export_csv": "Export CSV",
         "time": "Time",

meshview/lang/es.json

@@ -155,6 +155,7 @@
     "times_seen": "Visto (24h)",
     "avg_gateways": "Promedio de Gateways",
     "showing_nodes": "Mostrando",
+    "all_channels": "Todos los Canales",
     "nodes_suffix": "nodos"
   },
 
@@ -206,6 +207,8 @@
     "air_channel": "Utilización del aire y del canal",
     "environment": "Métricas Ambientales",
     "neighbors_chart": "Vecinos (Relación Señal/Ruido)",
+    "neighbors_chart_subtitle": "Últimos {days} días — {n} vecinos",
+    "neighbors_chart_subtitle_capped": "Últimos {days} días — los {n} vecinos con más datos, de {total}",
     "expand": "Ampliar",
     "export_csv": "Exportar CSV",
     "time": "Hora",

meshview/lang/ru.json

@@ -145,6 +145,7 @@
         "times_seen": "Принято (24ч)",
         "avg_gateways": "Среднее количество шлюзов",
         "showing_nodes": "Показать",
+        "all_channels": "Все каналы",
         "nodes_suffix": "ноды"
     },
     "nodegraph": {
@@ -201,6 +202,8 @@
         "air_channel": "Использование сети и каналов",
         "environment": "Метрики среды",
         "neighbors_chart": "Соседи (Соотношение сигнал/шум)",
+        "neighbors_chart_subtitle": "Последние {days} дн. — {n} соседей",
+        "neighbors_chart_subtitle_capped": "Последние {days} дн. — {n} соседей с наибольшим числом точек, из {total}",
         "expand": "Расширить",
         "export_csv": "Экспорт в CSV",
         "time": "Время",

meshview/mqtt_reader.py

@@ -12,7 +12,10 @@ from meshtastic.protobuf.mesh_pb2 import Data
 from meshtastic.protobuf.mqtt_pb2 import ServiceEnvelope
 from meshview.config import CONFIG
 
-PRIMARY_KEY = base64.b64decode("1PG7OiApB1nwvP+rz05pAQ==")
+# Meshtastic's well-known default PSK, i.e. the expansion of the single byte PSK index 1.
+DEFAULT_PSK = base64.b64decode("1PG7OiApB1nwvP+rz05pAQ==")
+PRIMARY_KEY = DEFAULT_PSK
+VALID_KEY_LENGTHS = (16, 32)
 
 logging.basicConfig(
     level=logging.INFO,
@@ -52,6 +55,10 @@ def _strip_quotes(value):
     return value
 
 
+def _expand_short_psk(index):
+    return DEFAULT_PSK[:-1] + bytes([(DEFAULT_PSK[-1] + index - 1) % 256])
+
+
 def _parse_secondary_keys():
     mqtt_config = CONFIG.get("mqtt", {})
     raw_value = mqtt_config.get("secondary_keys", "")
@@ -68,12 +75,33 @@ def _parse_secondary_keys():
 
     keys = []
     for value in values:
+        cleaned = _strip_quotes(str(value).strip())
+        if not cleaned:
+            continue
+
         try:
-            cleaned = _strip_quotes(str(value).strip())
-            if cleaned:
-                keys.append(base64.b64decode(cleaned))
+            key = base64.b64decode(cleaned)
         except (TypeError, ValueError):
             logger.warning("Invalid base64 key in mqtt.secondary_keys: %s", value)
+            continue
+
+        if len(key) == 1:
+            if key[0] == 0:
+                logger.warning(
+                    "Skipping PSK %s in mqtt.secondary_keys: index 0 means no encryption", value
+                )
+                continue
+            key = _expand_short_psk(key[0])
+
+        if len(key) not in VALID_KEY_LENGTHS:
+            logger.error(
+                "Skipping key %s in mqtt.secondary_keys: decoded to %d bytes, expected 16 or 32",
+                value,
+                len(key),
+            )
+            continue
+
+        keys.append(key)
     return keys
 
 

meshview/mqtt_store.py

@@ -1,7 +1,7 @@
 import logging
 import re
 import time
-from datetime import UTC, datetime
+from datetime import UTC, datetime, timedelta
 
 from sqlalchemy import func, select, update
 from sqlalchemy.dialects.postgresql import insert as pg_insert
@@ -17,24 +17,139 @@ from meshview.models import DailySnapshot, Node, Packet, PacketSeen, Traceroute
 logger = logging.getLogger(__name__)
 
 MQTT_GATEWAY_CACHE: set[int] = set()
+UNKNOWN_NODE_NAME = "unknown name"
+NODE_NAME_CONTROL_CHARS = re.compile(r"[\x00-\x1f\x7f]")
+MQTT_DIRECT_NODE_ID = 1
+# Meshtastic node ids are "!" + up to 8 hex digits; be a little lenient on length but the
+# result must still fit the BigInteger columns.
+NODE_ID_PATTERN = re.compile(r"^!([0-9a-fA-F]{1,16})$")
+MAX_NODE_ID = 2**63 - 1
+
+
+def parse_node_id(value: str | None) -> int | None:
+    """Turn a "!deadbeef" style id into an int, or None when it is not one.
+
+    Gateway and user ids arrive over MQTT and are not trustworthy; a malformed one must
+    not raise, otherwise the whole ingest loop goes down with it.
+    """
+    if not value:
+        return None
+    match = NODE_ID_PATTERN.match(value)
+    if not match:
+        return None
+    node_id = int(match.group(1), 16)
+    if node_id > MAX_NODE_ID:
+        return None
+    return node_id
+
+
+def has_valid_location(lat: int | None, lon: int | None) -> bool:
+    return lat is not None and lon is not None and abs(lat) > 1 and abs(lon) > 1
+
+
+def node_user_id(node_id: int) -> str:
+    return f"!{node_id:0{8}x}"
+
+
+def placeholder_short_name(node_id: int) -> str:
+    return node_user_id(node_id)[-4:]
+
+
+def normalize_node_name(name: str) -> str:
+    name = name.strip()
+    if not name or NODE_NAME_CONTROL_CHARS.search(name):
+        return UNKNOWN_NODE_NAME
+    return name
+
+
+def storage_packet_id(packet) -> int | None:
+    if packet.id:
+        return packet.id
+
+    if packet.decoded.portnum != PortNum.MAP_REPORT_APP:
+        return None
+
+    from_node_id = getattr(packet, "from", 0) or 0
+    now_us = int(time.time() * 1_000_000)
+    return -(now_us * 2048 + (from_node_id & 0x7FF))
+
+
+def storage_to_node_id(packet) -> int:
+    if packet.decoded.portnum == PortNum.MAP_REPORT_APP:
+        return MQTT_DIRECT_NODE_ID
+    return packet.to
+
+
+async def ensure_node_exists(session, node_id: int | None, channel: str | None) -> Node | None:
+    if node_id is None:
+        return None
+
+    node = (await session.execute(select(Node).where(Node.node_id == node_id))).scalar_one_or_none()
+    now_us = int(time.time() * 1_000_000)
+
+    if node:
+        node.last_seen_us = now_us
+        if node.first_seen_us is None:
+            node.first_seen_us = now_us
+        if channel and not node.channel:
+            node.channel = channel
+        return node
+
+    display_name = str(node_id)
+    node = Node(
+        id=node_user_id(node_id),
+        node_id=node_id,
+        long_name=display_name,
+        short_name=placeholder_short_name(node_id),
+        channel=channel,
+        first_seen_us=now_us,
+        last_seen_us=now_us,
+    )
+    session.add(node)
+    return node
 
 
 async def capture_daily_snapshot() -> None:
-    today = datetime.now(UTC).date()
+    """Record packet traffic and active nodes/gateways for the previous UTC day."""
+    day_end = datetime.now(UTC).replace(hour=0, minute=0, second=0, microsecond=0)
+    day_start = day_end - timedelta(days=1)
+    start_us = int(day_start.timestamp() * 1_000_000)
+    end_us = int(day_end.timestamp() * 1_000_000)
+    snapshot_date = day_start.date()
 
     async with mqtt_database.async_session() as session:
-        node_count = (await session.execute(select(func.count()).select_from(Node))).scalar_one()
+        # Include repeated observations of older packets as activity, too. Node.last_seen_us
+        # cannot answer this after a node has already been seen on the following day.
+        active_nodes = (
+            select(Packet.from_node_id)
+            .where(Packet.import_time_us >= start_us, Packet.import_time_us < end_us)
+            .union(
+                select(Packet.from_node_id)
+                .join(PacketSeen, PacketSeen.packet_id == Packet.id)
+                .where(PacketSeen.import_time_us >= start_us, PacketSeen.import_time_us < end_us)
+            )
+            .subquery()
+        )
+        node_count = (
+            await session.execute(select(func.count(active_nodes.c.from_node_id)))
+        ).scalar_one()
         packet_count = (
-            await session.execute(select(func.count()).select_from(Packet))
+            await session.execute(
+                select(func.count())
+                .select_from(Packet)
+                .where(Packet.import_time_us >= start_us, Packet.import_time_us < end_us)
+            )
         ).scalar_one()
         gateway_count = (
             await session.execute(
-                select(func.count()).select_from(Node).where(Node.is_mqtt_gateway.is_(True))
+                select(func.count(func.distinct(PacketSeen.node_id))).where(
+                    PacketSeen.import_time_us >= start_us, PacketSeen.import_time_us < end_us
+                )
             )
         ).scalar_one()
         captured_at_us = int(time.time() * 1_000_000)
         values = {
-            "snapshot_date": today,
+            "snapshot_date": snapshot_date,
             "node_count": node_count,
             "packet_count": packet_count,
             "gateway_count": gateway_count,
@@ -61,7 +176,7 @@ async def capture_daily_snapshot() -> None:
         else:
             snapshot = (
                 await session.execute(
-                    select(DailySnapshot).where(DailySnapshot.snapshot_date == today)
+                    select(DailySnapshot).where(DailySnapshot.snapshot_date == snapshot_date)
                 )
             ).scalar_one_or_none()
             if snapshot is None:
@@ -102,6 +217,7 @@ async def process_envelope(topic, env):
                 ).scalar_one_or_none()
 
                 now_us = int(time.time() * 1_000_000)
+                has_location = has_valid_location(map_report.latitude_i, map_report.longitude_i)
 
                 if node:
                     node.node_id = node_id
@@ -109,9 +225,11 @@ async def process_envelope(topic, env):
                     node.short_name = map_report.short_name
                     node.hw_model = hw_model
                     node.role = role
-                    node.channel = env.channel_id
-                    node.last_lat = map_report.latitude_i
-                    node.last_long = map_report.longitude_i
+                    if not node.channel:
+                        node.channel = env.channel_id
+                    if has_location:
+                        node.last_lat = map_report.latitude_i
+                        node.last_long = map_report.longitude_i
                     node.firmware = map_report.firmware_version
                     node.last_seen_us = now_us
                     if node.first_seen_us is None:
@@ -126,31 +244,35 @@ async def process_envelope(topic, env):
                         role=role,
                         channel=env.channel_id,
                         firmware=map_report.firmware_version,
-                        last_lat=map_report.latitude_i,
-                        last_long=map_report.longitude_i,
                         first_seen_us=now_us,
                         last_seen_us=now_us,
                     )
+                    if has_location:
+                        node.last_lat = map_report.latitude_i
+                        node.last_long = map_report.longitude_i
                     session.add(node)
             except Exception as e:
                 print(f"Error processing MAP_REPORT_APP: {e}")
 
             await session.commit()
 
-    if not env.packet.id:
+    packet_id = storage_packet_id(env.packet)
+    if packet_id is None:
         return
 
     async with mqtt_database.async_session() as session:
         # --- Packet insert with ON CONFLICT DO NOTHING
-        result = await session.execute(select(Packet).where(Packet.id == env.packet.id))
+        from_node_id = getattr(env.packet, "from", None)
+
+        result = await session.execute(select(Packet).where(Packet.id == packet_id))
         packet = result.scalar_one_or_none()
         if not packet:
             now_us = int(time.time() * 1_000_000)
             packet_values = {
-                "id": env.packet.id,
+                "id": packet_id,
                 "portnum": env.packet.decoded.portnum,
-                "from_node_id": getattr(env.packet, "from"),
-                "to_node_id": env.packet.to,
+                "from_node_id": from_node_id,
+                "to_node_id": storage_to_node_id(env.packet),
                 "payload": env.packet.SerializeToString(),
                 "import_time_us": now_us,
                 "channel": env.channel_id,
@@ -183,23 +305,25 @@ async def process_envelope(topic, env):
 
         # --- PacketSeen insert with conflict-safe handling
 
-        if not env.gateway_id:
-            print("WARNING: Missing gateway_id, skipping PacketSeen entry")
+        gateway_node_id = parse_node_id(env.gateway_id)
+        if gateway_node_id is None:
             # Most likely a misconfiguration of a mqtt publisher?
+            logger.warning(
+                "Skipping PacketSeen entry: missing or malformed gateway_id %r", env.gateway_id
+            )
+            await session.commit()
             return
-        else:
-            node_id = int(env.gateway_id[1:], 16)
 
-        if node_id not in MQTT_GATEWAY_CACHE:
-            MQTT_GATEWAY_CACHE.add(node_id)
+        if gateway_node_id not in MQTT_GATEWAY_CACHE:
+            MQTT_GATEWAY_CACHE.add(gateway_node_id)
             await session.execute(
-                update(Node).where(Node.node_id == node_id).values(is_mqtt_gateway=True)
+                update(Node).where(Node.node_id == gateway_node_id).values(is_mqtt_gateway=True)
             )
 
         now_us = int(time.time() * 1_000_000)
         seen_values = {
-            "packet_id": env.packet.id,
-            "node_id": int(env.gateway_id[1:], 16),
+            "packet_id": packet_id,
+            "node_id": gateway_node_id,
             "channel": env.channel_id,
             "rx_time": env.packet.rx_time,
             "rx_snr": env.packet.rx_snr,
@@ -241,9 +365,8 @@ async def process_envelope(topic, env):
                     PortNum.NODEINFO_APP, env.packet.decoded.payload
                 )
                 if user and user.id:
-                    if user.id[0] == "!" and re.fullmatch(r"[0-9a-fA-F]+", user.id[1:]):
-                        node_id = int(user.id[1:], 16)
-                    else:
+                    node_id = parse_node_id(user.id)
+                    if node_id is None:
                         logger.warning(
                             "Skipping NODEINFO_APP packet %s: unable to determine node_id",
                             env.packet.id,
@@ -265,13 +388,22 @@ async def process_envelope(topic, env):
                     node = (
                         await session.execute(select(Node).where(Node.id == user.id))
                     ).scalar_one_or_none()
+                    if node is None:
+                        # The same node may already be stored under a differently
+                        # spelled id (e.g. "!00abcd" vs "!abcd"); node_id is unique, so
+                        # update that row instead of inserting a duplicate.
+                        node = (
+                            await session.execute(select(Node).where(Node.node_id == node_id))
+                        ).scalar_one_or_none()
 
                     now_us = int(time.time() * 1_000_000)
+                    long_name = normalize_node_name(user.long_name)
+                    short_name = normalize_node_name(user.short_name)
 
                     if node:
                         node.node_id = node_id
-                        node.long_name = user.long_name
-                        node.short_name = user.short_name
+                        node.long_name = long_name
+                        node.short_name = short_name
                         node.hw_model = hw_model
                         node.role = role
                         node.channel = env.channel_id
@@ -282,8 +414,8 @@ async def process_envelope(topic, env):
                         node = Node(
                             id=user.id,
                             node_id=node_id,
-                            long_name=user.long_name,
-                            short_name=user.short_name,
+                            long_name=long_name,
+                            short_name=short_name,
                             hw_model=hw_model,
                             role=role,
                             channel=env.channel_id,
@@ -300,11 +432,10 @@ async def process_envelope(topic, env):
             position = decode_payload.decode_payload(
                 PortNum.POSITION_APP, env.packet.decoded.payload
             )
-            if position and position.latitude_i and position.longitude_i:
-                from_node_id = getattr(env.packet, "from")
-                node = (
-                    await session.execute(select(Node).where(Node.node_id == from_node_id))
-                ).scalar_one_or_none()
+            if position and has_valid_location(position.latitude_i, position.longitude_i):
+                node = await ensure_node_exists(
+                    session, getattr(env.packet, "from", None), env.channel_id
+                )
                 if node:
                     now_us = int(time.time() * 1_000_000)
                     node.last_lat = position.latitude_i
@@ -324,7 +455,7 @@ async def process_envelope(topic, env):
                         packet_id=packet_id,
                         route=env.packet.decoded.payload,
                         done=not env.packet.decoded.want_response,
-                        gateway_node_id=int(env.gateway_id[1:], 16),
+                        gateway_node_id=gateway_node_id,
                         import_time_us=now_us,
                     )
                 )

meshview/radio/coverage.py

@@ -42,7 +42,12 @@ def destination_point(
     return math.degrees(lat2), math.degrees(lon2)
 
 
-@lru_cache(maxsize=512)
+def _loop_is_safe(radius_km: float, step_km: float) -> bool:
+    """A non-positive step or a non-finite radius would make the distance loop endless."""
+    return math.isfinite(radius_km) and math.isfinite(step_km) and step_km > 0
+
+
+@lru_cache(maxsize=64)
 def compute_coverage(
     lat: float,
     lon: float,
@@ -54,7 +59,7 @@ def compute_coverage(
     step_km: float,
     reliability: float,
 ) -> list[tuple[float, float, float]]:
-    if not ITM_AVAILABLE:
+    if not ITM_AVAILABLE or not _loop_is_safe(radius_km, step_km):
         return []
 
     points = []
@@ -90,7 +95,7 @@ def compute_coverage(
     return points
 
 
-@lru_cache(maxsize=512)
+@lru_cache(maxsize=64)
 def compute_perimeter(
     lat: float,
     lon: float,
@@ -103,7 +108,7 @@ def compute_perimeter(
     reliability: float,
     threshold_dbm: float,
 ) -> list[tuple[float, float]]:
-    if not ITM_AVAILABLE:
+    if not ITM_AVAILABLE or not _loop_is_safe(radius_km, step_km):
         return []
 
     perimeter = []

meshview/static/html-escape.js

@@ -0,0 +1,10 @@
+/* Shared HTML escaping for strings that come from the mesh (node names, channel ids,
+   firmware strings, message text). Anything a radio can send must go through this
+   before it is placed in innerHTML, insertAdjacentHTML or a Leaflet popup/tooltip. */
+(function () {
+    const MAP = { "&": "&amp;", "<": "&lt;", ">": "&gt;", '"': "&quot;", "'": "&#39;" };
+    window.escapeHtml = function escapeHtml(value) {
+        if (value === null || value === undefined) return "";
+        return String(value).replace(/[&<>"']/g, ch => MAP[ch]);
+    };
+})();

meshview/static/kiosk.html

@@ -45,6 +45,7 @@ body { margin: 0; font-family: monospace; background: #121212; color: #eee; }
 <script src="https://unpkg.com/leaflet@1.9.4/dist/leaflet.js" crossorigin></script>
 <script src="https://unpkg.com/leaflet-polylinedecorator@1.6.0/dist/leaflet.polylinedecorator.js" crossorigin></script>
 <script src="/static/portmaps.js"></script>
+<script src="/static/html-escape.js"></script>
 
 <script>
 (async function(){
@@ -115,12 +116,12 @@ body { margin: 0; font-family: monospace; background: #121212; color: #eee; }
         marker.originalColor = color;
         markerById[node.node_id]=marker;
 
-        let popupContent=`<b>${node.long_name} (${node.short_name})</b><br>
-                          <b>Channel:</b> ${node.channel}<br>
-                          <b>Model:</b> ${node.hw_model}<br>
-                          <b>Role:</b> ${node.role}<br>`;
+        let popupContent=`<b>${escapeHtml(node.long_name)} (${escapeHtml(node.short_name)})</b><br>
+                          <b>Channel:</b> ${escapeHtml(node.channel)}<br>
+                          <b>Model:</b> ${escapeHtml(node.hw_model)}<br>
+                          <b>Role:</b> ${escapeHtml(node.role)}<br>`;
         if(node.last_seen_us) popupContent+=`<b>Last seen:</b> ${timeAgoFromUs(node.last_seen_us)}<br>`;
-        if(node.firmware) popupContent+=`<b>Firmware:</b> ${node.firmware}<br>`;
+        if(node.firmware) popupContent+=`<b>Firmware:</b> ${escapeHtml(node.firmware)}<br>`;
 
         marker.on('click', e=>{
             e.originalEvent.stopPropagation();
@@ -139,7 +140,7 @@ body { margin: 0; font-family: monospace; background: #121212; color: #eee; }
         const color=hashToColor(channel);
         const label=document.createElement('label');
         label.style.color=color;
-        label.innerHTML=`<input type="checkbox" class="filter-checkbox" id="${id}" checked> ${channel}`;
+        label.innerHTML=`<input type="checkbox" class="filter-checkbox" id="${escapeHtml(id)}" checked> ${escapeHtml(channel)}`;
         filterContainer.appendChild(label);
     });
 
@@ -178,7 +179,7 @@ body { margin: 0; font-family: monospace; background: #121212; color: #eee; }
         let count=0;
         const portName=portMap[portnum]||`Port ${portnum}`;
         const tooltip=L.tooltip({permanent:true,direction:'top',offset:[0,-marker.options.radius-5],className:'blinking-tooltip'})
-                        .setContent(`${longName} (${portName})`).setLatLng(marker.getLatLng());
+                        .setContent(`${escapeHtml(longName)} (${escapeHtml(portName)})`).setLatLng(marker.getLatLng());
         tooltip.addTo(map); marker.tooltip=tooltip;
         const interval=setInterval(()=>{
             if(map.hasLayer(marker)){

meshview/store.py

@@ -4,11 +4,14 @@ from datetime import datetime, timedelta, timezone
 from sqlalchemy import Text, and_, cast, func, or_, select
 from sqlalchemy.orm import lazyload
 
+from meshtastic.protobuf.portnums_pb2 import PortNum
 from meshview import database, models
 from meshview.models import Node, Packet, PacketSeen, Traceroute
 
 logger = logging.getLogger(__name__)
 
+BROADCAST_NODE_ID = 0xFFFFFFFF
+
 
 async def get_node(node_id):
     async with database.async_session() as session:
@@ -70,6 +73,14 @@ async def get_packets(
             contains_lower = f"%{contains.lower()}%"
             payload_text = cast(models.Packet.payload, Text)
             conditions.append(func.lower(payload_text).like(contains_lower))
+            # Direct text messages are shown as "<redacted>" by the API, so they must not be
+            # searchable either: a match would reveal their contents one substring at a time.
+            conditions.append(
+                or_(
+                    models.Packet.portnum != PortNum.TEXT_MESSAGE_APP,
+                    models.Packet.to_node_id == BROADCAST_NODE_ID,
+                )
+            )
 
         # Apply WHERE conditions
         if conditions:

meshview/templates/base.html

@@ -10,6 +10,7 @@
     <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.3/dist/css/bootstrap.min.css" rel="stylesheet" crossorigin="anonymous">
     <link rel="stylesheet" href="https://unpkg.com/leaflet@1.9.4/dist/leaflet.css" crossorigin=""/>
     <script src="https://unpkg.com/leaflet@1.9.4/dist/leaflet.js" crossorigin=""></script>
+    <script src="/static/html-escape.js"></script>
 
     {% block head %}{% endblock %}
 
@@ -39,8 +40,9 @@
             height: 500px;
         }
 
-        {% block css %}{% endblock %}
     </style>
+
+    {% block css %}{% endblock %}
 </head>
 
 <body>

meshview/templates/chat.html

@@ -1,6 +1,7 @@
 {% extends "base.html" %}
 
 {% block css %}
+<style>
 .timestamp {
     min-width: 10em;
     color: #ccc;
@@ -53,6 +54,7 @@
 .chat-filter {
     min-width: 220px;
 }
+</style>
 {% endblock %}
 
 {% block body %}

meshview/templates/firehose.html

@@ -1,6 +1,7 @@
 {% extends "base.html" %}
 
 {% block css %}
+<style>
 .container {
     margin: 0 auto;
     padding: 10px;
@@ -90,6 +91,7 @@
 .inline-link:hover {
     color: #c7e6ff;
 }
+</style>
 {% endblock %}
 
 {% block body %}
@@ -199,9 +201,7 @@ function portLabel(portnum, payload, linksHtml) {
     const name = PORT_MAP[portnum] || "Unknown";
     const color = PORT_COLORS[portnum] || "#6c757d";
 
-    const safePayload = payload
-        ? payload.replace(/"/g, "&quot;")
-        : "";
+    const safePayload = escapeHtml(payload || "");
 
     return `
         <span class="port-tag" style="background-color:${color}" title="${safePayload}">
@@ -241,7 +241,7 @@ function logPacketTimes(packet) {
     const times = formatTimes(packet.import_time_us);
     console.log(
         "[firehose] packet time",
-        "id=" + packet.id,
+        "id=" + (packet.packet_id ?? packet.id),
         "epoch_us=" + times.epoch,
         "local=" + times.local,
         "utc=" + times.utc
@@ -325,7 +325,7 @@ async function fetchUpdates() {
                            ${firehoseTranslations.all_broadcast || "All"}
                        </span>`
                     : `<a href="/node/${pkt.from_node_id}" style="text-decoration:underline; color:inherit;">
-                           ${nodeMap[pkt.from_node_id] || pkt.from_node_id}
+                           ${escapeHtml(nodeMap[pkt.from_node_id] || pkt.from_node_id)}
                        </a>`;
 
             /* TO — includes translation */
@@ -339,7 +339,7 @@ async function fetchUpdates() {
                            ${firehoseTranslations.all_broadcast || "All"}
                        </span>`
                     : `<a href="/node/${pkt.to_node_id}" style="text-decoration:underline; color:inherit;">
-                           ${nodeMap[pkt.to_node_id] || pkt.to_node_id}
+                           ${escapeHtml(nodeMap[pkt.to_node_id] || pkt.to_node_id)}
                        </a>`;
 
             let inlineLinks = "";
@@ -365,13 +365,19 @@ async function fetchUpdates() {
                 if (match) traceId = match[1];
 
                 inlineLinks += ` <a class="inline-link"
-                                    href="/graph/traceroute/${traceId}"
-                                    target="_blank">⮕</a>`;
+                                    href="/traceroute/${traceId}"
+                                    >⮕</a>`;
             }
 
-            const safePayload = (pkt.payload || "")
-                .replace(/</g, "&lt;")
-                .replace(/>/g, "&gt;");
+            const safePayload = escapeHtml(pkt.payload || "");
+            const packetIdCell = pkt.packet_id
+                ? `<a href="/packet/${pkt.storage_id || pkt.id}"
+                      style="text-decoration:underline; color:inherit;">
+                       ${pkt.packet_id}
+                   </a>`
+                : pkt.portnum === 73
+                ? `<span class="text-secondary" title="MQTT map report">Not at Packet</span>`
+                : `<span class="text-secondary" title="No Meshtastic packet ID">—</span>`;
 
             const html = `
 <tr class="packet-row">
@@ -382,15 +388,12 @@ async function fetchUpdates() {
 
     <td>
         <span class="toggle-btn">▶</span>
-        <a href="/packet/${pkt.id}"
-           style="text-decoration:underline; color:inherit;">
-            ${pkt.id}
-        </a>
+        ${packetIdCell}
     </td>
 
     <td>${from}</td>
     <td>${to}</td>
-    <td>${pkt.channel || "—"}</td>
+    <td>${escapeHtml(pkt.channel || "—")}</td>
     <td>${portLabel(pkt.portnum, pkt.payload, inlineLinks)}</td>
 
 </tr>

meshview/templates/map.html

@@ -1,6 +1,7 @@
 {% extends "base.html" %}
 
 {% block css %}
+<style>
 <link rel="stylesheet" href="https://unpkg.com/leaflet@1.9.4/dist/leaflet.css"
       integrity="sha256-p4NxAoJBhIIN+hmNHrzRCf9tD/miZyoHS5obTRR9BMY="
       crossorigin=""/>
@@ -27,9 +28,9 @@
         background: white;
         color: black;
         border: 1px solid #111;
-        border-radius: 6px;
-        padding: 6px 10px;
-        font-size: 14px;
+        border-radius: 4px;
+        padding: 3px 6px;
+        font-size: 12px;
         font-weight: 400;
         box-shadow: 0 2px 8px rgba(0,0,0,0.25);
     }
@@ -37,6 +38,33 @@
         animation: textPulse 1.1s ease-in-out 6;
         border-color: #ff8c00;
     }
+    .packet-seen-trace {
+        stroke-dasharray: 0.01px var(--packet-trace-gap, 999px);
+        stroke-linecap: round;
+        filter: drop-shadow(0 0 1px rgba(0, 0, 0, 0.65));
+        animation: packetTrace var(--packet-trace-duration, 6.5s) linear 1 forwards;
+    }
+    .packet-seen-landing {
+        transform-box: fill-box;
+        transform-origin: center;
+        animation: packetSeenLanding 1s ease-out 1 forwards;
+    }
+    @keyframes packetTrace {
+        from { stroke-dashoffset: 0; }
+        to { stroke-dashoffset: calc(-1 * var(--packet-trace-distance, 1000px)); }
+    }
+    @keyframes packetSeenLanding {
+        from {
+            stroke-opacity: 0.7;
+            fill-opacity: 0.24;
+            transform: scale(0.3);
+        }
+        to {
+            stroke-opacity: 0;
+            fill-opacity: 0;
+            transform: scale(1.6);
+        }
+    }
     @keyframes textPulse {
         0% { box-shadow: 0 2px 8px rgba(0,0,0,0.25); }
         50% { box-shadow: 0 4px 14px rgba(255,140,0,0.45); }
@@ -90,6 +118,7 @@
     .unmapped-node { font-weight: 400; color: #000; }
     .unmapped-empty { color: #666; font-style: italic; }
 </style>
+</style>
 {% endblock %}
 
 {% block body %}
@@ -148,6 +177,7 @@
 <script src="https://unpkg.com/leaflet-polylinedecorator@1.6.0/dist/leaflet.polylinedecorator.js"
         integrity="sha384-FhPn/2P/fJGhQLeNWDn9B/2Gml2bPOrKJwFqJXgR3xOPYxWg5mYQ5XZdhUSugZT0"
         crossorigin></script>
+<script src="https://unpkg.com/leaflet.heat/dist/leaflet-heat.js"></script>
 <script src="/static/portmaps.js"></script>
 
 <script>
@@ -188,39 +218,68 @@ function applyTranslationsMap(root = document) {
    ====================================================== */
 
 var map = L.map('map');
-L.tileLayer('https://tile.openstreetmap.org/{z}/{x}/{y}.png',
-    { maxZoom:19, attribution:'&copy; OpenStreetMap' }).addTo(map);
+var baseLayers = {
+    "OpenStreetMap": L.tileLayer('https://tile.openstreetmap.org/{z}/{x}/{y}.png', {
+        maxZoom: 19,
+        attribution: '&copy; OpenStreetMap'
+    }),
+    "OpenTopoMap": L.tileLayer('https://{s}.tile.opentopomap.org/{z}/{x}/{y}.png', {
+        maxZoom: 17,
+        attribution: 'Map data: &copy; OpenStreetMap, SRTM | Map style: &copy; OpenTopoMap'
+    })
+};
+baseLayers["OpenStreetMap"].addTo(map);
 L.control.scale({
     position: 'bottomleft',
     metric: true,
     imperial: true
 }).addTo(map);
+var nodeLayer = L.layerGroup().addTo(map);
+var edgeLayer = L.layerGroup().addTo(map);
+var packetSeenLayer = L.layerGroup();
+var trafficHeatLayer = L.heatLayer([], {
+    radius: 34,
+    blur: 18,
+    maxZoom: 12,
+    minOpacity: 0.3,
+    gradient: {
+        0.2: "#0b1f4d",
+        0.45: "#12436b",
+        0.7: "#7f2704",
+        1.0: "#4d0000"
+    }
+});
+var overlayLayers = {
+    "Nodes": nodeLayer,
+    "Packets Reach": packetSeenLayer,
+    "Traffic HeatMap": trafficHeatLayer
+};
+L.control.layers(baseLayers, overlayLayers, { collapsed: false }).addTo(map);
 
 // Data structures
 var nodes = [], markers = {}, markerById = {}, nodeMap = new Map();
-var edgeLayer = L.layerGroup().addTo(map), selectedNodeId = null;
+var selectedNodeId = null;
 var activeBlinks = new Map(), lastImportTime = null;
+var activePacketSeenHighlights = [];
 var mapInterval = 0;
 var unmappedPackets = [];
+var trafficHeatLoaded = false;
+var trafficHeatLoading = false;
 const UNMAPPED_LIMIT = 50;
 const UNMAPPED_TTL_MS = 5000;
+const PACKET_SEEN_DELAY_MS = 15000;
+const PACKET_TRACE_MIN_DURATION_MS = 1500;
+const PACKET_TRACE_MAX_DURATION_MS = 6500;
+const PACKET_TRACE_MS_PER_PIXEL = 18;
+const PACKET_LANDING_DURATION_MS = 1000;
 
 const portMap = window.PORT_LABEL_MAP;
 
-const palette = ["#bcf60c","#fabebe","#e6beff","#9a6324","#fffac8","#800000","#aaffc3","#ffd8b1",
-                 "#808080"];
+// Generated colors for channels without a configured override.
+const palette = ["#18aa18","#976049","#a5508c","#1898dc","#3a8846","#dc1884",
+                 "#b8813d","#b64f35","#408b18","#4c719e","#be379c","#18aa52"];
 
-const channelColorOverrides = {
-    longfast: "#4363d8",
-    longslow: "#000075",
-    verylongslow: "#808000",
-    mediumslow: "#008080",
-    mediumfast: "#e6194b",
-    shortslow: "#911eb4",
-    shortfast: "#f58231",
-    longmoderate: "#46f0f0",
-    shortturbo: "#f032e6"
-};
+const channelColorOverrides = new Map();
 
 const colorMap = new Map();
 const channelSet = new Set();
@@ -245,7 +304,7 @@ function hashToColor(str){
     const channel = String(str || "Unknown");
     if(colorMap.has(channel)) return colorMap.get(channel);
 
-    const override = channelColorOverrides[normalizeChannelName(channel)];
+    const override = channelColorOverrides.get(normalizeChannelName(channel));
     const c = override || palette[Math.floor(hashToUnit(channel) * palette.length) % palette.length];
     colorMap.set(channel,c);
     return c;
@@ -361,6 +420,7 @@ function fetchNewPackets(){
               } else {
                   addUnmappedPacket(pkt, nodeData);
               }
+              schedulePacketSeenNodes(pkt);
           });
 
           lastImportTime = latest;
@@ -389,12 +449,25 @@ document.addEventListener("visibilitychange",()=>{
     document.hidden?stopPacketFetcher():startPacketFetcher();
 });
 
+map.on("overlayadd", e => {
+    if(e.layer === nodeLayer) updateNodeVisibility();
+    if(e.layer === trafficHeatLayer) loadTrafficHeat();
+});
+
+map.on("overlayremove", e => {
+    if(e.layer === nodeLayer) clearActiveBlinks();
+    if(e.layer === packetSeenLayer) clearPacketSeenHighlights();
+});
+
 async function waitForConfig() {
     while (typeof window._siteConfigPromise === "undefined") {
         await new Promise(r => setTimeout(r, 100));
     }
     try {
         const cfg = await window._siteConfigPromise;
+        for(const [channel, color] of Object.entries(cfg.site?.map_channel_colors || {})){
+            channelColorOverrides.set(normalizeChannelName(channel), color);
+        }
         return cfg.site || {};
     } catch (err) {
         console.error("Error loading site config:", err);
@@ -443,7 +516,8 @@ initMapPolling();
 
 fetch('/api/nodes?days_active=3')
     .then(r=>r.json())
-    .then(data=>{
+    .then(async data=>{
+        await waitForConfig();
         if(!data.nodes) return;
 
         nodes = data.nodes.map(n=>({
@@ -470,6 +544,9 @@ fetch('/api/nodes?days_active=3')
 
         renderNodesOnMap();
         createChannelFilters();
+        if(map.hasLayer(trafficHeatLayer)){
+            loadTrafficHeat();
+        }
     })
     .catch(console.error);
 
@@ -490,7 +567,7 @@ function renderNodesOnMap(){
             fillColor: color,
             fillOpacity: 1,
             weight: 0.7
-        }).addTo(map);
+        }).addTo(nodeLayer);
 
         marker.nodeId = node.key;
         marker.originalColor = color;
@@ -513,6 +590,56 @@ function renderNodesOnMap(){
     setTimeout(() => applyTranslationsMap(), 50);
 }
 
+/* ======================================================
+   TRAFFIC HEAT LAYER
+   ====================================================== */
+
+async function loadTrafficHeat(){
+    if(trafficHeatLoaded || trafficHeatLoading) return;
+    if(nodeMap.size === 0) return;
+
+    trafficHeatLoading = true;
+
+    try {
+        const url = new URL("/api/packets", window.location.origin);
+        url.searchParams.set("limit", 1000);
+
+        const res = await fetch(url);
+        if(!res.ok) return;
+
+        const data = await res.json();
+        const counts = new Map();
+
+        (data.packets || []).forEach(pkt => {
+            const nodeId = pkt.from_node_id;
+            if(nodeId == null) return;
+            counts.set(nodeId, (counts.get(nodeId) || 0) + 1);
+        });
+
+        const maxCount = Math.max(...counts.values(), 0);
+        const heatPoints = [];
+
+        counts.forEach((count, nodeId) => {
+            const node = nodeMap.get(nodeId);
+            if(!node || isInvalidCoord(node)) return;
+
+            const markerLatLng = markerById[nodeId]?.getLatLng();
+            const lat = markerLatLng?.lat ?? node.lat;
+            const lng = markerLatLng?.lng ?? node.long;
+            const intensity = maxCount > 0 ? count / maxCount : 0;
+
+            heatPoints.push([lat, lng, Math.max(0.15, intensity)]);
+        });
+
+        trafficHeatLayer.setLatLngs(heatPoints);
+        trafficHeatLoaded = true;
+    } catch(err) {
+        console.error("Failed to load traffic heat layer:", err);
+    } finally {
+        trafficHeatLoading = false;
+    }
+}
+
 /* ======================================================
    UNMAPPED PACKETS LIST
    ====================================================== */
@@ -651,18 +778,131 @@ function isTextPort(portnum){
     return portnum === 1 || portnum === 7;
 }
 
+function isNodeVisibleForPackets(marker){
+    return map.hasLayer(nodeLayer) && nodeLayer.hasLayer(marker);
+}
+
+function clearBlinkForMarker(marker){
+    if(activeBlinks.has(marker)){
+        const blink = activeBlinks.get(marker);
+        clearInterval(blink.interval);
+        if(blink.tooltipTimeout) clearTimeout(blink.tooltipTimeout);
+        activeBlinks.delete(marker);
+    }
+    marker.setStyle({ fillColor: marker.originalColor });
+    if(marker.tooltip){
+        map.removeLayer(marker.tooltip);
+        marker.tooltip = null;
+    }
+}
+
+function clearActiveBlinks(){
+    activeBlinks.forEach((interval, marker) => clearBlinkForMarker(marker));
+    activeBlinks.clear();
+}
+
+function clearPacketSeenHighlights(){
+    activePacketSeenHighlights.forEach(timeout => clearTimeout(timeout));
+    activePacketSeenHighlights = [];
+    packetSeenLayer.clearLayers();
+}
+
+function schedulePacketSeenNodes(pkt){
+    if(!map.hasLayer(packetSeenLayer) || pkt.id == null) return;
+
+    const sourceNode = nodeMap.get(pkt.from_node_id);
+    if(!sourceNode || isInvalidCoord(sourceNode)) return;
+
+    const timeout = setTimeout(() => showPacketSeenNodes(pkt), PACKET_SEEN_DELAY_MS);
+    activePacketSeenHighlights.push(timeout);
+}
+
+async function showPacketSeenNodes(pkt){
+    if(!map.hasLayer(packetSeenLayer) || pkt.id == null) return;
+
+    try {
+        const res = await fetch(`/api/packets_seen/${pkt.id}`);
+        if(!res.ok) return;
+
+        const data = await res.json();
+        const seen = data.seen || [];
+        const sourceId = pkt.from_node_id;
+        const sourceNode = nodeMap.get(sourceId);
+        const sourceLatLng = sourceNode && !isInvalidCoord(sourceNode)
+            ? getNodeLatLng(sourceNode)
+            : null;
+
+        seen.forEach(row => {
+            const node = nodeMap.get(row.node_id);
+            if(!node || isInvalidCoord(node)) return;
+
+            const isSource = row.node_id === sourceId;
+            if(!sourceLatLng || isSource) return;
+
+            const markerLatLng = markerById[row.node_id]?.getLatLng();
+            const lat = markerLatLng?.lat ?? node.lat;
+            const lng = markerLatLng?.lng ?? node.long;
+            const trace = L.polyline([[sourceLatLng.lat, sourceLatLng.lng], [lat, lng]], {
+                color: "#000",
+                weight: 3,
+                opacity: 0.9,
+                className: "packet-seen-trace",
+                interactive: false
+            }).addTo(packetSeenLayer);
+            const traceEl = trace.getElement();
+            let traceDurationMs = PACKET_TRACE_MAX_DURATION_MS;
+            if(traceEl){
+                const traceLength = traceEl.getTotalLength();
+                traceDurationMs = Math.min(
+                    PACKET_TRACE_MAX_DURATION_MS,
+                    Math.max(PACKET_TRACE_MIN_DURATION_MS, traceLength * PACKET_TRACE_MS_PER_PIXEL)
+                );
+                traceEl.style.setProperty("--packet-trace-gap", `${traceLength + 20}px`);
+                traceEl.style.setProperty("--packet-trace-distance", `${traceLength + 20}px`);
+                traceEl.style.setProperty("--packet-trace-duration", `${traceDurationMs}ms`);
+            }
+
+            const landingTimeout = setTimeout(() => {
+                const landing = L.circleMarker([lat, lng], {
+                    radius: 16,
+                    color: "#000",
+                    fillColor: "#000",
+                    fillOpacity: 0.24,
+                    opacity: 0.7,
+                    weight: 2,
+                    className: "packet-seen-landing",
+                    interactive: false
+                }).addTo(packetSeenLayer);
+
+                const clearLandingTimeout = setTimeout(() => {
+                    packetSeenLayer.removeLayer(landing);
+                }, PACKET_LANDING_DURATION_MS);
+                activePacketSeenHighlights.push(clearLandingTimeout);
+            }, traceDurationMs);
+            activePacketSeenHighlights.push(landingTimeout);
+
+            const timeout = setTimeout(() => {
+                packetSeenLayer.removeLayer(trace);
+            }, traceDurationMs + PACKET_LANDING_DURATION_MS);
+            activePacketSeenHighlights.push(timeout);
+        });
+    } catch(err) {
+        console.error("Failed to load packet seen nodes", pkt.id, err);
+    }
+}
+
 function blinkNode(marker,longName,portnum,payload){
-    if(!map.hasLayer(marker)) return;
+    if(!isNodeVisibleForPackets(marker)) return;
 
     if(activeBlinks.has(marker)){
-        clearInterval(activeBlinks.get(marker));
-        marker.setStyle({ fillColor: marker.originalColor });
-        if(marker.tooltip) map.removeLayer(marker.tooltip);
+        clearBlinkForMarker(marker);
     }
 
-    let blinkCount = 0;
     const blinkStart = Date.now();
-    const blinkDurationMs = 7000;
+    const tooltipDurationMs = 5000;
+    const blinkDurationMs = map.hasLayer(packetSeenLayer) ? 15000 : tooltipDurationMs;
+    const fastBlinkStartMs = blinkDurationMs - 3000;
+    const blinkColor = "yellow";
     const safeName = escapeHtml(longName);
     const portLabel = portMap[portnum] || `Port ${portnum ?? "?"}`;
     const payloadText = (payload || "").trim();
@@ -683,26 +923,39 @@ function blinkNode(marker,longName,portnum,payload){
     .addTo(map);
 
     marker.tooltip = tooltip;
+    const tooltipTimeout = setTimeout(() => {
+        if(blinkDurationMs === tooltipDurationMs){
+            clearBlinkForMarker(marker);
+            return;
+        }
+        if(marker.tooltip){
+            map.removeLayer(marker.tooltip);
+            marker.tooltip = null;
+        }
+    }, tooltipDurationMs);
 
     const interval = setInterval(()=>{
-        if(map.hasLayer(marker)){
+        const elapsedMs = Date.now() - blinkStart;
+        const blinkPeriodMs = elapsedMs >= fastBlinkStartMs ? 250 : 500;
+        if(isNodeVisibleForPackets(marker)){
             marker.setStyle({
-                fillColor: blinkCount%2===0 ? 'yellow' : marker.originalColor
+                fillColor: Math.floor(elapsedMs / blinkPeriodMs)%2===0
+                    ? blinkColor
+                    : marker.originalColor
             });
             marker.bringToFront();
-        }
-        blinkCount++;
-
-        if(Date.now() - blinkStart > blinkDurationMs){
-            clearInterval(interval);
-            marker.setStyle({ fillColor: marker.originalColor });
-            map.removeLayer(tooltip);
-            activeBlinks.delete(marker);
+        } else {
+            clearBlinkForMarker(marker);
+            return;
         }
 
-    },500);
+        if(elapsedMs > blinkDurationMs){
+            clearBlinkForMarker(marker);
+        }
 
-    activeBlinks.set(marker, interval);
+    },250);
+
+    activeBlinks.set(marker, { interval, tooltipTimeout });
 }
 
 /* ======================================================
@@ -757,6 +1010,8 @@ function saveFiltersToLocalStorage(){
 }
 
 function updateNodeVisibility(){
+    if(!map.hasLayer(nodeLayer)) return;
+
     const routerOnly = document.getElementById("filter-routers-only").checked;
     const mqttOnly = document.getElementById("filter-mqtt-only").checked;
     const activeChannels = [...channelSet].filter(ch =>
@@ -771,7 +1026,12 @@ function updateNodeVisibility(){
                 (!mqttOnly || n.is_mqtt_gateway) &&
                 activeChannels.includes(n.channel);
 
-            visible ? map.addLayer(marker) : map.removeLayer(marker);
+            if(visible){
+                nodeLayer.addLayer(marker);
+            } else {
+                clearBlinkForMarker(marker);
+                nodeLayer.removeLayer(marker);
+            }
         }
     });
 }

meshview/templates/net.html

@@ -1,6 +1,7 @@
 {% extends "base.html" %}
 
 {% block css %}
+<style>
 .timestamp { min-width: 10em; color: #ccc; }
 
 .chat-packet:nth-of-type(odd) { background-color: #3a3a3a; }
@@ -26,6 +27,7 @@
 #weekly-message { margin: 15px 0; font-weight: bold; color: #ffeb3b; }
 #total-count { margin-bottom: 10px; font-style: italic; color: #ccc; }
 #net-note { margin: 16px 0 0; font-size: 0.9rem; color: #aaa; }
+</style>
 {% endblock %}
 
 {% block body %}

meshview/templates/node.html

@@ -1,6 +1,7 @@
 {% extends "base.html" %}
 
 {% block css %}
+<style>
 {{ super() }}
 
 /* --- Map --- */
@@ -295,7 +296,7 @@
     border-color:#48bb78;
     color:#fff;
 }
-
+</style>
 {% endblock %}
 
 {% block body %}
@@ -548,9 +549,9 @@ function makeNodePopup(node) {
     return `
         <div style="font-size:0.9em">
             <a href="/node/${node.node_id}" style="color:inherit; text-decoration:underline;">
-                <b>${node.long_name || node.short_name || node.node_id}</b>
+                <b>${escapeHtml(node.long_name || node.short_name || node.node_id)}</b>
             </a>
-            ${node.short_name ? ` (${node.short_name})` : ""}<br>
+            ${node.short_name ? ` (${escapeHtml(node.short_name)})` : ""}<br>
 
             <b><span data-translate-lang="node_id">
                 ${nodeTranslations.node_id || "Node ID"}:
@@ -558,19 +559,19 @@ function makeNodePopup(node) {
 
             <b><span data-translate-lang="hw_model">
                 ${nodeTranslations.hw_model || "HW Model"}:
-            </span></b> ${node.hw_model ?? "—"}<br>
+            </span></b> ${escapeHtml(node.hw_model ?? "—")}<br>
 
             <b><span data-translate-lang="channel">
                 ${nodeTranslations.channel || "Channel"}:
-            </span></b> ${node.channel ?? "—"}<br>
+            </span></b> ${escapeHtml(node.channel ?? "—")}<br>
 
             <b><span data-translate-lang="role">
                 ${nodeTranslations.role || "Role"}:
-            </span></b> ${node.role ?? "—"}<br>
+            </span></b> ${escapeHtml(node.role ?? "—")}<br>
 
             <b><span data-translate-lang="firmware">
                 ${nodeTranslations.firmware || "Firmware"}:
-            </span></b> ${node.firmware ?? "—"}<br>
+            </span></b> ${escapeHtml(node.firmware ?? "—")}<br>
 
             <b><span data-translate-lang="last_update">
                 ${nodeTranslations.last_update || "Last Update"}:
@@ -745,7 +746,7 @@ function nodeLink(id, labelOverride = null) {
     const label = labelOverride || nodeMap[id] || id;
 
     return `<a href="/node/${id}" style="text-decoration:underline; color:inherit;">
-                ${label}
+                ${escapeHtml(label)}
             </a>`;
 }
 
@@ -855,7 +856,7 @@ function addMarker(id, lat, lon, color = "red", node = null) {
         node = nodeCache[id] || null;
     }
 
-    const popupHtml = node ? makeNodePopup(node) : `<b>${id}</b>`;
+    const popupHtml = node ? makeNodePopup(node) : `<b>${escapeHtml(id)}</b>`;
 
     const m = L.circleMarker([lat, lon], {
         radius: 6,
@@ -1046,8 +1047,7 @@ async function loadPackets(filters = {}) {
         // ================================
         // TABLE ROW
         // ================================
-        const safePayload = (pkt.payload || "")
-            .replace(/[<>]/g, m => (m === "<" ? "&lt;" : "&gt;"));
+        const safePayload = escapeHtml(pkt.payload || "");
 
         const localTime = formatLocalTime(pkt.import_time_us);
         const fromCell  = nodeLink(pkt.from_node_id, pkt.long_name);
@@ -1071,18 +1071,24 @@ async function loadPackets(filters = {}) {
             const match = pkt.payload?.match(/ID:\s*(\d+)/i);
             if (match) traceId = match[1];
             inlineLinks +=
-                ` <a class="inline-link" href="/graph/traceroute/${traceId}" target="_blank">⮕</a>`;
+                ` <a class="inline-link" href="/traceroute/${traceId}" target="_blank">⮕</a>`;
         }
 
         const sizeBytes = packetSizeBytes(pkt);
+        const packetIdCell = pkt.packet_id
+            ? `<a href="/packet/${pkt.storage_id || pkt.id}"
+                  style="text-decoration:underline; color:inherit;">
+                   ${pkt.packet_id}
+               </a>`
+            : pkt.portnum === 73
+            ? `<span class="text-secondary" title="MQTT map report">Not a Packet</span>`
+            : `<span class="text-secondary" title="No Meshtastic packet ID">—</span>`;
 
         list.insertAdjacentHTML("afterbegin", `
         <tr class="packet-row">
             <td>${localTime}</td>
             <td><span class="toggle-btn">▶</span>
-                <a href="/packet/${pkt.id}" style="text-decoration:underline; color:inherit;">
-                    ${pkt.id}
-                </a>
+                ${packetIdCell}
             </td>
             <td>${fromCell}</td>
             <td>${toCell}</td>
@@ -1288,13 +1294,26 @@ async function loadLatestNeighborIds() {
 
 /* ======================================================
     NEIGHBOR CHART (portnum=71)
+
+    The request is bounded to NEIGHBOR_WINDOW_DAYS through the
+    API's `since` parameter, and only the NEIGHBOR_MAX_SERIES
+    neighbors with the most data points in that window are
+    drawn. Without both limits a busy node produces hundreds of
+    legend entries covering weeks of history.
    ====================================================== */
 
+const NEIGHBOR_WINDOW_DAYS = 7;
+const NEIGHBOR_MAX_SERIES  = 15;
+
 async function loadNeighborTimeSeries() {
     const container = document.getElementById("neighbor_chart_container");
     const chartEl   = document.getElementById("chart_neighbors");
 
-    const url = `/api/packets?portnum=71&from_node_id=${fromNodeId}&limit=500`;
+    // Start of the window, in microseconds (import_time_us is epoch-us).
+    const sinceUs = (Date.now() - NEIGHBOR_WINDOW_DAYS * 86400 * 1000) * 1000;
+
+    const url = `/api/packets?portnum=71&from_node_id=${fromNodeId}`
+              + `&since=${sinceUs}&limit=500`;
     const res = await fetch(url);
 
     if (!res.ok) {
@@ -1303,7 +1322,11 @@ async function loadNeighborTimeSeries() {
     }
 
     const data = await res.json();
-    const packets = data.packets || [];
+
+    // Re-apply the window here too, so the chart cannot drift past
+    // it if the packet source ever returns something older.
+    const packets = (data.packets || [])
+        .filter(p => (p.import_time_us || 0) >= sinceUs);
 
     if (!packets.length) {
         container.style.display = "none";
@@ -1313,7 +1336,8 @@ async function loadNeighborTimeSeries() {
     // Sort packets chronologically (microseconds)
     packets.sort((a, b) => (a.import_time_us || 0) - (b.import_time_us || 0));
 
-    const neighborHistory = {}; // node_id -> { name, times[], snr[] }
+    // --- Pass 1: collect the readings, without looking up names yet ---
+    const neighborHistory = {}; // node_id -> { times[], snr[] }
 
     for (const pkt of packets) {
         if (!pkt.import_time_us || !pkt.payload) continue;
@@ -1332,39 +1356,56 @@ async function loadNeighborTimeSeries() {
             if (!idMatch || !snrMatch) continue;
 
             const nid = parseInt(idMatch[1], 10);
-            const snr = parseFloat(snrMatch[1]);
-
-            // Fetch neighbor metadata once
-            const neighbor = await fetchNodeFromApi(nid);
 
             if (!neighborHistory[nid]) {
-                neighborHistory[nid] = {
-                    name: neighbor?.short_name ||
-                          neighbor?.long_name ||
-                          `Node ${nid}`,
-                    times: [],
-                    snr: []
-                };
+                neighborHistory[nid] = { times: [], snr: [] };
             }
 
             neighborHistory[nid].times.push(ts);
-            neighborHistory[nid].snr.push(snr);
+            neighborHistory[nid].snr.push(parseFloat(snrMatch[1]));
         }
     }
 
-    // Collect ALL timestamps across neighbors
-    const allTimes = new Set();
-    Object.values(neighborHistory).forEach(entry => {
-        entry.times.forEach(t => allTimes.add(t));
-    });
+    // --- Keep the neighbors with the most data points in the window ---
+    const ranked = Object.entries(neighborHistory)
+        .sort((a, b) => b[1].times.length - a[1].times.length);
 
-    // Sort timestamps numerically
-    const xTimes = Array.from(allTimes).sort((a, b) => a - b);
+    const kept    = ranked.slice(0, NEIGHBOR_MAX_SERIES);
+    const dropped = ranked.length - kept.length;
+
+    if (!kept.length) {
+        container.style.display = "none";
+        return;
+    }
+
+    // --- Pass 2: names, only for the series that are drawn ---
+    await Promise.all(kept.map(async ([nid, entry]) => {
+        const neighbor = await fetchNodeFromApi(nid);
+        entry.name = neighbor?.short_name ||
+                     neighbor?.long_name  ||
+                     `Node ${nid}`;
+    }));
+
+    // ECharts identifies a series by its name, so two neighbors sharing
+    // a short name would be drawn as one line. Keep them apart.
+    const usedNames = new Set();
+    for (const [nid, entry] of kept) {
+        if (usedNames.has(entry.name)) entry.name = `${entry.name} (${nid})`;
+        usedNames.add(entry.name);
+    }
+
+    // Timestamps across the series that are drawn
+    const xTimes = Array.from(
+        new Set(kept.flatMap(([, entry]) => entry.times))
+    ).sort((a, b) => a - b);
 
     const legend = [];
     const series = [];
 
-    for (const entry of Object.values(neighborHistory)) {
+    for (const [, entry] of kept) {
+        const byTime = new Map();
+        entry.times.forEach((t, i) => byTime.set(t, entry.snr[i]));
+
         legend.push(entry.name);
 
         series.push({
@@ -1373,21 +1414,46 @@ async function loadNeighborTimeSeries() {
             smooth: true,
             connectNulls: true,
             showSymbol: false,
-            data: xTimes.map(t => {
-                const idx = entry.times.indexOf(t);
-                return idx >= 0 ? entry.snr[idx] : null;
-            })
+            data: xTimes.map(t => (byTime.has(t) ? byTime.get(t) : null))
         });
     }
 
+    // Values behind the "Export CSV" button: the most recent SNR
+    // in the window, per neighbor.
+    neighborData = {
+        ids:   kept.map(([nid]) => nid),
+        names: kept.map(([, entry]) => entry.name),
+        snrs:  kept.map(([, entry]) => entry.snr[entry.snr.length - 1])
+    };
+
+    // Say which neighbors are on screen, so a missing line is not a mystery.
+    const subtextTemplate = dropped > 0
+        ? (nodeTranslations.neighbors_chart_subtitle_capped ||
+           "Last {days} days — the {n} neighbors with the most data points, of {total}")
+        : (nodeTranslations.neighbors_chart_subtitle ||
+           "Last {days} days — {n} neighbors");
+
+    const subtext = subtextTemplate
+        .replace("{days}",  NEIGHBOR_WINDOW_DAYS)
+        .replace("{n}",     kept.length)
+        .replace("{total}", ranked.length);
+
     const chart = echarts.init(chartEl);
 
     chart.setOption({
+        title: {
+            subtext,
+            left: "center",
+            bottom: 0,
+            subtextStyle: { color: "#888", fontSize: 11 }
+        },
+        grid: { bottom: 60 },
         tooltip: {
             trigger: "axis",
             axisPointer: { type: "line" }
         },
         legend: {
+            type: "scroll",
             data: legend,
             textStyle: { color: "#ccc" }
         },
@@ -1706,6 +1772,7 @@ async function loadNodeStats(nodeId) {
             : "";
 
         const portName = PORT_LABEL_MAP[pkt.portnum] || `Port ${pkt.portnum}`;
+        const displayPacketId = pkt.packet_id || (pkt.portnum === 73 ? "Not a Packet" : "");
 
         // Escape quotes + line breaks for CSV safety
         const payload = (pkt.payload || "")
@@ -1714,7 +1781,7 @@ async function loadNodeStats(nodeId) {
 
         rows.push([
             time,
-            pkt.id,
+            displayPacketId,
             pkt.from_node_id,
             pkt.to_node_id,
             pkt.portnum,

meshview/templates/nodegraph.html

@@ -5,6 +5,7 @@
 {% endblock %}
 
 {% block css %}
+<style>
 #mynetwork {
     width: 100%;
     height: 100vh;
@@ -82,6 +83,7 @@
     border-radius: 3px;
 }
 .circle { border-radius: 6px; }
+</style>
 {% endblock %}
 
 {% block body %}

meshview/templates/nodelist.html

@@ -652,14 +652,14 @@ document.addEventListener("DOMContentLoaded", async function() {
                 // DESKTOP TABLE ROW
                 const row = document.createElement("tr");
                 row.innerHTML = `
-                    <td>${node.short_name || "N/A"}</td>
-                    <td><a href="/node/${node.node_id}">${node.long_name || "N/A"}</a></td>
-                    <td>${node.hw_model || "N/A"}</td>
-                    <td>${node.firmware || "N/A"}</td>
-                    <td>${node.role || "N/A"}</td>
+                    <td>${escapeHtml(node.short_name || "N/A")}</td>
+                    <td><a href="/node/${node.node_id}">${escapeHtml(node.long_name || "N/A")}</a></td>
+                    <td>${escapeHtml(node.hw_model || "N/A")}</td>
+                    <td>${escapeHtml(node.firmware || "N/A")}</td>
+                    <td>${escapeHtml(node.role || "N/A")}</td>
                     <td>${node.last_lat ? (node.last_lat / 1e7).toFixed(7) : "N/A"}</td>
                     <td>${node.last_long ? (node.last_long / 1e7).toFixed(7) : "N/A"}</td>
-                    <td>${node.channel || "N/A"}</td>
+                    <td>${escapeHtml(node.channel || "N/A")}</td>
                     <td>${node.is_mqtt_gateway ? (nodelistTranslations.yes || "Yes") : (nodelistTranslations.no || "No")}</td>
                     <td>${timeAgoFromMs(node.last_seen_ms)}</td>
                     <td style="text-align:center;">
@@ -675,22 +675,22 @@ document.addEventListener("DOMContentLoaded", async function() {
                 card.className = "node-card";
                 card.innerHTML = `
                     <div class="node-card-header">
-                        <span>${node.short_name || node.long_name || node.node_id}</span>
+                        <span>${escapeHtml(node.short_name || node.long_name || node.node_id)}</span>
                         <span class="favorite-star ${fav ? "active" : ""}" data-node-id="${node.node_id}">
                             ${star}
                         </span>
                     </div>
 
                     <div class="node-card-field"><b>ID:</b> ${node.node_id}</div>
-                    <div class="node-card-field"><b>Name:</b> ${node.long_name || "N/A"}</div>
-                    <div class="node-card-field"><b>HW:</b> ${node.hw_model || "N/A"}</div>
-                    <div class="node-card-field"><b>Firmware:</b> ${node.firmware || "N/A"}</div>
-                    <div class="node-card-field"><b>Role:</b> ${node.role || "N/A"}</div>
+                    <div class="node-card-field"><b>Name:</b> ${escapeHtml(node.long_name || "N/A")}</div>
+                    <div class="node-card-field"><b>HW:</b> ${escapeHtml(node.hw_model || "N/A")}</div>
+                    <div class="node-card-field"><b>Firmware:</b> ${escapeHtml(node.firmware || "N/A")}</div>
+                    <div class="node-card-field"><b>Role:</b> ${escapeHtml(node.role || "N/A")}</div>
                     <div class="node-card-field"><b>Location:</b>
                         ${node.last_lat ? (node.last_lat / 1e7).toFixed(5) : "N/A"},
                         ${node.last_long ? (node.last_long / 1e7).toFixed(5) : "N/A"}
                     </div>
-                    <div class="node-card-field"><b>Channel:</b> ${node.channel || "N/A"}</div>
+                    <div class="node-card-field"><b>Channel:</b> ${escapeHtml(node.channel || "N/A")}</div>
                     <div class="node-card-field"><b>Last Seen:</b> ${timeAgoFromMs(node.last_seen_ms)}</div>
 
                     <a href="/node/${node.node_id}" style="color:#9fd4ff; text-decoration:underline; margin-top:5px; display:block;">

meshview/templates/packet.html

@@ -206,12 +206,12 @@ document.addEventListener("DOMContentLoaded", async () => {
     const fromNodeObj = nodeLookup[p.from_node_id];
     const toNodeObj   = nodeLookup[p.to_node_id];
 
-    const fromNodeLabel = fromNodeObj?.long_name || p.from_node_id;
+    const fromNodeLabel = escapeHtml(fromNodeObj?.long_name || p.from_node_id);
 
     const toNodeLabel =
         p.to_node_id == 4294967295
             ? (packetTranslations.all_broadcast || "All")
-            : (toNodeObj?.long_name || p.to_node_id);
+            : escapeHtml(toNodeObj?.long_name || p.to_node_id);
 
     /* ---------------------------------------------
        Parse payload for lat/lon
@@ -267,7 +267,7 @@ document.addEventListener("DOMContentLoaded", async () => {
                 }</dd>
 
                 <dt data-translate-lang="channel">${packetTranslations.channel || "Channel"}:</dt>
-                <dd>${p.channel ?? "—"}</dd>
+                <dd>${escapeHtml(p.channel ?? "—")}</dd>
 
                 <dt data-translate-lang="port">${packetTranslations.port || "Port"}:</dt>
                 <dd><i>${PORT_NAMES[p.portnum] || "UNKNOWN APP"}</i> (${p.portnum})</dd>
@@ -338,7 +338,7 @@ document.addEventListener("DOMContentLoaded", async () => {
                 Lat: ${srcLat.toFixed(6)}<br>
                 Lon: ${srcLon.toFixed(6)}<br>
                 <span data-translate-lang="from_node">${packetTranslations.from_node || "From Node:"}</span> ${fromNodeLabel}<br>
-                <span data-translate-lang="channel">${packetTranslations.channel || "Channel:"}</span> ${p.channel ?? "—"}<br>
+                <span data-translate-lang="channel">${packetTranslations.channel || "Channel:"}</span> ${escapeHtml(p.channel ?? "—")}<br>
                 <span data-translate-lang="port">${packetTranslations.port || "Port:"}</span> ${PORT_NAMES[p.portnum] || "UNKNOWN"} (${p.portnum})
             </div>
         `);
@@ -427,7 +427,7 @@ seenTableBody.innerHTML = Object.keys(hopGroups)
 
         const rows = hopGroups[hopKey].map(s => {
             const node = nodeLookup[s.node_id];
-            const label = node?.long_name || s.node_id;
+            const label = escapeHtml(node?.long_name || s.node_id);
 
             const hopDisplay = formatHopDisplay(hopKey, s.hop_start);
             const timeStr = s.import_time_us
@@ -473,8 +473,8 @@ seenTableBody.innerHTML = Object.keys(hopGroups)
                     <b>${label}</b><br>
                     <span data-translate-lang="node_id_short">Node ID</span>:
                     <a href="/node/${s.node_id}">${s.node_id}</a><br>
-                    HW: ${node?.hw_model ?? "—"}<br>
-                    <span data-translate-lang="channel">Channel</span>: ${s.channel ?? "—"}<br>
+                    HW: ${escapeHtml(node?.hw_model ?? "—")}<br>
+                    <span data-translate-lang="channel">Channel</span>: ${escapeHtml(s.channel ?? "—")}<br>
                     ${
                         distanceKm !== null
                             ? `<span data-translate-lang="distance">Distance</span>:
@@ -500,7 +500,7 @@ seenTableBody.innerHTML = Object.keys(hopGroups)
                     <td>${s.rx_rssi ?? "—"}</td>
                     <td>${s.rx_snr ?? "—"}</td>
                     <td>${hopDisplay}</td>
-                    <td>${s.channel ?? "—"}</td>
+                    <td>${escapeHtml(s.channel ?? "—")}</td>
                     <td>${timeStr}</td>
                 </tr>
             `;
@@ -535,7 +535,7 @@ seenTableBody.innerHTML = Object.keys(hopGroups)
        Escape HTML helper
     ----------------------------------------------*/
     function escapeHtml(unsafe) {
-        return (unsafe ?? "").replace(/[&<"'>]/g, m => ({
+        return String(unsafe ?? "").replace(/[&<"'>]/g, m => ({
             "&":"&amp;",
             "<":"&lt;",
             ">":"&gt;",

meshview/templates/reliability.html

@@ -1,6 +1,7 @@
 {% extends "base.html" %}
 
 {% block css %}
+<style>
 .reach-page {
     max-width: 1200px;
     margin: 0 auto;
@@ -258,6 +259,7 @@
         grid-template-columns: 1fr;
     }
 }
+</style>
 {% endblock %}
 
 {% block body %}

meshview/templates/stats.html

@@ -1,6 +1,7 @@
 {% extends "base.html" %}
 
 {% block css %}
+<style>
 #packet_details {
     height: 95vh;
     overflow: auto;
@@ -94,6 +95,7 @@
     border: none;
     border-radius: 4px;
 }
+</style>
 {% endblock %}
 
 {% block head %}

meshview/templates/top.html

@@ -151,6 +151,11 @@ async function loadChannels() {
     const sel = document.getElementById("channelFilter");
 
     sel.innerHTML = "";
+    const allOpt = document.createElement("option");
+    allOpt.value = "";
+    allOpt.textContent = topTranslations.all_channels || "All channels";
+    sel.appendChild(allOpt);
+
     for (const ch of data.channels || []) {
         const opt = document.createElement("option");
         opt.value = ch;
@@ -158,7 +163,7 @@ async function loadChannels() {
         sel.appendChild(opt);
     }
 
-    sel.value = "MediumFast";
+    sel.value = "";
 }
 
 /* ======================================================
@@ -189,11 +194,11 @@ async function renderTable() {
             <td>
                 <a class="node-link" href="/node/${n.node_id}"
                    onclick="event.stopPropagation()">
-                    ${n.long_name || n.node_id}
+                    ${escapeHtml(n.long_name || n.node_id)}
                 </a>
             </td>
-            <td>${n.short_name || ""}</td>
-            <td>${n.channel || ""}</td>
+            <td>${escapeHtml(n.short_name || "")}</td>
+            <td>${escapeHtml(n.channel || "")}</td>
             <td>${n.sent}</td>
             <td>${n.seen}</td>
             <td><span class="${avgClass(n.avg)}">${n.avg.toFixed(1)}</span></td>

meshview/templates/traceroute.html

@@ -5,14 +5,56 @@
 {% endblock %}
 
 {% block css %}
+<style>
 #traceroute-graph {
     width: 100%;
-    height: 85vh;
+    height: calc(100vh - 270px);
     border: 1px solid #2a2f36;
     background: linear-gradient(135deg, #0f1216 0%, #171b22 100%);
     border-radius: 10px;
 }
 
+#traceroute-graph-wrap {
+    position: relative;
+}
+
+#traceroute-legend {
+    position: absolute;
+    right: 16px;
+    bottom: 16px;
+    z-index: 2;
+    display: grid;
+    gap: 8px;
+    padding: 10px 12px;
+    color: #d6dde6;
+    background: rgba(15, 18, 22, 0.88);
+    border: 1px solid #303743;
+    border-radius: 8px;
+    font-size: 13px;
+}
+
+.traceroute-legend-row {
+    display: grid;
+    grid-template-columns: 34px auto;
+    align-items: center;
+    gap: 8px;
+    white-space: nowrap;
+}
+
+.traceroute-legend-line {
+    height: 0;
+    border-top: 3px solid var(--line-color);
+}
+
+.traceroute-legend-line.muted {
+    border-top-width: 2px;
+    opacity: 0.55;
+}
+
+.traceroute-legend-line.dashed {
+    border-top-style: dashed;
+}
+
 #traceroute-meta {
     padding: 12px 16px;
     color: #c8d0da;
@@ -21,6 +63,7 @@
 #traceroute-error {
     color: #ff6b6b;
 }
+</style>
 {% endblock %}
 
 {% block body %}
@@ -28,7 +71,23 @@
     <div><b>Traceroute</b> <span id="traceroute-title"></span></div>
     <div id="traceroute-error"></div>
 </div>
-<div id="traceroute-graph"></div>
+<div id="traceroute-graph-wrap">
+    <div id="traceroute-graph"></div>
+    <div id="traceroute-legend" aria-label="Traceroute edge legend">
+        <div class="traceroute-legend-row">
+            <span class="traceroute-legend-line" style="--line-color: #34c759"></span>
+            <span>Winning forward</span>
+        </div>
+        <div class="traceroute-legend-row">
+            <span class="traceroute-legend-line dashed" style="--line-color: #ff3b30"></span>
+            <span>Reported reverse</span>
+        </div>
+        <div class="traceroute-legend-row">
+            <span class="traceroute-legend-line muted" style="--line-color: #7c858f"></span>
+            <span>Reported route</span>
+        </div>
+    </div>
+</div>
 
 <script>
 const el = document.getElementById("traceroute-graph");
@@ -49,6 +108,35 @@ function addPathEdges(path, edges, style) {
     }
 }
 
+function pathKey(path) {
+    return path.map(id => String(id)).join(">");
+}
+
+function addMissingEndpoints(path, startId, endId) {
+    const fullPath = path.map(id => String(id));
+    if (startId && (!fullPath.length || fullPath[0] !== startId)) {
+        fullPath.unshift(startId);
+    }
+    if (endId && (!fullPath.length || fullPath[fullPath.length - 1] !== endId)) {
+        fullPath.push(endId);
+    }
+    return fullPath;
+}
+
+// These lists are deduplicated across observations. A forward path is only known to
+// reach the target if some response reported it; a reverse path carries no such marker.
+function uniqueForwardPaths(data, originId, targetId) {
+    return (data?.unique_forward_paths || [])
+        .map(item => addMissingEndpoints(item.path || [], originId, item.complete ? targetId : null))
+        .filter(path => path.length > 1);
+}
+
+function uniqueReversePaths(data, originId, targetId) {
+    return (data?.unique_reverse_paths || [])
+        .map(path => addMissingEndpoints(path || [], targetId, null))
+        .filter(path => path.length > 1);
+}
+
 async function loadTraceroute() {
     const packetId = packetIdFromPath();
     document.getElementById("traceroute-title").textContent = `#${packetId}`;
@@ -73,26 +161,62 @@ async function loadTraceroute() {
     const nodes = new Map();
     const edges = [];
 
-    const forwardPaths = data?.winning_paths?.forward || [];
-    const reversePaths = data?.winning_paths?.reverse || [];
-    const originId = data?.packet?.from != null ? String(data.packet.from) : null;
-    const targetId = data?.packet?.to != null ? String(data.packet.to) : null;
+    // A response packet carries from/to reversed, so use the endpoints the API resolved.
+    const originId = data?.packet?.initiator != null ? String(data.packet.initiator) : null;
+    const targetId = data?.packet?.target != null ? String(data.packet.target) : null;
+    const forwardPaths = (data?.winning_paths?.forward || []).map(path => path.map(id => String(id)));
+    const reversePaths = (data?.winning_paths?.reverse || []).map(path => path.map(id => String(id)));
+    const winningPathKeys = new Set([
+        ...forwardPaths.map(pathKey),
+        ...reversePaths.map(pathKey)
+    ]);
+    const winningForwardNodeIds = new Set(forwardPaths.flat());
+    const hasWinningReversePath = reversePaths.some(path => path.length > 1);
+    const winningForwardPath = forwardPaths.find(path => path.length > 1);
+    const colorStartId = winningForwardPath ? winningForwardPath[0] : originId;
+    const colorEndId = winningForwardPath ? winningForwardPath[winningForwardPath.length - 1] : targetId;
+    const startNodeColor = hasWinningReversePath ? "#ff3b30" : "#34c759";
+    const endNodeColor = hasWinningReversePath ? "#34c759" : "#ff3b30";
+
+    uniqueForwardPaths(data, originId, targetId).forEach(path => {
+        if (winningPathKeys.has(pathKey(path))) {
+            return;
+        }
+        path.forEach(id => nodes.set(String(id), { name: String(id) }));
+        addPathEdges(path, edges, { color: "#7c858f", width: 1.5, opacity: 0.45 });
+    });
+
+    uniqueReversePaths(data, originId, targetId).forEach(path => {
+        if (winningPathKeys.has(pathKey(path))) {
+            return;
+        }
+        path.forEach(id => nodes.set(String(id), { name: String(id) }));
+        addPathEdges(path, edges, { color: "#7c858f", width: 1.5, opacity: 0.45, type: "dashed" });
+    });
 
     forwardPaths.forEach(path => {
         path.forEach(id => nodes.set(String(id), { name: String(id) }));
-        addPathEdges(path, edges, { color: "#ff5733", width: 3 });
+        addPathEdges(path, edges, { color: "#34c759", width: 3 });
     });
 
     reversePaths.forEach(path => {
         path.forEach(id => nodes.set(String(id), { name: String(id) }));
-        addPathEdges(path, edges, { color: "#00c3ff", width: 2, type: "dashed" });
+        addPathEdges(path, edges, { color: "#ff3b30", width: 2, type: "dashed" });
     });
 
     const graphNodes = Array.from(nodes.values()).map(n => {
-        const isOrigin = originId && n.name === originId;
-        const isTarget = targetId && n.name === targetId;
-        const color = isOrigin ? "#ff3b30" : isTarget ? "#34c759" : "#8aa4c8";
-        const size = isOrigin || isTarget ? 44 : 36;
+        const isColorStart = colorStartId && n.name === colorStartId;
+        const isColorEnd = colorEndId && n.name === colorEndId;
+        const isWinningForwardNode = winningForwardNodeIds.has(n.name);
+        let color = "#3f454d";
+        if (isColorStart) {
+            color = startNodeColor;
+        } else if (isColorEnd) {
+            color = endNodeColor;
+        } else if (isWinningForwardNode) {
+            color = "#b88a2a";
+        }
+        const size = isColorStart || isColorEnd ? 44 : 36;
         return {
             id: n.name,
             name: nodeShortNameById.get(n.name) || n.name,
@@ -104,7 +228,10 @@ async function loadTraceroute() {
                 fontWeight: "bold"
             },
             tooltip: {
-                formatter: () => nodeLongNameById.get(n.name) || n.name
+                formatter: () => {
+                    const longName = nodeLongNameById.get(n.name) || n.name;
+                    return longName === n.name ? n.name : `${longName} (${n.name})`;
+                }
             }
         };
     });

meshview/traceroute.py

@@ -0,0 +1,55 @@
+"""Assemble node paths from Meshtastic RouteDiscovery payloads.
+
+A traceroute response carries the endpoints reversed with respect to the path
+being traced: `to` is the node that started the traceroute and `from` is the node
+that answered. Only the response direction is affected, so both orientations are
+handled here rather than at each call site.
+"""
+
+# Firmware inserts this into route[] as a placeholder for a hop it could not identify.
+NODENUM_BROADCAST = 0xFFFFFFFF
+
+
+def return_is_complete(route):
+    """True when the response was seen after it reached the node that started it.
+
+    Relays append both their ID and their SNR, but the final recipient appends only its
+    SNR, so one spare SNR entry means the return trip finished. Gateways usually report
+    a response that is still in flight, where the remaining hops are not known yet.
+    """
+    return len(route.snr_back) == len(route.route_back) + 1
+
+
+def forward_path(from_node_id, to_node_id, hops, done, gateway_node_id=None):
+    """Path towards the traced node, starting at the node that began the traceroute."""
+    if done:
+        return [node for node in (to_node_id, *hops, from_node_id) if node is not None]
+
+    path = [node for node in (from_node_id, *hops) if node is not None]
+    # Some nodes add themselves to the route before uplinking.
+    if gateway_node_id is not None and (not path or path[-1] != gateway_node_id):
+        path.append(gateway_node_id)
+    return path
+
+
+def return_path(from_node_id, to_node_id, hops, complete):
+    """Path back from the traced node, starting at the node that answered."""
+    path = [node for node in (from_node_id, *hops) if node is not None]
+    if complete and to_node_id is not None:
+        path.append(to_node_id)
+    return path
+
+
+def edges(path):
+    """Consecutive pairs of a path, skipping links that run through an unknown hop."""
+    for a, b in zip(path, path[1:], strict=False):
+        if NODENUM_BROADCAST in (a, b):
+            continue
+        yield a, b
+
+
+def endpoints(from_node_id, to_node_id, done):
+    """The (initiator, target) of the traceroute, regardless of packet direction."""
+    if done:
+        return to_node_id, from_node_id
+    return from_node_id, to_node_id

meshview/web.py

@@ -16,8 +16,9 @@ from google.protobuf.message import Message
 from jinja2 import Environment, PackageLoader, Undefined, select_autoescape
 from markupsafe import Markup
 
+from meshtastic.protobuf.mesh_pb2 import Routing
 from meshtastic.protobuf.portnums_pb2 import PortNum
-from meshview import config, database, decode_payload, migrations, models, store
+from meshview import config, database, decode_payload, migrations, models, store, traceroute
 from meshview.__version__ import (
     __version_string__,
 )
@@ -53,6 +54,7 @@ class Packet:
     """UI-friendly packet wrapper for templates and API payloads."""
 
     id: int
+    packet_id: int | None
     from_node_id: int
     from_node: models.Node
     to_node_id: int
@@ -80,6 +82,17 @@ class Packet:
 
         if payload is None:
             text_payload = "Did not decode"
+        elif (
+            packet.portnum == PortNum.ROUTING_APP
+            and isinstance(payload, Routing)
+            and payload.WhichOneof("variant") == "error_reason"
+        ):
+            # error_reason NONE is the implicit ACK, not a failure.
+            text_payload = (
+                "ACK"
+                if payload.error_reason == Routing.Error.NONE
+                else f"NAK: {Routing.Error.Name(payload.error_reason)}"
+            )
         elif isinstance(payload, Message):
             text_payload = text_format.MessageToString(payload)
         elif packet.portnum == PortNum.TEXT_MESSAGE_APP and packet.to_node_id != 0xFFFFFFFF:
@@ -99,8 +112,13 @@ class Packet:
                     f'<a href="https://www.google.com/maps/search/?api=1&query={payload.latitude_i * 1e-7},{payload.longitude_i * 1e-7}" target="_blank">map</a>'
                 )
 
+        packet_id = None
+        if mesh_packet and mesh_packet.id:
+            packet_id = mesh_packet.id
+
         return cls(
             id=packet.id,
+            packet_id=packet_id,
             from_node=packet.from_node,
             from_node_id=packet.from_node_id,
             to_node=packet.to_node,
@@ -223,8 +241,11 @@ async def serve_page(request):
     if not page.endswith(".html"):
         page = f"{page}.html"
 
-    html_file = pathlib.Path(__file__).parent / "static" / page
-    if not html_file.exists():
+    # aiohttp lets an encoded slash (%2F) through a single path segment, so resolve the
+    # final path and make sure it is still inside the static directory.
+    static_root = (pathlib.Path(__file__).parent / "static").resolve()
+    html_file = (static_root / page).resolve()
+    if static_root not in html_file.parents or not html_file.is_file():
         raise web.HTTPNotFound(text=f"Page '{page}' not found")
 
     content = html_file.read_text(encoding="utf-8")
@@ -389,14 +410,17 @@ async def graph_traceroute(request):
         route = decode_payload.decode_payload(PortNum.TRACEROUTE_APP, tr.route)
         if route is None:
             continue
-        path = [packet.from_node_id]
-        path.extend(route.route)
+        path = traceroute.forward_path(
+            packet.from_node_id,
+            packet.to_node_id,
+            route.route,
+            tr.done,
+            tr.gateway_node_id,
+        )
+        if not path:
+            continue
         if tr.done:
-            dest = packet.to_node_id
-            path.append(packet.to_node_id)
-        elif path[-1] != tr.gateway_node_id:
-            # It seems some nodes add them self to the list before uplinking
-            path.append(tr.gateway_node_id)
+            dest = path[-1]
 
         if not tr.done and tr.gateway_node_id not in node_seen_time and tr.import_time_us:
             node_seen_time[path[-1]] = tr.import_time_us

meshview/web_api/api.py

@@ -1,16 +1,18 @@
 """API endpoints for MeshView."""
 
+import asyncio
 import datetime
 import json
 import logging
 import math
 import os
+import re
 
 from aiohttp import web
 from sqlalchemy import func, select
 
 from meshtastic.protobuf.portnums_pb2 import PortNum
-from meshview import database, decode_payload, store
+from meshview import database, decode_payload, store, traceroute
 from meshview.__version__ import __version__, _git_revision_short, get_version_info
 from meshview.config import CONFIG
 from meshview.models import DailySnapshot, Node
@@ -37,6 +39,99 @@ _LANG_CACHE = {}
 # Create dedicated route table for API endpoints
 routes = web.RouteTableDef()
 
+# Language codes are short identifiers such as "en", "es", "pt-BR".
+LANG_CODE_RE = re.compile(r"^[A-Za-z]{2,3}(?:[-_][A-Za-z0-9]{2,8})?$")
+
+
+def _config_bool(section: str, key: str, default: bool = False) -> bool:
+    return str(CONFIG.get(section, {}).get(key, default)).lower() in ("1", "true", "yes", "on")
+
+
+def _config_int(section: str, key: str, default: int = 0) -> int:
+    try:
+        return int(CONFIG.get(section, {}).get(key, default))
+    except (TypeError, ValueError):
+        return default
+
+
+def _config_str(section: str, key: str, default: str = "") -> str:
+    value = CONFIG.get(section, {}).get(key, default)
+    return str(value) if value is not None else default
+
+
+def _cleanup_status_file() -> str:
+    cleanup_logfile = CONFIG.get("logging", {}).get("db_cleanup_logfile", "dbcleanup.log")
+    path_without_extension, _ = os.path.splitext(cleanup_logfile)
+    return f"{path_without_extension}.status.json"
+
+
+def _backup_status_file() -> str:
+    cleanup_logfile = CONFIG.get("logging", {}).get("db_cleanup_logfile", "dbcleanup.log")
+    return os.path.join(os.path.dirname(cleanup_logfile), "dbbackup.status.json")
+
+
+def _get_cleanup_health() -> dict:
+    cleanup_health = {
+        "enabled": _config_bool("cleanup", "enabled", False),
+        "days_to_keep": _config_int("cleanup", "days_to_keep", 14),
+        "scheduled_time": (
+            f"{_config_int('cleanup', 'hour', 2):02d}:{_config_int('cleanup', 'minute', 0):02d}"
+        ),
+        "vacuum": _config_bool("cleanup", "vacuum", False),
+        "status": "disabled",
+    }
+
+    if cleanup_health["enabled"]:
+        cleanup_health["status"] = "unknown"
+
+    status_file = _cleanup_status_file()
+    cleanup_health["status_file"] = status_file
+    if not os.path.exists(status_file):
+        return cleanup_health
+
+    try:
+        with open(status_file, encoding="utf-8") as f:
+            last_run = json.load(f)
+    except Exception as e:
+        cleanup_health["status"] = "error"
+        cleanup_health["error"] = f"Unable to read cleanup status: {e}"
+        return cleanup_health
+
+    cleanup_health["status"] = last_run.get("status", cleanup_health["status"])
+    cleanup_health["last_run"] = last_run
+    return cleanup_health
+
+
+def _get_backup_health() -> dict:
+    backup_hour = _config_int("cleanup", "backup_hour", _config_int("cleanup", "hour", 2))
+    backup_minute = _config_int("cleanup", "backup_minute", _config_int("cleanup", "minute", 0))
+    backup_health = {
+        "enabled": _config_bool("cleanup", "backup_enabled", False),
+        "backup_dir": _config_str("cleanup", "backup_dir", "./backups"),
+        "scheduled_time": f"{backup_hour:02d}:{backup_minute:02d}",
+        "status": "disabled",
+    }
+
+    if backup_health["enabled"]:
+        backup_health["status"] = "unknown"
+
+    status_file = _backup_status_file()
+    backup_health["status_file"] = status_file
+    if not os.path.exists(status_file):
+        return backup_health
+
+    try:
+        with open(status_file, encoding="utf-8") as f:
+            last_run = json.load(f)
+    except Exception as e:
+        backup_health["status"] = "error"
+        backup_health["error"] = f"Unable to read backup status: {e}"
+        return backup_health
+
+    backup_health["status"] = last_run.get("status", backup_health["status"])
+    backup_health["last_run"] = last_run
+    return backup_health
+
 
 def _haversine_km(lat1, lon1, lat2, lon2):
     r = 6371.0
@@ -60,6 +155,20 @@ def _bearing_deg(lat1, lon1, lat2, lon2):
 
 OBSERVED_MAX_DISTANCE_KM = 50.0
 
+# Bounds for /api/coverage query parameters. The ITM loop runs once per (distance step x
+# bearing), so the request size must be capped or a single call can hold the CPU for minutes.
+COVERAGE_LIMITS = {
+    "freq_mhz": (1.0, 10_000.0),
+    "tx_dbm": (-50.0, 60.0),
+    "tx_height_m": (0.0, 1_000.0),
+    "rx_height_m": (0.0, 1_000.0),
+    "radius_km": (1.0, 200.0),
+    "step_km": (0.05, 50.0),
+    "reliability": (0.01, 0.99),
+    "threshold_dbm": (-200.0, 0.0),
+}
+COVERAGE_MAX_STEPS = 4_000  # radius_km / step_km
+
 
 def init_api_module(packet_class, seq_regex, lang_dir):
     """Initialize API module with dependencies from main web module."""
@@ -160,6 +269,8 @@ async def api_packets(request):
             p = Packet.from_model(packet)
             data = {
                 "id": p.id,
+                "storage_id": p.id,
+                "packet_id": p.packet_id,
                 "from_node_id": p.from_node_id,
                 "to_node_id": p.to_node_id,
                 "portnum": int(p.portnum) if p.portnum is not None else None,
@@ -249,6 +360,8 @@ async def api_packets(request):
         for p in ui_packets:
             packet_dict = {
                 "id": p.id,
+                "storage_id": p.id,
+                "packet_id": p.packet_id,
                 "import_time_us": p.import_time_us,
                 "channel": p.channel,
                 "from_node_id": p.from_node_id,
@@ -536,10 +649,15 @@ async def api_edges(request):
             if route is None or tr.packet is None:
                 continue
 
-            path = [tr.packet.from_node_id] + list(route.route)
-            path.append(tr.packet.to_node_id if tr.done else tr.gateway_node_id)
+            path = traceroute.forward_path(
+                tr.packet.from_node_id,
+                tr.packet.to_node_id,
+                route.route,
+                tr.done,
+                tr.gateway_node_id,
+            )
 
-            for a, b in zip(path, path[1:], strict=False):
+            for a, b in traceroute.edges(path):
                 if (a, b) not in edges:
                     edges[(a, b)] = "traceroute"
                     edges_added_tr += 1
@@ -626,6 +744,11 @@ async def api_config(request):
             "map_bottom_right_lat": get_float(site, "map_bottom_right_lat", 36.0),
             "map_bottom_right_lon": get_float(site, "map_bottom_right_lon", -121.0),
             "map_interval": get_int(site, "map_interval", 3),
+            "map_channel_colors": {
+                channel: color.strip()
+                for channel, color in CONFIG.get("map_channel_colors", {}).items()
+                if re.fullmatch(r"#[0-9a-fA-F]{6}", color.strip())
+            },
             "firehose_interval": get_int(site, "firehose_interval", 3),
             "weekly_net_message": get_str(
                 site, "weekly_net_message", "Weekly Mesh check-in message."
@@ -680,6 +803,10 @@ async def api_lang(request):
     lang_code = request.query.get("lang") or CONFIG.get("site", {}).get("language", "en")
     section = request.query.get("section")
 
+    # The code names a file, so it must stay a plain language code. Anything else (a path,
+    # an absolute filename) falls back to English just like an unknown language does.
+    if not LANG_CODE_RE.match(lang_code):
+        lang_code = "en"
     lang_file = os.path.join(LANG_DIR, f"{lang_code}.json")
     if not os.path.exists(lang_file):
         lang_file = os.path.join(LANG_DIR, "en.json")
@@ -720,6 +847,8 @@ async def health_check(request):
         "timestamp": datetime.datetime.now(datetime.UTC).isoformat(),
         "version": __version__,
         "git_revision": _git_revision_short,
+        "cleanup": _get_cleanup_health(),
+        "backup": _get_backup_health(),
     }
 
     # Check database connectivity
@@ -849,6 +978,7 @@ async def api_traceroute(request):
                 "done": tr.done,
                 "forward_hops": forward_list,
                 "reverse_hops": reverse_list,
+                "reverse_complete": traceroute.return_is_complete(route),
             }
         )
 
@@ -859,6 +989,7 @@ async def api_traceroute(request):
 
     forward_paths = []
     reverse_paths = []
+    forward_complete = set()
     winning_forward_paths = []
     winning_reverse_paths = []
 
@@ -872,11 +1003,11 @@ async def api_traceroute(request):
         if tr["reverse_hops"]:
             reverse_paths.append(r)
 
+        # A response proves the request reached the target, whatever its return trip did.
         if tr["done"]:
-            if tr["forward_hops"]:
-                winning_forward_paths.append(f)
-            if tr["reverse_hops"]:
-                winning_reverse_paths.append(r)
+            forward_complete.add(f)
+            winning_forward_paths.append(f)
+            winning_reverse_paths.append((r, tr["reverse_complete"]))
 
     # Deduplicate
     unique_forward_paths = sorted(set(forward_paths))
@@ -887,30 +1018,26 @@ async def api_traceroute(request):
 
     # Convert for JSON output
     unique_forward_paths_json = [
-        {"path": list(p), "count": forward_counts[p]} for p in unique_forward_paths
+        {"path": list(p), "count": forward_counts[p], "complete": p in forward_complete}
+        for p in unique_forward_paths
     ]
 
     unique_reverse_paths_json = [list(p) for p in unique_reverse_paths]
 
     from_node_id = packet.from_node_id
     to_node_id = packet.to_node_id
+    # Winning paths are only built from responses, which carry the endpoints reversed.
     winning_forward_with_endpoints = []
-    for path in set(winning_forward_paths):
-        full_path = list(path)
-        if from_node_id is not None and (not full_path or full_path[0] != from_node_id):
-            full_path = [from_node_id, *full_path]
-        if to_node_id is not None and (not full_path or full_path[-1] != to_node_id):
-            full_path = [*full_path, to_node_id]
-        winning_forward_with_endpoints.append(full_path)
+    for hops in dict.fromkeys(winning_forward_paths):
+        path = traceroute.forward_path(from_node_id, to_node_id, hops, done=True)
+        if len(path) > 1:
+            winning_forward_with_endpoints.append(path)
 
     winning_reverse_with_endpoints = []
-    for path in set(winning_reverse_paths):
-        full_path = list(path)
-        if to_node_id is not None and (not full_path or full_path[0] != to_node_id):
-            full_path = [to_node_id, *full_path]
-        if from_node_id is not None and (not full_path or full_path[-1] != from_node_id):
-            full_path = [*full_path, from_node_id]
-        winning_reverse_with_endpoints.append(full_path)
+    for hops, complete in winning_reverse_paths:
+        path = traceroute.return_path(from_node_id, to_node_id, hops, complete)
+        if len(path) > 1 and path not in winning_reverse_with_endpoints:
+            winning_reverse_with_endpoints.append(path)
 
     winning_paths_json = {
         "forward": winning_forward_with_endpoints,
@@ -920,12 +1047,17 @@ async def api_traceroute(request):
     # --------------------------------------------
     # Final API output
     # --------------------------------------------
+    initiator, target = traceroute.endpoints(
+        from_node_id, to_node_id, any(tr["done"] for tr in tr_groups)
+    )
     return web.json_response(
         {
             "packet": {
                 "id": packet.id,
                 "from": packet.from_node_id,
                 "to": packet.to_node_id,
+                "initiator": initiator,
+                "target": target,
                 "channel": packet.channel,
             },
             "traceroute_packets": tr_groups,
@@ -1115,12 +1247,19 @@ async def api_coverage(request):
         if value is None:
             return default
         try:
-            return float(value)
+            parsed = float(value)
         except ValueError as exc:
             raise web.HTTPBadRequest(
                 text=json.dumps({"error": f"{name} must be a number"}),
                 content_type="application/json",
             ) from exc
+        low, high = COVERAGE_LIMITS[name]
+        if not math.isfinite(parsed) or not (low <= parsed <= high):
+            raise web.HTTPBadRequest(
+                text=json.dumps({"error": f"{name} must be between {low} and {high}"}),
+                content_type="application/json",
+            )
+        return parsed
 
     try:
         freq_mhz = parse_float("freq_mhz", 907.0)
@@ -1134,6 +1273,11 @@ async def api_coverage(request):
     except web.HTTPBadRequest as exc:
         raise exc
 
+    if radius_km / step_km > COVERAGE_MAX_STEPS:
+        return web.json_response(
+            {"error": f"radius_km / step_km must not exceed {COVERAGE_MAX_STEPS}"}, status=400
+        )
+
     node = await store.get_node(node_id)
     if not node or not node.last_lat or not node.last_long:
         return web.json_response({"error": "Node not found or missing location"}, status=404)
@@ -1142,8 +1286,11 @@ async def api_coverage(request):
     lon = node.last_long * 1e-7
 
     mode = request.query.get("mode", "perimeter")
+    # The ITM model is pure CPU work; keep it off the event loop so other requests
+    # are still served while it runs.
     if mode == "perimeter":
-        perimeter = compute_perimeter(
+        perimeter = await asyncio.to_thread(
+            compute_perimeter,
             lat=round(lat, 7),
             lon=round(lon, 7),
             freq_mhz=round(freq_mhz, 3),
@@ -1159,7 +1306,8 @@ async def api_coverage(request):
             {"mode": "perimeter", "threshold_dbm": threshold_dbm, "perimeter": perimeter}
         )
 
-    points = compute_coverage(
+    points = await asyncio.to_thread(
+        compute_coverage,
         lat=round(lat, 7),
         lon=round(lon, 7),
         freq_mhz=round(freq_mhz, 3),

sample.config.ini

@@ -59,6 +59,19 @@ firehose_interal=3
 weekly_net_message = Weekly Mesh check-in. We will keep it open on every Wednesday from 5:00pm for checkins. The message format should be (LONG NAME) - (CITY YOU ARE IN) #BayMeshNet.
 net_tag = #BayMeshNet
 
+# Map channel color overrides. Use six-digit hex colors; other channels get generated colors.
+# Names ignore case, spaces, hyphens, and underscores. Restart the web server after editing.
+[map_channel_colors]
+LongFast = #4363d8
+LongSlow = #000075
+VeryLongSlow = #808000
+MediumSlow = #008080
+MediumFast = #e6194b
+ShortSlow = #911eb4
+ShortFast = #f58231
+LongModerate = #46f0f0
+ShortTurbo = #f032e6
+
 # -------------------------
 # MQTT Broker Configuration
 # -------------------------

startdb.py

@@ -33,6 +33,8 @@ file_handler.setLevel(logging.INFO)
 formatter = logging.Formatter('%(asctime)s [%(levelname)s] %(message)s')
 file_handler.setFormatter(formatter)
 cleanup_logger.addHandler(file_handler)
+cleanup_status_file = str(Path(cleanup_logfile).with_suffix(".status.json"))
+backup_status_file = str(Path(cleanup_logfile).with_name("dbbackup.status.json"))
 
 
 # -------------------------
@@ -49,6 +51,32 @@ def get_int(config, section, key, default=0):
         return default
 
 
+def _rowcount(value):
+    return value if value is not None and value >= 0 else None
+
+
+def write_cleanup_status(status: dict) -> None:
+    status_path = Path(cleanup_status_file)
+    tmp_path = status_path.with_suffix(f"{status_path.suffix}.tmp")
+    try:
+        with open(tmp_path, "w", encoding="utf-8") as f:
+            json.dump(status, f, indent=2)
+        tmp_path.replace(status_path)
+    except Exception as e:
+        cleanup_logger.warning(f"Failed to write cleanup status file: {e}")
+
+
+def write_backup_status(status: dict) -> None:
+    status_path = Path(backup_status_file)
+    tmp_path = status_path.with_suffix(f"{status_path.suffix}.tmp")
+    try:
+        with open(tmp_path, "w", encoding="utf-8") as f:
+            json.dump(status, f, indent=2)
+        tmp_path.replace(status_path)
+    except Exception as e:
+        cleanup_logger.warning(f"Failed to write backup status file: {e}")
+
+
 # -------------------------
 # Shared DB lock
 # -------------------------
@@ -66,19 +94,40 @@ async def backup_database(database_url: str, backup_dir: str = ".") -> None:
         database_url: SQLAlchemy connection string
         backup_dir: Directory to store backups (default: current directory)
     """
+    backup_status = {
+        "status": "running",
+        "started_at": datetime.datetime.now(datetime.UTC).isoformat(),
+        "completed_at": None,
+        "backup_dir": backup_dir,
+        "database_path": None,
+        "backup_file": None,
+        "original_size_bytes": None,
+        "compressed_size_bytes": None,
+        "compression_percent": None,
+        "error": None,
+    }
+    write_backup_status(backup_status)
+
     try:
         url = make_url(database_url)
         if not url.drivername.startswith("sqlite"):
             cleanup_logger.warning("Backup only supported for SQLite databases")
+            backup_status["status"] = "unsupported"
+            backup_status["error"] = "Backup only supported for SQLite databases"
             return
 
         if not url.database or url.database == ":memory:":
             cleanup_logger.error("Could not extract database path from connection string")
+            backup_status["status"] = "error"
+            backup_status["error"] = "Could not extract database path from connection string"
             return
 
         db_file = Path(url.database)
+        backup_status["database_path"] = str(db_file)
         if not db_file.exists():
             cleanup_logger.error(f"Database file not found: {db_file}")
+            backup_status["status"] = "error"
+            backup_status["error"] = f"Database file not found: {db_file}"
             return
 
         # Create backup directory if it doesn't exist
@@ -89,6 +138,7 @@ async def backup_database(database_url: str, backup_dir: str = ".") -> None:
         timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
         backup_filename = f"{db_file.stem}_backup_{timestamp}.db.gz"
         backup_file = backup_path / backup_filename
+        backup_status["backup_file"] = str(backup_file)
 
         cleanup_logger.info(f"Creating backup: {backup_file}")
 
@@ -98,9 +148,15 @@ async def backup_database(database_url: str, backup_dir: str = ".") -> None:
                 shutil.copyfileobj(f_in, f_out)
 
         # Get file sizes for logging
-        original_size = db_file.stat().st_size / (1024 * 1024)  # MB
-        compressed_size = backup_file.stat().st_size / (1024 * 1024)  # MB
+        original_size_bytes = db_file.stat().st_size
+        compressed_size_bytes = backup_file.stat().st_size
+        original_size = original_size_bytes / (1024 * 1024)  # MB
+        compressed_size = compressed_size_bytes / (1024 * 1024)  # MB
         compression_ratio = (1 - compressed_size / original_size) * 100 if original_size > 0 else 0
+        backup_status["original_size_bytes"] = original_size_bytes
+        backup_status["compressed_size_bytes"] = compressed_size_bytes
+        backup_status["compression_percent"] = round(compression_ratio, 1)
+        backup_status["status"] = "ok"
 
         cleanup_logger.info(
             f"Backup created successfully: {backup_file.name} "
@@ -110,6 +166,11 @@ async def backup_database(database_url: str, backup_dir: str = ".") -> None:
 
     except Exception as e:
         cleanup_logger.error(f"Error creating database backup: {e}")
+        backup_status["status"] = "error"
+        backup_status["error"] = str(e)
+    finally:
+        backup_status["completed_at"] = datetime.datetime.now(datetime.UTC).isoformat()
+        write_backup_status(backup_status)
 
 
 # -------------------------
@@ -179,6 +240,24 @@ async def daily_cleanup_at(
         ).replace(tzinfo=None)
         cutoff_us = int(cutoff_dt.timestamp() * 1_000_000)
         cleanup_logger.info(f"Running cleanup for records older than {cutoff_dt.isoformat()}...")
+        rows_deleted = {
+            "packet": None,
+            "packet_seen": None,
+            "traceroute": None,
+            "node": None,
+        }
+        cleanup_status = {
+            "status": "running",
+            "started_at": datetime.datetime.now(datetime.UTC).isoformat(),
+            "completed_at": None,
+            "cutoff_at": cutoff_dt.replace(tzinfo=datetime.UTC).isoformat(),
+            "days_to_keep": days_to_keep,
+            "vacuum_requested": vacuum_db,
+            "vacuum_completed": False,
+            "rows_deleted": rows_deleted,
+            "error": None,
+        }
+        write_cleanup_status(cleanup_status)
 
         try:
             async with db_lock:  # Pause ingestion
@@ -191,6 +270,7 @@ async def daily_cleanup_at(
                     result = await session.execute(
                         delete(models.Packet).where(models.Packet.import_time_us < cutoff_us)
                     )
+                    rows_deleted["packet"] = _rowcount(result.rowcount)
                     cleanup_logger.info(f"Deleted {result.rowcount} rows from Packet")
 
                     # -------------------------
@@ -201,6 +281,7 @@ async def daily_cleanup_at(
                             models.PacketSeen.import_time_us < cutoff_us
                         )
                     )
+                    rows_deleted["packet_seen"] = _rowcount(result.rowcount)
                     cleanup_logger.info(f"Deleted {result.rowcount} rows from PacketSeen")
 
                     # -------------------------
@@ -211,6 +292,7 @@ async def daily_cleanup_at(
                             models.Traceroute.import_time_us < cutoff_us
                         )
                     )
+                    rows_deleted["traceroute"] = _rowcount(result.rowcount)
                     cleanup_logger.info(f"Deleted {result.rowcount} rows from Traceroute")
 
                     # -------------------------
@@ -219,6 +301,7 @@ async def daily_cleanup_at(
                     result = await session.execute(
                         delete(models.Node).where(models.Node.last_seen_us < cutoff_us)
                     )
+                    rows_deleted["node"] = _rowcount(result.rowcount)
                     cleanup_logger.info(f"Deleted {result.rowcount} rows from Node")
 
                     await session.commit()
@@ -228,14 +311,21 @@ async def daily_cleanup_at(
                     async with mqtt_database.engine.begin() as conn:
                         await conn.exec_driver_sql("VACUUM;")
                     cleanup_logger.info("VACUUM completed.")
+                    cleanup_status["vacuum_completed"] = True
                 elif vacuum_db:
                     cleanup_logger.info("VACUUM skipped (not supported for this database).")
 
                 cleanup_logger.info("Cleanup completed successfully.")
                 cleanup_logger.info("Ingestion resumed after cleanup.")
+                cleanup_status["status"] = "ok"
 
         except Exception as e:
             cleanup_logger.error(f"Error during cleanup: {e}")
+            cleanup_status["status"] = "error"
+            cleanup_status["error"] = str(e)
+        finally:
+            cleanup_status["completed_at"] = datetime.datetime.now(datetime.UTC).isoformat()
+            write_cleanup_status(cleanup_status)
 
 
 # -------------------------
@@ -248,11 +338,16 @@ async def load_database_from_mqtt(
     mqtt_user: str | None = None,
     mqtt_passwd: str | None = None,
 ):
+    logger = logging.getLogger(__name__)
     async for topic, env in mqtt_reader.get_topic_envelopes(
         mqtt_server, mqtt_port, topics, mqtt_user, mqtt_passwd
     ):
         async with db_lock:  # Block if cleanup is running
-            await mqtt_store.process_envelope(topic, env)
+            try:
+                await mqtt_store.process_envelope(topic, env)
+            except Exception:
+                # One bad or unexpected message must not stop ingestion for everyone.
+                logger.exception("Failed to store a message from topic %s; skipping it", topic)
 
 
 # -------------------------