feat: HTTP Cache-Control, dashboard TTL bump, and route event-hash dedup

Bundles four independent improvements to API caching and route evaluator
accuracy that all touch the same files in the cache/routes stack.

## HTTP Cache-Control headers (api_cache_control_enabled)

The API now emits HTTP Cache-Control on every /api/v1/* response and
handles ETag / If-None-Match -> 304 Not Modified on @cached endpoints.

- @cached GETs: private, max-age=<redis_ttl> + strong SHA-256 ETag.
  Matching If-None-Match returns 304 with empty body.
- Uncached GETs under /api/v1/*: private, max-age=0, must-revalidate.
- POST/PUT/DELETE/PATCH and /health*: no-store.
- All private (several @cached endpoints are role-aware).
- @cached decorator stores new envelope {body, etag} in Redis; legacy
  bare-JSON entries still read (hashed on the fly) and auto-migrate on
  the next miss.
- X-Cache: HIT|MISS observability header is always emitted.
- Kill switch: API_CACHE_CONTROL_ENABLED (default true).

## Dashboard cache TTL bump (30s -> 300s)

REDIS_CACHE_TTL_DASHBOARD default raised from 30s to 300s. Covers all
/dashboard/* endpoints and /api/v1/routes/{id}/history. These return
trend/aggregation data where minute-level staleness is invisible but
every MISS costs seconds of SQL/aggregation. Also resolves reported
short-TTL behaviour on the per-route healthchart (which shares this
setting). Operators with explicit env var: unchanged. Old Redis entries
expire naturally within <=30s; no flush needed.

## Route recent-packets hop truncation

In the routes page expanded card, path hops are now truncated to
2 + ellipsis + 2 when length > 5, matching the packet-detail page
styling. Reuses the existing packets.hops_hidden i18n key as the
tooltip on the ellipsis badge.

## Route event-hash deduplication

A single advert/message/telemetry/trace retransmitted or flooded
through the mesh produces one RawPacket per on-air copy, each with a
fresh wire packet_hash. Counting those as distinct matches let a single
underlying event satisfy packet_count_threshold within seconds and
biased route health.

- Add nullable event_hash column to raw_packets and packet_path_hops
  (alembic 6b3430fd84f4). No backfill; legacy rows keep NULL and the
  evaluator falls back to wire packet_hash until they age out of
  window_hours.
- Subscriber denormalizes the structured event's event_hash onto the
  captured raw packet after handler dispatch.
- Route evaluator prefers event_hash when set, collapsing all
  receptions of the same underlying event into one match.
This commit is contained in:
Louis King committed 2026-07-18 12:43:01 +01:00
1 parent 412bb1207b
commit 3810e029ce
26 files changed
+1587 -129

No files matched your search

@@ -0,0 +1,63 @@
"""add event_hash to raw packets and path hops
Revision ID: 6b3430fd84f4
Revises: ec40c67c8c83
Create Date: 2026-07-18 12:00:00.000000+00:00
Adds a nullable ``event_hash`` column to ``raw_packets`` and
``packet_path_hops``. The column denormalizes the underlying structured
event's identity (advertisement / message / telemetry / trace
``event_hash``) so the route evaluator can deduplicate matches by their
underlying event rather than by per-transmission wire hash.
Background: a single advert or message retransmitted (or flooded) through
the mesh produces one ``RawPacket`` row per on-air copy, each with a
fresh wire ``packet_hash``. Counting those as distinct matches let a
single underlying event satisfy ``packet_count_threshold`` within seconds
and bias the route's health. Joining through ``event_hash`` collapses
all receptions of the same underlying event into one match.
Rows captured before this migration (and any unclassified wire packets)
keep ``event_hash IS NULL``; the evaluator falls back to the wire
``packet_hash`` for those, preserving today's behaviour until they age
out of the configured ``window_hours``. No data backfill is performed.
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "6b3430fd84f4"
down_revision: Union[str, None] = "ec40c67c8c83"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
with op.batch_alter_table("raw_packets", schema=None) as batch_op:
batch_op.add_column(
sa.Column("event_hash", sa.String(length=32), nullable=True)
)
batch_op.create_index("ix_raw_packets_event_hash", ["event_hash"], unique=False)
with op.batch_alter_table("packet_path_hops", schema=None) as batch_op:
batch_op.add_column(
sa.Column("event_hash", sa.String(length=32), nullable=True)
)
batch_op.create_index(
"ix_packet_path_hops_event_hash_received_at",
["event_hash", "received_at"],
unique=False,
)
def downgrade() -> None:
with op.batch_alter_table("packet_path_hops", schema=None) as batch_op:
batch_op.drop_index("ix_packet_path_hops_event_hash_received_at")
batch_op.drop_column("event_hash")
with op.batch_alter_table("raw_packets", schema=None) as batch_op:
batch_op.drop_index("ix_raw_packets_event_hash")
batch_op.drop_column("event_hash")