perf: force timestamp range scan for windowed packet-stats GROUP BYs

The GROUP BY type and GROUP BY drop_reason sub-queries in
get_packet_stats (and get_packet_type_stats) filter by a time window but
let the planner pick idx_packets_type / idx_packets_transmitted to get
grouping for free. It then heap-checks the timestamp filter across the
entire table, turning a bounded window into a full scan: on a 1.5M-row
packets table the 24h stats load spent ~4.8s (type) + ~2.3s
(drop_reason) instead of ~0.1s each.

Pin these to idx_packets_timestamp with INDEXED BY so they range-scan the
window (~50k rows) and group via a small temp b-tree. Verified on the
live DB: 4.80s -> 0.10s and 2.32s -> 0.10s. Unlike a covering index this
adds no write-path cost on the packet-insert hot path.
This commit is contained in:
agessaman
2026-07-08 00:02:49 -07:00
committed by Rightup
parent 676e2cea30
commit e22514882f
+11 -3
View File
@@ -973,10 +973,16 @@ class SQLiteHandler:
(cutoff,),
).fetchone()
# INDEXED BY forces the timestamp range scan. Without it the
# planner picks idx_packets_type / idx_packets_transmitted to get
# grouping for free, then heap-checks the timestamp filter across
# the entire table — turning a bounded window into a full scan
# (~5s vs ~0.1s at 1.5M rows). A small temp b-tree over the
# windowed rows is far cheaper.
types = conn.execute(
"""
SELECT type, COUNT(*) as count
FROM packets
FROM packets INDEXED BY idx_packets_timestamp
WHERE timestamp > ?
GROUP BY type
ORDER BY count DESC
@@ -987,7 +993,7 @@ class SQLiteHandler:
drop_reasons = conn.execute(
"""
SELECT drop_reason, COUNT(*) as count
FROM packets
FROM packets INDEXED BY idx_packets_timestamp
WHERE timestamp > ? AND transmitted = 0 AND drop_reason IS NOT NULL
GROUP BY drop_reason
ORDER BY count DESC
@@ -1314,10 +1320,12 @@ class SQLiteHandler:
with self._connect() as conn:
conn.row_factory = sqlite3.Row
# See get_packet_stats: force the timestamp range scan so the
# windowed GROUP BY doesn't degrade into a full-table scan.
type_rows = conn.execute(
"""
SELECT type, COUNT(*) as count
FROM packets
FROM packets INDEXED BY idx_packets_timestamp
WHERE timestamp > ?
GROUP BY type
""",