perf: answer /api/status with one targeted query

/api/status is polled by every open page (on load, every 60s, and on
visibility resume) and cost ~266-450ms in-container while doing no device
I/O at all — check_connection() is just an attribute read, and /health
returns in ~1ms, so all of it was SQLite.

Two calls were over-fetching:

- get_stats() ran COUNT(*) over 11 tables to answer a question about 2.
  COUNT(*) FROM echoes alone was ~200ms over 19k rows and the result was
  discarded. That table only grows, so the endpoint kept getting slower.
- get_channel_messages(limit=1) was a SELECT * fetching every column,
  raw_packet included, to read one timestamp. ORDER BY timestamp has no
  usable index (idx_cm_channel_ts leads with channel_idx), so the plan
  was a full SCAN through two temp B-trees. MAX(timestamp) is served off
  that index as a covering scan instead.

Replaced with Database.get_status_summary(): three scalar subqueries on
one connection. ~371ms -> ~46ms for the DB work; endpoint median ~57ms
in-container, ~78ms from the page. Response verified byte-identical.

This does not address the multi-tab congestion — at one poll per tab per
60s /api/status was never that cause — but it removes a recurring cost
and re-baselines the probe those measurements are taken with.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
MarekWo
2026-07-31 08:35:53 +02:00
parent 1e82a1ed7c
commit ab64ef72f2
3 changed files with 33 additions and 6 deletions
+29
View File
@@ -1480,6 +1480,35 @@ class Database:
# Maintenance
# ================================================================
def get_status_summary(self) -> Dict[str, Any]:
"""Message count + newest channel timestamp, for /api/status.
/api/status is polled by every open tab, so it gets a dedicated query
instead of reusing get_stats() + get_channel_messages(limit=1):
- get_stats() counts 11 tables to answer a question about 2. The
COUNT(*) over echoes alone costs ~200ms on a 20k-row table and its
result was discarded — and echoes only ever grow.
- get_channel_messages(limit=1) was a `SELECT *` whose ORDER BY
timestamp has no usable index (idx_cm_channel_ts leads with
channel_idx), so it scanned the table through two temp B-trees and
materialized every column, raw_packet included, to read one field.
MAX(timestamp) is served straight off that index instead.
Together with folding both into a single connection this is ~46ms
where the pair cost ~371ms.
"""
with self._connect() as conn:
row = conn.execute(
"""SELECT (SELECT COUNT(*) FROM channel_messages) AS channel_count,
(SELECT COUNT(*) FROM direct_messages) AS dm_count,
(SELECT MAX(timestamp) FROM channel_messages) AS latest_timestamp"""
).fetchone()
return {
'message_count': (row['channel_count'] or 0) + (row['dm_count'] or 0),
'latest_message_timestamp': row['latest_timestamp'],
}
def get_stats(self) -> Dict[str, Any]:
"""Get row counts for all tables."""
tables = ['device', 'contacts', 'channels', 'channel_messages',
+3 -6
View File
@@ -887,12 +887,9 @@ def get_status():
message_count = 0
latest_timestamp = None
if db:
stats = db.get_stats()
message_count = stats.get('channel_messages', 0) + stats.get('direct_messages', 0)
# Get latest channel message timestamp
recent = db.get_channel_messages(limit=1)
if recent:
latest_timestamp = recent[0].get('timestamp')
summary = db.get_status_summary()
message_count = summary['message_count']
latest_timestamp = summary['latest_message_timestamp']
else:
message_count = parser.count_messages()
latest = parser.get_latest_message()
+1
View File
@@ -14,6 +14,7 @@ For deep technical notes, see [architecture.md](architecture.md). For the full g
- **Messages no longer go missing after the app has been in the background.** Coming back to a minimised app — or to a phone that had been asleep — could show a chat that quietly stopped at whatever message arrived last before the screen went off, with everything since then missing until the app was force-stopped and reopened. New messages reach an open page over a live connection, and Android tears that connection down while the app sits in the background; nothing then went back to ask the server what had been missed. Now every way back from a gap re-reads the list: the connection coming back, the app returning to the foreground, and a heartbeat that notices when the page has been frozen. The same applies to direct messages, and to a browser tab that lost its network for a while.
- **A Refresh item in the menu.** The browser's pull-to-refresh has no equivalent in the Android app, so there is now a **Refresh** entry at the top of the menu that reloads the messages from the server on demand — in the app, and everywhere else too.
- **The connection-status check is about five times cheaper.** Every open page asks the server how the mesh device is doing — on load, once a minute after that, and each time you come back to a tab you had left. Answering that took roughly a quarter of a second, almost none of it spent on the device itself: the server was counting the rows of every table in the database to report two numbers, and the row it needed for the "last message" timestamp was found by reading the entire message table and sorting it. The heaviest part, counting the radio-echo records, was thrown away unused — and that table only ever grows, so the check was getting slower the longer an instance had been running. It now asks for exactly the three values it needs, in one go. This is a smaller effect than the bundling below and it does not change what you see on screen; it does take a recurring cost off the server on every open page.
- **Far less load on the server while you have messages on screen.** Whenever new radio traffic came in, the page asked the server about every message on screen separately — hundreds of individual requests at a time, repeated every few seconds, and the same messages over and over. On a busy channel that was thousands of requests a minute from a single tab, which left the server little room to answer anything else; the worst of it looked like the mesh device had dropped off, with the message list stuck on "Loading messages…" and the status on "Connecting…", while the device was connected the whole time. Those requests are now bundled into one. A page that needed hundreds of requests per update now needs a single one. Keeping mc-webui open in several windows at once is still not recommended — that can still slow things down for a different reason — but the app is considerably lighter on the server than it was.
---