- Updated byte representations in tests to use lowercase hex format for consistency.
- Reformatted code for better readability, including line breaks and indentation adjustments.
- Consolidated multiple lines into single lines where appropriate to enhance clarity.
- Ensured that all test cases maintain consistent formatting and style across the test suite.
Problem
-------
update_packet_metrics() called rrdtool.info() (cached for 5 s) to get the
RRD's last_update timestamp. rrdtool.info() returns a massive Python dict:
17 data sources × 5 RRAs × ~8 fields each = ~700+ dict entries per call.
tracemalloc showed +10696 new allocations / +251 KB at this exact line,
flagged as "Investigate" in the memory diagnostics dashboard.
The rrdtool.info() approach was also unnecessarily complex: it required a
5-second secondary cache, a _pending_rrd_update buffer, and two extra
instance attributes — all to answer one question ("did we already write
this period?") that we can answer ourselves with a single integer.
Fix
---
Replace _last_rrd_info_cache / _last_rrd_info_time / _pending_rrd_update
with a single self._last_rrd_update: int = 0 that stores the timestamp of
the last successful rrdtool.update() call. The throttle check becomes:
if timestamp <= self._last_rrd_update:
return
On success: self._last_rrd_update = timestamp
Zero dict allocations per call. The only downside vs rrdtool.info() is
that _last_rrd_update resets to 0 on process restart, meaning the first
packet after a restart always triggers a write — correct behaviour.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Problem
-------
rrdtool.fetch() is a blocking C library call that reads 24 hours of RRD
data from disk. The dashboard can call get_data() on every page refresh.
On an SD card each fetch can cost several milliseconds of I/O, and because
the RRD step is 60 seconds the data cannot change more often than that —
any fetch within the same 60-second window returns identical data.
The combined-optimizations branch had a 60-second read cache; rightup's
batching refactor inadvertently removed it. This PR restores it.
Solution
--------
* Add self._get_data_cache: tuple = (0.0, None) to __init__
* In get_data(): set use_cache = (start_time is None and end_time is None)
- if use_cache and cache is < 60 s old: return cached result immediately
- after a successful live fetch with use_cache: store (now, result)
* Explicit start_time / end_time callers always bypass the cache so
fine-grained or historical queries are never stale
Why 60 s TTL?
The RRD step is 60 s, so the database cannot hold a newer sample until
the next step boundary. A 60-second cache is tight enough that the
dashboard always shows data ≤ one step stale, and loose enough that
a burst of refreshes costs one disk read instead of N.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Enhanced readability by adding descriptive suffixes to packet type labels in both rrdtool_handler.py and sqlite_handler.py.
- Aligned packet type definitions with the new naming conventions from pyMC_core, ensuring consistency across the codebase.
- Added `SQLiteHandler` for managing packet and advert storage in SQLite database.
- Implemented `RRDToolHandler` for creating and updating RRD databases for metrics.
- Developed `MQTTHandler` for publishing data to MQTT broker.
- Created `StorageCollector` to integrate SQLite, RRDTool, and MQTT functionalities.
- Added methods for recording packets, adverts, and noise floor data.
- Implemented data retrieval methods for packet statistics, recent packets, and noise floor history.
- Established database schema with appropriate tables and indices for efficient data access.
- Included error handling and logging for database operations and MQTT communications.
Lloyd