mirror of
https://github.com/jorijn/meshcore-stats.git
synced 2026-05-04 04:22:49 +02:00
* test: add comprehensive pytest test suite with 95% coverage Add full unit and integration test coverage for the meshcore-stats project: - 1020 tests covering all modules (db, charts, html, reports, client, etc.) - 95.95% code coverage with pytest-cov (95% threshold enforced) - GitHub Actions CI workflow for automated testing on push/PR - Proper mocking of external dependencies (meshcore, serial, filesystem) - SVG snapshot infrastructure for chart regression testing - Integration tests for collection and rendering pipelines Test organization: - tests/charts/: Chart rendering and statistics - tests/client/: MeshCore client and connection handling - tests/config/: Environment and configuration parsing - tests/database/: SQLite operations and migrations - tests/html/: HTML generation and Jinja templates - tests/reports/: Report generation and formatting - tests/retry/: Circuit breaker and retry logic - tests/unit/: Pure unit tests for utilities - tests/integration/: End-to-end pipeline tests 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: add test-engineer agent configuration Add project-local test-engineer agent for pytest test development, coverage analysis, and test review tasks. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: comprehensive test suite review with 956 tests analyzed Conducted thorough review of all 956 test cases across 47 test files: - Unit Tests: 338 tests (battery, metrics, log, telemetry, env, charts, html, reports, formatters) - Config Tests: 53 tests (env loading, config file parsing) - Database Tests: 115 tests (init, insert, queries, migrations, maintenance, validation) - Retry Tests: 59 tests (circuit breaker, async retries, factory) - Charts Tests: 76 tests (transforms, statistics, timeseries, rendering, I/O) - HTML Tests: 81 tests (site generation, Jinja2, metrics builders, reports index) - Reports Tests: 149 tests (location, JSON/TXT formatting, aggregation, counter totals) - Client Tests: 63 tests (contacts, connection, meshcore availability, commands) - Integration Tests: 22 tests (reports, collection, rendering pipelines) Results: - Overall Pass Rate: 99.7% (953/956) - 3 tests marked for improvement (empty test bodies in client tests) - 0 tests requiring fixes Key findings documented in test_review/tests.md including quality observations, F.I.R.S.T. principle adherence, and recommendations. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: implement snapshot testing for charts and reports Add comprehensive snapshot testing infrastructure: SVG Chart Snapshots: - Deterministic fixtures with fixed timestamps (2024-01-15 12:00:00) - Tests for gauge/counter metrics in light/dark themes - Empty chart and single-point edge cases - Extended normalize_svg_for_snapshot_full() for reproducible comparisons TXT Report Snapshots: - Monthly/yearly report snapshots for repeater and companion - Empty report handling tests - Tests in tests/reports/test_snapshots.py Infrastructure: - tests/snapshots/conftest.py with shared fixtures - UPDATE_SNAPSHOTS=1 environment variable for regeneration - scripts/generate_snapshots.py for batch snapshot generation Run `UPDATE_SNAPSHOTS=1 pytest tests/charts/test_chart_render.py::TestSvgSnapshots` to generate initial snapshots. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: fix SVG normalization and generate initial snapshots Fix normalize_svg_for_snapshot() to handle: - clipPath IDs like id="p47c77a2a6e" - url(#p...) references - xlink:href="#p..." references - <dc:date> timestamps Generated initial snapshot files: - 7 SVG chart snapshots (gauge, counter, empty, single-point in light/dark) - 6 TXT report snapshots (monthly/yearly for repeater/companion + empty) All 13 snapshot tests now pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: fix SVG normalization to preserve axis rendering The SVG normalization was replacing all matplotlib-generated IDs with the same value, causing duplicate IDs that broke SVG rendering: - Font glyphs, clipPaths, and tick marks all got id="normalized" - References couldn't resolve to the correct elements - X and Y axes failed to render in normalized snapshots Fix uses type-specific prefixes with sequential numbering: - glyph_N for font glyphs (DejaVuSans-XX patterns) - clip_N for clipPath definitions (p[0-9a-f]{8,} patterns) - tick_N for tick marks (m[0-9a-f]{8,} patterns) This ensures all IDs remain unique while still being deterministic for snapshot comparison. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: add coverage and pytest artifacts to gitignore Add .coverage, .coverage.*, htmlcov/, and .pytest_cache/ to prevent test artifacts from being committed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style: fix all ruff lint errors across codebase - Sort and organize imports (I001) - Use modern type annotations (X | Y instead of Union, collections.abc) - Remove unused imports (F401) - Combine nested if statements (SIM102) - Use ternary operators where appropriate (SIM108) - Combine nested with statements (SIM117) - Use contextlib.suppress instead of try-except-pass (SIM105) - Add noqa comments for intentional SIM115 violations (file locks) - Add TYPE_CHECKING import for forward references - Fix exception chaining (B904) All 1033 tests pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: add TDD workflow and pre-commit requirements to CLAUDE.md - Add mandatory test-driven development workflow (write tests first) - Add pre-commit requirements (must run lint and tests before committing) - Document test organization and running commands - Document 95% coverage requirement 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: resolve mypy type checking errors with proper structural fixes - charts.py: Create PeriodConfig dataclass for type-safe period configuration, use mdates.date2num() for matplotlib datetime handling, fix x-axis limits for single-point charts - db.py: Add explicit int() conversion with None handling for SQLite returns - env.py: Add class-level type annotations to Config class - html.py: Add MetricDisplay TypedDict, fix import order, add proper type annotations for table data functions - meshcore_client.py: Add return type annotation Update tests to use new dataclass attribute access and regenerate SVG snapshots. Add mypy step to CLAUDE.md pre-commit requirements. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: cast Jinja2 template.render() to str for mypy Jinja2's type stubs declare render() as returning Any, but it actually returns str. Wrap with str() to satisfy mypy's no-any-return check. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * ci: improve workflow security and reliability - test.yml: Pin all actions by SHA, add concurrency control to cancel in-progress runs on rapid pushes - release-please.yml: Pin action by SHA, add 10-minute timeout - conftest.py: Fix snapshot_base_time to use explicit UTC timezone for consistent behavior across CI and local environments Regenerate SVG snapshots with UTC-aware timestamps. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add mypy command to permissions in settings.local.json * test: add comprehensive script tests with coroutine warning fixes - Add tests/scripts/ with tests for collect_companion, collect_repeater, and render scripts (1135 tests total, 96% coverage) - Fix unawaited coroutine warnings by using AsyncMock properly for async functions and async_context_manager_factory fixture for context managers - Add --cov=scripts to CI workflow and pyproject.toml coverage config - Omit scripts/generate_snapshots.py from coverage (dev utility) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: migrate claude setup to codex skills * feat: migrate dependencies to uv (#31) * fix: run tests through uv * test: fix ruff lint issues in tests Consolidate patch context managers and clean unused imports/variables Use datetime.UTC in snapshot fixtures * test: avoid unawaited async mocks in entrypoint tests * ci: replace codecov with github coverage artifacts Add junit XML output and coverage summary in job output Upload HTML and XML coverage artifacts (3.12 only) on every run --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
270 lines
8.6 KiB
Python
270 lines
8.6 KiB
Python
"""Environment variable parsing and configuration."""
|
|
|
|
import os
|
|
import re
|
|
import warnings
|
|
from pathlib import Path
|
|
|
|
|
|
def _parse_config_value(value: str) -> str:
|
|
"""Parse a shell-style value, handling quotes and inline comments."""
|
|
value = value.strip()
|
|
|
|
if not value:
|
|
return ""
|
|
|
|
# Handle double-quoted strings
|
|
if value.startswith('"'):
|
|
end = value.find('"', 1)
|
|
if end != -1:
|
|
return value[1:end]
|
|
return value[1:] # No closing quote
|
|
|
|
# Handle single-quoted strings
|
|
if value.startswith("'"):
|
|
end = value.find("'", 1)
|
|
if end != -1:
|
|
return value[1:end]
|
|
return value[1:]
|
|
|
|
# Unquoted value - strip inline comments (# preceded by whitespace)
|
|
comment_match = re.search(r"\s+#", value)
|
|
if comment_match:
|
|
value = value[: comment_match.start()]
|
|
|
|
return value.strip()
|
|
|
|
|
|
def _load_config_file() -> None:
|
|
"""Load meshcore.conf if it exists. Env vars take precedence.
|
|
|
|
The config file is expected in the project root (three levels up from this module).
|
|
Scripts should be run from the project directory via cron: cd $MESHCORE && .venv/bin/python ...
|
|
"""
|
|
config_path = Path(__file__).resolve().parent.parent.parent / "meshcore.conf"
|
|
|
|
if not config_path.exists():
|
|
return
|
|
|
|
try:
|
|
with open(config_path, encoding="utf-8") as f:
|
|
for line in f:
|
|
line = line.strip()
|
|
|
|
# Skip comments and empty lines
|
|
if not line or line.startswith("#"):
|
|
continue
|
|
|
|
# Remove 'export ' prefix
|
|
if line.startswith("export "):
|
|
line = line[7:].lstrip()
|
|
|
|
# Must have KEY=value format
|
|
if "=" not in line:
|
|
continue
|
|
|
|
key, _, value = line.partition("=")
|
|
key = key.strip()
|
|
|
|
# Validate key is a valid shell identifier
|
|
if not key or not re.match(r"^[A-Za-z_][A-Za-z0-9_]*$", key):
|
|
continue
|
|
|
|
# Parse value (handles quotes, inline comments)
|
|
value = _parse_config_value(value)
|
|
|
|
# Only set if not already in environment
|
|
if key not in os.environ:
|
|
os.environ[key] = value
|
|
|
|
except (OSError, UnicodeDecodeError) as e:
|
|
warnings.warn(f"Failed to load {config_path}: {e}", stacklevel=2)
|
|
|
|
|
|
# Load config file at module import time, before Config is instantiated
|
|
_load_config_file()
|
|
|
|
|
|
def get_str(key: str, default: str | None = None) -> str | None:
|
|
"""Get string env var."""
|
|
return os.environ.get(key, default)
|
|
|
|
|
|
def get_int(key: str, default: int) -> int:
|
|
"""Get integer env var."""
|
|
val = os.environ.get(key)
|
|
if val is None:
|
|
return default
|
|
try:
|
|
return int(val)
|
|
except ValueError:
|
|
return default
|
|
|
|
|
|
def get_bool(key: str, default: bool = False) -> bool:
|
|
"""Get boolean env var (0/1, true/false, yes/no)."""
|
|
val = os.environ.get(key, "").lower()
|
|
if not val:
|
|
return default
|
|
return val in ("1", "true", "yes", "on")
|
|
|
|
|
|
def get_float(key: str, default: float) -> float:
|
|
"""Get float env var."""
|
|
val = os.environ.get(key)
|
|
if val is None:
|
|
return default
|
|
try:
|
|
return float(val)
|
|
except ValueError:
|
|
return default
|
|
|
|
|
|
def get_path(key: str, default: str) -> Path:
|
|
"""Get path env var, expanding user and making absolute."""
|
|
val = os.environ.get(key, default)
|
|
return Path(val).expanduser().resolve()
|
|
|
|
|
|
class Config:
|
|
"""Configuration loaded from environment variables."""
|
|
|
|
# Connection settings
|
|
mesh_transport: str
|
|
mesh_serial_port: str | None
|
|
mesh_serial_baud: int
|
|
mesh_tcp_host: str | None
|
|
mesh_tcp_port: int
|
|
mesh_ble_addr: str | None
|
|
mesh_ble_pin: str | None
|
|
mesh_debug: bool
|
|
|
|
# Remote repeater identity
|
|
repeater_name: str | None
|
|
repeater_key_prefix: str | None
|
|
repeater_password: str | None
|
|
|
|
# Intervals and timeouts
|
|
companion_step: int
|
|
repeater_step: int
|
|
remote_timeout_s: int
|
|
remote_retry_attempts: int
|
|
remote_retry_backoff_s: int
|
|
remote_cb_fails: int
|
|
remote_cb_cooldown_s: int
|
|
|
|
# Telemetry
|
|
telemetry_enabled: bool
|
|
telemetry_timeout_s: int
|
|
telemetry_retry_attempts: int
|
|
telemetry_retry_backoff_s: int
|
|
|
|
# Paths
|
|
state_dir: Path
|
|
out_dir: Path
|
|
|
|
# Report location metadata
|
|
report_location_name: str | None
|
|
report_location_short: str | None
|
|
report_lat: float
|
|
report_lon: float
|
|
report_elev: float
|
|
report_elev_unit: str | None
|
|
|
|
# Node display names
|
|
repeater_display_name: str | None
|
|
companion_display_name: str | None
|
|
repeater_pubkey_prefix: str | None
|
|
companion_pubkey_prefix: str | None
|
|
repeater_hardware: str | None
|
|
companion_hardware: str | None
|
|
|
|
# Radio configuration
|
|
radio_frequency: str | None
|
|
radio_bandwidth: str | None
|
|
radio_spread_factor: str | None
|
|
radio_coding_rate: str | None
|
|
|
|
def __init__(self) -> None:
|
|
# Connection settings
|
|
self.mesh_transport = get_str("MESH_TRANSPORT", "serial") or "serial"
|
|
self.mesh_serial_port = get_str("MESH_SERIAL_PORT") # None = auto-detect
|
|
self.mesh_serial_baud = get_int("MESH_SERIAL_BAUD", 115200)
|
|
self.mesh_tcp_host = get_str("MESH_TCP_HOST", "localhost")
|
|
self.mesh_tcp_port = get_int("MESH_TCP_PORT", 5000)
|
|
self.mesh_ble_addr = get_str("MESH_BLE_ADDR")
|
|
self.mesh_ble_pin = get_str("MESH_BLE_PIN")
|
|
self.mesh_debug = get_bool("MESH_DEBUG", False)
|
|
|
|
# Remote repeater identity
|
|
self.repeater_name = get_str("REPEATER_NAME")
|
|
self.repeater_key_prefix = get_str("REPEATER_KEY_PREFIX")
|
|
self.repeater_password = get_str("REPEATER_PASSWORD")
|
|
|
|
# Intervals and timeouts
|
|
self.companion_step = get_int("COMPANION_STEP", 60)
|
|
self.repeater_step = get_int("REPEATER_STEP", 900)
|
|
self.remote_timeout_s = get_int("REMOTE_TIMEOUT_S", 10)
|
|
self.remote_retry_attempts = get_int("REMOTE_RETRY_ATTEMPTS", 2)
|
|
self.remote_retry_backoff_s = get_int("REMOTE_RETRY_BACKOFF_S", 4)
|
|
self.remote_cb_fails = get_int("REMOTE_CB_FAILS", 6)
|
|
self.remote_cb_cooldown_s = get_int("REMOTE_CB_COOLDOWN_S", 3600)
|
|
|
|
# Telemetry collection (requires sensor board on repeater)
|
|
self.telemetry_enabled = get_bool("TELEMETRY_ENABLED", False)
|
|
# Separate settings allow tuning if telemetry proves problematic
|
|
# Defaults match status settings - tune down if needed
|
|
self.telemetry_timeout_s = get_int("TELEMETRY_TIMEOUT_S", 10)
|
|
self.telemetry_retry_attempts = get_int("TELEMETRY_RETRY_ATTEMPTS", 2)
|
|
self.telemetry_retry_backoff_s = get_int("TELEMETRY_RETRY_BACKOFF_S", 4)
|
|
|
|
# Paths (defaults are Docker container paths; native installs override via config)
|
|
self.state_dir = get_path("STATE_DIR", "/data/state")
|
|
self.out_dir = get_path("OUT_DIR", "/out")
|
|
|
|
# Report location metadata
|
|
self.report_location_name = get_str(
|
|
"REPORT_LOCATION_NAME", "Your Location"
|
|
)
|
|
self.report_location_short = get_str(
|
|
"REPORT_LOCATION_SHORT", "Your Location"
|
|
)
|
|
self.report_lat = get_float("REPORT_LAT", 0.0)
|
|
self.report_lon = get_float("REPORT_LON", 0.0)
|
|
self.report_elev = get_float("REPORT_ELEV", 0.0)
|
|
self.report_elev_unit = get_str("REPORT_ELEV_UNIT", "m") # "m" or "ft"
|
|
|
|
# Node display names for UI
|
|
self.repeater_display_name = get_str(
|
|
"REPEATER_DISPLAY_NAME", "Repeater Node"
|
|
)
|
|
self.companion_display_name = get_str(
|
|
"COMPANION_DISPLAY_NAME", "Companion Node"
|
|
)
|
|
|
|
# Public key prefixes for display (e.g., "!a1b2c3d4")
|
|
self.repeater_pubkey_prefix = get_str("REPEATER_PUBKEY_PREFIX")
|
|
self.companion_pubkey_prefix = get_str("COMPANION_PUBKEY_PREFIX")
|
|
|
|
# Hardware info for sidebar
|
|
self.repeater_hardware = get_str("REPEATER_HARDWARE", "LoRa Repeater")
|
|
self.companion_hardware = get_str("COMPANION_HARDWARE", "LoRa Node")
|
|
|
|
# Radio configuration (for display in sidebar)
|
|
self.radio_frequency = get_str("RADIO_FREQUENCY", "869.618 MHz")
|
|
self.radio_bandwidth = get_str("RADIO_BANDWIDTH", "62.5 kHz")
|
|
self.radio_spread_factor = get_str("RADIO_SPREAD_FACTOR", "SF8")
|
|
self.radio_coding_rate = get_str("RADIO_CODING_RATE", "CR8")
|
|
|
|
|
|
# Global config instance
|
|
_config: Config | None = None
|
|
|
|
|
|
def get_config() -> Config:
|
|
"""Get or create the global config instance."""
|
|
global _config
|
|
if _config is None:
|
|
_config = Config()
|
|
return _config
|