# MeshCore Stats A Python-based monitoring system for a MeshCore repeater node and its companion. Collects metrics from both devices, stores them in a SQLite database, and generates a static website with interactive SVG charts and statistics. **Live demo:** [meshcore.jorijn.com](https://meshcore.jorijn.com)

MeshCore Stats Dashboard MeshCore Stats Reports

## Features - **Data Collection** - Collect metrics from companion (local) and repeater (remote) nodes - **Chart Rendering** - Generate interactive SVG charts from the database using matplotlib - **Static Site** - Generate a static HTML website with day/week/month/year views - **Reports** - Generate monthly and yearly statistics reports ## Requirements ### Python Dependencies - Python 3.10+ - meshcore >= 2.2.3 - pyserial >= 3.5 - jinja2 >= 3.1.0 - matplotlib >= 3.8.0 ### System Dependencies - sqlite3 (for database maintenance script) ## Setup ### 1. Create Virtual Environment ```bash cd /path/to/meshcore-stats python3 -m venv .venv source .venv/bin/activate pip install -r requirements.txt ``` ### 2. Configure Copy the example configuration file and customize it: ```bash cp meshcore.conf.example meshcore.conf # Edit meshcore.conf with your settings ``` The configuration file is automatically loaded by the scripts. Key settings to configure: - **Connection**: `MESH_SERIAL_PORT`, `MESH_TRANSPORT` - **Repeater Identity**: `REPEATER_NAME`, `REPEATER_PASSWORD` - **Display Names**: `REPEATER_DISPLAY_NAME`, `COMPANION_DISPLAY_NAME` - **Location**: `REPORT_LOCATION_NAME`, `REPORT_LAT`, `REPORT_LON`, `REPORT_ELEV` - **Hardware Info**: `REPEATER_HARDWARE`, `COMPANION_HARDWARE` - **Radio Config**: `RADIO_FREQUENCY`, `RADIO_BANDWIDTH`, etc. (includes presets for different regions) See `meshcore.conf.example` for all available options with documentation. ## Usage ### Manual Execution ```bash cd /path/to/meshcore-stats source .venv/bin/activate # Collect companion data python scripts/collect_companion.py # Collect repeater data python scripts/collect_repeater.py # Generate static site (includes chart rendering) python scripts/render_site.py # Generate reports python scripts/render_reports.py ``` The configuration is automatically loaded from `meshcore.conf`. ### Cron Setup Add these entries to your crontab (`crontab -e`): ```cron # MeshCore Stats - adjust path as needed MESHCORE=/home/user/meshcore-stats # Every minute: collect companion data * * * * * cd $MESHCORE && flock -w 60 /tmp/meshcore.lock .venv/bin/python scripts/collect_companion.py # Every 15 minutes: collect repeater data 1,16,31,46 * * * * cd $MESHCORE && flock -w 60 /tmp/meshcore.lock .venv/bin/python scripts/collect_repeater.py # Every 5 minutes: render site */5 * * * * cd $MESHCORE && .venv/bin/python scripts/render_site.py # Daily at midnight: generate reports 0 0 * * * cd $MESHCORE && .venv/bin/python scripts/render_reports.py # Monthly at 3 AM on the 1st: database maintenance 0 3 1 * * $MESHCORE/scripts/db_maintenance.sh ``` **Notes:** - `cd $MESHCORE` is required because paths in the config are relative to the project root - `flock` prevents USB serial conflicts when companion and repeater collection overlap ### Docker / Container Usage When running in Docker, you can skip the config file and pass environment variables directly: ```bash docker run -e MESH_SERIAL_PORT=/dev/ttyUSB0 -e REPEATER_NAME="My Repeater" ... ``` Environment variables always take precedence over `meshcore.conf`. ### Serving the Site The static site is generated in the `out/` directory. You can serve it with any web server: ```bash # Simple Python server for testing cd out && python3 -m http.server 8080 # Or configure nginx/caddy to serve the out/ directory ``` ## Project Structure ``` meshcore-stats/ ├── requirements.txt ├── README.md ├── meshcore.conf.example # Example configuration ├── meshcore.conf # Your configuration (create this) ├── src/meshmon/ │ ├── __init__.py │ ├── env.py # Environment variable parsing │ ├── log.py # Logging helper │ ├── meshcore_client.py # MeshCore connection and commands │ ├── db.py # SQLite database module │ ├── retry.py # Retry logic and circuit breaker │ ├── charts.py # Matplotlib SVG chart generation │ ├── html.py # HTML rendering │ ├── reports.py # Report generation │ ├── metrics.py # Metric type definitions │ ├── battery.py # Battery voltage to percentage conversion │ ├── migrations/ # SQL schema migrations │ │ ├── 001_initial_schema.sql │ │ └── 002_eav_schema.sql │ └── templates/ # Jinja2 HTML templates ├── scripts/ │ ├── collect_companion.py # Collect metrics from companion node │ ├── collect_repeater.py # Collect metrics from repeater node │ ├── render_charts.py # Generate SVG charts from database │ ├── render_site.py # Generate static HTML site │ ├── render_reports.py # Generate monthly/yearly reports │ └── db_maintenance.sh # Database VACUUM/ANALYZE ├── data/ │ └── state/ │ ├── metrics.db # SQLite database (WAL mode) │ └── repeater_circuit.json └── out/ # Generated site ├── .htaccess # Apache config (DirectoryIndex, caching) ├── styles.css # Stylesheet ├── chart-tooltip.js # Chart tooltip enhancement ├── day.html # Repeater pages (entry point) ├── week.html ├── month.html ├── year.html ├── companion/ │ ├── day.html │ ├── week.html │ ├── month.html │ └── year.html └── reports/ ├── index.html ├── repeater/ # YYYY/MM reports └── companion/ ``` ## Chart Features Charts are rendered as inline SVG using matplotlib with the following features: - **Theme Support**: Automatic light/dark mode via CSS `prefers-color-scheme` - **Interactive Tooltips**: Hover to see exact values and timestamps - **Data Point Indicator**: Visual marker shows position on the chart line - **Mobile Support**: Touch-friendly tooltips - **Statistics**: Min/Avg/Max values displayed below each chart - **Period Views**: Day, week, month, and year time ranges ## Troubleshooting ### Serial Device Not Found If you see "No serial ports found" or connection fails: 1. Check that your device is connected: ```bash ls -la /dev/ttyUSB* /dev/ttyACM* ``` 2. Check permissions (add user to dialout group): ```bash sudo usermod -a -G dialout $USER # Log out and back in for changes to take effect ``` 3. Try specifying the port explicitly: ```bash export MESH_SERIAL_PORT=/dev/ttyACM0 ``` 4. Check dmesg for device detection: ```bash dmesg | tail -20 ``` ### Repeater Not Found If the script cannot find the repeater contact: 1. The script will print all discovered contacts - check for the correct name 2. Verify REPEATER_NAME matches exactly (case-sensitive) 3. Try using REPEATER_KEY_PREFIX instead with the first 6-12 hex chars of the public key ### Circuit Breaker If repeater collection shows "cooldown active": 1. This is normal after multiple failed remote requests 2. Wait for the cooldown period (default 1 hour) or reset manually: ```bash rm data/state/repeater_circuit.json ``` ## Environment Variables Reference | Variable | Default | Description | |----------|---------|-------------| | **Connection** | | | | `MESH_TRANSPORT` | serial | Connection type: serial, tcp, ble | | `MESH_SERIAL_PORT` | (auto) | Serial port path | | `MESH_SERIAL_BAUD` | 115200 | Baud rate | | `MESH_TCP_HOST` | localhost | TCP host | | `MESH_TCP_PORT` | 5000 | TCP port | | `MESH_BLE_ADDR` | - | BLE device address | | `MESH_BLE_PIN` | - | BLE PIN | | `MESH_DEBUG` | 0 | Enable debug output | | **Repeater Identity** | | | | `REPEATER_NAME` | - | Repeater advertised name | | `REPEATER_KEY_PREFIX` | - | Repeater public key prefix | | `REPEATER_PASSWORD` | - | Repeater login password | | `REPEATER_FETCH_ACL` | 0 | Also fetch ACL from repeater | | **Display Names** | | | | `REPEATER_DISPLAY_NAME` | Repeater Node | Display name for repeater in UI | | `COMPANION_DISPLAY_NAME` | Companion Node | Display name for companion in UI | | **Location** | | | | `REPORT_LOCATION_NAME` | Your Location | Full location name for reports | | `REPORT_LOCATION_SHORT` | Your Location | Short location for sidebar/meta | | `REPORT_LAT` | 0.0 | Latitude in decimal degrees | | `REPORT_LON` | 0.0 | Longitude in decimal degrees | | `REPORT_ELEV` | 0.0 | Elevation | | `REPORT_ELEV_UNIT` | m | Elevation unit: "m" or "ft" | | **Hardware Info** | | | | `REPEATER_HARDWARE` | LoRa Repeater | Repeater hardware model for sidebar | | `COMPANION_HARDWARE` | LoRa Node | Companion hardware model for sidebar | | **Radio Config** | | | | `RADIO_FREQUENCY` | 869.618 MHz | Radio frequency for display | | `RADIO_BANDWIDTH` | 62.5 kHz | Radio bandwidth for display | | `RADIO_SPREAD_FACTOR` | SF8 | Spread factor for display | | `RADIO_CODING_RATE` | CR8 | Coding rate for display | | **Intervals** | | | | `COMPANION_STEP` | 60 | Companion data collection interval (seconds) | | `REPEATER_STEP` | 900 | Repeater data collection interval (seconds) | | `REMOTE_TIMEOUT_S` | 10 | Remote request timeout | | `REMOTE_RETRY_ATTEMPTS` | 2 | Max retry attempts | | `REMOTE_RETRY_BACKOFF_S` | 4 | Retry backoff delay | | `REMOTE_CB_FAILS` | 6 | Failures before circuit opens | | `REMOTE_CB_COOLDOWN_S` | 3600 | Circuit breaker cooldown | | **Paths** | | | | `STATE_DIR` | ./data/state | State file path | | `OUT_DIR` | ./out | Output site path | ## Metrics Reference The system uses an EAV (Entity-Attribute-Value) schema where firmware field names are stored directly in the database. This allows new metrics to be captured automatically without schema changes. ### Repeater Metrics | Metric | Type | Display Unit | Description | |--------|------|--------------|-------------| | `bat` | Gauge | Voltage (V) | Battery voltage (stored in mV, displayed as V) | | `bat_pct` | Gauge | Battery (%) | Battery percentage (computed from voltage) | | `last_rssi` | Gauge | RSSI (dBm) | Signal strength of last packet | | `last_snr` | Gauge | SNR (dB) | Signal-to-noise ratio | | `noise_floor` | Gauge | dBm | Background RF noise | | `uptime` | Gauge | Days | Time since reboot (seconds ÷ 86400) | | `tx_queue_len` | Gauge | Queue depth | TX queue length | | `nb_recv` | Counter | Packets/min | Total packets received | | `nb_sent` | Counter | Packets/min | Total packets transmitted | | `airtime` | Counter | Seconds/min | TX airtime rate | | `rx_airtime` | Counter | Seconds/min | RX airtime rate | | `flood_dups` | Counter | Packets/min | Flood duplicate packets | | `direct_dups` | Counter | Packets/min | Direct duplicate packets | | `sent_flood` | Counter | Packets/min | Flood packets transmitted | | `recv_flood` | Counter | Packets/min | Flood packets received | | `sent_direct` | Counter | Packets/min | Direct packets transmitted | | `recv_direct` | Counter | Packets/min | Direct packets received | ### Companion Metrics | Metric | Type | Display Unit | Description | |--------|------|--------------|-------------| | `battery_mv` | Gauge | Voltage (V) | Battery voltage (stored in mV, displayed as V) | | `bat_pct` | Gauge | Battery (%) | Battery percentage (computed from voltage) | | `contacts` | Gauge | Count | Known mesh nodes | | `uptime_secs` | Gauge | Days | Time since reboot (seconds ÷ 86400) | | `recv` | Counter | Packets/min | Total packets received | | `sent` | Counter | Packets/min | Total packets transmitted | ### Metric Types - **Gauge**: Instantaneous values stored as-is (battery voltage, RSSI, queue depth) - **Counter**: Cumulative values where the rate of change is calculated (packets, airtime). Charts display per-minute rates. ## Database Metrics are stored in a SQLite database at `data/state/metrics.db` with WAL mode enabled for concurrent read/write access. ### Schema Migrations Database migrations are stored as SQL files in `src/meshmon/migrations/` and are applied automatically when the database is initialized. Migration files follow the naming convention `NNN_description.sql` (e.g., `001_initial_schema.sql`). ## Public Instances A list of publicly accessible MeshCore Stats installations. Want to add yours? [Open a pull request](https://github.com/jorijn/meshcore-stats/pulls)! | URL | Hardware | Location | |-----|----------|----------| | [meshcore.jorijn.com](https://meshcore.jorijn.com) | SenseCAP Solar Node P1 Pro + 6.5dBi Mikrotik antenna | Oosterhout, The Netherlands | ## License MIT