mirror of
https://github.com/jorijn/meshcore-stats.git
synced 2026-03-28 17:42:55 +01:00
Compare commits
10 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6afc14e007 | ||
|
|
4c5a408604 | ||
|
|
3c5eace220 | ||
|
|
7eee23ec40 | ||
|
|
30de7c20f3 | ||
|
|
19fa04c202 | ||
|
|
6ac52629d3 | ||
|
|
5b43f9ed12 | ||
|
|
0fe6c66ed8 | ||
|
|
2730a9d906 |
13
.github/workflows/release-please.yml
vendored
13
.github/workflows/release-please.yml
vendored
@@ -5,6 +5,17 @@ on:
|
||||
branches:
|
||||
- main
|
||||
|
||||
# Note: We use a fine-grained PAT (RELEASE_PLEASE_TOKEN) instead of GITHUB_TOKEN
|
||||
# because GITHUB_TOKEN cannot trigger other workflows (like docker-publish.yml).
|
||||
# This is a GitHub security feature to prevent infinite workflow loops.
|
||||
#
|
||||
# The PAT requires these permissions (scoped to this repository only):
|
||||
# - Contents: Read and write (for creating releases and pushing tags)
|
||||
# - Pull requests: Read and write (for creating/updating release PRs)
|
||||
#
|
||||
# To rotate: Settings > Developer settings > Fine-grained tokens
|
||||
# Recommended rotation: Every 90 days
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
@@ -16,6 +27,6 @@ jobs:
|
||||
- name: Release Please
|
||||
uses: googleapis/release-please-action@v4
|
||||
with:
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
token: ${{ secrets.RELEASE_PLEASE_TOKEN }}
|
||||
config-file: release-please-config.json
|
||||
manifest-file: .release-please-manifest.json
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
{
|
||||
".": "0.2.2"
|
||||
".": "0.2.5"
|
||||
}
|
||||
|
||||
26
CHANGELOG.md
26
CHANGELOG.md
@@ -4,6 +4,32 @@ All notable changes to this project will be documented in this file.
|
||||
|
||||
This changelog is automatically generated by [release-please](https://github.com/googleapis/release-please) based on [Conventional Commits](https://www.conventionalcommits.org/).
|
||||
|
||||
## [0.2.5](https://github.com/jorijn/meshcore-stats/compare/v0.2.4...v0.2.5) (2026-01-05)
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* add automatic serial port locking to prevent concurrent access ([3c5eace](https://github.com/jorijn/meshcore-stats/commit/3c5eace2207279c55401dd8fa27294d5a94bb682))
|
||||
|
||||
|
||||
### Documentation
|
||||
|
||||
* fix formatting in architecture diagram ([7eee23e](https://github.com/jorijn/meshcore-stats/commit/7eee23ec40ff9441515b4ac18fbb7cd3f87fa4b5))
|
||||
|
||||
## [0.2.4](https://github.com/jorijn/meshcore-stats/compare/v0.2.3...v0.2.4) (2026-01-05)
|
||||
|
||||
|
||||
### Documentation
|
||||
|
||||
* rewrite README with Docker-first installation guide ([6ac5262](https://github.com/jorijn/meshcore-stats/commit/6ac52629d3025db69f9334d3185b97ce16cd3e4b))
|
||||
|
||||
## [0.2.3](https://github.com/jorijn/meshcore-stats/compare/v0.2.2...v0.2.3) (2026-01-05)
|
||||
|
||||
|
||||
### Continuous Integration
|
||||
|
||||
* use fine-grained PAT for release-please to trigger Docker builds ([2730a9d](https://github.com/jorijn/meshcore-stats/commit/2730a9d906eeb5761af29dd69e8d4ebbfca50491))
|
||||
|
||||
## [0.2.2](https://github.com/jorijn/meshcore-stats/compare/v0.2.1...v0.2.2) (2026-01-05)
|
||||
|
||||
|
||||
|
||||
@@ -694,16 +694,14 @@ meshcore-cli -s /dev/ttyACM0 reset_path "repeater name"
|
||||
|
||||
## Cron Setup (Example)
|
||||
|
||||
Use `flock` to prevent USB serial conflicts when companion and repeater collection overlap.
|
||||
|
||||
```cron
|
||||
MESHCORE=/path/to/meshcore-stats
|
||||
|
||||
# Companion: every minute
|
||||
* * * * * cd $MESHCORE && flock -w 60 /tmp/meshcore.lock .venv/bin/python scripts/collect_companion.py
|
||||
* * * * * cd $MESHCORE && .venv/bin/python scripts/collect_companion.py
|
||||
|
||||
# Repeater: every 15 minutes (offset by 1 min for staggering)
|
||||
1,16,31,46 * * * * cd $MESHCORE && flock -w 60 /tmp/meshcore.lock .venv/bin/python scripts/collect_repeater.py
|
||||
1,16,31,46 * * * * cd $MESHCORE && .venv/bin/python scripts/collect_repeater.py
|
||||
|
||||
# Charts: every 5 minutes (generates SVG charts from database)
|
||||
*/5 * * * * cd $MESHCORE && .venv/bin/python scripts/render_charts.py
|
||||
@@ -717,7 +715,7 @@ MESHCORE=/path/to/meshcore-stats
|
||||
|
||||
**Notes:**
|
||||
- `cd $MESHCORE` is required because paths in the config are relative to the project root
|
||||
- `flock -w 60` waits up to 60 seconds for the lock, preventing USB serial conflicts
|
||||
- Serial port locking is handled automatically via `fcntl.flock()` in Python (no external `flock` needed)
|
||||
|
||||
## Adding New Metrics
|
||||
|
||||
|
||||
697
README.md
697
README.md
@@ -1,6 +1,6 @@
|
||||
# MeshCore Stats
|
||||
|
||||
A Python-based monitoring system for a MeshCore repeater node and its companion. Collects metrics from both devices, stores them in a SQLite database, and generates a static website with interactive SVG charts and statistics.
|
||||
A monitoring system for MeshCore LoRa mesh networks. Collects metrics from companion and repeater nodes, stores them in SQLite, and generates a static dashboard with interactive charts.
|
||||
|
||||
**Live demo:** [meshcore.jorijn.com](https://meshcore.jorijn.com)
|
||||
|
||||
@@ -9,500 +9,359 @@ A Python-based monitoring system for a MeshCore repeater node and its companion.
|
||||
<img src="docs/screenshot-2.png" width="49%" alt="MeshCore Stats Reports">
|
||||
</p>
|
||||
|
||||
## Features
|
||||
## Quick Start
|
||||
|
||||
- **Data Collection** - Collect metrics from companion (local) and repeater (remote) nodes
|
||||
- **Chart Rendering** - Generate interactive SVG charts from the database using matplotlib
|
||||
- **Static Site** - Generate a static HTML website with day/week/month/year views
|
||||
- **Reports** - Generate monthly and yearly statistics reports
|
||||
|
||||
## Requirements
|
||||
|
||||
### Python Dependencies
|
||||
|
||||
- Python 3.10+
|
||||
- meshcore >= 2.2.3
|
||||
- pyserial >= 3.5
|
||||
- jinja2 >= 3.1.0
|
||||
- matplotlib >= 3.8.0
|
||||
|
||||
### System Dependencies
|
||||
|
||||
- sqlite3 (for database maintenance script)
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Create Virtual Environment
|
||||
> **Linux only** - macOS and Windows users see [Platform Notes](#platform-notes) first.
|
||||
|
||||
```bash
|
||||
cd /path/to/meshcore-stats
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### 2. Configure
|
||||
|
||||
Copy the example configuration file and customize it:
|
||||
|
||||
```bash
|
||||
cp meshcore.conf.example meshcore.conf
|
||||
# Edit meshcore.conf with your settings
|
||||
```
|
||||
|
||||
The configuration file is automatically loaded by the scripts. Key settings to configure:
|
||||
|
||||
- **Connection**: `MESH_SERIAL_PORT`, `MESH_TRANSPORT`
|
||||
- **Repeater Identity**: `REPEATER_NAME`, `REPEATER_PASSWORD`
|
||||
- **Display Names**: `REPEATER_DISPLAY_NAME`, `COMPANION_DISPLAY_NAME`
|
||||
- **Location**: `REPORT_LOCATION_NAME`, `REPORT_LAT`, `REPORT_LON`, `REPORT_ELEV`
|
||||
- **Hardware Info**: `REPEATER_HARDWARE`, `COMPANION_HARDWARE`
|
||||
- **Radio Config**: `RADIO_FREQUENCY`, `RADIO_BANDWIDTH`, etc. (includes presets for different regions)
|
||||
|
||||
See `meshcore.conf.example` for all available options with documentation.
|
||||
|
||||
## Usage
|
||||
|
||||
### Manual Execution
|
||||
|
||||
```bash
|
||||
cd /path/to/meshcore-stats
|
||||
source .venv/bin/activate
|
||||
|
||||
# Collect companion data
|
||||
python scripts/collect_companion.py
|
||||
|
||||
# Collect repeater data
|
||||
python scripts/collect_repeater.py
|
||||
|
||||
# Generate static site (includes chart rendering)
|
||||
python scripts/render_site.py
|
||||
|
||||
# Generate reports
|
||||
python scripts/render_reports.py
|
||||
```
|
||||
|
||||
The configuration is automatically loaded from `meshcore.conf`.
|
||||
|
||||
### Cron Setup
|
||||
|
||||
Add these entries to your crontab (`crontab -e`):
|
||||
|
||||
```cron
|
||||
# MeshCore Stats - adjust path as needed
|
||||
MESHCORE=/home/user/meshcore-stats
|
||||
|
||||
# Every minute: collect companion data
|
||||
* * * * * cd $MESHCORE && flock -w 60 /tmp/meshcore.lock .venv/bin/python scripts/collect_companion.py
|
||||
|
||||
# Every 15 minutes: collect repeater data
|
||||
1,16,31,46 * * * * cd $MESHCORE && flock -w 60 /tmp/meshcore.lock .venv/bin/python scripts/collect_repeater.py
|
||||
|
||||
# Every 5 minutes: render site
|
||||
*/5 * * * * cd $MESHCORE && .venv/bin/python scripts/render_site.py
|
||||
|
||||
# Daily at midnight: generate reports
|
||||
0 0 * * * cd $MESHCORE && .venv/bin/python scripts/render_reports.py
|
||||
|
||||
# Monthly at 3 AM on the 1st: database maintenance
|
||||
0 3 1 * * $MESHCORE/scripts/db_maintenance.sh
|
||||
```
|
||||
|
||||
**Notes:**
|
||||
- `cd $MESHCORE` is required because paths in the config are relative to the project root
|
||||
- `flock` prevents USB serial conflicts when companion and repeater collection overlap
|
||||
|
||||
### Docker Installation
|
||||
|
||||
The recommended way to run MeshCore Stats is with Docker Compose. This provides automatic scheduling of all collection and rendering tasks.
|
||||
|
||||
#### Quick Start
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
# Clone and configure
|
||||
git clone https://github.com/jorijn/meshcore-stats.git
|
||||
cd meshcore-stats
|
||||
|
||||
# Create configuration
|
||||
cp meshcore.conf.example meshcore.conf
|
||||
# Edit meshcore.conf with your settings
|
||||
# Edit meshcore.conf with your repeater name and password
|
||||
|
||||
# Create data directories with correct ownership for container (UID 1000)
|
||||
# Create data directories (container runs as UID 1000)
|
||||
mkdir -p data/state out
|
||||
sudo chown -R 1000:1000 data out
|
||||
# Alternative: chmod -R 777 data out (less secure, use chown if possible)
|
||||
|
||||
# Start the containers
|
||||
docker compose up -d
|
||||
|
||||
# View logs
|
||||
docker compose logs -f
|
||||
```
|
||||
|
||||
The web interface will be available at `http://localhost:8080`.
|
||||
|
||||
#### Architecture
|
||||
|
||||
The Docker setup uses two containers:
|
||||
|
||||
| Container | Purpose |
|
||||
|-----------|---------|
|
||||
| `meshcore-stats` | Runs Ofelia scheduler for data collection and rendering |
|
||||
| `nginx` | Serves the static website |
|
||||
|
||||
#### Configuration
|
||||
|
||||
Configuration is loaded from `meshcore.conf` via the `env_file` directive. Key settings:
|
||||
|
||||
```bash
|
||||
# Required: Serial device for companion node
|
||||
MESH_SERIAL_PORT=/dev/ttyUSB0 # Adjust for your system
|
||||
|
||||
# Required: Repeater identity
|
||||
REPEATER_NAME="Your Repeater Name"
|
||||
REPEATER_PASSWORD="your-password"
|
||||
|
||||
# Display names (shown in UI)
|
||||
REPEATER_DISPLAY_NAME="My Repeater"
|
||||
COMPANION_DISPLAY_NAME="My Companion"
|
||||
```
|
||||
|
||||
See `meshcore.conf.example` for all available options.
|
||||
|
||||
#### Serial Device Access
|
||||
|
||||
For serial transport, the container needs access to your USB serial device. Create a `docker-compose.override.yml` file (gitignored) to specify your device:
|
||||
|
||||
```yaml
|
||||
# docker-compose.override.yml - Local device configuration (not tracked in git)
|
||||
# Add your serial device
|
||||
cat > docker-compose.override.yml << 'EOF'
|
||||
services:
|
||||
meshcore-stats:
|
||||
devices:
|
||||
- /dev/ttyUSB0:/dev/ttyUSB0:rw # Linux example
|
||||
# - /dev/ttyACM0:/dev/ttyACM0:rw # Alternative Linux device
|
||||
- /dev/ttyACM0:/dev/ttyACM0
|
||||
EOF
|
||||
|
||||
# Start
|
||||
docker compose up -d
|
||||
|
||||
# Verify it's working. The various collection and render jobs will trigger after a few minutes.
|
||||
docker compose ps
|
||||
docker compose logs meshcore-stats | head -20
|
||||
|
||||
# View dashboard at http://localhost:8080
|
||||
```
|
||||
|
||||
This file is automatically merged with `docker-compose.yml` when running `docker compose up`.
|
||||
## Features
|
||||
|
||||
> **Note**: TCP transport users (e.g., macOS with socat) don't need a devices section - just configure `MESH_TRANSPORT=tcp` in your `meshcore.conf`.
|
||||
- **Data Collection** - Metrics from local companion and remote repeater nodes
|
||||
- **Interactive Charts** - SVG charts with day/week/month/year views and tooltips
|
||||
- **Statistics Reports** - Monthly and yearly report generation
|
||||
- **Light/Dark Theme** - Automatic theme switching based on system preference
|
||||
|
||||
On the host, ensure the device is accessible:
|
||||
## Prerequisites
|
||||
|
||||
- Docker and Docker Compose V2
|
||||
- MeshCore companion node connected via USB serial
|
||||
- Remote repeater node reachable via LoRa from the companion
|
||||
|
||||
**Resource requirements:** ~100MB memory, ~100MB disk per year of data.
|
||||
|
||||
## Installation
|
||||
|
||||
### Docker (Recommended)
|
||||
|
||||
#### 1. Clone the Repository
|
||||
|
||||
```bash
|
||||
# Add user to dialout group (Linux)
|
||||
sudo usermod -a -G dialout $USER
|
||||
git clone https://github.com/jorijn/meshcore-stats.git
|
||||
cd meshcore-stats
|
||||
```
|
||||
|
||||
#### 2. Configure
|
||||
|
||||
Copy the example configuration and edit it:
|
||||
|
||||
```bash
|
||||
cp meshcore.conf.example meshcore.conf
|
||||
```
|
||||
|
||||
**Minimal required settings:**
|
||||
|
||||
```ini
|
||||
# Repeater identity (required)
|
||||
REPEATER_NAME=Your Repeater Name
|
||||
REPEATER_PASSWORD=your-admin-password
|
||||
|
||||
# Display names
|
||||
REPEATER_DISPLAY_NAME=My Repeater
|
||||
COMPANION_DISPLAY_NAME=My Companion
|
||||
```
|
||||
|
||||
See [meshcore.conf.example](meshcore.conf.example) for all available options.
|
||||
|
||||
#### 3. Create Data Directories
|
||||
|
||||
```bash
|
||||
mkdir -p data/state out
|
||||
sudo chown -R 1000:1000 data out
|
||||
```
|
||||
|
||||
The container runs as UID 1000, so directories must be writable by this user. If `sudo` is not available, you can relaxed the permissions using `chmod 777 data out`, but this is less secure.
|
||||
|
||||
#### 4. Configure Serial Device
|
||||
|
||||
Create `docker-compose.override.yml` to specify your serial device:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
meshcore-stats:
|
||||
devices:
|
||||
- /dev/ttyACM0:/dev/ttyACM0
|
||||
```
|
||||
|
||||
Ensure your user has serial port access:
|
||||
|
||||
```bash
|
||||
sudo usermod -aG dialout $USER
|
||||
# Log out and back in for changes to take effect
|
||||
```
|
||||
|
||||
#### Development Mode
|
||||
|
||||
For local development with live code changes:
|
||||
#### 5. Start the Containers
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
This mounts `src/` and `scripts/` into the container, so changes take effect immediately without rebuilding.
|
||||
After the various collection and render jobs has run, the dashboard will be available at **http://localhost:8080**.
|
||||
|
||||
#### Image Tags
|
||||
#### Verify Installation
|
||||
|
||||
Images are published to `ghcr.io/jorijn/meshcore-stats`:
|
||||
```bash
|
||||
# Check container status
|
||||
docker compose ps
|
||||
|
||||
| Tag | Description |
|
||||
|-----|-------------|
|
||||
| `X.Y.Z` | Specific version (e.g., `0.3.0`) |
|
||||
| `latest` | Latest release |
|
||||
| `nightly` | Latest release rebuilt with OS patches |
|
||||
| `nightly-YYYYMMDD` | Dated nightly build |
|
||||
# View logs
|
||||
docker compose logs -f meshcore-stats
|
||||
```
|
||||
|
||||
Version tags are rebuilt nightly to include OS security patches. For reproducible deployments, pin by SHA digest:
|
||||
### Common Docker Commands
|
||||
|
||||
```bash
|
||||
# View real-time logs
|
||||
docker compose logs -f meshcore-stats
|
||||
|
||||
# Restart after configuration changes
|
||||
docker compose restart meshcore-stats
|
||||
|
||||
# Update to latest version (database migrations are automatic)
|
||||
docker compose pull && docker compose up -d
|
||||
|
||||
# Stop all containers
|
||||
docker compose down
|
||||
|
||||
# Backup database
|
||||
cp data/state/metrics.db data/state/metrics.db.backup
|
||||
```
|
||||
|
||||
> **Note**: `docker compose down` preserves your data. Use `docker compose down -v` only if you want to delete everything.
|
||||
|
||||
### Manual Installation (Alternative)
|
||||
|
||||
For environments where Docker is not available.
|
||||
|
||||
#### Requirements
|
||||
|
||||
- Python 3.10+
|
||||
- SQLite3
|
||||
|
||||
#### Setup
|
||||
|
||||
```bash
|
||||
cd meshcore-stats
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
cp meshcore.conf.example meshcore.conf
|
||||
# Edit meshcore.conf with your settings
|
||||
```
|
||||
|
||||
#### Cron Setup
|
||||
|
||||
Add to your crontab (`crontab -e`):
|
||||
|
||||
```cron
|
||||
MESHCORE=/path/to/meshcore-stats
|
||||
|
||||
# Companion: every minute
|
||||
* * * * * cd $MESHCORE && .venv/bin/python scripts/collect_companion.py
|
||||
|
||||
# Repeater: every 15 minutes
|
||||
1,16,31,46 * * * * cd $MESHCORE && .venv/bin/python scripts/collect_repeater.py
|
||||
|
||||
# Charts: every 5 minutes
|
||||
*/5 * * * * cd $MESHCORE && .venv/bin/python scripts/render_charts.py
|
||||
|
||||
# Site: every 5 minutes
|
||||
*/5 * * * * cd $MESHCORE && .venv/bin/python scripts/render_site.py
|
||||
|
||||
# Reports: daily at midnight
|
||||
0 0 * * * cd $MESHCORE && .venv/bin/python scripts/render_reports.py
|
||||
```
|
||||
|
||||
Serve the `out/` directory with any web server.
|
||||
|
||||
## Platform Notes
|
||||
|
||||
<details>
|
||||
<summary><strong>Linux</strong></summary>
|
||||
|
||||
Docker can access USB serial devices directly. Add your device to `docker-compose.override.yml`:
|
||||
|
||||
```yaml
|
||||
image: ghcr.io/jorijn/meshcore-stats@sha256:abc123...
|
||||
services:
|
||||
meshcore-stats:
|
||||
devices:
|
||||
- /dev/ttyACM0:/dev/ttyACM0
|
||||
```
|
||||
|
||||
#### Volumes
|
||||
Common device paths:
|
||||
- `/dev/ttyACM0` - Arduino/native USB
|
||||
- `/dev/ttyUSB0` - USB-to-serial adapters
|
||||
|
||||
| Path | Purpose |
|
||||
|------|---------|
|
||||
| `./data/state` | SQLite database and circuit breaker state |
|
||||
| `./out` | Generated static site (served by nginx) |
|
||||
</details>
|
||||
|
||||
Both directories must be writable by UID 1000 (the container user). See Quick Start for setup.
|
||||
<details>
|
||||
<summary><strong>macOS</strong></summary>
|
||||
|
||||
#### Resource Limits
|
||||
Docker Desktop for macOS runs in a Linux VM and **cannot directly access USB serial devices**.
|
||||
|
||||
Default resource limits in `docker-compose.yml`:
|
||||
**Option 1: TCP Bridge (Recommended)**
|
||||
|
||||
| Container | CPU | Memory |
|
||||
|-----------|-----|--------|
|
||||
| meshcore-stats | 1.0 | 512MB |
|
||||
| nginx | 0.5 | 64MB |
|
||||
|
||||
Adjust in `docker-compose.yml` if needed.
|
||||
|
||||
#### Important Notes
|
||||
|
||||
- **Single instance only**: SQLite uses WAL mode which requires exclusive access. Do not run multiple container instances.
|
||||
- **Persistent storage**: Mount `./data/state` to preserve your database across container restarts.
|
||||
- **Health checks**: Both containers have health checks. Use `docker compose ps` to verify status.
|
||||
|
||||
Environment variables always take precedence over `meshcore.conf`.
|
||||
|
||||
### Serving the Site
|
||||
|
||||
The static site is generated in the `out/` directory. You can serve it with any web server:
|
||||
Expose the serial port over TCP using socat:
|
||||
|
||||
```bash
|
||||
# Simple Python server for testing
|
||||
cd out && python3 -m http.server 8080
|
||||
# Install socat
|
||||
brew install socat
|
||||
|
||||
# Or configure nginx/caddy to serve the out/ directory
|
||||
# Bridge serial to TCP (run in background)
|
||||
socat TCP-LISTEN:5000,fork,reuseaddr OPEN:/dev/cu.usbserial-0001,rawer,nonblock,ispeed=115200,ospeed=115200
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
Configure in `meshcore.conf`:
|
||||
|
||||
```
|
||||
meshcore-stats/
|
||||
├── requirements.txt
|
||||
├── README.md
|
||||
├── meshcore.conf.example # Example configuration
|
||||
├── meshcore.conf # Your configuration (create this)
|
||||
├── src/meshmon/
|
||||
│ ├── __init__.py
|
||||
│ ├── env.py # Environment variable parsing
|
||||
│ ├── log.py # Logging helper
|
||||
│ ├── meshcore_client.py # MeshCore connection and commands
|
||||
│ ├── db.py # SQLite database module
|
||||
│ ├── retry.py # Retry logic and circuit breaker
|
||||
│ ├── charts.py # Matplotlib SVG chart generation
|
||||
│ ├── html.py # HTML rendering
|
||||
│ ├── reports.py # Report generation
|
||||
│ ├── metrics.py # Metric type definitions
|
||||
│ ├── battery.py # Battery voltage to percentage conversion
|
||||
│ ├── migrations/ # SQL schema migrations
|
||||
│ │ ├── 001_initial_schema.sql
|
||||
│ │ └── 002_eav_schema.sql
|
||||
│ └── templates/ # Jinja2 HTML templates
|
||||
├── scripts/
|
||||
│ ├── collect_companion.py # Collect metrics from companion node
|
||||
│ ├── collect_repeater.py # Collect metrics from repeater node
|
||||
│ ├── render_charts.py # Generate SVG charts from database
|
||||
│ ├── render_site.py # Generate static HTML site
|
||||
│ ├── render_reports.py # Generate monthly/yearly reports
|
||||
│ └── db_maintenance.sh # Database VACUUM/ANALYZE
|
||||
├── data/
|
||||
│ └── state/
|
||||
│ ├── metrics.db # SQLite database (WAL mode)
|
||||
│ └── repeater_circuit.json
|
||||
└── out/ # Generated site
|
||||
├── .htaccess # Apache config (DirectoryIndex, caching)
|
||||
├── styles.css # Stylesheet
|
||||
├── chart-tooltip.js # Chart tooltip enhancement
|
||||
├── day.html # Repeater pages (entry point)
|
||||
├── week.html
|
||||
├── month.html
|
||||
├── year.html
|
||||
├── companion/
|
||||
│ ├── day.html
|
||||
│ ├── week.html
|
||||
│ ├── month.html
|
||||
│ └── year.html
|
||||
└── reports/
|
||||
├── index.html
|
||||
├── repeater/ # YYYY/MM reports
|
||||
└── companion/
|
||||
```ini
|
||||
MESH_TRANSPORT=tcp
|
||||
MESH_TCP_HOST=host.docker.internal
|
||||
MESH_TCP_PORT=5000
|
||||
```
|
||||
|
||||
## Chart Features
|
||||
**Option 2: Native Installation**
|
||||
|
||||
Charts are rendered as inline SVG using matplotlib with the following features:
|
||||
Use the manual installation method with cron instead of Docker.
|
||||
|
||||
- **Theme Support**: Automatic light/dark mode via CSS `prefers-color-scheme`
|
||||
- **Interactive Tooltips**: Hover to see exact values and timestamps
|
||||
- **Data Point Indicator**: Visual marker shows position on the chart line
|
||||
- **Mobile Support**: Touch-friendly tooltips
|
||||
- **Statistics**: Min/Avg/Max values displayed below each chart
|
||||
- **Period Views**: Day, week, month, and year time ranges
|
||||
</details>
|
||||
|
||||
## Troubleshooting
|
||||
<details>
|
||||
<summary><strong>Windows (WSL2)</strong></summary>
|
||||
|
||||
### Serial Device Not Found
|
||||
WSL2 and Docker Desktop for Windows cannot directly access COM ports.
|
||||
|
||||
If you see "No serial ports found" or connection fails:
|
||||
Use the TCP bridge approach (similar to macOS) or native installation.
|
||||
|
||||
1. Check that your device is connected:
|
||||
```bash
|
||||
ls -la /dev/ttyUSB* /dev/ttyACM*
|
||||
```
|
||||
</details>
|
||||
|
||||
2. Check permissions (add user to dialout group):
|
||||
```bash
|
||||
sudo usermod -a -G dialout $USER
|
||||
# Log out and back in for changes to take effect
|
||||
```
|
||||
|
||||
3. Try specifying the port explicitly:
|
||||
```bash
|
||||
export MESH_SERIAL_PORT=/dev/ttyACM0
|
||||
```
|
||||
|
||||
4. Check dmesg for device detection:
|
||||
```bash
|
||||
dmesg | tail -20
|
||||
```
|
||||
|
||||
### Repeater Not Found
|
||||
|
||||
If the script cannot find the repeater contact:
|
||||
|
||||
1. The script will print all discovered contacts - check for the correct name
|
||||
2. Verify REPEATER_NAME matches exactly (case-sensitive)
|
||||
3. Try using REPEATER_KEY_PREFIX instead with the first 6-12 hex chars of the public key
|
||||
|
||||
### Circuit Breaker
|
||||
|
||||
If repeater collection shows "cooldown active":
|
||||
|
||||
1. This is normal after multiple failed remote requests
|
||||
2. Wait for the cooldown period (default 1 hour) or reset manually:
|
||||
```bash
|
||||
rm data/state/repeater_circuit.json
|
||||
```
|
||||
|
||||
### Docker on macOS: Serial Devices Not Available
|
||||
|
||||
Docker on macOS (including Docker Desktop and OrbStack) runs containers inside a Linux virtual machine. USB and serial devices connected to the Mac host cannot be passed through to this VM, so the `devices:` section in docker-compose.yml will fail with:
|
||||
|
||||
```
|
||||
error gathering device information while adding custom device "/dev/cu.usbserial-0001": no such file or directory
|
||||
```
|
||||
|
||||
**Workarounds:**
|
||||
|
||||
1. **Use TCP transport**: Run a serial-to-TCP bridge on the host and configure the container to connect via TCP:
|
||||
```bash
|
||||
# On macOS host, expose serial port over TCP (install socat via Homebrew)
|
||||
socat TCP-LISTEN:5000,fork,reuseaddr OPEN:/dev/cu.usbserial-0001,rawer,nonblock,ispeed=115200,ospeed=115200
|
||||
```
|
||||
Then configure in meshcore.conf:
|
||||
```bash
|
||||
MESH_TRANSPORT=tcp
|
||||
MESH_TCP_HOST=host.docker.internal
|
||||
MESH_TCP_PORT=5000
|
||||
```
|
||||
|
||||
2. **Run natively on macOS**: Use the cron-based setup instead of Docker (see "Cron Setup" section).
|
||||
|
||||
3. **Use a Linux host**: Docker on Linux can pass through USB devices directly.
|
||||
|
||||
Note: OrbStack has [USB passthrough on their roadmap](https://github.com/orbstack/orbstack/issues/89) but it is not yet available.
|
||||
|
||||
## Environment Variables Reference
|
||||
## Configuration Reference
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| **Connection** | | |
|
||||
| `MESH_TRANSPORT` | serial | Connection type: serial, tcp, ble |
|
||||
| `MESH_SERIAL_PORT` | (auto) | Serial port path |
|
||||
| `MESH_SERIAL_BAUD` | 115200 | Baud rate |
|
||||
| `MESH_TCP_HOST` | localhost | TCP host |
|
||||
| `MESH_TCP_PORT` | 5000 | TCP port |
|
||||
| `MESH_BLE_ADDR` | - | BLE device address |
|
||||
| `MESH_BLE_PIN` | - | BLE PIN |
|
||||
| `MESH_DEBUG` | 0 | Enable debug output |
|
||||
| **Repeater Identity** | | |
|
||||
| `REPEATER_NAME` | - | Repeater advertised name |
|
||||
| `REPEATER_KEY_PREFIX` | - | Repeater public key prefix |
|
||||
| `REPEATER_PASSWORD` | - | Repeater login password |
|
||||
| **Display Names** | | |
|
||||
| `REPEATER_DISPLAY_NAME` | Repeater Node | Display name for repeater in UI |
|
||||
| `COMPANION_DISPLAY_NAME` | Companion Node | Display name for companion in UI |
|
||||
| `REPEATER_NAME` | *required* | Advertised name to find in contacts |
|
||||
| `REPEATER_PASSWORD` | *required* | Admin password for repeater |
|
||||
| `REPEATER_KEY_PREFIX` | - | Alternative to `REPEATER_NAME`: hex prefix of public key |
|
||||
| **Connection** | | |
|
||||
| `MESH_TRANSPORT` | serial | Transport type: `serial`, `tcp`, or `ble` |
|
||||
| `MESH_SERIAL_PORT` | auto | Serial port path |
|
||||
| `MESH_TCP_HOST` | localhost | TCP host (for TCP transport) |
|
||||
| `MESH_TCP_PORT` | 5000 | TCP port (for TCP transport) |
|
||||
| **Display** | | |
|
||||
| `REPEATER_DISPLAY_NAME` | Repeater Node | Name shown in UI |
|
||||
| `COMPANION_DISPLAY_NAME` | Companion Node | Name shown in UI |
|
||||
| `REPEATER_HARDWARE` | LoRa Repeater | Hardware model for sidebar |
|
||||
| `COMPANION_HARDWARE` | LoRa Node | Hardware model for sidebar |
|
||||
| **Location** | | |
|
||||
| `REPORT_LOCATION_NAME` | Your Location | Full location name for reports |
|
||||
| `REPORT_LOCATION_SHORT` | Your Location | Short location for sidebar/meta |
|
||||
| `REPORT_LAT` | 0.0 | Latitude in decimal degrees |
|
||||
| `REPORT_LON` | 0.0 | Longitude in decimal degrees |
|
||||
| `REPORT_LOCATION_NAME` | Your Location | Full location for reports |
|
||||
| `REPORT_LAT` | 0.0 | Latitude |
|
||||
| `REPORT_LON` | 0.0 | Longitude |
|
||||
| `REPORT_ELEV` | 0.0 | Elevation |
|
||||
| `REPORT_ELEV_UNIT` | m | Elevation unit: "m" or "ft" |
|
||||
| **Hardware Info** | | |
|
||||
| `REPEATER_HARDWARE` | LoRa Repeater | Repeater hardware model for sidebar |
|
||||
| `COMPANION_HARDWARE` | LoRa Node | Companion hardware model for sidebar |
|
||||
| **Radio Config** | | |
|
||||
| `RADIO_FREQUENCY` | 869.618 MHz | Radio frequency for display |
|
||||
| `RADIO_BANDWIDTH` | 62.5 kHz | Radio bandwidth for display |
|
||||
| `RADIO_SPREAD_FACTOR` | SF8 | Spread factor for display |
|
||||
| `RADIO_CODING_RATE` | CR8 | Coding rate for display |
|
||||
| **Intervals** | | |
|
||||
| `COMPANION_STEP` | 60 | Companion data collection interval (seconds) |
|
||||
| `REPEATER_STEP` | 900 | Repeater data collection interval (seconds) |
|
||||
| `REMOTE_TIMEOUT_S` | 10 | Remote request timeout |
|
||||
| `REMOTE_RETRY_ATTEMPTS` | 2 | Max retry attempts |
|
||||
| `REMOTE_RETRY_BACKOFF_S` | 4 | Retry backoff delay |
|
||||
| `REMOTE_CB_FAILS` | 6 | Failures before circuit opens |
|
||||
| `REMOTE_CB_COOLDOWN_S` | 3600 | Circuit breaker cooldown |
|
||||
| **Paths** | | |
|
||||
| `STATE_DIR` | ./data/state | State file path |
|
||||
| `OUT_DIR` | ./out | Output site path |
|
||||
| **Radio** (display only) | | |
|
||||
| `RADIO_FREQUENCY` | 869.618 MHz | Frequency shown in sidebar |
|
||||
| `RADIO_BANDWIDTH` | 62.5 kHz | Bandwidth |
|
||||
| `RADIO_SPREAD_FACTOR` | SF8 | Spread factor |
|
||||
|
||||
## Metrics Reference
|
||||
See [meshcore.conf.example](meshcore.conf.example) for all options with regional radio presets.
|
||||
|
||||
The system uses an EAV (Entity-Attribute-Value) schema where firmware field names are stored directly in the database. This allows new metrics to be captured automatically without schema changes.
|
||||
## Troubleshooting
|
||||
|
||||
### Repeater Metrics
|
||||
| Symptom | Cause | Solution |
|
||||
|---------|-------|----------|
|
||||
| "Permission denied" on serial port | User not in dialout group | `sudo usermod -aG dialout $USER` then re-login |
|
||||
| Repeater shows "offline" status | No data or circuit breaker tripped | Check logs; delete `data/state/repeater_circuit.json` to reset |
|
||||
| Empty charts | Not enough data collected | Wait for 2+ collection cycles |
|
||||
| Container exits immediately | Missing or invalid configuration | Verify `meshcore.conf` exists and has required values |
|
||||
| "No serial ports found" | Device not connected/detected | Check `ls /dev/tty*` and device permissions |
|
||||
| Device path changed after reboot | USB enumeration order changed | Update path in `docker-compose.override.yml` or use udev rules |
|
||||
| "database is locked" errors | Maintenance script running | Wait for completion; check if VACUUM is running |
|
||||
|
||||
| Metric | Type | Display Unit | Description |
|
||||
|--------|------|--------------|-------------|
|
||||
| `bat` | Gauge | Voltage (V) | Battery voltage (stored in mV, displayed as V) |
|
||||
| `bat_pct` | Gauge | Battery (%) | Battery percentage (computed from voltage) |
|
||||
| `last_rssi` | Gauge | RSSI (dBm) | Signal strength of last packet |
|
||||
| `last_snr` | Gauge | SNR (dB) | Signal-to-noise ratio |
|
||||
| `noise_floor` | Gauge | dBm | Background RF noise |
|
||||
| `uptime` | Gauge | Days | Time since reboot (seconds ÷ 86400) |
|
||||
| `tx_queue_len` | Gauge | Queue depth | TX queue length |
|
||||
| `nb_recv` | Counter | Packets/min | Total packets received |
|
||||
| `nb_sent` | Counter | Packets/min | Total packets transmitted |
|
||||
| `airtime` | Counter | Seconds/min | TX airtime rate |
|
||||
| `rx_airtime` | Counter | Seconds/min | RX airtime rate |
|
||||
| `flood_dups` | Counter | Packets/min | Flood duplicate packets |
|
||||
| `direct_dups` | Counter | Packets/min | Direct duplicate packets |
|
||||
| `sent_flood` | Counter | Packets/min | Flood packets transmitted |
|
||||
| `recv_flood` | Counter | Packets/min | Flood packets received |
|
||||
| `sent_direct` | Counter | Packets/min | Direct packets transmitted |
|
||||
| `recv_direct` | Counter | Packets/min | Direct packets received |
|
||||
### Debug Logging
|
||||
|
||||
### Companion Metrics
|
||||
```bash
|
||||
# Enable debug mode in meshcore.conf
|
||||
MESH_DEBUG=1
|
||||
|
||||
| Metric | Type | Display Unit | Description |
|
||||
|--------|------|--------------|-------------|
|
||||
| `battery_mv` | Gauge | Voltage (V) | Battery voltage (stored in mV, displayed as V) |
|
||||
| `bat_pct` | Gauge | Battery (%) | Battery percentage (computed from voltage) |
|
||||
| `contacts` | Gauge | Count | Known mesh nodes |
|
||||
| `uptime_secs` | Gauge | Days | Time since reboot (seconds ÷ 86400) |
|
||||
| `recv` | Counter | Packets/min | Total packets received |
|
||||
| `sent` | Counter | Packets/min | Total packets transmitted |
|
||||
# View detailed logs
|
||||
docker compose logs -f meshcore-stats
|
||||
```
|
||||
|
||||
### Metric Types
|
||||
### Circuit Breaker
|
||||
|
||||
- **Gauge**: Instantaneous values stored as-is (battery voltage, RSSI, queue depth)
|
||||
- **Counter**: Cumulative values where the rate of change is calculated (packets, airtime). Charts display per-minute rates.
|
||||
The repeater collector uses a circuit breaker to avoid spamming LoRa when the repeater is unreachable. After multiple failures, it enters a cooldown period (default: 1 hour).
|
||||
|
||||
## Database
|
||||
To reset manually:
|
||||
|
||||
Metrics are stored in a SQLite database at `data/state/metrics.db` with WAL mode enabled for concurrent read/write access.
|
||||
```bash
|
||||
rm data/state/repeater_circuit.json
|
||||
docker compose restart meshcore-stats
|
||||
```
|
||||
|
||||
### Schema Migrations
|
||||
## Architecture
|
||||
|
||||
Database migrations are stored as SQL files in `src/meshmon/migrations/` and are applied automatically when the database is initialized. Migration files follow the naming convention `NNN_description.sql` (e.g., `001_initial_schema.sql`).
|
||||
```
|
||||
┌─────────────────┐ LoRa ┌─────────────────┐
|
||||
│ Companion │◄─────────────►│ Repeater │
|
||||
│ (USB Serial) │ │ (Remote) │
|
||||
└────────┬────────┘ └─────────────────┘
|
||||
│
|
||||
│ Serial/TCP
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Docker Host │
|
||||
│ ┌───────────┐ │
|
||||
│ │ meshcore- │ │ ┌─────────┐
|
||||
│ │ stats │──┼────►│ nginx │──► :8080
|
||||
│ └───────────┘ │ └─────────┘
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ SQLite + SVG │
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
## Public Instances
|
||||
The system runs two containers:
|
||||
- **meshcore-stats**: Collects data on schedule (Ofelia) and generates charts
|
||||
- **nginx**: Serves the static dashboard
|
||||
|
||||
A list of publicly accessible MeshCore Stats installations. Want to add yours? [Open a pull request](https://github.com/jorijn/meshcore-stats/pulls)!
|
||||
## Documentation
|
||||
|
||||
| URL | Hardware | Location |
|
||||
|-----|----------|----------|
|
||||
| [meshcore.jorijn.com](https://meshcore.jorijn.com) | SenseCAP Solar Node P1 Pro + 6.5dBi Mikrotik antenna | Oosterhout, The Netherlands |
|
||||
- [docs/firmware-responses.md](docs/firmware-responses.md) - MeshCore firmware response formats
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
|
||||
## Public Instances
|
||||
|
||||
Public MeshCore Stats installations. Want to add yours? [Open a pull request](https://github.com/jorijn/meshcore-stats/pulls)!
|
||||
|
||||
| URL | Hardware | Location |
|
||||
|-----|----------|----------|
|
||||
| [meshcore.jorijn.com](https://meshcore.jorijn.com) | SenseCAP Solar Node P1 Pro + 6.5dBi Mikrotik antenna | Oosterhout, The Netherlands |
|
||||
|
||||
@@ -15,7 +15,7 @@ services:
|
||||
# MeshCore Stats - Data collection and rendering
|
||||
# ==========================================================================
|
||||
meshcore-stats:
|
||||
image: ghcr.io/jorijn/meshcore-stats:0.2.2 # x-release-please-version
|
||||
image: ghcr.io/jorijn/meshcore-stats:0.2.5 # x-release-please-version
|
||||
container_name: meshcore-stats
|
||||
restart: unless-stopped
|
||||
|
||||
|
||||
@@ -25,7 +25,7 @@ sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
|
||||
|
||||
from meshmon.env import get_config
|
||||
from meshmon import log
|
||||
from meshmon.meshcore_client import connect_from_env, run_command
|
||||
from meshmon.meshcore_client import connect_with_lock, run_command
|
||||
from meshmon.db import init_db, insert_metrics
|
||||
|
||||
|
||||
@@ -39,138 +39,131 @@ async def collect_companion() -> int:
|
||||
cfg = get_config()
|
||||
ts = int(time.time())
|
||||
|
||||
log.debug("Connecting to companion node...")
|
||||
mc = await connect_from_env()
|
||||
|
||||
if mc is None:
|
||||
log.error("Failed to connect to companion node")
|
||||
return 1
|
||||
|
||||
# Metrics to insert (firmware field names)
|
||||
metrics: dict[str, float] = {}
|
||||
commands_succeeded = 0
|
||||
|
||||
# Commands are accessed via mc.commands
|
||||
cmd = mc.commands
|
||||
log.debug("Connecting to companion node...")
|
||||
async with connect_with_lock() as mc:
|
||||
if mc is None:
|
||||
log.error("Failed to connect to companion node")
|
||||
return 1
|
||||
|
||||
try:
|
||||
# send_appstart (already called during connect, but call again to get self_info)
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.send_appstart(), "send_appstart"
|
||||
)
|
||||
if ok:
|
||||
commands_succeeded += 1
|
||||
log.debug(f"appstart: {evt_type}")
|
||||
else:
|
||||
log.error(f"appstart failed: {err}")
|
||||
# Commands are accessed via mc.commands
|
||||
cmd = mc.commands
|
||||
|
||||
# send_device_query
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.send_device_query(), "send_device_query"
|
||||
)
|
||||
if ok:
|
||||
commands_succeeded += 1
|
||||
log.debug(f"device_query: {payload}")
|
||||
else:
|
||||
log.error(f"device_query failed: {err}")
|
||||
try:
|
||||
# send_appstart (already called during connect, but call again to get self_info)
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.send_appstart(), "send_appstart"
|
||||
)
|
||||
if ok:
|
||||
commands_succeeded += 1
|
||||
log.debug(f"appstart: {evt_type}")
|
||||
else:
|
||||
log.error(f"appstart failed: {err}")
|
||||
|
||||
# get_bat
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.get_bat(), "get_bat"
|
||||
)
|
||||
if ok:
|
||||
commands_succeeded += 1
|
||||
log.debug(f"get_bat: {payload}")
|
||||
else:
|
||||
log.error(f"get_bat failed: {err}")
|
||||
# send_device_query
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.send_device_query(), "send_device_query"
|
||||
)
|
||||
if ok:
|
||||
commands_succeeded += 1
|
||||
log.debug(f"device_query: {payload}")
|
||||
else:
|
||||
log.error(f"device_query failed: {err}")
|
||||
|
||||
# get_time
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.get_time(), "get_time"
|
||||
)
|
||||
if ok:
|
||||
commands_succeeded += 1
|
||||
log.debug(f"get_time: {payload}")
|
||||
else:
|
||||
log.error(f"get_time failed: {err}")
|
||||
# get_bat
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.get_bat(), "get_bat"
|
||||
)
|
||||
if ok:
|
||||
commands_succeeded += 1
|
||||
log.debug(f"get_bat: {payload}")
|
||||
else:
|
||||
log.error(f"get_bat failed: {err}")
|
||||
|
||||
# get_self_telemetry
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.get_self_telemetry(), "get_self_telemetry"
|
||||
)
|
||||
if ok:
|
||||
commands_succeeded += 1
|
||||
log.debug(f"get_self_telemetry: {payload}")
|
||||
else:
|
||||
log.error(f"get_self_telemetry failed: {err}")
|
||||
# get_time
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.get_time(), "get_time"
|
||||
)
|
||||
if ok:
|
||||
commands_succeeded += 1
|
||||
log.debug(f"get_time: {payload}")
|
||||
else:
|
||||
log.error(f"get_time failed: {err}")
|
||||
|
||||
# get_custom_vars
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.get_custom_vars(), "get_custom_vars"
|
||||
)
|
||||
if ok:
|
||||
commands_succeeded += 1
|
||||
log.debug(f"get_custom_vars: {payload}")
|
||||
else:
|
||||
log.debug(f"get_custom_vars failed: {err}")
|
||||
# get_self_telemetry
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.get_self_telemetry(), "get_self_telemetry"
|
||||
)
|
||||
if ok:
|
||||
commands_succeeded += 1
|
||||
log.debug(f"get_self_telemetry: {payload}")
|
||||
else:
|
||||
log.error(f"get_self_telemetry failed: {err}")
|
||||
|
||||
# get_contacts - count contacts
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.get_contacts(), "get_contacts"
|
||||
)
|
||||
if ok:
|
||||
commands_succeeded += 1
|
||||
contacts_count = len(payload) if payload else 0
|
||||
metrics["contacts"] = float(contacts_count)
|
||||
log.debug(f"get_contacts: found {contacts_count} contacts")
|
||||
else:
|
||||
log.error(f"get_contacts failed: {err}")
|
||||
# get_custom_vars
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.get_custom_vars(), "get_custom_vars"
|
||||
)
|
||||
if ok:
|
||||
commands_succeeded += 1
|
||||
log.debug(f"get_custom_vars: {payload}")
|
||||
else:
|
||||
log.debug(f"get_custom_vars failed: {err}")
|
||||
|
||||
# Get statistics - these contain the main metrics
|
||||
# Core stats (battery_mv, uptime_secs, errors, queue_len)
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.get_stats_core(), "get_stats_core"
|
||||
)
|
||||
if ok and payload and isinstance(payload, dict):
|
||||
commands_succeeded += 1
|
||||
# Insert all numeric fields from stats_core
|
||||
for key, value in payload.items():
|
||||
if isinstance(value, (int, float)):
|
||||
metrics[key] = float(value)
|
||||
log.debug(f"stats_core: {payload}")
|
||||
# get_contacts - count contacts
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.get_contacts(), "get_contacts"
|
||||
)
|
||||
if ok:
|
||||
commands_succeeded += 1
|
||||
contacts_count = len(payload) if payload else 0
|
||||
metrics["contacts"] = float(contacts_count)
|
||||
log.debug(f"get_contacts: found {contacts_count} contacts")
|
||||
else:
|
||||
log.error(f"get_contacts failed: {err}")
|
||||
|
||||
# Radio stats (noise_floor, last_rssi, last_snr, tx_air_secs, rx_air_secs)
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.get_stats_radio(), "get_stats_radio"
|
||||
)
|
||||
if ok and payload and isinstance(payload, dict):
|
||||
commands_succeeded += 1
|
||||
for key, value in payload.items():
|
||||
if isinstance(value, (int, float)):
|
||||
metrics[key] = float(value)
|
||||
log.debug(f"stats_radio: {payload}")
|
||||
# Get statistics - these contain the main metrics
|
||||
# Core stats (battery_mv, uptime_secs, errors, queue_len)
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.get_stats_core(), "get_stats_core"
|
||||
)
|
||||
if ok and payload and isinstance(payload, dict):
|
||||
commands_succeeded += 1
|
||||
# Insert all numeric fields from stats_core
|
||||
for key, value in payload.items():
|
||||
if isinstance(value, (int, float)):
|
||||
metrics[key] = float(value)
|
||||
log.debug(f"stats_core: {payload}")
|
||||
|
||||
# Packet stats (recv, sent, flood_tx, direct_tx, flood_rx, direct_rx)
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.get_stats_packets(), "get_stats_packets"
|
||||
)
|
||||
if ok and payload and isinstance(payload, dict):
|
||||
commands_succeeded += 1
|
||||
for key, value in payload.items():
|
||||
if isinstance(value, (int, float)):
|
||||
metrics[key] = float(value)
|
||||
log.debug(f"stats_packets: {payload}")
|
||||
# Radio stats (noise_floor, last_rssi, last_snr, tx_air_secs, rx_air_secs)
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.get_stats_radio(), "get_stats_radio"
|
||||
)
|
||||
if ok and payload and isinstance(payload, dict):
|
||||
commands_succeeded += 1
|
||||
for key, value in payload.items():
|
||||
if isinstance(value, (int, float)):
|
||||
metrics[key] = float(value)
|
||||
log.debug(f"stats_radio: {payload}")
|
||||
|
||||
except Exception as e:
|
||||
log.error(f"Error during collection: {e}")
|
||||
# Packet stats (recv, sent, flood_tx, direct_tx, flood_rx, direct_rx)
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.get_stats_packets(), "get_stats_packets"
|
||||
)
|
||||
if ok and payload and isinstance(payload, dict):
|
||||
commands_succeeded += 1
|
||||
for key, value in payload.items():
|
||||
if isinstance(value, (int, float)):
|
||||
metrics[key] = float(value)
|
||||
log.debug(f"stats_packets: {payload}")
|
||||
|
||||
finally:
|
||||
# Close connection
|
||||
if hasattr(mc, "disconnect"):
|
||||
try:
|
||||
await mc.disconnect()
|
||||
except Exception:
|
||||
pass
|
||||
except Exception as e:
|
||||
log.error(f"Error during collection: {e}")
|
||||
|
||||
# Connection closed and lock released by context manager
|
||||
|
||||
# Print summary
|
||||
summary_parts = [f"ts={ts}"]
|
||||
|
||||
@@ -27,7 +27,7 @@ sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
|
||||
from meshmon.env import get_config
|
||||
from meshmon import log
|
||||
from meshmon.meshcore_client import (
|
||||
connect_from_env,
|
||||
connect_with_lock,
|
||||
run_command,
|
||||
get_contact_by_name,
|
||||
get_contact_by_key_prefix,
|
||||
@@ -161,97 +161,90 @@ async def collect_repeater() -> int:
|
||||
# Skip collection - no metrics to write
|
||||
return 0
|
||||
|
||||
# Connect to companion
|
||||
log.debug("Connecting to companion node...")
|
||||
mc = await connect_from_env()
|
||||
|
||||
if mc is None:
|
||||
log.error("Failed to connect to companion node")
|
||||
return 1
|
||||
|
||||
# Metrics to insert (firmware field names from req_status_sync)
|
||||
metrics: dict[str, float] = {}
|
||||
node_name = "unknown"
|
||||
status_ok = False
|
||||
|
||||
# Commands are accessed via mc.commands
|
||||
cmd = mc.commands
|
||||
|
||||
try:
|
||||
# Initialize (appstart already called during connect)
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.send_appstart(), "send_appstart"
|
||||
)
|
||||
if not ok:
|
||||
log.error(f"appstart failed: {err}")
|
||||
|
||||
# Find repeater contact
|
||||
contact = await find_repeater_contact(mc)
|
||||
|
||||
if contact is None:
|
||||
log.error("Cannot find repeater contact")
|
||||
# Connect to companion
|
||||
log.debug("Connecting to companion node...")
|
||||
async with connect_with_lock() as mc:
|
||||
if mc is None:
|
||||
log.error("Failed to connect to companion node")
|
||||
return 1
|
||||
|
||||
# Store contact info
|
||||
contact_info = extract_contact_info(contact)
|
||||
node_name = contact_info.get("adv_name", "unknown")
|
||||
# Commands are accessed via mc.commands
|
||||
cmd = mc.commands
|
||||
|
||||
log.debug(f"Found repeater: {node_name}")
|
||||
try:
|
||||
# Initialize (appstart already called during connect)
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc, cmd.send_appstart(), "send_appstart"
|
||||
)
|
||||
if not ok:
|
||||
log.error(f"appstart failed: {err}")
|
||||
|
||||
# Optional login (if command exists)
|
||||
if cfg.repeater_password and hasattr(cmd, "send_login"):
|
||||
log.debug("Attempting login...")
|
||||
try:
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc,
|
||||
cmd.send_login(contact, cfg.repeater_password),
|
||||
"send_login",
|
||||
)
|
||||
if ok:
|
||||
log.debug("Login successful")
|
||||
else:
|
||||
log.debug(f"Login failed or not supported: {err}")
|
||||
except Exception as e:
|
||||
log.debug(f"Login not supported: {e}")
|
||||
# Find repeater contact
|
||||
contact = await find_repeater_contact(mc)
|
||||
|
||||
# Query status (using _sync version which returns payload directly)
|
||||
# Use timeout=0 to let the device suggest timeout, with min_timeout as floor
|
||||
log.debug("Querying repeater status...")
|
||||
success, payload, err = await query_repeater_with_retry(
|
||||
mc,
|
||||
contact,
|
||||
"req_status_sync",
|
||||
lambda: cmd.req_status_sync(contact, timeout=0, min_timeout=cfg.remote_timeout_s),
|
||||
)
|
||||
if success and payload and isinstance(payload, dict):
|
||||
status_ok = True
|
||||
# Insert all numeric fields from status response
|
||||
for key, value in payload.items():
|
||||
if isinstance(value, (int, float)):
|
||||
metrics[key] = float(value)
|
||||
log.debug(f"req_status_sync: {payload}")
|
||||
else:
|
||||
log.warn(f"req_status_sync failed: {err}")
|
||||
if contact is None:
|
||||
log.error("Cannot find repeater contact")
|
||||
return 1
|
||||
|
||||
# Update circuit breaker
|
||||
if status_ok:
|
||||
cb.record_success()
|
||||
log.debug("Circuit breaker: recorded success")
|
||||
else:
|
||||
# Store contact info
|
||||
contact_info = extract_contact_info(contact)
|
||||
node_name = contact_info.get("adv_name", "unknown")
|
||||
|
||||
log.debug(f"Found repeater: {node_name}")
|
||||
|
||||
# Optional login (if command exists)
|
||||
if cfg.repeater_password and hasattr(cmd, "send_login"):
|
||||
log.debug("Attempting login...")
|
||||
try:
|
||||
ok, evt_type, payload, err = await run_command(
|
||||
mc,
|
||||
cmd.send_login(contact, cfg.repeater_password),
|
||||
"send_login",
|
||||
)
|
||||
if ok:
|
||||
log.debug("Login successful")
|
||||
else:
|
||||
log.debug(f"Login failed or not supported: {err}")
|
||||
except Exception as e:
|
||||
log.debug(f"Login not supported: {e}")
|
||||
|
||||
# Query status (using _sync version which returns payload directly)
|
||||
# Use timeout=0 to let the device suggest timeout, with min_timeout as floor
|
||||
log.debug("Querying repeater status...")
|
||||
success, payload, err = await query_repeater_with_retry(
|
||||
mc,
|
||||
contact,
|
||||
"req_status_sync",
|
||||
lambda: cmd.req_status_sync(contact, timeout=0, min_timeout=cfg.remote_timeout_s),
|
||||
)
|
||||
if success and payload and isinstance(payload, dict):
|
||||
status_ok = True
|
||||
# Insert all numeric fields from status response
|
||||
for key, value in payload.items():
|
||||
if isinstance(value, (int, float)):
|
||||
metrics[key] = float(value)
|
||||
log.debug(f"req_status_sync: {payload}")
|
||||
else:
|
||||
log.warn(f"req_status_sync failed: {err}")
|
||||
|
||||
# Update circuit breaker
|
||||
if status_ok:
|
||||
cb.record_success()
|
||||
log.debug("Circuit breaker: recorded success")
|
||||
else:
|
||||
cb.record_failure(cfg.remote_cb_fails, cfg.remote_cb_cooldown_s)
|
||||
log.debug(f"Circuit breaker: recorded failure ({cb.consecutive_failures}/{cfg.remote_cb_fails})")
|
||||
|
||||
except Exception as e:
|
||||
log.error(f"Error during collection: {e}")
|
||||
cb.record_failure(cfg.remote_cb_fails, cfg.remote_cb_cooldown_s)
|
||||
log.debug(f"Circuit breaker: recorded failure ({cb.consecutive_failures}/{cfg.remote_cb_fails})")
|
||||
|
||||
except Exception as e:
|
||||
log.error(f"Error during collection: {e}")
|
||||
cb.record_failure(cfg.remote_cb_fails, cfg.remote_cb_cooldown_s)
|
||||
|
||||
finally:
|
||||
# Close connection
|
||||
if hasattr(mc, "disconnect"):
|
||||
try:
|
||||
await mc.disconnect()
|
||||
except Exception:
|
||||
pass
|
||||
# Connection closed and lock released by context manager
|
||||
|
||||
# Print summary
|
||||
summary_parts = [f"ts={ts}"]
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
"""MeshCore network monitoring library."""
|
||||
|
||||
__version__ = "0.2.2" # x-release-please-version
|
||||
__version__ = "0.2.5" # x-release-please-version
|
||||
|
||||
@@ -1,7 +1,10 @@
|
||||
"""MeshCore client wrapper with safe command execution and contact lookup."""
|
||||
|
||||
import asyncio
|
||||
from typing import Any, Optional, Callable, Coroutine
|
||||
import fcntl
|
||||
from contextlib import asynccontextmanager
|
||||
from pathlib import Path
|
||||
from typing import Any, AsyncIterator, Callable, Coroutine, Optional
|
||||
|
||||
from .env import get_config
|
||||
from . import log
|
||||
@@ -100,6 +103,92 @@ async def connect_from_env() -> Optional[Any]:
|
||||
return None
|
||||
|
||||
|
||||
async def _acquire_lock_async(
|
||||
lock_file,
|
||||
timeout: float = 60.0,
|
||||
poll_interval: float = 0.1,
|
||||
) -> None:
|
||||
"""Acquire exclusive file lock without blocking the event loop.
|
||||
|
||||
Uses non-blocking LOCK_NB with async polling to avoid freezing the event loop.
|
||||
|
||||
Args:
|
||||
lock_file: Open file handle to lock
|
||||
timeout: Maximum seconds to wait for lock
|
||||
poll_interval: Seconds between lock attempts
|
||||
|
||||
Raises:
|
||||
TimeoutError: If lock cannot be acquired within timeout
|
||||
"""
|
||||
loop = asyncio.get_running_loop()
|
||||
deadline = loop.time() + timeout
|
||||
|
||||
while True:
|
||||
try:
|
||||
fcntl.flock(lock_file.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
return
|
||||
except BlockingIOError:
|
||||
if loop.time() >= deadline:
|
||||
raise TimeoutError(
|
||||
f"Could not acquire serial lock within {timeout}s. "
|
||||
"Another process may be using the serial port."
|
||||
)
|
||||
await asyncio.sleep(poll_interval)
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def connect_with_lock(
|
||||
lock_timeout: float = 60.0,
|
||||
) -> AsyncIterator[Optional[Any]]:
|
||||
"""Connect to MeshCore with serial port locking to prevent concurrent access.
|
||||
|
||||
For serial transport: Acquires exclusive file lock before connecting.
|
||||
For TCP/BLE: No locking needed (protocol handles multiple connections).
|
||||
|
||||
Args:
|
||||
lock_timeout: Maximum seconds to wait for serial lock
|
||||
|
||||
Yields:
|
||||
MeshCore client instance, or None if connection failed
|
||||
"""
|
||||
cfg = get_config()
|
||||
lock_file = None
|
||||
mc = None
|
||||
needs_lock = cfg.mesh_transport.lower() == "serial"
|
||||
|
||||
try:
|
||||
if needs_lock:
|
||||
lock_path: Path = cfg.state_dir / "serial.lock"
|
||||
lock_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Use 'a' mode: doesn't truncate, creates if missing
|
||||
lock_file = open(lock_path, "a")
|
||||
try:
|
||||
await _acquire_lock_async(lock_file, timeout=lock_timeout)
|
||||
log.debug(f"Acquired serial lock: {lock_path}")
|
||||
except Exception:
|
||||
# If lock acquisition fails, close file before re-raising
|
||||
lock_file.close()
|
||||
lock_file = None
|
||||
raise
|
||||
|
||||
mc = await connect_from_env()
|
||||
yield mc
|
||||
|
||||
finally:
|
||||
# Disconnect first (while we still hold the lock)
|
||||
if mc is not None and hasattr(mc, "disconnect"):
|
||||
try:
|
||||
await mc.disconnect()
|
||||
except Exception as e:
|
||||
log.debug(f"Error during disconnect (ignored): {e}")
|
||||
|
||||
# Release lock by closing the file (close() auto-releases flock)
|
||||
if lock_file is not None:
|
||||
lock_file.close()
|
||||
log.debug("Released serial lock")
|
||||
|
||||
|
||||
async def run_command(
|
||||
mc: Any,
|
||||
cmd_coro: Coroutine,
|
||||
|
||||
Reference in New Issue
Block a user