Compare commits

...

18 Commits

Author SHA1 Message Date
Jorijn Schrijvershof
30de7c20f3 Merge pull request #20 from jorijn/release-please--branches--main--components--meshcore-stats
chore(main): release 0.2.4
2026-01-05 09:32:23 +01:00
Jorijn Schrijvershof
19fa04c202 chore(main): release 0.2.4 2026-01-05 09:24:32 +01:00
Jorijn Schrijvershof
6ac52629d3 docs: rewrite README with Docker-first installation guide
Completely restructured README.md to prioritize Docker installation:
- Added Quick Start section with copy-pasteable commands
- Reorganized with Docker as recommended, manual as alternative
- Added Platform Notes (Linux/macOS/Windows) with collapsible sections
- Streamlined configuration reference table
- Added troubleshooting table with common issues
- Included resource requirements and backup instructions
- Moved metrics reference to CLAUDE.md (linked from docs section)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 09:24:07 +01:00
Jorijn Schrijvershof
5b43f9ed12 Merge pull request #19 from jorijn/release-please--branches--main--components--meshcore-stats
chore(main): release 0.2.3
2026-01-05 08:58:53 +01:00
Jorijn Schrijvershof
0fe6c66ed8 chore(main): release 0.2.3 2026-01-05 08:57:16 +01:00
Jorijn Schrijvershof
2730a9d906 ci: use fine-grained PAT for release-please to trigger Docker builds
GITHUB_TOKEN cannot trigger other workflows by design. Using a
fine-grained PAT (RELEASE_PLEASE_TOKEN) scoped to this repo only
allows releases to properly trigger the docker-publish workflow.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 08:57:01 +01:00
Jorijn Schrijvershof
5e5d63fca3 Merge pull request #18 from jorijn/release-please--branches--main--components--meshcore-stats
chore(main): release 0.2.2
2026-01-05 08:48:14 +01:00
github-actions[bot]
92b2286e18 chore(main): release 0.2.2 2026-01-05 07:47:54 +00:00
Jorijn Schrijvershof
6776c2c429 fix: move serial device config to override file
Docker Compose merges arrays instead of replacing them, so having
a default device in docker-compose.yml caused conflicts with override
files. Serial device configuration now requires docker-compose.override.yml,
which is cleaner since:
- Device paths vary per system
- TCP transport users don't need devices at all

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 08:47:34 +01:00
Jorijn Schrijvershof
e3a1df4aa6 fix(ci): skip smoke test when manual push is disabled 2026-01-05 08:41:58 +01:00
Jorijn Schrijvershof
f7f3889e41 Merge pull request #16 from jorijn/release-please--branches--main--components--meshcore-stats
chore(main): release 0.2.1
2026-01-05 08:24:24 +01:00
github-actions[bot]
be86404d8b chore(main): release 0.2.1 2026-01-05 07:23:46 +00:00
Jorijn Schrijvershof
7ba5ed37d4 Merge pull request #17 from jorijn/feat/docker-containerization
feat: add Docker containerization with GitHub Actions CI/CD
2026-01-05 08:23:34 +01:00
Jorijn Schrijvershof
046d7ead70 refactor: rename docker-compose.development.yml to docker-compose.dev.yml
- Rename to shorter docker-compose.dev.yml
- Add docker-compose.override.yml to .gitignore for local customizations
- Document override file pattern for device path customization
- Update README and CLAUDE.md with new naming convention

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 08:16:26 +01:00
Jorijn Schrijvershof
ee959d95a1 fix: improve Docker configuration and documentation
- Change Python path defaults to Docker paths (/data/state, /out)
- Remove STATE_DIR/OUT_DIR from Dockerfile ENV (Python defaults now correct)
- Remove REPEATER_FETCH_ACL feature (unsupported)
- Fix nginx tmpfs permissions with uid=101,gid=101
- Remove Ofelia [global] save=true (caused config parse error)
- Switch to bind mounts for ./out instead of named volume
- Comment out devices section (not available on macOS Docker)
- Add TCP and BLE transport options to meshcore.conf.example
- Document correct macOS socat command for serial-over-TCP
- Update README with macOS Docker workaround instructions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 07:56:58 +01:00
Jorijn Schrijvershof
7a181e4b1a feat: add Docker containerization with GitHub Actions CI/CD
- Multi-stage Dockerfile with Python 3.12 + Ofelia scheduler
- docker-compose.yml for production (ghcr.io image)
- docker-compose.development.yml for local builds
- GitHub Actions workflow for multi-arch builds (amd64/arm64)
- Security hardening: non-root user, cap_drop, read_only filesystem
- Trivy vulnerability scanning and SBOM generation
- Nightly rebuilds for OS security patches

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 07:12:59 +01:00
Jorijn Schrijvershof
2bf04ce3f7 Merge branch 'main' of github.com:jorijn/meshcore-stats
* 'main' of github.com:jorijn/meshcore-stats:
  chore(main): release 0.2.0
2026-01-04 20:15:33 +01:00
Jorijn Schrijvershof
f47916cf82 chore: clean up permissions in settings.local.json 2026-01-04 20:14:49 +01:00
19 changed files with 1206 additions and 377 deletions

View File

@@ -1,29 +1,20 @@
{
"permissions": {
"allow": [
"Bash(meshcore-cli:*)",
"Bash(.direnv/python-3.14/bin/pip index:*)",
"Bash(.direnv/python-3.14/bin/pip install:*)",
"Bash(.direnv/python-3.12/bin/pip:*)",
"Bash(rrdtool info:*)",
"Bash(rrdtool fetch:*)",
"Bash(cat:*)",
"Bash(xargs cat:*)",
"Bash(done)",
"Bash(ls:*)",
"Bash(source .envrc)",
"Bash(.direnv/python-3.12/bin/python:*)",
"Bash(xargs:*)",
"Bash(git add:*)",
"Bash(git commit:*)",
"Bash(git push)",
"Bash(find:*)",
"Bash(tree:*)",
"Skill(frontend-design:frontend-design)",
"Skill(frontend-design:frontend-design:*)",
"Bash(direnv exec:*)",
"Skill(frontend-design)",
"Skill(frontend-design:*)"
"Skill(frontend-design:*)",
"Bash(gh run view:*)",
"Bash(gh run list:*)",
"Bash(gh release view:*)",
"Bash(gh release list:*)",
"Bash(gh workflow list:*)"
]
}
}

42
.dockerignore Normal file
View File

@@ -0,0 +1,42 @@
# Git
.git
.gitignore
# Python
__pycache__
*.py[cod]
*$py.class
*.so
.Python
.venv
venv
ENV
env
# IDE
.idea
.vscode
*.swp
*.swo
*~
# Project directories (generated/runtime)
data/
out/
# Configuration (use environment variables in Docker)
.envrc
.direnv
meshcore.conf
# Documentation
docs/
*.md
!README.md
# Development files
.claude/
*.log
# macOS
.DS_Store

234
.github/workflows/docker-publish.yml vendored Normal file
View File

@@ -0,0 +1,234 @@
# Build and publish Docker images to GitHub Container Registry
#
# Triggers:
# - On release: Build with version tags (X.Y.Z, X.Y, latest)
# - On schedule: Rebuild all tags with fresh base image (OS patches)
# - Manual: For testing, optional push
#
# Security:
# - All actions pinned by SHA
# - Vulnerability scanning with Trivy
# - SBOM and provenance attestation
name: Docker Build and Publish
on:
release:
types: [published]
schedule:
# Daily at 4 AM UTC - rebuild with fresh base image
- cron: "0 4 * * *"
workflow_dispatch:
inputs:
push:
description: "Push image to registry"
required: false
default: false
type: boolean
permissions:
contents: read
packages: write
id-token: write
attestations: write
concurrency:
group: docker-${{ github.ref }}
cancel-in-progress: true
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
build:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
# For nightly builds, get the latest release version
- name: Get latest release version
id: get-version
if: github.event_name == 'schedule'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
# Get latest release tag
LATEST_TAG=$(gh release view --json tagName -q '.tagName' 2>/dev/null || echo "")
if [ -z "$LATEST_TAG" ]; then
echo "No releases found, skipping nightly build"
echo "skip=true" >> $GITHUB_OUTPUT
exit 0
fi
# Strip 'v' prefix if present
VERSION=$(echo "$LATEST_TAG" | sed 's/^v//')
echo "version=$VERSION" >> $GITHUB_OUTPUT
echo "skip=false" >> $GITHUB_OUTPUT
- name: Skip if no releases
if: github.event_name == 'schedule' && steps.get-version.outputs.skip == 'true'
run: |
echo "No releases found, skipping nightly build"
exit 0
- name: Set up QEMU
if: "!(github.event_name == 'schedule' && steps.get-version.outputs.skip == 'true')"
uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3.7.0
- name: Set up Docker Buildx
if: "!(github.event_name == 'schedule' && steps.get-version.outputs.skip == 'true')"
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
- name: Log in to Container Registry
if: "!(github.event_name == 'schedule' && steps.get-version.outputs.skip == 'true')"
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
# Generate tags based on event type
- name: Extract metadata (release)
id: meta-release
if: github.event_name == 'release'
uses: docker/metadata-action@c299e40c65443455700f0fdfc63efafe5b349051 # v5.10.0
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
# X.Y.Z
type=semver,pattern={{version}}
# X.Y
type=semver,pattern={{major}}.{{minor}}
# latest
type=raw,value=latest
- name: Extract metadata (nightly)
id: meta-nightly
if: github.event_name == 'schedule' && steps.get-version.outputs.skip != 'true'
uses: docker/metadata-action@c299e40c65443455700f0fdfc63efafe5b349051 # v5.10.0
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
# Rebuild version tags with OS patches
type=raw,value=${{ steps.get-version.outputs.version }}
# Nightly tags
type=raw,value=nightly
type=raw,value=nightly-{{date 'YYYYMMDD'}}
# Also update latest with security patches
type=raw,value=latest
- name: Extract metadata (manual)
id: meta-manual
if: github.event_name == 'workflow_dispatch'
uses: docker/metadata-action@c299e40c65443455700f0fdfc63efafe5b349051 # v5.10.0
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=sha,prefix=sha-
# Build image (release - with cache)
- name: Build and push (release)
id: build-release
if: github.event_name == 'release'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta-release.outputs.tags }}
labels: ${{ steps.meta-release.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
provenance: true
sbom: true
# Build image (nightly - no cache, fresh base)
- name: Build and push (nightly)
id: build-nightly
if: github.event_name == 'schedule' && steps.get-version.outputs.skip != 'true'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta-nightly.outputs.tags }}
labels: ${{ steps.meta-nightly.outputs.labels }}
pull: true
no-cache: true
provenance: true
sbom: true
# Build image (manual)
- name: Build and push (manual)
id: build-manual
if: github.event_name == 'workflow_dispatch'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: .
platforms: linux/amd64,linux/arm64
push: ${{ inputs.push }}
tags: ${{ steps.meta-manual.outputs.tags }}
labels: ${{ steps.meta-manual.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
provenance: true
sbom: true
# Determine image tag for scanning and testing
- name: Determine image tag
id: image-tag
if: "!(github.event_name == 'schedule' && steps.get-version.outputs.skip == 'true')"
run: |
if [ "${{ github.event_name }}" = "release" ]; then
# Strip 'v' prefix to match semver tag format from metadata-action
echo "tag=$(echo '${{ github.event.release.tag_name }}' | sed 's/^v//')" >> $GITHUB_OUTPUT
elif [ "${{ github.event_name }}" = "schedule" ]; then
echo "tag=nightly" >> $GITHUB_OUTPUT
else
echo "tag=sha-${{ github.sha }}" >> $GITHUB_OUTPUT
fi
# Vulnerability scanning
- name: Run Trivy vulnerability scanner
if: "!(github.event_name == 'schedule' && steps.get-version.outputs.skip == 'true')"
uses: aquasecurity/trivy-action@b6643a29fecd7f34b3597bc6acb0a98b03d33ff8 # v0.33.1
with:
image-ref: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.image-tag.outputs.tag }}
format: "sarif"
output: "trivy-results.sarif"
severity: "CRITICAL,HIGH"
continue-on-error: true
- name: Upload Trivy scan results
if: "!(github.event_name == 'schedule' && steps.get-version.outputs.skip == 'true')"
uses: github/codeql-action/upload-sarif@6e4b8622b82fab3c6ad2a7814fad1effc7615bc8 # v3.28.4
with:
sarif_file: "trivy-results.sarif"
continue-on-error: true
# Smoke test - verify image runs correctly
# Skip for manual runs when push is disabled (image not available to pull)
- name: Smoke test
if: "!(github.event_name == 'schedule' && steps.get-version.outputs.skip == 'true') && !(github.event_name == 'workflow_dispatch' && inputs.push == false)"
run: |
IMAGE_TAG="${{ steps.image-tag.outputs.tag }}"
docker pull ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${IMAGE_TAG}
# Test that Python and key modules are available
docker run --rm ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${IMAGE_TAG} \
python -c "from meshmon.db import init_db; from meshmon.env import get_config; print('Smoke test passed')"
# Attestation (releases only)
- name: Generate attestation
if: github.event_name == 'release'
uses: actions/attest-build-provenance@46a583fd92dfbf46b772907a9740f888f4324bb9 # v3.1.0
with:
subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
subject-digest: ${{ steps.build-release.outputs.digest }}
push-to-registry: true

View File

@@ -5,6 +5,17 @@ on:
branches:
- main
# Note: We use a fine-grained PAT (RELEASE_PLEASE_TOKEN) instead of GITHUB_TOKEN
# because GITHUB_TOKEN cannot trigger other workflows (like docker-publish.yml).
# This is a GitHub security feature to prevent infinite workflow loops.
#
# The PAT requires these permissions (scoped to this repository only):
# - Contents: Read and write (for creating releases and pushing tags)
# - Pull requests: Read and write (for creating/updating release PRs)
#
# To rotate: Settings > Developer settings > Fine-grained tokens
# Recommended rotation: Every 90 days
permissions:
contents: write
pull-requests: write
@@ -16,6 +27,6 @@ jobs:
- name: Release Please
uses: googleapis/release-please-action@v4
with:
token: ${{ secrets.GITHUB_TOKEN }}
token: ${{ secrets.RELEASE_PLEASE_TOKEN }}
config-file: release-please-config.json
manifest-file: .release-please-manifest.json

3
.gitignore vendored
View File

@@ -17,6 +17,9 @@ build/
.env
meshcore.conf
# Docker local overrides
docker-compose.override.yml
# Data directories (keep structure, ignore content)
data/snapshots/companion/**/*.json
data/snapshots/repeater/**/*.json

View File

@@ -1,3 +1,3 @@
{
".": "0.2.0"
".": "0.2.4"
}

View File

@@ -4,6 +4,51 @@ All notable changes to this project will be documented in this file.
This changelog is automatically generated by [release-please](https://github.com/googleapis/release-please) based on [Conventional Commits](https://www.conventionalcommits.org/).
## [0.2.4](https://github.com/jorijn/meshcore-stats/compare/v0.2.3...v0.2.4) (2026-01-05)
### Documentation
* rewrite README with Docker-first installation guide ([6ac5262](https://github.com/jorijn/meshcore-stats/commit/6ac52629d3025db69f9334d3185b97ce16cd3e4b))
## [0.2.3](https://github.com/jorijn/meshcore-stats/compare/v0.2.2...v0.2.3) (2026-01-05)
### Continuous Integration
* use fine-grained PAT for release-please to trigger Docker builds ([2730a9d](https://github.com/jorijn/meshcore-stats/commit/2730a9d906eeb5761af29dd69e8d4ebbfca50491))
## [0.2.2](https://github.com/jorijn/meshcore-stats/compare/v0.2.1...v0.2.2) (2026-01-05)
### Bug Fixes
* **ci:** skip smoke test when manual push is disabled ([e3a1df4](https://github.com/jorijn/meshcore-stats/commit/e3a1df4aa64bf87c32848be0d5c5e5ce16968186))
* move serial device config to override file ([6776c2c](https://github.com/jorijn/meshcore-stats/commit/6776c2c4293b71f4649a42dcf6c517f3b44469b5))
## [0.2.1](https://github.com/jorijn/meshcore-stats/compare/v0.2.0...v0.2.1) (2026-01-05)
### Features
* add Docker containerization with GitHub Actions CI/CD ([7ba5ed3](https://github.com/jorijn/meshcore-stats/commit/7ba5ed37d40d7c5e0a7e206cedcf6f70096759e5))
* add Docker containerization with GitHub Actions CI/CD ([7a181e4](https://github.com/jorijn/meshcore-stats/commit/7a181e4b1ac581b2b897cd70aa77c5c983c6e80a))
### Bug Fixes
* improve Docker configuration and documentation ([ee959d9](https://github.com/jorijn/meshcore-stats/commit/ee959d95a18afeeab47d41cc85ac435ba2a87016))
### Miscellaneous Chores
* clean up permissions in settings.local.json ([f47916c](https://github.com/jorijn/meshcore-stats/commit/f47916cf82118bb30d80d901773f0bfaf2de315a))
### Code Refactoring
* rename docker-compose.development.yml to docker-compose.dev.yml ([046d7ea](https://github.com/jorijn/meshcore-stats/commit/046d7ead708cc2807f10bcd36e1e1cf7494a8f45))
## [0.2.0](https://github.com/jorijn/meshcore-stats/compare/v0.1.0...v0.2.0) (2026-01-04)

100
CLAUDE.md
View File

@@ -171,6 +171,106 @@ Phase 3: Render → Static HTML site (inline SVG)
Phase 4: Render → Reports (monthly/yearly statistics)
```
## Docker Architecture
The project provides Docker containerization for easy deployment. Two containers work together:
```
┌─────────────────────────────────────────────────────────────────┐
│ Docker Compose │
│ ┌─────────────────────┐ ┌─────────────────────────────────┐ │
│ │ meshcore-stats │ │ nginx │ │
│ │ ┌───────────────┐ │ │ │ │
│ │ │ Ofelia │ │ │ Serves static site on :8080 │ │
│ │ │ (scheduler) │ │ │ │ │
│ │ └───────┬───────┘ │ └──────────────▲──────────────────┘ │
│ │ │ │ │ │
│ │ ┌──────▼──────┐ │ ┌─────────┴─────────┐ │
│ │ │ Python │ │ │ output_data │ │
│ │ │ Scripts │───┼────────►│ (named volume) │ │
│ │ └─────────────┘ │ └───────────────────┘ │
│ │ │ │ │
│ └──────────┼──────────┘ │
│ │ │
│ ┌────────▼────────┐ │
│ │ ./data/state │ │
│ │ (bind mount) │ │
│ └─────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
```
### Container Files
| File | Purpose |
|------|---------|
| `Dockerfile` | Multi-stage build: Python + Ofelia scheduler |
| `docker-compose.yml` | Production deployment using published ghcr.io image |
| `docker-compose.dev.yml` | Development override for local builds |
| `docker-compose.override.yml` | Local overrides (gitignored) |
| `docker/ofelia.ini` | Scheduler configuration (cron jobs) |
| `docker/nginx.conf` | nginx configuration for static site serving |
| `.dockerignore` | Files excluded from Docker build context |
### Docker Compose Files
**Production** (`docker-compose.yml`):
- Uses published image from `ghcr.io/jorijn/meshcore-stats`
- Image version managed by release-please via `x-release-please-version` placeholder
- Suitable for end users
**Development** (`docker-compose.dev.yml`):
- Override file that builds locally instead of pulling from registry
- Mounts `src/` and `scripts/` for live code changes
- Usage: `docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build`
**Local overrides** (`docker-compose.override.yml`):
- Gitignored file for local customizations (e.g., device paths, env_file)
- Automatically merged when running `docker compose up`
### Ofelia Scheduler
[Ofelia](https://github.com/mcuadros/ofelia) is a lightweight job scheduler designed for Docker. It replaces cron for container environments.
Jobs configured in `docker/ofelia.ini`:
- `collect-companion`: Every minute (with `no-overlap=true`)
- `collect-repeater`: Every 15 minutes at :01, :16, :31, :46 (with `no-overlap=true`)
- `render-charts`: Every 5 minutes
- `render-site`: Every 5 minutes
- `render-reports`: Daily at midnight
- `db-maintenance`: Monthly at 3 AM on the 1st
### GitHub Actions Workflow
`.github/workflows/docker-publish.yml` builds and publishes Docker images:
| Trigger | Tags Created |
|---------|--------------|
| Release | `X.Y.Z`, `X.Y`, `latest` |
| Nightly (4 AM UTC) | Rebuilds all version tags + `nightly`, `nightly-YYYYMMDD` |
| Manual | `sha-xxxxxx` |
**Nightly rebuilds** ensure version tags always include the latest OS security patches. This is a common pattern used by official Docker images (nginx, postgres, node). Users needing reproducibility should pin by SHA digest or use dated nightly tags.
All GitHub Actions are pinned by full SHA for security. Dependabot can be configured to update these automatically.
### Version Placeholder
The version in `docker-compose.yml` uses release-please's placeholder syntax:
```yaml
image: ghcr.io/jorijn/meshcore-stats:0.3.0 # x-release-please-version
```
This is automatically updated when a new release is created.
### Agent Review Guidelines
When reviewing Docker-related changes, always provide the **full plan or implementation** to review agents. Do not summarize or abbreviate - agents need complete context to provide accurate feedback.
Relevant agents for Docker reviews:
- **k8s-security-reviewer**: Container security, RBAC, secrets handling
- **cicd-pipeline-specialist**: GitHub Actions workflows, build pipelines
- **python-code-reviewer**: Dockerfile Python-specific issues (venv PATH, runtime libs)
## Directory Structure
```

112
Dockerfile Normal file
View File

@@ -0,0 +1,112 @@
# =============================================================================
# Stage 1: Build dependencies
# =============================================================================
FROM python:3.12-slim-bookworm AS builder
# Ofelia version and checksums (verified from GitHub releases)
ARG OFELIA_VERSION=0.3.12
ARG TARGETARCH
ARG OFELIA_SHA256_AMD64=cf06d2199abafbd3aa5afe0f8266e478818faacd11555b99200707321035c931
ARG OFELIA_SHA256_ARM64=57760ef7f17a2cd55b5b1e1946f79b91b24bde40d47e81a0d75fd1470d883c1a
# Install build dependencies for Python packages
RUN apt-get update && apt-get install -y --no-install-recommends \
gcc \
libfreetype6-dev \
libpng-dev \
curl \
&& rm -rf /var/lib/apt/lists/*
# Download and verify Ofelia binary in builder stage (keeps curl out of runtime)
RUN set -ex; \
if [ "$TARGETARCH" = "amd64" ]; then \
OFELIA_SHA256="$OFELIA_SHA256_AMD64"; \
elif [ "$TARGETARCH" = "arm64" ]; then \
OFELIA_SHA256="$OFELIA_SHA256_ARM64"; \
else \
echo "Unsupported architecture: $TARGETARCH" && exit 1; \
fi; \
curl -fsSL "https://github.com/mcuadros/ofelia/releases/download/v${OFELIA_VERSION}/ofelia_${OFELIA_VERSION}_linux_${TARGETARCH}.tar.gz" -o /tmp/ofelia.tar.gz \
&& echo "${OFELIA_SHA256} /tmp/ofelia.tar.gz" | sha256sum -c - \
&& tar -xzf /tmp/ofelia.tar.gz -C /usr/local/bin ofelia \
&& rm /tmp/ofelia.tar.gz \
&& chmod +x /usr/local/bin/ofelia
# Create virtual environment
RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
# Install Python dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir --upgrade pip && \
pip install --no-cache-dir -r requirements.txt
# =============================================================================
# Stage 2: Runtime
# =============================================================================
FROM python:3.12-slim-bookworm
# OCI Labels
LABEL org.opencontainers.image.source="https://github.com/jorijn/meshcore-stats"
LABEL org.opencontainers.image.description="MeshCore Stats - LoRa mesh network monitoring"
LABEL org.opencontainers.image.licenses="MIT"
# Install runtime dependencies
# - tini: init system for proper signal handling
# - libfreetype6, libpng16-16: matplotlib runtime libraries
# - fontconfig, fonts-dejavu-core: fonts for chart text rendering
RUN apt-get update && apt-get install -y --no-install-recommends \
tini \
libfreetype6 \
libpng16-16 \
fontconfig \
fonts-dejavu-core \
&& rm -rf /var/lib/apt/lists/* \
# Build font cache for matplotlib
&& fc-cache -f \
# Remove setuid/setgid binaries for security
&& find / -perm /6000 -type f -exec chmod a-s {} \; 2>/dev/null || true
# Create non-root user with dialout group for serial access
RUN groupadd -g 1000 meshmon \
&& useradd -u 1000 -g meshmon -G dialout -s /sbin/nologin meshmon \
&& mkdir -p /data/state /out /tmp/matplotlib \
&& chown -R meshmon:meshmon /data /out /tmp/matplotlib
# Copy Ofelia binary from builder (keeps curl out of runtime image)
COPY --from=builder /usr/local/bin/ofelia /usr/local/bin/ofelia
# Copy virtual environment from builder
COPY --from=builder /opt/venv /opt/venv
# Copy application code
COPY --chown=meshmon:meshmon src/ /app/src/
COPY --chown=meshmon:meshmon scripts/ /app/scripts/
COPY --chown=meshmon:meshmon docker/ofelia.ini /app/ofelia.ini
# Environment configuration
# - PATH: Include venv so Ofelia can run Python
# - PYTHONPATH: Allow imports from src/meshmon
# - PYTHONUNBUFFERED: Ensure logs are output immediately
# - PYTHONDONTWRITEBYTECODE: Don't create .pyc files
# - MPLCONFIGDIR: Matplotlib font cache directory
ENV PATH="/opt/venv/bin:$PATH" \
PYTHONPATH=/app/src \
PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
MPLCONFIGDIR=/tmp/matplotlib
WORKDIR /app
# Run as non-root user
USER meshmon
# Use tini as init system for proper signal handling
ENTRYPOINT ["/usr/bin/tini", "--"]
# Run Ofelia scheduler
CMD ["ofelia", "daemon", "--config=/app/ofelia.ini"]
# Health check - verify database is accessible
HEALTHCHECK --interval=5m --timeout=30s --start-period=60s --retries=3 \
CMD python -c "import sqlite3; sqlite3.connect('/data/state/metrics.db').execute('SELECT 1')" || exit 1

573
README.md
View File

@@ -1,6 +1,6 @@
# MeshCore Stats
A Python-based monitoring system for a MeshCore repeater node and its companion. Collects metrics from both devices, stores them in a SQLite database, and generates a static website with interactive SVG charts and statistics.
A monitoring system for MeshCore LoRa mesh networks. Collects metrics from companion and repeater nodes, stores them in SQLite, and generates a static dashboard with interactive charts.
**Live demo:** [meshcore.jorijn.com](https://meshcore.jorijn.com)
@@ -9,346 +9,359 @@ A Python-based monitoring system for a MeshCore repeater node and its companion.
<img src="docs/screenshot-2.png" width="49%" alt="MeshCore Stats Reports">
</p>
## Features
## Quick Start
- **Data Collection** - Collect metrics from companion (local) and repeater (remote) nodes
- **Chart Rendering** - Generate interactive SVG charts from the database using matplotlib
- **Static Site** - Generate a static HTML website with day/week/month/year views
- **Reports** - Generate monthly and yearly statistics reports
## Requirements
### Python Dependencies
- Python 3.10+
- meshcore >= 2.2.3
- pyserial >= 3.5
- jinja2 >= 3.1.0
- matplotlib >= 3.8.0
### System Dependencies
- sqlite3 (for database maintenance script)
## Setup
### 1. Create Virtual Environment
> **Linux only** - macOS and Windows users see [Platform Notes](#platform-notes) first.
```bash
cd /path/to/meshcore-stats
# Clone and configure
git clone https://github.com/jorijn/meshcore-stats.git
cd meshcore-stats
cp meshcore.conf.example meshcore.conf
# Edit meshcore.conf with your repeater name and password
# Create data directories (container runs as UID 1000)
mkdir -p data/state out
sudo chown -R 1000:1000 data out
# Add your serial device
cat > docker-compose.override.yml << 'EOF'
services:
meshcore-stats:
devices:
- /dev/ttyACM0:/dev/ttyACM0
EOF
# Start
docker compose up -d
# Verify it's working. The various collection and render jobs will trigger after a few minutes.
docker compose ps
docker compose logs meshcore-stats | head -20
# View dashboard at http://localhost:8080
```
## Features
- **Data Collection** - Metrics from local companion and remote repeater nodes
- **Interactive Charts** - SVG charts with day/week/month/year views and tooltips
- **Statistics Reports** - Monthly and yearly report generation
- **Light/Dark Theme** - Automatic theme switching based on system preference
## Prerequisites
- Docker and Docker Compose V2
- MeshCore companion node connected via USB serial
- Remote repeater node reachable via LoRa from the companion
**Resource requirements:** ~100MB memory, ~100MB disk per year of data.
## Installation
### Docker (Recommended)
#### 1. Clone the Repository
```bash
git clone https://github.com/jorijn/meshcore-stats.git
cd meshcore-stats
```
#### 2. Configure
Copy the example configuration and edit it:
```bash
cp meshcore.conf.example meshcore.conf
```
**Minimal required settings:**
```ini
# Repeater identity (required)
REPEATER_NAME=Your Repeater Name
REPEATER_PASSWORD=your-admin-password
# Display names
REPEATER_DISPLAY_NAME=My Repeater
COMPANION_DISPLAY_NAME=My Companion
```
See [meshcore.conf.example](meshcore.conf.example) for all available options.
#### 3. Create Data Directories
```bash
mkdir -p data/state out
sudo chown -R 1000:1000 data out
```
The container runs as UID 1000, so directories must be writable by this user. If `sudo` is not available, you can relaxed the permissions using `chmod 777 data out`, but this is less secure.
#### 4. Configure Serial Device
Create `docker-compose.override.yml` to specify your serial device:
```yaml
services:
meshcore-stats:
devices:
- /dev/ttyACM0:/dev/ttyACM0
```
Ensure your user has serial port access:
```bash
sudo usermod -aG dialout $USER
# Log out and back in for changes to take effect
```
#### 5. Start the Containers
```bash
docker compose up -d
```
After the various collection and render jobs has run, the dashboard will be available at **http://localhost:8080**.
#### Verify Installation
```bash
# Check container status
docker compose ps
# View logs
docker compose logs -f meshcore-stats
```
### Common Docker Commands
```bash
# View real-time logs
docker compose logs -f meshcore-stats
# Restart after configuration changes
docker compose restart meshcore-stats
# Update to latest version (database migrations are automatic)
docker compose pull && docker compose up -d
# Stop all containers
docker compose down
# Backup database
cp data/state/metrics.db data/state/metrics.db.backup
```
> **Note**: `docker compose down` preserves your data. Use `docker compose down -v` only if you want to delete everything.
### Manual Installation (Alternative)
For environments where Docker is not available.
#### Requirements
- Python 3.10+
- SQLite3
#### Setup
```bash
cd meshcore-stats
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```
### 2. Configure
Copy the example configuration file and customize it:
```bash
cp meshcore.conf.example meshcore.conf
# Edit meshcore.conf with your settings
```
The configuration file is automatically loaded by the scripts. Key settings to configure:
#### Cron Setup
- **Connection**: `MESH_SERIAL_PORT`, `MESH_TRANSPORT`
- **Repeater Identity**: `REPEATER_NAME`, `REPEATER_PASSWORD`
- **Display Names**: `REPEATER_DISPLAY_NAME`, `COMPANION_DISPLAY_NAME`
- **Location**: `REPORT_LOCATION_NAME`, `REPORT_LAT`, `REPORT_LON`, `REPORT_ELEV`
- **Hardware Info**: `REPEATER_HARDWARE`, `COMPANION_HARDWARE`
- **Radio Config**: `RADIO_FREQUENCY`, `RADIO_BANDWIDTH`, etc. (includes presets for different regions)
See `meshcore.conf.example` for all available options with documentation.
## Usage
### Manual Execution
```bash
cd /path/to/meshcore-stats
source .venv/bin/activate
# Collect companion data
python scripts/collect_companion.py
# Collect repeater data
python scripts/collect_repeater.py
# Generate static site (includes chart rendering)
python scripts/render_site.py
# Generate reports
python scripts/render_reports.py
```
The configuration is automatically loaded from `meshcore.conf`.
### Cron Setup
Add these entries to your crontab (`crontab -e`):
Add to your crontab (`crontab -e`):
```cron
# MeshCore Stats - adjust path as needed
MESHCORE=/home/user/meshcore-stats
MESHCORE=/path/to/meshcore-stats
# Every minute: collect companion data
# Companion: every minute
* * * * * cd $MESHCORE && flock -w 60 /tmp/meshcore.lock .venv/bin/python scripts/collect_companion.py
# Every 15 minutes: collect repeater data
# Repeater: every 15 minutes
1,16,31,46 * * * * cd $MESHCORE && flock -w 60 /tmp/meshcore.lock .venv/bin/python scripts/collect_repeater.py
# Every 5 minutes: render site
# Charts: every 5 minutes
*/5 * * * * cd $MESHCORE && .venv/bin/python scripts/render_charts.py
# Site: every 5 minutes
*/5 * * * * cd $MESHCORE && .venv/bin/python scripts/render_site.py
# Daily at midnight: generate reports
# Reports: daily at midnight
0 0 * * * cd $MESHCORE && .venv/bin/python scripts/render_reports.py
# Monthly at 3 AM on the 1st: database maintenance
0 3 1 * * $MESHCORE/scripts/db_maintenance.sh
```
**Notes:**
- `cd $MESHCORE` is required because paths in the config are relative to the project root
- `flock` prevents USB serial conflicts when companion and repeater collection overlap
Serve the `out/` directory with any web server.
### Docker / Container Usage
## Platform Notes
When running in Docker, you can skip the config file and pass environment variables directly:
<details>
<summary><strong>Linux</strong></summary>
Docker can access USB serial devices directly. Add your device to `docker-compose.override.yml`:
```yaml
services:
meshcore-stats:
devices:
- /dev/ttyACM0:/dev/ttyACM0
```
Common device paths:
- `/dev/ttyACM0` - Arduino/native USB
- `/dev/ttyUSB0` - USB-to-serial adapters
</details>
<details>
<summary><strong>macOS</strong></summary>
Docker Desktop for macOS runs in a Linux VM and **cannot directly access USB serial devices**.
**Option 1: TCP Bridge (Recommended)**
Expose the serial port over TCP using socat:
```bash
docker run -e MESH_SERIAL_PORT=/dev/ttyUSB0 -e REPEATER_NAME="My Repeater" ...
# Install socat
brew install socat
# Bridge serial to TCP (run in background)
socat TCP-LISTEN:5000,fork,reuseaddr OPEN:/dev/cu.usbserial-0001,rawer,nonblock,ispeed=115200,ospeed=115200
```
Environment variables always take precedence over `meshcore.conf`.
Configure in `meshcore.conf`:
### Serving the Site
The static site is generated in the `out/` directory. You can serve it with any web server:
```bash
# Simple Python server for testing
cd out && python3 -m http.server 8080
# Or configure nginx/caddy to serve the out/ directory
```ini
MESH_TRANSPORT=tcp
MESH_TCP_HOST=host.docker.internal
MESH_TCP_PORT=5000
```
## Project Structure
**Option 2: Native Installation**
```
meshcore-stats/
├── requirements.txt
├── README.md
├── meshcore.conf.example # Example configuration
├── meshcore.conf # Your configuration (create this)
├── src/meshmon/
│ ├── __init__.py
│ ├── env.py # Environment variable parsing
│ ├── log.py # Logging helper
│ ├── meshcore_client.py # MeshCore connection and commands
│ ├── db.py # SQLite database module
│ ├── retry.py # Retry logic and circuit breaker
│ ├── charts.py # Matplotlib SVG chart generation
│ ├── html.py # HTML rendering
│ ├── reports.py # Report generation
│ ├── metrics.py # Metric type definitions
│ ├── battery.py # Battery voltage to percentage conversion
│ ├── migrations/ # SQL schema migrations
│ │ ├── 001_initial_schema.sql
│ │ └── 002_eav_schema.sql
│ └── templates/ # Jinja2 HTML templates
├── scripts/
│ ├── collect_companion.py # Collect metrics from companion node
│ ├── collect_repeater.py # Collect metrics from repeater node
│ ├── render_charts.py # Generate SVG charts from database
│ ├── render_site.py # Generate static HTML site
│ ├── render_reports.py # Generate monthly/yearly reports
│ └── db_maintenance.sh # Database VACUUM/ANALYZE
├── data/
│ └── state/
│ ├── metrics.db # SQLite database (WAL mode)
│ └── repeater_circuit.json
└── out/ # Generated site
├── .htaccess # Apache config (DirectoryIndex, caching)
├── styles.css # Stylesheet
├── chart-tooltip.js # Chart tooltip enhancement
├── day.html # Repeater pages (entry point)
├── week.html
├── month.html
├── year.html
├── companion/
│ ├── day.html
│ ├── week.html
│ ├── month.html
│ └── year.html
└── reports/
├── index.html
├── repeater/ # YYYY/MM reports
└── companion/
```
Use the manual installation method with cron instead of Docker.
## Chart Features
</details>
Charts are rendered as inline SVG using matplotlib with the following features:
<details>
<summary><strong>Windows (WSL2)</strong></summary>
- **Theme Support**: Automatic light/dark mode via CSS `prefers-color-scheme`
- **Interactive Tooltips**: Hover to see exact values and timestamps
- **Data Point Indicator**: Visual marker shows position on the chart line
- **Mobile Support**: Touch-friendly tooltips
- **Statistics**: Min/Avg/Max values displayed below each chart
- **Period Views**: Day, week, month, and year time ranges
WSL2 and Docker Desktop for Windows cannot directly access COM ports.
## Troubleshooting
Use the TCP bridge approach (similar to macOS) or native installation.
### Serial Device Not Found
</details>
If you see "No serial ports found" or connection fails:
1. Check that your device is connected:
```bash
ls -la /dev/ttyUSB* /dev/ttyACM*
```
2. Check permissions (add user to dialout group):
```bash
sudo usermod -a -G dialout $USER
# Log out and back in for changes to take effect
```
3. Try specifying the port explicitly:
```bash
export MESH_SERIAL_PORT=/dev/ttyACM0
```
4. Check dmesg for device detection:
```bash
dmesg | tail -20
```
### Repeater Not Found
If the script cannot find the repeater contact:
1. The script will print all discovered contacts - check for the correct name
2. Verify REPEATER_NAME matches exactly (case-sensitive)
3. Try using REPEATER_KEY_PREFIX instead with the first 6-12 hex chars of the public key
### Circuit Breaker
If repeater collection shows "cooldown active":
1. This is normal after multiple failed remote requests
2. Wait for the cooldown period (default 1 hour) or reset manually:
```bash
rm data/state/repeater_circuit.json
```
## Environment Variables Reference
## Configuration Reference
| Variable | Default | Description |
|----------|---------|-------------|
| **Connection** | | |
| `MESH_TRANSPORT` | serial | Connection type: serial, tcp, ble |
| `MESH_SERIAL_PORT` | (auto) | Serial port path |
| `MESH_SERIAL_BAUD` | 115200 | Baud rate |
| `MESH_TCP_HOST` | localhost | TCP host |
| `MESH_TCP_PORT` | 5000 | TCP port |
| `MESH_BLE_ADDR` | - | BLE device address |
| `MESH_BLE_PIN` | - | BLE PIN |
| `MESH_DEBUG` | 0 | Enable debug output |
| **Repeater Identity** | | |
| `REPEATER_NAME` | - | Repeater advertised name |
| `REPEATER_KEY_PREFIX` | - | Repeater public key prefix |
| `REPEATER_PASSWORD` | - | Repeater login password |
| `REPEATER_FETCH_ACL` | 0 | Also fetch ACL from repeater |
| **Display Names** | | |
| `REPEATER_DISPLAY_NAME` | Repeater Node | Display name for repeater in UI |
| `COMPANION_DISPLAY_NAME` | Companion Node | Display name for companion in UI |
| `REPEATER_NAME` | *required* | Advertised name to find in contacts |
| `REPEATER_PASSWORD` | *required* | Admin password for repeater |
| `REPEATER_KEY_PREFIX` | - | Alternative to `REPEATER_NAME`: hex prefix of public key |
| **Connection** | | |
| `MESH_TRANSPORT` | serial | Transport type: `serial`, `tcp`, or `ble` |
| `MESH_SERIAL_PORT` | auto | Serial port path |
| `MESH_TCP_HOST` | localhost | TCP host (for TCP transport) |
| `MESH_TCP_PORT` | 5000 | TCP port (for TCP transport) |
| **Display** | | |
| `REPEATER_DISPLAY_NAME` | Repeater Node | Name shown in UI |
| `COMPANION_DISPLAY_NAME` | Companion Node | Name shown in UI |
| `REPEATER_HARDWARE` | LoRa Repeater | Hardware model for sidebar |
| `COMPANION_HARDWARE` | LoRa Node | Hardware model for sidebar |
| **Location** | | |
| `REPORT_LOCATION_NAME` | Your Location | Full location name for reports |
| `REPORT_LOCATION_SHORT` | Your Location | Short location for sidebar/meta |
| `REPORT_LAT` | 0.0 | Latitude in decimal degrees |
| `REPORT_LON` | 0.0 | Longitude in decimal degrees |
| `REPORT_LOCATION_NAME` | Your Location | Full location for reports |
| `REPORT_LAT` | 0.0 | Latitude |
| `REPORT_LON` | 0.0 | Longitude |
| `REPORT_ELEV` | 0.0 | Elevation |
| `REPORT_ELEV_UNIT` | m | Elevation unit: "m" or "ft" |
| **Hardware Info** | | |
| `REPEATER_HARDWARE` | LoRa Repeater | Repeater hardware model for sidebar |
| `COMPANION_HARDWARE` | LoRa Node | Companion hardware model for sidebar |
| **Radio Config** | | |
| `RADIO_FREQUENCY` | 869.618 MHz | Radio frequency for display |
| `RADIO_BANDWIDTH` | 62.5 kHz | Radio bandwidth for display |
| `RADIO_SPREAD_FACTOR` | SF8 | Spread factor for display |
| `RADIO_CODING_RATE` | CR8 | Coding rate for display |
| **Intervals** | | |
| `COMPANION_STEP` | 60 | Companion data collection interval (seconds) |
| `REPEATER_STEP` | 900 | Repeater data collection interval (seconds) |
| `REMOTE_TIMEOUT_S` | 10 | Remote request timeout |
| `REMOTE_RETRY_ATTEMPTS` | 2 | Max retry attempts |
| `REMOTE_RETRY_BACKOFF_S` | 4 | Retry backoff delay |
| `REMOTE_CB_FAILS` | 6 | Failures before circuit opens |
| `REMOTE_CB_COOLDOWN_S` | 3600 | Circuit breaker cooldown |
| **Paths** | | |
| `STATE_DIR` | ./data/state | State file path |
| `OUT_DIR` | ./out | Output site path |
| **Radio** (display only) | | |
| `RADIO_FREQUENCY` | 869.618 MHz | Frequency shown in sidebar |
| `RADIO_BANDWIDTH` | 62.5 kHz | Bandwidth |
| `RADIO_SPREAD_FACTOR` | SF8 | Spread factor |
## Metrics Reference
See [meshcore.conf.example](meshcore.conf.example) for all options with regional radio presets.
The system uses an EAV (Entity-Attribute-Value) schema where firmware field names are stored directly in the database. This allows new metrics to be captured automatically without schema changes.
## Troubleshooting
### Repeater Metrics
| Symptom | Cause | Solution |
|---------|-------|----------|
| "Permission denied" on serial port | User not in dialout group | `sudo usermod -aG dialout $USER` then re-login |
| Repeater shows "offline" status | No data or circuit breaker tripped | Check logs; delete `data/state/repeater_circuit.json` to reset |
| Empty charts | Not enough data collected | Wait for 2+ collection cycles |
| Container exits immediately | Missing or invalid configuration | Verify `meshcore.conf` exists and has required values |
| "No serial ports found" | Device not connected/detected | Check `ls /dev/tty*` and device permissions |
| Device path changed after reboot | USB enumeration order changed | Update path in `docker-compose.override.yml` or use udev rules |
| "database is locked" errors | Maintenance script running | Wait for completion; check if VACUUM is running |
| Metric | Type | Display Unit | Description |
|--------|------|--------------|-------------|
| `bat` | Gauge | Voltage (V) | Battery voltage (stored in mV, displayed as V) |
| `bat_pct` | Gauge | Battery (%) | Battery percentage (computed from voltage) |
| `last_rssi` | Gauge | RSSI (dBm) | Signal strength of last packet |
| `last_snr` | Gauge | SNR (dB) | Signal-to-noise ratio |
| `noise_floor` | Gauge | dBm | Background RF noise |
| `uptime` | Gauge | Days | Time since reboot (seconds ÷ 86400) |
| `tx_queue_len` | Gauge | Queue depth | TX queue length |
| `nb_recv` | Counter | Packets/min | Total packets received |
| `nb_sent` | Counter | Packets/min | Total packets transmitted |
| `airtime` | Counter | Seconds/min | TX airtime rate |
| `rx_airtime` | Counter | Seconds/min | RX airtime rate |
| `flood_dups` | Counter | Packets/min | Flood duplicate packets |
| `direct_dups` | Counter | Packets/min | Direct duplicate packets |
| `sent_flood` | Counter | Packets/min | Flood packets transmitted |
| `recv_flood` | Counter | Packets/min | Flood packets received |
| `sent_direct` | Counter | Packets/min | Direct packets transmitted |
| `recv_direct` | Counter | Packets/min | Direct packets received |
### Debug Logging
### Companion Metrics
```bash
# Enable debug mode in meshcore.conf
MESH_DEBUG=1
| Metric | Type | Display Unit | Description |
|--------|------|--------------|-------------|
| `battery_mv` | Gauge | Voltage (V) | Battery voltage (stored in mV, displayed as V) |
| `bat_pct` | Gauge | Battery (%) | Battery percentage (computed from voltage) |
| `contacts` | Gauge | Count | Known mesh nodes |
| `uptime_secs` | Gauge | Days | Time since reboot (seconds ÷ 86400) |
| `recv` | Counter | Packets/min | Total packets received |
| `sent` | Counter | Packets/min | Total packets transmitted |
# View detailed logs
docker compose logs -f meshcore-stats
```
### Metric Types
### Circuit Breaker
- **Gauge**: Instantaneous values stored as-is (battery voltage, RSSI, queue depth)
- **Counter**: Cumulative values where the rate of change is calculated (packets, airtime). Charts display per-minute rates.
The repeater collector uses a circuit breaker to avoid spamming LoRa when the repeater is unreachable. After multiple failures, it enters a cooldown period (default: 1 hour).
## Database
To reset manually:
Metrics are stored in a SQLite database at `data/state/metrics.db` with WAL mode enabled for concurrent read/write access.
```bash
rm data/state/repeater_circuit.json
docker compose restart meshcore-stats
```
### Schema Migrations
## Architecture
Database migrations are stored as SQL files in `src/meshmon/migrations/` and are applied automatically when the database is initialized. Migration files follow the naming convention `NNN_description.sql` (e.g., `001_initial_schema.sql`).
```
┌─────────────────┐ LoRa ┌─────────────────┐
│ Companion │◄────────────►│ Repeater │
│ (USB Serial) │ │ (Remote) │
└────────┬────────┘ └─────────────────┘
│ Serial/TCP
┌─────────────────┐
│ Docker Host │
│ ┌───────────┐ │
│ │ meshcore- │ │ ┌─────────┐
│ │ stats │──┼────►│ nginx │──► :8080
│ └───────────┘ │ └─────────┘
│ │ │
│ ▼ │
│ SQLite + SVG │
└─────────────────┘
```
## Public Instances
The system runs two containers:
- **meshcore-stats**: Collects data on schedule (Ofelia) and generates charts
- **nginx**: Serves the static dashboard
A list of publicly accessible MeshCore Stats installations. Want to add yours? [Open a pull request](https://github.com/jorijn/meshcore-stats/pulls)!
## Documentation
| URL | Hardware | Location |
|-----|----------|----------|
| [meshcore.jorijn.com](https://meshcore.jorijn.com) | SenseCAP Solar Node P1 Pro + 6.5dBi Mikrotik antenna | Oosterhout, The Netherlands |
- [docs/firmware-responses.md](docs/firmware-responses.md) - MeshCore firmware response formats
## License
MIT
## Public Instances
Public MeshCore Stats installations. Want to add yours? [Open a pull request](https://github.com/jorijn/meshcore-stats/pulls)!
| URL | Hardware | Location |
|-----|----------|----------|
| [meshcore.jorijn.com](https://meshcore.jorijn.com) | SenseCAP Solar Node P1 Pro + 6.5dBi Mikrotik antenna | Oosterhout, The Netherlands |

29
docker-compose.dev.yml Normal file
View File

@@ -0,0 +1,29 @@
# MeshCore Stats - Development Override
#
# Use this file for local development with live code changes.
#
# Usage:
# docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build
#
# This override:
# - Builds the image locally instead of pulling from ghcr.io
# - Mounts src/ and scripts/ for live code changes (no rebuild needed)
services:
meshcore-stats:
# Build locally instead of using published image
build:
context: .
dockerfile: Dockerfile
# Remove the image reference (use built image)
image: meshcore-stats:development
# Mount source code for live development
# Changes to Python files take effect immediately (no rebuild needed)
volumes:
- ./data/state:/data/state
- ./out:/out
# Development mounts (read-only to prevent accidental writes)
- ./src:/app/src:ro
- ./scripts:/app/scripts:ro

134
docker-compose.yml Normal file
View File

@@ -0,0 +1,134 @@
# MeshCore Stats - Docker Compose Configuration
#
# Production deployment using published container image.
# For local development, use: docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build
#
# Prerequisites:
# 1. Copy meshcore.conf.example to meshcore.conf and configure
# 2. For serial transport: Create docker-compose.override.yml with your device (see README)
# 3. Ensure your user has access to the serial device (dialout group)
# 4. Create data directories with correct ownership:
# mkdir -p ./data/state ./out && sudo chown -R 1000:1000 ./data ./out
services:
# ==========================================================================
# MeshCore Stats - Data collection and rendering
# ==========================================================================
meshcore-stats:
image: ghcr.io/jorijn/meshcore-stats:0.2.4 # x-release-please-version
container_name: meshcore-stats
restart: unless-stopped
# Load configuration from meshcore.conf
env_file:
- meshcore.conf
# NOTE: Serial device must be added via docker-compose.override.yml
# See README.md for examples. TCP transport users don't need devices.
volumes:
# Persistent storage for SQLite database and circuit breaker state
- ./data/state:/data/state
# Generated static site (served by nginx)
- ./out:/out
# Run as meshmon user (UID 1000)
user: "1000:1000"
# Add dialout group for serial port access
group_add:
- dialout
# Security hardening
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
read_only: true
tmpfs:
- /tmp:noexec,nosuid,size=64m
# Resource limits
deploy:
resources:
limits:
cpus: "1.0"
memory: 512M
reservations:
cpus: "0.1"
memory: 128M
# Logging limits to prevent disk exhaustion
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
# Health check
healthcheck:
test: ["CMD", "python", "-c", "import sqlite3; sqlite3.connect('/data/state/metrics.db').execute('SELECT 1')"]
interval: 5m
timeout: 30s
start_period: 60s
retries: 3
# ==========================================================================
# nginx - Static site server
# ==========================================================================
nginx:
image: nginx:1.27-alpine
container_name: meshcore-stats-nginx
restart: unless-stopped
# Run as nginx user (UID 101 in Alpine nginx image)
user: "101:101"
ports:
- "8080:8080"
volumes:
# Mount generated static site from meshcore-stats container
- ./out:/usr/share/nginx/html:ro
# Custom nginx configuration
- ./docker/nginx.conf:/etc/nginx/conf.d/default.conf:ro
# Security hardening
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
# NET_BIND_SERVICE not needed for port 8080 (unprivileged)
read_only: true
tmpfs:
- /var/cache/nginx:noexec,nosuid,size=16m,uid=101,gid=101
- /var/run:noexec,nosuid,size=1m,uid=101,gid=101
# Resource limits
deploy:
resources:
limits:
cpus: "0.5"
memory: 64M
reservations:
cpus: "0.05"
memory: 16M
# Logging limits
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
# Health check
healthcheck:
test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:8080/health"]
interval: 30s
timeout: 10s
start_period: 5s
retries: 3
depends_on:
meshcore-stats:
condition: service_healthy

67
docker/nginx.conf Normal file
View File

@@ -0,0 +1,67 @@
# nginx configuration for MeshCore Stats static site
# This file is used by the nginx container in docker-compose.yml
server {
listen 8080;
server_name _;
root /usr/share/nginx/html;
index day.html index.html;
# UTF-8 charset for all text files
charset utf-8;
charset_types text/plain text/css text/javascript application/json image/svg+xml;
# Gzip compression
gzip on;
gzip_vary on;
gzip_min_length 1024;
gzip_types text/plain text/css text/javascript application/json image/svg+xml;
# Security headers
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
# HTML, JSON, TXT files - no cache (frequently updated)
location ~* \.(html|json|txt)$ {
add_header Cache-Control "no-cache, no-store, must-revalidate" always;
add_header Pragma "no-cache" always;
add_header Expires "0" always;
# Re-add security headers (add_header in location blocks replaces parent)
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
}
# PNG files - no cache (charts are regenerated frequently)
location ~* \.png$ {
add_header Cache-Control "no-cache, no-store, must-revalidate" always;
add_header Pragma "no-cache" always;
add_header Expires "0" always;
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
}
# CSS, JS, SVG files - short cache (5 minutes)
location ~* \.(css|js|svg)$ {
expires 5m;
add_header Cache-Control "public, max-age=300" always;
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
}
# Default location
location / {
try_files $uri $uri/ =404;
}
# Health check endpoint
location /health {
access_log off;
return 200 "OK\n";
add_header Content-Type text/plain;
}
}

51
docker/ofelia.ini Normal file
View File

@@ -0,0 +1,51 @@
# Ofelia Job Scheduler Configuration
# https://github.com/mcuadros/ofelia
#
# This file defines the cron-like schedule for all MeshCore Stats tasks.
# Jobs run inside the same container (job-local).
# =============================================================================
# Data Collection Jobs
# =============================================================================
[job-local "collect-companion"]
# Collect metrics from companion node (USB serial)
schedule = @every 1m
command = python /app/scripts/collect_companion.py
no-overlap = true
[job-local "collect-repeater"]
# Collect metrics from repeater node (via LoRa)
# Offset by 1 second to avoid USB serial conflicts with companion collection
schedule = 1 1,16,31,46 * * * *
command = python /app/scripts/collect_repeater.py
no-overlap = true
# =============================================================================
# Rendering Jobs
# =============================================================================
[job-local "render-charts"]
# Generate SVG charts from database
schedule = @every 5m
command = python /app/scripts/render_charts.py
[job-local "render-site"]
# Generate static HTML site
schedule = @every 5m
command = python /app/scripts/render_site.py
[job-local "render-reports"]
# Generate monthly/yearly statistics reports
schedule = @daily
command = python /app/scripts/render_reports.py
# =============================================================================
# Maintenance Jobs
# =============================================================================
[job-local "db-maintenance"]
# Database VACUUM and ANALYZE for optimal performance
# Runs at 3 AM on the 1st of each month
schedule = 0 3 1 * *
command = python -c "import sqlite3; db=sqlite3.connect('/data/state/metrics.db'); db.execute('VACUUM'); db.execute('ANALYZE'); db.close(); print('Database maintenance complete')"

View File

@@ -2,55 +2,66 @@
# Copy this file to meshcore.conf and customize for your setup:
# cp meshcore.conf.example meshcore.conf
#
# This file is automatically loaded by the scripts. No need to source it manually.
# Environment variables always take precedence over this file (useful for Docker).
# Format: KEY=value (no 'export' keyword, no spaces around '=')
# This format is compatible with both Docker env_file and shell 'source' command.
# Comments start with # and blank lines are ignored.
# =============================================================================
# Connection Settings
# =============================================================================
export MESH_TRANSPORT=serial
export MESH_SERIAL_PORT=/dev/ttyUSB0 # Adjust for your system (e.g., /dev/ttyACM0, /dev/cu.usbserial-*)
export MESH_SERIAL_BAUD=115200
export MESH_DEBUG=0 # Set to 1 for verbose meshcore debug output
MESH_TRANSPORT=serial
MESH_SERIAL_PORT=/dev/ttyUSB0
# MESH_SERIAL_BAUD=115200
# MESH_DEBUG=0
# TCP transport (for macOS Docker or remote serial servers)
# MESH_TRANSPORT=tcp
# MESH_TCP_HOST=host.docker.internal
# MESH_TCP_PORT=5000
# BLE transport (Bluetooth Low Energy)
# MESH_TRANSPORT=ble
# MESH_BLE_ADDR=AA:BB:CC:DD:EE:FF
# MESH_BLE_PIN=123456
# =============================================================================
# Remote Repeater Identity
# =============================================================================
# At least REPEATER_NAME or REPEATER_KEY_PREFIX is required to identify your repeater
export REPEATER_NAME="Your Repeater Name" # Advertised name shown in contacts
# export REPEATER_KEY_PREFIX="a1b2c3" # Alternative: hex prefix of public key
export REPEATER_PASSWORD="your-password" # Admin password for repeater login
REPEATER_NAME=Your Repeater Name
# REPEATER_KEY_PREFIX=a1b2c3
REPEATER_PASSWORD=your-password
# =============================================================================
# Display Names (shown in UI)
# =============================================================================
export REPEATER_DISPLAY_NAME="My Repeater"
export COMPANION_DISPLAY_NAME="My Companion"
REPEATER_DISPLAY_NAME=My Repeater
COMPANION_DISPLAY_NAME=My Companion
# Public key prefixes (shown below node name in sidebar, e.g., "!a1b2c3d4")
# export REPEATER_PUBKEY_PREFIX="!a1b2c3d4"
# export COMPANION_PUBKEY_PREFIX="!e5f6g7h8"
# REPEATER_PUBKEY_PREFIX=!a1b2c3d4
# COMPANION_PUBKEY_PREFIX=!e5f6g7h8
# =============================================================================
# Location Metadata (for reports and sidebar display)
# =============================================================================
export REPORT_LOCATION_NAME="City, Country" # Full location name for reports
export REPORT_LOCATION_SHORT="City, XX" # Short version for sidebar/meta
export REPORT_LAT=0.0 # Latitude in decimal degrees
export REPORT_LON=0.0 # Longitude in decimal degrees
export REPORT_ELEV=0 # Elevation
export REPORT_ELEV_UNIT=m # "m" for meters, "ft" for feet
REPORT_LOCATION_NAME=City, Country
REPORT_LOCATION_SHORT=City, XX
REPORT_LAT=0.0
REPORT_LON=0.0
REPORT_ELEV=0
REPORT_ELEV_UNIT=m
# =============================================================================
# Hardware Info (shown in sidebar)
# =============================================================================
export REPEATER_HARDWARE="Your Repeater Model" # e.g., "SenseCAP P1-Pro", "LILYGO T-Beam"
export COMPANION_HARDWARE="Your Companion Model" # e.g., "Elecrow ThinkNode-M1", "Heltec V3"
REPEATER_HARDWARE=Your Repeater Model
COMPANION_HARDWARE=Your Companion Model
# =============================================================================
# Radio Configuration Presets
@@ -59,58 +70,54 @@ export COMPANION_HARDWARE="Your Companion Model" # e.g., "Elecrow ThinkNode-M1",
# or set custom values. These are for display purposes only.
# MeshCore EU/UK Narrow (default)
export RADIO_FREQUENCY="869.618 MHz"
export RADIO_BANDWIDTH="62.5 kHz"
export RADIO_SPREAD_FACTOR="SF8"
export RADIO_CODING_RATE="CR8"
RADIO_FREQUENCY=869.618 MHz
RADIO_BANDWIDTH=62.5 kHz
RADIO_SPREAD_FACTOR=SF8
RADIO_CODING_RATE=CR8
# # MeshCore EU/UK Wide
# export RADIO_FREQUENCY="869.525 MHz"
# export RADIO_BANDWIDTH="250 kHz"
# export RADIO_SPREAD_FACTOR="SF10"
# export RADIO_CODING_RATE="CR5"
# MeshCore EU/UK Wide
# RADIO_FREQUENCY=869.525 MHz
# RADIO_BANDWIDTH=250 kHz
# RADIO_SPREAD_FACTOR=SF10
# RADIO_CODING_RATE=CR5
# # MeshCore US Standard
# export RADIO_FREQUENCY="906.875 MHz"
# export RADIO_BANDWIDTH="250 kHz"
# export RADIO_SPREAD_FACTOR="SF10"
# export RADIO_CODING_RATE="CR5"
# MeshCore US Standard
# RADIO_FREQUENCY=906.875 MHz
# RADIO_BANDWIDTH=250 kHz
# RADIO_SPREAD_FACTOR=SF10
# RADIO_CODING_RATE=CR5
# # MeshCore US Fast
# export RADIO_FREQUENCY="906.875 MHz"
# export RADIO_BANDWIDTH="500 kHz"
# export RADIO_SPREAD_FACTOR="SF7"
# export RADIO_CODING_RATE="CR5"
# MeshCore US Fast
# RADIO_FREQUENCY=906.875 MHz
# RADIO_BANDWIDTH=500 kHz
# RADIO_SPREAD_FACTOR=SF7
# RADIO_CODING_RATE=CR5
# # MeshCore ANZ (Australia/New Zealand)
# export RADIO_FREQUENCY="917.0 MHz"
# export RADIO_BANDWIDTH="250 kHz"
# export RADIO_SPREAD_FACTOR="SF10"
# export RADIO_CODING_RATE="CR5"
# MeshCore ANZ (Australia/New Zealand)
# RADIO_FREQUENCY=917.0 MHz
# RADIO_BANDWIDTH=250 kHz
# RADIO_SPREAD_FACTOR=SF10
# RADIO_CODING_RATE=CR5
# =============================================================================
# Intervals and Timeouts
# =============================================================================
export COMPANION_STEP=60 # Collection interval for companion (seconds)
export REPEATER_STEP=900 # Collection interval for repeater (seconds, 15min default)
export REMOTE_TIMEOUT_S=10 # Minimum timeout for LoRa requests
export REMOTE_RETRY_ATTEMPTS=2 # Number of retry attempts
export REMOTE_RETRY_BACKOFF_S=4 # Seconds between retries
# COMPANION_STEP=60
# REPEATER_STEP=900
# REMOTE_TIMEOUT_S=10
# REMOTE_RETRY_ATTEMPTS=2
# REMOTE_RETRY_BACKOFF_S=4
# Circuit breaker settings (prevents spamming LoRa when repeater is unreachable)
export REMOTE_CB_FAILS=6 # Failures before circuit breaker opens
export REMOTE_CB_COOLDOWN_S=3600 # Cooldown period in seconds (1 hour)
# REMOTE_CB_FAILS=6
# REMOTE_CB_COOLDOWN_S=3600
# =============================================================================
# Paths
# Paths (Native installation only)
# =============================================================================
# Docker: Leave these commented. The container uses /data/state and /out by default.
# Native: Uncomment for local cron-based installation:
# STATE_DIR=./data/state
# OUT_DIR=./out
export STATE_DIR=./data/state # SQLite database and circuit breaker state
export OUT_DIR=./out # Generated static site output
# =============================================================================
# Optional
# =============================================================================
export REPEATER_FETCH_ACL=0 # Set to 1 to fetch ACL from repeater

View File

@@ -9,6 +9,11 @@
"type": "generic",
"path": "src/meshmon/__init__.py",
"glob": false
},
{
"type": "generic",
"path": "docker-compose.yml",
"glob": false
}
],
"changelog-sections": [

View File

@@ -233,20 +233,6 @@ async def collect_repeater() -> int:
else:
log.warn(f"req_status_sync failed: {err}")
# Optional ACL query (using _sync version)
if cfg.repeater_fetch_acl:
log.debug("Querying repeater ACL...")
success, payload, err = await query_repeater_with_retry(
mc,
contact,
"req_acl_sync",
lambda: cmd.req_acl_sync(contact, timeout=0, min_timeout=cfg.remote_timeout_s),
)
if success:
log.debug(f"req_acl_sync: {payload}")
else:
log.debug(f"req_acl_sync failed: {err}")
# Update circuit breaker
if status_ok:
cb.record_success()

View File

@@ -1,3 +1,3 @@
"""MeshCore network monitoring library."""
__version__ = "0.2.0" # x-release-please-version
__version__ = "0.2.4" # x-release-please-version

View File

@@ -145,7 +145,6 @@ class Config:
self.repeater_name = get_str("REPEATER_NAME")
self.repeater_key_prefix = get_str("REPEATER_KEY_PREFIX")
self.repeater_password = get_str("REPEATER_PASSWORD")
self.repeater_fetch_acl = get_bool("REPEATER_FETCH_ACL", False)
# Intervals and timeouts
self.companion_step = get_int("COMPANION_STEP", 60)
@@ -156,9 +155,9 @@ class Config:
self.remote_cb_fails = get_int("REMOTE_CB_FAILS", 6)
self.remote_cb_cooldown_s = get_int("REMOTE_CB_COOLDOWN_S", 3600)
# Paths
self.state_dir = get_path("STATE_DIR", "./data/state")
self.out_dir = get_path("OUT_DIR", "./out")
# Paths (defaults are Docker container paths; native installs override via config)
self.state_dir = get_path("STATE_DIR", "/data/state")
self.out_dir = get_path("OUT_DIR", "/out")
# Report location metadata
self.report_location_name = get_str(