adam/meshcore-hub
1
0
Fork 0
mirror of https://github.com/ipnet-mesh/meshcore-hub.git synced 2026-08-12 03:42:59 +02:00
Code Issues Packages Projects Releases Wiki Activity

Remove bundled Prometheus/Alertmanager, fix Getting Started and docker compose examples

Browse Source 
- Remove monitoring services from all docker compose files — monitoring is now
  user-managed infrastructure (users point their own Prometheus at /metrics)
- Remove metrics profile, prometheus/alertmanager volumes from Makefile
- Update README Getting Started: packet capture is included via --profile receiver,
  not a separate prerequisite; add remote observers guide
- Add --profile all to all docker compose command examples in README and UPGRADING.md
- Simplify UPGRADING.md backup/migration to only meshcore_hub_data
This commit is contained in:
Louis King
2026-04-16 22:47:07 +01:00
parent ac18c80675
commit 0a37010db6
9 changed files with 97 additions and 115 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+7 -2
View File
@@ -307,11 +307,16 @@ meshcore-hub/
│ ├── env.py
│ └── versions/
├── etc/
│ ├── docker/ # Docker configuration examples
│ │ └── meshcore-mqtt-broker/
│ ├── prometheus/ # Prometheus configuration
│ │ ├── prometheus.yml # Scrape and alerting config
│ │ └── alerts.yml # Alert rules
│ └── alertmanager/ # Alertmanager configuration
│ └── alertmanager.yml # Routing and receiver config
│ ├── alertmanager/ # Alertmanager configuration
│ │ └── alertmanager.yml # Routing and receiver config
│ └── systemd/ # Systemd service templates
│ ├── meshcore-hub-update@.service # Auto-update service
│ └── meshcore-hub-update@.timer # Auto-update timer
├── example/
│ ├── seed/ # Example seed data files
│ │ ├── node_tags.yaml # Example node tags
Makefile
-1
View File
@@ -7,7 +7,6 @@ COMPOSE_PROJECT_NAME ?= hub-dev
PROFILES ?= mqtt core
COMPOSE_FILES = -f docker-compose.yml -f docker-compose.dev.yml
VOLUMES = $(COMPOSE_PROJECT_NAME)_hub_data $(COMPOSE_PROJECT_NAME)_mqtt_broker_data \
$(COMPOSE_PROJECT_NAME)_prometheus_data $(COMPOSE_PROJECT_NAME)_alertmanager_data \
$(COMPOSE_PROJECT_NAME)_packetcapture_data
.PHONY: build up down logs backup restore
README.md
+63 -24
View File
@@ -75,10 +75,10 @@ flowchart LR
### Simple Self-Hosted Setup
The quickest way to get started is running the entire stack on a single machine alongside [meshcore-packet-capture](https://github.com/agessaman/meshcore-packet-capture).
The quickest way to get started is running the entire stack on a single machine with a connected LoRa radio.
**Prerequisites:**
1. Set up [meshcore-packet-capture](https://github.com/agessaman/meshcore-packet-capture) on a device with a compatible LoRa radio (e.g., Heltec V3, T-Beam) to observe MeshCore RF traffic
1. A compatible LoRa radio (e.g., Heltec V3, T-Beam) connected via serial
**Steps:**
```bash
@@ -93,16 +93,17 @@ wget https://raw.githubusercontent.com/ipnet-mesh/meshcore-hub/refs/heads/main/.
# Copy and configure environment
cp .env.example .env
# Edit .env: set MQTT_HOST to your MQTT broker if not using the local one
# Edit .env: set PACKETCAPTURE_IATA to your 3-letter airport code
# set SERIAL_PORT if not /dev/ttyUSB0
# Start the entire stack with local MQTT broker
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile mqtt --profile core up -d
# Start the entire stack with local MQTT broker and packet capture
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile mqtt --profile core --profile receiver up -d
# View the web dashboard
open http://localhost:8080
```
This starts all services: MQTT broker, collector, API, and web dashboard. MeshCore packet data is ingested via [meshcore-packet-capture](https://github.com/agessaman/meshcore-packet-capture), which publishes decoded packets to MQTT.
This starts all services: MQTT broker, collector, API, web dashboard, and packet capture. The `receiver` profile runs [meshcore-packet-capture](https://github.com/agessaman/meshcore-packet-capture) to observe MeshCore RF traffic and publish decoded packets to MQTT.
## Deployment
@@ -121,13 +122,13 @@ All `docker compose` commands require explicit file selection with `-f`:
```bash
# Development (default — exposes ports for local access)
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile all up -d
# Production (generic reverse proxy — nginx, caddy, etc.)
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
docker compose -f docker-compose.yml -f docker-compose.prod.yml --profile all up -d
# Production (Traefik)
docker compose -f docker-compose.yml -f docker-compose.prod.yml -f docker-compose.traefik.yml up -d
docker compose -f docker-compose.yml -f docker-compose.prod.yml -f docker-compose.traefik.yml --profile all up -d
```
Service profiles:
@@ -138,7 +139,6 @@ Service profiles:
| `core` | db-migrate, collector, api, web | Central server infrastructure |
| `mqtt` | meshcore-mqtt-broker | Local MQTT broker (optional) |
| `receiver` | packet capture observer | Observes RF traffic and publishes to MQTT |
| `metrics` | prometheus, alertmanager | Prometheus metrics and alerting |
| `seed` | seed | One-time seed data import |
| `migrate` | db-migrate | One-time database migration |
@@ -161,10 +161,55 @@ docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile core up
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile mqtt --profile core --profile receiver up -d
# View logs
docker compose -f docker-compose.yml -f docker-compose.dev.yml logs -f
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile all logs -f
# Stop services
docker compose -f docker-compose.yml -f docker-compose.dev.yml down
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile all down
```
### Adding Remote Observers
Other operators can run their own [meshcore-packet-capture](https://github.com/agessaman/meshcore-packet-capture) instance and publish decoded packets to your MeshCore Hub. They can also optionally contribute to the LetsMesh network.
> **Prerequisite:** Your MQTT broker must be accessible to remote observers. In production, this means exposing the WebSocket listener via a reverse proxy with TLS (e.g., `wss://mqtt.example.com/mqtt`).
#### Example: Observer contributing to LetsMesh and your community Hub
```bash
# In the observer's .env or docker-compose environment:
# Server 1 - Let's Mesh US (opt-in)
PACKETCAPTURE_MQTT1_ENABLED=true
PACKETCAPTURE_MQTT1_SERVER=mqtt-us-v1.letsmesh.net
PACKETCAPTURE_MQTT1_PORT=443
PACKETCAPTURE_MQTT1_TRANSPORT=websockets
PACKETCAPTURE_MQTT1_USE_TLS=true
PACKETCAPTURE_MQTT1_USE_AUTH_TOKEN=true
PACKETCAPTURE_MQTT1_TOKEN_AUDIENCE=mqtt-us-v1.letsmesh.net
# Server 2 - Let's Mesh EU (opt-in)
PACKETCAPTURE_MQTT2_ENABLED=false
# Server 3 - Your MeshCore Hub
PACKETCAPTURE_MQTT3_ENABLED=true
PACKETCAPTURE_MQTT3_SERVER=mqtt.example.com
PACKETCAPTURE_MQTT3_PORT=443
PACKETCAPTURE_MQTT3_TRANSPORT=websockets
PACKETCAPTURE_MQTT3_USE_TLS=true
PACKETCAPTURE_MQTT3_USE_AUTH_TOKEN=true
PACKETCAPTURE_MQTT3_TOKEN_AUDIENCE=mqtt.example.com
```
Replace `mqtt.example.com` with your public MQTT domain. The `TOKEN_AUDIENCE` must match the `MQTT_TOKEN_AUDIENCE` or `AUTH_EXPECTED_AUDIENCE` configured on your broker.
For local network observers (no TLS):
```bash
PACKETCAPTURE_MQTT3_SERVER=192.168.1.100
PACKETCAPTURE_MQTT3_PORT=1883
PACKETCAPTURE_MQTT3_TRANSPORT=websockets
PACKETCAPTURE_MQTT3_USE_TLS=false
PACKETCAPTURE_MQTT3_TOKEN_AUDIENCE=mqtt.localhost
```
### Multi-Instance Deployment
@@ -229,18 +274,12 @@ make restore FILE=backup/hub-dev_hub_data-20260414-120000.tar.gz
#### Using shell commands
```bash
# Back up all volumes
# Back up the database volume
source .env 2>/dev/null || true
mkdir -p backup
for vol in ${COMPOSE_PROJECT_NAME:-hub-dev}_hub_data \
${COMPOSE_PROJECT_NAME:-hub-dev}_mqtt_broker_data \
${COMPOSE_PROJECT_NAME:-hub-dev}_prometheus_data \
${COMPOSE_PROJECT_NAME:-hub-dev}_alertmanager_data \
${COMPOSE_PROJECT_NAME:-hub-dev}_packetcapture_data; do
echo "Backing up $vol..."
docker run --rm -v $vol:/data -v $(pwd)/backup:/backup \
alpine tar czf /backup/$vol-$(date +%Y%m%d-%H%M%S).tar.gz -C / data
done
vol=${COMPOSE_PROJECT_NAME:-hub-dev}_hub_data
docker run --rm -v $vol:/data -v $(pwd)/backup:/backup \
alpine tar czf /backup/$vol-$(date +%Y%m%d-%H%M%S).tar.gz -C / data
# Restore a specific volume (volume name derived from tarball filename)
source .env 2>/dev/null || true
@@ -250,7 +289,7 @@ docker run --rm -v $vol:/data -v $(pwd)/backup:/backup \
alpine sh -c "cd / && tar xzf /backup/$(basename $FILE)"
```
> **Note:** Replace `hub-dev` with your `COMPOSE_PROJECT_NAME` if using a different instance name.
> **Note:** Replace `hub-dev` with your `COMPOSE_PROJECT_NAME` if using a different instance name. Monitoring infrastructure (Prometheus, Alertmanager) manages its own data — consult your monitoring stack's documentation for backup procedures.
### Manual Installation
@@ -637,7 +676,7 @@ Health check endpoints are also available:
- **Health**: http://localhost:8000/health
- **Ready**: http://localhost:8000/health/ready (includes database check)
- **Metrics**: http://localhost:8000/metrics (Prometheus format)
- **Metrics**: http://localhost:8000/metrics (Prometheus format — point your Prometheus scraper here)
### Authentication
UPGRADING.md
+10 -32
View File
@@ -32,18 +32,6 @@ docker run --rm -v meshcore_hub_data:/data -v $(pwd)/backup:/backup \
alpine tar czf /backup/meshcore_hub_data-$(date +%Y%m%d-%H%M%S).tar.gz -C / data
```
Optionally back up monitoring volumes (only if you used the monitoring profile):
```bash
docker run --rm -v meshcore_prometheus_data:/data -v $(pwd)/backup:/backup \
alpine tar czf /backup/meshcore_prometheus_data-$(date +%Y%m%d-%H%M%S).tar.gz -C / data
docker run --rm -v meshcore_alertmanager_data:/data -v $(pwd)/backup:/backup \
alpine tar czf /backup/meshcore_alertmanager_data-$(date +%Y%m%d-%H%M%S).tar.gz -C / data
```
> **Note:** Only `meshcore_hub_data` (the database) is critical. The others contain metrics and alerting state that will be recreated.
To restore from backup if needed:
```bash
@@ -57,7 +45,7 @@ docker run --rm -v meshcore_hub_data:/data -v $(pwd)/backup:/backup \
Stop all services and remove orphaned containers from the old configuration:
```bash
docker compose down --remove-orphans
docker compose --profile all down --remove-orphans
```
> **Important:** Do NOT use `--volumes` / `-v`. That would delete your database. The `--remove-orphans` flag cleans up old services (like `interface-receiver`, `interface-sender`) that no longer exist in the new compose file.
@@ -79,10 +67,8 @@ These volumes always need migrating:
| Old Name | New Name |
|----------|----------|
| `meshcore_hub_data` | `hub-dev_hub_data` |
| `meshcore_prometheus_data` | `hub-dev_prometheus_data` |
| `meshcore_alertmanager_data` | `hub-dev_alertmanager_data` |
> **Note:** `packetcapture_data` and `mqtt_broker_data` are new — they are created automatically on first run and do not need migrating.
> **Note:** `packetcapture_data` and `mqtt_broker_data` are new — they are created automatically on first run and do not need migrating. Monitoring infrastructure (Prometheus, Alertmanager) is no longer bundled — if you used the previous `metrics` profile, manage those volumes separately.
### Option A: Rename (Docker Engine 23.0+)
@@ -90,8 +76,6 @@ These volumes always need migrating:
```bash
docker volume rename meshcore_hub_data hub-dev_hub_data
docker volume rename meshcore_prometheus_data hub-dev_prometheus_data
docker volume rename meshcore_alertmanager_data hub-dev_alertmanager_data
```
### Option B: Copy (all Docker versions)
@@ -99,18 +83,12 @@ docker volume rename meshcore_alertmanager_data hub-dev_alertmanager_data
If `docker volume rename` is not available in your Docker build:
```bash
# For each volume: create new, copy data, remove old
# Create new volume, copy data, remove old
docker volume create hub-dev_hub_data
docker run --rm -v meshcore_hub_data:/from -v hub-dev_hub_data:/to alpine sh -c "cp -a /from/. /to/"
docker volume create hub-dev_prometheus_data
docker run --rm -v meshcore_prometheus_data:/from -v hub-dev_prometheus_data:/to alpine sh -c "cp -a /from/. /to/"
docker volume create hub-dev_alertmanager_data
docker run --rm -v meshcore_alertmanager_data:/from -v hub-dev_alertmanager_data:/to alpine sh -c "cp -a /from/. /to/"
# Verify the new volumes have data, then remove old ones
docker volume rm meshcore_hub_data meshcore_prometheus_data meshcore_alertmanager_data
# Verify the new volume has data, then remove old one
docker volume rm meshcore_hub_data
```
> **Note:** If any volumes show "in use", remove any stopped containers first: `docker rm -f <container_id>`.
@@ -236,10 +214,10 @@ docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile core up
```bash
# Check all containers are running
docker compose -f docker-compose.yml -f docker-compose.dev.yml ps
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile all ps
# Check collector connected to MQTT
docker compose -f docker-compose.yml -f docker-compose.dev.yml logs collector | grep -i "connected to mqtt"
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile all logs collector | grep -i "connected to mqtt"
# Check the web dashboard
open http://localhost:8080
@@ -318,13 +296,13 @@ All `docker compose` commands now require explicit file selection:
```bash
# Development (exposes ports for local access)
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile all up -d
# Production (connects to reverse proxy network)
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
docker compose -f docker-compose.yml -f docker-compose.prod.yml --profile all up -d
# Production with Traefik
docker compose -f docker-compose.yml -f docker-compose.prod.yml -f docker-compose.traefik.yml up -d
docker compose -f docker-compose.yml -f docker-compose.prod.yml -f docker-compose.traefik.yml --profile all up -d
```
Container and volume names are parameterized via `COMPOSE_PROJECT_NAME` in `.env`. This enables multiple instances (e.g., `hub-prod`, `hub-beta`) on the same Docker host.
docker-compose.dev.yml
+7 -8
View File
@@ -7,6 +7,13 @@
# docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d
services:
collector:
depends_on:
db-migrate:
condition: service_completed_successfully
mqtt:
condition: service_healthy
mqtt:
ports:
- "${MQTT_PORT:-1883}:${MQTT_PORT:-1883}"
@@ -18,11 +25,3 @@ services:
web:
ports:
- "${WEB_PORT:-8080}:8080"
prometheus:
ports:
- "${PROMETHEUS_PORT:-9090}:9090"
alertmanager:
ports:
- "${ALERTMANAGER_PORT:-9093}:9093"
docker-compose.prod.yml
+6 -1
View File
@@ -1,6 +1,6 @@
# MeshCore Hub - Production Docker Compose Override
#
# Connects api and web services to an external proxy network for
# Connects core services to an external proxy network for
# reverse proxy access (Traefik, Nginx, Caddy, etc.).
# No ports are exposed directly — all traffic goes through the reverse proxy.
#
@@ -14,6 +14,11 @@
# docker compose -f docker-compose.yml -f docker-compose.prod.yml -f docker-compose.traefik.yml up -d
services:
collector:
networks:
- default
- proxy-net
api:
networks:
- default
docker-compose.traefik.yml
+1 -1
View File
@@ -1,6 +1,6 @@
# MeshCore Hub - Traefik Docker Compose Override
#
# Adds Traefik auto-discovery labels to api and web services.
# Adds Traefik auto-discovery labels to core services.
# Requires docker-compose.prod.yml (for the proxy-net network).
#
# Set TRAEFIK_DOMAIN in your .env file to the hostname for this instance.
docker-compose.yml
-44
View File
@@ -141,8 +141,6 @@ services:
depends_on:
db-migrate:
condition: service_completed_successfully
mqtt:
condition: service_healthy
restart: unless-stopped
volumes:
- hub_data:/data
@@ -348,44 +346,6 @@ services:
# Imports both node_tags.yaml and members.yaml if they exist
command: ["collector", "seed"]
# ==========================================================================
# Prometheus - Metrics collection and monitoring (optional, use --profile metrics)
# ==========================================================================
prometheus:
image: prom/prometheus:latest
container_name: ${COMPOSE_PROJECT_NAME:-hub-dev}-prometheus
profiles:
- all
- metrics
restart: unless-stopped
depends_on:
api:
condition: service_healthy
command:
- "--config.file=/etc/prometheus/prometheus.yml"
- "--storage.tsdb.retention.time=30d"
volumes:
- ./etc/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
- ./etc/prometheus/alerts.yml:/etc/prometheus/alerts.yml:ro
- prometheus_data:/prometheus
# ==========================================================================
# Alertmanager - Alert routing and notifications (optional, use --profile metrics)
# ==========================================================================
alertmanager:
image: prom/alertmanager:latest
container_name: ${COMPOSE_PROJECT_NAME:-hub-dev}-alertmanager
profiles:
- all
- metrics
restart: unless-stopped
volumes:
- ./etc/alertmanager/alertmanager.yml:/etc/alertmanager/alertmanager.yml:ro
- alertmanager_data:/alertmanager
command:
- "--config.file=/etc/alertmanager/alertmanager.yml"
- "--storage.path=/alertmanager"
# ==========================================================================
# Volumes
# ==========================================================================
@@ -394,9 +354,5 @@ volumes:
name: ${COMPOSE_PROJECT_NAME:-hub-dev}_hub_data
mqtt_broker_data:
name: ${COMPOSE_PROJECT_NAME:-hub-dev}_mqtt_broker_data
prometheus_data:
name: ${COMPOSE_PROJECT_NAME:-hub-dev}_prometheus_data
alertmanager_data:
name: ${COMPOSE_PROJECT_NAME:-hub-dev}_alertmanager_data
packetcapture_data:
name: ${COMPOSE_PROJECT_NAME:-hub-dev}_packetcapture_data
etc/prometheus/prometheus.yml
+3 -2
View File
@@ -1,7 +1,8 @@
# Prometheus scrape configuration for MeshCore Hub
#
# This file is used when running Prometheus via Docker Compose:
# docker compose --profile core --profile metrics up -d
# Example configuration — copy and adapt for your Prometheus deployment.
# Point your scraper at the MeshCore Hub API /metrics endpoint:
# http://api:8000/metrics
#
# The scrape interval matches the default metrics cache TTL (60s)
# to avoid unnecessary database queries.