mirror of
https://github.com/ipnet-mesh/meshcore-hub.git
synced 2026-03-28 17:42:56 +01:00
2617dace7b9fc9a24fe02d95b7ffb2b1f75465a3
This commit adds the complete Collector component for storing MeshCore events: MQTT Subscriber (collector/subscriber.py): - Subscribes to all event topics from MQTT broker - Routes events to appropriate handlers - Manages database persistence Event Handlers: - advertisement.py: Handles node advertisements, upserts nodes - message.py: Handles contact and channel messages - trace.py: Handles network trace path data - telemetry.py: Handles sensor telemetry responses - contacts.py: Handles contacts sync events - event_log.py: Generic handler for informational events CLI (collector/cli.py): - Click command for running the collector - Environment variable support for all options - Integrated with main CLI Tests: - Subscriber tests - Handler tests for advertisement, message, telemetry
MeshCore Hub
Python 3.11+ platform for managing and orchestrating MeshCore mesh networks.
Overview
MeshCore Hub provides a complete solution for monitoring, collecting, and interacting with MeshCore mesh networks. It consists of multiple components that work together:
| Component | Description |
|---|---|
| Interface | Connects to MeshCore companion nodes via Serial/USB, bridges events to/from MQTT |
| Collector | Subscribes to MQTT events and persists them to a database |
| API | REST API for querying data and sending commands to the network |
| Web Dashboard | User-friendly web interface for visualizing network status |
Architecture
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ MeshCore │ │ MeshCore │ │ MeshCore │
│ Device 1 │ │ Device 2 │ │ Device 3 │
└────────┬────────┘ └────────┬────────┘ └────────┬────────┘
│ │ │
│ Serial/USB │ Serial/USB │ Serial/USB
│ │ │
┌────────▼────────┐ ┌────────▼────────┐ ┌────────▼────────┐
│ Interface │ │ Interface │ │ Interface │
│ (RECEIVER) │ │ (RECEIVER) │ │ (SENDER) │
└────────┬────────┘ └────────┬────────┘ └────────▲────────┘
│ │ │
│ Publish │ Publish │ Subscribe
│ │ │
└───────────┬───────────┴───────────────────────┘
│
┌──────▼──────┐
│ MQTT │
│ Broker │
└──────┬──────┘
│
┌──────▼──────┐
│ Collector │
└──────┬──────┘
│
┌──────▼──────┐
│ Database │
└──────┬──────┘
│
┌───────────┴───────────┐
│ │
┌──────▼──────┐ ┌───────▼───────┐
│ API │◄──────│ Web Dashboard │
└─────────────┘ └───────────────┘
Features
- Multi-node Support: Connect multiple receiver nodes for better network coverage
- Event Persistence: Store messages, advertisements, telemetry, and trace data
- REST API: Query historical data with filtering and pagination
- Command Dispatch: Send messages and advertisements via the API
- Node Tagging: Add custom metadata to nodes for organization
- Web Dashboard: Visualize network status, node locations, and message history
- Docker Ready: Single image with all components, easy deployment
Quick Start
Using Docker Compose (Recommended)
# Clone the repository
git clone https://github.com/your-org/meshcore-hub.git
cd meshcore-hub
# Start all services
docker compose -f docker/docker-compose.yml up -d
# View logs
docker compose -f docker/docker-compose.yml logs -f
Manual Installation
# Create virtual environment
python -m venv .venv
source .venv/bin/activate # Linux/macOS
# .venv\Scripts\activate # Windows
# Install the package
pip install -e ".[dev]"
# Run database migrations
meshcore-hub db upgrade
# Start components (in separate terminals)
meshcore-hub interface --mode receiver --port /dev/ttyUSB0
meshcore-hub collector
meshcore-hub api
meshcore-hub web
Configuration
All components are configured via environment variables. Create a .env file or export variables:
Common Settings
| Variable | Default | Description |
|---|---|---|
LOG_LEVEL |
INFO |
Logging level (DEBUG, INFO, WARNING, ERROR) |
MQTT_HOST |
localhost |
MQTT broker hostname |
MQTT_PORT |
1883 |
MQTT broker port |
MQTT_PREFIX |
meshcore |
Topic prefix for all MQTT messages |
Interface Settings
| Variable | Default | Description |
|---|---|---|
INTERFACE_MODE |
RECEIVER |
Operating mode (RECEIVER or SENDER) |
SERIAL_PORT |
/dev/ttyUSB0 |
Serial port for MeshCore device |
SERIAL_BAUD |
115200 |
Serial baud rate |
MOCK_DEVICE |
false |
Use mock device for testing |
Collector Settings
| Variable | Default | Description |
|---|---|---|
DATABASE_URL |
sqlite:///./meshcore.db |
SQLAlchemy database URL |
API Settings
| Variable | Default | Description |
|---|---|---|
API_HOST |
0.0.0.0 |
API bind address |
API_PORT |
8000 |
API port |
API_READ_KEY |
(none) | Read-only API key |
API_ADMIN_KEY |
(none) | Admin API key (required for commands) |
Web Dashboard Settings
| Variable | Default | Description |
|---|---|---|
WEB_HOST |
0.0.0.0 |
Web server bind address |
WEB_PORT |
8080 |
Web server port |
API_BASE_URL |
http://localhost:8000 |
API endpoint URL |
NETWORK_NAME |
MeshCore Network |
Display name for the network |
NETWORK_CITY |
(none) | City where network is located |
NETWORK_COUNTRY |
(none) | Country code (ISO 3166-1 alpha-2) |
NETWORK_LOCATION |
(none) | Center coordinates (lat,lon) |
CLI Reference
# Show help
meshcore-hub --help
# Interface component
meshcore-hub interface --mode receiver --port /dev/ttyUSB0
meshcore-hub interface --mode sender --mock # Use mock device
# Collector component
meshcore-hub collector --database-url sqlite:///./data.db
# API component
meshcore-hub api --host 0.0.0.0 --port 8000
# Web dashboard
meshcore-hub web --port 8080 --network-name "My Network"
# Database management
meshcore-hub db upgrade # Run migrations
meshcore-hub db downgrade # Rollback one migration
meshcore-hub db current # Show current revision
API Documentation
When running, the API provides interactive documentation at:
- Swagger UI: http://localhost:8000/docs
- ReDoc: http://localhost:8000/redoc
- OpenAPI JSON: http://localhost:8000/openapi.json
Authentication
The API supports optional bearer token authentication:
# Read-only access
curl -H "Authorization: Bearer <API_READ_KEY>" http://localhost:8000/api/v1/nodes
# Admin access (required for commands)
curl -X POST \
-H "Authorization: Bearer <API_ADMIN_KEY>" \
-H "Content-Type: application/json" \
-d '{"destination": "abc123...", "text": "Hello!"}' \
http://localhost:8000/api/v1/commands/send-message
Example Endpoints
| Method | Endpoint | Description |
|---|---|---|
| GET | /api/v1/nodes |
List all known nodes |
| GET | /api/v1/nodes/{public_key} |
Get node details |
| GET | /api/v1/nodes/{public_key}/tags |
Get node tags |
| POST | /api/v1/nodes/{public_key}/tags |
Create node tag |
| GET | /api/v1/messages |
List messages with filters |
| GET | /api/v1/advertisements |
List advertisements |
| GET | /api/v1/telemetry |
List telemetry data |
| GET | /api/v1/trace-paths |
List trace paths |
| POST | /api/v1/commands/send-message |
Send direct message |
| POST | /api/v1/commands/send-channel-message |
Send channel message |
| GET | /api/v1/stats |
Get network statistics |
Development
Setup
# Clone and setup
git clone https://github.com/your-org/meshcore-hub.git
cd meshcore-hub
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
# Install pre-commit hooks
pre-commit install
Running Tests
# Run all tests
pytest
# Run with coverage
pytest --cov=meshcore_hub --cov-report=html
# Run specific test file
pytest tests/test_api/test_nodes.py
# Run tests matching pattern
pytest -k "test_list"
Code Quality
# Format code
black src/ tests/
# Lint
flake8 src/ tests/
# Type check
mypy src/
Creating Database Migrations
# Auto-generate migration from model changes
meshcore-hub db revision --autogenerate -m "Add new field to nodes"
# Create empty migration
meshcore-hub db revision -m "Custom migration"
# Apply migrations
meshcore-hub db upgrade
Project Structure
meshcore-hub/
├── src/meshcore_hub/ # Main package
│ ├── common/ # Shared code (models, schemas, config)
│ ├── interface/ # MeshCore device interface
│ ├── collector/ # MQTT event collector
│ ├── api/ # REST API
│ └── web/ # Web dashboard
├── tests/ # Test suite
├── alembic/ # Database migrations
├── docker/ # Docker configuration
├── PROMPT.md # Project specification
├── SCHEMAS.md # Event schema documentation
├── PLAN.md # Implementation plan
├── TASKS.md # Task tracker
└── AGENTS.md # AI assistant guidelines
Documentation
- PROMPT.md - Original project specification
- SCHEMAS.md - MeshCore event schemas
- PLAN.md - Architecture and implementation plan
- TASKS.md - Development task tracker
- AGENTS.md - Guidelines for AI coding assistants
Contributing
- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature) - Make your changes
- Run tests and linting (
pytest && black . && flake8) - Commit your changes (
git commit -m 'Add amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
License
See LICENSE for details.
Acknowledgments
- MeshCore - The mesh networking protocol
- meshcore_py - Python library for MeshCore devices
Languages
Python
80.3%
JavaScript
17.1%
HTML
1.5%
CSS
0.7%
Dockerfile
0.3%