Claude 2617dace7b Implement Phase 3: Collector Component
This commit adds the complete Collector component for storing MeshCore events:

MQTT Subscriber (collector/subscriber.py):
- Subscribes to all event topics from MQTT broker
- Routes events to appropriate handlers
- Manages database persistence

Event Handlers:
- advertisement.py: Handles node advertisements, upserts nodes
- message.py: Handles contact and channel messages
- trace.py: Handles network trace path data
- telemetry.py: Handles sensor telemetry responses
- contacts.py: Handles contacts sync events
- event_log.py: Generic handler for informational events

CLI (collector/cli.py):
- Click command for running the collector
- Environment variable support for all options
- Integrated with main CLI

Tests:
- Subscriber tests
- Handler tests for advertisement, message, telemetry
2025-12-02 23:21:10 +00:00
2025-12-02 22:58:16 +00:00
2025-12-02 21:52:34 +00:00
2025-12-02 22:58:16 +00:00
2025-12-02 22:43:19 +00:00
2025-12-02 22:58:16 +00:00
2025-12-02 22:43:19 +00:00
2025-12-02 22:58:16 +00:00

MeshCore Hub

Python 3.11+ platform for managing and orchestrating MeshCore mesh networks.

Overview

MeshCore Hub provides a complete solution for monitoring, collecting, and interacting with MeshCore mesh networks. It consists of multiple components that work together:

Component Description
Interface Connects to MeshCore companion nodes via Serial/USB, bridges events to/from MQTT
Collector Subscribes to MQTT events and persists them to a database
API REST API for querying data and sending commands to the network
Web Dashboard User-friendly web interface for visualizing network status

Architecture

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│    MeshCore     │     │    MeshCore     │     │    MeshCore     │
│    Device 1     │     │    Device 2     │     │    Device 3     │
└────────┬────────┘     └────────┬────────┘     └────────┬────────┘
         │                       │                       │
         │ Serial/USB            │ Serial/USB            │ Serial/USB
         │                       │                       │
┌────────▼────────┐     ┌────────▼────────┐     ┌────────▼────────┐
│   Interface     │     │   Interface     │     │   Interface     │
│   (RECEIVER)    │     │   (RECEIVER)    │     │   (SENDER)      │
└────────┬────────┘     └────────┬────────┘     └────────▲────────┘
         │                       │                       │
         │ Publish               │ Publish               │ Subscribe
         │                       │                       │
         └───────────┬───────────┴───────────────────────┘
                     │
              ┌──────▼──────┐
              │    MQTT     │
              │   Broker    │
              └──────┬──────┘
                     │
              ┌──────▼──────┐
              │  Collector  │
              └──────┬──────┘
                     │
              ┌──────▼──────┐
              │  Database   │
              └──────┬──────┘
                     │
         ┌───────────┴───────────┐
         │                       │
  ┌──────▼──────┐       ┌───────▼───────┐
  │     API     │◄──────│ Web Dashboard │
  └─────────────┘       └───────────────┘

Features

  • Multi-node Support: Connect multiple receiver nodes for better network coverage
  • Event Persistence: Store messages, advertisements, telemetry, and trace data
  • REST API: Query historical data with filtering and pagination
  • Command Dispatch: Send messages and advertisements via the API
  • Node Tagging: Add custom metadata to nodes for organization
  • Web Dashboard: Visualize network status, node locations, and message history
  • Docker Ready: Single image with all components, easy deployment

Quick Start

# Clone the repository
git clone https://github.com/your-org/meshcore-hub.git
cd meshcore-hub

# Start all services
docker compose -f docker/docker-compose.yml up -d

# View logs
docker compose -f docker/docker-compose.yml logs -f

Manual Installation

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # Linux/macOS
# .venv\Scripts\activate   # Windows

# Install the package
pip install -e ".[dev]"

# Run database migrations
meshcore-hub db upgrade

# Start components (in separate terminals)
meshcore-hub interface --mode receiver --port /dev/ttyUSB0
meshcore-hub collector
meshcore-hub api
meshcore-hub web

Configuration

All components are configured via environment variables. Create a .env file or export variables:

Common Settings

Variable Default Description
LOG_LEVEL INFO Logging level (DEBUG, INFO, WARNING, ERROR)
MQTT_HOST localhost MQTT broker hostname
MQTT_PORT 1883 MQTT broker port
MQTT_PREFIX meshcore Topic prefix for all MQTT messages

Interface Settings

Variable Default Description
INTERFACE_MODE RECEIVER Operating mode (RECEIVER or SENDER)
SERIAL_PORT /dev/ttyUSB0 Serial port for MeshCore device
SERIAL_BAUD 115200 Serial baud rate
MOCK_DEVICE false Use mock device for testing

Collector Settings

Variable Default Description
DATABASE_URL sqlite:///./meshcore.db SQLAlchemy database URL

API Settings

Variable Default Description
API_HOST 0.0.0.0 API bind address
API_PORT 8000 API port
API_READ_KEY (none) Read-only API key
API_ADMIN_KEY (none) Admin API key (required for commands)

Web Dashboard Settings

Variable Default Description
WEB_HOST 0.0.0.0 Web server bind address
WEB_PORT 8080 Web server port
API_BASE_URL http://localhost:8000 API endpoint URL
NETWORK_NAME MeshCore Network Display name for the network
NETWORK_CITY (none) City where network is located
NETWORK_COUNTRY (none) Country code (ISO 3166-1 alpha-2)
NETWORK_LOCATION (none) Center coordinates (lat,lon)

CLI Reference

# Show help
meshcore-hub --help

# Interface component
meshcore-hub interface --mode receiver --port /dev/ttyUSB0
meshcore-hub interface --mode sender --mock  # Use mock device

# Collector component
meshcore-hub collector --database-url sqlite:///./data.db

# API component
meshcore-hub api --host 0.0.0.0 --port 8000

# Web dashboard
meshcore-hub web --port 8080 --network-name "My Network"

# Database management
meshcore-hub db upgrade      # Run migrations
meshcore-hub db downgrade    # Rollback one migration
meshcore-hub db current      # Show current revision

API Documentation

When running, the API provides interactive documentation at:

Authentication

The API supports optional bearer token authentication:

# Read-only access
curl -H "Authorization: Bearer <API_READ_KEY>" http://localhost:8000/api/v1/nodes

# Admin access (required for commands)
curl -X POST \
  -H "Authorization: Bearer <API_ADMIN_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"destination": "abc123...", "text": "Hello!"}' \
  http://localhost:8000/api/v1/commands/send-message

Example Endpoints

Method Endpoint Description
GET /api/v1/nodes List all known nodes
GET /api/v1/nodes/{public_key} Get node details
GET /api/v1/nodes/{public_key}/tags Get node tags
POST /api/v1/nodes/{public_key}/tags Create node tag
GET /api/v1/messages List messages with filters
GET /api/v1/advertisements List advertisements
GET /api/v1/telemetry List telemetry data
GET /api/v1/trace-paths List trace paths
POST /api/v1/commands/send-message Send direct message
POST /api/v1/commands/send-channel-message Send channel message
GET /api/v1/stats Get network statistics

Development

Setup

# Clone and setup
git clone https://github.com/your-org/meshcore-hub.git
cd meshcore-hub
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

# Install pre-commit hooks
pre-commit install

Running Tests

# Run all tests
pytest

# Run with coverage
pytest --cov=meshcore_hub --cov-report=html

# Run specific test file
pytest tests/test_api/test_nodes.py

# Run tests matching pattern
pytest -k "test_list"

Code Quality

# Format code
black src/ tests/

# Lint
flake8 src/ tests/

# Type check
mypy src/

Creating Database Migrations

# Auto-generate migration from model changes
meshcore-hub db revision --autogenerate -m "Add new field to nodes"

# Create empty migration
meshcore-hub db revision -m "Custom migration"

# Apply migrations
meshcore-hub db upgrade

Project Structure

meshcore-hub/
├── src/meshcore_hub/       # Main package
│   ├── common/             # Shared code (models, schemas, config)
│   ├── interface/          # MeshCore device interface
│   ├── collector/          # MQTT event collector
│   ├── api/                # REST API
│   └── web/                # Web dashboard
├── tests/                  # Test suite
├── alembic/                # Database migrations
├── docker/                 # Docker configuration
├── PROMPT.md               # Project specification
├── SCHEMAS.md              # Event schema documentation
├── PLAN.md                 # Implementation plan
├── TASKS.md                # Task tracker
└── AGENTS.md               # AI assistant guidelines

Documentation

Contributing

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Make your changes
  4. Run tests and linting (pytest && black . && flake8)
  5. Commit your changes (git commit -m 'Add amazing feature')
  6. Push to the branch (git push origin feature/amazing-feature)
  7. Open a Pull Request

License

See LICENSE for details.

Acknowledgments

Languages
Python 80.3%
JavaScript 17.1%
HTML 1.5%
CSS 0.7%
Dockerfile 0.3%