Commit Graph

49 Commits

Author SHA1 Message Date
Louis King 9f64d5d0b5 Updates 2025-12-03 20:01:10 +00:00
JingleManSweep f09e89daaa Merge pull request #9 from ipnet-mesh/claude/add-data-home-config-01WnK1Q9vczq5atX7U5P3yJz
Add configurable data directory for services
2025-12-03 19:47:49 +00:00
Louis King 6bb134e473 Updates 2025-12-03 19:47:19 +00:00
Claude 862820bbd3 Add DATA_HOME configuration for centralized data directory management
- Add DATA_HOME setting to CommonSettings (default: ./data)
- Update CollectorSettings with:
  - effective_database_url property (default: sqlite:///{DATA_HOME}/collector/meshcore.db)
  - effective_tags_file property (default: {DATA_HOME}/collector/tags.json)
  - collector_data_dir property
- Update APISettings with effective_database_url property
- Update WebSettings with:
  - effective_members_file property (default: {DATA_HOME}/web/members.json)
  - web_data_dir property
- Update CLI commands (collector, api, web) to:
  - Accept --data-home option
  - Use effective_* properties for defaults
  - Auto-create data directories on startup
- Update docker-compose.yml.example to use DATA_HOME volume mounts
- Update .env.example with DATA_HOME documentation
- Update PLAN.md and AGENTS.md with data directory structure docs
- Add comprehensive tests for new configuration properties
2025-12-03 19:14:13 +00:00
Louis King a4a3a72fa9 Updates 2025-12-03 18:57:00 +00:00
JingleManSweep 2bc22b572b Merge pull request #8 from ipnet-mesh/claude/import-tags-json-011HsCgMWSVG65qhogLxXm4k
Import tags from JSON into database
2025-12-03 18:34:12 +00:00
Claude c56e15a5f8 Add documentation for node tag import feature
README.md:
- Add "Node Tags" section with comprehensive documentation
- Document CLI usage: meshcore-hub collector import-tags
- Document Docker Compose usage: docker compose --profile import-tags
- Document JSON format with field descriptions
- Document import options (--no-create-nodes)
- Document data directory structure for Docker
- Document tag management via REST API (CRUD operations)

AGENTS.md:
- Add tag_import.py to project structure
- Update example/data structure with collector/web subdirectories
- Update data directory structure documentation
2025-12-03 18:32:50 +00:00
Claude cbaf4f451c Add tests to verify Node API returns tags
- Add test_get_node_with_tags to verify GET /nodes/{pk} includes tags
- Add test_list_nodes_includes_tags to verify GET /nodes includes tags
- Update existing tests to assert tags field is present

The Node API was already correctly returning tags via the
lazy="selectin" relationship loading strategy. These tests
document and verify that behavior.
2025-12-03 18:26:32 +00:00
Claude fe1fd69904 Add node tag import functionality to collector
- Add tag_import.py module with JSON file parsing and database upsert
- Convert collector CLI to group with subcommands for extensibility
- Add 'import-tags' command to import tags from JSON file
- Update docker-compose.yml.example with separated data directories:
  - data/collector for tags.json
  - data/web for members.json
- Add import-tags Docker service for easy containerized imports
- Add example data files in example/data/collector and example/data/web
- Add comprehensive test coverage (20 tests) for tag import
2025-12-03 18:16:03 +00:00
Louis King df6f6b779c Update directory structure: move example data to example/data
- Update docker-compose.yml.example to use example/data paths
- Update .env.example with example/data paths and comments
- Update README.md and AGENTS.md project structure documentation
- Add data/ to .gitignore (runtime data directory)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-03 17:58:42 +00:00
Louis King 594ac14509 Add webhook configuration support for collector
Add environment variable configuration for forwarding events to external
HTTP endpoints. Supports separate webhooks for advertisements, channel
messages, and direct messages with configurable timeouts, retries, and
authentication via X-Webhook-Secret header.

- Add webhook fields to CollectorSettings (config.py)
- Add create_webhooks_from_settings() to build webhooks from env vars
- Integrate WebhookDispatcher into Subscriber with background processing
- Update collector CLI to load and display webhook configuration
- Document webhook settings in README, AGENTS.md, .env.example
- Update docker-compose.yml.example with webhook env vars

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-03 17:47:39 +00:00
Louis King 0cbd01fe05 Updates 2025-12-03 17:13:18 +00:00
Louis King e6b3ceb639 Updates 2025-12-03 17:02:57 +00:00
JingleManSweep 7ddff65570 Merge pull request #7 from ipnet-mesh/claude/complete-ta-tasks-01YEzC8yZg82b4gS37yrQGT4
Complete outstanding Task list tasks
2025-12-03 16:43:10 +00:00
Claude e57fe7a2d8 Disable e2e tests by default and fix mypy errors
- Add --e2e flag to pytest to run e2e tests
- E2E tests skip by default with clear message
- Fix type annotations in webhook.py for mypy compliance
- Add proper type hints for comparison operations
2025-12-03 16:37:14 +00:00
Claude 1588f7bc71 Complete remaining tasks: webhook dispatcher and health checks
- Add WebhookDispatcher for sending events to external services
  - Webhook configuration loading from dict config
  - JSONPath-like filter expression support for event filtering
  - Async HTTP POST sending with httpx
  - Retry logic with exponential backoff
  - Comprehensive test suite

- Add health check infrastructure for Interface and Collector
  - HealthReporter class for periodic status file updates
  - CLI commands: meshcore-hub health interface/collector
  - Updated Docker Compose to use CLI health checks
  - File-based health status for non-HTTP components

- Update TASKS.md progress to 99% (218/221 tasks)
  - Remaining 3 tasks are optional (docs/ directory)
2025-12-03 16:32:05 +00:00
Louis King 1ed62e26bd Updates 2025-12-03 16:13:59 +00:00
Louis King 601dd51cd0 Updates 2025-12-03 15:47:42 +00:00
JingleManSweep 844166c7a8 Merge pull request #6 from ipnet-mesh/claude/docker-compose-setup-01P96hkMwuGy9f2ZM5UwZVxf
Set up Docker with compose profiles
2025-12-03 15:38:54 +00:00
Claude 50a3b5be19 Complete Phase 6: Docker deployment and CI/CD
Health Checks (6.3):
- Add is_healthy property and get_health_status() to Receiver/Sender
- Add is_healthy property and get_health_status() to Collector Subscriber
- Track device, MQTT, and database connection status

Documentation (6.5):
- Update README with Docker Compose profiles documentation
- Add serial device access instructions
- Update API documentation URLs and add health check info

CI/CD (6.6):
- Add .github/workflows/ci.yml for linting, testing, and building
- Add .github/workflows/docker.yml for Docker image builds
- Support Python 3.11 and 3.12 in CI matrix
- Configure Codecov for coverage reporting

End-to-End Testing (6.7):
- Add tests/e2e/ directory with Docker Compose test configuration
- Add e2e test fixtures with service health waiting
- Add comprehensive e2e tests for API, Web, and auth flows
2025-12-03 15:38:02 +00:00
Claude 84b57a211f Add example members.json file for web dashboard
Remove members.json from .gitignore to provide a sample configuration
that users can customize for their network member listings.
2025-12-03 15:28:03 +00:00
Claude caa67059a3 Add Docker setup with Compose profiles for Phase 6
- Add multi-stage Dockerfile with python:3.11-slim base
- Add docker-compose.yml with profile support for selective service deployment:
  - mqtt: Eclipse Mosquitto MQTT broker
  - interface-receiver: MeshCore device receiver mode
  - interface-sender: MeshCore device sender mode
  - collector: MQTT subscriber and database storage
  - api: REST API server
  - web: Web dashboard
  - mock: Testing profile with mock devices
  - migrate: Database migration service
- Add mosquitto.conf with listener and persistence configuration
- Add .dockerignore for faster builds
- Add .env.example for Docker environment configuration
- Add sample members.json for web dashboard
2025-12-03 15:26:05 +00:00
JingleManSweep 0b1a971a39 Merge pull request #5 from ipnet-mesh/claude/web-tests-tasks-01GDCM3ptbX1xky72pg1ypNU
Continue Web Tests from TASKS.md
2025-12-03 15:16:44 +00:00
Claude 166f3b7384 Fix linting and type errors in web tests
- Remove unused imports (AsyncMock, patch, pytest)
- Fix type annotations: use Any instead of any
2025-12-03 15:15:05 +00:00
Claude 65c77afbe0 Add web dashboard tests for Phase 5.11
- Create conftest.py with MockHttpClient for testing web routes
- Add test_home.py with 9 tests for home page
- Add test_members.py with 11 tests for members page and load_members function
- Add test_network.py with 7 tests for network overview page
- Add test_nodes.py with 15 tests for nodes list and detail pages
- Add test_map.py with 12 tests for map page and data endpoint
- Add test_messages.py with 13 tests for messages page with filtering
- All 67 web tests pass, 184 total tests pass
- Update TASKS.md to mark Phase 5 as 100% complete (186/221 total)
2025-12-03 15:06:40 +00:00
JingleManSweep a98de503c7 Merge pull request #4 from ipnet-mesh/claude/fix-config-defaults-01J6hLPHTuF9VMsrdiGes6s7
Fix default configuration values in tests
2025-12-03 14:53:02 +00:00
Louis King 2be0c65f5d Updates 2025-12-03 14:50:56 +00:00
Claude 3ac5452a08 Fix dashboard route path from /dashboard/dashboard to /dashboard
The dashboard router was mounted with prefix /dashboard and the HTML
route was also /dashboard, making the full path /api/v1/dashboard/dashboard.
Changed the route to / so it's accessible at /api/v1/dashboard.
2025-12-03 14:49:14 +00:00
Claude abe8cad61a Add uv.lock for reproducible dependency resolution 2025-12-03 14:43:05 +00:00
Claude 79cb12287e Fix config tests to ignore .env file when testing defaults
Pass _env_file=None to settings classes to prevent pydantic-settings
from loading values from .env files, which would override the default
values the tests are meant to verify.
2025-12-03 14:42:17 +00:00
JingleManSweep 383e4f2b03 Merge pull request #3 from ipnet-mesh/claude/fix-linting-errors-01GKk8AffQQFixCbjZ4egqn2
Fix flake8 and mypy linting errors
2025-12-03 14:28:33 +00:00
Claude 0ac5ba567c Fix flake8 and mypy linting errors
- Update .flake8 and pre-commit config to properly use flake8 config
- Add B008 to ignored errors (FastAPI Depends pattern)
- Add E402 to ignored errors (intentional module-level imports)
- Remove unused imports from test files and source files
- Fix f-strings without placeholders
- Add type annotations to inner async functions
- Fix SQLAlchemy execute() to use text() wrapper
- Add type: ignore comments for alembic.command imports
- Exclude alembic/ directory from mypy in pre-commit
- Update mypy overrides for test files to not require type annotations
- Fix type annotations for params dicts in web routes
- Fix generator return type in test fixtures
2025-12-03 01:24:42 +00:00
Louis King c88ee99e55 Updates 2025-12-03 01:07:22 +00:00
JingleManSweep 10582fd60a Merge pull request #2 from ipnet-mesh/claude/fix-failing-tests-01P5ABdMT8oH6mUnPaxRdaVq
Fix failing pytest tests
2025-12-03 01:04:24 +00:00
Claude 8d1b0f0749 Fix failing API tests
- Return 401 instead of 403 for invalid API keys in require_read
- Add /dashboard prefix to dashboard router so routes are at /api/v1/dashboard/*
- Rename message filter param from 'type' to 'message_type' for clarity
- Add GET /nodes/{public_key}/tags/{key} endpoint for single tag retrieval
2025-12-03 01:03:41 +00:00
JingleManSweep 1fcabb5013 Merge pull request #1 from ipnet-mesh/claude/start-project-development-01H6a5BA6hagqsB7JvCBjcdn
Initial Project Setup
2025-12-03 00:48:53 +00:00
Louis King 8964e1b2d4 Fix meshcore library integration and add device initialization
- Fix event payload extraction: use event.payload instead of event.attributes
  to get full message data (text, pubkey_prefix, etc.)
- Fix command API: use mc.commands.* pattern instead of importing functions
- Add device initialization on receiver startup:
  - Set hardware clock to current Unix timestamp
  - Send local (non-flood) advertisement
  - Start automatic message fetching via start_auto_message_fetching()
- Add set_time() and start_message_fetching() methods to device interface
- Update AGENTS.md with meshcore library integration documentation
- Update README.md with correct meshcore library reference

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-03 00:47:48 +00:00
Claude 5a1b4fb855 Fix device public key retrieval and add node-address config option
- Use meshcore library's self_info property directly after connection
  instead of waiting for SELF_INFO event (which was already processed)
- Add --node-address CLI option to override device public key/address
- Support NODE_ADDRESS environment variable
- Add node_address parameter to DeviceConfig and all related functions
2025-12-03 00:26:21 +00:00
Claude 32fff7749a Fix MQTT authentication and integrate meshcore library
CLI fixes:
- Add --mqtt-username and --mqtt-password options to receiver/sender
  shortcut commands so they work with authenticated MQTT brokers
- These options read from MQTT_USERNAME/MQTT_PASSWORD env vars

Device integration:
- Integrate with meshcore>=2.2.0 library for actual serial device support
- Implement async-to-sync bridge for meshcore's async API
- Add proper event subscription mapping between meshcore and hub events
- Add meshcore>=2.2.0 to dependencies in pyproject.toml
2025-12-03 00:16:03 +00:00
Claude 9e88fcc0b6 Fix .env file loading for CLI environment variables
- Add load_dotenv() call at module import to load .env before Click
  processes envvar parameters
- Add python-dotenv>=1.0.0 as explicit dependency in pyproject.toml

Click's envvar only reads from OS environment, not .env files.
By loading dotenv early, all CLI commands now respect .env configuration.
2025-12-03 00:09:15 +00:00
Claude 8971b580f8 Update TASKS.md with completed phases 1-5 progress
Mark completed tasks for:
- Phase 1: Foundation (100%)
- Phase 2: Interface (100%)
- Phase 3: Collector (74% - webhook pending)
- Phase 4: API (100%)
- Phase 5: Web Dashboard (83% - tests pending)

Update progress summary: 179/221 tasks (81%) complete
Add session log entries for development history
2025-12-03 00:04:50 +00:00
Claude 8d1f4bb50e Phase 5: Implement Web Dashboard component
Add web dashboard with FastAPI and Jinja2 templates for visualizing
network status, nodes, messages, and members with an interactive map.

Features:
- FastAPI app with Jinja2 templating and httpx client for API
- Responsive UI using Tailwind CSS with DaisyUI components
- Interactive map with Leaflet.js for node visualization
- Pages: home, network stats, nodes list/detail, messages, map, members
- CLI with extensive configuration (network info, API, members file)
- Development mode with uvicorn auto-reload support
2025-12-02 23:56:05 +00:00
Claude aefa9b735f Phase 4: Implement REST API component
- Add FastAPI application with lifespan management
- Implement bearer token authentication (read/admin levels)
- Create comprehensive REST API routes:
  - Nodes: list, get by public key
  - Node tags: CRUD operations
  - Messages: list with filters, get by ID
  - Advertisements: list with filters, get by ID
  - Telemetry: list with filters, get by ID
  - Trace paths: list with filters, get by ID
  - Commands: send message, channel message, advertisement
  - Dashboard: stats API and HTML dashboard
- Add API CLI command for running the server
- Create API test suite with 44 passing tests

Routes use proper RESTful status codes (201 Created, 204 No Content).
Authentication is optional - when keys not configured, endpoints are open.
2025-12-02 23:41:32 +00:00
Claude 2617dace7b Implement Phase 3: Collector Component
This commit adds the complete Collector component for storing MeshCore events:

MQTT Subscriber (collector/subscriber.py):
- Subscribes to all event topics from MQTT broker
- Routes events to appropriate handlers
- Manages database persistence

Event Handlers:
- advertisement.py: Handles node advertisements, upserts nodes
- message.py: Handles contact and channel messages
- trace.py: Handles network trace path data
- telemetry.py: Handles sensor telemetry responses
- contacts.py: Handles contacts sync events
- event_log.py: Generic handler for informational events

CLI (collector/cli.py):
- Click command for running the collector
- Environment variable support for all options
- Integrated with main CLI

Tests:
- Subscriber tests
- Handler tests for advertisement, message, telemetry
2025-12-02 23:21:10 +00:00
Claude 15023b8d4a Implement Phase 2: Interface Component
This commit adds the complete Interface component for MeshCore device communication:

Device abstraction (interface/device.py):
- BaseMeshCoreDevice abstract class
- MeshCoreDevice for real hardware (placeholder for meshcore_py)
- DeviceConfig for connection settings
- EventType enumeration for all MeshCore events
- Event handler registration and dispatching

Mock device (interface/mock_device.py):
- MockMeshCoreDevice for testing without hardware
- Configurable event generation
- Simulated network with multiple mock nodes
- Support for injecting custom events

RECEIVER mode (interface/receiver.py):
- Subscribes to device events
- Publishes events to MQTT broker
- Signal handling for graceful shutdown

SENDER mode (interface/sender.py):
- Subscribes to MQTT command topics
- Dispatches commands to MeshCore device
- Handles send_msg, send_channel_msg, send_advert, etc.

CLI (interface/cli.py):
- Click commands for running interface
- Convenience commands for receiver/sender modes
- Environment variable support for all options

Tests:
- Device abstraction tests
- Mock device tests
- Receiver and sender mode tests
2025-12-02 23:16:58 +00:00
Claude 3c1625d4c9 Implement Phase 1: Foundation for MeshCore Hub
This commit establishes the complete foundation for the MeshCore Hub project:

- Project setup with pyproject.toml (Python 3.11+, all dependencies)
- Development tools: black, flake8, mypy, pytest configuration
- Pre-commit hooks for code quality
- Package structure with all components (interface, collector, api, web)

Common package includes:
- Pydantic settings for all component configurations
- SQLAlchemy models for nodes, messages, advertisements, traces, telemetry
- Pydantic schemas for events, API requests/responses, commands
- MQTT client utilities with topic builder
- Logging configuration

Database infrastructure:
- Alembic setup with initial migration for all tables
- Database manager with session handling

CLI entry point:
- Click-based CLI with subcommands for all components
- Database migration commands (upgrade, downgrade, revision)

Tests:
- Basic test suite for config and models
- pytest fixtures for in-memory database testing
2025-12-02 23:10:53 +00:00
Louis King 1d5377b639 Updates 2025-12-02 22:58:16 +00:00
Louis King 0ccf7619d1 Updates 2025-12-02 22:43:19 +00:00
JingleManSweep 66d3a9298f Initial commit 2025-12-02 21:52:34 +00:00