Commit Graph

21 Commits

Author SHA1 Message Date
Claude df05c3a462 Convert collector seed mechanism from JSON to YAML
- Replace JSON seed files with YAML format for better readability
- Auto-detect YAML primitive types (number, boolean, string) from values
- Add automatic seed import on collector startup
- Split lat/lon into separate tags instead of combined coordinate string
- Add PyYAML dependency and types-PyYAML for type checking
- Update example/seed and contrib/seed/ipnet with clean YAML format
- Update tests to verify YAML primitive type detection
2025-12-04 01:27:03 +00:00
Louis King e2d865f200 Fix nodes page test to match template output
The test was checking for adv_type values (REPEATER, CLIENT) but the
nodes.html template doesn't display that column. Updated to check for
public key prefixes instead.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 01:03:05 +00:00
Claude 0db0ebf9b2 Move members from web to collector layer with SEED_HOME
- Add Member database model with name, callsign, role, description, contact, and public_key fields
- Add Member Pydantic schemas (MemberCreate, MemberUpdate, MemberRead, MemberList)
- Add members table to initial migration
- Add members API endpoints (GET/POST/PUT/DELETE /api/v1/members)
- Add member_import.py for importing from JSON files
- Update web layer to fetch members from API instead of file
- Add SEED_HOME setting (defaults to ./seed) for seed data files
- Add 'collector seed' command to import node_tags.json and members.json
- Rename tags.json to node_tags.json for consistency
- Move example seed data from example/data/* to example/seed/
- Update tests and configuration
2025-12-03 23:42:16 +00:00
Louis King 6095f35bae Refactor tags.json format to use public_key as object key
Change tag import format from flat list with repeated public_keys to an
object keyed by public_key with nested tags. This makes the JSON more
intuitive and reduces redundancy.

New format supports both shorthand (string values) and full format
(with value and type):
{
  "0123456789abcdef...": {
    "friendly_name": "My Node",
    "location": {"value": "52.0,1.0", "type": "coordinate"}
  }
}

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-03 22:31:27 +00:00
Claude 862820bbd3 Add DATA_HOME configuration for centralized data directory management
- Add DATA_HOME setting to CommonSettings (default: ./data)
- Update CollectorSettings with:
  - effective_database_url property (default: sqlite:///{DATA_HOME}/collector/meshcore.db)
  - effective_tags_file property (default: {DATA_HOME}/collector/tags.json)
  - collector_data_dir property
- Update APISettings with effective_database_url property
- Update WebSettings with:
  - effective_members_file property (default: {DATA_HOME}/web/members.json)
  - web_data_dir property
- Update CLI commands (collector, api, web) to:
  - Accept --data-home option
  - Use effective_* properties for defaults
  - Auto-create data directories on startup
- Update docker-compose.yml.example to use DATA_HOME volume mounts
- Update .env.example with DATA_HOME documentation
- Update PLAN.md and AGENTS.md with data directory structure docs
- Add comprehensive tests for new configuration properties
2025-12-03 19:14:13 +00:00
Claude cbaf4f451c Add tests to verify Node API returns tags
- Add test_get_node_with_tags to verify GET /nodes/{pk} includes tags
- Add test_list_nodes_includes_tags to verify GET /nodes includes tags
- Update existing tests to assert tags field is present

The Node API was already correctly returning tags via the
lazy="selectin" relationship loading strategy. These tests
document and verify that behavior.
2025-12-03 18:26:32 +00:00
Claude fe1fd69904 Add node tag import functionality to collector
- Add tag_import.py module with JSON file parsing and database upsert
- Convert collector CLI to group with subcommands for extensibility
- Add 'import-tags' command to import tags from JSON file
- Update docker-compose.yml.example with separated data directories:
  - data/collector for tags.json
  - data/web for members.json
- Add import-tags Docker service for easy containerized imports
- Add example data files in example/data/collector and example/data/web
- Add comprehensive test coverage (20 tests) for tag import
2025-12-03 18:16:03 +00:00
Louis King e6b3ceb639 Updates 2025-12-03 17:02:57 +00:00
Claude e57fe7a2d8 Disable e2e tests by default and fix mypy errors
- Add --e2e flag to pytest to run e2e tests
- E2E tests skip by default with clear message
- Fix type annotations in webhook.py for mypy compliance
- Add proper type hints for comparison operations
2025-12-03 16:37:14 +00:00
Claude 1588f7bc71 Complete remaining tasks: webhook dispatcher and health checks
- Add WebhookDispatcher for sending events to external services
  - Webhook configuration loading from dict config
  - JSONPath-like filter expression support for event filtering
  - Async HTTP POST sending with httpx
  - Retry logic with exponential backoff
  - Comprehensive test suite

- Add health check infrastructure for Interface and Collector
  - HealthReporter class for periodic status file updates
  - CLI commands: meshcore-hub health interface/collector
  - Updated Docker Compose to use CLI health checks
  - File-based health status for non-HTTP components

- Update TASKS.md progress to 99% (218/221 tasks)
  - Remaining 3 tasks are optional (docs/ directory)
2025-12-03 16:32:05 +00:00
Claude 50a3b5be19 Complete Phase 6: Docker deployment and CI/CD
Health Checks (6.3):
- Add is_healthy property and get_health_status() to Receiver/Sender
- Add is_healthy property and get_health_status() to Collector Subscriber
- Track device, MQTT, and database connection status

Documentation (6.5):
- Update README with Docker Compose profiles documentation
- Add serial device access instructions
- Update API documentation URLs and add health check info

CI/CD (6.6):
- Add .github/workflows/ci.yml for linting, testing, and building
- Add .github/workflows/docker.yml for Docker image builds
- Support Python 3.11 and 3.12 in CI matrix
- Configure Codecov for coverage reporting

End-to-End Testing (6.7):
- Add tests/e2e/ directory with Docker Compose test configuration
- Add e2e test fixtures with service health waiting
- Add comprehensive e2e tests for API, Web, and auth flows
2025-12-03 15:38:02 +00:00
Claude 166f3b7384 Fix linting and type errors in web tests
- Remove unused imports (AsyncMock, patch, pytest)
- Fix type annotations: use Any instead of any
2025-12-03 15:15:05 +00:00
Claude 65c77afbe0 Add web dashboard tests for Phase 5.11
- Create conftest.py with MockHttpClient for testing web routes
- Add test_home.py with 9 tests for home page
- Add test_members.py with 11 tests for members page and load_members function
- Add test_network.py with 7 tests for network overview page
- Add test_nodes.py with 15 tests for nodes list and detail pages
- Add test_map.py with 12 tests for map page and data endpoint
- Add test_messages.py with 13 tests for messages page with filtering
- All 67 web tests pass, 184 total tests pass
- Update TASKS.md to mark Phase 5 as 100% complete (186/221 total)
2025-12-03 15:06:40 +00:00
Claude 3ac5452a08 Fix dashboard route path from /dashboard/dashboard to /dashboard
The dashboard router was mounted with prefix /dashboard and the HTML
route was also /dashboard, making the full path /api/v1/dashboard/dashboard.
Changed the route to / so it's accessible at /api/v1/dashboard.
2025-12-03 14:49:14 +00:00
Claude 79cb12287e Fix config tests to ignore .env file when testing defaults
Pass _env_file=None to settings classes to prevent pydantic-settings
from loading values from .env files, which would override the default
values the tests are meant to verify.
2025-12-03 14:42:17 +00:00
Claude 0ac5ba567c Fix flake8 and mypy linting errors
- Update .flake8 and pre-commit config to properly use flake8 config
- Add B008 to ignored errors (FastAPI Depends pattern)
- Add E402 to ignored errors (intentional module-level imports)
- Remove unused imports from test files and source files
- Fix f-strings without placeholders
- Add type annotations to inner async functions
- Fix SQLAlchemy execute() to use text() wrapper
- Add type: ignore comments for alembic.command imports
- Exclude alembic/ directory from mypy in pre-commit
- Update mypy overrides for test files to not require type annotations
- Fix type annotations for params dicts in web routes
- Fix generator return type in test fixtures
2025-12-03 01:24:42 +00:00
Louis King c88ee99e55 Updates 2025-12-03 01:07:22 +00:00
Claude aefa9b735f Phase 4: Implement REST API component
- Add FastAPI application with lifespan management
- Implement bearer token authentication (read/admin levels)
- Create comprehensive REST API routes:
  - Nodes: list, get by public key
  - Node tags: CRUD operations
  - Messages: list with filters, get by ID
  - Advertisements: list with filters, get by ID
  - Telemetry: list with filters, get by ID
  - Trace paths: list with filters, get by ID
  - Commands: send message, channel message, advertisement
  - Dashboard: stats API and HTML dashboard
- Add API CLI command for running the server
- Create API test suite with 44 passing tests

Routes use proper RESTful status codes (201 Created, 204 No Content).
Authentication is optional - when keys not configured, endpoints are open.
2025-12-02 23:41:32 +00:00
Claude 2617dace7b Implement Phase 3: Collector Component
This commit adds the complete Collector component for storing MeshCore events:

MQTT Subscriber (collector/subscriber.py):
- Subscribes to all event topics from MQTT broker
- Routes events to appropriate handlers
- Manages database persistence

Event Handlers:
- advertisement.py: Handles node advertisements, upserts nodes
- message.py: Handles contact and channel messages
- trace.py: Handles network trace path data
- telemetry.py: Handles sensor telemetry responses
- contacts.py: Handles contacts sync events
- event_log.py: Generic handler for informational events

CLI (collector/cli.py):
- Click command for running the collector
- Environment variable support for all options
- Integrated with main CLI

Tests:
- Subscriber tests
- Handler tests for advertisement, message, telemetry
2025-12-02 23:21:10 +00:00
Claude 15023b8d4a Implement Phase 2: Interface Component
This commit adds the complete Interface component for MeshCore device communication:

Device abstraction (interface/device.py):
- BaseMeshCoreDevice abstract class
- MeshCoreDevice for real hardware (placeholder for meshcore_py)
- DeviceConfig for connection settings
- EventType enumeration for all MeshCore events
- Event handler registration and dispatching

Mock device (interface/mock_device.py):
- MockMeshCoreDevice for testing without hardware
- Configurable event generation
- Simulated network with multiple mock nodes
- Support for injecting custom events

RECEIVER mode (interface/receiver.py):
- Subscribes to device events
- Publishes events to MQTT broker
- Signal handling for graceful shutdown

SENDER mode (interface/sender.py):
- Subscribes to MQTT command topics
- Dispatches commands to MeshCore device
- Handles send_msg, send_channel_msg, send_advert, etc.

CLI (interface/cli.py):
- Click commands for running interface
- Convenience commands for receiver/sender modes
- Environment variable support for all options

Tests:
- Device abstraction tests
- Mock device tests
- Receiver and sender mode tests
2025-12-02 23:16:58 +00:00
Claude 3c1625d4c9 Implement Phase 1: Foundation for MeshCore Hub
This commit establishes the complete foundation for the MeshCore Hub project:

- Project setup with pyproject.toml (Python 3.11+, all dependencies)
- Development tools: black, flake8, mypy, pytest configuration
- Pre-commit hooks for code quality
- Package structure with all components (interface, collector, api, web)

Common package includes:
- Pydantic settings for all component configurations
- SQLAlchemy models for nodes, messages, advertisements, traces, telemetry
- Pydantic schemas for events, API requests/responses, commands
- MQTT client utilities with topic builder
- Logging configuration

Database infrastructure:
- Alembic setup with initial migration for all tables
- Database manager with session handling

CLI entry point:
- Click-based CLI with subcommands for all components
- Database migration commands (upgrade, downgrade, revision)

Tests:
- Basic test suite for config and models
- pytest fixtures for in-memory database testing
2025-12-02 23:10:53 +00:00