Commit Graph

60 Commits

Author SHA1 Message Date
Louis King
2a153a5239 Add role label to node last seen metric and filter alerts by role
Joins NodeTag (key='role') to the node last seen Prometheus metric so
alert rules can target infrastructure nodes only (role="infra").

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 00:01:20 +00:00
Louis King
5a20da3afa Add Prometheus metrics endpoint, Alertmanager, and 1h stats window
Add /metrics endpoint with Prometheus gauges for nodes, messages,
advertisements, telemetry, trace paths, events, and members. Include
per-node last_seen timestamps for alerting. Add Alertmanager service
to Docker Compose metrics profile with default blackhole receiver.
Add NodeNotSeen alert rule (48h threshold). Add 1h time window to
all windowed metrics alongside existing 24h/7d/30d windows.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 23:06:07 +00:00
Louis King
189eb3a139 Add HTTP caching for web dashboard resources
Implement cache-control middleware to optimize browser caching and reduce
bandwidth usage. Static files are cached for 1 year when accessed with
version parameters, while dynamic content is never cached.

Changes:
- Add CacheControlMiddleware with path-based caching logic
- Register middleware in web app after ProxyHeadersMiddleware
- Add version query parameters to CSS, JS, and app.js references
- Create comprehensive test suite (20 tests) for all cache behaviors

Cache strategy:
- Static files with ?v=X.Y.Z: 1 year (immutable)
- Static files without version: 1 hour (fallback)
- SPA shell HTML: no-cache (dynamic config)
- Health endpoints: no-cache, no-store (always fresh)
- Map data: 5 minutes (location updates)
- Custom pages: 1 hour (stable markdown)
- API proxy: pass-through (backend controls)

All 458 tests passing, 95% middleware coverage.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-14 00:01:08 +00:00
Louis King
5272a72647 Refactor i18n, add translation guide, and audit documentation
## i18n Refactoring

- Refactor admin translations to use common composable patterns
- Add common patterns: delete_entity_confirm, entity_added_success, move_entity_to_another_node, etc.
- Remove 18 duplicate keys from admin_members and admin_node_tags sections
- Update all admin JavaScript files to use new common patterns with dynamic entity composition
- Fix label consistency: rename first_seen to first_seen_label to match naming convention

## Translation Documentation

- Create comprehensive translation reference guide (languages.md) with 200+ documented keys
- Add translation architecture documentation to AGENTS.md with examples and best practices
- Add "Help Translate" call-to-action section in README with link to translation guide
- Add i18n feature to README features list

## Documentation Audit

- Add undocumented config options: API_KEY, WEB_LOCALE, WEB_DOMAIN to README and .env.example
- Fix outdated CLI syntax: interface --mode receiver → interface receiver
- Update database migration commands to use CLI wrapper (meshcore-hub db) instead of direct alembic
- Add static/locales/ directory to project structure section
- Add i18n configuration (WEB_LOCALE, WEB_THEME) to docker-compose.yml

## Testing

- All 438 tests passing
- All pre-commit checks passing (black, flake8, mypy)
- Added tests for new common translation patterns

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-13 22:19:37 +00:00
Louis King
a15e91c754 Further refine i18n structure
- Remove "nav" section, use "entities" references instead
- Remove composite strings like "Total Nodes", "Recent Advertisements"
  - Use composed patterns: t('common.total_entity', { entity: t('entities.nodes') })
  - Use common.recent_entity, common.edit_entity, common.add_entity patterns
- Hardcode MeshCore tagline (official trademark, not configurable)
- Update all page components and templates to use entity-based translations
- Update tests to reflect new structure
- Remove redundant page-specific composite keys

This maximizes reusability and reduces duplication across translations.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-13 21:32:36 +00:00
Claude
127cd7adf6 Add i18n support for web dashboard
Implement lightweight i18n infrastructure with shared JSON translation
files used by both server-side Jinja2 templates and client-side SPA.

- Add custom i18n module (Python + JS, ~80 lines total, zero deps)
- Create en.json with ~200 translation keys covering all web strings
- Add WEB_LOCALE config setting (default: 'en', with localStorage override)
- Translate all navigation labels, page titles, and footer in spa.html
- Translate all 13 SPA page modules (home, dashboard, nodes, etc.)
- Translate shared components (pagination, relative time, charts)
- Translate all 3 admin pages (index, members, node-tags)
- Fix Adverts/Advertisements inconsistency (standardize to Advertisements)
- Add i18n unit tests with 100% coverage

https://claude.ai/code/session_01FbnUnwYAwPrsQmAh5EuSkF
2026-02-13 18:49:06 +00:00
Louis King
706c32ae01 Add feature flags to control web dashboard page visibility
Operators can now disable specific pages (Dashboard, Nodes, Advertisements,
Messages, Map, Members, Pages) via FEATURE_* environment variables. Disabled
features are fully hidden: removed from navigation, return 404 on routes,
and excluded from sitemap/robots.txt. Dashboard auto-disables when all of
Nodes/Advertisements/Messages are off. Map auto-disables when Nodes is off.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 15:43:23 +00:00
Claude
9b09e32d41 Fix admin authentication bypass in web dashboard
The admin pages only checked config.admin_enabled but not
config.is_authenticated, allowing unauthenticated users to access
admin functionality when WEB_ADMIN_ENABLED=true. Additionally, the
API proxy forwarded the service-level Bearer token on all requests
regardless of user authentication, granting full admin API access
to unauthenticated browsers.

Server-side: block POST/PUT/DELETE/PATCH through the API proxy when
admin is enabled and no X-Forwarded-User header is present.

Client-side: add is_authenticated check to all three admin pages,
showing a sign-in prompt instead of admin content.

https://claude.ai/code/session_01HYuz5XLjYZ6JaowWqz643A
2026-02-10 01:20:04 +00:00
Louis King
f1bceb5780 Rewrite web dashboard as Single Page Application
Replace server-side rendered Jinja2 page routes with a client-side SPA
using ES modules, lit-html templating, and a custom History API router.
All page rendering now happens in the browser with efficient DOM diffing.

Key changes:
- Add SPA router, API client, shared components, and 14 page modules
- Serve single spa.html shell template with catch-all route
- Remove server-side page routes (web/routes/) and legacy JS files
- Add centralized OKLCH color palette in CSS custom properties
- Add colored nav icons, navbar spacing, and loading spinner
- Add canonical URL and SEO path exclusions to SPA router
- Update charts.js to read from shared color palette
- Update tests for SPA architecture (template-agnostic assertions)
- Update AGENTS.md and README.md with SPA documentation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 00:23:45 +00:00
Louis King
8f660d6b94 Enhance map page with GPS fallback, infrastructure filter, and UI improvements
- Add GPS coordinate fallback: use tag coords, fall back to model coords
- Filter out nodes at (0, 0) coordinates (likely unset defaults)
- Add "Show" filter to toggle between All Nodes and Infrastructure Only
- Add "Show Labels" checkbox (labels hidden by default, appear on hover)
- Infrastructure nodes display network logo instead of emoji
- Add radius-based bounds filtering (20km) to prevent outlier zoom issues
- Position labels underneath pins, centered with transparent background
- Calculate and return infra_center for infrastructure node focus
- Initial map view focuses on infrastructure nodes when available
- Update popup button to outline style
- Add comprehensive tests for new functionality

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-07 20:05:56 +00:00
Louis King
b18b3c9aa4 Refactor PAGES_HOME to CONTENT_HOME and add custom logo support
- Replace PAGES_HOME with CONTENT_HOME configuration (default: ./content)
- Content directory now contains pages/ and media/ subdirectories
- Add support for custom logo at $CONTENT_HOME/media/images/logo.svg
- Custom logo replaces favicon and navbar/home logos when present
- Mount media directory as /media for serving custom assets
- Simplify default logo to generic WiFi-style radiating arcs
- Update documentation and example directory structure
- Update tests for new CONTENT_HOME structure

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-07 13:45:42 +00:00
Louis King
deaab9b9de Rename /network to /dashboard and add reusable icon macros
- Renamed network route, template, and tests to dashboard
- Added logo.svg for favicon and navbar branding
- Created reusable Jinja2 icon macros for navigation and UI elements
- Updated home page hero layout with centered content and larger logo
- Added Map button alongside Dashboard button in hero section
- Navigation menu items now display icons before labels

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-06 22:53:36 +00:00
Louis King
0f50bf4a41 Add custom markdown pages feature to web dashboard
Allows adding static content pages (About, FAQ, etc.) as markdown files
with YAML frontmatter. Pages are stored in PAGES_HOME directory (default:
./pages), automatically appear in navigation menu, and are included in
the sitemap.

- Add PageLoader class to parse markdown with frontmatter
- Add /pages/{slug} route for rendering custom pages
- Add PAGES_HOME config setting to WebSettings
- Add prose CSS styles for markdown content
- Add pages to navigation and sitemap
- Update docker-compose.yml with pages volume mount
- Add comprehensive tests for PageLoader and routes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-06 18:36:23 +00:00
Louis King
345ffd219b Separate API prefix search from exact match endpoint
- Add /api/v1/nodes/prefix/{prefix} for prefix-based node lookup
- Change /api/v1/nodes/{public_key} to exact match only
- /n/{prefix} now simply redirects to /nodes/{prefix}
- /nodes/{key} resolves prefixes via API and redirects to full key
2026-01-26 22:27:15 +00:00
Louis King
9661b22390 Fix node detail 404 to use custom error page 2026-01-26 22:11:48 +00:00
Louis King
0b3ac64845 Add prefix matching support to node API endpoint
Allow users to navigate to a node using any prefix of its public key
instead of requiring the full 64-character key. If multiple nodes match
the prefix, the first one alphabetically is returned.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 21:27:36 +00:00
Louis King
f7b4df13a7 Added more test coverage 2026-01-12 21:00:02 +00:00
Louis King
13bae5c8d7 Added more test coverage 2026-01-12 20:34:53 +00:00
Louis King
307f3935e0 Add access denied page for unauthenticated admin access
When users try to access /a/ without valid OAuth2Proxy headers (e.g.,
GitHub account not in org), they now see a friendly 403 page instead
of a 500 error. Added authentication checks to all admin routes.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 13:34:03 +00:00
Louis King
ed2cf09ff3 Improve admin UI and remove unused coordinate tag type
- Replace node type badge with icon in admin tag editor
- Add Edit/Add Tags button on node detail page (when admin enabled and authenticated)
- Remove automatic seed container startup to prevent overwriting user changes
- Remove unused 'coordinate' value type from node tags (only string, number, boolean remain)
2026-01-11 12:49:34 +00:00
Claude
d8a0f2abb8 Fix security vulnerabilities and add validation
- Fix XSS vulnerability by using data attributes instead of inline
  onclick handlers in node_tags.html template
- Fix URL injection by using urlencode for all redirect URL parameters
- Add validation to reject moves where source and destination nodes
  are the same (returns 400 Bad Request)
- Add error handling for response.json() calls that may fail
- Add missing test coverage for update endpoint error scenarios
2026-01-11 11:51:57 +00:00
Claude
367f838371 Add admin interface for managing node tags
Implement CRUD operations for NodeTags in the admin interface:

- Add NodeTagMove schema for moving tags between nodes
- Add PUT /nodes/{public_key}/tags/{key}/move API endpoint
- Add web routes at /a/node-tags for tag management
- Create admin templates with node selector and tag management UI
- Support editing, adding, moving, and deleting tags via API calls
- Add comprehensive tests for new functionality

The interface allows selecting a node from a dropdown, viewing its
tags, and performing all CRUD operations including moving a tag
to a different node without having to delete and recreate it.
2026-01-11 01:34:07 +00:00
Louis King
61d6b6287e Add contact cleanup to interface RECEIVER mode
- Add CONTACT_CLEANUP_ENABLED and CONTACT_CLEANUP_DAYS settings
- Implement remove_contact and schedule_remove_contact on device classes
- During contact sync, remove stale contacts from companion node
- Stale contacts (not advertised for > N days) not published to MQTT
- Update Python version to 3.13 across project config
- Remove brittle config tests that assumed default env values
2026-01-08 10:22:27 +00:00
Louis King
a290db0491 Updated chart stats 2025-12-08 19:37:45 +00:00
Louis King
9e621c0029 Fixed test 2025-12-08 16:42:13 +00:00
Louis King
57f51c741c Fixed Member model 2025-12-08 15:13:24 +00:00
Louis King
65b8418af4 Fixed last seen issue 2025-12-08 00:15:25 +00:00
Louis King
64ec1a7135 Receiver nodes now sync contacts to MQTT on every advert received 2025-12-07 23:34:33 +00:00
Louis King
fbd29ff78e Removed friendly name support and tidied tags 2025-12-07 23:02:19 +00:00
Louis King
84b8614e29 Updates 2025-12-06 21:42:33 +00:00
Louis King
3bc47a33bc Added data retention and node cleanup 2025-12-06 21:27:19 +00:00
Louis King
d715e4e4f0 Updates 2025-12-06 13:33:02 +00:00
Claude
6e3b86a1ad Add collector-level event deduplication using content hashes
Replace presentation-layer deduplication with collector-level approach:
- Add event_hash column to messages, advertisements, trace_paths, telemetry tables
- Handlers compute content hashes and skip duplicate events at insertion time
- Use 5-minute time buckets for advertisements and telemetry
- Include Alembic migration for schema changes
2025-12-06 12:23:14 +00:00
Claude
c80986fe67 Add event deduplication at presentation layer
When multiple receiver nodes are running, the same mesh events (messages,
advertisements) are reported multiple times. This causes duplicate entries
in the Web UI.

Changes:
- Add hash_utils.py with deterministic hash functions for each event type
- Add `dedupe` parameter to messages and advertisements API endpoints (default: True)
- Update dashboard stats to use distinct counts for messages/advertisements
- Deduplicate recent advertisements and channel messages in dashboard
- Add comprehensive tests for hash utilities

Hash strategy:
- Messages: hash of text + pubkey_prefix + channel_idx + sender_timestamp + txt_type
- Advertisements: hash of public_key + name + adv_type + flags + 5-minute time bucket
2025-12-06 12:08:07 +00:00
Louis King
23f6c290c9 Updates 2025-12-05 21:17:34 +00:00
Claude
a4b13d3456 Add member-node association support
Members can now have multiple associated nodes, each with a public_key
and node_role (e.g., 'chat', 'repeater'). This replaces the single
public_key field on members with a one-to-many relationship.

Changes:
- Add MemberNode model for member-node associations
- Update Member model to remove public_key, add nodes relationship
- Update Pydantic schemas with MemberNodeCreate/MemberNodeRead
- Update member_import.py to handle nodes list in seed files
- Update API routes to handle nodes in create/update/read operations
- Add Alembic migration to create member_nodes table and migrate data
- Update example seed file with new format
2025-12-05 20:34:09 +00:00
Louis King
0b8fc6e707 Charts 2025-12-05 19:50:22 +00:00
Claude
796e303665 Remove internal UUID fields from API responses
Internal database UUIDs (id, node_id, receiver_node_id) were being
exposed in API responses. These are implementation details that should
not be visible to API consumers. The canonical identifier for nodes
is the 64-char hex public_key.

Changes:
- Remove id, node_id from NodeTagRead, NodeRead schemas
- Remove id from MemberRead schema
- Remove id, receiver_node_id, node_id from MessageRead, AdvertisementRead,
  TracePathRead, TelemetryRead schemas
- Update web map component to use public_key instead of member.id
  for owner filtering
- Update tests to not assert on removed fields
2025-12-05 16:50:21 +00:00
Louis King
d7152a5359 Updates 2025-12-04 19:34:18 +00:00
Claude
df05c3a462 Convert collector seed mechanism from JSON to YAML
- Replace JSON seed files with YAML format for better readability
- Auto-detect YAML primitive types (number, boolean, string) from values
- Add automatic seed import on collector startup
- Split lat/lon into separate tags instead of combined coordinate string
- Add PyYAML dependency and types-PyYAML for type checking
- Update example/seed and contrib/seed/ipnet with clean YAML format
- Update tests to verify YAML primitive type detection
2025-12-04 01:27:03 +00:00
Louis King
e2d865f200 Fix nodes page test to match template output
The test was checking for adv_type values (REPEATER, CLIENT) but the
nodes.html template doesn't display that column. Updated to check for
public key prefixes instead.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 01:03:05 +00:00
Claude
0db0ebf9b2 Move members from web to collector layer with SEED_HOME
- Add Member database model with name, callsign, role, description, contact, and public_key fields
- Add Member Pydantic schemas (MemberCreate, MemberUpdate, MemberRead, MemberList)
- Add members table to initial migration
- Add members API endpoints (GET/POST/PUT/DELETE /api/v1/members)
- Add member_import.py for importing from JSON files
- Update web layer to fetch members from API instead of file
- Add SEED_HOME setting (defaults to ./seed) for seed data files
- Add 'collector seed' command to import node_tags.json and members.json
- Rename tags.json to node_tags.json for consistency
- Move example seed data from example/data/* to example/seed/
- Update tests and configuration
2025-12-03 23:42:16 +00:00
Louis King
6095f35bae Refactor tags.json format to use public_key as object key
Change tag import format from flat list with repeated public_keys to an
object keyed by public_key with nested tags. This makes the JSON more
intuitive and reduces redundancy.

New format supports both shorthand (string values) and full format
(with value and type):
{
  "0123456789abcdef...": {
    "friendly_name": "My Node",
    "location": {"value": "52.0,1.0", "type": "coordinate"}
  }
}

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-03 22:31:27 +00:00
Claude
862820bbd3 Add DATA_HOME configuration for centralized data directory management
- Add DATA_HOME setting to CommonSettings (default: ./data)
- Update CollectorSettings with:
  - effective_database_url property (default: sqlite:///{DATA_HOME}/collector/meshcore.db)
  - effective_tags_file property (default: {DATA_HOME}/collector/tags.json)
  - collector_data_dir property
- Update APISettings with effective_database_url property
- Update WebSettings with:
  - effective_members_file property (default: {DATA_HOME}/web/members.json)
  - web_data_dir property
- Update CLI commands (collector, api, web) to:
  - Accept --data-home option
  - Use effective_* properties for defaults
  - Auto-create data directories on startup
- Update docker-compose.yml.example to use DATA_HOME volume mounts
- Update .env.example with DATA_HOME documentation
- Update PLAN.md and AGENTS.md with data directory structure docs
- Add comprehensive tests for new configuration properties
2025-12-03 19:14:13 +00:00
Claude
cbaf4f451c Add tests to verify Node API returns tags
- Add test_get_node_with_tags to verify GET /nodes/{pk} includes tags
- Add test_list_nodes_includes_tags to verify GET /nodes includes tags
- Update existing tests to assert tags field is present

The Node API was already correctly returning tags via the
lazy="selectin" relationship loading strategy. These tests
document and verify that behavior.
2025-12-03 18:26:32 +00:00
Claude
fe1fd69904 Add node tag import functionality to collector
- Add tag_import.py module with JSON file parsing and database upsert
- Convert collector CLI to group with subcommands for extensibility
- Add 'import-tags' command to import tags from JSON file
- Update docker-compose.yml.example with separated data directories:
  - data/collector for tags.json
  - data/web for members.json
- Add import-tags Docker service for easy containerized imports
- Add example data files in example/data/collector and example/data/web
- Add comprehensive test coverage (20 tests) for tag import
2025-12-03 18:16:03 +00:00
Louis King
e6b3ceb639 Updates 2025-12-03 17:02:57 +00:00
Claude
e57fe7a2d8 Disable e2e tests by default and fix mypy errors
- Add --e2e flag to pytest to run e2e tests
- E2E tests skip by default with clear message
- Fix type annotations in webhook.py for mypy compliance
- Add proper type hints for comparison operations
2025-12-03 16:37:14 +00:00
Claude
1588f7bc71 Complete remaining tasks: webhook dispatcher and health checks
- Add WebhookDispatcher for sending events to external services
  - Webhook configuration loading from dict config
  - JSONPath-like filter expression support for event filtering
  - Async HTTP POST sending with httpx
  - Retry logic with exponential backoff
  - Comprehensive test suite

- Add health check infrastructure for Interface and Collector
  - HealthReporter class for periodic status file updates
  - CLI commands: meshcore-hub health interface/collector
  - Updated Docker Compose to use CLI health checks
  - File-based health status for non-HTTP components

- Update TASKS.md progress to 99% (218/221 tasks)
  - Remaining 3 tasks are optional (docs/ directory)
2025-12-03 16:32:05 +00:00
Claude
50a3b5be19 Complete Phase 6: Docker deployment and CI/CD
Health Checks (6.3):
- Add is_healthy property and get_health_status() to Receiver/Sender
- Add is_healthy property and get_health_status() to Collector Subscriber
- Track device, MQTT, and database connection status

Documentation (6.5):
- Update README with Docker Compose profiles documentation
- Add serial device access instructions
- Update API documentation URLs and add health check info

CI/CD (6.6):
- Add .github/workflows/ci.yml for linting, testing, and building
- Add .github/workflows/docker.yml for Docker image builds
- Support Python 3.11 and 3.12 in CI matrix
- Configure Codecov for coverage reporting

End-to-End Testing (6.7):
- Add tests/e2e/ directory with Docker Compose test configuration
- Add e2e test fixtures with service health waiting
- Add comprehensive e2e tests for API, Web, and auth flows
2025-12-03 15:38:02 +00:00