Commit Graph

45 Commits

Author SHA1 Message Date
Louis King 385d1ab141 feat: add optional Redis caching layer for API endpoints
Add Redis-backed response caching for read-heavy API endpoints (nodes,
advertisements, messages, channels, dashboard, profiles) with configurable
TTL, key prefix isolation, and graceful fallback when Redis is unavailable.

New files:
- common/redis.py: CacheBackend, NullCache, RedisCacheBackend
- api/cache.py: @cached decorator, sorted_query_string helper
- tests/test_api/test_cache.py: 23 unit tests

Changes:
- pyproject.toml: add redis[hiredis] dependency
- common/config.py: 8 Redis settings on APISettings
- api/cli.py: Redis Click options + startup banner
- api/app.py: Redis lifespan init/cleanup, X-Cache middleware, health check
- 6 route files: apply @cached decorator to list endpoints
- docker-compose.yml: Redis service (cache profile), env vars
- docker-compose.dev.yml: Redis port exposure
- .env.example, README.md, AGENTS.md, docs/upgrading.md: documentation

Redis is disabled by default (REDIS_ENABLED=false). Enable with
--profile cache and REDIS_ENABLED=true.
2026-06-09 23:08:49 +01:00
Louis King f7d9901c9b Split NETWORK_RADIO_CONFIG into individual env vars and add FEATURE_RADIO_CONFIG flag
- Replace single NETWORK_RADIO_CONFIG comma-delimited string with six
  individual environment variables: NETWORK_RADIO_PROFILE, _FREQUENCY,
  _BANDWIDTH, _SPREADING_FACTOR, _CODING_RATE, _TX_POWER
- Radio config fields now use raw numeric types (float/int) with units
  applied dynamically via RadioConfig.format_for_display()
- Add FEATURE_RADIO_CONFIG feature flag to control radio config panel
  visibility on the home page (default: enabled)
- Remove from_config_string class method (no backwards compatibility)
- Update Click CLI options, create_app() signature, and _build_config_json()
- Update docker-compose.yml, .env.example, README.md, AGENTS.md
- Add upgrading.md v0.12.0 section with migration instructions
- Add test coverage for schema, config, and feature flag
2026-06-07 14:35:40 +01:00
Louis King 5f6d44c7b8 Add database-backed channels with role-based visibility and web dashboard
Replaces env-var channel keys with a Channel database model and periodic
DB refresh in the collector. Adds Channels dashboard page with QR codes,
channel visibility filtering on messages/dashboard APIs, and channel card
navigation to filtered messages view.
2026-05-20 00:37:05 +01:00
Louis King dd36a240ba feat: add network announcement flash banner with Markdown support
Add NETWORK_ANNOUNCEMENT env var that displays a dismissible flash banner
on every page when set. Announcement text supports Markdown (bold, italic,
links, inline code) rendered to HTML server-side at startup.
2026-05-09 12:27:20 +01:00
Louis King 28255261fb fix: correct NODE_CLEANUP_DAYS default from 7 to 30 and add missing env vars to docs
NODE_CLEANUP_DAYS source of truth in Pydantic Settings is 30, not 7. Fixed in README, .env.example, docker-compose.yml, and docker-source-guide.md. Also added missing OIDC_POST_LOGOUT_REDIRECT_URI, WEB_AUTO_REFRESH_SECONDS, NETWORK_DOMAIN to docker-compose.yml web service. Added WEB_LOCALE and WEB_DATETIME_LOCALE to AGENTS.md env vars list.
2026-05-05 14:27:52 +01:00
Louis King d37b30a05b Replace Member model with UserProfile-backed data
Remove the static Member model/table, CRUD API, YAML seed files, and
admin UI. Replace with UserProfile-driven members page that reads roles
from OIDC identity provider. Key changes:

- Drop members table, add roles column to user_profiles (Alembic migration)
- Add GET /api/v1/user/profiles (paginated, no user_id exposed)
- Add GET /api/v1/user/profile/me (auto-creates profile for current user)
- Replace member_id node tag filter with adopted_by (profile UUID)
- Members page now shows profiles grouped by operator/member roles
- Profile page supports public view (/profile/:id) and owner edit (/profile)
- Node detail page shows adoption card side-by-side with public key card
- Auto-create user profile during OIDC login callback
- Hide Adopted Nodes section for non-operator/admin users
- Add member since date to profile cards
- Add role badges and adopted node badges to member tiles
- Add antenna/users icons to Members page group headers
2026-04-30 20:57:26 +01:00
Louis King 31418e6847 Add user profiles with node adoption via /v1/adoptions endpoint
Move adopt/release from profile routes to dedicated /v1/adoptions endpoint.
Node API now returns adopted_by field. Profile page shows read-only adopted
nodes. Node detail page has adopt/release buttons (operator adopts, admin
can release any). Admin release bypasses ownership check.
2026-04-30 00:07:49 +01:00
Louis King 02c0a8f1b7 Add OIDC/OAuth2 authentication via Authlib
Replace WEB_ADMIN_ENABLED with full OIDC support using Authlib.
Admin access now requires authenticated sessions with IdP-assigned
roles instead of an open toggle.

- Add authlib and itsdangerous dependencies
- Add OIDC settings to WebSettings (13 env vars)
- Create web/oidc.py module (OAuth registry, session helpers)
- Add /auth/login, /auth/callback, /auth/logout, /auth/user routes
- Gate API proxy writes to admin sessions when OIDC enabled
- Protect /a/ routes with session check (redirect to login)
- Add SessionMiddleware for signed session cookies
- Add renderAuthSection navbar component (login/avatar dropdown)
- Add 401/403 interceptor in api.js for auto-redirect
- Exclude /auth/ from SPA client-side router interception
- Render auth section after translations load (fixes raw key display)
- Add custom error pages for 500s (standalone HTML, no JS deps)
- Update docker-compose.yml to pass OIDC_* env vars to web container
- Update .env.example, README, AGENTS.md, upgrading.md, i18n.md
- Add auth.* and errors.* i18n keys
- Add 200 tests (OIDC, admin, error pages)
2026-04-28 17:36:44 +01:00
Louis King f0b44d28ab Rename Docker volumes: hub_data→data, mqtt_broker_data→mqtt_data; rename db-migrate service to migrate 2026-04-16 23:47:34 +01:00
Louis King b5fc4c06a0 Rename receiver profile to observer, packet-capture service to observer, hub-dev fallback to hub
- Rename compose profile 'receiver' -> 'observer' across all docs
- Rename docker-compose service 'packet-capture' -> 'observer' and volume
  'packetcapture_data' -> 'observer_data'
- Change COMPOSE_PROJECT_NAME fallback from 'hub-dev' to 'hub' across all
  compose files, Makefile, .env, docs
- Remove legacy interface-mock service from test compose file
2026-04-16 23:12:22 +01:00
Louis King 0a37010db6 Remove bundled Prometheus/Alertmanager, fix Getting Started and docker compose examples
- Remove monitoring services from all docker compose files — monitoring is now
  user-managed infrastructure (users point their own Prometheus at /metrics)
- Remove metrics profile, prometheus/alertmanager volumes from Makefile
- Update README Getting Started: packet capture is included via --profile receiver,
  not a separate prerequisite; add remote observers guide
- Add --profile all to all docker compose command examples in README and UPGRADING.md
- Simplify UPGRADING.md backup/migration to only meshcore_hub_data
2026-04-16 22:47:07 +01:00
Louis King 1e4a75f074 Rename COLLECTOR_LETSMESH_DECODER_KEYS to COLLECTOR_CHANNEL_KEYS
Simplify the variable name to remove the legacy LetsMesh decoder prefix.
Also fix unparenthesized except tuples in web/app.py and promote the
parenthesized-exception rule to a prominent position in AGENTS.md.
2026-04-14 22:41:27 +01:00
Louis King f4648d7fe7 Split Docker Compose into base/dev/prod/traefik overrides with multi-instance support
- Split docker-compose.yml into base config + environment overrides
  - docker-compose.dev.yml: port mappings for local development
  - docker-compose.prod.yml: external proxy-net network, no exposed ports
  - docker-compose.traefik.yml: optional Traefik auto-discovery labels
- Parameterize container and volume names with COMPOSE_PROJECT_NAME
  - Default: hub-dev (containers: hub-dev-api, volumes: hub-dev_hub_data)
  - Override per instance for multi-instance deployments (hub-prod, hub-beta)
- Add Makefile with build/up/down/logs/backup/restore targets
- Add TRAEFIK_DOMAIN env var for Traefik routing configuration
- Update UPGRADING.md with volume migration instructions (rename + copy methods)
- Update README.md with multi-instance deployment and backup/restore sections
2026-04-14 20:41:26 +01:00
Louis King f2c8581b10 Remove stale COLLECTOR_INGEST_MODE reference from docker-compose comment 2026-04-13 22:50:07 +01:00
Louis King dbd68e9a9f Replace obsolete MQTT broker image with custom build, add connection retry and upgrade docs
- Add custom Dockerfile for meshcore-mqtt-broker (Node 22 Alpine, built from michaelhart/meshcore-mqtt-broker source)
- Add GitHub Actions workflow for weekly multi-arch MQTT broker image builds
- Add local build script (etc/docker/meshcore-mqtt-broker/build.sh)
- Update docker-compose.yml and test compose to use new ghcr.io image
- Add MQTT connection retry logic with exponential backoff to collector subscriber
- Create UPGRADING.md with migration guide for breaking changes
- Update README.md and AGENTS.md for accuracy (Python 3.14, removed commands, fixed MQTT defaults)
- Remove obsolete files (etc/mosquitto.conf, .agentmap.yaml, CLAUDE.md)
2026-04-13 22:27:39 +01:00
Louis King bf6b86696c Migrate from Node.js meshcore-decoder CLI to native Python meshcoredecoder library
Replace subprocess-based packet decoding with native Python meshcoredecoder>=0.3.2.
The decoder is now always enabled (no toggle), removing ENABLED/COMMAND/TIMEOUT
config vars. Adds _enrich_payload_decoded() to compensate for payload classes that
lack to_dict() overrides in the library. Removes Node.js/npm from Dockerfile and
deletes the patches/ directory. Adds parenthesized exception rule to AGENTS.md.
2026-04-12 18:03:20 +01:00
Louis King 58499c420b Replace native interface with external packet capture and rename receiver to observer
Remove the meshcore_interface component in favor of external
meshcore-packet-capture for data ingestion. Rename receiver_node_id
to observer_node_id across all models, schemas, handlers, and API
routes. Add Alembic migration for the column/table renames. Fix
frontend JS property name mismatch that prevented the Receiver column
from displaying observer data.
2026-04-12 14:07:14 +01:00
Louis King c7655b5242 Add external packet capture receiver to Docker Compose
Integrate the meshcore-packet-capture image as an alternative to the
native interface-receiver, reorganize compose profiles so the receiver
profile uses packet capture while native-receiver covers the built-in
and mock receivers, and switch the default collector ingest mode to
letsmesh_upload.
2026-04-11 18:37:32 +01:00
yellowcooln 2f40b4a730 Add LetsMesh compatibility ingest, decoder integration, and admin auth updates 2026-03-03 16:18:54 -05:00
Louis King 5a20da3afa Add Prometheus metrics endpoint, Alertmanager, and 1h stats window
Add /metrics endpoint with Prometheus gauges for nodes, messages,
advertisements, telemetry, trace paths, events, and members. Include
per-node last_seen timestamps for alerting. Add Alertmanager service
to Docker Compose metrics profile with default blackhole receiver.
Add NodeNotSeen alert rule (48h threshold). Add 1h time window to
all windowed metrics alongside existing 24h/7d/30d windows.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 23:06:07 +00:00
Louis King 5272a72647 Refactor i18n, add translation guide, and audit documentation
## i18n Refactoring

- Refactor admin translations to use common composable patterns
- Add common patterns: delete_entity_confirm, entity_added_success, move_entity_to_another_node, etc.
- Remove 18 duplicate keys from admin_members and admin_node_tags sections
- Update all admin JavaScript files to use new common patterns with dynamic entity composition
- Fix label consistency: rename first_seen to first_seen_label to match naming convention

## Translation Documentation

- Create comprehensive translation reference guide (languages.md) with 200+ documented keys
- Add translation architecture documentation to AGENTS.md with examples and best practices
- Add "Help Translate" call-to-action section in README with link to translation guide
- Add i18n feature to README features list

## Documentation Audit

- Add undocumented config options: API_KEY, WEB_LOCALE, WEB_DOMAIN to README and .env.example
- Fix outdated CLI syntax: interface --mode receiver → interface receiver
- Update database migration commands to use CLI wrapper (meshcore-hub db) instead of direct alembic
- Add static/locales/ directory to project structure section
- Add i18n configuration (WEB_LOCALE, WEB_THEME) to docker-compose.yml

## Testing

- All 438 tests passing
- All pre-commit checks passing (black, flake8, mypy)
- Added tests for new common translation patterns

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-13 22:19:37 +00:00
Louis King 706c32ae01 Add feature flags to control web dashboard page visibility
Operators can now disable specific pages (Dashboard, Nodes, Advertisements,
Messages, Map, Members, Pages) via FEATURE_* environment variables. Disabled
features are fully hidden: removed from navigation, return 404 on routes,
and excluded from sitemap/robots.txt. Dashboard auto-disables when all of
Nodes/Advertisements/Messages are off. Map auto-disables when Nodes is off.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 15:43:23 +00:00
Louis King 76717179c2 Add timezone support for web dashboard date/time display
- Add TZ environment variable support (standard Linux timezone)
- Create Jinja2 filters for timezone-aware formatting (localtime, localdate, etc.)
- Update all templates to use timezone filters with abbreviation suffix
- Pass TZ through docker-compose for web service
- Document TZ setting in README and AGENTS.md

Timestamps remain stored as UTC; only display is converted.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-09 00:34:57 +00:00
Louis King 8fbac2cbd6 Add NETWORK_CONTACT_YOUTUBE config for footer link
Add YouTube channel URL configuration option alongside existing
GitHub/Discord/Email contact links. Also crop logo SVG to content
bounds and pass YouTube env var through docker-compose.
2026-02-08 23:36:40 +00:00
Louis King b18b3c9aa4 Refactor PAGES_HOME to CONTENT_HOME and add custom logo support
- Replace PAGES_HOME with CONTENT_HOME configuration (default: ./content)
- Content directory now contains pages/ and media/ subdirectories
- Add support for custom logo at $CONTENT_HOME/media/images/logo.svg
- Custom logo replaces favicon and navbar/home logos when present
- Mount media directory as /media for serving custom assets
- Simplify default logo to generic WiFi-style radiating arcs
- Update documentation and example directory structure
- Update tests for new CONTENT_HOME structure

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-07 13:45:42 +00:00
Louis King 0f50bf4a41 Add custom markdown pages feature to web dashboard
Allows adding static content pages (About, FAQ, etc.) as markdown files
with YAML frontmatter. Pages are stored in PAGES_HOME directory (default:
./pages), automatically appear in navigation menu, and are included in
the sitemap.

- Add PageLoader class to parse markdown with frontmatter
- Add /pages/{slug} route for rendering custom pages
- Add PAGES_HOME config setting to WebSettings
- Add prose CSS styles for markdown content
- Add pages to navigation and sitemap
- Update docker-compose.yml with pages volume mount
- Add comprehensive tests for PageLoader and routes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-06 18:36:23 +00:00
Louis King 927fcd6efb Fixed README and Docker Compose 2026-02-03 22:58:58 +00:00
Louis King 96e4215c29 Fixed Compose dependencies and switched to Docker managed volume 2026-01-28 21:53:36 +00:00
Louis King ed2cf09ff3 Improve admin UI and remove unused coordinate tag type
- Replace node type badge with icon in admin tag editor
- Add Edit/Add Tags button on node detail page (when admin enabled and authenticated)
- Remove automatic seed container startup to prevent overwriting user changes
- Remove unused 'coordinate' value type from node tags (only string, number, boolean remain)
2026-01-11 12:49:34 +00:00
Claude 1457360703 Use API_ADMIN_KEY for web service to enable admin operations
The web admin interface needs write permissions to create, update,
move, and delete node tags. Changed to use API_ADMIN_KEY with
fallback to API_READ_KEY if admin key is not configured.
2026-01-11 11:55:15 +00:00
Louis King 741dd3ce84 Initial admin commit 2026-01-11 00:42:57 +00:00
Louis King 57f51c741c Fixed Member model 2025-12-08 15:13:24 +00:00
JingleManSweep f0cee14bd8 Merge pull request #48 from ipnet-mesh/feature/mqtt-tls
Added support for MQTT TLS
2025-12-07 21:16:13 +00:00
Louis King 5ff8d16bcb Added support for MQTT TLS 2025-12-07 21:15:05 +00:00
Louis King 3bc47a33bc Added data retention and node cleanup 2025-12-06 21:27:19 +00:00
Louis King 714c3cbbd2 Set sensible Docker tag label 2025-12-06 15:32:15 +00:00
Louis King dd0b4c73c5 More fixes 2025-12-06 15:10:03 +00:00
Louis King 9cd1d50bf6 Updates 2025-12-06 14:38:53 +00:00
Louis King 733342a9ec Fixed README and Compose 2025-12-06 14:21:17 +00:00
Louis King d7152a5359 Updates 2025-12-04 19:34:18 +00:00
Louis King cf2c3350cc Updates 2025-12-04 18:10:29 +00:00
Louis King fff04e4b99 Updates 2025-12-04 01:33:25 +00:00
JingleManSweep c0988338c9 Updates 2025-12-04 01:24:33 +01:00
Louis King 234e1ba39f Updates 2025-12-04 00:21:09 +00:00
Louis King 8d05189629 Updates 2025-12-04 00:06:06 +00:00