adam/meshcore-hub
1
0
Fork 0
mirror of https://github.com/ipnet-mesh/meshcore-hub.git synced 2026-08-10 10:52:48 +02:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #222 from ipnet-mesh/feat/channel-model

Browse Source 
Add database-backed channels with role-based visibility and web dashboard
This commit is contained in:
JingleManSweep
2026-06-04 14:39:55 +01:00
committed by GitHub
parent d2579dd51d 9f79ceac14
commit 187b0b7ce1
54 changed files with 4312 additions and 1153 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/skills/docs-sync/SKILL.md
-175
View File
@@ -1,175 +0,0 @@
---
name: docs-sync
description: "Audits and fixes discrepancies between project source code (Python config, Docker Compose files) and primary documentation files (README.md, AGENTS.md, docs/upgrading.md, .env.example, SCHEMAS.md). Extracts environment variables from Pydantic Settings, Click CLI options, and os.getenv calls; parses Docker Compose services, profiles, volumes, and env passthroughs; verifies feature flags, CLI commands, and file paths referenced in documentation. Produces a structured audit report and applies fixes to keep documentation accurate and up-to-date. Invoke after any config change, env var addition/removal, Docker service modification, feature flag change, or when documentation drift is suspected."
license: MIT
compatibility: opencode
metadata:
author: https://github.com/agessaman
version: "0.1.0"
domain: quality
triggers: documentation sync, docs audit, env vars, config drift, .env.example, README, AGENTS.md, docs/upgrading.md, SCHEMAS.md, docker compose docs, feature flags, documentation update, keep docs in sync, documentation accuracy
role: specialist
scope: review
output-format: report
related-skills: code-review, docs-writer
---
# Docs Sync
Documentation accuracy specialist that keeps project docs in sync with source code and Docker configuration.
## When to Use This Skill
- After adding, removing, or renaming environment variables in Python config
- After modifying Docker Compose services, profiles, volumes, or port mappings
- After adding or removing feature flags
- After adding or removing CLI commands or subcommands
- After changing Pydantic Settings defaults or types
- When documentation drift is suspected
- Before releases to ensure docs are accurate
- When AGENTS.md or README.md references stale files or commands
## Primary Documentation Files
The following files are the documentation targets. All must be kept in sync:
| File | Role |
|------|------|
| `README.md` | User-facing reference: env var tables, Docker instructions, feature list |
| `AGENTS.md` | AI agent instructions: env var list, project structure, conventions |
| `docs/upgrading.md` | Upgrade guide: deprecated vars, new vars, migration steps |
| `docs/letsmesh.md` | LetsMesh packet decoding: normalization, channel keys, message handling |
| `docs/hosting/nginx-proxy-manager.md` | Nginx Proxy Manager admin authentication setup guide |
| `docs/seeding.md` | Seed data: node tags and members YAML format, directory structure, import process |
| `docs/webhooks.md` | Webhook configuration: URLs, secrets, retries, payload format |
| `docs/content.md` | Custom content: markdown pages, media files, logos, frontmatter fields |
| `docs/i18n.md` | Translation reference: all i18n keys, variable interpolation, translation tips |
| `.env.example` | Example environment file with comments and defaults |
| `SCHEMAS.md` | Event JSON schemas and database column mappings |
## Core Workflow
1. **Extract config from Python source** — Parse all environment variables from three sources: Pydantic Settings classes in `common/config.py`, Click `envvar=` parameters in CLI modules, and direct `os.getenv()`/`os.environ` calls. Build a complete inventory with field names, defaults, types, and descriptions. See `references/config-source-guide.md`.
2. **Extract Docker configuration** — Parse all `docker-compose*.yml` files for services, compose profiles, volumes, port mappings, environment variable references (with defaults), and device mappings. Distinguish hub-consumed vars from passthrough vars (e.g., `PACKETCAPTURE_*`). See `references/docker-source-guide.md`.
3. **Extract features and commands** — Verify feature flags have corresponding UI routes and config fields. Verify CLI commands documented in README.md and AGENTS.md still exist. Verify file paths and directory structures referenced in docs actually exist.
4. **Cross-reference against documentation** — For each of the 5 primary doc files, check every env var, Docker service, feature, command, and path reference against the source-of-truth inventories from steps 1-3. See `references/documentation-checklist.md`.
5. **Verify inline comments** — Check that all comments in `.env.example` and `docker-compose*.yml` accurately describe the values they annotate. Verify default values in comments match actual defaults from source code.
6. **Produce report and apply fixes** — Generate a structured discrepancy report. For each discrepancy, apply the fix to the relevant documentation file. Summarize all changes made.
## Reference Guide
Load detailed guidance based on context:
| Topic | Reference | Load When |
|-------|-----------|-----------|
| Config Source Extraction | `references/config-source-guide.md` | Extracting env vars from Python source |
| Docker Source Extraction | `references/docker-source-guide.md` | Parsing Docker Compose files |
| Documentation Checklist | `references/documentation-checklist.md` | Cross-referencing against each doc file |
## Discrepancy Categories
| Category | Severity | Description |
|----------|----------|-------------|
| Missing env var in docs | High | Variable exists in source but not in a doc file that should list it |
| Stale env var in docs | High | Variable documented but no longer exists in source |
| Wrong default value | High | Documented default doesn't match actual default |
| Wrong type or description | Medium | Documented type or description doesn't match source |
| Missing Docker service/profile | High | Service or profile exists in compose but not documented |
| Stale Docker service/profile | High | Documented service or profile no longer exists |
| Stale file path reference | Medium | Referenced file or directory doesn't exist |
| Stale CLI command | High | Documented command no longer exists |
| Stale feature reference | Medium | Documented feature flag doesn't exist in config |
| Inaccurate inline comment | Low | Comment doesn't accurately describe the value |
| Missing comment | Low | Value lacks a descriptive comment |
| Stale doc cross-reference | Medium | Reference to removed file (e.g., PLAN.md, TASKS.md) |
## Report Template
```
# Docs Sync Audit Report
## Summary
- Total discrepancies found: N
- High: N | Medium: N | Low: N
- Files modified: N
## Environment Variables
### Missing from documentation
| Variable | Default | Missing from |
|----------|---------|-------------|
| ... | ... | README.md, .env.example |
### Stale (removed from source)
| Variable | Still in |
|----------|---------|
| ... | AGENTS.md, README.md |
### Wrong defaults
| Variable | Documented | Actual |
|----------|-----------|--------|
| ... | ... | ... |
## Docker Configuration
### Services/Profiles
[Discrepancies between compose files and docs]
### Environment Passthroughs
[Missing or stale env vars in .env.example for Docker]
## Features & Commands
### Stale features
[Documented features that don't exist]
### Missing features
[Features in source but not documented]
### Stale commands
[Documented CLI commands that don't exist]
### Stale file references
[Paths referenced in docs that don't exist]
## Inline Comments
### Inaccurate comments
| File | Line | Current | Correct |
|------|------|---------|---------|
| ... | ... | ... | ... |
## Changes Applied
[List of edits made to each file]
```
## Constraints
### MUST DO
- Treat Python source code (`common/config.py`, CLI modules, `os.getenv` calls) as the single source of truth for environment variables
- Treat `docker-compose*.yml` files as the source of truth for Docker configuration
- Check ALL primary documentation files on every audit
- Include `.env.example` comment verification
- Include `docker-compose*.yml` inline comment verification
- Verify default values match exactly (type-aware: `true` vs `"true"`, port numbers as strings vs ints)
- Flag references to removed files (PLAN.md, TASKS.md) in AGENTS.md and README.md
- Apply fixes to documentation files after reporting
- Preserve existing formatting and section structure in doc files
- For AGENTS.md env var sections, maintain the existing table format and grouping
### MUST NOT DO
- Modify Python source code or Docker Compose files (only documentation files)
- Add documentation for variables that don't exist in source
- Remove content from documentation without confirming it's stale in source
- Change the formatting style of existing documentation (match surrounding content)
- Modify docs/upgrading.md historical content (deprecated var lists, old instructions)
- Skip any of the primary documentation files
- Guess at defaults — always verify against actual source code
- Treat test compose files (`tests/e2e/`) as documentation targets (they are test fixtures)
## Knowledge Reference
Pydantic Settings (BaseSettings, env_file, field defaults), Click (envvar parameter), Docker Compose (profiles, volumes, environment, depends_on), YAML parsing, environment variable naming conventions (UPPER_SNAKE_CASE), MeshCore Hub architecture (collector, API, web, MQTT broker, packet capture observer)
.agents/skills/docs-sync/references/config-source-guide.md
-139
View File
@@ -1,139 +0,0 @@
# Config Source Guide
How to extract the complete environment variable inventory from Python source code.
## Source Files
These files contain the authoritative definition of all environment variables consumed by the MeshCore Hub application. Check them in this order:
### 1. Pydantic Settings — `src/meshcore_hub/common/config.py`
**Primary source of truth.** Contains four Settings classes that define env vars via Pydantic fields:
| Class | Inherits | Component | Env Vars |
|-------|----------|-----------|----------|
| `CommonSettings` | `BaseSettings` | All services | Base config (MQTT, logging, paths) |
| `CollectorSettings` | `CommonSettings` | Collector | Database, webhooks, retention, cleanup, channel keys |
| `APISettings` | `CommonSettings` | API server | Host, port, auth keys, metrics |
| `WebSettings` | `CommonSettings` | Web dashboard | Host, port, theme, locale, features, network info |
#### Extraction Method
Read each class and extract:
- **Field name** — the Python attribute name (e.g., `mqtt_host`)
- **Env var name** — the uppercased field name (e.g., `MQTT_HOST`), unless explicitly overridden via `Field(alias=...)` or `alias` in `model_config`
- **Default value** — the `= value` in the field declaration, or `Field(default=...)`
- **Type** — the type annotation (e.g., `str`, `int`, `bool`, `Optional[str]`, constrained types)
- **Description** — the `Field(description=...)` or docstring, if any
- **Constraints** — `Field(ge=1, le=100)`, `min_length`, `max_length`, etc.
#### Pydantic-to-Env Mapping Rules
- Field `mqtt_host` → env var `MQTT_HOST` (automatic uppercasing)
- Field `api_base_url` → env var `API_BASE_URL`
- `Optional[str]` fields with `None` default → env var is optional, no default
- `bool` fields use Pydantic's built-in coercion: `"true"`, `"1"`, `"yes"` → `True`; `"false"`, `"0"`, `"no"` → `False`
- Enum fields (e.g., `LogLevel`, `MQTTTransport`) accept their member values as strings
#### Computed Properties
Some settings have computed properties (e.g., `database_url` falls back to `sqlite:///{DATA_HOME}/collector/meshcore.db`). These should be documented as having a computed default.
#### Settings Inheritance
`CollectorSettings`, `APISettings`, and `WebSettings` all inherit from `CommonSettings`. The shared env vars (`MQTT_*`, `LOG_LEVEL`, `DATA_HOME`, etc.) should appear once under "Common Settings" in documentation, not repeated per component.
### 2. Click CLI `envvar` — CLI Entry Points
These files define Click commands that accept env var overrides via `envvar=` or `envvar=[...]`:
| File | CLI Group | Key Env Vars |
|------|-----------|--------------|
| `src/meshcore_hub/__main__.py` | Root CLI | `DATA_HOME`, `DATABASE_URL` (also sets into `os.environ` for Alembic) |
| `src/meshcore_hub/collector/cli.py` | `collector` | `DATA_HOME`, `DATABASE_URL`, `SEED_HOME` |
| `src/meshcore_hub/api/cli.py` | `api` | `DATA_HOME`, `DATABASE_URL`, `CORS_ORIGINS`, `MQTT_PREFIX`/`MQTT_TOPIC_PREFIX` |
| `src/meshcore_hub/web/cli.py` | `web` | `DATA_HOME`, `API_BASE_URL` |
#### Extraction Method
Grep for `envvar=` in each file. Each `envvar=` parameter maps a Click option to an environment variable.
#### Key Patterns
- **Single envvar:** `envvar="CORS_ORIGINS"` — maps `--cors-origins` flag to `CORS_ORIGINS` env var
- **Multiple envvars (alias):** `envvar=["MQTT_PREFIX", "MQTT_TOPIC_PREFIX"]` — first is primary, rest are backward-compat aliases
- **Setting os.environ:** `__main__.py` sets `os.environ["DATABASE_URL"]` from CLI args so Alembic can pick it up. This is a pass-through, not a new env var.
#### Edge Case: `CORS_ORIGINS`
`CORS_ORIGINS` exists only as a Click `envvar` in `api/cli.py` — it has **no** Pydantic Settings field. It should still be documented as an env var consumed by the API component, but note that it only works when launched via the CLI (not when running with `--reload` which bypasses Click).
### 3. Direct `os.getenv()` / `os.environ` Access
Some code bypasses Pydantic Settings entirely and reads env vars directly:
| File | Env Var | Default | Purpose |
|------|---------|---------|---------|
| `src/meshcore_hub/web/app.py` | `COLLECTOR_CHANNEL_KEYS` | `None` | Reads channel keys for web UI label building |
| `src/meshcore_hub/web/app.py` | `COLLECTOR_INCLUDE_TEST_CHANNEL` | `"false"` | Reads test channel flag for web UI |
| `src/meshcore_hub/common/health.py` | `HEALTH_DIR` | `/tmp/meshcore-hub` | Health status file directory |
| `src/meshcore_hub/alembic/env.py` | `DATABASE_URL` | Falls to config | Alembic migration DB URL |
| `src/meshcore_hub/alembic/env.py` | `DATA_HOME` | Falls to config | Alembic fallback for computing DB URL |
#### Extraction Method
Grep for these patterns across `src/meshcore_hub/`:
- `os.getenv(`
- `os.environ.get(`
- `os.environ[`
- `os.environ.setdefault(`
#### Key Observation
`HEALTH_DIR` is defined in `common/health.py` but has **no** Pydantic Settings field. It should be documented in AGENTS.md and .env.example if it's user-configurable, or noted as an internal variable if not.
## Classification: Hub Vars vs. Passthrough Vars
Not all env vars in `.env.example` are consumed by the Hub's Python code. Some are passed through to external containers via Docker Compose:
### Hub-Consumed Variables
Read by `meshcore-hub` Python code. Source: `config.py`, CLI modules, direct `os.getenv`.
These MUST be documented in all 5 doc files.
### Docker Passthrough Variables
Read by external containers (packet-capture, MQTT broker). Source: `docker-compose.yml` `environment:` blocks.
These MUST be in `.env.example` and `README.md` Docker sections, but NOT in AGENTS.md env var sections (since the Hub doesn't consume them).
Passthrough prefixes:
- `PACKETCAPTURE_*` — consumed by `ghcr.io/agessaman/meshcore-packet-capture`
- `MQTT_TOKEN_AUDIENCE` — consumed by `ghcr.io/ipnet-mesh/meshcore-mqtt-broker`
- `COMPOSE_PROJECT_NAME` — consumed by Docker Compose itself
- `IMAGE_VERSION`, `PACKETCAPTURE_IMAGE_VERSION` — Docker image tags
- `TRAEFIK_DOMAIN` — consumed by Traefik labels in `docker-compose.traefik.yml`
- `SERIAL_PORT` — device mapping for packet capture container
- `PROMETHEUS_PORT`, `ALERTMANAGER_PORT` — port mappings for monitoring stack
## Complete Extraction Checklist
For a full audit, run these extractions:
1. **Read `common/config.py`** — extract all fields from `CommonSettings`, `CollectorSettings`, `APISettings`, `WebSettings`
2. **Read `__main__.py`** — extract Click `envvar=` parameters and `os.environ` writes
3. **Read `collector/cli.py`** — extract Click `envvar=` parameters
4. **Read `api/cli.py`** — extract Click `envvar=` parameters (especially `CORS_ORIGINS`)
5. **Read `web/cli.py`** — extract Click `envvar=` parameters
6. **Read `web/app.py`** — extract `os.getenv()` calls (channel keys, test channel)
7. **Read `common/health.py`** — extract `os.environ.get()` calls (HEALTH_DIR)
8. **Read `alembic/env.py`** — extract `os.environ.get()` calls (DATABASE_URL, DATA_HOME)
9. **Deduplicate** — merge all sources, noting which vars appear in multiple places
10. **Classify** — tag each var as hub-consumed vs. Docker passthrough
## Three-Layer Config Precedence
When documenting defaults, understand the precedence:
1. **Click `envvar=` CLI option** — highest priority, overrides everything
2. **Pydantic Settings field default** — used when no CLI option or env var is set
3. **`.env` file** — loaded by `python-dotenv` at `__main__.py` startup, feeds into Pydantic Settings
The documented default should be the Pydantic Settings field default, since that's what applies in the general case.
.agents/skills/docs-sync/references/docker-source-guide.md
-291
View File
@@ -1,291 +0,0 @@
# Docker Source Guide
How to extract Docker configuration from all Compose files and verify documentation accuracy.
## Compose Files
| File | Purpose | Scope |
|------|---------|-------|
| `docker-compose.yml` | Base shared config | All services, profiles, volumes |
| `docker-compose.dev.yml` | Development overrides | Port mappings, dev dependencies |
| `docker-compose.prod.yml` | Production overrides | External proxy network |
| `docker-compose.traefik.yml` | Traefik auto-discovery labels | HTTP routing, TLS |
**Out of scope:** `tests/e2e/docker-compose.test.yml` is a test fixture with hardcoded values. Do NOT audit it against documentation.
## Services
### Service Inventory
Extract from `docker-compose.yml`:
| Service | Image | Profiles | Command |
|---------|-------|----------|---------|
| `mqtt` | `ghcr.io/ipnet-mesh/meshcore-mqtt-broker:latest` | `all`, `mqtt` | (default) |
| `observer` | `ghcr.io/agessaman/meshcore-packet-capture:${PACKETCAPTURE_IMAGE_VERSION}` | `all`, `observer` | (default) |
| `collector` | `ghcr.io/ipnet-mesh/meshcore-hub:${IMAGE_VERSION}` | `all`, `core` | `["collector"]` |
| `api` | `ghcr.io/ipnet-mesh/meshcore-hub:${IMAGE_VERSION}` | `all`, `core` | `["api"]` |
| `web` | `ghcr.io/ipnet-mesh/meshcore-hub:${IMAGE_VERSION}` | `all`, `core` | `["web"]` |
| `migrate` | `ghcr.io/ipnet-mesh/meshcore-hub:${IMAGE_VERSION}` | `all`, `core`, `migrate` | `["db", "upgrade"]` |
| `seed` | `ghcr.io/ipnet-mesh/meshcore-hub:${IMAGE_VERSION}` | `seed` | `["collector", "seed"]` |
### Verification Checklist
For each service, verify documentation includes:
- [ ] Service name and purpose
- [ ] Correct compose profile membership
- [ ] Dependencies (`depends_on`)
- [ ] Volumes mounted
- [ ] Key environment variables
## Compose Profiles
Extract from `docker-compose.yml` `profiles:` keys:
| Profile | Services | Use Case |
|---------|----------|----------|
| `all` | mqtt, observer, collector, api, web, migrate | Everything on one host |
| `core` | collector, api, web, migrate | Central server (no local MQTT or observer) |
| `mqtt` | mqtt | Local MQTT broker only |
| `observer` | observer | Packet capture observer only |
| `migrate` | migrate | Database migration only |
| `seed` | seed | Seed data import only |
### Verification
- [ ] Profile table in README.md matches compose file
- [ ] Profile table in AGENTS.md matches compose file
- [ ] All profile names documented
- [ ] All services within each profile listed
- [ ] Use cases described accurately
## Volumes
### Named Volumes
| Volume | Services | Purpose |
|--------|----------|---------|
| `data` | collector, api, migrate, seed | SQLite database + runtime data |
| `mqtt_data` | mqtt | MQTT broker persistence |
| `observer_data` | observer | Packet capture data |
Volume names are prefixed: `${COMPOSE_PROJECT_NAME:-hub}_data`, `${COMPOSE_PROJECT_NAME:-hub}_mqtt_data`, etc.
### Bind Mounts
| Host Path | Container Path | Service | Mode |
|-----------|---------------|---------|------|
| `${SEED_HOME:-./seed}` | `/seed` | collector | rw |
| `${SEED_HOME:-./seed}` | `/seed` | seed | ro |
| `${CONTENT_HOME:-./content}` | `/content` | web | ro |
### Verification
- [ ] All named volumes documented in README.md
- [ ] Bind mount paths match `SEED_HOME` and `CONTENT_HOME` env var defaults
- [ ] Volume naming convention (COMPOSE_PROJECT_NAME prefix) documented
## Port Mappings
### Development (docker-compose.dev.yml)
| Service | Host Port | Container Port | Variable |
|---------|-----------|----------------|----------|
| mqtt | `${MQTT_PORT:-1883}` | `${MQTT_PORT:-1883}` | `MQTT_PORT` |
| api | `${API_PORT:-8000}` | `8000` | `API_PORT` |
| web | `${WEB_PORT:-8080}` | `8080` | `WEB_PORT` |
### Production (docker-compose.prod.yml)
No ports exposed. Services connect to external `proxy-net` Docker network.
### Traefik (docker-compose.traefik.yml)
Routing via labels. API handles `/api`, `/metrics`, `/health`. Web handles everything else.
### Verification
- [ ] Port mappings in README.md match `docker-compose.dev.yml`
- [ ] Production routing description matches `docker-compose.traefik.yml`
- [ ] Container ports match `Dockerfile` exposed ports (8000, 8080)
## Environment Variables in Docker
### Per-Service Env Var Extraction
For each service's `environment:` block, extract every variable. Classify each:
**Hardcoded (container-internal):** Set to a fixed value in compose file, not configurable via `.env`. These should NOT appear in `.env.example` but SHOULD be noted in README.md service descriptions.
- Example: `DATA_HOME=/data` (collector), `API_HOST=0.0.0.0` (api), `CONTENT_HOME=/content` (web)
**Variable substitution (user-configurable):** Use `${VAR:-default}` syntax. These MUST appear in `.env.example` with the same default.
- Example: `MQTT_HOST=${MQTT_HOST:-mqtt}`, `LOG_LEVEL=${LOG_LEVEL:-INFO}`
**Passthrough (no default):** Reference `${VAR}` without a default. These MUST appear in `.env.example` (typically commented out or with empty value).
- Example: `API_READ_KEY`, `API_ADMIN_KEY`, `WEBHOOK_ADVERTISEMENT_URL`
### Collector Service Env Vars
Complete list from `docker-compose.yml` collector `environment:` block:
| Variable | Default in Compose | Category |
|----------|--------------------|----------|
| `LOG_LEVEL` | `INFO` | Common |
| `MQTT_HOST` | `mqtt` | MQTT |
| `MQTT_PORT` | `1883` | MQTT |
| `MQTT_USERNAME` | (empty) | MQTT |
| `MQTT_PASSWORD` | (empty) | MQTT |
| `MQTT_PREFIX` | `meshcore` | MQTT |
| `MQTT_TLS` | `false` | MQTT |
| `MQTT_TRANSPORT` | `websockets` | MQTT |
| `MQTT_WS_PATH` | `/` | MQTT |
| `DATA_HOME` | `/data` (hardcoded) | Path |
| `SEED_HOME` | `/seed` (hardcoded) | Path |
| `COLLECTOR_CHANNEL_KEYS` | (empty) | Collector |
| `COLLECTOR_INCLUDE_TEST_CHANNEL` | `false` | Collector |
| `WEBHOOK_ADVERTISEMENT_URL` | (passthrough) | Webhook |
| `WEBHOOK_ADVERTISEMENT_SECRET` | (passthrough) | Webhook |
| `WEBHOOK_MESSAGE_URL` | (passthrough) | Webhook |
| `WEBHOOK_MESSAGE_SECRET` | (passthrough) | Webhook |
| `WEBHOOK_CHANNEL_MESSAGE_URL` | (passthrough) | Webhook |
| `WEBHOOK_CHANNEL_MESSAGE_SECRET` | (passthrough) | Webhook |
| `WEBHOOK_DIRECT_MESSAGE_URL` | (passthrough) | Webhook |
| `WEBHOOK_DIRECT_MESSAGE_SECRET` | (passthrough) | Webhook |
| `WEBHOOK_TIMEOUT` | `10.0` | Webhook |
| `WEBHOOK_MAX_RETRIES` | `3` | Webhook |
| `WEBHOOK_RETRY_BACKOFF` | `2.0` | Webhook |
| `DATA_RETENTION_ENABLED` | `true` | Retention |
| `DATA_RETENTION_DAYS` | `30` | Retention |
| `DATA_RETENTION_INTERVAL_HOURS` | `24` | Retention |
| `NODE_CLEANUP_ENABLED` | `true` | Node Cleanup |
| `NODE_CLEANUP_DAYS` | `30` | Node Cleanup |
### API Service Env Vars
| Variable | Default in Compose | Category |
|----------|--------------------|----------|
| `LOG_LEVEL` | `INFO` | Common |
| `MQTT_HOST` | `mqtt` | MQTT |
| `MQTT_PORT` | `1883` | MQTT |
| `MQTT_USERNAME` | (empty) | MQTT |
| `MQTT_PASSWORD` | (empty) | MQTT |
| `MQTT_PREFIX` | `meshcore` | MQTT |
| `MQTT_TLS` | `false` | MQTT |
| `MQTT_TRANSPORT` | `websockets` | MQTT |
| `MQTT_WS_PATH` | `/` | MQTT |
| `DATA_HOME` | `/data` (hardcoded) | Path |
| `API_HOST` | `0.0.0.0` (hardcoded) | API |
| `API_PORT` | `8000` (hardcoded) | API |
| `API_READ_KEY` | (passthrough) | Auth |
| `API_ADMIN_KEY` | (passthrough) | Auth |
| `METRICS_ENABLED` | `true` | Metrics |
| `METRICS_CACHE_TTL` | `60` | Metrics |
### Web Service Env Vars
| Variable | Default in Compose | Category |
|----------|--------------------|----------|
| `LOG_LEVEL` | `INFO` | Common |
| `API_BASE_URL` | `http://api:8000` (hardcoded) | API |
| `API_ADMIN_KEY` / `API_READ_KEY` | (cascading passthrough) | Auth |
| `WEB_HOST` | `0.0.0.0` (hardcoded) | Web |
| `WEB_PORT` | `8080` (hardcoded) | Web |
| `WEB_THEME` | `dark` | Theme |
| `WEB_LOCALE` | `en` | Locale |
| `WEB_DATETIME_LOCALE` | `en-US` | Locale |
| `OIDC_ENABLED` | `false` | Auth |
| `OIDC_CLIENT_ID` | (empty) | Auth |
| `OIDC_CLIENT_SECRET` | (empty) | Auth |
| `OIDC_DISCOVERY_URL` | (empty) | Auth |
| `OIDC_REDIRECT_URI` | (empty) | Auth |
| `OIDC_POST_LOGOUT_REDIRECT_URI` | (empty) | Auth |
| `OIDC_SCOPES` | `openid email profile` | Auth |
| `OIDC_ROLES_CLAIM` | `roles` | Auth |
| `OIDC_ROLE_ADMIN` | `admin` | Auth |
| `OIDC_ROLE_OPERATOR` | `operator` | Auth |
| `OIDC_ROLE_MEMBER` | `member` | Auth |
| `OIDC_SESSION_SECRET` | (empty) | Auth |
| `OIDC_SESSION_MAX_AGE` | `86400` | Auth |
| `OIDC_COOKIE_SECURE` | `false` | Auth |
| `WEB_AUTO_REFRESH_SECONDS` | `30` | Display |
| `WEB_DEBUG` | `false` | Display |
| `NETWORK_DOMAIN` | (empty) | Network |
| `NETWORK_NAME` | `MeshCore Network` | Network |
| `NETWORK_CITY` | (empty) | Network |
| `NETWORK_COUNTRY` | (empty) | Network |
| `NETWORK_RADIO_CONFIG` | (empty) | Network |
| `NETWORK_CONTACT_EMAIL` | (empty) | Network |
| `NETWORK_CONTACT_DISCORD` | (empty) | Network |
| `NETWORK_CONTACT_GITHUB` | (empty) | Network |
| `NETWORK_CONTACT_YOUTUBE` | (empty) | Network |
| `NETWORK_WELCOME_TEXT` | (empty) | Network |
| `CONTENT_HOME` | `/content` (hardcoded) | Path |
| `TZ` | `UTC` | Display |
| `COLLECTOR_CHANNEL_KEYS` | (empty) | Display |
| `COLLECTOR_INCLUDE_TEST_CHANNEL` | `false` | Display |
| `FEATURE_DASHBOARD` | `true` | Feature |
| `FEATURE_NODES` | `true` | Feature |
| `FEATURE_ADVERTISEMENTS` | `true` | Feature |
| `FEATURE_MESSAGES` | `true` | Feature |
| `FEATURE_MAP` | `true` | Feature |
| `FEATURE_MEMBERS` | `true` | Feature |
| `FEATURE_PAGES` | `true` | Feature |
### Observer (Packet Capture) Env Vars
These are ALL passthrough vars consumed by the external packet capture image. None are read by Hub Python code.
Grouped by function:
- **Connection:** `SERIAL_PORT`, `PACKETCAPTURE_TIMEOUT`, `PACKETCAPTURE_MAX_CONNECTION_RETRIES`, `PACKETCAPTURE_CONNECTION_RETRY_DELAY`, `PACKETCAPTURE_HEALTH_CHECK_INTERVAL`
- **Identity:** `PACKETCAPTURE_IATA`, `PACKETCAPTURE_ORIGIN`
- **Behavior:** `PACKETCAPTURE_ADVERT_INTERVAL_HOURS`, `PACKETCAPTURE_RF_DATA_TIMEOUT`
- **MQTT Broker 1 (Let's Mesh US):** `PACKETCAPTURE_MQTT1_ENABLED`, `PACKETCAPTURE_MQTT1_SERVER`, `PACKETCAPTURE_MQTT1_PORT`, `PACKETCAPTURE_MQTT1_USE_TLS`, `PACKETCAPTURE_MQTT1_USE_AUTH_TOKEN`, `PACKETCAPTURE_MQTT1_TOKEN_AUDIENCE`, `PACKETCAPTURE_MQTT1_KEEPALIVE`
- **MQTT Broker 2 (Let's Mesh EU):** `PACKETCAPTURE_MQTT2_ENABLED`, `PACKETCAPTURE_MQTT2_SERVER`, `PACKETCAPTURE_MQTT2_PORT`, `PACKETCAPTURE_MQTT2_USE_TLS`, `PACKETCAPTURE_MQTT2_USE_AUTH_TOKEN`, `PACKETCAPTURE_MQTT2_TOKEN_AUDIENCE`, `PACKETCAPTURE_MQTT2_KEEPALIVE`
- **MQTT Broker 3 (Local):** `PACKETCAPTURE_MQTT3_ENABLED`, `PACKETCAPTURE_MQTT3_KEEPALIVE`
- **MQTT Reconnection:** `PACKETCAPTURE_MAX_MQTT_RETRIES`, `PACKETCAPTURE_MQTT_RETRY_DELAY`, `PACKETCAPTURE_EXIT_ON_RECONNECT_FAIL`
Note: Broker 3 is wired to hub's MQTT vars: `MQTT_HOST`, `MQTT_PORT`, `MQTT_USERNAME`, `MQTT_PASSWORD`, `MQTT_TLS`, `MQTT_TOKEN_AUDIENCE`.
## Infrastructure-Only Variables
These appear in `.env.example` and compose files but are NOT consumed by any Python code:
| Variable | Consumer | Default |
|----------|----------|---------|
| `COMPOSE_PROJECT_NAME` | Docker Compose | `hub` |
| `IMAGE_VERSION` | Docker Compose | `latest` |
| `PACKETCAPTURE_IMAGE_VERSION` | Docker Compose | `latest` |
| `TRAEFIK_DOMAIN` | Traefik labels | (required when using traefik compose) |
| `SERIAL_PORT` | Observer container + device mapping | `/dev/ttyUSB0` |
| `MQTT_TOKEN_AUDIENCE` | MQTT broker container | `mqtt.localhost` |
| `PROMETHEUS_PORT` | Docker port mapping | `9090` |
| `ALERTMANAGER_PORT` | Docker port mapping | `9093` |
These MUST be in `.env.example` and README.md but NOT in AGENTS.md "Environment Variables" section (since AGENTS.md documents Hub-consumed vars).
## Dockerfile Verification
The `Dockerfile` sets ENV defaults that should match compose file defaults:
| Dockerfile ENV | Value | Must Match |
|----------------|-------|------------|
| `LOG_LEVEL` | `INFO` | Compose `LOG_LEVEL` default |
| `MQTT_HOST` | `mqtt` | Compose `MQTT_HOST` default |
| `MQTT_PORT` | `1883` | Compose `MQTT_PORT` default |
| `MQTT_PREFIX` | `meshcore` | Compose `MQTT_PREFIX` default |
| `DATA_HOME` | `/data` | Compose hardcoded `DATA_HOME` |
| `API_HOST` | `0.0.0.0` | Compose hardcoded `API_HOST` |
| `API_PORT` | `8000` | Compose hardcoded `API_PORT` |
| `WEB_HOST` | `0.0.0.0` | Compose hardcoded `WEB_HOST` |
| `WEB_PORT` | `8080` | Compose hardcoded `WEB_PORT` |
| `API_BASE_URL` | `http://api:8000` | Compose hardcoded `API_BASE_URL` |
## Inline Comment Verification
For `docker-compose.yml` and `.env.example`, verify every `# comment` accurately describes the value it annotates. Check:
1. **Default values in comments** match actual `${VAR:-default}` values
2. **Descriptions** accurately describe what the variable does
3. **Section headers** correctly group related variables
4. **References** to other files or sections are still valid (e.g., "see README.md" references)
5. **Examples** use current, valid values (not outdated formats)
.agents/skills/docs-sync/references/documentation-checklist.md
-338
View File
@@ -1,338 +0,0 @@
# Documentation Checklist
Per-file verification checklists for each of the 5 primary documentation files.
## 1. README.md
### Environment Variable Tables
README.md contains env var tables grouped by component. For each table:
- [ ] **Common Settings table** — every var from `CommonSettings` is listed with correct default
- [ ] **Collector Settings table** — every collector-specific var from `CollectorSettings` is listed
- [ ] **Webhook reference** — webhook config links to `docs/webhooks.md`
- [ ] **Data Retention table** — all retention and node cleanup vars listed
- [ ] **API Settings table** — all API-specific vars from `APISettings` listed
- [ ] **Web Dashboard Settings table** — all web-specific vars from `WebSettings` listed
- [ ] **Feature Flags table** — all 7 feature flags listed
- [ ] **Network Info vars** — all `NETWORK_*` vars listed
- [ ] **Contact Info vars** — all `NETWORK_CONTACT_*` vars listed
For each variable in each table:
- [ ] Default value matches Pydantic Settings field default exactly
- [ ] Description is accurate and matches field purpose
- [ ] No stale/removed variables remain
- [ ] No variables are duplicated across tables
### Docker Section
- [ ] Compose files listed: base, dev, prod, traefik (4 files)
- [ ] Service profiles table matches `docker-compose.yml` exactly
- [ ] All 7 services documented: mqtt, observer, collector, api, web, migrate, seed
- [ ] Port mappings match `docker-compose.dev.yml`
- [ ] Volume names documented with `COMPOSE_PROJECT_NAME` prefix convention
- [ ] Bind mounts documented (SEED_HOME, CONTENT_HOME)
- [ ] Traefik integration instructions reference `TRAEFIK_DOMAIN`
- [ ] Reverse Proxy section documents proxy setup and production network
- [ ] Quick start examples use correct current commands
### Features Section
- [ ] Each listed feature corresponds to actual code
- [ ] Feature flag dependency rules documented correctly (Dashboard auto-disables when Nodes/Ads/Messages all off; Map auto-disables when Nodes off)
### CLI Commands
- [ ] `meshcore-hub collector` — verify still exists
- [ ] `meshcore-hub api` — verify still exists
- [ ] `meshcore-hub web` — verify still exists
- [ ] `meshcore-hub db upgrade` — verify still exists
- [ ] `meshcore-hub collector seed` — verify still exists
- [ ] `meshcore-hub collector cleanup` — verify still exists
- [ ] All example Docker Compose commands use valid profile names
### File Paths
- [ ] `src/meshcore_hub/` structure matches actual layout
- [ ] Seed data directory structure (`node_tags.yaml`) documented in `docs/seeding.md`
- [ ] Custom content directory structure (`pages/`, `media/`) documented in `docs/content.md`
- [ ] Webhook configuration documented in `docs/webhooks.md`
- [ ] Translation files location (`src/meshcore_hub/web/static/locales/`) documented
- [ ] Translation reference guide (`docs/i18n.md`) linked from README and AGENTS.md
- [ ] No references to removed files (PLAN.md, TASKS.md)
## 2. AGENTS.md
### Environment Variables Section
AGENTS.md has a "Key variables" subsection under "Environment Variables". Verify:
- [ ] All hub-consumed env vars listed (from Pydantic Settings + Click + os.getenv)
- [ ] Passthrough vars (`PACKETCAPTURE_*`, `COMPOSE_PROJECT_NAME`, etc.) NOT listed as Hub vars
- [ ] Defaults match Pydantic Settings defaults
- [ ] Descriptions match field purpose
- [ ] Grouping matches the Settings class hierarchy (Common, Collector, API, Web)
### Project Structure
- [ ] Directory tree matches actual layout
- [ ] All listed files exist
- [ ] No removed files listed (e.g., PLAN.md, TASKS.md)
- [ ] All new directories/files included
### Features Documented
- [ ] Feature flags listed in "Environment Variables" match `WebSettings` fields
- [ ] Dependency rules documented (Dashboard/Map auto-disable logic)
- [ ] Admin auth mechanism documented accurately
### Code Examples
- [ ] Import paths use current module structure
- [ ] Class names match current models
- [ ] CLI commands match current Click definitions
- [ ] Async patterns match current codebase conventions
### Cross-References
- [ ] References to `PLAN.md` — should be removed (file deleted)
- [ ] References to `TASKS.md` — should be removed (file deleted)
- [ ] References to `SCHEMAS.md` — should remain (file exists)
- [ ] References to `docs/upgrading.md` — should remain (file exists)
- [ ] References to `docs/letsmesh.md` — should remain (file exists)
- [ ] References to `docs/webhooks.md` — should remain (file exists)
- [ ] References to `docs/content.md` — should remain (file exists)
## 3. docs/upgrading.md
### Deprecated Variables
docs/upgrading.md lists variables to remove during upgrade. Verify:
- [ ] Each deprecated var truly no longer exists in `config.py` or any CLI module
- [ ] Removal instructions are clear
- [ ] No currently-active vars are listed as deprecated
### New Variables
- [ ] Each new var listed actually exists in current `config.py`
- [ ] Defaults for new vars match Pydantic Settings defaults
- [ ] Migration instructions for adding new vars are correct
### Renamed Variables
- [ ] Old name no longer exists anywhere in codebase
- [ ] New name matches current `config.py` field name
### Docker Migration
- [ ] Volume rename instructions accurate
- [ ] Old service names truly removed from compose files
- [ ] New compose file structure documented correctly
- [ ] Migration commands still valid for current Docker versions
### Database Migration
- [ ] Column renames documented accurately (e.g., `receiver_node_id` → `observer_node_id`)
- [ ] Table renames documented accurately (e.g., `event_receivers` → `event_observers`)
## 3b. docs/letsmesh.md
### Packet Decoding Documentation
docs/letsmesh.md documents the LetsMesh packet normalization and decoding behavior. Verify:
- [ ] MQTT subscription topics match `subscriber.py` topic patterns
- [ ] Payload type mappings match `letsmesh_decoder.py` and `letsmesh_normalizer.py` logic
- [ ] Channel key handling documented matches `COLLECTOR_CHANNEL_KEYS` config behavior
- [ ] Known channel indexes (`17 -> Public`, `217 -> #test`) match built-in defaults in decoder
- [ ] Message normalization rules match collector handler implementations
- [ ] GPS/location update behavior documented matches advertisement handler logic
- [ ] No stale decoder behavior documented (e.g., references to Node.js decoder)
## 3c. docs/seeding.md
### Seed Data Documentation
docs/seeding.md documents the seed data format and import process for node tags and network members. Verify:
- [ ] Running the Seed Process section references correct Docker Compose command (`--profile seed`)
- [ ] Seed files listed match `tag_import.py` and `member_import.py` expected filenames
- [ ] Directory structure shows `SEED_HOME` and `DATA_HOME` correctly
- [ ] Node Tags YAML format matches `tag_import.py` parsing logic
- [ ] Tag value types documented match supported types in `tag_import.py`
- [ ] Members YAML format matches `member_import.py` parsing logic
- [ ] Member field table fields match `MemberCreate` Pydantic schema
- [ ] Example seed files referenced in `example/seed/` exist
## 3d. docs/i18n.md
### Translation Reference Guide
docs/i18n.md is a comprehensive reference for translators. Verify:
- [ ] Translation file location path (`src/meshcore_hub/web/static/locales/`) is correct
- [ ] Each translation section key matches a top-level key in `en.json`
- [ ] Entity keys (`entities.*`) match current `en.json` values
- [ ] Common pattern keys (`common.*`) match current `en.json` values
- [ ] Composite pattern examples produce correct output with current entity names
- [ ] Variable interpolation syntax (`{{variable}}`) matches actual usage in `en.json`
- [ ] Admin section keys match current admin page translations
- [ ] No stale translation keys documented (removed from `en.json`)
- [ ] Translation tips are accurate for current i18n system
## 3e. docs/webhooks.md
### Webhook Configuration Documentation
docs/webhooks.md documents the webhook configuration, URL routing logic, and payload format. Verify:
- [ ] All 11 webhook environment variables listed with correct defaults and descriptions
- [ ] URL routing rules documented correctly (MESSAGE_URL as default, CHANNEL/DIRECT overrides)
- [ ] Secret header mechanism documented (`X-Webhook-Secret`)
- [ ] Retry behavior documented (max retries, exponential backoff)
- [ ] Payload format JSON example matches `webhook.py` dispatcher output
- [ ] Event types listed match actual webhook event types
- [ ] Configuration examples use correct env var names
## 3f. docs/content.md
### Custom Content Documentation
docs/content.md documents the custom content system for the web dashboard. Verify:
- [ ] Directory structure shows `pages/` and `media/` subdirectories correctly
- [ ] Custom logo options (`logo.svg`, `logo-invert.svg`) documented with behavior per theme
- [ ] Frontmatter field table matches `pages.py` parser expectations
- [ ] Default values for frontmatter fields are correct (title, slug, menu_order)
- [ ] Setup examples create valid markdown pages
- [ ] Docker volume mounting instructions match `docker-compose.yml` bind mount config
- [ ] `CONTENT_HOME` default value documented correctly
## 4. .env.example
### Section Structure
Verify sections exist and are correctly ordered:
1. [ ] Quick Start header with observer node example
2. [ ] Common Settings (`COMPOSE_PROJECT_NAME`, `TRAEFIK_DOMAIN`, `IMAGE_VERSION`, `LOG_LEVEL`, `DATA_HOME`, `SEED_HOME`)
3. [ ] MQTT Settings (`MQTT_HOST`, `MQTT_PORT`, `MQTT_USERNAME`, `MQTT_PASSWORD`, `MQTT_PREFIX`, `MQTT_TLS`, `MQTT_TRANSPORT`, `MQTT_WS_PATH`, `MQTT_TOKEN_AUDIENCE`)
4. [ ] Packet Capture Settings (all `PACKETCAPTURE_*` vars + `SERIAL_PORT`)
5. [ ] Collector Settings (`COLLECTOR_CHANNEL_KEYS`, `COLLECTOR_INCLUDE_TEST_CHANNEL`, webhooks, retention, cleanup)
6. [ ] API Settings (`API_PORT`, `API_READ_KEY`, `API_ADMIN_KEY`, metrics)
7. [ ] Web Dashboard Settings (`WEB_PORT`, `API_BASE_URL`, `API_KEY`, theme, locale, auto-refresh, admin, TZ, content home, network info, feature flags, contact info)
### Per-Variable Checks
For every variable in `.env.example`:
- [ ] Variable name matches the env var name exactly (UPPER_SNAKE_CASE)
- [ ] Default value matches Pydantic Settings default OR compose file default (for passthrough vars)
- [ ] Comment above the variable accurately describes its purpose
- [ ] Comment includes valid range/options where applicable (e.g., "DEBUG, INFO, WARNING, ERROR, CRITICAL")
- [ ] Optional vars are commented out (`# VAR=`) with a note about the default
- [ ] Required vars have an uncommented assignment with the default value
- [ ] No duplicate variable entries
- [ ] No removed/stale variables
### Comment Accuracy
- [ ] `COMPOSE_PROJECT_NAME` comment mentions container/volume prefix
- [ ] `MQTT_TRANSPORT` comment states WebSocket is required by MeshCore broker
- [ ] `MQTT_WS_PATH` comment notes default `/` vs production `/mqtt`
- [ ] `MQTT_TOKEN_AUDIENCE` comment explains it must match broker config
- [ ] `PACKETCAPTURE_*` comments reference the external packet capture image
- [ ] `COLLECTOR_CHANNEL_KEYS` comment explains label=hex format
- [ ] `WEB_*` comments reference web dashboard behavior
- [ ] `FEATURE_*` comments explain what each flag controls
- [ ] `NETWORK_*` comments explain where values appear in UI
- [ ] `PROMETHEUS_PORT` / `ALERTMANAGER_PORT` comments reference the monitoring profile
### Missing Variables Check
- [ ] Every hub-consumed var from `config.py` appears (or is commented out) in `.env.example`
- [ ] Every passthrough var from compose files appears in `.env.example`
- [ ] No extra variables that don't exist in any compose file or Python source
## 5. SCHEMAS.md
### Event Schema Verification
SCHEMAS.md documents the JSON schemas for events stored in the database. Verify:
- [ ] Each event type documented has a corresponding handler in `src/meshcore_hub/collector/handlers/`
- [ ] Each documented field exists in the corresponding Pydantic schema or SQLAlchemy model
- [ ] Field types match current code (e.g., `str`, `int`, `Optional[str]`)
- [ ] Required vs optional fields match current schema definitions
- [ ] Database column names match current SQLAlchemy model definitions
### Database Table Verification
For each table documented in SCHEMAS.md:
- [ ] Table name matches `__tablename__` in the corresponding SQLAlchemy model
- [ ] Column names match `mapped_column()` field names
- [ ] Column types match (String length, DateTime, Text, Integer, etc.)
- [ ] Foreign key relationships documented correctly
- [ ] Indexes and unique constraints documented correctly
- [ ] New columns from recent migrations are included
- [ ] Removed columns are not documented
### MQTT Topic Schema
- [ ] Topic structure documented matches what the collector subscribes to
- [ ] Upload topic format (`<prefix>/<IATA>/<public_key>/<feed_type>`) is correct
- [ ] Subscriber subscriptions listed match `subscriber.py` topic patterns
## Cross-File Consistency Checks
These checks ensure all primary documentation files are consistent with each other:
### Env Var Coverage Matrix
Every hub-consumed env var should appear in:
| File | Required | Format |
|------|----------|--------|
| `config.py` | Yes (source of truth) | Pydantic field |
| `README.md` | Yes | Table row with default + description |
| `AGENTS.md` | Yes | Mentioned in env vars section |
| `.env.example` | Yes | Entry with default + comment |
| `docs/upgrading.md` | Only if new/renamed/deprecated | Migration instruction |
Every passthrough env var should appear in:
| File | Required |
|------|----------|
| `docker-compose.yml` | Yes (source of truth) |
| `README.md` | Yes |
| `.env.example` | Yes |
| `AGENTS.md` | No |
### Default Value Consistency
- [ ] Same default in README.md tables, .env.example values, and config.py
- [ ] No contradictions between files
- [ ] Type representations consistent (e.g., don't mix `true`/`True`/`1` for booleans)
### Stale Reference Sweep
Check all primary documentation files for references to removed items:
- [ ] `PLAN.md` — removed, references should be deleted from AGENTS.md
- [ ] `TASKS.md` — removed, references should be deleted from AGENTS.md
- [ ] Old compose profiles (`receiver`, `sender`, `mock`) — should only exist in docs/upgrading.md as deprecated
- [ ] Old service names (`interface-receiver`, `interface-sender`) — should only exist in docs/upgrading.md as deprecated
- [ ] Old env var names (`COLLECTOR_LETSMESH_DECODER_*`, `SERIAL_BAUD`, etc.) — should only exist in docs/upgrading.md as deprecated
## Applying Fixes
When discrepancies are found:
1. **Identify the source of truth** — config.py for env vars, docker-compose.yml for Docker config
2. **Determine scope** — which files need updating
3. **Make minimal edits** — only change what's wrong, preserve surrounding formatting
4. **Match existing style** — tables use same column order, comments use same format, sections use same headers
5. **Preserve historical content** — docs/upgrading.md deprecated var lists are historical, do not remove them
6. **Verify after editing** — re-read the changed section to confirm accuracy
.env.example
+3 -12
View File
@@ -194,18 +194,8 @@ PACKETCAPTURE_EXIT_ON_RECONNECT_FAIL=true
# =============================================================================
# The collector subscribes to MQTT events and stores them in the database
# LetsMesh decoder support
# The native Python decoder is always enabled.
# Optional: channel secret keys (comma or space separated) used to decrypt GroupText
# packets. This supports unlimited keys.
# Note: Public + #test keys are built into the collector code by default.
# To show friendly channel names in the web feed, use label=hex (example: bot=ABCDEF...).
# Without keys, encrypted packets cannot be shown as plaintext.
# COLLECTOR_CHANNEL_KEYS=
# Include built-in 'test' channel messages (channel_idx 217)
# Default: false (test channel messages are discarded)
# COLLECTOR_INCLUDE_TEST_CHANNEL=false
# Refresh interval for reloading channel keys from the database (seconds).
# CHANNEL_REFRESH_INTERVAL_SECONDS=300
# -------------------
# Webhook Settings
@@ -457,6 +447,7 @@ NETWORK_ANNOUNCEMENT=
# FEATURE_MAP=true
# FEATURE_MEMBERS=true
# FEATURE_PAGES=true
# FEATURE_CHANNELS=true
# -------------------
# Contact Information
AGENTS.md
+7 -5
View File
@@ -264,11 +264,13 @@ meshcore-hub/
│ │ ├── hash_utils.py # Hash utility functions
│ │ ├── models/ # SQLAlchemy models
│ │ │ ├── node.py # Node model
│ │ │ ├── channel.py # Channel model (encryption keys)
│ │ │ ├── user_profile.py # User profile model (OIDC users)
│ │ │ ├── user_profile_node.py # User-node adoption join table
│ │ │ └── ...
│ │ └── schemas/ # Pydantic schemas
│ │ ├── user_profiles.py # User profile API schemas
│ │ ├── channels.py # Channel API schemas
│ │ └── ...
│ ├── collector/
│ │ ├── cli.py # Collector CLI with seed commands
@@ -287,8 +289,9 @@ meshcore-hub/
│ │ ├── metrics.py # Prometheus metrics endpoint
│ │ └── routes/ # API routes
│ │ ├── user_profiles.py # User profile endpoints (GET/PUT profile)
│ │ ├── adoptions.py # Node adoption endpoints (POST adopt, DELETE release)
│ │ └── ...
│ │ ├── adoptions.py # Node adoption endpoints (POST adopt, DELETE release)
│ │ ├── channels.py # Channel CRUD endpoints (GET/POST/PUT/DELETE channels)
│ │ └── ...
│ └── web/
│ ├── cli.py
│ ├── app.py # FastAPI app
@@ -635,8 +638,7 @@ Key variables:
- `MQTT_TRANSPORT` - MQTT transport protocol (default: `websockets`)
- `MQTT_WS_PATH` - WebSocket path (default: `/`)
- `MQTT_TLS` - Enable TLS/SSL for MQTT (default: `false`, set `true` for `wss://`)
- `COLLECTOR_CHANNEL_KEYS` - Additional decoder channel keys for decrypting GroupText packets
- `COLLECTOR_INCLUDE_TEST_CHANNEL` - Include built-in 'test' channel messages (default: `false`)
- `CHANNEL_REFRESH_INTERVAL_SECONDS` - Seconds between channel key refresh from database (default: `300`, min: `10`)
- `API_HOST` - API server bind address (default: `0.0.0.0`)
- `API_PORT` - API server port (default: `8000`)
- `API_READ_KEY`, `API_ADMIN_KEY` - API authentication keys
@@ -668,7 +670,7 @@ Key variables:
- `WEB_AUTO_REFRESH_SECONDS` - Auto-refresh interval in seconds for list pages (default: `30`, `0` to disable)
- `WEB_DEBUG` - Enable debug mode in the web dashboard (default: `false`)
- `TZ` - Timezone for web dashboard date/time display (default: `UTC`, e.g., `America/New_York`, `Europe/London`)
- `FEATURE_DASHBOARD`, `FEATURE_NODES`, `FEATURE_ADVERTISEMENTS`, `FEATURE_MESSAGES`, `FEATURE_MAP`, `FEATURE_MEMBERS`, `FEATURE_PAGES` - Feature flags to enable/disable specific web dashboard pages (default: all `true`). Dependencies: Dashboard auto-disables when all of Nodes/Advertisements/Messages are disabled. Map auto-disables when Nodes is disabled.
- `FEATURE_DASHBOARD`, `FEATURE_NODES`, `FEATURE_ADVERTISEMENTS`, `FEATURE_MESSAGES`, `FEATURE_MAP`, `FEATURE_MEMBERS`, `FEATURE_PAGES`, `FEATURE_CHANNELS` - Feature flags to enable/disable specific web dashboard pages (default: all `true`). Dependencies: Dashboard auto-disables when all of Nodes/Advertisements/Messages are disabled. Map auto-disables when Nodes is disabled.
- `NETWORK_DOMAIN` - Network domain name (default: none)
- `NETWORK_NAME` - Network display name (default: `MeshCore Network`)
- `NETWORK_CITY` - Network city location (default: none)
README.md
+3 -4
View File
@@ -330,10 +330,9 @@ All components are configured via environment variables. Create a `.env` file or
### Collector Settings
| Variable | Default | Description |
| -------------------------------- | -------- | -------------------------------------------------------------------- |
| `COLLECTOR_CHANNEL_KEYS` | _(none)_ | Additional decoder channel keys (`label=hex`, `label:hex`, or `hex`) |
| `COLLECTOR_INCLUDE_TEST_CHANNEL` | `false` | Include built-in 'test' channel messages |
| Variable | Default | Description |
| ---------------------------------- | ------- | -------------------------------------------------------- |
| `CHANNEL_REFRESH_INTERVAL_SECONDS` | `300` | Seconds between channel key refresh from database (min 10) |
#### LetsMesh Packet Decoding
SCHEMAS.md
+2 -2
View File
@@ -154,7 +154,7 @@ Group/broadcast messages on specific channels.
**Field Descriptions**:
- `channel_idx`: Channel number (0-255) when available
- `channel_name`: Channel display label (e.g., `"Public"`, `"#test"`) when available
- `channel_name`: Channel display label (e.g., `"Public"`, `"Community"`, `"#test"`) when available
- `pubkey_prefix`: First 12 characters of the source public key prefix, used for message identification when available
- `path_len`: Number of hops message traveled
- `txt_type`: Message type indicator (0=plain, 2=signed, etc.)
@@ -186,7 +186,7 @@ Group/broadcast messages on specific channels.
- In LetsMesh upload compatibility mode, packet type `5` is normalized to `CHANNEL_MSG_RECV` and packet types `1`, `2`, and `7` are normalized to `CONTACT_MSG_RECV` when decryptable text is available.
- LetsMesh packets without decryptable message text are treated as informational `letsmesh_packet` events instead of message events.
- For UI labels, known channel indexes are mapped (`17 -> Public`, `217 -> #test`) and preferred over ambiguous/stale channel-name hints.
- Additional channel labels can be provided through `COLLECTOR_CHANNEL_KEYS` using `label=hex` entries.
- Additional channel labels are loaded from the `channels` database table via the collector's periodic refresh.
- When decoder output includes a human sender (`payload.decoded.decrypted.sender`), message text is normalized to `Name: Message`; sender identity remains unknown when only hash/prefix metadata is available.
**Compatibility ingest note (advertisements)**:
alembic/versions/20260519_2125_82dff87d6576_add_channels_table.py
+58
View File
@@ -0,0 +1,58 @@
"""add channels table
Revision ID: 82dff87d6576
Revises: 20260515_1920
Create Date: 2026-05-19 21:25:58.828179+00:00
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = "82dff87d6576"
down_revision: Union[str, None] = "20260515_1920"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
op.create_table(
"channels",
sa.Column("name", sa.String(length=100), nullable=False),
sa.Column("key_hex", sa.String(length=64), nullable=False),
sa.Column("channel_hash", sa.String(length=2), nullable=False),
sa.Column("visibility", sa.String(length=20), nullable=False),
sa.Column("enabled", sa.Boolean(), nullable=False),
sa.Column("id", sa.String(), nullable=False),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("(CURRENT_TIMESTAMP)"),
nullable=False,
),
sa.Column(
"updated_at",
sa.DateTime(timezone=True),
server_default=sa.text("(CURRENT_TIMESTAMP)"),
nullable=False,
),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("key_hex"),
)
with op.batch_alter_table("channels", schema=None) as batch_op:
batch_op.create_index(batch_op.f("ix_channels_name"), ["name"], unique=True)
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("channels", schema=None) as batch_op:
batch_op.drop_index(batch_op.f("ix_channels_name"))
op.drop_table("channels")
# ### end Alembic commands ###
alembic/versions/20260604_1200_rename_public_to_community.py
+29
View File
@@ -0,0 +1,29 @@
"""rename channel visibility public to community
Revision ID: 20260604_1200
Revises: 82dff87d6576
Create Date: 2026-06-04 12:00:00.000000+00:00
"""
from typing import Sequence, Union
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "20260604_1200"
down_revision: Union[str, None] = "82dff87d6576"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.execute(
"UPDATE channels SET visibility = 'community' WHERE visibility = 'public'"
)
def downgrade() -> None:
op.execute(
"UPDATE channels SET visibility = 'public' WHERE visibility = 'community'"
)
docker-compose.yml
+2 -4
View File
@@ -155,8 +155,7 @@ services:
- MQTT_TLS=${MQTT_TLS:-false}
- MQTT_TRANSPORT=${MQTT_TRANSPORT:-websockets}
- MQTT_WS_PATH=${MQTT_WS_PATH:-/}
- COLLECTOR_CHANNEL_KEYS=${COLLECTOR_CHANNEL_KEYS:-}
- COLLECTOR_INCLUDE_TEST_CHANNEL=${COLLECTOR_INCLUDE_TEST_CHANNEL:-false}
- CHANNEL_REFRESH_INTERVAL_SECONDS=${CHANNEL_REFRESH_INTERVAL_SECONDS:-300}
- DATA_HOME=/data
- SEED_HOME=/seed
# Webhook configuration
@@ -295,8 +294,6 @@ services:
- NETWORK_ANNOUNCEMENT=${NETWORK_ANNOUNCEMENT:-}
- CONTENT_HOME=/content
- TZ=${TZ:-UTC}
- COLLECTOR_CHANNEL_KEYS=${COLLECTOR_CHANNEL_KEYS:-}
- COLLECTOR_INCLUDE_TEST_CHANNEL=${COLLECTOR_INCLUDE_TEST_CHANNEL:-false}
# Feature flags (set to false to disable specific pages)
- FEATURE_DASHBOARD=${FEATURE_DASHBOARD:-true}
- FEATURE_NODES=${FEATURE_NODES:-true}
@@ -305,6 +302,7 @@ services:
- FEATURE_MAP=${FEATURE_MAP:-true}
- FEATURE_MEMBERS=${FEATURE_MEMBERS:-true}
- FEATURE_PAGES=${FEATURE_PAGES:-true}
- FEATURE_CHANNELS=${FEATURE_CHANNELS:-true}
command: ["web"]
healthcheck:
test:
docs/i18n.md
+32 -8
View File
@@ -32,8 +32,8 @@ Core entity names used throughout the application. These are referenced by other
| `nodes` | Nodes | Mesh network nodes (plural) |
| `node` | Node | Single mesh network node |
| `node_detail` | Node Detail | Node details page |
| `advertisements` | Advertisements | Network advertisements (plural) |
| `advertisement` | Advertisement | Single advertisement |
| `advertisements` | Adverts | Network advertisements (plural, used in nav menus and hero cards) |
| `advertisement` | Advert | Single advertisement |
| `messages` | Messages | Network messages (plural) |
| `message` | Message | Single message |
| `map` | Map | Network map page |
@@ -251,7 +251,7 @@ Homepage-specific content:
| `spreading_factor` | Spreading Factor | LoRa spreading factor label |
| `coding_rate` | Coding Rate | LoRa coding rate label |
| `tx_power` | TX Power | Transmit power label |
| `advertisements` | Advertisements | Homepage stat label |
| `advertisements` | Adverts | Homepage stat label |
| `messages` | Messages | Homepage stat label |
**Note:** MeshCore tagline "Off-Grid, Open-Source Encrypted Messaging" is hardcoded in English and should not be translated (trademark).
@@ -376,7 +376,31 @@ Members page content:
| `empty_state_description` | No members yet. | Empty state heading |
| `empty_description` | Members will appear here once users log in and adopt nodes. | Empty state description |
### 14. `not_found`
### 14. `channels`
Channel management and filter UI:
| Key | English | Context |
|-----|---------|---------|
| `title` | Channels | Page title |
| `add_channel` | Add Channel | Add button label |
| `edit_channel` | Edit Channel | Edit modal title |
| `delete_channel` | Delete Channel | Delete modal title |
| `delete_confirm` | Are you sure you want to delete channel {{name}}? | Delete confirmation message |
| `name_label` | Channel Name | Form label |
| `key_label` | Channel Key (hex) | Form label |
| `visibility_label` | Visibility | Form label |
| `visibility_community` | Community | Community visibility section heading |
| `visibility_member` | Member | Member visibility section heading |
| `visibility_operator` | Operator | Operator visibility section heading |
| `visibility_admin` | Admin | Admin visibility section heading |
| `enabled_label` | Enabled | Form label |
| `channel_hash_label` | Hash | Column header |
| `disabled` | Disabled | Disabled channel badge |
| `optgroup_standard` | Standard | Optgroup label for built-in channels (Public, Test) in channel filter dropdown |
| `optgroup_custom` | Custom | Optgroup label for user-defined channels in channel filter dropdown |
### 15. `not_found`
404 page content:
@@ -384,7 +408,7 @@ Members page content:
|-----|---------|---------|
| `description` | The page you're looking for doesn't exist or has been moved. | 404 description |
### 15. `custom_page`
### 16. `custom_page`
Custom markdown page errors:
@@ -392,7 +416,7 @@ Custom markdown page errors:
|-----|---------|---------|
| `failed_to_load` | Failed to load page | Page load error |
### 16. `auth`
### 17. `auth`
Authentication UI:
@@ -408,7 +432,7 @@ Authentication UI:
| `role_admin` | admin | Admin role badge text |
| `role_member` | member | Member role badge text |
### 17. `footer`
### 18. `footer`
Footer content:
@@ -416,7 +440,7 @@ Footer content:
|-----|---------|---------|
| `powered_by` | Powered by | "Powered by" attribution |
### 18. `user_profile`
### 19. `user_profile`
User profile page (OIDC authenticated users):
docs/letsmesh.md
+2 -2
View File
@@ -21,8 +21,8 @@ The collector subscribes to packets published by [meshcore-packet-capture](https
- For channel packets, if a channel key is available, a channel label is attached (for example `Public` or `#test`) for UI display.
- In the messages feed and dashboard channel sections, known channel indexes are preferred for labels (`17 -> Public`, `217 -> #test`) to avoid stale channel-name mismatches.
- Additional channel names are loaded from `COLLECTOR_CHANNEL_KEYS` when entries are provided as `label=hex` (for example `bot=<key>`).
- The collector keeps built-in keys for `Public` and `#test`, and merges any additional keys from `COLLECTOR_CHANNEL_KEYS`.
- Additional channel names are loaded from the `channels` database table (managed via CLI, API, or seed YAML).
- The collector keeps built-in keys for `Public` and `#test`, and merges any additional keys from enabled database channel rows.
## Location and Messages
docs/plans/20260519-2051-channel-model-db-decrypt/plan.md
+298
View File
@@ -0,0 +1,298 @@
# Channel Model: Database-Backed Decrypt Keys with Permission-Based Visibility
## Summary
Add a `Channel` database model to replace the `COLLECTOR_CHANNEL_KEYS` environment variable entirely. Channels store their name, secret key, computed channel hash, and a **visibility/permission level** (`public`, `member`, `operator`, `admin`). The collector loads keys from the database at startup and periodically refreshes them without restart. The web dashboard and API enforce permission-based visibility: "public" channels are visible to everyone (including logged-out users), "member" channels only to authenticated members, and so on.
A new **Channels page** in the web dashboard presents channels as cards (desktop and mobile), each showing a **QR code** for easy joining (`meshcore://channel/add?name=...&key=...`). The page is always visible regardless of OIDC status. When OIDC is enabled, admin users see inline channel management (add/edit/delete) and non-admin users see read-only cards filtered by their role. When OIDC is disabled, all channels are `public` by default, no admin UI is shown, and channels can only be configured via the seed mechanism.
## Background & Motivation
### Current State
Channel decryption keys flow through the system as follows:
1. `COLLECTOR_CHANNEL_KEYS` env var (comma/space-separated hex strings, e.g. `"MyChannel=ABC123...,Other=DEF456..."`)
2. Parsed by `CollectorSettings.collector_channel_keys_list` into a `list[str]` (`config.py:185-193`)
3. Passed to `create_subscriber(channel_keys=...)` and then `LetsMeshPacketDecoder(channel_keys=...)` (`subscriber.py:88-90`)
4. The decoder builds a `MeshCoreKeyStore` with `add_channel_secrets()` and uses it to decrypt GroupText (type 5) packets via the `meshcoredecoder` library (`letsmesh_decoder.py:63-68`)
5. Built-in keys (`Public`, `test`) are always included via `BUILTIN_CHANNEL_KEYS` (`letsmesh_decoder.py:32-35`)
6. The web app independently builds channel labels via `_build_channel_labels()` in `web/app.py:171-185`, reading the same env var to create a decoder instance just for label resolution
### Problems
- **Restart required**: Adding or changing a channel key requires editing `.env` and restarting the collector process.
- **No permission model**: All channels are visible to all users. There is no way to restrict sensitive channels (e.g., operator-only coordination channels) from public view.
- **No API/CLI management**: There is no way to add/remove channel keys at runtime.
- **No audit trail**: Keys exist only in config; there is no record of when a key was added or by whom.
- **Duplicated decoder construction**: The web app builds its own `LetsMeshPacketDecoder` just to resolve channel labels (`web/app.py:179-182`). With a database source, both collector and web can query the same `channels` table.
- **No user-facing channel info**: Users cannot discover available channels, see their names, or get QR codes to join them from their devices.
### Why Now
The collector already queries the database for cleanup and event persistence. The web app already has a QR code library (`qrcodejs`) loaded globally and used on the node detail page (`node-detail.js:361-374`) with the `meshcore://` URL scheme. The API proxy already has a role-based access control framework (`_build_endpoint_access` / `check_api_access` in `web/app.py:68-161`). Making channels a first-class database entity unblocks permission-based message filtering, a channels management page, and QR code distribution.
## Goals
- Introduce a `Channel` SQLAlchemy model with name, key, channel hash, **visibility/permission level**, and enabled flag
- Replace `COLLECTOR_CHANNEL_KEYS` env var entirely with database-backed channels
- Have the collector load keys from the `channels` table at startup and refresh periodically (no restart)
- Enforce permission-based visibility in messages view and dashboard: only show messages on channels the user has access to
- Provide a **Channels page** (`/channels`) that is always visible (with or without OIDC)
- When OIDC is enabled: show admin-only inline channel management; filter channel visibility by user role
- When OIDC is disabled: show all (public-only) channels read-only; channels configured only via seed
- Provide CLI commands and API endpoints for channel CRUD
- Support seeding channels from YAML (visibility defaults to `public`; no visibility field in seed data)
- Remove `COLLECTOR_CHANNEL_KEYS` and related config plumbing
## Non-Goals
- Changing the `meshcoredecoder` library or the decryption logic itself
- Storing node-specific private keys (this is about channel shared secrets)
- Encrypting channel keys at rest in the database
- End-to-end encryption or per-user channel access control beyond the role-based visibility model
- Filtering messages at the collector level (the collector decrypts all channels; filtering is done at the API/web layer)
- Admin UI for channels when OIDC is disabled (seed-only configuration)
## Requirements
### Functional Requirements
- **FR-1**: A `Channel` database model with fields:
- `id` (UUID primary key)
- `name` (String(100), unique, non-empty)
- `key_hex` (String(64), uppercase hex, unique — supports both AES-128 and AES-256 keys)
- `channel_hash` (String(2), computed: first byte of SHA-256 of `key_hex`)
- `visibility` (Enum: `public`, `member`, `operator`, `admin`; default `public`)
- `enabled` (Boolean, default `true`)
- `created_at`, `updated_at` (timestamps)
- **FR-2**: On startup, the collector queries all `Channel` rows where `enabled=true`, merges them with the hardcoded built-in keys (`Public`, `test` — both always available to the decoder), and builds the `MeshCoreKeyStore`. The `Public` built-in key always has `visibility=public` and cannot be overridden. The `test` built-in key is always loaded into the decoder for decryption, **but** test channel messages are **discarded by the normalizer by default** — they are only stored when a `test` channel row exists in the DB with `enabled=true` (added by an admin via CLI/API/seed). This replaces the `COLLECTOR_INCLUDE_TEST_CHANNEL` env var.
- **FR-3**: The collector periodically refreshes its key store from the database (configurable interval, default 5 minutes).
- **FR-4**: **Permission-based message visibility**: The API `/messages` endpoint accepts the user's role context (via OIDC X-User-Roles header from the web proxy, or API key auth for direct calls) and filters channel messages so that:
- No OIDC / OIDC disabled: all channel messages visible (all channels are `public` in this mode)
- Logged-out (OIDC enabled): only messages on `public` channels
- `member` role: messages on `public` + `member` channels
- `operator` role: messages on `public` + `member` + `operator` channels
- `admin` role: all messages (all channels)
- Direct messages (non-channel) remain governed by existing read access rules.
- **FR-4b**: **Dashboard channel activity filtering**: The `/dashboard/stats` and `/dashboard/message-activity` endpoints filter channel-related data by the same role-based visibility rules as `/messages`. Channel message counts, channel-specific message lists, and activity charts only include data from channels visible to the requesting user.
- **FR-5**: The dashboard and messages page channel filter dropdowns only show channels visible to the current user's role. When OIDC is disabled, all channels appear.
- **FR-6**: **Channels page** (`/channels`) -- always visible regardless of OIDC status:
- **OIDC disabled**: Read-only card grid showing all channels (all `public`). No add/edit/delete UI. No visibility badges.
- **OIDC enabled, no auth**: Read-only cards showing only `public` channels.
- **OIDC enabled, logged in**: Read-only cards showing channels up to the user's role level, with visibility badges.
- **OIDC enabled, admin**: Full management -- "Add Channel" button, edit/delete buttons per card, visibility select in forms.
- Card layout (responsive, works on desktop and mobile)
- Each card shows: channel name, channel hash, visibility badge (if OIDC enabled), QR code, enabled status
- QR code format: `meshcore://channel/add?name=<encoded_name>&key=<key_hex>`
- Card shows masked key (first/last 4 chars) with a reveal toggle for admins
- **FR-7**: **Admin inline channel management** (OIDC enabled + admin role only): Add/edit/delete channels via modal dialogs (following the tag editor pattern in `node-detail.js`). Only the `admin` role can perform these operations. This is not available when OIDC is disabled.
- **FR-8**: CLI commands: `meshcore-hub collector channel list`, `channel add --name X --key HEX --visibility public`, `channel remove --name X`, `channel enable/disable --name X`.
- **FR-9**: API endpoints:
- `GET /channels` -- list channels; filtered by user role visibility when OIDC enabled; returns all public channels when OIDC disabled
- `POST /channels` -- create (admin only, OIDC required)
- `PUT /channels/{id}` -- update (admin only, OIDC required)
- `DELETE /channels/{id}` -- delete (admin only, OIDC required)
- The web proxy (`_build_endpoint_access`) guards mutations behind the `admin` role; `GET` is `_OPEN`
- **FR-10**: Channel seeding from `${SEED_HOME}/channels.yaml` via `meshcore-hub collector seed`. Seed format does not include a `visibility` field -- seeded channels always get `visibility=public`. This is the only way to configure channels when OIDC is disabled.
- **FR-11**: `COLLECTOR_CHANNEL_KEYS` env var and related config (`collector_channel_keys`, `collector_channel_keys_list`) are removed. Migration guide documents the removal.
- **FR-12**: The web app's `_build_channel_labels()` in `web/app.py` is updated to query the `channels` table from the shared database instead of re-parsing the env var.
- **FR-13**: The `FEATURE_CHANNELS` feature flag controls page visibility. It does not depend on OIDC being enabled (unlike `feature_members` which requires OIDC). The page is available to all users when the flag is `true`.
### Technical Requirements
- **TR-1**: New model `Channel` in `src/meshcore_hub/common/models/channel.py`, exported from `models/__init__.py`.
- **TR-2**: Alembic migration to create the `channels` table.
- **TR-3**: Pydantic schemas for channel CRUD in `src/meshcore_hub/common/schemas/channels.py`.
- **TR-4**: `LetsMeshPacketDecoder.reload_keys(channel_keys: list[str])` method that rebuilds `MeshCoreKeyStore` and `_channel_names_by_hash` without discarding the decode cache. Thread-safe via atomic reference swap.
- **TR-5**: `Subscriber` gains a `_start_channel_refresh_scheduler()` method following the cleanup scheduler pattern (`subscriber.py:245-357`). Uses `DatabaseManager.async_session()` to query channels.
- **TR-6**: Message filtering at the API layer: the `/messages` route resolves the user's highest role, queries `channels` table for visible channel hashes, then filters channel messages. Filtering logic:
- OIDC disabled (no auth roles): no filtering — all channels treated as `public`.
- OIDC enabled: query DB for all channel hashes up to the user's visibility level. The query filter is: `(message_type != 'channel') OR (channel_idx IN (visible_hashes_as_ints)) OR (channel_idx NOT IN (all_known_hashes_as_ints))`. This shows direct messages always, channels at/below the user's visibility level, and unknown channels (treated as `public`). No pre-filtering means channels visible in the filter dropdown may differ from visible messages, but the `<10 channel count makes this negligible.
- **TR-7**: New SPA page module `src/meshcore_hub/web/static/js/spa/pages/channels.js` with card layout, QR code generation (reusing the `QRCode` library from `qrcodejs`), and admin modal editors.
- **TR-8**: Navigation placement: **All navigation surfaces** use the order `Messages → Channels → Members → Map`:
- `spa.html` desktop sidebar and mobile menu: insert Channels `<li>` between Messages and Members
- `app.js` dynamic nav: insert Channels `if (features.channels)` block between Messages and Members blocks
- `home.js` hero card grid: insert Channels `renderNavCard()` between Messages and Members cards
- Add CSS custom property `--color-channels` in `app.css` for hero card accent color
- **TR-9**: `FEATURE_CHANNELS` feature flag in `WebSettings` (default `true`), registered in the `features` property. Unlike `feature_members`, it does not gate on `oidc_enabled`.
- **TR-10**: The `channel_labels` config passed to the web frontend via `/config` endpoint stays in its current format (`{str(channel_idx): label}`). It is built from the `channels` DB table instead of parsing `COLLECTOR_CHANNEL_KEYS`, using a synchronous SQLAlchemy engine (SQLite allows concurrent reads). The existing `getChannelLabelsMap()` function in `components.js` continues to work unchanged. Channel visibility is fetched separately by the Channels page via `/api/v1/channels` — it does not go through the `/config` endpoint.
- **TR-11**: Remove `COLLECTOR_CHANNEL_KEYS` and `COLLECTOR_INCLUDE_TEST_CHANNEL` from `CollectorSettings`, remove `collector_channel_keys_list` property, remove `_parse_decoder_key_entries()` from `web/app.py`.
- **TR-12**: i18n keys for channel-related UI strings added to `en.json` and documented in `docs/i18n.md`.
- **TR-13**: Web proxy access mapping in `_build_endpoint_access()` updated:
- `"v1/channels": { "GET": _OPEN }` -- anyone can list
- `"v1/channels/": { "POST": frozenset({role_admin}), "PUT": frozenset({role_admin}), "DELETE": frozenset({role_admin}) }` -- admin-only mutations
- **TR-14**: `CHANNEL_REFRESH_INTERVAL_SECONDS` env var added to `CollectorSettings` (default `300`). Not in `WebSettings`.
## Implementation Plan
### Phase 1: Channel Model & Migration
- Create `src/meshcore_hub/common/models/channel.py`:
```python
class ChannelVisibility(str, Enum):
PUBLIC = "public"
MEMBER = "member"
OPERATOR = "operator"
ADMIN = "admin"
class Channel(Base, UUIDMixin, TimestampMixin):
__tablename__ = "channels"
name: Mapped[str] = mapped_column(String(100), unique=True, nullable=False)
key_hex: Mapped[str] = mapped_column(String(64), unique=True, nullable=False)
channel_hash: Mapped[str] = mapped_column(String(2), nullable=False)
visibility: Mapped[str] = mapped_column(String(20), default="public")
enabled: Mapped[bool] = mapped_column(Boolean, default=True)
```
- Export from `models/__init__.py`
- Generate Alembic migration: `meshcore-hub db revision --autogenerate -m "add channels table"`
- Unit tests for the model
### Phase 2: Decoder Reload Support
- Add `reload_keys(channel_keys: list[str])` to `LetsMeshPacketDecoder`
- Normalize new key list, rebuild `MeshCoreKeyStore`, update `_channel_names_by_hash`
- Preserve decode cache across reloads
- **Thread safety**: Add a `threading.Lock` (`_state_lock`) to guard access to `_key_store` and `_channel_names_by_hash`. The MQTT message callback thread reads these during decode; the refresh thread writes during reload. Lock is only held during the atomic swap (not during key normalization/KeyStore construction).
- Unit tests for reload behavior
### Phase 3: Collector DB Key Loading & Refresh
- On startup (`Subscriber.__init__` or `start()`), query `Channel` table for `enabled=true` rows via `self.db.session_scope()`
- Merge DB channels with the hardcoded built-in keys (`Public`, `test` — both always available to the decoder). The `test` key is always loaded into the decoder but test messages are discarded unless a DB row exists (see FR-2).
- The `_include_test_channel` flag moves from env var to a DB query: `self.db.async_session()->query(Channel).filter(name="test", enabled=True).first() is not None`. Evaluated once at startup and on refresh.
- Add `_start_channel_refresh_scheduler()` to `Subscriber`, following the cleanup scheduler pattern (`subscriber.py:245-357`)
- Add `channel_refresh_interval_seconds` field to `CollectorSettings` (env var `CHANNEL_REFRESH_INTERVAL_SECONDS`, default `300`)
- Pass interval to `Subscriber.__init__` alongside other scheduler params (cleanup_enabled, cleanup_retention_days, etc.)
- Remove `channel_keys` parameter from `Subscriber.__init__`, `create_subscriber()`, and `run_collector()`
- Remove `COLLECTOR_CHANNEL_KEYS` from `CollectorSettings` and related parsing
- Integration tests
### Phase 4: API Endpoints & Message Filtering
- Create `src/meshcore_hub/common/schemas/channels.py`: `ChannelCreate`, `ChannelRead`, `ChannelUpdate`, `ChannelList`
- Create `src/meshcore_hub/api/routes/channels.py`:
- `GET /channels` -- list channels, filtered by user role. When no OIDC roles are present (OIDC disabled or not logged in), returns only `public` channels
- `POST /channels` -- create (admin only, uses `RequireAdmin` dependency on API side)
- `PUT /channels/{id}` -- update (admin only)
- `DELETE /channels/{id}` -- delete (admin only)
- Add role-aware message filtering to `GET /messages`:
- Resolve user's highest role from auth context (X-User-Roles header or API key)
- When no roles available (OIDC disabled): no filtering (all channels visible)
- When OIDC enabled: query visible channel hashes from `channels` table based on role hierarchy (role → visibility levels up to that role)
- Channel hash to channel_idx conversion: `channel_idx = int(channel_hash, 16)` (both are 0-255)
- Query filter: `(message_type != 'channel') OR (channel_idx IN (visible_idxs)) OR (channel_idx NOT IN (all_known_idxs))`
- Unknown channel hashes (not in DB) are treated as `public` and pass through the third clause
- Add role-aware channel filtering to `GET /dashboard/stats` and `/dashboard/message-activity`:
- Resolve user's highest role from auth context
- Filter channel counts and channel activity lists by visible channels using the same visibility logic as `/messages`
- When OIDC disabled: show all channels
- Update `_build_channel_labels()` in `web/app.py` to query DB using a synchronous SQLAlchemy engine against the shared SQLite database. This is safe because SQLite allows concurrent readers. The function is called once at startup; results are stored in `app.state.channel_labels`. The existing format (`{str(channel_idx): label}`) is preserved.
- Add entries to `_build_endpoint_access()`:
- `"v1/channels": { "GET": _OPEN }`
- `"v1/channels/": { "POST": frozenset({role_admin}), "PUT": frozenset({role_admin}), "DELETE": frozenset({role_admin}) }`
- Register router in `api/routes/__init__.py`
- Tests for all endpoints and filtering
### Phase 5: CLI Commands
- Add `channel` subgroup to collector CLI in `cli.py`
- `meshcore-hub collector channel list` -- name, masked key, hash, visibility, enabled
- `meshcore-hub collector channel add --name NAME --key HEX --visibility public`
- `meshcore-hub collector channel remove --name NAME`
- `meshcore-hub collector channel enable/disable --name NAME`
- Remove `channel_keys` and `include_test_channel` params from `_run_collector_service()`
- Tests for each command
### Phase 6: Web Dashboard -- Channels Page
- Create `src/meshcore_hub/web/static/js/spa/pages/channels.js`:
- Fetch `/api/v1/channels` (API returns only channels the user can see)
- Render responsive card grid (DaisyUI `card` component)
- Each card: channel name, channel hash badge, visibility badge (OIDC only), QR code, masked key
- QR code: `meshcore://channel/add?name=<encoded>&key=<hex>` using `QRCode` library
- **OIDC disabled**: read-only cards, no visibility badges, no add/edit/delete controls
- **OIDC enabled, admin**: "Add Channel" button, edit/delete buttons per card
- **OIDC enabled, non-admin**: read-only cards filtered by role
- Add/edit modal (following tag editor pattern from `node-detail.js`): name, key (hex), visibility select, enabled toggle -- only rendered when `hasRole('admin')`
- Delete confirmation modal (following `tagDeleteModal` pattern)
- Use `getConfig().oidc_enabled` to conditionally show/hide admin controls and visibility badges
- **Navigation ordering** -- Channels appears **after Messages and before Members** in all navigation surfaces:
- `spa.html` desktop sidebar and mobile menu: insert Channels `<li>` between Messages and Members
- `app.js` dynamic nav (`renderNavItems()`): insert Channels `if (features.channels !== false)` block between Messages and Members blocks
- `home.js` hero card grid: insert Channels `renderNavCard()` between Messages and Members cards
- Add CSS custom property `--color-channels` in `app.css` for hero card accent color
- **Icon**: Use the existing `iconChannel` SVG function from `icons.js` (hash/# icon, already defined at `icons.js:77-79`). Import it in `channels.js`, `home.js`, `app.js`, and use it inline in `spa.html` nav links.
- Register route in `app.js`: `router.addRoute('/channels', pageHandler(pages.channels))`
- No OIDC dependency in route registration (unlike members which gates on `features.members`)
- Add `FEATURE_CHANNELS` feature flag to `WebSettings` -- does NOT gate on `oidc_enabled`:
```python
feature_channels: bool = Field(default=True, description="Enable the /channels page")
# In features property:
"channels": self.feature_channels, # no oidc_enabled guard
```
- Add page title handling in `updatePageTitle()` in `app.js`
- Add i18n keys to `en.json` (including `entities.channels`, `entities.channel`) and update `docs/i18n.md`
- Tests in `tests/test_web/`
### Phase 7: Seeding, Config Cleanup & Docs
- Add `channels.yaml` support to seed importer in `cli.py`
- Shorthand format: `name: HEX` — value is a hex string, treated as the channel key
- Expanded format: `name: { key: HEX, enabled: true }` — value is a dict
- The parser distinguishes by type: `str` → shorthand (treat value as `key_hex`), `dict` → expanded (read `key` and optional `enabled` fields)
- No `visibility` field in seed format — always defaults to `public`
- This is the primary configuration path when OIDC is disabled
- Remove `COLLECTOR_CHANNEL_KEYS` and `COLLECTOR_INCLUDE_TEST_CHANNEL` from:
- `CollectorSettings` in `config.py`
- `collector_channel_keys_list` property
- `_parse_decoder_key_entries()` in `web/app.py`
- `_run_collector_service()` in `cli.py`
- `Subscriber.__init__`, `create_subscriber()`, `run_collector()` signatures
- `AGENTS.md` env var list
- `.env.example` (lines 204 and 208)
- Update `docs/seeding.md` with channels.yaml documentation (emphasize: visibility always `public`, admin-only channels require OIDC + API/CLI)
- Update `docs/upgrading.md` with migration guide:
- Run `meshcore-hub db upgrade`
- Convert any `COLLECTOR_CHANNEL_KEYS` values to `channels.yaml` seed file or DB rows via CLI
- Remove env var from `.env`
- Note: existing seeded channels will be `public` visibility
- Update `AGENTS.md` with new model, page, feature flag, and API routes
- Update `SCHEMAS.md` if channel event schemas are affected
## Review
**Status**: Approved with Changes
**Reviewed**: 2026-05-19
### Resolutions
- **Nav ordering**: All navigation surfaces use `Messages → Channels → Members → Map` (not `Messages → Channels → Map → Members`). Updated TR-8 and Phase 6.
- **`key_hex` column size**: Changed from `String(32)` to `String(64)` to support both AES-128 and AES-256 keys. Updated FR-1 and Phase 1 model.
- **Frontend config format**: `channel_labels` stays as `{str(idx): label}` — unchanged format. Channels page fetches visibility from `/api/v1/channels` directly. Updated TR-10.
- **`_build_channel_labels()` sync approach**: Uses a synchronous SQLAlchemy engine for a one-time startup query against the shared SQLite database (safe because SQLite allows concurrent readers). Updated Phase 4.
- **Dashboard channel filtering**: Yes, dashboard channel activity is filtered by role visibility (consistent with `/messages`). Added FR-4b and updated Phase 4.
- **Thread safety**: Specified `threading.Lock` (`_state_lock`) on decoder for atomic swap between MQTT callback and refresh threads. Updated Phase 2.
- **`CHANNEL_REFRESH_INTERVAL_SECONDS`**: Added as `CollectorSettings` field (not `WebSettings`). Added TR-14 and updated Phase 3.
- **Message filtering logic**: Changed from simple `IN (...)` to three-clause filter: direct messages always visible, known channels filtered by visibility, unknown channels treated as `public`. Updated TR-6 and Phase 4.
- **Seed format**: Parser distinguishes shorthand (`str` → key_hex) vs expanded (`dict` → read `key` and optional `enabled`). No `visibility` field. Clarified in Phase 7.
### Remaining Action Items
- **QR code URL format**: The `meshcore://channel/add?name=...&key=...` scheme is proposed by analogy with the existing contact QR. Confirm with MeshCore app devs whether this scheme is supported or planned. (Baked into plan as-is; QR codes work if/when app supports them.)
- **Test channel excluded by default**: `test` built-in key always loaded into decoder (for decryption), but normalizer discards test messages unless a `Channel` row with `name="test"` and `enabled=true` exists in DB. NOT created automatically — admin must explicitly add it. Replaces `COLLECTOR_INCLUDE_TEST_CHANNEL`. Updated FR-2 and Phase 3.
- **Existing test updates**: Tests that reference `channel_labels["17"] == "Public"` etc. must continue to pass. Since format is unchanged, they should, but verify during Phase 4.
## References
- `src/meshcore_hub/collector/letsmesh_decoder.py` -- decoder with static key init, `BUILTIN_CHANNEL_KEYS`, `channel_labels_by_index()`
- `src/meshcore_hub/collector/subscriber.py` -- subscriber with cleanup scheduler pattern (thread + async session) to follow
- `src/meshcore_hub/common/models/node_tag.py` -- model pattern (UUIDMixin, TimestampMixin)
- `src/meshcore_hub/common/config.py:141-193` -- current `COLLECTOR_CHANNEL_KEYS` config (to be removed)
- `src/meshcore_hub/web/app.py:68-161` -- `_build_endpoint_access()` and `check_api_access()` for proxy auth guards
- `src/meshcore_hub/web/app.py:164-185` -- `_build_channel_labels()` (to be updated to query DB)
- `src/meshcore_hub/web/static/js/spa/pages/node-detail.js:361-374` -- QR code pattern using `QRCode` library and `meshcore://` scheme
- `src/meshcore_hub/web/static/js/spa/pages/node-detail.js:25-71` -- modal dialog patterns for tag edit/delete
- `src/meshcore_hub/api/auth.py` -- auth dependencies (`RequireRead`, `RequireAdmin`, `require_operator_or_admin`)
- `src/meshcore_hub/web/static/js/spa/components.js` -- `hasRole()`, `getChannelLabelsMap()`, `resolveChannelLabel()`
docs/plans/20260519-2051-channel-model-db-decrypt/tasks.md
+256
View File
@@ -0,0 +1,256 @@
# Tasks: Channel Model — Database-Backed Decrypt Keys with Permission-Based Visibility
> Generated from `plan.md` on 2026-05-19
## 1. Database Schema & Migration
- [ ] 1.1 Create `Channel` SQLAlchemy model
- [ ] 1.1.1 Define `ChannelVisibility` enum (`public`, `member`, `operator`, `admin`)
- [ ] 1.1.2 Create `Channel` class in `src/meshcore_hub/common/models/channel.py` (fields: `id`, `name`, `key_hex`, `channel_hash`, `visibility`, `enabled`, `created_at`, `updated_at`)
- [ ] 1.1.3 `key_hex` must be `String(64)` (supports AES-128 and AES-256 keys)
- [ ] 1.1.4 `channel_hash` must be `String(2)` (first byte of SHA-256 of `key_hex`, uppercase hex)
- [ ] 1.1.5 `name` must be `String(100)`, unique, non-nullable
- [ ] 1.1.6 Export `Channel`, `ChannelVisibility` from `models/__init__.py`
- [ ] 1.2 Generate Alembic migration
- [ ] 1.2.1 Run `meshcore-hub db revision --autogenerate -m "add channels table"`
- [ ] 1.2.2 Review generated migration for correctness (unique constraints on `name` and `key_hex`)
- [ ] 1.2.3 Test migration: `meshcore-hub db upgrade` and verify table creation
- [ ] 1.3 Create Pydantic schemas
- [ ] 1.3.1 Create `src/meshcore_hub/common/schemas/channels.py` with `ChannelCreate`, `ChannelRead`, `ChannelUpdate`, `ChannelList`
- [ ] 1.3.2 `ChannelCreate`: validate `name`, `key_hex` (uppercase hex, 32 or 64 chars), optional `visibility` (default `public`), optional `enabled` (default `true`)
- [ ] 1.3.3 `ChannelRead`: include `id`, `name`, `channel_hash`, `visibility`, `enabled`, `created_at`, `updated_at`, but NOT `key_hex` (mask first/last 4 chars for read)
- [ ] 1.3.4 `ChannelUpdate`: all fields optional except `name` immutable
- [ ] 1.3.5 Add `masked_key` computed property on `ChannelRead` (e.g. `"ABCD...EF01"`)
- [ ] 1.4 Write unit tests for the model and schemas
- [ ] 1.4.1 Test `Channel` model instantiation and defaults
- [ ] 1.4.2 Test unique constraint enforcement on `name` and `key_hex`
- [ ] 1.4.3 Test `ChannelCreate` schema validation (valid keys, invalid keys, name length)
- [ ] 1.4.4 Test `ChannelRead.masked_key` formatting
## 2. Decoder Reload Support
- [ ] 2.1 Add `reload_keys()` method to `LetsMeshPacketDecoder`
- [ ] 2.1.1 Add `threading.Lock` (`_state_lock`) to the decoder class
- [ ] 2.1.2 Implement `reload_keys(channel_keys: list[str])` — normalize new keys, rebuild `MeshCoreKeyStore`, update `_channel_names_by_hash`
- [ ] 2.1.3 Preserve the decode cache (`_decode_cache`) across reloads
- [ ] 2.1.4 Use `_state_lock` for atomic swap of `_key_store` and `_channel_names_by_hash` (hold lock only during swap, not during key normalization/KeyStore construction)
- [ ] 2.1.5 Update `channel_labels_by_index()` and `resolve_channel_name()` to use `_state_lock` when reading shared state
- [ ] 2.2 Write unit tests for reload behavior
- [ ] 2.2.1 Test that reload with new keys enables decryption of messages on the new channel
- [ ] 2.2.2 Test that decode cache persists across reloads
- [ ] 2.2.3 Test thread safety: concurrent decode reads while reload is in progress (mock/threading test)
## 3. Collector DB Key Loading & Refresh
- [ ] 3.1 Update `Subscriber` to load keys from database on startup
- [ ] 3.1.1 Query all `enabled=true` channels from DB via `self.db.session_scope()`
- [ ] 3.1.2 Merge DB channels with hardcoded built-in keys (`Public`, `test` — always loaded into decoder)
- [ ] 3.1.3 Move `_include_test_channel` from env var to DB query: check if `Channel(name="test", enabled=True)` row exists
- [ ] 3.1.4 Pass merged key list to decoder (replacing old `channel_keys` constructor param)
- [ ] 3.2 Update `letsmesh_normalizer.py` test channel filter
- [ ] 3.2.1 Replace env-var-based `_include_test_channel` check with DB lookup in the normalizer
- [ ] 3.2.2 Test messages are always decrypted (key in decoder), but discarded by normalizer unless DB row exists
- [ ] 3.3 Add `_start_channel_refresh_scheduler()` to `Subscriber`
- [ ] 3.3.1 Follow the cleanup scheduler pattern (`subscriber.py:245-357`): daemon thread + async session
- [ ] 3.3.2 Query enabled channels from DB on each cycle
- [ ] 3.3.3 Call `decoder.reload_keys()` with updated key list
- [ ] 3.3.4 Handle graceful shutdown (stop event)
- [ ] 3.4 Add `CHANNEL_REFRESH_INTERVAL_SECONDS` to `CollectorSettings`
- [ ] 3.4.1 Add field to `CollectorSettings` in `config.py` (default `300`, env var `CHANNEL_REFRESH_INTERVAL_SECONDS`)
- [ ] 3.4.2 Pass interval to `Subscriber.__init__` alongside other scheduler params
- [ ] 3.5 Remove `channel_keys` from collector plumbing
- [ ] 3.5.1 Remove `channel_keys` param from `Subscriber.__init__`, `create_subscriber()`, `run_collector()`
- [ ] 3.5.2 Remove `COLLECTOR_CHANNEL_KEYS` from `CollectorSettings`
- [ ] 3.5.3 Remove `collector_channel_keys_list` property from `CollectorSettings`
- [ ] 3.5.4 Remove `COLLECTOR_INCLUDE_TEST_CHANNEL` from `CollectorSettings`
- [ ] 3.6 Write integration tests
- [ ] 3.6.1 Test that collector loads keys from DB on startup
- [ ] 3.6.2 Test that collector refreshes keys on schedule
- [ ] 3.6.3 Test that test channel messages are discarded when no DB row exists
- [ ] 3.6.4 Test that test channel messages are stored when DB row with `enabled=true` exists
## 4. API Endpoints & Message Filtering
- [ ] 4.1 Create API routes for channels
- [ ] 4.1.1 Create `src/meshcore_hub/api/routes/channels.py`
- [ ] 4.1.2 Implement `GET /channels` — list channels filtered by user role visibility; when no OIDC roles, return only `public` channels
- [ ] 4.1.3 Implement `POST /channels` — create channel (admin only)
- [ ] 4.1.4 Implement `PUT /channels/{id}` — update channel (admin only, `name` immutable)
- [ ] 4.1.5 Implement `DELETE /channels/{id}` — delete channel (admin only)
- [ ] 4.1.6 Register router in `api/routes/__init__.py`
- [ ] 4.2 Add role-aware message filtering to `GET /messages`
- [ ] 4.2.1 Resolve user's highest role from auth context (X-User-Roles header or API key)
- [ ] 4.2.2 When no roles available (OIDC disabled): no filtering (all channels treated as public)
- [ ] 4.2.3 When OIDC enabled: query visible channel hashes from `channels` table based on role hierarchy
- [ ] 4.2.4 Build visibility set: compute `channel_idx = int(channel_hash, 16)` for each visible channel
- [ ] 4.2.5 Build full known set: all `channel_idx` values from all channels in DB (for "unknown = public" clause)
- [ ] 4.2.6 Apply three-clause filter: `(message_type != 'channel') OR (channel_idx IN (visible_idxs)) OR (channel_idx NOT IN (all_known_idxs))`
- [ ] 4.3 Add role-aware channel filtering to dashboard endpoints
- [ ] 4.3.1 Resolve user's highest role from auth context (same logic as `/messages`)
- [ ] 4.3.2 Filter `GET /dashboard/stats` channel message counts by visible channels
- [ ] 4.3.3 Filter `GET /dashboard/message-activity` channel activity lists by visible channels
- [ ] 4.3.4 When OIDC disabled: show all channels (no filtering)
- [ ] 4.4 Update `_build_channel_labels()` in `web/app.py`
- [ ] 4.4.1 Replace env-var parsing with database query using a synchronous SQLAlchemy engine
- [ ] 4.4.2 Maintain existing format: `{str(channel_idx): label}`
- [ ] 4.4.3 Include both built-in `Public` and all `enabled=true` channels from DB
- [ ] 4.4.4 Remove `_parse_decoder_key_entries()` helper function
- [ ] 4.5 Update web proxy `_build_endpoint_access()`
- [ ] 4.5.1 Add `"v1/channels": { "GET": _OPEN }` — anyone can list (filtering is server-side)
- [ ] 4.5.2 Add `"v1/channels/": { "POST": frozenset({role_admin}), "PUT": frozenset({role_admin}), "DELETE": frozenset({role_admin}) }` — admin-only mutations
- [ ] 4.5.3 Verify longest-prefix matching: `v1/channels/` takes precedence for POST/PUT/DELETE over `v1/channels`
- [ ] 4.6 Write tests for API endpoints and filtering
- [ ] 4.6.1 Test `GET /channels` returns only public channels when no auth
- [ ] 4.6.2 Test `GET /channels` returns appropriate channels per role
- [ ] 4.6.3 Test `POST/PUT/DELETE /channels` restricted to admin only
- [ ] 4.6.4 Test message filtering: only visible channel messages returned per role
- [ ] 4.6.5 Test message filtering: unknown channels pass through (treated as public)
- [ ] 4.6.6 Test message filtering: direct messages always visible
- [ ] 4.6.7 Test dashboard channel counts filtered by role
- [ ] 4.6.8 Verify existing tests referencing `channel_labels["17"] == "Public"` still pass
## 5. CLI Commands
- [ ] 5.1 Add `channel` subgroup to collector CLI
- [ ] 5.1.1 Create `channel` Click group in `src/meshcore_hub/collector/cli.py`
- [ ] 5.1.2 Implement `meshcore-hub collector channel list` — table output: name, masked key, hash, visibility, enabled
- [ ] 5.1.3 Implement `meshcore-hub collector channel add --name NAME --key HEX --visibility public` — create channel row
- [ ] 5.1.4 Implement `meshcore-hub collector channel remove --name NAME` — delete channel by name
- [ ] 5.1.5 Implement `meshcore-hub collector channel enable --name NAME` — set `enabled=true`
- [ ] 5.1.6 Implement `meshcore-hub collector channel disable --name NAME` — set `enabled=false`
- [ ] 5.1.7 Remove `channel_keys` and `include_test_channel` params from `_run_collector_service()`
- [ ] 5.2 Write tests for CLI commands
- [ ] 5.2.1 Test `channel list` output format
- [ ] 5.2.2 Test `channel add` creates row with correct fields and `channel_hash` computed
- [ ] 5.2.3 Test `channel add` rejects invalid keys (non-hex, wrong length)
- [ ] 5.2.4 Test `channel add` rejects duplicate names
- [ ] 5.2.5 Test `channel remove` deletes row
- [ ] 5.2.6 Test `channel enable/disable` toggles `enabled` flag
- [ ] 5.2.7 Test that old `--channel-keys` option is removed (CLI help output)
## 6. Web Dashboard — Channels Page
- [ ] 6.1 Add `FEATURE_CHANNELS` feature flag
- [ ] 6.1.1 Add `feature_channels: bool` to `WebSettings` (default `true`, env var `FEATURE_CHANNELS`)
- [ ] 6.1.2 Add `"channels": self.feature_channels` to `features` property (NO `oidc_enabled` guard)
- [ ] 6.1.3 Expose in `/config` endpoint response
- [ ] 6.2 Create Channels page module
- [ ] 6.2.1 Create `src/meshcore_hub/web/static/js/spa/pages/channels.js`
- [ ] 6.2.2 Implement `render(container, params, router)` — fetch `/api/v1/channels`, render card grid
- [ ] 6.2.3 Return a cleanup function if any resources are created
- [ ] 6.2.4 Responsive card layout: DaisyUI `card` component, grid adapts to screen width
- [ ] 6.3 Channel card UI
- [ ] 6.3.1 Card content: channel name, channel hash badge, visibility badge (OIDC only), QR code, masked key
- [ ] 6.3.2 QR code generation: `new QRCode(canvas, { text: "meshcore://channel/add?name=<encoded>&key=<hex>" })` using existing `qrcodejs` library
- [ ] 6.3.3 Masked key display: `{first4}...{last4}` with reveal toggle for admin users
- [ ] 6.3.4 Visibility badge: colored badge (e.g., green=public, yellow=member, orange=operator, red=admin)
- [ ] 6.3.5 Use `getConfig().oidc_enabled` to conditionally show admin controls and visibility badges
- [ ] 6.4 Admin inline channel management modals
- [ ] 6.4.1 Add Channel modal (follow tag editor pattern from `node-detail.js:25-71`)
- [ ] 6.4.2 Fields: name (text), key_hex (text, validated as hex), visibility (select: public/member/operator/admin), enabled (toggle)
- [ ] 6.4.3 Edit Channel modal: pre-populate fields, name read-only
- [ ] 6.4.4 Delete confirmation modal (follow `tagDeleteModal` pattern)
- [ ] 6.4.5 All modal actions gated behind `hasRole('admin')` — only rendered when admin
- [ ] 6.5 Conditional rendering modes
- [ ] 6.5.1 OIDC disabled: read-only cards, all channels public, no visibility badges, no add/edit/delete UI
- [ ] 6.5.2 OIDC enabled, not logged in: read-only cards, only public channels shown
- [ ] 6.5.3 OIDC enabled, non-admin: read-only cards, channels filtered by role, visibility badges shown
- [ ] 6.5.4 OIDC enabled, admin: full management — "Add Channel" button, edit/delete per card, visibility select in forms
- [ ] 6.6 Update channel filter dropdowns across the SPA
- [ ] 6.6.1 Dashboard channel filter: only show channels visible to user's role
- [ ] 6.6.2 Messages page channel filter: only show channels visible to user's role
- [ ] 6.6.3 When OIDC disabled, all channels appear in both dropdowns
- [ ] 6.7 Navigation placement — Channels between Messages and Members on all surfaces
- [ ] 6.7.1 `spa.html` desktop sidebar: insert `<li>` for `/channels` with `iconChannel()` between Messages and Members
- [ ] 6.7.2 `spa.html` mobile menu: insert `<li>` for `/channels` between Messages and Members
- [ ] 6.7.3 `app.js` dynamic nav: insert `if (features.channels !== false)` block between Messages and Members blocks
- [ ] 6.7.4 `app.js` route registration: `router.addRoute('/channels', pageHandler(pages.channels))` (no OIDC gate)
- [ ] 6.7.5 `app.js` `updatePageTitle()`: add 'channels' case
- [ ] 6.7.6 `home.js` hero card grid: insert `renderNavCard()` for Channels between Messages and Members
- [ ] 6.7.7 Add `--color-channels` CSS custom property in `app.css` for hero card accent color
- [ ] 6.7.8 Import `iconChannel` from `icons.js` in `channels.js`, `home.js`, `app.js`; use inline in `spa.html`
- [ ] 6.8 i18n
- [ ] 6.8.1 Add channel-related keys to `src/meshcore_hub/web/static/locales/en.json`:
- `entities.channel`, `entities.channels`
- `channels.title`, `channels.add_channel`, `channels.edit_channel`, `channels.delete_channel`
- `channels.name_label`, `channels.key_label`, `channels.visibility_label`, `channels.enabled_label`
- `channels.channel_hash_label`, `channels.qr_code_label`
- `channels.visibility_public`, `channels.visibility_member`, `channels.visibility_operator`, `channels.visibility_admin`
- `common.no_entity_found` for channels (composed pattern)
- [ ] 6.8.2 Add tests for new i18n keys in `tests/test_common/test_i18n.py`
- [ ] 6.8.3 Update `docs/i18n.md` with new keys and usage context
## 7. Seeding, Config Cleanup & Docs
- [ ] 7.1 Add `channels.yaml` seed support
- [ ] 7.1.1 Add channels seeding to `seed` command in `src/meshcore_hub/collector/cli.py`
- [ ] 7.1.2 Read `${SEED_HOME}/channels.yaml`
- [ ] 7.1.3 Parse shorthand format: `name: HEX` — value is `str`, treated as `key_hex`
- [ ] 7.1.4 Parse expanded format: `name: { key: HEX, enabled: true }` — value is `dict`
- [ ] 7.1.5 Parser distinguishes by type: `isinstance(value, str)` vs `isinstance(value, dict)`
- [ ] 7.1.6 No `visibility` field — always defaults to `public`
- [ ] 7.1.7 Upsert logic: update existing channel by `name`, insert new ones; never delete
- [ ] 7.2 Remove `COLLECTOR_CHANNEL_KEYS` and `COLLECTOR_INCLUDE_TEST_CHANNEL` from all files
- [ ] 7.2.1 `src/meshcore_hub/common/config.py` — `CollectorSettings`
- [ ] 7.2.2 `src/meshcore_hub/web/app.py` — `_parse_decoder_key_entries()`, `_build_channel_labels()`
- [ ] 7.2.3 `src/meshcore_hub/collector/cli.py` — `_run_collector_service()`
- [ ] 7.2.4 `src/meshcore_hub/collector/subscriber.py` — `Subscriber.__init__`, `create_subscriber()`, `run_collector()`
- [ ] 7.2.5 `.env.example` — lines 204, 208
- [ ] 7.2.6 `AGENTS.md` — env var list
- [ ] 7.3 Update documentation
- [ ] 7.3.1 Update `docs/seeding.md` with `channels.yaml` format, examples, and note that visibility is always `public`
- [ ] 7.3.2 Update `docs/upgrading.md` with migration guide:
- Run `meshcore-hub db upgrade`
- Convert `COLLECTOR_CHANNEL_KEYS` values to `channels.yaml` or DB rows via CLI
- Remove env var from `.env`
- Note all seeded channels are `public`
- [ ] 7.3.3 Update `AGENTS.md`: add `Channel` model to model list, `FEATURE_CHANNELS` to feature flags, `/channels` to API routes
- [ ] 7.3.4 Update `SCHEMAS.md` if channel event schemas are affected
- [ ] 7.3.5 Create example `channels.yaml` in `example/seed/channels.yaml`
## 8. Verification
- [ ] 8.1 Code quality
- [ ] 8.1.1 Run `pre-commit run --all-files` and fix all issues
- [ ] 8.1.2 Ensure no `except ValueError, TypeError:` patterns (use parenthesized tuples)
- [ ] 8.2 Component tests
- [ ] 8.2.1 Run `pytest tests/test_collector/` for collector-side changes
- [ ] 8.2.2 Run `pytest tests/test_api/` for API endpoints and message filtering
- [ ] 8.2.3 Run `pytest tests/test_web/` for web dashboard changes
- [ ] 8.2.4 Run `pytest tests/test_common/` for model and schema changes
- [ ] 8.2.5 Run `pytest tests/test_common/test_i18n.py` for i18n keys
- [ ] 8.2.6 Run full `pytest` to verify no regressions
- [ ] 8.3 Manual verification
- [ ] 8.3.1 Start collector with empty channels table — verify only `Public` channel messages decrypted and stored
- [ ] 8.3.2 Add a channel via CLI — verify collector picks it up at next refresh without restart
- [ ] 8.3.3 Seed channels from `channels.yaml` — verify rows created with `visibility=public`
- [ ] 8.3.4 Verify dashboard loads without errors and no features are lost
- [ ] 8.3.5 Verify Channels page renders correctly in all modes (OIDC disabled, logged out, admin)
- [ ] 8.3.6 Verify message filtering by role (different roles see different channel messages)
- [ ] 8.3.7 Verify QR code renders on channel cards
docs/seeding.md
+48 -1
View File
@@ -13,12 +13,14 @@ docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile seed up
This imports data from the following files (if they exist):
- `{SEED_HOME}/node_tags.yaml` - Node tag definitions
- `{SEED_HOME}/channels.yaml` - Channel decryption keys
## Directory Structure
```
seed/ # SEED_HOME (seed data files)
└── node_tags.yaml # Node tags for import
├── node_tags.yaml # Node tags for import
└── channels.yaml # Channel keys for import
data/ # DATA_HOME (runtime data)
└── collector/
@@ -59,3 +61,48 @@ Tag values can be:
```
Supported types: `string`, `number`, `boolean`
## Channels
Channel keys are used to decrypt encrypted mesh messages. They are stored in the database and loaded by the collector at startup, with periodic refresh.
### Channels YAML Format
Channels support two formats:
**Shorthand** (name → hex key string):
```yaml
MyChannel: AABBCCDD11223344AABBCCDD11223344
```
**Expanded** (name → dict with options):
```yaml
MyChannel:
key: AABBCCDD11223344AABBCCDD11223344
enabled: true
```
Key rules:
- Keys must be uppercase hex, 32 characters (AES-128) or 64 characters (AES-256)
- Seeded channels always have `visibility: community` — to set member/operator/admin visibility, use the CLI or API
- The `Public` and `test` built-in keys are always loaded into the decoder regardless of database contents
- Test channel messages are only stored when a `test` channel row exists in the database with `enabled: true`
### Managing Channels via CLI
```bash
# List all channels
meshcore-hub collector channel list
# Add a channel
meshcore-hub collector channel add --name MyChannel --key AABBCCDD11223344AABBCCDD11223344
# Enable/disable a channel
meshcore-hub collector channel enable --name MyChannel
meshcore-hub collector channel disable --name MyChannel
# Remove a channel
meshcore-hub collector channel remove --name MyChannel
```
docs/upgrading.md
+35 -19
View File
@@ -2,38 +2,54 @@
This guide covers upgrading from a previous MeshCore Hub release to the current version. Check the relevant version section below before upgrading.
## v0.12.0
## v0.11.0
### Advertisement Route Type & Deduplication Improvements
### Channel Visibility Rename: "public" → "community"
This release adds route type tracking and improves advertisement deduplication to better distinguish between flood and zero-hop (local) advertisements.
The channel visibility level `"public"` has been renamed to `"community"` to avoid confusion with MeshCore's concept of public channels. All MeshCore channels are private (encrypted) in protocol terms, so "community" better reflects the access level.
**New database columns on `advertisements` table:**
The Alembic migration automatically updates existing `visibility='public'` rows to `visibility='community'`. No manual database changes are required.
API consumers that filter channels by `visibility=public` must update to `visibility=community`.
### Database-Backed Channel Keys
Channel decryption keys are now managed via the `channels` database table instead of the `COLLECTOR_CHANNEL_KEYS` environment variable. This enables runtime key management, permission-based visibility, and a Channels dashboard page.
**New database table: `channels`**
| Column | Type | Description |
|--------|------|-------------|
| `route_type` | `VARCHAR(20), nullable` | Route type: `flood`, `transport_flood`, `direct`, `transport_direct` |
| `advert_timestamp` | `DATETIME, nullable` | Node's own Unix timestamp from the advert payload |
| `id` | `VARCHAR(36), PK` | UUID primary key |
| `name` | `VARCHAR(100), UNIQUE` | Channel display name |
| `key_hex` | `VARCHAR(64), UNIQUE` | Uppercase hex key (32 or 64 chars) |
| `channel_hash` | `VARCHAR(2)` | First byte of SHA-256 of key |
| `visibility` | `VARCHAR(20)` | `community`, `member`, `operator`, or `admin` |
| `enabled` | `BOOLEAN` | Whether the channel is active |
| `created_at`, `updated_at` | `DATETIME` | Timestamps |
Both columns are nullable — existing records will have `NULL` values. The Alembic migration adds these columns automatically.
**Removed environment variables:**
- `COLLECTOR_CHANNEL_KEYS` — replaced by database channels table
- `COLLECTOR_INCLUDE_TEST_CHANNEL` — replaced by presence of a `test` channel row in the database
**Default API filter change:**
**New environment variables:**
- `CHANNEL_REFRESH_INTERVAL_SECONDS` — seconds between key refresh (default: `300`)
- `FEATURE_CHANNELS` — enable/disable the /channels page (default: `true`)
`GET /api/v1/advertisements` now defaults to `route_type=flood,transport_flood`, showing only flood advertisements. Existing records with `route_type=NULL` are included in all default queries to avoid hiding historical data. Pass `route_type=all` to see all types.
**Migration steps:**
**Dashboard metrics now flood-only:**
1. Run `meshcore-hub db upgrade` to create the `channels` table and update visibility values
2. Convert any `COLLECTOR_CHANNEL_KEYS` values to either:
- A `channels.yaml` seed file in `SEED_HOME` (see `docs/seeding.md`)
- Database rows via CLI: `meshcore-hub collector channel add --name X --key HEX`
3. Remove `COLLECTOR_CHANNEL_KEYS` and `COLLECTOR_INCLUDE_TEST_CHANNEL` from your `.env`
4. If you previously relied on test channel messages, add a test channel: `meshcore-hub collector channel add --name test --key 9CD8FCF22A47333B591D96A2B848B73F`
All dashboard advertisement counts (`total_advertisements`, `advertisements_24h`, `advertisements_7d`, `recent_advertisements`, and `/activity`) now count only flood/transport_flood adverts plus NULL (historical records).
**Test channel behavior change:** Test channel messages (channel_idx 217) are now discarded by default unless a `test` channel row exists in the database with `enabled=true`. Previously this was controlled by `COLLECTOR_INCLUDE_TEST_CHANNEL`.
**Deduplication bucket increased from 120s to 300s:**
### Advertisement Route Type & Deduplication
Both `compute_advertisement_hash()` and `compute_telemetry_hash()` now use a 5-minute (300-second) deduplication bucket instead of the previous 2-minute (120-second) bucket. This reduces duplicate records when multiple observers report the same event within a 5-minute window.
**Advertisement deduplication now uses node timestamp:**
When available, the node's own `advert_timestamp` is used for deduplication bucketing instead of `received_at`. This means the same flood advertisement observed by multiple receivers will correctly deduplicate even if received several minutes apart. Node timestamps that deviate by more than 4 hours from `received_at` are rejected for bucketing (the raw value is still stored).
## v0.11.0
Advertisement route type tracking and improved deduplication are included. New `route_type` and `advert_timestamp` columns are added to the `advertisements` table automatically by the migration. The API defaults to showing flood advertisements only. Deduplication uses a 5-minute bucket with node timestamps when available.
### Async SQLite Foreign Key Fix
example/seed/channels.yaml
+14
View File
@@ -0,0 +1,14 @@
# Channel seed data for MeshCore Hub
#
# Format options:
# Shorthand: name: HEX_KEY
# Expanded: name: { key: HEX_KEY, enabled: true }
#
# Visibility is always 'public' for seeded channels.
# To set member/operator/admin visibility, use the CLI or API.
#
# Example:
# MyChannel: AABBCCDD11223344AABBCCDD11223344
# PrivateChannel:
# key: 11223344AABBCCDD11223344AABBCCDD11223344AABBCCDD11223344AABBCCDD
# enabled: true
src/meshcore_hub/api/channel_visibility.py
+68
View File
@@ -0,0 +1,68 @@
"""Shared channel visibility helpers for API routes.
Resolves user roles from proxy-injected headers and determines which
channel indices are visible based on channel visibility levels.
"""
from fastapi import Request
from sqlalchemy import select
from meshcore_hub.api.dependencies import DbSession
from meshcore_hub.common.models.channel import Channel
VISIBILITY_LEVELS = {"community": 0, "member": 1, "operator": 2, "admin": 3}
def resolve_user_role(request: Request) -> str | None:
"""Resolve the user's highest role from X-User-Roles header."""
roles_header = request.headers.get("x-user-roles", "")
if not roles_header:
return None
roles = {r.strip() for r in roles_header.split(",") if r.strip()}
admin_role = getattr(request.app.state, "oidc_role_admin", "admin")
operator_role = getattr(request.app.state, "oidc_role_operator", "operator")
member_role = getattr(request.app.state, "oidc_role_member", "member")
if admin_role in roles:
return "admin"
if operator_role in roles:
return "operator"
if member_role in roles:
return "member"
return None
def get_max_visibility_level(role: str | None) -> int:
"""Get the maximum visibility level for a given role.
Returns 0 for anonymous users (community only).
"""
if role is None:
return 0
return VISIBILITY_LEVELS.get(role, 0)
def get_visible_channel_indices(
session: DbSession,
max_level: int,
) -> set[int]:
"""Get set of visible channel_idx values based on visibility level.
Only returns indices for channels whose visibility level is at most
max_level (lower number = more permissive). The built-in Public
channel (idx 17) is always included.
"""
channels = session.execute(select(Channel)).scalars().all()
visible: set[int] = set()
for ch in channels:
level = VISIBILITY_LEVELS.get(ch.visibility, 0)
if level <= max_level:
idx = int(ch.channel_hash, 16)
visible.add(idx)
visible.add(17) # Built-in Public channel is always visible
return visible
def get_all_known_channel_indices(session: DbSession) -> set[int]:
"""Get set of all known channel_idx values from DB."""
channels = session.execute(select(Channel)).scalars().all()
return {int(ch.channel_hash, 16) for ch in channels}
src/meshcore_hub/api/routes/__init__.py
+2
View File
@@ -11,6 +11,7 @@ from meshcore_hub.api.routes.telemetry import router as telemetry_router
from meshcore_hub.api.routes.dashboard import router as dashboard_router
from meshcore_hub.api.routes.user_profiles import router as user_profiles_router
from meshcore_hub.api.routes.adoptions import router as adoptions_router
from meshcore_hub.api.routes.channels import router as channels_router
api_router = APIRouter()
@@ -28,3 +29,4 @@ api_router.include_router(telemetry_router, prefix="/telemetry", tags=["Telemetr
api_router.include_router(dashboard_router, prefix="/dashboard", tags=["Dashboard"])
api_router.include_router(user_profiles_router, prefix="/user", tags=["User"])
api_router.include_router(adoptions_router, prefix="/adoptions", tags=["Adoptions"])
api_router.include_router(channels_router, prefix="/channels", tags=["Channels"])
src/meshcore_hub/api/routes/channels.py
+158
View File
@@ -0,0 +1,158 @@
"""Channel API routes."""
from fastapi import APIRouter, HTTPException, Request
from sqlalchemy import select
from meshcore_hub.api.auth import RequireAdmin, RequireRead
from meshcore_hub.api.channel_visibility import (
VISIBILITY_LEVELS,
get_max_visibility_level,
resolve_user_role,
)
from meshcore_hub.api.dependencies import DbSession
from meshcore_hub.common.models.channel import Channel
from meshcore_hub.common.schemas.channels import (
ChannelCreate,
ChannelList,
ChannelRead,
ChannelUpdate,
)
router = APIRouter()
def _channel_to_read(channel: Channel, include_key: bool = False) -> ChannelRead:
"""Convert a Channel model to ChannelRead schema."""
return ChannelRead(
id=channel.id,
name=channel.name,
channel_hash=channel.channel_hash,
visibility=channel.visibility,
enabled=channel.enabled,
masked_key=channel.masked_key,
key_hex=channel.key_hex if include_key else None,
created_at=channel.created_at,
updated_at=channel.updated_at,
)
@router.get("", response_model=ChannelList)
async def list_channels(
_: RequireRead,
session: DbSession,
request: Request,
) -> ChannelList:
"""List channels, filtered by user role visibility.
When no OIDC roles are present (OIDC disabled or not logged in),
returns only public channels.
"""
role = resolve_user_role(request)
query = select(Channel).order_by(Channel.name)
channels = session.execute(query).scalars().all()
max_level = get_max_visibility_level(role)
filtered = []
for ch in channels:
level = VISIBILITY_LEVELS.get(ch.visibility, 0)
if level <= max_level:
filtered.append(_channel_to_read(ch, include_key=True))
return ChannelList(items=filtered, total=len(filtered))
@router.post("", response_model=ChannelRead, status_code=201)
async def create_channel(
__: RequireAdmin,
session: DbSession,
body: ChannelCreate,
) -> ChannelRead:
"""Create a new channel (admin only)."""
existing = session.execute(
select(Channel).where(Channel.name == body.name)
).scalar_one_or_none()
if existing:
raise HTTPException(
status_code=409, detail=f"Channel '{body.name}' already exists"
)
existing_key = session.execute(
select(Channel).where(Channel.key_hex == body.key_hex)
).scalar_one_or_none()
if existing_key:
raise HTTPException(
status_code=409, detail="Key already in use by another channel"
)
channel_hash = Channel.compute_channel_hash(body.key_hex)
channel = Channel(
name=body.name,
key_hex=body.key_hex,
channel_hash=channel_hash,
visibility=body.visibility,
enabled=body.enabled,
)
session.add(channel)
session.commit()
session.refresh(channel)
return _channel_to_read(channel, include_key=True)
@router.put("/{channel_id}", response_model=ChannelRead)
async def update_channel(
__: RequireAdmin,
session: DbSession,
channel_id: str,
body: ChannelUpdate,
) -> ChannelRead:
"""Update a channel (admin only, name is immutable)."""
channel = session.execute(
select(Channel).where(Channel.id == channel_id)
).scalar_one_or_none()
if not channel:
raise HTTPException(status_code=404, detail="Channel not found")
if body.key_hex is not None:
existing_key = session.execute(
select(Channel).where(
Channel.key_hex == body.key_hex, Channel.id != channel_id
)
).scalar_one_or_none()
if existing_key:
raise HTTPException(
status_code=409, detail="Key already in use by another channel"
)
channel.key_hex = body.key_hex
channel.channel_hash = Channel.compute_channel_hash(body.key_hex)
if body.visibility is not None:
channel.visibility = body.visibility
if body.enabled is not None:
channel.enabled = body.enabled
session.commit()
session.refresh(channel)
return _channel_to_read(channel, include_key=True)
@router.delete("/{channel_id}", status_code=204)
async def delete_channel(
__: RequireAdmin,
session: DbSession,
channel_id: str,
) -> None:
"""Delete a channel (admin only)."""
channel = session.execute(
select(Channel).where(Channel.id == channel_id)
).scalar_one_or_none()
if not channel:
raise HTTPException(status_code=404, detail="Channel not found")
session.delete(channel)
session.commit()
src/meshcore_hub/api/routes/dashboard.py
+42 -5
View File
@@ -2,11 +2,16 @@
from datetime import datetime, timedelta, timezone
from fastapi import APIRouter
from fastapi import APIRouter, Request
from sqlalchemy import func, or_, select
from sqlalchemy.sql.elements import ColumnElement
from meshcore_hub.api.auth import RequireRead
from meshcore_hub.api.channel_visibility import (
get_max_visibility_level,
get_visible_channel_indices,
resolve_user_role,
)
from meshcore_hub.api.dependencies import DbSession
from meshcore_hub.common.models import (
Advertisement,
@@ -47,6 +52,7 @@ def _flood_only_filter(
async def get_stats(
_: RequireRead,
session: DbSession,
request: Request,
) -> DashboardStats:
"""Get dashboard statistics."""
now = datetime.now(timezone.utc)
@@ -54,6 +60,21 @@ async def get_stats(
yesterday = now - timedelta(days=1)
seven_days_ago = now - timedelta(days=7)
# Resolve channel visibility
role = resolve_user_role(request)
max_level = get_max_visibility_level(role)
visible_indices = get_visible_channel_indices(session, max_level)
# Build channel message visibility filter
def _channel_visible_filter(
model: type[Message] = Message,
) -> ColumnElement[bool]:
return or_(
model.message_type != "channel",
model.channel_idx.is_(None),
model.channel_idx.in_(visible_indices),
)
# Total nodes
total_nodes = session.execute(select(func.count()).select_from(Node)).scalar() or 0
@@ -67,7 +88,10 @@ async def get_stats(
# Total messages
total_messages = (
session.execute(select(func.count()).select_from(Message)).scalar() or 0
session.execute(
select(func.count()).select_from(Message).where(_channel_visible_filter())
).scalar()
or 0
)
# Messages today
@@ -76,6 +100,7 @@ async def get_stats(
select(func.count())
.select_from(Message)
.where(Message.received_at >= today_start)
.where(_channel_visible_filter())
).scalar()
or 0
)
@@ -118,6 +143,7 @@ async def get_stats(
select(func.count())
.select_from(Message)
.where(Message.received_at >= seven_days_ago)
.where(_channel_visible_filter())
).scalar()
or 0
)
@@ -171,11 +197,12 @@ async def get_stats(
for ad in recent_ads
]
# Channel message counts
# Channel message counts (only visible channels)
channel_counts_query = (
select(Message.channel_idx, func.count())
.where(Message.message_type == "channel")
.where(Message.channel_idx.isnot(None))
.where(Message.channel_idx.in_(visible_indices))
.group_by(Message.channel_idx)
)
channel_results = session.execute(channel_counts_query).all()
@@ -336,6 +363,7 @@ async def get_activity(
async def get_message_activity(
_: RequireRead,
session: DbSession,
request: Request,
days: int = 30,
) -> MessageActivity:
"""Get daily message activity for the specified period.
@@ -349,11 +377,13 @@ async def get_message_activity(
days = min(days, 90)
now = datetime.now(timezone.utc)
# End at start of today (exclude today's incomplete data)
end_date = now.replace(hour=0, minute=0, second=0, microsecond=0)
start_date = end_date - timedelta(days=days)
# Query message counts grouped by date
role = resolve_user_role(request)
max_level = get_max_visibility_level(role)
visible_indices = get_visible_channel_indices(session, max_level)
date_expr = func.date(Message.received_at)
query = (
@@ -363,6 +393,13 @@ async def get_message_activity(
)
.where(Message.received_at >= start_date)
.where(Message.received_at < end_date)
.where(
or_(
Message.message_type != "channel",
Message.channel_idx.is_(None),
Message.channel_idx.in_(visible_indices),
)
)
.group_by(date_expr)
.order_by(date_expr)
)
src/meshcore_hub/api/routes/messages.py
+29 -2
View File
@@ -3,11 +3,16 @@
from datetime import datetime
from typing import Optional
from fastapi import APIRouter, HTTPException, Query
from sqlalchemy import func, select
from fastapi import APIRouter, HTTPException, Query, Request
from sqlalchemy import func, or_, select
from sqlalchemy.orm import aliased, selectinload
from meshcore_hub.api.auth import RequireRead
from meshcore_hub.api.channel_visibility import (
get_max_visibility_level,
get_visible_channel_indices,
resolve_user_role,
)
from meshcore_hub.api.dependencies import DbSession
from meshcore_hub.api.observer_utils import fetch_observers_for_events
from meshcore_hub.common.models import Message, Node, NodeTag
@@ -32,6 +37,7 @@ def _get_tag_name(node: Optional[Node]) -> Optional[str]:
async def list_messages(
_: RequireRead,
session: DbSession,
request: Request,
message_type: Optional[str] = Query(None, description="Filter by message type"),
pubkey_prefix: Optional[str] = Query(None, description="Filter by sender prefix"),
channel_idx: Optional[int] = Query(None, description="Filter by channel"),
@@ -79,6 +85,18 @@ async def list_messages(
if search:
query = query.where(Message.text.ilike(f"%{search}%"))
# Apply channel visibility filtering
role = resolve_user_role(request)
max_level = get_max_visibility_level(role)
visible_indices = get_visible_channel_indices(session, max_level)
query = query.where(
or_(
Message.message_type != "channel",
Message.channel_idx.is_(None),
Message.channel_idx.in_(visible_indices),
)
)
# Get total count
count_query = select(func.count()).select_from(query.subquery())
total = session.execute(count_query).scalar() or 0
@@ -211,6 +229,7 @@ async def list_messages(
async def get_message(
_: RequireRead,
session: DbSession,
request: Request,
message_id: str,
) -> MessageRead:
"""Get a single message by ID."""
@@ -227,6 +246,14 @@ async def get_message(
message, observer_pk = result
# Apply channel visibility filter
if message.message_type == "channel" and message.channel_idx is not None:
role = resolve_user_role(request)
max_level = get_max_visibility_level(role)
visible_indices = get_visible_channel_indices(session, max_level)
if message.channel_idx not in visible_indices:
raise HTTPException(status_code=404, detail="Message not found")
# Fetch observers for this message
observers = []
if message.event_hash:
src/meshcore_hub/collector/cli.py
+244 -4
View File
@@ -260,8 +260,7 @@ def _run_collector_service(
click.echo("")
builtin_keys = len(LetsMeshPacketDecoder.BUILTIN_CHANNEL_KEYS)
env_keys = len(settings.collector_channel_keys_list)
click.echo(f"Packet decoder: {builtin_keys} built-in keys, {env_keys} from .env")
click.echo(f"Packet decoder: {builtin_keys} built-in keys, loading from database")
click.echo("")
click.echo("Starting MQTT subscriber...")
@@ -281,8 +280,7 @@ def _run_collector_service(
cleanup_interval_hours=settings.data_retention_interval_hours,
node_cleanup_enabled=settings.node_cleanup_enabled,
node_cleanup_days=settings.node_cleanup_days,
channel_keys=settings.collector_channel_keys_list,
include_test_channel=settings.collector_include_test_channel,
channel_refresh_interval_seconds=settings.channel_refresh_interval_seconds,
)
@@ -309,6 +307,157 @@ def run_cmd(ctx: click.Context) -> None:
)
@collector.group("channel")
@click.pass_context
def channel_group(ctx: click.Context) -> None:
"""Manage decryption channels in the database."""
pass
@channel_group.command("list")
@click.pass_context
def channel_list_cmd(ctx: click.Context) -> None:
"""List all channels in the database."""
configure_logging(level=ctx.obj["log_level"])
from meshcore_hub.common.database import DatabaseManager
from meshcore_hub.common.models.channel import Channel
db = DatabaseManager(ctx.obj["database_url"])
try:
with db.session_scope() as session:
channels = session.query(Channel).order_by(Channel.name).all()
if not channels:
click.echo("No channels found.")
else:
click.echo(
f"{'Name':<20} {'Key':<16} {'Hash':<6} "
f"{'Visibility':<12} {'Enabled'}"
)
click.echo("-" * 70)
for ch in channels:
click.echo(
f"{ch.name:<20} {ch.masked_key:<16} {ch.channel_hash:<6} "
f"{ch.visibility:<12} {'Yes' if ch.enabled else 'No'}"
)
finally:
db.dispose()
@channel_group.command("add")
@click.option("--name", required=True, help="Channel display name")
@click.option(
"--key", "key_hex", required=True, help="Channel key as hex (32 or 64 chars)"
)
@click.option(
"--visibility",
type=click.Choice(["community", "member", "operator", "admin"]),
default="community",
help="Channel visibility level (default: community)",
)
@click.pass_context
def channel_add_cmd(
ctx: click.Context,
name: str,
key_hex: str,
visibility: str,
) -> None:
"""Add a new channel to the database."""
configure_logging(level=ctx.obj["log_level"])
from meshcore_hub.common.database import DatabaseManager
from meshcore_hub.common.models.channel import Channel
db = DatabaseManager(ctx.obj["database_url"])
try:
with db.session_scope() as session:
existing = session.query(Channel).filter(Channel.name == name).first()
if existing:
click.echo(f"Error: Channel '{name}' already exists.", err=True)
return
channel = Channel(
name=name,
key_hex=key_hex.upper(),
channel_hash=Channel.compute_channel_hash(key_hex.upper()),
visibility=visibility,
enabled=True,
)
session.add(channel)
click.echo(f"Channel '{name}' added (hash={channel.channel_hash})")
finally:
db.dispose()
@channel_group.command("remove")
@click.option("--name", required=True, help="Channel name to remove")
@click.pass_context
def channel_remove_cmd(ctx: click.Context, name: str) -> None:
"""Remove a channel from the database."""
configure_logging(level=ctx.obj["log_level"])
from meshcore_hub.common.database import DatabaseManager
from meshcore_hub.common.models.channel import Channel
db = DatabaseManager(ctx.obj["database_url"])
try:
with db.session_scope() as session:
channel = session.query(Channel).filter(Channel.name == name).first()
if not channel:
click.echo(f"Error: Channel '{name}' not found.", err=True)
return
session.delete(channel)
click.echo(f"Channel '{name}' removed.")
finally:
db.dispose()
@channel_group.command("enable")
@click.option("--name", required=True, help="Channel name to enable")
@click.pass_context
def channel_enable_cmd(ctx: click.Context, name: str) -> None:
"""Enable a channel."""
configure_logging(level=ctx.obj["log_level"])
from meshcore_hub.common.database import DatabaseManager
from meshcore_hub.common.models.channel import Channel
db = DatabaseManager(ctx.obj["database_url"])
try:
with db.session_scope() as session:
channel = session.query(Channel).filter(Channel.name == name).first()
if not channel:
click.echo(f"Error: Channel '{name}' not found.", err=True)
return
channel.enabled = True
click.echo(f"Channel '{name}' enabled.")
finally:
db.dispose()
@channel_group.command("disable")
@click.option("--name", required=True, help="Channel name to disable")
@click.pass_context
def channel_disable_cmd(ctx: click.Context, name: str) -> None:
"""Disable a channel."""
configure_logging(level=ctx.obj["log_level"])
from meshcore_hub.common.database import DatabaseManager
from meshcore_hub.common.models.channel import Channel
db = DatabaseManager(ctx.obj["database_url"])
try:
with db.session_scope() as session:
channel = session.query(Channel).filter(Channel.name == name).first()
if not channel:
click.echo(f"Error: Channel '{name}' not found.", err=True)
return
channel.enabled = False
click.echo(f"Channel '{name}' disabled.")
finally:
db.dispose()
@collector.command("seed")
@click.option(
"--no-create-nodes",
@@ -408,9 +557,100 @@ def _run_seed_import(
elif verbose:
click.echo(f"\nNo node_tags.yaml found in {seed_home}")
# Import channels if file exists
channels_file = Path(seed_home) / "channels.yaml"
if channels_file.exists():
if verbose:
click.echo(f"\nImporting channels from: {channels_file}")
channel_stats = _import_channels(
file_path=str(channels_file),
db=db,
verbose=verbose,
)
if verbose:
click.echo(
f" Channels: {channel_stats['created']} created, "
f"{channel_stats['updated']} updated"
)
if channel_stats["errors"]:
for error in channel_stats["errors"]: # type: ignore[union-attr]
click.echo(f" Error: {error}", err=True)
imported_any = True
elif verbose:
click.echo(f"\nNo channels.yaml found in {seed_home}")
return imported_any
def _import_channels(
file_path: str,
db: "DatabaseManager",
verbose: bool = False,
) -> dict[str, int | list[str]]:
"""Import channels from a YAML file.
Supports two formats:
- Shorthand: name: HEX (value is string, treated as key_hex)
- Expanded: name: { key: HEX, enabled: true } (value is dict)
Visibility is always 'public' for seeded channels.
Returns:
Dict with 'created', 'updated', and 'errors' counts.
"""
import yaml
from meshcore_hub.common.models.channel import Channel
created: int = 0
updated: int = 0
errors: list[str] = []
with open(file_path) as f:
data = yaml.safe_load(f)
if not data or not isinstance(data, dict):
return {"created": created, "updated": updated, "errors": errors}
with db.session_scope() as session:
for name, value in data.items():
try:
if isinstance(value, str):
key_hex = value.strip().upper()
enabled = True
elif isinstance(value, dict):
key_hex = value.get("key", "").strip().upper()
enabled = value.get("enabled", True)
else:
errors.append(f"Invalid format for channel '{name}'")
continue
if not key_hex:
errors.append(f"Empty key for channel '{name}'")
continue
existing = session.query(Channel).filter(Channel.name == name).first()
if existing:
existing.key_hex = key_hex
existing.channel_hash = Channel.compute_channel_hash(key_hex)
existing.enabled = enabled
updated += 1
else:
channel = Channel(
name=name,
key_hex=key_hex,
channel_hash=Channel.compute_channel_hash(key_hex),
visibility="community",
enabled=enabled,
)
session.add(channel)
created += 1
except Exception as e:
errors.append(f"Channel '{name}': {e}")
return {"created": created, "updated": updated, "errors": errors}
@collector.command("import-tags")
@click.argument("file", type=click.Path(), required=False, default=None)
@click.option(
src/meshcore_hub/collector/letsmesh_decoder.py
+44 -4
View File
@@ -10,6 +10,7 @@ from __future__ import annotations
import hashlib
import logging
import string
import threading
from typing import Any, NamedTuple
from meshcoredecoder import MeshCoreDecoder
@@ -31,7 +32,7 @@ class LetsMeshPacketDecoder:
BUILTIN_CHANNEL_KEYS: tuple[tuple[str, str], ...] = (
("Public", "8B3387E9C5CDEA6AC9E5EDBAA115CD72"),
("test", "9CD8FCF22A47333B591D96A2B848B73F"),
("Test", "9CD8FCF22A47333B591D96A2B848B73F"),
)
TEST_CHANNEL_HASH: str = "D9"
@@ -41,6 +42,7 @@ class LetsMeshPacketDecoder:
self,
channel_keys: list[str] | None = None,
) -> None:
self._state_lock = threading.Lock()
self._channel_key_infos = self._normalize_channel_keys(channel_keys or [])
self._channel_keys = [info.key_hex for info in self._channel_key_infos]
self._channel_names_by_hash = {
@@ -67,6 +69,39 @@ class LetsMeshPacketDecoder:
key_store.add_channel_secrets(self._channel_keys)
return key_store
def reload_keys(self, channel_keys: list[str]) -> None:
"""Reload channel keys from a new key list (thread-safe).
Rebuilds the key store and channel name map without discarding the
decode cache. The state lock is held only during the atomic swap
of ``_key_store`` and ``_channel_names_by_hash``, not during key
normalization or KeyStore construction.
Args:
channel_keys: New list of channel key entries to load.
"""
new_infos = self._normalize_channel_keys(channel_keys)
new_keys = [info.key_hex for info in new_infos]
new_names = {info.channel_hash: info.label for info in new_infos if info.label}
new_store = MeshCoreKeyStore()
if new_keys:
new_store.add_channel_secrets(new_keys)
with self._state_lock:
self._channel_key_infos = new_infos
self._channel_keys = new_keys
self._channel_names_by_hash = new_names
self._key_store = new_store
logger.debug(
"LetsMesh decoder reloaded: %d channel keys (%s)",
len(new_infos),
", ".join(
f"{info.label or 'unlabeled'}=0x{info.channel_hash}"
for info in new_infos
),
)
@classmethod
def _normalize_channel_keys(cls, values: list[str]) -> list[ChannelKey]:
"""Normalize key list (labels + key + channel hash, deduplicated)."""
@@ -160,12 +195,15 @@ class LetsMeshPacketDecoder:
if not isinstance(channel_hash, str):
return None
return self._channel_names_by_hash.get(channel_hash.upper())
with self._state_lock:
return self._channel_names_by_hash.get(channel_hash.upper())
def channel_labels_by_index(self) -> dict[int, str]:
"""Return channel labels keyed by numeric channel index (0-255)."""
labels: dict[int, str] = {}
for info in self._channel_key_infos:
with self._state_lock:
infos = list(self._channel_key_infos)
for info in infos:
if not info.label:
continue
@@ -202,8 +240,10 @@ class LetsMeshPacketDecoder:
def _decode_raw(self, raw_hex: str) -> dict[str, Any] | None:
"""Decode raw packet hex with native Python decoder (cached per packet hex)."""
try:
with self._state_lock:
key_store = self._key_store
options = DecryptionOptions(
key_store=self._key_store,
key_store=key_store,
attempt_decryption=True,
)
result = MeshCoreDecoder.decode(raw_hex, options)
src/meshcore_hub/collector/subscriber.py
+116 -18
View File
@@ -47,8 +47,7 @@ class Subscriber(LetsMeshNormalizer):
cleanup_interval_hours: int = 24,
node_cleanup_enabled: bool = False,
node_cleanup_days: int = 90,
channel_keys: list[str] | None = None,
include_test_channel: bool = False,
channel_refresh_interval_seconds: int = 300,
):
"""Initialize subscriber.
@@ -61,8 +60,7 @@ class Subscriber(LetsMeshNormalizer):
cleanup_interval_hours: Hours between cleanup runs
node_cleanup_enabled: Enable automatic cleanup of inactive nodes
node_cleanup_days: Remove nodes not seen for this many days
channel_keys: Optional channel keys for decrypting group text
include_test_channel: Include built-in test channel messages
channel_refresh_interval_seconds: Seconds between channel key refresh
"""
self.mqtt = mqtt_client
self.db = db_manager
@@ -85,10 +83,14 @@ class Subscriber(LetsMeshNormalizer):
self._node_cleanup_days = node_cleanup_days
self._cleanup_thread: Optional[threading.Thread] = None
self._last_cleanup: Optional[datetime] = None
# Channel key refresh
self._channel_refresh_interval_seconds = channel_refresh_interval_seconds
self._channel_refresh_thread: Optional[threading.Thread] = None
# Load initial channel keys from database
self._include_test_channel = self._load_channel_keys_from_db()
self._letsmesh_decoder = LetsMeshPacketDecoder(
channel_keys=channel_keys,
channel_keys=self._db_channel_keys,
)
self._include_test_channel = include_test_channel
@property
def is_healthy(self) -> bool:
@@ -99,6 +101,68 @@ class Subscriber(LetsMeshNormalizer):
"""
return self._running and self._mqtt_connected and self._db_connected
def _load_channel_keys_from_db(self) -> bool:
"""Load channel keys from the database (synchronous).
Queries enabled channels, merges with built-in keys, and
determines whether the test channel should be included.
Returns:
True if test channel should be included (DB row exists with enabled=True).
"""
self._db_channel_keys: list[str] = []
include_test = False
try:
from meshcore_hub.common.models.channel import Channel
with self.db.session_scope() as session:
channels = (
session.query(Channel)
.filter(Channel.enabled == True) # noqa: E712
.all()
)
for ch in channels:
self._db_channel_keys.append(f"{ch.name}={ch.key_hex}")
if ch.name.lower() == "test":
include_test = True
logger.info(
"Loaded %d channel keys from database (include_test=%s)",
len(self._db_channel_keys),
include_test,
)
except Exception as e:
logger.warning("Failed to load channel keys from database: %s", e)
self._db_channel_keys = []
return include_test
def _refresh_channel_keys_from_db(self) -> None:
"""Refresh channel keys from the database and reload the decoder."""
new_keys: list[str] = []
include_test = False
try:
from meshcore_hub.common.models.channel import Channel
with self.db.session_scope() as session:
channels = (
session.query(Channel)
.filter(Channel.enabled == True) # noqa: E712
.all()
)
for ch in channels:
new_keys.append(f"{ch.name}={ch.key_hex}")
if ch.name.lower() == "test":
include_test = True
self._db_channel_keys = new_keys
self._include_test_channel = include_test
self._letsmesh_decoder.reload_keys(new_keys)
logger.info(
"Refreshed %d channel keys from database (include_test=%s)",
len(new_keys),
include_test,
)
except Exception as e:
logger.error("Failed to refresh channel keys from database: %s", e)
def get_health_status(self) -> dict[str, Any]:
"""Get detailed health status.
@@ -364,6 +428,40 @@ class Subscriber(LetsMeshNormalizer):
if self._cleanup_thread.is_alive():
logger.warning("Cleanup scheduler thread did not stop cleanly")
def _start_channel_refresh_scheduler(self) -> None:
"""Start background thread for periodic channel key refresh."""
interval = self._channel_refresh_interval_seconds
if interval <= 0:
logger.info("Channel key refresh is disabled (interval=0)")
return
logger.info("Starting channel refresh scheduler (interval=%ds)", interval)
def run_refresh_loop() -> None:
"""Periodically refresh channel keys from database."""
while self._running:
for _ in range(interval):
if not self._running:
break
time.sleep(1)
if self._running:
try:
self._refresh_channel_keys_from_db()
except Exception as e:
logger.error("Channel refresh error: %s", e, exc_info=True)
self._channel_refresh_thread = threading.Thread(
target=run_refresh_loop, daemon=True, name="channel-refresh"
)
self._channel_refresh_thread.start()
def _stop_channel_refresh_scheduler(self) -> None:
"""Stop the channel refresh scheduler thread."""
if self._channel_refresh_thread and self._channel_refresh_thread.is_alive():
self._channel_refresh_thread.join(timeout=5.0)
if self._channel_refresh_thread.is_alive():
logger.warning("Channel refresh thread did not stop cleanly")
def start(self) -> None:
"""Start the subscriber."""
logger.info("Starting collector subscriber")
@@ -428,6 +526,9 @@ class Subscriber(LetsMeshNormalizer):
# Start cleanup scheduler if configured
self._start_cleanup_scheduler()
# Start channel key refresh scheduler
self._start_channel_refresh_scheduler()
# Start health reporter for Docker health checks
self._health_reporter = HealthReporter(
component="collector",
@@ -463,6 +564,9 @@ class Subscriber(LetsMeshNormalizer):
# Stop cleanup scheduler
self._stop_cleanup_scheduler()
# Stop channel refresh scheduler
self._stop_channel_refresh_scheduler()
# Stop webhook processor
self._stop_webhook_processor()
@@ -495,8 +599,7 @@ def create_subscriber(
cleanup_interval_hours: int = 24,
node_cleanup_enabled: bool = False,
node_cleanup_days: int = 90,
channel_keys: list[str] | None = None,
include_test_channel: bool = False,
channel_refresh_interval_seconds: int = 300,
) -> Subscriber:
"""Create a configured subscriber instance.
@@ -516,8 +619,7 @@ def create_subscriber(
cleanup_interval_hours: Hours between cleanup runs
node_cleanup_enabled: Enable automatic cleanup of inactive nodes
node_cleanup_days: Remove nodes not seen for this many days
channel_keys: Optional channel keys for decrypting group text
include_test_channel: Include built-in test channel messages
channel_refresh_interval_seconds: Seconds between channel key refresh
Returns:
Configured Subscriber instance
@@ -550,8 +652,7 @@ def create_subscriber(
cleanup_interval_hours=cleanup_interval_hours,
node_cleanup_enabled=node_cleanup_enabled,
node_cleanup_days=node_cleanup_days,
channel_keys=channel_keys,
include_test_channel=include_test_channel,
channel_refresh_interval_seconds=channel_refresh_interval_seconds,
)
# Register handlers
@@ -578,8 +679,7 @@ def run_collector(
cleanup_interval_hours: int = 24,
node_cleanup_enabled: bool = False,
node_cleanup_days: int = 90,
channel_keys: list[str] | None = None,
include_test_channel: bool = False,
channel_refresh_interval_seconds: int = 300,
) -> None:
"""Run the collector (blocking).
@@ -599,8 +699,7 @@ def run_collector(
cleanup_interval_hours: Hours between cleanup runs
node_cleanup_enabled: Enable automatic cleanup of inactive nodes
node_cleanup_days: Remove nodes not seen for this many days
channel_keys: Optional channel keys for decrypting group text
include_test_channel: Include built-in test channel messages
channel_refresh_interval_seconds: Seconds between channel key refresh
"""
subscriber = create_subscriber(
mqtt_host=mqtt_host,
@@ -618,8 +717,7 @@ def run_collector(
cleanup_interval_hours=cleanup_interval_hours,
node_cleanup_enabled=node_cleanup_enabled,
node_cleanup_days=node_cleanup_days,
channel_keys=channel_keys,
include_test_channel=include_test_channel,
channel_refresh_interval_seconds=channel_refresh_interval_seconds,
)
# Set up signal handlers
src/meshcore_hub/common/config.py
+13 -20
View File
@@ -1,7 +1,6 @@
"""Pydantic Settings for MeshCore Hub configuration."""
from enum import Enum
import re
from typing import Optional
from pydantic import Field, field_validator
@@ -138,16 +137,10 @@ class CollectorSettings(CommonSettings):
description="Remove nodes not seen for this many days (last_seen)",
ge=1,
)
collector_channel_keys: Optional[str] = Field(
default=None,
description=(
"Optional channel secret keys for message decryption. "
"Provide as comma/space separated hex values."
),
)
collector_include_test_channel: bool = Field(
default=False,
description="Include built-in 'test' channel messages (channel_idx 217).",
channel_refresh_interval_seconds: int = Field(
default=300,
description="Seconds between channel key refresh from database",
ge=10,
)
@property
@@ -182,15 +175,11 @@ class CollectorSettings(CommonSettings):
return str(Path(self.effective_seed_home) / "node_tags.yaml")
@property
def collector_channel_keys_list(self) -> list[str]:
"""Parse configured channel keys into a normalized list."""
if not self.collector_channel_keys:
return []
return [
part.strip()
for part in re.split(r"[,\s]+", self.collector_channel_keys)
if part.strip()
]
def channels_file(self) -> str:
"""Get the path to channels.yaml in seed_home."""
from pathlib import Path
return str(Path(self.effective_seed_home) / "channels.yaml")
@field_validator("database_url")
@classmethod
@@ -383,6 +372,9 @@ class WebSettings(CommonSettings):
default=True, description="Enable the /map page and /map/data endpoint"
)
feature_members: bool = Field(default=True, description="Enable the /members page")
feature_channels: bool = Field(
default=True, description="Enable the /channels page"
)
feature_pages: bool = Field(
default=True, description="Enable custom markdown pages"
)
@@ -412,6 +404,7 @@ class WebSettings(CommonSettings):
"messages": self.feature_messages,
"map": self.feature_map and self.feature_nodes,
"members": self.feature_members and self.oidc_enabled,
"channels": self.feature_channels,
"pages": self.feature_pages,
}
src/meshcore_hub/common/database.py
+27 -12
View File
@@ -1,11 +1,11 @@
"""Database connection and session management."""
from contextlib import asynccontextmanager, contextmanager
from typing import AsyncGenerator, Generator
from typing import Any, AsyncGenerator, Generator
from sqlalchemy import create_engine, event
from sqlalchemy.engine import Engine
from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine
from sqlalchemy.ext.asyncio import AsyncEngine, AsyncSession, create_async_engine
from sqlalchemy.orm import Session, sessionmaker
from meshcore_hub.common.models.base import Base
@@ -88,6 +88,8 @@ class DatabaseManager:
"""Database connection manager.
Manages database engine and session creation for a component.
The async engine is created lazily on first async session access
to avoid leaking connections when only sync operations are needed.
"""
def __init__(self, database_url: str, echo: bool = False):
@@ -98,6 +100,7 @@ class DatabaseManager:
echo: Enable SQL query logging
"""
self.database_url = database_url
self._echo = echo
# Ensure parent directory exists for SQLite databases
if database_url.startswith("sqlite:///"):
@@ -110,14 +113,24 @@ class DatabaseManager:
self.engine = create_database_engine(database_url, echo=echo)
self.session_factory = create_session_factory(self.engine)
# Create async engine for async operations
async_url = database_url.replace("sqlite://", "sqlite+aiosqlite://")
self.async_engine = create_async_engine(async_url, echo=echo)
# Lazy-initialized async engine (created on first async_session call)
self._async_engine: AsyncEngine | None = None
self._async_session_factory: Any = None
def _ensure_async_engine(self) -> None:
"""Create the async engine and session factory on first use."""
if self._async_engine is not None:
return
from sqlalchemy.ext.asyncio import async_sessionmaker
async_url = self.database_url.replace("sqlite://", "sqlite+aiosqlite://")
self._async_engine = create_async_engine(async_url, echo=self._echo)
# Enable foreign keys for async SQLite engine
if database_url.startswith("sqlite"):
if self.database_url.startswith("sqlite"):
@event.listens_for(self.async_engine.sync_engine, "connect")
@event.listens_for(self._async_engine.sync_engine, "connect")
def set_sqlite_pragma_async(
dbapi_connection: object, connection_record: object
) -> None:
@@ -125,10 +138,8 @@ class DatabaseManager:
cursor.execute("PRAGMA foreign_keys=ON")
cursor.close()
from sqlalchemy.ext.asyncio import async_sessionmaker
self.async_session_factory = async_sessionmaker(
self.async_engine,
self._async_session_factory = async_sessionmaker(
self._async_engine,
class_=AsyncSession,
expire_on_commit=False,
)
@@ -183,12 +194,16 @@ class DatabaseManager:
result = await session.execute(select(Node))
await session.commit()
"""
async with self.async_session_factory() as session:
self._ensure_async_engine()
assert self._async_session_factory is not None
async with self._async_session_factory() as session:
yield session
def dispose(self) -> None:
"""Dispose of the database engine and connection pool."""
self.engine.dispose()
if self._async_engine is not None:
self._async_engine.sync_engine.dispose()
# Global database manager instance (initialized at runtime)
src/meshcore_hub/common/models/__init__.py
+3
View File
@@ -11,6 +11,7 @@ from meshcore_hub.common.models.event_log import EventLog
from meshcore_hub.common.models.user_profile import UserProfile
from meshcore_hub.common.models.user_profile_node import UserProfileNode
from meshcore_hub.common.models.event_observer import EventObserver, add_event_observer
from meshcore_hub.common.models.channel import Channel, ChannelVisibility
__all__ = [
"Base",
@@ -26,4 +27,6 @@ __all__ = [
"UserProfileNode",
"EventObserver",
"add_event_observer",
"Channel",
"ChannelVisibility",
]
src/meshcore_hub/common/models/channel.py
+60
View File
@@ -0,0 +1,60 @@
"""Channel model for database-backed decrypt keys."""
import hashlib
from enum import Enum
from sqlalchemy import Boolean, String
from sqlalchemy.orm import Mapped, mapped_column
from meshcore_hub.common.models.base import Base, TimestampMixin, UUIDMixin
class ChannelVisibility(str, Enum):
"""Channel visibility/permission levels."""
COMMUNITY = "community"
MEMBER = "member"
OPERATOR = "operator"
ADMIN = "admin"
class Channel(Base, UUIDMixin, TimestampMixin):
"""Channel model for database-backed decrypt keys with permission-based visibility.
Attributes:
id: UUID primary key
name: Channel display name (unique, non-empty)
key_hex: Secret key as uppercase hex (supports AES-128 and AES-256)
channel_hash: First byte of SHA-256 of key_hex (2-char uppercase hex)
visibility: Permission level (community, member, operator, admin)
enabled: Whether the channel is active
created_at: Record creation timestamp
updated_at: Record update timestamp
"""
__tablename__ = "channels"
name: Mapped[str] = mapped_column(
String(100), unique=True, nullable=False, index=True
)
key_hex: Mapped[str] = mapped_column(String(64), unique=True, nullable=False)
channel_hash: Mapped[str] = mapped_column(String(2), nullable=False)
visibility: Mapped[str] = mapped_column(
String(20), default=ChannelVisibility.COMMUNITY.value, nullable=False
)
enabled: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
def __repr__(self) -> str:
return f"<Channel(name={self.name}, hash={self.channel_hash}, visibility={self.visibility})>"
@staticmethod
def compute_channel_hash(key_hex: str) -> str:
"""Compute channel hash (first byte of SHA-256 of key_hex)."""
return hashlib.sha256(bytes.fromhex(key_hex)).digest()[:1].hex().upper()
@property
def masked_key(self) -> str:
"""Return masked key showing first/last 4 chars."""
if len(self.key_hex) <= 8:
return self.key_hex
return f"{self.key_hex[:4]}...{self.key_hex[-4:]}"
src/meshcore_hub/common/schemas/channels.py
+101
View File
@@ -0,0 +1,101 @@
"""Pydantic schemas for channel API endpoints."""
import re
from datetime import datetime
from typing import Literal, Optional
from pydantic import BaseModel, Field, field_validator
class ChannelCreate(BaseModel):
"""Schema for creating a channel."""
name: str = Field(
...,
min_length=1,
max_length=100,
description="Channel display name",
)
key_hex: str = Field(
...,
min_length=32,
max_length=64,
description="Channel secret key as uppercase hex (32 or 64 chars)",
)
visibility: Literal["community", "member", "operator", "admin"] = Field(
default="community",
description="Channel visibility/permission level",
)
enabled: bool = Field(
default=True,
description="Whether the channel is active",
)
@field_validator("key_hex")
@classmethod
def validate_key_hex(cls, v: str) -> str:
"""Validate key is uppercase hex and correct length."""
v = v.strip().upper()
if not re.fullmatch(r"[0-9A-F]+", v):
raise ValueError("key_hex must contain only hexadecimal characters")
if len(v) not in (32, 64):
raise ValueError("key_hex must be 32 or 64 hex characters")
return v
class ChannelUpdate(BaseModel):
"""Schema for updating a channel."""
key_hex: Optional[str] = Field(
default=None,
min_length=32,
max_length=64,
description="Channel secret key as uppercase hex",
)
visibility: Optional[Literal["community", "member", "operator", "admin"]] = Field(
default=None,
description="Channel visibility/permission level",
)
enabled: Optional[bool] = Field(
default=None,
description="Whether the channel is active",
)
@field_validator("key_hex")
@classmethod
def validate_key_hex(cls, v: str | None) -> str | None:
"""Validate key is uppercase hex and correct length."""
if v is None:
return v
v = v.strip().upper()
if not re.fullmatch(r"[0-9A-F]+", v):
raise ValueError("key_hex must contain only hexadecimal characters")
if len(v) not in (32, 64):
raise ValueError("key_hex must be 32 or 64 hex characters")
return v
class ChannelRead(BaseModel):
"""Schema for reading a channel."""
id: str = Field(..., description="Channel UUID")
name: str = Field(..., description="Channel display name")
channel_hash: str = Field(..., description="Channel hash (2-char hex)")
visibility: str = Field(..., description="Visibility level")
enabled: bool = Field(..., description="Whether the channel is active")
masked_key: str = Field(..., description="Masked key (first/last 4 chars)")
key_hex: Optional[str] = Field(
default=None,
description="Full key hex (visible to users with channel access)",
)
created_at: datetime = Field(..., description="Creation timestamp")
updated_at: datetime = Field(..., description="Last update timestamp")
model_config = {"from_attributes": True}
class ChannelList(BaseModel):
"""Schema for paginated channel list response."""
items: list[ChannelRead] = Field(..., description="List of channels")
total: int = Field(..., description="Total number of channels")
src/meshcore_hub/web/app.py
+44 -21
View File
@@ -2,8 +2,6 @@
import json
import logging
import os
import re
from contextlib import asynccontextmanager
from datetime import datetime
from pathlib import Path
@@ -124,6 +122,15 @@ def _build_endpoint_access(
"GET": _OPEN,
"PUT": _AUTHENTICATED,
},
"v1/channels": {
"GET": _OPEN,
"POST": frozenset({role_admin}),
},
"v1/channels/": {
"POST": frozenset({role_admin}),
"PUT": frozenset({role_admin}),
"DELETE": frozenset({role_admin}),
},
}
@@ -161,27 +168,35 @@ def check_api_access(
return False
def _parse_decoder_key_entries(raw: str | None) -> list[str]:
"""Parse COLLECTOR_CHANNEL_KEYS into key entries."""
if not raw:
return []
return [part.strip() for part in re.split(r"[,\s]+", raw) if part.strip()]
def _build_channel_labels() -> dict[str, str]:
"""Build UI channel labels from built-in + configured decoder keys."""
raw_keys = os.getenv("COLLECTOR_CHANNEL_KEYS")
include_test = os.getenv("COLLECTOR_INCLUDE_TEST_CHANNEL", "false").lower() in (
"true",
"1",
"yes",
)
decoder = LetsMeshPacketDecoder(
channel_keys=_parse_decoder_key_entries(raw_keys),
)
"""Build UI channel labels from built-in + database channel keys."""
decoder = LetsMeshPacketDecoder(channel_keys=[])
labels = decoder.channel_labels_by_index()
if not include_test:
labels.pop(LetsMeshPacketDecoder.TEST_CHANNEL_IDX, None)
try:
from meshcore_hub.common.config import get_collector_settings
settings = get_collector_settings()
from meshcore_hub.common.database import DatabaseManager
db = DatabaseManager(settings.effective_database_url)
try:
from meshcore_hub.common.models.channel import Channel
with db.session_scope() as session:
channels = (
session.query(Channel).filter(Channel.enabled.is_(True)).all()
)
db_decoder = LetsMeshPacketDecoder(
channel_keys=[f"{ch.name}={ch.key_hex}" for ch in channels]
)
db_labels = db_decoder.channel_labels_by_index()
labels.update(db_labels)
finally:
db.dispose()
except Exception as e:
logger.warning("Failed to load channel labels from database: %s", e)
return {str(idx): label for idx, label in sorted(labels.items())}
@@ -683,6 +698,13 @@ def create_app(
):
resp_headers[k] = v
if (
response.status_code < 300
and path.startswith("v1/channels")
and request.method in ("POST", "PUT", "DELETE")
):
request.app.state.channel_labels = _build_channel_labels()
return Response(
content=response.content,
status_code=response.status_code,
@@ -933,6 +955,7 @@ def create_app(
("/dashboard", "hourly", "0.9", "dashboard"),
("/nodes", "hourly", "0.9", "nodes"),
("/advertisements", "hourly", "0.8", "advertisements"),
("/channels", "daily", "0.7", "channels"),
("/map", "daily", "0.7", "map"),
("/members", "weekly", "0.6", "members"),
]
src/meshcore_hub/web/static/css/app.css
+2
View File
@@ -23,6 +23,7 @@
--color-nodes: oklch(0.65 0.24 265); /* violet */
--color-adverts: oklch(0.7 0.17 330); /* magenta */
--color-messages: oklch(0.75 0.18 180); /* teal */
--color-channels: oklch(0.72 0.15 300); /* purple */
--color-map: oklch(0.8471 0.199 83.87); /* yellow (matches btn-warning) */
--color-members: oklch(0.72 0.17 50); /* orange */
--color-neutral: oklch(0.3 0.01 250); /* subtle dark grey */
@@ -35,6 +36,7 @@
--color-nodes: oklch(0.50 0.24 265);
--color-adverts: oklch(0.55 0.17 330);
--color-messages: oklch(0.55 0.18 180);
--color-channels: oklch(0.55 0.15 300);
--color-map: oklch(0.58 0.16 45);
--color-members: oklch(0.55 0.18 25);
--color-neutral: oklch(0.85 0.01 250);
src/meshcore_hub/web/static/js/spa/app.js
+9 -1
View File
@@ -8,7 +8,7 @@
import { Router } from './router.js';
import { html, litRender, getConfig, hasRole, renderAuthSection } from './components.js';
import { loadLocale, t } from './i18n.js';
import { iconHome, iconDashboard, iconNodes, iconAdvertisements, iconMessages, iconMap, iconMembers, iconPage } from './icons.js';
import { iconHome, iconDashboard, iconNodes, iconAdvertisements, iconMessages, iconMap, iconMembers, iconPage, iconChannel } from './icons.js';
// Page modules (lazy-loaded)
const pages = {
@@ -20,6 +20,7 @@ const pages = {
advertisements: () => import('./pages/advertisements.js'),
map: () => import('./pages/map.js'),
members: () => import('./pages/members.js'),
channels: () => import('./pages/channels.js'),
customPage: () => import('./pages/custom-page.js'),
notFound: () => import('./pages/not-found.js'),
profile: () => import('./pages/profile.js'),
@@ -70,6 +71,9 @@ if (features.nodes !== false) {
router.navigate(`/nodes/${params.prefix}`, true);
});
}
if (features.channels !== false) {
router.addRoute('/channels', pageHandler(pages.channels));
}
if (features.messages !== false) {
router.addRoute('/messages', pageHandler(pages.messages));
}
@@ -149,6 +153,7 @@ function updatePageTitle(pathname) {
// Add feature-dependent titles
if (features.dashboard !== false) titles['/dashboard'] = composePageTitle('entities.dashboard');
if (features.nodes !== false) titles['/nodes'] = composePageTitle('entities.nodes');
if (features.channels !== false) titles['/channels'] = composePageTitle('entities.channels');
if (features.messages !== false) titles['/messages'] = composePageTitle('entities.messages');
if (features.advertisements !== false) titles['/advertisements'] = composePageTitle('entities.advertisements');
if (features.map !== false) titles['/map'] = composePageTitle('entities.map');
@@ -198,6 +203,9 @@ function renderMobileNav(config) {
if (features.advertisements !== false) {
items.push(html`<li><a href="/advertisements" data-nav-link>${iconAdvertisements('h-5 w-5 nav-icon-adverts')} ${t('entities.advertisements')}</a></li>`);
}
if (features.channels !== false) {
items.push(html`<li><a href="/channels" data-nav-link>${iconChannel('h-5 w-5')} ${t('entities.channels')}</a></li>`);
}
if (features.messages !== false) {
items.push(html`<li><a href="/messages" data-nav-link>${iconMessages('h-5 w-5 nav-icon-messages')} ${t('entities.messages')}</a></li>`);
}
src/meshcore_hub/web/static/js/spa/pages/channels.js
+286
View File
@@ -0,0 +1,286 @@
import { apiGet, apiPost, apiPut, apiDelete } from '../api.js';
import { html, litRender, nothing, t, errorAlert, getConfig, hasRole } from '../components.js';
import { iconChannel, iconPlus, iconEdit, iconTrash, iconLock } from '../icons.js';
const VISIBILITY_ORDER = ['community', 'member', 'operator', 'admin'];
function renderVisibilityBadge(visibility, oidcEnabled) {
if (!oidcEnabled) return nothing;
return html`<span class="badge badge-primary badge-sm">${visibility}</span>`;
}
function renderChannelCard(channel, { oidcEnabled, isAdmin, onDelete, onEdit, onNavigate }) {
const visibilityBadge = renderVisibilityBadge(channel.visibility, oidcEnabled);
const enabledBadge = !channel.enabled
? html`<span class="badge badge-ghost badge-sm">${t('channels.disabled')}</span>`
: nothing;
const channelIdx = parseInt(channel.channel_hash, 16);
const qrId = `qr-${channel.id}`;
const adminButtons = isAdmin
? html`<div class="flex gap-2 mt-2">
<button class="btn btn-xs btn-outline" @click=${(e) => { e.stopPropagation(); onEdit(channel); }}>
${iconEdit('h-3 w-3')} ${t('common.edit')}
</button>
<button class="btn btn-xs btn-outline btn-error" @click=${(e) => { e.stopPropagation(); onDelete(channel); }}>
${iconTrash('h-3 w-3')} ${t('common.delete')}
</button>
</div>`
: nothing;
const keyDisplay = channel.key_hex
? html`<div class="font-mono text-xs opacity-70 mt-1 break-all select-all">${channel.key_hex.toLowerCase()}</div>`
: nothing;
const qrPlaceholder = channel.key_hex
? html`<div id="${qrId}" class="qr-container"></div>`
: nothing;
return html`<div class="card bg-base-100 shadow-xl cursor-pointer" @click=${() => onNavigate(channelIdx)}>
<div class="card-body flex-row gap-4">
<div class="flex-1 min-w-0">
<h2 class="card-title flex items-center gap-2">
${channel.name}
${visibilityBadge}
${enabledBadge}
</h2>
${keyDisplay}
${adminButtons}
</div>
<div class="flex-shrink-0 self-center">
${qrPlaceholder}
</div>
</div>
</div>`;
}
function renderAddButton(onAdd) {
return html`<button class="btn btn-primary btn-sm" @click=${onAdd}>
${iconPlus('h-4 w-4')} ${t('channels.add_channel')}
</button>`;
}
function renderChannelModal({ channel, isEdit, onSave, onCancel }) {
const title = isEdit ? t('channels.edit_channel') : t('channels.add_channel');
return html`<dialog open class="modal modal-open">
<div class="modal-box">
<h3 class="font-bold text-lg mb-4">${title}</h3>
<form @submit=${(e) => { e.preventDefault(); onSave(); }}>
<div class="grid grid-cols-[auto_1fr] gap-x-4 gap-y-3 items-center mb-4">
<label class="label-text text-right">${t('channels.name_label')}</label>
<input type="text" id="channel-modal-name" class="input input-bordered input-sm"
.value=${isEdit ? channel.name : ''}
?disabled=${isEdit}
placeholder="${t('channels.name_label')}"
required maxlength="100" />
${!isEdit ? html`
<label class="label-text text-right">${t('channels.key_label')}</label>
<input type="text" id="channel-modal-key" class="input input-bordered input-sm font-mono"
placeholder="e.g. ABCDEF0123456789..."
required minlength="32" maxlength="64"
pattern="[0-9A-Fa-f]{32,64}" />` : nothing}
<label class="label-text text-right">${t('channels.visibility_label')}</label>
<select id="channel-modal-visibility" class="select select-bordered select-sm">
<option value="community" .selected=${channel?.visibility === 'community' || !channel}>community</option>
<option value="member" .selected=${channel?.visibility === 'member'}>member</option>
<option value="operator" .selected=${channel?.visibility === 'operator'}>operator</option>
<option value="admin" .selected=${channel?.visibility === 'admin'}>admin</option>
</select>
<div></div>
<label class="label cursor-pointer justify-start gap-3">
<input type="checkbox" id="channel-modal-enabled" class="checkbox checkbox-sm"
.checked=${channel?.enabled !== false} />
<span class="label-text">${t('channels.enabled_label')}</span>
</label>
</div>
<div class="modal-action">
<button type="button" class="btn btn-ghost" @click=${onCancel}>${t('common.cancel')}</button>
<button type="submit" class="btn btn-primary">${t('common.save')}</button>
</div>
</form>
</div>
<form method="dialog" class="modal-backdrop"><button @click=${onCancel}></button></form>
</dialog>`;
}
function renderDeleteModal({ channel, onConfirm, onCancel }) {
return html`<dialog open class="modal modal-open">
<div class="modal-box">
<h3 class="font-bold text-lg mb-4">${t('channels.delete_channel')}</h3>
<p>${t('channels.delete_confirm', { name: channel.name })}</p>
<div class="modal-action">
<button class="btn btn-ghost" @click=${onCancel}>${t('common.cancel')}</button>
<button class="btn btn-error" @click=${onConfirm}>${t('common.delete')}</button>
</div>
</div>
<form method="dialog" class="modal-backdrop"><button @click=${onCancel}></button></form>
</dialog>`;
}
export async function render(container, params, router) {
try {
const config = getConfig();
const oidcEnabled = config.oidc_enabled;
const isAdmin = hasRole('admin');
const data = await apiGet('/api/v1/channels');
const channels = data.items || [];
let modalState = null;
async function refresh() {
const newData = await apiGet('/api/v1/channels');
renderPage(newData.items || []);
}
function renderPage(channelsList) {
const adminHeader = isAdmin
? html`<div class="flex justify-end mb-4">${renderAddButton(handleAdd)}</div>`
: nothing;
const emptyMessage = channelsList.length === 0
? html`<div class="text-center py-10 opacity-60">
${t('common.no_entity_found', { entity: t('entities.channels').toLowerCase() })}
</div>`
: nothing;
const groups = new Map();
for (const vis of VISIBILITY_ORDER) {
groups.set(vis, []);
}
for (const ch of channelsList) {
const vis = ch.visibility || 'community';
if (!groups.has(vis)) groups.set(vis, []);
groups.get(vis).push(ch);
}
const cardOpts = {
oidcEnabled,
isAdmin,
onDelete: handleDeleteClick,
onEdit: handleEditClick,
onNavigate: (idx) => router.navigate(`/messages?channel_idx=${idx}`),
};
const groupedSections = [];
for (const vis of VISIBILITY_ORDER) {
const group = groups.get(vis);
if (!group || group.length === 0) continue;
groupedSections.push(html`
<h2 class="text-lg font-semibold mt-6 mb-3 opacity-70">${t(`channels.visibility_${vis}`)}</h2>
<div class="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-4">
${group.map(ch => renderChannelCard(ch, cardOpts))}
</div>
`);
}
let modalHtml = nothing;
if (modalState?.type === 'add' || modalState?.type === 'edit') {
modalHtml = renderChannelModal({
channel: modalState.channel,
isEdit: modalState.type === 'edit',
onSave: handleSave,
onCancel: () => { modalState = null; renderPage(channelsList); },
});
} else if (modalState?.type === 'delete') {
modalHtml = renderDeleteModal({
channel: modalState.channel,
onConfirm: handleDeleteConfirm,
onCancel: () => { modalState = null; renderPage(channelsList); },
});
}
litRender(html`
<div class="mb-4">
<h1 class="text-2xl font-bold flex items-center gap-2">
${iconChannel('h-7 w-7')}
${t('channels.title')}
</h1>
</div>
${adminHeader}
${emptyMessage}
${groupedSections}
${modalHtml}
`, container);
channelsList.forEach(ch => {
const qrEl = document.getElementById(`qr-${ch.id}`);
if (qrEl && !qrEl.hasChildNodes() && ch.key_hex) {
const qrUrl = `meshcore://channel/add?name=${encodeURIComponent(ch.name)}&secret=${ch.key_hex.toLowerCase()}`;
new QRCode(qrEl, {
text: qrUrl,
width: 128,
height: 128,
correctLevel: QRCode.CorrectLevel.M,
});
}
});
}
function handleAdd() {
modalState = { type: 'add', channel: { visibility: 'community', enabled: true } };
renderPage(channels);
}
function handleEditClick(channel) {
modalState = { type: 'edit', channel };
renderPage(channels);
}
function handleDeleteClick(channel) {
modalState = { type: 'delete', channel };
renderPage(channels);
}
async function handleSave() {
const nameEl = document.getElementById('channel-modal-name');
const keyEl = document.getElementById('channel-modal-key');
const visEl = document.getElementById('channel-modal-visibility');
const enabledEl = document.getElementById('channel-modal-enabled');
const isEdit = modalState.type === 'edit';
const body = {
visibility: visEl.value,
enabled: enabledEl.checked,
};
if (!isEdit) {
body.name = nameEl.value.trim();
body.key_hex = keyEl.value.trim().toUpperCase();
} else {
if (keyEl && keyEl.value) {
body.key_hex = keyEl.value.trim().toUpperCase();
}
}
try {
if (isEdit) {
await apiPut(`/api/v1/channels/${modalState.channel.id}`, body);
} else {
await apiPost('/api/v1/channels', body);
}
modalState = null;
await refresh();
} catch (e) {
alert(e.message || 'Failed to save channel');
}
}
async function handleDeleteConfirm() {
try {
await apiDelete(`/api/v1/channels/${modalState.channel.id}`);
modalState = null;
await refresh();
} catch (e) {
alert(e.message || 'Failed to delete channel');
}
}
renderPage(channels);
} catch (e) {
litRender(errorAlert(e.message || t('common.failed_to_load_page')), container);
}
}
src/meshcore_hub/web/static/js/spa/pages/dashboard.js
+9 -2
View File
@@ -160,17 +160,24 @@ function renderChartCards({ showNodes, showAdverts, showMessages }) {
export async function render(container, params, router) {
try {
const config = getConfig();
const channelLabels = getChannelLabelsMap(config);
let channelLabels = new Map();
const features = config.features || {};
const showNodes = features.nodes !== false;
const showAdverts = features.advertisements !== false;
const showMessages = features.messages !== false;
const [stats, advertActivity, messageActivity, nodeCount] = await Promise.all([
const [stats, advertActivity, messageActivity, nodeCount, channelsData] = await Promise.all([
apiGet('/api/v1/dashboard/stats'),
apiGet('/api/v1/dashboard/activity', { days: 7 }),
apiGet('/api/v1/dashboard/message-activity', { days: 7 }),
apiGet('/api/v1/dashboard/node-count', { days: 7 }),
apiGet('/api/v1/channels'),
]);
channelLabels = new Map([
...getChannelLabelsMap(config),
...(channelsData.items || [])
.map(ch => [parseInt(ch.channel_hash, 16), ch.name])
.filter(([idx]) => Number.isInteger(idx)),
]);
// Top section: stats + charts
src/meshcore_hub/web/static/js/spa/pages/home.js
+10 -4
View File
@@ -5,7 +5,7 @@ import {
} from '../components.js';
import {
iconDashboard, iconNodes, iconAdvertisements, iconMessages, iconMembers, iconMap,
iconPage, iconInfo, iconChart, iconAntenna, iconUsers,
iconPage, iconInfo, iconChart, iconAntenna, iconUsers, iconChannel,
iconSettings, iconFrequency, iconBandwidth, iconSpreadingFactor, iconCodingRate, iconTxPower,
} from '../icons.js';
@@ -35,14 +35,14 @@ function renderRadioTiles(rc) {
function renderNavCard({ href, icon, label, colorVar }) {
return html`
<a href="${href}" class="w-28 h-28 sm:w-32 sm:h-32
<a href="${href}" class="w-24 h-24 sm:w-28 sm:h-28
border border-base-content/20 rounded-box
hover:scale-105 hover:border-base-content/40
transition-all duration-200 ease-out
flex flex-col items-center justify-center gap-2
bg-base-200/50 hover:bg-base-200
group">
<span class="w-8 h-8 sm:w-10 sm:h-10 flex items-center justify-center"
<span class="w-7 h-7 sm:w-9 sm:h-9 flex items-center justify-center"
style="${colorVar ? `color: var(${colorVar})` : ''}">
${icon}
</span>
@@ -74,7 +74,7 @@ function renderHeroSection({ networkName, logoUrl, logoInvertLight, networkCity,
</div>
${welcomeText}
<div class="flex-1"></div>
<div class="flex flex-wrap justify-center gap-3 sm:gap-4 mt-auto">
<div class="flex flex-wrap justify-center gap-2 sm:gap-3 mt-auto">
${features.dashboard !== false ? renderNavCard({
href: '/dashboard',
icon: iconDashboard('w-full h-full'),
@@ -93,6 +93,12 @@ function renderHeroSection({ networkName, logoUrl, logoInvertLight, networkCity,
label: t('entities.advertisements'),
colorVar: '--color-adverts',
}) : nothing}
${features.channels !== false ? renderNavCard({
href: '/channels',
icon: iconChannel('w-full h-full'),
label: t('entities.channels'),
colorVar: '--color-channels',
}) : nothing}
${features.messages !== false ? renderNavCard({
href: '/messages',
icon: iconMessages('w-full h-full'),
src/meshcore_hub/web/static/js/spa/pages/messages.js
+15 -4
View File
@@ -24,7 +24,7 @@ export async function render(container, params, router) {
const order = query.order || 'desc';
const config = getConfig();
const channelLabels = getChannelLabelsMap(config);
let channelLabels = new Map();
const tz = config.timezone || '';
const tzBadge = tz && tz !== 'UTC' ? html`<span class="text-sm opacity-60">${tz}</span>` : nothing;
const navigate = (url) => router.navigate(url);
@@ -206,10 +206,18 @@ ${displayContent}`, container);
try {
const apiParams = { limit, offset, message_type, channel_idx, sort, order };
if (observed_by.length > 0) apiParams.observed_by = observed_by;
const [data, nodesData] = await Promise.all([
const [data, nodesData, channelsData] = await Promise.all([
apiGet('/api/v1/messages', apiParams),
apiGet('/api/v1/nodes', { limit: 500, observer: true }),
apiGet('/api/v1/channels'),
]);
const builtinLabels = getChannelLabelsMap(config);
const customLabels = new Map(
(channelsData.items || [])
.map(ch => [parseInt(ch.channel_hash, 16), ch.name])
.filter(([idx]) => Number.isInteger(idx)),
);
channelLabels = new Map([...builtinLabels, ...customLabels]);
const messages = dedupeBySignature(data.items || []);
const allNodes = nodesData.items || [];
@@ -356,9 +364,12 @@ ${displayContent}`, container);
</label>
<select name="channel_idx" class="select select-bordered select-sm" @change=${autoSubmit}>
<option value="">${t('common.all_channels')}</option>
${[...channelLabels.entries()].map(([idx, label]) =>
${builtinLabels.size > 0 ? html`<optgroup label=${t('channels.optgroup_standard')}>${[...builtinLabels.entries()].map(([idx, label]) =>
html`<option value=${idx} ?selected=${channel_idx === String(idx)}>${label}</option>`
)}
)}</optgroup>` : nothing}
${customLabels.size > 0 ? html`<optgroup label=${t('channels.optgroup_custom')}>${[...customLabels.entries()].map(([idx, label]) =>
html`<option value=${idx} ?selected=${channel_idx === String(idx)}>${label}</option>`
)}</optgroup>` : nothing}
</select>
</div>`,
];
src/meshcore_hub/web/static/locales/en.json
+23 -3
View File
@@ -5,8 +5,8 @@
"nodes": "Nodes",
"node": "Node",
"node_detail": "Node Detail",
"advertisements": "Advertisements",
"advertisement": "Advertisement",
"advertisements": "Adverts",
"advertisement": "Advert",
"messages": "Messages",
"message": "Message",
"map": "Map",
@@ -14,7 +14,8 @@
"member": "Member",
"tags": "Tags",
"tag": "Tag",
"channel": "Channel"
"channel": "Channel",
"channels": "Channels"
},
"common": {
"filter": "Filter",
@@ -223,6 +224,25 @@
"empty_state_description": "No members yet.",
"empty_description": "Members will appear here once users log in and adopt nodes."
},
"channels": {
"title": "Channels",
"add_channel": "Add Channel",
"edit_channel": "Edit Channel",
"delete_channel": "Delete Channel",
"delete_confirm": "Are you sure you want to delete channel {{name}}?",
"name_label": "Channel Name",
"key_label": "Channel Key (hex)",
"visibility_label": "Visibility",
"visibility_community": "Community",
"visibility_member": "Member",
"visibility_operator": "Operator",
"visibility_admin": "Admin",
"enabled_label": "Enabled",
"channel_hash_label": "Hash",
"disabled": "Disabled",
"optgroup_standard": "Standard",
"optgroup_custom": "Custom"
},
"not_found": {
"description": "The page you're looking for doesn't exist or has been moved."
},
src/meshcore_hub/web/static/locales/nl.json
+6
View File
@@ -170,6 +170,12 @@
"empty_state_description": "Nog geen leden.",
"empty_description": "Leden verschijnen hier zodra gebruikers inloggen en knooppunten adopteren."
},
"channels": {
"visibility_community": "Community",
"visibility_member": "Lid",
"visibility_operator": "Operator",
"visibility_admin": "Beheerder"
},
"not_found": {
"description": "De pagina die u zoekt bestaat niet of is verplaatst."
},
src/meshcore_hub/web/templates/spa.html
+3
View File
@@ -67,6 +67,9 @@
{% if features.advertisements %}
<li><a href="/advertisements" data-nav-link><svg xmlns="http://www.w3.org/2000/svg" class="h-4 w-4 nav-icon-adverts" fill="none" viewBox="0 0 24 24" stroke="currentColor"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M11 5.882V19.24a1.76 1.76 0 01-3.417.592l-2.147-6.15M18 13a3 3 0 100-6M5.436 13.683A4.001 4.001 0 017 6h1.832c4.1 0 7.625-1.234 9.168-3v14c-1.543-1.766-5.067-3-9.168-3H7a3.988 3.988 0 01-1.564-.317z" /></svg> {{ t('entities.advertisements') }}</a></li>
{% endif %}
{% if features.channels %}
<li><a href="/channels" data-nav-link><svg xmlns="http://www.w3.org/2000/svg" class="h-4 w-4" fill="none" viewBox="0 0 24 24" stroke="currentColor"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M7 20l4-16m2 16l4-16M6 9h14M4 15h14" /></svg> {{ t('entities.channels') }}</a></li>
{% endif %}
{% if features.messages %}
<li><a href="/messages" data-nav-link><svg xmlns="http://www.w3.org/2000/svg" class="h-4 w-4 nav-icon-messages" fill="none" viewBox="0 0 24 24" stroke="currentColor"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M8 10h.01M12 10h.01M16 10h.01M9 16H5a2 2 0 01-2-2V6a2 2 0 012-2h14a2 2 0 012 2v8a2 2 0 01-2 2h-5l-5 5v-5z" /></svg> {{ t('entities.messages') }}</a></li>
{% endif %}
tests/test_api/conftest.py
+52 -1
View File
@@ -21,6 +21,7 @@ from meshcore_hub.common.database import DatabaseManager
from meshcore_hub.common.models import (
Advertisement,
Base,
Channel,
Message,
Node,
NodeTag,
@@ -311,7 +312,7 @@ def sample_message_with_receiver(api_db_session, receiver_node):
"""Create a message with a receiver node."""
message = Message(
message_type="channel",
channel_idx=1,
channel_idx=17,
pubkey_prefix="xyz789",
text="Channel message with receiver",
received_at=datetime.now(timezone.utc),
@@ -445,3 +446,53 @@ def sample_adopted_node(api_db_session, sample_user_profile, sample_node):
api_db_session.commit()
api_db_session.refresh(association)
return association
@pytest.fixture
def sample_channel(api_db_session):
"""Create a sample community channel in the database."""
channel = Channel(
name="TestChannel",
key_hex="AABBCCDDEEFF00112233445566778899",
channel_hash=Channel.compute_channel_hash("AABBCCDDEEFF00112233445566778899"),
visibility="community",
enabled=True,
)
api_db_session.add(channel)
api_db_session.commit()
api_db_session.refresh(channel)
return channel
@pytest.fixture
def sample_member_channel(api_db_session):
"""Create a sample member-only channel in the database."""
key = "11223344556677889900AABBCCDDEEFF"
channel = Channel(
name="MemberChannel",
key_hex=key,
channel_hash=Channel.compute_channel_hash(key),
visibility="member",
enabled=True,
)
api_db_session.add(channel)
api_db_session.commit()
api_db_session.refresh(channel)
return channel
@pytest.fixture
def sample_admin_channel(api_db_session):
"""Create a sample admin-only channel in the database."""
key = "FFEEDDCCBBAA99887766554433221100"
channel = Channel(
name="AdminChannel",
key_hex=key,
channel_hash=Channel.compute_channel_hash(key),
visibility="admin",
enabled=True,
)
api_db_session.add(channel)
api_db_session.commit()
api_db_session.refresh(channel)
return channel
tests/test_api/test_channel_visibility.py
+312
View File
@@ -0,0 +1,312 @@
"""Tests for channel_visibility helpers."""
from unittest.mock import MagicMock
import pytest
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from meshcore_hub.api.channel_visibility import (
get_all_known_channel_indices,
get_max_visibility_level,
get_visible_channel_indices,
resolve_user_role,
)
from meshcore_hub.common.models import Base
from meshcore_hub.common.models.channel import Channel
@pytest.fixture
def db_session():
"""Create an in-memory SQLite database session."""
engine = create_engine(
"sqlite:///:memory:",
connect_args={"check_same_thread": False},
)
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
yield session
session.close()
Base.metadata.drop_all(engine)
engine.dispose()
def _make_request(
headers: dict | None = None, app_state: dict | None = None
) -> MagicMock:
"""Create a mock FastAPI Request."""
from types import SimpleNamespace
request = MagicMock()
request.headers = headers or {}
state = SimpleNamespace(**(app_state or {}))
request.app.state = state
return request
class TestResolveUserRole:
"""Tests for resolve_user_role()."""
def test_no_header_returns_none(self) -> None:
"""No X-User-Roles header returns None."""
request = _make_request(headers={})
assert resolve_user_role(request) is None
def test_empty_header_returns_none(self) -> None:
"""Empty X-User-Roles header returns None."""
request = _make_request(headers={"x-user-roles": ""})
assert resolve_user_role(request) is None
def test_admin_role(self) -> None:
"""Admin role is resolved correctly."""
request = _make_request(headers={"x-user-roles": "admin"})
assert resolve_user_role(request) == "admin"
def test_operator_role(self) -> None:
"""Operator role is resolved correctly."""
request = _make_request(headers={"x-user-roles": "operator"})
assert resolve_user_role(request) == "operator"
def test_member_role(self) -> None:
"""Member role is resolved correctly."""
request = _make_request(headers={"x-user-roles": "member"})
assert resolve_user_role(request) == "member"
def test_admin_takes_precedence_over_member(self) -> None:
"""Admin takes precedence when multiple roles present."""
request = _make_request(headers={"x-user-roles": "member,admin"})
assert resolve_user_role(request) == "admin"
def test_operator_takes_precedence_over_member(self) -> None:
"""Operator takes precedence over member."""
request = _make_request(headers={"x-user-roles": "member,operator"})
assert resolve_user_role(request) == "operator"
def test_admin_takes_precedence_over_all(self) -> None:
"""Admin takes precedence over operator and member."""
request = _make_request(headers={"x-user-roles": "member,operator,admin"})
assert resolve_user_role(request) == "admin"
def test_unknown_role_returns_none(self) -> None:
"""Unknown role returns None."""
request = _make_request(headers={"x-user-roles": "viewer"})
assert resolve_user_role(request) is None
def test_custom_role_names(self) -> None:
"""Custom OIDC role names from app.state are recognized."""
request = _make_request(
headers={"x-user-roles": "superadmin,moderator"},
app_state={
"oidc_role_admin": "superadmin",
"oidc_role_operator": "moderator",
"oidc_role_member": "user",
},
)
assert resolve_user_role(request) == "admin"
def test_custom_member_role_name(self) -> None:
"""Custom member role name is recognized."""
request = _make_request(
headers={"x-user-roles": "user"},
app_state={
"oidc_role_member": "user",
},
)
assert resolve_user_role(request) == "member"
def test_whitespace_in_header(self) -> None:
"""Whitespace around role names is handled."""
request = _make_request(headers={"x-user-roles": " admin , member "})
assert resolve_user_role(request) == "admin"
class TestGetMaxVisibilityLevel:
"""Tests for get_max_visibility_level()."""
def test_none_returns_zero(self) -> None:
"""Anonymous users get level 0 (community only)."""
assert get_max_visibility_level(None) == 0
def test_community_returns_zero(self) -> None:
assert get_max_visibility_level("community") == 0
def test_member_returns_one(self) -> None:
assert get_max_visibility_level("member") == 1
def test_operator_returns_two(self) -> None:
assert get_max_visibility_level("operator") == 2
def test_admin_returns_three(self) -> None:
assert get_max_visibility_level("admin") == 3
def test_unknown_returns_zero(self) -> None:
assert get_max_visibility_level("unknown") == 0
class TestGetVisibleChannelIndices:
"""Tests for get_visible_channel_indices()."""
def test_always_includes_idx_17(self, db_session) -> None:
"""Built-in Public channel (idx 17) is always visible."""
indices = get_visible_channel_indices(db_session, 0)
assert 17 in indices
def test_community_channels_visible_at_level_0(self, db_session) -> None:
"""Community channels are visible at level 0."""
key = "AABBCCDDEEFF00112233445566778899"
ch = Channel(
name="Community",
key_hex=key,
channel_hash=Channel.compute_channel_hash(key),
visibility="community",
)
db_session.add(ch)
db_session.commit()
indices = get_visible_channel_indices(db_session, 0)
expected_idx = int(ch.channel_hash, 16)
assert expected_idx in indices
assert 17 in indices
def test_member_channels_hidden_at_level_0(self, db_session) -> None:
"""Member channels are hidden at level 0."""
key = "11223344556677889900AABBCCDDEEFF"
ch = Channel(
name="MembersOnly",
key_hex=key,
channel_hash=Channel.compute_channel_hash(key),
visibility="member",
)
db_session.add(ch)
db_session.commit()
indices = get_visible_channel_indices(db_session, 0)
ch_idx = int(ch.channel_hash, 16)
assert ch_idx not in indices
def test_member_channels_visible_at_level_1(self, db_session) -> None:
"""Member channels are visible at level 1."""
key = "11223344556677889900AABBCCDDEEFF"
ch = Channel(
name="MemberCh",
key_hex=key,
channel_hash=Channel.compute_channel_hash(key),
visibility="member",
)
db_session.add(ch)
db_session.commit()
indices = get_visible_channel_indices(db_session, 1)
ch_idx = int(ch.channel_hash, 16)
assert ch_idx in indices
def test_admin_channels_visible_at_level_3(self, db_session) -> None:
"""Admin channels are visible at level 3."""
key = "FFEEDDCCBBAA99887766554433221100"
ch = Channel(
name="AdminCh",
key_hex=key,
channel_hash=Channel.compute_channel_hash(key),
visibility="admin",
)
db_session.add(ch)
db_session.commit()
indices = get_visible_channel_indices(db_session, 3)
ch_idx = int(ch.channel_hash, 16)
assert ch_idx in indices
def test_admin_channels_hidden_at_level_1(self, db_session) -> None:
"""Admin channels are hidden at level 1."""
key = "FFEEDDCCBBAA99887766554433221100"
ch = Channel(
name="AdminCh",
key_hex=key,
channel_hash=Channel.compute_channel_hash(key),
visibility="admin",
)
db_session.add(ch)
db_session.commit()
indices = get_visible_channel_indices(db_session, 1)
ch_idx = int(ch.channel_hash, 16)
assert ch_idx not in indices
def test_mixed_visibility_channels(self, db_session) -> None:
"""Multiple channels with different visibility levels."""
pub_key = "AABBCCDDEEFF00112233445566778899"
mem_key = "11223344556677889900AABBCCDDEEFF"
adm_key = "FFEEDDCCBBAA99887766554433221100"
for name, key, vis in [
("Community", pub_key, "community"),
("Member", mem_key, "member"),
("Admin", adm_key, "admin"),
]:
db_session.add(
Channel(
name=name,
key_hex=key,
channel_hash=Channel.compute_channel_hash(key),
visibility=vis,
)
)
db_session.commit()
level_0 = get_visible_channel_indices(db_session, 0)
level_1 = get_visible_channel_indices(db_session, 1)
level_3 = get_visible_channel_indices(db_session, 3)
pub_idx = int(Channel.compute_channel_hash(pub_key), 16)
mem_idx = int(Channel.compute_channel_hash(mem_key), 16)
adm_idx = int(Channel.compute_channel_hash(adm_key), 16)
assert pub_idx in level_0
assert mem_idx not in level_0
assert adm_idx not in level_0
assert pub_idx in level_1
assert mem_idx in level_1
assert adm_idx not in level_1
assert pub_idx in level_3
assert mem_idx in level_3
assert adm_idx in level_3
assert 17 in level_0
assert 17 in level_1
assert 17 in level_3
class TestGetAllKnownChannelIndices:
"""Tests for get_all_known_channel_indices()."""
def test_empty_db(self, db_session) -> None:
"""Empty DB returns empty set."""
indices = get_all_known_channel_indices(db_session)
assert indices == set()
def test_returns_all_indices(self, db_session) -> None:
"""Returns all channel indices from DB."""
key1 = "AABBCCDDEEFF00112233445566778899"
key2 = "11223344556677889900AABBCCDDEEFF"
for name, key in [("Ch1", key1), ("Ch2", key2)]:
db_session.add(
Channel(
name=name,
key_hex=key,
channel_hash=Channel.compute_channel_hash(key),
)
)
db_session.commit()
indices = get_all_known_channel_indices(db_session)
idx1 = int(Channel.compute_channel_hash(key1), 16)
idx2 = int(Channel.compute_channel_hash(key2), 16)
assert indices == {idx1, idx2}
def test_does_not_include_builtin_17(self, db_session) -> None:
"""Does not include the built-in Public channel (17) unless in DB."""
indices = get_all_known_channel_indices(db_session)
assert 17 not in indices
tests/test_api/test_channels.py
+358
View File
@@ -0,0 +1,358 @@
"""Tests for channel API routes."""
from meshcore_hub.common.models import Channel
VALID_KEY_32 = "A" * 32
VALID_KEY_64 = "B" * 64
ALT_KEY_32 = "C" * 32
class TestListChannels:
"""Tests for GET /channels endpoint."""
def test_list_channels_empty(self, client_no_auth):
"""Test listing channels when database is empty."""
response = client_no_auth.get("/api/v1/channels")
assert response.status_code == 200
data = response.json()
assert data["items"] == []
assert data["total"] == 0
def test_list_channels_with_data(self, client_no_auth, sample_channel):
"""Test listing channels with data in database."""
response = client_no_auth.get("/api/v1/channels")
assert response.status_code == 200
data = response.json()
assert len(data["items"]) == 1
assert data["total"] == 1
assert data["items"][0]["name"] == "TestChannel"
assert data["items"][0]["key_hex"] is not None
assert data["items"][0]["masked_key"] is not None
def test_list_channels_anonymous_only_community(
self, client_no_auth, api_db_session
):
"""Anonymous users only see community channels."""
pub_key = "AABBCCDDEEFF00112233445566778899"
mem_key = "11223344556677889900AABBCCDDEEFF"
for name, key, vis in [
("Community", pub_key, "community"),
("Secret", mem_key, "member"),
]:
ch = Channel(
name=name,
key_hex=key,
channel_hash=Channel.compute_channel_hash(key),
visibility=vis,
enabled=True,
)
api_db_session.add(ch)
api_db_session.commit()
response = client_no_auth.get("/api/v1/channels")
assert response.status_code == 200
data = response.json()
assert data["total"] == 1
assert data["items"][0]["name"] == "Community"
def test_list_channels_admin_sees_all(self, client_no_auth, api_db_session):
"""Admin role header allows seeing all channels."""
pub_key = "AABBCCDDEEFF00112233445566778899"
mem_key = "11223344556677889900AABBCCDDEEFF"
adm_key = "FFEEDDCCBBAA99887766554433221100"
for name, key, vis in [
("Community", pub_key, "community"),
("Member", mem_key, "member"),
("Admin", adm_key, "admin"),
]:
ch = Channel(
name=name,
key_hex=key,
channel_hash=Channel.compute_channel_hash(key),
visibility=vis,
enabled=True,
)
api_db_session.add(ch)
api_db_session.commit()
response = client_no_auth.get(
"/api/v1/channels",
headers={"X-User-Roles": "admin"},
)
assert response.status_code == 200
data = response.json()
assert data["total"] == 3
def test_list_channels_member_sees_community_and_member(
self, client_no_auth, api_db_session
):
"""Member role sees community and member channels, not admin."""
pub_key = "AABBCCDDEEFF00112233445566778899"
mem_key = "11223344556677889900AABBCCDDEEFF"
adm_key = "FFEEDDCCBBAA99887766554433221100"
for name, key, vis in [
("Community", pub_key, "community"),
("MemberCh", mem_key, "member"),
("AdminCh", adm_key, "admin"),
]:
ch = Channel(
name=name,
key_hex=key,
channel_hash=Channel.compute_channel_hash(key),
visibility=vis,
enabled=True,
)
api_db_session.add(ch)
api_db_session.commit()
response = client_no_auth.get(
"/api/v1/channels",
headers={"X-User-Roles": "member"},
)
assert response.status_code == 200
data = response.json()
assert data["total"] == 2
names = {item["name"] for item in data["items"]}
assert names == {"Community", "MemberCh"}
def test_list_channels_operator_sees_community_member_operator(
self, client_no_auth, api_db_session
):
"""Operator role sees community, member, and operator channels."""
pub_key = "AABBCCDDEEFF00112233445566778899"
mem_key = "11223344556677889900AABBCCDDEEFF"
op_key = "0A0B0C0D0E0F10111213141516171819"
adm_key = "FFEEDDCCBBAA99887766554433221100"
for name, key, vis in [
("Community", pub_key, "community"),
("MemberCh", mem_key, "member"),
("OperatorCh", op_key, "operator"),
("AdminCh", adm_key, "admin"),
]:
ch = Channel(
name=name,
key_hex=key,
channel_hash=Channel.compute_channel_hash(key),
visibility=vis,
enabled=True,
)
api_db_session.add(ch)
api_db_session.commit()
response = client_no_auth.get(
"/api/v1/channels",
headers={"X-User-Roles": "operator"},
)
assert response.status_code == 200
data = response.json()
assert data["total"] == 3
names = {item["name"] for item in data["items"]}
assert names == {"Community", "MemberCh", "OperatorCh"}
class TestCreateChannel:
"""Tests for POST /channels endpoint."""
def test_create_channel_success(self, client_no_auth):
"""Test creating a channel successfully."""
response = client_no_auth.post(
"/api/v1/channels",
json={
"name": "NewChannel",
"key_hex": VALID_KEY_32,
"visibility": "community",
"enabled": True,
},
)
assert response.status_code == 201
data = response.json()
assert data["name"] == "NewChannel"
assert data["visibility"] == "community"
assert data["enabled"] is True
assert data["key_hex"] == VALID_KEY_32
assert data["masked_key"] == f"{VALID_KEY_32[:4]}...{VALID_KEY_32[-4:]}"
assert data["channel_hash"] == Channel.compute_channel_hash(VALID_KEY_32)
assert data["id"] is not None
assert data["created_at"] is not None
def test_create_channel_duplicate_name(self, client_no_auth, sample_channel):
"""Test creating channel with duplicate name returns 409."""
response = client_no_auth.post(
"/api/v1/channels",
json={
"name": "TestChannel",
"key_hex": ALT_KEY_32,
},
)
assert response.status_code == 409
assert "already exists" in response.json()["detail"]
def test_create_channel_duplicate_key(self, client_no_auth, sample_channel):
"""Test creating channel with duplicate key returns 409."""
response = client_no_auth.post(
"/api/v1/channels",
json={
"name": "DifferentName",
"key_hex": sample_channel.key_hex,
},
)
assert response.status_code == 409
assert "Key already in use" in response.json()["detail"]
def test_create_channel_invalid_key(self, client_no_auth):
"""Test creating channel with invalid key returns 422."""
response = client_no_auth.post(
"/api/v1/channels",
json={
"name": "BadKey",
"key_hex": "NOT-HEX",
},
)
assert response.status_code == 422
def test_create_channel_aes256_key(self, client_no_auth):
"""Test creating channel with AES-256 key (64 hex chars)."""
response = client_no_auth.post(
"/api/v1/channels",
json={
"name": "AES256",
"key_hex": VALID_KEY_64,
},
)
assert response.status_code == 201
assert response.json()["key_hex"] == VALID_KEY_64
def test_create_channel_with_auth(self, client_with_auth):
"""Test creating channel requires admin key."""
response = client_with_auth.post(
"/api/v1/channels",
json={
"name": "AuthChannel",
"key_hex": VALID_KEY_32,
},
)
assert response.status_code == 401
response = client_with_auth.post(
"/api/v1/channels",
headers={"Authorization": "Bearer test-admin-key"},
json={
"name": "AuthChannel",
"key_hex": VALID_KEY_32,
},
)
assert response.status_code == 201
class TestUpdateChannel:
"""Tests for PUT /channels/{channel_id} endpoint."""
def test_update_channel_visibility(self, client_no_auth, sample_channel):
"""Test updating channel visibility."""
response = client_no_auth.put(
f"/api/v1/channels/{sample_channel.id}",
json={"visibility": "member"},
)
assert response.status_code == 200
assert response.json()["visibility"] == "member"
def test_update_channel_key(self, client_no_auth, sample_channel):
"""Test updating channel key regenerates hash."""
response = client_no_auth.put(
f"/api/v1/channels/{sample_channel.id}",
json={"key_hex": ALT_KEY_32},
)
assert response.status_code == 200
data = response.json()
assert data["key_hex"] == ALT_KEY_32
assert data["channel_hash"] == Channel.compute_channel_hash(ALT_KEY_32)
def test_update_channel_enabled(self, client_no_auth, sample_channel):
"""Test disabling a channel."""
response = client_no_auth.put(
f"/api/v1/channels/{sample_channel.id}",
json={"enabled": False},
)
assert response.status_code == 200
assert response.json()["enabled"] is False
def test_update_channel_not_found(self, client_no_auth):
"""Test updating non-existent channel returns 404."""
response = client_no_auth.put(
"/api/v1/channels/nonexistent-id",
json={"visibility": "admin"},
)
assert response.status_code == 404
def test_update_channel_duplicate_key(self, client_no_auth, api_db_session):
"""Test updating key to one already in use returns 409."""
key1 = "AABBCCDDEEFF00112233445566778899"
key2 = "11223344556677889900AABBCCDDEEFF"
ch1 = Channel(
name="Ch1",
key_hex=key1,
channel_hash=Channel.compute_channel_hash(key1),
)
ch2 = Channel(
name="Ch2",
key_hex=key2,
channel_hash=Channel.compute_channel_hash(key2),
)
api_db_session.add_all([ch1, ch2])
api_db_session.commit()
response = client_no_auth.put(
f"/api/v1/channels/{ch1.id}",
json={"key_hex": key2},
)
assert response.status_code == 409
def test_update_channel_same_key_allowed(self, client_no_auth, sample_channel):
"""Test updating channel with its own key is allowed."""
response = client_no_auth.put(
f"/api/v1/channels/{sample_channel.id}",
json={"key_hex": sample_channel.key_hex},
)
assert response.status_code == 200
class TestDeleteChannel:
"""Tests for DELETE /channels/{channel_id} endpoint."""
def test_delete_channel_success(self, client_no_auth, sample_channel):
"""Test deleting a channel."""
response = client_no_auth.delete(f"/api/v1/channels/{sample_channel.id}")
assert response.status_code == 204
response = client_no_auth.get("/api/v1/channels")
assert response.status_code == 200
assert response.json()["total"] == 0
def test_delete_channel_not_found(self, client_no_auth):
"""Test deleting non-existent channel returns 404."""
response = client_no_auth.delete("/api/v1/channels/nonexistent-id")
assert response.status_code == 404
def test_delete_channel_with_auth(self, client_with_auth, api_db_session):
"""Test deleting channel requires admin key."""
key = "AABBCCDDEEFF00112233445566778899"
ch = Channel(
name="ToDelete",
key_hex=key,
channel_hash=Channel.compute_channel_hash(key),
)
api_db_session.add(ch)
api_db_session.commit()
response = client_with_auth.delete(f"/api/v1/channels/{ch.id}")
assert response.status_code == 401
response = client_with_auth.delete(
f"/api/v1/channels/{ch.id}",
headers={"Authorization": "Bearer test-admin-key"},
)
assert response.status_code == 204
tests/test_api/test_dashboard.py
+228 -1
View File
@@ -5,7 +5,13 @@ from unittest.mock import patch
import pytest
from meshcore_hub.common.models import Advertisement, Message, Node
from meshcore_hub.common.models import (
Advertisement,
Message,
Node,
NodeTag,
Channel,
)
from meshcore_hub.common.models import UserProfile
@@ -439,3 +445,224 @@ class TestDashboardFloodOnlyFilter:
data = response.json()
total_count = sum(point["count"] for point in data["data"])
assert total_count == 1
class TestDashboardChannelVisibility:
"""Tests for channel visibility filtering on dashboard stats."""
@pytest.fixture
def channels_with_messages(self, api_db_session):
"""Create public and admin channels with messages."""
pub_key = "AABBCCDDEEFF00112233445566778899"
adm_key = "FFEEDDCCBBAA99887766554433221100"
pub_idx = int(Channel.compute_channel_hash(pub_key), 16)
adm_idx = int(Channel.compute_channel_hash(adm_key), 16)
pub_ch = Channel(
name="CommunityCh",
key_hex=pub_key,
channel_hash=Channel.compute_channel_hash(pub_key),
visibility="community",
enabled=True,
)
adm_ch = Channel(
name="AdminCh",
key_hex=adm_key,
channel_hash=Channel.compute_channel_hash(adm_key),
visibility="admin",
enabled=True,
)
api_db_session.add_all([pub_ch, adm_ch])
pub_msg = Message(
message_type="channel",
channel_idx=pub_idx,
text="Public message",
received_at=datetime.now(timezone.utc),
)
adm_msg = Message(
message_type="channel",
channel_idx=adm_idx,
text="Admin message",
received_at=datetime.now(timezone.utc),
)
direct_msg = Message(
message_type="direct",
pubkey_prefix="abc123",
text="Direct message",
received_at=datetime.now(timezone.utc),
)
api_db_session.add_all([pub_msg, adm_msg, direct_msg])
api_db_session.commit()
return pub_idx, adm_idx
def test_anonymous_sees_only_community_messages(
self, client_no_auth, channels_with_messages
):
"""Anonymous users only see community and direct messages in stats."""
response = client_no_auth.get("/api/v1/dashboard/stats")
assert response.status_code == 200
data = response.json()
assert data["total_messages"] == 2
def test_admin_sees_all_messages(self, client_no_auth, channels_with_messages):
"""Admin users see all messages in stats."""
response = client_no_auth.get(
"/api/v1/dashboard/stats",
headers={"X-User-Roles": "admin"},
)
assert response.status_code == 200
data = response.json()
assert data["total_messages"] == 3
def test_channel_message_counts_filtered(
self, client_no_auth, channels_with_messages
):
"""Channel message counts exclude hidden channels."""
pub_idx, adm_idx = channels_with_messages
response = client_no_auth.get("/api/v1/dashboard/stats")
assert response.status_code == 200
data = response.json()
assert str(pub_idx) in data["channel_message_counts"]
assert str(adm_idx) not in data["channel_message_counts"]
def test_admin_channel_message_counts_all(
self, client_no_auth, channels_with_messages
):
"""Admin users see all channel message counts."""
pub_idx, adm_idx = channels_with_messages
response = client_no_auth.get(
"/api/v1/dashboard/stats",
headers={"X-User-Roles": "admin"},
)
assert response.status_code == 200
data = response.json()
assert str(pub_idx) in data["channel_message_counts"]
assert str(adm_idx) in data["channel_message_counts"]
def test_message_activity_respects_visibility(self, client_no_auth, api_db_session):
"""Message activity endpoint filters by channel visibility."""
adm_key = "FFEEDDCCBBAA99887766554433221100"
adm_idx = int(Channel.compute_channel_hash(adm_key), 16)
adm_ch = Channel(
name="AdminCh",
key_hex=adm_key,
channel_hash=Channel.compute_channel_hash(adm_key),
visibility="admin",
enabled=True,
)
api_db_session.add(adm_ch)
yesterday = datetime.now(timezone.utc) - timedelta(days=1)
adm_msg = Message(
message_type="channel",
channel_idx=adm_idx,
text="Admin msg",
received_at=yesterday,
)
api_db_session.add(adm_msg)
api_db_session.commit()
response_anon = client_no_auth.get("/api/v1/dashboard/message-activity")
assert response_anon.status_code == 200
anon_data = response_anon.json()
anon_total = sum(p["count"] for p in anon_data["data"])
assert anon_total == 0
response_admin = client_no_auth.get(
"/api/v1/dashboard/message-activity",
headers={"X-User-Roles": "admin"},
)
assert response_admin.status_code == 200
admin_data = response_admin.json()
admin_total = sum(p["count"] for p in admin_data["data"])
assert admin_total >= 1
def test_recent_advertisements_includes_tag_name(
self, client_no_auth, api_db_session
):
"""Recent advertisements resolve tag_name from name tags."""
now = datetime.now(timezone.utc)
pub_key = "aa" * 16
node = Node(
public_key=pub_key,
name="NodeName",
adv_type="CLIENT",
first_seen=now,
last_seen=now,
)
api_db_session.add(node)
api_db_session.commit()
tag = NodeTag(node_id=node.id, key="name", value="TagName")
api_db_session.add(tag)
ad = Advertisement(
public_key=pub_key,
name=None,
adv_type="CLIENT",
received_at=now,
route_type="flood",
)
api_db_session.add(ad)
api_db_session.commit()
response = client_no_auth.get("/api/v1/dashboard/stats")
assert response.status_code == 200
data = response.json()
assert len(data["recent_advertisements"]) == 1
assert data["recent_advertisements"][0]["tag_name"] == "TagName"
def test_operator_sees_community_and_member_channel_counts(
self, client_no_auth, api_db_session
):
"""Operator role sees community + member channels but not admin in stats."""
pub_key = "AABBCCDDEEFF00112233445566778899"
mem_key = "11223344556677889900AABBCCDDEEFF"
adm_key = "FFEEDDCCBBAA99887766554433221100"
pub_idx = int(Channel.compute_channel_hash(pub_key), 16)
mem_idx = int(Channel.compute_channel_hash(mem_key), 16)
adm_idx = int(Channel.compute_channel_hash(adm_key), 16)
for name, key, vis in [
("Community", pub_key, "community"),
("Member", mem_key, "member"),
("Admin", adm_key, "admin"),
]:
ch = Channel(
name=name,
key_hex=key,
channel_hash=Channel.compute_channel_hash(key),
visibility=vis,
enabled=True,
)
api_db_session.add(ch)
api_db_session.commit()
for idx, text in [
(pub_idx, "Pub msg"),
(mem_idx, "Mem msg"),
(adm_idx, "Adm msg"),
]:
msg = Message(
message_type="channel",
channel_idx=idx,
text=text,
received_at=datetime.now(timezone.utc),
)
api_db_session.add(msg)
api_db_session.commit()
response = client_no_auth.get(
"/api/v1/dashboard/stats",
headers={"X-User-Roles": "operator"},
)
assert response.status_code == 200
data = response.json()
assert str(pub_idx) in data["channel_message_counts"]
assert str(mem_idx) in data["channel_message_counts"]
assert str(adm_idx) not in data["channel_message_counts"]
tests/test_api/test_messages.py
+260 -8
View File
@@ -2,7 +2,9 @@
from datetime import datetime, timedelta, timezone
from meshcore_hub.common.models import EventObserver, Message, Node, NodeTag
import pytest
from meshcore_hub.common.models import EventObserver, Message, Node, NodeTag, Channel
class TestListMessages:
@@ -112,7 +114,7 @@ class TestListMessages:
"""Messages include observers list in response."""
msg = Message(
message_type="channel",
channel_idx=1,
channel_idx=17,
text="Msg with observer",
received_at=datetime.now(timezone.utc),
observer_node_id=receiver_node.id,
@@ -158,7 +160,7 @@ class TestGetMessage:
"""Get message includes observers list."""
msg = Message(
message_type="channel",
channel_idx=1,
channel_idx=17,
text="Msg for get observer test",
received_at=datetime.now(timezone.utc),
observer_node_id=receiver_node.id,
@@ -204,11 +206,11 @@ class TestListMessagesFilters:
):
"""Test filtering messages by channel_idx."""
# Channel 1 should match sample_message_with_receiver
response = client_no_auth.get("/api/v1/messages?channel_idx=1")
response = client_no_auth.get("/api/v1/messages?channel_idx=17")
assert response.status_code == 200
data = response.json()
assert len(data["items"]) == 1
assert data["items"][0]["channel_idx"] == 1
assert data["items"][0]["channel_idx"] == 17
# Channel 0 should return no results
response = client_no_auth.get("/api/v1/messages?channel_idx=0")
@@ -251,14 +253,14 @@ class TestListMessagesFilters:
# Create two messages, each observed by a different receiver
msg1 = Message(
message_type="channel",
channel_idx=1,
channel_idx=17,
text="Msg from receiver A",
received_at=datetime.now(timezone.utc),
observer_node_id=receiver_node.id,
)
msg2 = Message(
message_type="channel",
channel_idx=2,
channel_idx=17,
text="Msg from receiver B",
received_at=datetime.now(timezone.utc),
observer_node_id=second_receiver.id,
@@ -381,7 +383,7 @@ class TestMessageSort:
now = datetime.now(timezone.utc)
msg_ch = Message(
message_type="channel",
channel_idx=1,
channel_idx=17,
text="Channel msg",
received_at=now,
)
@@ -445,6 +447,99 @@ class TestMessageSort:
assert items[0]["text"] == "Alpha message"
assert items[1]["text"] == "Zebra message"
def test_sort_by_type_desc(self, client_no_auth, api_db_session):
"""sort=type&order=desc sorts by message_type descending."""
now = datetime.now(timezone.utc)
msg_ch = Message(
message_type="channel",
channel_idx=17,
text="Channel msg",
received_at=now,
)
msg_ct = Message(
message_type="contact",
text="Contact msg",
received_at=now,
)
api_db_session.add_all([msg_ch, msg_ct])
api_db_session.commit()
response = client_no_auth.get("/api/v1/messages?sort=type&order=desc")
assert response.status_code == 200
items = response.json()["items"]
assert items[0]["message_type"] == "contact"
assert items[1]["message_type"] == "channel"
def test_sort_by_from_desc(self, client_no_auth, api_db_session):
"""sort=from&order=desc sorts by pubkey_prefix descending."""
now = datetime.now(timezone.utc)
msg_b = Message(
message_type="direct",
pubkey_prefix="bb_prefix",
text="From B",
received_at=now,
)
msg_a = Message(
message_type="direct",
pubkey_prefix="aa_prefix",
text="From A",
received_at=now,
)
api_db_session.add_all([msg_b, msg_a])
api_db_session.commit()
response = client_no_auth.get("/api/v1/messages?sort=from&order=desc")
assert response.status_code == 200
items = response.json()["items"]
assert items[0]["text"] == "From B"
assert items[1]["text"] == "From A"
def test_sort_by_message_desc(self, client_no_auth, api_db_session):
"""sort=message&order=desc sorts by text descending."""
now = datetime.now(timezone.utc)
msg_b = Message(
message_type="direct",
text="Zebra message",
received_at=now,
)
msg_a = Message(
message_type="direct",
text="Alpha message",
received_at=now,
)
api_db_session.add_all([msg_b, msg_a])
api_db_session.commit()
response = client_no_auth.get("/api/v1/messages?sort=message&order=desc")
assert response.status_code == 200
items = response.json()["items"]
assert items[0]["text"] == "Zebra message"
assert items[1]["text"] == "Alpha message"
def test_sort_by_time_asc(self, client_no_auth, api_db_session):
"""sort=time&order=asc sorts by received_at ascending."""
now = datetime.now(timezone.utc)
msg_old = Message(
message_type="direct",
pubkey_prefix="aa",
text="Old msg",
received_at=now - timedelta(hours=1),
)
msg_new = Message(
message_type="direct",
pubkey_prefix="bb",
text="New msg",
received_at=now,
)
api_db_session.add_all([msg_old, msg_new])
api_db_session.commit()
response = client_no_auth.get("/api/v1/messages?sort=time&order=asc")
assert response.status_code == 200
items = response.json()["items"]
assert items[0]["text"] == "Old msg"
assert items[1]["text"] == "New msg"
def test_sort_invalid_ignored(self, client_no_auth, api_db_session):
"""Invalid sort value falls back to default (time desc)."""
now = datetime.now(timezone.utc)
@@ -465,3 +560,160 @@ class TestMessageSort:
assert response.status_code == 200
items = response.json()["items"]
assert items[0]["text"] == "New"
class TestMessageChannelVisibility:
"""Tests for channel visibility filtering on messages."""
@pytest.fixture
def messages_with_visibility(self, api_db_session):
"""Create messages on public and admin channels."""
pub_key = "AABBCCDDEEFF00112233445566778899"
adm_key = "FFEEDDCCBBAA99887766554433221100"
pub_idx = int(Channel.compute_channel_hash(pub_key), 16)
adm_idx = int(Channel.compute_channel_hash(adm_key), 16)
pub_ch = Channel(
name="CommunityCh",
key_hex=pub_key,
channel_hash=Channel.compute_channel_hash(pub_key),
visibility="community",
enabled=True,
)
adm_ch = Channel(
name="AdminCh",
key_hex=adm_key,
channel_hash=Channel.compute_channel_hash(adm_key),
visibility="admin",
enabled=True,
)
api_db_session.add_all([pub_ch, adm_ch])
pub_msg = Message(
message_type="channel",
channel_idx=pub_idx,
text="Community channel message",
received_at=datetime.now(timezone.utc),
)
adm_msg = Message(
message_type="channel",
channel_idx=adm_idx,
text="Admin channel message",
received_at=datetime.now(timezone.utc),
)
direct_msg = Message(
message_type="direct",
pubkey_prefix="abc123",
text="Direct message",
received_at=datetime.now(timezone.utc),
)
api_db_session.add_all([pub_msg, adm_msg, direct_msg])
api_db_session.commit()
return pub_msg, adm_msg, direct_msg
def test_anonymous_sees_only_community_channel_messages(
self, client_no_auth, messages_with_visibility
):
"""Anonymous users see community channel and direct messages only."""
response = client_no_auth.get("/api/v1/messages")
assert response.status_code == 200
data = response.json()
assert data["total"] == 2
texts = {item["text"] for item in data["items"]}
assert "Community channel message" in texts
assert "Direct message" in texts
assert "Admin channel message" not in texts
def test_admin_sees_all_channel_messages(
self, client_no_auth, messages_with_visibility
):
"""Admin users see all channel messages."""
response = client_no_auth.get(
"/api/v1/messages",
headers={"X-User-Roles": "admin"},
)
assert response.status_code == 200
data = response.json()
assert data["total"] == 3
texts = {item["text"] for item in data["items"]}
assert "Community channel message" in texts
assert "Admin channel message" in texts
assert "Direct message" in texts
def test_get_message_hidden_channel_returns_404(
self, client_no_auth, messages_with_visibility
):
"""Getting a message on a hidden channel returns 404."""
pub_msg, adm_msg, direct_msg = messages_with_visibility
response = client_no_auth.get(f"/api/v1/messages/{adm_msg.id}")
assert response.status_code == 404
def test_get_message_hidden_channel_visible_to_admin(
self, client_no_auth, messages_with_visibility
):
"""Admin can get a message on an admin channel."""
pub_msg, adm_msg, direct_msg = messages_with_visibility
response = client_no_auth.get(
f"/api/v1/messages/{adm_msg.id}",
headers={"X-User-Roles": "admin"},
)
assert response.status_code == 200
assert response.json()["text"] == "Admin channel message"
def test_get_message_community_channel_visible(
self, client_no_auth, messages_with_visibility
):
"""Anonymous can get a message on a community channel."""
pub_msg, adm_msg, direct_msg = messages_with_visibility
response = client_no_auth.get(f"/api/v1/messages/{pub_msg.id}")
assert response.status_code == 200
assert response.json()["text"] == "Community channel message"
def test_direct_messages_always_visible(
self, client_no_auth, messages_with_visibility
):
"""Direct messages are always visible regardless of channel visibility."""
pub_msg, adm_msg, direct_msg = messages_with_visibility
response = client_no_auth.get(f"/api/v1/messages/{direct_msg.id}")
assert response.status_code == 200
assert response.json()["text"] == "Direct message"
def test_get_message_channel_null_idx_not_filtered(
self, client_no_auth, api_db_session
):
"""Channel message with channel_idx=None bypasses visibility filter."""
msg = Message(
message_type="channel",
channel_idx=None,
text="Channel msg no idx",
received_at=datetime.now(timezone.utc),
)
api_db_session.add(msg)
api_db_session.commit()
response = client_no_auth.get(f"/api/v1/messages/{msg.id}")
assert response.status_code == 200
assert response.json()["text"] == "Channel msg no idx"
def test_get_message_no_observers_without_event_hash(
self, client_no_auth, api_db_session
):
"""Message without event_hash returns empty observers list."""
msg = Message(
message_type="direct",
pubkey_prefix="nohash1",
text="No hash msg",
received_at=datetime.now(timezone.utc),
event_hash=None,
)
api_db_session.add(msg)
api_db_session.commit()
response = client_no_auth.get(f"/api/v1/messages/{msg.id}")
assert response.status_code == 200
assert response.json()["observers"] == []
tests/test_collector/test_cli.py
+374 -29
View File
@@ -2,23 +2,45 @@
from unittest.mock import MagicMock, patch
import pytest
from click.testing import CliRunner
from meshcore_hub.collector.cli import collector
from meshcore_hub.collector.cli import _import_channels, collector
from meshcore_hub.common.database import DatabaseManager
from meshcore_hub.common.models.channel import Channel
def _make_mock_settings(db_url: str, seed_home: str = "/tmp/seed") -> MagicMock:
mock_settings = MagicMock(
data_home="/tmp/data",
effective_seed_home=seed_home,
effective_database_url=db_url,
node_tags_file=f"{seed_home}/node_tags.yaml",
channels_file=f"{seed_home}/channels.yaml",
)
mock_settings.model_copy.return_value = mock_settings
return mock_settings
def _invoke_channel_cmd(runner: CliRunner, db_url: str, args: list[str]):
mock_settings = _make_mock_settings(db_url)
with patch(
"meshcore_hub.common.config.get_collector_settings",
return_value=mock_settings,
):
return runner.invoke(
collector,
["--database-url", db_url] + args,
catch_exceptions=False,
)
class TestCollectorGroup:
"""Tests for the collector group command."""
def test_collector_without_subcommand_calls_run_service(self):
"""Invoking collector without subcommand calls _run_collector_service."""
runner = CliRunner()
mock_settings = MagicMock(
data_home="/tmp/data",
effective_seed_home="/tmp/seed",
effective_database_url="sqlite:///tmp/test.db",
)
mock_settings.model_copy.return_value = mock_settings
mock_settings = _make_mock_settings("sqlite:///tmp/test.db")
with (
patch(
@@ -35,14 +57,8 @@ class TestCollectorGroup:
mock_run.assert_called_once()
def test_collector_with_data_home_override(self):
"""--data-home overrides the default data home."""
runner = CliRunner()
mock_settings = MagicMock(
data_home="/default",
effective_seed_home="/default/seed",
effective_database_url="sqlite:///default/db",
)
mock_settings.model_copy.return_value = mock_settings
mock_settings = _make_mock_settings("sqlite:///default/db")
with (
patch(
@@ -68,14 +84,8 @@ class TestCollectorRunSubcommand:
"""Tests for the 'collector run' subcommand."""
def test_run_subcommand_calls_run_service(self):
"""'collector run' delegates to _run_collector_service."""
runner = CliRunner()
mock_settings = MagicMock(
data_home="/tmp/data",
effective_seed_home="/tmp/seed",
effective_database_url="sqlite:///tmp/test.db",
)
mock_settings.model_copy.return_value = mock_settings
mock_settings = _make_mock_settings("sqlite:///tmp/test.db")
with (
patch(
@@ -94,14 +104,8 @@ class TestCollectorSeedSubcommand:
"""Tests for the 'collector seed' subcommand."""
def test_seed_command_help(self):
"""'collector seed --help' shows usage."""
runner = CliRunner()
mock_settings = MagicMock(
data_home="/tmp/data",
effective_seed_home="/tmp/seed",
effective_database_url="sqlite:///tmp/test.db",
)
mock_settings.model_copy.return_value = mock_settings
mock_settings = _make_mock_settings("sqlite:///tmp/test.db")
with patch(
"meshcore_hub.common.config.get_collector_settings",
@@ -111,3 +115,344 @@ class TestCollectorSeedSubcommand:
assert result.exit_code == 0
assert "seed" in result.output.lower() or "import" in result.output.lower()
class TestChannelCommands:
"""Integration tests for channel CLI commands using real SQLite."""
@pytest.fixture
def cli_db_url(self, tmp_path):
db_path = tmp_path / "test.db"
db_url = f"sqlite:///{db_path}"
db = DatabaseManager(db_url)
db.create_tables()
db.dispose()
return db_url
def test_channel_list_empty(self, cli_db_url):
runner = CliRunner()
result = _invoke_channel_cmd(runner, cli_db_url, ["channel", "list"])
assert result.exit_code == 0
assert "No channels found." in result.output
def test_channel_add_success(self, cli_db_url):
runner = CliRunner()
result = _invoke_channel_cmd(
runner,
cli_db_url,
["channel", "add", "--name", "TestCh", "--key", "AABB" * 8],
)
assert result.exit_code == 0
assert "TestCh" in result.output
assert "added" in result.output
result = _invoke_channel_cmd(runner, cli_db_url, ["channel", "list"])
assert "TestCh" in result.output
def test_channel_add_duplicate_name(self, cli_db_url):
runner = CliRunner()
_invoke_channel_cmd(
runner, cli_db_url, ["channel", "add", "--name", "Dup", "--key", "A" * 32]
)
result = _invoke_channel_cmd(
runner, cli_db_url, ["channel", "add", "--name", "Dup", "--key", "B" * 32]
)
assert result.exit_code == 0
assert "already exists" in result.output
def test_channel_add_custom_visibility(self, cli_db_url):
runner = CliRunner()
result = _invoke_channel_cmd(
runner,
cli_db_url,
[
"channel",
"add",
"--name",
"PrivateCh",
"--key",
"C" * 32,
"--visibility",
"member",
],
)
assert result.exit_code == 0
assert "added" in result.output
db = DatabaseManager(cli_db_url)
with db.session_scope() as session:
ch = session.query(Channel).filter(Channel.name == "PrivateCh").first()
assert ch is not None
assert ch.visibility == "member"
db.dispose()
def test_channel_list_with_data(self, cli_db_url):
runner = CliRunner()
_invoke_channel_cmd(
runner,
cli_db_url,
["channel", "add", "--name", "Alpha", "--key", "D" * 32],
)
result = _invoke_channel_cmd(runner, cli_db_url, ["channel", "list"])
assert result.exit_code == 0
assert "Alpha" in result.output
assert "Yes" in result.output
def test_channel_remove_success(self, cli_db_url):
runner = CliRunner()
_invoke_channel_cmd(
runner,
cli_db_url,
["channel", "add", "--name", "Gone", "--key", "E" * 32],
)
result = _invoke_channel_cmd(
runner, cli_db_url, ["channel", "remove", "--name", "Gone"]
)
assert result.exit_code == 0
assert "removed" in result.output
result = _invoke_channel_cmd(runner, cli_db_url, ["channel", "list"])
assert "Gone" not in result.output
def test_channel_remove_not_found(self, cli_db_url):
runner = CliRunner()
result = _invoke_channel_cmd(
runner, cli_db_url, ["channel", "remove", "--name", "Missing"]
)
assert result.exit_code == 0
assert "not found" in result.output
def test_channel_disable_then_enable(self, cli_db_url):
runner = CliRunner()
_invoke_channel_cmd(
runner,
cli_db_url,
["channel", "add", "--name", "Toggle", "--key", "F" * 32],
)
result = _invoke_channel_cmd(
runner, cli_db_url, ["channel", "disable", "--name", "Toggle"]
)
assert result.exit_code == 0
assert "disabled" in result.output
db = DatabaseManager(cli_db_url)
with db.session_scope() as session:
ch = session.query(Channel).filter(Channel.name == "Toggle").first()
assert ch is not None
assert ch.enabled is False
db.dispose()
result = _invoke_channel_cmd(
runner, cli_db_url, ["channel", "enable", "--name", "Toggle"]
)
assert result.exit_code == 0
assert "enabled" in result.output
db = DatabaseManager(cli_db_url)
with db.session_scope() as session:
ch = session.query(Channel).filter(Channel.name == "Toggle").first()
assert ch is not None
assert ch.enabled is True
db.dispose()
def test_channel_enable_not_found(self, cli_db_url):
runner = CliRunner()
result = _invoke_channel_cmd(
runner, cli_db_url, ["channel", "enable", "--name", "Missing"]
)
assert result.exit_code == 0
assert "not found" in result.output
def test_channel_disable_not_found(self, cli_db_url):
runner = CliRunner()
result = _invoke_channel_cmd(
runner, cli_db_url, ["channel", "disable", "--name", "Missing"]
)
assert result.exit_code == 0
assert "not found" in result.output
class TestImportChannels:
"""Unit tests for _import_channels YAML import function."""
@pytest.fixture
def import_db(self, tmp_path):
db_path = tmp_path / "import.db"
db = DatabaseManager(f"sqlite:///{db_path}")
db.create_tables()
yield db
db.dispose()
def _write_yaml(self, tmp_path, content: str) -> str:
yaml_file = tmp_path / "channels.yaml"
yaml_file.write_text(content)
return str(yaml_file)
def test_import_shorthand_format(self, import_db, tmp_path):
key = "AABBCCDDEEFF00112233445566778899"
path = self._write_yaml(tmp_path, f"TestCh: {key}\n")
result = _import_channels(path, import_db)
assert result["created"] == 1
assert result["updated"] == 0
assert result["errors"] == []
with import_db.session_scope() as session:
ch = session.query(Channel).filter(Channel.name == "TestCh").first()
assert ch is not None
assert ch.key_hex == key.upper()
assert ch.visibility == "community"
assert ch.enabled is True
def test_import_expanded_format(self, import_db, tmp_path):
key = "11223344556677889900AABBCCDDEEFF"
path = self._write_yaml(tmp_path, f"Expanded: {{key: {key}, enabled: false}}\n")
result = _import_channels(path, import_db)
assert result["created"] == 1
assert result["updated"] == 0
with import_db.session_scope() as session:
ch = session.query(Channel).filter(Channel.name == "Expanded").first()
assert ch is not None
assert ch.enabled is False
def test_import_updates_existing(self, import_db, tmp_path):
old_key = "A" * 32
new_key = "B" * 32
path1 = self._write_yaml(tmp_path, f"MyCh: {old_key}\n")
_import_channels(path1, import_db)
path2 = self._write_yaml(tmp_path, f"MyCh: {new_key}\n")
result = _import_channels(path2, import_db)
assert result["created"] == 0
assert result["updated"] == 1
with import_db.session_scope() as session:
ch = session.query(Channel).filter(Channel.name == "MyCh").first()
assert ch.key_hex == new_key.upper()
assert ch.channel_hash == Channel.compute_channel_hash(new_key.upper())
def test_import_empty_yaml(self, import_db, tmp_path):
path = self._write_yaml(tmp_path, "")
result = _import_channels(path, import_db)
assert result["created"] == 0
assert result["updated"] == 0
errors: list[str] = result["errors"] # type: ignore[assignment]
assert errors == []
def test_import_invalid_format(self, import_db, tmp_path):
path = self._write_yaml(tmp_path, "BadCh: 12345\n")
result = _import_channels(path, import_db)
assert result["created"] == 0
errors: list[str] = result["errors"] # type: ignore[assignment]
assert len(errors) == 1
assert "Invalid format" in errors[0]
def test_import_empty_key(self, import_db, tmp_path):
path = self._write_yaml(tmp_path, "EmptyKey: {key: ''}\n")
result = _import_channels(path, import_db)
assert result["created"] == 0
errors: list[str] = result["errors"] # type: ignore[assignment]
assert len(errors) == 1
assert "Empty key" in errors[0]
def test_import_exception_handling(self, import_db, tmp_path):
path = self._write_yaml(tmp_path, "Boom: AABBCCDDEEFF00112233445566778899\n")
with patch(
"meshcore_hub.common.models.channel.Channel",
side_effect=RuntimeError("db boom"),
):
result = _import_channels(path, import_db)
errors: list[str] = result["errors"] # type: ignore[assignment]
assert len(errors) == 1
assert "Boom" in errors[0]
def test_import_multiple_channels(self, import_db, tmp_path):
path = self._write_yaml(
tmp_path,
"Ch1: AABBCCDDEEFF00112233445566778899\n"
"Ch2: 11223344556677889900AABBCCDDEEFF\n",
)
result = _import_channels(path, import_db)
assert result["created"] == 2
assert result["updated"] == 0
class TestChannelSeedImport:
"""Integration tests for seed command with channels.yaml."""
def test_seed_imports_channels_yaml(self, tmp_path):
runner = CliRunner()
seed_dir = tmp_path / "seed"
seed_dir.mkdir()
(seed_dir / "channels.yaml").write_text(
"SeededCh: AABBCCDDEEFF00112233445566778899\n"
)
db_path = tmp_path / "seed_test.db"
db_url = f"sqlite:///{db_path}"
db = DatabaseManager(db_url)
db.create_tables()
db.dispose()
mock_settings = _make_mock_settings(db_url, seed_home=str(seed_dir))
with patch(
"meshcore_hub.common.config.get_collector_settings",
return_value=mock_settings,
):
result = runner.invoke(
collector,
["--database-url", db_url, "--seed-home", str(seed_dir), "seed"],
catch_exceptions=False,
)
assert result.exit_code == 0
assert "Channels: 1 created" in result.output
db = DatabaseManager(db_url)
with db.session_scope() as session:
ch = session.query(Channel).filter(Channel.name == "SeededCh").first()
assert ch is not None
assert ch.visibility == "community"
db.dispose()
def test_seed_no_seed_files(self, tmp_path):
runner = CliRunner()
empty_seed = tmp_path / "empty_seed"
empty_seed.mkdir()
db_path = tmp_path / "noseed.db"
db_url = f"sqlite:///{db_path}"
db = DatabaseManager(db_url)
db.create_tables()
db.dispose()
mock_settings = _make_mock_settings(db_url, seed_home=str(empty_seed))
with patch(
"meshcore_hub.common.config.get_collector_settings",
return_value=mock_settings,
):
result = runner.invoke(
collector,
[
"--database-url",
db_url,
"--seed-home",
str(empty_seed),
"seed",
],
catch_exceptions=False,
)
assert result.exit_code == 0
assert "No seed files found" in result.output
tests/test_collector/test_letsmesh_decoder.py
+80 -1
View File
@@ -118,7 +118,7 @@ def test_channel_labels_by_index_includes_labeled_entries() -> None:
labels = decoder.channel_labels_by_index()
assert labels[17] == "Public"
assert labels[217] == "test"
assert labels[217] == "Test"
assert labels[202] == "bot"
assert labels[184] == "chat"
@@ -195,3 +195,82 @@ def test_decode_payload_returns_none_for_non_string_raw() -> None:
decoder = LetsMeshPacketDecoder()
assert decoder.decode_payload({"raw": 12345}) is None
assert decoder.decode_payload({"raw": None}) is None
def test_reload_keys_replaces_existing_keys() -> None:
"""reload_keys replaces channel keys and names."""
decoder = LetsMeshPacketDecoder(
channel_keys=["bot=EB50A1BCB3E4E5D7BF69A57C9DADA211"]
)
new_key = "D0BDD6D71538138ED979EEC00D98AD97"
decoder.reload_keys([f"chat={new_key}"])
assert new_key.upper() in decoder._channel_keys
assert "EB50A1BCB3E4E5D7BF69A57C9DADA211" not in decoder._channel_keys
new_hash = LetsMeshPacketDecoder._compute_channel_hash(new_key)
assert decoder._channel_names_by_hash.get(new_hash) == "chat"
def test_reload_keys_with_empty_list_keeps_builtins() -> None:
"""Reloading with empty list retains builtin keys."""
decoder = LetsMeshPacketDecoder(
channel_keys=["bot=EB50A1BCB3E4E5D7BF69A57C9DADA211"]
)
decoder.reload_keys([])
assert "8B3387E9C5CDEA6AC9E5EDBAA115CD72" in decoder._channel_keys
assert "9CD8FCF22A47333B591D96A2B848B73F" in decoder._channel_keys
assert "EB50A1BCB3E4E5D7BF69A57C9DADA211" not in decoder._channel_keys
def test_enrich_payload_decoded_merges_attributes() -> None:
"""_enrich_payload_decoded merges payload object attributes into dict."""
payload_obj = MagicMock()
payload_obj.channel_hash = "AB"
payload_obj.decrypted = {"message": "hello"}
payload_obj.sender_public_key = "DEADBEEF"
payload_obj.cipher_mac = None
payload_obj.ciphertext = None
payload_obj.ciphertext_length = None
payload_obj.destination_hash = None
payload_obj.source_hash = None
payload_obj.path_length = None
payload_obj.path_hashes = None
payload_obj.extra_type = None
payload_obj.extra_data = None
payload_obj.checksum = None
decoded_dict = {
"payload": {
"decoded": {
"type": 5,
}
}
}
LetsMeshPacketDecoder._enrich_payload_decoded(decoded_dict, payload_obj)
decoded = decoded_dict["payload"]["decoded"]
assert decoded["channelHash"] == "AB"
assert decoded["decrypted"] == {"message": "hello"}
assert decoded["senderPublicKey"] == "DEADBEEF"
assert "cipherMac" not in decoded
def test_channel_name_from_decoded_returns_none_for_non_dict() -> None:
"""channel_name_from_decoded returns None for non-dict inputs."""
decoder = LetsMeshPacketDecoder()
assert decoder.channel_name_from_decoded(None) is None
assert decoder.channel_name_from_decoded("string") is None # type: ignore[arg-type]
assert decoder.channel_name_from_decoded(42) is None # type: ignore[arg-type]
assert decoder.channel_name_from_decoded({"payload": "not a dict"}) is None
assert (
decoder.channel_name_from_decoded({"payload": {"decoded": "not a dict"}})
is None
)
assert (
decoder.channel_name_from_decoded(
{"payload": {"decoded": {"channelHash": 123}}}
)
is None
)
tests/test_collector/test_subscriber.py
+202 -1
View File
@@ -402,8 +402,8 @@ class TestSubscriber:
subscriber = Subscriber(
mock_mqtt_client,
db_manager,
include_test_channel=True,
)
subscriber._include_test_channel = True
handler = MagicMock()
subscriber.register_handler("channel_msg_recv", handler)
subscriber.start()
@@ -984,3 +984,204 @@ class TestCreateSubscriber:
assert subscriber is not None
MockMQTT.assert_called_once()
class TestChannelKeyRefresh:
"""Tests for channel key loading and refresh from database."""
@pytest.fixture
def mock_mqtt_client(self):
"""Create a mock MQTT client."""
client = MagicMock()
client.topic_builder = MagicMock()
client.topic_builder.prefix = "meshcore"
client.topic_builder.parse_letsmesh_upload_topic.return_value = (
"a" * 64,
"status",
)
return client
def test_load_channel_keys_from_db(self, mock_mqtt_client, db_manager):
"""Test loading channel keys from database."""
from meshcore_hub.common.models.channel import Channel
with db_manager.session_scope() as session:
ch = Channel(
name="TestCh",
key_hex="AABBCCDDEEFF00112233445566778899",
channel_hash=Channel.compute_channel_hash(
"AABBCCDDEEFF00112233445566778899"
),
visibility="community",
enabled=True,
)
session.add(ch)
subscriber = Subscriber(mock_mqtt_client, db_manager)
assert len(subscriber._db_channel_keys) == 1
assert "TestCh=AABBCCDDEEFF00112233445566778899" in subscriber._db_channel_keys
def test_load_channel_keys_detects_test_channel(self, mock_mqtt_client, db_manager):
"""Test that test channel is detected by name."""
from meshcore_hub.common.models.channel import Channel
with db_manager.session_scope() as session:
ch = Channel(
name="Test",
key_hex="AABBCCDDEEFF00112233445566778899",
channel_hash=Channel.compute_channel_hash(
"AABBCCDDEEFF00112233445566778899"
),
visibility="community",
enabled=True,
)
session.add(ch)
subscriber = Subscriber(mock_mqtt_client, db_manager)
assert subscriber._include_test_channel is True
def test_load_channel_keys_only_enabled(self, mock_mqtt_client, db_manager):
"""Test that only enabled channels are loaded."""
from meshcore_hub.common.models.channel import Channel
with db_manager.session_scope() as session:
ch1 = Channel(
name="Enabled",
key_hex="AABBCCDDEEFF00112233445566778899",
channel_hash=Channel.compute_channel_hash(
"AABBCCDDEEFF00112233445566778899"
),
enabled=True,
)
ch2 = Channel(
name="Disabled",
key_hex="11223344556677889900AABBCCDDEEFF",
channel_hash=Channel.compute_channel_hash(
"11223344556677889900AABBCCDDEEFF"
),
enabled=False,
)
session.add_all([ch1, ch2])
subscriber = Subscriber(mock_mqtt_client, db_manager)
assert len(subscriber._db_channel_keys) == 1
assert "Enabled=" in subscriber._db_channel_keys[0]
def test_load_channel_keys_handles_db_error(self, mock_mqtt_client, db_manager):
"""Test graceful handling of database errors during key loading."""
broken_db = MagicMock()
broken_db.session_scope.side_effect = Exception("DB connection failed")
subscriber = Subscriber(mock_mqtt_client, broken_db)
assert subscriber._db_channel_keys == []
assert subscriber._include_test_channel is False
def test_refresh_channel_keys_from_db(self, mock_mqtt_client, db_manager):
"""Test refreshing channel keys reloads the decoder."""
from meshcore_hub.common.models.channel import Channel
subscriber = Subscriber(mock_mqtt_client, db_manager)
assert len(subscriber._db_channel_keys) == 0
with db_manager.session_scope() as session:
ch = Channel(
name="NewCh",
key_hex="CCDDEEFF00112233445566778899AABB",
channel_hash=Channel.compute_channel_hash(
"CCDDEEFF00112233445566778899AABB"
),
visibility="community",
enabled=True,
)
session.add(ch)
with patch.object(subscriber._letsmesh_decoder, "reload_keys") as mock_reload:
subscriber._refresh_channel_keys_from_db()
assert len(subscriber._db_channel_keys) == 1
mock_reload.assert_called_once_with(subscriber._db_channel_keys)
def test_refresh_handles_db_error(self, mock_mqtt_client, db_manager):
"""Test refresh handles database errors gracefully."""
broken_db = MagicMock()
def broken_scope():
raise Exception("DB error")
broken_db.session_scope.side_effect = broken_scope
subscriber = Subscriber(broken_db, broken_db)
with patch.object(subscriber._letsmesh_decoder, "reload_keys"):
subscriber._refresh_channel_keys_from_db()
assert subscriber._db_channel_keys == []
def test_channel_refresh_scheduler_starts(self, mock_mqtt_client, db_manager):
"""Test channel refresh scheduler starts a daemon thread."""
subscriber = Subscriber(
mock_mqtt_client, db_manager, channel_refresh_interval_seconds=300
)
subscriber._running = True
subscriber._start_channel_refresh_scheduler()
assert subscriber._channel_refresh_thread is not None
assert subscriber._channel_refresh_thread.daemon is True
subscriber._running = False
subscriber._channel_refresh_thread.join(timeout=2.0)
def test_channel_refresh_scheduler_disabled(self, mock_mqtt_client, db_manager):
"""Test channel refresh scheduler is disabled when interval is 0."""
subscriber = Subscriber(
mock_mqtt_client, db_manager, channel_refresh_interval_seconds=0
)
subscriber._running = True
subscriber._start_channel_refresh_scheduler()
assert subscriber._channel_refresh_thread is None
subscriber._running = False
def test_channel_refresh_scheduler_stop(self, mock_mqtt_client, db_manager):
"""Test stopping the channel refresh scheduler."""
subscriber = Subscriber(
mock_mqtt_client, db_manager, channel_refresh_interval_seconds=300
)
subscriber._running = True
subscriber._start_channel_refresh_scheduler()
subscriber._running = False
subscriber._stop_channel_refresh_scheduler()
assert subscriber._channel_refresh_thread is not None
assert not subscriber._channel_refresh_thread.is_alive()
def test_load_channel_keys_empty_db(self, mock_mqtt_client, db_manager):
"""Test loading channel keys from empty database."""
subscriber = Subscriber(mock_mqtt_client, db_manager)
assert subscriber._db_channel_keys == []
assert subscriber._include_test_channel is False
def test_decoder_initialized_with_db_keys(self, mock_mqtt_client, db_manager):
"""Test decoder is initialized with database channel keys."""
from meshcore_hub.common.models.channel import Channel
key_hex = "DDEEFF00112233445566778899AABBCC"
with db_manager.session_scope() as session:
ch = Channel(
name="DecCh",
key_hex=key_hex,
channel_hash=Channel.compute_channel_hash(key_hex),
visibility="community",
enabled=True,
)
session.add(ch)
subscriber = Subscriber(mock_mqtt_client, db_manager)
assert key_hex in subscriber._letsmesh_decoder._channel_keys
tests/test_common/test_channel_model.py
+241
View File
@@ -0,0 +1,241 @@
"""Tests for Channel model and channel Pydantic schemas."""
import hashlib
import pytest
from pydantic import ValidationError
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from meshcore_hub.common.models import Base, Channel, ChannelVisibility
from meshcore_hub.common.schemas.channels import ChannelCreate, ChannelUpdate
@pytest.fixture
def db_session():
"""Create an in-memory SQLite database session."""
engine = create_engine(
"sqlite:///:memory:",
connect_args={"check_same_thread": False},
)
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
yield session
session.close()
Base.metadata.drop_all(engine)
engine.dispose()
class TestChannelModel:
"""Tests for Channel SQLAlchemy model."""
def test_create_channel(self, db_session) -> None:
"""Test creating a channel in the database."""
key_hex = "AABBCCDDEEFF00112233445566778899"
channel = Channel(
name="TestCh",
key_hex=key_hex,
channel_hash=Channel.compute_channel_hash(key_hex),
visibility="community",
enabled=True,
)
db_session.add(channel)
db_session.commit()
assert channel.id is not None
assert channel.name == "TestCh"
assert channel.key_hex == key_hex
assert channel.visibility == "community"
assert channel.enabled is True
def test_channel_repr(self, db_session) -> None:
"""Test channel string representation."""
channel = Channel(
name="MyChannel",
key_hex="AA" * 16,
channel_hash="AB",
visibility="member",
)
assert repr(channel) == "<Channel(name=MyChannel, hash=AB, visibility=member)>"
def test_compute_channel_hash_aes128(self) -> None:
"""Test channel hash computation for AES-128 key (32 hex chars)."""
key_hex = "8B3387E9C5CDEA6AC9E5EDBAA115CD72"
expected = hashlib.sha256(bytes.fromhex(key_hex)).digest()[:1].hex().upper()
result = Channel.compute_channel_hash(key_hex)
assert result == expected
assert len(result) == 2
def test_compute_channel_hash_aes256(self) -> None:
"""Test channel hash computation for AES-256 key (64 hex chars)."""
key_hex = "A" * 64
expected = hashlib.sha256(bytes.fromhex(key_hex)).digest()[:1].hex().upper()
result = Channel.compute_channel_hash(key_hex)
assert result == expected
assert len(result) == 2
def test_masked_key_normal(self) -> None:
"""Test masked key shows first/last 4 chars for long keys."""
channel = Channel(
name="Ch",
key_hex="AABBCCDDEEFF00112233445566778899",
channel_hash="AB",
)
assert channel.masked_key == "AABB...8899"
def test_masked_key_short(self) -> None:
"""Test masked key returns full key when <= 8 chars."""
channel = Channel(
name="Ch",
key_hex="AABBCCDD",
channel_hash="AB",
)
assert channel.masked_key == "AABBCCDD"
def test_channel_visibility_enum(self) -> None:
"""Test ChannelVisibility enum values."""
assert ChannelVisibility.COMMUNITY.value == "community"
assert ChannelVisibility.MEMBER.value == "member"
assert ChannelVisibility.OPERATOR.value == "operator"
assert ChannelVisibility.ADMIN.value == "admin"
def test_channel_unique_name_constraint(self, db_session) -> None:
"""Test that duplicate channel names raise an error."""
key1 = "AABBCCDDEEFF00112233445566778899"
key2 = "11223344556677889900AABBCCDDEEFF"
ch1 = Channel(
name="Unique",
key_hex=key1,
channel_hash=Channel.compute_channel_hash(key1),
)
ch2 = Channel(
name="Unique",
key_hex=key2,
channel_hash=Channel.compute_channel_hash(key2),
)
db_session.add(ch1)
db_session.commit()
db_session.add(ch2)
with pytest.raises(Exception, match=""):
db_session.commit()
def test_channel_default_values(self, db_session) -> None:
"""Test channel default visibility and enabled."""
key_hex = "AABBCCDDEEFF00112233445566778899"
channel = Channel(
name="Defaults",
key_hex=key_hex,
channel_hash=Channel.compute_channel_hash(key_hex),
)
db_session.add(channel)
db_session.commit()
assert channel.visibility == "community"
assert channel.enabled is True
class TestChannelCreateSchema:
"""Tests for ChannelCreate Pydantic schema."""
def test_valid_32_char_key(self) -> None:
"""Test valid AES-128 key (32 hex chars)."""
schema = ChannelCreate(
name="Test",
key_hex="aabbccddeeff00112233445566778899",
)
assert schema.key_hex == "AABBCCDDEEFF00112233445566778899"
def test_valid_64_char_key(self) -> None:
"""Test valid AES-256 key (64 hex chars)."""
key = "A" * 64
schema = ChannelCreate(name="Test", key_hex=key)
assert schema.key_hex == key
def test_key_hex_uppercases(self) -> None:
"""Test that key_hex is normalized to uppercase."""
schema = ChannelCreate(
name="Test",
key_hex="aabbccddeeff00112233445566778899",
)
assert schema.key_hex == "AABBCCDDEEFF00112233445566778899"
def test_key_hex_strips_whitespace(self) -> None:
"""Test that key_hex strips whitespace."""
schema = ChannelCreate(
name="Test",
key_hex=" AABBCCDDEEFF00112233445566778899 ",
)
assert schema.key_hex == "AABBCCDDEEFF00112233445566778899"
def test_invalid_key_non_hex(self) -> None:
"""Test that non-hex characters are rejected."""
with pytest.raises(ValidationError, match="hexadecimal"):
ChannelCreate(name="Test", key_hex="G" * 32)
def test_invalid_key_wrong_length(self) -> None:
"""Test that wrong-length keys are rejected."""
with pytest.raises(ValidationError):
ChannelCreate(name="Test", key_hex="A" * 16)
def test_invalid_key_too_long(self) -> None:
"""Test that keys longer than 64 chars are rejected."""
with pytest.raises(ValidationError):
ChannelCreate(name="Test", key_hex="A" * 128)
def test_name_required(self) -> None:
"""Test that name is required."""
with pytest.raises(ValidationError):
ChannelCreate(key_hex="A" * 32) # type: ignore[call-arg]
def test_name_too_long(self) -> None:
"""Test that name max length is 100."""
with pytest.raises(ValidationError):
ChannelCreate(name="X" * 101, key_hex="A" * 32)
def test_default_visibility(self) -> None:
"""Test default visibility is community."""
schema = ChannelCreate(name="Test", key_hex="A" * 32)
assert schema.visibility == "community"
def test_default_enabled(self) -> None:
"""Test default enabled is True."""
schema = ChannelCreate(name="Test", key_hex="A" * 32)
assert schema.enabled is True
class TestChannelUpdateSchema:
"""Tests for ChannelUpdate Pydantic schema."""
def test_key_hex_none_passthrough(self) -> None:
"""Test that None key_hex is allowed."""
schema = ChannelUpdate()
assert schema.key_hex is None
def test_valid_key_hex(self) -> None:
"""Test valid key_hex update."""
schema = ChannelUpdate(key_hex="A" * 32)
assert schema.key_hex == "A" * 32
def test_key_hex_uppercases(self) -> None:
"""Test that key_hex is normalized to uppercase."""
schema = ChannelUpdate(key_hex="a" * 32)
assert schema.key_hex == "A" * 32
def test_invalid_key_non_hex(self) -> None:
"""Test that non-hex characters are rejected."""
with pytest.raises(ValidationError, match="hexadecimal"):
ChannelUpdate(key_hex="Z" * 32)
def test_invalid_key_wrong_length(self) -> None:
"""Test that wrong-length keys are rejected."""
with pytest.raises(ValidationError):
ChannelUpdate(key_hex="A" * 48)
def test_all_fields_optional(self) -> None:
"""Test that all fields are optional."""
schema = ChannelUpdate()
assert schema.key_hex is None
assert schema.visibility is None
assert schema.enabled is None
tests/test_common/test_config.py
+88 -8
View File
@@ -59,18 +59,33 @@ class TestCollectorSettings:
assert settings.effective_seed_home == "/seed/data"
assert settings.node_tags_file == "/seed/data/node_tags.yaml"
def test_collector_channel_keys_list(self) -> None:
"""Channel keys are parsed from comma/space-separated env values."""
def test_channel_refresh_interval_seconds(self) -> None:
"""Channel refresh interval defaults to 300."""
settings = CollectorSettings(_env_file=None)
assert settings.channel_refresh_interval_seconds == 300
def test_channel_refresh_interval_seconds_custom(self) -> None:
"""Channel refresh interval can be overridden."""
settings = CollectorSettings(
_env_file=None,
collector_channel_keys="aa11, bb22 cc33",
channel_refresh_interval_seconds=60,
)
assert settings.collector_channel_keys_list == [
"aa11",
"bb22",
"cc33",
]
assert settings.channel_refresh_interval_seconds == 60
def test_channels_file_path(self) -> None:
"""channels_file property resolves to seed_home/channels.yaml."""
settings = CollectorSettings(_env_file=None, seed_home="/seed/data")
assert settings.channels_file == "/seed/data/channels.yaml"
def test_channels_file_default(self) -> None:
"""channels_file uses default seed_home."""
settings = CollectorSettings(_env_file=None)
assert settings.channels_file.endswith("channels.yaml")
assert "seed" in settings.channels_file
class TestAPISettings:
@@ -107,3 +122,68 @@ class TestWebSettings:
settings = WebSettings(_env_file=None)
assert settings.network_announcement is None
def test_feature_channels_default_true(self) -> None:
"""Test that feature_channels defaults to True."""
settings = WebSettings(_env_file=None)
assert settings.feature_channels is True
def test_feature_channels_override(self) -> None:
"""Test that feature_channels can be disabled."""
settings = WebSettings(_env_file=None, feature_channels=False)
assert settings.feature_channels is False
def test_features_dict_includes_channels(self) -> None:
"""Test that features dict includes channels key."""
settings = WebSettings(_env_file=None)
features = settings.features
assert "channels" in features
assert features["channels"] is True
def test_features_dashboard_auto_disables(self) -> None:
"""Dashboard disables when nodes, ads, and messages all off."""
settings = WebSettings(
_env_file=None,
feature_dashboard=True,
feature_nodes=False,
feature_advertisements=False,
feature_messages=False,
)
assert settings.features["dashboard"] is False
def test_features_map_auto_disables_without_nodes(self) -> None:
"""Map disables when nodes feature is off."""
settings = WebSettings(
_env_file=None,
feature_map=True,
feature_nodes=False,
)
assert settings.features["map"] is False
def test_features_members_auto_disables_without_oidc(self) -> None:
"""Members disables when OIDC is not enabled."""
settings = WebSettings(
_env_file=None,
feature_members=True,
oidc_enabled=False,
)
assert settings.features["members"] is False
def test_features_all_enabled_by_default(self) -> None:
"""All features are enabled with default settings."""
settings = WebSettings(
_env_file=None,
oidc_enabled=True,
)
features = settings.features
assert features["dashboard"] is True
assert features["nodes"] is True
assert features["advertisements"] is True
assert features["messages"] is True
assert features["map"] is True
assert features["members"] is True
assert features["channels"] is True
assert features["pages"] is True
tests/test_common/test_i18n.py
+13 -1
View File
@@ -48,7 +48,7 @@ class TestTranslation:
def test_nested_key(self):
"""Deeply nested keys resolve correctly."""
assert t("entities.advertisements") == "Advertisements"
assert t("entities.advertisements") == "Adverts"
def test_missing_key_returns_key(self):
"""Missing key returns the key itself as fallback."""
@@ -152,3 +152,15 @@ class TestEnJsonCompleteness:
!= "advertisements.route_type_unknown"
)
assert t("advertisements.col_route_type") != "advertisements.col_route_type"
def test_channels_optgroup_keys(self):
"""Channel optgroup labels exist and resolve correctly."""
assert t("channels.optgroup_standard") == "Standard"
assert t("channels.optgroup_custom") == "Custom"
def test_channels_visibility_keys(self):
"""Channel visibility level labels exist and resolve correctly."""
assert t("channels.visibility_community") == "Community"
assert t("channels.visibility_member") == "Member"
assert t("channels.visibility_operator") == "Operator"
assert t("channels.visibility_admin") == "Admin"
tests/test_web/conftest.py
-2
View File
@@ -320,7 +320,6 @@ def web_app(mock_http_client: MockHttpClient, monkeypatch: pytest.MonkeyPatch) -
"""Create a web app with mocked HTTP client."""
# Ensure tests use a consistent locale regardless of local .env
monkeypatch.setenv("WEB_DATETIME_LOCALE", "en-US")
monkeypatch.setenv("COLLECTOR_INCLUDE_TEST_CHANNEL", "true")
monkeypatch.setenv("OIDC_ENABLED", "false")
monkeypatch.setenv("NETWORK_ANNOUNCEMENT", "")
app = create_app(
@@ -366,7 +365,6 @@ def web_app_with_oidc(
)
monkeypatch.setenv("OIDC_SESSION_SECRET", "test-session-secret")
monkeypatch.setenv("WEB_DATETIME_LOCALE", "en-US")
monkeypatch.setenv("COLLECTOR_INCLUDE_TEST_CHANNEL", "true")
monkeypatch.setenv("NETWORK_ANNOUNCEMENT", "")
app = create_app(
tests/test_web/test_messages.py
+1 -1
View File
@@ -103,4 +103,4 @@ class TestMessagesConfig:
config = json.loads(text[config_start:config_end])
assert config["channel_labels"]["17"] == "Public"
assert config["channel_labels"]["217"] == "test"
assert config["channel_labels"]["217"] == "Test"