adam/meshcore-hub
1
0
Fork 0
mirror of https://github.com/ipnet-mesh/meshcore-hub.git synced 2026-08-09 18:32:44 +02:00
Code Issues Packages Projects Releases Wiki Activity

Removed old agent skills

Browse Source
This commit is contained in:
Louis King
2026-05-20 14:13:22 +01:00
parent 5f6d44c7b8
commit 3ad288433d
4 changed files with 0 additions and 940 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/skills/docs-sync/SKILL.md
-175
View File
@@ -1,175 +0,0 @@
---
name: docs-sync
description: "Audits and fixes discrepancies between project source code (Python config, Docker Compose files) and primary documentation files (README.md, AGENTS.md, docs/upgrading.md, .env.example, SCHEMAS.md). Extracts environment variables from Pydantic Settings, Click CLI options, and os.getenv calls; parses Docker Compose services, profiles, volumes, and env passthroughs; verifies feature flags, CLI commands, and file paths referenced in documentation. Produces a structured audit report and applies fixes to keep documentation accurate and up-to-date. Invoke after any config change, env var addition/removal, Docker service modification, feature flag change, or when documentation drift is suspected."
license: MIT
compatibility: opencode
metadata:
author: https://github.com/agessaman
version: "0.1.0"
domain: quality
triggers: documentation sync, docs audit, env vars, config drift, .env.example, README, AGENTS.md, docs/upgrading.md, SCHEMAS.md, docker compose docs, feature flags, documentation update, keep docs in sync, documentation accuracy
role: specialist
scope: review
output-format: report
related-skills: code-review, docs-writer
---
# Docs Sync
Documentation accuracy specialist that keeps project docs in sync with source code and Docker configuration.
## When to Use This Skill
- After adding, removing, or renaming environment variables in Python config
- After modifying Docker Compose services, profiles, volumes, or port mappings
- After adding or removing feature flags
- After adding or removing CLI commands or subcommands
- After changing Pydantic Settings defaults or types
- When documentation drift is suspected
- Before releases to ensure docs are accurate
- When AGENTS.md or README.md references stale files or commands
## Primary Documentation Files
The following files are the documentation targets. All must be kept in sync:
| File | Role |
|------|------|
| `README.md` | User-facing reference: env var tables, Docker instructions, feature list |
| `AGENTS.md` | AI agent instructions: env var list, project structure, conventions |
| `docs/upgrading.md` | Upgrade guide: deprecated vars, new vars, migration steps |
| `docs/letsmesh.md` | LetsMesh packet decoding: normalization, channel keys, message handling |
| `docs/hosting/nginx-proxy-manager.md` | Nginx Proxy Manager admin authentication setup guide |
| `docs/seeding.md` | Seed data: node tags and members YAML format, directory structure, import process |
| `docs/webhooks.md` | Webhook configuration: URLs, secrets, retries, payload format |
| `docs/content.md` | Custom content: markdown pages, media files, logos, frontmatter fields |
| `docs/i18n.md` | Translation reference: all i18n keys, variable interpolation, translation tips |
| `.env.example` | Example environment file with comments and defaults |
| `SCHEMAS.md` | Event JSON schemas and database column mappings |
## Core Workflow
1. **Extract config from Python source** — Parse all environment variables from three sources: Pydantic Settings classes in `common/config.py`, Click `envvar=` parameters in CLI modules, and direct `os.getenv()`/`os.environ` calls. Build a complete inventory with field names, defaults, types, and descriptions. See `references/config-source-guide.md`.
2. **Extract Docker configuration** — Parse all `docker-compose*.yml` files for services, compose profiles, volumes, port mappings, environment variable references (with defaults), and device mappings. Distinguish hub-consumed vars from passthrough vars (e.g., `PACKETCAPTURE_*`). See `references/docker-source-guide.md`.
3. **Extract features and commands** — Verify feature flags have corresponding UI routes and config fields. Verify CLI commands documented in README.md and AGENTS.md still exist. Verify file paths and directory structures referenced in docs actually exist.
4. **Cross-reference against documentation** — For each of the 5 primary doc files, check every env var, Docker service, feature, command, and path reference against the source-of-truth inventories from steps 1-3. See `references/documentation-checklist.md`.
5. **Verify inline comments** — Check that all comments in `.env.example` and `docker-compose*.yml` accurately describe the values they annotate. Verify default values in comments match actual defaults from source code.
6. **Produce report and apply fixes** — Generate a structured discrepancy report. For each discrepancy, apply the fix to the relevant documentation file. Summarize all changes made.
## Reference Guide
Load detailed guidance based on context:
| Topic | Reference | Load When |
|-------|-----------|-----------|
| Config Source Extraction | `references/config-source-guide.md` | Extracting env vars from Python source |
| Docker Source Extraction | `references/docker-source-guide.md` | Parsing Docker Compose files |
| Documentation Checklist | `references/documentation-checklist.md` | Cross-referencing against each doc file |
## Discrepancy Categories
| Category | Severity | Description |
|----------|----------|-------------|
| Missing env var in docs | High | Variable exists in source but not in a doc file that should list it |
| Stale env var in docs | High | Variable documented but no longer exists in source |
| Wrong default value | High | Documented default doesn't match actual default |
| Wrong type or description | Medium | Documented type or description doesn't match source |
| Missing Docker service/profile | High | Service or profile exists in compose but not documented |
| Stale Docker service/profile | High | Documented service or profile no longer exists |
| Stale file path reference | Medium | Referenced file or directory doesn't exist |
| Stale CLI command | High | Documented command no longer exists |
| Stale feature reference | Medium | Documented feature flag doesn't exist in config |
| Inaccurate inline comment | Low | Comment doesn't accurately describe the value |
| Missing comment | Low | Value lacks a descriptive comment |
| Stale doc cross-reference | Medium | Reference to removed file (e.g., PLAN.md, TASKS.md) |
## Report Template
```
# Docs Sync Audit Report
## Summary
- Total discrepancies found: N
- High: N | Medium: N | Low: N
- Files modified: N
## Environment Variables
### Missing from documentation
| Variable | Default | Missing from |
|----------|---------|-------------|
| ... | ... | README.md, .env.example |
### Stale (removed from source)
| Variable | Still in |
|----------|---------|
| ... | AGENTS.md, README.md |
### Wrong defaults
| Variable | Documented | Actual |
|----------|-----------|--------|
| ... | ... | ... |
## Docker Configuration
### Services/Profiles
[Discrepancies between compose files and docs]
### Environment Passthroughs
[Missing or stale env vars in .env.example for Docker]
## Features & Commands
### Stale features
[Documented features that don't exist]
### Missing features
[Features in source but not documented]
### Stale commands
[Documented CLI commands that don't exist]
### Stale file references
[Paths referenced in docs that don't exist]
## Inline Comments
### Inaccurate comments
| File | Line | Current | Correct |
|------|------|---------|---------|
| ... | ... | ... | ... |
## Changes Applied
[List of edits made to each file]
```
## Constraints
### MUST DO
- Treat Python source code (`common/config.py`, CLI modules, `os.getenv` calls) as the single source of truth for environment variables
- Treat `docker-compose*.yml` files as the source of truth for Docker configuration
- Check ALL primary documentation files on every audit
- Include `.env.example` comment verification
- Include `docker-compose*.yml` inline comment verification
- Verify default values match exactly (type-aware: `true` vs `"true"`, port numbers as strings vs ints)
- Flag references to removed files (PLAN.md, TASKS.md) in AGENTS.md and README.md
- Apply fixes to documentation files after reporting
- Preserve existing formatting and section structure in doc files
- For AGENTS.md env var sections, maintain the existing table format and grouping
### MUST NOT DO
- Modify Python source code or Docker Compose files (only documentation files)
- Add documentation for variables that don't exist in source
- Remove content from documentation without confirming it's stale in source
- Change the formatting style of existing documentation (match surrounding content)
- Modify docs/upgrading.md historical content (deprecated var lists, old instructions)
- Skip any of the primary documentation files
- Guess at defaults — always verify against actual source code
- Treat test compose files (`tests/e2e/`) as documentation targets (they are test fixtures)
## Knowledge Reference
Pydantic Settings (BaseSettings, env_file, field defaults), Click (envvar parameter), Docker Compose (profiles, volumes, environment, depends_on), YAML parsing, environment variable naming conventions (UPPER_SNAKE_CASE), MeshCore Hub architecture (collector, API, web, MQTT broker, packet capture observer)
.agents/skills/docs-sync/references/config-source-guide.md
-137
View File
@@ -1,137 +0,0 @@
# Config Source Guide
How to extract the complete environment variable inventory from Python source code.
## Source Files
These files contain the authoritative definition of all environment variables consumed by the MeshCore Hub application. Check them in this order:
### 1. Pydantic Settings — `src/meshcore_hub/common/config.py`
**Primary source of truth.** Contains four Settings classes that define env vars via Pydantic fields:
| Class | Inherits | Component | Env Vars |
|-------|----------|-----------|----------|
| `CommonSettings` | `BaseSettings` | All services | Base config (MQTT, logging, paths) |
| `CollectorSettings` | `CommonSettings` | Collector | Database, webhooks, retention, cleanup, channel keys |
| `APISettings` | `CommonSettings` | API server | Host, port, auth keys, metrics |
| `WebSettings` | `CommonSettings` | Web dashboard | Host, port, theme, locale, features, network info |
#### Extraction Method
Read each class and extract:
- **Field name** — the Python attribute name (e.g., `mqtt_host`)
- **Env var name** — the uppercased field name (e.g., `MQTT_HOST`), unless explicitly overridden via `Field(alias=...)` or `alias` in `model_config`
- **Default value** — the `= value` in the field declaration, or `Field(default=...)`
- **Type** — the type annotation (e.g., `str`, `int`, `bool`, `Optional[str]`, constrained types)
- **Description** — the `Field(description=...)` or docstring, if any
- **Constraints** — `Field(ge=1, le=100)`, `min_length`, `max_length`, etc.
#### Pydantic-to-Env Mapping Rules
- Field `mqtt_host` → env var `MQTT_HOST` (automatic uppercasing)
- Field `api_base_url` → env var `API_BASE_URL`
- `Optional[str]` fields with `None` default → env var is optional, no default
- `bool` fields use Pydantic's built-in coercion: `"true"`, `"1"`, `"yes"` → `True`; `"false"`, `"0"`, `"no"` → `False`
- Enum fields (e.g., `LogLevel`, `MQTTTransport`) accept their member values as strings
#### Computed Properties
Some settings have computed properties (e.g., `database_url` falls back to `sqlite:///{DATA_HOME}/collector/meshcore.db`). These should be documented as having a computed default.
#### Settings Inheritance
`CollectorSettings`, `APISettings`, and `WebSettings` all inherit from `CommonSettings`. The shared env vars (`MQTT_*`, `LOG_LEVEL`, `DATA_HOME`, etc.) should appear once under "Common Settings" in documentation, not repeated per component.
### 2. Click CLI `envvar` — CLI Entry Points
These files define Click commands that accept env var overrides via `envvar=` or `envvar=[...]`:
| File | CLI Group | Key Env Vars |
|------|-----------|--------------|
| `src/meshcore_hub/__main__.py` | Root CLI | `DATA_HOME`, `DATABASE_URL` (also sets into `os.environ` for Alembic) |
| `src/meshcore_hub/collector/cli.py` | `collector` | `DATA_HOME`, `DATABASE_URL`, `SEED_HOME` |
| `src/meshcore_hub/api/cli.py` | `api` | `DATA_HOME`, `DATABASE_URL`, `CORS_ORIGINS`, `MQTT_PREFIX`/`MQTT_TOPIC_PREFIX` |
| `src/meshcore_hub/web/cli.py` | `web` | `DATA_HOME`, `API_BASE_URL` |
#### Extraction Method
Grep for `envvar=` in each file. Each `envvar=` parameter maps a Click option to an environment variable.
#### Key Patterns
- **Single envvar:** `envvar="CORS_ORIGINS"` — maps `--cors-origins` flag to `CORS_ORIGINS` env var
- **Multiple envvars (alias):** `envvar=["MQTT_PREFIX", "MQTT_TOPIC_PREFIX"]` — first is primary, rest are backward-compat aliases
- **Setting os.environ:** `__main__.py` sets `os.environ["DATABASE_URL"]` from CLI args so Alembic can pick it up. This is a pass-through, not a new env var.
#### Edge Case: `CORS_ORIGINS`
`CORS_ORIGINS` exists only as a Click `envvar` in `api/cli.py` — it has **no** Pydantic Settings field. It should still be documented as an env var consumed by the API component, but note that it only works when launched via the CLI (not when running with `--reload` which bypasses Click).
### 3. Direct `os.getenv()` / `os.environ` Access
Some code bypasses Pydantic Settings entirely and reads env vars directly:
| File | Env Var | Default | Purpose |
|------|---------|---------|---------|
| `src/meshcore_hub/common/health.py` | `HEALTH_DIR` | `/tmp/meshcore-hub` | Health status file directory |
| `src/meshcore_hub/alembic/env.py` | `DATABASE_URL` | Falls to config | Alembic migration DB URL |
| `src/meshcore_hub/alembic/env.py` | `DATA_HOME` | Falls to config | Alembic fallback for computing DB URL |
#### Extraction Method
Grep for these patterns across `src/meshcore_hub/`:
- `os.getenv(`
- `os.environ.get(`
- `os.environ[`
- `os.environ.setdefault(`
#### Key Observation
`HEALTH_DIR` is defined in `common/health.py` but has **no** Pydantic Settings field. It should be documented in AGENTS.md and .env.example if it's user-configurable, or noted as an internal variable if not.
## Classification: Hub Vars vs. Passthrough Vars
Not all env vars in `.env.example` are consumed by the Hub's Python code. Some are passed through to external containers via Docker Compose:
### Hub-Consumed Variables
Read by `meshcore-hub` Python code. Source: `config.py`, CLI modules, direct `os.getenv`.
These MUST be documented in all 5 doc files.
### Docker Passthrough Variables
Read by external containers (packet-capture, MQTT broker). Source: `docker-compose.yml` `environment:` blocks.
These MUST be in `.env.example` and `README.md` Docker sections, but NOT in AGENTS.md env var sections (since the Hub doesn't consume them).
Passthrough prefixes:
- `PACKETCAPTURE_*` — consumed by `ghcr.io/agessaman/meshcore-packet-capture`
- `MQTT_TOKEN_AUDIENCE` — consumed by `ghcr.io/ipnet-mesh/meshcore-mqtt-broker`
- `COMPOSE_PROJECT_NAME` — consumed by Docker Compose itself
- `IMAGE_VERSION`, `PACKETCAPTURE_IMAGE_VERSION` — Docker image tags
- `TRAEFIK_DOMAIN` — consumed by Traefik labels in `docker-compose.traefik.yml`
- `SERIAL_PORT` — device mapping for packet capture container
- `PROMETHEUS_PORT`, `ALERTMANAGER_PORT` — port mappings for monitoring stack
## Complete Extraction Checklist
For a full audit, run these extractions:
1. **Read `common/config.py`** — extract all fields from `CommonSettings`, `CollectorSettings`, `APISettings`, `WebSettings`
2. **Read `__main__.py`** — extract Click `envvar=` parameters and `os.environ` writes
3. **Read `collector/cli.py`** — extract Click `envvar=` parameters
4. **Read `api/cli.py`** — extract Click `envvar=` parameters (especially `CORS_ORIGINS`)
5. **Read `web/cli.py`** — extract Click `envvar=` parameters
6. **Read `web/app.py`** — extract `os.getenv()` calls (channel keys, test channel)
7. **Read `common/health.py`** — extract `os.environ.get()` calls (HEALTH_DIR)
8. **Read `alembic/env.py`** — extract `os.environ.get()` calls (DATABASE_URL, DATA_HOME)
9. **Deduplicate** — merge all sources, noting which vars appear in multiple places
10. **Classify** — tag each var as hub-consumed vs. Docker passthrough
## Three-Layer Config Precedence
When documenting defaults, understand the precedence:
1. **Click `envvar=` CLI option** — highest priority, overrides everything
2. **Pydantic Settings field default** — used when no CLI option or env var is set
3. **`.env` file** — loaded by `python-dotenv` at `__main__.py` startup, feeds into Pydantic Settings
The documented default should be the Pydantic Settings field default, since that's what applies in the general case.
.agents/skills/docs-sync/references/docker-source-guide.md
-290
View File
@@ -1,290 +0,0 @@
# Docker Source Guide
How to extract Docker configuration from all Compose files and verify documentation accuracy.
## Compose Files
| File | Purpose | Scope |
|------|---------|-------|
| `docker-compose.yml` | Base shared config | All services, profiles, volumes |
| `docker-compose.dev.yml` | Development overrides | Port mappings, dev dependencies |
| `docker-compose.prod.yml` | Production overrides | External proxy network |
| `docker-compose.traefik.yml` | Traefik auto-discovery labels | HTTP routing, TLS |
**Out of scope:** `tests/e2e/docker-compose.test.yml` is a test fixture with hardcoded values. Do NOT audit it against documentation.
## Services
### Service Inventory
Extract from `docker-compose.yml`:
| Service | Image | Profiles | Command |
|---------|-------|----------|---------|
| `mqtt` | `ghcr.io/ipnet-mesh/meshcore-mqtt-broker:latest` | `all`, `mqtt` | (default) |
| `observer` | `ghcr.io/agessaman/meshcore-packet-capture:${PACKETCAPTURE_IMAGE_VERSION}` | `all`, `observer` | (default) |
| `collector` | `ghcr.io/ipnet-mesh/meshcore-hub:${IMAGE_VERSION}` | `all`, `core` | `["collector"]` |
| `api` | `ghcr.io/ipnet-mesh/meshcore-hub:${IMAGE_VERSION}` | `all`, `core` | `["api"]` |
| `web` | `ghcr.io/ipnet-mesh/meshcore-hub:${IMAGE_VERSION}` | `all`, `core` | `["web"]` |
| `migrate` | `ghcr.io/ipnet-mesh/meshcore-hub:${IMAGE_VERSION}` | `all`, `core`, `migrate` | `["db", "upgrade"]` |
| `seed` | `ghcr.io/ipnet-mesh/meshcore-hub:${IMAGE_VERSION}` | `seed` | `["collector", "seed"]` |
### Verification Checklist
For each service, verify documentation includes:
- [ ] Service name and purpose
- [ ] Correct compose profile membership
- [ ] Dependencies (`depends_on`)
- [ ] Volumes mounted
- [ ] Key environment variables
## Compose Profiles
Extract from `docker-compose.yml` `profiles:` keys:
| Profile | Services | Use Case |
|---------|----------|----------|
| `all` | mqtt, observer, collector, api, web, migrate | Everything on one host |
| `core` | collector, api, web, migrate | Central server (no local MQTT or observer) |
| `mqtt` | mqtt | Local MQTT broker only |
| `observer` | observer | Packet capture observer only |
| `migrate` | migrate | Database migration only |
| `seed` | seed | Seed data import only |
### Verification
- [ ] Profile table in README.md matches compose file
- [ ] Profile table in AGENTS.md matches compose file
- [ ] All profile names documented
- [ ] All services within each profile listed
- [ ] Use cases described accurately
## Volumes
### Named Volumes
| Volume | Services | Purpose |
|--------|----------|---------|
| `data` | collector, api, migrate, seed | SQLite database + runtime data |
| `mqtt_data` | mqtt | MQTT broker persistence |
| `observer_data` | observer | Packet capture data |
Volume names are prefixed: `${COMPOSE_PROJECT_NAME:-hub}_data`, `${COMPOSE_PROJECT_NAME:-hub}_mqtt_data`, etc.
### Bind Mounts
| Host Path | Container Path | Service | Mode |
|-----------|---------------|---------|------|
| `${SEED_HOME:-./seed}` | `/seed` | collector | rw |
| `${SEED_HOME:-./seed}` | `/seed` | seed | ro |
| `${CONTENT_HOME:-./content}` | `/content` | web | ro |
### Verification
- [ ] All named volumes documented in README.md
- [ ] Bind mount paths match `SEED_HOME` and `CONTENT_HOME` env var defaults
- [ ] Volume naming convention (COMPOSE_PROJECT_NAME prefix) documented
## Port Mappings
### Development (docker-compose.dev.yml)
| Service | Host Port | Container Port | Variable |
|---------|-----------|----------------|----------|
| mqtt | `${MQTT_PORT:-1883}` | `${MQTT_PORT:-1883}` | `MQTT_PORT` |
| api | `${API_PORT:-8000}` | `8000` | `API_PORT` |
| web | `${WEB_PORT:-8080}` | `8080` | `WEB_PORT` |
### Production (docker-compose.prod.yml)
No ports exposed. Services connect to external `proxy-net` Docker network.
### Traefik (docker-compose.traefik.yml)
Routing via labels. API handles `/api`, `/metrics`, `/health`. Web handles everything else.
### Verification
- [ ] Port mappings in README.md match `docker-compose.dev.yml`
- [ ] Production routing description matches `docker-compose.traefik.yml`
- [ ] Container ports match `Dockerfile` exposed ports (8000, 8080)
## Environment Variables in Docker
### Per-Service Env Var Extraction
For each service's `environment:` block, extract every variable. Classify each:
**Hardcoded (container-internal):** Set to a fixed value in compose file, not configurable via `.env`. These should NOT appear in `.env.example` but SHOULD be noted in README.md service descriptions.
- Example: `DATA_HOME=/data` (collector), `API_HOST=0.0.0.0` (api), `CONTENT_HOME=/content` (web)
**Variable substitution (user-configurable):** Use `${VAR:-default}` syntax. These MUST appear in `.env.example` with the same default.
- Example: `MQTT_HOST=${MQTT_HOST:-mqtt}`, `LOG_LEVEL=${LOG_LEVEL:-INFO}`
**Passthrough (no default):** Reference `${VAR}` without a default. These MUST appear in `.env.example` (typically commented out or with empty value).
- Example: `API_READ_KEY`, `API_ADMIN_KEY`, `WEBHOOK_ADVERTISEMENT_URL`
### Collector Service Env Vars
Complete list from `docker-compose.yml` collector `environment:` block:
| Variable | Default in Compose | Category |
|----------|--------------------|----------|
| `LOG_LEVEL` | `INFO` | Common |
| `MQTT_HOST` | `mqtt` | MQTT |
| `MQTT_PORT` | `1883` | MQTT |
| `MQTT_USERNAME` | (empty) | MQTT |
| `MQTT_PASSWORD` | (empty) | MQTT |
| `MQTT_PREFIX` | `meshcore` | MQTT |
| `MQTT_TLS` | `false` | MQTT |
| `MQTT_TRANSPORT` | `websockets` | MQTT |
| `MQTT_WS_PATH` | `/` | MQTT |
| `DATA_HOME` | `/data` (hardcoded) | Path |
| `SEED_HOME` | `/seed` (hardcoded) | Path |
| `CHANNEL_REFRESH_INTERVAL_SECONDS` | `300` | Collector |
| `WEBHOOK_ADVERTISEMENT_URL` | (passthrough) | Webhook |
| `WEBHOOK_ADVERTISEMENT_SECRET` | (passthrough) | Webhook |
| `WEBHOOK_MESSAGE_URL` | (passthrough) | Webhook |
| `WEBHOOK_MESSAGE_SECRET` | (passthrough) | Webhook |
| `WEBHOOK_CHANNEL_MESSAGE_URL` | (passthrough) | Webhook |
| `WEBHOOK_CHANNEL_MESSAGE_SECRET` | (passthrough) | Webhook |
| `WEBHOOK_DIRECT_MESSAGE_URL` | (passthrough) | Webhook |
| `WEBHOOK_DIRECT_MESSAGE_SECRET` | (passthrough) | Webhook |
| `WEBHOOK_TIMEOUT` | `10.0` | Webhook |
| `WEBHOOK_MAX_RETRIES` | `3` | Webhook |
| `WEBHOOK_RETRY_BACKOFF` | `2.0` | Webhook |
| `DATA_RETENTION_ENABLED` | `true` | Retention |
| `DATA_RETENTION_DAYS` | `30` | Retention |
| `DATA_RETENTION_INTERVAL_HOURS` | `24` | Retention |
| `NODE_CLEANUP_ENABLED` | `true` | Node Cleanup |
| `NODE_CLEANUP_DAYS` | `30` | Node Cleanup |
### API Service Env Vars
| Variable | Default in Compose | Category |
|----------|--------------------|----------|
| `LOG_LEVEL` | `INFO` | Common |
| `MQTT_HOST` | `mqtt` | MQTT |
| `MQTT_PORT` | `1883` | MQTT |
| `MQTT_USERNAME` | (empty) | MQTT |
| `MQTT_PASSWORD` | (empty) | MQTT |
| `MQTT_PREFIX` | `meshcore` | MQTT |
| `MQTT_TLS` | `false` | MQTT |
| `MQTT_TRANSPORT` | `websockets` | MQTT |
| `MQTT_WS_PATH` | `/` | MQTT |
| `DATA_HOME` | `/data` (hardcoded) | Path |
| `API_HOST` | `0.0.0.0` (hardcoded) | API |
| `API_PORT` | `8000` (hardcoded) | API |
| `API_READ_KEY` | (passthrough) | Auth |
| `API_ADMIN_KEY` | (passthrough) | Auth |
| `METRICS_ENABLED` | `true` | Metrics |
| `METRICS_CACHE_TTL` | `60` | Metrics |
### Web Service Env Vars
| Variable | Default in Compose | Category |
|----------|--------------------|----------|
| `LOG_LEVEL` | `INFO` | Common |
| `API_BASE_URL` | `http://api:8000` (hardcoded) | API |
| `API_ADMIN_KEY` / `API_READ_KEY` | (cascading passthrough) | Auth |
| `WEB_HOST` | `0.0.0.0` (hardcoded) | Web |
| `WEB_PORT` | `8080` (hardcoded) | Web |
| `WEB_THEME` | `dark` | Theme |
| `WEB_LOCALE` | `en` | Locale |
| `WEB_DATETIME_LOCALE` | `en-US` | Locale |
| `OIDC_ENABLED` | `false` | Auth |
| `OIDC_CLIENT_ID` | (empty) | Auth |
| `OIDC_CLIENT_SECRET` | (empty) | Auth |
| `OIDC_DISCOVERY_URL` | (empty) | Auth |
| `OIDC_REDIRECT_URI` | (empty) | Auth |
| `OIDC_POST_LOGOUT_REDIRECT_URI` | (empty) | Auth |
| `OIDC_SCOPES` | `openid email profile` | Auth |
| `OIDC_ROLES_CLAIM` | `roles` | Auth |
| `OIDC_ROLE_ADMIN` | `admin` | Auth |
| `OIDC_ROLE_OPERATOR` | `operator` | Auth |
| `OIDC_ROLE_MEMBER` | `member` | Auth |
| `OIDC_SESSION_SECRET` | (empty) | Auth |
| `OIDC_SESSION_MAX_AGE` | `86400` | Auth |
| `OIDC_COOKIE_SECURE` | `false` | Auth |
| `WEB_AUTO_REFRESH_SECONDS` | `30` | Display |
| `WEB_DEBUG` | `false` | Display |
| `NETWORK_DOMAIN` | (empty) | Network |
| `NETWORK_NAME` | `MeshCore Network` | Network |
| `NETWORK_CITY` | (empty) | Network |
| `NETWORK_COUNTRY` | (empty) | Network |
| `NETWORK_RADIO_CONFIG` | (empty) | Network |
| `NETWORK_CONTACT_EMAIL` | (empty) | Network |
| `NETWORK_CONTACT_DISCORD` | (empty) | Network |
| `NETWORK_CONTACT_GITHUB` | (empty) | Network |
| `NETWORK_CONTACT_YOUTUBE` | (empty) | Network |
| `NETWORK_WELCOME_TEXT` | (empty) | Network |
| `CONTENT_HOME` | `/content` (hardcoded) | Path |
| `TZ` | `UTC` | Display |
| `FEATURE_CHANNELS` | `true` | Feature |
| `FEATURE_DASHBOARD` | `true` | Feature |
| `FEATURE_NODES` | `true` | Feature |
| `FEATURE_ADVERTISEMENTS` | `true` | Feature |
| `FEATURE_MESSAGES` | `true` | Feature |
| `FEATURE_MAP` | `true` | Feature |
| `FEATURE_MEMBERS` | `true` | Feature |
| `FEATURE_PAGES` | `true` | Feature |
| `FEATURE_CHANNELS` | `true` | Feature |
### Observer (Packet Capture) Env Vars
These are ALL passthrough vars consumed by the external packet capture image. None are read by Hub Python code.
Grouped by function:
- **Connection:** `SERIAL_PORT`, `PACKETCAPTURE_TIMEOUT`, `PACKETCAPTURE_MAX_CONNECTION_RETRIES`, `PACKETCAPTURE_CONNECTION_RETRY_DELAY`, `PACKETCAPTURE_HEALTH_CHECK_INTERVAL`
- **Identity:** `PACKETCAPTURE_IATA`, `PACKETCAPTURE_ORIGIN`
- **Behavior:** `PACKETCAPTURE_ADVERT_INTERVAL_HOURS`, `PACKETCAPTURE_RF_DATA_TIMEOUT`
- **MQTT Broker 1 (Let's Mesh US):** `PACKETCAPTURE_MQTT1_ENABLED`, `PACKETCAPTURE_MQTT1_SERVER`, `PACKETCAPTURE_MQTT1_PORT`, `PACKETCAPTURE_MQTT1_USE_TLS`, `PACKETCAPTURE_MQTT1_USE_AUTH_TOKEN`, `PACKETCAPTURE_MQTT1_TOKEN_AUDIENCE`, `PACKETCAPTURE_MQTT1_KEEPALIVE`
- **MQTT Broker 2 (Let's Mesh EU):** `PACKETCAPTURE_MQTT2_ENABLED`, `PACKETCAPTURE_MQTT2_SERVER`, `PACKETCAPTURE_MQTT2_PORT`, `PACKETCAPTURE_MQTT2_USE_TLS`, `PACKETCAPTURE_MQTT2_USE_AUTH_TOKEN`, `PACKETCAPTURE_MQTT2_TOKEN_AUDIENCE`, `PACKETCAPTURE_MQTT2_KEEPALIVE`
- **MQTT Broker 3 (Local):** `PACKETCAPTURE_MQTT3_ENABLED`, `PACKETCAPTURE_MQTT3_KEEPALIVE`
- **MQTT Reconnection:** `PACKETCAPTURE_MAX_MQTT_RETRIES`, `PACKETCAPTURE_MQTT_RETRY_DELAY`, `PACKETCAPTURE_EXIT_ON_RECONNECT_FAIL`
Note: Broker 3 is wired to hub's MQTT vars: `MQTT_HOST`, `MQTT_PORT`, `MQTT_USERNAME`, `MQTT_PASSWORD`, `MQTT_TLS`, `MQTT_TOKEN_AUDIENCE`.
## Infrastructure-Only Variables
These appear in `.env.example` and compose files but are NOT consumed by any Python code:
| Variable | Consumer | Default |
|----------|----------|---------|
| `COMPOSE_PROJECT_NAME` | Docker Compose | `hub` |
| `IMAGE_VERSION` | Docker Compose | `latest` |
| `PACKETCAPTURE_IMAGE_VERSION` | Docker Compose | `latest` |
| `TRAEFIK_DOMAIN` | Traefik labels | (required when using traefik compose) |
| `SERIAL_PORT` | Observer container + device mapping | `/dev/ttyUSB0` |
| `MQTT_TOKEN_AUDIENCE` | MQTT broker container | `mqtt.localhost` |
| `PROMETHEUS_PORT` | Docker port mapping | `9090` |
| `ALERTMANAGER_PORT` | Docker port mapping | `9093` |
These MUST be in `.env.example` and README.md but NOT in AGENTS.md "Environment Variables" section (since AGENTS.md documents Hub-consumed vars).
## Dockerfile Verification
The `Dockerfile` sets ENV defaults that should match compose file defaults:
| Dockerfile ENV | Value | Must Match |
|----------------|-------|------------|
| `LOG_LEVEL` | `INFO` | Compose `LOG_LEVEL` default |
| `MQTT_HOST` | `mqtt` | Compose `MQTT_HOST` default |
| `MQTT_PORT` | `1883` | Compose `MQTT_PORT` default |
| `MQTT_PREFIX` | `meshcore` | Compose `MQTT_PREFIX` default |
| `DATA_HOME` | `/data` | Compose hardcoded `DATA_HOME` |
| `API_HOST` | `0.0.0.0` | Compose hardcoded `API_HOST` |
| `API_PORT` | `8000` | Compose hardcoded `API_PORT` |
| `WEB_HOST` | `0.0.0.0` | Compose hardcoded `WEB_HOST` |
| `WEB_PORT` | `8080` | Compose hardcoded `WEB_PORT` |
| `API_BASE_URL` | `http://api:8000` | Compose hardcoded `API_BASE_URL` |
## Inline Comment Verification
For `docker-compose.yml` and `.env.example`, verify every `# comment` accurately describes the value it annotates. Check:
1. **Default values in comments** match actual `${VAR:-default}` values
2. **Descriptions** accurately describe what the variable does
3. **Section headers** correctly group related variables
4. **References** to other files or sections are still valid (e.g., "see README.md" references)
5. **Examples** use current, valid values (not outdated formats)
.agents/skills/docs-sync/references/documentation-checklist.md
-338
View File
@@ -1,338 +0,0 @@
# Documentation Checklist
Per-file verification checklists for each of the 5 primary documentation files.
## 1. README.md
### Environment Variable Tables
README.md contains env var tables grouped by component. For each table:
- [ ] **Common Settings table** — every var from `CommonSettings` is listed with correct default
- [ ] **Collector Settings table** — every collector-specific var from `CollectorSettings` is listed
- [ ] **Webhook reference** — webhook config links to `docs/webhooks.md`
- [ ] **Data Retention table** — all retention and node cleanup vars listed
- [ ] **API Settings table** — all API-specific vars from `APISettings` listed
- [ ] **Web Dashboard Settings table** — all web-specific vars from `WebSettings` listed
- [ ] **Feature Flags table** — all 7 feature flags listed
- [ ] **Network Info vars** — all `NETWORK_*` vars listed
- [ ] **Contact Info vars** — all `NETWORK_CONTACT_*` vars listed
For each variable in each table:
- [ ] Default value matches Pydantic Settings field default exactly
- [ ] Description is accurate and matches field purpose
- [ ] No stale/removed variables remain
- [ ] No variables are duplicated across tables
### Docker Section
- [ ] Compose files listed: base, dev, prod, traefik (4 files)
- [ ] Service profiles table matches `docker-compose.yml` exactly
- [ ] All 7 services documented: mqtt, observer, collector, api, web, migrate, seed
- [ ] Port mappings match `docker-compose.dev.yml`
- [ ] Volume names documented with `COMPOSE_PROJECT_NAME` prefix convention
- [ ] Bind mounts documented (SEED_HOME, CONTENT_HOME)
- [ ] Traefik integration instructions reference `TRAEFIK_DOMAIN`
- [ ] Reverse Proxy section documents proxy setup and production network
- [ ] Quick start examples use correct current commands
### Features Section
- [ ] Each listed feature corresponds to actual code
- [ ] Feature flag dependency rules documented correctly (Dashboard auto-disables when Nodes/Ads/Messages all off; Map auto-disables when Nodes off)
### CLI Commands
- [ ] `meshcore-hub collector` — verify still exists
- [ ] `meshcore-hub api` — verify still exists
- [ ] `meshcore-hub web` — verify still exists
- [ ] `meshcore-hub db upgrade` — verify still exists
- [ ] `meshcore-hub collector seed` — verify still exists
- [ ] `meshcore-hub collector cleanup` — verify still exists
- [ ] All example Docker Compose commands use valid profile names
### File Paths
- [ ] `src/meshcore_hub/` structure matches actual layout
- [ ] Seed data directory structure (`node_tags.yaml`) documented in `docs/seeding.md`
- [ ] Custom content directory structure (`pages/`, `media/`) documented in `docs/content.md`
- [ ] Webhook configuration documented in `docs/webhooks.md`
- [ ] Translation files location (`src/meshcore_hub/web/static/locales/`) documented
- [ ] Translation reference guide (`docs/i18n.md`) linked from README and AGENTS.md
- [ ] No references to removed files (PLAN.md, TASKS.md)
## 2. AGENTS.md
### Environment Variables Section
AGENTS.md has a "Key variables" subsection under "Environment Variables". Verify:
- [ ] All hub-consumed env vars listed (from Pydantic Settings + Click + os.getenv)
- [ ] Passthrough vars (`PACKETCAPTURE_*`, `COMPOSE_PROJECT_NAME`, etc.) NOT listed as Hub vars
- [ ] Defaults match Pydantic Settings defaults
- [ ] Descriptions match field purpose
- [ ] Grouping matches the Settings class hierarchy (Common, Collector, API, Web)
### Project Structure
- [ ] Directory tree matches actual layout
- [ ] All listed files exist
- [ ] No removed files listed (e.g., PLAN.md, TASKS.md)
- [ ] All new directories/files included
### Features Documented
- [ ] Feature flags listed in "Environment Variables" match `WebSettings` fields
- [ ] Dependency rules documented (Dashboard/Map auto-disable logic)
- [ ] Admin auth mechanism documented accurately
### Code Examples
- [ ] Import paths use current module structure
- [ ] Class names match current models
- [ ] CLI commands match current Click definitions
- [ ] Async patterns match current codebase conventions
### Cross-References
- [ ] References to `PLAN.md` — should be removed (file deleted)
- [ ] References to `TASKS.md` — should be removed (file deleted)
- [ ] References to `SCHEMAS.md` — should remain (file exists)
- [ ] References to `docs/upgrading.md` — should remain (file exists)
- [ ] References to `docs/letsmesh.md` — should remain (file exists)
- [ ] References to `docs/webhooks.md` — should remain (file exists)
- [ ] References to `docs/content.md` — should remain (file exists)
## 3. docs/upgrading.md
### Deprecated Variables
docs/upgrading.md lists variables to remove during upgrade. Verify:
- [ ] Each deprecated var truly no longer exists in `config.py` or any CLI module
- [ ] Removal instructions are clear
- [ ] No currently-active vars are listed as deprecated
### New Variables
- [ ] Each new var listed actually exists in current `config.py`
- [ ] Defaults for new vars match Pydantic Settings defaults
- [ ] Migration instructions for adding new vars are correct
### Renamed Variables
- [ ] Old name no longer exists anywhere in codebase
- [ ] New name matches current `config.py` field name
### Docker Migration
- [ ] Volume rename instructions accurate
- [ ] Old service names truly removed from compose files
- [ ] New compose file structure documented correctly
- [ ] Migration commands still valid for current Docker versions
### Database Migration
- [ ] Column renames documented accurately (e.g., `receiver_node_id` → `observer_node_id`)
- [ ] Table renames documented accurately (e.g., `event_receivers` → `event_observers`)
## 3b. docs/letsmesh.md
### Packet Decoding Documentation
docs/letsmesh.md documents the LetsMesh packet normalization and decoding behavior. Verify:
- [ ] MQTT subscription topics match `subscriber.py` topic patterns
- [ ] Payload type mappings match `letsmesh_decoder.py` and `letsmesh_normalizer.py` logic
- [ ] Channel key handling documented matches database-backed channels (channels table, CHANNEL_REFRESH_INTERVAL_SECONDS)
- [ ] Known channel indexes (`17 -> Public`, `217 -> #test`) match built-in defaults in decoder
- [ ] Message normalization rules match collector handler implementations
- [ ] GPS/location update behavior documented matches advertisement handler logic
- [ ] No stale decoder behavior documented (e.g., references to Node.js decoder)
## 3c. docs/seeding.md
### Seed Data Documentation
docs/seeding.md documents the seed data format and import process for node tags and network members. Verify:
- [ ] Running the Seed Process section references correct Docker Compose command (`--profile seed`)
- [ ] Seed files listed match `tag_import.py` and `member_import.py` expected filenames
- [ ] Directory structure shows `SEED_HOME` and `DATA_HOME` correctly
- [ ] Node Tags YAML format matches `tag_import.py` parsing logic
- [ ] Tag value types documented match supported types in `tag_import.py`
- [ ] Members YAML format matches `member_import.py` parsing logic
- [ ] Member field table fields match `MemberCreate` Pydantic schema
- [ ] Example seed files referenced in `example/seed/` exist
## 3d. docs/i18n.md
### Translation Reference Guide
docs/i18n.md is a comprehensive reference for translators. Verify:
- [ ] Translation file location path (`src/meshcore_hub/web/static/locales/`) is correct
- [ ] Each translation section key matches a top-level key in `en.json`
- [ ] Entity keys (`entities.*`) match current `en.json` values
- [ ] Common pattern keys (`common.*`) match current `en.json` values
- [ ] Composite pattern examples produce correct output with current entity names
- [ ] Variable interpolation syntax (`{{variable}}`) matches actual usage in `en.json`
- [ ] Admin section keys match current admin page translations
- [ ] No stale translation keys documented (removed from `en.json`)
- [ ] Translation tips are accurate for current i18n system
## 3e. docs/webhooks.md
### Webhook Configuration Documentation
docs/webhooks.md documents the webhook configuration, URL routing logic, and payload format. Verify:
- [ ] All 11 webhook environment variables listed with correct defaults and descriptions
- [ ] URL routing rules documented correctly (MESSAGE_URL as default, CHANNEL/DIRECT overrides)
- [ ] Secret header mechanism documented (`X-Webhook-Secret`)
- [ ] Retry behavior documented (max retries, exponential backoff)
- [ ] Payload format JSON example matches `webhook.py` dispatcher output
- [ ] Event types listed match actual webhook event types
- [ ] Configuration examples use correct env var names
## 3f. docs/content.md
### Custom Content Documentation
docs/content.md documents the custom content system for the web dashboard. Verify:
- [ ] Directory structure shows `pages/` and `media/` subdirectories correctly
- [ ] Custom logo options (`logo.svg`, `logo-invert.svg`) documented with behavior per theme
- [ ] Frontmatter field table matches `pages.py` parser expectations
- [ ] Default values for frontmatter fields are correct (title, slug, menu_order)
- [ ] Setup examples create valid markdown pages
- [ ] Docker volume mounting instructions match `docker-compose.yml` bind mount config
- [ ] `CONTENT_HOME` default value documented correctly
## 4. .env.example
### Section Structure
Verify sections exist and are correctly ordered:
1. [ ] Quick Start header with observer node example
2. [ ] Common Settings (`COMPOSE_PROJECT_NAME`, `TRAEFIK_DOMAIN`, `IMAGE_VERSION`, `LOG_LEVEL`, `DATA_HOME`, `SEED_HOME`)
3. [ ] MQTT Settings (`MQTT_HOST`, `MQTT_PORT`, `MQTT_USERNAME`, `MQTT_PASSWORD`, `MQTT_PREFIX`, `MQTT_TLS`, `MQTT_TRANSPORT`, `MQTT_WS_PATH`, `MQTT_TOKEN_AUDIENCE`)
4. [ ] Packet Capture Settings (all `PACKETCAPTURE_*` vars + `SERIAL_PORT`)
5. [ ] Collector Settings (`CHANNEL_REFRESH_INTERVAL_SECONDS`, webhooks, retention, cleanup)
6. [ ] API Settings (`API_PORT`, `API_READ_KEY`, `API_ADMIN_KEY`, metrics)
7. [ ] Web Dashboard Settings (`WEB_PORT`, `API_BASE_URL`, `API_KEY`, theme, locale, auto-refresh, admin, TZ, content home, network info, feature flags, contact info)
### Per-Variable Checks
For every variable in `.env.example`:
- [ ] Variable name matches the env var name exactly (UPPER_SNAKE_CASE)
- [ ] Default value matches Pydantic Settings default OR compose file default (for passthrough vars)
- [ ] Comment above the variable accurately describes its purpose
- [ ] Comment includes valid range/options where applicable (e.g., "DEBUG, INFO, WARNING, ERROR, CRITICAL")
- [ ] Optional vars are commented out (`# VAR=`) with a note about the default
- [ ] Required vars have an uncommented assignment with the default value
- [ ] No duplicate variable entries
- [ ] No removed/stale variables
### Comment Accuracy
- [ ] `COMPOSE_PROJECT_NAME` comment mentions container/volume prefix
- [ ] `MQTT_TRANSPORT` comment states WebSocket is required by MeshCore broker
- [ ] `MQTT_WS_PATH` comment notes default `/` vs production `/mqtt`
- [ ] `MQTT_TOKEN_AUDIENCE` comment explains it must match broker config
- [ ] `PACKETCAPTURE_*` comments reference the external packet capture image
- [ ] `CHANNEL_REFRESH_INTERVAL_SECONDS` comment explains channel key refresh from database
- [ ] `WEB_*` comments reference web dashboard behavior
- [ ] `FEATURE_*` comments explain what each flag controls
- [ ] `NETWORK_*` comments explain where values appear in UI
- [ ] `PROMETHEUS_PORT` / `ALERTMANAGER_PORT` comments reference the monitoring profile
### Missing Variables Check
- [ ] Every hub-consumed var from `config.py` appears (or is commented out) in `.env.example`
- [ ] Every passthrough var from compose files appears in `.env.example`
- [ ] No extra variables that don't exist in any compose file or Python source
## 5. SCHEMAS.md
### Event Schema Verification
SCHEMAS.md documents the JSON schemas for events stored in the database. Verify:
- [ ] Each event type documented has a corresponding handler in `src/meshcore_hub/collector/handlers/`
- [ ] Each documented field exists in the corresponding Pydantic schema or SQLAlchemy model
- [ ] Field types match current code (e.g., `str`, `int`, `Optional[str]`)
- [ ] Required vs optional fields match current schema definitions
- [ ] Database column names match current SQLAlchemy model definitions
### Database Table Verification
For each table documented in SCHEMAS.md:
- [ ] Table name matches `__tablename__` in the corresponding SQLAlchemy model
- [ ] Column names match `mapped_column()` field names
- [ ] Column types match (String length, DateTime, Text, Integer, etc.)
- [ ] Foreign key relationships documented correctly
- [ ] Indexes and unique constraints documented correctly
- [ ] New columns from recent migrations are included
- [ ] Removed columns are not documented
### MQTT Topic Schema
- [ ] Topic structure documented matches what the collector subscribes to
- [ ] Upload topic format (`<prefix>/<IATA>/<public_key>/<feed_type>`) is correct
- [ ] Subscriber subscriptions listed match `subscriber.py` topic patterns
## Cross-File Consistency Checks
These checks ensure all primary documentation files are consistent with each other:
### Env Var Coverage Matrix
Every hub-consumed env var should appear in:
| File | Required | Format |
|------|----------|--------|
| `config.py` | Yes (source of truth) | Pydantic field |
| `README.md` | Yes | Table row with default + description |
| `AGENTS.md` | Yes | Mentioned in env vars section |
| `.env.example` | Yes | Entry with default + comment |
| `docs/upgrading.md` | Only if new/renamed/deprecated | Migration instruction |
Every passthrough env var should appear in:
| File | Required |
|------|----------|
| `docker-compose.yml` | Yes (source of truth) |
| `README.md` | Yes |
| `.env.example` | Yes |
| `AGENTS.md` | No |
### Default Value Consistency
- [ ] Same default in README.md tables, .env.example values, and config.py
- [ ] No contradictions between files
- [ ] Type representations consistent (e.g., don't mix `true`/`True`/`1` for booleans)
### Stale Reference Sweep
Check all primary documentation files for references to removed items:
- [ ] `PLAN.md` — removed, references should be deleted from AGENTS.md
- [ ] `TASKS.md` — removed, references should be deleted from AGENTS.md
- [ ] Old compose profiles (`receiver`, `sender`, `mock`) — should only exist in docs/upgrading.md as deprecated
- [ ] Old service names (`interface-receiver`, `interface-sender`) — should only exist in docs/upgrading.md as deprecated
- [ ] Old env var names (`COLLECTOR_LETSMESH_DECODER_*`, `SERIAL_BAUD`, etc.) — should only exist in docs/upgrading.md as deprecated
## Applying Fixes
When discrepancies are found:
1. **Identify the source of truth** — config.py for env vars, docker-compose.yml for Docker config
2. **Determine scope** — which files need updating
3. **Make minimal edits** — only change what's wrong, preserve surrounding formatting
4. **Match existing style** — tables use same column order, comments use same format, sections use same headers
5. **Preserve historical content** — docs/upgrading.md deprecated var lists are historical, do not remove them
6. **Verify after editing** — re-read the changed section to confirm accuracy