docs: Reorganize and simplify documentation structure

- Simplify README.md from ~925 to ~217 lines - Create docs/user-guide.md with detailed feature documentation - Create docs/architecture.md with technical details and API reference - Create docs/troubleshooting.md (merged from COMMON_ISSUES.md + README) - Move DOCKER_INSTALL.md to docs/docker-install.md - Remove COMMON_ISSUES.md (content merged into troubleshooting.md) - Add Documentation section with links to all docs The README now focuses on quick start and installation, while detailed documentation is organized in docs/ folder. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-28 17:42:45 +01:00 · 2026-01-07 10:21:55 +01:00
parent f6b858195b
commit 0a44a9d792
6 changed files with 1096 additions and 946 deletions
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -0,0 +1,301 @@
+# mc-webui Architecture
+
+Technical documentation for mc-webui, covering system architecture, project structure, and internal APIs.
+
+## Table of Contents
+
+- [Tech Stack](#tech-stack)
+- [Container Architecture](#container-architecture)
+- [Bridge Session Architecture](#bridge-session-architecture)
+- [Project Structure](#project-structure)
+- [Message File Format](#message-file-format)
+- [API Reference](#api-reference)
+- [Offline Support](#offline-support)
+
+---
+
+## Tech Stack
+
+- **Backend:** Python 3.11+, Flask
+- **Frontend:** HTML5, Bootstrap 5, vanilla JavaScript
+- **Deployment:** Docker / Docker Compose (2-container architecture)
+- **Communication:** HTTP bridge to meshcore-cli (USB isolation for stability)
+- **Data source:** `~/.config/meshcore/<device_name>.msgs` (JSON Lines)
+
+---
+
+## Container Architecture
+
+mc-webui uses a **2-container architecture** for improved USB stability:
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                     Docker Network                           │
+│                                                              │
+│  ┌─────────────────────┐      ┌─────────────────────────┐  │
+│  │   meshcore-bridge   │      │       mc-webui          │  │
+│  │                     │      │                         │  │
+│  │  - USB device access│ HTTP │  - Flask web app        │  │
+│  │  - meshcli process  │◄────►│  - User interface       │  │
+│  │  - Port 5001        │      │  - Port 5000            │  │
+│  │                     │      │                         │  │
+│  └─────────┬───────────┘      └─────────────────────────┘  │
+│            │                                                 │
+└────────────┼─────────────────────────────────────────────────┘
+             │
+             ▼
+      ┌──────────────┐
+      │  USB Device  │
+      │  (Heltec V4) │
+      └──────────────┘
+```
+
+### meshcore-bridge (Port 5001 - internal)
+
+Lightweight service with exclusive USB device access:
+
+- Maintains a **persistent meshcli session** (single long-lived process)
+- Multiplexes stdout: JSON adverts → `.adverts.jsonl` log, CLI commands → HTTP responses
+- Real-time message reception via `msgs_subscribe` (no polling)
+- Thread-safe command queue with event-based synchronization
+- Watchdog thread for automatic crash recovery
+- Exposes HTTP API on port 5001 (internal only)
+
+### mc-webui (Port 5000 - external)
+
+Main web application:
+
+- Flask-based web interface
+- Communicates with bridge via HTTP
+- No direct USB access (prevents device locking)
+
+This separation solves USB timeout/deadlock issues common in Docker + VM environments.
+
+---
+
+## Bridge Session Architecture
+
+The meshcore-bridge maintains a **single persistent meshcli session** instead of spawning new processes per request:
+
+- **Single subprocess.Popen** - One long-lived meshcli process with stdin/stdout pipes
+- **Multiplexing** - Intelligently routes output:
+  - JSON adverts (with `payload_typename: "ADVERT"`) → logged to `{device_name}.adverts.jsonl`
+  - CLI command responses → returned via HTTP API
+- **Real-time messages** - `msgs_subscribe` command enables instant message reception without polling
+- **Thread-safe queue** - Commands are serialized through a queue.Queue for FIFO execution
+- **Timeout-based detection** - Response completion detected when no new lines arrive for 300ms
+- **Auto-restart watchdog** - Monitors process health and restarts on crash
+
+This architecture enables advanced features like pending contact management (`manual_add_contacts`) and provides better stability and performance.
+
+---
+
+## Project Structure
+
+```
+mc-webui/
+├── Dockerfile                      # Main app Docker image
+├── docker-compose.yml              # Multi-container orchestration
+├── meshcore-bridge/
+│   ├── Dockerfile                  # Bridge service image
+│   ├── bridge.py                   # HTTP API wrapper for meshcli
+│   └── requirements.txt            # Bridge dependencies (Flask only)
+├── app/
+│   ├── __init__.py
+│   ├── main.py                     # Flask entry point
+│   ├── config.py                   # Configuration from env vars
+│   ├── read_status.py              # Server-side read status manager
+│   ├── meshcore/
+│   │   ├── __init__.py
+│   │   ├── cli.py                  # HTTP client for bridge API
+│   │   └── parser.py               # .msgs file parser
+│   ├── archiver/
+│   │   └── manager.py              # Archive scheduler and management
+│   ├── routes/
+│   │   ├── __init__.py
+│   │   ├── api.py                  # REST API endpoints
+│   │   └── views.py                # HTML views
+│   ├── static/
+│   │   ├── css/
+│   │   │   └── style.css           # Custom styles
+│   │   ├── js/
+│   │   │   ├── app.js              # Main page frontend logic
+│   │   │   ├── dm.js               # Direct Messages page logic
+│   │   │   ├── contacts.js         # Contact Management logic
+│   │   │   ├── message-utils.js    # Message content processing
+│   │   │   └── sw.js               # Service Worker for PWA
+│   │   ├── vendor/                 # Local vendor libraries (offline)
+│   │   │   ├── bootstrap/          # Bootstrap CSS/JS
+│   │   │   ├── bootstrap-icons/    # Icon fonts
+│   │   │   └── emoji-picker-element/
+│   │   └── manifest.json           # PWA manifest
+│   └── templates/
+│       ├── base.html               # Base template
+│       ├── index.html              # Main chat view
+│       ├── dm.html                 # Direct Messages view
+│       ├── contacts_base.html      # Contact pages base template
+│       ├── contacts-manage.html    # Contact Management settings
+│       ├── contacts-pending.html   # Pending contacts view
+│       └── contacts-existing.html  # Existing contacts view
+├── docs/                           # Documentation
+├── images/                         # Screenshots and diagrams
+├── requirements.txt                # Python dependencies
+├── .env.example                    # Example environment config
+└── README.md
+```
+
+---
+
+## Message File Format
+
+Location: `~/.config/meshcore/<device_name>.msgs` (JSON Lines)
+
+### Message Types
+
+**Channel messages:**
+```json
+{"type": "CHAN", "text": "User: message", "timestamp": 1766300846}
+{"type": "SENT_CHAN", "text": "my message", "name": "DeviceName", "timestamp": 1766309432}
+```
+
+**Private messages:**
+```json
+{"type": "PRIV", "text": "message", "sender_timestamp": 1766300846, "pubkey_prefix": "abc123", "sender": "User"}
+{"type": "SENT_MSG", "text": "message", "recipient": "User", "expected_ack": "xyz", "suggested_timeout": 30000}
+```
+
+**Note on SENT_MSG:** Requires meshcore-cli >= 1.3.12 for correct format with both `recipient` and `sender` fields.
+
+---
+
+## API Reference
+
+### Main Web UI Endpoints
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| GET | `/api/messages` | List messages (supports `?archive_date`, `?days`, `?channel_idx`) |
+| POST | `/api/messages` | Send message (`{text, channel_idx, reply_to?}`) |
+| GET | `/api/messages/updates` | Check for new messages (smart refresh) |
+| GET | `/api/status` | Connection status |
+| GET | `/api/contacts` | List contacts |
+| GET | `/api/contacts/detailed` | Full contact_info data |
+| POST | `/api/contacts/delete` | Delete contact by name |
+| GET | `/api/contacts/pending` | List pending contacts |
+| POST | `/api/contacts/pending/approve` | Approve pending contact |
+| POST | `/api/contacts/preview-cleanup` | Preview cleanup matches |
+| POST | `/api/contacts/cleanup` | Execute contact cleanup |
+| GET | `/api/channels` | List all channels |
+| POST | `/api/channels` | Create new channel |
+| POST | `/api/channels/join` | Join existing channel |
+| DELETE | `/api/channels/<index>` | Remove channel |
+| GET | `/api/channels/<index>/qr` | Generate QR code |
+| GET | `/api/dm/conversations` | List DM conversations |
+| GET | `/api/dm/messages` | Get messages for conversation |
+| POST | `/api/dm/messages` | Send DM |
+| GET | `/api/dm/updates` | Check for new DMs |
+| GET | `/api/device/info` | Device information |
+| GET | `/api/device/settings` | Get device settings |
+| POST | `/api/device/settings` | Update device settings |
+| POST | `/api/device/command` | Execute special command |
+| GET | `/api/read_status` | Get server-side read status |
+| POST | `/api/read_status/mark_read` | Mark messages as read |
+| GET | `/api/archives` | List available archives |
+| POST | `/api/archive/trigger` | Manually trigger archiving |
+
+### Bridge Internal API (Port 5001)
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| POST | `/cli` | Execute meshcli command |
+| GET | `/health` | Bridge health check |
+| GET | `/pending_contacts` | List pending contacts |
+| POST | `/add_pending` | Approve pending contact |
+| GET | `/device/settings` | Get device settings |
+| POST | `/device/settings` | Update device settings |
+
+**Base URL:** `http://meshcore-bridge:5001` (internal Docker network only)
+
+---
+
+## Offline Support
+
+The application works completely offline without internet connection - perfect for mesh networks in remote or emergency scenarios.
+
+### Local Vendor Libraries
+
+| Library | Size | Location |
+|---------|------|----------|
+| Bootstrap 5.3.2 CSS | ~227 KB | `static/vendor/bootstrap/css/` |
+| Bootstrap 5.3.2 JS | ~80 KB | `static/vendor/bootstrap/js/` |
+| Bootstrap Icons 1.11.2 | ~398 KB | `static/vendor/bootstrap-icons/` |
+| Emoji Picker Element | ~529 KB | `static/vendor/emoji-picker-element/` |
+
+**Total offline package size:** ~1.2 MB
+
+### Service Worker Caching
+
+- **Cache version:** `mc-webui-v3`
+- **Strategy:** Hybrid caching
+  - **Cache-first** for vendor libraries (static, unchanging)
+  - **Network-first** for app code (dynamic, needs updates)
+
+### How It Works
+
+1. On first visit (online), Service Worker installs and caches all assets
+2. Vendor libraries (Bootstrap, Icons, Emoji Picker) loaded from cache instantly
+3. App code checks network first, falls back to cache if offline
+4. Complete UI functionality available offline
+5. Only API calls (messages, channels, contacts) require connectivity
+
+---
+
+## Environment Variables
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `MC_SERIAL_PORT` | Serial device path | - |
+| `MC_DEVICE_NAME` | Device name (for files) | - |
+| `MC_CONFIG_DIR` | Configuration directory | `./data/meshcore` |
+| `MC_ARCHIVE_DIR` | Archive directory | `./data/archive` |
+| `MC_ARCHIVE_ENABLED` | Enable automatic archiving | `true` |
+| `MC_ARCHIVE_RETENTION_DAYS` | Days to show in live view | `7` |
+| `FLASK_HOST` | Listen address | `0.0.0.0` |
+| `FLASK_PORT` | Web server port | `5000` |
+| `FLASK_DEBUG` | Debug mode | `false` |
+| `TZ` | Timezone for logs | `UTC` |
+
+---
+
+## Persistent Settings
+
+### Settings File
+
+**Location:** `MC_CONFIG_DIR/.webui_settings.json`
+
+```json
+{
+  "manual_add_contacts": false
+}
+```
+
+### Read Status File
+
+**Location:** `MC_CONFIG_DIR/.read_status.json`
+
+Stores per-channel and per-conversation read timestamps for cross-device synchronization.
+
+```json
+{
+  "channels": {"0": 1735900000, "1": 1735900100},
+  "dm": {"name_User1": 1735900200}
+}
+```
+
+---
+
+## Related Documentation
+
+- [User Guide](user-guide.md) - How to use all features
+- [Troubleshooting](troubleshooting.md) - Common issues and solutions
+- [Docker Installation](docker-install.md) - How to install Docker