adam/Remote-Terminal-for-MeshCore
1
0
Fork 0
mirror of https://github.com/jkingsman/Remote-Terminal-for-MeshCore.git synced 2026-08-06 16:53:38 +02:00
Code Issues Packages Projects Releases Wiki Activity

Outgoing WS now echoes, websock reclamation after unmount cleanup, hash fix for empty contacts, no double bot broadcast, AGENTS.md + test fixes (this should have been more than one commit lol)

Browse Source
This commit is contained in:
Jack Kingsman
2026-02-11 23:59:05 -08:00
parent fef18a943c
commit f73fa54532
14 changed files with 816 additions and 1391 deletions
Download Patch File Download Diff File Expand all files Collapse all files
app/AGENTS.md
+158 -668
View File
@@ -1,717 +1,207 @@
# Backend AGENTS.md
This document provides context for AI assistants and developers working on the FastAPI backend.
This document is the backend working guide for agents and developers.
Keep it aligned with `app/` source files and router behavior.
## Technology Stack
## Stack
- **FastAPI** - Async web framework with automatic OpenAPI docs
- **aiosqlite** - Async SQLite driver
- **meshcore** - MeshCore radio library (local dependency at `../meshcore_py`)
- **Pydantic** - Data validation and settings management
- **PyCryptodome** - AES-128 encryption for packet decryption
- **UV** - Python package manager
- FastAPI
- aiosqlite
- Pydantic
- MeshCore Python library (`references/meshcore_py`)
- PyCryptodome
## Directory Structure
## Backend Map
```
```text
app/
├── main.py # FastAPI app, lifespan, router registration, static file serving
├── config.py # Pydantic settings (env vars: MESHCORE_*)
├── database.py # SQLite schema, connection management, runs migrations
├── migrations.py # Database migrations using SQLite user_version pragma
├── models.py # Pydantic models for API request/response
├── repository.py # Database CRUD (ContactRepository, ChannelRepository, etc.)
├── radio.py # RadioManager - serial connection to MeshCore device
├── radio_sync.py # Periodic sync, contact auto-loading to radio
├── decoder.py # Packet decryption (channel + direct messages)
├── packet_processor.py # Raw packet processing, advertisement handling
├── keystore.py # Ephemeral key store (private key in memory only)
├── event_handlers.py # Radio event subscriptions, ACK tracking, repeat detection
├── websocket.py # WebSocketManager for real-time client updates
└── routers/ # All routes prefixed with /api
├── health.py # GET /api/health
├── radio.py # Radio config, advertise, private key, reboot
├── contacts.py # Contact CRUD, radio sync, mark-read
├── channels.py # Channel CRUD, radio sync, mark-read
├── messages.py # Message list and send (direct/channel)
├── packets.py # Raw packet endpoints, historical decryption
├── read_state.py # Read state: unread counts, mark-all-read
├── settings.py # App settings (max_radio_contacts, experimental_channel_double_send, etc.)
└── ws.py # WebSocket endpoint at /api/ws
├── main.py # App startup/lifespan, router registration, static frontend mounting
├── config.py # Env-driven runtime settings
├── database.py # SQLite connection + base schema + migration runner
├── migrations.py # Schema migrations (SQLite user_version)
├── models.py # Pydantic request/response models
├── repository.py # Data access layer
├── radio.py # RadioManager + auto-reconnect monitor
├── radio_sync.py # Polling, sync, periodic advertisement loop
├── decoder.py # Packet parsing/decryption
├── packet_processor.py # Raw packet pipeline, dedup, path handling
├── event_handlers.py # MeshCore event subscriptions and ACK tracking
├── websocket.py # WS manager + broadcast helpers
├── bot.py # Bot execution and outbound bot sends
├── dependencies.py # Shared FastAPI dependency providers
├── frontend_static.py # Mount/serve built frontend (production)
└── routers/
├── health.py
├── radio.py
├── contacts.py
├── channels.py
├── messages.py
├── packets.py
├── read_state.py
├── settings.py
└── ws.py
```
## Intentional Security Design Decisions
## Core Runtime Flows
The following are **deliberate design choices**, not bugs. They are documented in the README with appropriate warnings. Do not "fix" these or flag them as vulnerabilities.
### Incoming data
1. **No CORS restrictions**: `CORSMiddleware` in `main.py` allows all origins, methods, and headers. This lets users access their radio from any device/origin on their network.
2. **No authentication or authorization**: All API endpoints and the WebSocket are openly accessible. The app is designed for trusted networks only (home LAN, VPN).
3. **Arbitrary bot code execution**: `bot.py` uses `exec()` with full `__builtins__` to run user-provided Python code. This is intentional — bots are a power-user automation feature. Safeguards are limited to timeouts and concurrency limits, not sandboxing.
1. Radio emits events.
2. `on_rx_log_data` stores raw packet and tries decrypt/pipeline handling.
3. Decrypted messages are inserted into `messages` and broadcast over WS.
4. `CONTACT_MSG_RECV` is a fallback DM path when packet pipeline cannot decrypt.
## Key Architectural Patterns
### Outgoing messages
### Repository Pattern
1. Send endpoints in `routers/messages.py` call MeshCore commands.
2. Message is persisted as outgoing.
3. Endpoint broadcasts WS `message` event so all live clients update.
4. ACK/repeat updates arrive later as `message_acked` events.
All database operations go through repository classes in `repository.py`:
### Connection lifecycle
```python
from app.repository import ContactRepository, ChannelRepository, MessageRepository, RawPacketRepository, AppSettingsRepository
- `RadioManager.start_connection_monitor()` checks health every 5s.
- On reconnect, monitor runs `post_connect_setup()` before broadcasting healthy state.
- Setup includes handler registration, key export, time sync, contact/channel sync, polling/advert tasks.
# Examples
contact = await ContactRepository.get_by_key_prefix("abc123")
await MessageRepository.create(msg_type="PRIV", text="Hello", received_at=timestamp)
await RawPacketRepository.mark_decrypted(packet_id, message_id)
## Important Behaviors
# App settings (single-row pattern)
settings = await AppSettingsRepository.get()
await AppSettingsRepository.update(auto_decrypt_dm_on_advert=True)
await AppSettingsRepository.update(experimental_channel_double_send=True)
await AppSettingsRepository.add_favorite("contact", public_key)
```
### Read/unread state
### Radio Connection
- Server is source of truth (`contacts.last_read_at`, `channels.last_read_at`).
- `GET /api/read-state/unreads` returns counts, mention flags, and `last_message_times`.
`RadioManager` in `radio.py` handles radio connection over Serial, TCP, or BLE:
### Echo/repeat dedup
```python
from app.radio import radio_manager
- Message uniqueness: `(type, conversation_key, text, sender_timestamp)`.
- Duplicate insert is treated as an echo/repeat; ACK count/path list is updated.
# Access meshcore instance
if radio_manager.meshcore:
await radio_manager.meshcore.commands.send_msg(dst, msg)
### Periodic advertisement
# Check connection info (e.g. "Serial: /dev/ttyUSB0", "TCP: 192.168.1.1:4000", "BLE: AA:BB:CC:DD:EE:FF")
print(radio_manager.connection_info)
```
- Controlled by `app_settings.advert_interval` (seconds).
- `0` means disabled.
- Last send time tracked in `app_settings.last_advert_time`.
Transport is configured via env vars (see root AGENTS.md). When no transport env vars are set, serial auto-detection is used.
### Event-Driven Architecture
Radio events flow through `event_handlers.py`:
| Event | Handler | Actions |
|-------|---------|---------|
| `CONTACT_MSG_RECV` | `on_contact_message` | **Fallback only** - stores DM if packet processor didn't handle it |
| `RX_LOG_DATA` | `on_rx_log_data` | Store packet, decrypt channels/DMs, broadcast via WS |
| `PATH_UPDATE` | `on_path_update` | Update contact path info |
| `NEW_CONTACT` | `on_new_contact` | Sync contact from radio's internal database |
| `ACK` | `on_ack` | Match pending ACKs, mark message acked, broadcast |
**Note on DM handling**: Direct messages are primarily handled by the packet processor via
`RX_LOG_DATA`, which decrypts using the exported private key. The `CONTACT_MSG_RECV` handler
exists as a fallback for radios without `ENABLE_PRIVATE_KEY_EXPORT=1` in firmware.
### WebSocket Broadcasting
Real-time updates use `ws_manager` singleton:
```python
from app.websocket import ws_manager
# Broadcast to all connected clients
await ws_manager.broadcast("message", {"id": 1, "text": "Hello"})
```
Event types: `health`, `contacts`, `channels`, `message`, `contact`, `raw_packet`, `message_acked`, `error`
**Note:** The WebSocket initial connect only sends `health`. Contacts and channels are fetched
via REST (`GET /api/contacts`, `GET /api/channels`) for faster parallel loading. The WS still
broadcasts real-time `contacts`/`channels` updates when data changes.
Helper functions for common broadcasts:
```python
from app.websocket import broadcast_error, broadcast_health
# Notify clients of errors (shows toast in frontend)
broadcast_error("Operation failed", "Additional details")
# Notify clients of connection status change
broadcast_health(radio_connected=True, connection_info="Serial: /dev/ttyUSB0")
```
### Connection Monitoring
`RadioManager` includes a background task that monitors connection status:
- Checks connection every 5 seconds
- Broadcasts `health` event on status change
- Attempts automatic reconnection when connection lost
- **Runs full `post_connect_setup()` after successful reconnect** (event handlers, key export, time sync, contact/channel sync, advertisements, message polling)
- Resilient to transient errors (logs and continues rather than crashing)
- Supports manual reconnection via `POST /api/radio/reconnect`
```python
from app.radio import radio_manager
# Manual reconnection
success = await radio_manager.reconnect()
# Background monitor (started automatically in app lifespan)
await radio_manager.start_connection_monitor()
await radio_manager.stop_connection_monitor()
```
### Message Polling
Periodic message polling serves as a fallback for platforms where push events are unreliable.
Use `pause_polling()` to temporarily suspend polling during operations that need exclusive
radio access (e.g., repeater CLI commands):
```python
from app.radio_sync import pause_polling, is_polling_paused
# Pause polling during sensitive operations (supports nesting)
async with pause_polling():
# Polling is paused here
await do_repeater_operation()
async with pause_polling():
# Still paused (nested)
await do_another_operation()
# Still paused (outer context active)
# Polling resumes when all contexts exit
# Check current state
if is_polling_paused():
print("Polling is currently paused")
```
### Periodic Advertisement
The server automatically sends an advertisement every hour to announce presence on the mesh.
This helps maintain visibility to other nodes and refreshes routing information.
- Started automatically on radio connection
- Interval: 1 hour (3600 seconds)
- Uses flood mode for maximum reach
```python
from app.radio_sync import start_periodic_advert, stop_periodic_advert, send_advertisement
# Start/stop periodic advertising
start_periodic_advert() # Started automatically in lifespan
await stop_periodic_advert()
# Manual advertisement
await send_advertisement() # Returns True on success
```
## Database Schema
```sql
contacts (
public_key TEXT PRIMARY KEY, -- 64-char hex
name TEXT,
type INTEGER DEFAULT 0, -- 0=unknown, 1=client, 2=repeater, 3=room
flags INTEGER DEFAULT 0,
last_path TEXT, -- Routing path hex
last_path_len INTEGER DEFAULT -1,
last_advert INTEGER, -- Unix timestamp of last advertisement
lat REAL, lon REAL,
last_seen INTEGER,
on_radio INTEGER DEFAULT 0, -- Boolean: contact loaded on radio
last_contacted INTEGER, -- Unix timestamp of last message sent/received
last_read_at INTEGER -- Unix timestamp when conversation was last read
)
channels (
key TEXT PRIMARY KEY, -- 32-char hex channel key
name TEXT NOT NULL,
is_hashtag INTEGER DEFAULT 0, -- Key derived from SHA256(name)[:16]
on_radio INTEGER DEFAULT 0,
last_read_at INTEGER -- Unix timestamp when channel was last read
)
messages (
id INTEGER PRIMARY KEY,
type TEXT NOT NULL, -- 'PRIV' or 'CHAN'
conversation_key TEXT NOT NULL, -- User pubkey for PRIV, channel key for CHAN
text TEXT NOT NULL,
sender_timestamp INTEGER,
received_at INTEGER NOT NULL,
paths TEXT, -- JSON array of {path, received_at} for multiple delivery paths
txt_type INTEGER DEFAULT 0,
signature TEXT,
outgoing INTEGER DEFAULT 0,
acked INTEGER DEFAULT 0,
UNIQUE(type, conversation_key, text, sender_timestamp) -- Deduplication
)
raw_packets (
id INTEGER PRIMARY KEY,
timestamp INTEGER NOT NULL,
data BLOB NOT NULL, -- Raw packet bytes
message_id INTEGER, -- FK to messages if decrypted
payload_hash TEXT, -- SHA256 of payload for deduplication (UNIQUE index)
FOREIGN KEY (message_id) REFERENCES messages(id)
)
app_settings (
id INTEGER PRIMARY KEY CHECK (id = 1), -- Single-row pattern
max_radio_contacts INTEGER DEFAULT 200,
experimental_channel_double_send INTEGER DEFAULT 0, -- Experimental delayed byte-perfect channel resend
favorites TEXT DEFAULT '[]', -- JSON array of {type, id}
auto_decrypt_dm_on_advert INTEGER DEFAULT 0,
sidebar_sort_order TEXT DEFAULT 'recent', -- 'recent' or 'alpha'
last_message_times TEXT DEFAULT '{}', -- JSON object of state_key -> timestamp
preferences_migrated INTEGER DEFAULT 0 -- One-time migration flag
)
```
## Database Migrations (`migrations.py`)
Schema migrations use SQLite's `user_version` pragma for version tracking:
```python
from app.migrations import get_version, set_version, run_migrations
# Check current schema version
version = await get_version(conn) # Returns int (0 for new/unmigrated DB)
# Run pending migrations (called automatically on startup)
applied = await run_migrations(conn) # Returns number of migrations applied
```
### How It Works
1. `database.py` calls `run_migrations()` after schema initialization
2. Each migration checks `user_version` and runs if needed
3. Migrations are idempotent (safe to run multiple times)
4. `ALTER TABLE ADD COLUMN` handles existing columns gracefully
### Adding a New Migration
```python
# In migrations.py
async def run_migrations(conn: aiosqlite.Connection) -> int:
version = await get_version(conn)
applied = 0
if version < 1:
await _migrate_001_add_last_read_at(conn)
await set_version(conn, 1)
applied += 1
# Add new migrations here:
# if version < 2:
# await _migrate_002_something(conn)
# await set_version(conn, 2)
# applied += 1
return applied
```
## Packet Decryption (`decoder.py`)
The decoder handles MeshCore packet decryption for historical packet analysis:
### Packet Types
```python
class PayloadType(IntEnum):
GROUP_TEXT = 0x05 # Channel messages (decryptable)
TEXT_MESSAGE = 0x02 # Direct messages
ACK = 0x03
ADVERT = 0x04
# ... see decoder.py for full list
```
### Channel Key Derivation
Hashtag channels derive keys from name:
```python
channel_key = hashlib.sha256(b"#channelname").digest()[:16]
```
### Decryption Flow
1. Parse packet header to get payload type
2. For `GROUP_TEXT`: extract channel_hash (1 byte), cipher_mac (2 bytes), ciphertext
3. Verify HMAC-SHA256 using 32-byte secret (key + 16 zero bytes)
4. Decrypt with AES-128 ECB
5. Parse decrypted content: timestamp (4 bytes), flags (1 byte), "sender: message" text
```python
from app.decoder import try_decrypt_packet_with_channel_key
result = try_decrypt_packet_with_channel_key(raw_bytes, channel_key)
if result:
print(f"{result.sender}: {result.message}")
```
### Direct Message Decryption
Direct messages use ECDH key exchange (Ed25519 → X25519) for shared secret derivation.
**Key storage**: The private key is exported from the radio on startup and stored in memory
via `keystore.py`. This enables server-side DM decryption even when contacts aren't loaded
on the radio.
**Primary path (packet processor)**: When an `RX_LOG_DATA` event contains a `TEXT_MESSAGE`
packet, `packet_processor.py` handles the complete flow:
1. Decrypts using known contact public keys and stored private key
2. Filters CLI responses (txt_type=1 in flags)
3. Stores message in database
4. Broadcasts via WebSocket
5. Updates contact's last_contacted timestamp
6. Triggers bot if enabled
**Fallback path (event handler)**: If the packet processor can't decrypt (no private key
export, unknown contact), `on_contact_message` handles DMs from the MeshCore library's
`CONTACT_MSG_RECV` event. DB deduplication prevents double-storage when both paths fire.
**Prefix-stored DMs (edge case)**: A rare scenario can occur when the radio can decrypt
a DM (contact is on the radio) but the server cannot (private key not exported or
contact not yet known server-side). The `CONTACT_MSG_RECV` payload may only include a
pubkey prefix. If the full key isn't known yet, the message is stored with the prefix
as `conversation_key`. When a full contact key becomes known (via advertisement or
radio sync), the server attempts to **claim** those prefix messages and upgrade them
to the full key. Claims only occur when the prefix matches exactly one contact to
avoid mis-attribution in large contact sets. Until claimed, these DMs will not show
in the UI because conversations are keyed by full public keys.
**Outgoing DMs**: Outgoing direct messages are only sent via the app's REST API
(`POST /api/messages/direct`), which stores the plaintext directly in the database.
No decryption is needed for outgoing DMs. The real-time packet processor may also see
the outgoing packet via `RX_LOG_DATA`, but the DB UNIQUE constraint deduplicates it
against the already-stored plaintext. Historical decryption intentionally skips outgoing
packets for the same reason — the app already has the plaintext.
**Historical decryption**: When creating a contact with `try_historical=True`, the server
attempts to decrypt all stored `TEXT_MESSAGE` packets for that contact. This only recovers
**incoming** messages; outgoing DMs are already stored as plaintext by the send endpoint.
**Direction detection**: The decoder uses the 1-byte dest_hash and src_hash to determine
if a message is incoming or outgoing. Edge case: when both bytes match (1/256 chance),
defaults to treating as incoming.
```python
from app.decoder import try_decrypt_dm
result = try_decrypt_dm(raw_bytes, private_key, contact_public_key)
if result:
print(f"{result.message} (timestamp={result.timestamp})")
```
## Advertisement Parsing (`decoder.py`)
Advertisement packets contain contact information including optional GPS coordinates.
### Packet Structure
```
Bytes 0-31: Public key (32 bytes)
Bytes 32-35: Timestamp (4 bytes, little-endian Unix timestamp)
Bytes 36-99: Signature (64 bytes)
Byte 100: App flags
Bytes 101+: Optional fields (location, name) based on flags
```
### App Flags (byte 100)
- Bits 0-3: Device role (1=Chat, 2=Repeater, 3=Room, 4=Sensor)
- Bit 4: Has location (lat/lon follow)
- Bit 5: Has feature 1
- Bit 6: Has feature 2
- Bit 7: Has name (null-terminated string at end)
### GPS Extraction
When bit 4 is set, latitude and longitude follow as signed int32 little-endian values,
divided by 1,000,000 to get decimal degrees:
```python
from app.decoder import parse_advertisement
advert = parse_advertisement(payload_bytes)
if advert:
print(f"Device role: {advert.device_role}") # 1=Chat, 2=Repeater
if advert.lat and advert.lon:
print(f"Location: {advert.lat}, {advert.lon}")
```
### Data Flow
1. `event_handlers.py` receives ADVERTISEMENT event
2. `packet_processor.py` calls `parse_advertisement()` to extract data
3. Contact is upserted with location data (`lat`, `lon`) and `device_role` as `type`
4. Frontend MapView displays contacts with GPS coordinates
## ACK and Repeat Detection
The `acked` field is an integer count, not a boolean:
- `0` = not acked
- `1` = one ACK/echo received
- `2+` = multiple flood echoes received
### Direct Message ACKs
When sending a direct message, an expected ACK code is tracked:
```python
from app.event_handlers import track_pending_ack
track_pending_ack(expected_ack="abc123", message_id=42, timeout_ms=30000)
```
When ACK event arrives, the message's ack count is incremented.
### Channel Message Repeats
Flood messages echo back through repeaters. Repeats are detected via the database
UNIQUE constraint on `(type, conversation_key, text, sender_timestamp)`. When an INSERT
hits a duplicate, `_handle_duplicate_message()` in `packet_processor.py` increments the
ack count and adds the new delivery path. There is no timestamp-windowed matching;
deduplication is exact-match only.
Each repeat increments the ack count. The frontend displays:
- `?` = no acks
- `✓` = 1 echo
- `✓2`, `✓3`, etc. = multiple echoes (real-time updates via WebSocket)
### Auto-Contact Sync to Radio
To enable the radio to auto-ACK incoming DMs, recent non-repeater contacts are
automatically loaded to the radio. Configured via `max_radio_contacts` setting (default 200).
- Triggered on each advertisement from a non-repeater contact
- Loads most recently contacted non-repeaters (by `last_contacted` timestamp)
- Throttled to at most once per 30 seconds
- `last_contacted` updated on message send/receive
```python
from app.radio_sync import sync_recent_contacts_to_radio
result = await sync_recent_contacts_to_radio(force=True)
# Returns: {"loaded": 5, "already_on_radio": 195, "failed": 0}
```
## API Endpoints
All endpoints are prefixed with `/api`.
## API Surface (all under `/api`)
### Health
- `GET /api/health` - Connection status, connection info
- `GET /health`
### Radio
- `GET /api/radio/config` - Read config (public key, name, radio params)
- `PATCH /api/radio/config` - Update name, lat/lon, tx_power, radio params
- `PUT /api/radio/private-key` - Import private key to radio (write-only)
- `POST /api/radio/advertise?flood=true` - Send advertisement
- `POST /api/radio/reboot` - Reboot radio or reconnect if disconnected
- `POST /api/radio/reconnect` - Manual reconnection attempt
- `GET /radio/config`
- `PATCH /radio/config`
- `PUT /radio/private-key`
- `POST /radio/advertise`
- `POST /radio/reboot`
- `POST /radio/reconnect`
### Contacts
- `GET /api/contacts` - List from database
- `GET /api/contacts/{key}` - Get by public key or prefix
- `POST /api/contacts` - Create contact (optionally trigger historical DM decryption)
- `POST /api/contacts/sync` - Pull from radio to database
- `POST /api/contacts/{key}/add-to-radio` - Push to radio
- `POST /api/contacts/{key}/remove-from-radio` - Remove from radio
- `POST /api/contacts/{key}/mark-read` - Mark conversation as read (updates last_read_at)
- `POST /api/contacts/{key}/telemetry` - Request telemetry from repeater (see below)
- `GET /contacts`
- `GET /contacts/{public_key}`
- `POST /contacts`
- `DELETE /contacts/{public_key}`
- `POST /contacts/sync`
- `POST /contacts/{public_key}/add-to-radio`
- `POST /contacts/{public_key}/remove-from-radio`
- `POST /contacts/{public_key}/mark-read`
- `POST /contacts/{public_key}/telemetry`
- `POST /contacts/{public_key}/command`
- `POST /contacts/{public_key}/trace`
### Channels
- `GET /api/channels` - List from database
- `GET /api/channels/{key}` - Get by channel key
- `POST /api/channels` - Create (hashtag if name starts with # or no key provided)
- `POST /api/channels/sync` - Pull from radio
- `POST /api/channels/{key}/mark-read` - Mark channel as read (updates last_read_at)
- `DELETE /api/channels/{key}` - Delete channel
### Read State
- `GET /api/read-state/unreads?name=X` - Server-computed unread counts, mention flags, and last message times
- `POST /api/read-state/mark-all-read` - Mark all contacts and channels as read
- `GET /channels`
- `GET /channels/{key}`
- `POST /channels`
- `DELETE /channels/{key}`
- `POST /channels/sync`
- `POST /channels/{key}/mark-read`
### Messages
- `GET /api/messages?type=&conversation_key=&limit=&offset=` - List with filters
- `POST /api/messages/direct` - Send direct message
- `POST /api/messages/channel` - Send channel message (stores outgoing immediately; response includes current ack count)
- `GET /messages`
- `POST /messages/direct`
- `POST /messages/channel`
### Packets
- `GET /api/packets/undecrypted/count` - Count of undecrypted packets
- `POST /api/packets/decrypt/historical` - Try decrypting old packets with new key
- `GET /packets/undecrypted/count`
- `POST /packets/decrypt/historical`
- `POST /packets/maintenance`
### Read state
- `GET /read-state/unreads`
- `POST /read-state/mark-all-read`
### Settings
- `GET /api/settings` - Get all app settings
- `PATCH /api/settings` - Update settings (max_radio_contacts, experimental_channel_double_send, auto_decrypt_dm_on_advert, sidebar_sort_order)
- `POST /api/settings/favorites` - Add a favorite
- `DELETE /api/settings/favorites` - Remove a favorite
- `POST /api/settings/favorites/toggle` - Toggle favorite status
- `POST /api/settings/migrate` - One-time migration from frontend localStorage
- `GET /settings`
- `PATCH /settings`
- `POST /settings/favorites/toggle`
- `POST /settings/migrate`
### WebSocket
- `WS /api/ws` - Real-time updates (health, contacts, channels, messages, raw packets)
- `WS /ws`
### Static Files (Production)
In production, the backend serves the frontend if `frontend/dist` exists. Users must build the
frontend first (`cd frontend && npm install && npm run build`):
- `/` - Serves `frontend/dist/index.html`
- `/assets/*` - Serves compiled JS/CSS from `frontend/dist/assets/`
- `/*` - Falls back to `index.html` for SPA routing
## WebSocket Events
- `health`
- `contacts`
- `channels`
- `contact`
- `message`
- `message_acked`
- `raw_packet`
- `error`
- `success`
Initial WS connect sends `health` only. Contacts/channels are loaded by REST.
## Data Model Notes
Main tables:
- `contacts`
- `channels`
- `messages`
- `raw_packets`
- `app_settings`
`app_settings` fields in active model:
- `max_radio_contacts`
- `experimental_channel_double_send`
- `favorites`
- `auto_decrypt_dm_on_advert`
- `sidebar_sort_order`
- `last_message_times`
- `preferences_migrated`
- `advert_interval`
- `last_advert_time`
- `bots`
## Security Posture (intentional)
- No authn/authz.
- No CORS restriction (`*`).
- Bot code executes user-provided Python via `exec()`.
These are product decisions for trusted-network deployments; do not flag as accidental vulnerabilities.
## Testing
Run tests with:
Run backend tests:
```bash
PYTHONPATH=. uv run pytest tests/ -v
```
Key test files:
- `tests/test_decoder.py` - Channel + direct message decryption, key exchange, real-world test vectors
- `tests/test_keystore.py` - Ephemeral key store operations
- `tests/test_event_handlers.py` - ACK tracking, repeat detection, CLI response filtering
- `tests/test_api.py` - API endpoint tests, read state tracking
- `tests/test_migrations.py` - Migration system, schema versioning
High-signal suites:
- `tests/test_packet_pipeline.py`
- `tests/test_event_handlers.py`
- `tests/test_send_messages.py`
- `tests/test_radio.py`
- `tests/test_api.py`
- `tests/test_migrations.py`
## Common Tasks
## Editing Checklist
### Adding a New Endpoint
1. Create or update router in `app/routers/`
2. Define Pydantic models in `app/models.py` if needed
3. Add repository methods in `app/repository.py` for database operations
4. Register router in `app/main.py` if new file
5. Add tests in `tests/`
### Adding a New Event Handler
1. Define handler in `app/event_handlers.py`
2. Register in `register_event_handlers()` function
3. Broadcast updates via `ws_manager` as needed
### Working with Radio Commands
```python
# Available via radio_manager.meshcore.commands
await mc.commands.send_msg(dst, msg)
await mc.commands.send_chan_msg(chan, msg)
await mc.commands.get_contacts()
await mc.commands.add_contact(contact_dict)
await mc.commands.set_channel(idx, name, key)
await mc.commands.send_advert(flood=True)
```
## Repeater Telemetry
The `POST /api/contacts/{key}/telemetry` endpoint fetches status, neighbors, and ACL from repeaters (contact type=2).
### Request Flow
1. Verify contact exists and is a repeater (type=2)
2. Add contact to radio with stored path data (from advertisements)
3. Send login with password
4. Request status with retries (3 attempts, 10s timeout)
5. Fetch neighbors with `fetch_all_neighbours()` (handles pagination)
6. Fetch ACL with `req_acl_sync()`
7. Resolve pubkey prefixes to contact names from database
### ACL Permission Levels
```python
ACL_PERMISSION_NAMES = {
0: "Guest",
1: "Read-only",
2: "Read-write",
3: "Admin",
}
```
### Response Models
```python
class NeighborInfo(BaseModel):
pubkey_prefix: str # 4-12 char prefix
name: str | None # Resolved contact name
snr: float # Signal-to-noise ratio in dB
last_heard_seconds: int # Seconds since last heard
class AclEntry(BaseModel):
pubkey_prefix: str # 12 char prefix
name: str | None # Resolved contact name
permission: int # 0-3
permission_name: str # Human-readable name
class TelemetryResponse(BaseModel):
# Status fields
pubkey_prefix: str
battery_volts: float # Converted from mV
uptime_seconds: int
# ... signal quality, packet counts, etc.
# Related data
neighbors: list[NeighborInfo]
acl: list[AclEntry]
```
## Repeater CLI Commands
After login via telemetry endpoint, you can send CLI commands to repeaters:
### Endpoint
`POST /api/contacts/{key}/command` - Send a CLI command (assumes already logged in)
### Request/Response
```python
class CommandRequest(BaseModel):
command: str # CLI command to send
class CommandResponse(BaseModel):
command: str # Echo of sent command
response: str # Response from repeater
sender_timestamp: int | None # Timestamp from response
```
### Common Commands
```
get name / set name <value> # Repeater name
get tx / set tx <dbm> # TX power
get radio / set radio <freq,bw,sf,cr> # Radio params
tempradio <freq,bw,sf,cr,mins> # Temporary radio change
setperm <pubkey> <0-3> # ACL: 0=guest, 1=ro, 2=rw, 3=admin
clock / clock sync # Get/sync time
ver # Firmware version
reboot # Restart repeater
```
### CLI Response Filtering
CLI responses have `txt_type=1` (vs `txt_type=0` for normal messages). Both DM handling
paths filter these to prevent storage—the command endpoint returns the response directly.
**Packet processor path** (primary):
```python
# In create_dm_message_from_decrypted()
txt_type = decrypted.flags & 0x0F
if txt_type == 1:
return None # Skip CLI responses
```
**Event handler path** (fallback):
```python
# In on_contact_message()
txt_type = payload.get("txt_type", 0)
if txt_type == 1:
return # Skip CLI responses
```
### Helper Function
`prepare_repeater_connection()` handles the login dance:
1. Add contact to radio with stored path from DB (`out_path`, `out_path_len`)
2. Send login with password
3. Wait for key exchange to complete
### Contact Path Tracking
When advertisements are received, path data is extracted and stored:
- `last_path`: Hex string of routing path bytes
- `last_path_len`: Number of hops (-1=flood/unknown, 0=direct, >0=hops through repeaters)
**Shortest path selection**: When receiving echoed advertisements within 60 seconds, the shortest path is kept. This ensures we use the most efficient route when multiple paths exist.
When changing backend behavior:
1. Update/add router and repository tests.
2. Confirm WS event contracts when payload shape changes.
3. Run `PYTHONPATH=. uv run pytest tests/ -v`.
4. If API contract changed, update frontend types and AGENTS docs.
app/bot.py
+2 -7
View File
@@ -173,7 +173,6 @@ async def _send_single_bot_message(
from app.models import SendChannelMessageRequest, SendDirectMessageRequest
from app.routers.messages import send_channel_message, send_direct_message
from app.websocket import broadcast_event
# Serialize bot sends and enforce minimum spacing
async with _bot_send_lock:
@@ -190,15 +189,11 @@ async def _send_single_bot_message(
if is_dm:
logger.info("Bot sending DM reply to %s", sender_key[:12])
request = SendDirectMessageRequest(destination=sender_key, text=message_text)
message = await send_direct_message(request)
# Broadcast to WebSocket (endpoint returns to HTTP caller, bot needs explicit broadcast)
broadcast_event("message", message.model_dump())
await send_direct_message(request)
elif channel_key:
logger.info("Bot sending channel reply to %s", channel_key[:8])
request = SendChannelMessageRequest(channel_key=channel_key, text=message_text)
message = await send_channel_message(request)
# Broadcast to WebSocket
broadcast_event("message", message.model_dump())
await send_channel_message(request)
else:
logger.warning("Cannot send bot response: no destination")
return # Don't update timestamp if we didn't send
app/radio.py
+10 -4
View File
@@ -365,7 +365,7 @@ class RadioManager:
self._meshcore = None
logger.debug("Radio disconnected")
async def reconnect(self) -> bool:
async def reconnect(self, *, broadcast_on_success: bool = True) -> bool:
"""Attempt to reconnect to the radio.
Returns True if reconnection was successful, False otherwise.
@@ -399,7 +399,8 @@ class RadioManager:
if self.is_connected:
logger.info("Radio reconnected successfully at %s", self._connection_info)
broadcast_health(True, self._connection_info)
if broadcast_on_success:
broadcast_health(True, self._connection_info)
return True
else:
logger.warning("Reconnection failed: not connected after connect()")
@@ -435,13 +436,18 @@ class RadioManager:
if not current_connected:
# Attempt reconnection on every loop while disconnected
if not self.is_reconnecting and await self.reconnect():
if not self.is_reconnecting and await self.reconnect(
broadcast_on_success=False
):
await self.post_connect_setup()
broadcast_health(True, self._connection_info)
self._last_connected = True
elif not self._last_connected and current_connected:
# Connection restored (might have reconnected automatically)
# Connection restored (might have reconnected automatically).
# Always run setup before reporting healthy.
logger.info("Radio connection restored")
await self.post_connect_setup()
broadcast_health(True, self._connection_info)
self._last_connected = True
app/routers/messages.py
+7
View File
@@ -10,6 +10,7 @@ from app.event_handlers import track_pending_ack
from app.models import Message, SendChannelMessageRequest, SendDirectMessageRequest
from app.radio import radio_manager
from app.repository import AmbiguousPublicKeyPrefixError, MessageRepository
from app.websocket import broadcast_event
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/messages", tags=["messages"])
@@ -132,6 +133,9 @@ async def send_direct_message(request: SendDirectMessageRequest) -> Message:
acked=0,
)
# Broadcast so all connected clients (not just sender) see the outgoing message immediately.
broadcast_event("message", message.model_dump())
# Trigger bots for outgoing DMs (runs in background, doesn't block response)
from app.bot import run_bot_for_message
@@ -281,6 +285,9 @@ async def send_channel_message(request: SendChannelMessageRequest) -> Message:
acked=acked_count,
)
# Broadcast so all connected clients (not just sender) see the outgoing message immediately.
broadcast_event("message", message.model_dump())
# Trigger bots for outgoing channel messages (runs in background, doesn't block response)
from app.bot import run_bot_for_message