From 973d6953a1e460cbabb17ac7bb4a981e28aed4e2 Mon Sep 17 00:00:00 2001 From: Jack Kingsman Date: Tue, 20 Jan 2026 15:59:20 -0800 Subject: [PATCH] Standardize on AGENTS.md --- AGENTS.md | 316 +++++++++++++++++++++++++++++ CLAUDE.md | 316 +---------------------------- app/{CLAUDE.md => AGENTS.md} | 2 +- frontend/{CLAUDE.md => AGENTS.md} | 2 +- frontend/src/components/AGENTS.md | 326 ++++++++++++++++++++++++++++++ 5 files changed, 647 insertions(+), 315 deletions(-) create mode 100644 AGENTS.md rename app/{CLAUDE.md => AGENTS.md} (99%) rename frontend/{CLAUDE.md => AGENTS.md} (99%) create mode 100644 frontend/src/components/AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..ac18700 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,316 @@ +# RemoteTerm for MeshCore + +## Important Rules + +**NEVER make git commits.** A human must make all commits. You may stage files and prepare commit messages, but do not run `git commit`. + +If instructed to "run all tests" or "get ready for a commit" or other summative, work ending directives, make sure you run the following and that they all pass green: + +```bash +uv run ruff check app/ tests/ --fix # check for python violations +uv run ruff format app/ tests/ # format python +uv run pyright app/ # type check python +PYTHONPATH=. uv run pytest tests/ -v # test python + +cd frontend/ # move to frontend directory +npm run lint:fix # fix lint violations +npm run format # format the code +npm run build # run a frontend build +``` + +## Overview + +A web interface for MeshCore mesh radio networks. The backend connects to a MeshCore-compatible radio over serial and exposes REST/WebSocket APIs. The React frontend provides real-time messaging and radio configuration. + +**For detailed component documentation, see:** +- `app/AGENTS.md` - Backend (FastAPI, database, radio connection, packet decryption) +- `frontend/AGENTS.md` - Frontend (React, state management, WebSocket, components) +- `frontend/src/components/AGENTS.md` - Frontend visualizer feature (a particularly complex and long force-directed graph visualizer component; can skip this file unless you're working on that feature) + +## Architecture Overview + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ Frontend (React) │ +│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │ +│ │ StatusBar│ │ Sidebar │ │MessageList│ │ MessageInput │ │ +│ └──────────┘ └──────────┘ └──────────┘ └──────────────────┘ │ +│ ┌────────────────────────────────────────────────────────────┐ │ +│ │ CrackerPanel (global collapsible, WebGPU cracking) │ │ +│ └────────────────────────────────────────────────────────────┘ │ +│ │ │ +│ useWebSocket ←──── Real-time updates │ +│ │ │ +│ api.ts ←──── REST API calls │ +└───────────────────────────┼──────────────────────────────────────┘ + │ HTTP + WebSocket (/api/*) +┌───────────────────────────┼──────────────────────────────────────┐ +│ Backend (FastAPI) │ +│ ┌──────────┐ ┌──────────────┐ ┌────────────┐ ┌───────────┐ │ +│ │ Routers │→ │ Repositories │→ │ SQLite DB │ │ WebSocket │ │ +│ └──────────┘ └──────────────┘ └────────────┘ │ Manager │ │ +│ ↓ └───────────┘ │ +│ ┌──────────────────────────────────────────────────────────┐ │ +│ │ RadioManager + Event Handlers │ │ +│ └──────────────────────────────────────────────────────────┘ │ +└───────────────────────────┼──────────────────────────────────────┘ + │ Serial + ┌──────┴──────┐ + │ MeshCore │ + │ Radio │ + └─────────────┘ +``` + +## Key Design Principles + +1. **Store-and-serve**: Backend stores all packets even when no client is connected +2. **Parallel storage**: Messages stored both decrypted (when possible) and as raw packets +3. **Extended capacity**: Server stores contacts/channels beyond radio limits (~350 contacts, ~40 channels) +4. **Real-time updates**: WebSocket pushes events; REST for actions +5. **Offline-capable**: Radio operates independently; server syncs when connected +6. **Auto-reconnect**: Background monitor detects disconnection and attempts reconnection + +## Data Flow + +### Incoming Messages + +1. Radio receives message → MeshCore library emits event +2. `event_handlers.py` catches event → stores in database +3. `ws_manager` broadcasts to connected clients +4. Frontend `useWebSocket` receives → updates React state + +### Outgoing Messages + +1. User types message → clicks send +2. `api.sendChannelMessage()` → POST to backend +3. Backend calls `radio_manager.meshcore.commands.send_chan_msg()` +4. Message stored in database with `outgoing=true` +5. For direct messages: ACK tracked; for channel: repeat detection + +### ACK and Repeat Detection + +**Direct messages**: Expected ACK code is tracked. When ACK event arrives, message marked as acked. + +**Channel messages**: Flood messages echo back. The decoder identifies repeats by matching (channel_idx, text_hash, timestamp ±5s) and marks the original as "acked". + +## Directory Structure + +``` +. +├── app/ # FastAPI backend +│ ├── AGENTS.md # Backend documentation +│ ├── main.py # App entry, lifespan +│ ├── routers/ # API endpoints +│ ├── repository.py # Database CRUD +│ ├── event_handlers.py # Radio events +│ ├── decoder.py # Packet decryption +│ └── websocket.py # Real-time broadcasts +├── frontend/ # React frontend +│ ├── AGENTS.md # Frontend documentation +│ ├── src/ +│ │ ├── App.tsx # Main component +│ │ ├── api.ts # REST client +│ │ ├── useWebSocket.ts # WebSocket hook +│ │ └── components/ +│ │ ├── CrackerPanel.tsx # WebGPU key cracking +│ │ ├── MapView.tsx # Leaflet map showing node locations +│ │ └── ... +│ └── vite.config.ts +├── references/meshcore_py/ # MeshCore Python library +├── tests/ # Backend tests (pytest) +├── data/ # SQLite database (runtime) +├── integration_test.html # Browser-based API tests +└── pyproject.toml # Python dependencies +``` + +## Development Setup + +### Backend + +```bash +# Install dependencies +uv sync + +# Run server (auto-detects radio) +uv run uvicorn app.main:app --reload + +# Or specify port +MESHCORE_SERIAL_PORT=/dev/cu.usbserial-0001 uv run uvicorn app.main:app --reload +``` + +### Frontend + +```bash +cd frontend +npm install +npm run dev # http://localhost:5173, proxies /api to :8000 +``` + +### Both Together (Development) + +Terminal 1: `uv run uvicorn app.main:app --reload` +Terminal 2: `cd frontend && npm run dev` + +### Production + +In production, the FastAPI backend serves the compiled frontend: + +```bash +cd frontend && npm run build && cd .. +uv run uvicorn app.main:app --host 0.0.0.0 --port 8000 +``` + +Access at `http://localhost:8000`. All API routes are prefixed with `/api`. + +## Testing + +### Backend (pytest) + +```bash +PYTHONPATH=. uv run pytest tests/ -v +``` + +Key test files: +- `tests/test_decoder.py` - Channel + direct message decryption, key exchange +- `tests/test_keystore.py` - Ephemeral key store +- `tests/test_event_handlers.py` - ACK tracking, repeat detection +- `tests/test_api.py` - API endpoints, read state tracking +- `tests/test_migrations.py` - Database migration system + +### Frontend (Vitest) + +```bash +cd frontend +npm run test:run +``` + +### Integration Tests + +Open `integration_test.html` in a browser with the backend running. + +### Before Completing Changes + +**Always run both backend and frontend validation before finishing any changes:** + +```bash +# From project root - run backend tests +PYTHONPATH=. uv run pytest tests/ -v + +# From project root - run frontend tests and build +cd frontend && npm run test:run && npm run build +``` + +This catches: +- Type mismatches between frontend and backend (e.g., missing fields in TypeScript interfaces) +- Breaking changes to shared types or API contracts +- Runtime errors that only surface during compilation + +## API Summary + +All endpoints are prefixed with `/api` (e.g., `/api/health`). + +| Method | Endpoint | Description | +|--------|----------|-------------| +| GET | `/api/health` | Connection status | +| GET | `/api/radio/config` | Radio configuration | +| PATCH | `/api/radio/config` | Update name, location, radio params | +| POST | `/api/radio/advertise` | Send advertisement | +| POST | `/api/radio/reconnect` | Manual radio reconnection | +| POST | `/api/radio/reboot` | Reboot radio or reconnect if disconnected | +| PUT | `/api/radio/private-key` | Import private key to radio | +| GET | `/api/contacts` | List contacts | +| POST | `/api/contacts` | Create contact (optionally trigger historical DM decrypt) | +| POST | `/api/contacts/sync` | Pull from radio | +| POST | `/api/contacts/{key}/telemetry` | Request telemetry from repeater | +| POST | `/api/contacts/{key}/command` | Send CLI command to repeater | +| GET | `/api/channels` | List channels | +| POST | `/api/channels` | Create channel | +| GET | `/api/messages` | List with filters | +| POST | `/api/messages/direct` | Send direct message | +| POST | `/api/messages/channel` | Send channel message | +| POST | `/api/packets/decrypt/historical` | Decrypt stored packets | +| GET | `/api/packets/decrypt/progress` | Get historical decryption progress | +| POST | `/api/packets/maintenance` | Delete old packets (cleanup) | +| POST | `/api/contacts/{key}/mark-read` | Mark contact conversation as read | +| POST | `/api/channels/{key}/mark-read` | Mark channel as read | +| POST | `/api/read-state/mark-all-read` | Mark all conversations as read | +| GET | `/api/settings` | Get app settings | +| PATCH | `/api/settings` | Update app settings | +| WS | `/api/ws` | Real-time updates | + +## Key Concepts + +### Contact Public Keys + +- Full key: 64-character hex string +- Prefix: 12-character hex (used for matching) +- Lookups use `LIKE 'prefix%'` for matching + +### Contact Types + +- `0` - Unknown +- `1` - Client (regular node) +- `2` - Repeater +- `3` - Room + +### Channel Keys + +- Stored as 32-character hex string (TEXT PRIMARY KEY) +- Hashtag channels: `SHA256("#name")[:16]` converted to hex +- Custom channels: User-provided or generated + +### Message Types + +- `PRIV` - Direct messages +- `CHAN` - Channel messages +- Both use `conversation_key` (user pubkey for PRIV, channel key for CHAN) + +### Read State Tracking + +Read state (`last_read_at`) is tracked **server-side** for consistency across devices: +- Stored as Unix timestamp in `contacts.last_read_at` and `channels.last_read_at` +- Updated via `POST /api/contacts/{key}/mark-read` and `POST /api/channels/{key}/mark-read` +- Bulk update via `POST /api/read-state/mark-all-read` +- Frontend compares `last_read_at` with message `received_at` to count unreads + +**State Tracking Keys (Frontend)**: Generated by `getStateKey()` for message times (sidebar sorting): +- Channels: `channel-{channel_key}` +- Contacts: `contact-{12-char-pubkey-prefix}` + +**Note:** These are NOT the same as `Message.conversation_key` (the database field). + +### Server-Side Decryption + +The server can decrypt packets using stored keys, both in real-time and for historical packets. + +**Channel messages**: Decrypted automatically when a matching channel key is available. + +**Direct messages**: Decrypted server-side using the private key exported from the radio on startup. This enables DM decryption even when the contact isn't loaded on the radio. The private key is stored in memory only (see `keystore.py`). + +## MeshCore Library + +The `meshcore_py` library provides radio communication. Key patterns: + +```python +# Connection +mc = await MeshCore.create_serial(port="/dev/ttyUSB0") + +# Commands +await mc.commands.send_msg(dst, msg) +await mc.commands.send_chan_msg(channel_idx, msg) +await mc.commands.get_contacts() +await mc.commands.set_channel(idx, name, key) + +# Events +mc.subscribe(EventType.CONTACT_MSG_RECV, handler) +mc.subscribe(EventType.CHANNEL_MSG_RECV, handler) +mc.subscribe(EventType.ACK, handler) +``` + +## Environment Variables + +| Variable | Default | Description | +|----------|---------|-------------| +| `MESHCORE_SERIAL_PORT` | auto-detect | Serial port for radio | +| `MESHCORE_DATABASE_PATH` | `data/meshcore.db` | SQLite database location | +| `MESHCORE_MAX_RADIO_CONTACTS` | `200` | Max recent contacts to keep on radio for DM ACKs | diff --git a/CLAUDE.md b/CLAUDE.md index 31c0bb6..058e7cf 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,315 +1,5 @@ -# RemoteTerm for MeshCore - -## Important Rules - -**NEVER make git commits.** A human must make all commits. You may stage files and prepare commit messages, but do not run `git commit`. - -If instructed to "run all tests" or "get ready for a commit" or other summative, work ending directives, make sure you run the following and that they all pass green: - -```bash -uv run ruff check app/ tests/ --fix # check for python violations -uv run ruff format app/ tests/ # format python -uv run pyright app/ # type check python -PYTHONPATH=. uv run pytest tests/ -v # test python - -cd frontend/ # move to frontend directory -npm run lint:fix # fix lint violations -npm run format # format the code -npm run build # run a frontend build -``` - -## Overview - -A web interface for MeshCore mesh radio networks. The backend connects to a MeshCore-compatible radio over serial and exposes REST/WebSocket APIs. The React frontend provides real-time messaging and radio configuration. - **For detailed component documentation, see:** -- `app/CLAUDE.md` - Backend (FastAPI, database, radio connection, packet decryption) -- `frontend/CLAUDE.md` - Frontend (React, state management, WebSocket, components) -## Architecture Overview - -``` -┌─────────────────────────────────────────────────────────────────┐ -│ Frontend (React) │ -│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │ -│ │ StatusBar│ │ Sidebar │ │MessageList│ │ MessageInput │ │ -│ └──────────┘ └──────────┘ └──────────┘ └──────────────────┘ │ -│ ┌────────────────────────────────────────────────────────────┐ │ -│ │ CrackerPanel (global collapsible, WebGPU cracking) │ │ -│ └────────────────────────────────────────────────────────────┘ │ -│ │ │ -│ useWebSocket ←──── Real-time updates │ -│ │ │ -│ api.ts ←──── REST API calls │ -└───────────────────────────┼──────────────────────────────────────┘ - │ HTTP + WebSocket (/api/*) -┌───────────────────────────┼──────────────────────────────────────┐ -│ Backend (FastAPI) │ -│ ┌──────────┐ ┌──────────────┐ ┌────────────┐ ┌───────────┐ │ -│ │ Routers │→ │ Repositories │→ │ SQLite DB │ │ WebSocket │ │ -│ └──────────┘ └──────────────┘ └────────────┘ │ Manager │ │ -│ ↓ └───────────┘ │ -│ ┌──────────────────────────────────────────────────────────┐ │ -│ │ RadioManager + Event Handlers │ │ -│ └──────────────────────────────────────────────────────────┘ │ -└───────────────────────────┼──────────────────────────────────────┘ - │ Serial - ┌──────┴──────┐ - │ MeshCore │ - │ Radio │ - └─────────────┘ -``` - -## Key Design Principles - -1. **Store-and-serve**: Backend stores all packets even when no client is connected -2. **Parallel storage**: Messages stored both decrypted (when possible) and as raw packets -3. **Extended capacity**: Server stores contacts/channels beyond radio limits (~350 contacts, ~40 channels) -4. **Real-time updates**: WebSocket pushes events; REST for actions -5. **Offline-capable**: Radio operates independently; server syncs when connected -6. **Auto-reconnect**: Background monitor detects disconnection and attempts reconnection - -## Data Flow - -### Incoming Messages - -1. Radio receives message → MeshCore library emits event -2. `event_handlers.py` catches event → stores in database -3. `ws_manager` broadcasts to connected clients -4. Frontend `useWebSocket` receives → updates React state - -### Outgoing Messages - -1. User types message → clicks send -2. `api.sendChannelMessage()` → POST to backend -3. Backend calls `radio_manager.meshcore.commands.send_chan_msg()` -4. Message stored in database with `outgoing=true` -5. For direct messages: ACK tracked; for channel: repeat detection - -### ACK and Repeat Detection - -**Direct messages**: Expected ACK code is tracked. When ACK event arrives, message marked as acked. - -**Channel messages**: Flood messages echo back. The decoder identifies repeats by matching (channel_idx, text_hash, timestamp ±5s) and marks the original as "acked". - -## Directory Structure - -``` -. -├── app/ # FastAPI backend -│ ├── CLAUDE.md # Backend documentation -│ ├── main.py # App entry, lifespan -│ ├── routers/ # API endpoints -│ ├── repository.py # Database CRUD -│ ├── event_handlers.py # Radio events -│ ├── decoder.py # Packet decryption -│ └── websocket.py # Real-time broadcasts -├── frontend/ # React frontend -│ ├── CLAUDE.md # Frontend documentation -│ ├── src/ -│ │ ├── App.tsx # Main component -│ │ ├── api.ts # REST client -│ │ ├── useWebSocket.ts # WebSocket hook -│ │ └── components/ -│ │ ├── CrackerPanel.tsx # WebGPU key cracking -│ │ ├── MapView.tsx # Leaflet map showing node locations -│ │ └── ... -│ └── vite.config.ts -├── references/meshcore_py/ # MeshCore Python library -├── tests/ # Backend tests (pytest) -├── data/ # SQLite database (runtime) -├── integration_test.html # Browser-based API tests -└── pyproject.toml # Python dependencies -``` - -## Development Setup - -### Backend - -```bash -# Install dependencies -uv sync - -# Run server (auto-detects radio) -uv run uvicorn app.main:app --reload - -# Or specify port -MESHCORE_SERIAL_PORT=/dev/cu.usbserial-0001 uv run uvicorn app.main:app --reload -``` - -### Frontend - -```bash -cd frontend -npm install -npm run dev # http://localhost:5173, proxies /api to :8000 -``` - -### Both Together (Development) - -Terminal 1: `uv run uvicorn app.main:app --reload` -Terminal 2: `cd frontend && npm run dev` - -### Production - -In production, the FastAPI backend serves the compiled frontend: - -```bash -cd frontend && npm run build && cd .. -uv run uvicorn app.main:app --host 0.0.0.0 --port 8000 -``` - -Access at `http://localhost:8000`. All API routes are prefixed with `/api`. - -## Testing - -### Backend (pytest) - -```bash -PYTHONPATH=. uv run pytest tests/ -v -``` - -Key test files: -- `tests/test_decoder.py` - Channel + direct message decryption, key exchange -- `tests/test_keystore.py` - Ephemeral key store -- `tests/test_event_handlers.py` - ACK tracking, repeat detection -- `tests/test_api.py` - API endpoints, read state tracking -- `tests/test_migrations.py` - Database migration system - -### Frontend (Vitest) - -```bash -cd frontend -npm run test:run -``` - -### Integration Tests - -Open `integration_test.html` in a browser with the backend running. - -### Before Completing Changes - -**Always run both backend and frontend validation before finishing any changes:** - -```bash -# From project root - run backend tests -PYTHONPATH=. uv run pytest tests/ -v - -# From project root - run frontend tests and build -cd frontend && npm run test:run && npm run build -``` - -This catches: -- Type mismatches between frontend and backend (e.g., missing fields in TypeScript interfaces) -- Breaking changes to shared types or API contracts -- Runtime errors that only surface during compilation - -## API Summary - -All endpoints are prefixed with `/api` (e.g., `/api/health`). - -| Method | Endpoint | Description | -|--------|----------|-------------| -| GET | `/api/health` | Connection status | -| GET | `/api/radio/config` | Radio configuration | -| PATCH | `/api/radio/config` | Update name, location, radio params | -| POST | `/api/radio/advertise` | Send advertisement | -| POST | `/api/radio/reconnect` | Manual radio reconnection | -| POST | `/api/radio/reboot` | Reboot radio or reconnect if disconnected | -| PUT | `/api/radio/private-key` | Import private key to radio | -| GET | `/api/contacts` | List contacts | -| POST | `/api/contacts` | Create contact (optionally trigger historical DM decrypt) | -| POST | `/api/contacts/sync` | Pull from radio | -| POST | `/api/contacts/{key}/telemetry` | Request telemetry from repeater | -| POST | `/api/contacts/{key}/command` | Send CLI command to repeater | -| GET | `/api/channels` | List channels | -| POST | `/api/channels` | Create channel | -| GET | `/api/messages` | List with filters | -| POST | `/api/messages/direct` | Send direct message | -| POST | `/api/messages/channel` | Send channel message | -| POST | `/api/packets/decrypt/historical` | Decrypt stored packets | -| GET | `/api/packets/decrypt/progress` | Get historical decryption progress | -| POST | `/api/packets/maintenance` | Delete old packets (cleanup) | -| POST | `/api/contacts/{key}/mark-read` | Mark contact conversation as read | -| POST | `/api/channels/{key}/mark-read` | Mark channel as read | -| POST | `/api/read-state/mark-all-read` | Mark all conversations as read | -| GET | `/api/settings` | Get app settings | -| PATCH | `/api/settings` | Update app settings | -| WS | `/api/ws` | Real-time updates | - -## Key Concepts - -### Contact Public Keys - -- Full key: 64-character hex string -- Prefix: 12-character hex (used for matching) -- Lookups use `LIKE 'prefix%'` for matching - -### Contact Types - -- `0` - Unknown -- `1` - Client (regular node) -- `2` - Repeater -- `3` - Room - -### Channel Keys - -- Stored as 32-character hex string (TEXT PRIMARY KEY) -- Hashtag channels: `SHA256("#name")[:16]` converted to hex -- Custom channels: User-provided or generated - -### Message Types - -- `PRIV` - Direct messages -- `CHAN` - Channel messages -- Both use `conversation_key` (user pubkey for PRIV, channel key for CHAN) - -### Read State Tracking - -Read state (`last_read_at`) is tracked **server-side** for consistency across devices: -- Stored as Unix timestamp in `contacts.last_read_at` and `channels.last_read_at` -- Updated via `POST /api/contacts/{key}/mark-read` and `POST /api/channels/{key}/mark-read` -- Bulk update via `POST /api/read-state/mark-all-read` -- Frontend compares `last_read_at` with message `received_at` to count unreads - -**State Tracking Keys (Frontend)**: Generated by `getStateKey()` for message times (sidebar sorting): -- Channels: `channel-{channel_key}` -- Contacts: `contact-{12-char-pubkey-prefix}` - -**Note:** These are NOT the same as `Message.conversation_key` (the database field). - -### Server-Side Decryption - -The server can decrypt packets using stored keys, both in real-time and for historical packets. - -**Channel messages**: Decrypted automatically when a matching channel key is available. - -**Direct messages**: Decrypted server-side using the private key exported from the radio on startup. This enables DM decryption even when the contact isn't loaded on the radio. The private key is stored in memory only (see `keystore.py`). - -## MeshCore Library - -The `meshcore_py` library provides radio communication. Key patterns: - -```python -# Connection -mc = await MeshCore.create_serial(port="/dev/ttyUSB0") - -# Commands -await mc.commands.send_msg(dst, msg) -await mc.commands.send_chan_msg(channel_idx, msg) -await mc.commands.get_contacts() -await mc.commands.set_channel(idx, name, key) - -# Events -mc.subscribe(EventType.CONTACT_MSG_RECV, handler) -mc.subscribe(EventType.CHANNEL_MSG_RECV, handler) -mc.subscribe(EventType.ACK, handler) -``` - -## Environment Variables - -| Variable | Default | Description | -|----------|---------|-------------| -| `MESHCORE_SERIAL_PORT` | auto-detect | Serial port for radio | -| `MESHCORE_DATABASE_PATH` | `data/meshcore.db` | SQLite database location | -| `MESHCORE_MAX_RADIO_CONTACTS` | `200` | Max recent contacts to keep on radio for DM ACKs | +- `./AGENTS.md` (general project information) +- `app/AGENTS.md` - Backend (FastAPI, database, radio connection, packet decryption) +- `frontend/AGENTS.md` - Frontend (React, state management, WebSocket, components) diff --git a/app/CLAUDE.md b/app/AGENTS.md similarity index 99% rename from app/CLAUDE.md rename to app/AGENTS.md index 2aa6dc2..a36a98a 100644 --- a/app/CLAUDE.md +++ b/app/AGENTS.md @@ -1,4 +1,4 @@ -# Backend CLAUDE.md +# Backend AGENTS.md This document provides context for AI assistants and developers working on the FastAPI backend. diff --git a/frontend/CLAUDE.md b/frontend/AGENTS.md similarity index 99% rename from frontend/CLAUDE.md rename to frontend/AGENTS.md index cb1ea46..7d1a1b0 100644 --- a/frontend/CLAUDE.md +++ b/frontend/AGENTS.md @@ -1,4 +1,4 @@ -# Frontend CLAUDE.md +# Frontend AGENTS.md This document provides context for AI assistants and developers working on the React frontend. diff --git a/frontend/src/components/AGENTS.md b/frontend/src/components/AGENTS.md new file mode 100644 index 0000000..544b198 --- /dev/null +++ b/frontend/src/components/AGENTS.md @@ -0,0 +1,326 @@ +# PacketVisualizer Architecture + +This document explains the architecture and design of the PacketVisualizer component, which renders a real-time force-directed graph visualization of mesh network packet traffic. + +## Overview + +The PacketVisualizer displays: + +- **Nodes**: Network participants (self, repeaters, clients) +- **Links**: Connections between nodes based on observed packet paths +- **Particles**: Animated dots traveling along links representing packets in transit + +## Architecture: Data Layer vs Rendering Layer + +The component is split into two distinct layers to enable future rendering engine swaps (e.g., WebGL, Three.js): + +### Data Layer (`useVisualizerData` hook) + +The custom hook manages all graph state and simulation logic: + +``` +Packets → Parse → Aggregate by key → Observation window → Publish → Animate +``` + +**Key responsibilities:** + +- Maintains node and link maps (`nodesRef`, `linksRef`) +- Runs D3 force simulation for layout +- Processes incoming packets with deduplication +- Aggregates packet repeats across multiple paths +- Manages particle queue and animation timing + +**State:** + +- `nodesRef`: Map of node ID → GraphNode +- `linksRef`: Map of link key → GraphLink +- `particlesRef`: Array of active Particle objects +- `simulationRef`: D3 force simulation instance +- `pendingRef`: Packets in observation window awaiting animation +- `timersRef`: Per-packet publish timers + +### Rendering Layer (canvas drawing functions) + +Separate pure functions handle all canvas rendering: + +- `renderLinks()`: Draws connections between nodes +- `renderParticles()`: Draws animated packets with labels +- `renderNodes()`: Draws node circles with emojis/text + +The main component orchestrates rendering via `requestAnimationFrame`. + +## Packet Processing Pipeline + +### 1. Packet Arrival + +When a new packet arrives from the WebSocket: + +```typescript +packets.forEach((packet) => { + if (processedRef.current.has(packet.id)) return; // Skip duplicates + processedRef.current.add(packet.id); + + const parsed = parsePacket(packet.data); + const key = generatePacketKey(parsed, packet); + // ... +}); +``` + +### 2. Key Generation + +Packets are grouped by a unique key to aggregate repeats: + +| Packet Type | Key Format | +| -------------- | ----------------------------------------- | +| Advertisement | `ad:{pubkey_prefix_12}` | +| Group Text | `gt:{channel}:{sender}:{content_hash}` | +| Direct Message | `dm:{src_hash}:{dst_hash}:{content_hash}` | +| Other | `other:{data_hash}` | + +### 3. Observation Window + +Same packets arriving via different paths are aggregated: + +```typescript +if (existing && now < existing.expiresAt) { + // Append path to existing entry + existing.paths.push({ nodes: path, snr: packet.snr, timestamp: now }); +} else { + // Create new pending entry with 2-second observation window + pendingPacketsRef.current.set(key, { + key, + label, + paths: [{ nodes: path, ... }], + expiresAt: now + OBSERVATION_WINDOW_MS, + }); +} +``` + +### 4. Publishing & Animation + +When the observation window expires, all paths animate simultaneously: + +```typescript +function publishPacket(pending: PendingPacket) { + // Ensure all nodes exist in graph + // Create links between consecutive nodes + // Queue particles for ALL paths at once + + for (const observedPath of pending.paths) { + for (let i = 0; i < path.length - 1; i++) { + // Spawn particle with negative initial progress for smooth flow + particlesRef.current.push({ + progress: -(i * HOP_DELAY), // Stagger by hop index + // ... + }); + } + } +} +``` + +**Key insight:** Particles start with negative progress. This creates smooth flow through multi-hop paths without pausing at intermediate nodes. + +## D3 Force Simulation + +The layout uses D3's force simulation with these forces: + +| Force | Purpose | +| ------------- | ---------------------------------------------------- | +| `link` | Pulls connected nodes together | +| `charge` | Repels nodes from each other (self node 6x stronger) | +| `center` | Gently pulls graph toward center | +| `collide` | Prevents node overlap | +| `selfX/selfY` | Anchors self node near center | + +### Shuffle Layout + +The "Shuffle layout" button randomizes all node positions (except self, which stays centered) and reheats the simulation to alpha=1. This lets users try different random starting configurations to find a cleaner layout. + +### Continuous Drift + +When "Let 'em drift" is enabled, `alphaTarget(0.05)` keeps the simulation running indefinitely, allowing the graph to continuously reorganize into better layouts. + +## Node Resolution + +Nodes are resolved from various sources: + +```typescript +function resolveNode(source, isRepeater, showAmbiguous): string | null { + // source.type can be: 'pubkey', 'prefix', or 'name' + // Try to find matching contact + // If found: use full 12-char prefix as node ID + // If not found and showAmbiguous: create "?prefix" node + // Otherwise: return null (path terminates) +} +``` + +### Ambiguous Nodes + +When only a 1-byte prefix is known (from packet path bytes), the node is marked ambiguous and shown with a `?` prefix and gray styling. + +### Traffic Pattern Splitting (Experimental) + +**Problem:** Multiple physical repeaters can share the same 1-byte prefix (collision). Since packet paths only contain 1-byte hashes, we can't directly distinguish them. However, traffic patterns provide a heuristic. + +**Key Insight:** If packets from different sources all route through prefix `32` to the same next hop, it's likely the same physical node. But if `32` routes to different next hops depending on the source, those are likely different physical nodes. + +**Example:** + +``` +ae -> 32 -> ba -> self +c1 -> 32 -> ba -> self +d1 -> 32 -> 60 -> self +d1 -> 32 -> 60 -> self +``` + +Here we can deduce: + +- The `32` that routes to `ba` is likely one physical repeater +- The `32` that routes to `60` is likely a DIFFERENT physical repeater + +**Algorithm:** When "Split by traffic pattern" is enabled: + +1. **Intermediate repeaters** (has next hop in path): Node ID includes the next hop suffix + - `?32:>ba` - the `32` that routes to `ba` + - `?32:>60` - the `32` that routes to `60` + +2. **Final repeaters** (no next hop, connects directly to destination): No suffix added + - Stays as simple `?ba`, `?60` etc. + - Rationale: The last repeater before you is clearly a single physical node regardless of where traffic originates + +**Why only NEXT hop matters:** + +- We DON'T key on the previous node because multiple sources going through the same repeater to the same destination = same physical node +- We DO key on next hop because a repeater routing to different destinations suggests prefix collision + +**Node ID format:** + +- Without splitting: `?XX` (e.g., `?32`) +- With splitting (intermediate): `?XX:>YY` (e.g., `?32:>ba`) +- With splitting (final): `?XX` (unchanged, no suffix) + +## Path Building + +Paths are constructed from packet data: + +```typescript +function buildPath(parsed, packet, myPrefix): string[] { + const path = []; + + // 1. Add source node (from advert pubkey, DM src hash, or group text sender) + // 2. Add repeater path (from path bytes in packet header) + // 3. Add destination (self for incoming, or DM dst hash for outgoing) + + return dedupeConsecutive(path); // Remove consecutive duplicates +} +``` + +## Packet Types & Colors + +| Label | Type | Color | +| ----- | -------------- | ---------------- | +| AD | Advertisement | Amber (#f59e0b) | +| GT | Group Text | Cyan (#06b6d4) | +| DM | Direct Message | Purple (#8b5cf6) | +| ACK | Acknowledgment | Green (#22c55e) | +| TR | Trace | Orange (#f97316) | +| RQ | Request | Pink (#ec4899) | +| RS | Response | Teal (#14b8a6) | +| ? | Unknown | Gray (#6b7280) | + +### Sender Extraction by Packet Type + +Different packet types provide different levels of sender identification: + +| Packet Type | Sender Info Available | Resolution | +| -------------- | ------------------------------ | ------------------------------ | +| Advertisement | Full 32-byte public key | Exact contact match | +| AnonRequest | Full 32-byte public key | Exact contact match | +| Group Text | Sender name (after decryption) | Name lookup | +| Direct Message | 1-byte source hash | Ambiguous (may match multiple) | +| Request | 1-byte source hash | Ambiguous | +| Other | None | Path bytes only | + +**AnonRequest packets** are particularly useful because they include the sender's full public key (unlike regular Request packets which only have a 1-byte hash). This allows exact identification of who is making the request. + +## Canvas Rendering + +### Coordinate Transformation + +Pan and zoom are applied via transform matrix: + +```typescript +ctx.setTransform(dpr * scale, 0, 0, dpr * scale, dpr * (x + panX), dpr * (y + panY)); +``` + +### Render Order + +1. Clear canvas with background +2. Draw links (gray lines) +3. Draw particles (colored dots with labels) +4. Draw nodes (circles with emojis) +5. Draw hover tooltip if applicable + +## Configuration Options + +| Option | Default | Description | +| -------------------------- | ------- | --------------------------------------------------------- | +| Ambiguous path repeaters | On | Show nodes when only partial prefix known | +| Ambiguous sender/recipient | Off | Show placeholder nodes for unknown senders | +| Split by traffic pattern | Off | Split ambiguous repeaters by next-hop routing (see above) | +| Hide repeaters >48hrs | Off | Filter out old repeaters | +| Observation window | 15 sec | Wait time for duplicate packets before animating (1-60s) | +| Let 'em drift | On | Continuous layout optimization | +| Repulsion | 200 | Force strength (50-2500) | +| Packet speed | 2x | Particle animation speed multiplier (1x-5x) | +| Shuffle layout | - | Button to randomize node positions and reheat sim | +| Oooh Big Stretch! | - | Button to temporarily increase repulsion then relax | +| Hide UI | Off | Hide legends and most controls for cleaner view | +| Full screen | Off | Hide the packet feed panel (desktop only) | + +## File Structure + +``` +PacketVisualizer.tsx +├── TYPES (GraphNode, GraphLink, Particle, etc.) +├── CONSTANTS (colors, timing, legend items) +├── UTILITY FUNCTIONS +│ ├── simpleHash() +│ ├── parsePacket() +│ ├── getPacketLabel() +│ ├── generatePacketKey() +│ ├── findContactBy*() +│ └── dedupeConsecutive() +├── DATA LAYER HOOK (useVisualizerData) +│ ├── Refs (nodes, links, particles, simulation, pending, timers) +│ ├── Simulation initialization +│ ├── Node/link management (addNode, addLink, syncSimulation) +│ ├── Path building (resolveNode, buildPath) +│ └── Packet processing & publishing +├── RENDERING FUNCTIONS +│ ├── renderLinks() +│ ├── renderParticles() +│ └── renderNodes() +└── MAIN COMPONENT (PacketVisualizer) + ├── State (dimensions, options, transform, hover) + ├── Event handlers (mouse, wheel) + ├── Animation loop + └── JSX (canvas, legend, settings panel) +``` + +## Performance Considerations + +- **Observation window**: 2 seconds balances latency vs. path aggregation +- **Max links**: Capped at 100 to prevent graph explosion +- **Particle culling**: Particles removed when progress > 1 +- **Node filtering**: Old repeaters can be hidden to reduce clutter +- **requestAnimationFrame**: Render loop tied to display refresh rate + +## Future Improvements + +The data/rendering split enables: + +- WebGL rendering for larger graphs +- 3D visualization +- Different layout algorithms +- Export to other formats