Docs updates

This commit is contained in:
Jack Kingsman
2026-03-30 21:24:36 -07:00
parent 44d6fcac24
commit 7b9d8f6a23
5 changed files with 102 additions and 35 deletions

View File

@@ -277,23 +277,23 @@ PYTHONPATH=. uv run pytest tests/ -v
```
Key test files:
- `tests/test_decoder.py` - Channel + direct message decryption, key exchange
- `tests/test_keystore.py` - Ephemeral key store
- `tests/test_event_handlers.py` - ACK tracking, repeat detection
- `tests/test_packet_pipeline.py` - End-to-end packet processing
- `tests/test_api.py` - API endpoints, read state tracking
- `tests/test_migrations.py` - Database migration system
- `tests/test_frontend_static.py` - Frontend static route registration (missing `dist`/`index.html` handling)
- `tests/test_messages_search.py` - Message search, around endpoint, forward pagination
- `tests/test_rx_log_data.py` - on_rx_log_data event handler integration
- `tests/test_ack_tracking_wiring.py` - DM ACK tracking extraction and wiring
- `tests/test_radio_lifecycle_service.py` - Radio reconnect/setup orchestration helpers
- `tests/test_radio_commands_service.py` - Radio config/private-key service workflows
- `tests/test_health_mqtt_status.py` - Health endpoint MQTT status field
- `tests/test_community_mqtt.py` - Community MQTT publisher (JWT, packet format, hash, broadcast)
- `tests/test_radio_sync.py` - Radio sync, periodic tasks, and contact offload back to the radio
- `tests/test_real_crypto.py` - Real cryptographic operations
- `tests/test_disable_bots.py` - MESHCORE_DISABLE_BOTS=true feature
- `tests/test_api.py` - Broad API integration coverage across routers and read-state flows
- `tests/test_packet_pipeline.py` - End-to-end packet processing, decrypt, dedup, and message creation
- `tests/test_event_handlers.py` - ACK tracking, fallback DM handling, and event subscription cleanup
- `tests/test_send_messages.py` - Outgoing DM/channel send workflows, retries, and bot-trigger wiring
- `tests/test_packets_router.py` - Historical decrypt, maintenance, and raw-packet detail endpoints
- `tests/test_repeater_routes.py` - Repeater command/telemetry/trace pane endpoints
- `tests/test_room_routes.py` - Room-server login/status/ACL/telemetry endpoints
- `tests/test_radio_router.py` - Radio config, advert, discovery, trace, and reconnect endpoints
- `tests/test_radio_sync.py` - Radio sync, periodic tasks, contact offload/reload, and pending-message flushes
- `tests/test_fanout.py` - Fanout config CRUD, scope matching, and manager dispatch
- `tests/test_fanout_integration.py` - Integration-module lifecycle and delivery behavior
- `tests/test_statistics.py` - Aggregated mesh/network statistics and noise-floor snapshots
- `tests/test_version_info.py` - Version/build metadata resolution
- `tests/test_websocket.py` - WS manager broadcast and cleanup behavior
- `tests/test_frontend_static.py` - Frontend static route registration and fallback behavior
For the fuller backend inventory, see `app/AGENTS.md`. For frontend-specific suites, see `frontend/AGENTS.md`.
### Frontend (Vitest)
@@ -319,6 +319,7 @@ All endpoints are prefixed with `/api` (e.g., `/api/health`).
| PUT | `/api/radio/private-key` | Import private key to radio |
| POST | `/api/radio/advertise` | Send advertisement (`mode`: `flood` or `zero_hop`, default `flood`) |
| POST | `/api/radio/discover` | Run a short mesh discovery sweep for nearby repeaters/sensors |
| POST | `/api/radio/trace` | Send a multi-hop trace loop through known repeaters and back to the local radio |
| POST | `/api/radio/reboot` | Reboot radio or reconnect if disconnected |
| POST | `/api/radio/disconnect` | Disconnect from radio and pause automatic reconnect attempts |
| POST | `/api/radio/reconnect` | Manual radio reconnection |
@@ -341,6 +342,10 @@ All endpoints are prefixed with `/api` (e.g., `/api/health`).
| POST | `/api/contacts/{public_key}/repeater/radio-settings` | Fetch repeater radio config via CLI |
| POST | `/api/contacts/{public_key}/repeater/advert-intervals` | Fetch advert intervals |
| POST | `/api/contacts/{public_key}/repeater/owner-info` | Fetch owner info |
| POST | `/api/contacts/{public_key}/room/login` | Log in to a room server |
| POST | `/api/contacts/{public_key}/room/status` | Fetch room-server status telemetry |
| POST | `/api/contacts/{public_key}/room/lpp-telemetry` | Fetch room-server CayenneLPP sensor data |
| POST | `/api/contacts/{public_key}/room/acl` | Fetch room-server ACL entries |
| GET | `/api/channels` | List channels |
| GET | `/api/channels/{key}/detail` | Comprehensive channel profile (message stats, top senders) |
@@ -354,6 +359,7 @@ All endpoints are prefixed with `/api` (e.g., `/api/health`).
| POST | `/api/messages/channel` | Send channel message |
| POST | `/api/messages/channel/{message_id}/resend` | Resend channel message (default: byte-perfect within 30s; `?new_timestamp=true`: fresh timestamp, no time limit, creates new message row) |
| GET | `/api/packets/undecrypted/count` | Count of undecrypted packets |
| GET | `/api/packets/{packet_id}` | Fetch one stored raw packet by row ID for on-demand inspection |
| POST | `/api/packets/decrypt/historical` | Decrypt stored packets |
| POST | `/api/packets/maintenance` | Delete old packets and vacuum |
| GET | `/api/read-state/unreads` | Server-computed unread counts, mentions, last message times, and `last_read_ats` boundaries |
@@ -368,6 +374,7 @@ All endpoints are prefixed with `/api` (e.g., `/api/health`).
| POST | `/api/fanout` | Create new fanout config |
| PATCH | `/api/fanout/{id}` | Update fanout config (triggers module reload) |
| DELETE | `/api/fanout/{id}` | Delete fanout config (stops module) |
| POST | `/api/fanout/bots/disable-until-restart` | Stop bot fanout modules and keep bots disabled until the process restarts |
| GET | `/api/statistics` | Aggregated mesh network statistics |
| WS | `/api/ws` | Real-time updates |

View File

@@ -78,6 +78,7 @@ These tests are only guaranteed to run correctly in a narrow subset of environme
```bash
cd tests/e2e
npm install
npx playwright test # headless
npx playwright test --headed # you can probably guess
```

View File

@@ -25,18 +25,22 @@ Keep it aligned with `app/` source files and router behavior.
app/
├── main.py # App startup/lifespan, router registration, static frontend mounting
├── config.py # Env-driven runtime settings
├── channel_constants.py # Public/default channel constants shared across sync/send logic
├── database.py # SQLite connection + base schema + migration runner
├── migrations.py # Schema migrations (SQLite user_version)
├── models.py # Pydantic request/response models and typed write contracts (for example ContactUpsert)
├── version_info.py # Unified version/build metadata resolution for debug + startup surfaces
├── repository/ # Data access layer (contacts, channels, messages, raw_packets, settings, fanout)
├── services/ # Shared orchestration/domain services
│ ├── messages.py # Shared message creation, dedup, ACK application
│ ├── message_send.py # Direct send, channel send, resend workflows
│ ├── dm_ingest.py # Shared direct-message ingest / dedup seam for packet + fallback paths
│ ├── dm_ack_apply.py # Shared DM ACK application over pending/buffered ACK state
│ ├── dm_ack_tracker.py # Pending DM ACK state
│ ├── contact_reconciliation.py # Prefix-claim, sender-key backfill, name-history wiring
│ ├── radio_lifecycle.py # Post-connect setup and reconnect/setup helpers
│ ├── radio_commands.py # Radio config/private-key command workflows
│ ├── radio_noise_floor.py # In-memory local radio noise-floor sampling/history
│ └── radio_runtime.py # Router/dependency seam over the global RadioManager
├── radio.py # RadioManager transport/session state + lock management
├── radio_sync.py # Polling, sync, periodic advertisement loop
@@ -61,6 +65,8 @@ app/
├── messages.py
├── packets.py
├── read_state.py
├── rooms.py
├── server_control.py
├── settings.py
├── fanout.py
├── repeaters.py
@@ -174,6 +180,7 @@ app/
- `PUT /radio/private-key`
- `POST /radio/advertise` — manual advert send; request body may set `mode` to `flood` or `zero_hop` (defaults to `flood`)
- `POST /radio/discover` — short mesh discovery sweep for nearby repeaters/sensors
- `POST /radio/trace` — send a multi-hop trace loop through known repeaters and back to the local radio
- `POST /radio/disconnect`
- `POST /radio/reboot`
- `POST /radio/reconnect`
@@ -198,6 +205,10 @@ app/
- `POST /contacts/{public_key}/repeater/radio-settings`
- `POST /contacts/{public_key}/repeater/advert-intervals`
- `POST /contacts/{public_key}/repeater/owner-info`
- `POST /contacts/{public_key}/room/login`
- `POST /contacts/{public_key}/room/status`
- `POST /contacts/{public_key}/room/lpp-telemetry`
- `POST /contacts/{public_key}/room/acl`
### Channels
- `GET /channels`
@@ -216,6 +227,7 @@ app/
### Packets
- `GET /packets/undecrypted/count`
- `GET /packets/{packet_id}` — fetch one stored raw packet by row ID for on-demand inspection
- `POST /packets/decrypt/historical`
- `POST /packets/maintenance`
@@ -236,6 +248,7 @@ app/
- `POST /fanout` — create new fanout config
- `PATCH /fanout/{id}` — update fanout config (triggers module reload)
- `DELETE /fanout/{id}` — delete fanout config (stops module)
- `POST /fanout/bots/disable-until-restart` — stop bot modules and keep bots disabled until restart
### Statistics
- `GET /statistics` — aggregated mesh network stats (entity counts, message/packet splits, activity windows, busiest channels)
@@ -322,9 +335,11 @@ tests/
├── conftest.py # Shared fixtures
├── test_ack_tracking_wiring.py # DM ACK tracking extraction and wiring
├── test_api.py # REST endpoint integration tests
├── test_block_lists.py # Blocked keys/names filtering across list/search surfaces
├── test_bot.py # Bot execution and sandboxing
├── test_channels_router.py # Channels router endpoints
├── test_channel_sender_backfill.py # Sender-key backfill uniqueness rules for channel messages
├── test_channels_router.py # Channels router endpoints
├── test_community_mqtt.py # Community MQTT publisher (JWT, packet format, hash, broadcast)
├── test_config.py # Configuration validation
├── test_contact_reconciliation_service.py # Prefix/contact reconciliation service helpers
├── test_contacts_router.py # Contacts router endpoints
@@ -332,40 +347,41 @@ tests/
├── test_disable_bots.py # MESHCORE_DISABLE_BOTS=true feature
├── test_echo_dedup.py # Echo/repeat deduplication (incl. concurrent)
├── test_fanout.py # Fanout bus CRUD, scope matching, manager dispatch
├── test_fanout_integration.py # Fanout integration tests
├── test_fanout_hitlist.py # Fanout-related hitlist regression tests
├── test_fanout_integration.py # Fanout integration tests
├── test_event_handlers.py # ACK tracking, event registration, cleanup
├── test_frontend_static.py # Frontend static file serving
├── test_health_mqtt_status.py # Health endpoint MQTT status field
├── test_http_quality.py # Cache-control / gzip / basic-auth HTTP quality checks
├── test_key_normalization.py # Public key normalization
├── test_keystore.py # Ephemeral keystore
├── test_main_startup.py # App startup and lifespan
├── test_map_upload.py # Map upload fanout module
├── test_message_pagination.py # Cursor-based message pagination
├── test_message_prefix_claim.py # Message prefix claim logic
├── test_migrations.py # Schema migration system
├── test_community_mqtt.py # Community MQTT publisher (JWT, packet format, hash, broadcast)
├── test_mqtt.py # MQTT publisher topic routing and lifecycle
├── test_messages_search.py # Message search, around, forward pagination
├── test_migrations.py # Schema migration system
├── test_packet_pipeline.py # End-to-end packet processing
├── test_packets_router.py # Packets router endpoints (decrypt, maintenance)
├── test_path_utils.py # Path hex rendering helpers
├── test_radio.py # RadioManager, serial detection
├── test_radio_commands_service.py # Radio config/private-key service workflows
├── test_radio_lifecycle_service.py # Reconnect/setup orchestration helpers
├── test_radio_runtime_service.py # radio_runtime seam behavior and helpers
├── test_real_crypto.py # Real cryptographic operations
├── test_radio_operation.py # radio_operation() context manager
├── test_radio_router.py # Radio router endpoints
├── test_radio_runtime_service.py # radio_runtime seam behavior and helpers
├── test_radio_sync.py # Polling, sync, advertisement
├── test_real_crypto.py # Real cryptographic operations
├── test_repeater_routes.py # Repeater command/telemetry/trace + granular pane endpoints
├── test_repository.py # Data access layer
├── test_room_routes.py # Room-server login/status/telemetry/ACL endpoints
├── test_rx_log_data.py # on_rx_log_data event handler integration
├── test_messages_search.py # Message search, around, forward pagination
├── test_block_lists.py # Blocked keys/names filtering
├── test_security.py # Optional Basic Auth middleware / config behavior
├── test_send_messages.py # Outgoing messages, bot triggers, concurrent sends
├── test_settings_router.py # Settings endpoints, advert validation
├── test_statistics.py # Statistics aggregation
├── test_main_startup.py # App startup and lifespan
├── test_path_utils.py # Path hex rendering helpers
├── test_version_info.py # Version/build metadata resolution
├── test_websocket.py # WS manager broadcast/cleanup
└── test_websocket_route.py # WS endpoint lifecycle
```

View File

@@ -264,9 +264,10 @@ async def process_raw_packet(
This is the main entry point for all incoming RF packets.
Note: Packets are deduplicated by payload hash in the database. If we receive
a duplicate packet (same payload, different path), we still broadcast it to
the frontend (for the real-time packet feed) but skip decryption processing
since the original packet was already processed.
a duplicate payload (same payload, different path), we still broadcast it to
the frontend for realtime packet-feed fidelity. Some payload types are also
intentionally reprocessed on duplicate arrival so message-level dedup/path
merge logic and advert/path-history tracking still see each observation.
"""
ts = timestamp or int(time.time())
observation_id = next(_raw_observation_counter)

View File

@@ -39,6 +39,8 @@ frontend/src/
├── index.css # Global styles/utilities
├── styles.css # Additional global app styles
├── themes.css # Color theme definitions
├── contexts/
│ └── DistanceUnitContext.tsx # Browser-local distance-unit context/provider
├── lib/
│ └── utils.ts # cn() — clsx + tailwind-merge helper
├── hooks/
@@ -53,10 +55,14 @@ frontend/src/
│ ├── useRadioControl.ts # Radio health/config state, reconnection, mesh discovery sweeps
│ ├── useAppSettings.ts # Settings, favorites, preferences migration
│ ├── useConversationRouter.ts # URL hash → active conversation routing
── useContactsAndChannels.ts # Contact/channel loading, creation, deletion
── useContactsAndChannels.ts # Contact/channel loading, creation, deletion
│ ├── useBrowserNotifications.ts # Per-conversation browser notification preferences + dispatch
│ ├── useFaviconBadge.ts # Browser tab unread badge state
│ ├── useRawPacketStatsSession.ts # Session-scoped packet-feed stats history
│ └── useRememberedServerPassword.ts # Browser-local repeater/room password persistence
├── components/
│ ├── AppShell.tsx # App-shell layout: status, sidebar, search/settings panes, cracker, modals
│ ├── ConversationPane.tsx # Active conversation surface selection (map/raw/repeater/chat/empty)
│ ├── AppShell.tsx # App-shell layout: status, sidebar, search/settings panes, cracker, modals, security warning
│ ├── ConversationPane.tsx # Active conversation surface selection (map/raw/trace/repeater/room/chat/empty)
│ ├── visualizer/
│ │ ├── useVisualizerData3D.ts # Packet→graph data pipeline, repeat aggregation, simulation state
│ │ ├── useVisualizer3DScene.ts # Three.js scene lifecycle, buffers, hover/pin interaction
@@ -73,14 +79,17 @@ frontend/src/
│ ├── pubkey.ts # getContactDisplayName (12-char prefix fallback)
│ ├── contactAvatar.ts # Avatar color derivation from public key
│ ├── rawPacketIdentity.ts # observation_id vs id dedup helpers
│ ├── rawPacketStats.ts # Session packet stats windows, rankings, and coverage helpers
│ ├── regionScope.ts # Regional flood-scope label/normalization helpers
│ ├── visualizerUtils.ts # 3D visualizer node types, colors, particles
│ ├── visualizerSettings.ts # LocalStorage persistence for visualizer options
│ ├── a11y.ts # Keyboard accessibility helper
│ ├── distanceUnits.ts # Browser-local distance unit persistence/helpers
│ ├── lastViewedConversation.ts # localStorage for last-viewed conversation
│ ├── contactMerge.ts # Merge WS contact updates into list
│ ├── localLabel.ts # Local label (text + color) in localStorage
│ ├── radioPresets.ts # LoRa radio preset configurations
│ ├── publicChannel.ts # Public-channel resolution helpers for routing/hash defaults
│ ├── fontScale.ts # Browser-local relative font scale persistence/application
│ └── theme.ts # Theme switching helpers
├── components/
@@ -92,8 +101,12 @@ frontend/src/
│ ├── NewMessageModal.tsx
│ ├── SearchView.tsx # Full-text message search pane
│ ├── SettingsModal.tsx # Layout shell — delegates to settings/ sections
│ ├── SecurityWarningModal.tsx # Startup warning for trusted-network / bot execution posture
│ ├── RawPacketList.tsx
│ ├── RawPacketFeedView.tsx # Live raw packet feed + session stats drawer
│ ├── RawPacketDetailModal.tsx # On-demand packet inspector dialog
│ ├── MapView.tsx
│ ├── TracePane.tsx # Multi-hop route trace builder/results view
│ ├── VisualizerView.tsx
│ ├── PacketVisualizer3D.tsx
│ ├── PathModal.tsx
@@ -103,9 +116,14 @@ frontend/src/
│ ├── ContactAvatar.tsx
│ ├── ContactInfoPane.tsx # Contact detail sheet (stats, name history, paths)
│ ├── ContactStatusInfo.tsx # Contact status info component
│ ├── ContactPathDiscoveryModal.tsx # Forward/return path discovery dialog
│ ├── ContactRoutingOverrideModal.tsx # Manual direct-route override editor
│ ├── RepeaterDashboard.tsx # Layout shell — delegates to repeater/ panes
│ ├── RepeaterLogin.tsx # Repeater login form (password + guest)
│ ├── RoomServerPanel.tsx # Room-server auth gate + status banner ahead of room chat
│ ├── ServerLoginStatusBanner.tsx # Shared repeater/room login state banner
│ ├── ChannelInfoPane.tsx # Channel detail sheet (stats, top senders)
│ ├── ChannelFloodScopeOverrideModal.tsx # Per-channel flood-scope override editor
│ ├── DirectTraceIcon.tsx # Shared direct-trace glyph used in header/dashboard
│ ├── NeighborsMiniMap.tsx # Leaflet mini-map for repeater neighbor locations
│ ├── settings/
@@ -131,12 +149,13 @@ frontend/src/
│ └── ui/ # shadcn/ui primitives
├── types/
│ └── d3-force-3d.d.ts # Type declarations for d3-force-3d
└── test/
└── test/ # Representative frontend test suites (not an exhaustive listing)
├── setup.ts
├── fixtures/websocket_events.json
├── api.test.ts
├── appFavorites.test.tsx
├── appStartupHash.test.tsx
├── conversationPane.test.tsx
├── contactAvatar.test.ts
├── contactInfoPane.test.tsx
├── integration.test.ts
@@ -147,18 +166,23 @@ frontend/src/
├── rawPacketList.test.tsx
├── pathUtils.test.ts
├── prefetch.test.ts
├── rawPacketDetailModal.test.tsx
├── rawPacketFeedView.test.tsx
├── radioPresets.test.ts
├── rawPacketIdentity.test.ts
├── repeaterDashboard.test.tsx
├── repeaterFormatters.test.ts
├── repeaterLogin.test.tsx
├── repeaterMessageParsing.test.ts
├── roomServerPanel.test.tsx
├── securityWarningModal.test.tsx
├── localLabel.test.ts
├── messageInput.test.tsx
├── newMessageModal.test.tsx
├── settingsModal.test.tsx
├── sidebar.test.tsx
├── statusBar.test.tsx
├── tracePane.test.tsx
├── unreadCounts.test.ts
├── urlHash.test.ts
├── appSearchJump.test.tsx
@@ -170,12 +194,17 @@ frontend/src/
├── useConversationMessages.race.test.ts
├── useConversationNavigation.test.ts
├── useAppShell.test.ts
├── useBrowserNotifications.test.ts
├── useFaviconBadge.test.ts
├── useRepeaterDashboard.test.ts
├── useRememberedServerPassword.test.ts
├── useContactsAndChannels.test.ts
├── useRealtimeAppState.test.ts
├── useUnreadCounts.test.ts
├── useWebSocket.dispatch.test.ts
├── useWebSocket.lifecycle.test.ts
├── rawPacketStats.test.ts
├── fontScale.test.ts
└── wsEvents.test.ts
```
@@ -191,6 +220,7 @@ frontend/src/
- search/settings surface switching
- global cracker mount/focus behavior
- new-message modal and info panes
- trusted-network `SecurityWarningModal`
High-level state is delegated to hooks:
- `useAppShell`: app-shell view state (settings section, sidebar, cracker, new-message modal)
@@ -212,7 +242,9 @@ High-level state is delegated to hooks:
- map view
- visualizer
- raw packet feed
- trace view
- repeater dashboard
- room-server auth/status gate before room chat
- normal chat chrome (`ChatHeader` + `MessageList` + `MessageInput`)
### Initial load + realtime
@@ -273,12 +305,16 @@ Supported routes:
- `#map/focus/{pubkey_or_prefix}`
- `#visualizer`
- `#search`
- `#trace`
- `#settings/{section}`
- `#channel/{channelKey}`
- `#channel/{channelKey}/{label}`
- `#contact/{publicKey}`
- `#contact/{publicKey}/{label}`
Legacy name-based hashes are still accepted for compatibility.
Where `{section}` is one of `radio`, `local`, `fanout`, `database`, `statistics`, or `about`.
Legacy name-based channel/contact hashes are still accepted for compatibility.
## Conversation State Keys (`utils/conversationState.ts`)
@@ -378,6 +414,12 @@ For repeater contacts (`type=2`), `ConversationPane.tsx` renders `RepeaterDashbo
All state is managed by `useRepeaterDashboard` hook. State resets on conversation change.
## Room Server Panel
For room contacts (`type=3`), `ConversationPane.tsx` keeps the normal chat surface but inserts `RoomServerPanel` above it. That panel handles room-server login/status messaging and gates room chat behind the room-authenticated state when required.
`ServerLoginStatusBanner` is shared between repeater and room login surfaces for inline status/error display.
## Message Search Pane
The `SearchView` component (`components/SearchView.tsx`) provides full-text search across all DMs and channel messages. Key behaviors: