Remote-Terminal-for-MeshCore/app/AGENTS.md

Backend AGENTS.md

This document is the backend working guide for agents and developers. Keep it aligned with app/ source files and router behavior.

Stack

• FastAPI
• aiosqlite
• Pydantic
• MeshCore Python library (references/meshcore_py)
• PyCryptodome

Backend Map

app/
├── main.py              # App startup/lifespan, router registration, static frontend mounting
├── config.py            # Env-driven runtime settings
├── database.py          # SQLite connection + base schema + migration runner
├── migrations.py        # Schema migrations (SQLite user_version)
├── models.py            # Pydantic request/response models
├── repository.py        # Data access layer
├── radio.py             # RadioManager + auto-reconnect monitor
├── radio_sync.py        # Polling, sync, periodic advertisement loop
├── decoder.py           # Packet parsing/decryption
├── packet_processor.py  # Raw packet pipeline, dedup, path handling
├── event_handlers.py    # MeshCore event subscriptions and ACK tracking
├── websocket.py         # WS manager + broadcast helpers
├── bot.py               # Bot execution and outbound bot sends
├── dependencies.py      # Shared FastAPI dependency providers
├── keystore.py          # Ephemeral private/public key storage for DM decryption
├── frontend_static.py   # Mount/serve built frontend (production)
└── routers/
    ├── health.py
    ├── radio.py
    ├── contacts.py
    ├── channels.py
    ├── messages.py
    ├── packets.py
    ├── read_state.py
    ├── settings.py
    ├── statistics.py
    └── ws.py

Core Runtime Flows

Incoming data

1. Radio emits events.
2. on_rx_log_data stores raw packet and tries decrypt/pipeline handling.
3. Decrypted messages are inserted into messages and broadcast over WS.
4. CONTACT_MSG_RECV is a fallback DM path when packet pipeline cannot decrypt.

Outgoing messages

1. Send endpoints in routers/messages.py call MeshCore commands.
2. Message is persisted as outgoing.
3. Endpoint broadcasts WS message event so all live clients update.
4. ACK/repeat updates arrive later as message_acked events.
5. Channel resend (POST /messages/channel/{id}/resend) strips the sender name prefix by exact match against the current radio name. This assumes the radio name hasn't changed between the original send and the resend — a safe assumption since name changes require a radio config update and are not something that happens mid-conversation.

Connection lifecycle

• RadioManager.start_connection_monitor() checks health every 5s.
• Monitor reconnect path runs post_connect_setup() before broadcasting healthy state.
• Manual reconnect/reboot endpoints call reconnect() then post_connect_setup().
• Setup includes handler registration, key export, time sync, contact/channel sync, polling/advert tasks.

Important Behaviors

Read/unread state

• Server is source of truth (contacts.last_read_at, channels.last_read_at).
• GET /api/read-state/unreads returns counts, mention flags, and last_message_times.

Echo/repeat dedup

• Message uniqueness: (type, conversation_key, text, sender_timestamp).
• Duplicate insert is treated as an echo/repeat; ACK count/path list is updated.

Raw packet dedup policy

• Raw packet storage deduplicates by payload hash (RawPacketRepository.create), excluding routing/path bytes.
• Stored packet id is therefore a payload identity, not a per-arrival identity.
• Realtime raw-packet WS broadcasts include observation_id (unique per RF arrival) in addition to id.
• Frontend packet-feed features should key/dedupe by observation_id; use id only as the storage reference.
• Message-layer repeat handling (_handle_duplicate_message + MessageRepository.add_path) is separate from raw-packet storage dedup.

Contact sync throttle

• sync_recent_contacts_to_radio() sets _last_contact_sync = now before the sync completes.
• This is intentional: if sync fails, the next attempt is still throttled to prevent a retry-storm against a flaky radio. Contacts will resync on the next scheduled cycle or on reconnect.

Periodic advertisement

• Controlled by app_settings.advert_interval (seconds).
• 0 means disabled.
• Last send time tracked in app_settings.last_advert_time.

API Surface (all under /api)

Health

• GET /health

Radio

• GET /radio/config
• PATCH /radio/config
• PUT /radio/private-key
• POST /radio/advertise
• POST /radio/reboot
• POST /radio/reconnect

Contacts

• GET /contacts
• GET /contacts/{public_key}
• POST /contacts
• DELETE /contacts/{public_key}
• POST /contacts/sync
• POST /contacts/{public_key}/add-to-radio
• POST /contacts/{public_key}/remove-from-radio
• POST /contacts/{public_key}/mark-read
• POST /contacts/{public_key}/telemetry
• POST /contacts/{public_key}/command
• POST /contacts/{public_key}/trace

Channels

• GET /channels
• GET /channels/{key}
• POST /channels
• DELETE /channels/{key}
• POST /channels/sync
• POST /channels/{key}/mark-read

Messages

• GET /messages
• POST /messages/direct
• POST /messages/channel
• POST /messages/channel/{message_id}/resend

Packets

• GET /packets/undecrypted/count
• POST /packets/decrypt/historical
• POST /packets/maintenance

Read state

• GET /read-state/unreads
• POST /read-state/mark-all-read

Settings

• GET /settings
• PATCH /settings
• POST /settings/favorites/toggle
• POST /settings/migrate

Statistics

• GET /statistics — aggregated mesh network stats (entity counts, message/packet splits, activity windows, busiest channels)

WebSocket

• WS /ws

WebSocket Events

• health — radio connection status (broadcast on change, personal on connect)
• contact — single contact upsert (from advertisements and radio sync)
• message — new message (channel or DM, from packet processor or send endpoints)
• message_acked — ACK/echo update for existing message (ack count + paths)
• raw_packet — every incoming RF packet (for real-time packet feed UI)
• error — toast notification (reconnect failure, missing private key, etc.)
• success — toast notification (historical decrypt complete, etc.)

Initial WS connect sends health only. Contacts/channels are loaded by REST. Client sends "ping" text; server replies {"type":"pong"}.

Data Model Notes

Main tables:

• contacts
• channels
• messages
• raw_packets
• app_settings

app_settings fields in active model:

• max_radio_contacts
• favorites
• auto_decrypt_dm_on_advert
• sidebar_sort_order
• last_message_times
• preferences_migrated
• advert_interval
• last_advert_time
• bots

Security Posture (intentional)

• No authn/authz.
• No CORS restriction (*).
• Bot code executes user-provided Python via exec().

These are product decisions for trusted-network deployments; do not flag as accidental vulnerabilities.

Testing

Run backend tests:

PYTHONPATH=. uv run pytest tests/ -v

Test suites:

tests/
├── conftest.py                 # Shared fixtures
├── test_api.py                 # REST endpoint integration tests
├── test_bot.py                 # Bot execution and sandboxing
├── test_config.py              # Configuration validation
├── test_contacts_router.py     # Contacts router endpoints
├── test_decoder.py             # Packet parsing/decryption
├── test_echo_dedup.py          # Echo/repeat deduplication (incl. concurrent)
├── test_event_handlers.py      # ACK tracking, event registration, cleanup
├── test_frontend_static.py     # Frontend static file serving
├── test_key_normalization.py   # Public key normalization
├── test_keystore.py            # Ephemeral keystore
├── test_message_pagination.py  # Cursor-based message pagination
├── test_message_prefix_claim.py # Message prefix claim logic
├── test_migrations.py          # Schema migration system
├── test_packet_pipeline.py     # End-to-end packet processing
├── test_radio.py               # RadioManager, serial detection
├── test_radio_operation.py     # radio_operation() context manager
├── test_radio_router.py        # Radio router endpoints
├── test_radio_sync.py          # Polling, sync, advertisement
├── test_repeater_routes.py     # Repeater command/telemetry/trace
├── test_repository.py          # Data access layer
├── test_send_messages.py       # Outgoing messages, bot triggers, concurrent sends
├── test_settings_router.py     # Settings endpoints, advert validation
├── test_statistics.py          # Statistics aggregation
├── test_websocket.py           # WS manager broadcast/cleanup
└── test_websocket_route.py     # WS endpoint lifecycle

Editing Checklist

When changing backend behavior:

1. Update/add router and repository tests.
2. Confirm WS event contracts when payload shape changes.
3. Run PYTHONPATH=. uv run pytest tests/ -v.
4. If API contract changed, update frontend types and AGENTS docs.