diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 256d09a..fce4ce7 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -70,17 +70,111 @@ npm run test:run npm run build ``` +## Quality + Publishing Scripts + +
+scripts/quality/ + +| Script | Purpose | +|--------|---------| +| `all_quality.sh` | Repo-standard gate: autofix (ruff, eslint, prettier), then pyright, pytest, vitest, and frontend build. Run before finishing any code change. | +| `extended_quality.sh` | `all_quality.sh` plus e2e tests and Docker build matrix. Used for release validation. | +| `e2e.sh` | Thin wrapper that runs Playwright e2e tests from `tests/e2e/`. | +| `docker_ci.sh` | Builds the Docker image and runs a smoke test against it. | +| `test_aur_package.sh` | Builds the AUR package in an Arch container, then installs and boots it in a second container with port 8000 exposed (hang finish). | +| `run_aur_with_radio.sh` | Like `test_aur_package.sh` but passes through the host serial device for testing with a real radio (hang finish). | + +
+ +
+scripts/build/ + +| Script | Purpose | +|--------|---------| +| `publish.sh` | Full release ceremony: quality gate, version bump, changelog, frontend build, Docker multi-arch push, GitHub release. | +| `release_common.sh` | Shared shell helpers (version validation, formatting) sourced by other build scripts. | +| `package_release_artifact.sh` | Builds the prebuilt-frontend release zip attached to GitHub releases. | +| `push_docker_multiarch.sh` | Builds and pushes multi-arch Docker images (amd64 + arm64). | +| `create_github_release.sh` | Creates a GitHub release with changelog notes and the release artifact. | +| `extract_release_notes.sh` | Extracts the latest version's notes from `CHANGELOG.md` for the release body. | +| `collect_licenses.sh` | Gathers third-party license attributions into `LICENSES.md`. | +| `print_frontend_licenses.cjs` | Helper that extracts frontend npm dependency licenses. | +| `dump_api_specs.py` | Dumps the OpenAPI spec from the running backend (developer utility). | + +
+ ## E2E Testing -E2E coverage exists, but it is intentionally not part of the normal development path. +E2E tests exercise the full stack (backend + frontend + real radio hardware) via Playwright. -These tests are only guaranteed to run correctly in a narrow subset of environments; they require a busy mesh with messages arriving constantly, an available autodetect-able radio, and a contact in the test database (which you can provide in `tests/e2e/.tmp/e2e-test.db` after an initial run). E2E tests are generally not necessary to run for normal development work. +> [!WARNING] +> E2E tests are **not part of the normal development path** — most contributors will never need to run them. They exist to catch integration issues that unit tests can't and generally only need to be run by maintainers. + +### Hardware requirements + +- A MeshCore radio connected via serial (auto-detected, or set `MESHCORE_SERIAL_PORT`) +- The radio must be powered on and past its startup sequence before tests begin + +### Running ```bash cd tests/e2e npm install -npx playwright test # headless -npx playwright test --headed # you can probably guess +npx playwright install chromium # first time only +npx playwright test # headless +npx playwright test --headed # watch it run +``` + +The test harness starts its own uvicorn instance on port 8001 with a fresh temporary database. Your development server (port 8000) is unaffected. + +### Test tiers + +**Most tests (22 of 28) are fully self-contained.** They seed their own data via API calls or direct DB writes and need only a connected radio. These cover messaging, pagination, search, favorites, settings, fanout integrations, historical decryption, and all UI-only views. + +**Mesh-traffic tests (tagged `@mesh-traffic`)** wait up to 3 minutes for an incoming message from another node on the network. If no traffic arrives, they fail with an advisory that the failure may be RF conditions, not a bug. These are: `incoming-message` and `packet-feed` (second test only). + +**The partner-radio DM ACK test (tagged `@partner-radio`)** validates direct-route learning by sending a DM and waiting for an ACK. It requires a second radio in range that has your test radio in its contacts. Configure the partner node's public key and name via `E2E_PARTNER_RADIO_PUBKEY` and `E2E_PARTNER_RADIO_NAME`. + +### Making mesh-traffic tests reliable: the echo bot + +The most practical way to guarantee incoming traffic is to run an **echo bot on a second radio** monitoring a known channel. When the test suite starts a `@mesh-traffic` test, it sends a trigger message to that channel. If a bot on another radio is listening, it replies — generating the incoming RF packet the test needs within seconds instead of waiting for organic mesh traffic. + +The test suite sends `!echo please give incoming message` to the echo channel (default `#flightless`) at the start of each `@mesh-traffic` test. The trigger message is configurable via `E2E_ECHO_TRIGGER_MESSAGE`. + +Setup: +1. Set up a second MeshCore radio within RF range of your test radio +2. Run a RemoteTerm instance on the second radio +3. Configure a bot on the second radio that monitors the echo channel and replies when it sees the trigger. Example bot code: + ```python + def bot(sender_name, sender_key, message_text, is_dm, + channel_key, channel_name, sender_timestamp, path): + if "!echo" in message_text.lower(): + return f"[ECHO] {message_text}" + return None + ``` +4. The test suite calls `nudgeEchoBot()` automatically — no manual intervention needed + +Without the echo bot, `@mesh-traffic` tests rely on organic traffic from other nodes. In a quiet RF environment they will time out. + +### Environment variables + +All E2E environment configuration is centralized in `tests/e2e/helpers/env.ts` with defaults that work for the maintainer's test rig. Override via environment variables: + +| Variable | Default | Purpose | +|----------|---------|---------| +| `MESHCORE_SERIAL_PORT` | auto-detect | Serial port for the test radio | +| `E2E_ECHO_CHANNEL` | `#flightless` | Channel the echo bot monitors for traffic generation | +| `E2E_ECHO_TRIGGER_MESSAGE` | `!echo please give incoming message` | Message sent to nudge the echo bot | +| `E2E_PARTNER_RADIO_PUBKEY` | *(maintainer's test node)* | 64-char hex public key of a node that will ACK DMs from your radio | +| `E2E_PARTNER_RADIO_NAME` | *(maintainer's test node)* | Display name of that node (used in UI assertions) | + +Example for a contributor with their own two-radio setup: + +```bash +E2E_ECHO_CHANNEL="#mytest" \ +E2E_PARTNER_RADIO_PUBKEY="abcd1234...full64charhexkey..." \ +E2E_PARTNER_RADIO_NAME="MyTestNode" \ +npx playwright test ``` ## Pull Request Expectations diff --git a/scripts/quality/run_aur_with_radio.sh b/scripts/quality/run_aur_with_radio.sh old mode 100644 new mode 100755 diff --git a/tests/e2e/helpers/api.ts b/tests/e2e/helpers/api.ts index fe0b10e..bf2a061 100644 --- a/tests/e2e/helpers/api.ts +++ b/tests/e2e/helpers/api.ts @@ -282,14 +282,19 @@ export function deleteFanoutConfig(id: string): Promise<{ deleted: boolean }> { // --- Helpers --- /** - * Ensure #flightless channel exists, creating it if needed. + * Ensure a channel exists by name, creating it if needed. * Returns the channel object. */ -export async function ensureFlightlessChannel(): Promise { +export async function ensureChannel(name: string): Promise { const channels = await getChannels(); - const existing = channels.find((c) => c.name === '#flightless'); + const existing = channels.find((c) => c.name === name); if (existing) return existing; - return createChannel('#flightless'); + return createChannel(name); +} + +/** Convenience alias — ensures #flightless exists. */ +export async function ensureFlightlessChannel(): Promise { + return ensureChannel('#flightless'); } /** diff --git a/tests/e2e/helpers/env.ts b/tests/e2e/helpers/env.ts new file mode 100644 index 0000000..f91c23f --- /dev/null +++ b/tests/e2e/helpers/env.ts @@ -0,0 +1,46 @@ +/** + * Centralized E2E environment configuration. + * + * All environment-dependent values live here with sensible defaults that + * match the maintainer's test rig. Contributors can override any of these + * via environment variables to match their own hardware setup. + * + * See CONTRIBUTING.md § "E2E Testing" for what each variable means and + * how to set up a test environment from scratch. + */ + +/** + * Channel used to trigger echo-bot traffic generation. + * + * The echo bot (running on a second "partner" radio) should monitor this + * channel and reply to any message, generating incoming RF traffic that + * mesh-traffic tests can observe. The channel is created automatically if + * it doesn't exist in the test database. + */ +export const E2E_ECHO_CHANNEL = + process.env.E2E_ECHO_CHANNEL ?? '#flightless'; + +/** + * Message sent to the echo channel to nudge the bot into replying. + * The bot just needs to see *any* message and respond; the exact text + * doesn't matter as long as the bot doesn't filter it out. + */ +export const E2E_ECHO_TRIGGER_MESSAGE = + process.env.E2E_ECHO_TRIGGER_MESSAGE ?? '!echo please give incoming message'; + +/** + * Public key (64-char hex) of a nearby node that will ACK direct messages + * sent by the test radio. This node must have the test radio's public key + * in its contact list. Used only by the partner-radio DM ACK test. + */ +export const E2E_PARTNER_RADIO_PUBKEY = + process.env.E2E_PARTNER_RADIO_PUBKEY ?? + 'ae92577bae6c269a1da3c87b5333e1bdb007e372b66e94204b9f92a6b52a62b1'; + +/** + * Display name for the partner radio node above. Used in UI assertions + * (searching the sidebar, verifying the conversation header, etc.). + */ +export const E2E_PARTNER_RADIO_NAME = + process.env.E2E_PARTNER_RADIO_NAME ?? 'FlightlessDt\u{1F95D}'; + diff --git a/tests/e2e/helpers/meshTrafficTest.ts b/tests/e2e/helpers/meshTrafficTest.ts index 1acc651..06c6ae6 100644 --- a/tests/e2e/helpers/meshTrafficTest.ts +++ b/tests/e2e/helpers/meshTrafficTest.ts @@ -17,7 +17,8 @@ * long polling timeout for environments without the bot. */ import { test as base, expect } from '@playwright/test'; -import { ensureFlightlessChannel, sendChannelMessage } from './api'; +import { ensureChannel, sendChannelMessage } from './api'; +import { E2E_ECHO_CHANNEL, E2E_ECHO_TRIGGER_MESSAGE } from './env'; export { expect }; @@ -26,15 +27,18 @@ const TRAFFIC_ADVISORY = 'network. Failure may indicate insufficient mesh traffic rather than a bug.'; /** - * Best-effort: send a message to #flightless that triggers a remote echo - * bot. If the bot is within radio range it will reply, generating the - * incoming traffic the test needs. Failures are silently ignored — the - * test will fall back to waiting for organic mesh traffic. + * Best-effort: send a message to the echo channel that triggers a remote + * echo bot on a partner radio. If the bot is within radio range it will + * reply, generating the incoming traffic the test needs. Failures are + * silently ignored — the test will fall back to waiting for organic mesh + * traffic. + * + * Configure the channel via E2E_ECHO_CHANNEL (default: #flightless). */ export async function nudgeEchoBot(): Promise { try { - const channel = await ensureFlightlessChannel(); - await sendChannelMessage(channel.key, '!echo please give incoming message'); + const channel = await ensureChannel(E2E_ECHO_CHANNEL); + await sendChannelMessage(channel.key, E2E_ECHO_TRIGGER_MESSAGE); } catch { // Best-effort — bot may not be reachable } diff --git a/tests/e2e/specs/dev-flightless-direct-route.spec.ts b/tests/e2e/specs/dev-flightless-direct-route.spec.ts index fd48b5a..77724b8 100644 --- a/tests/e2e/specs/dev-flightless-direct-route.spec.ts +++ b/tests/e2e/specs/dev-flightless-direct-route.spec.ts @@ -6,50 +6,46 @@ import { getMessages, setContactRoutingOverride, } from '../helpers/api'; +import { + E2E_PARTNER_RADIO_PUBKEY, + E2E_PARTNER_RADIO_NAME, +} from '../helpers/env'; -const DEV_ONLY_ENV = 'MESHCORE_ENABLE_DEV_FLIGHTLESS_ROUTE_E2E'; -const FLIGHTLESS_NAME = 'FlightlessDt🥝'; -const FLIGHTLESS_PUBLIC_KEY = - 'ae92577bae6c269a1da3c87b5333e1bdb007e372b66e94204b9f92a6b52a62b1'; -const DEVELOPER_ONLY_NOTICE = - `Developer-only hardware test. This scenario assumes ${FLIGHTLESS_NAME} ` + - `(${FLIGHTLESS_PUBLIC_KEY.slice(0, 12)}...) is a nearby reachable node for the author's test radio. ` + - `Set ${DEV_ONLY_ENV}=1 to run it intentionally.`; +const PARTNER_RADIO_NOTICE = + `Partner-radio hardware test. Requires a nearby node "${E2E_PARTNER_RADIO_NAME}" ` + + `(${E2E_PARTNER_RADIO_PUBKEY.slice(0, 12)}...) that will ACK DMs from this radio. ` + + `Set E2E_USE_PARTNER_RADIO_FOR_DM_ACK_TEST=1 to run, and override ` + + `E2E_PARTNER_RADIO_PUBKEY / E2E_PARTNER_RADIO_NAME to match your hardware.`; function escapeRegex(value: string): string { return value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); } -test.describe('Developer-only direct-route learning for FlightlessDt🥝', () => { - test('zero-hop adverts then DM ACK learns a direct route', { tag: '@developer-only' }, async ({ +test.describe('Partner-radio direct-route learning via DM ACK', () => { + test('zero-hop adverts then DM ACK learns a direct route', { tag: '@partner-radio' }, async ({ page, }, testInfo) => { - testInfo.annotations.push({ type: 'notice', description: DEVELOPER_ONLY_NOTICE }); - if (process.env[DEV_ONLY_ENV] !== '1') { - test.skip(true, DEVELOPER_ONLY_NOTICE); - } - + testInfo.annotations.push({ type: 'notice', description: PARTNER_RADIO_NOTICE }); test.setTimeout(180_000); - console.warn(`[developer-only e2e] ${DEVELOPER_ONLY_NOTICE}`); try { - await deleteContact(FLIGHTLESS_PUBLIC_KEY); + await deleteContact(E2E_PARTNER_RADIO_PUBKEY); } catch { // Best-effort reset; the contact may not exist yet in the temp E2E DB. } - await createContact(FLIGHTLESS_PUBLIC_KEY, FLIGHTLESS_NAME); - await setContactRoutingOverride(FLIGHTLESS_PUBLIC_KEY, ''); + await createContact(E2E_PARTNER_RADIO_PUBKEY, E2E_PARTNER_RADIO_NAME); + await setContactRoutingOverride(E2E_PARTNER_RADIO_PUBKEY, ''); await expect .poll( async () => { - const contact = await getContactByKey(FLIGHTLESS_PUBLIC_KEY); + const contact = await getContactByKey(E2E_PARTNER_RADIO_PUBKEY); return contact?.direct_path_len ?? null; }, { timeout: 10_000, - message: 'Waiting for recreated FlightlessDt contact to start in flood mode', + message: 'Waiting for recreated partner contact to start in flood mode', } ) .toBe(-1); @@ -74,22 +70,22 @@ test.describe('Developer-only direct-route learning for FlightlessDt🥝', () => }); const searchInput = page.getByLabel('Search conversations'); - await searchInput.fill(FLIGHTLESS_PUBLIC_KEY.slice(0, 12)); - await expect(page.getByText(FLIGHTLESS_NAME, { exact: true })).toBeVisible({ + await searchInput.fill(E2E_PARTNER_RADIO_PUBKEY.slice(0, 12)); + await expect(page.getByText(E2E_PARTNER_RADIO_NAME, { exact: true })).toBeVisible({ timeout: 15_000, }); - await page.getByText(FLIGHTLESS_NAME, { exact: true }).click(); + await page.getByText(E2E_PARTNER_RADIO_NAME, { exact: true }).click(); await expect .poll(() => page.url(), { timeout: 15_000, - message: 'Waiting for FlightlessDt conversation route to load', + message: 'Waiting for partner contact conversation route to load', }) - .toContain(`#contact/${encodeURIComponent(FLIGHTLESS_PUBLIC_KEY)}`); + .toContain(`#contact/${encodeURIComponent(E2E_PARTNER_RADIO_PUBKEY)}`); await expect( - page.getByPlaceholder(new RegExp(`message\\s+${escapeRegex(FLIGHTLESS_NAME)}`, 'i')) + page.getByPlaceholder(new RegExp(`message\\s+${escapeRegex(E2E_PARTNER_RADIO_NAME)}`, 'i')) ).toBeVisible({ timeout: 15_000 }); - const text = `dev-flightless-direct-${Date.now()}`; + const text = `dm-ack-route-test-${Date.now()}`; const input = page.getByPlaceholder(/message/i); await input.fill(text); await page.getByRole('button', { name: 'Send', exact: true }).click(); @@ -100,7 +96,7 @@ test.describe('Developer-only direct-route learning for FlightlessDt🥝', () => async () => { const messages = await getMessages({ type: 'PRIV', - conversation_key: FLIGHTLESS_PUBLIC_KEY, + conversation_key: E2E_PARTNER_RADIO_PUBKEY, limit: 25, }); const match = messages.find((message) => message.outgoing && message.text === text); @@ -108,7 +104,7 @@ test.describe('Developer-only direct-route learning for FlightlessDt🥝', () => }, { timeout: 90_000, - message: 'Waiting for FlightlessDt DM ACK', + message: 'Waiting for partner radio DM ACK', } ) .toBeGreaterThan(0); @@ -116,17 +112,17 @@ test.describe('Developer-only direct-route learning for FlightlessDt🥝', () => await expect .poll( async () => { - const contact = await getContactByKey(FLIGHTLESS_PUBLIC_KEY); + const contact = await getContactByKey(E2E_PARTNER_RADIO_PUBKEY); return contact?.direct_path_len ?? null; }, { timeout: 90_000, - message: 'Waiting for FlightlessDt route to update from flood to direct', + message: 'Waiting for partner radio route to update from flood to direct', } ) .toBe(0); - const learnedContact = await getContactByKey(FLIGHTLESS_PUBLIC_KEY); + const learnedContact = await getContactByKey(E2E_PARTNER_RADIO_PUBKEY); expect(learnedContact?.direct_path ?? '').toBe(''); await page.locator('[title="View contact info"]').click();