meshcore-hub/docs/database.md

Database

MeshCore Hub supports two database backends: SQLite (the zero-config default) and PostgreSQL (optional, for write scaling and multi-host deployments). Postgres is opt-in — leave the DATABASE_* variables unset to keep using SQLite.

Note

As of v0.14, SQLite is deprecated in favour of PostgreSQL. SQLite remains the default and continues to work unchanged; support will be removed in a future release (at least 3 months out). New deployments that need to scale across hosts should pick Postgres. Existing SQLite deployments can move to Postgres with a one-command migration — see Migrating from SQLite to PostgreSQL and upgrading.md.

SQLite (default)

SQLite needs no configuration. The database file is created automatically on first run and lives under DATA_HOME:

${DATA_HOME}/collector/meshcore.db

In Docker Compose this is the hub_data volume (${COMPOSE_PROJECT_NAME:-hub}_data). WAL mode is enabled automatically, allowing concurrent readers alongside the collector's single writer.

Limitations: writes are serialised to one process, and SQLite's file locking does not work over network filesystems — it caps you at a single host. To scale writes or run the stack across multiple hosts, switch to PostgreSQL.

PostgreSQL

Set DATABASE_BACKEND=postgres and fill in the DATABASE_* connection variables. Postgres is never selected implicitly — the explicit switch avoids a silent backend change.

Environment variables

Variable	Default	Description
DATABASE_BACKEND	sqlite	sqlite or postgres. Explicit switch — Postgres is never selected implicitly.
DATABASE_HOST	postgres	Postgres hostname (postgres = bundled container service name)
DATABASE_PORT	5432	Postgres port
DATABASE_NAME	meshcorehub	Database name
DATABASE_SCHEMA	meshcorehub	Schema (search_path). Set a distinct value per instance on a shared cluster
DATABASE_USER	meshcorehub	Role name
DATABASE_PASSWORD	(none)	Required for Postgres. Generate one, e.g. openssl rand -base64 32
DATABASE_URL	(none)	Advanced: full SQLAlchemy URL; overrides all of the above

The bundled postgres container derives its POSTGRES_USER / POSTGRES_PASSWORD / POSTGRES_DB from the same DATABASE_USER / DATABASE_PASSWORD / DATABASE_NAME values — one source of truth.

Docker (bundled container)

Postgres is bundled behind the postgres compose profile:

# Start the stack on Postgres (bundled container)
docker compose -f docker-compose.yml -f docker-compose.dev.yml \
  --profile postgres --profile core up -d

# Start on SQLite (default — no postgres profile)
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile core up -d

Production provisioning (role and database)

The bundled container provisions the role and database for you on first start from the DATABASE_* values. For a managed or external Postgres, create them once before pointing Hub at it. This mirrors the init script used in the ipnet-mesh/infrastructure cluster:

-- Run once as a superuser/admin role on the target cluster
CREATE DATABASE meshcorehub;
CREATE ROLE meshcorehub LOGIN PASSWORD 'your-password';
GRANT ALL PRIVILEGES ON DATABASE meshcorehub TO meshcorehub;

The application schema and tables are created automatically by db upgrade (run by the migrate service on startup); the role just needs CREATE privilege on the database. Hub only ever connects as DATABASE_USER — no admin or bootstrap credentials are needed at runtime.

Managed or external Postgres

To point Hub at an already-running Postgres (e.g. a managed cloud instance), set DATABASE_HOST at it and do not activate the postgres profile:

DATABASE_BACKEND=postgres
DATABASE_HOST=your-managed-postgres.example.com
DATABASE_PORT=5432
DATABASE_NAME=meshcorehub
DATABASE_USER=meshcorehub
DATABASE_PASSWORD=your-password

For advanced cases (custom driver, extra query params), set a full SQLAlchemy URL instead — it takes precedence over all the component variables:

DATABASE_URL=postgresql+psycopg2://meshcorehub:your-password@host:5432/meshcorehub

Schema-per-instance (search_path)

Each Hub instance is isolated to its own Postgres schema via the connection's search_path, rather than its own database. This lets several instances (e.g. prod, stg) share one Postgres cluster without colliding — each gets its own tables and its own alembic_version.

Give every instance a distinct DATABASE_SCHEMA:

# Production (.env)
COMPOSE_PROJECT_NAME=hub
DATABASE_BACKEND=postgres
DATABASE_SCHEMA=meshcorehub_prod

# Staging (.env, separate directory)
COMPOSE_PROJECT_NAME=hub-beta
DATABASE_BACKEND=postgres
DATABASE_SCHEMA=meshcorehub_stg

The schema is created automatically on db upgrade if it does not exist, so no manual CREATE SCHEMA is required. Connect both instances to the same DATABASE_HOST / DATABASE_NAME / DATABASE_USER; only DATABASE_SCHEMA (and COMPOSE_PROJECT_NAME) differ.

Note: This is the database-level isolation for instances sharing a Postgres cluster. For running multiple instances on the same Docker host (separate volumes, Traefik routing), see Multi-Instance Deployments in the README.

Migrating from SQLite to PostgreSQL

Existing SQLite deployments can be moved to Postgres with a single built-in command (meshcore-hub db migrate-to-postgres), which copies every table in foreign-key order through the ORM and prints a per-table row-count reconciliation. Downtime is required while writers are stopped; the source SQLite file is never modified.

See the v0.14 upgrade guide in upgrading.md for the full step-by-step runbook (backup, stop writers, bring up Postgres, run the migration, restart on Postgres).