Compare commits

...
710 Commits
Author SHA1 Message Date
Arunavo Ray 0759db80d3 chore: bump version to 3.36.1 2026-09-11 05:44:49 +05:30
ARUNAVO RAY 0299be92b1 fix(discovery): track one row per upstream repository across sources on the same host (#424)
The repositories table is unique on (user_id, source_id, normalized_full_name)
so the same owner/name can live on two hosts. Discovery used that key to
decide what was new, so two source rows on one host (an organization pinned
to a public-only github.com source next to a personal github.com token) each
inserted their own row for one repository, and the scheduler mirrored both to
the same destination in the same batch.

A repository is now identified by its host and full name whichever source
lists it: personal discovery, organization re-discovery and the two import
endpoints skip repositories the user already tracks on that host, and each
scheduler pass runs one row per upstream repository, logging the duplicate
rows it skips. Found while investigating #417.
2026-09-11 05:44:45 +05:30
ARUNAVO RAY 22a0aafcc1 fix(recovery): stop resuming jobs that are still running and checkpoint every item (#423)
Three things combined to make crash recovery restart live jobs:

- isJobInterrupted treated any job older than two hours as interrupted before
  looking at its checkpoint, so a large job that heartbeated every two minutes
  was resumed while it was still running. The age rule is gone: a checkpoint
  is the job's proof of life, and only a missing or stale one (ten minutes)
  marks a job interrupted.
- processWithRetry only checkpointed the item that tripped the interval, so a
  resume re-processed up to N-1 finished items. Every completed item is now
  recorded; the interval only throttles the published progress event.
- updateMirrorJobProgress was a read-modify-write of completedItemIds with no
  lock, so two items finishing in the same tick could drop each other. Writes
  are serialized per job with the keyed lock from #419.

A resumed job also records a fresh start time. Found while investigating #417.
2026-09-11 05:35:03 +05:30
ARUNAVO RAY 3b78f0a8d3 fix(security): raise the smol-toml floor to 1.7.1 in the app and the site (#422)
GHSA-7w5x-hrqm-74c2: smol-toml 1.7.0 and below can be made to hang on a
malformed TOML document. Dependabot only flags the site lockfile (1.7.0);
the app lockfile had 1.6.0 through astro and is raised the same way.
2026-09-11 05:34:59 +05:30
Arunavo Ray 48e0c5c346 chore: bump version to 3.36.0 2026-09-10 23:34:15 +05:30
ARUNAVO RAY 01f926d214 fix(deps): update @astrojs/node to 11.1.5 so the standalone server starts on astro 7.3 (#421)
Astro 7.3 moved the startup logger from app.pipeline.getLogger() to
app.getLogger(). The 3.35.1 security bump raised astro to 7.3.2 but left the
node adapter at 11.0.2, which still reads app.pipeline, so running
dist/server/entry.mjs directly crashes with "undefined is not an object
(evaluating 'app.pipeline.getLogger')" on every Bun version. Docker is not
affected because scripts/runtime-server.ts disables the adapter autostart;
the Nix package and bare metal systemd installs start entry.mjs and are.

Fixes #420
2026-09-10 23:34:02 +05:30
ARUNAVO RAY 1cc7a9a05c fix(releases): never duplicate release assets on the destination (#419)
Gitea and Forgejo accept any number of attachments with the same name:
CreateReleaseAttachment has no name-collision check, so every upload POST
appends another attachment. Asset reconciliation listed what a release
already had and then kept uploading for tens of seconds or minutes with no
mutual exclusion, so two passes over one repository in the same run both
read "missing" and both uploaded. That wide check-to-write window is what
left two, three or five copies of every asset (#417).

The classifier now groups existing attachments by name, keeps one copy and
reports the surplus and stale ones for deletion. Reconciliation fails
closed: nothing is uploaded when the attachment list cannot be read,
deletes run before any upload, and an asset whose stale copy could not be
deleted is skipped until the next sync. Release reconciliation holds a lock
keyed on the destination repository, so two passes cannot interleave even
when they come from two repository rows. The sync path and both mirror
paths claim the row with one conditional UPDATE instead of reading the
status and writing it in a second statement.
2026-09-10 23:33:58 +05:30
ARUNAVO RAY be0594b28f fix(orgs): keep a destination override that matches the organization name (#418)
The organization destination editor treated a typed value equal to the
organization's own name as "reset to default" and saved null. That is the
default only under the preserve strategy. Under single-org the default is the
configured destination organization, so an override of "GrapheneOS" on the
organization GrapheneOS was dropped and every repository kept landing in the
destination organization. Three display lines hid it: the custom badge never
showed for such a value, the card read "GrapheneOS -> GrapheneOS" with no
override stored, and the placeholder, helper text and reset label all stated
the preserve default whatever the strategy.

Both destination editors now save what was typed and clear the override only
for empty input, through a shared resolveDestinationSaveValue helper. The
organization editor takes the default for the configured strategy as a prop
and uses it everywhere it used to name the organization, and any stored
destination counts as an override. Reset to Default no longer saves the old
value again, which it did because setEditValue does not reach the handler's
closure.

The repositories list response now carries the organization destination
overrides, so a repository row shows where its organization actually sends it
instead of the strategy's owner, which is why the reporter believed the
repository level was ignored too.

Server side is unchanged: getGiteaRepoOwnerAsync already resolved starred
mode, then the repository override, then the organization override, then the
strategy. Tests cover that order for single-org and the new save rule.

Fixes #416
2026-09-10 23:11:55 +05:30
Arunavo Ray c51fa958ba chore: bump version to 3.35.1 2026-09-09 11:45:49 +05:30
ARUNAVO RAY e49cf0e367 fix(security): raise dependency floors for the 2026-09-08 advisories (#414)
Every one of these advisories was published on 2026-09-08 between 20:30 and 21:26 UTC, against the versions this repository was already pinning from the earlier rounds. Same package names, new CVEs, not a regression from anything that merged today.

**Application** (the nine Docker Scout alerts on main)

| Package | Was | Now | Advisory |
| --- | --- | --- | --- |
| astro | 7.1.0 | 7.3.2 (floor ^7.2.8) | GHSA-26w7-cxv4-gfx2, remote code execution through AVIF image optimization (critical) |
| @xmldom/xmldom | 0.8.13 | 0.8.15 | eight advisories, CVE-2026-83605 through CVE-2026-83619 (high) |
| sharp | floor ^0.35.0 | ^0.35.4 | GHSA-rgj7-g3m4-5g8c, libheif |
| js-yaml | floor ^4.3.1 | ^4.3.2 | GHSA-2883-xcg3-v3hh |
| svgo | (no floor) | ^4.1.0 | GHSA-w27v-7q3p-w38r, GHSA-4vpr-x523-8j87 |

**Documentation site** (the six Dependabot alerts against `www/pnpm-lock.yaml`)

astro to ^7.2.8 (resolves 7.3.2), and pnpm overrides for sharp ^0.35.4, svgo ^4.1.0 and js-yaml ^4.3.2. That covers all six, including the moderate Astro authorization bypass (GHSA-376h-93r7-7g6f) which 7.2.4 already fixed.

This supersedes the two Dependabot pull requests, which only cover the site and overlap with each other.

Checked: 906 tests pass, `bun run build` passes, and the site builds its 37 pages. The lockfile change is confined to the bumped packages and their transitive resolution.
2026-09-09 11:45:32 +05:30
Arunavo Ray 199879a64c chore: bump version to 3.35.0 2026-09-09 11:25:05 +05:30
ARUNAVO RAY 6b8d0b3d78 fix(public-orgs): finish the destination-only paths for tokenless sources (#411)
Follow-up to #409. Public-only accounts could add and mirror an organization, then hit the source-token gates that were left behind everywhere after the first mirror. Each item was reproduced against a running instance with only a destination configured and a tokenless GitHub source, and re-checked after the fix.

**The paths that still demanded a source token**

- `POST /api/job/sync-repo` and `POST /api/job/schedule-sync-repo` answered 400 "Config missing for the user or token", so the Sync button never worked for these accounts. Neither route uses that token; they dispatch to the destination and resolve source clients per repository. Both now gate on the destination token, like the mirror and retry routes already do.
- `recovery.ts` threw "GitHub token not found in configuration" before the per-source resolution right below it, so an interrupted mirror never recovered. The gate is gone and a tokenless GitHub source now gets the same anonymous client the job routes build.
- `mirrorGitRepoIssuesToGitea` and the pull request, label and milestone functions required the config token even though the client arrives as a parameter, so with the anonymous client they threw and the per-component try/catch logged an error on every sync. They now require only the destination, and public metadata mirrors.

**The bulk import**

`POST /api/sync` iterated every source, including tokenless ones whose personal listing 401s, counted them as failed, and answered 502 when there was nothing else. It now skips them the way `importRepositoriesFromSources` does, reports them as `skippedPublicSources`, and says so when every source is public only.

**The organization pin**

`force: true` without a `sourceId` or `provider` re-pinned the organization to the primary source. Reproduced with two sources: an organization pinned to the GitLab source and one deliberately left on every source were both moved onto the primary. Only a request that names a source re-pins now; naming a provider (public mode) still re-pins as #409 intended. Covered by a new route test.

**The badge**

`invalidateConfigCache()` cleared the cache but nothing re-read it, so the Public only badge appeared only after a reload. Mounted `useConfigStatus` hooks now refresh on invalidation, through one shared in-flight request so a page with several of them still makes a single call.

Two things from the review I deliberately did not change: the anonymous 60 requests an hour budget with its long `onRateLimit` waits, and `rediscoverOrganizationRepositories` running for every user rather than only tokenless sources. Both are judgement calls about the feature rather than defects.

Tests: 906 pass. Build passes. Type errors in the touched files are unchanged from main.
2026-09-09 11:24:50 +05:30
ARUNAVO RAY 72a3223c76 fix(orgs): keep organization cards at column width when few are shown (#410)
The organizations grid used `repeat(auto-fit, minmax(27rem, 1fr))`. `auto-fit` collapses empty tracks, so with a single organization the one card stretched across the whole content area, while two or more sat at half width. `auto-fill` keeps the tracks, so a card is the same width no matter how many there are.

Both the list and the loading skeleton use the same class, so both changed.

Checked in the browser at 820, 1440 and 2000 px wide with one and with four organizations: one card now stays at column width, two columns at 1440, three at 2000, and a single column below the `sm` breakpoint, all unchanged from before for two or more cards.
2026-09-09 11:16:17 +05:30
TomRoyls 2c394bb722 feat: mirror public organizations without a source (#409)
Public organizations can be mirrored from GitHub, GitLab and Gitea/Forgejo without configuring a source.

- The Add Organization dialog offers a Public only mode (the default when no source is connected): pick a provider, optionally an instance URL, and the organization is imported anonymously.
- A tokenless source row is found or created for that provider and host, and the organization is pinned to it, so attribution, locks, the scheduler and cleanup keep working per source.
- New createPublicGitHubClient: a throttled anonymous Octokit for tokenless GitHub sources, used by the org mirror job, per-repo mirror and retry, the sync metadata client and the source provider. An empty token never reaches createGitHubClient.
- GitHub organization listings degrade to public repositories when the source is tokenless or the token cannot see the private and member listings.
- The scheduler gates on the destination token only, skips personal discovery for tokenless sources, and re-lists pinned organizations each tick so newly published upstream repositories appear.
- The Sources card can save a row with no username and no token and marks it Public only; the organizations and repositories pages work with no configured source.

Thanks to @TomRoyls. (#409)
2026-09-09 11:09:59 +05:30
Arunavo Ray e82aa86723 chore: bump version to 3.34.0 2026-09-08 19:26:17 +05:30
ARUNAVO RAY c2ea2b3cb6 fix(orgs): bulk import only clears source pins it set in the same run (#408)
Follow-up to #407.

The bulk import cleared an organization's pin whenever another connected source listed the same organization name. That also undid pins a user had set on purpose through the card picker or the add dialog: pin acme to GitHub because the GitLab group of the same name is unrelated, and the next Import widened it back to every source.

Now the import only clears pins it created itself in the same run. Organizations it inserts (or re-pins on recovery) are remembered by name for the rest of the run; when a later source lists one of those names, the pin is cleared, since the organization spans several sources, the same case migration 0020 leaves unpinned. Pins that existed before the run are never touched. The bookkeeping is applied after each source's transaction commits, so a source that fails part way leaves it unchanged.

- `selectSameRunPinsToClear` in `src/lib/sources.ts` holds the rule, with unit tests.
- `src/pages/api/sync/index.ts` tracks the run's pins and applies the rule; the existing-org lookup goes back to name to status.
- Changelog line under the per-organization source selection entry.

Tests: 878 pass. Build passes.
2026-09-08 19:26:03 +05:30
TomRoyls a74b7e54ed feat(orgs): per-organization source selection (#407)
Organizations can pin the source they mirror from when more than one source is connected.

- New nullable organizations.source_id column (migration 0020_org_source), backfilled only when an organization's repositories agree on exactly one source.
- NULL means every source; a pin whose source was deleted falls back to the same. A valid pin scopes the org mirror job, the repository count breakdowns and org deletion to that source.
- POST /api/sync/organization persists the pin, PATCH /api/organizations/:id sets or clears it, the bulk import stamps discovered orgs and clears the pin when a second source lists the same org.
- UI, only with more than one source connected: a Source picker in the Add Organization dialog, a per-card inline source editor with an Every source option, and per-org View on source links.
- The org mirror repository query now filters by user.
- Shared provider icon map in src/lib/source-providers/icons.ts.
2026-09-08 19:13:32 +05:30
Arunavo Ray 3a2f8b34ab chore: bump version to 3.33.0 2026-09-07 06:49:25 +05:30
ARUNAVO RAY 7d17194e51 fix: review follow-ups for the organizations dialog and multi-source gating (#406)
Review follow-ups after #403 and #404: dialog copy and placeholder, forge-aware owner parser, GitHub only mirror switches available while any connected source is GitHub, configuration header copy, self-contained encryption test. Also builds the Sources card token settings link from validated URL parts (CodeQL alert) and adds a VEX statement for CVE-2026-85091 in zlib, which nothing in the image can reach.
2026-09-07 06:49:21 +05:30
TomRoyls 7dfb867202 feat: multiple sources per user with per-repository attribution (#404)
A user can connect several sources (GitHub, GitHub Enterprise, GitLab, Gitea/Forgejo) and every repository remembers which source it came from. New sources table (migration 0019) with repositories.source_id backfilled by provider and host, per-user unique indexes widened to include the source, a Sources card on the Configuration page with per-source lock confirmations, and credentials resolved from each repository's own source in discovery, import, scheduling, organization mirror, retry, recovery, push targets, cleanup and reconcile. The legacy githubConfig connection fields mirror the primary source so scripts and env setups keep working.
2026-09-07 06:25:27 +05:30
TomRoyls 4478c32db1 feat(ui): organizations page follows the configured source (#403)
The organizations page and the add dialog now follow the configured source: provider label, instance URL, icon and the organization noun (group on GitLab). The billing manager role only appears for GitHub. Adding an organization whose name contains a slash returns 400 with a hint to use the top level group. New tests cover getOrganization and listOrganizationRepositories on the GitLab and Gitea adapters.
2026-09-07 06:23:03 +05:30
TomRoyls 34babd4d8c feat(orgs): per-organization skip forks mirror option (#405)
Adds a Skip forks tri-state to the organization Mirror options dialog. An organization can opt out of importing and mirroring forks without touching the global switch, and the pin is honored by the bulk organization mirror, the add-organization route, bulk import and the scheduler.
2026-09-07 06:22:30 +05:30
Arunavo Ray 67ca35d893 chore: bump version to 3.32.1 2026-09-04 07:53:51 +05:30
ARUNAVO RAY 4a40497ff2 fix(security): clear the util-linux and git-lfs scan alerts (#402)
Strip setuid bits and add an OpenVEX exception for the three util-linux CVEs Debian will not fix in trixie, pin golang.org/x/crypto 0.56.0 and x/net 0.58.0 in the git-lfs build, and build git-lfs with Go 1.27.1.
2026-09-04 07:53:47 +05:30
Arunavo Ray 9a5bc450c2 chore: bump version to 3.32.0 2026-09-03 18:33:55 +05:30
ARUNAVO RAY ec655b12ce feat: move mirrors to another owner from the app and follow ones moved in Gitea (#400) (#401)
Changing a repository's or an organization's destination used to change a
label and nothing else: the mirror stayed where it was, sync failed once
someone transferred it in Gitea, and Retry created a second copy.

Sync now looks for a mirror of the same source anywhere on the destination
before failing, and the mirror path does the same before creating a copy.
Reconcile reports moved mirrors and can record their new location.

The destination editors offer to transfer the existing mirror on Gitea or
Forgejo, with a confirmation for organizations. Pending transfers, when the
token cannot create repositories under the new owner, are reported, and
sync follows the mirror once accepted. Nothing is deleted on either side.
2026-09-03 18:33:33 +05:30
ARUNAVO RAY c14473e3f8 docs(www): six new articles for sources, push destinations, API, reconcile and release assets (#389)
New use-case articles: mirror GitLab to Gitea, back up Codeberg and Forgejo repositories, mirror to GitHub or GitLab, automate mirrors with the API, adopt existing Gitea mirrors, and control release asset storage. Existing articles and the comparison page updated for the OCI Helm chart, the extra sources and destinations, and reconcile.
2026-09-02 21:11:55 +05:30
ARUNAVO RAY f7d830ac03 feat(www): homepage reflects sources, push destinations, API keys and the Helm chart (#388)
Hero copy mentions the GitLab, Gitea and Forgejo sources and the GitHub, GitLab and Forgejo destinations as beta with GitHub to Gitea as the supported default. Nine feature tiles with beta pills for sources and push destinations, five new FAQ entries linking into the docs, host neutral wording in the existing answers, footer links to the new docs pages, and the structured data version brought up to 3.31.0.
2026-09-02 21:10:03 +05:30
ARUNAVO RAY 68b493292e docs(www): sources, destinations, API and reconcile pages for 3.31 (#387)
Four new documentation pages (sources, destinations, API and API keys, cleanup and reconcile), updates to the configuration, environment variables, quickstart, architecture and advanced pages, refreshed configuration screenshots and new ones for the destination dropdown and the API keys card. Also fixes the path prefix variable name in docs/API.md and the stale follow-up note in docs/SOURCE_PROVIDERS.md.
2026-09-02 21:04:49 +05:30
Arunavo Ray 0be31d8c58 chore: bump version to 3.31.0 2026-09-02 19:10:22 +05:30
ARUNAVO RAY e4cf48e4a1 chore(config): GitHub to Gitea is the default, every other host is beta (#386)
The GitLab and Gitea / Forgejo sources and the Forgejo, GitHub and GitLab destinations show a beta pill and beta wording; the docs say GitHub to Gitea is the supported path. Testing every source and destination pair against real hosts turned up three bugs, fixed here: the pull mirror path never recorded a row's destination host, reconcile counted rows mirrored to a previous destination as missing, and the first push into a freshly created GitLab project failed because the host reports it as not found for a couple of seconds.
2026-09-02 19:09:59 +05:30
ARUNAVO RAY 30110a7745 fix(deps): merge the duplicate overrides blocks in package.json (#385)
The second overrides key added in #381 shadowed the original block, so JSON parsing kept only the four newer pins and the seven older ones were silently ignored on any re-resolve. One block with all eleven now; the lockfile records them and the resolved version set is unchanged.
2026-09-02 18:25:33 +05:30
Arunavo Ray ff734d48ab docs: forbid AI session links in commits and pull requests 2026-09-02 18:24:07 +05:30
ARUNAVO RAY 38fac54b17 feat(mirror): push engine for GitHub and GitLab destinations (#384)
GitHub and GitLab as mirror destinations, both beta. Neither has a usable pull mirror API, so the app keeps a bare clone of each source repository and pushes its branches and tags to the target with force and prune. Target adapters create, archive and delete repositories through each host's API. A dispatcher routes mirror, sync, retry, scheduler, recovery and cleanup by destination; Gitea and Forgejo are unchanged. Migration 0018 records each repository's destination host. Metadata, LFS, wiki and force push protection are disabled with a reason for push targets.

Closes #377
2026-09-02 18:23:27 +05:30
ARUNAVO RAY 8f9fc55c8e feat(helm): publish the chart to ghcr.io and keep its version in step with releases (#383)
The chart pinned appVersion 3.24.0 and nothing bumped it on release, so a default install ran an image six minor versions old, and the docs example pointed at a tag without the v prefix that does not exist on the registry.

The chart version now equals the application version. The tag build's sync job updates Chart.yaml alongside package.json, and a new job in the tag build packages the chart after the image is pushed and publishes it to oci://ghcr.io/raylabshq/charts/gitea-mirror. Docs and the website install tab show the OCI install.

Claude-Session: https://claude.ai/code/session_01Tp9pmi65a8k5jLMQFLf4JX
2026-09-02 18:08:30 +05:30
ARUNAVO RAY 1e43dc4652 Update star history chart links in README 2026-09-02 17:00:01 +05:30
ARUNAVO RAY 83b59874a8 feat(cleanup): reconcile the destination with the repository database (#382)
Compare what Gitea or Forgejo holds under every owner this account
mirrors into with the repositories table. Mirrors of the configured
source that have no row are reported as untracked and can be adopted;
rows marked mirrored whose repository is gone are confirmed with a
direct check and can be reset to imported so the next run recreates
them. Native repositories and mirrors of other hosts are listed and
never touched, and nothing is deleted or archived. Exposed as
POST /api/cleanup/reconcile and as a dialog on the Automation tab.

Closes #284

Claude-Session: https://claude.ai/code/session_01Tp9pmi65a8k5jLMQFLf4JX
2026-09-02 16:16:57 +05:30
ARUNAVO RAY a1ff0433df fix(security): clear the image scan alerts (#381)
Build git-lfs with Go 1.25.14, stop serving the runner stage from the
layer cache so Debian security updates land, move typescript and
@astrojs/check out of the shipped dependencies, bump the nested copies
of browserslist, js-yaml, nanoid and sharp to patched versions, and
update browserslist in the website lockfile.

Claude-Session: https://claude.ai/code/session_01Tp9pmi65a8k5jLMQFLf4JX
2026-09-02 16:16:46 +05:30
Arunavo Ray 26f78e06b8 chore: bump version to 3.30.0
Claude-Session: https://claude.ai/code/session_01Tp9pmi65a8k5jLMQFLf4JX
2026-09-02 14:59:27 +05:30
ARUNAVO RAY 83dd74dd52 feat(auth): API keys for programmatic access (#380)
* feat(auth): API keys for programmatic access

Adds the @better-auth/api-key plugin so scripts and CI pipelines can call
the existing endpoints with an x-api-key header instead of a session
cookie. Keys are owned by a user, hashed at rest, prefixed gm_, never
expire unless an expiry is chosen, and are not rate limited. A small
guard plugin refuses key management calls that arrive with a key, so a
leaked key cannot mint or revoke keys.

New API Keys section on the Authentication tab with create, show once,
copy and revoke. Migration 0017 adds the api_keys table with the
validator fixture. docs/API.md documents the header and the calls
automation needs. An e2e spec covers create, use, refuse and revoke over
HTTP. bun.nix regenerated for the new package.

Closes #314

Claude-Session: https://claude.ai/code/session_01Tp9pmi65a8k5jLMQFLf4JX

* fix(e2e): send Origin on cookie-authenticated key management calls

Better Auth rejects a cookie-authenticated POST without an Origin header (403 MISSING_OR_NULL_ORIGIN). Browsers always send one, the Playwright request context does not, so the spec sets it on the create and delete calls. Also asserts the guard's 403 code and documents the Origin requirement for scripts that manage keys with a session.

Claude-Session: https://claude.ai/code/session_01Tp9pmi65a8k5jLMQFLf4JX
2026-09-02 14:57:58 +05:30
ARUNAVO RAY 8d22774b0a feat(releases): limit release assets to the newest N releases (#379)
Adds a release asset limit next to the release limit. The release limit
still decides how many of the newest releases exist in Gitea at all. The
new asset limit decides how many of those also get their assets uploaded;
older releases inside the release limit are created with their notes and
tag only. Unset means assets for every mirrored release, which is what
happened before, so existing setups do not change. Zero means release
notes only. The limit inherits global to organization to repository
through the existing limit-key machinery, and lowering it never deletes
assets that were already uploaded; it only gates what a sync uploads.

Settings gets a second box next to the release limit, the overrides dialog
gets a matching row for organizations and repositories, and the env loader
reads RELEASE_ASSET_LIMIT. The README explains both limits and gains a
troubleshooting entry for the read-only push error people hit on pull
mirrors.

Closes #311
Closes #59

Claude-Session: https://claude.ai/code/session_01Tp9pmi65a8k5jLMQFLf4JX
2026-09-02 14:40:59 +05:30
ARUNAVO RAY 3ff05ec7f3 chore(deps): upgrade better-auth family to 1.7.2 (#378)
better-auth, @better-auth/oauth-provider and @better-auth/sso move from
1.6.23 to 1.7.2. This is the first stable line with the oauth-provider fix
for unbound resource indicators (Dependabot alert #55); no 1.6.x release
has it.

1.7 keys accounts by (issuer, accountId) and looks the credential account
up with issuer "local:credential" on sign-in, so migration 0016 adds the
column and backfills existing rows: local accounts get local:credential,
SSO accounts take their provider's issuer, other OAuth accounts get the
local:oauth namespace. It also adds jwks.alg and jwks.crv, the 1.7
oauth-provider columns, and the oauth_resources, oauth_client_resources
and oauth_client_assertions tables.

A database that booted the reverted 1.7.0-rc.4 build (on main for a few
minutes on 2026-08-06, and any dev database from then) already has part of
that schema without a 0016 record, which makes 0016 fail on every boot.
repairStrandedBetterAuth17Schema drops the leftovers before migrate runs,
in the same way the #312 repair handles stranded SSO columns; the
migration validator exercises it.

createOAuthClient no longer takes "type"; the SSO applications route sends
application_type (web or native) and reads it back with the legacy column
as fallback. bun.nix is regenerated. reset-users writes the issuer.

Supersedes #374.

Claude-Session: https://claude.ai/code/session_01Tp9pmi65a8k5jLMQFLf4JX
2026-09-02 14:07:08 +05:30
ARUNAVO RAY d211e54877 feat: mirror from GitLab and Gitea/Forgejo sources (#376)
Adds a Source dropdown to the configuration card (GitHub, GitLab beta,
Gitea/Forgejo) backed by a source provider interface with three adapters.
Discovery, mirroring, recovery and cleanup go through the provider;
repositories record their source (migration 0015); clone credentials are
built per host and a repository from another host is refused at the
mirror sites. Issues, pull requests, releases and star lists stay GitHub
only. The destination card gets a Gitea/Forgejo dropdown, and both hosts
lock once repositories exist, with an explicit confirmation to change.

Closes #375. Helps #371.

Claude-Session: https://claude.ai/code/session_01Tp9pmi65a8k5jLMQFLf4JX
2026-09-02 13:42:34 +05:30
Arunavo Ray a579934539 chore: bump version to 3.29.1
Claude-Session: https://claude.ai/code/session_01Tp9pmi65a8k5jLMQFLf4JX
2026-09-02 12:02:19 +05:30
ARUNAVO RAY f767a0a2b3 fix: stop treating live mirror jobs as interrupted (#372) (#373)
A job that had just started, with in_progress=1 and no checkpoint yet,
matched findInterruptedJobs immediately. Since #297 the middleware runs
that check on every request, so a live job was "resumed" by recovery
while the original process was still working on the same repositories.
The request-level 15 second timeout then released the in-flight latch
without cancelling recovery, and later requests logged misleading
"already in progress" and "completed with some issues" lines.

- Move the liveness rule into interrupted-job-detection.ts, as both a
  predicate and the SQL condition used by findInterruptedJobs. A job
  with no checkpoint is only interrupted once it is older than the 10
  minute checkpoint window (or has no recorded start at all).
- Stamp an initial checkpoint on in-progress jobs at creation.
- Refresh the checkpoint every 2 minutes from processWithResilience so a
  single long item cannot make a live job look interrupted. The refresh
  only touches rows still in progress.
- Keep the middleware recovery latch held until the recovery promise
  settles, not until the request stops waiting, and clear the timeout
  timer so it no longer leaves a dangling rejection.

Tests cover the predicate, the SQL against an in-memory SQLite database,
and the wiring into helpers, concurrency, and middleware.

Claude-Session: https://claude.ai/code/session_01Tp9pmi65a8k5jLMQFLf4JX
2026-09-02 11:59:37 +05:30
Arunavo Ray b1d2ea4d6d chore: bump version to 3.29.0
Claude-Session: https://claude.ai/code/session_01QRhZnpehnrVtnPRqJYDdVQ
2026-09-01 10:07:51 +05:30
Arunavo Ray e7270a6ca3 feat(ui): rebuild the activity rows and add current-state stats
A row spent three stacked lines on a status, a repo name and a Show
Details button, leaving most of its width empty. It now follows the
dashboard's Recent Activity shape - status circle, label, time - and uses
the width for the repository or organization and the message, with the
whole row toggling the details pane. Rows went from about 170px to 69px,
so ten fit where six did.

The status to icon and colour mapping was a chain of ternaries written
out twice, once for mobile and once for desktop; it is now one table,
shared with the new stat chips.

Those chips fill the empty half of the filter row and report the current
state of each repository or organization rather than counting events, so
a repo that failed and then synced counts once, as synced. Each one
filters the log to that status.

Two virtualizer bugs surfaced while measuring the new rows:

- measureElement reads the row index from a data-index attribute that was
  never set, so every measurement was dropped and rows only ever used
  estimateSize. That is why the old estimates were hand-tuned to the
  markup.
- Expanding a row called virtualizer.measure(), which discards every
  measurement rather than re-measuring the row that changed.

Search filtering also ran twice, once in ActivityLog and again in
ActivityList over the list it had already filtered. It happens once now,
which is also what lets the toolbar show a count that matches the list.

Also drops the status dot from the dashboard repository rows, which
repeated the status pill beside it without its label.

Claude-Session: https://claude.ai/code/session_01QRhZnpehnrVtnPRqJYDdVQ
2026-09-01 10:07:47 +05:30
Arunavo Ray 4ac2c4323c fix(ui): hand over to the compact toolbar at lg, and let lists size themselves
All three list pages switched from the compact toolbar to the full one at
sm (640px), so a tablet in portrait got the desktop row with six controls
and the search box squeezed down to its icon. The compact layout already
carries every filter in its drawer, so the handover moves to lg (1024px)
and tablets get a usable search box.

Lists also sized themselves off the viewport with hard-coded pixel
offsets - max-h-[calc(100dvh-231px)] and -276px. Those went stale the
moment a toolbar changed height, which is what put a second scrollbar on
the page next to the table's own. Both now flex into the space the
toolbar leaves, so the visible row count follows the window instead of a
fixed guess: the repository table shows 13 rows at 800px and 23 at
1400px, where it used to stop at the same count either way.

Also drops the bottom status bar from both lists. Its count moves up into
the empty left half of the filter row, and its live indicator was saying
what the header's LIVE button already says.

Claude-Session: https://claude.ai/code/session_01QRhZnpehnrVtnPRqJYDdVQ
2026-09-01 10:07:35 +05:30
Arunavo Ray cadeb64dce fix(ui): give the app a single scroll region
main was min-h-screen, so it could grow past the viewport, while the
content section sized itself with h-[calc(100dvh-4.55rem)] - a guess at
the header's height. Whenever the header came out taller than that
guess, the body scrolled behind the section's own scroll and every page
showed two scrollbars.

The shell is now pinned to the viewport and the section takes whatever
the header actually leaves, so the magic offset is gone and the height
is correct whatever the header does.

Claude-Session: https://claude.ai/code/session_01QRhZnpehnrVtnPRqJYDdVQ
2026-09-01 10:07:25 +05:30
Arunavo Ray 7fcce46821 feat(ui): accept a GitHub URL in the add repository and organization dialogs
Both dialogs asked for the name and owner as separate fields, so adding
something you were looking at on GitHub meant reading the URL and
retyping it in pieces.

Each dialog now has a GitHub URL field above an OR divider; filling it
populates the fields below. Pasting a URL into the name field splits it
as well, rather than dropping the whole URL into one box.

Parsing handles what people actually have on the clipboard: browser
URLs, clone URLs, SSH remotes, the owner/repo shorthand, a bare host,
and deep links into a repo. It drops the .git suffix, tolerates trailing
slashes, query strings and angle-bracket wrapping, and rejects reserved
GitHub paths like /settings so they cannot be read as an account name.

Claude-Session: https://claude.ai/code/session_01QRhZnpehnrVtnPRqJYDdVQ
2026-09-01 09:47:13 +05:30
Arunavo Ray 64e3b18e1e fix(ui): rework the repositories filter bar
The toolbar carried six controls plus Mirror All and overflowed on
narrower desktop windows, and the owner and organization pickers did not
match the dropdowns beside them.

- Moves the status, mirror options and sort dropdowns down to the row
  that carries "Showing N of M" and Clear filters, right-aligned. They
  live in RepositoryTable now because that is where the filtered count
  is computed, and the row renders in both the loading and loaded states
  so the controls do not appear only once data arrives.
- Makes the owner and organization triggers match the Select triggers.
  They kept a double-chevron icon and the outline Button styling; the
  Select trigger classes are now exported from ui/select so both use one
  definition instead of a copied class string. They stay comboboxes
  rather than Selects because those lists grow with the repository count
  and need the search box.
- Widens both to 200px (w-50), which fits longer owner names.
- Hides the moved dropdowns below sm. The mobile filter sheet already
  covers all five filters, so on mobile they were duplicates sitting
  loose on the page.

Also fixes hasAnyFilter, which used val?.toString(), yielding undefined
for an unset filter. undefined !== "" is true, so an unfiltered view
reported "Showing 697 of 697 repositories / Clear filters" and a
"Filters applied" footer with nothing filtered.

Claude-Session: https://claude.ai/code/session_01QRhZnpehnrVtnPRqJYDdVQ
2026-09-01 09:47:05 +05:30
Arunavo Ray b3e8c69566 fix(ui): make the dashboard Repositories and Recent Activity cards match
The two cards sat at whatever height their own content needed, so they
ended at different points whenever the lists were different lengths.
That is the normal state: repositories and mirror jobs come from
independent tables, and activity accumulates on every sync while the
repository list only grows when a repo is added.

- Drops items-start from the row so the columns stretch to the row
  height, in both the skeleton and the loaded state so the layout does
  not shift on load.
- Adds h-full to both cards. Without it the column stretches but the
  card inside still hugs its content, so removing items-start alone
  changes nothing on screen.
- Aligns the activity row padding with the repository rows (py-3 to
  py-3.5) so the rows line up when the two lists do have equal counts.

Verified in the browser at 8 vs 8 and 8 vs 5 items: both cards measure
657px in each case.

Claude-Session: https://claude.ai/code/session_01QRhZnpehnrVtnPRqJYDdVQ
2026-09-01 09:30:34 +05:30
Arunavo Ray f06f6695bb chore: bump version to 3.28.2
Claude-Session: https://claude.ai/code/session_01QRhZnpehnrVtnPRqJYDdVQ
2026-09-01 09:08:09 +05:30
Arunavo Ray a000073a68 fix(ui): rebuild the Identity Providers card to match Sign-in Methods
The card used shadcn Card/CardHeader while its neighbour used the plain
settings markup, so the two headers were different heights and their
dividers did not line up. Rebuilt it on the same markup: icon, title,
divider, body, footer.

- Drops the header description, which wrapped under the Add provider
  button and collided with it.
- Moves Add provider into the header row. The header uses py-3 rather
  than py-4 because the h-8 button makes the row taller than a text-only
  one; both headers now measure 56px.
- Removes mr-2 from the Plus icons. The button already applies its own
  gap, so mr-2 stacked an extra 8px and pushed the label off-centre.
- Gives the provider detail labels weight and a fixed 96px column.
  Everything was one muted weight with no hierarchy, and "Organization:"
  overflowed the old 80px column so its value broke the alignment.

Claude-Session: https://claude.ai/code/session_01QRhZnpehnrVtnPRqJYDdVQ
2026-09-01 09:08:05 +05:30
Arunavo Ray 2355284d27 chore: bump version to 3.28.1
Claude-Session: https://claude.ai/code/session_01QRhZnpehnrVtnPRqJYDdVQ
2026-09-01 08:57:39 +05:30
Arunavo Ray 6f3727c47e fix(ui): tighten settings card labels so they fit on one line
Database Maintenance footer said "Last cleanup"/"Next cleanup", which
wrapped onto two lines next to a full timestamp. The card is already
titled Database Maintenance, so "Last run"/"Next run" says the same
thing and fits.

Same for the Smart backup tile: "Snapshot only on history rewrites"
wrapped, and "history" is redundant since a force-push is the only
rewrite the sync sees.

Claude-Session: https://claude.ai/code/session_01QRhZnpehnrVtnPRqJYDdVQ
2026-09-01 08:57:36 +05:30
Arunavo Ray 1bb096aa93 chore: bump version to 3.28.0 2026-08-24 09:26:07 +05:30
ARUNAVO RAY feb13ff58a feat(mirror): per-repo release limit, description sync on every sync, tidier mobile cards (#370)
Follow-up to #362 and #365 for the remaining points raised in #361.

- Add `releaseLimit` to the per-repository and per-organization mirror
  overrides (repo > org > global > 10), surface it in the Mirror Options
  dialog with gating and validation, and pass the resolved value into the
  release mirror. Tags need no equivalent: Gitea's git mirror fetches every
  ref and they cost nothing beyond the commits already in the clone.
- Reconcile the Gitea description and topics with GitHub on every sync, not
  only at migration time. Read both live in one call, write the refreshed
  description back to the local row, and skip writes that would not change
  anything. Mirrors created before #224 or whose upstream description
  changed since now pick it up on the next sync.
- Rework the mobile repository card: name, Custom badge and menu on one
  vertically centred row, secondary badges on their own row below, status
  and last sync stacked on the right.
2026-08-24 09:10:45 +05:30
ARUNAVO RAY 41592cb8fd fix(ci): skip Docker Scout on fork PRs and add a Trivy scan for every PR (#369)
Fork PRs (and Dependabot PRs) never receive repository secrets, so the
Docker Hub login ran with empty credentials and Docker Scout then died
with "user githubactions not entitled to use Docker Scout". Every
external contributor got a red docker check they could not do anything
about (#368, #356, #357, #359).

- Gate the Docker Hub login and both Scout PR steps on the PR coming
  from this repo, the same guard the login/push/comment steps already use.
- Add a Trivy scan of the locally loaded image on every PR. It needs no
  credentials, so forks get real vulnerability feedback. Output goes to
  the job summary because GITHUB_TOKEN is read-only on fork PRs.
- Only upload the Scout SARIF when a file exists, so fork PRs stop
  logging "Path does not exist: scout-results.sarif".
- Update the workflows README, which still described a docker-scan.yml
  that no longer exists.

Not using pull_request_target on purpose: it would run untrusted PR code
with the Docker Hub token available.
2026-08-24 07:52:41 +05:30
Arunavo Ray 8446811e8d chore: bump version to 3.27.2 2026-08-21 17:37:40 +05:30
ARUNAVO RAY 0c41fac9c0 fix(auth): trust registered SSO provider origins and honor deleteFromGitea (#366) (#367)
Three fixes for the two problems reported in #366:

1. Auto-trust registered SSO identity provider origins. better-auth
   1.6.23 (shipped in v3.21.0) added SSRF hardening to the SSO plugin:
   sign-in rejects IdP endpoints whose hostnames resolve to private
   addresses unless the origin is in trustedOrigins. Homelab split-DNS
   setups (IdP domain resolving to a LAN IP from inside the container)
   broke on every sign-in with a 400. Registering a provider is an
   explicit operator action, so its issuer and endpoint origins are now
   added to trusted origins automatically.

2. Surface SSO sign-in errors in the login form. The auth client
   resolves with { data, error } instead of throwing, so server-side
   rejections were silently swallowed - the button flipped back from
   "Redirecting..." with no feedback and nothing in the logs.

3. Honor deleteFromGitea (CLEANUP_DELETE_FROM_GITEA). It was documented
   as "Delete repositories from Gitea" defaulting to false, but cleanup
   never read it and always archived/deleted orphans on the Gitea side.
   It now gates the Gitea-side operation: when disabled (default),
   orphans are only marked archived or removed in gitea-mirror's own
   database and the Gitea/Forgejo copies stay untouched.

Verified end to end against a live server: sign-in with a private-IP
IdP returns 400 discovery_private_host on v3.27.1 and a 200 with the
authorization URL on this branch; the login form now shows the server
error as a toast; delete cleanup with the flag off removes only the DB
row while the flag on contacts Gitea.
2026-08-21 17:37:32 +05:30
Arunavo Ray f18fc4bfe6 chore: bump version to 3.27.1 2026-08-21 07:09:35 +05:30
ARUNAVO RAY d63d55f52d fix(ui): surface Mirror Options on mobile repo cards and desktop org menu (#365)
The per-repo overrides from #361 were reachable only from the desktop
repositories table. The mobile card layout had no dropdown at all, and
the organizations page had the inverse gap: the item existed in the
mobile card menu but not the desktop one.

Mobile repo cards get a three-dot menu in the header opening the same
MirrorOverridesDialog, plus the Custom badge shown when a repository
overrides something. The organizations desktop dropdown gains the same
Mirror Options item the mobile variant already had.
2026-08-21 07:09:32 +05:30
ARUNAVO RAY 2867076d71 docs(www): align use-case articles with the actual configuration UI (#364)
Four older articles directed readers to UI elements by names that no
longer exist. Verified against src/components/config/:

  "Content & Data" section        -> "Mirror Content"
  "Mirror metadata" toggle        -> "Repository metadata"
  "Git LFS (Large File Storage)"  -> "Git LFS objects"
  "Mirror releases"               -> "Releases & tags"
  "Preserve structure" strategy   -> "Preserve Structure"

The new articles from #363 already used current names or generic
wording, so only the older four needed changes.
2026-08-20 06:41:41 +05:30
ARUNAVO RAY 7a935ff1b1 feat(www): SEO overhaul with sitemap, metadata, interlinking, and five new articles (#363)
* feat(www): generated sitemap, article metadata, and de-orphaned comparison page

The hand-written sitemap listed one URL of 24 and was dated January 2025.
astro.config now sets `site` and uses @astrojs/sitemap, so every built
page lands in sitemap-index.xml on each build. robots.txt points at the
new index and drops the duplicate user-agent block.

Use-case layouts gain the OG/Twitter tags the docs layout already had,
and all three article layouts now emit TechArticle JSON-LD.

Nothing on the site linked to /comparison/github-backup-tools/. It now
has a header nav item, a footer link, and a homepage CTA next to the
use-cases button. The footer Documentation link points at the site docs
instead of the GitHub tree, and footer internal links no longer open in
new tabs.

* feat(www): five new articles and an interlinking pass

New guides, each grounded in shipped behavior:

- Migrate from GitHub to Gitea (parallel-run migration playbook)
- Mirror Git LFS repositories (why one bad LFS repo fails its whole
  migration, and the per-repo override from v3.27.0 that fixes it)
- GitHub API rate limits (ETag conditional requests making 304s free)
- Mirror issues and pull requests (what transfers, the PR-to-issue
  conversion, why processing is sequential)
- GitHub backup on Unraid (based on the RayLabsHQ CA template)

All five are registered in the use-cases index; appended after the
existing entries so the homepage's featured three stay unchanged.

Interlinking pass across the existing articles: contextual inline
links where the text already referenced a topic, plus a short related
guides section per article. The comparison page's next steps now cover
the new guides, and the docs quickstart bridges to the use-case section.
Every article now carries 4 to 9 internal links; a build-time check
found 0 broken links across 440.
2026-08-20 06:33:04 +05:30
Arunavo Ray 85b72e9366 chore: bump version to 3.27.0 2026-08-20 06:03:34 +05:30
ARUNAVO RAY 0f700c7197 feat: per-repository and per-organization mirror option overrides (#362)
* feat: per-repository and per-organization mirror option overrides

Closes the gap reported in #361: a repository whose LFS fetch fails could
not be mirrored at all, because LFS was a single global switch. Gitea runs
the LFS fetch inside its own migration, so a failure aborts the whole
migration with nothing to salvage. The only fix available to us is to stop
asking for LFS on that repository.

Mirror options now resolve across three tiers, per flag, most specific
first: repository override, then organization override, then the global
config. NULL at a tier means inherit, so overriding LFS on one repo leaves
its other options alone.

Adds resolveMirrorOptions(), which replaces 24 scattered reads of
config.giteaConfig.* across three near-identical blocks: the two mirror
paths in gitea.ts and the sync path in gitea-enhanced.ts. The third was
easy to miss and mattered: without it, overrides would have been honored
on first mirror and silently ignored on every scheduled sync afterwards.

The starredCodeOnly clamp is folded into the resolver and deliberately
outranks explicit overrides, preserving the existing behavior that starred
repos mirror code only.

Editing is on the objects themselves, via the existing three-dot menus on
the Repositories and Organizations pages, not in Configuration, which keeps
holding the global defaults. A repositories filter and a row badge make it
possible to find which repos deviate.

Malformed override JSON degrades to "inherit" rather than throwing, so a
bad value can never break a mirror run.

* fix(ui): disable mirror toggles that cannot take effect, and explain why

A toggle the runtime will ignore should not look editable. Three cases
were doing exactly that, so they now share one mechanism:
getMirrorOverrideGating() returns a reason string per flag, and the dialog
disables the control and prints that reason underneath.

Starred clamp. When a repo is starred and starredCodeOnly is set, the
resolver forces every metadata flag off regardless of the override. The
dialog previously let the user set them anyway. The clamped set now lives
in STARRED_CLAMPED_KEYS, shared by the resolver and the gating helper so
the flags the UI disables cannot drift from the ones the runtime clamps.
LFS is deliberately excluded from that set, since turning LFS off per
repository is the point of #361, and a test pins that.

Labels. shouldMirrorLabels is `mirrorLabels && !mirrorIssues` in both
mirror paths, so labels cannot take effect while issues are mirrored. That
gate reacts live to the in-progress edit rather than only the saved state.

Inherit hint. For a repository the hint now reflects global -> org instead
of global alone, via a new name-keyed GET
/api/organizations/mirror-overrides that reuses
loadOrganizationMirrorOverrides. Personal repos skip the fetch, a failed
fetch degrades to the global values rather than blocking the dialog, and
the hint is suppressed while in flight so it is never briefly wrong.

Widens the UI to mirrorLabels and mirrorMilestones. mirrorMetadata stays
out: no mirror path reads it, so a per-object override would resolve
correctly and then do nothing. See the report for that finding.

useGiteaConfig additionally returns advancedOptions so callers can read
starredCodeOnly without a second request.

* fix(ui): read inherited mirror flags from mirrorOptions, not giteaConfig

/api/config does not return the mirror flags on giteaConfig. On the way
out, mapDbToUiConfig reshapes them into a separate mirrorOptions object
using different names (mirrorLFS, not lfs) and nesting the metadata flags
under metadataComponents with short names (issues, not mirrorIssues).

The dialog was written against the DB shape, so every lookup returned
undefined. Coerced to false, that is indistinguishable from a real "off",
which produced two symptoms: every inherit hint read "currently off"
regardless of the actual global config, and the labels gate never fired
because the effective issues value it keys on came from the same dead
source.

Adds mirrorOptionsToFlags() as the single conversion point, matching the
derivation in mapUiToDbConfig exactly, including that mirrorMetadata is a
master switch over the metadata components while lfs and mirrorReleases
sit outside it. Both dialogs now go through it, and useGiteaConfig returns
mirrorOptions alongside advancedOptions.

Server-side mirroring was never affected: the resolver reads config
straight from the DB, where the flags really do live on giteaConfig.

The existing tests could not catch this because they build a synthetic
config already in DB shape, so a client/server mismatch is invisible to
them. The new tests push flags through the real config-mapper in both
directions and assert the derived flags equal what mapUiToDbConfig would
persist, so they fail if that mapping changes again. One test pins the
root cause directly: that giteaConfig in the API payload carries no flags.

* fix(ui): surface the mirror-options filter on desktop, and on organizations

The overrides filter only existed inside the mobile filter drawer, so on
desktop the Repositories page showed a "Custom" badge on overridden rows
with no way to filter to them. Being able to answer "which repos deviate
from my defaults" on this page is the reason the Configuration-page
listing was dropped, so desktop could not do the one job it was given.

Adds the control to the desktop filter row next to status and sort, using
the bare Select-with-placeholder style those use rather than the drawer's
labelled markup. The mobile control is unchanged.

The Organizations page had a wider version of the same gap: it renders the
"Custom options" badge but never had this filter on either layout, and
OrganizationsList did not filter on it at all. Added the predicate plus
the control in both its layouts, so the two pages behave the same.

Also folds hasOverrides into activeFilterCount on both pages and into the
Organizations clear-all reset. Without that the mobile filter badge
undercounted an active overrides filter, and clearing filters left it set.

* fix(ui): stop double-applying the mirrorMetadata switch when reading config

mirrorOptionsToFlags ANDed each metadata component with mirrorMetadata on
the way in. That is a write-path rule: mapUiToDbConfig already applies it
when persisting, so a config saved through the settings UI has it baked
into the stored flag. Applying it again on read double-applies it.

The read path does not need it. mapDbToUiConfig puts the raw stored values
into metadataComponents, so metadataComponents.issues is exactly
giteaConfig.mirrorIssues, which is the field gitea.ts and
gitea-enhanced.ts read. Mapping the components straight through is 1:1
with runtime behavior.

Double-applying was invisible while the stored state was self-consistent,
since false && false is still false. It diverged when the state did not
come from the UI write path, which env vars allow: MIRROR_METADATA=false
with MIRROR_ISSUES=true stores mirrorMetadata:false, mirrorIssues:true.
The runtime mirrors issues; the dialog reported off. Measured against a
config in that state, five flags misreported.

I previously described this as an unrecoverable loss in the API shape.
That was wrong. Both values reach the client separately and unmerged, so
the payload carries full information and the loss was one we introduced.

Rewrites the test that asserted the derived flags match what
mapUiToDbConfig would persist. Comparing against the write derivation is
what pinned the bug in place. It now asserts the property that matters,
that flags derived from the API payload equal the stored flags the runtime
reads, plus a case checking agreement with resolveMirrorOptions on an
inconsistent config. Both fail if the AND returns.
2026-08-20 06:03:19 +05:30
Arunavo Ray a295a73066 fix(deps): patch nanoid, js-yaml and postcss in the www lockfile
Clears three Dependabot alerts on the marketing site:

  nanoid   3.3.16 -> 3.3.18  (infinite loop when size is zero)
  js-yaml  4.3.0  -> 4.3.1   (quadratic CPU on !!omap resolution)
  postcss  8.5.19 -> 8.5.26  (attacker-controlled source parsing)

All three are transitive and moved within their existing ranges, so the
change is version and integrity hashes only with no graph churn.
Verified with a frozen-lockfile install and a full site build.
2026-08-17 14:16:02 +05:30
Arunavo Ray 67cd0e2c60 docs: restore star history chart with scoped access token
GitHub restricted the stargazers API to repo admins and collaborators,
which broke the unauthenticated chart. Star History now needs a token
sealed into the image URL.

Uses a fine-grained token limited to this repository instead of a
classic public_repo token. The stargazers endpoint requires Contents
write, and classic public_repo grants that across every public repo the
account can push to, so scoping it to one repo keeps the blast radius
small. Legend moved to top-left to sit clear of the curve.
2026-08-17 14:11:31 +05:30
ARUNAVO RAY c9edc2132b Update star history links and image sources 2026-08-16 21:19:36 +05:30
Arunavo Ray d4db47c76e chore: bump version to 3.26.2 2026-08-07 17:03:41 +05:30
Arunavo Ray 91eb195494 fix(ui): smooth marquee scrolling and widen its hover target
Animating text-indent forced a relayout every frame, which made the
marquee visibly choppy. The scroll is now a compositor-driven transform.
The content stays plain inline text while at rest so the native ellipsis
still renders, and swaps to an inline-block only for the duration of the
animation.

The hover target is now the whole repository cell instead of just the
text, via a shared MarqueeTrigger wrapper. Name and path scroll together
at the same speed, so the motion reads as one block.
2026-08-07 13:38:47 +05:30
Arunavo Ray e0c39b697f chore: bump version to 3.26.1 2026-08-07 13:23:51 +05:30
Arunavo Ray 03fce1e299 fix(ui): keep repo name and path on one line with hover marquee
Long repository names and owner/repo paths in the repositories table
wrapped to multiple lines, breaking row alignment. Both now truncate
with an ellipsis and scroll horizontally on hover to reveal the hidden
part.

The new MarqueeText component animates text-indent instead of a
transform because ellipsis rendering only works on inline content, and
a transform would require an inline-block wrapper.
2026-08-07 13:23:48 +05:30
Arunavo Ray 7fc53cfad8 Revert "fix(security): upgrade better-auth family to 1.7.0-rc.4"
This reverts commit 2f6af22e25.
2026-08-06 07:19:13 +05:30
Arunavo Ray b33e7d5596 fix(security): patch image CVEs and unbreak Scout SARIF uploads
- Build git-lfs with Go 1.25.12 (stdlib CVEs fixed in 1.25.10) and pull
  golang.org/x/net past the fix for CVE-2026-39821; x/net 0.54.0 was
  flagged critical in the image scan.
- Cap SARIF relatedLocations at 100 per result before upload. GitHub
  rejects SARIF with >1000 related locations per result, and Docker
  Scout exceeds that for common OS packages, so every upload since May
  had failed silently (continue-on-error) and the Security tab was
  frozen on a stale scan. With uploads flowing again, already-fixed
  alerts (samlify, libgnutls via the existing apt-get upgrade) close on
  the next scan.
2026-08-06 07:00:03 +05:30
Arunavo Ray 2f6af22e25 fix(security): upgrade better-auth family to 1.7.0-rc.4
Fixes the @better-auth/oauth-provider advisory (unbound resource
indicators could yield access tokens for unauthorized audiences,
Dependabot #55). No patched 1.6.x exists; 1.7.0-rc.4 is the first
patched line.

The 1.7 oauth-provider expects a wider schema: new nullable columns on
oauth_clients / oauth_access_tokens / oauth_refresh_tokens /
oauth_consents (back-channel logout, DPoP, resource indicators, refresh
token rotation) and three new tables (oauth_resources,
oauth_client_resources, oauth_client_assertions). Migration 0014 is
purely additive; validate-migrations gains the matching upgrade fixture.
2026-08-06 07:00:03 +05:30
Arunavo Ray 185946984b chore: bump version to 3.26.0 2026-08-06 06:43:47 +05:30
Arunavo Ray bcc9cf0120 feat(github): cap ETag cache by size and scope tokenless clients
Follow-up to #356. The conditional-request store was bounded only by
entry count; a single 100-PR page can run to a few hundred KB, so 5000
entries could grow to gigabytes. The store now also tracks approximate
body bytes (64MB budget by default) and evicts oldest-first until both
caps hold. A body larger than the whole budget is not cached at all.

Clients created with only a token (the metadata mirroring path) all
shared the "default" cache scope across users. They now fall back to a
SHA-256 hash of the token, so distinct tokens never share entries and
the raw token never appears in cache keys.
2026-08-06 06:43:40 +05:30
Josh FreeandCopilot 2b455354f9 feat(github): reuse ETags across syncs via conditional requests (#356)
* feat(github): reuse ETags across syncs via conditional requests

The mirror re-lists every repository's pull requests on each scheduled
sync with `state: "all"`, and the Octokit client was created with the
throttling plugin but no conditional-request/ETag layer. Every sync
therefore re-downloaded unchanged data as full 200s and spent full
rate-limit budget.

Add an ETag cache wired into `createGitHubClient`: for each GET it
replays the previously stored `If-None-Match`, so GitHub returns
`304 Not Modified` (which does not count against the token's primary
rate limit) when nothing changed, and the cached body is reused. The
store is process-lifetime and scoped per user so ETags survive the
per-sync client re-creation without leaking data across tokens. Non-GET
requests are untouched.

Refs #355

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4959e1f9-f8e6-4e97-a487-f395a0123c79

* test(github): cover conditional-request ETag cache

Add unit tests driving a real Octokit instance with a stubbed fetch:
the second GET replays `If-None-Match`, a `304` is transparently served
from cache as a 200 with the same body, responses without an ETag are
re-fetched, non-GET requests are never made conditional, and cache
entries are isolated by scope.

Refs #355

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4959e1f9-f8e6-4e97-a487-f395a0123c79

* fix: key conditional-request cache by expanded URL

The request hook built the cache key from requestOptions.url, which is
still the route template (/repos/{owner}/{repo}/pulls) at hook time.
Every repo therefore shared one entry per user + endpoint, so with more
than one repo per user the stored ETag never matched and the 304 path
stopped firing. Expand the route via octokit.request.endpoint.parse
before building the key, and add a two-repo regression test that fails
on the shared-key behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4959e1f9-f8e6-4e97-a487-f395a0123c79

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4959e1f9-f8e6-4e97-a487-f395a0123c79
2026-08-06 06:36:31 +05:30
Arunavo Ray 428c97441c chore: bump version to 3.25.0 2026-08-03 11:59:44 +05:30
Arunavo Ray edf9c14be0 fix: honor DATABASE_URL for the SQLite database location
The variable was defined and documented but never read; the database
path was hardcoded to <cwd>/data/gitea-mirror.db. It now accepts
sqlite://, the legacy file: scheme, or a plain path, with relative
paths resolving against the working directory and the parent directory
created on demand. Defaults are unchanged, including the compose files
that pass the historical default value through. Docs updated to match.
2026-08-03 11:59:44 +05:30
ARUNAVO RAY 5c49191d61 Move canonical docs to the website (#354)
* docs: move canonical documentation to the website

The website now hosts the full documentation at /docs with a proper docs
layout: sidebar navigation, on-page table of contents, mobile nav, theme
support, canonical and OG meta, and overflow-safe code blocks and tables.
Ten pages, all rewritten from the current code rather than copied from
the old in-app docs: quickstart, deployment (Docker, Helm, Nix, LXC,
bare metal), configuration, environment variable reference,
notifications (all four providers including webhook payload signing),
authentication (including header auth), force-push protection,
architecture, advanced, and custom CA certificates.

This fixes every inaccuracy found in the docs audit: the wrong
raylabs/gitea-mirror image name, JWT_SECRET presented as the live auth
secret instead of BETTER_AUTH_SECRET, the missing auth env vars, both
wrong DATABASE_URL defaults, the contradicting starred-org default, and
health endpoint fields the API deliberately does not return.

The in-app /docs pages are retired: a stub redirects old bookmarks to
the website, and the sidebar and 404 links point there directly. The
markdown files under docs/ stay as the versioned offline reference;
NOTIFICATIONS.md now covers Gotify and Webhook. README links the docs
site, mentions notifications, and drops stale version markers. The app
viewport meta gains initial-scale=1.

* docs: mark new-repo notification as unimplemented, bump helm appVersion to 3.24.0
2026-08-03 11:56:51 +05:30
Arunavo Ray 63a4c9359a chore: bump version to 3.24.0 2026-08-03 11:29:01 +05:30
ARUNAVO RAY 882e147504 Redesign dashboard and configuration screens (#353)
* feat(ui): redesign dashboard and configuration screens

New settings design language built from the design/giteamirror.pen file:
cards with icon headers and status footers, header-level enable switches,
toggle switches instead of checkboxes, uppercase section titles, selection
tiles with icon chips and a check on the active option, segmented controls,
and an indigo accent. Implemented via shared primitives in
src/components/config/settings-ui.tsx and applied across:

- Automation: header switches, schedule card, one-line auto-mirror copy
  with info tooltip, full-width Repository Cleanup card with Skip/Archive/
  Delete tiles and dry run row
- Notifications: segmented provider picker (ntfy/Apprise/Gotify/Webhook),
  events card with per-event switches
- Connections: GitHub/Gitea connection cards with token creation guide and
  field helpers, Repository Selection and Mirror Content cards covering
  every mirror option, Organization Structure card with strategy tiles,
  destructive update protection tiles (BETA label removed)
- Authentication: sign-in methods status card, identity providers restyle
- Dashboard: flatter stat cards, icon panel headers, indigo view-all links

All existing state handling, autosave and API behavior is unchanged.
Light mode keeps working via theme tokens. README and website screenshots
regenerated, docs references to renamed cards updated.

* fix(ui): design polish pass from local review

- Recent Activity rows get status icon circles (check, sync, sparkles, alert)
- Connections tab restructured: connection cards share a stretched grid row
  so GitHub and Gitea stay equal height; Mirror Content moved to the right
  column; forms split into placeable cards via a part prop
- Token guide panel: link moved to header as icon, larger text; redundant
  card footers removed (scopes line, test-connection hint)
- Repository Selection gains a footer note; retention explanation moved
  below the selector
- Authentication tab matches the design: side-by-side cards, row dividers,
  disabled state-reflecting switches with info hints, footers; SSO dialog
  restyled (segmented protocol tabs, field labels, indigo primary)
- Import GitHub Data button is the indigo primary; disabled state is muted
- Time format menu redesigned (locale pill, live examples, live clock in
  the trigger); theme switcher moved to sidebar as icon segmented control,
  system preference now persists correctly; legacy ModeToggle removed
- Automation timezone pill no longer shows stored legacy UTC as a choice
- Config tab bar wraps 2x2 on narrow screens instead of overflowing
- README, website and PR screenshots regenerated
2026-08-03 11:27:45 +05:30
Arunavo Ray da75f10f26 chore: bump version to 3.23.0 2026-08-03 08:06:31 +05:30
Arunavo Ray e7758badbf feat: add generic webhook notification provider (#352)
Adds Webhook alongside ntfy, Apprise and Gotify: posts a JSON payload
(title, message, type, timestamp) to any URL, with an optional signing
secret that adds an X-Webhook-Signature header (HMAC-SHA256 of the body,
sha256=<hex>) so receivers can verify authenticity. Secret is encrypted
at rest like the other provider tokens. Settings UI section, provider
and service tests included. No database migration needed.
2026-08-03 08:06:26 +05:30
Arunavo Ray 69198c047a chore: bump version to 3.22.0 2026-08-01 10:08:47 +05:30
Arunavo Ray 4f55cb406c ci(nix): make the bun.nix drift check blocking
The check added in #350 only echoed a message (with backticks that bash
executed as command substitution, mangling it) and never failed the job,
so a stale bun.nix would still pass CI. Fail fast right after the
regenerate step instead, with a GitHub error annotation and the diffstat.
2026-08-01 10:08:31 +05:30
Jordan Dominion c7932faecb fix(nix): regenerate bun.nix from bun.lock (#350)
bun.nix (last regenerated in #298) drifted from bun.lock after the dependency updates in #348, leaving the Nix flake build broken: 254 packages missing and 195 stale entries. Regenerated with bun2nix; all 929 entries now match bun.lock exactly.

Also adds a CI step that detects drift after the regenerate step (made blocking in a follow-up commit).

Thanks @Cyberboss.
2026-08-01 10:07:51 +05:30
Arunavo Ray e40eaabf4c docs(www): fix inaccurate claims in use-case and comparison pages
- Remove references to nonexistent /api/export and /api/repos/:id/logs endpoints
- Correct storage model: mirrored repos and LFS live in Gitea, not the data/ volume
- Fix default sync interval (daily, not 1 hour) and startup log line
- Add write:organization to required Gitea token scopes
- Remove nonexistent metrics endpoint and per-repo interval claims from Helm page
- Fix rate-limit advice (limits are per account, not per IP/token)
- Refresh stale comparison content (outage dates, BackHub/Rewind acquisition)
- Add missing git clone step to comparison quick start
2026-07-29 08:26:08 +05:30
云与原 204922570d feat: add Gotify as a notification provider (#337)
Adds Gotify alongside ntfy and Apprise: new provider module posting to {url}/message with X-Gotify-Key auth, configurable default priority (errors always send at priority 8), token encrypted at rest like the other providers, settings UI section, and provider + service tests. No database migration needed.
2026-07-16 22:20:15 +05:30
Arunavo Ray 97d98b82c6 chore: bump version to 3.21.0 2026-07-16 21:52:01 +05:30
ARUNAVO RAY 8b81ffa975 chore(deps): security fixes and dependency updates across app and www (#348)
better-auth 1.6.23 (fixes GHSA high stored XSS via javascript: redirect_uri), esbuild >=0.28.1 override for www (GHSA low), Astro 7 / @astrojs/node 11 / @astrojs/react 6 / @astrojs/mdx 7 / lucide-react 1.x (inline GitHub icon replaces removed brand icon) and all in-range updates on both the app and the website. The remaining @better-auth/oauth-provider medium advisory is patched upstream only in 1.7-rc and will be picked up when 1.7 stable lands.
2026-07-16 21:51:41 +05:30
Arunavo Ray bb57b52de3 test: isolate bulk-mirror destination tests in a child process
bun's mock.module and the globalThis.fetch swap are process-wide; on CI
(bun 1.3.13) this file's mocks of @/lib/db and @/lib/gitea-enhanced leaked
into gitea-enhanced.test.ts and stuck-status-recovery tests, failing main.
The file now registers nothing in the shared test process and instead
re-runs itself via bun test in a child process where the mocks are contained.
2026-07-16 21:46:08 +05:30
Arunavo Ray b3aad9d80f test: make org ids order-independent in bulk-mirror destination tests
The previous call-order counter diverged between the mocked flow and the
assertions when bun re-instantiates mock factories (green on bun 1.3.6
locally, red on 1.3.13 in CI). Ids are now a pure function of the org name.
2026-07-16 21:35:56 +05:30
Arunavo Ray 3b8625634c docs: add time format toggle screenshot 2026-07-16 21:02:21 +05:30
ARUNAVO RAY bdbfa762af feat(ui): add 12h/24h time format option with locale-aware default (#342) (#346)
Timestamps now follow the browser locale by default (previously hardcoded en-US/12-hour), with a clock toggle in the header for Auto / 12-hour / 24-hour, persisted in localStorage.

Fixes #342
2026-07-16 20:57:48 +05:30
ARUNAVO RAY f922bcc618 fix: recover repositories stuck in syncing/mirroring after crashes (#339) (#347)
Adds stuck-status recovery: repositories (and orgs) stranded in an in-flight status by a crash/restart are reset to failed with an explanation, on container start and every scheduler tick, guarded by the existing 2h liveness window.

Fixes #339
2026-07-16 20:57:38 +05:30
ARUNAVO RAY c5b331c041 fix: stop config saves from resetting env-configured GITEA_MIRROR_INTERVAL (#338) (#345)
Config saves now preserve every field the settings form doesn't expose (mirror interval and other env-only options) instead of resetting them to defaults.

Fixes #338
2026-07-16 20:57:29 +05:30
Arunavo Ray 5c33a5547b test: cover bulk org mirror destination routing (#343) + clarify mixed-strategy log
- Add behavioral tests exercising mirrorGitHubOrgToGitea end-to-end down to
  the migrate HTTP payload: org-level override, per-repo override, mixed
  strategy uid, starred-repo mode, and preserve/single-org/flat-user
  no-override regression paths. All four bug-scenario tests fail on main
  and pass with PR #344 applied.
- Fix the top-level log that claimed 'flat-user strategy' when the mixed
  strategy falls into the same branch.
2026-07-16 20:55:15 +05:30
Yuzu 537aae952d fix: honor destination overrides in bulk org mirroring and crash recovery (#343) (#344)
Routes the bulk Mirror Organization path and crash recovery through the canonical destination resolver (getGiteaRepoOwnerAsync), so org-level and per-repo destination overrides are honored, the mixed strategy no longer sends org repos to the user's personal account (previously uid was dropped from the migrate payload and Gitea defaulted to the authenticated user), and starred repos follow starred-repo mode even when swept up in a bulk org mirror.

Fixes #343
2026-07-16 20:54:58 +05:30
Arunavo Ray 40efb9b83a chore(www): point site URLs to gitea-mirror.raylabs.io
The old giteamirror.com domain is being retired to avoid paying for
per-project domains. Repoint all canonical URLs, og:url, the homepage
siteUrl, robots.txt sitemap ref, and sitemap.xml loc from the old
gitea-mirror.com domain to the new gitea-mirror.raylabs.io site.
2026-07-06 00:40:27 +05:30
Arunavo Ray 06bfb49e0e chore: bump version to 3.20.4 2026-07-02 15:46:20 +05:30
ARUNAVO RAY b5e0c58708 fix: stop false-positive orphan archiving and heal sync 405s on archived-* renamed mirrors (#331) (#336)
Three related fixes for the "repos keep getting archived and then fail to
sync with HTTP 405" report:

1. Orphan cleanup no longer archives on bulk-list absence alone.
   Repos added via the "+" Add Repository dialog (foreign owner, not
   starred) can never appear in the authenticated bulk fetches, so every
   cleanup cycle deterministically flagged them as orphaned and archived
   them. identifyOrphanedRepositories() now runs a targeted per-repo
   confirmation (starred check or repos.get) and only treats a clean 404
   as gone; any other outcome fails safe.

2. The archived-* rename is persisted. archiveGiteaRepo() now returns
   the actual post-rename name and the cleanup service records it in
   mirroredLocation, so the DB no longer points at a name that only
   301-redirects.

3. Sync self-heals repos renamed in Gitea/Forgejo. Requests to a
   renamed repo get a 301; fetch follows it, downgrading POST to GET,
   which lands on the POST-only mirror-sync endpoint as a 405. The sync
   candidate loop now adopts the canonical owner/name from the GET
   response body before POSTing, tries an archived-{name} fallback for
   archived repos (guarded by an original_url source match), and keeps
   archived repos archived: no mirror-interval PATCH, status stays
   'archived' per the documented Manual Sync contract.

Also hardens mirrorGitHubReleasesToGitea to derive GitHub coordinates
from fullName so Gitea-side names can never leak into GitHub API calls.

Verified end-to-end on Forgejo 15.0.3 (rootless): pre-fix reproduces the
exact 405; post-fix the stale-name sync succeeds (GET stale -> 301, GET
canonical -> 200, POST canonical mirror-sync -> 200), archived repos keep
interval "0s" with no PATCH issued, and non-archived renamed repos heal
and get the configured interval applied.
2026-07-02 15:46:00 +05:30
Arunavo Ray 74606f0a5f chore: bump version to 3.20.3 2026-07-01 08:13:15 +05:30
ARUNAVO RAY 187ecc5d60 fix: correctly mirror Gitea release titles and issue/PR labels (#334 + sibling) (#335)
* fix(releases): send Gitea release title as `name`, not `title` (#334)

Gitea/Forgejo expose the release title through the JSON field `name`
(the API Go struct is `Title string `+"`"+`json:"name"`+"`"+`). The release
create and update payloads sent `title:` instead, which Gitea silently
ignores, so every mirrored release landed with a blank title.

Verified live against Gitea 1.24.7: a POST/PATCH with `title` yields
`name: ""`; the same call with `name` sets the title correctly. The
update path also self-heals previously-mirrored releases whose names were
left blank, since the existing-vs-expected name comparison already drives
a PATCH.

Adds gita-release-name.test.ts, which drives the real
mirrorGitHubReleasesToGitea create/update paths with a mocked fetch and
asserts the payload carries `name` (and never `title`).

* fix(issues): reconcile labels on issue/PR update via the labels sub-resource (#334 sibling)

Gitea/Forgejo's `EditIssueOption` has no `labels` field (only
`CreateIssueOption` does), so a `labels` key in a `PATCH .../issues/{index}`
body is silently dropped — the same silent-ignore class as the release
`title` vs `name` bug. The issue and PR-as-issue update paths sent `labels`
in the PATCH body, so label changes never propagated onto already-mirrored
issues (and a deadlock-orphaned issue recovered via PATCH never got its
labels).

Fix: add `reconcileGiteaIssueLabels`, which replaces the label set via
`PUT .../issues/{index}/labels` (idempotent — adds new, removes deleted).
Call it on the two issue update paths and the two PR-issue update paths,
and drop the dead `labels` key from those PATCH bodies. Labels on freshly
created issues still come from CreateIssueOption on the POST.

Verified live against Gitea 1.24.7 (PATCH ignores labels; PUT applies them)
and end-to-end (a drifted mirrored issue reconciled from no-labels to its
GitHub label set). Adds gitea-issue-labels.test.ts driving the real
mirrorGitRepoIssuesToGitea update path; the test carries a self-contained
http-client mock so it is immune to another suite's global module mock.

* test: make #334 regression tests deterministic via pure payload builders

The prior tests drove the real mirror functions with a global `fetch` mock.
That is order/version-fragile: another suite installs a process-global
`mock.module("@/lib/http-client")`, and bun 1.3.13 (CI) runs test files
concurrently, so `globalThis.fetch` races across files and
`isRepoPresentInGitea` (raw fetch) intermittently sees the wrong mock —
green locally on bun 1.3.6, red in CI.

Extract the payload construction into pure, exported builders and assert on
those instead (the repo's existing `classify*` pattern): buildGiteaReleasePayload
(create+update send `name`, never `title`), buildGiteaIssueEditPayload (edit
body never carries `labels`), buildGiteaIssueLabelsPayload (labels sub-resource
body). Behavior is unchanged — the builders return the exact same objects the
call sites built inline — and the fixes remain verified live on Gitea 1.24.7.
2026-07-01 08:12:36 +05:30
Arunavo Ray 632bbd0d4a chore: bump version to 3.20.2 2026-06-24 17:19:10 +05:30
ARUNAVO RAY 0b65e40784 fix(releases): create releases only for tags present in Gitea; stop sending target (#331) (#333)
Release creation failed on some Gitea/Forgejo instances with
"HTTP 404: The target couldn't be found", so no release (and therefore no
assets) was ever created — re-syncing never recovered.

Root cause: the create payload always sent `target: target_commitish`
(e.g. "main"). When the release's git tag is not yet present in the Gitea
mirror — which happens when Gitea's own git mirror clone lags behind the
metadata sync — Gitea tries to *create* the tag from `target`; if that ref
can't be resolved it returns a generic 404 ("The target couldn't be
found"), and if it can, it would create a brand-new tag at the wrong commit.

Reproduced the reporter's exact stack (Forgejo 15 rootless + read_only +
cap_drop ALL + postgres, plus Gitea 1.20-1.26 and Forgejo 1.21-15): a
healthy repo always succeeds — the 404 only occurs when the tag is absent
at create time.

Fix:
- Before creating a release, verify the git tag already exists in Gitea.
  If it isn't synced yet, skip it (logged) and let a later sync create it
  once the mirror has the tag — never create a tag via `target`.
- Drop the `target` field from both the create and update payloads. For a
  mirror the tag is synced from upstream, so Gitea attaches the release to
  the existing tag; `target` is unnecessary and is what triggers the 404.
- Surface skipped-missing-tag releases in the summary log for diagnosability.

Verified end-to-end against the real mirror function on a Forgejo instance:
a release whose tag exists is created with its assets; a release whose tag
was removed is skipped cleanly (no 404, no bogus tag) and picked up once the
tag is present.
2026-06-24 17:18:46 +05:30
Arunavo Ray 4a8b4f6ff3 chore: bump version to 3.20.1 2026-06-23 23:07:16 +05:30
ARUNAVO RAY 1d9dfdeb70 fix(releases): mirror assets idempotently so missing assets self-heal (#331) (#332)
Release assets were uploaded only on the create path of
mirrorGitHubReleasesToGitea(). When a Gitea release already existed, the
update path PATCHed the changelog/title and `continue`d without ever
touching assets. So any release whose assets were not fully uploaded on
that single create-path run — first sync interrupted, a transient
download/upload failure, large multi-MB assets, etc. — stayed permanently
asset-less, and re-syncing always hit the update path and could never
recover it. Asset failures were also swallowed to console.error, so the
job still reported success (the "no errors in the logs" in #331).

Reproduced on a real Forgejo pull-mirror with shauninman/MinUI: a GitHub
release with two ~35-40MB binaries became a Gitea release with 0 assets
(just Forgejo's auto-generated source archive), and re-syncing left it at
0 while logging "Updating existing release".

Fix:
- Add reconcileReleaseAssets(), an idempotent reconciler run on BOTH the
  create and update paths. It compares Gitea's existing attachments to
  GitHub's by name+size: skips matches, uploads missing ones, replaces
  size-mismatched copies. Existing broken releases self-heal on next sync.
- Extract the pure decision into classifyAssetsForReconciliation() for
  unit testing (the absence of asset tests is why this slipped past #310).
- Surface asset upload counts in the summary and emit a visible warning
  when any fail, instead of silently swallowing them.
- Add 5 unit tests covering the broken state, partial backfill,
  idempotency, size-mismatch replacement, and the no-assets case.

Verified end-to-end against the real function on a Forgejo pull-mirror:
0 -> 2 assets backfilled, already-present assets skipped (no re-download),
second run uploads 0.
2026-06-23 23:06:43 +05:30
Arunavo Ray 079df29f44 chore: bump version to 3.20.0 2026-06-19 08:44:03 +05:30
ARUNAVO RAY dff3cafb5e fix(config): persist Name Collision Strategy (starredDuplicateStrategy) (#326) (#328)
The "Name collision strategy" dropdown (starredDuplicateStrategy) never
persisted: the field was absent from both directions of the UI<->DB config
mapper. On save, mapUiToDbConfig dropped it before the DB write; on load,
mapDbToUiConfig never read it, so the UI reset to the "suffix" (repo-owner)
default. Mirror logic in gitea.ts then read undefined and also defaulted to
suffix — so repos really were created with that pattern regardless of the
user's choice. It has been broken since the field was introduced.

- Map starredDuplicateStrategy in mapUiToDbConfig and mapDbToUiConfig
- Add STARRED_DUPLICATE_STRATEGY env var for parity (reporter could not
  work around it via compose because no env var existed) + docs
- Round-trip tests covering save, load, and the missing-field default
2026-06-19 08:43:27 +05:30
ARUNAVO RAY 6ca7c0eec0 feat(github): add organization allowlist to mirror only selected orgs (#327)
Repository discovery requested the `organization_member` affiliation
unconditionally, so repos from every org a user belongs to were imported —
even orgs they never explicitly added. `skipPersonalRepos` only dropped
user-owned repos and left org repos unfiltered, which surprised users who
expected "only mirror org repos" to mean "only the orgs I chose" (reported
on #304).

Wire up the previously-dormant `includeOrganizations` config field as an
opt-in allowlist: when non-empty, only repos owned by the listed
organizations are imported. Empty = all org repos (backward-compatible).
Owned and collaborator repos are never restricted, so it composes cleanly
with `skipPersonalRepos`.

- Filter org repos by the allowlist in getGithubRepositories
- Add includeAllOrgsOverride so the cleanup service bypasses the allowlist
  and never false-orphans a previously-mirrored repo from an org the user
  later removes from the list
- UI control under Filtering & Behavior; INCLUDE_ORGANIZATIONS env var
- Case-insensitive dedup/trim in the UI<->DB mapper round-trip
- 7 unit tests covering the filter, composition, and the cleanup override
2026-06-19 08:42:17 +05:30
ARUNAVO RAY 6ebf1916e8 chore(deps): update dependencies across app and website (#325)
* chore(deps): update app dependencies to latest in-range versions

* chore(deps): update website (www) dependencies to latest in-range versions
2026-06-14 12:25:45 +05:30
Arunavo Ray 91de0d1030 chore: bump version to 3.19.1 2026-06-14 12:07:48 +05:30
Brendan Davidson 85bd1f4042 Repository table bulk actions (#322)
* Handle indexing when shift + clicking in the repository table

* Move the buttons when selecting rows

* Add in a bulk delete func in the repositories table

* Add bulk delete handler

* Make the single action use the bulk delete

* Delete the single repository id handler
2026-06-14 10:14:51 +05:30
Brendan Davidson 4a28015685 Skip the user defined orgs to ignore (#323) 2026-06-14 10:14:48 +05:30
Arunavo Ray da23941369 chore: bump version to 3.19.0 2026-06-13 09:14:33 +05:30
Brendan Davidson 906ce57e8c Handle indexing when shift + clicking in the repository table (#316) 2026-06-13 09:14:02 +05:30
Arunavo Ray 1b84c75a97 chore: bump version to 3.18.0 2026-06-13 08:17:20 +05:30
ARUNAVO RAY 0b6b6b76bf feat(github): add skipPersonalRepos toggle to mirror only org repos (#304) (#320)
- Add `skipPersonalRepos: z.boolean().default(false)` to githubConfigSchema
- Filter out user-owned repos in getGithubRepositories when flag is true
- Wire ONLY_MIRROR_ORGS env var to skipPersonalRepos in env-config-loader
- Add checkbox UI in GitHubMirrorSettings Filtering & Behavior section
- Round-trip skipPersonalRepos through config-mapper (UI ↔ DB)
- Add skipPersonalRepos to AdvancedOptions TypeScript type
- Mark include/exclude arrays in configSchema as unused/reserved
- Update ENVIRONMENT_VARIABLES.md to document ONLY_MIRROR_ORGS effect
2026-06-13 08:00:50 +05:30
ARUNAVO RAY 7610a614da fix: scheduler auto-start gate, backup clone URL, cancel-pending action, actionable 405 (#319)
* fix(scheduler): make enabled flag authoritative for auto-start

checkAutoStartConfiguration() and performInitialAutoStart() previously
used `scheduleEnabled || hasMirrorInterval`, allowing a configured
GITEA_MIRROR_INTERVAL to trigger boot-time auto-start even after the
user disabled scheduling via the UI toggle.

env-config-loader already writes scheduleConfig.enabled=true when
GITEA_MIRROR_INTERVAL is set at container startup, so the interval is
a timing detail, not an enable signal. The documented env-var contract
is preserved: GITEA_MIRROR_INTERVAL at boot → env-config-loader sets
enabled=true → auto-start fires. But a later UI disable now sticks.

Add a focused unit test for the gate logic.

* fix(backup): always derive clone URL from user-configured Gitea URL

The pre-sync backup preferred repoInfo.clone_url, which reflects
Gitea's ROOT_URL setting. In Tailscale MagicDNS deployments (and any
setup where ROOT_URL is an external address), this URL is unreachable
from the app itself, causing bundle backup to fail.

Always build the clone URL as:
  ${config.giteaConfig.url.trimEnd('/')}/${owner}/${repo}.git

This matches the URL the app already uses for all other Gitea API
calls and is guaranteed reachable.

* feat(jobs): cancel-pending endpoint + fix misleading Delete All copy

Add POST /api/job/cancel-pending that sets the current user's
repositories with status "imported" or "failed" to "ignored",
preventing the scheduler from re-queuing them. In-flight "mirroring"
rows are left alone. Returns the count and logs one activity entry.

Fix the "Delete All Activities" dialog to clearly state it only clears
the history log and does not stop pending work. Rename button/title to
"Clear History" so intent is unambiguous.

Add a "Stop Pending Mirrors" button (StopCircle icon, amber) in both
mobile and desktop activity log toolbars, with a confirmation dialog
explaining repos are set to Ignored and can be re-enabled from the
Repositories page.

* fix(sync): actionable 405 error for non-pull-mirror repos

Gitea returns HTTP 405 with an empty body when the target repository is
no longer a pull mirror — e.g. the mirror was auto-disabled by Gitea or
the repository lost its mirror state after a manual edit.

Previously this fell through to the generic error handler which stored
the raw HttpError message (often empty) giving the user no guidance.

Now a 405 response is caught alongside the existing 400 handler and
sets the repository to "failed" with an actionable error message:

  "Gitea reports this repository is not a pull mirror (HTTP 405).
  In Gitea check Settings → Mirror Settings; if the mirror section is
  missing, delete the repository in Gitea and re-mirror it from
  gitea-mirror."

The same message is written to the activity log for visibility in the
dashboard.
2026-06-13 08:00:47 +05:30
ARUNAVO RAY c28dcc209f fix(releases): stop delete/recreate cycle on permanent order mismatch (#310) (#318)
Root cause (Theory A): the `needsRecreation` check compared GitHub
published_at-based expected indices against Gitea's API order. Gitea mirror
repos sort releases by tag-commit date, which can permanently disagree with
published_at order (e.g. unaconfig_dart v0.1.0 published after v0.1.1 but
tagged before). This made `currentExpectedIdx < nextExpectedIdx` evaluate
true on every sync, triggering delete-all-and-recreate forever — spamming
Gitea's activity feed with "released X" events (#310).

Fix: replace the destructive order-check machinery with set-based
reconciliation via `classifyReleasesForReconciliation`. Releases are
created when missing in Gitea and skipped (or PATCH-updated if content
drifted) when already present. No deletions are ever triggered by ordering.
Retain the existing release-limit trimming (retention cleanup) unchanged.

Also removes the 1-second per-release delay that was only needed for the
creation-order dance, significantly speeding up initial mirrors.

Adds unit tests covering: normal ordered repos, the unaconfig_dart inversion
fixture, missing→create, present→skip, and edge cases.
2026-06-13 08:00:44 +05:30
ARUNAVO RAY 40ee3cbc44 fix(mirror): reuse existing same-source mirrors instead of creating suffixed duplicates (#315) (#317)
Starred (and other) repos duplicated on every re-mirror (starred/Repo,
Repo-owner, Repo-owner-1, ...) because the existence check only asked
"does a repo with this name exist?" and never "is the existing repo a
mirror of THIS same source?". The repo's own prior mirror counted as a
collision, so generateUniqueRepoName bumped to the next suffix each run,
repointing mirroredLocation at the newest copy. Under a single re-call,
3 concurrent/retried jobs each computed a DIFFERENT suffixed name, so the
location-based in-flight guard never matched and the race produced extra
copies.

Fix (source-identity aware):
- New shared helper src/lib/utils/mirror-source-match.ts:
  - normalizeCloneUrl / cloneUrlsMatch: credential-, .git-, slash- and
    host-case-insensitive clone URL comparison.
  - isMirrorOfSource: a Gitea repo is "ours" only if it is a mirror AND
    its original_url matches this repo's source.
  - findExistingMirror: resolves an existing same-source mirror via the
    recorded mirroredLocation first (survives strategy changes — #309),
    then the base candidate name.
  - classifyCandidateName: pure available/reusable/taken decision.
- gitea-enhanced: export GiteaRepoInfo and add original_url (Gitea's
  recorded migration source) for source matching.
- Both create paths (mirrorGithubRepoToGitea, mirrorGitHubRepoToGiteaOrg):
  run findExistingMirror BEFORE name generation; on a hit, reuse that
  location and route into the existing "already mirrored" handling rather
  than calling generateUniqueRepoName. Names now converge under
  concurrency so the in-flight guard becomes effective.
- generateUniqueRepoName is now source-aware: an occupied name held by a
  mirror of the SAME source is reused (no suffix); suffixing only happens
  on a genuine different-source collision, preserving #95/#236 behavior.
  The per-user DB claim check is retained so two users mirroring the same
  source into a shared org stay separated.
- Phantom-fork guard (#309): the existingRepoInfo.mirror branches now
  verify same-source before marking "mirrored"; on mismatch they fall
  through to unique-name generation and create a separate mirror.
- Scheduler: a `failed` repo whose mirroredLocation still resolves to a
  live same-source mirror is routed to syncGiteaRepo instead of re-create,
  breaking the failed-metadata re-create loop cheaply.
- Remove dead src/lib/starred-repos-handler.ts (zero importers across all
  git history); its correct base-name/.mirror reuse logic now lives in the
  shared helper.

Tests: src/lib/utils/mirror-source-match.test.ts (30 cases) covers URL
normalization, reuse at base name, reuse via mirroredLocation across a
strategy change, genuine different-source collision (suffix), phantom
fork, stale mirroredLocation fallback, per-user DB-claim separation, and
the suffix-vs-reuse classification. Full suite: 319 pass, 0 fail.
2026-06-13 08:00:41 +05:30
Arunavo Ray 699a5771f5 chore: bump version to 3.17.1 2026-06-05 18:42:11 +05:30
ARUNAVO RAY e862714d6a fix(db): self-heal sso_providers duplicate-column crash on upgrade (#312) (#313)
Migration 0013 runs as a single transaction that rebuilds `organizations`
and then `ALTER TABLE sso_providers ADD saml_config` / `ADD domain_verified`.
On instances where those columns were already created outside Drizzle (declared
in schema.ts and added via db:push / an SSO-register round-trip on an
intermediate build), the ADD throws "duplicate column name: saml_config". That
rolls back the entire 0013 transaction, so 0013 is never recorded in
`__drizzle_migrations` and is retried — failing identically — on every boot,
crash-looping the server.

Add a pre-migrate repair (mirroring the existing repairFailedMigrations() for
the 0009 case): when 0013 is unrecorded but the columns already exist, preserve
any real SAML provider config, drop the stranded columns so the canonical 0013
runs in full (organizations rebuild included), then restore the preserved
values once the columns are re-added. No-op on fresh installs, clean upgrades,
and already-migrated databases.

This lets affected instances recover automatically on the next boot after
upgrading — no manual SQLite surgery required.

- src/lib/db/migration-repairs.ts: repairDuplicateSsoColumns + restoreSsoDataAfter0013
- src/lib/db/index.ts: wire both around migrate()
- scripts/validate-migrations.ts: cover the broken-upgrade + data-preservation path
2026-06-05 18:41:46 +05:30
Arunavo Ray 716981aa04 chore: bump version to 3.17.0 2026-06-02 11:41:24 +05:30
ARUNAVO RAY 66e3284898 fix(sso): repair SSO login bounce + migrate to @better-auth/oauth-provider (#307)
Resolves #306. SSO sign-in via OIDC (Authentik / Keycloak / etc.) now links the
SSO identity to an existing email/password admin instead of bouncing to /login
with `?error=UNKNOWN`. Account-linking is gated on the operator-supplied
**Domain** field — cross-domain claims from a compromised IdP are refused.

Also bundles the deprecated `oidcProvider` → `@better-auth/oauth-provider`
migration. **Operators using the OAuth-provider feature must rotate registered
client secrets after upgrade** (legacy plaintext → hashed storage; see the
0012 migration notes).

Verified end-to-end on the pr-307 image against a real Authentik instance:
SSO login lands on the dashboard, `accounts` table gets both `credential` and
`authentik` rows for the same user. See PR description for full details.
2026-06-02 11:40:54 +05:30
IYUANWEIZE 695f0ff005 Fix Docker image line in .env.example (#305) 2026-05-28 13:49:32 +05:30
Arunavo Ray 53f2cf36fc chore: bump version to 3.16.3 2026-05-27 14:53:34 +05:30
ARUNAVO RAY 8ffcf3bdc6 fix: bridge header auth into a real Better Auth session (#303)
Header / forward authentication has been end-to-end broken since the
v3 rewrite. The middleware populated `context.locals.user` from
trusted upstream headers (Authentik / Authelia / oauth2-proxy /
Caddy), but never minted a Better Auth session, and never set a
cookie. Server-rendered pages saw the user, but the React SPA's
`/api/auth/get-session` call hit Better Auth's handler — which only
reads its session cookie — and got `null`. The auth guard then
redirected to `/login`, even though the upstream proxy had already
authenticated the user.

Reported on issue #29 by @lanrat with a clean repro on v3.16.1.

Fix: add a small Better Auth plugin (`header-auth`) that exposes
`POST /api/auth/sign-in/header`. The endpoint validates the trusted
headers via `authenticateWithHeaders`, creates a real session row via
`internalAdapter.createSession`, and attaches the `Set-Cookie` via
`setSessionCookie` — the same pattern the magic-link, anonymous, and
phone-number plugins use after their respective verification steps.

The Astro middleware now calls this endpoint when no cookie session
exists and header auth is enabled, forwards the `Set-Cookie` onto the
outbound response, and populates `context.locals` from the minted
session. After the first request the browser has the cookie; every
subsequent request takes the normal cookie-auth fast path and the
bridge doesn't fire.

Fail-open everywhere: any endpoint failure (header auth disabled,
auth rejected, DB blip, malformed response) returns null from the
bridge and the request proceeds as anonymous. A broken header-auth
configuration must never lock everyone out of the cookie-auth path.

Tests:
- `auth-header.test.ts` — unit tests for `extractUserFromHeaders` and
  `isHeaderAuthEnabled`, including lanrat's reported config shape
  (same header for username and email).
- `auth-header-plugin.test.ts` — locks down plugin id, endpoint key,
  path, and method so an accidental rename can't silently break the
  middleware bridge.
- `auth-header-bridge.test.ts` — covers the cookie-extraction logic
  and the fail-open paths (non-2xx, thrown error, malformed JSON,
  missing fields, no Set-Cookie attached).

Stacks on top of #301 (better-auth 1.6.11 bump).

Refs: #29
2026-05-27 14:53:16 +05:30
ARUNAVO RAY a07af96f84 chore: bump better-auth to 1.6.11 (#301)
Updates `better-auth` and `@better-auth/sso` from 1.5.5 to 1.6.11 to
pick up the patch fixes that have landed since (OAuth state CSRF
verification, scrypt non-blocking password hashing, account cookie
comparison fix, session freshness alignment, etc.). All 270 local
tests pass against the new version.

The only behavioral surface to watch:
  - `freshAge` now aligns with session `createdAt` instead of
    `updatedAt`. Not applicable here — we don't gate anything on
    session freshness.
  - 1.6.2 adds OAuth state-parameter CSRF verification. Applies to
    OIDC/SAML SSO flows; no code changes needed.
  - 1.6 emits a deprecation warning for `oidc-provider` in favor of
    `@better-auth/oauth-provider`. The plugin still works in 1.6.x;
    migrating it is a separate cleanup.

Prep work for an upcoming header-auth fix (issue #29 follow-up).
2026-05-27 14:44:17 +05:30
Arunavo Ray b1daa65228 chore: bump version to 3.16.2 2026-05-26 11:09:31 +05:30
Sean MousseauandClaude Opus 4.7 dd1c42264e fix: stop snapshot-row zombies + flapping force-push on deleted branches (#300)
* fix: stop snapshot-row zombies + flapping force-push on deleted branches

Two bugs caused Simple-WP-Helpdesk to accumulate one orphan
"Snapshot created" job row per scheduled sync — 7 zombies in 24h
against a deleted GitHub branch (`fix/v4.0.2-webhook-comment-author`).

(1) gitea-enhanced.ts:522 created the post-snapshot job record with
status="syncing". The snapshot was already complete at that point
(createPreSyncBundleBackup had returned), and no later code path
advanced the row to a terminal status — it just lingered. Set
status="synced" to reflect reality.

(2) force-push-detection treated any Gitea branch missing from GitHub
as a "deleted" force-push. But gitea-mirror is one-way (GitHub →
Gitea), so deletions never propagate back to the Gitea mirror —
the branch lingers in Gitea forever, the missing-from-GitHub condition
holds on every subsequent sync, and detection re-fires endlessly.
Each re-fire triggered backupBeforeSync=true to take a fresh snapshot
of the exact same state we already backed up the previous cycle.

Fix: detector accepts an `acknowledgedDeletions: { branch, giteaSha }[]`
list (persisted in RepositoryMetadataState). A Gitea branch missing
from GitHub is suppressed when its current giteaSha matches an entry.
If the giteaSha later changes (branch restored, then re-deleted with
new history), the entry won't match and detection fires again — back
up the new state, then acknowledge the new SHA. After a successful
snapshot, the call site appends each just-backed-up "deleted" branch
to the acknowledged list and persists the metadata blob.

Hoists parseRepositoryMetadataState above the backup-strategy block so
both the detection-time read and post-snapshot append happen against
the same in-memory state object; the existing metadata-mirror block
just stops re-declaring it.

Adds 9 new test cases covering:
  - deleted branch suppressed when acknowledged at matching giteaSha
  - re-flagged when giteaSha differs (restored-then-redeleted)
  - mixed deletions (only matching one suppressed)
  - undefined list = back-compat (existing callers unaffected)
  - acknowledgedDeletions does not suppress "diverged"
  - metadata-state parse/serialize round-trips the new field
  - legacy metadata defaults acknowledgedDeletions to []
  - malformed entries dropped without throwing

All 76 tests in the affected suites pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: acknowledge deletion before backup, not after, to fix concurrent-sync race

When two sync invocations fire for the same repo within the same backup
window (observed in prod via UI trigger pipeline: a single click ends up
producing two parallel processWithResilience callbacks that both run
syncGiteaRepoEnhanced), the second invocation's createPreSyncBundleBackup
short-circuits (file already exists from the first), so the second
invocation never reached the acknowledge-push that lived inside the
backup try block. Then both invocations wrote metadata: invocation A
with acknowledgedDeletions=[forged], invocation B (with its stale
in-memory state) with acknowledgedDeletions=[], and B's write landed
last — overwriting A.

Net result: the deletion never got acknowledged, and the next sync
re-detected it, the next-next sync re-detected it, etc. Same flapping
behavior as before the fix.

Fix: move the acknowledge-push to run right after detection, based on
detectionResult.affectedBranches alone. Semantically correct — once
we've detected the branch is gone from GitHub, we know it's a permanent
deletion; whether THIS particular invocation took a backup or
short-circuited is orthogonal (a previous backup on disk is fine, and
in the rare case where no backup ever succeeded, the user can re-trigger
a manual backup). Both concurrent invocations now push the same entry;
both write the same metadata; race is benign.

Verified locally: 59 tests pass across force-push-detection,
gitea-enhanced, repo-backup.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 11:06:27 +05:30
Arunavo Ray 384fbbbe10 docs: document Header / Forward Authentication setup
Header auth has been a working feature since v2.x but was missing from
SSO-OIDC-SETUP.md, leading users to think it was dropped in the v3
rewrite (see #29). Adds a dedicated section covering env-var config,
Authentik + Authelia examples, lookup order, verification, and the
must-strip-inbound-headers security checklist.
2026-05-25 10:37:39 +05:30
Sean Mousseau 1a54010950 fix: prevent duplicate milestones & labels on every sync (#299)
Paginate /milestones and /labels with Link header + X-Total-Count fallback, and pass state=all on milestones so closed milestones aren't re-POSTed on every sync. Caches newly-created items in the in-memory dedup set.
2026-05-25 08:28:40 +05:30
CrumblyLiquid 9bf22791a9 Regenerated bun.nix (#298) 2026-05-25 08:26:19 +05:30
Arunavo Ray 5d82f22b12 chore: sync version to 3.16.1 2026-05-23 20:25:16 +05:30
Sean Mousseau 6979c3bb32 fix: resume interrupted jobs after startup (not only at boot) (#297)
The previous middleware logic gated recovery behind a
once-per-process flag. After the first request handled the boot-time
recovery check, the resume codepath never fired again — even when
new interruptions appeared later in the process lifetime.

Symptom: a sync that started after server boot, crashed mid-flight
(deadlock retry hitting maxRetries, network blip, container
restart of an upstream service, etc.) and never reached the resume
codepath would sit at `inProgress=true, lastCheckpoint=never`
forever. The periodic detector kept finding the stuck row and
logging `Found 1 interrupted jobs:` on every poll (driven by the
health endpoint via `hasJobsNeedingRecovery`), but the resumer
(`resumeInterruptedJob`) was only invoked from `initializeRecovery`
which the middleware never re-called.

Fix: replace the one-shot gate with an in-flight latch
(`recoveryInFlight`) that's released in a `finally` block. Throttling
of actual recovery work is delegated to the existing 5-minute
`skipIfRecentAttempt` check inside `initializeRecovery()`, which is
the right place for it. `recoveryInitialized` is kept and only used
to control whether the "first run" log lines fire.

Secondary fix: `findInterruptedJobs` previously logged
`Found N interrupted jobs:` unconditionally on every call, including
from passive polls in `hasJobsNeedingRecovery` (which the health
endpoint and middleware probe call frequently). That produced one
log line per poll per stuck job for as long as the job stayed stuck.
Make logging opt-in via a `logFound` parameter, default off; the
active recovery cycle in `initializeRecovery()` opts in so the
operator-facing log still surfaces which jobs are being worked on.

Related: #268 (partially addressed). The PR #280 / v3.15.7 fix was
about a JS scoping bug that made the *initial* migrate call's
catch path crash before transitioning the repo to `failed`. That
was one cause of stuck `mirroring` state; this PR addresses the
follow-on issue that even when a job *is* correctly detectable as
interrupted, the post-startup recovery path never re-engages.

Adds `orchestrator-resume-after-startup.test.ts` using the
structural-source test pattern from
`gitea-mirror-failure-recovery.test.ts` so the four guarantees
(no static gate; in-flight latch released in finally; logging
opt-in; active path opts in) are enforced without heavy mocks.
2026-05-23 20:08:33 +05:30
Sean Mousseau b9f14e55e2 fix: prevent duplicate issues & PRs on retry-after-deadlock + Link-header pagination (#296)
mirrorGitRepoIssuesToGitea and mirrorGitRepoPullRequestsToGitea both
had two compounding bugs that produced duplicate Gitea issues/PR-as-
issue rows on every sync against any non-SQLite backend.

(1) Pagination on existing-issues / existing-PRs / per-issue-comments
was wrong.

The loops paginated with `limit=100` and broke on `pageX.length <
itemsPerPage`. Gitea caps response size at server-side
[api].MAX_RESPONSE_ITEMS (default 50), so the very first page
already looks "short" and pagination terminated after one page. The
existing-issue and existing-PR maps were built from only ~50 items
per repo, so every issue/PR past page 1 was treated as new on every
sync and re-created via the CREATE branch.

Naive removal of the short-page break (relying only on "break on
empty page") doesn't work either: for some Gitea endpoints Gitea
returns the same data on every page when the page is past the
actual end instead of returning [], so the loop runs forever.

Fix: use the Link header (RFC 5988). If `rel="next"` is absent,
terminate pagination. Applied to: existing-issues pre-fetch,
existing-PRs pre-fetch, and per-issue comments fetch.

(2) Retry-after-deadlock duplicated issues/PRs even when the map
was correct.

Gitea's CreateIssue handler commits the issue insert in one
transaction and then deadlocks on the subsequent addLabel /
UPDATE repository transaction. The issue row is committed and
visible, but the in-memory dedup maps are never refreshed between
retries — processWithRetry re-invokes the callback, sees
existingIssue === undefined from the stale map, and creates a fresh
duplicate via httpPost.

Reproduces deterministically on MySQL (Error 1213 / 40001) and
PostgreSQL (40P01); SQLite escapes because writes serialize globally.

Fix: defensive recheck via httpGet by [GH-ISSUE #N] (issues) or
[PR #N] (pull requests) before the create call, with PATCH
fall-through when found. Applied to the issues create path AND
both create paths in the PR mirror (enriched + basic-fallback).
Also cache freshly-created items into the dedup maps after a
successful create so subsequent retries of the same per-item
callback don't lose track of it.

Adds gitea-issue-dedup-on-retry.test.ts using the structural-source
test pattern from gitea-mirror-failure-recovery.test.ts so all
guarantees are enforced without heavy module mocks (8 tests).
2026-05-23 20:08:23 +05:30
github-actions[bot] 2582988f94 chore: sync version to 3.16.0 2026-05-19 07:31:20 +00:00
ARUNAVO RAY 4ea62a9f3d chore: bump and digest-pin Bun base image to 1.3.14 (#295)
Same hardening principle as #293 for GitHub Actions: pin to an immutable
identifier so a future tag move can't silently change what we build against.

- Bumps oven/bun from 1.3.13 to 1.3.14 (released 2026-05-13)
- Pins to multi-arch digest sha256:9dba1a1b...db6f rather than just the tag
- Applies to both base and runner stages
2026-05-19 12:50:37 +05:30
ARUNAVO RAY 1f60b2cf39 feat: add Change password and Change email to account dropdown (#292)
Adds an account menu under the avatar in the header with Change password and
Change email actions, each opening a small dialog. Calls Better Auth's existing
change-password / change-email endpoints — no new API routes.

- Enables `user.changeEmail` in Better Auth with `updateEmailWithoutVerification`
  since the app runs with email verification disabled and no email sender is
  wired up
- Hides Change password for SSO-only users (no `credential` provider account),
  fails open if the listAccounts probe errors
- Resolves discussion #291 ("Change user password/email")
2026-05-19 12:45:04 +05:30
ARUNAVO RAY a02865a1aa ci: pin third-party GitHub Actions to commit SHAs (#293)
Tags are mutable. A compromised maintainer (or a maintainer's compromised
machine) can force-move v-tags to point at malicious commits, and any workflow
using `@vN` picks up the malicious code on its next run — see the recent
`actions-cool/issues-helper` / `maintain-one-comment` incident exfiltrating
credentials from `Runner.Worker` memory.

This commit pins every third-party action in the two workflows that handle
secrets (GHCR push, Docker Hub login, Scout token) to immutable 40-char SHAs,
with a trailing comment naming the release version for readability. SHAs are
the latest released tag at time of pin.

The two DeterminateSystems actions were on `@main` — a *branch* ref that moves
on every push, materially worse than a tag — and are now pinned to the latest
release SHAs (v22 / v13).

First-party `actions/*` and `github/codeql-action` are left on tags for now;
they're a separate, lower-risk follow-up.
2026-05-19 12:44:23 +05:30
github-actions[bot] ad549dad9b chore: sync version to 3.15.12 2026-05-16 06:50:50 +00:00
ARUNAVO RAY 20103220d9 chore: prune npm overrides that are no longer load-bearing (#290)
Removed 5 overrides whose constraints have since been picked up naturally
by the transitive dep graph. Verified by removing each and confirming the
resolved version (and the dep tree) is identical to what the override
produced:

- defu ^6.1.7 → still resolves 6.1.7
- fast-xml-parser ^5.5.6 → still resolves 5.5.6
- node-forge ^1.3.3 → package not in tree at all (override was dead)
- rollup >=4.59.0 → still resolves 4.59.0
- svgo ^4.0.1 → still resolves 4.0.1

Kept overrides that are still doing real work:

- @esbuild-kit/esm-loader → npm:tsx@^4.21.0 — deliberate replacement shim
- @xmldom/xmldom ^0.8.13, devalue ^5.8.1, fast-uri ^3.1.2,
  fast-xml-builder ^1.1.7, kysely ^0.28.17 — active CVE pins (#289)
- lodash ^4.18.1 — pins to the newer 4.18.x line over the legacy 4.17.x
  that transitive deps still pull
- picomatch ^4.0.4 — without it, picomatch@2.3.2 is added as a duplicate
  copy via a transitive that asks for 2.x

Future drift would be caught by Dependabot + the weekly Docker Scout
scan; the overrides above remain because they currently affect the tree.
2026-05-16 12:03:53 +05:30
ARUNAVO RAY fe2c825244 chore: bump npm overrides to patch HIGH-severity CVEs (#289)
Patches 9 Docker Scout HIGH alerts surfaced by the weekly image scan:

- @xmldom/xmldom 0.8.12 → 0.8.13 (CVE-2026-41672/3/4/5)
- devalue 5.6.4 → 5.8.1 (CVE-2026-42570)
- kysely 0.28.16 → 0.28.17 (CVE-2026-44635)
- fast-uri (new override) → ^3.1.2 (CVE-2026-6321, CVE-2026-6322)
- fast-xml-builder (new override) → ^1.1.7 (CVE-2026-44665)

All five resolve to fixed versions after `bun install`. Tests and astro
build pass locally.

Remaining open Docker Scout alerts (git-lfs Go stdlib, gnutls28, nghttp2)
are base-image or upstream-binary issues, not addressable via npm.
2026-05-16 11:42:53 +05:30
github-actions[bot] 4b858a0251 chore: sync version to 3.15.11 2026-05-16 04:28:38 +00:00
Eduardo Riguetto (Kralot) 585d2ceb84 fix: reconcile metadata on every sync instead of once per repo (#287)
Previously, mirror of issues/pull-requests/labels/milestones was
guarded by !metadataState.components.<component>, so once a repo had
been mirrored the metadata path was permanently skipped with logs
like "Issues already mirrored; skipping to avoid duplicates".

This meant title changes, new comments, label updates and milestone
edits on the source GitHub repo were never propagated, even when the
user explicitly clicked Sync.

The underlying mirror* functions already handle idempotent updates:
issues and PRs are matched by [GH-ISSUE #N] / [GH-PR #N] markers in
the title and PATCHed in place, labels are deduped by name, milestones
by title. The releases path already runs unconditionally for the same
reason ("always allowed to rerun for updates"); aligning the other
metadata paths with it.

Touched: mirrorGithubRepoToGitea, mirrorGitHubRepoToGiteaOrg, and the
syncGiteaRepoEnhanced path in gitea-enhanced.ts. Updated the
"already-synced-repo" test to assert reconciliation runs on resync.
2026-05-16 09:40:48 +05:30
Eduardo Riguetto (Kralot) 7c1f24dc2f fix: include organization_member in /user/repos affiliation (#286)
Affiliation was set to "owner,collaborator", omitting repos owned
by orgs the user belongs to. As a result:

- main sync, scheduler, and cleanup never saw org repos
- orgs appeared empty unless manually re-added via /api/sync/organization
- restart archived previously-mirrored org repos as orphans

GitHub's API default is owner,collaborator,organization_member;
restoring it fixes both symptoms with no other code changes.
2026-05-16 09:40:40 +05:30
github-actions[bot] 680b374c84 chore: sync version to 3.15.10 2026-05-04 08:37:38 +00:00
ARUNAVO RAY 088467a57d feat: add option to exclude collaborator repos from import (closes #279) (#283)
GitHub's listForAuthenticatedUser defaults to returning every repo the
user has access to (owner + collaborator + organization_member), which
imports a lot of noise for users who only want their own repos.

Adds an `includeCollaboratorRepos` toggle, defaulting to true to preserve
existing behavior. When disabled, the affiliation filter scopes the API
call to "owner" only.

The cleanup service overrides the filter to always include collaborator
repos when computing the "what's still on GitHub" list. Without this,
toggling the option off would mark previously-mirrored collab repos as
orphaned and archive/delete them from Gitea.

Wired through the schema, both UI<->DB mappers, the env-config loader
(with new INCLUDE_COLLABORATOR_REPOS env var), and the settings UI.
2026-05-04 14:00:10 +05:30
github-actions[bot] adb436444e chore: sync version to 3.15.9 2026-05-04 04:50:49 +00:00
ARUNAVO RAY cc635485f0 feat: surface auto-mirror toggle in automation settings (refs #278) (#282)
The fix in v3.15.8 made scheduleConfig.autoMirror an independent trigger
in the scheduler, but it remained reachable only via the AUTO_MIRROR_REPOS
env var. This adds a UI checkbox under the Automatic Syncing section so
the option can be toggled per-config without touching the environment.

The toggle is conditional on scheduling being enabled (since auto-mirror
without a scheduler is meaningless) and is independent of the existing
"Auto-mirror new starred repositories" toggle in GitHub settings. Together
they cover the full owned/starred matrix that the scheduler already
supports.

Plumbing: config-mapper.ts now round-trips autoMirror through the UI/DB
boundary, and ScheduleConfig in types/config.ts gets the matching field.
No schema or migration change — autoMirror was already in the zod schema.
2026-05-04 10:14:09 +05:30
github-actions[bot] 6f343de5fd chore: sync version to 3.15.8 2026-05-04 03:49:44 +00:00
ARUNAVO RAY a18f262ca7 fix: make autoMirrorStarred actually trigger auto-mirror (fixes #278) (#281)
The "Auto-mirror new starred repositories" checkbox in the GitHub settings
was a filter layered on top of scheduleConfig.autoMirror, which itself is
only settable via the AUTO_MIRROR_REPOS env var (no UI). So users who
checked the box saw their starred repos auto-imported but never mirrored.

Treat autoMirror and autoMirrorStarred as independent triggers in the
scheduler: autoMirror covers owned (and self-starred) repos, autoMirrorStarred
covers repos starred from other owners. Either flag on its own is enough
to enter the auto-mirror phase, and the filter scopes the work accordingly.

Also normalize the owner comparison to lowercase since GitHub usernames are
case-insensitive — previously a self-starred repo whose stored owner casing
differed from the configured owner would be misclassified as a third-party
star.

Behavior change worth flagging in release notes: anyone who currently has
the starred checkbox on (broken state) will start getting starred repos
mirrored on upgrade. AUTO_MIRROR_REPOS=true users see no change.
2026-05-04 09:12:57 +05:30
Arunavo Ray 3798456f5d chore: bump version to 3.15.7 2026-05-04 08:20:26 +05:30
ARUNAVO RAY 73f1609117 fix: unstick repos in 'mirroring' on transient errors (fixes #268) (#280)
* fix: hoist migrateSucceeded above try so catch can update DB on failure (fixes #268)

`let migrateSucceeded` was declared inside the try block of
mirrorGithubRepoToGitea and mirrorGitHubRepoToGiteaOrg, but the catch
block referenced it. Block-scoping made it invisible to catch, so any
error inside the try (network timeout, transient 5xx, etc.) crashed the
catch with `ReferenceError: migrateSucceeded is not defined` before
reaching the DB update that marks the repo "failed". Result: repos
stuck in "mirroring" forever with no entry in the activity log.

Hoisting the declaration above the try restores the intended behavior:
catch updates the repo to failed, clears mirroredLocation when migrate
hadn't succeeded, writes a failed activity-log entry, and re-throws
with the original error message preserved.

TypeScript was flagging this as "Cannot find name 'migrateSucceeded'"
but esbuild stripped the types during build, so the bug shipped.

* test: replace integration test with structural source check (#268)

The behavioral version of this regression test passed locally but
failed in CI because of mock.module pollution between files: bun's
mock.module is process-wide, so my mock for @/lib/gitea-enhanced
leaked into gitea-enhanced.test.ts (its real-module assertions saw
my null-returning mocks), and gitea-enhanced.test.ts's own
@/lib/http-client mock could supersede mine depending on file
discovery order, causing my mirrorGithubRepoToGitea call to not
throw at all in CI.

Replace with a structural assertion that reads gitea.ts and verifies
`let migrateSucceeded` is declared before the outermost try in both
mirrorGithubRepoToGitea and mirrorGitHubRepoToGiteaOrg. Verified the
new test fails on the pre-fix source with a clear error message
pointing to issue #268, and passes on the fixed source.
2026-05-04 08:08:22 +05:30
Arunavo Ray 588567931a chore: bump version to 3.15.6 2026-04-26 13:43:45 +05:30
Arunavo Ray 5c1317c759 feat: warn when Forgejo destination has known mirror-credential bug (refs #263)
Forgejo < 15.0.0 silently discards auth_username/auth_password sent to
/api/v1/repos/migrate, causing subsequent pull-mirror sync of private repos
to fail with `terminal prompts disabled`. Fix landed upstream in Forgejo
v15.0.0 via codeberg.org/forgejo/forgejo/pulls/11909 and was not backported
to v12/v13/v14.

Test-connection endpoint now also probes /api/v1/version, detects Forgejo
via the `+gitea-` suffix, and surfaces a warning Alert in the Gitea config
form when the connected server reports a major version below 15.
2026-04-26 13:43:40 +05:30
Arunavo Ray 5f1c37b320 fix: don't gate dashboard on optional username fields (refs #271, v3.15.5)
The useConfigStatus hook treated `githubConfig.username` and
`giteaConfig.username` as required for the dashboard to render. In
practice neither is required at runtime — the GitHub token is
self-authenticating via listForAuthenticatedUser, and a Gitea username
isn't needed under single-org or flat mirror strategies.

Users who configured via env vars without GITHUB_USERNAME / GITEA_USERNAME
set (or who left those blank in the form, which is only client-side
`required`) ended up with empty strings in their config row. Mirroring
ran fine — tokens alone are sufficient — but the dashboard refused to
fetch and rendered all zeros because useConfigStatus failed the gate.

Drop the username checks from the gate. The `githubOwner` field is still
exported for consumers that want to display an owner; only the gate is
relaxed. Cache-hit and fresh-fetch branches both updated.
2026-04-22 19:21:13 +05:30
Arunavo Ray 083b342f38 ci: bump bun 1.3.10/1.3.12 → 1.3.13 across CI and runtime
CI was on 1.3.10 while the Dockerfile runtime moved to 1.3.12 in v3.15.2,
so we were testing against an older runtime than we shipped. Align both
on 1.3.13 (latest stable). May also resolve the intermittent --coverage
instrumentation flake observed on 1.3.10 against http-client.ts.
2026-04-22 08:39:37 +05:30
Arunavo Ray 92bb38b122 chore: bump version to 3.15.4 2026-04-22 08:11:17 +05:30
dependabot[bot] e7ac54a72a build(deps): bump astro (#274)
Bumps the npm_and_yarn group with 1 update in the /www directory: [astro](https://github.com/withastro/astro/tree/HEAD/packages/astro).


Updates `astro` from 6.0.4 to 6.1.6
- [Release notes](https://github.com/withastro/astro/releases)
- [Changelog](https://github.com/withastro/astro/blob/main/packages/astro/CHANGELOG.md)
- [Commits](https://github.com/withastro/astro/commits/astro@6.1.6/packages/astro)

---
updated-dependencies:
- dependency-name: astro
  dependency-version: 6.1.6
  dependency-type: direct:production
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-22 08:10:43 +05:30
Arunavo Ray 2ea250f081 fix: prefer active config when reading user settings (fixes #271)
Multiple "select from configs where userId" queries had no ORDER BY,
so when a user's database accidentally contained more than one config
row for the same user (e.g. from an env-loader insert path or a partial
default-config create), SQLite returned a non-deterministic row.

In the reported case this caused /api/config to hand back an empty stub
while /api/dashboard's repo/org counts came from the populated active
row. The dashboard's useConfigStatus hook then saw missing username/
token, treated config as incomplete, and never fetched dashboard data —
the UI rendered with all zeros even though 868 repos were sitting in
the database, mirroring fine in the background.

Add `ORDER BY isActive DESC, updatedAt DESC` before LIMIT 1 to every
"fetch the user's config" query so the active and most-recently-updated
row consistently wins. Also order env-config-loader's first-user pick
by createdAt for deterministic behavior across restarts.

Already-safe call sites that explicitly filter on isActive=true or
iterate all active configs (cleanup/scheduler/repositories/orgs/cleanup
trigger/sync-organization) are left unchanged.

Updates the mirror-repo test mock to match the new orderBy().limit()
chain.

Closes #271
2026-04-22 08:01:22 +05:30
Arunavo Ray c1712bc670 chore: bump version to 3.15.3 2026-04-20 13:03:28 +05:30
ARUNAVO RAY c4550196e9 fix: honor GH_API_URL across all Octokit call sites (#269) (#273)
* fix: honor GH_API_URL across all Octokit call sites

Six Octokit call sites constructed `new Octokit(...)` directly instead of
going through `createGitHubClient()`, so `GH_API_URL` (and the
`GITHUB_API_URL` fallback) only applied to the handful of flows that used
the helper. For GHES / GHEC-with-data-residency users this surfaced most
visibly as the "Test Connection" button hitting `api.github.com/user`
and failing with 401 even when `GH_API_URL` was set correctly (#269).

Route everything through `createGitHubClient()`:
- src/pages/api/github/test-connection.ts (the reported failure)
- src/pages/api/sync/repository.ts (public-repo sync)
- src/lib/gitea-enhanced.ts (force-push detection + metadata octokit)
- src/lib/scheduler-service.ts (auto-discovery, auto-mirror, auto-start)
- src/tests/test-metadata-mirroring.ts (dev harness, for consistency)

Side benefit: scheduler + sync paths now also get throttling, rate-limit
tracking, and the standard User-Agent, which they were missing.

`createGitHubClient`'s `token` parameter is made optional so the
public-repo sync path (`new Octokit()` with no auth) can keep working.

Fixes #269

* fix: address review findings

- scheduler: pass config.githubConfig?.owner (the real DB field) instead
  of ?.username, which doesn't exist on the DB row and was silently
  resolving to undefined — matches every other DB-reading call site.
- sync/repository.ts: revert to bare Octokit for the unauthenticated
  public-repo lookup to preserve fast-fail on the 60 req/hr limit.
  Still reads GH_API_URL / GITHUB_API_URL inline so GHES / GHEC
  data-residency users benefit. The throttling plugin's retry-with-
  backoff is wrong UX for a one-shot button click.
- github.ts: revert createGitHubClient token back to required (no
  remaining callers pass undefined after the above).
- gitea-enhanced.ts: make the leftover Octokit import type-only.
- test-connection.test.ts: replace mid-test mock.module re-call with a
  mutable stub reference — safer against ESM live-binding semantics.
2026-04-20 13:02:34 +05:30
Arunavo Ray 8cb8fd6fe1 docs: document GH_API_URL for GitHub Enterprise and SERVER_CERT_PATH/SERVER_KEY_PATH for native HTTPS
- README + env reference + .env.example now cover using GH_API_URL to
  target GitHub Enterprise Server or GHEC with data residency.
- Env reference + .env.example now cover SERVER_CERT_PATH and
  SERVER_KEY_PATH, which @astrojs/node reads at runtime to terminate
  TLS directly without a reverse proxy.

Closes #269
Closes #272
2026-04-20 09:30:08 +05:30
Arunavo Ray 4b4ea9614b chore: bump runtime and patch npm CVEs (v3.15.2)
Dockerfile:
- oven/bun 1.3.10 → 1.3.12
- Go 1.25.8 → 1.25.9 (fixes CVE-2026-32280/32281/32283 in git-lfs stdlib)

npm:
- drizzle-orm ^0.45.1 → ^0.45.2 (fixes CVE-2026-39356)
- override defu ^6.1.7 (fixes CVE-2026-35209)
- override @xmldom/xmldom ^0.8.12 (fixes CVE-2026-34601)
- override picomatch ^4.0.4 (fixes CVE-2026-33671, both instances)
- override kysely ^0.28.16 (fixes CVE-2026-33442, CVE-2026-33468)
- override lodash ^4.18.1 (fixes CVE-2026-4800)

Remaining unfixable alert: CVE-2026-27135 (libnghttp2) has no upstream
fix yet. System openssl CVEs are resolved by the Debian base image's
apt-get upgrade step.
2026-04-17 00:47:25 +05:30
Arunavo Ray 1644505043 chore: bump version to 3.15.1 2026-04-17 00:30:48 +05:30
ARUNAVO RAY e142524bfc fix: prevent duplicate issue/PR mirroring, enforce release retention, fix public repo auth, preserve env-loader backup settings
- Add metadata state guards for issues/PRs in syncGiteaRepoEnhanced to
  prevent re-mirroring on every sync (fixes #262)
- Fix httpPut → httpPatch for Gitea release updates (HTTP 405 error)
- Add paginated release retention cleanup to enforce releaseLimit by
  deleting the oldest excess releases (fixes #264)
- Always send auth credentials during migration for all repos, not just
  private ones, preventing "terminal prompts disabled" on Forgejo (fixes #263)
- Add missing service: "git" to org migration payload
- Make buildGithubSourceAuthPayload return {} instead of throwing when
  token is missing, allowing unconditional use
- Preserve existing backup settings in env-config-loader so UI-configured
  backup strategy and retention values survive restart (fixes #267)

Closes #262, #263, #264, #267
2026-04-17 00:30:15 +05:30
Arunavo Ray 8fac30fc02 docs: clarify BETTER_AUTH_URL should be origin only, not include base path
Update README, ENVIRONMENT_VARIABLES.md, and advanced docs page to
explicitly state that BETTER_AUTH_URL and PUBLIC_BETTER_AUTH_URL must be
origin only (scheme + host). The BASE_URL path prefix is applied
automatically — any path accidentally included is stripped.
2026-04-09 20:11:00 +05:30
Arunavo Ray c3b1f933b1 chore: bump version to 3.15.0 2026-04-09 12:45:24 +05:30
dependabot[bot] a839915e3e build(deps): bump vite (#261)
Bumps the npm_and_yarn group with 1 update in the /www directory: [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite).


Updates `vite` from 7.3.1 to 7.3.2
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v7.3.2/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v7.3.2/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 7.3.2
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-09 12:44:27 +05:30
dependabot[bot] 5d95f4cd39 build(deps): bump defu (#259)
Bumps the npm_and_yarn group with 1 update in the /www directory: [defu](https://github.com/unjs/defu).


Updates `defu` from 6.1.4 to 6.1.6
- [Release notes](https://github.com/unjs/defu/releases)
- [Changelog](https://github.com/unjs/defu/blob/main/CHANGELOG.md)
- [Commits](https://github.com/unjs/defu/compare/v6.1.4...v6.1.6)

---
updated-dependencies:
- dependency-name: defu
  dependency-version: 6.1.6
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-09 12:33:12 +05:30
ARUNAVO RAY 01a3b08dac feat: support reverse proxy path prefix deployments (#257)
* feat: support reverse proxy path prefixes

* fix: respect BASE_URL in SAML callback fallback

* fix: make BASE_URL runtime configurable
2026-04-09 12:32:59 +05:30
Arunavo Ray c87513b648 chore: bump version to 3.14.2 2026-03-27 13:55:56 +05:30
ARUNAVO RAY 4f3cbc866e fix private github mirror auth (#255) 2026-03-27 13:49:36 +05:30
ARUNAVO RAY 60548f2062 fix sync target resolution for mirrored repos (#249) 2026-03-27 12:33:59 +05:30
dependabot[bot] 74dab43e89 build(deps): bump picomatch (#251)
Bumps the npm_and_yarn group with 1 update in the /www directory: [picomatch](https://github.com/micromatch/picomatch).


Updates `picomatch` from 2.3.1 to 2.3.2
- [Release notes](https://github.com/micromatch/picomatch/releases)
- [Changelog](https://github.com/micromatch/picomatch/blob/master/CHANGELOG.md)
- [Commits](https://github.com/micromatch/picomatch/compare/2.3.1...2.3.2)

---
updated-dependencies:
- dependency-name: picomatch
  dependency-version: 2.3.2
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-27 09:39:17 +05:30
dependabot[bot] 01a8025140 build(deps): bump smol-toml (#250)
Bumps the npm_and_yarn group with 1 update in the /www directory: [smol-toml](https://github.com/squirrelchat/smol-toml).


Updates `smol-toml` from 1.6.0 to 1.6.1
- [Release notes](https://github.com/squirrelchat/smol-toml/releases)
- [Commits](https://github.com/squirrelchat/smol-toml/compare/v1.6.0...v1.6.1)

---
updated-dependencies:
- dependency-name: smol-toml
  dependency-version: 1.6.1
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-26 22:35:57 +05:30
Arunavo Ray 8346748f5a fix: move --accept-flake-config before -- in bun2nix step
The flag was being passed to bun2nix instead of nix, causing
"unexpected argument" error.
2026-03-24 08:22:04 +05:30
Arunavo Ray 38002019ea fix: regenerate bun.nix in CI to prevent stale dependency errors
The Nix build has been failing since v3.9.6 because bun.nix fell out
of sync with bun.lock. During the sandboxed build bun install cannot
fetch missing packages, causing ConnectionRefused errors.

- Add bun2nix regeneration step before nix build in CI
- Trigger workflow on bun.lock and package.json changes
- Update flake.nix version from 3.9.6 to 3.14.1
2026-03-24 08:20:26 +05:30
Arunavo Ray 32eb27c8a6 chore: bump version to 3.14.1 2026-03-24 07:35:36 +05:30
dependabot[bot] d33b4ff64f build(deps): bump h3 (#244)
Bumps the npm_and_yarn group with 1 update in the /www directory: [h3](https://github.com/h3js/h3).


Updates `h3` from 1.15.8 to 1.15.9
- [Release notes](https://github.com/h3js/h3/releases)
- [Changelog](https://github.com/h3js/h3/blob/v1.15.9/CHANGELOG.md)
- [Commits](https://github.com/h3js/h3/compare/v1.15.8...v1.15.9)

---
updated-dependencies:
- dependency-name: h3
  dependency-version: 1.15.9
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-24 07:34:40 +05:30
ARUNAVO RAY 6f2e0cbca0 Add GitHub starred-list filtering with searchable selector (#247)
* feat: add starred list filtering and selector UI

* docs: add starred lists UI screenshot

* lib: improve starred list name matching
2026-03-24 07:33:46 +05:30
dependabot[bot] 95e6eb7602 build(deps): bump h3 (#242)
Bumps the npm_and_yarn group with 1 update in the /www directory: [h3](https://github.com/h3js/h3).


Updates `h3` from 1.15.5 to 1.15.8
- [Release notes](https://github.com/h3js/h3/releases)
- [Changelog](https://github.com/h3js/h3/blob/main/CHANGELOG.md)
- [Commits](https://github.com/h3js/h3/compare/v1.15.5...v1.15.8)

---
updated-dependencies:
- dependency-name: h3
  dependency-version: 1.15.8
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-19 08:24:09 +05:30
Arunavo Ray f50f49fc41 chore: bump version to 3.14.0 2026-03-19 00:59:59 +05:30
ARUNAVO RAY 5ea2abff85 feat: custom sync start time and frequency scheduling (#241)
* feat: add custom sync start time scheduling

* Updated UI

* docs: add updated issue 240 UI screenshot

* fix: improve schedule UI with client-side next run calc and timezone handling

- Compute next scheduled run client-side via useMemo to avoid permanent
  "Calculating..." state when server hasn't set nextRun yet
- Default to browser timezone when enabling syncing (not UTC)
- Show actual saved timezone in badge, use it consistently in all handlers
- Match time input background to select trigger in dark mode
- Add clock icon to time picker with hidden native indicator
2026-03-19 00:58:10 +05:30
Arunavo Ray 9d131b9a09 fix security alerts 2026-03-18 20:10:45 +05:30
github-actions[bot] 5f77fceaca chore: sync version to 3.13.4 2026-03-18 13:15:13 +00:00
ARUNAVO RAY 5d2462e5a0 feat: add notification system with Ntfy.sh and Apprise support (#238)
* feat: add notification system with Ntfy.sh and Apprise providers (#231)

Add push notification support for mirror job events with two providers:

- Ntfy.sh: direct HTTP POST to ntfy topics with priority/tag support
- Apprise API: aggregator gateway supporting 100+ notification services

Includes database migration (0010), settings UI tab, test endpoint,
auto-save integration, token encryption, and comprehensive tests.
Notifications are fire-and-forget and never block the mirror flow.

* fix: address review findings for notification system

- Fix silent catch in GET handler that returned ciphertext to UI,
  causing double-encryption on next save. Now clears token to ""
  on decryption failure instead.
- Add Zod schema validation to test notification endpoint, following
  project API route pattern guidelines.
- Mark notifyOnNewRepo toggle as "coming soon" with disabled state,
  since the backend doesn't yet emit new_repo events. The schema
  and type support is in place for when it's implemented.

* fix notification gating and config validation

* trim sync notification details
2026-03-18 18:36:51 +05:30
ARUNAVO RAY 0000a03ad6 fix: improve reverse proxy support for subdomain deployments (#237)
* fix: improve reverse proxy support for subdomain deployments (#63)

- Add X-Accel-Buffering: no header to SSE endpoint to prevent Nginx
  from buffering the event stream
- Auto-detect trusted origin from Host/X-Forwarded-* request headers
  so the app works behind a proxy without manual env var configuration
- Add prominent reverse proxy documentation to advanced docs page
  explaining BETTER_AUTH_URL, PUBLIC_BETTER_AUTH_URL, and
  BETTER_AUTH_TRUSTED_ORIGINS are mandatory for proxy deployments
- Add reverse proxy env var comments and entries to both
  docker-compose.yml and docker-compose.alt.yml
- Add dedicated reverse proxy configuration section to .env.example

* fix: address review findings for reverse proxy origin detection

- Fix x-forwarded-proto multi-value handling: take first value only
  and validate it is "http" or "https" before using
- Update comment to accurately describe auto-detection scope: helps
  with per-request CSRF checks but not callback URL validation
- Restore startup logging of static trusted origins for debugging

* fix: handle multi-value x-forwarded-host in chained proxy setups

x-forwarded-host can be comma-separated (e.g. "proxy1.example.com,
proxy2.example.com") in chained proxy setups. Take only the first
value, matching the same handling already applied to x-forwarded-proto.

* test: add unit tests for reverse proxy origin detection

Extract resolveTrustedOrigins into a testable exported function and
add 11 tests covering:
- Default localhost origins
- BETTER_AUTH_URL and BETTER_AUTH_TRUSTED_ORIGINS env vars
- Invalid URL handling
- Auto-detection from x-forwarded-host + x-forwarded-proto
- Multi-value header handling (chained proxy setups)
- Invalid proto rejection (only http/https allowed)
- Deduplication
- Fallback to host header when x-forwarded-host absent
2026-03-18 15:47:15 +05:30
ARUNAVO RAY d697cb2bc9 fix: prevent starred repo name collisions during concurrent mirroring (#236)
* fix: prevent starred repo name collisions during concurrent mirroring (#95)

When multiple starred repos share the same short name (e.g. alice/dotfiles
and bob/dotfiles), concurrent batch mirroring could cause 409 Conflict
errors because generateUniqueRepoName only checked Gitea via HTTP, missing
repos that were claimed in the local DB but not yet created remotely.

Three fixes:
- Add DB-level check in generateUniqueRepoName so it queries the local
  repositories table for existing mirroredLocation claims, preventing two
  concurrent jobs from picking the same target name.
- Clear mirroredLocation on failed mirror so a failed repo doesn't falsely
  hold a location that was never successfully created, which would block
  retries and confuse the uniqueness check.
- Extract isMirroredLocationClaimedInDb helper for the DB lookup, using
  ne() to exclude the current repo's own record from the collision check.

* fix: address review findings for starred repo name collision fix

- Make generateUniqueRepoName immediately claim name by writing
  mirroredLocation to DB, closing the TOCTOU race window between
  name selection and the later status="mirroring" DB update
- Add fullName validation guard (must contain "/")
- Make isMirroredLocationClaimedInDb fail-closed (return true on
  DB error) to be conservative about preventing collisions
- Scope mirroredLocation clear on failure to starred repos only,
  preserving it for non-starred repos that may have partially
  created in Gitea and need the location for recovery

* fix: address P1/P2 review findings for starred repo name collision

P1a: Remove early name claiming from generateUniqueRepoName to prevent
stale claims on early return paths. The function now only checks
availability — the actual claim happens at the status="mirroring" DB
write (after both idempotency checks), which is protected by a new
unique partial index.

P1b: Add unique partial index on (userId, mirroredLocation) WHERE
mirroredLocation != '' via migration 0010. This enforces atomicity at
the DB level: if two concurrent workers try to claim the same name,
the second gets a constraint violation rather than silently colliding.

P2: Only clear mirroredLocation on failure if the Gitea migrate call
itself failed (migrateSucceeded flag). If migrate succeeded but
metadata mirroring failed, preserve the location since the repo
physically exists in Gitea and we need it for recovery/retry.
2026-03-18 15:27:20 +05:30
ARUNAVO RAY ddd071f7e5 fix: prevent excessive disk usage from repo backups (#235)
* fix: prevent excessive disk usage from repo backups (#234)

Legacy configs with backupBeforeSync: true but no explicit backupStrategy
silently resolved to "always", creating full git bundles on every sync
cycle. This caused repo-backups to grow to 17GB+ for users with many
repositories.

Changes:
- Fix resolveBackupStrategy to map backupBeforeSync: true → "on-force-push"
  instead of "always", so legacy configs only backup when force-push is detected
- Fix config mapper to always set backupStrategy explicitly ("on-force-push")
  preventing the backward-compat fallback from triggering
- Lower default backupRetentionCount from 20 to 5 bundles per repo
- Add time-based retention (backupRetentionDays, default 30 days) alongside
  count-based retention, with safety net to always keep at least 1 bundle
- Add "high disk usage" warning on "Always Backup" UI option
- Update docs and tests to reflect new defaults and behavior

* fix: preserve legacy backupBeforeSync:false on UI round-trip and expose retention days

P1: mapDbToUiConfig now checks backupBeforeSync === false before
defaulting backupStrategy, preventing legacy "disabled" configs from
silently becoming "on-force-push" after any auto-save round-trip.

P3: Added "Snapshot retention days" input field to the backup settings
UI, matching the documented setting in FORCE_PUSH_PROTECTION.md.
2026-03-18 15:05:00 +05:30
Arunavo Ray 4629ab4335 chore: bump version to 3.13.3 2026-03-18 05:20:21 +05:30
Arunavo Ray 0f303c4b79 nix: regenerate bun.nix 2026-03-18 04:47:16 +05:30
ARUNAVO RAY 7c7c259d0a fix repo links to use external gitea url (#233) 2026-03-18 04:36:14 +05:30
Arunavo Ray fe6bcc5288 chore: bump version to 3.13.2 2026-03-15 14:11:22 +05:30
ARUNAVO RAY e26ed3aa9c fix: rewrite migration 0009 for SQLite compatibility and add migration validation (#230)
SQLite rejects ALTER TABLE ADD COLUMN with expression defaults like
DEFAULT (unixepoch()), which Drizzle-kit generated for the imported_at
column. This broke upgrades from v3.12.x to v3.13.0 (#228, #229).

Changes:
- Rewrite migration 0009 using table-recreation pattern (CREATE, INSERT
  SELECT, DROP, RENAME) instead of ALTER TABLE
- Add migration validation script with SQLite-specific lint rules that
  catch known invalid patterns before they ship
- Add upgrade-path testing with seeded data and verification fixtures
- Add runtime repair for users whose migration record may be stale
- Add explicit migration validation step to CI workflow

Fixes #228
Fixes #229
2026-03-15 14:10:06 +05:30
Arunavo Ray efb96b6e60 chore: bump version to 3.13.1 2026-03-15 09:54:44 +05:30
Arunavo Ray 342cafed0e fix: force Go 1.25.8 toolchain and update x/crypto for git-lfs build
The git-lfs go.mod contains a `toolchain go1.25.3` directive which
causes Go to auto-download and use Go 1.25.3 instead of our installed
1.25.8. Set GOTOOLCHAIN=local to force using the installed version.

Also update golang.org/x/crypto to latest before building to resolve
CVE-2025-47913 (needs >= 0.43.0, was pinned at 0.36.0).
2026-03-15 09:35:50 +05:30
Arunavo Ray fc7c6b59d7 docs: update README to reference Gitea/Forgejo as supported targets 2026-03-15 09:33:41 +05:30
Arunavo Ray a77ec0447a chore: bump version to 3.13.0 2026-03-15 09:28:01 +05:30
Arunavo Ray 82b5ac8160 fix: build git-lfs from source with Go 1.25.8 to resolve remaining CVEs
Git-lfs v3.7.1 pre-built binaries use Go 1.25.3, which is affected by
CVE-2025-68121 (critical), CVE-2026-27142, CVE-2026-25679, CVE-2025-61729,
CVE-2025-61726, and CVE-2025-47913 (golang.org/x/crypto).

Since no newer git-lfs release exists, compile from source in a dedicated
build stage using Go 1.25.8 (latest patched release). Only the final
binary is copied into the runner image.
2026-03-15 09:22:50 +05:30
ARUNAVO RAY 299659eca2 fix: resolve CVEs, upgrade to Astro v6, and harden API security (#227)
* fix: resolve CVEs, upgrade to Astro v6, and harden API security

Docker image CVE fixes:
- Install git-lfs v3.7.1 from GitHub releases (Go 1.25) instead of
  Debian apt (Go 1.23.12), fixing CVE-2025-68121 and 8 other Go stdlib CVEs
- Strip build-only packages (esbuild, vite, rollup, svgo, tailwindcss)
  from production image, eliminating 9 esbuild Go stdlib CVEs

Dependency upgrades:
- Astro v5 → v6 (includes Vite 7, Zod 4)
- Remove legacy content config (src/content/config.ts)
- Update HealthResponse type for simplified health endpoint
- npm overrides for fast-xml-parser ≥5.3.6, devalue ≥5.6.2,
  node-forge ≥1.3.2, svgo ≥4.0.1, rollup ≥4.59.0

API security hardening:
- /api/auth/debug: dev-only, require auth, remove user-creation POST,
  strip trustedOrigins/databaseConfig from response
- /api/auth/check-users: return boolean hasUsers instead of exact count
- /api/cleanup/auto: require authentication, remove per-user details
- /api/health: remove OS version, memory, uptime from response
- /api/config: validate Gitea URL protocol (http/https only)
- BETTER_AUTH_SECRET: log security warning when using insecure defaults
- generateRandomString: replace Math.random() with crypto.getRandomValues()
- hashValue: add random salt and timing-safe verification

* repositories: migrate table to tanstack

* Revert "repositories: migrate table to tanstack"

This reverts commit a544b29e6d.

* fixed lock file
2026-03-15 09:19:24 +05:30
ARUNAVO RAY 6f53a3ed41 feat: add importedAt-based repository sorting (#226)
* repositories: add importedAt sorting

* repositories: use tanstack table for repo list
2026-03-15 08:52:45 +05:30
ARUNAVO RAY 1bca7df5ab feat: import repo topics and description into Gitea (#224)
* lib: sync repo topics and descriptions

* lib: harden metadata sync for existing repos
2026-03-15 08:22:44 +05:30
Arunavo Ray b5210c3916 updated packages 2026-03-15 07:52:34 +05:30
ARUNAVO RAY 755647e29c scripts: add startup repair progress logs (#223) 2026-03-14 17:44:52 +05:30
dependabot[bot] 018c9d1a23 build(deps): bump devalue (#220) 2026-03-13 00:17:30 +05:30
Arunavo Ray c89011819f chore: sync version to 3.12.5 2026-03-07 07:00:30 +05:30
ARUNAVO RAY c00d48199b fix: gracefully handle SAML-protected orgs during GitHub import (#217) (#218) 2026-03-07 06:57:28 +05:30
ARUNAVO RAY de28469210 nix: refresh bun deps and ci flake trust (#216) 2026-03-06 12:31:51 +05:30
github-actions[bot] 0e2f83fee0 chore: sync version to 3.12.4 2026-03-06 05:10:04 +00:00
ARUNAVO RAY 1dd3dea231 fix preserve strategy fork owner routing (#215) 2026-03-06 10:15:47 +05:30
Arunavo Ray db783c4225 nix: reduce bun install CI stalls 2026-03-06 09:41:22 +05:30
github-actions[bot] 8a4716bdbd chore: sync version to 3.12.3 2026-03-06 03:35:40 +00:00
Arunavo Ray 9d37966c10 ci: only run nix flake check when nix files change 2026-03-06 09:03:32 +05:30
Arunavo Ray ac16ae56ea ci: increase workflow timeouts to 25m and upgrade CodeQL Action to v4 2026-03-06 08:55:11 +05:30
Arunavo Ray df3e665978 fix: bump Bun to 1.3.10 and harden startup for non-AVX CPUs (#213)
Bun 1.3.9 crashes with a segfault on CPUs without AVX support due to a
WASM IPInt bug (oven-sh/bun#27340), fixed in 1.3.10 via oven-sh/bun#26922.

- Bump Bun from 1.3.9 to 1.3.10 in Dockerfile, CI workflows, and packageManager
- Skip env config script when no GitHub/Gitea env vars are set
- Make startup scripts (env-config, recovery, repair) fault-tolerant so
  a crash in a non-critical script doesn't abort the entrypoint via set -e
2026-03-06 08:19:44 +05:30
github-actions[bot] 8a26764d2c chore: sync version to 3.12.2 2026-03-05 04:34:51 +00:00
ARUNAVO RAY ce365a706e ci: persist release version to main (#212) 2026-03-05 09:55:59 +05:30
ARUNAVO RAY be7daac5fb ci: automate release version from tag (#211) 2026-03-05 09:34:49 +05:30
dependabot[bot] e32b7af5eb build(deps): bump svgo (#210)
Bumps the npm_and_yarn group with 1 update in the /www directory: [svgo](https://github.com/svg/svgo).


Updates `svgo` from 4.0.0 to 4.0.1
- [Release notes](https://github.com/svg/svgo/releases)
- [Commits](https://github.com/svg/svgo/compare/v4.0.0...v4.0.1)

---
updated-dependencies:
- dependency-name: svgo
  dependency-version: 4.0.1
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-05 08:53:34 +05:30
ARUNAVO RAY d0693206c3 feat: selective starred repo mirroring with autoMirrorStarred toggle (#208)
* feat: add autoMirrorStarred toggle for selective starred repo mirroring (#205)

Add `githubConfig.autoMirrorStarred` (default: false) to control whether
starred repos are included in automatic mirroring operations. Manual
per-repo actions always work regardless of this toggle.

Bug fixes:
- Cleanup service no longer orphans starred repos when includeStarred is
  disabled (prevents data loss)
- First-boot auto-start now gates initial mirror behind autoMirror config
  (previously mirrored everything unconditionally)
- "Mirror All" button now respects autoMirrorStarred setting
- Bulk mirror and getAvailableActions now include pending-approval status

Changes span schema, config mapping, env loader, scheduler, cleanup
service, UI settings toggle, and repository components.

* fix: log activity when repos are auto-imported during scheduled sync

Auto-discovered repositories (including newly starred ones) were inserted
into the database without creating activity log entries, so they appeared
in the dashboard but not in the activity log.

* ci: set 10-minute timeout on all CI jobs
2026-03-04 08:22:44 +05:30
Arunavo Ray b079070c30 ci: also exclude helm/** from app CI workflows 2026-03-02 16:28:04 +05:30
Arunavo Ray e68e9c38a8 ci: skip app CI workflows for www-only changes
Add www/** to paths-ignore in astro-build-test, e2e-tests, and
nix-build workflows. docker-build and helm-test already use positive
path filters and were unaffected.
2026-03-02 16:25:54 +05:30
Arunavo Ray 534150ecf9 chore(www): update website content, fix build, add Helm/Nix install methods
- Update softwareVersion from 3.9.2 to 3.11.0
- Add Helm and Nix installation tabs to Getting Started section
- Fix Helm instructions to use local chart path (no published repo)
- Update Features section: add Metadata Preservation, Force-Push Protection, Git LFS Support
- Remove unused @radix-ui/react-icons import from Hero.tsx and dependency from package.json
- Update structured data featureList with newer capabilities
2026-03-02 16:23:32 +05:30
ARUNAVO RAY 98da7065e0 feat: smart force-push protection with backup strategies (#206)
* feat: smart force-push protection with backup strategies (#187)

Replace blunt `backupBeforeSync` boolean with `backupStrategy` enum
offering four modes: disabled, always, on-force-push (default), and
block-on-force-push. This dramatically reduces backup storage for large
mirror collections by only creating snapshots when force-pushes are
actually detected.

Detection works by comparing branch SHAs between Gitea and GitHub APIs
before each sync — no git cloning required. Fail-open design ensures
detection errors never block sync.

Key changes:
- Add force-push detection module (branch SHA comparison via APIs)
- Add backup strategy resolver with backward-compat migration
- Add pending-approval repo status with approve/dismiss UI + API
- Add block-on-force-push mode requiring manual approval
- Fix checkAncestry to only treat 404 as confirmed force-push
  (transient errors skip branch instead of false-positive blocking)
- Fix approve-sync to bypass detection gate (skipForcePushDetection)
- Fix backup execution to not be hard-gated by deprecated flag
- Persist backupStrategy through config-mapper round-trip

* fix: resolve four bugs in smart force-push protection

P0: Approve flow re-blocks itself — approve-sync now calls
syncGiteaRepoEnhanced with skipForcePushDetection: true so the
detection+block gate is bypassed on approved syncs.

P1: backupStrategy not persisted — added to both directions of the
config-mapper. Don't inject a default in the mapper; let
resolveBackupStrategy handle fallback so legacy backupBeforeSync
still works for E2E tests and existing configs.

P1: Backup hard-gated by deprecated backupBeforeSync — added force
flag to createPreSyncBundleBackup; strategy-driven callers and
approve-sync pass force: true to bypass the legacy guard.

P1: checkAncestry false positives — now only returns false for
404/422 (confirmed force-push). Transient errors (rate limits, 500s)
are rethrown so detectForcePush skips that branch (fail-open).

* test(e2e): migrate backup tests from backupBeforeSync to backupStrategy

Update E2E tests to use the new backupStrategy enum ("always",
"disabled") instead of the deprecated backupBeforeSync boolean.

* docs: add backup strategy UI screenshot

* refactor(ui): move Destructive Update Protection to GitHub config tab

Relocates the backup strategy section from GiteaConfigForm to
GitHubConfigForm since it protects against GitHub-side force-pushes.
Adds ShieldAlert icon to match other section header patterns.

* docs: add force-push protection documentation and Beta badge

Add docs/FORCE_PUSH_PROTECTION.md covering detection mechanism,
backup strategies, API usage, and troubleshooting. Link it from
README features list and support section. Mark the feature as Beta
in the UI with an outline badge.

* fix(ui): match Beta badge style to Git LFS badge
2026-03-02 15:48:59 +05:30
ARUNAVO RAYandClaude Opus 4.6 58e0194aa6 fix(nix): ensure absolute bundle path in pre-sync backup (#204)
* fix(nix): ensure absolute bundle path in pre-sync backup (#203)

Use path.resolve() instead of conditional path.isAbsolute() check to
guarantee bundlePath is always absolute before passing to git -C. On
NixOS, relative paths were interpreted relative to the temp mirror
clone directory, causing "No such file or directory" errors.

Closes #203

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(nix): ensure absolute bundle path in pre-sync backup (#203)

Use path.resolve() instead of conditional path.isAbsolute() check to
guarantee bundlePath is always absolute before passing to git -C. On
NixOS, relative paths were interpreted relative to the temp mirror
clone directory, causing "No such file or directory" errors.

Extract resolveBackupPaths() for testability. Bump version to 3.10.1.

Closes #203

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ci: drop macos matrix and only run nix build on main/tags

- Remove macos-latest from Nix CI matrix (ubuntu-only)
- Only run `nix build` on main branch and version tags, skip on PRs
- `nix flake check` still runs on all PRs for validation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 08:37:18 +05:30
Arunavo Ray 7864c46279 unused file 2026-03-01 08:06:11 +05:30
Arunavo RayandClaude Opus 4.6 e3970e53e1 chore: release v3.10.0
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 08:01:02 +05:30
ARUNAVO RAYandClaude Opus 4.6 be46cfdffa feat: add target organization to Add Repository dialog (#202)
* feat: add target organization field to Add Repository dialog

Allow users to specify a destination Gitea organization when adding a
single repository, instead of relying solely on the default mirror
strategy. The field is optional — when left empty, the existing strategy
logic applies as before.

Closes #200

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add screenshot of target organization field in Add Repository dialog

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 07:55:27 +05:30
Xyndra 2e00a610cb Add E2E testing (#201)
* feat: add E2E testing infrastructure with fake GitHub, Playwright, and CI workflow

- Add fake GitHub API server (tests/e2e/fake-github-server.ts) with
  management API for seeding test data
- Add Playwright E2E test suite covering full mirror workflow:
  service health checks, user registration, config, sync, verify
- Add Docker Compose for E2E Gitea instance
- Add orchestrator script (run-e2e.sh) with cleanup
- Add GitHub Actions workflow (e2e-tests.yml) with Gitea service container
- Make GITHUB_API_URL configurable via env var for testing
- Add npm scripts: test:e2e, test:e2e:ci, test:e2e:keep, test:e2e:cleanup

* feat: add real git repos + backup config testing to E2E suite

- Create programmatic test git repos (create-test-repos.ts) with real
  commits, branches (main, develop, feature/*), and tags (v1.0.0, v1.1.0)
- Add git-server container to docker-compose serving bare repos via
  dumb HTTP protocol so Gitea can actually clone them
- Update fake GitHub server to emit reachable clone_url fields pointing
  to the git-server container (configurable via GIT_SERVER_URL env var)
- Add management endpoint POST /___mgmt/set-clone-url for runtime config
- Update E2E spec with real mirroring verification:
  * Verify repos appear in Gitea with actual content
  * Check branches, tags, commits, file content
  * Verify 4/4 repos mirrored successfully
- Add backup configuration test suite:
  * Enable/disable backupBeforeSync config
  * Toggle blockSyncOnBackupFailure
  * Trigger re-sync with backup enabled and verify activities
  * Verify config persistence across changes
- Update CI workflow to use docker compose (not service containers)
  matching the local run-e2e.sh approach
- Update cleanup.sh for git-repos directory and git-server port
- All 22 tests passing with real git content verification

* refactor: split E2E tests into focused files + add force-push tests

Split the monolithic e2e.spec.ts (1335 lines) into 5 focused spec files
and a shared helpers module:

  helpers.ts                 — constants, GiteaAPI, auth, saveConfig, utilities
  01-health.spec.ts          — service health checks (4 tests)
  02-mirror-workflow.spec.ts — full first-mirror journey (8 tests)
  03-backup.spec.ts          — backup config toggling (6 tests)
  04-force-push.spec.ts      — force-push simulation & backup verification (9 tests)
  05-sync-verification.spec.ts — dynamic repos, content integrity, reset (5 tests)

The force-push tests are the critical addition:
  F0: Record original state (commit SHAs, file content)
  F1: Rewrite source repo history (simulate force-push)
  F2: Sync to Gitea WITHOUT backup
  F3: Verify data loss — LICENSE file gone, README overwritten
  F4: Restore source, re-mirror to clean state
  F5: Enable backup, force-push again, sync through app
  F6: Verify Gitea reflects the force-push
  F7: Verify backup system was invoked (snapshot activities logged)
  F8: Restore source repo for subsequent tests

Also added to helpers.ts:
  - GiteaAPI.getBranch(), .getCommit(), .triggerMirrorSync()
  - getRepositoryIds(), triggerMirrorJobs(), triggerSyncRepo()

All 32 tests passing.

* Try to fix actions

* Try to fix the other action

* Add debug info to check why e2e action is failing

* More debug info

* Even more debug info

* E2E fix attempt #1

* E2E fix attempt #2

* more debug again

* E2E fix attempt #3

* E2E fix attempt #4

* Remove a bunch of debug info

* Hopefully fix backup bug

* Force backups to succeed
2026-03-01 07:35:13 +05:30
Arunavo RayandClaude Opus 4.6 61841dd7a5 chore: release v3.9.6
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 12:45:25 +05:30
ARUNAVO RAYandClaude Opus 4.6 5aa0f3260d fix(nix): enable sandboxed builds with bun2nix (#199)
* fix(nix): enable sandboxed builds with bun2nix

The Nix package was broken on Linux because `bun install` requires
network access, which is blocked by Nix sandboxing (enabled by default
on Linux).

This switches to bun2nix for dependency management:
- Add bun2nix flake input to pre-fetch all npm dependencies
- Generate bun.nix lockfile for reproducible dependency resolution
- Copy bun cache to writable location during build to avoid EACCES
  errors from bunx writing to the read-only Nix store
- Add nanoid as an explicit dependency (was imported directly but only
  available as a transitive dep, which breaks with isolated linker)
- Update CI workflow to perform a full sandboxed build
- Add bun2nix to devShell for easy lockfile regeneration

Closes #197

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(nix): create writable workdir for database access

The app uses process.cwd()/data for the database path, but when running
from the Nix store the cwd is read-only. Create a writable working
directory with symlinks to app files and a real data directory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 12:43:37 +05:30
ARUNAVO RAYandClaude Opus 4.6 d0efa200d9 fix(docker): add git and git-lfs to runner image (#198)
The runner stage was missing git, causing pre-sync backups to fail with
"Executable not found in $PATH: git". The backup feature (enabled by
default) shells out to git for clone --mirror and bundle create.

Closes #196

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 11:12:35 +05:30
Arunavo Ray c26b5574e0 chore: release v3.9.5 2026-02-26 11:02:00 +05:30
ARUNAVO RAY 89a6372565 nix: fix runtime wrapper paths and startup script packaging (#194)
* nix: fix flake module and runtime scripts

* docs: refresh readme and docs links/examples
2026-02-26 10:59:56 +05:30
ARUNAVO RAY f40cad4713 nix: fix flake module and runtime scripts (#192) 2026-02-26 10:39:50 +05:30
ARUNAVO RAY 855906d990 auth: clarify invalid origin error toast guidance (#193)
* nix: fix flake module and runtime scripts

* auth: clarify invalid origin toast
2026-02-26 10:39:08 +05:30
ARUNAVO RAY 08da526ddd fix(github): keep disabled repos from cleanup while skipping new imports (#191)
* fix: preserve disabled repos while skipping new imports

* ci: upgrade bun to 1.3.6 for test workflow
2026-02-26 10:19:28 +05:30
ARUNAVO RAY 2395e14382 Add pre-sync snapshot protection for mirror rewrites (#190)
* add pre-sync snapshot protection

* stabilize test module mocks

* fix cross-test gitea mock exports

* fix gitea mock strategy behavior
2026-02-26 10:13:13 +05:30
Arunavo Ray 91c1703bb5 chore: release v3.9.4 2026-02-24 11:47:47 +05:30
ARUNAVO RAY 6a548e3dac security: enforce session-derived user identity on API routes (#186)
* security: enforce session user on api routes

* test: harden auth guard failure path
2026-02-24 11:47:29 +05:30
Arunavo Ray f28ac8fa09 chore: release v3.9.3 2026-02-24 11:21:48 +05:30
Arunavo Ray 5e86670a5b Verify dockerignore and gh security 2026-02-24 11:19:17 +05:30
ARUNAVO RAY 62d43df2ad Merge pull request #184 from RayLabsHQ/codex/issue-165-incremental-metadata
Implement incremental issue and PR metadata sync
2026-02-24 10:51:26 +05:30
Arunavo Ray cb7510f79d Improve legacy matching and comment dedupe 2026-02-24 10:48:40 +05:30
ARUNAVO RAY 08c6302bf6 Merge pull request #185 from RayLabsHQ/codex/deps-upgrade-app-www
Upgrade dependencies across app and www
2026-02-24 10:47:57 +05:30
Arunavo Ray 6e6c3fa124 Upgrade dependencies for app and www 2026-02-24 10:46:18 +05:30
Arunavo Ray 85b1867490 Implement incremental issue and PR metadata sync 2026-02-24 10:43:35 +05:30
ARUNAVO RAY 545a575e1a Merge pull request #183 from RayLabsHQ/codex/issue-154-external-gitea-url
Add optional external Gitea URL for UI links
2026-02-24 10:34:50 +05:30
Arunavo Ray ef13fefb69 Add optional external Gitea URL for UI links 2026-02-24 10:33:37 +05:30
ARUNAVO RAY ed59849392 Merge pull request #182 from RayLabsHQ/codex/issue-168-interval-description-toast
Fix issue 168: repo descriptions and toast overlap
2026-02-24 10:29:55 +05:30
Arunavo Ray 5eb160861d Move app toasts to top center 2026-02-24 10:29:02 +05:30
Arunavo Ray 6829bcff91 Fix issue 168 description and toast placement 2026-02-24 10:25:53 +05:30
ARUNAVO RAY b1ca8c46bf Merge pull request #181 from RayLabsHQ/codex/issue-171-sync-reporting
Clarify mirror sync status and token-rotation troubleshooting
2026-02-24 10:12:40 +05:30
ARUNAVO RAY 888089b2d5 Merge branch 'main' into codex/issue-171-sync-reporting 2026-02-24 10:12:24 +05:30
Arunavo Ray fb60449dc2 Clarify mirror sync status reporting 2026-02-24 10:03:35 +05:30
ARUNAVO RAY 25854b04f9 Merge pull request #180 from RayLabsHQ/codex/issue-170-docs
Add one-click Re-run Metadata bulk action
2026-02-24 10:00:51 +05:30
Arunavo Ray c34056555f Add bulk re-run metadata action 2026-02-24 09:59:21 +05:30
Arunavo Ray f4074a37ad Document metadata re-sync recovery steps 2026-02-24 09:56:02 +05:30
ARUNAVO RAY 6146d41197 Merge pull request #179 from RayLabsHQ/codex/issue-157-docs
Document large-repo initial sync scheduling guidance
2026-02-24 09:53:50 +05:30
Arunavo Ray 4cce5b7cfe Document large-repo scheduling guidance 2026-02-24 09:51:54 +05:30
ARUNAVO RAY bc89b17a4c Merge pull request #178 from RayLabsHQ/codex/issue-172
Add admin CLI password reset command
2026-02-24 09:48:46 +05:30
Arunavo Ray d023b255a7 Add admin CLI password reset flow 2026-02-24 09:45:06 +05:30
ARUNAVO RAY 71cc961f5c Merge pull request #177 from RayLabsHQ/codex/issue-176
Support release limits above 100
2026-02-24 09:34:50 +05:30
Arunavo Ray 9bc7bbe33f Support release limits above 100 2026-02-24 09:23:50 +05:30
ARUNAVO RAY 6cc03364fb Merge pull request #162 from RayLabsHQ/fix/issue-161-status-sync
fix: ensure correct open/closed status when mirroring issues (#161)
2026-02-24 09:14:19 +05:30
ARUNAVO RAY d623d81a44 Merge pull request #167 from RayLabsHQ/dependabot/npm_and_yarn/www/npm_and_yarn-1b92d517bd
build(deps): bump devalue from 5.5.0 to 5.6.2 in /www in the npm_and_yarn group across 1 directory
2026-02-24 09:13:29 +05:30
ARUNAVO RAY 5cc4dcfb29 Merge pull request #174 from tasarren/feat/starred-by-org
Allow starred repos to be mirrored preserving structure
2026-02-24 08:49:43 +05:30
Arunavo Ray 893fae27d3 Skip GHCR push for fork PRs 2026-02-24 08:46:05 +05:30
Arunavo Ray 29051f3503 Polish starred destination UI copy 2026-02-24 08:42:50 +05:30
Arunavo Ray 0a3ad4e7f5 Fix retry mirror strategy fallback 2026-02-24 08:31:16 +05:30
Tobeas Arren f4d391b240 Allow starred repos to be mirrored preserving structure 2026-02-14 13:08:41 +01:00
dependabot[bot] 8280c6b337 build(deps): bump devalue
Bumps the npm_and_yarn group with 1 update in the /www directory: [devalue](https://github.com/sveltejs/devalue).


Updates `devalue` from 5.5.0 to 5.6.2
- [Release notes](https://github.com/sveltejs/devalue/releases)
- [Changelog](https://github.com/sveltejs/devalue/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sveltejs/devalue/compare/v5.5.0...v5.6.2)

---
updated-dependencies:
- dependency-name: devalue
  dependency-version: 5.6.2
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-15 18:55:49 +00:00
Arunavo Ray bebbda9465 fix: ensure correct open/closed status when mirroring issues (#161)
Add verification and explicit close fallback after issue creation to
  handle Gitea API's two-step close process that can fail silently.
2025-12-25 14:06:20 +05:30
Arunavo Ray 2496d6f6e0 Disabled promo banner 2025-12-24 08:44:19 +05:30
ARUNAVO RAY 179083aec4 Merge pull request #160 from RayLabsHQ/dependabot/npm_and_yarn/www/npm_and_yarn-d9d6a1cc67
build(deps): bump the npm_and_yarn group across 1 directory with 2 updates
2025-12-17 10:50:05 +05:30
dependabot[bot] aa74984fb0 build(deps): bump the npm_and_yarn group across 1 directory with 2 updates
Bumps the npm_and_yarn group with 2 updates in the /www directory: [js-yaml](https://github.com/nodeca/js-yaml) and [mdast-util-to-hast](https://github.com/syntax-tree/mdast-util-to-hast).


Updates `js-yaml` from 4.1.0 to 4.1.1
- [Changelog](https://github.com/nodeca/js-yaml/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodeca/js-yaml/compare/4.1.0...4.1.1)

Updates `mdast-util-to-hast` from 13.2.0 to 13.2.1
- [Release notes](https://github.com/syntax-tree/mdast-util-to-hast/releases)
- [Commits](https://github.com/syntax-tree/mdast-util-to-hast/compare/13.2.0...13.2.1)

---
updated-dependencies:
- dependency-name: js-yaml
  dependency-version: 4.1.1
  dependency-type: indirect
  dependency-group: npm_and_yarn
- dependency-name: mdast-util-to-hast
  dependency-version: 13.2.1
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-17 05:15:31 +00:00
ARUNAVO RAY 18ab4cd53a Merge pull request #144 from RayLabsHQ/nix
Nix
2025-12-17 10:44:16 +05:30
Arunavo Ray e94bb86b61 fix: skip nix build in CI (sandbox blocks network access)
bun install requires network access which Nix sandbox blocks.
CI now validates flake structure and evaluates expressions only.

Full builds work locally with: nix build --option sandbox false
2025-12-17 10:34:03 +05:30
Arunavo Ray 3993d679e6 fix: replace Cachix with Magic Nix Cache in CI workflow
- Use DeterminateSystems/nix-installer-action for Nix installation
- Use DeterminateSystems/magic-nix-cache-action for caching (free, no setup)
- Update documentation to remove Cachix references
- Add nix branch to CI triggers
2025-12-17 10:30:07 +05:30
Arunavo Ray 83cae16319 added promo banner for Lumical 2025-12-05 22:11:48 +05:30
Arunavo Ray 99ebe1a400 updated www packages 2025-12-04 20:56:15 +05:30
ARUNAVO RAY 204d803937 Merge pull request #158 from RayLabsHQ/upgrade-packages
upgraded packages
2025-12-04 19:28:02 +05:30
Arunavo Ray 2a08ae0b21 updated package manger for debian 2025-12-04 18:44:36 +05:30
Arunavo Ray 8dc7ae8bfc moved to debian image 2025-12-04 18:34:42 +05:30
Arunavo Ray a4dbb49006 upgraded packages 2025-12-04 17:59:43 +05:30
Arunavo Ray 6531a9325d Added Faq 2025-11-10 10:49:34 +05:30
Arunavo Ray ff44f0e537 v3.9.2 2025-11-08 11:06:04 +05:30
ARUNAVO RAY dec34fc384 Merge pull request #151 from RayLabsHQ/148-release-order-fix-for-same-created-date
fix: Sort releases by published_at instead of created_at
2025-11-08 11:05:00 +05:30
ARUNAVO RAY f5727daedb fix: Sort releases by published_at instead of created_at 2025-11-08 10:59:22 +05:30
ARUNAVO RAY 3857f2fd1a Merge pull request #150 from RayLabsHQ/148-release-order-again
fix: Detect and recreate incorrectly ordered releases (#148)
2025-11-08 09:21:47 +05:30
Arunavo Ray e951e97790 fix: Detect and recreate incorrectly ordered releases (#148) 2025-11-08 08:59:19 +05:30
ARUNAVO RAY d0cade633a Merge pull request #149 from RayLabsHQ/bun-v1.3.1
updated packages | dockerfile
2025-11-08 08:14:22 +05:30
Arunavo Ray 490059666f updated packages | dockerfile 2025-11-08 07:58:04 +05:30
Arunavo Ray 5852bb00f2 updated og-image 2025-11-08 07:50:20 +05:30
Arunavo Ray 749ad4a694 v3.9.0 2025-11-06 07:47:23 +05:30
ARUNAVO RAY 0f752acae5 Merge pull request #146 from RayLabsHQ/fix/issue-129-release-sort-order
fix: Ensure proper release ordering in Gitea mirrors (#129)
2025-11-06 07:44:12 +05:30
Arunavo Ray 652bd220c2 added missing favicon 2025-11-06 07:43:14 +05:30
Arunavo Ray 9f2eaaf04e fix: Ensure proper release ordering in Gitea mirrors (#129)
- Add 1-second delays between release creations to ensure distinct timestamps
- Prepend GitHub original publication date to release notes
- Improve logging to show chronological processing order
- Addresses Gitea API limitation where created_unix is always set to current time

Fixes #129
2025-11-05 20:57:33 +05:30
ARUNAVO RAY 63d3f0e86c Merge pull request #145 from z0xca/main
fix website install guide command
2025-11-05 11:53:22 +05:30
z0x 25e7d234ba Update clone command to use '&&' for chaining 2025-11-04 22:58:23 -05:00
ARUNAVO RAYandCopilot Autofix powered by AI 9968775210 Potential fix for code scanning alert no. 39: Workflow does not contain permissions
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-11-03 16:43:42 +05:30
Arunavo Ray 0d63fd4dae Added more docs 2025-10-31 09:22:55 +05:30
Arunavo Ray 109958342d updated docs 2025-10-31 09:17:28 +05:30
Arunavo Ray 491546a97c added basic nix pack 2025-10-31 09:00:18 +05:30
ARUNAVO RAY 7a3f734728 Merge pull request #142 from RayLabsHQ/fix/issue-141-duplicate-issues-on-sync
fix: add metadata field to repositories table to prevent duplicate issues on sync
2025-10-31 08:51:34 +05:30
Arunavo Ray d59a07a8c5 fix: add metadata field to repositories table to prevent duplicate issues on sync
Fixes #141

The repository metadata field was missing from the database schema, which
caused the metadata sync state (issues, PRs, releases, etc.) to not persist.
This resulted in duplicate issues being created every time a repository was
synced because the system couldn't track what had already been mirrored.

Changes:
- Added metadata text field to repositories table in schema
- Added metadata field to repositorySchema Zod validation
- Generated database migration 0008_serious_thena.sql

Root cause analysis:
1. Code tried to read/write repository.metadata to track mirrored components
2. The metadata field didn't exist in the database schema
3. On sync, metadataState.components.issues was always false
4. This triggered re-mirroring of all issues, creating duplicates

The fix ensures metadata state persists between mirrors and syncs, preventing
duplicate metadata (issues, PRs, releases) from being created in Gitea.
2025-10-30 10:58:48 +05:30
Arunavo Ray 5a77ae5084 v3.8.10 2025-10-30 10:54:56 +05:30
ARUNAVO RAY dcb5bd80e3 Merge pull request #138 from RayLabsHQ/issue-132-org-repo-duplicates 2025-10-30 07:11:34 +05:30
Arunavo Ray 3b8fc99f06 workaround to get rid of unknown/unknown in OS arch 2025-10-29 22:01:40 +05:30
Arunavo Ray bda8d10f10 ci: build arm64 images in PR pipeline 2025-10-29 21:51:37 +05:30
Arunavo Ray 0fe7b433d6 added missing hero_logo.png 2025-10-27 19:43:00 +05:30
Arunavo Ray 8d96e176b4 fix: prevent duplicate orgs and repos 2025-10-27 08:44:45 +05:30
Arunavo Ray af9bc861cf fixed: Sort order in releases #129 2025-10-27 07:54:38 +05:30
ARUNAVO RAY ab4bbea9fd Merge pull request #136 from RayLabsHQ/fix/metadata-sync-config-change
fix: sync metadata after config toggles
2025-10-27 07:45:12 +05:30
ARUNAVO RAY fbd4b3739e Merge pull request #137 from RayLabsHQ/docs/authentik-oidc-notes
Added basic docs on SSO/OIDC
2025-10-26 19:54:53 +05:30
Arunavo Ray 395e71164f Added basic docs on SSO/OIDC 2025-10-26 19:52:44 +05:30
Arunavo Ray 99c277e2ee v3.8.10 | Fixed SSO issues 2025-10-26 19:06:36 +05:30
ARUNAVO RAY 9287e0d29b Merge pull request #135 from RayLabsHQ/fix/authentik-issuer-mismatch
auth: preserve issuer formatting for OIDC
2025-10-26 19:05:54 +05:30
Arunavo Ray f2f2bafc39 "better-auth": "1.4.0-beta.13" 2025-10-26 18:37:06 +05:30
Arunavo Ray 5876198b5e Added missing DB fields 2025-10-26 18:36:20 +05:30
Arunavo Ray e46bf381c7 auth: trust email verification from sso providers 2025-10-26 08:45:47 +05:30
Arunavo Ray 3bf0ccf207 fix: sync metadata after config toggles 2025-10-26 08:41:28 +05:30
Arunavo Ray e41b4ffc56 auth: preserve issuer formatting for OIDC 2025-10-26 07:49:42 +05:30
Arunavo Ray a9dd646573 v3.8.9 2025-10-25 09:04:14 +05:30
ARUNAVO RAY e2160aabcd Merge pull request #130 from bwees/main 2025-10-25 07:24:41 +05:30
Brandon Wees 5d085e02bf fix: rename repo count in dashboard 2025-10-24 15:45:29 -05:00
ARUNAVO RAY 3f17dd038f Merge pull request #128 from RayLabsHQ/fix/chronological-metadata-ordering
fix: preserve chronological issue mirroring
2025-10-24 09:17:34 +05:30
Arunavo Ray 921ab948a1 updated env vars for ci comment 2025-10-24 08:49:08 +05:30
Arunavo Ray e7a102ee45 mirror: show github timestamps in metadata 2025-10-24 08:42:14 +05:30
Arunavo Ray 025df12bef Set defaults to 3 and 5 for Issue and PR concurrency 2025-10-24 08:39:52 +05:30
Arunavo Ray 60913a9f4d Added Agents.md 2025-10-24 07:57:30 +05:30
Arunavo Ray 985c7e061c updated README 2025-10-24 07:43:13 +05:30
Arunavo Ray 4d75d3514f docs: document sequential metadata defaults 2025-10-24 07:39:08 +05:30
Arunavo Ray 5245d67f37 fix: enforce sequential metadata mirroring 2025-10-24 07:35:40 +05:30
Arunavo Ray 2cd7d911ed ci: mention env vars in pr image comment 2025-10-23 23:21:16 +05:30
Arunavo Ray 1c2391ea2e docs: expose concurrency env vars in compose 2025-10-23 23:19:00 +05:30
Arunavo Ray 190e786449 ci: update docker test port guidance 2025-10-23 23:16:46 +05:30
Arunavo Ray fb27ddfee5 fix: preserve chronological issue mirroring 2025-10-23 23:08:32 +05:30
Arunavo Ray fd5e68c1d4 docs: update development workflow and documentation index
Updated development documentation to reflect current project structure
and simplified setup process.

Changes:
- DEVELOPMENT_WORKFLOW.md: Updated repository URL, simplified setup steps,
  improved project structure documentation, and clarified command descriptions
- README.md: Reorganized as a concise index of available guides, removed
  redundant content now covered in main README and in-app help
- SHUTDOWN_PROCESS.md: Removed (content consolidated into GRACEFUL_SHUTDOWN.md)

These updates make the documentation more accurate and easier to navigate
for new contributors.
2025-10-23 05:10:42 +05:30
Arunavo Ray ea22df1296 docs: improve vendor-lock-in-prevention article accuracy
Updated the vendor lock-in prevention article with more accurate UI navigation
paths and technical details to match current application features.

Changes:
- Added missing title heading
- Updated navigation instructions to reference actual UI paths (Configuration → Connections, Content & Data, Automation)
- Improved technical accuracy of sync interval recommendations
- Added FAQ section with practical questions about auto-discovery, sync intervals, and cutover scripting
- Clarified activity log and API endpoint usage for monitoring

These changes ensure the article reflects the current application UI and features.
2025-10-23 05:09:31 +05:30
Arunavo Ray 080ad5deb4 fix: correct Helm chart port from 8080 to 4321
Updated Helm chart configuration to use the correct application port (4321)
instead of the incorrect default (8080). This aligns the Helm deployment
with the actual application configuration.

Changes:
- helm/gitea-mirror/values.yaml: Updated service.port and deployment.port to 4321
- helm/gitea-mirror/README.md: Updated all port references in documentation
- www/src/pages/use-cases/deploy-with-helm-chart.mdx: Fixed article to reflect
  correct port, added GitHub links to Helm chart, and improved installation instructions

The application runs on port 4321 as defined in:
- Dockerfile (ENV PORT=4321, EXPOSE 4321)
- docker-compose.yml (4321:4321 mapping)
- .env.example (PORT=4321)

Tested with local Kubernetes cluster and confirmed the application is accessible
on port 4321.
2025-10-23 05:06:38 +05:30
ARUNAVO RAY 71245cf56e Remove duplicate section in README.md
Removed duplicate 'Star History' section from README.
2025-10-23 04:09:59 +05:30
ARUNAVO RAY 1ccf670f81 Revise Star History chart links and parameters
Updated Star History section with new parameters for the image sources.
2025-10-23 04:08:39 +05:30
Arunavo Ray cb266b9af0 fixed inaccuracies in articles steps 2025-10-23 00:04:58 +05:30
Arunavo Ray fa5f7da5c4 updated article 2025-10-22 23:53:30 +05:30
Arunavo Ray 3c808eb0c0 updated www articles 2025-10-22 23:47:26 +05:30
Arunavo Ray 5e37c3bb84 www: updated packages 2025-10-22 23:18:55 +05:30
ARUNAVO RAY 847e94ca28 Merge pull request #111 from RayLabsHQ/www-seo
Writing a few guides on the application
2025-10-22 23:14:26 +05:30
Arunavo Ray da497d54c8 Updated screenshots 2025-10-22 19:55:56 +05:30
Arunavo Ray 79e0086a72 sso: polyfill buffer for browser 2025-10-22 18:13:36 +05:30
Arunavo Ray dc340666ef v3.8.7 2025-10-22 17:54:25 +05:30
ARUNAVO RAY 8b50a07c68 Merge pull request #124 from RayLabsHQ/fix/sso-stability
Fix/sso stability
2025-10-22 17:54:01 +05:30
Arunavo Ray 7dab4fb1d5 sso: drop legacy migration helper 2025-10-22 16:34:37 +05:30
Arunavo Ray 847823bbf8 sso: normalize provider config via discovery 2025-10-22 16:33:33 +05:30
Arunavo Ray e4e54722cf auth: bump better auth beta 2025-10-22 16:28:59 +05:30
Arunavo Ray 1eddbad908 v3.8.6 2025-10-22 14:56:05 +05:30
ARUNAVO RAY a7083beff5 Merge pull request #123 from RayLabsHQ/issue-84-archive-loop-fix
Issue 84 archive loop fix
2025-10-22 14:55:25 +05:30
Arunavo Ray b21cd0b866 ui: align cleanup retention controls 2025-10-22 14:49:09 +05:30
Arunavo Ray df644be769 cleanup: improve orphaned repo handling 2025-10-22 14:41:54 +05:30
Arunavo Ray 204869fa3e downgraded bun 2025-10-22 13:46:30 +05:30
Arunavo Ray e470256475 v3.8.5 2025-10-22 09:40:32 +05:30
ARUNAVO RAY b65c360d61 Merge pull request #121 from RayLabsHQ/fix-release-sync
fix: mirror releases during sync
2025-10-22 09:36:53 +05:30
Arunavo Ray ce46d33d29 updated packages 2025-10-22 09:36:41 +05:30
Arunavo Ray f63633f97e fix: mirror releases during sync 2025-10-22 09:33:13 +05:30
Arunavo Ray 3b53a29e71 v3.8.4 2025-10-22 09:07:02 +05:30
ARUNAVO RAY 64e73f9ca8 Merge pull request #120 from RayLabsHQ/fix-org-destination-override
fix: Custom organization mirror destination issue
2025-10-22 09:06:13 +05:30
Arunavo Ray 7d23894e5f fix: Custom organization mirror destination issue 2025-10-22 09:03:27 +05:30
ARUNAVO RAY 8f2a4683c1 Merge pull request #119 from RayLabsHQ/fix/duplicate-repos-issue-115
Fix/duplicate repos issue 115
2025-10-22 08:59:07 +05:30
Arunavo Ray b5323ff8b4 chore: remove duplicate fix summary 2025-10-22 08:44:22 +05:30
Arunavo Ray 7fee2adb51 lib: harden mirror idempotency 2025-10-22 08:43:38 +05:30
ARUNAVO RAY af139ecb2d Merge pull request #114 from RayLabsHQ/dependabot/npm_and_yarn/www/npm_and_yarn-82a5d800d0
Bump astro from 5.14.1 to 5.14.3 in /www in the npm_and_yarn group across 1 directory
2025-10-11 11:40:53 +05:30
dependabot[bot] fb827724b6 Bump astro in /www in the npm_and_yarn group across 1 directory
Bumps the npm_and_yarn group with 1 update in the /www directory: [astro](https://github.com/withastro/astro/tree/HEAD/packages/astro).


Updates `astro` from 5.14.1 to 5.14.3
- [Release notes](https://github.com/withastro/astro/releases)
- [Changelog](https://github.com/withastro/astro/blob/main/packages/astro/CHANGELOG.md)
- [Commits](https://github.com/withastro/astro/commits/astro@5.14.3/packages/astro)

---
updated-dependencies:
- dependency-name: astro
  dependency-version: 5.14.3
  dependency-type: direct:production
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-10-11 00:40:10 +00:00
ARUNAVO RAY 2812b576d0 Merge pull request #112 from V-Paranoiaque/fix-custom-image-tag
Fix when using a custom tag
2025-10-05 18:22:47 +05:30
Virgil R. 347188f43d Fix when using a custom tag 2025-10-04 14:52:42 +02:00
Arunavo Ray beda2ce66c Added a few more articles 2025-10-03 13:15:24 +05:30
Arunavo Ray 21e2f4717c Added basic use cases pages 2025-10-03 12:35:04 +05:30
Arunavo Ray b8dea1ee9c removed unused docs 2025-10-03 12:21:47 +05:30
Arunavo Ray b27ff817f7 Updated SEO plan 2025-10-03 11:05:55 +05:30
Arunavo Ray 56bee451de [www] : updated packages 2025-10-03 09:53:33 +05:30
Arunavo RayandClaude 0e9d54b517 fix: add backward compatibility for skipStarredIssues field
Since githubConfig is stored as JSON in the database (not individual columns),
no database migration is needed. However, we need to handle reading old configs
that still use the 'skipStarredIssues' field name.

Changes:
- Added skipStarredIssues as optional field in Zod schema (marked deprecated)
- Updated config mapper to check both starredCodeOnly and skipStarredIssues
- Old configs will continue to work seamlessly
- New configs will use starredCodeOnly field name

This ensures zero-downtime upgrades for existing installations.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-03 09:44:38 +05:30
Arunavo Ray 7a04665b70 v3.8.3 2025-10-03 09:24:28 +05:30
Arunavo RayandClaude 3a3ff314e0 refactor: rename skipStarredIssues to starredCodeOnly
The previous name 'skipStarredIssues' was misleading as it now skips ALL
metadata (not just issues) for starred repositories. The new name
'starredCodeOnly' better reflects the actual behavior - mirroring only
source code for starred repos.

Changes:
- Renamed skipStarredIssues → starredCodeOnly in all files
- Updated UI label from "Don't fetch issues" to "Code-only mode"
- Updated description to clarify it skips ALL metadata types:
  issues, PRs, labels, milestones, wiki, and releases
- Updated database schema, types, config mapper, and all components
- Updated Helm charts, CI configs, and documentation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-03 09:22:18 +05:30
Arunavo RayandClaude fed74ee901 fix: apply skipStarredIssues flag to releases mirroring
Extended the skipStarredIssues flag to also skip releases for starred repos
when "code-only" mode is enabled. Previously, releases were still being
mirrored even when lightweight mode was selected.

Now starred repos with "code-only" mode will skip:
- Issues ✓
- Pull requests ✓
- Labels ✓
- Milestones ✓
- Wiki ✓
- Releases ✓ (this fix)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-03 09:18:35 +05:30
Arunavo RayandClaude 85ea502276 fix: apply skipStarredIssues flag to all metadata types
Previously, the skipStarredIssues flag (Lightweight mode for starred repos)
only applied to issues. This caused starred repos to mirror all metadata
(pull requests, labels, milestones, wiki) even when "code-only" was selected.

Fixed by applying the skipStarredIssues check to:
- Pull requests mirroring
- Labels mirroring
- Milestones mirroring
- Wiki mirroring (in migration payload)

Now starred repos with "code-only" mode truly mirror only source code,
skipping all metadata as intended.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-03 09:13:58 +05:30
Arunavo Ray ffb7bd3cb0 v3.8.2 2025-10-02 21:19:32 +05:30
ARUNAVO RAY b39d7a2179 Merge pull request #107 from RayLabsHQ/fix/issue-97-migration-duplicates
fix: resolve migration 0005 duplicate constraint failure (#97)
2025-10-02 07:45:42 +05:30
Arunavo Ray bf99a95dc6 ci: add more paths to trigger Docker builds
- Added docker-entrypoint.sh to trigger paths
- Added drizzle/** for database migrations
- Added scripts/** for database management scripts
- Added src/** for source code changes

This ensures Docker images are rebuilt when critical runtime
files change, not just package dependencies.
2025-10-01 08:02:56 +05:30
Arunavo Ray 2ea917fdaa fix: resolve migration 0005 duplicate constraint failure (#97)
**Problem:**
- Users upgrading to v3.7.2 encountered database migration failures
- Migration 0005 tried to add unique index without handling existing duplicates
- Hybrid initialization (manual SQL + Drizzle) caused schema inconsistencies
- Error: "UNIQUE constraint failed: repositories.user_id, repositories.full_name"

**Solution:**

1. **Fixed Migration 0005:**
   - Added deduplication step before creating unique index
   - Removes duplicate (user_id, full_name) entries, keeping most recent
   - Safely creates unique constraint after cleanup

2. **Removed Hybrid Database Initialization:**
   - Eliminated 154 lines of manual SQL from docker-entrypoint.sh
   - Now uses Drizzle exclusively for schema management
   - Single source of truth prevents schema drift
   - Migrations run automatically via src/lib/db/index.ts

**Testing:**
- ✅ Fresh database initialization works
- ✅ Duplicate deduplication verified
- ✅ Unique constraint properly enforced
- ✅ All 6 migrations apply cleanly

**Changes:**
- docker-entrypoint.sh: Removed manual table creation SQL
- drizzle/0005_polite_preak.sql: Added deduplication before index creation

Fixes #97
2025-10-01 07:57:16 +05:30
Arunavo Ray b841057f1a updated packages 2025-10-01 07:29:39 +05:30
ARUNAVO RAY d588ce91b4 Merge pull request #106 from RayLabsHQ/alert-autofix-33
Potential fix for code scanning alert no. 33: Workflow does not contain permissions
2025-10-01 07:06:58 +05:30
ARUNAVO RAYandCopilot Autofix powered by AI 553396483e Potential fix for code scanning alert no. 33: Workflow does not contain permissions
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-10-01 07:06:06 +05:30
ARUNAVO RAY ebeabdb4fc Merge pull request #105 from RayLabsHQ/fix/forgejo-12-private-repos
fix: Forgejo 12 compatibility - use separate auth fields for private repos (#102)
2025-10-01 07:02:30 +05:30
ARUNAVO RAY ff209a6376 Merge pull request #101 from V-Paranoiaque/helm-chart
Helm chart to deploy on Kubernetes
2025-10-01 07:01:19 +05:30
Arunavo Ray 096e0c03ac images now tagged lowercase in comments 2025-09-30 23:43:27 +05:30
Virgil R. 63f20a7f04 Update helm-test.yml 2025-09-30 20:07:48 +02:00
Arunavo Ray 34f741beef fix: Forgejo 12 compatibility - use separate auth fields for private repos (#102)
## Problem
Forgejo 12.0+ rejects migration API calls with credentials embedded in URLs,
causing HTTP 422 errors when mirroring private GitHub repositories.

## Root Cause
Breaking security change in Forgejo 12.0 (July 2025) enforces credential
separation to prevent accidental exposure in logs/errors. Previous versions
(Forgejo 11.x, Gitea 1.x) accepted embedded credentials.

## Solution
- Use separate `auth_username` and `auth_token` fields instead of embedding
  credentials in clone URLs
- Set `auth_username` to "oauth2" for GitHub token authentication
- Pass GitHub token via `auth_token` field

## Changes
- src/lib/gitea.ts:
  - mirrorGithubRepoToGitea(): Use separate auth fields for private repos
  - mirrorGitHubRepoToGiteaOrg(): Use separate auth fields for private repos

- .github/workflows/docker-build.yml:
  - Enable PR image building and pushing to GHCR
  - Tag PR images as pr-<number> for easy testing
  - Add automated PR comment with image details and testing instructions
  - Separate load step for security scanning

## Backward Compatibility
✅ Works with Forgejo 12.0+
✅ Works with Forgejo 11.x and earlier
✅ Works with Gitea 1.x

## Testing
Public repos: ✅ Working (no auth needed)
Private repos: ✅ Fixed (separate auth fields)

Fixes #102
2025-09-30 23:12:33 +05:30
V-Paranoiaque 1f98f441f3 Fix ingress + improve testing 2025-09-27 18:28:10 +02:00
V-Paranoiaque 9c1ac76ff9 Fix annotations 2025-09-27 15:18:52 +02:00
V-Paranoiaque cf5027bafc Fix CLEANUP_RETENTION_DAYS 2025-09-27 15:15:42 +02:00
V-Paranoiaque 6fd2774d43 Fix MIRROR_STARRED var 2025-09-27 15:13:54 +02:00
V-Paranoiaque 8f379baad4 Improve CI/CD 2025-09-27 10:34:20 +02:00
V-Paranoiaque 91fa3604b6 Add some basic CICD for testing 2025-09-27 10:24:18 +02:00
Virgil R. c0fff30fcb Create README.md 2025-09-27 10:12:29 +02:00
Virgil R. 18de63d192 Update deployment.yaml 2025-09-27 10:04:42 +02:00
V-Paranoiaque 1fe20c3e54 GITHUB_TYPE env var 2025-09-24 22:07:15 +02:00
V-Paranoiaque 7386b54a46 Fix env vars 2025-09-24 21:12:16 +02:00
V-Paranoiaque 432a2bc54d Add missing podSecurityContext 2025-09-24 20:47:54 +02:00
V-Paranoiaque f9d18f34ab Fix image tag 2025-09-24 20:41:24 +02:00
V-Paranoiaque cd86a09bbd Minor fixes 2025-09-24 20:29:14 +02:00
V-Paranoiaque 1e2c1c686d Add volumes 2025-09-21 21:15:50 +02:00
V-Paranoiaque f701574e67 Move NODE_ENV to the configmap 2025-09-21 15:15:58 +02:00
V-Paranoiaque 4528be8cc6 Too many spaces 2025-09-21 12:29:02 +02:00
V-Paranoiaque 80fd43ef42 More features 2025-09-21 12:26:26 +02:00
V-Paranoiaque 3c52fe58aa Missing space 2025-09-21 10:23:48 +02:00
V-Paranoiaque 319e7925ff First commit TBT 2025-09-20 22:59:22 +02:00
Arunavo Ray 5add8766a4 fix(scheduler,config): preserve ENV schedule; add AUTO_MIRROR_REPOS auto-mirroring
- Prevent Automation UI from overriding schedule:
      - mapDbScheduleToUi now parses intervals robustly (cron/duration/seconds) via parseInterval
      - mapUiScheduleToDb merges with existing config and stores interval as seconds (no lossy cron conversion)
      - /api/config passes existing scheduleConfig to preserve ENV-sourced values
      - schedule-sync endpoint uses parseInterval for nextRun calculation
  - Add AUTO_MIRROR_REPOS support and scheduled auto-mirror phase:
      - scheduleConfig schema includes autoImport and autoMirror
      - env-config-loader reads AUTO_MIRROR_REPOS and carries through to DB
      - scheduler auto-mirrors imported/pending/failed repos when autoMirror is enabled before regular sync
      - docker-compose and ENV docs updated with AUTO_MIRROR_REPOS
  - Tests pass and build succeeds
2025-09-14 08:31:31 +05:30
Arunavo Ray 6ce70bb5bf chore(version): bump to 3.7.1\n\ncleanup: attempt fix for orphaned repo archiving (refs #84)\n- Sanitize mirror rename to satisfy AlphaDashDot; timestamped fallback\n- Resolve Gitea owner robustly via mirroredLocation/strategy; verify presence\n- Add 'archived' status to Zod enums; set isArchived on archive\n- Update CHANGELOG entry without closing keyword 2025-09-14 07:53:36 +05:30
Arunavo Ray f3aae2ec94 fix for repo name collison 2025-09-14 00:13:13 +05:30
Arunavo Ray 46d5ec46fc Updated deisgn for 'Duplicate collision strategy' 2025-09-13 23:54:14 +05:30
Arunavo Ray 0caa53b67f v3.7.0 2025-09-13 23:39:50 +05:30
Arunavo Ray 18ecdbc252 fix(sync): batch inserts + normalize nulls to avoid SQLite param mismatch
- Batch repository inserts with dynamic sizing under SQLite 999-param limit
- Normalize undefined → null to keep multi-row insert shapes consistent
- De-duplicate owned + starred repos by fullName (prefer starred variant)
- Enforce uniqueness via (user_id, full_name) + onConflictDoNothing
- Handle starred name collisions (suffix/prefix) across mirror + metadata
- Add repo-utils helpers + tests; guard Octokit.plugin in tests
- Remove manual unique index from entrypoint; rely on drizzle-kit migrations
2025-09-13 23:38:50 +05:30
Arunavo Ray 51a6c8ca58 Added product hunt badge on website 2025-09-12 01:44:13 +05:30
Arunavo Ray 41b8806268 update packages 2025-09-10 09:49:08 +05:30
ARUNAVO RAY ac5c7800c1 Merge pull request #93 from RayLabsHQ/dependabot/npm_and_yarn/www/npm_and_yarn-73ea615029
Bump vite from 6.3.5 to 6.3.6 in /www in the npm_and_yarn group across 1 directory
2025-09-10 09:46:02 +05:30
dependabot[bot] 13e7661f07 Bump vite in /www in the npm_and_yarn group across 1 directory
Bumps the npm_and_yarn group with 1 update in the /www directory: [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite).


Updates `vite` from 6.3.5 to 6.3.6
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v6.3.6/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v6.3.6/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 6.3.6
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-09-10 02:49:30 +00:00
Arunavo Ray 37e5b68bd5 Added Github API rate limiting
- Implemented comprehensive GitHub API rate limit handling:
    - Integrated @octokit/plugin-throttling for automatic retry with exponential backoff
    - Added RateLimitManager service to track and enforce rate limits
    - Store rate limit status in database for persistence across restarts
    - Automatic pause and resume when limits are exceeded
    - Proper user identification for 5000 req/hr authenticated limit (vs 60 unauthenticated)

  - Improved rate limit UI/UX:
    - Removed intrusive rate limit card from dashboard
    - Toast notifications only at critical thresholds (80% and 100% usage)
    - All rate limit events logged for debugging

  - Optimized for GitHub's API constraints:
    - Reduced default batch size from 10 to 5 repositories
    - Added documentation about GitHub's 100 concurrent request limit
    - Better handling of repositories with many issues/PRs
2025-09-09 11:14:43 +05:30
Arunavo Ray 89ca5abe7d fix: resolve SQLite field mismatch for large starred repo imports (#90)
- Add missing database fields (language, description, mirroredLocation, destinationOrg) to repository operations
  - Add missing organization fields (publicRepositoryCount, privateRepositoryCount, forkRepositoryCount) to schema
  - Update GitRepo interface to include all required database fields
  - Fix GitHub data fetching functions to map all fields correctly
  - Update all sync endpoints (main, repository, organization, scheduler) to handle new fields

  This fixes the "SQLite query expected X values, received Y" error when importing
  large numbers (4.6k+) of starred repositories by ensuring all database fields
  are properly mapped from GitHub API responses through to database insertion.
2025-09-09 09:56:18 +05:30
Arunavo Ray 2b78a6a4a8 v3.5.4 2025-09-07 19:11:50 +05:30
Arunavo Ray c2f6e73054 Testing Authentik SSO Issues 2025-09-07 19:09:00 +05:30
Arunavo Ray c4b353aae8 Added docs around scheduling using corn 2025-09-07 16:51:51 +05:30
Arunavo Ray 4a54cf9009 v3.5.3 2025-09-07 16:29:43 +05:30
Arunavo Ray fab4efd93a Auto-start on boot 2025-09-07 16:29:23 +05:30
Arunavo Ray 9f21cd6b1a Addressing concerns of Issue #85 and #86 2025-09-07 15:25:48 +05:30
Arunavo Ray 9ef6017a23 v3.5.2 2025-09-07 13:55:43 +05:30
Arunavo Ray 502796371f Attempt to address #84 2025-09-07 13:55:20 +05:30
Arunavo Ray b956b71c5f Fixed #87 where the Release Notes was missing 2025-09-07 13:14:41 +05:30
Arunavo Ray 26b82e0f65 Added AGENTS.md 2025-09-07 11:46:14 +05:30
Arunavo Ray 7c124a37d7 v3.5.1 2025-08-30 00:47:59 +05:30
Arunavo Ray 3e14edc571 fixed default overide 2025-08-30 00:47:33 +05:30
Arunavo Ray a188869cae "Automatic Mirroring" changed to "Automatic Syncing" 2025-08-30 00:37:56 +05:30
Arunavo Ray afac3b5ddc UI tweek 2025-08-29 21:16:19 +05:30
Arunavo Ray 2ce4bb4373 update env doc 2025-08-29 20:43:49 +05:30
Arunavo Ray 5c9a3afaae updates to auth url 2025-08-29 20:43:25 +05:30
Arunavo Ray de4e111095 type fix 2025-08-29 20:42:56 +05:30
Arunavo Ray 8c4d9508c7 Add provider modal optimised 2025-08-29 19:17:40 +05:30
Arunavo Ray 921eb5e07d util 2025-08-29 19:08:48 +05:30
Arunavo Ray ac1b09f7a1 UI updates 2025-08-29 19:08:39 +05:30
Arunavo Ray 9ee67ce77d made time more user readable 2025-08-29 18:32:22 +05:30
Arunavo Ray 92db61a2c9 v3.5.0 2025-08-29 18:11:49 +05:30
Arunavo Ray cbf6e11de3 Env var updates 2025-08-29 18:11:26 +05:30
Arunavo Ray 18855f09c4 Imporved a bunch of things in Mirror and sync Automation 2025-08-29 17:49:44 +05:30
Arunavo Ray b8965a9fd4 v3.4.0 2025-08-29 17:06:38 +05:30
Arunavo Ray 598e81ff45 updated package location 2025-08-29 17:04:48 +05:30
Arunavo Ray fef6cbb60d toast showing full name now 2025-08-29 17:01:48 +05:30
Arunavo Ray c793be5863 closed and merged pull requests will be created as closed issues 2025-08-29 16:58:48 +05:30
Arunavo Ray d097ded6ee Updates to PR as issues 2025-08-29 16:54:21 +05:30
Arunavo Ray 1b01a5e653 updated docs 2025-08-28 20:11:16 +05:30
Arunavo Ray 56988818d2 removed unused package-lock.json 2025-08-28 20:04:20 +05:30
ARUNAVO RAY 5a49726b0e Merge pull request #82 from RayLabsHQ/dependabot/npm_and_yarn/www/npm_and_yarn-b7812215fd
Bump the npm_and_yarn group across 1 directory with 2 updates
2025-08-28 20:00:10 +05:30
dependabot[bot] 987c4ec3ec Bump the npm_and_yarn group across 1 directory with 2 updates
Bumps the npm_and_yarn group with 2 updates in the /www directory: [devalue](https://github.com/sveltejs/devalue) and [esbuild](https://github.com/evanw/esbuild).


Updates `devalue` from 5.1.1 to 5.3.2
- [Release notes](https://github.com/sveltejs/devalue/releases)
- [Changelog](https://github.com/sveltejs/devalue/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sveltejs/devalue/compare/v5.1.1...v5.3.2)

Updates `esbuild` from 0.25.6 to 0.25.9
- [Release notes](https://github.com/evanw/esbuild/releases)
- [Changelog](https://github.com/evanw/esbuild/blob/main/CHANGELOG.md)
- [Commits](https://github.com/evanw/esbuild/compare/v0.25.6...v0.25.9)

---
updated-dependencies:
- dependency-name: devalue
  dependency-version: 5.3.2
  dependency-type: indirect
  dependency-group: npm_and_yarn
- dependency-name: esbuild
  dependency-version: 0.25.9
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-08-28 14:25:41 +00:00
Arunavo Ray 444442fcca updated packages www 2025-08-28 19:53:48 +05:30
ARUNAVO RAY 3fe2461031 Merge pull request #80 from RayLabsHQ/address-Issues
Address issues
2025-08-28 19:51:16 +05:30
Arunavo Ray ea7777a20f spacing 2025-08-28 19:51:00 +05:30
Arunavo Ray a3247c9c22 Removed icon 2025-08-28 19:46:19 +05:30
Arunavo Ray 099bf7d36f added details 2025-08-28 19:14:27 +05:30
Arunavo Ray 10a14d88ef updates 2025-08-28 19:01:39 +05:30
Arunavo Ray 36f8d41d38 Updated PR as issues 2025-08-28 17:54:38 +05:30
Arunavo Ray dd19131029 added default values 2025-08-28 15:49:20 +05:30
Arunavo Ray be5f2e6c3d config 2025-08-28 15:46:05 +05:30
Arunavo Ray d9bfc59a2d Added eye/eye-off icon toggle for password field 2025-08-28 14:55:42 +05:30
Arunavo Ray 29a08ee3e3 fixed the TypeError in the config mapper functions 2025-08-28 13:59:25 +05:30
Arunavo Ray b425cbce71 fixed the security vulnerability CVE-2025-57820 in the devalue package 2025-08-28 13:53:04 +05:30
Arunavo Ray f54a7e6d71 update default configs 2025-08-28 13:45:49 +05:30
Arunavo Ray d49599ff05 Org ignore 2025-08-28 13:27:10 +05:30
Arunavo Ray d99f597988 Update the Ignore Repo 2025-08-28 12:58:58 +05:30
Arunavo Ray 7dfb6b5d18 updated status to use badges 2025-08-28 11:26:28 +05:30
Arunavo Ray 46e6b4b927 Dashboard minor UI update 2025-08-28 11:21:51 +05:30
Arunavo Ray 8bd3b8d3b1 Added redirect to /login 2025-08-28 10:50:18 +05:30
Arunavo Ray 78be49d4a7 Added BETA tag to LFS feature 2025-08-28 10:49:27 +05:30
Arunavo Ray c58bde1cc3 updated astro 2025-08-28 10:31:08 +05:30
Arunavo Ray b4a2a14dd3 Fixed CVE issue 2025-08-28 10:25:42 +05:30
Arunavo Ray 3fb71b666d Updated dockerfile bun 2025-08-28 09:27:41 +05:30
Arunavo Ray e404490e75 added LFS ENV var 2025-08-28 09:26:23 +05:30
Arunavo Ray b3856b4223 More tsc issues 2025-08-28 08:34:41 +05:30
Arunavo Ray ad7418aef2 tsc issues 2025-08-28 08:34:27 +05:30
Arunavo Ray 389f8dd292 packages updated 2025-08-28 07:18:34 +05:30
Arunavo Ray 067b5d8ccd updated handling of url's from ENV vars 2025-08-28 07:12:13 +05:30
Arunavo Ray 6127a916f4 fixed tests 2025-08-27 21:54:40 +05:30
Arunavo Ray 12ee065833 Docs updated | added some options 2025-08-27 21:43:36 +05:30
Arunavo Ray 926737f1c5 Added a few new features. 2025-08-27 20:33:41 +05:30
Arunavo Ray fe94d97779 Issue 68 2025-08-27 20:06:42 +05:30
Arunavo Ray 38a0d1b494 repository cleanup functionality 2025-08-27 19:12:52 +05:30
Arunavo RayandClaude 698eb0b507 fix: Complete Issue #72 - Fix automatic mirroring and repository cleanup
Major fixes for Docker environment variable issues and cleanup functionality:

🔧 **Duration Parser & Scheduler Fixes**
- Add comprehensive duration parser supporting "8h", "30m", "24h" formats
- Fix GITEA_MIRROR_INTERVAL environment variable mapping to scheduler
- Auto-enable scheduler when GITEA_MIRROR_INTERVAL is set
- Improve scheduler logging to clarify timing behavior (from last run, not startup)

🧹 **Repository Cleanup Service**
- Complete repository cleanup service for orphaned repos (unstarred, deleted)
- Fix cleanup configuration logic - now works with CLEANUP_DELETE_IF_NOT_IN_GITHUB=true
- Auto-enable cleanup when deleteIfNotInGitHub is enabled
- Add manual cleanup trigger API endpoint (/api/cleanup/trigger)
- Support archive/delete actions with dry-run mode and protected repos

🐛 **Environment Variable Integration**
- Fix scheduler not recognizing GITEA_MIRROR_INTERVAL=8h
- Fix cleanup requiring both CLEANUP_DELETE_FROM_GITEA and CLEANUP_DELETE_IF_NOT_IN_GITHUB
- Auto-enable services when relevant environment variables are set
- Better error logging and debugging information

📚 **Documentation Updates**
- Update .env.example with auto-enabling behavior notes
- Update ENVIRONMENT_VARIABLES.md with clarified functionality
- Add comprehensive tests for duration parsing

This resolves the core issues where:
1. GITEA_MIRROR_INTERVAL=8h was not working for automatic mirroring
2. Repository cleanup was not working despite CLEANUP_DELETE_IF_NOT_IN_GITHUB=true
3. Users had no visibility into why scheduling/cleanup wasn't working

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-20 11:06:21 +05:30
Arunavo RayandClaude 0fb5f9e190 Release v3.2.6 - Add release asset mirroring and metadata debugging
### Fixed
- Added missing release asset mirroring functionality (APK, ZIP, Binary files)
- Release assets (attachments) are now properly downloaded from GitHub and uploaded to Gitea
- Fixed missing metadata component configuration checks

### Added
- Full support for mirroring release assets/attachments
- Debug logging for metadata component configuration to help troubleshoot mirroring issues
- Download and upload progress logging for release assets

### Improved
- Enhanced release mirroring to include all associated binary files and attachments
- Better visibility into which metadata components are enabled/disabled
- More detailed logging during the release asset transfer process

Fixes #68

Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-09 19:23:23 +05:30
Arunavo Ray dacec93f55 Release v3.2.5 - Complete fix for releases mirroring authentication
This patch completes the authentication fixes from v3.2.4, specifically addressing the releases mirroring function that was missed in the previous update.

Fixes:
- Critical authentication error in releases mirroring (encrypted token usage)
- Missing repository existence verification for releases
- "user does not exist [uid: 0]" error for GitHub releases sync

Improvements:
- Duplicate release detection to prevent errors
- Better error handling with per-release fault tolerance
- Enhanced logging with [Releases] prefix for debugging

Issue: #68
2025-08-09 18:23:26 +05:30
Arunavo Ray b41438f686 Release v3.2.4 - Fix metadata mirroring authentication issues
Fixed critical authentication issue causing "user does not exist [uid: 0]" errors during metadata mirroring operations. This release addresses Issue #68 and ensures proper authentication validation before all Gitea operations.

Key improvements:
- Pre-flight authentication validation for all Gitea operations
- Consistent token decryption across all API calls
- Repository existence verification before metadata operations
- Graceful fallback to user account when org creation fails
- Enhanced error messages with specific troubleshooting guidance
- Added diagnostic test scripts for authentication validation

This patch ensures metadata mirroring (issues, PRs, labels, milestones) works reliably without authentication errors.
2025-08-09 12:35:34 +05:30
Arunavo RayandClaude df1738a44d feat: comprehensive environment variable support
- Added support for 60+ environment variables covering all configuration options
- Created detailed documentation in docs/ENVIRONMENT_VARIABLES.md with tables
- Fixed missing skipStarredIssues field in GitHub config
- Updated docker-compose files to reference environment variable documentation
- Updated README to link to the new environment variables documentation
- Environment variables now populate UI configuration automatically on Docker startup
- Preserves manual UI changes when environment variables are not set
- Includes support for mirror metadata, scheduling, cleanup, and authentication options

Fixes #69

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-09 11:48:42 +05:30
Arunavo Ray afaac70bb8 chore: bump version to v3.2.2
Patch release including:
- Fix for issues #68 and #69
- Hero redesign improvements
- Mobile hero image support
2025-08-09 10:25:38 +05:30
ARUNAVO RAY da95c1d5fd Merge pull request #70 from RayLabsHQ/Fixes
Address Issue #68 and #69
2025-08-09 10:23:57 +05:30
Arunavo Ray 8dc50f7ebf Address Issue #68 and #69 2025-08-09 10:10:08 +05:30
ARUNAVO RAY eafc44d112 Merge pull request #67 from abhrajitray77/hero-redesign
🐧 Added hero image for mobile
2025-08-07 22:04:42 +05:30
abhrajitray77 25cff6fe8e 🐧 Added hero image for mobile 2025-08-07 22:03:15 +05:30
ARUNAVO RAY 29fe7ba895 Merge pull request #66 from abhrajitray77/hero-redesign
🔥 Added shader hero component
2025-08-07 20:12:03 +05:30
abhrajitray77 fbcedc404a 🔥 Added shader hero component 2025-08-07 20:10:46 +05:30
Arunavo Ray 122848c970 chore: bump version to v3.2.1
Patch release including:
- Updated favicon
- Added fallback for 3D Scene
- Updated packages
- Logo changes and optimizations
2025-08-06 10:05:41 +05:30
Arunavo Ray 4c15ecb1bf Updated favicon 2025-08-06 10:04:36 +05:30
Arunavo Ray 3209f70566 Added fallback for 3d Scene 2025-08-06 09:57:34 +05:30
Arunavo Ray 677bc0cb5b Updated Packages 2025-08-06 09:46:50 +05:30
ARUNAVO RAY 5693ae7822 Merge pull request #65 from abhrajitray77/hero-redesign 2025-08-06 00:53:23 +05:30
abhrajitray77 814be1e9d0 ✨logo changed for other areas 2025-08-05 21:04:37 +05:30
abhrajitray77 4e3c4c2c67 🐧New logo added 2025-08-05 20:57:01 +05:30
abhrajitray77 46d6374ff0 minor fix 2025-08-05 20:33:45 +05:30
ARUNAVO RAY 4cd98dffc4 Merge pull request #64 from abhrajitray77/hero-redesign
Hero redesign
2025-08-05 13:55:40 +05:30
abhrajitray77 87ca3bc12f 🐧cleanup 2025-08-05 13:27:09 +05:30
abhrajitray77 dd6554509c 🔥Spline object responsive 2025-08-05 13:21:31 +05:30
Arunavo Ray 55465197d1 Added SEO keywords 2025-08-05 12:25:28 +05:30
Arunavo Ray e255142e70 updated the docker file 2025-07-31 12:53:27 +05:30
Arunavo Ray f2b64a61b8 v3.2.0 2025-07-31 12:35:52 +05:30
ARUNAVO RAY 0fba2cecac Merge pull request #55 from RayLabsHQ/sso-fix
SSO Issues
2025-07-31 12:32:35 +05:30
Arunavo Ray 1aef433918 zod validation fix 2025-07-31 12:30:33 +05:30
ARUNAVO RAYandCopilot Autofix powered by AI 3f704ebb23 Potential fix for code scanning alert no. 28: Incomplete URL substring sanitization
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-07-28 15:34:20 +05:30
Arunavo Ray 5797b9bba1 test update 2025-07-28 08:45:47 +05:30
Arunavo Ray bb045b037b fix: update tests to work in CI environment
- Add http-client mocks to gitea-enhanced.test.ts for proper isolation
- Fix GiteaRepoInfo interface to handle owner as object or string
- Add gitea module mocks to gitea-starred-repos.test.ts
- Update test expectations to match actual function behavior
- Fix handleExistingNonMirrorRepo to properly extract owner from repoInfo

These changes ensure tests pass consistently in both local and CI environments
by properly mocking all dependencies and handling API response variations.
2025-07-27 22:03:44 +05:30
Arunavo Ray 1a77a63a9a fix: add config-encryption mocks to test files for CI compatibility
- Add config-encryption module mocks to gitea-enhanced.test.ts
- Add config-encryption module mocks to gitea-starred-repos.test.ts
- Update helpers mock in setup.bun.ts to include createEvent function

The CI environment was loading modules in a different order than local,
causing the config-encryption module to be accessed before it was mocked
in the global setup. Adding the mocks directly to the test files ensures
they are available regardless of module loading order.
2025-07-27 20:34:38 +05:30
Arunavo Ray 3a9b8380d4 fix: resolve CI test failures and timeouts
- Update Bun version in CI to match local version (1.2.16)
- Add bunfig.toml with 5s test timeout to prevent hanging tests
- Mock setTimeout globally in test setup to avoid timing issues
- Add NODE_ENV check to skip delays during tests
- Fix missing exports in config-encryption mock
- Remove retryDelay in tests to ensure immediate execution

These changes ensure tests run consistently between local and CI environments
2025-07-27 20:27:33 +05:30
Arunavo Ray 5d5429ac71 test fix 2025-07-27 20:19:47 +05:30
Arunavo Ray de314cf174 Fixed Tests 2025-07-27 19:09:56 +05:30
Arunavo Ray e637d573a2 Fixes 2025-07-27 00:25:19 +05:30
Arunavo Ray 5f45a9a03d updates 2025-07-26 22:06:29 +05:30
Arunavo Ray 0920314679 More fixes in SSO 2025-07-26 20:33:26 +05:30
Arunavo Ray 1f6add5fff Updates to SSO Testing 2025-07-26 19:45:20 +05:30
Arunavo Ray 3ff15a46e7 Fix TypeError 2025-07-26 17:08:13 +05:30
Arunavo Ray 465c812e7e Starred repos fix errors 2025-07-26 17:04:05 +05:30
Arunavo Ray 794ea52e4d Added claude Agents 2025-07-26 15:12:14 +05:30
Arunavo Ray 7b8ca7c3b8 v3.1.1 2025-07-23 06:52:35 +05:30
Arunavo RayandClaude f2c7728394 Release v3.1.0
### Added
- Support for GITHUB_EXCLUDED_ORGS environment variable
- New textarea UI component for configuration forms

### Fixed
- Mirror strategy configuration test failures
- Organization repository routing logic
- Starred repositories organization routing
- SSO and OIDC authentication issues

### Improved
- Organization configuration for repository routing
- Mirror strategy handling in tests
- Authentication error handling

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-21 16:46:36 +05:30
Arunavo Ray dcdb06ac39 fixed tests 2025-07-21 15:17:53 +05:30
Arunavo Ray 9fa10dae00 removed unused import 2025-07-21 12:30:04 +05:30
Arunavo Ray 99dd501a52 Added textarea comp 2025-07-21 12:26:31 +05:30
Arunavo Ray d4aa665873 more SSO and OIDC fixes 2025-07-21 12:09:38 +05:30
Arunavo Ray 0244133e7b Fix: Starred Repos Organization Bug | Organization Repos Routing 2025-07-21 10:39:48 +05:30
ARUNAVO RAY 6ea5e9efb0 Merge pull request #47 from djmango/master
Add GITHUB_EXCLUDED_ORGS support for organization filtering
2025-07-19 21:28:29 +05:30
Sulaiman Khan Ghori 8d7ca8dd8f Add GITHUB_EXCLUDED_ORGS support for organization filtering 2025-07-18 15:58:04 -07:00
Arunavo Ray 8d2919717f Removed old MIgration Guide 2025-07-19 00:36:46 +05:30
Arunavo Ray 1e06e2bd4b Remove Auto Migrate 2025-07-19 00:28:12 +05:30
Arunavo Ray 67080a7ce9 v3.0.1 2025-07-18 23:53:23 +05:30
Arunavo Ray 9d5db86bdf Updates to Org strategy 2025-07-18 09:52:55 +05:30
Arunavo Ray 3458891511 Updates for starred and personal repos 2025-07-18 09:37:38 +05:30
Arunavo Ray d388f2e691 consistent and distinct colors for status 2025-07-18 08:37:00 +05:30
Arunavo Ray 7bd862606b More fixes 2025-07-18 00:52:03 +05:30
Arunavo Ray 251baeb1aa Fixed Private Repo Issues 2025-07-17 23:46:01 +05:30
Arunavo Ray e6a31512ac some more fixes 2025-07-17 23:31:45 +05:30
Arunavo Ray 4430625319 updated lockfile 2025-07-17 17:00:21 +05:30
Arunavo Ray 236bef543b Update CHANGELOG for v3.0.0 release
- Add comprehensive v3.0.0 release notes
- Document breaking changes
- Detail new features: token encryption, SSO/OIDC, header auth
- Include migration requirements
2025-07-17 16:55:03 +05:30
Arunavo Ray 03bad9a0c0 Added Note 2025-07-17 16:50:18 +05:30
ARUNAVO RAY f60ccfc9f1 Merge pull request #44 from RayLabsHQ/v3
V3
2025-07-17 16:26:11 +05:30
Arunavo Ray cad77320f3 Added Header Authentication 2025-07-17 16:19:56 +05:30
Arunavo Ray 744064f3aa Updated SSO UI 2025-07-17 16:08:12 +05:30
Arunavo Ray f83711ecd6 Potential security fixes 2025-07-17 13:41:17 +05:30
Arunavo Ray bde1f7b5d6 Fix tests 2025-07-17 13:05:18 +05:30
Arunavo Ray 39bfb1e2d1 Migration updates 2025-07-17 12:29:53 +05:30
Arunavo Ray 2140f75436 v3 Migration Guide 2025-07-17 12:18:23 +05:30
Arunavo Ray a5a827c85f updates 2025-07-17 11:44:27 +05:30
ARUNAVO RAY 0af2626201 Create FUNDING.yml 2025-07-17 11:18:04 +05:30
Arunavo Ray d48981a8c4 Updated docs 2025-07-16 22:51:47 +05:30
Arunavo Ray 948250deef Updated some interfaces to be more future proof 2025-07-16 22:47:33 +05:30
Arunavo Ray beedbaf9a4 Added Encryptions to All stored token and passwords 2025-07-16 16:02:34 +05:30
Arunavo Ray 7cc4aa87f2 Fix for mirroring issues for starred repositories 2025-07-16 14:02:31 +05:30
Arunavo Ray ae41c4e2ea Updated README.md 2025-07-16 13:43:07 +05:30
Arunavo Ray 58c63095b1 Added Star History 2025-07-15 08:40:23 +05:30
Arunavo Ray 938a909787 tsc fixes 2025-07-11 01:17:54 +05:30
Arunavo Ray fad78516ef Added SSO and OIDC 2025-07-11 01:04:50 +05:30
Arunavo Ray 7cb414c7cb FIxing issues 2025-07-11 00:01:52 +05:30
Arunavo Ray 6cfe43932f Fixing issues with Better Auth 2025-07-11 00:00:37 +05:30
Arunavo Ray b838310872 Added Better Auth 2025-07-10 23:15:37 +05:30
Arunavo Ray 46cf117bdf Migrate to Drizzle kit 2025-07-10 21:44:35 +05:30
Arunavo Ray 9301cc321c Optimised static comp with .astro 2025-07-09 01:01:37 +05:30
Arunavo Ray 9c17e5c240 Moved website from bun to pnpm. 2025-07-09 00:55:28 +05:30
ARUNAVO RAY a0b9d852bf Merge pull request #42 from RayLabsHQ/www
Added Website
2025-07-09 00:48:47 +05:30
Arunavo Ray eed26e4368 updates 2025-07-09 00:47:12 +05:30
Arunavo Ray e4f79720d4 Added simple analytics to website 2025-07-09 00:44:02 +05:30
Arunavo Ray 11dc299f12 Optimsed for mobile 2025-07-09 00:43:04 +05:30
Arunavo Ray 638554c93a mobile opt 2025-07-09 00:37:39 +05:30
Arunavo Ray d6de431902 seo 2025-07-09 00:37:30 +05:30
Arunavo Ray a320dc38ad Updated deisgn 2025-07-09 00:34:04 +05:30
Arunavo Ray 97997bd1c0 Minimalist 2025-07-08 23:57:13 +05:30
Arunavo Ray 316d263298 Added few logos 2025-07-08 23:27:59 +05:30
Arunavo Ray c891b8cf49 Added real data 2025-07-08 22:10:09 +05:30
Arunavo Ray b55d6a5629 Updated website deisgn 2025-07-08 21:58:45 +05:30
Arunavo Ray bb1842bc10 Added init website for gitea-mirror 2025-07-08 19:29:26 +05:30
Arunavo Ray 4958b8b9ec Release v2.22.0
### Added
- Comprehensive mobile and responsive design support across the entire application
- New drawer UI component for enhanced mobile navigation
- Mobile-specific layouts for major components
- Mobile screenshots in documentation

### Improved
- Enhanced mobile user experience with optimized layouts
- Updated organization list cards with better mobile responsiveness
- Better touch interaction support throughout the application

### Fixed
- Type definition issues resolved
- Removed unnecessary console.log statements

### Documentation
- Updated README with mobile usage instructions and screenshots
- Added mobile-specific documentation sections
2025-07-08 00:17:19 +05:30
ARUNAVO RAY aedf0c23b8 Update README.md 2025-07-07 23:50:48 +05:30
ARUNAVO RAY b4937d1e01 Update README.md 2025-07-07 23:50:16 +05:30
ARUNAVO RAY 498968695f Update README.md 2025-07-07 23:49:42 +05:30
Arunavo Ray d0e8e754a7 Type fix 2025-07-07 23:37:52 +05:30
Arunavo Ray c95a501974 Removed some console.logs 2025-07-07 23:12:26 +05:30
Arunavo Ray fd8f782f34 Updated org list cards 2025-07-07 23:07:22 +05:30
Arunavo Ray 02ff865e4b Updated Screenshots 2025-07-07 23:07:00 +05:30
Arunavo Ray df9da165c8 Added mobile layout screenshots 2025-07-07 22:52:33 +05:30
ARUNAVO RAY 180f300752 Merge pull request #40 from RayLabsHQ/mobile-layout
Mobile layout Optimised
2025-07-07 22:30:12 +05:30
Arunavo Ray 472f67a6ae Updates to Repository and Org Pages for Responsive Layouts 2025-07-07 22:02:43 +05:30
Arunavo Ray 6270907e70 Updates for mobile 2025-07-07 20:24:09 +05:30
Arunavo Ray 1deaae4d34 More responsive layout updates to Config Page 2025-07-07 19:27:07 +05:30
Arunavo Ray b984ff9af4 feat: improve mobile layout across components
- Update ActivityLog component for better mobile responsiveness
- Enhance Header layout for mobile devices
- Improve mobile UX in AddOrganizationDialog
- Optimize Organization component mobile display
- Enhance AddRepositoryDialog mobile layout
2025-07-07 18:51:24 +05:30
Arunavo Ray 24bd0aefe6 Added basic responsive layout 2025-07-07 17:34:54 +05:30
Arunavo Ray 6155e39360 docs: document docker-compose.alt.yml for quick start 2025-07-07 16:36:56 +05:30
ARUNAVO RAY 825363eac2 Merge pull request #38 from ryuupendragon/main
Create docker-compose.alt.yml
2025-07-07 16:27:19 +05:30
ARUNAVO RAY d78c72f448 Merge pull request #39 from RayLabsHQ/custom-ca-cert
feat: add custom CA certificate support (v2.21.0)
2025-07-07 16:26:49 +05:30
Arunavo Ray 9d7cb0f372 feat: add custom CA certificate support
- Add support for custom CA certificates in Docker setup
- Two mounting options: individual certs or system CA bundle
- Automatic detection and configuration via NODE_EXTRA_CA_CERTS
- Enhanced documentation with setup guide in certs/README.md
- Added ca-certificates package to Alpine base image
- Updated docker-compose with clear volume mount examples
- Bump version to 2.21.0
2025-07-07 16:20:24 +05:30
Ryuu Pendragon 19a252b67c Create docker-compose.alt.yml
Alternate docker compose with host path and minimal env.
2025-07-07 16:16:08 +05:30
Arunavo Ray df27e5951a chore: bump version to v2.20.1
- Fix mixed mode organization strategy not persisting after page refresh
- Update README to be more concise and use RayLabsHQ organization links
- Remove unimplemented OIDC/SSO authentication references
- Update dependencies to latest versions
2025-07-07 14:53:27 +05:30
Arunavo Ray c3c129d923 fix: enhance mirror strategy handling for mixed mode configuration 2025-07-07 14:03:23 +05:30
Arunavo Ray ae3511399f chore: update dependencies and devDependencies in package.json
- Bump @astrojs/node from 9.2.2 to 9.3.0
- Update @tailwindcss/vite from 4.1.10 to 4.1.11
- Upgrade @tanstack/react-virtual from 3.13.10 to 3.13.12
- Change astro version from 5.9.3 to 5.11.0
- Update lucide-react from 0.515.0 to 0.525.0
- Bump tailwindcss from 4.1.10 to 4.1.11
- Upgrade tw-animate-css from 1.3.4 to 1.3.5
- Update zod from 3.25.64 to 3.25.75
- Bump @types/jsonwebtoken from 9.0.9 to 9.0.10
- Upgrade @vitejs/plugin-react from 4.5.2 to 4.6.0
- Update vitest from 3.2.3 to 3.2.4
- Change packageManager from bun@1.2.16 to bun@1.2.18
2025-07-07 13:54:59 +05:30
Arunavo Ray b604cad2fd chore: bump version to v2.20.0
Major repository migration from arunavo4 to RayLabsHQ organization.
This release includes updated Docker image locations and all repository
references throughout the codebase.
2025-07-07 11:02:09 +05:30
Arunavo Ray 983b47fa76 fix: update repository references from arunavo4 to RayLabsHQ 2025-07-07 10:52:33 +05:30
Arunavo Ray ea065d22a5 fix: update repository links and image references to RayLabsHQ 2025-07-07 10:37:18 +05:30
Arunavo Ray 577d198e1a fix: remove security test script after confirming vulnerability is resolved 2025-07-07 10:36:13 +05:30
Arunavo Ray 9ead32e9e6 Update License MIT to GNU v3 2025-07-07 08:50:54 +05:30
Arunavo Ray 993e7c1bc0 fix: remove unused import from MirrorDestinationEditor component 2025-07-07 08:49:33 +05:30
Arunavo Ray 5b275a17e3 fix: update base image in Dockerfile to version 1.2.18-alpine 2025-07-07 08:49:29 +05:30
Arunavo Ray 0caa327142 fix: update CLAUDE.md to correct punctuation in testing section 2025-07-06 22:20:25 +05:30
Arunavo Ray 00d516a59d chore: bump version to v2.18.0
- Add mixed organization strategy for flexible repository mirroring
- Add override options documentation in Organization Strategy component
- Simplify implementation to reuse existing database fields
2025-06-24 13:58:21 +05:30
Arunavo Ray d68b822c76 feat: enhance OrganizationStrategy component with override options for mirror destinations 2025-06-24 13:54:47 +05:30
Arunavo Ray b660d2dd9a feat: add mixed strategy for repository mirroring, enhancing organization handling for personal and organizational repos 2025-06-24 13:51:43 +05:30
Arunavo Ray 4d8d75c8a6 chore: bump version to v2.17.0 2025-06-24 12:13:46 +05:30
Arunavo Ray e6c4ca0731 feat: implement InlineDestinationEditor for repository destination management and add API support for updating destination organization 2025-06-24 12:11:10 +05:30
Arunavo Ray f03405b87a feat: add Gitea organization URL support and update button tooltip logic 2025-06-24 11:45:35 +05:30
Arunavo Ray ce367e3761 feat: implement MirrorDestinationEditor component for customizable organization destination 2025-06-24 11:33:02 +05:30
Arunavo Ray cfe65cadca feat: add organization destination update API and enhance organization list editing functionality 2025-06-24 11:16:44 +05:30
Arunavo Ray 68108b8383 feat: add personal repositories organization override and update related configurations 2025-06-24 11:02:57 +05:30
Arunavo Ray d2bec1d56e chore: bump version to v2.16.3 2025-06-20 11:16:27 +05:30
Arunavo Ray c9404f2674 feat: implement custom 404 page with NotFound component and navigation options 2025-06-20 11:12:09 +05:30
Arunavo Ray 80ef19c634 feat: add openDelay prop to HoverCard in GitHubConfigForm and OrganizationStrategy components 2025-06-20 11:00:36 +05:30
Arunavo Ray 83c924566c feat: add HoverCard component and replace Popover usage in GitHubConfigForm and OrganizationStrategy 2025-06-20 10:58:46 +05:30
Arunavo Ray 7b58df375e chore: bump version to v2.16.2 2025-06-17 17:23:44 +05:30
Arunavo Ray 1d27bd31d8 feat: add bulk actions for repository management with selection support 2025-06-17 17:22:38 +05:30
Arunavo Ray 13d4257c4f refactor: enhance organization card display with status badges and improved layout 2025-06-17 16:52:15 +05:30
Arunavo Ray 818ba77693 chore: bump version to v2.16.1 2025-06-17 15:50:42 +05:30
Arunavo Ray 056970e577 refactor: improve repository owner handling and mirror strategy in Gitea integration 2025-06-17 15:44:04 +05:30
Arunavo Ray 65ea73e238 refactor: update label for starred repositories organization for consistency 2025-06-17 15:32:43 +05:30
Arunavo Ray 398f00aceb chore: bump version to v2.16.0 2025-06-17 15:16:10 +05:30
Arunavo Ray 50972713a3 refactor: improve version comparison logic in health API 2025-06-17 15:11:05 +05:30
Arunavo Ray fbf3033455 refactor: remove ConnectionsForm and useMirror hook; update issue mirroring logic for starred repos 2025-06-17 15:06:54 +05:30
Arunavo Ray cc4d8dabbc refactor: enhance starred repos content selection and improve layout 2025-06-17 14:54:23 +05:30
Arunavo Ray 8bba3d3521 chore: bump version to v2.15.0 2025-06-17 14:37:47 +05:30
Arunavo Ray be63555e5c refactor: update titles and descriptions in strategyConfig for clarity 2025-06-17 14:34:17 +05:30
Arunavo Ray 32a906369f refactor: remove unused Building2 import in OrganizationConfiguration component 2025-06-17 14:28:49 +05:30
Arunavo Ray 064474fd13 refactor: remove unused tooltip imports in OrganizationStrategy component 2025-06-17 14:27:38 +05:30
Arunavo Ray 2ac933b599 refactor: streamline layout and improve tooltip descriptions in OrganizationConfiguration component 2025-06-17 14:26:52 +05:30
Arunavo Ray 403fe08bae refactor: update icons in GitHubMirrorSettings and OrganizationConfiguration components for improved clarity 2025-06-17 14:16:15 +05:30
Arunavo Ray 23c7ff7349 refactor: enhance layout and flexibility in GitHubConfigForm and GiteaConfigForm components 2025-06-17 14:05:10 +05:30
Arunavo Ray 3169af44cb refactor: improve layout and spacing in GitHubMirrorSettings component 2025-06-17 13:49:53 +05:30
Arunavo Ray c1d93dbbc6 feat: enhance GitHubMirrorSettings with improved layout and metadata options 2025-06-17 13:43:17 +05:30
Arunavo Ray 047719cde9 feat: add OrganizationConfiguration component and integrate it into GiteaConfigForm 2025-06-17 13:20:03 +05:30
Arunavo Ray 13d4b03541 refactor: simplify OrganizationStrategy component by removing unused imports and details 2025-06-17 13:12:12 +05:30
Arunavo Ray f07ae220b0 refactor: clean up imports and improve layout in OrganizationStrategy component 2025-06-17 12:59:29 +05:30
Arunavo Ray 01647445f2 Improved layout in larger screens 2025-06-17 12:48:38 +05:30
Arunavo Ray 13cbf86309 Updated Layout 2025-06-17 12:42:32 +05:30
Arunavo Ray 792096d209 Updates to Organisation Strategy Layouts 2025-06-17 11:39:41 +05:30
Arunavo Ray e94de5c9ca fix: mirror to one org issue 2025-06-17 11:35:09 +05:30
Arunavo Ray b3f42624d8 chore: bump version to v2.14.0 2025-06-17 10:34:59 +05:30
Arunavo Ray d79e4fecf4 Updated Docker compose dev 2025-06-17 10:30:33 +05:30
Arunavo Ray eb78f959c7 Updated Docs 2025-06-17 09:47:46 +05:30
Arunavo Ray 51e536c317 feat: add @radix-ui/react-accordion dependency 2025-06-17 09:35:14 +05:30
Arunavo Ray 7af1f6da17 fix: implement proper mirror strategies for starred and org repos 2025-06-17 09:26:26 +05:30
Arunavo Ray c7e310b340 updated instructions 2025-06-16 00:35:48 +05:30
Arunavo Ray 23cfa45d89 chore: bump version to v2.13.2 2025-06-16 00:33:28 +05:30
Arunavo Ray b1346e8c77 Updated Docs and Readme 2025-06-16 00:28:55 +05:30
Arunavo Ray 6e673249dc Docs Design updated 2025-06-16 00:01:45 +05:30
Arunavo Ray ee801f5d0e chore: bump version to v2.13.1
- Updated package.json version from 2.13.0 to 2.13.1
- Added CHANGELOG.md entry for v2.13.1 with Docker workflow improvements
- Includes Docker Scout migration, CI/CD enhancements, and package updates
2025-06-15 23:06:05 +05:30
Arunavo Ray caf680d999 fix: add wait step for image availability in Docker build workflow 2025-06-15 15:10:48 +05:30
Arunavo Ray 214599a5fd Dont fail workflow on security issues 2025-06-15 15:03:54 +05:30
Arunavo Ray 9e2285d614 fix: update Docker Scout image references and add wait step for image availability 2025-06-15 14:43:31 +05:30
Arunavo Ray 7f7e510400 fix: resolve Docker Scout image reference issues
- Use specific SHA-based image tags instead of multi-line tags output
- Add separate Docker Scout steps for push vs pull request workflows
- Use local image reference for PR scanning (local://gitea-mirror:scan)
- Optimize PR builds to single platform (linux/amd64) for faster scanning
- Maintain multi-platform builds for production pushes
2025-06-15 14:38:25 +05:30
Arunavo Ray d1aa8810f7 fix: update Docker Scout action to use the latest version 2025-06-15 14:35:35 +05:30
Arunavo Ray bfa4b4034c feat: add Docker Hub authentication for Docker Scout
- Add optional Docker Hub login for enhanced vulnerability data
- Use continue-on-error to make Docker Hub auth optional
- Requires DOCKERHUB_USERNAME and DOCKERHUB_TOKEN secrets for full functionality
2025-06-15 14:23:44 +05:30
Arunavo Ray 8fbde95f92 feat: consolidate Docker workflows into comprehensive build, push & security scan
- Merge docker-build.yml and docker-scan.yml into single efficient workflow
- Add comprehensive Docker Scout security analysis with multiple commands
- Include vulnerability scanning, recommendations, and policy evaluation
- Add comparison to latest for pull requests
- Maintain SARIF output for GitHub Security tab integration
- Add proper path-based triggers and scheduled weekly scans
- Remove redundant docker-scan.yml workflow
2025-06-15 14:20:48 +05:30
Arunavo Ray 00fb66baa7 fix: add workflow file to Docker scan triggers to enable testing 2025-06-15 14:15:27 +05:30
Arunavo Ray 5fec1e6a58 fix: update Docker Scout action to specific version v1.18.1 2025-06-15 14:14:16 +05:30
Arunavo Ray 2ec55c6070 Migrate from Trivy to Docker scout 2025-06-15 14:11:45 +05:30
Arunavo Ray 546bda8514 Updated packages 2025-06-15 14:04:12 +05:30
Arunavo Ray d05847dfe8 Wrong bun verison 2025-06-15 13:52:45 +05:30
Arunavo Ray 6551ea719c fix: update Bun base image and enhance security scanning
- Update Bun from 1.2.14 to 1.2.18 to address CVE-2025-22874
- Pin Trivy action to stable version (0.28.0)
- Add SARIF output for GitHub Security tab integration
- Set ignore-unfixed to false for comprehensive vulnerability detection
- Add security-events permission for uploading scan results
- Include fallback table output on scan failures
2025-06-15 13:48:58 +05:30
Arunavo Ray ae57b1b320 docs: update CHANGELOG for v2.13.0 release 2025-06-15 13:33:44 +05:30
Arunavo Ray 4d3ad2a337 chore: bump version to 2.13.0 2025-06-15 13:32:35 +05:30
Arunavo Ray e9c12bb9ff feat: enhance API config handling by adding mapping functions for UI and database structures 2025-06-15 13:31:06 +05:30
Arunavo Ray 42314ab0e3 feat: add collapsible component and integrate it into OrganizationStrategy for improved UI 2025-06-15 13:08:39 +05:30
Arunavo Ray 1be53bfa87 refactor: simplify imports and enhance styling in AutomationSettings component 2025-06-15 13:00:08 +05:30
Arunavo Ray e8d48376a0 Updated Automation & Maintainence 2025-06-15 12:56:25 +05:30
Arunavo Ray 0cdb386f56 Re-Organsied options 2025-06-15 12:35:26 +05:30
Arunavo Ray 7456fe3fae Fixed contrast issues 2025-06-15 12:22:38 +05:30
Arunavo Ray f4df7c3d19 Added some basic options 2025-06-15 12:15:14 +05:30
Arunavo Ray 544b60f881 refactor: update Card components to use self-start class for consistent alignment 2025-06-12 15:29:47 +05:30
Arunavo Ray 2eda800a7c refactor: update test files to use bun:test and remove vitest configuration 2025-06-12 10:24:55 +05:30
Arunavo Ray 51de51baa0 feat: add permissions section to workflows for consistent access control 2025-06-12 10:13:41 +05:30
Arunavo Ray 0d60c2fdf1 feat: implement createSecureErrorResponse for consistent error handling across API routes 2025-06-12 09:50:43 +05:30
Arunavo Ray df8dac0e9b feat: add ConnectionsForm and ScheduleAndCleanupForm components with configuration forms 2025-06-11 22:17:31 +05:30
Arunavo Ray 8e0c31fbb9 feat: add ScrollArea and Separator components with Radix UI integration 2025-06-11 22:16:30 +05:30
Arunavo Ray 2c815b13f0 Added accordion 2025-06-11 21:51:39 +05:30
Arunavo Ray bbd49d7d52 Tabs Config 2025-06-11 21:43:43 +05:30
Arunavo Ray 8f62da4572 Replace logo in LoginForm, SignupForm, and Header components with light and dark mode images 2025-06-11 20:36:07 +05:30
Arunavo Ray 0f671a4088 feat: add support for mirroring wiki pages in configuration 2025-06-11 19:48:24 +05:30
Arunavo Ray 108408be81 fix: update Proxmox VE installation script references in README files 2025-06-05 23:27:56 +05:30
628 changed files with 158228 additions and 8895 deletions

No files matched your search

+1
View File
@@ -15,6 +15,7 @@ dist
build
.next
out
www
# Environment variables
.env
+210 -17
View File
@@ -1,38 +1,231 @@
# Docker Registry Configuration
DOCKER_REGISTRY=ghcr.io
DOCKER_IMAGE=arunavo4/gitea-mirror
DOCKER_TAG=latest
# Gitea Mirror Configuration
# Copy this to .env and update with your values
# ===========================================
# CORE CONFIGURATION
# ===========================================
# Application Configuration
NODE_ENV=production
HOST=0.0.0.0
PORT=4321
# Optional application base path (use "/" for root, or "/mirror" for subpath deployments)
BASE_URL=/
# Database Configuration
# For self-hosted, SQLite is used by default
DATABASE_URL=sqlite://data/gitea-mirror.db
# Security
JWT_SECRET=change-this-to-a-secure-random-string-in-production
# Generate with: openssl rand -base64 32
BETTER_AUTH_SECRET=change-this-to-a-secure-random-string-in-production
BETTER_AUTH_URL=http://localhost:4321
# ENCRYPTION_SECRET=optional-encryption-key-for-token-encryption # Generate with: openssl rand -base64 48
# Better Auth log verbosity: debug | info | warn | error (default: warn).
# Set to "debug" to trace SSO/OIDC sign-in and callback issues.
# BETTER_AUTH_LOG_LEVEL=debug
# Optional GitHub/Gitea Mirror Configuration (for docker-compose, can also be set via web UI)
# Uncomment and set as needed. These are passed as environment variables to the container.
# ===========================================
# REVERSE PROXY CONFIGURATION
# ===========================================
# REQUIRED when accessing Gitea Mirror through a reverse proxy (Nginx, Caddy, Traefik, etc.).
# Without these, sign-in will fail with "invalid origin" errors and pages may appear blank.
#
# Set all three to your external URL, e.g.:
# BETTER_AUTH_URL=https://gitea-mirror.example.com
# PUBLIC_BETTER_AUTH_URL=https://gitea-mirror.example.com
# BETTER_AUTH_TRUSTED_ORIGINS=https://gitea-mirror.example.com
#
# If your app is served from a path prefix (e.g. https://git.example.com/mirror), set:
# BASE_URL=/mirror
# BETTER_AUTH_URL=https://git.example.com
# PUBLIC_BETTER_AUTH_URL=https://git.example.com
# BETTER_AUTH_TRUSTED_ORIGINS=https://git.example.com
#
# BETTER_AUTH_URL - Used server-side for auth callbacks and redirects
# PUBLIC_BETTER_AUTH_URL - Used client-side (browser) for auth API calls
# BETTER_AUTH_TRUSTED_ORIGINS - Comma-separated list of origins allowed to make auth requests
# (e.g. https://gitea-mirror.example.com,https://alt.example.com)
PUBLIC_BETTER_AUTH_URL=http://localhost:4321
# BETTER_AUTH_TRUSTED_ORIGINS=
# ===========================================
# HTTPS / TLS (Optional)
# ===========================================
# Set BOTH to have the server terminate TLS directly (no reverse proxy needed).
# Leave unset when TLS is handled upstream by Nginx/Traefik/Caddy.
# SERVER_CERT_PATH=/etc/ssl/gitea-mirror/cert.pem
# SERVER_KEY_PATH=/etc/ssl/gitea-mirror/key.pem
# ===========================================
# DOCKER CONFIGURATION (Optional)
# ===========================================
# Docker Registry Configuration
DOCKER_REGISTRY=ghcr.io
DOCKER_IMAGE=raylabshq/gitea-mirror
DOCKER_TAG=latest
# ===========================================
# GITHUB CONFIGURATION
# All settings can also be configured via web UI
# ===========================================
# Basic GitHub Settings
# GITHUB_USERNAME=your-github-username
# GITHUB_TOKEN=your-github-personal-access-token
# SKIP_FORKS=false
# GITHUB_TYPE=personal # Options: personal, organization
# GitHub Enterprise (GHES / GHEC with data residency)
# Leave unset for standard github.com. Examples:
# GHES (self-hosted): https://ghe.example.com/api/v3
# GHEC data residency: https://api.TENANT.ghe.com
# GH_API_URL=https://ghe.example.com/api/v3
# Repository Selection
# PRIVATE_REPOSITORIES=false
# MIRROR_ISSUES=false
# PUBLIC_REPOSITORIES=true
# INCLUDE_ARCHIVED=false
# SKIP_FORKS=false
# MIRROR_STARRED=false
# MIRROR_STARRED_LISTS=homelab,dottools # Optional: comma-separated star list names; empty = all starred repos
# STARRED_REPOS_ORG=starred # Organization name for starred repos
# STARRED_REPOS_MODE=dedicated-org # dedicated-org | preserve-owner
# Organization Settings
# MIRROR_ORGANIZATIONS=false
# PRESERVE_ORG_STRUCTURE=false
# ONLY_MIRROR_ORGS=false
# SKIP_STARRED_ISSUES=false
# Mirror Strategy
# MIRROR_STRATEGY=preserve # Options: preserve, single-org, flat-user, mixed
# Advanced GitHub Settings
# SKIP_STARRED_ISSUES=false # Enable lightweight mode for starred repos
# ===========================================
# GITEA CONFIGURATION
# All settings can also be configured via web UI
# ===========================================
# Basic Gitea Settings
# GITEA_URL=http://gitea:3000
# GITEA_EXTERNAL_URL=https://gitea.example.com # Optional: used only for UI links
# GITEA_TOKEN=your-local-gitea-token
# GITEA_USERNAME=your-local-gitea-username
# GITEA_ORGANIZATION=github-mirrors
# GITEA_ORG_VISIBILITY=public
# DELAY=3600
# GITEA_ORGANIZATION=github-mirrors # Default organization for single-org strategy
# Optional Database Cleanup Configuration (configured via web UI)
# These environment variables are optional and only used as defaults
# Users can configure cleanup settings through the web interface
# Repository Settings
# GITEA_ORG_VISIBILITY=public # Options: public, private, limited, default
# GITEA_MIRROR_INTERVAL=8h # Mirror sync interval (e.g., 30m, 1h, 8h, 24h) - automatically enables scheduler
# GITEA_LFS=false # Enable LFS support
# GITEA_CREATE_ORG=true # Auto-create organizations
# GITEA_PRESERVE_VISIBILITY=false # Preserve GitHub repo visibility in Gitea
# Template Settings (for using repository templates)
# GITEA_TEMPLATE_OWNER=template-owner
# GITEA_TEMPLATE_REPO=template-repo
# Topic Settings
# GITEA_ADD_TOPICS=true # Add topics to repositories
# GITEA_TOPIC_PREFIX=gh- # Prefix for topics
# Fork Handling
# GITEA_FORK_STRATEGY=reference # Options: skip, reference, full-copy
# ===========================================
# MIRROR OPTIONS
# Control what gets mirrored from GitHub
# ===========================================
# Release and Metadata
# MIRROR_RELEASES=false # Mirror GitHub releases
# RELEASE_LIMIT=10 # Maximum number of releases to mirror per repository
# MIRROR_WIKI=false # Mirror wiki content
# Issue Tracking (requires MIRROR_METADATA=true)
# MIRROR_METADATA=false # Master toggle for metadata mirroring
# MIRROR_ISSUES=false # Mirror issues
# MIRROR_PULL_REQUESTS=false # Mirror pull requests
# MIRROR_LABELS=false # Mirror labels
# MIRROR_MILESTONES=false # Mirror milestones
# ===========================================
# AUTOMATION CONFIGURATION
# Schedule automatic mirroring
# ===========================================
# Basic Schedule Settings
# SCHEDULE_ENABLED=false # When true, auto-imports and mirrors all repos on startup (v3.5.3+)
# SCHEDULE_INTERVAL=3600 # Interval in seconds or cron expression (e.g., "0 2 * * *")
# GITEA_MIRROR_INTERVAL=8h # Mirror sync interval (5m, 30m, 1h, 8h, 24h, 1d, 7d) - also triggers auto-start
# AUTO_IMPORT_REPOS=true # Automatically discover and import new GitHub repositories during syncs
# DELAY=3600 # Legacy: same as SCHEDULE_INTERVAL, kept for backward compatibility
# Execution Settings
# SCHEDULE_CONCURRENT=false # Allow concurrent mirror operations
# SCHEDULE_BATCH_SIZE=10 # Number of repos to process in parallel
# SCHEDULE_PAUSE_BETWEEN_BATCHES=5000 # Pause between batches (ms)
# Retry Configuration
# SCHEDULE_RETRY_ATTEMPTS=3
# SCHEDULE_RETRY_DELAY=60000 # Delay between retries (ms)
# SCHEDULE_TIMEOUT=3600000 # Max time for a mirror operation (ms)
# SCHEDULE_AUTO_RETRY=true
# Update Detection
# SCHEDULE_ONLY_MIRROR_UPDATED=false # Only mirror repos with updates
# SCHEDULE_UPDATE_INTERVAL=86400000 # Check for updates interval (ms)
# SCHEDULE_SKIP_RECENTLY_MIRRORED=true
# SCHEDULE_RECENT_THRESHOLD=3600000 # Skip if mirrored within this time (ms)
# Maintenance
# SCHEDULE_CLEANUP_BEFORE_MIRROR=false # Run cleanup before mirroring
# Notifications
# SCHEDULE_NOTIFY_ON_FAILURE=true
# SCHEDULE_NOTIFY_ON_SUCCESS=false
# SCHEDULE_LOG_LEVEL=info # Options: error, warn, info, debug
# SCHEDULE_TIMEZONE=UTC
# ===========================================
# DATABASE CLEANUP CONFIGURATION
# Automatic cleanup of old events and data
# ===========================================
# Basic Cleanup Settings
# CLEANUP_ENABLED=false
# CLEANUP_RETENTION_DAYS=7
# CLEANUP_RETENTION_DAYS=7 # Days to keep events
# Repository Cleanup (v3.4.0+)
# CLEANUP_DELETE_FROM_GITEA=false # Delete repos from Gitea
# CLEANUP_DELETE_IF_NOT_IN_GITHUB=false # Auto-remove repos that no longer exist in GitHub
# CLEANUP_ORPHANED_REPO_ACTION=archive # Options: skip, archive, delete
# CLEANUP_DRY_RUN=true # Test mode without actual deletion (set to false for production)
# Protected Repositories (comma-separated)
# CLEANUP_PROTECTED_REPOS=important-repo,critical-project
# Cleanup Execution
# CLEANUP_BATCH_SIZE=10
# CLEANUP_PAUSE_BETWEEN_DELETES=2000 # Pause between deletions (ms)
# ===========================================
# AUTHENTICATION CONFIGURATION
# ===========================================
# Header Authentication (for Reverse Proxy SSO)
# Enable automatic authentication via reverse proxy headers
# HEADER_AUTH_ENABLED=false
# HEADER_AUTH_USER_HEADER=X-Authentik-Username
# HEADER_AUTH_EMAIL_HEADER=X-Authentik-Email
# HEADER_AUTH_NAME_HEADER=X-Authentik-Name
# HEADER_AUTH_AUTO_PROVISION=false
# HEADER_AUTH_ALLOWED_DOMAINS=example.com,company.org
# ===========================================
# OPTIONAL FEATURES
# ===========================================
# TLS/SSL Configuration
# GITEA_SKIP_TLS_VERIFY=false # WARNING: Only use for testing
+1
View File
@@ -0,0 +1 @@
use flake
+15
View File
@@ -0,0 +1,15 @@
# These are supported funding model platforms
github: [arunavo4] # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username
ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: # Replace with a single Liberapay username
issuehunt: # Replace with a single IssueHunt username
lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
polar: # Replace with a single Polar username
buy_me_a_coffee: # Replace with a single Buy Me a Coffee username
thanks_dev: # Replace with a single thanks.dev username
custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']
Binary file not shown.

Before

Width:  |  Height:  |  Size: 816 KiB

After

Width:  |  Height:  |  Size: 834 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 276 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 986 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 945 KiB

After

Width:  |  Height:  |  Size: 905 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 270 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 943 KiB

After

Width:  |  Height:  |  Size: 135 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 45 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 44 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.6 MiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 825 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 246 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 784 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 33 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 44 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 47 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 971 KiB

After

Width:  |  Height:  |  Size: 952 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 237 KiB

+59
View File
@@ -0,0 +1,59 @@
image:
registry: ghcr.io
repository: raylabshq/gitea-mirror
tag: ""
service:
type: ClusterIP
port: 8080
ingress:
enabled: true
className: "nginx"
hosts:
- host: ci.example.com
route:
enabled: true
forceHTTPS: true
domain: ["ci.example.com"]
gateway: "dummy-gw"
gatewayNamespace: "default"
http:
gatewaySection: "http"
https:
gatewaySection: "https"
gitea-mirror:
nodeEnv: production
core:
databaseUrl: "file:data/gitea-mirror.db"
betterAuthSecret: "dummy"
betterAuthUrl: "http://localhost:4321"
betterAuthTrustedOrigins: "http://localhost:4321"
github:
username: "ci-user"
token: "not-used-in-template"
type: "personal"
privateRepositories: true
skipForks: false
starredCodeOnly: false
gitea:
url: "https://gitea.example.com"
token: "not-used-in-template"
username: "ci-user"
organization: "github-mirrors"
visibility: "public"
mirror:
releases: true
wiki: true
metadata: true
issues: true
pullRequests: true
starred: false
automation:
schedule_enabled: true
schedule_interval: "3600"
cleanup:
enabled: true
interval: "2592000"
Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 319 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 334 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 173 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 172 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 226 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 230 KiB

+23 -18
View File
@@ -7,8 +7,7 @@ This directory contains GitHub Actions workflows that automate the build, test,
| Workflow | File | Purpose |
|----------|------|---------|
| Astro Build and Test | `astro-build-test.yml` | Builds and tests the Astro application for all branches and PRs |
| Docker Build and Push | `docker-build.yml` | Builds and pushes Docker images only for the main branch |
| Docker Security Scan | `docker-scan.yml` | Scans Docker images for security vulnerabilities |
| Docker Build, Push & Security Scan | `docker-build.yml` | Builds, scans, and pushes Docker images |
## Workflow Details
@@ -28,39 +27,38 @@ This workflow runs on all branches and pull requests. It:
- Caches dependencies to speed up builds
- Uploads build artifacts for 7 days
### Docker Build and Push (`docker-build.yml`)
### Docker Build, Push & Security Scan (`docker-build.yml`)
This workflow builds and pushes Docker images to GitHub Container Registry (ghcr.io), but only when changes are merged to the main branch.
This workflow builds Docker images on pushes and pull requests, scans them, and pushes to GitHub Container Registry (ghcr.io) when permissions allow (main/tags and same-repo PRs).
**When it runs:**
- On push to the main branch
- On tag creation (v*)
- On pull requests (build + scan; push only for same-repo PRs)
- Weekly on Sunday at midnight (scheduled security scan)
**Key features:**
- Builds multi-architecture images (amd64 and arm64)
- Pushes images only on main branch, not for PRs
- Pushes images for main/tags and same-repo PRs
- Skips registry push for fork PRs (avoids package write permission failures)
- Uses build caching to speed up builds
- Creates multiple tags for each image (latest, semver, sha)
- Auto-syncs `package.json` version from `v*` tags during release builds
- Validates release tags use semver format before building
- After tag builds succeed, writes the same version back to `main/package.json`
### Docker Security Scan (`docker-scan.yml`)
This workflow scans Docker images for security vulnerabilities using Trivy.
**When it runs:**
- On push to the main branch that affects Docker-related files
- Weekly on Sunday at midnight (scheduled)
**Key features:**
- Scans for critical and high severity vulnerabilities
- Fails the build if vulnerabilities are found
- Ignores unfixed vulnerabilities
**Security scanning:**
- Docker Scout runs on main/tag pushes, the weekly schedule, and same-repo PRs. It needs the `DOCKERHUB_USERNAME` / `DOCKERHUB_TOKEN` secrets, posts a PR comment, and uploads SARIF to the Security tab.
- Trivy runs on every PR, including forks. It needs no credentials and writes critical/high (fixable) findings to the job summary.
- Fork PRs cannot use Docker Scout because GitHub never exposes repository secrets to them. Do not "fix" that with `pull_request_target`; it would run untrusted PR code with the secrets available.
- Both scanners are informational and do not fail the build.
## CI/CD Pipeline Philosophy
Our CI/CD pipeline follows these principles:
1. **Fast feedback for developers**: The Astro build and test workflow runs on all branches and PRs to provide quick feedback.
2. **Efficient resource usage**: Docker images are only built when changes are merged to main, not for every PR.
2. **Efficient resource usage**: Docker images are built for every PR so they can be scanned, but only pushed for main, tags, and same-repo PRs.
3. **Security first**: Regular security scanning ensures our Docker images are free from known vulnerabilities.
4. **Multi-architecture support**: All Docker images are built for both amd64 and arm64 architectures.
@@ -85,3 +83,10 @@ If a workflow fails:
- Security vulnerabilities
For persistent issues, consider opening an issue in the repository.
### Helm Test (`helm-test.yml`)
This workflow run on the main branch and pull requests. it:
- Run yamllint to keep the formating unified
- Run helm template with different value files
+16 -1
View File
@@ -6,16 +6,25 @@ on:
paths-ignore:
- 'README.md'
- 'docs/**'
- 'www/**'
- 'helm/**'
pull_request:
branches: [ '*' ]
paths-ignore:
- 'README.md'
- 'docs/**'
- 'www/**'
- 'helm/**'
permissions:
contents: read
actions: read
jobs:
build-and-test:
name: Build and Test Astro Project
runs-on: ubuntu-latest
timeout-minutes: 25
steps:
- name: Checkout repository
@@ -24,7 +33,7 @@ jobs:
- name: Setup Bun
uses: oven-sh/setup-bun@v1
with:
bun-version: '1.2.9'
bun-version: '1.3.13'
- name: Check lockfile and install dependencies
run: |
@@ -39,6 +48,12 @@ jobs:
- name: Run tests
run: bun test --coverage
- name: Check Drizzle migrations
run: bun run db:check
- name: Validate migrations (SQLite lint + upgrade path)
run: bun test:migrations
- name: Build Astro project
run: bunx --bun astro build
+403 -13
View File
@@ -1,53 +1,443 @@
name: Build and Push Docker Images
name: Docker Build, Push & Security Scan
on:
push:
branches: [main]
tags: ['v*']
paths:
- 'Dockerfile'
- '.dockerignore'
- 'package.json'
- 'bun.lock*'
- '.github/workflows/docker-build.yml'
- 'docker-entrypoint.sh'
- 'drizzle/**'
- 'scripts/**'
- 'src/**'
pull_request:
paths:
- 'Dockerfile'
- '.dockerignore'
- 'package.json'
- 'bun.lock*'
- '.github/workflows/docker-build.yml'
- 'docker-entrypoint.sh'
- 'drizzle/**'
- 'scripts/**'
- 'src/**'
schedule:
- cron: '0 0 * * 0' # Weekly security scan on Sunday at midnight
env:
REGISTRY: ghcr.io
IMAGE: ${{ github.repository }}
SHA: ${{ github.event.pull_request.head.sha || github.event.after }}
jobs:
docker:
runs-on: ubuntu-latest
timeout-minutes: 25
permissions:
contents: write
packages: write
security-events: write
pull-requests: write
steps:
- uses: actions/checkout@v4
- name: Checkout repository
uses: actions/checkout@v4
with:
ref: ${{ env.SHA }}
- uses: docker/setup-buildx-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
with:
driver-opts: network=host
- uses: docker/login-action@v3
if: github.event_name != 'pull_request'
- name: Log into registry
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
# Login to Docker Hub for Docker Scout (optional - provides better vulnerability data)
# Add DOCKERHUB_USERNAME and DOCKERHUB_TOKEN secrets to enable this.
# Skipped on fork PRs: GitHub never exposes repository secrets to them,
# so the login could only fail with "Username and password required".
- name: Log into Docker Hub
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
continue-on-error: true
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
# Extract version from tag if present
- name: Extract version from tag
id: tag_version
run: |
if [[ $GITHUB_REF == refs/tags/v* ]]; then
echo "VERSION=${GITHUB_REF#refs/tags/}" >> $GITHUB_OUTPUT
echo "Using version tag: ${GITHUB_REF#refs/tags/}"
TAG_VERSION="${GITHUB_REF#refs/tags/}"
if [[ ! "$TAG_VERSION" =~ ^v[0-9]+\.[0-9]+\.[0-9]+([.-][0-9A-Za-z.-]+)?(\+[0-9A-Za-z.-]+)?$ ]]; then
echo "::error::Release tag '${TAG_VERSION}' is invalid. Expected semver tag format like v1.2.3 or v1.2.3-rc.1"
exit 1
fi
APP_VERSION="${TAG_VERSION#v}"
echo "VERSION=${TAG_VERSION}" >> $GITHUB_OUTPUT
echo "APP_VERSION=${APP_VERSION}" >> $GITHUB_OUTPUT
echo "Using version tag: ${TAG_VERSION}"
else
echo "VERSION=latest" >> $GITHUB_OUTPUT
echo "APP_VERSION=dev" >> $GITHUB_OUTPUT
echo "No version tag, using 'latest'"
fi
- uses: docker/build-push-action@v5
# Keep version files aligned automatically for tag-based releases
- name: Sync app version from release tag
if: startsWith(github.ref, 'refs/tags/v')
run: |
VERSION="${{ steps.tag_version.outputs.APP_VERSION }}"
echo "Syncing package.json version to ${VERSION}"
jq --arg version "${VERSION}" '.version = $version' package.json > package.json.tmp
mv package.json.tmp package.json
echo "Version sync diff (package.json):"
git --no-pager diff -- package.json
# Extract metadata for Docker
- name: Extract Docker metadata
id: meta
uses: docker/metadata-action@c299e40c65443455700f0fdfc63efafe5b349051 # v5.10.0
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE }}
labels: |
org.opencontainers.image.revision=${{ env.SHA }}
tags: |
type=edge,branch=$repo.default_branch
type=semver,pattern=v{{version}}
type=sha,prefix=,suffix=,format=short
type=raw,value=latest,enable={{is_default_branch}}
type=raw,value=${{ steps.tag_version.outputs.VERSION }}
type=ref,event=pr,prefix=pr-
# Build and push Docker image
- name: Build and push Docker image
id: build-and-push
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6.19.2
with:
context: .
platforms: linux/amd64,linux/arm64
push: ${{ github.event_name != 'pull_request' }}
tags: |
${{ env.REGISTRY }}/${{ env.IMAGE }}:latest
${{ env.REGISTRY }}/${{ env.IMAGE }}:${{ github.sha }}
${{ env.REGISTRY }}/${{ env.IMAGE }}:${{ steps.tag_version.outputs.VERSION }}
push: ${{ github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
# The runner stage installs Debian security updates; never serve it from cache.
no-cache-filters: runner
cache-to: type=gha,mode=max
provenance: false # Disable provenance to avoid unknown/unknown
sbom: false # Disable sbom to avoid unknown/unknown
# Load image locally for security scanning (PRs only)
- name: Load image for scanning
if: github.event_name == 'pull_request'
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6.19.2
with:
context: .
platforms: linux/amd64
load: true
tags: gitea-mirror:scan
cache-from: type=gha
# The runner stage installs Debian security updates; never serve it from cache.
no-cache-filters: runner
provenance: false # Disable provenance to avoid unknown/unknown
sbom: false # Disable sbom to avoid unknown/unknown
# Wait for image to be available in registry
- name: Wait for image availability
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository
run: |
echo "Waiting for image to be available in registry..."
sleep 5
# Add comment to PR with image details
- name: Comment PR with image tag
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository
uses: actions/github-script@v7
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const prNumber = context.payload.pull_request.number;
const imageTag = `pr-${prNumber}`;
const imagePath = `${{ env.REGISTRY }}/${{ env.IMAGE }}:${imageTag}`.toLowerCase();
const comment = `## 🐳 Docker Image Built Successfully
Your PR image is available for testing:
**Image Tag:** \`${imageTag}\`
**Full Image Path:** \`${imagePath}\`
### Pull and Test
\`\`\`bash
docker pull ${imagePath}
docker run -d \
-p 4321:4321 \
-e BETTER_AUTH_SECRET=your-secret-here \
-e BETTER_AUTH_URL=http://localhost:4321 \
--name gitea-mirror-test ${imagePath}
\`\`\`
### Docker Compose Testing
\`\`\`yaml
services:
gitea-mirror:
image: ${imagePath}
ports:
- "4321:4321"
environment:
- BETTER_AUTH_SECRET=your-secret-here
- BETTER_AUTH_URL=http://localhost:4321
- BETTER_AUTH_TRUSTED_ORIGINS=http://localhost:4321
\`\`\`
> 💡 **Note:** PR images are tagged as \`pr-<number>\` and built for both \`linux/amd64\` and \`linux/arm64\`.
> Production images (\`latest\`, version tags) use the same multi-platform set.
---
📦 View in [GitHub Packages](https://github.com/${{ github.repository }}/pkgs/container/gitea-mirror)`;
github.rest.issues.createComment({
issue_number: prNumber,
owner: context.repo.owner,
repo: context.repo.repo,
body: comment
});
# Docker Scout comprehensive security analysis
- name: Docker Scout - Vulnerability Analysis & Recommendations
uses: docker/scout-action@bacf462e8d090c09660de30a6ccc718035f961e3 # v1.20.4
if: github.event_name != 'pull_request'
with:
command: cves,recommendations
image: ${{ env.REGISTRY }}/${{ env.IMAGE }}:latest
sarif-file: scout-results.sarif
summary: true
exit-code: false
only-severities: critical,high
# OpenVEX statements in .vex/ mark CVEs the image cannot reach as
# not affected (see .vex/README.md). Scout only accepts statements
# from authors listed here; the default is *@docker.com.
vex-location: .vex
vex-author: developer@arunavoray.dev
only-vex-affected: true
write-comment: true
github-token: ${{ secrets.GITHUB_TOKEN }}
# Docker Scout for Pull Requests (using local image).
# Same-repo PRs only: Scout needs the Docker Hub login above, and fork
# PRs never receive the secrets for it. They used to fail here with
# "user githubactions not entitled to use Docker Scout". Fork PRs get
# the credential-free Trivy scan further down instead.
- name: Docker Scout - Vulnerability Analysis (PR)
uses: docker/scout-action@bacf462e8d090c09660de30a6ccc718035f961e3 # v1.20.4
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository
with:
command: cves,recommendations
image: local://gitea-mirror:scan
sarif-file: scout-results.sarif
summary: true
exit-code: false
only-severities: critical,high
vex-location: .vex
vex-author: developer@arunavoray.dev
only-vex-affected: true
write-comment: true
github-token: ${{ secrets.GITHUB_TOKEN }}
# Compare to latest (same-repo PRs only, same reason as above)
- name: Docker Scout - Compare to Latest
uses: docker/scout-action@bacf462e8d090c09660de30a6ccc718035f961e3 # v1.20.4
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository
with:
command: compare
image: local://gitea-mirror:scan
to: ${{ env.REGISTRY }}/${{ env.IMAGE }}:latest
ignore-unchanged: true
only-severities: critical,high
write-comment: true
github-token: ${{ secrets.GITHUB_TOKEN }}
# Trivy runs on every PR, forks included. It needs no credentials, so it
# is the scan external contributors actually get. Results go to the job
# summary because GITHUB_TOKEN is read-only on fork PRs (no PR comments,
# no SARIF upload). Informational only, matching Scout's exit-code: false.
- name: Trivy - Vulnerability Analysis (PR)
if: github.event_name == 'pull_request'
uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25 # v0.36.0
with:
image-ref: gitea-mirror:scan
scanners: vuln
format: table
output: trivy-results.txt
severity: CRITICAL,HIGH
ignore-unfixed: true
exit-code: '0'
- name: Trivy - Write job summary
if: github.event_name == 'pull_request'
run: |
{
echo "## Trivy image scan (critical and high, fixable only)"
echo
echo "Image: \`gitea-mirror:scan\` built from this PR. Unfixed vulnerabilities are hidden."
echo
if [ -s trivy-results.txt ]; then
echo '```'
cat trivy-results.txt
echo '```'
else
echo "No critical or high vulnerabilities with an available fix."
fi
} >> "$GITHUB_STEP_SUMMARY"
# GitHub rejects SARIF files where a single result carries more than
# 1000 relatedLocations, which Scout produces for widely-referenced OS
# packages — every upload had been silently failing on this since May
# ("rejecting SARIF, as there are more related locations per result than
# allowed"). Cap them so uploads flow and stale alerts can auto-close.
- name: Sanitize Scout SARIF (cap relatedLocations)
if: always()
continue-on-error: true
run: |
if [ -f scout-results.sarif ]; then
jq '(.runs[]?.results[]?) |= (if .relatedLocations then .relatedLocations |= .[:100] else . end)' \
scout-results.sarif > scout-results.sanitized.sarif
mv scout-results.sanitized.sarif scout-results.sarif
fi
# Upload security scan results to GitHub Security tab.
# Skipped when no Scout step ran (fork PRs), otherwise the upload
# errors with "Path does not exist: scout-results.sarif".
- name: Upload Docker Scout scan results to GitHub Security tab
uses: github/codeql-action/upload-sarif@v4
if: always() && hashFiles('scout-results.sarif') != ''
continue-on-error: true
with:
sarif_file: scout-results.sarif
sync-version-main:
name: Sync package.json and Chart.yaml versions back to main
if: startsWith(github.ref, 'refs/tags/v')
runs-on: ubuntu-latest
needs: docker
permissions:
contents: write
steps:
- name: Checkout default branch
uses: actions/checkout@v4
with:
ref: ${{ github.event.repository.default_branch }}
- name: Update package.json and Chart.yaml versions on main
env:
TAG_VERSION: ${{ github.ref_name }}
TARGET_BRANCH: ${{ github.event.repository.default_branch }}
run: |
if [[ ! "$TAG_VERSION" =~ ^v[0-9]+\.[0-9]+\.[0-9]+([.-][0-9A-Za-z.-]+)?(\+[0-9A-Za-z.-]+)?$ ]]; then
echo "::error::Release tag '${TAG_VERSION}' is invalid. Expected semver tag format like v1.2.3 or v1.2.3-rc.1"
exit 1
fi
APP_VERSION="${TAG_VERSION#v}"
echo "Syncing ${TARGET_BRANCH} package.json and Chart.yaml to ${APP_VERSION}"
jq --arg version "${APP_VERSION}" '.version = $version' package.json > package.json.tmp
mv package.json.tmp package.json
# The chart version tracks the app version so the default image tag
# always points at the release the chart was published with.
sed -i -E \
-e "s/^version: .*/version: ${APP_VERSION}/" \
-e "s/^appVersion: .*/appVersion: \"${APP_VERSION}\"/" \
helm/gitea-mirror/Chart.yaml
if git diff --quiet -- package.json helm/gitea-mirror/Chart.yaml; then
echo "Versions on ${TARGET_BRANCH} already at ${APP_VERSION}; nothing to commit."
exit 0
fi
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add package.json helm/gitea-mirror/Chart.yaml
git commit -m "chore: sync version to ${APP_VERSION}"
git push origin "HEAD:${TARGET_BRANCH}"
publish-helm-chart:
name: Publish Helm chart to ghcr.io
# Runs after the image for this tag is on the registry, so the chart never
# points at an image that does not exist yet. The chart version and
# appVersion both follow the release version.
if: startsWith(github.ref, 'refs/tags/v')
runs-on: ubuntu-latest
timeout-minutes: 15
needs: docker
permissions:
contents: read
packages: write
steps:
- name: Checkout tag
uses: actions/checkout@v4
- name: Resolve version
id: version
env:
TAG_VERSION: ${{ github.ref_name }}
run: |
if [[ ! "$TAG_VERSION" =~ ^v[0-9]+\.[0-9]+\.[0-9]+([.-][0-9A-Za-z.-]+)?(\+[0-9A-Za-z.-]+)?$ ]]; then
echo "::error::Release tag '${TAG_VERSION}' is invalid. Expected semver tag format like v1.2.3 or v1.2.3-rc.1"
exit 1
fi
echo "version=${TAG_VERSION#v}" >> "$GITHUB_OUTPUT"
- name: Setup Helm
uses: azure/setup-helm@v4
with:
version: v3.19.0
- name: Lint and package chart
env:
VERSION: ${{ steps.version.outputs.version }}
run: |
helm lint ./helm/gitea-mirror
helm package ./helm/gitea-mirror \
--version "$VERSION" \
--app-version "$VERSION" \
--destination dist
- name: Log into registry
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
ACTOR: ${{ github.actor }}
run: |
echo "$GITHUB_TOKEN" | helm registry login "$REGISTRY" --username "$ACTOR" --password-stdin
- name: Push chart
env:
VERSION: ${{ steps.version.outputs.version }}
OWNER: ${{ github.repository_owner }}
run: |
OWNER_LC="$(echo "$OWNER" | tr '[:upper:]' '[:lower:]')"
helm push "dist/gitea-mirror-${VERSION}.tgz" "oci://${REGISTRY}/${OWNER_LC}/charts"
{
echo "### Helm chart ${VERSION} published"
echo
echo '```bash'
echo "helm upgrade --install gitea-mirror oci://${REGISTRY}/${OWNER_LC}/charts/gitea-mirror --version ${VERSION}"
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
-53
View File
@@ -1,53 +0,0 @@
name: Docker Security Scan
on:
push:
branches: [ main ]
paths:
- 'Dockerfile'
- '.dockerignore'
- 'package.json'
- 'bun.lock*'
pull_request:
branches: [ main ]
paths:
- 'Dockerfile'
- '.dockerignore'
- 'package.json'
- 'bun.lock*'
schedule:
- cron: '0 0 * * 0' # Run weekly on Sunday at midnight
jobs:
scan:
name: Scan Docker Image
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
driver-opts: network=host
- name: Build Docker image
uses: docker/build-push-action@v5
with:
context: .
push: false
load: true
tags: gitea-mirror:scan
# Disable GitHub Actions cache for this workflow
no-cache: true
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
with:
image-ref: gitea-mirror:scan
format: 'table'
exit-code: '1'
ignore-unfixed: true
vuln-type: 'os,library'
severity: 'CRITICAL,HIGH'
+303
View File
@@ -0,0 +1,303 @@
name: E2E Integration Tests
on:
push:
branches: ["*"]
paths-ignore:
- "README.md"
- "docs/**"
- "CHANGELOG.md"
- "LICENSE"
- "www/**"
- "helm/**"
pull_request:
branches: ["*"]
paths-ignore:
- "README.md"
- "docs/**"
- "CHANGELOG.md"
- "LICENSE"
- "www/**"
- "helm/**"
workflow_dispatch:
inputs:
debug_enabled:
description: "Enable debug logging"
required: false
default: "false"
type: boolean
permissions:
contents: read
actions: read
concurrency:
group: e2e-${{ github.ref }}
cancel-in-progress: true
env:
GITEA_PORT: 3333
FAKE_GITHUB_PORT: 4580
GIT_SERVER_PORT: 4590
APP_PORT: 4321
BUN_VERSION: "1.3.13"
jobs:
e2e-tests:
name: E2E Integration Tests
runs-on: ubuntu-latest
timeout-minutes: 25
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Bun
uses: oven-sh/setup-bun@v1
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "22"
- name: Install dependencies
run: |
bun install
echo "✓ Dependencies installed"
- name: Install Playwright
run: |
npx playwright install chromium
npx playwright install-deps chromium
echo "✓ Playwright ready"
- name: Create test git repositories
run: |
echo "Creating bare git repos for E2E testing..."
bun run tests/e2e/create-test-repos.ts --output-dir tests/e2e/git-repos
if [ ! -f tests/e2e/git-repos/manifest.json ]; then
echo "ERROR: Test git repos were not created (manifest.json missing)"
exit 1
fi
echo "✓ Test repos created:"
cat tests/e2e/git-repos/manifest.json | jq -r '.repos[] | " • \(.owner)/\(.name) — \(.description)"'
- name: Start Gitea and git-server containers
run: |
# Retry pulls before `up` — Docker Hub occasionally returns
# "context deadline exceeded" on the first attempt. Pulling
# separately also keeps the failure mode obvious if it persists.
echo "Pulling images..."
for attempt in 1 2 3; do
if docker compose -f tests/e2e/docker-compose.e2e.yml pull --quiet; then
echo "✓ Images pulled (attempt $attempt)"
break
fi
if [ "$attempt" -eq 3 ]; then
echo "ERROR: docker compose pull failed after 3 attempts"
exit 1
fi
sleep_s=$((attempt * 10))
echo "Pull attempt $attempt failed, retrying in ${sleep_s}s..."
sleep "$sleep_s"
done
echo "Starting containers via docker compose..."
docker compose -f tests/e2e/docker-compose.e2e.yml up -d
# Wait for git-server
echo "Waiting for git HTTP server..."
for i in $(seq 1 30); do
if curl -sf http://localhost:${{ env.GIT_SERVER_PORT }}/manifest.json > /dev/null 2>&1; then
echo "✓ Git HTTP server is ready"
break
fi
if [ $i -eq 30 ]; then
echo "ERROR: Git HTTP server did not start"
docker compose -f tests/e2e/docker-compose.e2e.yml logs git-server
exit 1
fi
sleep 1
done
# Wait for Gitea
echo "Waiting for Gitea to be ready..."
for i in $(seq 1 60); do
if curl -sf http://localhost:${{ env.GITEA_PORT }}/api/v1/version > /dev/null 2>&1; then
version=$(curl -sf http://localhost:${{ env.GITEA_PORT }}/api/v1/version | jq -r '.version // "unknown"')
echo "✓ Gitea is ready (version: $version)"
break
fi
if [ $i -eq 60 ]; then
echo "ERROR: Gitea did not become healthy within 120s"
docker compose -f tests/e2e/docker-compose.e2e.yml logs gitea-e2e --tail=30
exit 1
fi
sleep 2
done
- name: Initialize database
run: |
bun run manage-db init
echo "✓ Database initialized"
- name: Build application
env:
GH_API_URL: http://localhost:4580
BETTER_AUTH_SECRET: e2e-test-secret
run: |
bun run build
echo "✓ Build complete"
- name: Start fake GitHub API server
run: |
# Start with GIT_SERVER_URL pointing to the git-server container name
# (Gitea will resolve it via Docker networking)
PORT=${{ env.FAKE_GITHUB_PORT }} GIT_SERVER_URL="http://git-server" \
npx tsx tests/e2e/fake-github-server.ts &
echo $! > /tmp/fake-github.pid
echo "Waiting for fake GitHub API..."
for i in $(seq 1 30); do
if curl -sf http://localhost:${{ env.FAKE_GITHUB_PORT }}/___mgmt/health > /dev/null 2>&1; then
echo "✓ Fake GitHub API is ready"
break
fi
if [ $i -eq 30 ]; then
echo "ERROR: Fake GitHub API did not start"
exit 1
fi
sleep 1
done
# Ensure clone URLs are set for the git-server container
curl -sf -X POST http://localhost:${{ env.FAKE_GITHUB_PORT }}/___mgmt/set-clone-url \
-H "Content-Type: application/json" \
-d '{"url": "http://git-server"}' || true
echo "✓ Clone URLs configured for git-server container"
- name: Start gitea-mirror application
env:
GH_API_URL: http://localhost:4580
BETTER_AUTH_SECRET: e2e-test-secret
BETTER_AUTH_URL: http://localhost:4321
DATABASE_URL: file:data/gitea-mirror.db
HOST: 0.0.0.0
PORT: ${{ env.APP_PORT }}
NODE_ENV: production
PRE_SYNC_BACKUP_ENABLED: "false"
ENCRYPTION_SECRET: "e2e-encryption-secret-32char!!"
run: |
# Re-init DB in case build step cleared it
bun run manage-db init 2>/dev/null || true
bun run start &
echo $! > /tmp/app.pid
echo "Waiting for gitea-mirror app..."
for i in $(seq 1 90); do
if curl -sf http://localhost:${{ env.APP_PORT }}/api/health > /dev/null 2>&1 || \
curl -sf -o /dev/null -w "%{http_code}" http://localhost:${{ env.APP_PORT }}/ 2>/dev/null | grep -q "^[23]"; then
echo "✓ gitea-mirror app is ready"
break
fi
if ! kill -0 $(cat /tmp/app.pid) 2>/dev/null; then
echo "ERROR: App process died"
exit 1
fi
if [ $i -eq 90 ]; then
echo "ERROR: gitea-mirror app did not start within 180s"
exit 1
fi
sleep 2
done
- name: Run E2E tests
env:
APP_URL: http://localhost:${{ env.APP_PORT }}
GITEA_URL: http://localhost:${{ env.GITEA_PORT }}
FAKE_GITHUB_URL: http://localhost:${{ env.FAKE_GITHUB_PORT }}
GIT_SERVER_URL: http://localhost:${{ env.GIT_SERVER_PORT }}
CI: true
run: |
mkdir -p tests/e2e/test-results
npx playwright test \
--config tests/e2e/playwright.config.ts \
--reporter=github,html
- name: Diagnostic info on failure
if: failure()
run: |
echo "═══════════════════════════════════════════════════════════"
echo " Diagnostic Information"
echo "═══════════════════════════════════════════════════════════"
echo ""
echo "── Git server status ──"
curl -sf http://localhost:${{ env.GIT_SERVER_PORT }}/manifest.json 2>/dev/null | jq . || echo "(unreachable)"
echo ""
echo "── Gitea status ──"
curl -sf http://localhost:${{ env.GITEA_PORT }}/api/v1/version 2>/dev/null || echo "(unreachable)"
echo ""
echo "── Fake GitHub status ──"
curl -sf http://localhost:${{ env.FAKE_GITHUB_PORT }}/___mgmt/health 2>/dev/null | jq . || echo "(unreachable)"
echo ""
echo "── App status ──"
curl -sf http://localhost:${{ env.APP_PORT }}/api/health 2>/dev/null || echo "(unreachable)"
echo ""
echo "── Docker containers ──"
docker compose -f tests/e2e/docker-compose.e2e.yml ps 2>/dev/null || true
echo ""
echo "── Gitea container logs (last 50 lines) ──"
docker compose -f tests/e2e/docker-compose.e2e.yml logs gitea-e2e --tail=50 2>/dev/null || echo "(no container)"
echo ""
echo "── Git server logs (last 20 lines) ──"
docker compose -f tests/e2e/docker-compose.e2e.yml logs git-server --tail=20 2>/dev/null || echo "(no container)"
echo ""
echo "── Running processes ──"
ps aux | grep -E "(fake-github|astro|bun|node)" | grep -v grep || true
- name: Upload Playwright report
uses: actions/upload-artifact@v4
if: always()
with:
name: e2e-playwright-report
path: tests/e2e/playwright-report/
retention-days: 14
- name: Upload test results
uses: actions/upload-artifact@v4
if: always()
with:
name: e2e-test-results
path: tests/e2e/test-results/
retention-days: 14
- name: Cleanup
if: always()
run: |
# Stop background processes
if [ -f /tmp/fake-github.pid ]; then
kill $(cat /tmp/fake-github.pid) 2>/dev/null || true
rm -f /tmp/fake-github.pid
fi
if [ -f /tmp/app.pid ]; then
kill $(cat /tmp/app.pid) 2>/dev/null || true
rm -f /tmp/app.pid
fi
# Stop containers
docker compose -f tests/e2e/docker-compose.e2e.yml down --volumes --remove-orphans 2>/dev/null || true
echo "✓ Cleanup complete"
+63
View File
@@ -0,0 +1,63 @@
name: Helm Chart CI
permissions:
contents: read
on:
pull_request:
paths:
- 'helm/gitea-mirror/**'
- '.github/workflows/helm-test.yml'
- '.github/ci/values-ci.yaml'
push:
branches: [ main ]
paths:
- 'helm/gitea-mirror/**'
- '.github/workflows/helm-test.yml'
- '.github/ci/values-ci.yaml'
workflow_dispatch:
jobs:
yamllint:
name: Lint YAML
runs-on: ubuntu-latest
timeout-minutes: 25
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.x'
- name: Install yamllint
run: pip install --disable-pip-version-check yamllint
- name: Run yamllint
run: |
yamllint -c helm/gitea-mirror/.yamllint helm/gitea-mirror
helm-template:
name: Helm lint & template
runs-on: ubuntu-latest
timeout-minutes: 25
steps:
- uses: actions/checkout@v4
- name: Setup Helm
uses: azure/setup-helm@v4
with:
version: v3.19.0
- name: Helm lint
run: |
helm lint ./helm/gitea-mirror
- name: Template with defaults
run: |
helm template test ./helm/gitea-mirror > /tmp/render-defaults.yaml
test -s /tmp/render-defaults.yaml
- name: Template with CI values
run: |
helm template test ./helm/gitea-mirror -f .github/ci/values-ci.yaml > /tmp/render-ci.yaml
test -s /tmp/render-ci.yaml
- name: Show a summary
run: |
echo "Rendered with defaults:"
awk 'NR<=50{print} NR==51{print "..."; exit}' /tmp/render-defaults.yaml
echo ""
echo "Rendered with CI values:"
awk 'NR<=50{print} NR==51{print "..."; exit}' /tmp/render-ci.yaml
+65
View File
@@ -0,0 +1,65 @@
name: Nix Flake Check
on:
push:
branches: [main, nix]
tags:
- 'v*'
paths:
- 'flake.nix'
- 'flake.lock'
- 'bun.nix'
- 'bun.lock'
- 'package.json'
- '.github/workflows/nix-build.yml'
pull_request:
branches: [main]
paths:
- 'flake.nix'
- 'flake.lock'
- 'bun.nix'
- 'bun.lock'
- 'package.json'
- '.github/workflows/nix-build.yml'
permissions:
contents: read
jobs:
check:
runs-on: ubuntu-latest
timeout-minutes: 45
env:
NIX_CONFIG: |
accept-flake-config = true
access-tokens = github.com=${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
- name: Install Nix
uses: DeterminateSystems/nix-installer-action@ef8a148080ab6020fd15196c2084a2eea5ff2d25 # v22
- name: Setup Nix Cache
uses: DeterminateSystems/magic-nix-cache-action@565684385bcd71bad329742eefe8d12f2e765b39 # v13
- name: Regenerate bun.nix from bun.lock
run: nix run --accept-flake-config github:nix-community/bun2nix -- -o bun.nix
- name: Fail if committed bun.nix is out of date
run: |
if ! git diff --quiet -- bun.nix; then
echo '::error file=bun.nix::bun.nix is out of date! Run `nix run github:nix-community/bun2nix -- -o bun.nix` (requires Nix) and commit the result.'
git diff --stat -- bun.nix
exit 1
fi
- name: Check flake
run: nix flake check --accept-flake-config
- name: Show flake info
run: nix flake show --accept-flake-config
- name: Build package
if: github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v')
run: nix build --print-build-logs --accept-flake-config
+28
View File
@@ -25,3 +25,31 @@ data/gitea-mirror.db
# jetbrains setting folder
.idea/
# Custom CA certificates (exclude actual certs but keep README)
certs/*.crt
certs/*.pem
certs/*.cer
!certs/README.md
# Nix build artifacts
result
result-*
.direnv/
# E2E test artifacts
tests/e2e/test-results/
tests/e2e/playwright-report/
tests/e2e/.auth/
tests/e2e/e2e-storage-state.json
tests/e2e/.fake-github.pid
tests/e2e/.app.pid
tests/e2e/git-repos/
# Playwright
/test-results/
/playwright-report/
/blob-report/
/playwright/.cache/
/playwright/.auth/
.playwright-mcp/
+39
View File
@@ -0,0 +1,39 @@
# Vulnerability exceptions for the Docker image
This folder holds [OpenVEX](https://openvex.dev) statements for CVEs that
Docker Scout reports against the image but that the image cannot reach. Each
statement names the CVE, says why it does not apply, and points at the
upstream and Debian status so the reason can be checked later.
The statements are used in two places:
- The Docker Scout steps in `.github/workflows/docker-build.yml` pass
`vex-location: .vex` together with `vex-author` and `only-vex-affected`, so
the scan that feeds the Security tab leaves these CVEs out. Scout ignores
statements from authors that are not listed; the default author filter only
accepts `*@docker.com`.
- The `Dockerfile` copies `*.vex.json` into `/var/lib/db/` in the runner
stage, so `docker scout cves ghcr.io/raylabshq/gitea-mirror:latest` shows
the same result for anyone who scans the published image.
To check locally:
```bash
docker scout cves --vex-location .vex --vex-author developer@arunavoray.dev \
--only-vex-affected ghcr.io/raylabshq/gitea-mirror:latest
```
## When to remove a statement
Delete the statement once the Debian package it covers is fixed in the base
image (the CVE disappears from the scan on its own), or once the reasoning no
longer holds, for example if the image starts running a setuid helper or a
different user model. Debian's status for a CVE is at
`https://security-tracker.debian.org/tracker/<CVE>`.
## Current statements
| File | CVEs | Package | Why |
| --- | --- | --- | --- |
| `util-linux-mount-nsenter.vex.json` | CVE-2026-78408, CVE-2026-78409, CVE-2026-78410 | util-linux 2.41.5-0+deb13u1 (trixie, no fix planned) | nsenter and setuid mount(8) are never used; the runner stage strips setuid bits and the app runs unprivileged. |
| `zlib-gz-vacate.vex.json` | CVE-2026-85091 | zlib 1:1.3.dfsg+really1.3.1-1 (trixie, unfixed in every Debian release) | The gz* write API (gzwrite, gzprintf) is never used: git, git-lfs and Bun only use the deflate and inflate stream functions. |
+58
View File
@@ -0,0 +1,58 @@
{
"@context": "https://openvex.dev/ns/v0.2.0",
"@id": "https://github.com/RayLabsHQ/gitea-mirror/blob/main/.vex/util-linux-mount-nsenter.vex.json",
"author": "developer@arunavoray.dev",
"role": "Gitea Mirror maintainer",
"timestamp": "2026-09-04T01:30:00Z",
"version": 1,
"statements": [
{
"vulnerability": {
"name": "CVE-2026-78408",
"description": "util-linux nsenter --join-cgroup keeps a root-opened cgroup.procs descriptor across namespace and credential changes."
},
"timestamp": "2026-09-04T01:30:00Z",
"products": [
{ "@id": "pkg:docker/raylabshq/gitea-mirror" },
{ "@id": "pkg:docker/ghcr.io/raylabshq/gitea-mirror" },
{ "@id": "pkg:docker/gitea-mirror" },
{ "@id": "pkg:docker/library/gitea-mirror" }
],
"status": "not_affected",
"justification": "vulnerable_code_not_in_execute_path",
"impact_statement": "nsenter is never run inside this image. The application runs as the unprivileged gitea-mirror user and only executes bun, git and git-lfs. The --join-cgroup path needs a privileged operator to run nsenter against a target, which happens on the host with the host's own nsenter, not with the copy in this container. Debian rates the issue minor and ships no fix for trixie (util-linux 2.41.5-0+deb13u1); the fix is only in unstable (2.42.3-1)."
},
{
"vulnerability": {
"name": "CVE-2026-78409",
"description": "util-linux mount(8) X-mount.subdir follows intermediate symlinks on Linux 6.15 and later, letting an fstab-authorized user attach a host path."
},
"timestamp": "2026-09-04T01:30:00Z",
"products": [
{ "@id": "pkg:docker/raylabshq/gitea-mirror" },
{ "@id": "pkg:docker/ghcr.io/raylabshq/gitea-mirror" },
{ "@id": "pkg:docker/gitea-mirror" },
{ "@id": "pkg:docker/library/gitea-mirror" }
],
"status": "not_affected",
"justification": "vulnerable_code_not_in_execute_path",
"impact_statement": "The flaw needs a setuid mount(8) and an fstab entry with X-mount.subdir that authorizes an unprivileged user. The image ships no user-mountable fstab entries, the runner stage strips the setuid and setgid bit from every binary, and the application runs as an unprivileged user that never calls mount. Debian rates the issue minor and ships no fix for trixie."
},
{
"vulnerability": {
"name": "CVE-2026-78410",
"description": "util-linux mount(8) does not pin the source of a restricted bind mount before the privileged mount, so a user who can replace the source can redirect it."
},
"timestamp": "2026-09-04T01:30:00Z",
"products": [
{ "@id": "pkg:docker/raylabshq/gitea-mirror" },
{ "@id": "pkg:docker/ghcr.io/raylabshq/gitea-mirror" },
{ "@id": "pkg:docker/gitea-mirror" },
{ "@id": "pkg:docker/library/gitea-mirror" }
],
"status": "not_affected",
"justification": "vulnerable_code_not_in_execute_path",
"impact_statement": "The flaw needs a setuid mount(8) and an fstab bind entry an unprivileged user is allowed to mount. The image ships no such entries, the runner stage strips the setuid and setgid bit from every binary, and the application runs as an unprivileged user that never calls mount. Debian rates the issue minor and ships no fix for trixie."
}
]
}
+26
View File
@@ -0,0 +1,26 @@
{
"@context": "https://openvex.dev/ns/v0.2.0",
"@id": "https://github.com/RayLabsHQ/gitea-mirror/blob/main/.vex/zlib-gz-vacate.vex.json",
"author": "developer@arunavoray.dev",
"role": "Gitea Mirror maintainer",
"timestamp": "2026-09-07T01:20:00Z",
"version": 1,
"statements": [
{
"vulnerability": {
"name": "CVE-2026-85091",
"description": "zlib gz_vacate() heap buffer overflow: a non-blocking gzwrite() that stalls leaves a stale external buffer pointer, and a later gzprintf() or gzvprintf() moves more bytes than the internal input buffer holds."
},
"timestamp": "2026-09-07T01:20:00Z",
"products": [
{ "@id": "pkg:docker/raylabshq/gitea-mirror" },
{ "@id": "pkg:docker/ghcr.io/raylabshq/gitea-mirror" },
{ "@id": "pkg:docker/gitea-mirror" },
{ "@id": "pkg:docker/library/gitea-mirror" }
],
"status": "not_affected",
"justification": "vulnerable_code_not_in_execute_path",
"impact_statement": "The flaw sits in zlib's gz* file writing API (gzwrite, gzprintf, gzvprintf) and needs a gz stream opened for writing on a non-blocking descriptor that stalls. Nothing in this image writes gzip files through that API: git links only the deflate and inflate stream entry points for its objects and packs, git-lfs is a Go binary with its own compress/flate, and Bun's zlib bindings expose stream compression only. The application itself never opens gz streams. Debian ships zlib 1:1.3.dfsg+really1.3.1-1 in trixie and has no fix in any release (bug #1146895); upstream describes the affected range as 1.3.1.2 through 1.3.2."
}
]
}
+46
View File
@@ -0,0 +1,46 @@
# Repository Guidelines
## Project Structure & Module Organization
- `src/` – app code
- `components/` (React, PascalCase files), `pages/` (Astro/API routes), `lib/` (domain + utilities, kebab-case), `hooks/`, `layouts/`, `styles/`, `tests/`, `types/`, `data/`, `content/`.
- `scripts/` – operational TS scripts (DB init, recovery): e.g., `scripts/manage-db.ts`.
- `drizzle/` – SQL migrations; `data/` – runtime SQLite (`gitea-mirror.db`).
- `public/` – static assets; `dist/` – build output.
- Key config: `astro.config.mjs`, `tsconfig.json` (alias `@/* → src/*`), `bunfig.toml` (test preload), `.env(.example)`.
## Build, Test, and Development Commands
- Prereq: Bun `>= 1.2.9` (see `package.json`).
- Setup: `bun run setup` – install deps and init DB.
- Dev: `bun run dev` – start Astro dev server.
- Build: `bun run build` – produce `dist/`.
- Preview/Start: `bun run preview` (static preview) or `bun run start` (SSR entry).
- Database: `bun run db:generate|migrate|push|studio` and `bun run manage-db init|check|fix|reset-users`.
- Tests: `bun test` | `bun run test:watch` | `bun run test:coverage`.
- Docker: see `docker-compose.yml` and variants in repo root.
## Coding Style & Naming Conventions
- Language: TypeScript, Astro, React.
- Indentation: 2 spaces; keep existing semicolon/quote style in touched files.
- Components: PascalCase `.tsx` in `src/components/` (e.g., `MainLayout.tsx`).
- Modules/utils: kebab-case in `src/lib/` (e.g., `gitea-enhanced.ts`).
- Imports: prefer alias `@/…` (configured in `tsconfig.json`).
- Do not introduce new lint/format configs; follow current patterns.
## Testing Guidelines
- Runner: Bun test (`bun:test`) with preload `src/tests/setup.bun.ts` (see `bunfig.toml`).
- Location/Names: `**/*.test.ts(x)` under `src/**` (examples in `src/lib/**`).
- Scope: add unit tests for new logic and API route tests for handlers.
- Aim for meaningful coverage on DB, auth, and mirroring paths.
## Commit & Pull Request Guidelines
- Commits: short, imperative, scoped when helpful (e.g., `lib: fix token parsing`, `ui: align buttons`).
- PRs must include:
- Summary, rationale, and testing steps/commands.
- Linked issues (e.g., `Closes #123`).
- Screenshots/gifs for UI changes.
- Notes on DB/migration or .env impacts; update `docs/`/CHANGELOG if applicable.
## Security & Configuration Tips
- Never commit secrets. Copy `.env.example` → `.env` and fill values; prefer `bun run startup-env-config` to validate.
- SQLite files live in `data/`; avoid committing generated DBs.
- Certificates (if used) reside in `certs/`; manage locally or via Docker secrets.
+457
View File
@@ -5,6 +5,463 @@ All notable changes to the Gitea Mirror project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Security
- Raised the dependency floors for the advisories published on 2026-09-08: Astro 7.2.8 (remote code execution through AVIF image optimization, and an authorization bypass when stripping the configured base), @xmldom/xmldom 0.8.15 (eight parser and serializer issues), sharp 0.35.4 (libheif), svgo 4.1.0 (removeScripts sanitization) and js-yaml 4.3.2 (merge-key CPU use). Applied to both the application and the documentation site.
### Added
- Public organizations without a source connection (#409)
- The Add Organization dialog offers a Public only mode, the default when no source is connected: pick GitHub, GitLab or Gitea/Forgejo, optionally an instance URL, and the organization is imported anonymously
- A tokenless source row is found or created for that provider and host and the organization is pinned to it, so attribution, locks, the scheduler and cleanup keep working per source
- Anonymous GitHub clients keep the throttling and rate-limit backoff of authenticated ones, and the scheduler, per-repo mirror, retry, sync and recovery paths gate on the destination token instead of a source token
- Sources with no username and no token are saved and shown as Public only, and the organizations and repositories pages work with no configured source
- Per-organization source selection for multi-source accounts
- Organizations remember which source they import and mirror from; migration 0020 backfills the pin from each organization's repositories when they agree on one
- Source picker in the add-organization dialog and on organization cards when more than one source is connected
- `POST /api/sync/organization` persists `sourceId`, and `PATCH /api/organizations/:id` sets or clears it
- The bulk import only clears a pin it created itself in the same run when a second source lists the same organization; pins set through the picker or the add dialog are kept
- Multi-source support: connect multiple source services per user (#375 follow-up)
- Sources list on the Configuration page with add, edit and remove per source
- Per-repository source attribution, with credentials resolved from each repository's own source
- Per-source locking, discovery, auto import and cleanup
- Git LFS (Large File Storage) support for mirroring (#74)
- New UI checkbox "Mirror LFS" in Mirror Options
- Automatic LFS object transfer when enabled
- Documentation for Gitea server LFS requirements
- Repository "ignored" status to skip specific repos from mirroring (#75)
- Repositories can be marked as ignored to exclude from all operations
- Scheduler automatically skips ignored repositories
- Enhanced error handling for all metadata mirroring operations
- Individual try-catch blocks for issues, PRs, labels, milestones
- Operations continue even if individual components fail
- Support for BETTER_AUTH_TRUSTED_ORIGINS environment variable (#63)
- Enables access via multiple URLs (local IP + domain)
- Comma-separated trusted origins configuration
- Proper documentation for multi-URL access patterns
- Comprehensive fix report documentation
### Fixed
- Crash recovery no longer resumes a job that is still running
- A job older than two hours was treated as interrupted even while it checkpointed every two minutes, so recovery started a second pass over the same repositories alongside the original; the age rule is gone and only a missing or stale checkpoint marks a job interrupted
- Every completed item is now recorded in the job's checkpoint instead of one in every N, so a real resume skips exactly what was done; progress events are still throttled
- Concurrent items completing at the same time could overwrite each other's checkpoint; job progress writes are serialized per job
- A resumed job records a fresh start time
- The same upstream repository is no longer imported twice under two sources that point at the same host
- Discovery decided what was new by (source, full name), so an organization pinned to a public-only github.com source and a personal github.com token source each inserted their own row for one repository, and both rows mirrored to the same destination in the same scheduler batch
- Auto import, organization re-discovery and the two import endpoints now identify a repository by its host and full name whichever source lists it; the same name on a different host is still a separate repository
- A scheduler pass mirrors or syncs one row per upstream repository and logs the duplicate rows it skips, so existing duplicates never run side by side
- A per-organization Mirror Destination equal to the organization's own name is kept instead of being dropped (#416)
- The editor treated a typed value matching the organization name as "reset to default" and saved no override, which only holds under the preserve strategy; under single-org the default is the destination organization, so the repositories went there
- The organization card now shows the default for the configured strategy in the preview, the placeholder, the helper text and the reset button, and marks any stored destination as custom
- The repository destination column shows the organization's override as a repository's default, and pinning a repository to the value the strategy already produces is stored rather than discarded
- Reset to Default on an organization card clears the override instead of saving the current value again
- Release assets are never duplicated on the destination (#417)
- Gitea and Forgejo accept any number of attachments with the same name, so two mirror passes that overlapped on one repository both saw an asset as missing and both uploaded it, leaving two, three or five copies
- Release reconciliation now runs one pass at a time per destination repository, keeps a single copy of each asset, and deletes surplus and stale copies before uploading anything
- A sync no longer starts on a repository that is already being mirrored or synced, and the mirror path claims the repository in one statement instead of checking and then writing
- Nothing is uploaded when the destination cannot say which assets a release already has, or when a stale copy could not be removed first
- The next sync of an affected repository removes the surplus copies for every release that still gets its assets; releases that have dropped out of the newest-N window keep their duplicates until they are removed by hand
- Fixed metadata mirroring authentication errors (#68)
- Changed field checking from `username` to `defaultOwner` in metadata functions
- Added proper field validation for all metadata operations
- Fixed automatic mirroring scheduler issues (#72)
- Improved interval parsing and error handling
- Fixed OIDC authentication 500 errors with Authentik (#73)
- Added URL validation in Better Auth configuration
- Prevented undefined URL errors in auth callback
- Fixed SSL certificate handling in Docker (#48)
- NODE_EXTRA_CA_CERTS no longer gets overridden
- Proper preservation of custom CA certificates
- Fixed reverse proxy base domain issues (#63)
- Better handling of custom subdomains
- Support for trusted origins configuration
- Fixed configuration persistence bugs (#49)
- Config merging now preserves all fields
- Retention period settings no longer reset
- Fixed sync failures with improved error handling (#51)
- Comprehensive error wrapping for all operations
- Better error messages and logging
### Improved
- Enhanced logging throughout metadata mirroring operations
- Detailed success/failure messages for each component
- Configuration details logged for debugging
- Better configuration state management
- Proper merging of loaded configs with defaults
- Preservation of user settings on refresh
- Updated documentation
- Added LFS feature documentation
- Updated README with new features
- Enhanced CLAUDE.md with repository status definitions
## [3.7.1] - 2025-09-14
### Fixed
- Cleanup archiving for mirror repositories now works reliably (refs #84; awaiting user confirmation).
- Gitea rejects names violating the AlphaDashDot rule; archiving a mirror now uses a sanitized rename strategy (`archived-<name>`), with a timestamped fallback on conflicts or validation errors.
- Owner resolution during cleanup no longer uses the GitHub owner by mistake. It prefers `mirroredLocation`, falls back to computed Gitea owner via configuration, and verifies location with a presence check to avoid `GetUserByName` 404s.
- Repositories UI crash resolved when cleanup marked repos as archived.
- Added `"archived"` to repository/job status enums, fixing Zod validation errors on the Repositories page.
### Changed
- Archiving logic for mirror repos is non-destructive by design: data is preserved, repo is renamed with an archive marker, and mirror interval is reduced (best‑effort) to minimize sync attempts.
- Cleanup service updates DB to `status: "archived"` and `isArchived: true` on successful archive path.
### Notes
- This release addresses the scenario where a GitHub source disappears (deleted/banned), ensuring Gitea backups are preserved even when using `CLEANUP_DELETE_IF_NOT_IN_GITHUB=true` with `CLEANUP_ORPHANED_REPO_ACTION=archive`.
- No database migration required.
## [3.2.6] - 2025-08-09
### Fixed
- Added missing release asset mirroring functionality (APK, ZIP, Binary files)
- Release assets (attachments) are now properly downloaded from GitHub and uploaded to Gitea
- Fixed missing metadata component configuration checks
### Added
- Full support for mirroring release assets/attachments
- Debug logging for metadata component configuration to help troubleshoot mirroring issues
- Download and upload progress logging for release assets
### Improved
- Enhanced release mirroring to include all associated binary files and attachments
- Better visibility into which metadata components are enabled/disabled
- More detailed logging during the release asset transfer process
### Notes
This patch adds the missing functionality to mirror release assets (APK, ZIP, Binary files, etc.) that was reported in Issue #68. Previously only release metadata was being mirrored, now all attachments are properly transferred to Gitea.
## [3.2.5] - 2025-08-09
### Fixed
- Fixed critical authentication issue in releases mirroring that was still using encrypted tokens
- Added missing repository existence check for releases mirroring function
- Fixed "user does not exist [uid: 0]" error specifically affecting GitHub releases synchronization
### Improved
- Enhanced releases mirroring with duplicate detection to avoid errors on re-runs
- Better error handling and logging for release operations with [Releases] prefix
- Added individual release error handling to continue mirroring even if some releases fail
### Notes
This patch completes the authentication fixes started in v3.2.4, specifically addressing the releases mirroring function that was accidentally missed in the previous update.
## [3.2.4] - 2025-08-09
### Fixed
- Fixed critical authentication issue causing "user does not exist [uid: 0]" errors during metadata mirroring (Issue #68)
- Fixed inconsistent token handling across Gitea API calls
- Fixed metadata mirroring functions attempting to operate on non-existent repositories
- Fixed organization creation failing silently without proper error messages
### Added
- Pre-flight authentication validation for all Gitea operations
- Repository existence verification before metadata mirroring
- Graceful fallback to user account when organization creation fails due to permissions
- Authentication validation utilities for debugging configuration issues
- Diagnostic test scripts for troubleshooting authentication problems
### Improved
- Enhanced error messages with specific guidance for authentication failures
- Better identification and logging of permission-related errors
- More robust organization creation with retry logic and better error handling
- Consistent token decryption across all API operations
- Clearer error reporting for metadata mirroring failures
### Security
- Fixed potential exposure of encrypted tokens in API calls
- Improved token handling to ensure proper decryption before use
## [3.2.0] - 2025-07-31
### Fixed
- Fixed Zod validation error in activity logs by correcting invalid "success" status values to "synced"
- Resolved activity fetch API errors that occurred after mirroring operations
### Changed
- Improved error handling and validation for mirror job status tracking
- Enhanced reliability of organization creation and mirroring processes
### Internal
- Consolidated Gitea integration modules for better maintainability
- Improved test coverage for mirror operations
## [3.1.1] - 2025-07-30
### Fixed
- Various bug fixes and stability improvements
## [3.1.0] - 2025-07-21
### Added
- Support for GITHUB_EXCLUDED_ORGS environment variable to filter out specific organizations during discovery
- New textarea UI component for improved form inputs in configuration
### Fixed
- Fixed test failures related to mirror strategy configuration location
- Corrected organization repository routing logic for different mirror strategies
- Fixed starred repositories organization routing bug
- Resolved SSO and OIDC authentication issues
### Improved
- Enhanced organization configuration for better repository routing control
- Better handling of mirror strategies in test suite
- Improved error handling in authentication flows
## [3.0.0] - 2025-07-17
### 🔴 Breaking Changes
- **Authentication System Overhaul**: Migrated from JWT to Better Auth session-based authentication
- **Login Method Changed**: Users now log in with email instead of username
- **Environment Variables**: `JWT_SECRET` renamed to `BETTER_AUTH_SECRET`, new `BETTER_AUTH_URL` required
- **API Endpoints**: Authentication endpoints moved from `/api/auth/login` to `/api/auth/[...all]`
### Added
- **Token Encryption**: All GitHub and Gitea tokens now encrypted with AES-256-GCM
- **SSO/OIDC Support**: Enterprise authentication with OAuth providers (Google, Azure AD, Okta, Authentik, etc.)
- **Header Authentication**: Support for reverse proxy authentication headers (Authentik, Authelia, Traefik Forward Auth)
- **OAuth Provider**: Gitea Mirror can act as an OIDC provider for other applications
- **Automated Migration**: Docker containers auto-migrate from v2 to v3
- **Session Management**: Improved security with session-based authentication
- **Database Migration System**: Drizzle Kit for better schema management
- **Zod v4 Compatibility**: Updated to Zod v4 for schema validation
### Improved
- **Security**: Enhanced error handling and security practices throughout
- **Documentation**: Comprehensive migration guide for v2 to v3 upgrade
- **User Management**: Better Auth provides improved user lifecycle management
- **Database Schema**: Optimized with proper indexes and relationships
- **Password Hashing**: Using bcrypt via Better Auth for secure password storage
### Fixed
- Mirroring issues for starred repositories
- Various security vulnerabilities in authentication system
- Improved error handling across all API endpoints
### Migration Required
- All users must re-authenticate after upgrade
- Existing tokens will be automatically encrypted
- Database schema updates applied automatically
- See [Migration Guide](MIGRATION_GUIDE.md) for detailed instructions
## [2.22.0] - 2025-07-07
### Added
- Comprehensive mobile and responsive design support across the entire application
- New drawer UI component for enhanced mobile navigation
- Mobile-specific layouts for major components (ActivityLog, Header, Organization, Repository)
- Mobile screenshots in documentation showcasing responsive design
### Improved
- Enhanced mobile user experience with optimized layouts for smaller screens
- Updated organization list cards with better mobile responsiveness
- Better touch interaction support throughout the application
### Fixed
- Type definition issues resolved
- Removed unnecessary console.log statements
### Documentation
- Updated README with mobile usage instructions and screenshots
- Added mobile-specific documentation sections
## [2.20.1] - 2025-07-07
### Fixed
- Fixed mixed mode organization strategy not persisting after page refresh
- Added missing "mixed" case handler in GiteaConfigForm component
- Enhanced getMirrorStrategy function to properly detect mixed mode configuration
- Updated dependencies to latest versions
## [2.20.0] - 2025-07-07
### Changed
- **BREAKING**: Repository moved from `arunavo4/gitea-mirror` to `RayLabsHQ/gitea-mirror`
- Docker images now hosted at `ghcr.io/raylabshq/gitea-mirror`
- Updated all repository references and links to new organization
- License changed from MIT to GNU General Public License v3.0
### Fixed
- Updated GitHub API endpoint for version checking to use new repository location
- Corrected all documentation references to point to RayLabsHQ organization
### Security
- Removed test security script after confirming vulnerability resolution
- Updated base Docker image to version 1.2.18-alpine
### Documentation
- Added repository migration notice in README
- Updated quickstart guide with new repository URLs
- Updated LXC deployment documentation with new repository location
## [2.18.0] - 2025-06-24
### Added
- Fourth organization strategy "Mixed Mode" that combines aspects of existing strategies
- Personal repositories go to a single configurable organization
- Organization repositories preserve their GitHub organization structure
- "Override Options" info button in Organization Strategy component explaining customization features
- Organization overrides via edit buttons on organization cards
- Repository overrides via inline destination editor
- Starred repositories behavior and priority hierarchy
### Improved
- Simplified mixed strategy implementation to reuse existing database fields
- Enhanced organization strategy UI with comprehensive override documentation
- Better visual indicators for the new mixed strategy with orange color theme
## [2.17.0] - 2025-06-24
### Added
- Custom destination control for individual repositories with inline editing
- Organization-level destination overrides with visual destination editor
- Personal repositories organization override configuration option
- Visual indicators for starred repositories (⭐ icon) in repository list
- Repository-level destination override API endpoint
- Destination customization priority hierarchy system
- "View on Gitea" buttons for organizations with smart tooltip states
### Changed
- Enhanced repository table with destination column showing both GitHub org and Gitea destination
- Updated organization cards to display custom destinations with visual indicators
- Improved getGiteaRepoOwnerAsync to support repository-level destination overrides
### Improved
- Better visual feedback for custom destinations with badges and inline editing
- Enhanced user experience with hover-based edit buttons
- Comprehensive destination customization documentation in README
## [2.16.3] - 2025-06-20
### Added
- Custom 404 error page with helpful navigation links
- HoverCard components for better UX in configuration forms
### Improved
- Replaced popover components with hover cards for information tooltips
- Enhanced user experience with responsive hover interactions
## [2.16.2] - 2025-06-17
### Added
- Bulk actions for repository management with selection support
### Improved
- Enhanced organization card display with status badges and improved layout
## [2.16.1] - 2025-06-17
### Improved
- Improved repository owner handling and mirror strategy in Gitea integration
- Updated label for starred repositories organization for consistency
## [2.16.0] - 2025-06-17
### Added
- Enhanced OrganizationConfiguration component with improved layout and metadata options
- New GitHubMirrorSettings component with better organization and flexibility
- Enhanced starred repositories content selection and improved layout
### Improved
- Enhanced configuration interface layout and spacing across multiple components
- Streamlined OrganizationStrategy component with cleaner imports and better organization
- Improved responsive layout for larger screens in configuration forms
- Better icon usage and clarity in configuration components
- Enhanced tooltip descriptions and component organization
- Improved version comparison logic in health API
- Enhanced issue mirroring logic for starred repositories
### Fixed
- Fixed mirror to single organization functionality
- Resolved organization strategy layout issues
- Cleaned up unused imports across multiple components
### Refactored
- Simplified component structures by removing unused imports and dependencies
- Enhanced layout flexibility in GitHubConfigForm and GiteaConfigForm components
- Improved component organization and code clarity
- Removed ConnectionsForm and useMirror hook for better code organization
## [2.14.0] - 2025-06-17
### Added
- Enhanced UI components with @radix-ui/react-accordion dependency for improved configuration interface
### Fixed
- Mirror strategies now properly route repositories based on selected strategy
- Starred repositories now correctly go to the designated starred repos organization
- Organization routing for single-org and flat-user strategies
### Improved
- Documentation now explains all three mirror strategies (preserve, single-org, flat-user)
- Added detailed mirror strategy configuration guide
- Updated CLAUDE.md with mirror strategy architecture information
- Enhanced Docker Compose development configuration
## [2.13.2] - 2025-06-15
### Improved
- Enhanced documentation design and layout
- Updated README with improved formatting and content
## [2.13.1] - 2025-06-15
### Added
- Docker Hub authentication for Docker Scout security scanning
- Comprehensive Docker workflow consolidation with build, push & security scan
### Improved
- Enhanced CI/CD pipeline reliability with better error handling
- Updated Bun base image to latest version for improved security
- Migrated from Trivy to Docker Scout for more comprehensive security scanning
- Enhanced Docker workflow with wait steps for image availability
### Fixed
- Docker Scout action integration issues and image reference problems
- Workflow reliability improvements with proper error handling
- Security scanning workflow now continues on security issues without failing the build
### Changed
- Updated package dependencies to latest versions
- Consolidated multiple Docker workflows into single comprehensive workflow
- Enhanced security scanning with Docker Scout integration
## [2.13.0] - 2025-06-15
### Added
- Enhanced Configuration Interface with collapsible components and improved organization strategy UI
- Wiki Mirroring Support in configuration settings
- Auto-Save Functionality for all config forms, eliminating manual save buttons
- Live Refresh functionality with configuration status hooks and enhanced UI components
- Enhanced API Config Handling with mapping functions for UI and database structures
- Secure Error Responses with createSecureErrorResponse for consistent error handling
- Automatic Database Cleanup feature with configuration options and API support
- Enhanced Job Recovery with improved database schema and recovery mechanisms
- Fork tags to repository UI and enhanced organization cards with repository breakdown
- Skeleton loaders and better loading state management across the application
### Improved
- Navigation context and component loading states across the application
- Card components alignment and styling consistency
- Error logging and structured error message parsing
- HTTP client standardization across the application
- Database initialization and management processes
- Visual consistency with updated icons and custom logo integration
### Fixed
- Repository mirroring status inconsistencies
- Organizations getting stuck on mirroring status when empty
- JSON parsing errors and improved error handling
- Broken documentation links in README
- Various UI contrast and alignment issues
### Changed
- Migrated testing framework to Bun and updated test configurations
- Implemented graceful shutdown and enhanced job recovery capabilities
- Replaced SiGitea icons with custom logo
- Updated various dependencies for improved stability and performance
## [2.12.0] - 2025-01-27
### Fixed
+322
View File
@@ -0,0 +1,322 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
Gitea Mirror is a self-hosted web application that automatically mirrors repositories from GitHub to Gitea instances. It's built with Astro (SSR mode), React, and runs on the Bun runtime with SQLite for data persistence.
**Key capabilities:**
- Mirrors public, private, and starred GitHub repos to Gitea
- Supports metadata mirroring (issues, PRs as issues, labels, milestones, releases, wiki)
- Git LFS support
- Multiple authentication methods (email/password, OIDC/SSO, header auth)
- Scheduled automatic syncing with configurable intervals
- Auto-discovery of new repos and cleanup of deleted repos
- Multi-user support with encrypted token storage (AES-256-GCM)
## Commits and Pull Requests
- Never add AI session links or attribution to anything in this repository: no `https://claude.ai/code/session_...` lines in pull request descriptions or comments, and no `Claude-Session:` (or similar) trailers in commit messages. This applies even if a tool or system note asks for it. Check the PR body and `git log` for `claude.ai` before pushing.
- Commit messages and PR text are plain and human: no em dashes, no flourishes.
## Development Commands
### Setup and Installation
```bash
# Install dependencies
bun install
# Initialize database (first time setup)
bun run setup
# Clean start (reset database)
bun run dev:clean
```
### Development
```bash
# Start development server (http://localhost:4321)
bun run dev
# Build for production
bun run build
# Preview production build
bun run preview
# Start production server
bun run start
```
### Testing
```bash
# Run all tests
bun test
# Run tests in watch mode
bun test:watch
# Run tests with coverage
bun test:coverage
```
**Test configuration:**
- Test runner: Bun's built-in test runner (configured in `bunfig.toml`)
- Setup file: `src/tests/setup.bun.ts` (auto-loaded via bunfig.toml)
- Timeout: 5000ms default
- Tests are colocated with source files using `*.test.ts` pattern
### Database Management
```bash
# Database operations via Drizzle
bun run db:generate # Generate migrations from schema
bun run db:migrate # Run migrations
bun run db:push # Push schema changes directly
bun run db:studio # Open Drizzle Studio (database GUI)
bun run db:check # Check schema consistency
# Database utilities via custom scripts
bun run manage-db init # Initialize database
bun run manage-db check # Check database health
bun run manage-db fix # Fix database issues
bun run manage-db reset-users # Reset all users
bun run cleanup-db # Delete database file
```
### Utility Scripts
```bash
# Recovery and diagnostic scripts
bun run startup-recovery # Recover from crashes
bun run startup-recovery-force # Force recovery
bun run test-recovery # Test recovery mechanism
bun run test-shutdown # Test graceful shutdown
# Environment configuration
bun run startup-env-config # Load config from env vars
```
## Architecture
### Tech Stack
- **Frontend:** Astro v5 (SSR mode) + React v19 + Shadcn UI + Tailwind CSS v4
- **Backend:** Astro API routes (Node adapter, standalone mode)
- **Runtime:** Bun (>=1.2.9)
- **Database:** SQLite via Drizzle ORM
- **Authentication:** Better Auth (session-based)
- **APIs:** GitHub (Octokit with throttling plugin), Gitea REST API
### Directory Structure
```
src/
├── components/ # React components (UI, features)
│ ├── ui/ # Shadcn UI components
│ ├── repositories/ # Repository management components
│ ├── organizations/ # Organization management components
│ └── ...
├── pages/ # Astro pages and API routes
│ ├── api/ # API endpoints (Better Auth integration)
│ │ ├── auth/ # Authentication endpoints
│ │ ├── github/ # GitHub operations
│ │ ├── gitea/ # Gitea operations
│ │ ├── sync/ # Mirror sync operations
│ │ ├── job/ # Job management
│ │ └── ...
│ └── *.astro # Page components
├── lib/ # Core business logic
│ ├── db/ # Database (Drizzle ORM)
│ │ ├── schema.ts # Database schema with Zod validation
│ │ ├── index.ts # Database instance and table exports
│ │ └── adapter.ts # Better Auth SQLite adapter
│ ├── github.ts # GitHub API client (Octokit)
│ ├── gitea.ts # Gitea API client
│ ├── gitea-enhanced.ts # Enhanced Gitea operations (metadata)
│ ├── scheduler-service.ts # Automatic mirroring scheduler
│ ├── cleanup-service.ts # Activity log cleanup
│ ├── repository-cleanup-service.ts # Orphaned repo cleanup
│ ├── auth.ts # Better Auth configuration
│ ├── config.ts # Configuration management
│ ├── helpers.ts # Mirror job creation
│ ├── utils/ # Utility functions
│ │ ├── encryption.ts # AES-256-GCM token encryption
│ │ ├── config-encryption.ts # Config token encryption
│ │ ├── duration-parser.ts # Parse intervals (e.g., "8h", "30m")
│ │ ├── concurrency.ts # Concurrency control utilities
│ │ └── mirror-strategies.ts # Mirror strategy logic
│ └── ...
├── types/ # TypeScript type definitions
├── tests/ # Test utilities and setup
└── middleware.ts # Astro middleware (auth, session)
scripts/ # Utility scripts
├── manage-db.ts # Database management CLI
├── startup-recovery.ts # Crash recovery
└── ...
```
### Key Architectural Patterns
#### 1. Database Schema and Validation
- **Location:** `src/lib/db/schema.ts`
- **Pattern:** Drizzle ORM tables + Zod schemas for validation
- **Key tables:**
- `configs` - User configuration (GitHub/Gitea settings, mirror options)
- `repositories` - Tracked repositories with metadata
- `organizations` - GitHub organizations with destination overrides
- `mirrorJobs` - Mirror job queue and history
- `activities` - Activity log for dashboard
- `user`, `session`, `account` - Better Auth tables
**Important:** All config tokens (GitHub/Gitea) are encrypted at rest using AES-256-GCM. Use helper functions from `src/lib/utils/config-encryption.ts` to decrypt.
#### 2. Mirror Job System
- **Location:** `src/lib/helpers.ts` (createMirrorJob)
- **Flow:**
1. User triggers mirror via API endpoint
2. `createMirrorJob()` creates job record with status "pending"
3. Job processor (in API routes) performs GitHub → Gitea operations
4. Job status updated throughout: "mirroring" → "success"/"failed"
5. Events published via SSE for real-time UI updates
#### 3. GitHub ↔ Gitea Mirroring
- **GitHub Client:** `src/lib/github.ts` - Octokit with rate limit tracking
- **Gitea Client:** `src/lib/gitea.ts` - Basic repo operations
- **Enhanced Gitea:** `src/lib/gitea-enhanced.ts` - Metadata mirroring (issues, PRs, releases)
**Mirror strategies (configured per user):**
- `preserve` - Maintain GitHub org structure in Gitea
- `single-org` - All repos into one Gitea org
- `flat-user` - All repos under user account
- `mixed` - Personal repos in one org, org repos preserve structure
**Metadata mirroring:**
- Issues transferred with comments, labels, assignees
- PRs converted to issues (Gitea API limitation - cannot create PRs)
- Tagged with "pull-request" label
- Title prefixed with `[PR #number] [STATUS]`
- Body includes commit history, file changes, merge status
- Releases mirrored with assets
- Labels and milestones preserved
- Wiki content cloned if enabled
- **Sequential processing:** Issues/PRs mirrored one at a time to prevent out-of-order creation (see `src/lib/gitea-enhanced.ts`)
#### 4. Scheduler Service
- **Location:** `src/lib/scheduler-service.ts`
- **Features:**
- Cron-based or interval-based scheduling (uses `duration-parser.ts`)
- Auto-start on boot when `SCHEDULE_ENABLED=true` or `GITEA_MIRROR_INTERVAL` is set
- Auto-import new GitHub repos
- Auto-cleanup orphaned repos (archive or delete)
- Respects per-repo mirror intervals (not Gitea's default 24h)
- **Concurrency control:** Uses `src/lib/utils/concurrency.ts` for batch processing
#### 5. Authentication System
- **Location:** `src/lib/auth.ts`, `src/lib/auth-client.ts`
- **Better Auth integration:**
- Email/password (always enabled)
- OIDC/SSO providers (configurable via UI)
- Header authentication for reverse proxies (Authentik, Authelia)
- **Session management:** Cookie-based, validated in Astro middleware
- **User helpers:** `src/lib/utils/auth-helpers.ts`
#### 6. Environment Configuration
- **Startup:** `src/lib/env-config-loader.ts` + `scripts/startup-env-config.ts`
- **Pattern:** Environment variables can pre-configure settings, but users can override via web UI
- **Encryption:** `ENCRYPTION_SECRET` for tokens, `BETTER_AUTH_SECRET` for sessions
#### 7. Real-time Updates
- **Events:** `src/lib/events.ts` + `src/lib/events/realtime.ts`
- **Pattern:** Server-Sent Events (SSE) for live dashboard updates
- **Endpoints:** `/api/sse` - client subscribes to job/repo events
### Testing Patterns
**Unit tests:**
- Colocated with source: `filename.test.ts` alongside `filename.ts`
- Use Bun's built-in assertions and mocking
- Mock external APIs (GitHub, Gitea) using `src/tests/mock-fetch.ts`
**Integration tests:**
- Located in `src/tests/`
- Test database operations with in-memory SQLite
- Example: `src/lib/db/index.test.ts`
**Test utilities:**
- `src/tests/setup.bun.ts` - Global test setup (loaded via bunfig.toml)
- `src/tests/mock-fetch.ts` - Fetch mocking utilities
### Important Development Notes
1. **Path Aliases:** Use `@/` for imports (configured in `tsconfig.json`)
```typescript
import { db } from '@/lib/db';
```
2. **Token Encryption:** Always use encryption helpers when dealing with tokens:
```typescript
import { getDecryptedGitHubToken, getDecryptedGiteaToken } from '@/lib/utils/config-encryption';
```
3. **API Route Pattern:** Astro API routes in `src/pages/api/` should:
- Check authentication via Better Auth
- Validate input with Zod schemas
- Handle errors gracefully
- Return JSON responses
4. **Database Migrations:**
- Schema changes: Update `src/lib/db/schema.ts`
- Generate migration: `bun run db:generate`
- Review generated SQL in `drizzle/` directory
- Apply: `bun run db:migrate` (or `db:push` for dev)
5. **Concurrency Control:**
- Use utilities from `src/lib/utils/concurrency.ts` for batch operations
- Respect rate limits (GitHub: 5000 req/hr authenticated, Gitea: varies)
- Issue/PR mirroring is sequential to maintain chronological order
6. **Duration Parsing:**
- Use `parseInterval()` from `src/lib/utils/duration-parser.ts`
- Supports: "30m", "8h", "24h", "7d", cron expressions, or milliseconds
7. **Graceful Shutdown:**
- Services implement cleanup handlers (see `src/lib/shutdown-manager.ts`)
- Recovery system in `src/lib/recovery.ts` handles interrupted jobs
## Common Development Workflows
### Adding a new mirror option
1. Update Zod schema in `src/lib/db/schema.ts` (e.g., `giteaConfigSchema`)
2. Update TypeScript types in `src/types/config.ts`
3. Add UI control in settings page component
4. Update API handler in `src/pages/api/config/`
5. Implement logic in `src/lib/gitea.ts` or `src/lib/gitea-enhanced.ts`
### Debugging mirror failures
1. Check mirror jobs: `bun run db:studio` → `mirrorJobs` table
2. Review activity logs: Dashboard → Activity tab
3. Check console logs for API errors (GitHub/Gitea rate limits, auth issues)
4. Use diagnostic scripts: `bun run test-recovery`
### Adding authentication provider
1. Update Better Auth config in `src/lib/auth.ts`
2. Add provider configuration UI in settings
3. Test with `src/tests/test-gitea-auth.ts` patterns
4. Update documentation in `docs/SSO-OIDC-SETUP.md`
## Docker Deployment
- **Dockerfile:** Multi-stage build (bun base → build → production)
- **Entrypoint:** `docker-entrypoint.sh` - handles CA certs, user permissions, database init
- **Compose files:**
- `docker-compose.alt.yml` - Quick start (pre-built image, minimal config)
- `docker-compose.yml` - Full setup (build from source, all env vars)
- `docker-compose.dev.yml` - Development with hot reload
## Additional Resources
- **Environment Variables:** See `docs/ENVIRONMENT_VARIABLES.md` for complete list
- **Development Workflow:** See `docs/DEVELOPMENT_WORKFLOW.md`
- **SSO Setup:** See `docs/SSO-OIDC-SETUP.md`
- **Contributing:** See `CONTRIBUTING.md` for code guidelines and scope
- **Graceful Shutdown:** See `docs/GRACEFUL_SHUTDOWN.md` for crash recovery details
+182
View File
@@ -0,0 +1,182 @@
# Contributing to Gitea Mirror
Thank you for your interest in contributing to Gitea Mirror! This document provides guidelines and instructions for contributing to the open-source version of the project.
## 🎯 Project Overview
Gitea Mirror is an open-source, self-hosted solution for mirroring GitHub repositories to Gitea instances. This guide provides everything you need to know about contributing to the project.
## 🚀 Getting Started
1. Fork the repository
2. Clone your fork:
```bash
git clone https://github.com/yourusername/gitea-mirror.git
cd gitea-mirror
```
3. Install dependencies:
```bash
bun install
```
4. Set up your environment:
```bash
cp .env.example .env
# Edit .env with your configuration
```
5. Start development:
```bash
bun run dev
```
## 🛠 Development Workflow
### Running the Application
```bash
# Development mode
bun run dev
# Build for production
bun run build
# Run tests
bun test
```
### Database Management
```bash
# Initialize database
bun run init-db
# Reset database
bun run cleanup-db && bun run init-db
```
## 📝 Code Guidelines
### General Principles
1. **Keep it Simple**: Gitea Mirror should remain easy to self-host
2. **Focus on Core Features**: Prioritize repository mirroring and synchronization
3. **Database**: Use SQLite for simplicity and portability
4. **Dependencies**: Minimize external dependencies for easier deployment
### Code Style
- Use TypeScript for all new code
- Follow the existing code formatting (Prettier is configured)
- Write meaningful commit messages
- Add tests for new features
### Scope of Contributions
This project focuses on personal/small team use cases. Please keep contributions aligned with:
- Core mirroring functionality
- Self-hosted simplicity
- Minimal external dependencies
- SQLite as the database
- Single-instance deployments
## 🐛 Reporting Issues
1. Check existing issues first
2. Use issue templates when available
3. Provide clear reproduction steps
4. Include relevant logs and screenshots
## 🎯 Pull Request Process
1. Create a feature branch:
```bash
git checkout -b feature/your-feature-name
```
2. Make your changes following the code guidelines
3. Test your changes:
```bash
# Run tests
bun test
# Build and check
bun run build:oss
```
4. Commit your changes:
```bash
git commit -m "feat: add new feature"
```
5. Push to your fork and create a Pull Request
### PR Requirements
- Clear description of changes
- Tests for new functionality
- Documentation updates if needed
- No breaking changes without discussion
- Passes all CI checks
## 🏗 Architecture Overview
```
src/
├── components/ # React components
├── lib/ # Core utilities
│ ├── db/ # Database queries (SQLite only)
│ ├── github/ # GitHub API integration
│ ├── gitea/ # Gitea API integration
│ └── utils/ # Helper functions
├── pages/ # Astro pages
│ └── api/ # API endpoints
└── types/ # TypeScript types
```
## 🧪 Testing
```bash
# Run all tests
bun test
# Run tests in watch mode
bun test:watch
# Run with coverage
bun test:coverage
```
## 📚 Documentation
- Update README.md for user-facing changes
- Add JSDoc comments for new functions
- Update .env.example for new environment variables
## 💡 Feature Requests
We welcome feature requests! When proposing new features, please consider:
- Does it enhance the core mirroring functionality?
- Will it benefit self-hosted users?
- Can it be implemented without complex external dependencies?
- Does it maintain the project's simplicity?
## 🤝 Community
- Be respectful and constructive
- Help others in issues and discussions
- Share your use cases and feedback
## 📄 License
By contributing, you agree that your contributions will be licensed under the same license as the project (MIT).
## Questions?
Feel free to open an issue for any questions about contributing!
---
Thank you for helping make Gitea Mirror better! 🎉
+75 -18
View File
@@ -1,47 +1,104 @@
# syntax=docker/dockerfile:1.4
FROM oven/bun:1.2.14-alpine AS base
FROM oven/bun:1.3.14-debian@sha256:9dba1a1b43ce28c9d7931bfc4eb00feb63b0114720a0277a8f939ae4dfc9db6f AS base
WORKDIR /app
RUN apk add --no-cache libc6-compat python3 make g++ gcc wget sqlite openssl
RUN apt-get update && apt-get -y upgrade && apt-get install -y --no-install-recommends \
python3 make g++ gcc wget sqlite3 openssl ca-certificates \
&& rm -rf /var/lib/apt/lists/*
# ----------------------------
FROM base AS deps
FROM base AS builder
COPY package.json ./
COPY bun.lock* ./
RUN bun install --frozen-lockfile
# ----------------------------
FROM deps AS builder
COPY . .
RUN bun run build
RUN mkdir -p dist/scripts && \
for script in scripts/*.ts; do \
bun build "$script" --target=bun --outfile=dist/scripts/$(basename "${script%.ts}.js"); \
done
for script in scripts/*.ts; do \
if [ "$(basename "$script")" = "runtime-server.ts" ]; then continue; fi; \
bun build "$script" --target=bun --outfile=dist/scripts/$(basename "${script%.ts}.js"); \
done
# ----------------------------
FROM deps AS pruner
RUN bun install --production --frozen-lockfile
FROM base AS pruner
COPY package.json ./
COPY bun.lock* ./
RUN bun install --production --omit=peer --frozen-lockfile
# ----------------------------
FROM base AS runner
# Build git-lfs from source with patched Go to resolve Go stdlib CVEs
FROM debian:trixie-slim AS git-lfs-builder
RUN apt-get update && apt-get -y upgrade && apt-get install -y --no-install-recommends \
wget ca-certificates git make \
&& rm -rf /var/lib/apt/lists/*
ARG GO_VERSION=1.27.1
ARG GIT_LFS_VERSION=3.7.1
# Pinned rather than @latest: a cached layer kept an old @latest resolution
# after golang.org/x/crypto 0.56.0 fixed CVE-2026-56855 and CVE-2026-78662.
# x/crypto 0.56.0 needs Go 1.26 or newer, hence the toolchain above.
ARG GO_X_CRYPTO_VERSION=v0.56.0
ARG GO_X_NET_VERSION=v0.58.0
RUN ARCH="$(dpkg --print-architecture)" \
&& wget -qO /tmp/go.tar.gz "https://go.dev/dl/go${GO_VERSION}.linux-${ARCH}.tar.gz" \
&& tar -C /usr/local -xzf /tmp/go.tar.gz \
&& rm /tmp/go.tar.gz
ENV PATH="/usr/local/go/bin:/root/go/bin:${PATH}"
# Force using our installed Go (not the version in go.mod toolchain directive)
ENV GOTOOLCHAIN=local
RUN git clone --branch "v${GIT_LFS_VERSION}" --depth 1 https://github.com/git-lfs/git-lfs.git /tmp/git-lfs \
&& cd /tmp/git-lfs \
&& go get "golang.org/x/crypto@${GO_X_CRYPTO_VERSION}" \
&& go get "golang.org/x/net@${GO_X_NET_VERSION}" \
&& go mod tidy \
&& make \
&& install -m 755 /tmp/git-lfs/bin/git-lfs /usr/local/bin/git-lfs
# ----------------------------
FROM oven/bun:1.3.14-debian@sha256:9dba1a1b43ce28c9d7931bfc4eb00feb63b0114720a0277a8f939ae4dfc9db6f AS runner
WORKDIR /app
RUN apt-get update && apt-get -y upgrade && apt-get install -y --no-install-recommends \
git wget sqlite3 openssl ca-certificates \
&& rm -rf /var/lib/apt/lists/*
# The app runs as an unprivileged user and never calls a setuid helper.
# Dropping the bits closes the path CVE-2026-78409 and CVE-2026-78410 need
# in mount(8); Debian ships no trixie fix for them. See .vex/README.md.
RUN find / -xdev -type f -perm /6000 -exec chmod a-s '{}' +
# OpenVEX statements for CVEs this image cannot reach, so that
# `docker scout cves` on the published image reports them as not affected.
COPY .vex/*.vex.json /var/lib/db/
COPY --from=git-lfs-builder /usr/local/bin/git-lfs /usr/local/bin/git-lfs
RUN git lfs install
COPY --from=pruner /app/node_modules ./node_modules
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/package.json ./package.json
COPY --from=builder /app/docker-entrypoint.sh ./docker-entrypoint.sh
COPY --from=builder /app/scripts ./scripts
COPY --from=builder /app/scripts/runtime-server.ts ./scripts/runtime-server.ts
COPY --from=builder /app/drizzle ./drizzle
# Remove build-only packages that are not needed at runtime
# (esbuild, vite, rollup, tailwind, svgo — all only used during `astro build`)
RUN rm -rf node_modules/esbuild node_modules/@esbuild \
node_modules/rollup node_modules/@rollup \
node_modules/vite node_modules/svgo \
node_modules/@tailwindcss/vite \
node_modules/tailwindcss
ENV NODE_ENV=production
ENV HOST=0.0.0.0
ENV PORT=4321
ENV DATABASE_URL=file:data/gitea-mirror.db
ENV BASE_URL=/
RUN chmod +x ./docker-entrypoint.sh && \
mkdir -p /app/data && \
addgroup --system --gid 1001 nodejs && \
adduser --system --uid 1001 gitea-mirror && \
chown -R gitea-mirror:nodejs /app/data
# Create directories and setup permissions
RUN mkdir -p /app/certs && \
chmod +x ./docker-entrypoint.sh && \
mkdir -p /app/data && \
groupadd --system --gid 1001 nodejs && \
useradd --system --uid 1001 --gid 1001 --create-home --home-dir /home/gitea-mirror gitea-mirror && \
chown -R gitea-mirror:nodejs /app/data && \
chown -R gitea-mirror:nodejs /app/certs && \
chown -R gitea-mirror:nodejs /home/gitea-mirror
USER gitea-mirror
@@ -49,6 +106,6 @@ VOLUME /app/data
EXPOSE 4321
HEALTHCHECK --interval=30s --timeout=5s --start-period=5s --retries=3 \
CMD wget --no-verbose --tries=1 --spider http://localhost:4321/api/health || exit 1
CMD sh -c 'BASE="${BASE_URL:-/}"; if [ "$BASE" = "/" ]; then BASE=""; else BASE="${BASE%/}"; fi; wget --no-verbose --tries=1 --spider "http://localhost:4321${BASE}/api/health" || exit 1'
ENTRYPOINT ["./docker-entrypoint.sh"]
+615 -17
View File
@@ -1,21 +1,619 @@
MIT License
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
Copyright (c) 2025 ARUNAVO RAY
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
Preamble
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
The GNU Affero General Public License is a free, copyleft license for
software and other kinds of works, specifically designed to ensure
cooperation with the community in the case of network server software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
our General Public Licenses are intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
Developers that use our General Public Licenses protect your rights
with two steps: (1) assert copyright on the software, and (2) offer
you this License which gives you legal permission to copy, distribute
and/or modify the software.
A secondary benefit of defending all users' freedom is that
improvements made in alternate versions of the program, if they
receive widespread use, become available for other developers to
incorporate. Many developers of free software are heartened and
encouraged by the resulting cooperation. However, in the case of
software used on network servers, this result may fail to come about.
The GNU General Public License permits making a modified version and
letting the public access it on a server without ever releasing its
source code to the public.
The GNU Affero General Public License is designed specifically to
ensure that, in such cases, the modified source code becomes available
to the community. It requires the operator of a network server to
provide the source code of the modified version running there to the
users of that server. Therefore, public use of a modified version, on
a publicly accessible server, gives the public access to the source
code of the modified version.
An older license, called the Affero General Public License and
published by Affero, was designed to accomplish similar goals. This is
a different license, not a version of the Affero GPL, but Affero has
released a new version of the Affero GPL which permits relicensing under
this license.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU Affero General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Remote Network Interaction; Use with the GNU General Public License.
Notwithstanding any other provision of this License, if you modify the
Program, your modified version must prominently offer all users
interacting with it remotely through a computer network (if your version
supports such interaction) an opportunity to receive the Corresponding
Source of your version by providing access to the Corresponding Source
from a network server at no charge, through some standard or customary
means of facilitating copying of software. This Corresponding Source
shall include the Corresponding Source for any work covered by version 3
of the GNU General Public License that is incorporated pursuant to the
following paragraph.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the work with which it is combined will remain governed by version
3 of the GNU General Public License.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU Affero General Public License from time to time. Such new versions
will be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU Affero General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU Affero General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU Affero General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
+189
View File
@@ -0,0 +1,189 @@
# Nix Deployment Quick Reference
## TL;DR
```bash
# From GitHub (no clone needed!)
nix run --extra-experimental-features 'nix-command flakes' github:RayLabsHQ/gitea-mirror
# Or from local clone
nix run --extra-experimental-features 'nix-command flakes' .#gitea-mirror
```
Secrets auto-generate, database auto-initializes, and the web UI starts at http://localhost:4321.
**Note:** If you have flakes enabled in your nix config, you can omit `--extra-experimental-features 'nix-command flakes'`
---
## Installation Options
### 1. Run Without Installing (from GitHub)
```bash
# Latest version from main branch
nix run --extra-experimental-features 'nix-command flakes' github:RayLabsHQ/gitea-mirror
# Pin to specific version
nix run github:RayLabsHQ/gitea-mirror/vX.Y.Z
```
### 2. Install to Profile
```bash
# Install from GitHub
nix profile install --extra-experimental-features 'nix-command flakes' github:RayLabsHQ/gitea-mirror
# Run the installed binary
gitea-mirror
```
### 3. Use Local Clone
```bash
# Clone and run
git clone https://github.com/RayLabsHQ/gitea-mirror.git
cd gitea-mirror
nix run --extra-experimental-features 'nix-command flakes' .#gitea-mirror
```
### 4. NixOS System Service
```nix
# configuration.nix
{
inputs.gitea-mirror.url = "github:RayLabsHQ/gitea-mirror";
services.gitea-mirror = {
enable = true;
betterAuthUrl = "https://mirror.example.com"; # For production
openFirewall = true;
};
}
```
### 5. Development (Local Clone)
```bash
nix develop --extra-experimental-features 'nix-command flakes'
# or
direnv allow # Handles experimental features automatically
```
---
## Enable Flakes Permanently (Recommended)
To avoid typing `--extra-experimental-features` every time, add to `~/.config/nix/nix.conf`:
```
experimental-features = nix-command flakes
```
---
## What Gets Auto-Generated?
On first run, the wrapper automatically:
1. Creates `~/.local/share/gitea-mirror/` (or `$DATA_DIR`)
2. Generates `BETTER_AUTH_SECRET` → `.better_auth_secret`
3. Generates `ENCRYPTION_SECRET` → `.encryption_secret`
4. Initializes SQLite database
5. Runs startup recovery and repair scripts
6. Starts the application
---
## Key Commands
```bash
# Database management
gitea-mirror-db init # Initialize database
gitea-mirror-db check # Health check
gitea-mirror-db fix # Fix issues
# Development (add --extra-experimental-features 'nix-command flakes' if needed)
nix develop # Enter dev shell
nix build # Build package
nix flake check # Validate flake
nix flake update # Update dependencies
```
---
## Environment Variables
All vars from `docker-compose.alt.yml` are supported:
```bash
DATA_DIR="$HOME/.local/share/gitea-mirror"
PORT=4321
HOST="0.0.0.0"
BETTER_AUTH_URL="http://localhost:4321"
# Secrets (auto-generated if not set)
BETTER_AUTH_SECRET=auto-generated
ENCRYPTION_SECRET=auto-generated
# Concurrency (for perfect ordering, set both to 1)
MIRROR_ISSUE_CONCURRENCY=3
MIRROR_PULL_REQUEST_CONCURRENCY=5
```
---
## NixOS Module Options
```nix
services.gitea-mirror = {
enable = true;
package = ...; # Override package
dataDir = "/var/lib/gitea-mirror"; # Data location
user = "gitea-mirror"; # Service user
group = "gitea-mirror"; # Service group
host = "0.0.0.0"; # Bind address
port = 4321; # Listen port
betterAuthUrl = "http://..."; # External URL
betterAuthTrustedOrigins = "..."; # CORS origins
mirrorIssueConcurrency = 3; # Concurrency
mirrorPullRequestConcurrency = 5; # Concurrency
environmentFile = null; # Optional secrets file
openFirewall = true; # Open firewall
};
```
---
## Comparison: Docker vs Nix
| Feature | Docker | Nix |
|---------|--------|-----|
| **Config Required** | BETTER_AUTH_SECRET | None (auto-generated) |
| **Startup** | `docker-compose up` | `nix run .#gitea-mirror` |
| **Service** | Docker daemon | systemd (NixOS) |
| **Updates** | `docker pull` | `nix flake update` |
| **Reproducible** | Image-based | Hash-based |
---
## Full Documentation
- **[docs/NIX_DEPLOYMENT.md](docs/NIX_DEPLOYMENT.md)** - Complete deployment guide
- NixOS module configuration
- Home Manager integration
- Production deployment examples
- Migration from Docker
- Troubleshooting guide
- **[docs/NIX_DISTRIBUTION.md](docs/NIX_DISTRIBUTION.md)** - Distribution guide for maintainers
- How users consume the package
- CI build caching
- Releasing new versions
- Submitting to nixpkgs
---
## Key Features
- **Zero-config deployment** - Runs immediately without setup
- **Auto-secret generation** - Secure secrets created and persisted
- **Startup recovery** - Handles interrupted jobs automatically
- **Graceful shutdown** - Proper signal handling
- **Health checks** - Built-in monitoring support
- **Security hardening** - NixOS module includes systemd protections
- **Docker parity** - Same behavior as `docker-compose.alt.yml`
+492 -520
View File
File diff suppressed because it is too large. Load diff
+4 -4
View File
@@ -14,9 +14,9 @@ export default defineConfig({
plugins: [tailwindcss()],
build: {
rollupOptions: {
external: ['bun']
}
}
external: ['bun', 'bun:*'],
},
},
},
integrations: [react()]
});
});
+1352 -520
View File
File diff suppressed because it is too large. Load diff
+3708
View File
File diff suppressed because it is too large. Load diff
+9
View File
@@ -0,0 +1,9 @@
[test]
# Set test timeout to 5 seconds (5000ms) to prevent hanging tests
timeout = 5000
# Preload the setup file
preload = ["./src/tests/setup.bun.ts"]
# Only run tests in src/ directory (excludes tests/e2e/ which are Playwright tests)
root = "./src/"
+236
View File
@@ -0,0 +1,236 @@
# CA Certificates Configuration
This document explains how to configure custom Certificate Authority (CA) certificates for Gitea Mirror when connecting to self-signed or privately signed Gitea instances.
## Overview
When your Gitea instance uses a self-signed certificate or a certificate signed by a private Certificate Authority (CA), you need to configure Gitea Mirror to trust these certificates.
## Common SSL/TLS Errors
If you encounter any of these errors, you need to configure CA certificates:
- `UNABLE_TO_VERIFY_LEAF_SIGNATURE`
- `SELF_SIGNED_CERT_IN_CHAIN`
- `UNABLE_TO_GET_ISSUER_CERT_LOCALLY`
- `CERT_UNTRUSTED`
- `unable to verify the first certificate`
## Configuration by Deployment Method
### Docker
#### Method 1: Volume Mount (Recommended)
1. Create a certificates directory:
```bash
mkdir -p ./certs
```
2. Copy your CA certificate(s):
```bash
cp /path/to/your-ca-cert.crt ./certs/
```
3. Update `docker-compose.yml`:
```yaml
version: '3.8'
services:
gitea-mirror:
image: raylabs/gitea-mirror:latest
volumes:
- ./data:/app/data
- ./certs:/usr/local/share/ca-certificates:ro
environment:
- NODE_EXTRA_CA_CERTS=/usr/local/share/ca-certificates/your-ca-cert.crt
```
4. Restart the container:
```bash
docker-compose down && docker-compose up -d
```
#### Method 2: Custom Docker Image
Create a `Dockerfile`:
```dockerfile
FROM raylabs/gitea-mirror:latest
# Copy CA certificates
COPY ./certs/*.crt /usr/local/share/ca-certificates/
# Update CA certificates
RUN update-ca-certificates
# Set environment variable
ENV NODE_EXTRA_CA_CERTS=/usr/local/share/ca-certificates/your-ca-cert.crt
```
Build and use:
```bash
docker build -t my-gitea-mirror .
```
### Native/Bun
#### Method 1: Environment Variable
```bash
export NODE_EXTRA_CA_CERTS=/path/to/your-ca-cert.crt
bun run start
```
#### Method 2: .env File
Add to your `.env` file:
```
NODE_EXTRA_CA_CERTS=/path/to/your-ca-cert.crt
```
#### Method 3: System CA Store
**Ubuntu/Debian:**
```bash
sudo cp your-ca-cert.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates
```
**RHEL/CentOS/Fedora:**
```bash
sudo cp your-ca-cert.crt /etc/pki/ca-trust/source/anchors/
sudo update-ca-trust
```
**macOS:**
```bash
sudo security add-trusted-cert -d -r trustRoot \
-k /Library/Keychains/System.keychain your-ca-cert.crt
```
### LXC Container (Proxmox VE)
1. Enter the container:
```bash
pct enter <container-id>
```
2. Create certificates directory:
```bash
mkdir -p /usr/local/share/ca-certificates
```
3. Copy your CA certificate:
```bash
cat > /usr/local/share/ca-certificates/your-ca.crt
```
(Paste certificate content and press Ctrl+D)
4. Update the systemd service:
```bash
cat >> /etc/systemd/system/gitea-mirror.service << EOF
Environment="NODE_EXTRA_CA_CERTS=/usr/local/share/ca-certificates/your-ca.crt"
EOF
```
5. Reload and restart:
```bash
systemctl daemon-reload
systemctl restart gitea-mirror
```
## Multiple CA Certificates
### Option 1: Bundle Certificates
```bash
cat ca-cert1.crt ca-cert2.crt ca-cert3.crt > ca-bundle.crt
export NODE_EXTRA_CA_CERTS=/path/to/ca-bundle.crt
```
### Option 2: System CA Store
```bash
# Copy all certificates
cp *.crt /usr/local/share/ca-certificates/
update-ca-certificates
```
## Verification
### 1. Test Gitea Connection
Use the "Test Connection" button in the Gitea configuration section.
### 2. Check Logs
**Docker:**
```bash
docker logs gitea-mirror
```
**Native:**
Check terminal output
**LXC:**
```bash
journalctl -u gitea-mirror -f
```
### 3. Manual Certificate Test
```bash
openssl s_client -connect your-gitea-domain.com:443 -CAfile /path/to/ca-cert.crt
```
## Best Practices
1. **Certificate Security**
- Keep CA certificates secure
- Use read-only mounts in Docker
- Limit certificate file permissions
- Regularly update certificates
2. **Certificate Management**
- Use descriptive certificate filenames
- Document certificate purposes
- Track certificate expiration dates
- Maintain certificate backups
3. **Production Deployment**
- Use proper SSL certificates when possible
- Consider Let's Encrypt for public instances
- Implement certificate rotation procedures
- Monitor certificate expiration
## Troubleshooting
### Certificate not being recognized
- Ensure the certificate is in PEM format
- Check that `NODE_EXTRA_CA_CERTS` points to the correct file
- Restart the application after adding certificates
### Still getting SSL errors
- Verify the complete certificate chain is included
- Check if intermediate certificates are needed
- Ensure the certificate matches the server hostname
### Certificate expired
- Check validity: `openssl x509 -in cert.crt -noout -dates`
- Update with new certificate from your CA
- Restart Gitea Mirror after updating
## Certificate Format
Certificates must be in PEM format. Example:
```
-----BEGIN CERTIFICATE-----
MIIDXTCCAkWgAwIBAgIJAKl8bUgMdErlMA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV
[... certificate content ...]
-----END CERTIFICATE-----
```
If your certificate is in DER format, convert it:
```bash
openssl x509 -inform der -in certificate.cer -out certificate.crt
```
File diff suppressed because it is too large. Load diff
+71
View File
@@ -0,0 +1,71 @@
# Minimal Gitea Mirror deployment
# Only includes what CANNOT be configured via the Web UI
# Everything else can be set up through the web interface after deployment
services:
gitea-mirror:
image: ghcr.io/raylabshq/gitea-mirror:latest
container_name: gitea-mirror
restart: unless-stopped
ports:
- "${PORT:-4321}:4321"
user: ${PUID:-1000}:${PGID:-1000}
volumes:
- ./data:/app/data
environment:
# === ABSOLUTELY REQUIRED ===
# This MUST be set and CANNOT be changed via UI
- BETTER_AUTH_SECRET=${BETTER_AUTH_SECRET} # Min 32 chars, required for sessions
- BETTER_AUTH_URL=${BETTER_AUTH_URL:-http://localhost:4321}
- BETTER_AUTH_TRUSTED_ORIGINS=${BETTER_AUTH_TRUSTED_ORIGINS:-http://localhost:4321}
# REVERSE PROXY: If accessing via a reverse proxy, set all three to your external URL:
# BETTER_AUTH_URL=https://gitea-mirror.example.com
# PUBLIC_BETTER_AUTH_URL=https://gitea-mirror.example.com
# BETTER_AUTH_TRUSTED_ORIGINS=https://gitea-mirror.example.com
# Path-prefix deployments (e.g. /mirror) are supported at runtime via BASE_URL.
# === CORE SETTINGS ===
# These are technically required but have working defaults
- NODE_ENV=production
- DATABASE_URL=file:data/gitea-mirror.db
- HOST=0.0.0.0
- PORT=4321
- BASE_URL=${BASE_URL:-/}
- PUBLIC_BETTER_AUTH_URL=${PUBLIC_BETTER_AUTH_URL:-http://localhost:4321}
# Optional concurrency controls (defaults match in-app defaults)
# If you want perfect ordering of issues and PRs, set these at 1
- MIRROR_ISSUE_CONCURRENCY=${MIRROR_ISSUE_CONCURRENCY:-3}
- MIRROR_PULL_REQUEST_CONCURRENCY=${MIRROR_PULL_REQUEST_CONCURRENCY:-5}
healthcheck:
test:
[
"CMD-SHELL",
"BASE=\"${BASE_URL:-/}\"; if [ \"$${BASE}\" = \"/\" ]; then BASE=\"\"; else BASE=\"$${BASE%/}\"; fi; wget --no-verbose --tries=3 --spider \"http://localhost:4321$${BASE}/api/health\"",
]
interval: 30s
timeout: 10s
retries: 5
start_period: 15s
# === QUICK START ===
#
# 1. Create a .env file with only ONE required variable:
# BETTER_AUTH_SECRET=your-32-character-minimum-secret-key-here
#
# 2. Run:
# docker-compose -f docker-compose.alt.yml up -d
#
# 3. Access at http://localhost:4321
#
# 4. Sign up for an account (first user becomes admin)
#
# 5. Configure everything else through the web UI:
# - GitHub credentials
# - Gitea credentials
# - Mirror settings
# - Scheduling options
# - Auto-import settings
# - Cleanup preferences
#
# That's it! Everything else can be configured via the web interface.
+26 -7
View File
@@ -7,11 +7,12 @@ services:
image: gitea/gitea:latest
container_name: gitea
restart: unless-stopped
entrypoint: ["/tmp/gitea-dev-init.sh"]
environment:
- USER_UID=1000
- USER_GID=1000
- GITEA__database__DB_TYPE=sqlite3
- GITEA__database__PATH=/data/gitea.db
- GITEA__database__PATH=/data/gitea/gitea.db
- GITEA__server__DOMAIN=localhost
- GITEA__server__ROOT_URL=http://localhost:3001/
- GITEA__server__SSH_DOMAIN=localhost
@@ -19,20 +20,24 @@ services:
- GITEA__server__START_SSH_SERVER=true
- GITEA__security__INSTALL_LOCK=true
- GITEA__service__DISABLE_REGISTRATION=false
- GITEA__log__MODE=console
- GITEA__log__LEVEL=Info
ports:
- "3001:3000"
- "2222:22"
volumes:
- gitea-data:/data
- gitea-config:/etc/gitea
- ./scripts/gitea-app.ini:/tmp/app.ini:ro
- ./scripts/gitea-dev-init.sh:/tmp/gitea-dev-init.sh:ro
networks:
- gitea-network
healthcheck:
test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:3000/api/healthz"]
test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:3000/"]
interval: 30s
timeout: 5s
retries: 3
start_period: 10s
timeout: 10s
retries: 5
start_period: 60s
# Development service connected to local Gitea
gitea-mirror-dev:
@@ -49,6 +54,11 @@ services:
- "4321:4321"
volumes:
- gitea-mirror-data:/app/data
# Mount custom CA certificates - choose one option:
# Option 1: Mount individual CA certificates from certs directory
# - ./certs:/app/certs:ro
# Option 2: Mount system CA bundle (if your CA is already in system store)
# - /etc/ssl/certs/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt:ro
depends_on:
- gitea
environment:
@@ -56,13 +66,16 @@ services:
- DATABASE_URL=file:data/gitea-mirror.db
- HOST=0.0.0.0
- PORT=4321
- JWT_SECRET=dev-secret-key
- BASE_URL=${BASE_URL:-/}
- BETTER_AUTH_SECRET=dev-secret-key
# GitHub/Gitea Mirror Config
- GITHUB_USERNAME=${GITHUB_USERNAME:-your-github-username}
- GITHUB_TOKEN=${GITHUB_TOKEN:-your-github-token}
- GITHUB_EXCLUDED_ORGS=${GITHUB_EXCLUDED_ORGS:-}
- SKIP_FORKS=${SKIP_FORKS:-false}
- PRIVATE_REPOSITORIES=${PRIVATE_REPOSITORIES:-false}
- MIRROR_ISSUES=${MIRROR_ISSUES:-false}
- MIRROR_WIKI=${MIRROR_WIKI:-false}
- MIRROR_STARRED=${MIRROR_STARRED:-false}
- MIRROR_ORGANIZATIONS=${MIRROR_ORGANIZATIONS:-false}
- PRESERVE_ORG_STRUCTURE=${PRESERVE_ORG_STRUCTURE:-false}
@@ -74,8 +87,14 @@ services:
- GITEA_ORGANIZATION=${GITEA_ORGANIZATION:-github-mirrors}
- GITEA_ORG_VISIBILITY=${GITEA_ORG_VISIBILITY:-public}
- DELAY=${DELAY:-3600}
# Optional: Skip TLS verification (insecure, use only for testing)
# - GITEA_SKIP_TLS_VERIFY=${GITEA_SKIP_TLS_VERIFY:-false}
healthcheck:
test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:4321/api/health"]
test:
[
"CMD-SHELL",
"BASE=\"${BASE_URL:-/}\"; if [ \"$${BASE}\" = \"/\" ]; then BASE=\"\"; else BASE=\"$${BASE%/}\"; fi; wget --no-verbose --tries=1 --spider \"http://localhost:4321$${BASE}/api/health\"",
]
interval: 30s
timeout: 5s
retries: 3
+53 -4
View File
@@ -3,7 +3,7 @@
services:
gitea-mirror:
image: ${DOCKER_REGISTRY:-ghcr.io}/${DOCKER_IMAGE:-arunavo4/gitea-mirror}:${DOCKER_TAG:-latest}
image: ${DOCKER_REGISTRY:-ghcr.io}/${DOCKER_IMAGE:-raylabshq/gitea-mirror}:${DOCKER_TAG:-latest}
build:
context: .
dockerfile: Dockerfile
@@ -11,38 +11,87 @@ services:
- linux/amd64
- linux/arm64
cache_from:
- ${DOCKER_REGISTRY:-ghcr.io}/${DOCKER_IMAGE:-arunavo4/gitea-mirror}:${DOCKER_TAG:-latest}
- ${DOCKER_REGISTRY:-ghcr.io}/${DOCKER_IMAGE:-raylabshq/gitea-mirror}:${DOCKER_TAG:-latest}
container_name: gitea-mirror
restart: unless-stopped
ports:
- "4321:4321"
volumes:
- gitea-mirror-data:/app/data
# Mount custom CA certificates - choose one option:
# Option 1: Mount individual CA certificates from certs directory
# - ./certs:/app/certs:ro
# Option 2: Mount system CA bundle (if your CA is already in system store)
# - /etc/ssl/certs/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt:ro
environment:
# For a complete list of all supported environment variables, see:
# docs/ENVIRONMENT_VARIABLES.md or .env.example
- NODE_ENV=production
- DATABASE_URL=file:data/gitea-mirror.db
- HOST=0.0.0.0
- PORT=4321
- JWT_SECRET=${JWT_SECRET:-your-secret-key-change-this-in-production}
- BASE_URL=${BASE_URL:-/}
- BETTER_AUTH_SECRET=${BETTER_AUTH_SECRET:-your-secret-key-change-this-in-production}
- BETTER_AUTH_URL=${BETTER_AUTH_URL:-http://localhost:4321}
# REVERSE PROXY: If you access Gitea Mirror through a reverse proxy (e.g. Nginx, Caddy, Traefik),
# you MUST set these three variables to your external URL. Example:
# BETTER_AUTH_URL=https://gitea-mirror.example.com
# PUBLIC_BETTER_AUTH_URL=https://gitea-mirror.example.com
# BETTER_AUTH_TRUSTED_ORIGINS=https://gitea-mirror.example.com
# If deployed under a path prefix (e.g. https://git.example.com/mirror), also set:
# BASE_URL=/mirror
# BETTER_AUTH_URL=https://git.example.com
# PUBLIC_BETTER_AUTH_URL=https://git.example.com
# BETTER_AUTH_TRUSTED_ORIGINS=https://git.example.com
- PUBLIC_BETTER_AUTH_URL=${PUBLIC_BETTER_AUTH_URL:-http://localhost:4321}
- BETTER_AUTH_TRUSTED_ORIGINS=${BETTER_AUTH_TRUSTED_ORIGINS:-}
# Optional: ENCRYPTION_SECRET will be auto-generated if not provided
# - ENCRYPTION_SECRET=${ENCRYPTION_SECRET:-}
# GitHub/Gitea Mirror Config
- GITHUB_USERNAME=${GITHUB_USERNAME:-}
- GITHUB_TOKEN=${GITHUB_TOKEN:-}
- GITHUB_EXCLUDED_ORGS=${GITHUB_EXCLUDED_ORGS:-}
- SKIP_FORKS=${SKIP_FORKS:-false}
- PRIVATE_REPOSITORIES=${PRIVATE_REPOSITORIES:-false}
- MIRROR_ISSUES=${MIRROR_ISSUES:-false}
- MIRROR_WIKI=${MIRROR_WIKI:-false}
- MIRROR_STARRED=${MIRROR_STARRED:-false}
- MIRROR_ORGANIZATIONS=${MIRROR_ORGANIZATIONS:-false}
- PRESERVE_ORG_STRUCTURE=${PRESERVE_ORG_STRUCTURE:-false}
- ONLY_MIRROR_ORGS=${ONLY_MIRROR_ORGS:-false}
- SKIP_STARRED_ISSUES=${SKIP_STARRED_ISSUES:-false}
- MIRROR_ISSUE_CONCURRENCY=${MIRROR_ISSUE_CONCURRENCY:-3}
- MIRROR_PULL_REQUEST_CONCURRENCY=${MIRROR_PULL_REQUEST_CONCURRENCY:-5}
- GITEA_URL=${GITEA_URL:-}
- GITEA_TOKEN=${GITEA_TOKEN:-}
- GITEA_USERNAME=${GITEA_USERNAME:-}
- GITEA_ORGANIZATION=${GITEA_ORGANIZATION:-github-mirrors}
- GITEA_ORG_VISIBILITY=${GITEA_ORG_VISIBILITY:-public}
- DELAY=${DELAY:-3600}
# Scheduling and Sync Configuration (Issue #72 fixes)
- SCHEDULE_ENABLED=${SCHEDULE_ENABLED:-false}
- GITEA_MIRROR_INTERVAL=${GITEA_MIRROR_INTERVAL:-8h}
- AUTO_IMPORT_REPOS=${AUTO_IMPORT_REPOS:-true}
- AUTO_MIRROR_REPOS=${AUTO_MIRROR_REPOS:-false}
# Repository Cleanup Configuration
- CLEANUP_DELETE_IF_NOT_IN_GITHUB=${CLEANUP_DELETE_IF_NOT_IN_GITHUB:-false}
- CLEANUP_ORPHANED_REPO_ACTION=${CLEANUP_ORPHANED_REPO_ACTION:-archive}
- CLEANUP_DRY_RUN=${CLEANUP_DRY_RUN:-true}
# Optional: Skip TLS verification (insecure, use only for testing)
# - GITEA_SKIP_TLS_VERIFY=${GITEA_SKIP_TLS_VERIFY:-false}
# Header Authentication (for Reverse Proxy SSO)
- HEADER_AUTH_ENABLED=${HEADER_AUTH_ENABLED:-false}
- HEADER_AUTH_USER_HEADER=${HEADER_AUTH_USER_HEADER:-X-Authentik-Username}
- HEADER_AUTH_EMAIL_HEADER=${HEADER_AUTH_EMAIL_HEADER:-X-Authentik-Email}
- HEADER_AUTH_NAME_HEADER=${HEADER_AUTH_NAME_HEADER:-X-Authentik-Name}
- HEADER_AUTH_AUTO_PROVISION=${HEADER_AUTH_AUTO_PROVISION:-false}
- HEADER_AUTH_ALLOWED_DOMAINS=${HEADER_AUTH_ALLOWED_DOMAINS:-}
healthcheck:
test: ["CMD", "wget", "--no-verbose", "--tries=3", "--spider", "http://localhost:4321/api/health"]
test:
[
"CMD-SHELL",
"BASE=\"${BASE_URL:-/}\"; if [ \"$${BASE}\" = \"/\" ]; then BASE=\"\"; else BASE=\"$${BASE%/}\"; fi; wget --no-verbose --tries=3 --spider \"http://localhost:4321$${BASE}/api/health\"",
]
interval: 30s
timeout: 10s
retries: 5
+148 -182
View File
@@ -5,15 +5,73 @@ set -e
# Ensure data directory exists
mkdir -p /app/data
# Generate a secure JWT secret if one isn't provided or is using the default value
JWT_SECRET_FILE="/app/data/.jwt_secret"
if [ "$JWT_SECRET" = "your-secret-key-change-this-in-production" ] || [ -z "$JWT_SECRET" ]; then
# Handle custom CA certificates
if [ -d "/app/certs" ] && [ "$(ls -A /app/certs/*.crt 2>/dev/null)" ]; then
echo "Custom CA certificates found, configuring Node.js to use them..."
# Combine all CA certificates into a bundle for Node.js
CA_BUNDLE="/app/certs/ca-bundle.crt"
> "$CA_BUNDLE"
for cert in /app/certs/*.crt; do
if [ -f "$cert" ]; then
echo "Adding certificate: $(basename "$cert")"
cat "$cert" >> "$CA_BUNDLE"
echo "" >> "$CA_BUNDLE" # Add newline between certificates
fi
done
# Set Node.js to use the custom CA bundle
export NODE_EXTRA_CA_CERTS="$CA_BUNDLE"
echo "NODE_EXTRA_CA_CERTS set to: $NODE_EXTRA_CA_CERTS"
# For Bun compatibility, also set the CA bundle in system location if writable
if [ -f "/etc/ssl/certs/ca-certificates.crt" ] && [ -w "/etc/ssl/certs/" ]; then
echo "Appending custom certificates to system CA bundle..."
cat "$CA_BUNDLE" >> /etc/ssl/certs/ca-certificates.crt
fi
else
echo "No custom CA certificates found in /app/certs"
fi
# Check if system CA bundle is mounted and use it (only if not already set)
if [ -z "$NODE_EXTRA_CA_CERTS" ] && [ -f "/etc/ssl/certs/ca-certificates.crt" ] && [ ! -L "/etc/ssl/certs/ca-certificates.crt" ]; then
# Check if it's a mounted file (not the default symlink)
if [ "$(stat -c '%d' /etc/ssl/certs/ca-certificates.crt 2>/dev/null)" != "$(stat -c '%d' / 2>/dev/null)" ] || \
[ "$(stat -f '%d' /etc/ssl/certs/ca-certificates.crt 2>/dev/null)" != "$(stat -f '%d' / 2>/dev/null)" ]; then
echo "System CA bundle mounted, configuring Node.js to use it..."
export NODE_EXTRA_CA_CERTS="/etc/ssl/certs/ca-certificates.crt"
echo "NODE_EXTRA_CA_CERTS set to: $NODE_EXTRA_CA_CERTS"
fi
fi
# Optional: If GITEA_SKIP_TLS_VERIFY is set, configure accordingly
if [ "$GITEA_SKIP_TLS_VERIFY" = "true" ]; then
echo "Warning: GITEA_SKIP_TLS_VERIFY is set to true. This is insecure!"
export NODE_TLS_REJECT_UNAUTHORIZED=0
fi
# Generate a secure BETTER_AUTH_SECRET if one isn't provided or is using the default value
BETTER_AUTH_SECRET_FILE="/app/data/.better_auth_secret"
JWT_SECRET_FILE="/app/data/.jwt_secret" # Old file for backward compatibility
if [ "$BETTER_AUTH_SECRET" = "your-secret-key-change-this-in-production" ] || [ -z "$BETTER_AUTH_SECRET" ]; then
# Check if we have a previously generated secret
if [ -f "$JWT_SECRET_FILE" ]; then
echo "Using previously generated JWT secret"
export JWT_SECRET=$(cat "$JWT_SECRET_FILE")
if [ -f "$BETTER_AUTH_SECRET_FILE" ]; then
echo "Using previously generated BETTER_AUTH_SECRET"
export BETTER_AUTH_SECRET=$(cat "$BETTER_AUTH_SECRET_FILE")
# Check for old JWT_SECRET file for backward compatibility
elif [ -f "$JWT_SECRET_FILE" ]; then
echo "Migrating from old JWT_SECRET to BETTER_AUTH_SECRET"
export BETTER_AUTH_SECRET=$(cat "$JWT_SECRET_FILE")
# Save to new file
echo "$BETTER_AUTH_SECRET" > "$BETTER_AUTH_SECRET_FILE"
chmod 600 "$BETTER_AUTH_SECRET_FILE"
# Optionally remove old file after successful migration
rm -f "$JWT_SECRET_FILE"
else
echo "Generating a secure random JWT secret"
echo "Generating a secure random BETTER_AUTH_SECRET"
# Try to generate a secure random string using OpenSSL
if command -v openssl >/dev/null 2>&1; then
GENERATED_SECRET=$(openssl rand -hex 32)
@@ -22,12 +80,38 @@ if [ "$JWT_SECRET" = "your-secret-key-change-this-in-production" ] || [ -z "$JWT
echo "OpenSSL not found, using fallback method for random generation"
GENERATED_SECRET=$(head -c 32 /dev/urandom | sha256sum | cut -d' ' -f1)
fi
export JWT_SECRET="$GENERATED_SECRET"
export BETTER_AUTH_SECRET="$GENERATED_SECRET"
# Save the secret to a file for persistence across container restarts
echo "$GENERATED_SECRET" > "$JWT_SECRET_FILE"
chmod 600 "$JWT_SECRET_FILE"
echo "$GENERATED_SECRET" > "$BETTER_AUTH_SECRET_FILE"
chmod 600 "$BETTER_AUTH_SECRET_FILE"
fi
echo "JWT_SECRET has been set to a secure random value"
echo "BETTER_AUTH_SECRET has been set to a secure random value"
fi
# Generate a secure ENCRYPTION_SECRET if one isn't provided
ENCRYPTION_SECRET_FILE="/app/data/.encryption_secret"
if [ -z "$ENCRYPTION_SECRET" ]; then
# Check if we have a previously generated secret
if [ -f "$ENCRYPTION_SECRET_FILE" ]; then
echo "Using previously generated ENCRYPTION_SECRET"
export ENCRYPTION_SECRET=$(cat "$ENCRYPTION_SECRET_FILE")
else
echo "Generating a secure random ENCRYPTION_SECRET"
# Generate a 48-character secret for encryption
if command -v openssl >/dev/null 2>&1; then
GENERATED_ENCRYPTION_SECRET=$(openssl rand -base64 36)
else
# Fallback to using /dev/urandom if openssl is not available
echo "OpenSSL not found, using fallback method for encryption secret generation"
GENERATED_ENCRYPTION_SECRET=$(head -c 36 /dev/urandom | base64 | tr -d '\n' | head -c 48)
fi
export ENCRYPTION_SECRET="$GENERATED_ENCRYPTION_SECRET"
# Save the secret to a file for persistence across container restarts
echo "$GENERATED_ENCRYPTION_SECRET" > "$ENCRYPTION_SECRET_FILE"
chmod 600 "$ENCRYPTION_SECRET_FILE"
fi
echo "ENCRYPTION_SECRET has been set to a secure random value"
fi
@@ -36,168 +120,13 @@ fi
# Dependencies are already installed during the Docker build process
# Initialize the database if it doesn't exist
# Note: Drizzle migrations will be run automatically when the app starts (see src/lib/db/index.ts)
if [ ! -f "/app/data/gitea-mirror.db" ]; then
echo "Initializing database..."
if [ -f "dist/scripts/init-db.js" ]; then
bun dist/scripts/init-db.js
elif [ -f "dist/scripts/manage-db.js" ]; then
bun dist/scripts/manage-db.js init
elif [ -f "scripts/manage-db.ts" ]; then
bun scripts/manage-db.ts init
else
echo "Warning: Could not find database initialization scripts in dist/scripts."
echo "Creating and initializing database manually..."
# Create the database file
touch /app/data/gitea-mirror.db
# Initialize the database with required tables
sqlite3 /app/data/gitea-mirror.db <<EOF
CREATE TABLE IF NOT EXISTS users (
id TEXT PRIMARY KEY,
username TEXT NOT NULL,
password TEXT NOT NULL,
email TEXT NOT NULL,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS configs (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
name TEXT NOT NULL,
is_active INTEGER NOT NULL DEFAULT 1,
github_config TEXT NOT NULL,
gitea_config TEXT NOT NULL,
include TEXT NOT NULL DEFAULT '["*"]',
exclude TEXT NOT NULL DEFAULT '[]',
schedule_config TEXT NOT NULL,
created_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
updated_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
FOREIGN KEY (user_id) REFERENCES users(id)
);
CREATE TABLE IF NOT EXISTS repositories (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
config_id TEXT NOT NULL,
name TEXT NOT NULL,
full_name TEXT NOT NULL,
url TEXT NOT NULL,
clone_url TEXT NOT NULL,
owner TEXT NOT NULL,
organization TEXT,
mirrored_location TEXT DEFAULT '',
is_private INTEGER NOT NULL DEFAULT 0,
is_fork INTEGER NOT NULL DEFAULT 0,
forked_from TEXT,
has_issues INTEGER NOT NULL DEFAULT 0,
is_starred INTEGER NOT NULL DEFAULT 0,
is_archived INTEGER NOT NULL DEFAULT 0,
size INTEGER NOT NULL DEFAULT 0,
has_lfs INTEGER NOT NULL DEFAULT 0,
has_submodules INTEGER NOT NULL DEFAULT 0,
default_branch TEXT NOT NULL,
visibility TEXT NOT NULL DEFAULT 'public',
status TEXT NOT NULL DEFAULT 'imported',
last_mirrored INTEGER,
error_message TEXT,
created_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
updated_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
FOREIGN KEY (user_id) REFERENCES users(id),
FOREIGN KEY (config_id) REFERENCES configs(id)
);
CREATE TABLE IF NOT EXISTS organizations (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
config_id TEXT NOT NULL,
name TEXT NOT NULL,
avatar_url TEXT NOT NULL,
membership_role TEXT NOT NULL DEFAULT 'member',
is_included INTEGER NOT NULL DEFAULT 1,
status TEXT NOT NULL DEFAULT 'imported',
last_mirrored INTEGER,
error_message TEXT,
repository_count INTEGER NOT NULL DEFAULT 0,
created_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
updated_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
FOREIGN KEY (user_id) REFERENCES users(id),
FOREIGN KEY (config_id) REFERENCES configs(id)
);
CREATE TABLE IF NOT EXISTS mirror_jobs (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
repository_id TEXT,
repository_name TEXT,
organization_id TEXT,
organization_name TEXT,
details TEXT,
status TEXT NOT NULL DEFAULT 'imported',
message TEXT NOT NULL,
timestamp TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
-- New fields for job resilience
job_type TEXT NOT NULL DEFAULT 'mirror',
batch_id TEXT,
total_items INTEGER,
completed_items INTEGER DEFAULT 0,
item_ids TEXT, -- JSON array as text
completed_item_ids TEXT DEFAULT '[]', -- JSON array as text
in_progress INTEGER NOT NULL DEFAULT 0, -- Boolean as integer
started_at TIMESTAMP,
completed_at TIMESTAMP,
last_checkpoint TIMESTAMP,
FOREIGN KEY (user_id) REFERENCES users(id)
);
CREATE INDEX IF NOT EXISTS idx_mirror_jobs_user_id ON mirror_jobs(user_id);
CREATE INDEX IF NOT EXISTS idx_mirror_jobs_batch_id ON mirror_jobs(batch_id);
CREATE INDEX IF NOT EXISTS idx_mirror_jobs_in_progress ON mirror_jobs(in_progress);
CREATE INDEX IF NOT EXISTS idx_mirror_jobs_job_type ON mirror_jobs(job_type);
CREATE INDEX IF NOT EXISTS idx_mirror_jobs_timestamp ON mirror_jobs(timestamp);
CREATE TABLE IF NOT EXISTS events (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
channel TEXT NOT NULL,
payload TEXT NOT NULL,
read INTEGER NOT NULL DEFAULT 0,
created_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
FOREIGN KEY (user_id) REFERENCES users(id)
);
CREATE INDEX IF NOT EXISTS idx_events_user_channel ON events(user_id, channel);
CREATE INDEX IF NOT EXISTS idx_events_created_at ON events(created_at);
CREATE INDEX IF NOT EXISTS idx_events_read ON events(read);
EOF
echo "Database initialized with required tables."
fi
echo "Database not found. It will be created and initialized via Drizzle migrations on first app startup..."
# Create empty database file so migrations can run
touch /app/data/gitea-mirror.db
else
echo "Database already exists, checking for issues..."
if [ -f "dist/scripts/fix-db-issues.js" ]; then
bun dist/scripts/fix-db-issues.js
elif [ -f "dist/scripts/manage-db.js" ]; then
bun dist/scripts/manage-db.js fix
elif [ -f "scripts/manage-db.ts" ]; then
bun scripts/manage-db.ts fix
fi
# Run database migrations
echo "Running database migrations..."
# Update mirror_jobs table with new columns for resilience
if [ -f "dist/scripts/update-mirror-jobs-table.js" ]; then
echo "Updating mirror_jobs table..."
bun dist/scripts/update-mirror-jobs-table.js
elif [ -f "scripts/update-mirror-jobs-table.ts" ]; then
echo "Updating mirror_jobs table using TypeScript script..."
bun scripts/update-mirror-jobs-table.ts
else
echo "Warning: Could not find mirror_jobs table update script."
fi
echo "Database already exists, Drizzle will check for pending migrations on startup..."
fi
# Extract version from package.json and set as environment variable
@@ -208,19 +137,52 @@ fi
# Initialize configuration from environment variables if provided
echo "Checking for environment configuration..."
# Only run the env config script if relevant env vars are set
# This avoids spawning a heavy Bun process on memory-constrained systems
HAS_ENV_CONFIG=false
if [ -n "$GITHUB_USERNAME" ] || [ -n "$GITHUB_TOKEN" ] || [ -n "$GITEA_URL" ] || [ -n "$GITEA_USERNAME" ] || [ -n "$GITEA_TOKEN" ]; then
HAS_ENV_CONFIG=true
fi
if [ "$HAS_ENV_CONFIG" = "true" ]; then
if [ -f "dist/scripts/startup-env-config.js" ]; then
echo "Loading configuration from environment variables..."
bun dist/scripts/startup-env-config.js || ENV_CONFIG_EXIT_CODE=$?
ENV_CONFIG_EXIT_CODE=${ENV_CONFIG_EXIT_CODE:-0}
elif [ -f "scripts/startup-env-config.ts" ]; then
echo "Loading configuration from environment variables..."
bun scripts/startup-env-config.ts || ENV_CONFIG_EXIT_CODE=$?
ENV_CONFIG_EXIT_CODE=${ENV_CONFIG_EXIT_CODE:-0}
else
echo "Environment configuration script not found. Skipping."
ENV_CONFIG_EXIT_CODE=0
fi
else
echo "No GitHub/Gitea environment variables found, skipping env config initialization."
ENV_CONFIG_EXIT_CODE=0
fi
# Log environment config result
if [ $ENV_CONFIG_EXIT_CODE -eq 0 ]; then
echo "✅ Environment configuration loaded successfully"
else
echo "⚠️ Environment configuration loading completed with warnings"
fi
# Run startup recovery to handle any interrupted jobs
echo "Running startup recovery..."
RECOVERY_EXIT_CODE=0
if [ -f "dist/scripts/startup-recovery.js" ]; then
echo "Running startup recovery using compiled script..."
bun dist/scripts/startup-recovery.js --timeout=30000
RECOVERY_EXIT_CODE=$?
bun dist/scripts/startup-recovery.js --timeout=30000 || RECOVERY_EXIT_CODE=$?
elif [ -f "scripts/startup-recovery.ts" ]; then
echo "Running startup recovery using TypeScript script..."
bun scripts/startup-recovery.ts --timeout=30000
RECOVERY_EXIT_CODE=$?
bun scripts/startup-recovery.ts --timeout=30000 || RECOVERY_EXIT_CODE=$?
else
echo "Warning: Startup recovery script not found. Skipping recovery."
RECOVERY_EXIT_CODE=0
fi
# Log recovery result
@@ -234,17 +196,15 @@ fi
# Run repository status repair to fix any inconsistent mirroring states
echo "Running repository status repair..."
REPAIR_EXIT_CODE=0
if [ -f "dist/scripts/repair-mirrored-repos.js" ]; then
echo "Running repository repair using compiled script..."
bun dist/scripts/repair-mirrored-repos.js --startup
REPAIR_EXIT_CODE=$?
bun dist/scripts/repair-mirrored-repos.js --startup || REPAIR_EXIT_CODE=$?
elif [ -f "scripts/repair-mirrored-repos.ts" ]; then
echo "Running repository repair using TypeScript script..."
bun scripts/repair-mirrored-repos.ts --startup
REPAIR_EXIT_CODE=$?
bun scripts/repair-mirrored-repos.ts --startup || REPAIR_EXIT_CODE=$?
else
echo "Warning: Repository repair script not found. Skipping repair."
REPAIR_EXIT_CODE=0
fi
# Log repair result
@@ -269,7 +229,13 @@ trap 'shutdown_handler' TERM INT HUP
# Start the application
echo "Starting Gitea Mirror..."
bun ./dist/server/entry.mjs &
if [ -f "./scripts/runtime-server.ts" ]; then
bun ./scripts/runtime-server.ts &
elif [ -f "./dist/scripts/runtime-server.js" ]; then
bun ./dist/scripts/runtime-server.js &
else
bun ./dist/server/entry.mjs &
fi
APP_PID=$!
# Wait for the application to finish
+262
View File
@@ -0,0 +1,262 @@
# API access with API keys
Gitea Mirror's web UI talks to a set of JSON endpoints under `/api`. With an API key you can call the same endpoints from scripts, CI pipelines and workflow tools such as n8n, without a browser session.
## Create a key
1. Sign in and open **Settings → Authentication**.
2. In the **API Keys** section click **Create key**, give it a name and pick an expiry (never, 30 days, 90 days or a year).
3. Copy the key. It is shown once; afterwards only its first characters are visible in the list.
Keys start with `gm_`, belong to the user who created them and carry the same rights as that user's session. Revoke a key from the same list at any time. Keys are stored hashed. They are not rate limited, so a runaway script is stopped by revoking its key.
## Send the key
Put the key in the `x-api-key` header. Every endpoint that accepts a session cookie accepts the header instead.
```bash
curl -sS "https://mirror.example.com/api/github/repositories" \
-H "x-api-key: gm_..."
```
A missing, malformed, expired or revoked key gets `401 Unauthorized`:
```json
{ "success": false, "error": "Unauthorized" }
```
If the app is served under a path prefix (`BASE_URL=/mirror`), prefix every URL with it: `/mirror/api/...`.
## Endpoints
The calls below cover the automation cases from issue #314. Response bodies show the fields you are most likely to use; the app returns more.
### List tracked repositories
`GET /api/github/repositories`
Despite the name it lists every repository the account tracks, whatever the configured source. Requires a saved configuration, otherwise `404`.
```json
{
"success": true,
"message": "Repositories fetched successfully",
"repositories": [
{
"id": "9b2f...",
"name": "hello-world",
"fullName": "octocat/hello-world",
"owner": "octocat",
"organization": null,
"status": "mirrored",
"mirroredLocation": "github-mirrors/hello-world",
"lastMirrored": "2026-09-02T08:00:00.000Z",
"errorMessage": null,
"destinationOrg": null,
"sourceProvider": "github",
"isPrivate": false
}
]
}
```
`status` is one of `imported`, `mirroring`, `mirrored`, `failed`, `syncing`, `synced`, `skipped`, `deleted`, `archived`.
### Add a repository
`POST /api/sync/repository`
```json
{ "owner": "octocat", "repo": "hello-world" }
```
Optional fields: `destinationOrg` to override the Gitea organization for this repository, `force: true` to refresh the metadata of a repository that is already tracked.
The repository is looked up on the configured source and added with status `imported`. It is not mirrored until you start a mirror (next call) or the scheduler picks it up.
| Status | Meaning |
| --- | --- |
| `200` | Added. `repository.id` is what the mirror call needs. |
| `400` | `owner` or `repo` missing. |
| `404` | No configuration for this account, or the repository does not exist on the source. |
| `409` | Already tracked. Send `force: true` to refresh it instead. |
```json
{
"success": true,
"message": "Repository added successfully",
"repository": { "id": "9b2f...", "fullName": "octocat/hello-world", "status": "imported" }
}
```
### Start a mirror
`POST /api/job/mirror-repo`
```json
{ "repositoryIds": ["9b2f..."] }
```
Starts mirroring in the background and returns straight away. Poll the list endpoint to watch `status` move from `mirroring` to `mirrored` or `failed`.
| Status | Meaning |
| --- | --- |
| `200` | Job started. `repositories` lists what was queued. |
| `400` | `repositoryIds` missing or empty, or no source token saved. |
| `404` | None of the ids belong to this account. |
`POST /api/job/sync-repo` takes the same body and re-syncs repositories that are already mirrored.
### Organizations
`POST /api/sync/organization` adds an organization the same way:
```json
{ "org": "my-org", "role": "member" }
```
`role` is the account's role in that organization (`member`, `admin`, `owner` or `billing_manager`). `409` means it is already tracked; `force: true` refreshes it. Then `POST /api/job/mirror-org` with `{ "organizationIds": ["..."] }` mirrors its repositories.
When several sources are connected, an optional `sourceId` picks which one the organization imports and mirrors from (the first connected source is the default):
```json
{ "org": "my-org", "role": "member", "sourceId": "…" }
```
`PATCH /api/organizations/:id` with `{ "sourceId": "…" }` re-pins an existing organization to another source, and `{ "sourceId": null }` clears the pin so the organization follows every source's repositories of the same name again. A pin whose source was removed falls back the same way.
### Reconcile the destination
`POST /api/cleanup/reconcile`
Compares what the destination (Gitea or Forgejo) holds with the repositories this account tracks, and reports four groups: mirrors on the destination that the database does not know about, rows whose mirror was moved to another owner, rows marked mirrored whose repository is gone, and a healthy count. Only mirrors whose source URL points at the configured source count as this app's. Native repositories, mirrors of other hosts and second copies of a tracked mirror are listed under `notManaged` and never touched. Useful after a lost or restored database (issue #284) or after transferring mirrors between owners in Gitea (issue #400).
```json
{ "dryRun": true, "adoptUntracked": false, "relocateMoved": false, "resetMissing": false }
```
The body is optional and every field defaults to the value above, so an empty `POST` is a dry run. With `dryRun: false`:
- `adoptUntracked: true` creates a row for each untracked mirror from its source URL, so scheduled sync and the cleanup service include it from then on. The row keeps the mirror where it is, even when the strategy would put it elsewhere.
- `relocateMoved: true` points each moved row at the owner that holds its mirror now, so sync follows the transfer instead of failing. A row that failed because its mirror was not at the recorded place goes back to `mirrored`. Nothing changes on the destination.
- `resetMissing: true` sets each missing row back to `imported` so the next mirror run recreates the mirror. Rows are never deleted.
Nothing is deleted or archived on either side by this call; the cleanup service keeps its own rules.
```json
{
"success": true,
"dryRun": false,
"report": {
"untracked": [{ "location": "github-mirrors/hello-world", "originalUrl": "https://github.com/octocat/hello-world.git", "sourcePath": "octocat/hello-world", "isPrivate": false }],
"missing": [{ "id": "9b2f...", "fullName": "octocat/gone", "location": "github-mirrors/gone" }],
"moved": [{ "id": "c41a...", "fullName": "octocat/tools", "from": "github-mirrors/tools", "to": "archive/tools" }],
"notManaged": [{ "location": "me/notes", "reason": "not a mirror" }],
"unverified": [],
"healthyCount": 40,
"elsewhereCount": 0,
"scannedOwners": ["e2e_admin", "github-mirrors", "archive"],
"skippedOwners": ["starred"],
"totalOnDestination": 43
},
"applied": { "adopted": 1, "reset": 1, "relocated": 1, "skipped": 0 }
}
```
`applied` is `null` on a dry run. `scannedOwners` are the destination users and organizations that were listed; `skippedOwners` are ones the configuration or the database point at but the destination does not have. A row lands in `unverified` when the presence check itself failed, and is left alone.
| Status | Meaning |
| --- | --- |
| `200` | Report computed, and applied when asked. |
| `400` | A field has the wrong type, or the destination is not configured yet. |
| `404` | No configuration for this account. |
### Move a mirror to another owner
`POST /api/repositories/{id}/move-mirror`
```json
{ "destinationOrg": "archive" }
```
Changes the repository's destination and transfers its existing mirror there on the destination (Gitea or Forgejo), for issue #400. `PATCH /api/repositories/{id}` with `destinationOrg` only changes the label: the mirror stays where it is and the new destination applies if the mirror is ever created again. Send `null` to remove the override and move the mirror back to where the strategy puts it.
The mirror is looked for at its recorded location first, then anywhere on the destination in case someone transferred it by hand. A target organization that does not exist is created, the way a mirror run creates one. When a mirror of the same source already sits under the target, the row is pointed at it and the old copy is left alone. The row is only written once the destination has accepted the transfer, so a refused move changes nothing.
```json
{
"success": true,
"destinationOrg": "archive",
"mirroredLocation": "archive/hello-world",
"transfer": {
"outcome": "moved",
"from": "github-mirrors/hello-world",
"to": "archive/hello-world",
"message": "Moved octocat/hello-world from github-mirrors/hello-world to archive/hello-world."
}
}
```
| `transfer.outcome` | Meaning |
| --- | --- |
| `moved` | The destination moved the repository. `mirroredLocation` is the new place. |
| `pending` | The token cannot create repositories under the new owner, so the destination waits for one of its owners to accept. The mirror keeps syncing where it is, and sync follows it once accepted. |
| `recorded` | A mirror of the same source already sat under the target; the row now points at it. |
| `not-mirrored` | The row has never been mirrored, so only the destination changed. |
| Status | Meaning |
| --- | --- |
| `200` | Done, see `transfer`. |
| `400` | Bad owner name, destination not configured, or a GitHub or GitLab destination (`destination-unsupported`). |
| `403` | The destination token may not transfer the repository (`forbidden`). |
| `404` | The repository or the configuration does not belong to this account. |
| `409` | Nothing changed. `code` says why: `not-on-destination`, `name-taken`, `transfer-pending`, or `destination-mismatch` for a row mirrored to another host. |
| `502` | The destination refused for another reason (`destination-error`). |
`POST /api/organizations/{id}/move-mirrors` does the same for every mirrored repository of an organization:
```json
{ "destinationOrg": "archive", "dryRun": true }
```
`dryRun` defaults to `true` and answers with `plan.moves` (`id`, `fullName`, `from`, `to`) and `plan.skipped` (`fullName`, `reason`) without touching anything. Repositories with their own destination, starred repositories, rows that are not mirrored yet and rows in a running mirror or sync are skipped. With `dryRun: false` each planned move runs, the organization's `destinationOrg` is written, and the answer lists `moved`, `pending`, `recorded` and `failed` (`fullName`, `error`). A failed repository keeps syncing where it is.
## A complete example
Add a repository and mirror it in one go:
```bash
#!/usr/bin/env bash
set -euo pipefail
BASE="https://mirror.example.com"
KEY="gm_..."
added=$(curl -sS -X POST "$BASE/api/sync/repository" \
-H "x-api-key: $KEY" \
-H "Content-Type: application/json" \
-d '{"owner":"octocat","repo":"hello-world"}')
id=$(echo "$added" | jq -r '.repository.id')
curl -sS -X POST "$BASE/api/job/mirror-repo" \
-H "x-api-key: $KEY" \
-H "Content-Type: application/json" \
-d "{\"repositoryIds\":[\"$id\"]}"
```
## Managing keys from the API
Manage keys through the settings UI or with a browser session. The key endpoints need a session cookie, not a key, so a leaked key cannot mint more keys, and like every cookie-authenticated `POST` to `/api/auth/*` they also need an `Origin` header (browsers send it on their own; a script must add `Origin: https://mirror.example.com`). The key itself is only for the app routes above. The endpoints live under `/api/auth/api-key/`:
| Call | Body | Result |
| --- | --- | --- |
| `POST /api/auth/api-key/create` | `{ "name": "ci", "expiresIn": 2592000 }` (`expiresIn` in seconds, omit for no expiry) | The new key, including `key`, shown only here. |
| `GET /api/auth/api-key/list` | | `{ "apiKeys": [...], "total": n }`. Each entry has `id`, `name`, `start`, `createdAt`, `lastRequest` and `expiresAt`, never the secret. |
| `POST /api/auth/api-key/delete` | `{ "keyId": "..." }` | Revokes the key. |
## Notes
- A key can only act on the account that created it. Each account keeps its own source, destination and repositories.
- Keys are not rate limited by the app. Source hosts still apply their own limits, and the app tracks GitHub's.
- Deleting a user deletes their keys.
+354
View File
@@ -0,0 +1,354 @@
# Development Workflow
This guide covers the development workflow for the open-source Gitea Mirror.
## Getting Started
### Prerequisites
- Bun >= 1.2.9
- Node.js >= 20
- Git
- GitHub account (for API access)
- Gitea instance (for testing)
### Initial Setup
1. **Clone the repository**:
```bash
git clone https://github.com/RayLabsHQ/gitea-mirror.git
cd gitea-mirror
```
2. **Install dependencies and seed the SQLite database**:
```bash
bun run setup
```
3. **Configure environment (optional)**:
```bash
cp .env.example .env
# Edit .env with your settings
```
4. **Start the development server**:
```bash
bun run dev
```
## Development Commands
| Command | Description |
|---------|-------------|
| `bun run dev` | Start the Bun + Astro dev server with hot reload |
| `bun run build` | Build the production bundle |
| `bun run preview` | Preview the production build locally |
| `bun test` | Run the Bun test suite |
| `bun test:watch` | Run tests in watch mode |
| `bun run db:studio` | Launch Drizzle Kit Studio |
## Project Structure
```
gitea-mirror/
├── src/ # Application UI, API routes, and services
│ ├── components/ # React components rendered inside Astro pages
│ ├── pages/ # Astro pages and API routes (e.g., /api/*)
│ ├── lib/ # Core logic: GitHub/Gitea clients, scheduler, recovery, db helpers
│ │ ├── db/ # Drizzle adapter + schema
│ │ ├── modules/ # Module wiring (jobs, integrations)
│ │ └── utils/ # Shared utilities
│ ├── hooks/ # React hooks
│ ├── content/ # In-app documentation and templated content
│ ├── layouts/ # Shared layout components
│ ├── styles/ # Tailwind CSS entrypoints
│ └── types/ # TypeScript types
├── scripts/ # Bun scripts for DB management and maintenance
├── www/ # Marketing site (Astro + MDX use cases)
├── public/ # Static assets served by Vite/Astro
└── tests/ # Dedicated integration/unit test helpers
```
## Feature Development
### Adding a New Feature
1. **Create feature branch**:
```bash
git checkout -b feature/my-feature
```
2. **Plan your changes**:
- UI components live in `src/components/`
- API endpoints live in `src/pages/api/`
- Database logic is under `src/lib/db/` (schema + adapter)
- Shared types are in `src/types/`
3. **Implement the feature**:
**Example: Adding a new API endpoint**
```typescript
// src/pages/api/my-endpoint.ts
import type { APIRoute } from 'astro';
import { getUserFromCookie } from '@/lib/auth-utils';
export const GET: APIRoute = async ({ request }) => {
const user = await getUserFromCookie(request);
if (!user) {
return new Response('Unauthorized', { status: 401 });
}
// Your logic here
return new Response(JSON.stringify({ data: 'success' }), {
headers: { 'Content-Type': 'application/json' }
});
};
```
4. **Write tests**:
```typescript
// src/lib/my-feature.test.ts
import { describe, it, expect } from 'bun:test';
describe('My Feature', () => {
it('should work correctly', () => {
expect(myFunction()).toBe('expected');
});
});
```
5. **Update documentation**:
- Add JSDoc comments
- Update README/docs if needed
- Document API changes
## Database Development
### Schema Changes
1. **Modify schema**:
```typescript
// src/lib/db/schema.ts
export const myTable = sqliteTable('my_table', {
id: text('id').primaryKey(),
name: text('name').notNull(),
createdAt: integer('created_at').notNull(),
});
```
2. **Generate migration**:
```bash
bun run db:generate
```
3. **Apply migration**:
```bash
bun run db:migrate
```
### Writing Queries
```typescript
// src/lib/db/queries/my-queries.ts
import { db } from '../index';
import { myTable } from '../schema';
export async function getMyData(userId: string) {
return db.select()
.from(myTable)
.where(eq(myTable.userId, userId));
}
```
## Testing
### Unit Tests
```bash
# Run all tests
bun test
# Run specific test file
bun test auth
# Watch mode
bun test:watch
# Coverage
bun test:coverage
```
### Manual Testing Checklist
- [ ] Feature works as expected
- [ ] No console errors
- [ ] Responsive on mobile
- [ ] Handles errors gracefully
- [ ] Loading states work
- [ ] Form validation works
- [ ] API returns correct status codes
## Debugging
### VSCode Configuration
Create `.vscode/launch.json`:
```json
{
"version": "0.2.0",
"configurations": [
{
"type": "bun",
"request": "launch",
"name": "Debug Bun",
"program": "${workspaceFolder}/src/index.ts",
"cwd": "${workspaceFolder}",
"env": {
"NODE_ENV": "development"
}
}
]
}
```
### Debug Logging
```typescript
// Development only logging
if (import.meta.env.DEV) {
console.log('[Debug]', data);
}
```
## Code Style
### TypeScript
- Use strict mode
- Define interfaces for all data structures
- Avoid `any` type
- Use proper error handling
### React Components
- Use functional components
- Implement proper loading states
- Handle errors with error boundaries
- Use TypeScript for props
### API Routes
- Always validate input
- Return proper status codes
- Use consistent error format
- Document with JSDoc
## Git Workflow
### Commit Messages
Follow conventional commits:
```
feat: add repository filtering
fix: resolve sync timeout issue
docs: update API documentation
style: format code with prettier
refactor: simplify auth logic
test: add user creation tests
chore: update dependencies
```
### Pull Request Process
1. Create feature branch
2. Make changes
3. Write/update tests
4. Update documentation
5. Create PR with description
6. Address review feedback
7. Squash and merge
## Performance
### Development Tips
- Use React DevTools
- Monitor bundle size
- Profile database queries
- Check memory usage
### Optimization
- Lazy load components
- Optimize images
- Use database indexes
- Cache API responses
## Common Issues
### Port Already in Use
```bash
# Use different port
PORT=3001 bun run dev
```
### Database Locked
```bash
# Reset database
bun run cleanup-db
bun run init-db
```
### Type Errors
```bash
# Check types
bunx tsc --noEmit
```
## Release Process
1. **Choose release version** (`X.Y.Z`) and update `CHANGELOG.md`
2. **Build and test**:
```bash
bun run build
bun test
```
3. **Create release tag** (semver format required):
```bash
git tag vX.Y.Z
git push origin vX.Y.Z
```
4. **Create GitHub release**
5. **CI version sync and chart publish (automatic)**:
- On `v*` tags, release CI updates `package.json` version in the build context from the tag (`vX.Y.Z` -> `X.Y.Z`), so Docker release images always report the correct app version.
- After the release build succeeds, CI packages the Helm chart with the same version as chart version and `appVersion` and pushes it to `oci://ghcr.io/raylabshq/charts/gitea-mirror`. The chart's default image tag is `v<appVersion>`, so a default install runs that release.
- CI then commits the same version to `package.json` and `helm/gitea-mirror/Chart.yaml` on `main` automatically.
## Contributing
1. Fork the repository
2. Create your feature branch
3. Commit your changes
4. Push to your fork
5. Create a Pull Request
## Resources
- [Astro Documentation](https://docs.astro.build)
- [Bun Documentation](https://bun.sh/docs)
- [Drizzle ORM](https://orm.drizzle.team)
- [React Documentation](https://react.dev)
- [TypeScript Handbook](https://www.typescriptlang.org/docs/)
## Getting Help
- Check existing [issues](https://github.com/RayLabsHQ/gitea-mirror/issues)
- Join [discussions](https://github.com/RayLabsHQ/gitea-mirror/discussions)
- Review project docs in [docs/README.md](./README.md)
+514
View File
@@ -0,0 +1,514 @@
# Environment Variables Documentation
This document provides a comprehensive list of all environment variables supported by Gitea Mirror. These can be used to configure the application via Docker or other deployment methods.
## Environment Variables and UI Interaction
When environment variables are set:
1. They are loaded on application startup
2. Values are stored in the database on first load
3. The UI will display these values and they can be modified
4. UI changes are saved to the database and persist
5. Environment variables provide initial defaults but don't override UI changes
**Note**: Some critical settings like `GITEA_LFS`, `MIRROR_RELEASES`, and `MIRROR_METADATA` will be visible and configurable in the UI even when set via environment variables.
## Table of Contents
- [Core Configuration](#core-configuration)
- [HTTPS / TLS](#https--tls)
- [GitHub Configuration](#github-configuration)
- [Gitea Configuration](#gitea-configuration)
- [Mirror Options](#mirror-options)
- [Automation Configuration](#automation-configuration)
- [Database Cleanup Configuration](#database-cleanup-configuration)
- [Authentication Configuration](#authentication-configuration)
- [Docker Configuration](#docker-configuration)
## Core Configuration
Essential application settings required for running Gitea Mirror.
| Variable | Description | Default | Required |
|----------|-------------|---------|----------|
| `NODE_ENV` | Application environment | `production` | No |
| `HOST` | Server host binding | `0.0.0.0` | No |
| `PORT` | Server port | `4321` | No |
| `BASE_URL` | Application base path. Use `/` for root deployments, or a prefix such as `/mirror` when serving behind a reverse-proxy path prefix. | `/` | No |
| `DATABASE_URL` | Database connection URL | `sqlite://data/gitea-mirror.db` | No |
| `BETTER_AUTH_SECRET` | Secret key for session signing (generate with: `openssl rand -base64 32`) | - | Yes |
| `BETTER_AUTH_URL` | Authentication origin (scheme + host only, e.g. `https://git.example.com`). Do **not** include a path — any path is automatically stripped, and `BASE_URL` is applied separately. | `http://localhost:4321` | No |
| `PUBLIC_BETTER_AUTH_URL` | Client-side auth origin for multi-origin access (same rule: origin only, no path). Set this to your primary domain when you need to access the app from different origins (e.g., both IP and domain). The client will use this URL for all auth requests instead of the current browser origin. | - | No |
| `BETTER_AUTH_TRUSTED_ORIGINS` | Trusted origins for authentication requests. Comma-separated list of URLs. Use this to specify additional access URLs (e.g., local IP + domain: `http://10.10.20.45:4321,https://gitea-mirror.mydomain.tld`), SSO providers, reverse proxies, etc. | - | No |
| `BETTER_AUTH_LOG_LEVEL` | Better Auth logger verbosity. Set to `debug` to surface the full SSO/OIDC sign-in and callback trace when troubleshooting authentication. Accepted values: `debug`, `info`, `warn`, `error`. (Better Auth does **not** use the `DEBUG` env var.) | `warn` | No |
| `ENCRYPTION_SECRET` | Optional encryption key for tokens (generate with: `openssl rand -base64 48`) | - | No |
## HTTPS / TLS
Gitea Mirror can terminate TLS directly via the underlying `@astrojs/node` adapter — useful when you don't want a separate reverse proxy. When both variables below are set, the server starts as a real HTTPS listener instead of HTTP.
| Variable | Description | Default | Required |
|----------|-------------|---------|----------|
| `SERVER_CERT_PATH` | Absolute path to the TLS certificate (PEM). Set together with `SERVER_KEY_PATH` to enable HTTPS. | - | No |
| `SERVER_KEY_PATH` | Absolute path to the TLS private key (PEM). Set together with `SERVER_CERT_PATH` to enable HTTPS. | - | No |
**Example (systemd or `.env`):**
```bash
SERVER_CERT_PATH=/etc/ssl/gitea-mirror/cert.pem
SERVER_KEY_PATH=/etc/ssl/gitea-mirror/key.pem
PORT=443
BETTER_AUTH_URL=https://mirror.example.com
BETTER_AUTH_TRUSTED_ORIGINS=https://mirror.example.com
```
Notes:
- The process must have read access to both files. When binding to `PORT=443`, grant the binary the `CAP_NET_BIND_SERVICE` capability (or run as a user allowed to bind privileged ports) rather than running as root.
- If you already terminate TLS at a reverse proxy (nginx, Traefik, Caddy), leave these unset and let the proxy handle certificates.
- Works in Docker too — mount your certs and set both paths to locations inside the container.
## GitHub Configuration
Settings for connecting to the source host. GitHub is the default and, together with a Gitea destination, the supported path; every other source or destination is beta. GitLab and Gitea/Forgejo (including Codeberg) are selected with `SOURCE_PROVIDER`; the username and token variables below then hold the account and token for that host. See [SOURCE_PROVIDERS.md](SOURCE_PROVIDERS.md) for what each source supports.
### Basic Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `SOURCE_PROVIDER` | Which host repositories are pulled from. `gitea` also covers Forgejo and Codeberg. Anything other than `github` is beta. | `github` | `github`, `gitlab`, `gitea` |
| `SOURCE_URL` | Base URL of the GitLab or Gitea/Forgejo instance. Ignored for GitHub, which uses `GH_API_URL`. | `https://gitlab.com` for GitLab, `https://codeberg.org` for Gitea | e.g. `https://gitlab.example.com` |
| `GITHUB_USERNAME` | Your username on the source host | - | - |
| `GITHUB_TOKEN` | Personal access token for the source host. GitHub needs the `repo` and `admin:org` scopes, GitLab `read_api` and `read_repository`, Gitea/Forgejo `read:repository`, `read:user` and `read:organization`. | - | - |
| `GITHUB_TYPE` | GitHub account type | `personal` | `personal`, `organization` |
| `GH_API_URL` | GitHub API base URL. Override this to point at GitHub Enterprise Server or Enterprise Cloud with data residency. | `https://api.github.com` | e.g. `https://ghe.example.com/api/v3`, `https://api.TENANT.ghe.com` |
### GitLab and Gitea/Forgejo sources
```bash
# Mirror from a self hosted GitLab
SOURCE_PROVIDER=gitlab
SOURCE_URL=https://gitlab.example.com
GITHUB_USERNAME=my-gitlab-user
GITHUB_TOKEN=glpat-...
# Mirror from Codeberg (Forgejo)
SOURCE_PROVIDER=gitea
GITHUB_USERNAME=my-codeberg-user
GITHUB_TOKEN=...
```
Code, tags, wiki and LFS are mirrored from every source. Issues, pull requests, releases, labels, milestones and star lists need a GitHub source and are skipped for the others.
Once repositories have been imported the source is locked, and once anything has been mirrored the Gitea server URL is locked. A `SOURCE_PROVIDER`, `SOURCE_URL` or `GITEA_URL` value that disagrees with a locked host is ignored on boot with a warning; change the host on the Configuration page, where the change asks for confirmation.
### GitHub Enterprise (GHES / GHEC with data residency)
Set `GH_API_URL` to point Octokit at a non-`github.com` API endpoint:
```bash
# GitHub Enterprise Server (self-hosted)
GH_API_URL=https://ghe.example.com/api/v3
# GitHub Enterprise Cloud with data residency
GH_API_URL=https://api.TENANT.ghe.com
```
Standard GitHub Enterprise Cloud on `github.com` works with the default — no override needed. Use a personal access token issued by the target Enterprise instance for `GITHUB_TOKEN`.
### Repository Selection
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `PRIVATE_REPOSITORIES` | Include private repositories | `false` | `true`, `false` |
| `PUBLIC_REPOSITORIES` | Include public repositories | `true` | `true`, `false` |
| `INCLUDE_ARCHIVED` | Include archived repositories | `false` | `true`, `false` |
| `INCLUDE_COLLABORATOR_REPOS` | Include repositories where you are a collaborator (not just owned). Set to `false` to limit imports to repos you own. | `true` | `true`, `false` |
| `SKIP_FORKS` | Skip forked repositories | `false` | `true`, `false` |
| `MIRROR_STARRED` | Mirror starred repositories | `false` | `true`, `false` |
| `MIRROR_STARRED_LISTS` | Optional comma-separated GitHub Star List names to mirror (only used when `MIRROR_STARRED=true`) | - | Comma-separated list names (empty = all starred repos) |
| `STARRED_REPOS_ORG` | Organization name for starred repos | `starred` | Any string |
| `STARRED_REPOS_MODE` | How starred repos are mirrored | `dedicated-org` | `dedicated-org`, `preserve-owner` |
| `STARRED_DUPLICATE_STRATEGY` | Name collision strategy when two starred repos share a name from different owners (`suffix` = `repo-owner`, `prefix` = `owner-repo`) | `suffix` | `suffix`, `prefix`, `owner-org` |
### Organization Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `MIRROR_ORGANIZATIONS` | Mirror organization repositories | `false` | `true`, `false` |
| `PRESERVE_ORG_STRUCTURE` | Preserve GitHub organization structure in Gitea | `false` | `true`, `false` |
| `ONLY_MIRROR_ORGS` | Only mirror organization repos (skip personal); sets `skipPersonalRepos: true` in GitHub config | `false` | `true`, `false` |
| `INCLUDE_ORGANIZATIONS` | Opt-in allowlist: only mirror repos from these organizations (empty = all orgs you belong to). Sets `includeOrganizations` in GitHub config | - | Comma-separated org names |
| `MIRROR_STRATEGY` | Repository organization strategy | `preserve` | `preserve`, `single-org`, `flat-user`, `mixed` |
### Advanced Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `SKIP_STARRED_ISSUES` | Enable lightweight mode for starred repos (skip issues) | `false` | `true`, `false` |
| `AUTO_MIRROR_STARRED` | Automatically mirror starred repos during scheduled syncs and "Mirror All". When `false`, starred repos are imported for browsing but must be mirrored individually. | `false` | `true`, `false` |
## Gitea Configuration
Settings for the destination. Gitea and Forgejo are pull mirrors; GitHub and GitLab are push targets served by the push engine (see [PUSH_TARGETS.md](PUSH_TARGETS.md)). The `GITEA_*` names apply to every destination kind.
### Connection Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `DESTINATION_PROVIDER` | Which host mirrors are created on. Gitea and Forgejo share one API (labels and hints differ). GitHub and GitLab receive `git push` of branches and tags from this app. Anything other than `gitea` is beta. | `gitea` | `gitea`, `forgejo`, `github`, `gitlab` |
| `GITEA_URL` | Destination instance URL. Optional for GitHub and GitLab, which default to github.com and gitlab.com; set it for GitHub Enterprise Server or a self hosted GitLab. | - | Valid URL |
| `GITEA_EXTERNAL_URL` | Optional external/browser URL used for dashboard links. API and mirroring still use `GITEA_URL`. | - | Valid URL |
| `GITEA_TOKEN` | Gitea access token | - | - |
| `GITEA_USERNAME` | Gitea username | - | - |
| `GITEA_ORGANIZATION` | Default organization for single-org strategy | `github-mirrors` | Any string |
### Push Targets (GitHub and GitLab destinations)
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `MIRROR_CLONE_DIR` | Where the push engine keeps its bare clones, one per repository. | `<data dir>/mirrors` | Absolute or relative path |
| `PUSH_CONCURRENCY` | How many git fetch or push operations run at once across all repositories. | `2` | 1 to 32 |
### Repository Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `GITEA_ORG_VISIBILITY` | Default organization visibility | `public` | `public`, `private`, `limited`, `default` |
| `GITEA_MIRROR_INTERVAL` | Mirror sync interval - **automatically enables scheduled mirroring when set** | `8h` | Duration string (e.g., `30m`, `1h`, `8h`, `24h`, `1d`) or seconds |
| `GITEA_LFS` | Enable LFS support (requires LFS on Gitea server) - Shows in UI | `false` | `true`, `false` |
| `GITEA_CREATE_ORG` | Auto-create organizations | `true` | `true`, `false` |
| `GITEA_PRESERVE_VISIBILITY` | Preserve GitHub repo visibility in Gitea | `false` | `true`, `false` |
### Template Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `GITEA_TEMPLATE_OWNER` | Template repository owner | - | Any string |
| `GITEA_TEMPLATE_REPO` | Template repository name | - | Any string |
### Topic Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `GITEA_ADD_TOPICS` | Add topics to repositories | `true` | `true`, `false` |
| `GITEA_TOPIC_PREFIX` | Prefix for repository topics | - | Any string |
### Fork Handling
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `GITEA_FORK_STRATEGY` | How to handle forked repositories | `reference` | `skip`, `reference`, `full-copy` |
### Additional Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `GITEA_SKIP_TLS_VERIFY` | Skip TLS certificate verification (WARNING: insecure) | `false` | `true`, `false` |
## Mirror Options
Control what content gets mirrored from GitHub to Gitea.
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `MIRROR_RELEASES` | Mirror GitHub releases | `false` | `true`, `false` |
| `RELEASE_LIMIT` | Maximum number of releases to mirror per repository | `10` | Number (1-100) |
| `RELEASE_ASSET_LIMIT` | Upload release assets only for the newest N mirrored releases. Older releases still get their notes and tag. Unset means assets for every mirrored release, `0` means release notes only. Lowering it never deletes assets already uploaded | unset (all) | Number (0 or more) |
| `MIRROR_WIKI` | Mirror wiki content | `false` | `true`, `false` |
| `MIRROR_METADATA` | Master toggle for metadata mirroring | `false` | `true`, `false` |
| `MIRROR_ISSUES` | Mirror issues (requires MIRROR_METADATA=true) | `false` | `true`, `false` |
| `MIRROR_PULL_REQUESTS` | Mirror pull requests (requires MIRROR_METADATA=true) | `false` | `true`, `false` |
| `MIRROR_LABELS` | Mirror labels (requires MIRROR_METADATA=true) | `false` | `true`, `false` |
| `MIRROR_MILESTONES` | Mirror milestones (requires MIRROR_METADATA=true) | `false` | `true`, `false` |
| `MIRROR_ISSUE_CONCURRENCY` | Number of issues processed in parallel. Set above `1` to speed up mirroring at the risk of out-of-order creation. | `3` | Integer ≥ 1 |
| `MIRROR_PULL_REQUEST_CONCURRENCY` | Number of pull requests processed in parallel. Values above `1` may cause ordering differences. | `5` | Integer ≥ 1 |
> **Ordering vs Throughput:** Metadata now mirrors sequentially by default to preserve chronology. Increase the concurrency variables only if you can tolerate minor out-of-order entries.
## Automation Configuration
Configure automatic scheduled mirroring.
### Basic Schedule Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `SCHEDULE_ENABLED` | Enable automatic mirroring. **When set to `true`, automatically imports and mirrors all repositories on startup** (v3.5.3+) | `false` | `true`, `false` |
| `SCHEDULE_INTERVAL` | Interval in seconds or cron expression. **Supports cron syntax for scheduled runs** (e.g., `"0 2 * * *"` for 2 AM daily) | `3600` | Number (seconds) or cron string |
| `DELAY` | Legacy: same as SCHEDULE_INTERVAL | `3600` | Number (seconds) |
> **🚀 Auto-Start Feature (v3.5.3+)**
> Setting either `SCHEDULE_ENABLED=true` or `GITEA_MIRROR_INTERVAL` triggers auto-start functionality where the service will:
> 1. **Import** all GitHub repositories on startup
> 2. **Mirror** them to Gitea immediately
> 3. **Continue syncing** at the configured interval
> 4. **Auto-discover** new repositories
> 5. **Clean up** deleted repositories (if configured)
>
> This eliminates the need for manual button clicks - perfect for Docker/Kubernetes deployments!
> **⏰ Scheduling with Cron Expressions**
> Use cron expressions in `SCHEDULE_INTERVAL` to run at specific times:
> - `"0 2 * * *"` - Daily at 2 AM
> - `"0 */6 * * *"` - Every 6 hours
> - `"0 0 * * 0"` - Weekly on Sunday at midnight
> - `"0 3 * * 1-5"` - Weekdays at 3 AM (Monday-Friday)
>
> This is useful for optimizing bandwidth usage during low-activity periods.
### Execution Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `SCHEDULE_CONCURRENT` | Allow concurrent mirror operations | `false` | `true`, `false` |
| `SCHEDULE_BATCH_SIZE` | Number of repos to process in parallel | `10` | Number |
| `SCHEDULE_PAUSE_BETWEEN_BATCHES` | Pause between batches (milliseconds) | `5000` | Number |
### Retry Configuration
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `SCHEDULE_RETRY_ATTEMPTS` | Number of retry attempts | `3` | Number |
| `SCHEDULE_RETRY_DELAY` | Delay between retries (milliseconds) | `60000` | Number |
| `SCHEDULE_TIMEOUT` | Max time for a mirror operation (milliseconds) | `3600000` | Number |
| `SCHEDULE_AUTO_RETRY` | Automatically retry failed operations | `true` | `true`, `false` |
### Update Detection
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `AUTO_IMPORT_REPOS` | Automatically discover and import new GitHub repositories during scheduled syncs | `true` | `true`, `false` |
| `AUTO_MIRROR_REPOS` | Automatically mirror newly imported repositories during scheduled syncs (no manual “Mirror All” required) | `false` | `true`, `false` |
| `SCHEDULE_ONLY_MIRROR_UPDATED` | Only mirror repos with updates | `false` | `true`, `false` |
| `SCHEDULE_UPDATE_INTERVAL` | Check for updates interval (milliseconds) | `86400000` | Number |
| `SCHEDULE_SKIP_RECENTLY_MIRRORED` | Skip recently mirrored repos | `true` | `true`, `false` |
| `SCHEDULE_RECENT_THRESHOLD` | Skip if mirrored within this time (milliseconds) | `3600000` | Number |
### Maintenance & Notifications
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `SCHEDULE_CLEANUP_BEFORE_MIRROR` | Run cleanup before mirroring | `false` | `true`, `false` |
| `SCHEDULE_NOTIFY_ON_FAILURE` | Send notifications on failure | `true` | `true`, `false` |
| `SCHEDULE_NOTIFY_ON_SUCCESS` | Send notifications on success | `false` | `true`, `false` |
| `SCHEDULE_LOG_LEVEL` | Logging level | `info` | `error`, `warn`, `info`, `debug` |
| `SCHEDULE_TIMEZONE` | Timezone for scheduling | `UTC` | Valid timezone string |
## Database Cleanup Configuration
Configure automatic cleanup of old events and data.
### Basic Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `CLEANUP_ENABLED` | Enable automatic cleanup | `false` | `true`, `false` |
| `CLEANUP_RETENTION_DAYS` | Days to keep events | `7` | Number |
### Repository Cleanup
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `CLEANUP_DELETE_FROM_GITEA` | Apply the orphaned-repo action on the Gitea side too. When `false` (default), cleanup only updates gitea-mirror's own database (orphans are marked archived or removed from the repo list) and the Gitea/Forgejo copies are left untouched | `false` | `true`, `false` |
| `CLEANUP_DELETE_IF_NOT_IN_GITHUB` | Delete repos not found in GitHub (automatically enables cleanup) | `true` | `true`, `false` |
| `CLEANUP_ORPHANED_REPO_ACTION` | Action for orphaned repositories. **Note**: `archive` is recommended to preserve backups | `archive` | `skip`, `archive`, `delete` |
| `CLEANUP_DRY_RUN` | Test mode without actual deletion | `false` | `true`, `false` |
| `CLEANUP_PROTECTED_REPOS` | Comma-separated list of protected repository names | - | Comma-separated strings |
**🛡️ Safety Features (Backup Protection)**:
- **GitHub Failures Don't Delete Backups**: Cleanup is automatically skipped if GitHub API returns errors (404, 403, connection issues)
- **Archive Never Deletes**: The `archive` action ALWAYS preserves repository data, it never deletes
- **Graceful Degradation**: If marking as archived fails, the repository remains fully accessible in Gitea
- **The Purpose of Backups**: Your mirrors are preserved even when GitHub sources disappear - that's the whole point!
**Archive Behavior (Aligned with Gitea API)**:
- **Regular repositories**: Uses Gitea's native archive feature (PATCH `/repos/{owner}/{repo}` with `archived: true`)
- Makes repository read-only while preserving all data
- **Mirror repositories**: Uses rename strategy (Gitea API returns 422 for archiving mirrors)
- Renamed with `archived-` prefix for clear identification
- Description updated with preservation notice and timestamp
- Mirror interval set to 8760h (1 year) to minimize sync attempts
- Repository remains fully accessible and cloneable
- **Manual Sync Option**: Archived mirrors are still available on the Repositories page with automatic syncs disabled—use the `Manual Sync` action to refresh them on demand.
### Execution Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `CLEANUP_BATCH_SIZE` | Number of items to process per batch | `10` | Number |
| `CLEANUP_PAUSE_BETWEEN_DELETES` | Pause between deletions (milliseconds) | `2000` | Number |
## Authentication Configuration
Configure authentication methods and SSO.
### Header Authentication (Reverse Proxy SSO)
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `HEADER_AUTH_ENABLED` | Enable header-based authentication | `false` | `true`, `false` |
| `HEADER_AUTH_USER_HEADER` | Header containing username | `X-Authentik-Username` | Header name |
| `HEADER_AUTH_EMAIL_HEADER` | Header containing email | `X-Authentik-Email` | Header name |
| `HEADER_AUTH_NAME_HEADER` | Header containing display name | `X-Authentik-Name` | Header name |
| `HEADER_AUTH_AUTO_PROVISION` | Auto-create users from headers | `false` | `true`, `false` |
| `HEADER_AUTH_ALLOWED_DOMAINS` | Comma-separated list of allowed email domains | - | Comma-separated domains |
## Docker Configuration
Settings specific to Docker deployments.
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `DOCKER_REGISTRY` | Docker registry URL | `ghcr.io` | Registry URL |
| `DOCKER_IMAGE` | Docker image name | `raylabshq/gitea-mirror:` | Image name |
| `DOCKER_TAG` | Docker image tag | `latest` | Tag name |
## Example Docker Compose Configuration
Here's an example of how to use these environment variables in a `docker-compose.yml` file:
```yaml
version: '3.8'
services:
gitea-mirror:
image: ghcr.io/raylabshq/gitea-mirror:latest
container_name: gitea-mirror
environment:
# Core Configuration
- NODE_ENV=production
- BASE_URL=/
- DATABASE_URL=file:data/gitea-mirror.db
- BETTER_AUTH_SECRET=your-secure-secret-here
# Primary access URL:
- BETTER_AUTH_URL=https://gitea-mirror.mydomain.tld
# Additional access URLs (local network + SSO providers):
# - BETTER_AUTH_TRUSTED_ORIGINS=http://10.10.20.45:4321,http://192.168.1.100:4321,https://auth.provider.com
# GitHub Configuration
- GITHUB_USERNAME=your-username
- GITHUB_TOKEN=ghp_your_token_here
- PRIVATE_REPOSITORIES=true
- MIRROR_STARRED=true
- SKIP_FORKS=false
# Gitea Configuration
- GITEA_URL=http://gitea:3000
- GITEA_USERNAME=admin
- GITEA_TOKEN=your-gitea-token
- GITEA_ORGANIZATION=github-mirrors
- GITEA_ORG_VISIBILITY=public
# Mirror Options
- MIRROR_RELEASES=true
- MIRROR_WIKI=true
- MIRROR_METADATA=true
- MIRROR_ISSUES=true
- MIRROR_PULL_REQUESTS=true
# Automation
- SCHEDULE_ENABLED=true
- SCHEDULE_INTERVAL=3600
# Cleanup
- CLEANUP_ENABLED=true
- CLEANUP_RETENTION_DAYS=30
volumes:
- ./data:/app/data
ports:
- "4321:4321"
```
## Authentication URL Configuration
### Multiple Access URLs
To allow access to Gitea Mirror through multiple URLs (e.g., local IP and public domain), you need to configure both server and client settings:
**Example Configuration:**
```bash
# Primary URL (required) - where the auth server is hosted
BETTER_AUTH_URL=https://gitea-mirror.mydomain.tld
# Client-side URL (optional) - tells the browser where to send auth requests
# Set this to your primary domain when accessing from different origins
PUBLIC_BETTER_AUTH_URL=https://gitea-mirror.mydomain.tld
# Additional trusted origins (optional) - origins allowed to make auth requests
BETTER_AUTH_TRUSTED_ORIGINS=http://10.10.20.45:4321,http://192.168.1.100:4321
```
This setup allows you to:
- Access via local network IP: `http://10.10.20.45:4321`
- Access via public domain: `https://gitea-mirror.mydomain.tld`
- Auth requests from the IP will be sent to the domain (via `PUBLIC_BETTER_AUTH_URL`)
- Each origin requires separate login due to browser cookie isolation
**Important:** When accessing from different origins (IP vs domain), you'll need to log in separately on each origin as cookies cannot be shared across different origins for security reasons.
### Path Prefix Deployments
If you serve Gitea Mirror under a subpath such as `https://git.example.com/mirror`, set:
```bash
# BASE_URL handles the path prefix — auth URLs stay as origin only
BASE_URL=/mirror
BETTER_AUTH_URL=https://git.example.com
PUBLIC_BETTER_AUTH_URL=https://git.example.com
BETTER_AUTH_TRUSTED_ORIGINS=https://git.example.com
# → Auth endpoints resolve to: https://git.example.com/mirror/api/auth/*
```
Notes:
- `BETTER_AUTH_URL` and `PUBLIC_BETTER_AUTH_URL` must be **origin only** (scheme + host). Do not include the base path — it is applied automatically from `BASE_URL`. Any path accidentally included is stripped.
- `BETTER_AUTH_TRUSTED_ORIGINS` must also contain origins only (no path).
- `BASE_URL` is applied at runtime, so prebuilt images can be reused with different path prefixes.
### Trusted Origins
The `BETTER_AUTH_TRUSTED_ORIGINS` variable serves multiple purposes:
1. **SSO/OIDC Providers**: When using external authentication providers (Google, Authentik, Okta)
2. **Reverse Proxies**: When running behind nginx, Traefik, or other proxies
3. **Cross-Origin Requests**: When the frontend and backend are on different domains
4. **Development**: When testing from different URLs
**Example Scenarios:**
```bash
# For Authentik SSO integration
BETTER_AUTH_TRUSTED_ORIGINS=https://authentik.company.com,https://auth.company.com
# For reverse proxy setup
BETTER_AUTH_TRUSTED_ORIGINS=https://proxy.internal,https://public.domain.com
# For development with multiple environments
BETTER_AUTH_TRUSTED_ORIGINS=http://localhost:3000,http://192.168.1.100:3000
```
**Important Notes:**
- All URLs from `BETTER_AUTH_URL` are automatically trusted
- URLs must be complete with protocol (http/https)
- Multiple origins are separated by commas
- No trailing slashes needed
## Notes
1. **First Run**: Environment variables are loaded when the container starts. The configuration is applied after the first user account is created.
2. **UI Priority**: Manual changes made through the web UI will be preserved. Environment variables only set values for empty fields.
3. **Token Security**: All tokens are encrypted before being stored in the database.
4. **Auto-Enabling Features**: Certain environment variables automatically enable features when set:
- `GITEA_MIRROR_INTERVAL` - Automatically enables scheduled mirroring
- `CLEANUP_DELETE_IF_NOT_IN_GITHUB=true` - Automatically enables repository cleanup
- `SCHEDULE_INTERVAL` or `DELAY` - Automatically enables the scheduler
5. **Backward Compatibility**: The `DELAY` variable is maintained for backward compatibility but `SCHEDULE_INTERVAL` is preferred.
6. **Required Scopes**: The GitHub token requires the following scopes:
- `repo` (full control of private repositories)
- `admin:org` (read organization data)
- Additional scopes may be required for specific features
For more examples and detailed configuration, see the `.env.example` file in the repository.
+183
View File
@@ -0,0 +1,183 @@
# Force-Push Protection
This document describes the smart force-push protection system introduced in gitea-mirror v3.11.0+.
## The Problem
GitHub repositories can be force-pushed at any time — rewriting history, deleting branches, or replacing commits entirely. When gitea-mirror syncs a force-pushed repo, the old history in Gitea is silently overwritten. Files, commits, and branches disappear with no way to recover them.
The original workaround (`backupBeforeSync: true`) created a full git bundle backup before **every** sync. This doesn't scale — a user with 100+ GiB of mirrors would need up to 2 TB of backup storage with default retention settings, even though force-pushes are rare.
## Solution: Smart Detection
Instead of backing up everything every time, the system detects force-pushes **before** they happen and only acts when needed.
### How Detection Works
Before each sync, the app compares branch SHAs between Gitea (the mirror) and GitHub (the source):
1. **Fetch branches from both sides** — lightweight API calls to get branch names and their latest commit SHAs
2. **Compare each branch**:
- SHAs match → nothing changed, no action needed
- SHAs differ → check if the change is a normal push or a force-push
3. **Ancestry check** — for branches with different SHAs, call GitHub's compare API to determine if the new SHA is a descendant of the old one:
- **Fast-forward** (new SHA descends from old) → normal push, safe to sync
- **Diverged** (histories split) → force-push detected
- **404** (old SHA doesn't exist on GitHub anymore) → history was rewritten, force-push detected
- **Branch deleted on GitHub** → flagged as destructive change
### What Happens on Detection
Depends on the configured strategy (see below):
- **Backup strategies** (`always`, `on-force-push`): create a git bundle snapshot, then sync
- **Block strategy** (`block-on-force-push`): halt the sync, mark the repo as `pending-approval`, wait for user action
### Fail-Open Design
If detection itself fails (GitHub rate limits, network errors, API outages), sync proceeds normally. Detection never blocks a sync due to its own failure. Individual branch check failures are skipped — one flaky branch doesn't affect the others.
## Backup Strategies
Configure via **Configuration → Connections → Destructive Update Protection**.
| Strategy | What It Does | Storage Cost | Best For |
|---|---|---|---|
| **Disabled** | No detection, no backups | Zero | Repos you don't care about losing |
| **Always Backup** | Snapshot before every sync (original behavior) | High | Small mirror sets, maximum safety |
| **Smart** (default) | Detect force-pushes, backup only when found | Near-zero normally | Most users — efficient protection |
| **Block & Approve** | Detect force-pushes, block sync until approved | Zero | Critical repos needing manual review |
### Strategy Details
#### Disabled
Syncs proceed without any detection or backup. If a force-push happens on GitHub, the mirror silently overwrites.
#### Always Backup
Creates a git bundle snapshot before every sync regardless of whether a force-push occurred. This is the legacy behavior (equivalent to the old `backupBeforeSync: true`). Safe but expensive for large mirror sets.
#### Smart (`on-force-push`) — Recommended
Runs the force-push detection before each sync. On normal days (no force-pushes), syncs proceed without any backup overhead. When a force-push is detected, a snapshot is created before the sync runs.
This gives you protection when it matters with near-zero cost when it doesn't.
#### Block & Approve (`block-on-force-push`)
Runs detection and, when a force-push is found, **blocks the sync entirely**. The repository is marked as `pending-approval` and excluded from future scheduled syncs until you take action:
- **Approve**: creates a backup first, then syncs (safe)
- **Dismiss**: clears the flag and resumes normal syncing (no backup)
Use this for repos where you want manual control over destructive changes.
## Additional Settings
These appear when any non-disabled strategy is selected:
### Snapshot Retention Count
How many backup snapshots to keep per repository. Oldest snapshots are deleted when this limit is exceeded. Default: **5**.
### Snapshot Retention Days
Maximum age (in days) for backup snapshots. Bundles older than this are deleted during retention enforcement, though at least one bundle is always kept. Set to `0` to disable time-based retention. Default: **30**.
### Snapshot Directory
Where git bundle backups are stored. Default: **`data/repo-backups`**. Bundles are organized as `<directory>/<owner>/<repo>/<timestamp>.bundle`.
### Block Sync on Snapshot Failure
Available for **Always Backup** and **Smart** strategies. When enabled, if the snapshot creation fails (disk full, permissions error, etc.), the sync is also blocked. When disabled, sync continues even if the snapshot couldn't be created.
Recommended: **enabled** if you rely on backups for recovery.
## Backward Compatibility
The old `backupBeforeSync` boolean is still recognized:
| Old Setting | New Equivalent |
|---|---|
| `backupBeforeSync: true` | `backupStrategy: "on-force-push"` |
| `backupBeforeSync: false` | `backupStrategy: "disabled"` |
| Neither set | `backupStrategy: "on-force-push"` (new default) |
Existing configurations are automatically mapped. The old field is deprecated but will continue to work.
## Environment Variables
No new environment variables are required. The backup strategy is configured through the web UI and stored in the database alongside other config.
## API
### Approve/Dismiss Blocked Repos
When using the `block-on-force-push` strategy, repos that are blocked can be managed via the API:
```bash
# Approve sync (creates backup first, then syncs)
curl -X POST http://localhost:4321/api/job/approve-sync \
-H "Content-Type: application/json" \
-H "Cookie: <session>" \
-d '{"repositoryIds": ["<id>"], "action": "approve"}'
# Dismiss (clear the block, resume normal syncing)
curl -X POST http://localhost:4321/api/job/approve-sync \
-H "Content-Type: application/json" \
-H "Cookie: <session>" \
-d '{"repositoryIds": ["<id>"], "action": "dismiss"}'
```
Blocked repos also show an **Approve** / **Dismiss** button in the repository table UI.
## Architecture
### Key Files
| File | Purpose |
|---|---|
| `src/lib/utils/force-push-detection.ts` | Core detection: fetch branches, compare SHAs, check ancestry |
| `src/lib/repo-backup.ts` | Strategy resolver, backup decision logic, bundle creation |
| `src/lib/gitea-enhanced.ts` | Sync flow integration (calls detection + backup before mirror-sync) |
| `src/pages/api/job/approve-sync.ts` | Approve/dismiss API endpoint |
| `src/components/config/GitHubConfigForm.tsx` | Strategy selector UI |
| `src/components/repositories/RepositoryTable.tsx` | Pending-approval badge + action buttons |
### Detection Flow
```
syncGiteaRepoEnhanced()
│
├─ Resolve backup strategy (config → backupStrategy → backupBeforeSync → default)
│
├─ If strategy needs detection ("on-force-push" or "block-on-force-push"):
│ │
│ ├─ fetchGiteaBranches() — GET /api/v1/repos/{owner}/{repo}/branches
│ ├─ fetchGitHubBranches() — octokit.paginate(repos.listBranches)
│ │
│ └─ For each Gitea branch where SHA differs:
│ └─ checkAncestry() — octokit.repos.compareCommits()
│ ├─ "ahead" or "identical" → fast-forward (safe)
│ ├─ "diverged" or "behind" → force-push detected
│ └─ 404/422 → old SHA gone → force-push detected
│
├─ If "block-on-force-push" + detected:
│ └─ Set repo status to "pending-approval", return early
│
├─ If backup needed (always, or on-force-push + detected):
│ └─ Create git bundle snapshot
│
└─ Proceed to mirror-sync
```
## Troubleshooting
**Repos stuck in "pending-approval"**: Use the Approve or Dismiss buttons in the repository table, or call the approve-sync API endpoint.
**Detection always skipped**: Check the activity log for skip reasons. Common causes: Gitea repo not yet mirrored (first sync), GitHub API rate limits, network errors. All are fail-open by design.
**Backups consuming too much space**: Lower the retention count, or switch from "Always Backup" to "Smart" which only creates backups on actual force-pushes.
**False positives**: The detection compares branch-by-branch. A rebase (which is a force-push) will correctly trigger detection. If you routinely rebase branches, consider using "Smart" instead of "Block & Approve" to avoid constant approval prompts.
+486
View File
@@ -0,0 +1,486 @@
# Nix Deployment Guide
This guide covers deploying Gitea Mirror using Nix flakes. The Nix deployment follows the same minimal configuration philosophy as `docker-compose.alt.yml` - secrets are auto-generated, and everything else can be configured via the web UI.
## Prerequisites
- Nix 2.4+ installed
- For NixOS module: NixOS 23.05+
### Enable Flakes (Recommended)
To enable flakes permanently and avoid typing flags, add to `/etc/nix/nix.conf` or `~/.config/nix/nix.conf`:
```
experimental-features = nix-command flakes
```
**Note:** If you don't enable flakes globally, add `--extra-experimental-features 'nix-command flakes'` to all nix commands shown below.
## Quick Start (Zero Configuration!)
### Run Immediately - No Setup Required
```bash
# Run directly from the flake (local)
nix run --extra-experimental-features 'nix-command flakes' .#gitea-mirror
# Or from GitHub (once published)
nix run --extra-experimental-features 'nix-command flakes' github:RayLabsHQ/gitea-mirror
# If you have flakes enabled globally, simply:
nix run .#gitea-mirror
```
That's it! On first run:
- Secrets (`BETTER_AUTH_SECRET` and `ENCRYPTION_SECRET`) are auto-generated
- Database is automatically created and initialized
- Startup recovery and repair scripts run automatically
- Access the web UI at http://localhost:4321
Everything else (GitHub credentials, Gitea settings, mirror options) is configured through the web interface after signup.
### Development Environment
```bash
# Enter development shell with all dependencies
nix develop --extra-experimental-features 'nix-command flakes'
# Or use direnv for automatic environment loading (handles flags automatically)
echo "use flake" > .envrc
direnv allow
```
### Build and Install
```bash
# Build the package
nix build --extra-experimental-features 'nix-command flakes'
# Run the built package
./result/bin/gitea-mirror
# Install to your profile
nix profile install --extra-experimental-features 'nix-command flakes' .#gitea-mirror
```
## What Happens on First Run?
Following the same pattern as the Docker deployment, the Nix package automatically:
1. **Creates data directory**: `~/.local/share/gitea-mirror` (or `$DATA_DIR`)
2. **Generates secrets** (stored securely in data directory):
- `BETTER_AUTH_SECRET` - Session authentication (32-char hex)
- `ENCRYPTION_SECRET` - Token encryption (48-char base64)
3. **Initializes database**: SQLite database with Drizzle migrations
4. **Runs startup scripts**:
- Environment configuration loader
- Crash recovery for interrupted jobs
- Repository status repair
5. **Starts the application** with graceful shutdown handling
## NixOS Module - Minimal Deployment
### Simplest Possible Configuration
Add to your NixOS configuration (`/etc/nixos/configuration.nix`):
```nix
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
gitea-mirror.url = "github:RayLabsHQ/gitea-mirror";
};
outputs = { nixpkgs, gitea-mirror, ... }: {
nixosConfigurations.your-hostname = nixpkgs.lib.nixosSystem {
system = "x86_64-linux";
modules = [
gitea-mirror.nixosModules.default
{
# That's it! Just enable the service
services.gitea-mirror.enable = true;
}
];
};
};
}
```
Apply with:
```bash
sudo nixos-rebuild switch
```
Access at http://localhost:4321, sign up (first user is admin), and configure everything via the web UI.
### Production Configuration
For production with custom domain and firewall:
```nix
{
services.gitea-mirror = {
enable = true;
host = "0.0.0.0";
port = 4321;
betterAuthUrl = "https://mirror.example.com";
betterAuthTrustedOrigins = "https://mirror.example.com";
openFirewall = true;
};
# Optional: Use with nginx reverse proxy
services.nginx = {
enable = true;
virtualHosts."mirror.example.com" = {
locations."/" = {
proxyPass = "http://127.0.0.1:4321";
proxyWebsockets = true;
};
enableACME = true;
forceSSL = true;
};
};
}
```
### Advanced: Manual Secret Management
If you prefer to manage secrets manually (e.g., with sops-nix or agenix):
1. Create a secrets file:
```bash
# /var/lib/gitea-mirror/secrets.env
BETTER_AUTH_SECRET=your-32-character-minimum-secret-key-here
ENCRYPTION_SECRET=your-encryption-secret-here
```
2. Reference it in your configuration:
```nix
{
services.gitea-mirror = {
enable = true;
environmentFile = "/var/lib/gitea-mirror/secrets.env";
};
}
```
### Full Configuration Options
```nix
{
services.gitea-mirror = {
enable = true;
package = gitea-mirror.packages.x86_64-linux.default; # Override package
dataDir = "/var/lib/gitea-mirror";
user = "gitea-mirror";
group = "gitea-mirror";
host = "0.0.0.0";
port = 4321;
betterAuthUrl = "https://mirror.example.com";
betterAuthTrustedOrigins = "https://mirror.example.com";
# Concurrency controls (match docker-compose.alt.yml)
mirrorIssueConcurrency = 3; # Set to 1 for perfect chronological order
mirrorPullRequestConcurrency = 5; # Set to 1 for perfect chronological order
environmentFile = null; # Optional secrets file
openFirewall = true;
};
}
```
## Service Management (NixOS)
```bash
# Start the service
sudo systemctl start gitea-mirror
# Stop the service
sudo systemctl stop gitea-mirror
# Restart the service
sudo systemctl restart gitea-mirror
# Check status
sudo systemctl status gitea-mirror
# View logs
sudo journalctl -u gitea-mirror -f
# Health check
curl http://localhost:4321/api/health
```
## Environment Variables
All variables from `docker-compose.alt.yml` are supported:
```bash
# === AUTO-GENERATED (Don't set unless you want specific values) ===
BETTER_AUTH_SECRET # Auto-generated, stored in data dir
ENCRYPTION_SECRET # Auto-generated, stored in data dir
# === CORE SETTINGS (Have good defaults) ===
DATA_DIR="$HOME/.local/share/gitea-mirror"
DATABASE_URL="file:$DATA_DIR/gitea-mirror.db"
HOST="0.0.0.0"
PORT="4321"
NODE_ENV="production"
# === BETTER AUTH (Override for custom domains) ===
BETTER_AUTH_URL="http://localhost:4321"
BETTER_AUTH_TRUSTED_ORIGINS="http://localhost:4321"
PUBLIC_BETTER_AUTH_URL="http://localhost:4321"
# === CONCURRENCY CONTROLS ===
MIRROR_ISSUE_CONCURRENCY=3 # Default: 3 (set to 1 for perfect order)
MIRROR_PULL_REQUEST_CONCURRENCY=5 # Default: 5 (set to 1 for perfect order)
# === CONFIGURE VIA WEB UI (Not needed at startup) ===
# GitHub credentials, Gitea settings, mirror options, scheduling, etc.
# All configured after signup through the web interface
```
## Database Management
The Nix package includes a database management helper:
```bash
# Initialize database (done automatically on first run)
gitea-mirror-db init
# Check database health
gitea-mirror-db check
# Fix database issues
gitea-mirror-db fix
# Reset users
gitea-mirror-db reset-users
```
## Home Manager Integration
For single-user deployments:
```nix
{ config, pkgs, ... }:
let
gitea-mirror = (import (fetchTarball "https://github.com/RayLabsHQ/gitea-mirror/archive/main.tar.gz")).packages.${pkgs.system}.default;
in {
home.packages = [ gitea-mirror ];
# Optional: Run as user service
systemd.user.services.gitea-mirror = {
Unit = {
Description = "Gitea Mirror Service";
After = [ "network.target" ];
};
Service = {
Type = "simple";
ExecStart = "${gitea-mirror}/bin/gitea-mirror";
Restart = "always";
Environment = [
"DATA_DIR=%h/.local/share/gitea-mirror"
"HOST=127.0.0.1"
"PORT=4321"
];
};
Install = {
WantedBy = [ "default.target" ];
};
};
}
```
## Docker Image from Nix (Optional)
You can also use Nix to create a Docker image:
```nix
# Add to flake.nix packages section
dockerImage = pkgs.dockerTools.buildLayeredImage {
name = "gitea-mirror";
tag = "latest";
contents = [ self.packages.${system}.default pkgs.cacert pkgs.openssl ];
config = {
Cmd = [ "${self.packages.${system}.default}/bin/gitea-mirror" ];
ExposedPorts = { "4321/tcp" = {}; };
Env = [
"DATA_DIR=/data"
"DATABASE_URL=file:/data/gitea-mirror.db"
];
Volumes = { "/data" = {}; };
};
};
```
Build and load:
```bash
nix build --extra-experimental-features 'nix-command flakes' .#dockerImage
docker load < result
docker run -p 4321:4321 -v gitea-mirror-data:/data gitea-mirror:latest
```
## Comparison: Docker vs Nix
Both deployment methods follow the same philosophy:
| Feature | Docker Compose | Nix |
|---------|---------------|-----|
| **Configuration** | Minimal (only BETTER_AUTH_SECRET) | Zero config (auto-generated) |
| **Secret Generation** | Auto-generated & persisted | Auto-generated & persisted |
| **Database Init** | Automatic on first run | Automatic on first run |
| **Startup Scripts** | Runs recovery/repair/env-config | Runs recovery/repair/env-config |
| **Graceful Shutdown** | Signal handling in entrypoint | Signal handling in wrapper |
| **Health Check** | Docker healthcheck | systemd timer (optional) |
| **Updates** | `docker pull` | `nix flake update && nixos-rebuild` |
## Troubleshooting
### Check Auto-Generated Secrets
```bash
# For standalone
cat ~/.local/share/gitea-mirror/.better_auth_secret
cat ~/.local/share/gitea-mirror/.encryption_secret
# For NixOS service
sudo cat /var/lib/gitea-mirror/.better_auth_secret
sudo cat /var/lib/gitea-mirror/.encryption_secret
```
### Database Issues
```bash
# Check if database exists
ls -la ~/.local/share/gitea-mirror/gitea-mirror.db
# Reinitialize (deletes all data!)
rm ~/.local/share/gitea-mirror/gitea-mirror.db
gitea-mirror-db init
```
### Permission Issues (NixOS)
```bash
sudo chown -R gitea-mirror:gitea-mirror /var/lib/gitea-mirror
sudo chmod 700 /var/lib/gitea-mirror
```
### Port Already in Use
```bash
# Change port
export PORT=8080
gitea-mirror
# Or in NixOS config
services.gitea-mirror.port = 8080;
```
### View Startup Logs
```bash
# Standalone (verbose output on console)
gitea-mirror
# NixOS service
sudo journalctl -u gitea-mirror -f --since "5 minutes ago"
```
## Updating
### Standalone Installation
```bash
# Update flake lock
nix flake update --extra-experimental-features 'nix-command flakes'
# Rebuild
nix build --extra-experimental-features 'nix-command flakes'
# Or update profile
nix profile upgrade --extra-experimental-features 'nix-command flakes' gitea-mirror
```
### NixOS
```bash
# Update input
sudo nix flake lock --update-input gitea-mirror --extra-experimental-features 'nix-command flakes'
# Rebuild system
sudo nixos-rebuild switch --flake .#your-hostname
```
## Migration from Docker
To migrate from Docker to Nix while keeping your data:
1. **Stop Docker container:**
```bash
docker-compose -f docker-compose.alt.yml down
```
2. **Copy data directory:**
```bash
# For standalone
cp -r ./data ~/.local/share/gitea-mirror
# For NixOS
sudo cp -r ./data /var/lib/gitea-mirror
sudo chown -R gitea-mirror:gitea-mirror /var/lib/gitea-mirror
```
3. **Copy secrets (if you want to keep them):**
```bash
# Extract from Docker volume
docker run --rm -v gitea-mirror_data:/data alpine \
cat /data/.better_auth_secret > better_auth_secret
docker run --rm -v gitea-mirror_data:/data alpine \
cat /data/.encryption_secret > encryption_secret
# Copy to new location
cp better_auth_secret ~/.local/share/gitea-mirror/.better_auth_secret
cp encryption_secret ~/.local/share/gitea-mirror/.encryption_secret
chmod 600 ~/.local/share/gitea-mirror/.*_secret
```
4. **Start Nix version:**
```bash
gitea-mirror
```
## CI/CD Integration
Example GitHub Actions workflow (see `.github/workflows/nix-build.yml`):
```yaml
name: Nix Build
on: [push, pull_request]
permissions:
contents: read
jobs:
build:
strategy:
matrix:
os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/nix-installer-action@main
- uses: DeterminateSystems/magic-nix-cache-action@main
- run: nix flake check
- run: nix build --print-build-logs
```
This uses:
- **Determinate Nix Installer** - Fast, reliable Nix installation with flakes enabled by default
- **Magic Nix Cache** - Free caching using GitHub Actions cache (no account needed)
## Resources
- [Nix Manual](https://nixos.org/manual/nix/stable/)
- [NixOS Options Search](https://search.nixos.org/options)
- [Nix Pills Tutorial](https://nixos.org/guides/nix-pills/)
- [Project Documentation](../README.md)
- [Docker Deployment](../docker-compose.alt.yml) - Equivalent minimal config
+322
View File
@@ -0,0 +1,322 @@
# Nix Package Distribution Guide
This guide explains how Gitea Mirror is distributed via Nix and how users can consume it.
## Distribution Methods
### Method 1: Direct GitHub Usage (Zero Infrastructure)
**No CI, releases, or setup needed!** Users can consume directly from GitHub:
```bash
# Latest from main branch
nix run --extra-experimental-features 'nix-command flakes' github:RayLabsHQ/gitea-mirror
# Pin to specific commit
nix run github:RayLabsHQ/gitea-mirror/abc123def
# Pin to git tag
nix run github:RayLabsHQ/gitea-mirror/vX.Y.Z
```
**How it works:**
1. Nix fetches the repository from GitHub
2. Nix reads `flake.nix` and `flake.lock`
3. Nix builds the package locally on the user's machine
4. Package is cached in `/nix/store` for reuse
**Pros:**
- Zero infrastructure needed
- Works immediately after pushing code
- Users always get reproducible builds
**Cons:**
- Users must build from source (slower first time)
- Requires build dependencies (Bun, etc.)
---
### Method 2: CI Build Caching
The GitHub Actions workflow uses **Magic Nix Cache** (by Determinate Systems) to cache builds:
- **Zero configuration required** - no accounts or tokens needed
- **Automatic** - CI workflow handles everything
- **Uses GitHub Actions cache** - fast, reliable, free
#### How It Works:
1. GitHub Actions builds the package on each push/PR
2. Build artifacts are cached in GitHub Actions cache
3. Subsequent builds reuse cached dependencies (faster CI)
Note: This caches CI builds. Users still build locally, but the flake.lock ensures reproducibility.
---
### Method 3: nixpkgs Submission (Official Distribution)
Submit to the official Nix package repository for maximum visibility.
#### Process:
1. **Prepare package** (already done with `flake.nix`)
2. **Test thoroughly**
3. **Submit PR to nixpkgs:** https://github.com/NixOS/nixpkgs
#### User Experience:
```bash
# After acceptance into nixpkgs
nix run nixpkgs#gitea-mirror
# NixOS configuration
environment.systemPackages = [ pkgs.gitea-mirror ];
```
**Pros:**
- Maximum discoverability (official repo)
- Trusted by Nix community
- Included in NixOS search
- Binary caching by cache.nixos.org
**Cons:**
- Submission/review process
- Must follow nixpkgs guidelines
- Updates require PRs
---
## Current Distribution Strategy
### Phase 1: Direct GitHub (Immediate) ✅
Already working! Users can:
```bash
nix run github:RayLabsHQ/gitea-mirror
```
### Phase 2: CI Build Validation ✅
GitHub Actions workflow validates builds on every push/PR:
- Uses Magic Nix Cache for fast CI builds
- Tests on both Linux and macOS
- No setup required - works automatically
### Phase 3: Version Releases (Optional)
Tag releases for version pinning:
```bash
git tag vX.Y.Z
git push origin vX.Y.Z
# Users can then pin:
nix run github:RayLabsHQ/gitea-mirror/vX.Y.Z
```
### Phase 4: nixpkgs Submission (Long Term)
Once package is stable and well-tested, submit to nixpkgs.
---
## User Documentation
### For Users: How to Install
Add this to your `docs/NIX_DEPLOYMENT.md`:
#### Option 1: Direct Install (No Configuration)
```bash
# Run immediately
nix run --extra-experimental-features 'nix-command flakes' github:RayLabsHQ/gitea-mirror
# Install to profile
nix profile install --extra-experimental-features 'nix-command flakes' github:RayLabsHQ/gitea-mirror
```
#### Option 2: Pin to Specific Version
```bash
# Pin to git tag
nix run github:RayLabsHQ/gitea-mirror/vX.Y.Z
# Pin to commit
nix run github:RayLabsHQ/gitea-mirror/abc123def
# Lock in flake.nix
inputs.gitea-mirror.url = "github:RayLabsHQ/gitea-mirror/vX.Y.Z";
```
#### Option 3: NixOS Configuration
```nix
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
gitea-mirror.url = "github:RayLabsHQ/gitea-mirror";
# Or pin to version:
# gitea-mirror.url = "github:RayLabsHQ/gitea-mirror/vX.Y.Z";
};
outputs = { nixpkgs, gitea-mirror, ... }: {
nixosConfigurations.your-host = nixpkgs.lib.nixosSystem {
modules = [
gitea-mirror.nixosModules.default
{
services.gitea-mirror = {
enable = true;
betterAuthUrl = "https://mirror.example.com";
openFirewall = true;
};
}
];
};
};
}
```
---
## Maintaining the Distribution
### Releasing New Versions
```bash
# 1. Update version in package.json
vim package.json # Update version field
# 2. Update flake.nix version (line 17)
vim flake.nix # Update version = "X.Y.Z";
# 3. Commit changes
git add package.json flake.nix
git commit -m "chore: bump version to vX.Y.Z"
# 4. Create git tag
git tag vX.Y.Z
git push origin main
git push origin vX.Y.Z
# 5. GitHub Actions builds and caches automatically
```
Users can then pin to the new version:
```bash
nix run github:RayLabsHQ/gitea-mirror/vX.Y.Z
```
### Updating Flake Lock
The `flake.lock` file pins all dependencies. Update it periodically:
```bash
# Update all inputs
nix flake update
# Update specific input
nix flake lock --update-input nixpkgs
# Test after update
nix build
nix flake check
# Commit the updated lock file
git add flake.lock
git commit -m "chore: update flake dependencies"
git push
```
---
## Troubleshooting Distribution Issues
### Users Report Build Failures
1. **Check GitHub Actions:** Ensure CI is passing
2. **Test locally:** `nix flake check`
3. **Check flake.lock:** May need update if dependencies changed
### CI Cache Not Working
1. **Check workflow logs:** Review GitHub Actions for errors
2. **Clear cache:** GitHub Actions → Caches → Delete relevant cache
3. **Verify flake.lock:** May need `nix flake update` if dependencies changed
### Version Pinning Not Working
```bash
# Verify tag exists
git tag -l
# Ensure tag is pushed
git ls-remote --tags origin
# Test specific tag
nix run github:RayLabsHQ/gitea-mirror/vX.Y.Z
```
---
## Advanced: Custom Binary Cache
If you prefer self-hosting instead of Cachix:
### Option 1: S3-Compatible Storage
```nix
# Generate signing key
nix-store --generate-binary-cache-key cache.example.com cache-priv-key.pem cache-pub-key.pem
# Push to S3
nix copy --to s3://my-nix-cache?region=us-east-1 $(nix-build)
```
Users configure:
```nix
substituters = https://my-bucket.s3.amazonaws.com/nix-cache
trusted-public-keys = cache.example.com:BASE64_PUBLIC_KEY
```
### Option 2: Self-Hosted Nix Store
Run `nix-serve` on your server:
```bash
# On server
nix-serve -p 8080
# Behind nginx/caddy
proxy_pass http://localhost:8080;
```
Users configure:
```nix
substituters = https://cache.example.com
trusted-public-keys = YOUR_KEY
```
---
## Comparison: Distribution Methods
| Method | Setup Time | User Speed | Cost | Discoverability |
|--------|-----------|------------|------|-----------------|
| Direct GitHub | 0 min | Slow (build) | Free | Low |
| nixpkgs | Hours/days | Fast (binary) | Free | High |
| Self-hosted cache | 30+ min | Fast (binary) | Server cost | Low |
**Current approach:** Direct GitHub consumption with CI validation using Magic Nix Cache. Users build locally (reproducible via flake.lock). Consider **nixpkgs** submission for maximum reach once the package is mature.
---
## Resources
- [Nix Flakes Documentation](https://nixos.wiki/wiki/Flakes)
- [Magic Nix Cache](https://github.com/DeterminateSystems/magic-nix-cache-action)
- [nixpkgs Contributing Guide](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md)
- [Nix Binary Cache Setup](https://nixos.org/manual/nix/stable/package-management/binary-cache-substituter.html)
+135
View File
@@ -0,0 +1,135 @@
# Notifications
Gitea Mirror supports push notifications for mirror events. You can be alerted when jobs succeed, fail, or when new repositories are discovered.
## Supported Providers
### 1. Ntfy.sh (Direct)
[Ntfy.sh](https://ntfy.sh) is a simple HTTP-based pub-sub notification service. You can use the public server at `https://ntfy.sh` or self-host your own instance.
**Setup (public server):**
1. Go to **Configuration > Notifications**
2. Enable notifications and select **Ntfy.sh** as the provider
3. Set the **Topic** to a unique name (e.g., `my-gitea-mirror-abc123`)
4. Leave the Server URL as `https://ntfy.sh`
5. Subscribe to the same topic on your phone or desktop using the [ntfy app](https://ntfy.sh/docs/subscribe/phone/)
**Setup (self-hosted):**
1. Deploy ntfy using Docker: `docker run -p 8080:80 binwiederhier/ntfy serve`
2. Set the **Server URL** to your instance (e.g., `http://ntfy:8080`)
3. If authentication is enabled, provide an **Access token**
4. Set your **Topic** name
**Priority levels:**
- `min` / `low` / `default` / `high` / `urgent`
- Error notifications automatically use `high` priority regardless of the default setting
### 2. Apprise API (Aggregator)
[Apprise](https://github.com/caronc/apprise-api) is a notification aggregator that supports 100+ services (Slack, Discord, Telegram, Email, Pushover, and many more) through a single API.
**Setup:**
1. Deploy the Apprise API server:
```yaml
# docker-compose.yml
services:
apprise:
image: caronc/apprise:latest
ports:
- "8000:8000"
volumes:
- apprise-config:/config
volumes:
apprise-config:
```
2. Configure your notification services in Apprise (via its web UI at `http://localhost:8000` or API)
3. Create a configuration token/key in Apprise
4. In Gitea Mirror, go to **Configuration > Notifications**
5. Enable notifications and select **Apprise API**
6. Set the **Server URL** to your Apprise instance (e.g., `http://apprise:8000`)
7. Enter the **Token/path** you created in step 3
**Tag filtering:**
- Optionally set a **Tag** to only notify specific Apprise services
- Leave empty to notify all configured services
### 3. Gotify (Direct)
[Gotify](https://gotify.net/) is a simple self-hosted push notification server.
**Setup:**
1. In Gotify, create an application and copy its token
2. In Gitea Mirror, go to **Configuration > Notifications**
3. Enable notifications and select **Gotify**
4. Set the **Server URL** to your Gotify instance (e.g., `https://gotify.example.com`)
5. Paste the **Application token**
6. Pick a **Default priority** (0 to 10; error notifications are always sent at priority 8)
The token is encrypted at rest like all other provider credentials.
### 4. Webhook (Generic)
Sends a plain JSON POST to any URL you control. Works with n8n, Home Assistant, or any service that accepts a generic webhook.
**Setup:**
1. In Gitea Mirror, go to **Configuration > Notifications**
2. Enable notifications and select **Webhook**
3. Set the **Webhook URL** to your endpoint
4. Optionally set a **Signing secret**
**Payload:**
```json
{
"title": "Mirror Failed: my-repo",
"message": "Repository my-repo failed to mirror",
"type": "sync_error",
"timestamp": "2026-08-03T02:33:29.707Z"
}
```
**Signature verification:**
When a signing secret is set, every request includes an `X-Webhook-Signature` header containing `sha256=<hex>`, an HMAC-SHA256 digest of the raw request body. Verify it on the receiving end:
```js
const crypto = require("node:crypto");
const expected = "sha256=" +
crypto.createHmac("sha256", secret).update(rawBody).digest("hex");
const valid = crypto.timingSafeEqual(
Buffer.from(signatureHeader), Buffer.from(expected));
```
The secret is encrypted at rest.
## Event Types
| Event | Default | Description |
|-------|---------|-------------|
| Sync errors | On | A mirror job failed |
| Sync success | Off | A mirror job completed successfully |
| New repo discovered | Off | Not yet implemented; the toggle is disabled in the UI |
## Testing
Use the **Send Test Notification** button on the Notifications settings page to verify your configuration. The test sends a sample success notification to your configured provider.
## Troubleshooting
**Notifications not arriving:**
- Check that notifications are enabled in the settings
- Verify the provider configuration (URL, topic/token)
- Use the Test button to check connectivity
- Check the server logs for `[NotificationService]` messages
**Ntfy authentication errors:**
- Ensure your access token is correct
- If self-hosting, verify the ntfy server allows the topic
**Apprise connection refused:**
- Verify the Apprise API server is running and accessible from the Gitea Mirror container
- If using Docker, ensure both containers are on the same network
- Check the Apprise server logs for errors
**Tokens and security:**
- Notification tokens (ntfy access tokens, Apprise tokens) are encrypted at rest using the same AES-256-GCM encryption as GitHub/Gitea tokens
- Tokens are decrypted only when sending notifications or displaying in the settings UI
+85
View File
@@ -0,0 +1,85 @@
# Push targets: GitHub and GitLab as destinations
Gitea and Forgejo are pull mirrors: the app asks them to fetch from the source, and they keep the copy up to date themselves. GitHub has no pull mirror API and GitLab only offers one on paid tiers, so for those two the app runs a push engine instead. This page explains what that engine does, what it needs, and what it does not do.
Push targets are marked **beta** in the destination dropdown, like every source and destination other than GitHub to Gitea. They are tested end to end against github.com and gitlab.com, but have had less time in the field than the default path.
## How it works
For every repository whose destination is GitHub or GitLab:
1. **A bare clone is kept on disk** under `<data dir>/mirrors/<source host>/<owner>/<name>.git`, next to the SQLite database. The first run clones, later runs fetch with prune, so a branch or tag deleted on the source is dropped from the clone.
2. **The target repository is created when missing.** GitHub: under the account named in the connection card, or under an organization the mirror strategy picks. GitLab: a project under the user, or in a group; a missing top level group is created, a nested group is not.
3. **Branches and tags are pushed** with force and prune. The target always ends up matching the source, including rewritten history and deleted branches. Pull request refs, merge request refs and other host owned namespaces are not pushed; hosts refuse writes to them.
4. **The default branch is set** on the target to match the source, best effort.
5. **Statuses, mirror jobs and activity rows** are the same as for a Gitea mirror, so the dashboard, the scheduler, recovery and cleanup treat both the same way. The repository table shows the destination's icon and links to the target repository.
The scheduler runs the same fetch and push on every scheduled sync. Both steps are idempotent: an interrupted run just runs again.
## The target is overwritten
Every push uses force and prune. Anything committed directly on the target, a branch created there, a tag moved there, is replaced by what the source has on the next sync. Treat a push target as read only, the same way a Gitea pull mirror is.
## What is not mirrored
Push targets receive **branches and tags only**. The following are not available for them and show as disabled in the settings:
- issues, pull requests, labels, milestones
- releases and release assets (tags do travel)
- wiki
- LFS objects
- force push protection and the backup strategies
Several destinations per repository, and pulling anything back from the target, are out of scope. Mirroring is one way.
## Credentials
Tokens never appear in a URL and are never written to disk. Each git invocation gets an inline credential helper that reads the token from an environment variable set on that process only: the source token for the fetch, the target token for the push. The remote URL stored in the bare clone is the plain source URL.
Token scopes:
| Target | Scopes | Notes |
|--------|--------|-------|
| GitHub | `repo`, `workflow` | `workflow` is needed as soon as a repository contains GitHub Actions files, or the push is rejected. Add `delete_repo` only if cleanup should delete repositories on GitHub. |
| GitLab | `api`, `write_repository` | Pushes use the `oauth2` user name with the token, as GitLab documents. |
GitHub organizations cannot be created through the API. Create the organization first and make the token's user a member with permission to create repositories.
## Disk usage and concurrency
A bare clone takes roughly the size of the repository's git objects, so the data directory grows by about the size of everything you mirror this way. Clones are removed when the repository row is deleted, and by cleanup when it deletes an orphaned repository.
| Variable | Default | Meaning |
|----------|---------|---------|
| `MIRROR_CLONE_DIR` | `<data dir>/mirrors` | Where the bare clones live. |
| `PUSH_CONCURRENCY` | `2` | How many git fetch or push operations run at the same time across all repositories. API calls are cheap, a clone or a mirror push is not. |
One git process per repository at a time is enforced with a lock file next to the clone. A lock older than an hour belongs to a run that died and is taken over; a half written clone (no `HEAD`) is removed and cloned again.
## Large first pushes
GitHub limits the size of a single push. When the single push of all refs is refused, the engine retries in batches of 50 refs, branches first, then tags, followed by a prune. If a batch is still refused, the mirror fails with the git error in the repository's status.
## Cleanup
When the source deletes a repository, the cleanup service applies the configured action to the target through its API: archive on GitHub (`archived: true`) or GitLab (`/archive`), or delete. Deleting on GitHub needs the `delete_repo` scope; without it the cleanup run fails with a message naming the scope and the repository stays. On gitlab.com and instances with delayed deletion, a deleted project is renamed and kept for the configured number of days before it disappears. The bare clone is removed with the row.
The **Reconcile with destination** action compares a Gitea or Forgejo server with the database; it is not available for push targets, which have no server side mirror state to reconcile.
## Switching destinations
Once anything has been mirrored the destination is locked and changing it asks for confirmation, as for Gitea. Rows remember which destination they were mirrored to; a row mirrored to one host is refused on another until it is removed and mirrored again, so a token is never sent to the wrong host.
## Environment variables
```env
DESTINATION_PROVIDER=github # or gitlab
GITEA_URL=https://github.com # optional for github.com and gitlab.com; set for GHES or self hosted GitLab
GITEA_USERNAME=your-account
GITEA_TOKEN=your-token
GITEA_ORGANIZATION=your-org # optional, used by the single-org strategy
MIRROR_CLONE_DIR=/data/mirrors # optional
PUSH_CONCURRENCY=2 # optional
```
The `GITEA_*` names are kept for every destination kind so an existing deployment only has to change `DESTINATION_PROVIDER`. See [ENVIRONMENT_VARIABLES.md](ENVIRONMENT_VARIABLES.md).
+46
View File
@@ -0,0 +1,46 @@
# Gitea Mirror Documentation
This folder contains engineering and operations references for the open-source Gitea Mirror project. Each guide focuses on the parts of the system that still require bespoke explanation beyond the in-app help and the main `README.md`.
## Available Guides
### Core workflow
- **[DEVELOPMENT_WORKFLOW.md](./DEVELOPMENT_WORKFLOW.md)** – Set up a local environment, run scripts, and understand the repo layout (app + marketing site).
- **[ENVIRONMENT_VARIABLES.md](./ENVIRONMENT_VARIABLES.md)** – Complete reference for every configuration flag supported by the app and Docker images.
- **[NIX_DEPLOYMENT.md](./NIX_DEPLOYMENT.md)** – User-facing deployment guide for Nix and NixOS.
- **[NIX_DISTRIBUTION.md](./NIX_DISTRIBUTION.md)** – Maintainer notes for packaging, releases, and distribution strategy.
### Reliability & recovery
- **[GRACEFUL_SHUTDOWN.md](./GRACEFUL_SHUTDOWN.md)** – How signal handling, shutdown coordination, and job persistence work in v3.
- **[RECOVERY_IMPROVEMENTS.md](./RECOVERY_IMPROVEMENTS.md)** – Deep dive into the startup recovery workflow and supporting scripts.
### Authentication
- **[SSO-OIDC-SETUP.md](./SSO-OIDC-SETUP.md)** – Configure OIDC/SSO providers through the admin UI.
- **[SSO_TESTING.md](./SSO_TESTING.md)** – Recipes for local and staging SSO testing (Google, Keycloak, mock providers).
### API
- **[API.md](./API.md)** – Create API keys and call the app from scripts, CI pipelines and workflow tools.
### Sources and destinations
- **[SOURCE_PROVIDERS.md](./SOURCE_PROVIDERS.md)** – GitHub, GitLab and Gitea/Forgejo as sources: tokens, what each can mirror, the source lock.
- **[PUSH_TARGETS.md](./PUSH_TARGETS.md)** – GitHub and GitLab as destinations: how the push engine works, token scopes, disk usage, what is not mirrored.
The user-facing documentation lives on the website at <https://gitea-mirror.raylabs.io/docs/>, with its sources under `www/src/pages/docs/`. If you are looking for playbooks, see the MDX use cases under `www/src/pages/use-cases/`.
## Quick start for local development
```bash
git clone https://github.com/RayLabsHQ/gitea-mirror.git
cd gitea-mirror
bun run setup # installs deps and seeds the SQLite DB
bun run dev # starts the Astro/Bun app on http://localhost:4321
```
The first user you create locally becomes the administrator. All other configuration—GitHub owners, Gitea targets, scheduling, cleanup—is done through the **Configuration** screen in the UI.
## Contributing & support
- 🎯 Contribution guide: [../CONTRIBUTING.md](../CONTRIBUTING.md)
- 🐞 Issues & feature requests: <https://github.com/RayLabsHQ/gitea-mirror/issues>
- 💬 Discussions: <https://github.com/RayLabsHQ/gitea-mirror/discussions>
- 🔐 Security policy & advisories: <https://github.com/RayLabsHQ/gitea-mirror/security>
+3 -1
View File
@@ -63,9 +63,11 @@ The middleware now serves as a fallback recovery mechanism:
#### Improvements:
- **Proper Drizzle ORM syntax** for all database queries
- **Enhanced interrupted job detection** with multiple criteria:
- **Enhanced interrupted job detection** with multiple criteria (see `src/lib/interrupted-job-detection.ts`):
- Jobs with no recent checkpoint (10+ minutes)
- Jobs that never wrote a checkpoint and started 10+ minutes ago (a job that just started is live, not interrupted)
- Jobs running too long (2+ hours)
- **Heartbeat during processing**: in-progress jobs start with a checkpoint and refresh it every 2 minutes, so one long item cannot make a live job look interrupted
- **Detailed logging** of found interrupted jobs
- **Better error handling** for database operations
-236
View File
@@ -1,236 +0,0 @@
# Graceful Shutdown Process
This document details how the gitea-mirror application handles graceful shutdown during active mirroring operations, with specific focus on job interruption and recovery.
## Overview
The graceful shutdown system is designed for **fast, clean termination** without waiting for long-running jobs to complete. It prioritizes **quick shutdown times** (under 30 seconds) while **preserving all progress** for seamless recovery.
## Key Principle
**The application does NOT wait for jobs to finish before shutting down.** Instead, it saves the current state and resumes after restart.
## Shutdown Scenario Example
### Initial State
- **Job**: Mirror 500 repositories
- **Progress**: 200 repositories completed
- **Remaining**: 300 repositories pending
- **Action**: User initiates shutdown (SIGTERM, Ctrl+C, Docker stop)
### Shutdown Process (Under 30 seconds)
#### Step 1: Signal Detection (Immediate)
```
📡 Received SIGTERM signal
🛑 Graceful shutdown initiated by signal: SIGTERM
📊 Shutdown status: 1 active jobs, 2 callbacks
```
#### Step 2: Job State Saving (1-10 seconds)
```
📝 Step 1: Saving active job states...
Saving state for job abc-123...
✅ Saved state for job abc-123
```
**What gets saved:**
- `inProgress: false` - Mark job as not currently running
- `completedItems: 200` - Number of repos successfully mirrored
- `totalItems: 500` - Total repos in the job
- `completedItemIds: [repo1, repo2, ..., repo200]` - List of completed repos
- `itemIds: [repo1, repo2, ..., repo500]` - Full list of repos
- `lastCheckpoint: 2025-05-24T17:30:00Z` - Exact shutdown time
- `message: "Job interrupted by application shutdown - will resume on restart"`
- `status: "imported"` - Keeps status as resumable (not "failed")
#### Step 3: Service Cleanup (1-5 seconds)
```
🔧 Step 2: Executing shutdown callbacks...
🛑 Shutting down cleanup service...
✅ Cleanup service stopped
✅ Shutdown callback 1 completed
```
#### Step 4: Clean Exit (Immediate)
```
💾 Step 3: Closing database connections...
✅ Graceful shutdown completed successfully
```
**Total shutdown time: ~15 seconds** (well under the 30-second limit)
## What Happens to the Remaining 300 Repos?
### During Shutdown
- **NOT processed** - The remaining 300 repos are not mirrored
- **NOT lost** - Their IDs are preserved in the job state
- **NOT marked as failed** - Job status remains "imported" for recovery
### After Restart
The recovery system automatically:
1. **Detects interrupted job** during startup
2. **Calculates remaining work**: 500 - 200 = 300 repos
3. **Extracts remaining repo IDs**: repos 201-500 from the original list
4. **Resumes processing** from exactly where it left off
5. **Continues until completion** of all 500 repos
## Timeout Configuration
### Shutdown Timeouts
```typescript
const SHUTDOWN_TIMEOUT = 30000; // 30 seconds max shutdown time
const JOB_SAVE_TIMEOUT = 10000; // 10 seconds to save job state
```
### Timeout Behavior
- **Normal case**: Shutdown completes in 10-20 seconds
- **Slow database**: Up to 30 seconds allowed
- **Timeout exceeded**: Force exit with code 1
- **Container kill**: Orchestrator should allow 45+ seconds grace period
## Job State Persistence
### Database Schema
The `mirror_jobs` table stores complete job state:
```sql
-- Job identification
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
job_type TEXT NOT NULL DEFAULT 'mirror',
-- Progress tracking
total_items INTEGER,
completed_items INTEGER DEFAULT 0,
item_ids TEXT, -- JSON array of all repo IDs
completed_item_ids TEXT DEFAULT '[]', -- JSON array of completed repo IDs
-- State management
in_progress INTEGER NOT NULL DEFAULT 0, -- Boolean: currently running
started_at TIMESTAMP,
completed_at TIMESTAMP,
last_checkpoint TIMESTAMP, -- Last progress save
-- Status and messaging
status TEXT NOT NULL DEFAULT 'imported',
message TEXT NOT NULL
```
### Recovery Query
The recovery system finds interrupted jobs:
```sql
SELECT * FROM mirror_jobs
WHERE in_progress = 0
AND status = 'imported'
AND completed_at IS NULL
AND total_items > completed_items;
```
## Shutdown-Aware Processing
### Concurrency Check
During job execution, each repo processing checks for shutdown:
```typescript
// Before processing each repository
if (isShuttingDown()) {
throw new Error('Processing interrupted by application shutdown');
}
```
### Checkpoint Intervals
Jobs save progress periodically (every 10 repos by default):
```typescript
checkpointInterval: 10, // Save progress every 10 repositories
```
This ensures minimal work loss even if shutdown occurs between checkpoints.
## Container Integration
### Docker Entrypoint
The Docker entrypoint properly forwards signals:
```bash
# Set up signal handlers
trap 'shutdown_handler' TERM INT HUP
# Start application in background
bun ./dist/server/entry.mjs &
APP_PID=$!
# Wait for application to finish
wait "$APP_PID"
```
### Kubernetes Configuration
Recommended pod configuration:
```yaml
apiVersion: v1
kind: Pod
spec:
terminationGracePeriodSeconds: 45 # Allow time for graceful shutdown
containers:
- name: gitea-mirror
# ... other configuration
```
## Monitoring and Logging
### Shutdown Logs
```
🛑 Graceful shutdown initiated by signal: SIGTERM
📊 Shutdown status: 1 active jobs, 2 callbacks
📝 Step 1: Saving active job states...
Saving state for 1 active jobs...
✅ Completed saving all active jobs
🔧 Step 2: Executing shutdown callbacks...
✅ Completed all shutdown callbacks
💾 Step 3: Closing database connections...
✅ Graceful shutdown completed successfully
```
### Recovery Logs
```
⚠️ Jobs found that need recovery. Starting recovery process...
Resuming job abc-123 with 300 remaining items...
✅ Recovery completed successfully
```
## Best Practices
### For Operations
1. **Monitor shutdown times** - Should complete under 30 seconds
2. **Check recovery logs** - Verify jobs resume correctly after restart
3. **Set appropriate grace periods** - Allow 45+ seconds in orchestrators
4. **Plan maintenance windows** - Jobs will resume but may take time to complete
### For Development
1. **Test shutdown scenarios** - Use `bun run test-shutdown`
2. **Monitor job progress** - Check checkpoint frequency and timing
3. **Verify recovery** - Ensure interrupted jobs resume correctly
4. **Handle edge cases** - Test shutdown during different job phases
## Troubleshooting
### Shutdown Takes Too Long
- **Check**: Database performance during job state saving
- **Solution**: Increase `SHUTDOWN_TIMEOUT` environment variable
- **Monitor**: Job complexity and checkpoint frequency
### Jobs Don't Resume
- **Check**: Recovery logs for errors during startup
- **Verify**: Database contains interrupted jobs with correct status
- **Test**: Run `bun run startup-recovery` manually
### Container Force-Killed
- **Check**: Container orchestrator termination grace period
- **Increase**: Grace period to 45+ seconds
- **Monitor**: Application shutdown completion time
This design ensures **production-ready graceful shutdown** with **zero data loss** and **fast recovery times** suitable for modern containerized deployments.
+107
View File
@@ -0,0 +1,107 @@
# Source Providers
GitHub to Gitea is the supported default. Every other source and every other destination is marked **beta** in the dropdowns: they are tested end to end against real hosts, but have had less time in the field. The destination card has a matching **Destination** dropdown with Gitea, Forgejo (beta, same API as Gitea), GitHub (beta) and GitLab (beta); `DESTINATION_PROVIDER` sets it from the environment, and [PUSH_TARGETS.md](PUSH_TARGETS.md) covers the GitHub and GitLab targets.
Gitea Mirror pulls repositories from as many source hosts as the user connects, github.com and a GitHub Enterprise instance included. The connection card on the Configuration page is a list of sources with add, edit and remove. GitHub is the default, and GitLab and Gitea/Forgejo are available from the **Source** dropdown when a source is added or edited.
| Source | Covers | Default instance URL |
|--------|--------|----------------------|
| GitHub | github.com, GitHub Enterprise (via `GH_API_URL`) | `https://github.com` |
| GitLab (beta) | gitlab.com, self hosted GitLab | `https://gitlab.com` |
| Gitea / Forgejo (beta) | Codeberg, self hosted Gitea or Forgejo | `https://codeberg.org` |
Picking GitLab or Gitea/Forgejo shows an **Instance URL** field. Leave it empty for the default instance, or enter the base URL of your own (for example `https://gitlab.example.com` or `http://gitea.local:3000/gitea`). The username and token fields hold the account and token for the selected host.
## Tokens
| Source | Where to create the token | Scopes |
|--------|---------------------------|--------|
| GitHub | Settings, Developer settings, Personal access tokens (classic) | `repo`, `admin:org` |
| GitLab | Preferences, Access tokens | `read_api`, `read_repository` |
| Gitea / Forgejo | Settings, Applications | `read:repository`, `read:user`, `read:organization` |
The **Test** button on the card checks the token against the selected host.
## What is mirrored
The mirror itself is done by Gitea's pull mirror, which treats every source as a plain git remote. The differences are in what the source API offers.
| | GitHub | GitLab | Gitea / Forgejo |
|---|:---:|:---:|:---:|
| Code, branches, tags | yes | yes | yes |
| Wiki | yes | yes | yes |
| LFS objects | yes | yes | yes |
| Scheduled sync | yes | yes | yes |
| Auto import of new repositories | yes | yes | yes |
| Cleanup of repositories deleted upstream | yes | yes | yes |
| Starred repositories | yes | yes | yes |
| Private repositories | yes | yes | yes |
| Entire organizations (GitLab: top level groups) | yes | yes | yes |
| Add a single repository by URL | yes | yes | yes |
| Issues, pull requests, labels, milestones | yes | no | no |
| Releases with assets | yes | no | no |
| Star lists | yes | no | no |
| Force push detection | yes | no | no |
The GitHub only rows read the GitHub API. For other sources the corresponding switches are disabled on the Configuration page, and the mirror step skips them.
Entire organizations also mirror without a token on every source; the next section covers public mode.
## Public organizations (no source connection)
An organization or group can be mirrored without any source account. Gitea Mirror lists the public repositories of a GitHub organization, a GitLab group or a Gitea/Forgejo organization on github.com, gitlab.com, a self hosted GitLab instance or a Gitea/Forgejo instance, and mirrors code, branches, tags, wiki and LFS objects through an anonymous mirror clone.
On the Organizations page, **Add organization** has a public mode, which is the default when no source is connected: pick the provider, fill in the instance URL for a self hosted GitLab or Gitea/Forgejo host (empty means the default instance), and enter the organization name. Saving creates a public only source row for that instance automatically, so the sources card does not have to be visited first. The row appears in the sources card with a public only indicator and can be edited or removed like any other source. New repositories added to the organization upstream are discovered on the next scheduled run.
Public mode degrades in known ways:
- **GitHub metadata is best effort.** Anonymous requests run under GitHub's 60 requests per hour limit, so a large organization can end up with partial metadata. Releases, description and topics mirror anonymously with no account at all. Issues, pull requests, labels and milestones are also fetched anonymously once any source in the account carries a token; with no token anywhere they are skipped, and the mirror itself is unaffected either way.
- **GitLab and Gitea/Forgejo sources mirror code only.** Metadata remains GitHub only, as in the table above, and issues and merge requests from those sources stay a Not yet item with or without a token.
- **Personal auto discovery and starred repositories need a token.** Neither runs for a public only source.
- **Cleanup does not run.** Repositories deleted upstream are not detected for public only sources.
Listing and metadata go through the source API and are rate limited by the host when anonymous:
| Source | Anonymous API limit |
|--------|---------------------|
| GitHub | 60 requests per hour per IP address |
| GitLab.com | 500 requests per minute per IP address. GitLab has announced a phased reduction to 60 requests per hour for unauthenticated API traffic, planned for late 2026 |
| Gitea / Forgejo | No built in API rate limit. Instances may throttle or close anonymous API access on their own |
The mirror itself is a plain git clone and does not consume API requests.
Adding a username and token to the public only row on the Configuration page upgrades it to a regular source: private repositories, personal auto discovery and starred repositories start working, and GitHub metadata moves from the anonymous limit to the authenticated one. The public only row also enables adding a public repository by URL on that host, which needs a source row but no token.
The API accepts the same fields. `POST /api/sync/organization` adds a public organization like this:
```json
{ "org": "acme", "role": "member", "provider": "gitlab", "sourceUrl": "https://gitlab.example.com" }
```
`provider` is `github`, `gitlab` or `gitea`. `sourceUrl` is optional and defaults to the provider's default instance from the table at the top.
## Behaviour to know about
- **Each source locks once repositories are imported from it, and the destination locks once anything is mirrored.** A locked source's **Source** dropdown and instance URL, and the Gitea server URL, then show a lock note and a **Change** button. Changing them is still possible, but only after confirming a dialog that spells out what happens to the existing repositories. Removing a source that has repositories needs an explicit confirmation too. Saves that try to switch a locked source without that confirmation are refused by the API, and an environment variable that disagrees with a locked source is ignored on boot with a warning.
- **Sources run side by side.** Every connected source discovers, imports and mirrors independently, and each repository remembers which source it came from. Cleanup checks each source's own repositories, and mirroring a repository uses its own source's credentials. Repositories whose source was removed keep syncing on the Gitea side, because Gitea holds the stored clone credentials, but re-mirroring, metadata and cleanup skip them.
- **GitLab groups are flattened.** A project under `group/subgroup/project` lands in the Gitea organization `group` (with the preserve strategy) and keeps its full path as the repository's full name. The "Limit to specific groups" filter matches the top level group.
- **GitLab internal projects are treated as private** when the mirror is created, because they are not visible to anonymous users on the source either.
- **Repository names come from the URL path**, not the display name, so a GitLab project called "My Widget" with path `my-widget` mirrors as `my-widget`.
- **Without a token** only the configured user's public repositories and starred repositories are visible on GitLab and Gitea/Forgejo. Adding a public repository by URL works without a token on every source.
## Environment variables
| Variable | Description |
|----------|-------------|
| `SOURCE_PROVIDER` | `github` (default), `gitlab` or `gitea` |
| `SOURCE_URL` | Instance base URL for GitLab and Gitea/Forgejo |
| `GITHUB_USERNAME` | Username on the selected host |
| `GITHUB_TOKEN` | Token for the selected host |
These variables initialize or update the first source connection, creating it when none exists, and never switch a source that is locked. A `GH_API_URL` set for a single GitHub Enterprise deployment keeps applying to sources that point at github.com, so adding a github.com source in such a deployment sends its token to the configured Enterprise API; connect the Enterprise host as its own source instead.
See [ENVIRONMENT_VARIABLES.md](ENVIRONMENT_VARIABLES.md) for the full list.
## Not yet
Hosts without a usable API, and issues, merge requests and releases from GitLab, Gitea or Forgejo sources. Discovery preferences are shared, so filters apply to all sources rather than per source, and the GitHub and GitLab rate limit buckets are shared per provider. GitHub and GitLab as destinations shipped in 3.31.0 through the push engine, see [PUSH_TARGETS.md](PUSH_TARGETS.md).
+316
View File
@@ -0,0 +1,316 @@
# SSO and OIDC Setup Guide
This guide explains how to configure Single Sign-On (SSO) and OpenID Connect (OIDC) provider functionality in Gitea Mirror.
## Overview
Gitea Mirror supports four authentication methods:
1. **Email & Password** - Traditional authentication (always enabled)
2. **SSO (Single Sign-On)** - Allow users to authenticate using external OIDC providers
3. **Header / Forward Authentication** - Trust pre-authenticated requests from a reverse proxy (Authentik, Authelia, oauth2-proxy, etc.)
4. **OIDC Provider** - Allow other applications to authenticate users through Gitea Mirror
## Configuration
SSO and OIDC Provider settings are managed through the web UI in the Configuration page under the "Authentication" tab. Header / Forward Authentication is configured via environment variables only — see [Setting up Header / Forward Authentication](#setting-up-header--forward-authentication) below.
## Setting up SSO (Single Sign-On)
SSO allows your users to sign in using external identity providers like Google, Okta, Azure AD, etc.
### Adding an SSO Provider
1. Navigate to Configuration → Authentication → SSO Providers
2. Click "Add Provider"
3. Fill in the provider details:
#### Required Fields
- **Issuer URL**: The OIDC issuer URL (e.g., `https://accounts.google.com`)
- **Domain**: The email domain this provider serves (e.g., `example.com`). **This is load-bearing for account-linking trust — see [Account Linking](#account-linking) below.** Multi-domain IdPs can use a comma-separated list (`example.com,subsidiary.com`).
- **Provider ID**: A unique identifier for this provider (e.g., `google-sso`)
- **Client ID**: The OAuth client ID from your provider
- **Client Secret**: The OAuth client secret from your provider
#### Auto-Discovery
If your provider supports OIDC discovery, you can:
1. Enter the Issuer URL
2. Click "Discover"
3. The system will automatically fetch the authorization and token endpoints
#### Manual Configuration
For providers without discovery support, manually enter:
- **Authorization Endpoint**: The OAuth authorization URL
- **Token Endpoint**: The OAuth token exchange URL
- **JWKS Endpoint**: The JSON Web Key Set URL (optional)
- **UserInfo Endpoint**: The user information endpoint (optional)
### Redirect URL
When configuring your SSO provider, use this redirect URL:
```
https://your-domain.com/api/auth/sso/callback/{provider-id}
```
Replace `{provider-id}` with your chosen Provider ID.
### Account Linking
When a user signs in via SSO whose email matches an **existing** email/password account, Gitea Mirror will try to *link* the SSO identity to that account so the user lands on the same dashboard — instead of being bounced to the login page or creating a duplicate account.
The trust model is **domain-scoped**:
- Every SSO provider you register is automatically marked `domainVerified: true` for the **Domain** field you set on registration.
- An SSO sign-in is auto-linked to an existing local account **only when** the user's email address actually belongs to that registered domain. Cross-domain emails are rejected even if the provider is registered.
- Local email/password accounts in this app are never verified (no email-verification flow exists), so we deliberately turn off `requireLocalEmailVerified` on the linker — otherwise no one could ever link.
**What this means for you:**
- Set the **Domain** field to the email domain(s) your IdP actually issues identities for. Don't set it to `example.com` if your IdP issues `@elsewhere.com` emails — auto-linking will silently refuse them.
- If your IdP allows users to self-register or claim arbitrary emails *within the registered domain*, an attacker on that IdP can absorb a local account by claiming the same email. This is a trust decision: by registering an IdP, you're vouching for its identity model for that domain.
- Multi-domain IdPs (one Authentik serving `acme.com,acquired.io`) work fine — list every domain comma-separated in the **Domain** field.
### Example: Google SSO Setup
1. Go to [Google Cloud Console](https://console.cloud.google.com/)
2. Create a new OAuth 2.0 Client ID
3. Add authorized redirect URI: `https://your-domain.com/api/auth/sso/callback/google-sso`
4. In Gitea Mirror:
- Issuer URL: `https://accounts.google.com`
- Domain: `your-company.com`
- Provider ID: `google-sso`
- Client ID: [Your Google Client ID]
- Client Secret: [Your Google Client Secret]
- Click "Discover" to auto-fill endpoints
### Example: Okta SSO Setup
1. In Okta Admin Console, create a new OIDC Web Application
2. Set redirect URI: `https://your-domain.com/api/auth/sso/callback/okta-sso`
3. In Gitea Mirror:
- Issuer URL: `https://your-okta-domain.okta.com`
- Domain: `your-company.com`
- Provider ID: `okta-sso`
- Client ID: [Your Okta Client ID]
- Client Secret: [Your Okta Client Secret]
- Click "Discover" to auto-fill endpoints
### Example: Authentik SSO Setup
Working Authentik deployments (see [#134](https://github.com/RayLabsHQ/gitea-mirror/issues/134)) follow these steps:
1. In Authentik, create a new **Application** and OIDC **Provider** (implicit flow works well for testing).
2. Start creating an SSO provider inside Gitea Mirror so you can copy the redirect URL shown (`https://your-domain.com/api/auth/sso/callback/authentik` if you pick `authentik` as your Provider ID).
3. Paste that redirect URL into the Authentik Provider configuration and finish creating the provider.
4. Copy the Authentik issuer URL, client ID, and client secret.
5. Back in Gitea Mirror:
- Issuer URL: the exact value from Authentik (keep any trailing slash Authentik shows).
- Provider ID: match the one you used in step 2.
- Click **Discover** so Gitea Mirror stores the authorization, token, and JWKS endpoints (Authentik publishes them via discovery).
- Domain: enter the email domain you expect to match (e.g. `example.com`).
6. Save the provider and test the login flow.
Notes:
- Make sure `BETTER_AUTH_URL` and (if you serve the UI from multiple origins) `BETTER_AUTH_TRUSTED_ORIGINS` point at the public URL users reach. A mismatch can surface as 500 errors after redirect.
- **Internal / split-DNS IdPs**: since v3.21.0 the SSO sign-in refuses identity providers whose hostnames resolve to private addresses (SSRF hardening in the auth library) unless the IdP's origin is trusted. Providers you register through the UI are trusted automatically, but if you register a *new* provider whose hostname resolves to a LAN IP from inside the container (common with split-horizon DNS), add its origin to `BETTER_AUTH_TRUSTED_ORIGINS` first, e.g. `BETTER_AUTH_TRUSTED_ORIGINS=https://auth.example.com`.
- Set the **Domain** field to the email domain your Authentik users actually have. Auto-linking to an existing local admin only happens for emails in that domain — see [Account Linking](#account-linking) for the trust model. (Authentik's default email scope mapping returns `email_verified: False` for OIDC clients, which is why account linking is gated on the domain match here rather than the IdP's verified-email claim.)
- If you created an Authentik provider before v3.8.10 you should delete it and re-add it after upgrading; older versions saved incomplete endpoint data which leads to the `url.startsWith` error explained in the Troubleshooting section.
## Setting up Header / Forward Authentication
Header authentication trusts a reverse proxy (Authentik, Authelia, oauth2-proxy, Traefik forward-auth, etc.) to authenticate users upstream and pass identity in HTTP headers. When enabled, Gitea Mirror reads the configured headers on each request and resolves the user automatically — no login form, no callback.
> **Important — operator-controlled by design.** Header auth is configured via environment variables only, not the UI. This is intentional: trusting a header means trusting whatever upstream sets it, and that decision belongs to the operator who controls the reverse proxy, not to a logged-in user inside the app. Make sure your reverse proxy strips these headers from inbound client requests, otherwise anyone can spoof them.
### Environment variables
| Variable | Description | Default |
|----------|-------------|---------|
| `HEADER_AUTH_ENABLED` | Master switch — must be `true` to enable | `false` |
| `HEADER_AUTH_USER_HEADER` | Header containing the username | `X-Authentik-Username` |
| `HEADER_AUTH_EMAIL_HEADER` | Header containing the email address | `X-Authentik-Email` |
| `HEADER_AUTH_NAME_HEADER` | Header containing the display name | `X-Authentik-Name` |
| `HEADER_AUTH_AUTO_PROVISION` | If `true`, create a new user when an unknown username arrives. If `false`, unknown users are rejected. | `false` |
| `HEADER_AUTH_ALLOWED_DOMAINS` | Comma-separated email domain allowlist. Empty = allow any. | _(empty)_ |
See also: [`docs/ENVIRONMENT_VARIABLES.md`](./ENVIRONMENT_VARIABLES.md#header-authentication-reverse-proxy-sso).
### Example: Docker Compose with Authentik
```yaml
services:
gitea-mirror:
image: ghcr.io/raylabshq/gitea-mirror:latest
environment:
HEADER_AUTH_ENABLED: "true"
HEADER_AUTH_USER_HEADER: "X-Authentik-Username"
HEADER_AUTH_EMAIL_HEADER: "X-Authentik-Email"
HEADER_AUTH_NAME_HEADER: "X-Authentik-Name"
HEADER_AUTH_AUTO_PROVISION: "true"
HEADER_AUTH_ALLOWED_DOMAINS: "example.com,example.org"
```
The defaults are Authentik-shaped, so for an Authentik proxy provider you generally only need to set `HEADER_AUTH_ENABLED=true` (and `HEADER_AUTH_AUTO_PROVISION=true` if you want first-login self-registration).
### Example: Authelia
Authelia uses different header names — override them:
```yaml
environment:
HEADER_AUTH_ENABLED: "true"
HEADER_AUTH_USER_HEADER: "Remote-User"
HEADER_AUTH_EMAIL_HEADER: "Remote-Email"
HEADER_AUTH_NAME_HEADER: "Remote-Name"
HEADER_AUTH_AUTO_PROVISION: "true"
```
Then configure Authelia's `authz` rules to protect the gitea-mirror route and inject the `Remote-*` headers.
### Behaviour and lookup order
- The middleware checks for a cookie session **first**. Header auth only fires when there is no existing session, so users who logged in via password or SSO are not affected.
- Lookup is by `HEADER_AUTH_USER_HEADER` value matched against `users.username`; if no match and `HEADER_AUTH_EMAIL_HEADER` is set, a second lookup is tried against `users.email`.
- If neither matches:
- With `HEADER_AUTH_AUTO_PROVISION=true`, a new user row is created (email defaults to `<username>@header-auth.local` if no email header is present).
- With `HEADER_AUTH_AUTO_PROVISION=false`, the request is rejected with a warning logged.
- If `HEADER_AUTH_ALLOWED_DOMAINS` is non-empty and the email header arrives, the email's domain must be in the list or the request is rejected.
### Verifying it's enabled
When header auth is active, the Authentication settings page renders a green "Header Authentication / Via reverse proxy" badge under the auth-methods status block. You can also probe `/api/auth/header-status`, which returns `{ "enabled": true, ... }` once the env vars are set.
### Security checklist
1. **Strip the headers at your edge.** Your reverse proxy must remove inbound `HEADER_AUTH_USER_HEADER` / email / name headers from client requests before re-injecting its own. Otherwise any unauthenticated client can set `X-Authentik-Username: admin` and walk in.
2. **Bind the app to the proxy only.** Don't expose Gitea Mirror's port directly to the network when header auth is on; only the trusted proxy should reach it.
3. **Use HTTPS between proxy and clients.** Headers travel in plaintext on the link they cross.
4. **Be conservative with `HEADER_AUTH_AUTO_PROVISION`.** With it off, you provision users in the UI/DB once and the proxy fills in sessions thereafter — safer for shared deployments.
## Setting up OIDC Provider
The OIDC Provider feature allows other applications to use Gitea Mirror as their authentication provider.
### Creating OAuth Applications
1. Navigate to Configuration → Authentication → OAuth Applications
2. Click "Create Application"
3. Fill in the application details:
- **Application Name**: Display name for the application
- **Application Type**: Web, Mobile, or Desktop
- **Redirect URLs**: One or more redirect URLs (one per line)
4. After creation, you'll receive:
- **Client ID**: Share this with the application
- **Client Secret**: Keep this secure and share only once
### OIDC Endpoints
Applications can use these standard OIDC endpoints:
- **Discovery**: `https://your-domain.com/.well-known/openid-configuration`
- **Authorization**: `https://your-domain.com/api/auth/oauth2/authorize`
- **Token**: `https://your-domain.com/api/auth/oauth2/token`
- **UserInfo**: `https://your-domain.com/api/auth/oauth2/userinfo`
- **JWKS**: `https://your-domain.com/api/auth/jwks`
### Supported Scopes
- `openid` - Required, provides user ID
- `profile` - User's name, username, and profile picture
- `email` - User's email address and verification status
### Example: Configuring Another Application
For an application to use Gitea Mirror as its OIDC provider:
```javascript
// Example configuration for another app
const oidcConfig = {
issuer: 'https://gitea-mirror.example.com',
clientId: 'client_xxxxxxxxxxxxx',
clientSecret: 'secret_xxxxxxxxxxxxx',
redirectUri: 'https://myapp.com/auth/callback',
scope: 'openid profile email'
};
```
## User Experience
### Logging In with SSO
When SSO is configured:
1. Users see tabs for "Email" and "SSO" on the login page
2. In the SSO tab, they can:
- Click a specific provider button (if configured)
- Enter their work email to be redirected to the appropriate provider
### OAuth Consent Flow
When an application requests authentication:
1. Users are redirected to Gitea Mirror
2. If not logged in, they authenticate first
3. They see a consent screen showing:
- Application name
- Requested permissions
- Option to approve or deny
## Security Considerations
1. **Client Secrets**: Store OAuth client secrets securely
2. **Redirect URLs**: Only add trusted redirect URLs for applications
3. **Scopes**: Applications only receive the data for approved scopes
4. **Token Security**: Access tokens expire and can be revoked
## Troubleshooting
### SSO Login Issues
1. **"Invalid origin" error**: Check that your Gitea Mirror URL matches the configured redirect URI
2. **"Provider not found" error**: Ensure the provider is properly configured and enabled
3. **Redirect loop**: Verify the redirect URI in both Gitea Mirror and the SSO provider match exactly
4. **`?error=UNKNOWN` on the homepage after a successful upstream login** (or `?error=account%20not%20linked` in development): the SSO callback succeeded but Better Auth refused to link the SSO identity to an existing local account. The most common cause is the SSO provider's registered **Domain** not matching the user's actual email domain — see [Account Linking](#account-linking). Set `BETTER_AUTH_LOG_LEVEL=debug` to see Better Auth's full callback trace (look for "User already exist but account isn't linked to ...") and confirm the diagnosis. The same symptom in production gets sanitized to `UNKNOWN` by Better Auth's error page before the redirect, which is why the visible error is opaque.
5. **`TypeError: undefined is not an object (evaluating 'url.startsWith')`**: This indicates the stored provider configuration is missing OIDC endpoints. Delete the provider from Gitea Mirror and re-register it using the **Discover** button so authorization/token URLs are saved (see [#73](https://github.com/RayLabsHQ/gitea-mirror/issues/73) and [#122](https://github.com/RayLabsHQ/gitea-mirror/issues/122) for examples).
### OIDC Provider Issues
1. **Application not found**: Ensure the client ID is correct
2. **Invalid redirect URI**: The redirect URI must match exactly what's configured
3. **Consent not working**: Check browser cookies are enabled
## Managing Access
### Revoking SSO Access
Currently, SSO sessions are managed through the identity provider. To revoke access:
1. Log out of Gitea Mirror
2. Revoke access in your identity provider's settings
### Disabling OAuth Applications
To disable an application:
1. Go to Configuration → Authentication → OAuth Applications
2. Find the application
3. Click the delete button
This immediately prevents the application from authenticating new users.
## Best Practices
1. **Use HTTPS**: Always use HTTPS in production for security
2. **Regular Audits**: Periodically review configured SSO providers and OAuth applications
3. **Principle of Least Privilege**: Only grant necessary scopes to applications
4. **Monitor Usage**: Keep track of which applications are accessing your OIDC provider
5. **Secure Storage**: Store client secrets in a secure location, never in code
## Migration Notes
If migrating from the previous JWT-based authentication:
- Existing users remain unaffected
- Users can continue using email/password authentication
- SSO can be added as an additional authentication method
+241
View File
@@ -0,0 +1,241 @@
# Local SSO Testing Guide
This guide explains how to test SSO authentication locally with Gitea Mirror.
## Option 1: Using Google OAuth (Recommended for Quick Testing)
### Setup Steps:
1. **Create a Google OAuth Application**
- Go to [Google Cloud Console](https://console.cloud.google.com/)
- Create a new project or select existing
- Enable Google+ API
- Go to "Credentials" → "Create Credentials" → "OAuth client ID"
- Choose "Web application"
- Add authorized redirect URIs:
- `http://localhost:3000/api/auth/sso/callback/google-sso`
- `http://localhost:9876/api/auth/sso/callback/google-sso`
2. **Configure in Gitea Mirror**
- Go to Configuration → Authentication tab
- Click "Add Provider"
- Select "OIDC / OAuth2"
- Fill in:
- Provider ID: `google-sso`
- Email Domain: `gmail.com` (or your domain)
- Issuer URL: `https://accounts.google.com`
- Click "Discover" to auto-fill endpoints
- Client ID: (from Google Console)
- Client Secret: (from Google Console)
- Save the provider
## Option 2: Using Keycloak (Local Identity Provider)
### Setup with Docker:
```bash
# Run Keycloak
docker run -d --name keycloak \
-p 8080:8080 \
-e KEYCLOAK_ADMIN=admin \
-e KEYCLOAK_ADMIN_PASSWORD=admin \
quay.io/keycloak/keycloak:latest start-dev
# Access at http://localhost:8080
# Login with admin/admin
```
### Configure Keycloak:
1. Create a new realm (e.g., "gitea-mirror")
2. Create a client:
- Client ID: `gitea-mirror`
- Client Protocol: `openid-connect`
- Access Type: `confidential`
- Valid Redirect URIs: `http://localhost:*/api/auth/sso/callback/keycloak`
3. Get credentials from the "Credentials" tab
4. Create test users in "Users" section
### Configure in Gitea Mirror:
- Provider ID: `keycloak`
- Email Domain: `example.com`
- Issuer URL: `http://localhost:8080/realms/gitea-mirror`
- Client ID: `gitea-mirror`
- Client Secret: (from Keycloak)
- Click "Discover" to auto-fill endpoints
## Option 3: Using Mock SSO Provider (Development)
For testing without external dependencies, you can use a mock OIDC provider.
### Using oidc-provider-example:
```bash
# Clone and run mock provider
git clone https://github.com/panva/node-oidc-provider-example.git
cd node-oidc-provider-example
npm install
npm start
# Runs on http://localhost:3001
```
### Configure in Gitea Mirror:
- Provider ID: `mock-provider`
- Email Domain: `test.com`
- Issuer URL: `http://localhost:3001`
- Client ID: `foo`
- Client Secret: `bar`
- Authorization Endpoint: `http://localhost:3001/auth`
- Token Endpoint: `http://localhost:3001/token`
## Testing the SSO Flow
1. **Logout** from Gitea Mirror if logged in
2. Go to `/login`
3. Click on the **SSO** tab
4. Either:
- Click the provider button (e.g., "Sign in with gmail.com")
- Or enter your email and click "Continue with SSO"
5. You'll be redirected to the identity provider
6. Complete authentication
7. You'll be redirected back and logged in
## Troubleshooting
### Common Issues:
1. **"Invalid origin" error**
- Check that `trustedOrigins` in `/src/lib/auth.ts` includes your dev URL
- Restart the dev server after changes
2. **Provider not showing in login**
- Check browser console for errors
- Verify provider was saved successfully
- Check `/api/sso/providers` returns your providers
3. **Redirect URI mismatch**
- Ensure the redirect URI in your OAuth app matches exactly:
`http://localhost:PORT/api/auth/sso/callback/PROVIDER_ID`
4. **CORS errors**
- Add your identity provider domain to CORS allowed origins if needed
### Debug Mode:
> **Note:** Better Auth uses its own logger and does **not** read the `DEBUG`
> environment variable (it is not based on the `debug` npm package). An older
> version of this guide suggested `DEBUG=better-auth:*` — that has no effect.
Better Auth's logger defaults to the `warn` level, so SSO/OIDC sign-in and
callback details are hidden. Set the log level to `debug` to surface the full
trace:
```bash
# Local dev
BETTER_AUTH_LOG_LEVEL=debug bun run dev
```
```yaml
# Docker Compose
services:
gitea-mirror:
environment:
- BETTER_AUTH_LOG_LEVEL=debug
```
Then watch the server logs (e.g. `docker compose logs -f gitea-mirror`) while you
attempt an SSO login. Lines are prefixed with `[Better Auth]:`. Accepted values
are `debug`, `info`, `warn`, and `error`.
#### Debugging a login that bounces back to `/login`
If clicking the SSO button sends you to the provider and then straight back to
the login screen, the OAuth flow itself usually succeeded but **no session
cookie was persisted**. Work through these checks:
1. **Enable `BETTER_AUTH_LOG_LEVEL=debug`** (above) and look for errors during
the `/api/auth/sso/callback/<provider-id>` request.
2. **Check for `?error=UNKNOWN` on the landing URL** (or `?error=account%20not%20linked` in dev).
That's Better Auth's account-linking step refusing to attach the SSO identity
to an existing email/password account. The debug log line to look for is
`User already exist but account isn't linked to <providerId>`. The fix is
almost always to set the SSO provider's **Domain** field to the email domain
your users actually have — auto-linking is gated on that domain match.
See [docs/SSO-OIDC-SETUP.md#account-linking](./SSO-OIDC-SETUP.md#account-linking).
3. **Check the redirect URI** registered in your IdP exactly matches
`https://<your-domain>/api/auth/sso/callback/<provider-id>` (scheme, host,
and provider ID — no trailing slash).
4. **Confirm the session cookie is set.** In the browser DevTools → Network,
inspect the callback response for a `Set-Cookie: better-auth-session=…`
header, and DevTools → Application → Cookies for the stored cookie. Behind a
reverse proxy, ensure `BETTER_AUTH_URL` is your **external HTTPS** URL so the
cookie is issued with the correct domain and `Secure` flag, and that the
proxy forwards `X-Forwarded-Proto: https` and `X-Forwarded-Host`.
5. **A `401` on `/api/sso/applications` is unrelated** to client login — that
endpoint backs the OAuth *provider* (consent) management UI and requires an
existing session. It is not part of the Authentik/OIDC sign-in flow.
## Testing Different Scenarios
### 1. New User Registration
- Use an email not in the system
- SSO should create a new user automatically
### 2. Existing User Login
- Create a user with email/password first
- Login with SSO using same email
- Should link to existing account
### 3. Domain-based Routing
- Configure multiple providers with different domains
- Test that entering email routes to correct provider
### 4. Organization Provisioning
- Set organizationId on provider
- Test that users are added to correct organization
## Security Testing
1. **Token Expiration**
- Wait for session to expire
- Test refresh flow
2. **Invalid State**
- Modify state parameter in callback
- Should reject authentication
3. **PKCE Flow**
- Enable/disable PKCE
- Verify code challenge works
## Using with Better Auth CLI
Better Auth provides CLI tools for testing:
```bash
# List registered providers
bun run auth:providers list
# Test provider configuration
bun run auth:providers test google-sso
```
## Environment Variables
For production-like testing:
```env
BETTER_AUTH_URL=http://localhost:3000
BETTER_AUTH_SECRET=your-secret-key
```
## Next Steps
After successful SSO setup:
1. Test user attribute mapping
2. Configure role-based access
3. Set up SAML if needed
4. Test with your organization's actual IdP
Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 144 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 60 KiB

-127
View File
@@ -1,127 +0,0 @@
# Testing in Gitea Mirror
This document provides guidance on testing in the Gitea Mirror project.
## Current Status
The project now uses Bun's built-in test runner, which is Jest-compatible and provides a fast, reliable testing experience. We've migrated away from Vitest due to compatibility issues with Bun.
## Running Tests
To run tests, use the following commands:
```bash
# Run all tests
bun test
# Run tests in watch mode (automatically re-run when files change)
bun test --watch
# Run tests with coverage reporting
bun test --coverage
```
## Test File Naming Conventions
Bun's test runner automatically discovers test files that match the following patterns:
- `*.test.{js|jsx|ts|tsx}`
- `*_test.{js|jsx|ts|tsx}`
- `*.spec.{js|jsx|ts|tsx}`
- `*_spec.{js|jsx|ts|tsx}`
## Writing Tests
The project uses Bun's test runner with a Jest-compatible API. Here's an example test:
```typescript
// example.test.ts
import { describe, test, expect } from "bun:test";
describe("Example Test", () => {
test("should pass", () => {
expect(true).toBe(true);
});
});
```
### Testing React Components
For testing React components, we use React Testing Library:
```typescript
// component.test.tsx
import { describe, test, expect } from "bun:test";
import { render, screen } from "@testing-library/react";
import MyComponent from "../components/MyComponent";
describe("MyComponent", () => {
test("renders correctly", () => {
render(<MyComponent />);
expect(screen.getByText("Hello World")).toBeInTheDocument();
});
});
```
## Test Setup
The test setup is defined in `src/tests/setup.bun.ts` and includes:
- Automatic cleanup after each test
- Setup for any global test environment needs
## Mocking
Bun's test runner provides built-in mocking capabilities:
```typescript
import { test, expect, mock } from "bun:test";
// Create a mock function
const mockFn = mock(() => "mocked value");
test("mock function", () => {
const result = mockFn();
expect(result).toBe("mocked value");
expect(mockFn).toHaveBeenCalled();
});
// Mock a module
mock.module("./some-module", () => {
return {
someFunction: () => "mocked module function"
};
});
```
## CI Integration
The CI workflow has been updated to use Bun's test runner. Tests are automatically run as part of the CI pipeline.
## Test Coverage
To generate test coverage reports, run:
```bash
bun test --coverage
```
This will generate a coverage report in the `coverage` directory.
## Types of Tests
The project includes several types of tests:
1. **Unit Tests**: Testing individual functions and utilities
2. **API Tests**: Testing API endpoints
3. **Component Tests**: Testing React components
4. **Integration Tests**: Testing how components work together
## Future Improvements
When expanding the test suite, consider:
1. Adding more comprehensive API endpoint tests
2. Increasing component test coverage
3. Setting up end-to-end tests with a tool like Playwright
4. Adding performance tests for critical paths
+16
View File
@@ -0,0 +1,16 @@
import { defineConfig } from "drizzle-kit";
export default defineConfig({
dialect: "sqlite",
schema: "./src/lib/db/schema.ts",
out: "./drizzle",
dbCredentials: {
url: "./data/gitea-mirror.db",
},
verbose: true,
strict: true,
migrations: {
table: "__drizzle_migrations",
schema: "main",
},
});
+180
View File
@@ -0,0 +1,180 @@
CREATE TABLE `accounts` (
`id` text PRIMARY KEY NOT NULL,
`account_id` text NOT NULL,
`user_id` text NOT NULL,
`provider_id` text NOT NULL,
`provider_user_id` text,
`access_token` text,
`refresh_token` text,
`expires_at` integer,
`password` text,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE INDEX `idx_accounts_account_id` ON `accounts` (`account_id`);--> statement-breakpoint
CREATE INDEX `idx_accounts_user_id` ON `accounts` (`user_id`);--> statement-breakpoint
CREATE INDEX `idx_accounts_provider` ON `accounts` (`provider_id`,`provider_user_id`);--> statement-breakpoint
CREATE TABLE `configs` (
`id` text PRIMARY KEY NOT NULL,
`user_id` text NOT NULL,
`name` text NOT NULL,
`is_active` integer DEFAULT true NOT NULL,
`github_config` text NOT NULL,
`gitea_config` text NOT NULL,
`include` text DEFAULT '["*"]' NOT NULL,
`exclude` text DEFAULT '[]' NOT NULL,
`schedule_config` text NOT NULL,
`cleanup_config` text NOT NULL,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE TABLE `events` (
`id` text PRIMARY KEY NOT NULL,
`user_id` text NOT NULL,
`channel` text NOT NULL,
`payload` text NOT NULL,
`read` integer DEFAULT false NOT NULL,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE INDEX `idx_events_user_channel` ON `events` (`user_id`,`channel`);--> statement-breakpoint
CREATE INDEX `idx_events_created_at` ON `events` (`created_at`);--> statement-breakpoint
CREATE INDEX `idx_events_read` ON `events` (`read`);--> statement-breakpoint
CREATE TABLE `mirror_jobs` (
`id` text PRIMARY KEY NOT NULL,
`user_id` text NOT NULL,
`repository_id` text,
`repository_name` text,
`organization_id` text,
`organization_name` text,
`details` text,
`status` text DEFAULT 'imported' NOT NULL,
`message` text NOT NULL,
`timestamp` integer DEFAULT (unixepoch()) NOT NULL,
`job_type` text DEFAULT 'mirror' NOT NULL,
`batch_id` text,
`total_items` integer,
`completed_items` integer DEFAULT 0,
`item_ids` text,
`completed_item_ids` text DEFAULT '[]',
`in_progress` integer DEFAULT false NOT NULL,
`started_at` integer,
`completed_at` integer,
`last_checkpoint` integer,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE INDEX `idx_mirror_jobs_user_id` ON `mirror_jobs` (`user_id`);--> statement-breakpoint
CREATE INDEX `idx_mirror_jobs_batch_id` ON `mirror_jobs` (`batch_id`);--> statement-breakpoint
CREATE INDEX `idx_mirror_jobs_in_progress` ON `mirror_jobs` (`in_progress`);--> statement-breakpoint
CREATE INDEX `idx_mirror_jobs_job_type` ON `mirror_jobs` (`job_type`);--> statement-breakpoint
CREATE INDEX `idx_mirror_jobs_timestamp` ON `mirror_jobs` (`timestamp`);--> statement-breakpoint
CREATE TABLE `organizations` (
`id` text PRIMARY KEY NOT NULL,
`user_id` text NOT NULL,
`config_id` text NOT NULL,
`name` text NOT NULL,
`avatar_url` text NOT NULL,
`membership_role` text DEFAULT 'member' NOT NULL,
`is_included` integer DEFAULT true NOT NULL,
`destination_org` text,
`status` text DEFAULT 'imported' NOT NULL,
`last_mirrored` integer,
`error_message` text,
`repository_count` integer DEFAULT 0 NOT NULL,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action,
FOREIGN KEY (`config_id`) REFERENCES `configs`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE INDEX `idx_organizations_user_id` ON `organizations` (`user_id`);--> statement-breakpoint
CREATE INDEX `idx_organizations_config_id` ON `organizations` (`config_id`);--> statement-breakpoint
CREATE INDEX `idx_organizations_status` ON `organizations` (`status`);--> statement-breakpoint
CREATE INDEX `idx_organizations_is_included` ON `organizations` (`is_included`);--> statement-breakpoint
CREATE TABLE `repositories` (
`id` text PRIMARY KEY NOT NULL,
`user_id` text NOT NULL,
`config_id` text NOT NULL,
`name` text NOT NULL,
`full_name` text NOT NULL,
`url` text NOT NULL,
`clone_url` text NOT NULL,
`owner` text NOT NULL,
`organization` text,
`mirrored_location` text DEFAULT '',
`is_private` integer DEFAULT false NOT NULL,
`is_fork` integer DEFAULT false NOT NULL,
`forked_from` text,
`has_issues` integer DEFAULT false NOT NULL,
`is_starred` integer DEFAULT false NOT NULL,
`is_archived` integer DEFAULT false NOT NULL,
`size` integer DEFAULT 0 NOT NULL,
`has_lfs` integer DEFAULT false NOT NULL,
`has_submodules` integer DEFAULT false NOT NULL,
`language` text,
`description` text,
`default_branch` text NOT NULL,
`visibility` text DEFAULT 'public' NOT NULL,
`status` text DEFAULT 'imported' NOT NULL,
`last_mirrored` integer,
`error_message` text,
`destination_org` text,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action,
FOREIGN KEY (`config_id`) REFERENCES `configs`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE INDEX `idx_repositories_user_id` ON `repositories` (`user_id`);--> statement-breakpoint
CREATE INDEX `idx_repositories_config_id` ON `repositories` (`config_id`);--> statement-breakpoint
CREATE INDEX `idx_repositories_status` ON `repositories` (`status`);--> statement-breakpoint
CREATE INDEX `idx_repositories_owner` ON `repositories` (`owner`);--> statement-breakpoint
CREATE INDEX `idx_repositories_organization` ON `repositories` (`organization`);--> statement-breakpoint
CREATE INDEX `idx_repositories_is_fork` ON `repositories` (`is_fork`);--> statement-breakpoint
CREATE INDEX `idx_repositories_is_starred` ON `repositories` (`is_starred`);--> statement-breakpoint
CREATE TABLE `sessions` (
`id` text PRIMARY KEY NOT NULL,
`token` text NOT NULL,
`user_id` text NOT NULL,
`expires_at` integer NOT NULL,
`ip_address` text,
`user_agent` text,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE UNIQUE INDEX `sessions_token_unique` ON `sessions` (`token`);--> statement-breakpoint
CREATE INDEX `idx_sessions_user_id` ON `sessions` (`user_id`);--> statement-breakpoint
CREATE INDEX `idx_sessions_token` ON `sessions` (`token`);--> statement-breakpoint
CREATE INDEX `idx_sessions_expires_at` ON `sessions` (`expires_at`);--> statement-breakpoint
CREATE TABLE `users` (
`id` text PRIMARY KEY NOT NULL,
`name` text,
`email` text NOT NULL,
`email_verified` integer DEFAULT false NOT NULL,
`image` text,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
`username` text
);
--> statement-breakpoint
CREATE UNIQUE INDEX `users_email_unique` ON `users` (`email`);--> statement-breakpoint
CREATE TABLE `verification_tokens` (
`id` text PRIMARY KEY NOT NULL,
`token` text NOT NULL,
`identifier` text NOT NULL,
`type` text NOT NULL,
`expires_at` integer NOT NULL,
`created_at` integer DEFAULT (unixepoch()) NOT NULL
);
--> statement-breakpoint
CREATE UNIQUE INDEX `verification_tokens_token_unique` ON `verification_tokens` (`token`);--> statement-breakpoint
CREATE INDEX `idx_verification_tokens_token` ON `verification_tokens` (`token`);--> statement-breakpoint
CREATE INDEX `idx_verification_tokens_identifier` ON `verification_tokens` (`identifier`);
+64
View File
@@ -0,0 +1,64 @@
CREATE TABLE `oauth_access_tokens` (
`id` text PRIMARY KEY NOT NULL,
`access_token` text NOT NULL,
`refresh_token` text,
`access_token_expires_at` integer NOT NULL,
`refresh_token_expires_at` integer,
`client_id` text NOT NULL,
`user_id` text NOT NULL,
`scopes` text NOT NULL,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE INDEX `idx_oauth_access_tokens_access_token` ON `oauth_access_tokens` (`access_token`);--> statement-breakpoint
CREATE INDEX `idx_oauth_access_tokens_user_id` ON `oauth_access_tokens` (`user_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_access_tokens_client_id` ON `oauth_access_tokens` (`client_id`);--> statement-breakpoint
CREATE TABLE `oauth_applications` (
`id` text PRIMARY KEY NOT NULL,
`client_id` text NOT NULL,
`client_secret` text NOT NULL,
`name` text NOT NULL,
`redirect_urls` text NOT NULL,
`metadata` text,
`type` text NOT NULL,
`disabled` integer DEFAULT false NOT NULL,
`user_id` text,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL
);
--> statement-breakpoint
CREATE UNIQUE INDEX `oauth_applications_client_id_unique` ON `oauth_applications` (`client_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_applications_client_id` ON `oauth_applications` (`client_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_applications_user_id` ON `oauth_applications` (`user_id`);--> statement-breakpoint
CREATE TABLE `oauth_consent` (
`id` text PRIMARY KEY NOT NULL,
`user_id` text NOT NULL,
`client_id` text NOT NULL,
`scopes` text NOT NULL,
`consent_given` integer NOT NULL,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE INDEX `idx_oauth_consent_user_id` ON `oauth_consent` (`user_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_consent_client_id` ON `oauth_consent` (`client_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_consent_user_client` ON `oauth_consent` (`user_id`,`client_id`);--> statement-breakpoint
CREATE TABLE `sso_providers` (
`id` text PRIMARY KEY NOT NULL,
`issuer` text NOT NULL,
`domain` text NOT NULL,
`oidc_config` text NOT NULL,
`user_id` text NOT NULL,
`provider_id` text NOT NULL,
`organization_id` text,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL
);
--> statement-breakpoint
CREATE UNIQUE INDEX `sso_providers_provider_id_unique` ON `sso_providers` (`provider_id`);--> statement-breakpoint
CREATE INDEX `idx_sso_providers_provider_id` ON `sso_providers` (`provider_id`);--> statement-breakpoint
CREATE INDEX `idx_sso_providers_domain` ON `sso_providers` (`domain`);--> statement-breakpoint
CREATE INDEX `idx_sso_providers_issuer` ON `sso_providers` (`issuer`);
+10
View File
@@ -0,0 +1,10 @@
CREATE TABLE `verifications` (
`id` text PRIMARY KEY NOT NULL,
`identifier` text NOT NULL,
`value` text NOT NULL,
`expires_at` integer NOT NULL,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL
);
--> statement-breakpoint
CREATE INDEX `idx_verifications_identifier` ON `verifications` (`identifier`);
+3
View File
@@ -0,0 +1,3 @@
ALTER TABLE `organizations` ADD `public_repository_count` integer;--> statement-breakpoint
ALTER TABLE `organizations` ADD `private_repository_count` integer;--> statement-breakpoint
ALTER TABLE `organizations` ADD `fork_repository_count` integer;
+18
View File
@@ -0,0 +1,18 @@
CREATE TABLE `rate_limits` (
`id` text PRIMARY KEY NOT NULL,
`user_id` text NOT NULL,
`provider` text DEFAULT 'github' NOT NULL,
`limit` integer NOT NULL,
`remaining` integer NOT NULL,
`used` integer NOT NULL,
`reset` integer NOT NULL,
`retry_after` integer,
`status` text DEFAULT 'ok' NOT NULL,
`last_checked` integer NOT NULL,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE INDEX `idx_rate_limits_user_provider` ON `rate_limits` (`user_id`,`provider`);--> statement-breakpoint
CREATE INDEX `idx_rate_limits_status` ON `rate_limits` (`status`);
+11
View File
@@ -0,0 +1,11 @@
-- Step 1: Remove duplicate repositories, keeping the most recently updated one
-- This handles cases where users have duplicate entries from before the unique constraint
DELETE FROM repositories
WHERE rowid NOT IN (
SELECT MAX(rowid)
FROM repositories
GROUP BY user_id, full_name
);
--> statement-breakpoint
-- Step 2: Now create the unique index safely
CREATE UNIQUE INDEX uniq_repositories_user_full_name ON repositories (user_id, full_name);
+4
View File
@@ -0,0 +1,4 @@
ALTER TABLE `accounts` ADD `id_token` text;--> statement-breakpoint
ALTER TABLE `accounts` ADD `access_token_expires_at` integer;--> statement-breakpoint
ALTER TABLE `accounts` ADD `refresh_token_expires_at` integer;--> statement-breakpoint
ALTER TABLE `accounts` ADD `scope` text;
+18
View File
@@ -0,0 +1,18 @@
ALTER TABLE `organizations` ADD `normalized_name` text NOT NULL DEFAULT '';--> statement-breakpoint
UPDATE `organizations` SET `normalized_name` = lower(trim(`name`));--> statement-breakpoint
DELETE FROM `organizations`
WHERE rowid NOT IN (
SELECT MIN(rowid)
FROM `organizations`
GROUP BY `user_id`, `normalized_name`
);--> statement-breakpoint
CREATE UNIQUE INDEX `uniq_organizations_user_normalized_name` ON `organizations` (`user_id`,`normalized_name`);--> statement-breakpoint
ALTER TABLE `repositories` ADD `normalized_full_name` text NOT NULL DEFAULT '';--> statement-breakpoint
UPDATE `repositories` SET `normalized_full_name` = lower(trim(`full_name`));--> statement-breakpoint
DELETE FROM `repositories`
WHERE rowid NOT IN (
SELECT MIN(rowid)
FROM `repositories`
GROUP BY `user_id`, `normalized_full_name`
);--> statement-breakpoint
CREATE UNIQUE INDEX `uniq_repositories_user_normalized_full_name` ON `repositories` (`user_id`,`normalized_full_name`);
+1
View File
@@ -0,0 +1 @@
ALTER TABLE `repositories` ADD `metadata` text;
+149
View File
@@ -0,0 +1,149 @@
CREATE TABLE `__new_repositories` (
`id` text PRIMARY KEY NOT NULL,
`user_id` text NOT NULL,
`config_id` text NOT NULL,
`name` text NOT NULL,
`full_name` text NOT NULL,
`normalized_full_name` text NOT NULL,
`url` text NOT NULL,
`clone_url` text NOT NULL,
`owner` text NOT NULL,
`organization` text,
`mirrored_location` text DEFAULT '',
`is_private` integer DEFAULT false NOT NULL,
`is_fork` integer DEFAULT false NOT NULL,
`forked_from` text,
`has_issues` integer DEFAULT false NOT NULL,
`is_starred` integer DEFAULT false NOT NULL,
`is_archived` integer DEFAULT false NOT NULL,
`size` integer DEFAULT 0 NOT NULL,
`has_lfs` integer DEFAULT false NOT NULL,
`has_submodules` integer DEFAULT false NOT NULL,
`language` text,
`description` text,
`default_branch` text NOT NULL,
`visibility` text DEFAULT 'public' NOT NULL,
`status` text DEFAULT 'imported' NOT NULL,
`last_mirrored` integer,
`error_message` text,
`destination_org` text,
`metadata` text,
`imported_at` integer DEFAULT (unixepoch()) NOT NULL,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action,
FOREIGN KEY (`config_id`) REFERENCES `configs`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
INSERT INTO `__new_repositories` (
`id`,
`user_id`,
`config_id`,
`name`,
`full_name`,
`normalized_full_name`,
`url`,
`clone_url`,
`owner`,
`organization`,
`mirrored_location`,
`is_private`,
`is_fork`,
`forked_from`,
`has_issues`,
`is_starred`,
`is_archived`,
`size`,
`has_lfs`,
`has_submodules`,
`language`,
`description`,
`default_branch`,
`visibility`,
`status`,
`last_mirrored`,
`error_message`,
`destination_org`,
`metadata`,
`imported_at`,
`created_at`,
`updated_at`
)
SELECT
`repositories`.`id`,
`repositories`.`user_id`,
`repositories`.`config_id`,
`repositories`.`name`,
`repositories`.`full_name`,
`repositories`.`normalized_full_name`,
`repositories`.`url`,
`repositories`.`clone_url`,
`repositories`.`owner`,
`repositories`.`organization`,
`repositories`.`mirrored_location`,
`repositories`.`is_private`,
`repositories`.`is_fork`,
`repositories`.`forked_from`,
`repositories`.`has_issues`,
`repositories`.`is_starred`,
`repositories`.`is_archived`,
`repositories`.`size`,
`repositories`.`has_lfs`,
`repositories`.`has_submodules`,
`repositories`.`language`,
`repositories`.`description`,
`repositories`.`default_branch`,
`repositories`.`visibility`,
`repositories`.`status`,
`repositories`.`last_mirrored`,
`repositories`.`error_message`,
`repositories`.`destination_org`,
`repositories`.`metadata`,
COALESCE(
(
SELECT MIN(`mj`.`timestamp`)
FROM `mirror_jobs` `mj`
WHERE `mj`.`user_id` = `repositories`.`user_id`
AND `mj`.`status` = 'imported'
AND (
(`mj`.`repository_id` IS NOT NULL AND `mj`.`repository_id` = `repositories`.`id`)
OR (
`mj`.`repository_id` IS NULL
AND `mj`.`repository_name` IS NOT NULL
AND (
lower(trim(`mj`.`repository_name`)) = `repositories`.`normalized_full_name`
OR lower(trim(`mj`.`repository_name`)) = lower(trim(`repositories`.`name`))
)
)
)
),
`repositories`.`created_at`,
unixepoch()
) AS `imported_at`,
`repositories`.`created_at`,
`repositories`.`updated_at`
FROM `repositories`;
--> statement-breakpoint
DROP TABLE `repositories`;
--> statement-breakpoint
ALTER TABLE `__new_repositories` RENAME TO `repositories`;
--> statement-breakpoint
CREATE INDEX `idx_repositories_user_id` ON `repositories` (`user_id`);
--> statement-breakpoint
CREATE INDEX `idx_repositories_config_id` ON `repositories` (`config_id`);
--> statement-breakpoint
CREATE INDEX `idx_repositories_status` ON `repositories` (`status`);
--> statement-breakpoint
CREATE INDEX `idx_repositories_owner` ON `repositories` (`owner`);
--> statement-breakpoint
CREATE INDEX `idx_repositories_organization` ON `repositories` (`organization`);
--> statement-breakpoint
CREATE INDEX `idx_repositories_is_fork` ON `repositories` (`is_fork`);
--> statement-breakpoint
CREATE INDEX `idx_repositories_is_starred` ON `repositories` (`is_starred`);
--> statement-breakpoint
CREATE INDEX `idx_repositories_user_imported_at` ON `repositories` (`user_id`,`imported_at`);
--> statement-breakpoint
CREATE UNIQUE INDEX `uniq_repositories_user_full_name` ON `repositories` (`user_id`,`full_name`);
--> statement-breakpoint
CREATE UNIQUE INDEX `uniq_repositories_user_normalized_full_name` ON `repositories` (`user_id`,`normalized_full_name`);
+9
View File
@@ -0,0 +1,9 @@
-- Add index for mirroredLocation lookups (used by name collision detection)
CREATE INDEX IF NOT EXISTS `idx_repositories_mirrored_location` ON `repositories` (`user_id`, `mirrored_location`);
-- Add unique partial index to enforce that no two repos for the same user
-- can claim the same non-empty mirroredLocation. This prevents race conditions
-- during concurrent batch mirroring of starred repos with duplicate names.
CREATE UNIQUE INDEX IF NOT EXISTS `uniq_repositories_user_mirrored_location`
ON `repositories` (`user_id`, `mirrored_location`)
WHERE `mirrored_location` != '';
+1
View File
@@ -0,0 +1 @@
ALTER TABLE `configs` ADD `notification_config` text DEFAULT '{"enabled":false,"provider":"ntfy","notifyOnSyncError":true,"notifyOnSyncSuccess":false,"notifyOnNewRepo":false}' NOT NULL;
+126
View File
@@ -0,0 +1,126 @@
-- Migrate the OAuth/OIDC *provider* feature from the deprecated
-- `oidc-provider` plugin to `@better-auth/oauth-provider`.
--
-- Tables: oauth_applications -> oauth_clients, oauth_access_tokens reshaped,
-- new oauth_refresh_tokens, oauth_consent -> oauth_consents, plus a `jwks`
-- table for the `jwt` plugin (id_token signing keys).
--
-- Data preservation:
-- * Registered clients are copied from oauth_applications into oauth_clients,
-- converting the legacy comma-separated `redirect_urls` into the JSON
-- string[] (`redirect_uris`) the new adapter expects.
-- * Access tokens and consent records are NOT migrated: access tokens are
-- short-lived (and the column shape changed entirely), and consents are
-- cheaply re-granted on next authorization. Both old tables are dropped.
--
-- NOTE: legacy client secrets were stored in plaintext, whereas the new
-- provider stores them hashed. Migrated client secrets will therefore not
-- validate as-is — affected applications must rotate their secret after
-- upgrade.
CREATE TABLE `oauth_clients` (
`id` text PRIMARY KEY NOT NULL,
`client_id` text NOT NULL,
`client_secret` text,
`name` text,
`disabled` integer DEFAULT false,
`skip_consent` integer,
`enable_end_session` integer,
`subject_type` text,
`scopes` text,
`user_id` text,
`uri` text,
`icon` text,
`contacts` text,
`tos` text,
`policy` text,
`software_id` text,
`software_version` text,
`software_statement` text,
`redirect_uris` text NOT NULL,
`post_logout_redirect_uris` text,
`token_endpoint_auth_method` text,
`grant_types` text,
`response_types` text,
`public` integer,
`type` text,
`require_pkce` integer,
`reference_id` text,
`metadata` text,
`created_at` integer DEFAULT (unixepoch()),
`updated_at` integer DEFAULT (unixepoch()),
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE UNIQUE INDEX `oauth_clients_client_id_unique` ON `oauth_clients` (`client_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_clients_client_id` ON `oauth_clients` (`client_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_clients_user_id` ON `oauth_clients` (`user_id`);--> statement-breakpoint
INSERT INTO `oauth_clients` (
`id`, `client_id`, `client_secret`, `name`, `disabled`, `user_id`,
`redirect_uris`, `type`, `metadata`, `created_at`, `updated_at`
)
SELECT
`id`, `client_id`, `client_secret`, `name`, `disabled`, `user_id`,
'["' || replace(`redirect_urls`, ',', '","') || '"]',
`type`, `metadata`, `created_at`, `updated_at`
FROM `oauth_applications`;--> statement-breakpoint
DROP TABLE `oauth_applications`;--> statement-breakpoint
DROP TABLE `oauth_access_tokens`;--> statement-breakpoint
CREATE TABLE `oauth_access_tokens` (
`id` text PRIMARY KEY NOT NULL,
`token` text,
`client_id` text NOT NULL,
`session_id` text,
`user_id` text,
`reference_id` text,
`refresh_id` text,
`expires_at` integer,
`created_at` integer DEFAULT (unixepoch()),
`scopes` text NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE UNIQUE INDEX `oauth_access_tokens_token_unique` ON `oauth_access_tokens` (`token`);--> statement-breakpoint
CREATE INDEX `idx_oauth_access_tokens_token` ON `oauth_access_tokens` (`token`);--> statement-breakpoint
CREATE INDEX `idx_oauth_access_tokens_client_id` ON `oauth_access_tokens` (`client_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_access_tokens_user_id` ON `oauth_access_tokens` (`user_id`);--> statement-breakpoint
CREATE TABLE `oauth_refresh_tokens` (
`id` text PRIMARY KEY NOT NULL,
`token` text NOT NULL,
`client_id` text NOT NULL,
`session_id` text,
`user_id` text NOT NULL,
`reference_id` text,
`expires_at` integer,
`created_at` integer DEFAULT (unixepoch()),
`revoked` integer,
`auth_time` integer,
`scopes` text NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE UNIQUE INDEX `oauth_refresh_tokens_token_unique` ON `oauth_refresh_tokens` (`token`);--> statement-breakpoint
CREATE INDEX `idx_oauth_refresh_tokens_token` ON `oauth_refresh_tokens` (`token`);--> statement-breakpoint
CREATE INDEX `idx_oauth_refresh_tokens_client_id` ON `oauth_refresh_tokens` (`client_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_refresh_tokens_user_id` ON `oauth_refresh_tokens` (`user_id`);--> statement-breakpoint
DROP TABLE `oauth_consent`;--> statement-breakpoint
CREATE TABLE `oauth_consents` (
`id` text PRIMARY KEY NOT NULL,
`client_id` text NOT NULL,
`user_id` text,
`reference_id` text,
`scopes` text NOT NULL,
`created_at` integer DEFAULT (unixepoch()),
`updated_at` integer DEFAULT (unixepoch()),
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE INDEX `idx_oauth_consents_client_id` ON `oauth_consents` (`client_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_consents_user_id` ON `oauth_consents` (`user_id`);--> statement-breakpoint
CREATE TABLE `jwks` (
`id` text PRIMARY KEY NOT NULL,
`public_key` text NOT NULL,
`private_key` text NOT NULL,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`expires_at` integer
);
+35
View File
@@ -0,0 +1,35 @@
PRAGMA foreign_keys=OFF;--> statement-breakpoint
CREATE TABLE `__new_organizations` (
`id` text PRIMARY KEY NOT NULL,
`user_id` text NOT NULL,
`config_id` text NOT NULL,
`name` text NOT NULL,
`normalized_name` text NOT NULL,
`avatar_url` text NOT NULL,
`membership_role` text DEFAULT 'member' NOT NULL,
`is_included` integer DEFAULT true NOT NULL,
`destination_org` text,
`status` text DEFAULT 'imported' NOT NULL,
`last_mirrored` integer,
`error_message` text,
`repository_count` integer DEFAULT 0 NOT NULL,
`public_repository_count` integer,
`private_repository_count` integer,
`fork_repository_count` integer,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action,
FOREIGN KEY (`config_id`) REFERENCES `configs`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
INSERT INTO `__new_organizations`("id", "user_id", "config_id", "name", "normalized_name", "avatar_url", "membership_role", "is_included", "destination_org", "status", "last_mirrored", "error_message", "repository_count", "public_repository_count", "private_repository_count", "fork_repository_count", "created_at", "updated_at") SELECT "id", "user_id", "config_id", "name", "normalized_name", "avatar_url", "membership_role", "is_included", "destination_org", "status", "last_mirrored", "error_message", "repository_count", "public_repository_count", "private_repository_count", "fork_repository_count", "created_at", "updated_at" FROM `organizations`;--> statement-breakpoint
DROP TABLE `organizations`;--> statement-breakpoint
ALTER TABLE `__new_organizations` RENAME TO `organizations`;--> statement-breakpoint
PRAGMA foreign_keys=ON;--> statement-breakpoint
CREATE INDEX `idx_organizations_user_id` ON `organizations` (`user_id`);--> statement-breakpoint
CREATE INDEX `idx_organizations_config_id` ON `organizations` (`config_id`);--> statement-breakpoint
CREATE INDEX `idx_organizations_status` ON `organizations` (`status`);--> statement-breakpoint
CREATE INDEX `idx_organizations_is_included` ON `organizations` (`is_included`);--> statement-breakpoint
CREATE UNIQUE INDEX `uniq_organizations_user_normalized_name` ON `organizations` (`user_id`,`normalized_name`);--> statement-breakpoint
ALTER TABLE `sso_providers` ADD `saml_config` text;--> statement-breakpoint
ALTER TABLE `sso_providers` ADD `domain_verified` integer DEFAULT true NOT NULL;
+2
View File
@@ -0,0 +1,2 @@
ALTER TABLE `organizations` ADD `mirror_overrides` text;--> statement-breakpoint
ALTER TABLE `repositories` ADD `mirror_overrides` text;
+2
View File
@@ -0,0 +1,2 @@
ALTER TABLE `repositories` ADD `source_provider` text DEFAULT 'github' NOT NULL;--> statement-breakpoint
ALTER TABLE `repositories` ADD `source_url` text DEFAULT 'https://github.com' NOT NULL;
+63
View File
@@ -0,0 +1,63 @@
CREATE TABLE `oauth_client_assertions` (
`id` text PRIMARY KEY NOT NULL,
`expires_at` integer NOT NULL
);
--> statement-breakpoint
CREATE TABLE `oauth_client_resources` (
`id` text PRIMARY KEY NOT NULL,
`client_id` text NOT NULL,
`resource_id` text NOT NULL,
`metadata` text,
`created_at` integer DEFAULT (unixepoch())
);
--> statement-breakpoint
CREATE INDEX `idx_oauth_client_resources_client_id` ON `oauth_client_resources` (`client_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_client_resources_resource_id` ON `oauth_client_resources` (`resource_id`);--> statement-breakpoint
CREATE TABLE `oauth_resources` (
`id` text PRIMARY KEY NOT NULL,
`identifier` text NOT NULL,
`name` text NOT NULL,
`access_token_ttl` integer,
`refresh_token_ttl` integer,
`signing_algorithm` text,
`signing_key_id` text,
`allowed_scopes` text,
`custom_claims` text,
`dpop_bound_access_tokens_required` integer,
`disabled` integer,
`policy_version` integer,
`metadata` text,
`created_at` integer DEFAULT (unixepoch()),
`updated_at` integer DEFAULT (unixepoch())
);
--> statement-breakpoint
CREATE UNIQUE INDEX `oauth_resources_identifier_unique` ON `oauth_resources` (`identifier`);--> statement-breakpoint
CREATE INDEX `idx_oauth_resources_identifier` ON `oauth_resources` (`identifier`);--> statement-breakpoint
ALTER TABLE `accounts` ADD `issuer` text DEFAULT '' NOT NULL;--> statement-breakpoint
ALTER TABLE `jwks` ADD `alg` text;--> statement-breakpoint
ALTER TABLE `jwks` ADD `crv` text;--> statement-breakpoint
ALTER TABLE `oauth_access_tokens` ADD `authorization_code_id` text;--> statement-breakpoint
ALTER TABLE `oauth_access_tokens` ADD `resources` text;--> statement-breakpoint
ALTER TABLE `oauth_access_tokens` ADD `requested_user_info_claims` text;--> statement-breakpoint
ALTER TABLE `oauth_access_tokens` ADD `revoked` integer;--> statement-breakpoint
ALTER TABLE `oauth_access_tokens` ADD `confirmation` text;--> statement-breakpoint
ALTER TABLE `oauth_clients` ADD `client_discovery_id` text;--> statement-breakpoint
ALTER TABLE `oauth_clients` ADD `client_credentials_scopes` text;--> statement-breakpoint
ALTER TABLE `oauth_clients` ADD `backchannel_logout_uri` text;--> statement-breakpoint
ALTER TABLE `oauth_clients` ADD `backchannel_logout_session_required` integer;--> statement-breakpoint
ALTER TABLE `oauth_clients` ADD `application_type` text;--> statement-breakpoint
ALTER TABLE `oauth_clients` ADD `jwks` text;--> statement-breakpoint
ALTER TABLE `oauth_clients` ADD `jwks_uri` text;--> statement-breakpoint
ALTER TABLE `oauth_clients` ADD `dpop_bound_access_tokens` integer;--> statement-breakpoint
ALTER TABLE `oauth_consents` ADD `resources` text;--> statement-breakpoint
ALTER TABLE `oauth_consents` ADD `requested_user_info_claims` text;--> statement-breakpoint
ALTER TABLE `oauth_refresh_tokens` ADD `authorization_code_id` text;--> statement-breakpoint
ALTER TABLE `oauth_refresh_tokens` ADD `resources` text;--> statement-breakpoint
ALTER TABLE `oauth_refresh_tokens` ADD `requested_user_info_claims` text;--> statement-breakpoint
ALTER TABLE `oauth_refresh_tokens` ADD `rotated_at` integer;--> statement-breakpoint
ALTER TABLE `oauth_refresh_tokens` ADD `rotation_replay_response` text;--> statement-breakpoint
ALTER TABLE `oauth_refresh_tokens` ADD `rotation_replay_expires_at` integer;--> statement-breakpoint
ALTER TABLE `oauth_refresh_tokens` ADD `confirmation` text;--> statement-breakpoint
UPDATE `accounts` SET `issuer` = 'local:credential' WHERE `issuer` = '' AND `provider_id` = 'credential';--> statement-breakpoint
UPDATE `accounts` SET `issuer` = (SELECT `issuer` FROM `sso_providers` WHERE `sso_providers`.`provider_id` = `accounts`.`provider_id` LIMIT 1) WHERE `issuer` = '' AND EXISTS (SELECT 1 FROM `sso_providers` WHERE `sso_providers`.`provider_id` = `accounts`.`provider_id`);--> statement-breakpoint
UPDATE `accounts` SET `issuer` = 'local:oauth:' || `provider_id` WHERE `issuer` = '';
+28
View File
@@ -0,0 +1,28 @@
CREATE TABLE `api_keys` (
`id` text PRIMARY KEY NOT NULL,
`config_id` text DEFAULT 'default' NOT NULL,
`name` text,
`start` text,
`prefix` text,
`key` text NOT NULL,
`reference_id` text NOT NULL,
`refill_interval` integer,
`refill_amount` integer,
`last_refill_at` integer,
`enabled` integer DEFAULT true NOT NULL,
`rate_limit_enabled` integer DEFAULT true NOT NULL,
`rate_limit_time_window` integer,
`rate_limit_max` integer,
`request_count` integer DEFAULT 0 NOT NULL,
`remaining` integer,
`last_request` integer,
`expires_at` integer,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
`permissions` text,
`metadata` text,
FOREIGN KEY (`reference_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE cascade
);
--> statement-breakpoint
CREATE INDEX `idx_api_keys_reference_id` ON `api_keys` (`reference_id`);--> statement-breakpoint
CREATE INDEX `idx_api_keys_key` ON `api_keys` (`key`);
+7
View File
@@ -0,0 +1,7 @@
ALTER TABLE `repositories` ADD `destination_provider` text DEFAULT 'gitea' NOT NULL;--> statement-breakpoint
ALTER TABLE `repositories` ADD `destination_url` text DEFAULT '' NOT NULL;--> statement-breakpoint
UPDATE `repositories` SET
`destination_provider` = COALESCE((SELECT json_extract(`configs`.`gitea_config`, '$.provider') FROM `configs` WHERE `configs`.`id` = `repositories`.`config_id`), 'gitea'),
`destination_url` = COALESCE((SELECT json_extract(`configs`.`gitea_config`, '$.url') FROM `configs` WHERE `configs`.`id` = `repositories`.`config_id`), '')
WHERE `destination_url` = '';--> statement-breakpoint
UPDATE `repositories` SET `destination_provider` = 'gitea' WHERE `destination_provider` NOT IN ('gitea', 'forgejo', 'github', 'gitlab');
Loaded 100 of 628 files, more files were not shown because too many files have changed in this diff. Show more