Release assets were uploaded only on the create path of
mirrorGitHubReleasesToGitea(). When a Gitea release already existed, the
update path PATCHed the changelog/title and `continue`d without ever
touching assets. So any release whose assets were not fully uploaded on
that single create-path run — first sync interrupted, a transient
download/upload failure, large multi-MB assets, etc. — stayed permanently
asset-less, and re-syncing always hit the update path and could never
recover it. Asset failures were also swallowed to console.error, so the
job still reported success (the "no errors in the logs" in #331).
Reproduced on a real Forgejo pull-mirror with shauninman/MinUI: a GitHub
release with two ~35-40MB binaries became a Gitea release with 0 assets
(just Forgejo's auto-generated source archive), and re-syncing left it at
0 while logging "Updating existing release".
Fix:
- Add reconcileReleaseAssets(), an idempotent reconciler run on BOTH the
create and update paths. It compares Gitea's existing attachments to
GitHub's by name+size: skips matches, uploads missing ones, replaces
size-mismatched copies. Existing broken releases self-heal on next sync.
- Extract the pure decision into classifyAssetsForReconciliation() for
unit testing (the absence of asset tests is why this slipped past #310).
- Surface asset upload counts in the summary and emit a visible warning
when any fail, instead of silently swallowing them.
- Add 5 unit tests covering the broken state, partial backfill,
idempotency, size-mismatch replacement, and the no-assets case.
Verified end-to-end against the real function on a Forgejo pull-mirror:
0 -> 2 assets backfilled, already-present assets skipped (no re-download),
second run uploads 0.
The "Name collision strategy" dropdown (starredDuplicateStrategy) never
persisted: the field was absent from both directions of the UI<->DB config
mapper. On save, mapUiToDbConfig dropped it before the DB write; on load,
mapDbToUiConfig never read it, so the UI reset to the "suffix" (repo-owner)
default. Mirror logic in gitea.ts then read undefined and also defaulted to
suffix — so repos really were created with that pattern regardless of the
user's choice. It has been broken since the field was introduced.
- Map starredDuplicateStrategy in mapUiToDbConfig and mapDbToUiConfig
- Add STARRED_DUPLICATE_STRATEGY env var for parity (reporter could not
work around it via compose because no env var existed) + docs
- Round-trip tests covering save, load, and the missing-field default
Repository discovery requested the `organization_member` affiliation
unconditionally, so repos from every org a user belongs to were imported —
even orgs they never explicitly added. `skipPersonalRepos` only dropped
user-owned repos and left org repos unfiltered, which surprised users who
expected "only mirror org repos" to mean "only the orgs I chose" (reported
on #304).
Wire up the previously-dormant `includeOrganizations` config field as an
opt-in allowlist: when non-empty, only repos owned by the listed
organizations are imported. Empty = all org repos (backward-compatible).
Owned and collaborator repos are never restricted, so it composes cleanly
with `skipPersonalRepos`.
- Filter org repos by the allowlist in getGithubRepositories
- Add includeAllOrgsOverride so the cleanup service bypasses the allowlist
and never false-orphans a previously-mirrored repo from an org the user
later removes from the list
- UI control under Filtering & Behavior; INCLUDE_ORGANIZATIONS env var
- Case-insensitive dedup/trim in the UI<->DB mapper round-trip
- 7 unit tests covering the filter, composition, and the cleanup override
* Handle indexing when shift + clicking in the repository table
* Move the buttons when selecting rows
* Add in a bulk delete func in the repositories table
* Add bulk delete handler
* Make the single action use the bulk delete
* Delete the single repository id handler
- Add `skipPersonalRepos: z.boolean().default(false)` to githubConfigSchema
- Filter out user-owned repos in getGithubRepositories when flag is true
- Wire ONLY_MIRROR_ORGS env var to skipPersonalRepos in env-config-loader
- Add checkbox UI in GitHubMirrorSettings Filtering & Behavior section
- Round-trip skipPersonalRepos through config-mapper (UI ↔ DB)
- Add skipPersonalRepos to AdvancedOptions TypeScript type
- Mark include/exclude arrays in configSchema as unused/reserved
- Update ENVIRONMENT_VARIABLES.md to document ONLY_MIRROR_ORGS effect
* fix(scheduler): make enabled flag authoritative for auto-start
checkAutoStartConfiguration() and performInitialAutoStart() previously
used `scheduleEnabled || hasMirrorInterval`, allowing a configured
GITEA_MIRROR_INTERVAL to trigger boot-time auto-start even after the
user disabled scheduling via the UI toggle.
env-config-loader already writes scheduleConfig.enabled=true when
GITEA_MIRROR_INTERVAL is set at container startup, so the interval is
a timing detail, not an enable signal. The documented env-var contract
is preserved: GITEA_MIRROR_INTERVAL at boot → env-config-loader sets
enabled=true → auto-start fires. But a later UI disable now sticks.
Add a focused unit test for the gate logic.
* fix(backup): always derive clone URL from user-configured Gitea URL
The pre-sync backup preferred repoInfo.clone_url, which reflects
Gitea's ROOT_URL setting. In Tailscale MagicDNS deployments (and any
setup where ROOT_URL is an external address), this URL is unreachable
from the app itself, causing bundle backup to fail.
Always build the clone URL as:
${config.giteaConfig.url.trimEnd('/')}/${owner}/${repo}.git
This matches the URL the app already uses for all other Gitea API
calls and is guaranteed reachable.
* feat(jobs): cancel-pending endpoint + fix misleading Delete All copy
Add POST /api/job/cancel-pending that sets the current user's
repositories with status "imported" or "failed" to "ignored",
preventing the scheduler from re-queuing them. In-flight "mirroring"
rows are left alone. Returns the count and logs one activity entry.
Fix the "Delete All Activities" dialog to clearly state it only clears
the history log and does not stop pending work. Rename button/title to
"Clear History" so intent is unambiguous.
Add a "Stop Pending Mirrors" button (StopCircle icon, amber) in both
mobile and desktop activity log toolbars, with a confirmation dialog
explaining repos are set to Ignored and can be re-enabled from the
Repositories page.
* fix(sync): actionable 405 error for non-pull-mirror repos
Gitea returns HTTP 405 with an empty body when the target repository is
no longer a pull mirror — e.g. the mirror was auto-disabled by Gitea or
the repository lost its mirror state after a manual edit.
Previously this fell through to the generic error handler which stored
the raw HttpError message (often empty) giving the user no guidance.
Now a 405 response is caught alongside the existing 400 handler and
sets the repository to "failed" with an actionable error message:
"Gitea reports this repository is not a pull mirror (HTTP 405).
In Gitea check Settings → Mirror Settings; if the mirror section is
missing, delete the repository in Gitea and re-mirror it from
gitea-mirror."
The same message is written to the activity log for visibility in the
dashboard.
Root cause (Theory A): the `needsRecreation` check compared GitHub
published_at-based expected indices against Gitea's API order. Gitea mirror
repos sort releases by tag-commit date, which can permanently disagree with
published_at order (e.g. unaconfig_dart v0.1.0 published after v0.1.1 but
tagged before). This made `currentExpectedIdx < nextExpectedIdx` evaluate
true on every sync, triggering delete-all-and-recreate forever — spamming
Gitea's activity feed with "released X" events (#310).
Fix: replace the destructive order-check machinery with set-based
reconciliation via `classifyReleasesForReconciliation`. Releases are
created when missing in Gitea and skipped (or PATCH-updated if content
drifted) when already present. No deletions are ever triggered by ordering.
Retain the existing release-limit trimming (retention cleanup) unchanged.
Also removes the 1-second per-release delay that was only needed for the
creation-order dance, significantly speeding up initial mirrors.
Adds unit tests covering: normal ordered repos, the unaconfig_dart inversion
fixture, missing→create, present→skip, and edge cases.
Starred (and other) repos duplicated on every re-mirror (starred/Repo,
Repo-owner, Repo-owner-1, ...) because the existence check only asked
"does a repo with this name exist?" and never "is the existing repo a
mirror of THIS same source?". The repo's own prior mirror counted as a
collision, so generateUniqueRepoName bumped to the next suffix each run,
repointing mirroredLocation at the newest copy. Under a single re-call,
3 concurrent/retried jobs each computed a DIFFERENT suffixed name, so the
location-based in-flight guard never matched and the race produced extra
copies.
Fix (source-identity aware):
- New shared helper src/lib/utils/mirror-source-match.ts:
- normalizeCloneUrl / cloneUrlsMatch: credential-, .git-, slash- and
host-case-insensitive clone URL comparison.
- isMirrorOfSource: a Gitea repo is "ours" only if it is a mirror AND
its original_url matches this repo's source.
- findExistingMirror: resolves an existing same-source mirror via the
recorded mirroredLocation first (survives strategy changes — #309),
then the base candidate name.
- classifyCandidateName: pure available/reusable/taken decision.
- gitea-enhanced: export GiteaRepoInfo and add original_url (Gitea's
recorded migration source) for source matching.
- Both create paths (mirrorGithubRepoToGitea, mirrorGitHubRepoToGiteaOrg):
run findExistingMirror BEFORE name generation; on a hit, reuse that
location and route into the existing "already mirrored" handling rather
than calling generateUniqueRepoName. Names now converge under
concurrency so the in-flight guard becomes effective.
- generateUniqueRepoName is now source-aware: an occupied name held by a
mirror of the SAME source is reused (no suffix); suffixing only happens
on a genuine different-source collision, preserving #95/#236 behavior.
The per-user DB claim check is retained so two users mirroring the same
source into a shared org stay separated.
- Phantom-fork guard (#309): the existingRepoInfo.mirror branches now
verify same-source before marking "mirrored"; on mismatch they fall
through to unique-name generation and create a separate mirror.
- Scheduler: a `failed` repo whose mirroredLocation still resolves to a
live same-source mirror is routed to syncGiteaRepo instead of re-create,
breaking the failed-metadata re-create loop cheaply.
- Remove dead src/lib/starred-repos-handler.ts (zero importers across all
git history); its correct base-name/.mirror reuse logic now lives in the
shared helper.
Tests: src/lib/utils/mirror-source-match.test.ts (30 cases) covers URL
normalization, reuse at base name, reuse via mirroredLocation across a
strategy change, genuine different-source collision (suffix), phantom
fork, stale mirroredLocation fallback, per-user DB-claim separation, and
the suffix-vs-reuse classification. Full suite: 319 pass, 0 fail.
Migration 0013 runs as a single transaction that rebuilds `organizations`
and then `ALTER TABLE sso_providers ADD saml_config` / `ADD domain_verified`.
On instances where those columns were already created outside Drizzle (declared
in schema.ts and added via db:push / an SSO-register round-trip on an
intermediate build), the ADD throws "duplicate column name: saml_config". That
rolls back the entire 0013 transaction, so 0013 is never recorded in
`__drizzle_migrations` and is retried — failing identically — on every boot,
crash-looping the server.
Add a pre-migrate repair (mirroring the existing repairFailedMigrations() for
the 0009 case): when 0013 is unrecorded but the columns already exist, preserve
any real SAML provider config, drop the stranded columns so the canonical 0013
runs in full (organizations rebuild included), then restore the preserved
values once the columns are re-added. No-op on fresh installs, clean upgrades,
and already-migrated databases.
This lets affected instances recover automatically on the next boot after
upgrading — no manual SQLite surgery required.
- src/lib/db/migration-repairs.ts: repairDuplicateSsoColumns + restoreSsoDataAfter0013
- src/lib/db/index.ts: wire both around migrate()
- scripts/validate-migrations.ts: cover the broken-upgrade + data-preservation path
Resolves#306. SSO sign-in via OIDC (Authentik / Keycloak / etc.) now links the
SSO identity to an existing email/password admin instead of bouncing to /login
with `?error=UNKNOWN`. Account-linking is gated on the operator-supplied
**Domain** field — cross-domain claims from a compromised IdP are refused.
Also bundles the deprecated `oidcProvider` → `@better-auth/oauth-provider`
migration. **Operators using the OAuth-provider feature must rotate registered
client secrets after upgrade** (legacy plaintext → hashed storage; see the
0012 migration notes).
Verified end-to-end on the pr-307 image against a real Authentik instance:
SSO login lands on the dashboard, `accounts` table gets both `credential` and
`authentik` rows for the same user. See PR description for full details.
Header / forward authentication has been end-to-end broken since the
v3 rewrite. The middleware populated `context.locals.user` from
trusted upstream headers (Authentik / Authelia / oauth2-proxy /
Caddy), but never minted a Better Auth session, and never set a
cookie. Server-rendered pages saw the user, but the React SPA's
`/api/auth/get-session` call hit Better Auth's handler — which only
reads its session cookie — and got `null`. The auth guard then
redirected to `/login`, even though the upstream proxy had already
authenticated the user.
Reported on issue #29 by @lanrat with a clean repro on v3.16.1.
Fix: add a small Better Auth plugin (`header-auth`) that exposes
`POST /api/auth/sign-in/header`. The endpoint validates the trusted
headers via `authenticateWithHeaders`, creates a real session row via
`internalAdapter.createSession`, and attaches the `Set-Cookie` via
`setSessionCookie` — the same pattern the magic-link, anonymous, and
phone-number plugins use after their respective verification steps.
The Astro middleware now calls this endpoint when no cookie session
exists and header auth is enabled, forwards the `Set-Cookie` onto the
outbound response, and populates `context.locals` from the minted
session. After the first request the browser has the cookie; every
subsequent request takes the normal cookie-auth fast path and the
bridge doesn't fire.
Fail-open everywhere: any endpoint failure (header auth disabled,
auth rejected, DB blip, malformed response) returns null from the
bridge and the request proceeds as anonymous. A broken header-auth
configuration must never lock everyone out of the cookie-auth path.
Tests:
- `auth-header.test.ts` — unit tests for `extractUserFromHeaders` and
`isHeaderAuthEnabled`, including lanrat's reported config shape
(same header for username and email).
- `auth-header-plugin.test.ts` — locks down plugin id, endpoint key,
path, and method so an accidental rename can't silently break the
middleware bridge.
- `auth-header-bridge.test.ts` — covers the cookie-extraction logic
and the fail-open paths (non-2xx, thrown error, malformed JSON,
missing fields, no Set-Cookie attached).
Stacks on top of #301 (better-auth 1.6.11 bump).
Refs: #29
* fix: stop snapshot-row zombies + flapping force-push on deleted branches
Two bugs caused Simple-WP-Helpdesk to accumulate one orphan
"Snapshot created" job row per scheduled sync — 7 zombies in 24h
against a deleted GitHub branch (`fix/v4.0.2-webhook-comment-author`).
(1) gitea-enhanced.ts:522 created the post-snapshot job record with
status="syncing". The snapshot was already complete at that point
(createPreSyncBundleBackup had returned), and no later code path
advanced the row to a terminal status — it just lingered. Set
status="synced" to reflect reality.
(2) force-push-detection treated any Gitea branch missing from GitHub
as a "deleted" force-push. But gitea-mirror is one-way (GitHub →
Gitea), so deletions never propagate back to the Gitea mirror —
the branch lingers in Gitea forever, the missing-from-GitHub condition
holds on every subsequent sync, and detection re-fires endlessly.
Each re-fire triggered backupBeforeSync=true to take a fresh snapshot
of the exact same state we already backed up the previous cycle.
Fix: detector accepts an `acknowledgedDeletions: { branch, giteaSha }[]`
list (persisted in RepositoryMetadataState). A Gitea branch missing
from GitHub is suppressed when its current giteaSha matches an entry.
If the giteaSha later changes (branch restored, then re-deleted with
new history), the entry won't match and detection fires again — back
up the new state, then acknowledge the new SHA. After a successful
snapshot, the call site appends each just-backed-up "deleted" branch
to the acknowledged list and persists the metadata blob.
Hoists parseRepositoryMetadataState above the backup-strategy block so
both the detection-time read and post-snapshot append happen against
the same in-memory state object; the existing metadata-mirror block
just stops re-declaring it.
Adds 9 new test cases covering:
- deleted branch suppressed when acknowledged at matching giteaSha
- re-flagged when giteaSha differs (restored-then-redeleted)
- mixed deletions (only matching one suppressed)
- undefined list = back-compat (existing callers unaffected)
- acknowledgedDeletions does not suppress "diverged"
- metadata-state parse/serialize round-trips the new field
- legacy metadata defaults acknowledgedDeletions to []
- malformed entries dropped without throwing
All 76 tests in the affected suites pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: acknowledge deletion before backup, not after, to fix concurrent-sync race
When two sync invocations fire for the same repo within the same backup
window (observed in prod via UI trigger pipeline: a single click ends up
producing two parallel processWithResilience callbacks that both run
syncGiteaRepoEnhanced), the second invocation's createPreSyncBundleBackup
short-circuits (file already exists from the first), so the second
invocation never reached the acknowledge-push that lived inside the
backup try block. Then both invocations wrote metadata: invocation A
with acknowledgedDeletions=[forged], invocation B (with its stale
in-memory state) with acknowledgedDeletions=[], and B's write landed
last — overwriting A.
Net result: the deletion never got acknowledged, and the next sync
re-detected it, the next-next sync re-detected it, etc. Same flapping
behavior as before the fix.
Fix: move the acknowledge-push to run right after detection, based on
detectionResult.affectedBranches alone. Semantically correct — once
we've detected the branch is gone from GitHub, we know it's a permanent
deletion; whether THIS particular invocation took a backup or
short-circuited is orthogonal (a previous backup on disk is fine, and
in the rare case where no backup ever succeeded, the user can re-trigger
a manual backup). Both concurrent invocations now push the same entry;
both write the same metadata; race is benign.
Verified locally: 59 tests pass across force-push-detection,
gitea-enhanced, repo-backup.
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Paginate /milestones and /labels with Link header + X-Total-Count fallback, and pass state=all on milestones so closed milestones aren't re-POSTed on every sync. Caches newly-created items in the in-memory dedup set.
The previous middleware logic gated recovery behind a
once-per-process flag. After the first request handled the boot-time
recovery check, the resume codepath never fired again — even when
new interruptions appeared later in the process lifetime.
Symptom: a sync that started after server boot, crashed mid-flight
(deadlock retry hitting maxRetries, network blip, container
restart of an upstream service, etc.) and never reached the resume
codepath would sit at `inProgress=true, lastCheckpoint=never`
forever. The periodic detector kept finding the stuck row and
logging `Found 1 interrupted jobs:` on every poll (driven by the
health endpoint via `hasJobsNeedingRecovery`), but the resumer
(`resumeInterruptedJob`) was only invoked from `initializeRecovery`
which the middleware never re-called.
Fix: replace the one-shot gate with an in-flight latch
(`recoveryInFlight`) that's released in a `finally` block. Throttling
of actual recovery work is delegated to the existing 5-minute
`skipIfRecentAttempt` check inside `initializeRecovery()`, which is
the right place for it. `recoveryInitialized` is kept and only used
to control whether the "first run" log lines fire.
Secondary fix: `findInterruptedJobs` previously logged
`Found N interrupted jobs:` unconditionally on every call, including
from passive polls in `hasJobsNeedingRecovery` (which the health
endpoint and middleware probe call frequently). That produced one
log line per poll per stuck job for as long as the job stayed stuck.
Make logging opt-in via a `logFound` parameter, default off; the
active recovery cycle in `initializeRecovery()` opts in so the
operator-facing log still surfaces which jobs are being worked on.
Related: #268 (partially addressed). The PR #280 / v3.15.7 fix was
about a JS scoping bug that made the *initial* migrate call's
catch path crash before transitioning the repo to `failed`. That
was one cause of stuck `mirroring` state; this PR addresses the
follow-on issue that even when a job *is* correctly detectable as
interrupted, the post-startup recovery path never re-engages.
Adds `orchestrator-resume-after-startup.test.ts` using the
structural-source test pattern from
`gitea-mirror-failure-recovery.test.ts` so the four guarantees
(no static gate; in-flight latch released in finally; logging
opt-in; active path opts in) are enforced without heavy mocks.
mirrorGitRepoIssuesToGitea and mirrorGitRepoPullRequestsToGitea both
had two compounding bugs that produced duplicate Gitea issues/PR-as-
issue rows on every sync against any non-SQLite backend.
(1) Pagination on existing-issues / existing-PRs / per-issue-comments
was wrong.
The loops paginated with `limit=100` and broke on `pageX.length <
itemsPerPage`. Gitea caps response size at server-side
[api].MAX_RESPONSE_ITEMS (default 50), so the very first page
already looks "short" and pagination terminated after one page. The
existing-issue and existing-PR maps were built from only ~50 items
per repo, so every issue/PR past page 1 was treated as new on every
sync and re-created via the CREATE branch.
Naive removal of the short-page break (relying only on "break on
empty page") doesn't work either: for some Gitea endpoints Gitea
returns the same data on every page when the page is past the
actual end instead of returning [], so the loop runs forever.
Fix: use the Link header (RFC 5988). If `rel="next"` is absent,
terminate pagination. Applied to: existing-issues pre-fetch,
existing-PRs pre-fetch, and per-issue comments fetch.
(2) Retry-after-deadlock duplicated issues/PRs even when the map
was correct.
Gitea's CreateIssue handler commits the issue insert in one
transaction and then deadlocks on the subsequent addLabel /
UPDATE repository transaction. The issue row is committed and
visible, but the in-memory dedup maps are never refreshed between
retries — processWithRetry re-invokes the callback, sees
existingIssue === undefined from the stale map, and creates a fresh
duplicate via httpPost.
Reproduces deterministically on MySQL (Error 1213 / 40001) and
PostgreSQL (40P01); SQLite escapes because writes serialize globally.
Fix: defensive recheck via httpGet by [GH-ISSUE #N] (issues) or
[PR #N] (pull requests) before the create call, with PATCH
fall-through when found. Applied to the issues create path AND
both create paths in the PR mirror (enriched + basic-fallback).
Also cache freshly-created items into the dedup maps after a
successful create so subsequent retries of the same per-item
callback don't lose track of it.
Adds gitea-issue-dedup-on-retry.test.ts using the structural-source
test pattern from gitea-mirror-failure-recovery.test.ts so all
guarantees are enforced without heavy module mocks (8 tests).
Adds an account menu under the avatar in the header with Change password and
Change email actions, each opening a small dialog. Calls Better Auth's existing
change-password / change-email endpoints — no new API routes.
- Enables `user.changeEmail` in Better Auth with `updateEmailWithoutVerification`
since the app runs with email verification disabled and no email sender is
wired up
- Hides Change password for SSO-only users (no `credential` provider account),
fails open if the listAccounts probe errors
- Resolves discussion #291 ("Change user password/email")
Previously, mirror of issues/pull-requests/labels/milestones was
guarded by !metadataState.components.<component>, so once a repo had
been mirrored the metadata path was permanently skipped with logs
like "Issues already mirrored; skipping to avoid duplicates".
This meant title changes, new comments, label updates and milestone
edits on the source GitHub repo were never propagated, even when the
user explicitly clicked Sync.
The underlying mirror* functions already handle idempotent updates:
issues and PRs are matched by [GH-ISSUE #N] / [GH-PR #N] markers in
the title and PATCHed in place, labels are deduped by name, milestones
by title. The releases path already runs unconditionally for the same
reason ("always allowed to rerun for updates"); aligning the other
metadata paths with it.
Touched: mirrorGithubRepoToGitea, mirrorGitHubRepoToGiteaOrg, and the
syncGiteaRepoEnhanced path in gitea-enhanced.ts. Updated the
"already-synced-repo" test to assert reconciliation runs on resync.
Affiliation was set to "owner,collaborator", omitting repos owned
by orgs the user belongs to. As a result:
- main sync, scheduler, and cleanup never saw org repos
- orgs appeared empty unless manually re-added via /api/sync/organization
- restart archived previously-mirrored org repos as orphans
GitHub's API default is owner,collaborator,organization_member;
restoring it fixes both symptoms with no other code changes.
GitHub's listForAuthenticatedUser defaults to returning every repo the
user has access to (owner + collaborator + organization_member), which
imports a lot of noise for users who only want their own repos.
Adds an `includeCollaboratorRepos` toggle, defaulting to true to preserve
existing behavior. When disabled, the affiliation filter scopes the API
call to "owner" only.
The cleanup service overrides the filter to always include collaborator
repos when computing the "what's still on GitHub" list. Without this,
toggling the option off would mark previously-mirrored collab repos as
orphaned and archive/delete them from Gitea.
Wired through the schema, both UI<->DB mappers, the env-config loader
(with new INCLUDE_COLLABORATOR_REPOS env var), and the settings UI.
The fix in v3.15.8 made scheduleConfig.autoMirror an independent trigger
in the scheduler, but it remained reachable only via the AUTO_MIRROR_REPOS
env var. This adds a UI checkbox under the Automatic Syncing section so
the option can be toggled per-config without touching the environment.
The toggle is conditional on scheduling being enabled (since auto-mirror
without a scheduler is meaningless) and is independent of the existing
"Auto-mirror new starred repositories" toggle in GitHub settings. Together
they cover the full owned/starred matrix that the scheduler already
supports.
Plumbing: config-mapper.ts now round-trips autoMirror through the UI/DB
boundary, and ScheduleConfig in types/config.ts gets the matching field.
No schema or migration change — autoMirror was already in the zod schema.
The "Auto-mirror new starred repositories" checkbox in the GitHub settings
was a filter layered on top of scheduleConfig.autoMirror, which itself is
only settable via the AUTO_MIRROR_REPOS env var (no UI). So users who
checked the box saw their starred repos auto-imported but never mirrored.
Treat autoMirror and autoMirrorStarred as independent triggers in the
scheduler: autoMirror covers owned (and self-starred) repos, autoMirrorStarred
covers repos starred from other owners. Either flag on its own is enough
to enter the auto-mirror phase, and the filter scopes the work accordingly.
Also normalize the owner comparison to lowercase since GitHub usernames are
case-insensitive — previously a self-starred repo whose stored owner casing
differed from the configured owner would be misclassified as a third-party
star.
Behavior change worth flagging in release notes: anyone who currently has
the starred checkbox on (broken state) will start getting starred repos
mirrored on upgrade. AUTO_MIRROR_REPOS=true users see no change.
* fix: hoist migrateSucceeded above try so catch can update DB on failure (fixes#268)
`let migrateSucceeded` was declared inside the try block of
mirrorGithubRepoToGitea and mirrorGitHubRepoToGiteaOrg, but the catch
block referenced it. Block-scoping made it invisible to catch, so any
error inside the try (network timeout, transient 5xx, etc.) crashed the
catch with `ReferenceError: migrateSucceeded is not defined` before
reaching the DB update that marks the repo "failed". Result: repos
stuck in "mirroring" forever with no entry in the activity log.
Hoisting the declaration above the try restores the intended behavior:
catch updates the repo to failed, clears mirroredLocation when migrate
hadn't succeeded, writes a failed activity-log entry, and re-throws
with the original error message preserved.
TypeScript was flagging this as "Cannot find name 'migrateSucceeded'"
but esbuild stripped the types during build, so the bug shipped.
* test: replace integration test with structural source check (#268)
The behavioral version of this regression test passed locally but
failed in CI because of mock.module pollution between files: bun's
mock.module is process-wide, so my mock for @/lib/gitea-enhanced
leaked into gitea-enhanced.test.ts (its real-module assertions saw
my null-returning mocks), and gitea-enhanced.test.ts's own
@/lib/http-client mock could supersede mine depending on file
discovery order, causing my mirrorGithubRepoToGitea call to not
throw at all in CI.
Replace with a structural assertion that reads gitea.ts and verifies
`let migrateSucceeded` is declared before the outermost try in both
mirrorGithubRepoToGitea and mirrorGitHubRepoToGiteaOrg. Verified the
new test fails on the pre-fix source with a clear error message
pointing to issue #268, and passes on the fixed source.
Forgejo < 15.0.0 silently discards auth_username/auth_password sent to
/api/v1/repos/migrate, causing subsequent pull-mirror sync of private repos
to fail with `terminal prompts disabled`. Fix landed upstream in Forgejo
v15.0.0 via codeberg.org/forgejo/forgejo/pulls/11909 and was not backported
to v12/v13/v14.
Test-connection endpoint now also probes /api/v1/version, detects Forgejo
via the `+gitea-` suffix, and surfaces a warning Alert in the Gitea config
form when the connected server reports a major version below 15.
The useConfigStatus hook treated `githubConfig.username` and
`giteaConfig.username` as required for the dashboard to render. In
practice neither is required at runtime — the GitHub token is
self-authenticating via listForAuthenticatedUser, and a Gitea username
isn't needed under single-org or flat mirror strategies.
Users who configured via env vars without GITHUB_USERNAME / GITEA_USERNAME
set (or who left those blank in the form, which is only client-side
`required`) ended up with empty strings in their config row. Mirroring
ran fine — tokens alone are sufficient — but the dashboard refused to
fetch and rendered all zeros because useConfigStatus failed the gate.
Drop the username checks from the gate. The `githubOwner` field is still
exported for consumers that want to display an owner; only the gate is
relaxed. Cache-hit and fresh-fetch branches both updated.
Multiple "select from configs where userId" queries had no ORDER BY,
so when a user's database accidentally contained more than one config
row for the same user (e.g. from an env-loader insert path or a partial
default-config create), SQLite returned a non-deterministic row.
In the reported case this caused /api/config to hand back an empty stub
while /api/dashboard's repo/org counts came from the populated active
row. The dashboard's useConfigStatus hook then saw missing username/
token, treated config as incomplete, and never fetched dashboard data —
the UI rendered with all zeros even though 868 repos were sitting in
the database, mirroring fine in the background.
Add `ORDER BY isActive DESC, updatedAt DESC` before LIMIT 1 to every
"fetch the user's config" query so the active and most-recently-updated
row consistently wins. Also order env-config-loader's first-user pick
by createdAt for deterministic behavior across restarts.
Already-safe call sites that explicitly filter on isActive=true or
iterate all active configs (cleanup/scheduler/repositories/orgs/cleanup
trigger/sync-organization) are left unchanged.
Updates the mirror-repo test mock to match the new orderBy().limit()
chain.
Closes#271
* fix: honor GH_API_URL across all Octokit call sites
Six Octokit call sites constructed `new Octokit(...)` directly instead of
going through `createGitHubClient()`, so `GH_API_URL` (and the
`GITHUB_API_URL` fallback) only applied to the handful of flows that used
the helper. For GHES / GHEC-with-data-residency users this surfaced most
visibly as the "Test Connection" button hitting `api.github.com/user`
and failing with 401 even when `GH_API_URL` was set correctly (#269).
Route everything through `createGitHubClient()`:
- src/pages/api/github/test-connection.ts (the reported failure)
- src/pages/api/sync/repository.ts (public-repo sync)
- src/lib/gitea-enhanced.ts (force-push detection + metadata octokit)
- src/lib/scheduler-service.ts (auto-discovery, auto-mirror, auto-start)
- src/tests/test-metadata-mirroring.ts (dev harness, for consistency)
Side benefit: scheduler + sync paths now also get throttling, rate-limit
tracking, and the standard User-Agent, which they were missing.
`createGitHubClient`'s `token` parameter is made optional so the
public-repo sync path (`new Octokit()` with no auth) can keep working.
Fixes#269
* fix: address review findings
- scheduler: pass config.githubConfig?.owner (the real DB field) instead
of ?.username, which doesn't exist on the DB row and was silently
resolving to undefined — matches every other DB-reading call site.
- sync/repository.ts: revert to bare Octokit for the unauthenticated
public-repo lookup to preserve fast-fail on the 60 req/hr limit.
Still reads GH_API_URL / GITHUB_API_URL inline so GHES / GHEC
data-residency users benefit. The throttling plugin's retry-with-
backoff is wrong UX for a one-shot button click.
- github.ts: revert createGitHubClient token back to required (no
remaining callers pass undefined after the above).
- gitea-enhanced.ts: make the leftover Octokit import type-only.
- test-connection.test.ts: replace mid-test mock.module re-call with a
mutable stub reference — safer against ESM live-binding semantics.
- Add metadata state guards for issues/PRs in syncGiteaRepoEnhanced to
prevent re-mirroring on every sync (fixes#262)
- Fix httpPut → httpPatch for Gitea release updates (HTTP 405 error)
- Add paginated release retention cleanup to enforce releaseLimit by
deleting the oldest excess releases (fixes#264)
- Always send auth credentials during migration for all repos, not just
private ones, preventing "terminal prompts disabled" on Forgejo (fixes#263)
- Add missing service: "git" to org migration payload
- Make buildGithubSourceAuthPayload return {} instead of throwing when
token is missing, allowing unconditional use
- Preserve existing backup settings in env-config-loader so UI-configured
backup strategy and retention values survive restart (fixes#267)
Closes#262, #263, #264, #267
Update README, ENVIRONMENT_VARIABLES.md, and advanced docs page to
explicitly state that BETTER_AUTH_URL and PUBLIC_BETTER_AUTH_URL must be
origin only (scheme + host). The BASE_URL path prefix is applied
automatically — any path accidentally included is stripped.
* feat: add custom sync start time scheduling
* Updated UI
* docs: add updated issue 240 UI screenshot
* fix: improve schedule UI with client-side next run calc and timezone handling
- Compute next scheduled run client-side via useMemo to avoid permanent
"Calculating..." state when server hasn't set nextRun yet
- Default to browser timezone when enabling syncing (not UTC)
- Show actual saved timezone in badge, use it consistently in all handlers
- Match time input background to select trigger in dark mode
- Add clock icon to time picker with hidden native indicator
* feat: add notification system with Ntfy.sh and Apprise providers (#231)
Add push notification support for mirror job events with two providers:
- Ntfy.sh: direct HTTP POST to ntfy topics with priority/tag support
- Apprise API: aggregator gateway supporting 100+ notification services
Includes database migration (0010), settings UI tab, test endpoint,
auto-save integration, token encryption, and comprehensive tests.
Notifications are fire-and-forget and never block the mirror flow.
* fix: address review findings for notification system
- Fix silent catch in GET handler that returned ciphertext to UI,
causing double-encryption on next save. Now clears token to ""
on decryption failure instead.
- Add Zod schema validation to test notification endpoint, following
project API route pattern guidelines.
- Mark notifyOnNewRepo toggle as "coming soon" with disabled state,
since the backend doesn't yet emit new_repo events. The schema
and type support is in place for when it's implemented.
* fix notification gating and config validation
* trim sync notification details
* fix: improve reverse proxy support for subdomain deployments (#63)
- Add X-Accel-Buffering: no header to SSE endpoint to prevent Nginx
from buffering the event stream
- Auto-detect trusted origin from Host/X-Forwarded-* request headers
so the app works behind a proxy without manual env var configuration
- Add prominent reverse proxy documentation to advanced docs page
explaining BETTER_AUTH_URL, PUBLIC_BETTER_AUTH_URL, and
BETTER_AUTH_TRUSTED_ORIGINS are mandatory for proxy deployments
- Add reverse proxy env var comments and entries to both
docker-compose.yml and docker-compose.alt.yml
- Add dedicated reverse proxy configuration section to .env.example
* fix: address review findings for reverse proxy origin detection
- Fix x-forwarded-proto multi-value handling: take first value only
and validate it is "http" or "https" before using
- Update comment to accurately describe auto-detection scope: helps
with per-request CSRF checks but not callback URL validation
- Restore startup logging of static trusted origins for debugging
* fix: handle multi-value x-forwarded-host in chained proxy setups
x-forwarded-host can be comma-separated (e.g. "proxy1.example.com,
proxy2.example.com") in chained proxy setups. Take only the first
value, matching the same handling already applied to x-forwarded-proto.
* test: add unit tests for reverse proxy origin detection
Extract resolveTrustedOrigins into a testable exported function and
add 11 tests covering:
- Default localhost origins
- BETTER_AUTH_URL and BETTER_AUTH_TRUSTED_ORIGINS env vars
- Invalid URL handling
- Auto-detection from x-forwarded-host + x-forwarded-proto
- Multi-value header handling (chained proxy setups)
- Invalid proto rejection (only http/https allowed)
- Deduplication
- Fallback to host header when x-forwarded-host absent
* fix: prevent starred repo name collisions during concurrent mirroring (#95)
When multiple starred repos share the same short name (e.g. alice/dotfiles
and bob/dotfiles), concurrent batch mirroring could cause 409 Conflict
errors because generateUniqueRepoName only checked Gitea via HTTP, missing
repos that were claimed in the local DB but not yet created remotely.
Three fixes:
- Add DB-level check in generateUniqueRepoName so it queries the local
repositories table for existing mirroredLocation claims, preventing two
concurrent jobs from picking the same target name.
- Clear mirroredLocation on failed mirror so a failed repo doesn't falsely
hold a location that was never successfully created, which would block
retries and confuse the uniqueness check.
- Extract isMirroredLocationClaimedInDb helper for the DB lookup, using
ne() to exclude the current repo's own record from the collision check.
* fix: address review findings for starred repo name collision fix
- Make generateUniqueRepoName immediately claim name by writing
mirroredLocation to DB, closing the TOCTOU race window between
name selection and the later status="mirroring" DB update
- Add fullName validation guard (must contain "/")
- Make isMirroredLocationClaimedInDb fail-closed (return true on
DB error) to be conservative about preventing collisions
- Scope mirroredLocation clear on failure to starred repos only,
preserving it for non-starred repos that may have partially
created in Gitea and need the location for recovery
* fix: address P1/P2 review findings for starred repo name collision
P1a: Remove early name claiming from generateUniqueRepoName to prevent
stale claims on early return paths. The function now only checks
availability — the actual claim happens at the status="mirroring" DB
write (after both idempotency checks), which is protected by a new
unique partial index.
P1b: Add unique partial index on (userId, mirroredLocation) WHERE
mirroredLocation != '' via migration 0010. This enforces atomicity at
the DB level: if two concurrent workers try to claim the same name,
the second gets a constraint violation rather than silently colliding.
P2: Only clear mirroredLocation on failure if the Gitea migrate call
itself failed (migrateSucceeded flag). If migrate succeeded but
metadata mirroring failed, preserve the location since the repo
physically exists in Gitea and we need it for recovery/retry.
* fix: prevent excessive disk usage from repo backups (#234)
Legacy configs with backupBeforeSync: true but no explicit backupStrategy
silently resolved to "always", creating full git bundles on every sync
cycle. This caused repo-backups to grow to 17GB+ for users with many
repositories.
Changes:
- Fix resolveBackupStrategy to map backupBeforeSync: true → "on-force-push"
instead of "always", so legacy configs only backup when force-push is detected
- Fix config mapper to always set backupStrategy explicitly ("on-force-push")
preventing the backward-compat fallback from triggering
- Lower default backupRetentionCount from 20 to 5 bundles per repo
- Add time-based retention (backupRetentionDays, default 30 days) alongside
count-based retention, with safety net to always keep at least 1 bundle
- Add "high disk usage" warning on "Always Backup" UI option
- Update docs and tests to reflect new defaults and behavior
* fix: preserve legacy backupBeforeSync:false on UI round-trip and expose retention days
P1: mapDbToUiConfig now checks backupBeforeSync === false before
defaulting backupStrategy, preventing legacy "disabled" configs from
silently becoming "on-force-push" after any auto-save round-trip.
P3: Added "Snapshot retention days" input field to the backup settings
UI, matching the documented setting in FORCE_PUSH_PROTECTION.md.
SQLite rejects ALTER TABLE ADD COLUMN with expression defaults like
DEFAULT (unixepoch()), which Drizzle-kit generated for the imported_at
column. This broke upgrades from v3.12.x to v3.13.0 (#228, #229).
Changes:
- Rewrite migration 0009 using table-recreation pattern (CREATE, INSERT
SELECT, DROP, RENAME) instead of ALTER TABLE
- Add migration validation script with SQLite-specific lint rules that
catch known invalid patterns before they ship
- Add upgrade-path testing with seeded data and verification fixtures
- Add runtime repair for users whose migration record may be stale
- Add explicit migration validation step to CI workflow
Fixes#228Fixes#229
* feat: add autoMirrorStarred toggle for selective starred repo mirroring (#205)
Add `githubConfig.autoMirrorStarred` (default: false) to control whether
starred repos are included in automatic mirroring operations. Manual
per-repo actions always work regardless of this toggle.
Bug fixes:
- Cleanup service no longer orphans starred repos when includeStarred is
disabled (prevents data loss)
- First-boot auto-start now gates initial mirror behind autoMirror config
(previously mirrored everything unconditionally)
- "Mirror All" button now respects autoMirrorStarred setting
- Bulk mirror and getAvailableActions now include pending-approval status
Changes span schema, config mapping, env loader, scheduler, cleanup
service, UI settings toggle, and repository components.
* fix: log activity when repos are auto-imported during scheduled sync
Auto-discovered repositories (including newly starred ones) were inserted
into the database without creating activity log entries, so they appeared
in the dashboard but not in the activity log.
* ci: set 10-minute timeout on all CI jobs
* feat: smart force-push protection with backup strategies (#187)
Replace blunt `backupBeforeSync` boolean with `backupStrategy` enum
offering four modes: disabled, always, on-force-push (default), and
block-on-force-push. This dramatically reduces backup storage for large
mirror collections by only creating snapshots when force-pushes are
actually detected.
Detection works by comparing branch SHAs between Gitea and GitHub APIs
before each sync — no git cloning required. Fail-open design ensures
detection errors never block sync.
Key changes:
- Add force-push detection module (branch SHA comparison via APIs)
- Add backup strategy resolver with backward-compat migration
- Add pending-approval repo status with approve/dismiss UI + API
- Add block-on-force-push mode requiring manual approval
- Fix checkAncestry to only treat 404 as confirmed force-push
(transient errors skip branch instead of false-positive blocking)
- Fix approve-sync to bypass detection gate (skipForcePushDetection)
- Fix backup execution to not be hard-gated by deprecated flag
- Persist backupStrategy through config-mapper round-trip
* fix: resolve four bugs in smart force-push protection
P0: Approve flow re-blocks itself — approve-sync now calls
syncGiteaRepoEnhanced with skipForcePushDetection: true so the
detection+block gate is bypassed on approved syncs.
P1: backupStrategy not persisted — added to both directions of the
config-mapper. Don't inject a default in the mapper; let
resolveBackupStrategy handle fallback so legacy backupBeforeSync
still works for E2E tests and existing configs.
P1: Backup hard-gated by deprecated backupBeforeSync — added force
flag to createPreSyncBundleBackup; strategy-driven callers and
approve-sync pass force: true to bypass the legacy guard.
P1: checkAncestry false positives — now only returns false for
404/422 (confirmed force-push). Transient errors (rate limits, 500s)
are rethrown so detectForcePush skips that branch (fail-open).
* test(e2e): migrate backup tests from backupBeforeSync to backupStrategy
Update E2E tests to use the new backupStrategy enum ("always",
"disabled") instead of the deprecated backupBeforeSync boolean.
* docs: add backup strategy UI screenshot
* refactor(ui): move Destructive Update Protection to GitHub config tab
Relocates the backup strategy section from GiteaConfigForm to
GitHubConfigForm since it protects against GitHub-side force-pushes.
Adds ShieldAlert icon to match other section header patterns.
* docs: add force-push protection documentation and Beta badge
Add docs/FORCE_PUSH_PROTECTION.md covering detection mechanism,
backup strategies, API usage, and troubleshooting. Link it from
README features list and support section. Mark the feature as Beta
in the UI with an outline badge.
* fix(ui): match Beta badge style to Git LFS badge
* fix(nix): ensure absolute bundle path in pre-sync backup (#203)
Use path.resolve() instead of conditional path.isAbsolute() check to
guarantee bundlePath is always absolute before passing to git -C. On
NixOS, relative paths were interpreted relative to the temp mirror
clone directory, causing "No such file or directory" errors.
Closes#203
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(nix): ensure absolute bundle path in pre-sync backup (#203)
Use path.resolve() instead of conditional path.isAbsolute() check to
guarantee bundlePath is always absolute before passing to git -C. On
NixOS, relative paths were interpreted relative to the temp mirror
clone directory, causing "No such file or directory" errors.
Extract resolveBackupPaths() for testability. Bump version to 3.10.1.
Closes#203
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* ci: drop macos matrix and only run nix build on main/tags
- Remove macos-latest from Nix CI matrix (ubuntu-only)
- Only run `nix build` on main branch and version tags, skip on PRs
- `nix flake check` still runs on all PRs for validation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
ARUNAVO RAY