Commit Graph

57 Commits

Author SHA1 Message Date
ARUNAVO RAY dff3cafb5e fix(config): persist Name Collision Strategy (starredDuplicateStrategy) (#326) (#328)
The "Name collision strategy" dropdown (starredDuplicateStrategy) never
persisted: the field was absent from both directions of the UI<->DB config
mapper. On save, mapUiToDbConfig dropped it before the DB write; on load,
mapDbToUiConfig never read it, so the UI reset to the "suffix" (repo-owner)
default. Mirror logic in gitea.ts then read undefined and also defaulted to
suffix — so repos really were created with that pattern regardless of the
user's choice. It has been broken since the field was introduced.

- Map starredDuplicateStrategy in mapUiToDbConfig and mapDbToUiConfig
- Add STARRED_DUPLICATE_STRATEGY env var for parity (reporter could not
  work around it via compose because no env var existed) + docs
- Round-trip tests covering save, load, and the missing-field default
2026-06-19 08:43:27 +05:30
ARUNAVO RAY 6ca7c0eec0 feat(github): add organization allowlist to mirror only selected orgs (#327)
Repository discovery requested the `organization_member` affiliation
unconditionally, so repos from every org a user belongs to were imported —
even orgs they never explicitly added. `skipPersonalRepos` only dropped
user-owned repos and left org repos unfiltered, which surprised users who
expected "only mirror org repos" to mean "only the orgs I chose" (reported
on #304).

Wire up the previously-dormant `includeOrganizations` config field as an
opt-in allowlist: when non-empty, only repos owned by the listed
organizations are imported. Empty = all org repos (backward-compatible).
Owned and collaborator repos are never restricted, so it composes cleanly
with `skipPersonalRepos`.

- Filter org repos by the allowlist in getGithubRepositories
- Add includeAllOrgsOverride so the cleanup service bypasses the allowlist
  and never false-orphans a previously-mirrored repo from an org the user
  later removes from the list
- UI control under Filtering & Behavior; INCLUDE_ORGANIZATIONS env var
- Case-insensitive dedup/trim in the UI<->DB mapper round-trip
- 7 unit tests covering the filter, composition, and the cleanup override
2026-06-19 08:42:17 +05:30
ARUNAVO RAY 0b6b6b76bf feat(github): add skipPersonalRepos toggle to mirror only org repos (#304) (#320)
- Add `skipPersonalRepos: z.boolean().default(false)` to githubConfigSchema
- Filter out user-owned repos in getGithubRepositories when flag is true
- Wire ONLY_MIRROR_ORGS env var to skipPersonalRepos in env-config-loader
- Add checkbox UI in GitHubMirrorSettings Filtering & Behavior section
- Round-trip skipPersonalRepos through config-mapper (UI ↔ DB)
- Add skipPersonalRepos to AdvancedOptions TypeScript type
- Mark include/exclude arrays in configSchema as unused/reserved
- Update ENVIRONMENT_VARIABLES.md to document ONLY_MIRROR_ORGS effect
2026-06-13 08:00:50 +05:30
ARUNAVO RAY 66e3284898 fix(sso): repair SSO login bounce + migrate to @better-auth/oauth-provider (#307)
Resolves #306. SSO sign-in via OIDC (Authentik / Keycloak / etc.) now links the
SSO identity to an existing email/password admin instead of bouncing to /login
with `?error=UNKNOWN`. Account-linking is gated on the operator-supplied
**Domain** field — cross-domain claims from a compromised IdP are refused.

Also bundles the deprecated `oidcProvider` → `@better-auth/oauth-provider`
migration. **Operators using the OAuth-provider feature must rotate registered
client secrets after upgrade** (legacy plaintext → hashed storage; see the
0012 migration notes).

Verified end-to-end on the pr-307 image against a real Authentik instance:
SSO login lands on the dashboard, `accounts` table gets both `credential` and
`authentik` rows for the same user. See PR description for full details.
2026-06-02 11:40:54 +05:30
Arunavo Ray 384fbbbe10 docs: document Header / Forward Authentication setup
Header auth has been a working feature since v2.x but was missing from
SSO-OIDC-SETUP.md, leading users to think it was dropped in the v3
rewrite (see #29). Adds a dedicated section covering env-var config,
Authentik + Authelia examples, lookup order, verification, and the
must-strip-inbound-headers security checklist.
2026-05-25 10:37:39 +05:30
ARUNAVO RAY 088467a57d feat: add option to exclude collaborator repos from import (closes #279) (#283)
GitHub's listForAuthenticatedUser defaults to returning every repo the
user has access to (owner + collaborator + organization_member), which
imports a lot of noise for users who only want their own repos.

Adds an `includeCollaboratorRepos` toggle, defaulting to true to preserve
existing behavior. When disabled, the affiliation filter scopes the API
call to "owner" only.

The cleanup service overrides the filter to always include collaborator
repos when computing the "what's still on GitHub" list. Without this,
toggling the option off would mark previously-mirrored collab repos as
orphaned and archive/delete them from Gitea.

Wired through the schema, both UI<->DB mappers, the env-config loader
(with new INCLUDE_COLLABORATOR_REPOS env var), and the settings UI.
2026-05-04 14:00:10 +05:30
Arunavo Ray 8cb8fd6fe1 docs: document GH_API_URL for GitHub Enterprise and SERVER_CERT_PATH/SERVER_KEY_PATH for native HTTPS
- README + env reference + .env.example now cover using GH_API_URL to
  target GitHub Enterprise Server or GHEC with data residency.
- Env reference + .env.example now cover SERVER_CERT_PATH and
  SERVER_KEY_PATH, which @astrojs/node reads at runtime to terminate
  TLS directly without a reverse proxy.

Closes #269
Closes #272
2026-04-20 09:30:08 +05:30
Arunavo Ray 8fac30fc02 docs: clarify BETTER_AUTH_URL should be origin only, not include base path
Update README, ENVIRONMENT_VARIABLES.md, and advanced docs page to
explicitly state that BETTER_AUTH_URL and PUBLIC_BETTER_AUTH_URL must be
origin only (scheme + host). The BASE_URL path prefix is applied
automatically — any path accidentally included is stripped.
2026-04-09 20:11:00 +05:30
ARUNAVO RAY 01a3b08dac feat: support reverse proxy path prefix deployments (#257)
* feat: support reverse proxy path prefixes

* fix: respect BASE_URL in SAML callback fallback

* fix: make BASE_URL runtime configurable
2026-04-09 12:32:59 +05:30
ARUNAVO RAY 6f2e0cbca0 Add GitHub starred-list filtering with searchable selector (#247)
* feat: add starred list filtering and selector UI

* docs: add starred lists UI screenshot

* lib: improve starred list name matching
2026-03-24 07:33:46 +05:30
ARUNAVO RAY 5ea2abff85 feat: custom sync start time and frequency scheduling (#241)
* feat: add custom sync start time scheduling

* Updated UI

* docs: add updated issue 240 UI screenshot

* fix: improve schedule UI with client-side next run calc and timezone handling

- Compute next scheduled run client-side via useMemo to avoid permanent
  "Calculating..." state when server hasn't set nextRun yet
- Default to browser timezone when enabling syncing (not UTC)
- Show actual saved timezone in badge, use it consistently in all handlers
- Match time input background to select trigger in dark mode
- Add clock icon to time picker with hidden native indicator
2026-03-19 00:58:10 +05:30
ARUNAVO RAY 5d2462e5a0 feat: add notification system with Ntfy.sh and Apprise support (#238)
* feat: add notification system with Ntfy.sh and Apprise providers (#231)

Add push notification support for mirror job events with two providers:

- Ntfy.sh: direct HTTP POST to ntfy topics with priority/tag support
- Apprise API: aggregator gateway supporting 100+ notification services

Includes database migration (0010), settings UI tab, test endpoint,
auto-save integration, token encryption, and comprehensive tests.
Notifications are fire-and-forget and never block the mirror flow.

* fix: address review findings for notification system

- Fix silent catch in GET handler that returned ciphertext to UI,
  causing double-encryption on next save. Now clears token to ""
  on decryption failure instead.
- Add Zod schema validation to test notification endpoint, following
  project API route pattern guidelines.
- Mark notifyOnNewRepo toggle as "coming soon" with disabled state,
  since the backend doesn't yet emit new_repo events. The schema
  and type support is in place for when it's implemented.

* fix notification gating and config validation

* trim sync notification details
2026-03-18 18:36:51 +05:30
ARUNAVO RAY ddd071f7e5 fix: prevent excessive disk usage from repo backups (#235)
* fix: prevent excessive disk usage from repo backups (#234)

Legacy configs with backupBeforeSync: true but no explicit backupStrategy
silently resolved to "always", creating full git bundles on every sync
cycle. This caused repo-backups to grow to 17GB+ for users with many
repositories.

Changes:
- Fix resolveBackupStrategy to map backupBeforeSync: true → "on-force-push"
  instead of "always", so legacy configs only backup when force-push is detected
- Fix config mapper to always set backupStrategy explicitly ("on-force-push")
  preventing the backward-compat fallback from triggering
- Lower default backupRetentionCount from 20 to 5 bundles per repo
- Add time-based retention (backupRetentionDays, default 30 days) alongside
  count-based retention, with safety net to always keep at least 1 bundle
- Add "high disk usage" warning on "Always Backup" UI option
- Update docs and tests to reflect new defaults and behavior

* fix: preserve legacy backupBeforeSync:false on UI round-trip and expose retention days

P1: mapDbToUiConfig now checks backupBeforeSync === false before
defaulting backupStrategy, preventing legacy "disabled" configs from
silently becoming "on-force-push" after any auto-save round-trip.

P3: Added "Snapshot retention days" input field to the backup settings
UI, matching the documented setting in FORCE_PUSH_PROTECTION.md.
2026-03-18 15:05:00 +05:30
ARUNAVO RAY ce365a706e ci: persist release version to main (#212) 2026-03-05 09:55:59 +05:30
ARUNAVO RAY be7daac5fb ci: automate release version from tag (#211) 2026-03-05 09:34:49 +05:30
ARUNAVO RAY d0693206c3 feat: selective starred repo mirroring with autoMirrorStarred toggle (#208)
* feat: add autoMirrorStarred toggle for selective starred repo mirroring (#205)

Add `githubConfig.autoMirrorStarred` (default: false) to control whether
starred repos are included in automatic mirroring operations. Manual
per-repo actions always work regardless of this toggle.

Bug fixes:
- Cleanup service no longer orphans starred repos when includeStarred is
  disabled (prevents data loss)
- First-boot auto-start now gates initial mirror behind autoMirror config
  (previously mirrored everything unconditionally)
- "Mirror All" button now respects autoMirrorStarred setting
- Bulk mirror and getAvailableActions now include pending-approval status

Changes span schema, config mapping, env loader, scheduler, cleanup
service, UI settings toggle, and repository components.

* fix: log activity when repos are auto-imported during scheduled sync

Auto-discovered repositories (including newly starred ones) were inserted
into the database without creating activity log entries, so they appeared
in the dashboard but not in the activity log.

* ci: set 10-minute timeout on all CI jobs
2026-03-04 08:22:44 +05:30
ARUNAVO RAY 98da7065e0 feat: smart force-push protection with backup strategies (#206)
* feat: smart force-push protection with backup strategies (#187)

Replace blunt `backupBeforeSync` boolean with `backupStrategy` enum
offering four modes: disabled, always, on-force-push (default), and
block-on-force-push. This dramatically reduces backup storage for large
mirror collections by only creating snapshots when force-pushes are
actually detected.

Detection works by comparing branch SHAs between Gitea and GitHub APIs
before each sync — no git cloning required. Fail-open design ensures
detection errors never block sync.

Key changes:
- Add force-push detection module (branch SHA comparison via APIs)
- Add backup strategy resolver with backward-compat migration
- Add pending-approval repo status with approve/dismiss UI + API
- Add block-on-force-push mode requiring manual approval
- Fix checkAncestry to only treat 404 as confirmed force-push
  (transient errors skip branch instead of false-positive blocking)
- Fix approve-sync to bypass detection gate (skipForcePushDetection)
- Fix backup execution to not be hard-gated by deprecated flag
- Persist backupStrategy through config-mapper round-trip

* fix: resolve four bugs in smart force-push protection

P0: Approve flow re-blocks itself — approve-sync now calls
syncGiteaRepoEnhanced with skipForcePushDetection: true so the
detection+block gate is bypassed on approved syncs.

P1: backupStrategy not persisted — added to both directions of the
config-mapper. Don't inject a default in the mapper; let
resolveBackupStrategy handle fallback so legacy backupBeforeSync
still works for E2E tests and existing configs.

P1: Backup hard-gated by deprecated backupBeforeSync — added force
flag to createPreSyncBundleBackup; strategy-driven callers and
approve-sync pass force: true to bypass the legacy guard.

P1: checkAncestry false positives — now only returns false for
404/422 (confirmed force-push). Transient errors (rate limits, 500s)
are rethrown so detectForcePush skips that branch (fail-open).

* test(e2e): migrate backup tests from backupBeforeSync to backupStrategy

Update E2E tests to use the new backupStrategy enum ("always",
"disabled") instead of the deprecated backupBeforeSync boolean.

* docs: add backup strategy UI screenshot

* refactor(ui): move Destructive Update Protection to GitHub config tab

Relocates the backup strategy section from GiteaConfigForm to
GitHubConfigForm since it protects against GitHub-side force-pushes.
Adds ShieldAlert icon to match other section header patterns.

* docs: add force-push protection documentation and Beta badge

Add docs/FORCE_PUSH_PROTECTION.md covering detection mechanism,
backup strategies, API usage, and troubleshooting. Link it from
README features list and support section. Mark the feature as Beta
in the UI with an outline badge.

* fix(ui): match Beta badge style to Git LFS badge
2026-03-02 15:48:59 +05:30
ARUNAVO RAY be46cfdffa feat: add target organization to Add Repository dialog (#202)
* feat: add target organization field to Add Repository dialog

Allow users to specify a destination Gitea organization when adding a
single repository, instead of relying solely on the default mirror
strategy. The field is optional — when left empty, the existing strategy
logic applies as before.

Closes #200

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add screenshot of target organization field in Add Repository dialog

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 07:55:27 +05:30
ARUNAVO RAY 89a6372565 nix: fix runtime wrapper paths and startup script packaging (#194)
* nix: fix flake module and runtime scripts

* docs: refresh readme and docs links/examples
2026-02-26 10:59:56 +05:30
Arunavo Ray ef13fefb69 Add optional external Gitea URL for UI links 2026-02-24 10:33:37 +05:30
Tobeas Arren f4d391b240 Allow starred repos to be mirrored preserving structure 2026-02-14 13:08:41 +01:00
Arunavo Ray 3993d679e6 fix: replace Cachix with Magic Nix Cache in CI workflow
- Use DeterminateSystems/nix-installer-action for Nix installation
- Use DeterminateSystems/magic-nix-cache-action for caching (free, no setup)
- Update documentation to remove Cachix references
- Add nix branch to CI triggers
2025-12-17 10:30:07 +05:30
Arunavo Ray 0d63fd4dae Added more docs 2025-10-31 09:22:55 +05:30
Arunavo Ray 109958342d updated docs 2025-10-31 09:17:28 +05:30
Arunavo Ray 491546a97c added basic nix pack 2025-10-31 09:00:18 +05:30
Arunavo Ray 395e71164f Added basic docs on SSO/OIDC 2025-10-26 19:52:44 +05:30
Arunavo Ray 025df12bef Set defaults to 3 and 5 for Issue and PR concurrency 2025-10-24 08:39:52 +05:30
Arunavo Ray 4d75d3514f docs: document sequential metadata defaults 2025-10-24 07:39:08 +05:30
Arunavo Ray fd5e68c1d4 docs: update development workflow and documentation index
Updated development documentation to reflect current project structure
and simplified setup process.

Changes:
- DEVELOPMENT_WORKFLOW.md: Updated repository URL, simplified setup steps,
  improved project structure documentation, and clarified command descriptions
- README.md: Reorganized as a concise index of available guides, removed
  redundant content now covered in main README and in-app help
- SHUTDOWN_PROCESS.md: Removed (content consolidated into GRACEFUL_SHUTDOWN.md)

These updates make the documentation more accurate and easier to navigate
for new contributors.
2025-10-23 05:10:42 +05:30
ARUNAVO RAY 847e94ca28 Merge pull request #111 from RayLabsHQ/www-seo
Writing a few guides on the application
2025-10-22 23:14:26 +05:30
Arunavo Ray df644be769 cleanup: improve orphaned repo handling 2025-10-22 14:41:54 +05:30
Arunavo Ray 21e2f4717c Added basic use cases pages 2025-10-03 12:35:04 +05:30
Arunavo Ray b8dea1ee9c removed unused docs 2025-10-03 12:21:47 +05:30
Arunavo Ray 5add8766a4 fix(scheduler,config): preserve ENV schedule; add AUTO_MIRROR_REPOS auto-mirroring
- Prevent Automation UI from overriding schedule:
      - mapDbScheduleToUi now parses intervals robustly (cron/duration/seconds) via parseInterval
      - mapUiScheduleToDb merges with existing config and stores interval as seconds (no lossy cron conversion)
      - /api/config passes existing scheduleConfig to preserve ENV-sourced values
      - schedule-sync endpoint uses parseInterval for nextRun calculation
  - Add AUTO_MIRROR_REPOS support and scheduled auto-mirror phase:
      - scheduleConfig schema includes autoImport and autoMirror
      - env-config-loader reads AUTO_MIRROR_REPOS and carries through to DB
      - scheduler auto-mirrors imported/pending/failed repos when autoMirror is enabled before regular sync
      - docker-compose and ENV docs updated with AUTO_MIRROR_REPOS
  - Tests pass and build succeeds
2025-09-14 08:31:31 +05:30
Arunavo Ray c4b353aae8 Added docs around scheduling using corn 2025-09-07 16:51:51 +05:30
Arunavo Ray fab4efd93a Auto-start on boot 2025-09-07 16:29:23 +05:30
Arunavo Ray 9f21cd6b1a Addressing concerns of Issue #85 and #86 2025-09-07 15:25:48 +05:30
Arunavo Ray 502796371f Attempt to address #84 2025-09-07 13:55:20 +05:30
Arunavo Ray a188869cae "Automatic Mirroring" changed to "Automatic Syncing" 2025-08-30 00:37:56 +05:30
Arunavo Ray 2ce4bb4373 update env doc 2025-08-29 20:43:49 +05:30
Arunavo Ray 598e81ff45 updated package location 2025-08-29 17:04:48 +05:30
Arunavo Ray 1b01a5e653 updated docs 2025-08-28 20:11:16 +05:30
Arunavo Ray e404490e75 added LFS ENV var 2025-08-28 09:26:23 +05:30
Arunavo Ray 067b5d8ccd updated handling of url's from ENV vars 2025-08-28 07:12:13 +05:30
Arunavo Ray 12ee065833 Docs updated | added some options 2025-08-27 21:43:36 +05:30
Arunavo Ray 698eb0b507 fix: Complete Issue #72 - Fix automatic mirroring and repository cleanup
Major fixes for Docker environment variable issues and cleanup functionality:

🔧 **Duration Parser & Scheduler Fixes**
- Add comprehensive duration parser supporting "8h", "30m", "24h" formats
- Fix GITEA_MIRROR_INTERVAL environment variable mapping to scheduler
- Auto-enable scheduler when GITEA_MIRROR_INTERVAL is set
- Improve scheduler logging to clarify timing behavior (from last run, not startup)

🧹 **Repository Cleanup Service**
- Complete repository cleanup service for orphaned repos (unstarred, deleted)
- Fix cleanup configuration logic - now works with CLEANUP_DELETE_IF_NOT_IN_GITHUB=true
- Auto-enable cleanup when deleteIfNotInGitHub is enabled
- Add manual cleanup trigger API endpoint (/api/cleanup/trigger)
- Support archive/delete actions with dry-run mode and protected repos

🐛 **Environment Variable Integration**
- Fix scheduler not recognizing GITEA_MIRROR_INTERVAL=8h
- Fix cleanup requiring both CLEANUP_DELETE_FROM_GITEA and CLEANUP_DELETE_IF_NOT_IN_GITHUB
- Auto-enable services when relevant environment variables are set
- Better error logging and debugging information

📚 **Documentation Updates**
- Update .env.example with auto-enabling behavior notes
- Update ENVIRONMENT_VARIABLES.md with clarified functionality
- Add comprehensive tests for duration parsing

This resolves the core issues where:
1. GITEA_MIRROR_INTERVAL=8h was not working for automatic mirroring
2. Repository cleanup was not working despite CLEANUP_DELETE_IF_NOT_IN_GITHUB=true
3. Users had no visibility into why scheduling/cleanup wasn't working

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-20 11:06:21 +05:30
Arunavo Ray df1738a44d feat: comprehensive environment variable support
- Added support for 60+ environment variables covering all configuration options
- Created detailed documentation in docs/ENVIRONMENT_VARIABLES.md with tables
- Fixed missing skipStarredIssues field in GitHub config
- Updated docker-compose files to reference environment variable documentation
- Updated README to link to the new environment variables documentation
- Environment variables now populate UI configuration automatically on Docker startup
- Preserves manual UI changes when environment variables are not set
- Includes support for mirror metadata, scheduling, cleanup, and authentication options

Fixes #69

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-09 11:48:42 +05:30
Arunavo Ray 1f6add5fff Updates to SSO Testing 2025-07-26 19:45:20 +05:30
Sulaiman Khan Ghori 8d7ca8dd8f Add GITHUB_EXCLUDED_ORGS support for organization filtering 2025-07-18 15:58:04 -07:00
Arunavo Ray d48981a8c4 Updated docs 2025-07-16 22:51:47 +05:30