Commit Graph

30 Commits

Author SHA1 Message Date
ARUNAVO RAY ddd071f7e5 fix: prevent excessive disk usage from repo backups (#235)
* fix: prevent excessive disk usage from repo backups (#234)

Legacy configs with backupBeforeSync: true but no explicit backupStrategy
silently resolved to "always", creating full git bundles on every sync
cycle. This caused repo-backups to grow to 17GB+ for users with many
repositories.

Changes:
- Fix resolveBackupStrategy to map backupBeforeSync: true → "on-force-push"
  instead of "always", so legacy configs only backup when force-push is detected
- Fix config mapper to always set backupStrategy explicitly ("on-force-push")
  preventing the backward-compat fallback from triggering
- Lower default backupRetentionCount from 20 to 5 bundles per repo
- Add time-based retention (backupRetentionDays, default 30 days) alongside
  count-based retention, with safety net to always keep at least 1 bundle
- Add "high disk usage" warning on "Always Backup" UI option
- Update docs and tests to reflect new defaults and behavior

* fix: preserve legacy backupBeforeSync:false on UI round-trip and expose retention days

P1: mapDbToUiConfig now checks backupBeforeSync === false before
defaulting backupStrategy, preventing legacy "disabled" configs from
silently becoming "on-force-push" after any auto-save round-trip.

P3: Added "Snapshot retention days" input field to the backup settings
UI, matching the documented setting in FORCE_PUSH_PROTECTION.md.
2026-03-18 15:05:00 +05:30
ARUNAVO RAY 6f53a3ed41 feat: add importedAt-based repository sorting (#226)
* repositories: add importedAt sorting

* repositories: use tanstack table for repo list
2026-03-15 08:52:45 +05:30
ARUNAVO RAY d0693206c3 feat: selective starred repo mirroring with autoMirrorStarred toggle (#208)
* feat: add autoMirrorStarred toggle for selective starred repo mirroring (#205)

Add `githubConfig.autoMirrorStarred` (default: false) to control whether
starred repos are included in automatic mirroring operations. Manual
per-repo actions always work regardless of this toggle.

Bug fixes:
- Cleanup service no longer orphans starred repos when includeStarred is
  disabled (prevents data loss)
- First-boot auto-start now gates initial mirror behind autoMirror config
  (previously mirrored everything unconditionally)
- "Mirror All" button now respects autoMirrorStarred setting
- Bulk mirror and getAvailableActions now include pending-approval status

Changes span schema, config mapping, env loader, scheduler, cleanup
service, UI settings toggle, and repository components.

* fix: log activity when repos are auto-imported during scheduled sync

Auto-discovered repositories (including newly starred ones) were inserted
into the database without creating activity log entries, so they appeared
in the dashboard but not in the activity log.

* ci: set 10-minute timeout on all CI jobs
2026-03-04 08:22:44 +05:30
ARUNAVO RAY 98da7065e0 feat: smart force-push protection with backup strategies (#206)
* feat: smart force-push protection with backup strategies (#187)

Replace blunt `backupBeforeSync` boolean with `backupStrategy` enum
offering four modes: disabled, always, on-force-push (default), and
block-on-force-push. This dramatically reduces backup storage for large
mirror collections by only creating snapshots when force-pushes are
actually detected.

Detection works by comparing branch SHAs between Gitea and GitHub APIs
before each sync — no git cloning required. Fail-open design ensures
detection errors never block sync.

Key changes:
- Add force-push detection module (branch SHA comparison via APIs)
- Add backup strategy resolver with backward-compat migration
- Add pending-approval repo status with approve/dismiss UI + API
- Add block-on-force-push mode requiring manual approval
- Fix checkAncestry to only treat 404 as confirmed force-push
  (transient errors skip branch instead of false-positive blocking)
- Fix approve-sync to bypass detection gate (skipForcePushDetection)
- Fix backup execution to not be hard-gated by deprecated flag
- Persist backupStrategy through config-mapper round-trip

* fix: resolve four bugs in smart force-push protection

P0: Approve flow re-blocks itself — approve-sync now calls
syncGiteaRepoEnhanced with skipForcePushDetection: true so the
detection+block gate is bypassed on approved syncs.

P1: backupStrategy not persisted — added to both directions of the
config-mapper. Don't inject a default in the mapper; let
resolveBackupStrategy handle fallback so legacy backupBeforeSync
still works for E2E tests and existing configs.

P1: Backup hard-gated by deprecated backupBeforeSync — added force
flag to createPreSyncBundleBackup; strategy-driven callers and
approve-sync pass force: true to bypass the legacy guard.

P1: checkAncestry false positives — now only returns false for
404/422 (confirmed force-push). Transient errors (rate limits, 500s)
are rethrown so detectForcePush skips that branch (fail-open).

* test(e2e): migrate backup tests from backupBeforeSync to backupStrategy

Update E2E tests to use the new backupStrategy enum ("always",
"disabled") instead of the deprecated backupBeforeSync boolean.

* docs: add backup strategy UI screenshot

* refactor(ui): move Destructive Update Protection to GitHub config tab

Relocates the backup strategy section from GiteaConfigForm to
GitHubConfigForm since it protects against GitHub-side force-pushes.
Adds ShieldAlert icon to match other section header patterns.

* docs: add force-push protection documentation and Beta badge

Add docs/FORCE_PUSH_PROTECTION.md covering detection mechanism,
backup strategies, API usage, and troubleshooting. Link it from
README features list and support section. Mark the feature as Beta
in the UI with an outline badge.

* fix(ui): match Beta badge style to Git LFS badge
2026-03-02 15:48:59 +05:30
ARUNAVO RAY be46cfdffa feat: add target organization to Add Repository dialog (#202)
* feat: add target organization field to Add Repository dialog

Allow users to specify a destination Gitea organization when adding a
single repository, instead of relying solely on the default mirror
strategy. The field is optional — when left empty, the existing strategy
logic applies as before.

Closes #200

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add screenshot of target organization field in Add Repository dialog

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 07:55:27 +05:30
ARUNAVO RAY 08da526ddd fix(github): keep disabled repos from cleanup while skipping new imports (#191)
* fix: preserve disabled repos while skipping new imports

* ci: upgrade bun to 1.3.6 for test workflow
2026-02-26 10:19:28 +05:30
ARUNAVO RAY 2395e14382 Add pre-sync snapshot protection for mirror rewrites (#190)
* add pre-sync snapshot protection

* stabilize test module mocks

* fix cross-test gitea mock exports

* fix gitea mock strategy behavior
2026-02-26 10:13:13 +05:30
Arunavo Ray ef13fefb69 Add optional external Gitea URL for UI links 2026-02-24 10:33:37 +05:30
Arunavo Ray c34056555f Add bulk re-run metadata action 2026-02-24 09:59:21 +05:30
Tobeas Arren f4d391b240 Allow starred repos to be mirrored preserving structure 2026-02-14 13:08:41 +01:00
Arunavo Ray 8d96e176b4 fix: prevent duplicate orgs and repos 2025-10-27 08:44:45 +05:30
Arunavo Ray fb27ddfee5 fix: preserve chronological issue mirroring 2025-10-23 23:08:32 +05:30
Arunavo Ray df644be769 cleanup: improve orphaned repo handling 2025-10-22 14:41:54 +05:30
Arunavo Ray 7d23894e5f fix: Custom organization mirror destination issue 2025-10-22 09:03:27 +05:30
Arunavo Ray 3a3ff314e0 refactor: rename skipStarredIssues to starredCodeOnly
The previous name 'skipStarredIssues' was misleading as it now skips ALL
metadata (not just issues) for starred repositories. The new name
'starredCodeOnly' better reflects the actual behavior - mirroring only
source code for starred repos.

Changes:
- Renamed skipStarredIssues → starredCodeOnly in all files
- Updated UI label from "Don't fetch issues" to "Code-only mode"
- Updated description to clarify it skips ALL metadata types:
  issues, PRs, labels, milestones, wiki, and releases
- Updated database schema, types, config mapper, and all components
- Updated Helm charts, CI configs, and documentation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-03 09:22:18 +05:30
Arunavo Ray 6ce70bb5bf chore(version): bump to 3.7.1\n\ncleanup: attempt fix for orphaned repo archiving (refs #84)\n- Sanitize mirror rename to satisfy AlphaDashDot; timestamped fallback\n- Resolve Gitea owner robustly via mirroredLocation/strategy; verify presence\n- Add 'archived' status to Zod enums; set isArchived on archive\n- Update CHANGELOG entry without closing keyword 2025-09-14 07:53:36 +05:30
Arunavo Ray 18ecdbc252 fix(sync): batch inserts + normalize nulls to avoid SQLite param mismatch
- Batch repository inserts with dynamic sizing under SQLite 999-param limit
- Normalize undefined → null to keep multi-row insert shapes consistent
- De-duplicate owned + starred repos by fullName (prefer starred variant)
- Enforce uniqueness via (user_id, full_name) + onConflictDoNothing
- Handle starred name collisions (suffix/prefix) across mirror + metadata
- Add repo-utils helpers + tests; guard Octokit.plugin in tests
- Remove manual unique index from entrypoint; rely on drizzle-kit migrations
2025-09-13 23:38:50 +05:30
Arunavo Ray 89ca5abe7d fix: resolve SQLite field mismatch for large starred repo imports (#90)
- Add missing database fields (language, description, mirroredLocation, destinationOrg) to repository operations
  - Add missing organization fields (publicRepositoryCount, privateRepositoryCount, forkRepositoryCount) to schema
  - Update GitRepo interface to include all required database fields
  - Fix GitHub data fetching functions to map all fields correctly
  - Update all sync endpoints (main, repository, organization, scheduler) to handle new fields

  This fixes the "SQLite query expected X values, received Y" error when importing
  large numbers (4.6k+) of starred repositories by ensuring all database fields
  are properly mapped from GitHub API responses through to database insertion.
2025-09-09 09:56:18 +05:30
Arunavo Ray 10a14d88ef updates 2025-08-28 19:01:39 +05:30
Arunavo Ray 12ee065833 Docs updated | added some options 2025-08-27 21:43:36 +05:30
Arunavo Ray 926737f1c5 Added a few new features. 2025-08-27 20:33:41 +05:30
Arunavo Ray b660d2dd9a feat: add mixed strategy for repository mirroring, enhancing organization handling for personal and organizational repos 2025-06-24 13:51:43 +05:30
Arunavo Ray 68108b8383 feat: add personal repositories organization override and update related configurations 2025-06-24 11:02:57 +05:30
Arunavo Ray f4df7c3d19 Added some basic options 2025-06-15 12:15:14 +05:30
Arunavo Ray bbd49d7d52 Tabs Config 2025-06-11 21:43:43 +05:30
Arunavo Ray 0f671a4088 feat: add support for mirroring wiki pages in configuration 2025-06-11 19:48:24 +05:30
Arunavo Ray 5b60cffaae Add fork tags to repository UI and enhance organization cards with repository breakdown
- Add fork tags to repository table and dashboard list components
- Display 'Fork' badge for repositories where isForked is true
- Enhance organization cards to show breakdown of public, private, and fork repositories
- Update organization API to respect user configuration filters (private repos, forks)
- Add visual indicators with colored dots for each repository type
- Ensure consistent filtering between repository and organization APIs
- Fix issue where private repositories weren't showing due to configuration filtering
2025-05-28 12:53:32 +05:30
Arunavo Ray a28a766f8b refactor: update cleanup and schedule config to use seconds for retentionDays and improve nextRun calculation 2025-05-24 20:12:27 +05:30
Arunavo Ray 47e1c7b493 feat: Implement automatic database cleanup feature with configuration options and API support 2025-05-24 18:33:59 +05:30
Arunavo Ray 5d40023de0 🎉 Gitea Mirror: Added 2025-05-18 09:31:23 +05:30